Skip to main content

Pod restart

Kubernetes pods are designed to maintain a desired state defined in YAML configurations. When changes occur to configurations that affect a pod's runtime environment, Kubernetes initiates pod restarts to ensure the actual state aligns with the desired state. This reconciliation process is fundamental to Kubernetes' operation.

Specifically, modifications to resources such as AgentConfiguration, AgentConnection, and AgentInjector trigger these restarts. These resources define critical aspects of the pod's behavior, including connectivity, settings, and injection rules. When these configurations are updated, Kubernetes detects the discrepancy between the current pod state and the new desired state.

The impact extends to ClusterAgentConnection, ClusterAgentConfiguration, and ClusterAgentInjector too. These cluster-level resources generate AgentConnections, AgentConfigurations, and AgentInjectors within specified namespaces.

Any change to the pod's configuration, whether direct or indirect through related resources, necessitates a restart to guarantee that the pod reflects the latest defined state within the cluster. This ensures consistency and reliability across the application. It is essential to understand that only the pods affected explicitly by the configuration change will be restarted. For instance, if an AgentInjector targets deployments using a specific label selector, only the pods within those targeted deployments will be restarted, not all pods in the cluster.

Reduce pod restarts

To have pods lazily update agent versions there are a few options:

  • Use the latest for the AgentInjector version (the default). This will use the latest agent when a pods starts.

  • Tags for the major and minor versions are published alongside the exact version tag in the docker registry. You can use the major/minor version in the AgentInjector to have pods use the newest agent in that range. For example, 2 will select the highest 2.x.x version available.

  • If using a custom registry, you can set the version on the AgentInjector to latest and update the latest tag to point to a new image in the registry.

  • Setting ClusterAgentInjector.reconcilePolicy: OnCreate or AgentInjector.reconcilePolicy: OnCreate will patch a workload only when it first becomes matched. After that, the agent-operator leaves the workload alone on settings, version, connection, configuration, or secret drift, and new settings reach pods as they are naturally recreated by deploys, autoscaling, evictions, and node replacement.

In this section: