Limit the number of agent deployments
When you deploy agents to monitor applications with IAST (Assess), you want consider their impact on performance. In cases where your web traffic is load balanced across a farm of servers, you may not need to install an agent on every server. If your load balancer distributes traffic evenly and routes all traffic through every server in the farm, you could achieve the same level of monitoring or vulnerability detection by deploying an agent to just one server. Similarly, you could have sample points at each back-end destination based on your load balancing topology.
This topic provides examples of how you can limit the deployment of agents to improve performance.
Reasons for limiting agent deployments
Performance optimization: Deploying agents across all servers can add unnecessary overhead, especially if your applications are latency-sensitive or resource-constrained. By reducing the number of deployed agents, you minimize CPU and memory usage, which can improve response times and overall application performance.
Sufficient traffic coverage: Since your load balancer evenly distributes traffic across all servers, an agent on one server would still monitor representative traffic. In many cases, this can provide enough data for security and monitoring purposes without the need to instrument every instance.
Deploy selected agents in Kubernetes (K8s)
In a Kubernetes environment, you can manage which pods in a deployment are instrumented with an agent by controlling the number of replicas that run with the agent. Use this strategy as a guide:
Nodes taints: Use node taints and tolerations to dedicate specific nodes for pods with agents. Label one node with a specific taint and schedule the agent-enabled pod to run on that node, leaving the others unaffected.
Pod affinity: Use pod affinity and anti-affinity to control where your agent-enabled pods run. You can create an affinity rule that only schedules the agent-enabled pod to a particular node or pod, ensuring it doesn’t replicate across the entire farm.
Deployment overrides: Use a custom deployment manifest where you set up two deployments: one for the regular application pods and another for the agent-enabled pods. For the agent-enabled deployment, you can set replicas: on to ensure only one instance runs with the agent.
Example: Selective agent deployment in K8s
In this YAML example,only one replica of the application has the agent enabled.
apiVersion: apps/v1
kind: Deployment
metadata:
name: app-with-agent
spec:
replicas: 1
template:
metadata:
labels:
app: my-app
agent: enabled
spec:
containers:
- name: app-container
image: my-app-image
env:
- name: ENABLE_AGENT
value: "true"Deploy selected agents with Docker Swarm
Similar to what you can do with K8s, you can use service scaling and constraints in Docker Swarm, as described in this strategy:
Service constraints: Use service constraints to deploy the agent-enabled container to a specific node or set of nodes. For example, you can label a node and ensure that only one agent-enabled service runs on that node.
Bash example
This example deploys the agent-enabled instance to a node with the label
agent == true, leaving other nodes unaffected.docker service create \ --replicas 1 \ --constraint "node.labels.agent == true" \ --env ENABLE_AGENT=true \ my-app-image
Task replication: In Docker Swarm, you can use two services: one with the regular application and one with the agent. Scale the regular service across all nodes and the agent-enabled service to just one replica.
Bash example
This example shows how creates a setup where most of the containers run without the agent, but one instance includes the agent for monitoring purposes.
docker service create --replicas 5 my-app-image # Regular service docker service create --replicas 1 --env ENABLE_AGENT=true my-app-image # Agent-enabled service
Troubleshooting and testing recommendations
Deploying the agent to only one server in a web farm can introduce issues with traffic routing. Since your load balancer distributes traffic across all servers, you can’t always guarantee that test traffic will consistently go to the server with the installed agent. Verifying results can be more difficult, because some traffic may bypass the instrumented server. You may observe incomplete data or gaps in monitoring because not all traffic is routed to the correct server.
In such cases, you need to investigate your load balancer settings: Consider the following :
Session affinity (sticky sessions): Ensure that your load balancer supports session affinity, which can route repeat requests from the same client to the same server. By enabling sticky sessions, you can direct all test traffic from a specific user or IP address to the server with the agent, improving consistency during testing.
Targeted testing: You might need to use a load balancer feature that allows for weighted routing or manual targeting of specific servers for certain traffic. This feature would let you send your test traffic to the server with the agent, ensuring that the agent processes it.