Azure DevOps Dynamic Agent (Using EC2)
Introduction
Azure DevOps offers Microsoft-hosted and self-hosted agent to execute pipelines.
With Microsoft-hosted agents, maintenance and upgrades are taken care of for you. Each time you run a pipeline, you get a fresh virtual machine. The virtual machine is discarded after one use. Microsoft-hosted agents can run jobs directly on the VM or in a container. Azure Pipelines provides a pre-defined agent pool named Azure Pipelines with Microsoft-hosted agents.
An agent that you set up and manage on your own to run jobs is a self-hosted agent. Self-hosted agents give you more control to install dependent software needed for your builds and deployments. Also, machine-level caches and configuration persist from run to run, which can boost speed.
Challenge with Self-Hosted agent
Assume you have requirement to use self-hosted agent. There could be multiple reasons but I am not covering this in this article. So, either we purchase license to execute parallel job or go for multiple self-hosted agent. But this solution is either expensive or not extensible when team grows in size.
For example, you have application A (multi module maven project or any enterprise application) that takes 15–30 min to complete CI process (build, unit test, database test, code quality etc). What if 10–15 developer checks the code, obviously in their respective feature branch. If I am having one self-hosted agent (in pool — app-testpool) without parallel job (won’t prefer to buy it as its incur more money and secondly increase more load on agent machine), then all jobs will run sequentially, one after another.
This is frustrating for dev team and not at solution to sustain in long term. So we need solution t have dynamic scaling of self-hosted agent.
Dynamic scaling
Agent Pool: Collection of Agents. You can create an agent pool for your organization and share it among multiple projects. In our case, it is a collection of init agents (dynamic agents).
Process Flow
- Spin up self-hosted Agent: This is the phase that creates an agent as per your need (I am using Centos based EC2 using custom AMI). Under the hood, I am using customized AMI to avoid any installation on EC2. Simply need to run command to join Agent in DevOps pool. Use Init Script to call script. Use appropriate tag as mentioned above, to identify the agent in later phases.
Note: Init agent creates a new agent (EC2) with the unique name <pipeline name>-$Build.BuildId) which will be used as a demand in the next phase.
2. Build or Release: This is the actual phase that will run your build or release. It is very important to pick the agent which was created earlier during the SpinUp Agent phase. To achieve that, we can use the agent name as a demand.
3. Delete Agent: In this phase, we will just delete the agent (pod) which was created during the SpinUp Agent phase. We have to make sure, we are deleting the correct agent, which can be done by deleting the helm deployment by name (<pipeline name>-$(Build.BuildId)).
This phase must run on the ‘init’ agent.
Important: We need to preserve the sequential order which we specify for the phases as every phase is dependent on the previous phase.
For instance, the Build or Release phase depends on the Spin up Agent and its success. Moreover, Delete Agent follows the above sequential order but will continue to run on an error: continueOnError: true