[HOWTO] Limit the number of Microsoft-hosted agents that can be used by an Azure DevOps YAML pipeline

Usually an Azure DevOps Services organization contains multiple projects. Often for some of these projects YAML pipelines are set up, which are triggered on every pull request and commit to the develop and main branches. Let’s assume that in one or more of these Azure DevOps projects a whole bunch of YAML pipelines is set up for a specific Git repository where multiple developers actively contribute to. For example a pipeline for building the backend source code and deploying the artifacts to a specific stage, a pipeline for the frontend part, a pipeline for validating DB migrations, a pipeline for end to end tests, …

Based on the amount of purchased parallel jobs and depending on the amount of contributions and the execution time of the different pipelines it can lead to the situation, that a single Azure DevOps project uses all Microsoft-hosted agents available for a certain time frame. This can be a very annoying situation. Especially if production deployments get postponed due to that fact for let’s say 20 minutes.

While searching for possible improvements for this problem, I came across the following pragmatic approach.

You can define so called User-defined capabilities for Microsoft-hosted agents and adjust the pipelines that are usually blocking others by acquiring all agents to require these capabilities. This is a simple way to limit the number of agents for a particular pipeline.

Demands and capabilities are designed for use with self-hosted agents so that jobs can be matched with an agent that meets the requirements of the job. When using Microsoft-hosted agents, you select an image for the agent that matches the requirements of the job, so although it is possible to add capabilities to a Microsoft-hosted agent, you don’t need to use capabilities with Microsoft-hosted agents.

Azure Pipelines Agents – Capabilities

To do so, proceed as follows:

Add user-defined capability

1. Go to your Azure DevOps Services organization
2. Open Organization Settings
3. Navigate to Agent pools under Pipelines
4. Select pool Azure Pipelines
5. Switch to tab Agents
6. Click on the desired agent
7. Switch to tab Capabilities
8. Click + button
9. Enter a name and a value

Add demand for capability to pipeline

1. Open the YAML file of the desired pipeline
2. Add a demand for the before specified capability to the pipeline

pool:
  vmImage: desiredImage
  demands:
  - nameOfYourCapability [-equals valueOfYourCapability]

Update 11.11.2022

If you use vmImage windows-latest, you probably get the following error:

##[error]No agent found in pool Hosted Windows 2019 with VS2019 which satisfies the following demand: nameOfYourCapability. All demands: ...

To resolve that, just specify the name of the pool as follows

pool:
  name: 'Azure Pipelines'
  vmImage: desiredImage
  demands:
  - nameOfYourCapability [-equals valueOfYourCapability]

For more details see here