Yesterday I was confronted with the situation that an Azure DevOps pipeline, which is executed on a self-hosted agent, failed with the following error.
##[error]There is not enough space on the disk. : 'D:\Agent01_diag\pages\00000000-0000-0000-0000-000000000000_1.log'
,##[error]Error reported in diagnostic logs. Please examine the log for more details.
I checked drive D:\ of the VM on which the agents are running. It was full, as expected according to the error message.
A more detailed analysis revealed that there were many large but old directories in the _work directories of the various agent folders (on drive D:\). I had some doubts about deleting these folders manually. Therefore I investigated a bit deeper and came across the maintenance job feature of Azure DevOps which was disabled for the corresponding agent pool on the Azure DevOps organizations that use the agent pool with the self-hosted agents.
Maintenance job
The maintenance job feature is supported by agent pools that contain self-hosted agents. If enabled, stale working directories and repositories are periodically cleaned which will reduce the risk that agents run out of disk space.
Besides the Days to keep unused working directories setting there are some general settings regarding timeout, concurrency and job history. Furthermore the scheduling has to be defined based on your needs.
I recommend scheduling the maintenance jobs when the agents are usually not busy at all.
For more details regarding maintenance job, see here.
blog.rufer.be 
Leave a comment