Azure DevOps Clean after pipeline run

Question

I have self-hosted build agents using Azure DevOps. We have the clean flag in the checkout step, however we run out of space sometimes.

What is the preferred way of cleaning the agents' work directories?

The problem is, that the clean process is in the beginning of a pipeline, and leaves everything behind. This is even worse if we have two parallel builds. (First clean then clones the repo, then the second cleans and clones the repo, then first build, then there is no space for the second build). Unless the first build is scheduled again, the build outputs stays on the agent.

We tried to schedule a deletion task every day, but it doesn't solve our problems.

We tried to add a script at the end of our yaml file that cleans the working dir. Is it recommended?

    - powershell: Remove-Item $(Pipeline.Workspace)/** -Force
      displayName: Clean after publish

Also it would be great if we could clean up even if the build fails, or someone cancels it midway.

score 2 · Answer 1 · edited Mar 08 '24 at 20:56

2

- task: DeleteFiles@1
  displayName: 'clean *'
  inputs:
    SourceFolder: '$(Pipeline.Workspace)'
    Contents: '**/*'
    RemoveDotFiles: true
  condition: always()

will clean everything in '$(Pipeline.Workspace)/notClean/allClean'.

edited Mar 08 '24 at 20:56

Community

1

answered Nov 30 '22 at 04:31

lioncub

21
2

lg2de · Answer 2 · 2023-11-29T10:42:38.333

I tried to use DeleteFiles@1, like @lionclub. But the implementation is very slow for large repositories.

So, I decided to use PowerShell:

- task: PowerShell@2
  displayName: 'Cleanup source folder'
  inputs:
    targetType: 'inline'
    script: 'Get-ChildItem -Recurse | foreach { Write-Host $_.FullName ; Remove-Item $_.FullName -Force -Recurse }'
    failOnStderr: true
    pwsh: true
    workingDirectory: '$(Build.SourcesDirectory)'

There are variations:

Using failOnStderr or not
Using PowerShell or PowerShell Core (pwsh: true)
Add separate tasks for $(Build.ArtifactStagingDirectory) and $(Build.BinariesDirectory)
Clear complete working folder e.g. by using variable $(Agent.BuildDirectory)
Adding condition: always() to always clear folder, incl. failed runs

score 1 · Accepted Answer · answered Dec 12 '22 at 10:34

There is a clean option within the job properties. See the job definition.

jobs:
- job:
...
 workspace:  # Workspace options on the agent.
   clean: string # Which parts of the workspace should be scorched before fetching.  (outputs, resources, all)

As it would be called before fetching, there will be always space for the new build.

Azure DevOps Clean after pipeline run

3 Answers3