Optimizing Disk Space on Microsoft-Hosted Agents (14 GB Limit) in Azure DevOps Pipelines for Dynamics 365 FO

Azure DevOps pipelines for Dynamics 365 for Finance and Operations (D365 FO) often involve managing large build processes, including numerous NuGet dependencies and ISV solutions. With Microsoft-hosted agents offering limited disk space (14 GB), it's crucial to optimize disk usage to prevent pipeline failures and keep builds efficient.

In this blog post, we’ll explore:

1. Why optimizing disk space is important.
2. Practical steps to free up disk space during pipeline runs.
3. How to implement these optimizations in your YAML pipeline.

Limit is mentioned in the official documentation Microsoft-hosted agents

Why Optimize Disk Space?

Disk space issues are rare but can occur in pipelines involving Dynamics 365 FO due to the nature of ISV solutions and dependencies:

• ISV solutions in version control often include precompiled binary-only packages. These are necessary for delivering the code and enabling builds but can significantly increase workspace size. Once these binaries are copied to the designated build directory, they can be safely removed from the checkout folder to free up space.
• NuGet packages are essential for compilation but are not always needed in later pipeline steps, such as deployable package creation. Removing unused packages after the build phase ensures that disk space is available for subsequent tasks.

Failing to optimize disk space can result in errors during critical tasks. For example, the following error was encountered when creating a deployable package due to insufficient disk space: [error]Exception calling "MergePackage" with "5" argument(s): "There is not enough space on the disk."

By strategically managing ISV binaries and NuGet packages, you can optimize the limited disk space on Microsoft-hosted agents and avoid potential pipeline failures.

Practical Steps to Optimize Disk Space

To address disk space challenges, here is a clear and actionable approach:

Checkout Only the Necessary Repository

Minimizing the data fetched from the repository is a good first step. Use options like clean: true to remove leftover files from previous runs, and fetchDepth: 1 to download only the latest commit.

Yaml Example

- checkout: self
  clean: true
  fetchDepth: 1

Remove ISV and Binary-Only Files from Checkout Folder Once Copied

ISV solutions are important during the build phase but are no longer needed in the checkout folder once their binaries are copied to the designated build directory. Removing these files at this stage helps free up disk space.

YAML Example

# Copy ISV and other third-party codes (binaries) to 'Build.BinariesDirectory' folder
- task: CopyFiles@2
  displayName: 'Copy ISV codes to Build.BinariesDirectory'
  inputs:
    SourceFolder: '$(Build.SourcesDirectory)/App/BinaryOnly'
    TargetFolder: '$(Build.BinariesDirectory)'

# Remove ISV source codes after copying Build.BinariesDirectory
- task: DeleteFiles@1
  displayName: 'Remove BinaryOnly folder to free up disk space'
  inputs:
    SourceFolder: '$(Build.SourcesDirectory)/App'
    Contents: 'BinaryOnly'

Remove Unnecessary NuGet Packages Post-Build

NuGet packages restored during the build are often required only for compilation. Removing those not needed for later tasks, such as deployable package creation, can free up significant space and avoid unnecessary disk usage.

YAML Example

# Remove unused NuGet packages after the build
- task: DeleteFiles@1
  displayName: 'Remove unnecessary NuGet packages post-build'
  inputs:
    SourceFolder: '$(Pipeline.Workspace)/NuGets'
    Contents: |
      Microsoft.Dynamics.AX.Application1.DevALM.BuildXpp
      Microsoft.Dynamics.AX.Application2.DevALM.BuildXpp
      Microsoft.Dynamics.AX.ApplicationSuite.DevALM.BuildXpp
      Microsoft.Dynamics.AX.Platform.DevALM.BuildXpp

Pipeline Overview

Here’s how the entire pipeline could look after implementing these optimizations:

- checkout: self
  clean: true
  fetchDepth: 1

# Copy ISV codes (binaries) to 'Build.BinariesDirectory' folder
- task: CopyFiles@2
  displayName: 'Copy ISV codes to Build.BinariesDirectory'
  inputs:
    SourceFolder: '$(Build.SourcesDirectory)/App/BinaryOnly'
    TargetFolder: '$(Build.BinariesDirectory)'

# Remove ISV source codes after copying Build.BinariesDirectory
- task: DeleteFiles@1
  displayName: 'Remove BinaryOnly folder to free up disk space'
  inputs:
    SourceFolder: '$(Build.SourcesDirectory)/App'
    Contents: 'BinaryOnly'

# Build application
- task: VSBuild@1
  displayName: 'Build solution'
  inputs:
    solution: '$(BuildSolutionPath)'
    msbuildArgs: >
      ...
      /p:OutputDirectory="$(Build.BinariesDirectory)"

# Remove unused NuGet packages after the build
- task: DeleteFiles@1
  displayName: 'Remove unnecessary NuGet packages post-build'
  inputs:
    SourceFolder: '$(Pipeline.Workspace)/NuGets'
    Contents: |
      Microsoft.Dynamics.AX.Application1.DevALM.BuildXpp
      Microsoft.Dynamics.AX.Application2.DevALM.BuildXpp
      Microsoft.Dynamics.AX.ApplicationSuite.DevALM.BuildXpp
      Microsoft.Dynamics.AX.Platform.DevALM.BuildXpp

# Create deployable package
- task: XppCreatePackage@2
  displayName: 'Create deployable package'
  inputs:
    XppToolsPath: '$(Pipeline.Workspace)/NuGets/Microsoft.Dynamics.AX.Platform.CompilerPackage'
    XppBinariesPath: '$(Build.BinariesDirectory)'
    DeployablePackagePath: '$(Build.ArtifactStagingDirectory)/LCS/AllInOne_$(Build.BuildNumber).zip'

Conclusion

Optimizing disk space is a critical part of managing Azure DevOps pipelines for Dynamics 365 FO, especially for large builds with ISV solutions and extensive dependencies. By implementing the following strategies:

• Fetching only the necessary repository content to minimize workspace size.
• Removing ISV codes from the checkout folder after copying binaries.
• Cleaning up unused NuGet packages post-build.

You can efficiently use the limited 14 GB disk space available on Microsoft-hosted agents, reduce build failures, and maintain a streamlined workflow.

If you’ve faced similar disk space challenges or have other optimization insights, connect with me on LinkedIn to discuss and share your experiences!