How can I speed up Nuget package restore on Azure DevOps?

Nuget package restores take forever and fail frequently, can anything be done?

Additional Posts

Caveat: This approach can have issues with the Cache plugin running out of storage.

One option is to cache Nuget package dependencies. In DevOps, you add two steps:

1. Hash all the packages.config files together in a Powershell script.
2. Use the Cache plugin to cache the contents of the packages folder based on the hash of the packages.config files.

      - task: PowerShell@2
        inputs:
          targetType: 'inline'
          script: 'Get-FileHash -Algorithm MD5 -Path (Get-ChildItem packages.config -Recurse) >> hash.txt'
          workingDirectory: './src_dir_if_necessary/'
      
      - task: Cache@2
        inputs:
          key: 'nuget|1|$(Agent.OS)|$(Build.SourcesDirectory)\src_dir_if_necessary\hash.txt'
          path: '$(Build.SourcesDirectory)\packages'
          cacheHitVar: 'nuget_cache_hit'

Of course, replace src_dir_if_necessary if necessary, if your projects are in a particular directory. Otherwise you can leave it out. Be sure that you don't exclude any packages.config files that exist in the project, as changes to any of them should invalidate the cache.

The packages folder will be cached for up to 7 days. If any packages.config file changes, Nuget will need to restore all packages from sources, but that folder will be cached going forward.

Comments

• This seems to have issues with storage limits. Probably don't do this one. :(

If the Cache plugin in DevOps is having issues, you can also fake it by caching packages in Azure storage instead. You do this in two steps:

At the top of your build pipeline, use an Azure CLI step to pull your packages folder down from storage. There is a package-cache container in the blendinteractive storage account for this purpose, just be sure you use the right paths so we don’t have different projects using the same packages folders.

For example, from the Blend Developer site (in YAML form):

    - task: AzureCLI@2
      inputs:
        azureSubscription: 'Blend Azure Access'
        scriptType: 'ps'
        scriptLocation: 'inlineScript'
        inlineScript: |
          az storage blob download-batch -d "$($env:Build_SourcesDirectory)" --pattern blend-developer-site/Packages/* -s package-cache --account-name blendinteractivestorage
          Move-Item -Path "$($env:Build_SourcesDirectory)/blend-developer-site/Packages" -Destination "$($env:Build_SourcesDirectory)/packages"
        powerShellErrorActionPreference: 'continue'

You will need to update blend-developer-site to a folder name that represents the project you're working on. You will potentially also need to update Blend Azure Access to a service principal with access rights to the Blend subscription.

Note that when this downloads the packages, they will be in a blend-developer-site/Packages folder, so there is a second step that moves the packages folder to the correct location.

Then after the rest of the build has completed, you will need to sync any new or updated packages back to Azure storage.

Again, example from the Blend Developer site:

- task: AzureCLI@2
  inputs:
    azureSubscription: 'Blend Azure Access'
    scriptType: 'ps'
    scriptLocation: 'inlineScript'
    inlineScript: |
      az storage blob sync -c package-cache --account-name blendinteractivestorage -s "$($env:Build_SourcesDirectory)\packages\" -d blend-developer-site/Packages

And again, you will need to update blend-developer-site and potentially Blend Azure Access.

Pulling packages from Azure Storage is weirdly slow, but (hopefully) more reliable than restoring from Nuget every time.

On a sidenote, with this approach you can also prime the cache by manually copying your own packages folder to the appropriate place in Azure Storage, if necessary.