Skip to content

[Operational Issue]: darc-pub-* feeds can cause package restores races in stable branches #5609

@akoeplinger

Description

@akoeplinger

Error

In #5417 we saw the build failing with this error randomly:

Overriding System.Private.CoreLib.dll with a newer version is not supported. Attempted to use D:\a_work\1\s\src\runtime\artifacts.packages\microsoft.netcore.app.runtime.win-x64\10.0.5\runtimes\win-x64\lib\net10.0\System.Private.CoreLib.dll instead of D:\a_work\1\s\src\runtime\artifacts.packages\microsoft.netcore.app.runtime.nativeaot.win-x64\10.0.5\runtimes\win-x64\native\System.Private.CoreLib.dll.

Which was weird since they both referenced the same version of the runtime packs.

The root cause turned out to be a race depending on whether one of the packs was restored from dotnet-public or a darc-pub feed:

The working restore was getting these packages (i.e. both runtime packs from the same darc-pub-dotnet-dotnet-8d7e7d73 feed):

Installed Microsoft.NETCore.App.Runtime.win-x86 10.0.5 from https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-dotnet-8d7e7d73/nuget/v3/index.json to D:\a\_work\1\s\src\runtime\artifacts\.packages\microsoft.netcore.app.runtime.win-x86\10.0.5 with content hash R9TK837/ekTlFcIfgifXbm7BKFDrV1TQWxgS249SQwnRn4pxsI8pbCwGyHfoHwvYGTiA+FXHx9wnRZAtcb1J+w==.
Installed Microsoft.NETCore.App.Runtime.NativeAOT.win-x86 10.0.5 from https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-dotnet-8d7e7d73/nuget/v3/index.json to D:\a\_work\1\s\src\runtime\artifacts\.packages\microsoft.netcore.app.runtime.nativeaot.win-x86\10.0.5 with content hash pwMpFPaWjrgfgwwdNbZADEvWetRGHHuMtmzxpKJA9xlp6bfC6W4BSw0TA7yl9e076DerKmjlO4EdRHMQg2QnxQ==.

The failing one however got one of them from the dotnet-public feed:

Installed Microsoft.NETCore.App.Runtime.win-x86 10.0.5 from https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget/v3/index.json to D:\a\_work\1\s\src\runtime\artifacts\.packages\microsoft.netcore.app.runtime.win-x86\10.0.5 with content hash oQN9eBUcb27U6D+tD4buA2dqlxRQsFuSm8xng4QemvSvscxCzySNhfwbm8hVk808xaWiiVwBT47OxlZs+pGCyw==.
Installed Microsoft.NETCore.App.Runtime.NativeAOT.win-x86 10.0.5 from https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-dotnet-8d7e7d73/nuget/v3/index.json to D:\a\_work\1\s\src\runtime\artifacts\.packages\microsoft.netcore.app.runtime.nativeaot.win-x86\10.0.5 with content hash pwMpFPaWjrgfgwwdNbZADEvWetRGHHuMtmzxpKJA9xlp6bfC6W4BSw0TA7yl9e076DerKmjlO4EdRHMQg2QnxQ==.

So that's where the different file versions came from since we likely pushed a different build of 10.0.5 to nuget.org (dotnet-public).

We need to find a solution for this so we don't have randomly failing builds when we update stable versions in servicing branches.

Links

No response

Root Cause

No response

Blocking

  • This issue is blocking

Affected Releases

  • .NET 10
  • .NET 10 (SDK band build)
  • .NET 11

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions