dotnet watch fails with MSB3021 (locked file) when project references custom source generator

[bdukes]

Describe the bug

When a project references a custom Source Generator, changes to the project while using dotnet watch can cause warning MSB3026, error MSB3027, and error MSB3021. This is due to the source generator project's output DLL being locked.

To Reproduce

I've created a repro at https://github.com/bdukes/lockrepro. There's a script which starts dotnet watch and then switches branches to simulate making changes to the files.

Exceptions (if any)

Error output

dotnet watch ⌚ File changed: .\MyExample.WidgetApi\MyExample.WidgetApi.csproj.
dotnet watch ⌚ Exited
dotnet watch 🔧 Building...
  MyExample.ThisIsASourceGenerator failed with 2 error(s) and 10 warning(s) (12.8s)
    C:\Program Files\dotnet\sdk\8.0.300\Microsoft.Common.CurrentVersion.targets(4806,5): warning MSB3026: Could not copy "obj\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll" to "bin\Any CPU\Debug\net8.0\MyExample.This
IsASourceGenerator.dll". Beginning retry 1 in 1000ms. The process cannot access the file 'I:\code\lockrepro\MyExample.ThisIsASourceGenerator\bin\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll' because it is being used by another process. The file is locked by: ".NET Host (20364)"
    C:\Program Files\dotnet\sdk\8.0.300\Microsoft.Common.CurrentVersion.targets(4806,5): warning MSB3026: Could not copy "obj\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll" to "bin\Any CPU\Debug\net8.0\MyExample.This
IsASourceGenerator.dll". Beginning retry 2 in 1000ms. The process cannot access the file 'I:\code\lockrepro\MyExample.ThisIsASourceGenerator\bin\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll' because it is being used by another process. The file is locked by: ".NET Host (20364)"
    C:\Program Files\dotnet\sdk\8.0.300\Microsoft.Common.CurrentVersion.targets(4806,5): warning MSB3026: Could not copy "obj\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll" to "bin\Any CPU\Debug\net8.0\MyExample.This
IsASourceGenerator.dll". Beginning retry 3 in 1000ms. The process cannot access the file 'I:\code\lockrepro\MyExample.ThisIsASourceGenerator\bin\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll' because it is being used by another process. The file is locked by: ".NET Host (20364)"
    C:\Program Files\dotnet\sdk\8.0.300\Microsoft.Common.CurrentVersion.targets(4806,5): warning MSB3026: Could not copy "obj\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll" to "bin\Any CPU\Debug\net8.0\MyExample.This
IsASourceGenerator.dll". Beginning retry 4 in 1000ms. The process cannot access the file 'I:\code\lockrepro\MyExample.ThisIsASourceGenerator\bin\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll' because it is being used by another process. The file is locked by: ".NET Host (20364)"
    C:\Program Files\dotnet\sdk\8.0.300\Microsoft.Common.CurrentVersion.targets(4806,5): warning MSB3026: Could not copy "obj\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll" to "bin\Any CPU\Debug\net8.0\MyExample.This
IsASourceGenerator.dll". Beginning retry 5 in 1000ms. The process cannot access the file 'I:\code\lockrepro\MyExample.ThisIsASourceGenerator\bin\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll' because it is being used by another process. The file is locked by: ".NET Host (20364)"
    C:\Program Files\dotnet\sdk\8.0.300\Microsoft.Common.CurrentVersion.targets(4806,5): warning MSB3026: Could not copy "obj\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll" to "bin\Any CPU\Debug\net8.0\MyExample.This
IsASourceGenerator.dll". Beginning retry 6 in 1000ms. The process cannot access the file 'I:\code\lockrepro\MyExample.ThisIsASourceGenerator\bin\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll' because it is being used by another process. The file is locked by: ".NET Host (20364)"
    C:\Program Files\dotnet\sdk\8.0.300\Microsoft.Common.CurrentVersion.targets(4806,5): warning MSB3026: Could not copy "obj\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll" to "bin\Any CPU\Debug\net8.0\MyExample.This
IsASourceGenerator.dll". Beginning retry 7 in 1000ms. The process cannot access the file 'I:\code\lockrepro\MyExample.ThisIsASourceGenerator\bin\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll' because it is being used by another process. The file is locked by: ".NET Host (20364)"
    C:\Program Files\dotnet\sdk\8.0.300\Microsoft.Common.CurrentVersion.targets(4806,5): warning MSB3026: Could not copy "obj\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll" to "bin\Any CPU\Debug\net8.0\MyExample.This
IsASourceGenerator.dll". Beginning retry 8 in 1000ms. The process cannot access the file 'I:\code\lockrepro\MyExample.ThisIsASourceGenerator\bin\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll' because it is being used by another process. The file is locked by: ".NET Host (20364)"
    C:\Program Files\dotnet\sdk\8.0.300\Microsoft.Common.CurrentVersion.targets(4806,5): warning MSB3026: Could not copy "obj\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll" to "bin\Any CPU\Debug\net8.0\MyExample.This
IsASourceGenerator.dll". Beginning retry 9 in 1000ms. The process cannot access the file 'I:\code\lockrepro\MyExample.ThisIsASourceGenerator\bin\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll' because it is being used by another process. The file is locked by: ".NET Host (20364)"
    C:\Program Files\dotnet\sdk\8.0.300\Microsoft.Common.CurrentVersion.targets(4806,5): warning MSB3026: Could not copy "obj\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll" to "bin\Any CPU\Debug\net8.0\MyExample.This
IsASourceGenerator.dll". Beginning retry 10 in 1000ms. The process cannot access the file 'I:\code\lockrepro\MyExample.ThisIsASourceGenerator\bin\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll' because it is being used by another process. The file is locked by: ".NET Host (20364)"
    C:\Program Files\dotnet\sdk\8.0.300\Microsoft.Common.CurrentVersion.targets(4806,5): error MSB3027: Could not copy "obj\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll" to "bin\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll". Exceeded retry count of 10. Failed. The file is locked by: ".NET Host (20364)"
    C:\Program Files\dotnet\sdk\8.0.300\Microsoft.Common.CurrentVersion.targets(4806,5): error MSB3021: Unable to copy file "obj\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll" to "bin\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll". The process cannot access the file 'I:\code\lockrepro\MyExample.ThisIsASourceGenerator\bin\Any CPU\Debug\net8.0\MyExample.ThisIsASourceGenerator.dll' because it is being used by another process.      
Build failed with 2 error(s) and 10 warning(s) in 14.1s

Further technical details

dotnet --info

.NET SDK:
 Version:           8.0.300
 Commit:            326f6e68b2
 Workload version:  8.0.300-manifests.818b3449
 MSBuild version:   17.10.4+10fbfbf2e
Runtime Environment:

OS Name:     Windows

OS Version:  10.0.22631

OS Platform: Windows

RID:         win-x64

Base Path:   C:\Program Files\dotnet\sdk\8.0.300\
.NET workloads installed:

There are no installed workloads to display.
Host:

Version:      8.0.5

Architecture: x64

Commit:       087e15321b
.NET SDKs installed:

6.0.422 [C:\Program Files\dotnet\sdk]

8.0.205 [C:\Program Files\dotnet\sdk]

8.0.300 [C:\Program Files\dotnet\sdk]
.NET runtimes installed:

Microsoft.AspNetCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]

Microsoft.AspNetCore.App 6.0.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]

Microsoft.AspNetCore.App 7.0.19 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]

Microsoft.AspNetCore.App 8.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]

Microsoft.NETCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

Microsoft.NETCore.App 6.0.30 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

Microsoft.NETCore.App 7.0.19 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

Microsoft.NETCore.App 8.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

Microsoft.WindowsDesktop.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Microsoft.WindowsDesktop.App 6.0.30 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Microsoft.WindowsDesktop.App 7.0.19 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Microsoft.WindowsDesktop.App 8.0.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Other architectures found:

x86   [C:\Program Files (x86)\dotnet]

registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]
Environment variables:

Not set
global.json file:

Not found
Learn more:

https://aka.ms/dotnet/info
Download .NET:

https://aka.ms/dotnet/download

[bdukes]

Our entire team started seeing this issue become much more prevalent when we moved from .NET 7 to .NET 8.