Nondeterministic resolution (?) causing dependency to flap in Cargo.lock

[ijackson]

Problem

We have a situation where cargo wants to keep changng the version of one of our dependencies. Probably related is that our Cargo.tomls say things like this

heck = ">=0.4, <0.6"

Steps

1. Starting point

git clone https://gitlab.torproject.org/Diziet/rust-derive-deftly
cd rust-derive-deftly
git checkout 84faa3c30c05bb20bd9b1f54e2e97ecbf844ead4

2. Observe that cargo is happy and makes no changes

cargo fetch
git diff
# observe no output

3. Make a minor change which doesn't require different deps

git cherry-pick a20d8994f74e091a0df864211e087e5ed5f15615
git show # See that we change the version number of one crate

4. Ask cargo what it thinks

cargo --locked fetch
# observe: error "Cargo.lock needs to be updated", expected
cargo fetch
# lockfile is updated

5. Look at Cargo.lock diff:

git diff

6. Expected changes

Many changes of the form

- "derive-deftly 0.11.0",
+ "derive-deftly 0.10.666",

are expected.

The changes of the form

- "derive-deftly 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)",
+ "derive-deftly 0.11.0",

are OK - that's just removing disambiguations.

7. Unexpected change

- "heck 0.5.0",
+ "heck 0.4.1",

This hunk is undesirable. Also, in our experience, cargo sometimes wants to change this from 0.4 to 0.5 and sometimes from 0.5 to 0.4. (I tried to repro it flapping, but I wasn't able to.)

Possible Solution(s)

Ideally cargo wouldn't make unecessary changes.

Notes

The use of a relaxed version dependency seems to be implicated.

Our dependency stack includes heck 0.4 because of strum.

Version

cargo 1.79.0-beta.3 (8a7384119 2024-05-03)
release: 1.79.0-beta.3
commit-hash: 8a7384119af18fa4b3311c310661acd81f1b4885
commit-date: 2024-05-03
host: x86_64-unknown-linux-gnu
libgit2: 1.7.2 (sys:0.18.3 vendored)
libcurl: 8.6.0-DEV (sys:0.4.72+curl-8.6.0 vendored ssl:OpenSSL/1.1.1w)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: Debian 10.0.0 (buster) [64-bit]

[ijackson]

(This is derive-deftly ticket 82)

[epage]

This mentions 1.79.0-beta.3. I've reproduced this on 1.64

[epage]

So the reproduction step has us cherry pick a20d8994f74e091a0df864211e087e5ed5f15615

diff --git c/Cargo.toml w/Cargo.toml
index 80f8e24..48d3569 100644
--- c/Cargo.toml
+++ w/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "derive-deftly"
-version = "0.11.0"
+version = "0.10.666"
 edition = "2021"
 license="MIT"
 authors=["Ian Jackson <ijackson@chiark.greenend.org.uk>",

• This is why changes in version are expected
• There are registry dependencies on derive-deftly 0.11.0, so that is why the disambiguation changes are there

On the baseline commit:

$ cargo tree -i heck@0.4
heck v0.4.1
└── strum_macros v0.26.2 (proc-macro)
    └── strum v0.26.2
        └── derive-deftly-macros v0.11.0 (proc-macro) (/home/epage/src/personal dump/rust-derive-deftly/macros)
            └── derive-deftly v0.11.0 (/home/epage/src/personal/dump/rust-derive-deftly)
$ cargo tree -i heck@0.5
heck v0.5.0
├── derive-deftly v0.11.0 (/home/epage/src/personal/dump/rust-derive-deftly)
└── derive-deftly-macros v0.11.0 (proc-macro) (/home/epage/src/personal/dump/rust-derive-deftly/macros)
    └── derive-deftly v0.11.0 (/home/epage/src/personal/dump/rust-derive-deftly)

• strum_macros depends on heck = "0.4.1"
• both derive-deftly and derive-deftly-macros depends on heck = ">=0.4, <0.6"
  • The registry version has the same version requirement

However, if I run

$ cargo generate-lockfile

then the version of heck doesn't change.

So overall, I'm not finding anything out of the ordinary than wide version ranges.
My general assumption has always been that we pick the highest version for each version range. However, what is likely happening here is that we are resolving derive-deflty freshly and seeing an existing locked resolve of heck and picking that.

We have #13535 for tweaking the logic which might make this more consistent.