Vendored dependencies

[joseandro]

We were exposed to supply chain attacks because our Rust dependencies were not pinned. This PR fixes that by pinning the kimchi stubs dependencies and also incorporating all vendored dependencies available in proof-systems.

Explain your changes:

• Vendored all Kimchi stubs dependencies to a separate repository (i.e., kimchi-stubs-vendors).
• Incorporated the kimchi-stubs-vendors repository as a dependency through git submodules.
• Updated the Kimchi stubs dune script with proper instructions to use the vendored dependencies.
• Pulled in all the vendored dependencies available in proof-systems:berkeley.

Explain how you tested your changes:

• Verified that our build scripts no longer downloads crates from remote sources, both on CI and locally.
• Ensured that Cargo always compiles the pinned dependencies available in our vendored submodules, both on CI and locally.
• Checked that the version of the dependencies that Cargo builds matches exactly what we specify in Cargo.toml.

Checklist:

• Dependency versions are unchanged
  • Notify Velocity team if dependencies must change in CI
• Modified the current draft of release notes with details on what is completed or incomplete within this project
• Document code purpose, how to use it
  • Mention expected invariants, implicit constraints
• Tests were added for the new behavior
  • Document test purpose, significance of failures
  • Test names should reflect their purpose
• All tests pass (CI will check this if you didn't)
• Serialized types are in stable-versioned modules
• Does this close issues? List them

• Closes Berkeley nightly failing due to rust library updates #15623

[joseandro]

!ci-build-me

[joseandro]

!ci-build-me

[joseandro]

!ci-build-me

[joseandro]

!ci-build-me

[joseandro]

!ci-build-me

[joseandro]

!ci-build-me

[joseandro]

!ci-build-me

[joseandro]

!ci-build-me

[joseandro]

!ci-build-me

[dannywillems]

I have:

ile "src/lib/crypto/kimchi_bindings/stubs/dune", line 51, characters 0-615:
51 | (rule
52 |  (enabled_if
53 |   (= %{env:MARLIN_PLONK_STUBS=n} n))
....
74 |     cp
75 |     %{read:dune-build-root}/cargo_kimchi_stubs/release/libwires_15_stubs.a
76 |     .))))
error: failed to select a version for the requirement `libc = "^0.2.0"` (locked to 0.2.154)
candidate versions found which didn't match: 0.2.147
location searched: directory source `/home/soc/codes/o1-labs/mina-berkeley/_build/default/src/lib/crypto/kimchi_bindings/stubs/kimchi-stubs-vendors` (which is replacing registry `crates-io`)
required by package `wires_15_stubs v0.1.0 (/home/soc/codes/o1-labs/mina-berkeley/_build/default/src/lib/crypto/kimchi_bindings/stubs)`
perhaps a crate was updated and forgotten to be re-vendored?

while building.

Also, i have the following (unrelated) error:

Error: optcomp: cannot open imported file: /src/config.mlh: /src/config.mlh: No such file or directory
File "src/lib/genesis_constants/genesis_constants.ml", line 1, characters 3-9:
1 | [%%import "/src/config.mlh"]

Do you know where it comes from?

Commands I use to test:

git clone git@github.com:MinaProtocol/mina/ mina-berkeley
cd mina-berkeley
git checkout vendoring
gvm use go1.20
git submodule init
opam switch create ./ 4.14.0
git submodule update --recursive && eval $(opam env)
opam switch import opam.export -y
./scripts/pin-external-packages.sh
make build

[dannywillems]

I ran #15603 (comment) and it seems to work, not downloading anything. LGTM.

[joseandro]

Building it one last time to make sure everything is fine using the berkeley branch in proof-systems

[joseandro]

!ci-build-me

[joseandro]

!ci-build-me

[joseandro]

!ci-build-me

[joseandro]

!ci-build-me

[joseandro]

!ci-build-me

[mrmr1993]

!approved-for-mainnet