Upgrade to hyper-rustls 0.27/rustls 0.23 #2225

djc · 2024-03-29T14:23:35Z

Sticks with ring as the default crypto provider for rustls for now (see #2136).

This is also a prerequisite for re-enabling a Quinn-based H3 backend, since Quinn is skipping rustls 0.22.

seanmonstar · 2024-04-01T13:24:04Z

Thanks so much! I do have some questions, to help plan how we could allow using the different crypto providers.

If no provider is enabled, what happens? I tried removing "ring" from the dependencies in this PR locally, and it still builds...
- Oh I see, I just ran the simple example, it panics at runtime.
I suppose we could introduce rustls-tls-ring and rustls-tls-awslc features eventually. We don't need to do so yet, but is there anything to take into consideration?

djc · 2024-04-02T05:47:58Z

Thanks so much! I do have some questions, to help plan how we could allow using the different crypto providers.

If no provider is enabled, what happens? I tried removing "ring" from the dependencies in this PR locally, and it still builds...

Oh I see, I just ran the simple example, it panics at runtime.

Yes. To be safe at compile time, we could use the builder_with_provider() API and pass rustls::crypto::ring::default_provider() explicitly instead.

I suppose we could introduce rustls-tls-ring and rustls-tls-awslc features eventually. We don't need to do so yet, but is there anything to take into consideration?

Those features would be non-additive, because rustls will panic if both are enabled. Also note that people might want to use a third-party crypto provider -- this is of course already possible via the preconfigured_tls() path.

seanmonstar · 2024-04-02T14:32:26Z

Those features would be non-additive, because rustls will panic if both are enabled.

Oh, so reqwest would be picking the crypto provider for the application as a whole? If any other library depends on rustls and doesn't disable the default features, will the application start to see panics when they upgrade reqwest?

djc · 2024-04-02T15:47:00Z

The ClientConfig::builder() indirectly calls into here:

https://github.com/rustls/rustls/blob/main/rustls/src/crypto/mod.rs#L254

This will:

If a process-wide crypto provider has already been installed, use it
If no provider has been installed and only one has been enabled via Cargo features, use it and install it as the default
Otherwise, panic

As such, I think the idea is that reqwest should rely on this and if people run into a run-time panic they should call CryptoProvider::install_default() first before setting up reqwest's Client/ClientBuilder.

See more discussion in rustls/rustls#1877.

NobodyXu · 2024-04-02T23:30:30Z

Won't it be more reasonable, to prefer one crypto to another instead of panicking at runtime?

Or maybe a compile_error! is also better than runtime panic.

djc · 2024-04-03T05:08:38Z

Feel free to contribute to the discussion in the upstream issue.

stormshield-gt · 2024-04-04T10:10:40Z

I would like to suggest a change to be able to setup a custom provider without having to compile both ring and the custom provider.

do not activate the ring feature on hyper-rustls, rustls and tokio-rustls
activate the ring feature only if rustls-tls feature is enabled. rustls-tls = ["rustls-tls-webpki-roots", "rustls/ring"]
optionally, add another feature that uses rustls but don't activate the ring feature. rustls-tls-providerless = ["rustls-tls-webpki-roots"]. This is optional as users could use the rustls-tls-webpki-roots directly

djc · 2024-04-04T10:16:18Z

FWIW, this PR was just intended to do the rustls upgrade while making as few other changes as possible. Obviously there could be a number of new Cargo features that help control the crypto provider used by rustls. But I think reqwest will want to pick a default crypto provider to make adopting rustls as easy as possible.

stormshield-gt · 2024-04-04T10:22:35Z

My suggested changes still pick up ring as a default provider without adding any new feature, and leaving the possibility to control the crypto provider used by rustls. That's why I think it's interesting to consider this now. This can also be, of course, in a follow-up PR as you suggest.

seanmonstar · 2024-04-16T14:39:05Z

I do want this to merge. As to the status of this upgrade: I'm hesitant to merge and publish since anyone upgrading that was already using rustls v0.23 may suddenly start seeing a panic, because reqwest enables the alternative backend. (And I don't really want to "fix" that by switching to aws-lc.)

My feeling is that I'd want to wait for a rustls version that doesn't have that issue. Discussion is in rustls/rustls#1877.

djc · 2024-04-22T12:55:36Z

@seanmonstar I think we can work around that issue by using ClientConfig::builder_with_provider(), explicitly passing whichever provider we have selected into the config builder explicitly. This should never panic, independent of which Cargo features other libraries have selected. Does that sound like a suitable mechanism for now?

I feel like a solution to rustls/rustls#1877 is unlikely to happen soon.

djc · 2024-05-12T13:00:42Z

I've revised this to explicitly select a ring (using the existing Cargo features) or aws-lc-rs (using a parallel set of Cargo features) provider and use the selected crypto provider explicitly when constructing a rustls ClientConfig. This setup should also enable preventing a dependency on either ring or aws-lc-rs by using a new rustls-base feature (renamed from __rustls) -- the latest release depends on ring (via rustls) unconditionally so even though one could wire up a preconfigured rustls ClientConfig using aws-lc-rs or another crypto provider, there would still be a ring dependency.

This is motivated in part by rustup concerns for making rustls the default TLS implementation, since some scenarios need P521 cipher suites which only aws-lc-rs supports.

djc · 2024-05-12T13:05:00Z

So the MSRV build fails with this error:

error: Package reqwest v0.12.4 (/Users/djc/src/reqwest) does not have feature rustls. It has an optional dependency > with that name, but that dependency uses the "dep:" syntax in the features table, so it does not have an implicit feature > with that name.

Not entirely sure what's up with that? From looking at the state of Cargo.toml after these changes I don't think there are any references to a rustls feature left?

(Seems to work okay with current stable.)

asonix · 2024-05-12T16:00:57Z

fwiw just running cargo c on this branch with rust 1.63 results in the same error (ubuntu 24.04 container)

asonix · 2024-05-12T16:02:48Z

Cargo.toml

+__rustls_crypto_ring = ["rustls-base", "rustls/ring"]
+__rustls_crypto_aws_lc = ["rustls-base", "rustls/aws_lc_rs"]


Suggested change

__rustls_crypto_ring = ["rustls-base", "rustls/ring"]

__rustls_crypto_aws_lc = ["rustls-base", "rustls/aws_lc_rs"]

__rustls_crypto_ring = ["rustls-base", "rustls?/ring"]

__rustls_crypto_aws_lc = ["rustls-base", "rustls?/aws_lc_rs"]

a few well-placed questionmarks seem to allow this to work on 1.63

asonix · 2024-05-12T16:05:57Z

hyper-rustls 0.27.1 also bumps msrv from 1.63 to 1.64

asonix · 2024-05-12T16:09:54Z

src/async_impl/client.rs

-                #[cfg(feature = "__rustls")]
+                #[cfg(any(
+                    feature = "__rustls_crypto_ring",
+                    feature = "__rustls_crypto_aws_lc-rs"


Suggested change

feature = "__rustls_crypto_aws_lc-rs"

feature = "__rustls_crypto_aws_lc"

Feature name is wrong

asonix · 2024-05-12T16:10:14Z

src/async_impl/client.rs

+                    let provider = rustls::crypto::ring::default_provider();
+
+                    #[cfg(all(
+                        feature = "__rustls_crypto_aws_lc-rs",


Suggested change

feature = "__rustls_crypto_aws_lc-rs",

feature = "__rustls_crypto_aws_lc",

feature name is wrong here too

asonix · 2024-05-12T16:10:29Z

src/async_impl/client.rs

-    #[cfg_attr(docsrs, doc(cfg(feature = "rustls-tls")))]
+    #[cfg(any(
+        feature = "__rustls_crypto_ring",
+        feature = "__rustls_crypto_aws_lc-rs"


Suggested change

feature = "__rustls_crypto_aws_lc-rs"

feature = "__rustls_crypto_aws_lc"

and here

asonix · 2024-05-12T16:10:44Z

src/async_impl/client.rs

+        docsrs,
+        doc(cfg(any(
+            feature = "__rustls_crypto_ring",
+            feature = "__rustls_crypto_aws_lc-rs"


Suggested change

feature = "__rustls_crypto_aws_lc-rs"

feature = "__rustls_crypto_aws_lc"

here

asonix · 2024-05-12T16:11:07Z

src/tls.rs

-    #[cfg(feature = "__rustls")]
+    #[cfg(any(
+        feature = "__rustls_crypto_ring",
+        feature = "__rustls_crypto_aws_lc-rs"


Suggested change

feature = "__rustls_crypto_aws_lc-rs"

feature = "__rustls_crypto_aws_lc"

at this point it's probably easier to add -rs to Cargo.toml

actually I'd suggest changing this just to rustls-base since it doesn't compile with only that feature otherwise

Either that or gate the Debug impl behind this same cfg(any())

asonix · 2024-05-12T16:32:30Z

From my look into this PR, I think TlsBackend can't have a Default impl if only rustls-base is enabled, since the TlsBackend::Rustls variant indicates that either aws-lc-rs or ring has been selected, and requires the call to default_provider() for one of the backends.

In that scenario it is required that a user pass in a a rustls-compatible clientconfig so that the BuildRustls variant is set on the client.

Would it make sense to set UnknownPreconfigured as the default in that case? That could lead to confusing error messages, though, so maybe a new variant UnsetRustls or something could be created in order to tell users that rustls was enabled but no backend was configured

asonix · 2024-05-12T16:35:34Z

I think these compile issues could have been caught by CI if the new features were added to the github actions workflow, also

djc changed the title ~~Upgrade to hyper-rustls 0.23~~ Upgrade to hyper-rustls 0.27/rustls 0.23 Mar 29, 2024

tmpfs mentioned this pull request Apr 8, 2024

Update dependencies (rustls and axum-server) n0-computer/tokio-rustls-acme#5

Open

aumetra mentioned this pull request Apr 8, 2024

Upgrade to Hyper 1.0 & Axum 0.7 hyperium/tonic#1670

Open

17 tasks

This comment was marked as off-topic.

Sign in to view

charliermarsh mentioned this pull request Apr 23, 2024

Update Rust crate rustls to 0.23.0 astral-sh/uv#2884

Closed

1 task

dignifiedquire mentioned this pull request May 8, 2024

upgrade to quinn@0.11 & rustls@0.23 n0-computer/iroh#2273

Open

Alvenix mentioned this pull request May 8, 2024

Feature Request: Avoid panicking when ring and aws_lc_rs are both specified rustls/rustls#1877

Open

1 task

rami3l mentioned this pull request May 11, 2024

Lack of support for p521 signatures with the ring-based reqwest/rustls backend rust-lang/rustup#3820

Open

djc force-pushed the rustls-0.23 branch 2 times, most recently from 0b8a99a to b9240eb Compare May 12, 2024 12:43

Upgrade to hyper-rustls 0.27 and rustls 0.23

bd909b8

djc force-pushed the rustls-0.23 branch from b9240eb to bd909b8 Compare May 12, 2024 12:53

asonix reviewed May 12, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade to hyper-rustls 0.27/rustls 0.23 #2225

Upgrade to hyper-rustls 0.27/rustls 0.23 #2225

djc commented Mar 29, 2024 •

edited

seanmonstar commented Apr 1, 2024 •

edited

djc commented Apr 2, 2024 •

edited

seanmonstar commented Apr 2, 2024

djc commented Apr 2, 2024

NobodyXu commented Apr 2, 2024

djc commented Apr 3, 2024

stormshield-gt commented Apr 4, 2024

djc commented Apr 4, 2024

stormshield-gt commented Apr 4, 2024

This comment was marked as off-topic.

seanmonstar commented Apr 16, 2024

djc commented Apr 22, 2024 •

edited

djc commented May 12, 2024 •

edited

djc commented May 12, 2024 •

edited

asonix commented May 12, 2024

asonix May 12, 2024

asonix commented May 12, 2024

asonix May 12, 2024

asonix May 12, 2024

asonix May 12, 2024

asonix May 12, 2024

asonix May 12, 2024

asonix May 12, 2024

asonix May 12, 2024

asonix commented May 12, 2024

asonix commented May 12, 2024

		__rustls_crypto_ring = ["rustls-base", "rustls/ring"]
		__rustls_crypto_aws_lc = ["rustls-base", "rustls/aws_lc_rs"]

	feature = "__rustls_crypto_aws_lc-rs"
	feature = "__rustls_crypto_aws_lc"

	feature = "__rustls_crypto_aws_lc-rs",
	feature = "__rustls_crypto_aws_lc",

Upgrade to hyper-rustls 0.27/rustls 0.23 #2225

Are you sure you want to change the base?

Upgrade to hyper-rustls 0.27/rustls 0.23 #2225

Conversation

djc commented Mar 29, 2024 • edited

seanmonstar commented Apr 1, 2024 • edited

djc commented Apr 2, 2024 • edited

seanmonstar commented Apr 2, 2024

djc commented Apr 2, 2024

NobodyXu commented Apr 2, 2024

djc commented Apr 3, 2024

stormshield-gt commented Apr 4, 2024

djc commented Apr 4, 2024

stormshield-gt commented Apr 4, 2024

This comment was marked as off-topic.

seanmonstar commented Apr 16, 2024

djc commented Apr 22, 2024 • edited

djc commented May 12, 2024 • edited

djc commented May 12, 2024 • edited

asonix commented May 12, 2024

Choose a reason for hiding this comment

asonix commented May 12, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

asonix commented May 12, 2024

asonix commented May 12, 2024

djc commented Mar 29, 2024 •

edited

seanmonstar commented Apr 1, 2024 •

edited

djc commented Apr 2, 2024 •

edited

djc commented Apr 22, 2024 •

edited

djc commented May 12, 2024 •

edited

djc commented May 12, 2024 •

edited