client-side encrypted client hello (ECH) support #485
Conversation
The MSRV tests may have shaken out now? The other option is to hit cloudflare's DNS API? like |
I would be minded to minimise this API as much as possible. I haven't fully thought this through, but how about:
(I had a quick look at curl's ECH docs, there is no way of specifying any HPKE suite, or seeing what suite is used AFAIK) |
Yup!
I gave this more thought and realized that it's beneficial to do as much lifting as possible in Rust code because it's cross-platform and avoids me having to fight through Powershell on Windows ( 🤢 ). I think the Hickory DNS helper util is probably the best option with that in mind.
This was helpful input. I implemented the API you suggested and it was much simpler. I collapsed everything into one commit as a result. PTAL.
I figured this out and wrote an update in the PR description. TLDR: the existing |
I was able to reproduce the problem upstream in Rustls building the ech client example with 1.71 and 1.72. I reported the ICE to the Rust team here: rust-lang/rust#133578 We can either block on a resolution for this upstream, land a |
cpu
force-pushed
the
cpu-client-ech
branch
3 times, most recently
from
4da2330 to
2f47234
Compare
The upstream Rust project isn't interested in fixing bugs in old compilers (sensible!) so I think we're left with ~two options:
I vote for the latter option and have pushed a corresponding update. |
cpu
changed the title
We can't derive `Default` on `ClientConfigBuilder` because we want to enable SNI by default. Instead, implement it by hand. This allows simplifying the construction sites to use the default except for the ~1/2 fields needing to be customized. Along the way, change where we initialize the default protocol versions. Previously we did this in `rustls_client_config_builder_new()`, but when we introduce ECH config it will be beneficial to know if the user has customized protocol versions using `rustls_client_config_builder_new_custom()` or just wants sensible defaults. This is easier to deduce if we defer populating the default protocol values to the time of `rustls_client_config_builder_build()` when we find the builder's protocol versions vec to be empty, indicating defaults are desired (explicitly customizing configuration for no protocols makes no sense).
Previously there were three. Let's reduce to one.
This commit adds initial support for customizing a `rustls_client_config_builder` instance to use encrypted client hello (ECH). Presently this is only supported with the `aws-lc-rs` crypto provider. In sum it adds: * New `rustls_hpke` type for representing a collection of HPKE suites as an opaque struct. * New `rustls_supported_hpke()` function for returning a const pointer to a `rustls_hpke` if available. With the aws-lc-rs backend this is a wrapper around `rustls::crypto::aws_lc_rs::hpke::ALL_SUPPORTED_SUITES`. For other providers it returns NULL. * New `rustls_client_config_builder_enable_ech_grease` client config builder function for configuring ECH GREASE using a `rustls_hpke`. This chooses the HPKE suite at random, and so brings in a dep on the `rand` crate. * New `rustls_client_config_builder_enable_ech` client config builder function for configuring ECH using a `rustls_hpke` and a TLS encoded ECH config. Additionally the `client.c` example is updated to add `RUSTLS_ECH_GREASE` and `RUSTLS_ECH_CONFIG` env vars. The former when non-zero configures ECH GREASE if possible, the latter configures ECH with the TLS encoded config list read from the provided file path. To facilitate a reliable cross-platform way to fetch ECH configs a small `ech-fetch` test program is added that accepts a domain and (optionally) a file name and writes the TLS encoded ECH config for that domain to the file after performing the appropriate DNS-over-HTTPS lookup. This requires a new dev-only dep on HickoryDNS and Tokio.
1.71 and 1.72 both hit an ICE building with a static reference to Rustls' aws-lc-rs HPKE suites. Upstream (sensibly) isn't interested in fixing bugs in old compilers. We also don't want to bump the upstream Rustls MSRV for niche errors, or add excessive workarounds. So: let's do the easy thing and increase rustls-ffi's MSRV to 1.73. The rustls-ffi project has few downstream consumers at this point and so is more agile for MSRV bumps than the core crate. Along the way, match the formatter to the MSRV. It had fallen behind.
|
I think this is self-contained enough and has been up for review long enough that I'm comfortable merging with just Ctz's review. Happy to iterate further if there's any new feedback. I don't plan to cut a release tag for a bit so there's still time to adjust.
Going to admin merge this and then fix the MSRV in the branch rules (it's now 1.73) |
Done. |
Client ECH support
This branch adds initial support for customizing a
rustls_client_config_builderinstance to use encrypted client hello(ECH). Presently this is only supported with the
aws-lc-rscrypto provider.In sum it adds:
rustls_hpketype for representing a collection of HPKE suites as an opaque struct.rustls_supported_hpke()function for returning a const pointer to arustls_hpkeif available. With the aws-lc-rs backend this is a wrapper aroundrustls::crypto::aws_lc_rs::hpke::ALL_SUPPORTED_SUITES. For other providers it returns NULL.rustls_client_config_builder_enable_ech_greaseclient config builder function for configuring ECH GREASE using arustls_hpke. This chooses the HPKE suite at random, and so brings in a dep on therandcrate. We could come up with a simpler selection mechanism if avoiding the dependency is preferred.rustls_client_config_builder_enable_echclient config builder function for configuring ECH using arustls_hpkeand a TLS encoded ECH config.Additionally the
client.cexample is updated to addRUSTLS_ECH_GREASEandRUSTLS_ECH_CONFIGenv vars. The former when non-zero configures ECH GREASE if possible, the latter configures ECH with the TLS encoded config list read from the provided file path.To facilitate a reliable cross-platform way to fetch ECH configs a small
ech-fetchtest program is added that accepts a domain and (optionally) a file name and writes the TLS encoded ECH config for that domain to the file after performing the appropriate DNS-over-HTTPS lookup. This requires a new dev-only dep on HickoryDNS and Tokio.Testing
Test locally with:
You should see output that includes:
You may have to run this a few times to avoid an error of the form: "content length header not found". The reason for this is that the existing
client.cHTTP handling is somewhat primitive and can't handle chunked responses like this server generates.The existing client code only successfully handles an HTTP response if either:
send_request_and_read_response. In this case we dump the plaintext to stdout without processing any HTTP headers and so it "works" with chunked encoding responses (mostly by accident).defo.ieandresearch.cloudflare.com) because both use chunked encoding and so never send a content length header.This buggy handling can be replicated without ECH under the right conditions, but with ECH, it's much more frequent that we miss the happy accident case (number 1 above.).
Because of this I've not added CI coverage of running the test client. I think it makes more sense to land this as-is and break out a separate refactoring of the client.c HTTP handling. The code is a little bit messy and could use a more thoughtful rework (maybe as a state machine?).
TODO
Add CI coverage for client.c test w/ ECH - blocked on makingTracking in Add CI coverage for ECH client #498client.creliable in more circumstances.