Hybrid post-quantum cryptography and TLS 1.3.

Enabling a hybrid group is a configuration change. Establishing that clients actually negotiate it, authenticate the intended server, and tolerate its transport costs is an engineering exercise with several separate measurements — and "PQC enabled" is not one of them. This chapter uses RFC 10024's three standardized ML-KEM hybrid groups to work through exact wire costs and component ordering, where the combined secret enters the TLS key schedule, HelloRetryRequest's cost, and closes with a reproducible local OpenSSL lab whose real handshake output — including an incompatible-group rejection test — is reported below.

1. Standards checkpoint and learning goals

This chapter uses RFC 10024, published August 2026 on the Standards Track, for three ML-KEM hybrid groups. RFC 9954 provides the Informational hybrid-construction framework. The current TLS 1.3 specification is RFC 9846, which obsoletes RFC 8446 while retaining the "TLS 1.3" version number. These statuses were checked as of this writing.

You should already understand ECDHE, KEMs, transcript authentication, HKDF, and authenticated encryption from Parts 2 and 7. The goal here is to connect those concepts to actual bytes and observable protocol behavior. We distinguish group negotiation from certificate signatures, calculate share overhead exactly, examine HelloRetryRequest, and run an authenticated local exchange using native OpenSSL 3.5.8.

2. Why combine classical and post-quantum mechanisms?

A hybrid exchange combines independently generated contributions from a traditional mechanism and a post-quantum mechanism through a specified construction. The intended security property is preservation of shared-secret security when at least one component remains secure, subject to the construction's assumptions and a correct implementation. This reduces dependence on either component alone during migration (Part 5 §10).

The claim has real boundaries. If an attacker steals both ephemeral private states from an endpoint, the combiner cannot restore secrecy. If a shared random generator is compromised, both contributions may be affected simultaneously. If the protocol fails to authenticate its negotiation, an attacker may change what the peers think they selected. Hybrid cryptography does not remove the endpoint-security and composition parts of the threat model — it addresses one specific failure mode, not every failure mode.

A standardized hybrid group packages the component encodings and shared-secret combination into one negotiated identifier. It does not negotiate two unrelated groups and hope the application combines them correctly on its own. The exact order, lengths, validation rules, and TLS key-schedule integration are all defined by the profile — this is exactly the "combiner needs its own security argument" point from Part 5 §10, made concrete.

3. Three groups and their wire costs

The following sizes describe the key_exchange payload inside a KeyShareEntry. They exclude the group identifier, length fields, extension framing, other ClientHello fields, TLS records, and transport headers.

Group	Code point	Client share	Server share	Combined secret
X25519MLKEM768	4588 / 0x11EC	1,216 bytes	1,120 bytes	64 bytes
SecP256r1MLKEM768	4587 / 0x11EB	1,249 bytes	1,153 bytes	64 bytes
SecP384r1MLKEM1024	4589 / 0x11ED	1,665 bytes	1,665 bytes	80 bytes

For X25519MLKEM768, the client sends the 1,184-byte ML-KEM-768 encapsulation key followed by a 32-byte X25519 share. The server returns the 1,088-byte ML-KEM ciphertext followed by its 32-byte X25519 share. The combined secret places the 32-byte ML-KEM output first, followed by the 32-byte X25519 secret.

4. Registry recommendation and obsolete experiments

The IANA TLS registry marks X25519MLKEM768 Recommended-Y and the two NIST-curve hybrids Recommended-N at this checkpoint. All three are standardized by RFC 10024. A registry recommendation flag is distinct from whether a mechanism has a standard, whether it's appropriate under a particular policy, or whether its cryptography is broken — these are three separate facts often conflated into one green checkmark.

RFC 10024 also addresses obsolete experimental groups, including X25519Kyber768Draft00 at 0x6399 and SecP256r1Kyber768Draft00 at 0x639A. Those names refer to earlier, pre-standard Kyber-based experiments (Part 1 §7). They should not be presented as alternate spellings for the final ML-KEM groups, or retained as an undocumented fallback after migration completes.

Inventory systems should record both the human-readable group name and numeric code point. This helps distinguish old telemetry labels, draft experiments, and standardized deployments from each other. A dashboard that normalizes every name containing "Kyber" or "MLKEM" into one green "PQC" status can conceal a real interoperability or policy gap sitting right underneath it. See the IANA TLS registry.

5. Where the secret enters the TLS key schedule

TLS 1.3 separates the cipher suite from key establishment and signature algorithms entirely. A suite such as TLS_AES_256_GCM_SHA384 identifies record protection and the associated hash — it does not tell you whether the handshake used X25519 or a hybrid group at all. Group and signature telemetry are two separate fields, and conflating them in logging misreports the connection's actual PQC status.

Conceptually, HKDF-Extract combines the negotiated key-establishment secret with a derived value from the early stage of the schedule (Part 2 §7). Transcript-bound HKDF expansions derive client and server handshake traffic secrets. Later stages derive application traffic secrets and other purpose-specific outputs. The combined hybrid secret occupies the key-establishment input defined by the protocol — it is never used directly as an AES key.

6. The hybrid group protects key establishment, not authentication

The transcript binds negotiation and handshake messages to authentication. CertificateVerify proves possession of the server's signing key over the required context and transcript, while Finished authenticates the transcript under a derived handshake secret. They serve genuinely different purposes, and both need the surrounding certificate and identity checks to be meaningful at all (Part 2 §11).

The hybrid group protects key establishment specifically. A server may still authenticate with a classical ECDSA certificate. That deployment can improve resistance to passive harvest-now-decrypt-later attacks (Part 1 §5) without providing any post-quantum signature authentication against a future active attacker. Report these two properties separately — never call the entire connection "fully quantum safe" when only key establishment is upgraded.

7. Normal handshake and HelloRetryRequest

Client                                           Server
ClientHello:
  supported_groups includes hybrid
  key_share contains hybrid share ---------------->
                                      validate encapsulation key
                                      compute ECDHE + encapsulate
                         <-------------- ServerHello hybrid share
                         <-------------- encrypted authentication
validate certificate and hostname
compute ECDHE + decapsulate
verify transcript authentication
Finished --------------------------------------->
application data <-----------------------------> application data

A client can advertise support for a group without sending its share in the first ClientHello. Sending every possible share increases initial bytes and computation for every connection. If the server selects a supported group for which the client hasn't supplied a share, it can use HelloRetryRequest to ask for the needed group, within the protocol's constraints.

The retry adds another client flight and usually another full round trip. That may be inexpensive on a local network and significant on a mobile or satellite path. The best initial-share strategy therefore depends on server selection behavior and client population — measure actual retry frequency instead of assuming that advertising support eliminates handshake cost.

8. Validation, implicit rejection, and downgrade policy

The server validates the ML-KEM encapsulation key as required by FIPS 203 and the TLS profile together. Both sides validate the classical contribution, including X25519's all-zero-secret check where applicable. The client checks ML-KEM ciphertext length and handles decapsulation errors through the specified TLS behavior. Every component's checks remain necessary in a hybrid exchange — a hybrid group doesn't relax any single component's validation requirements.

ML-KEM implicit rejection still applies exactly as in Part 7 §7: a same-length corrupted ciphertext may yield fallback secret material, causing subsequent handshake authentication to fail downstream rather than an immediate visible error. That differs from a malformed ciphertext length or a direct API failure. The TLS implementation must map errors correctly without exposing internal validity information through ad hoc diagnostics.

A supported-groups list is not necessarily a minimum-security policy. If a service permits classical fallback for compatibility, successful connections may still end up classical. Conversely, requiring a hybrid group can reject clients that lack support entirely. Define the intended behavior for each audience explicitly, measure it, and make fallback visible in telemetry rather than hidden inside an aggregate success rate.

An attacker who tampers with a correctly authenticated handshake cannot simply edit the negotiated group without detection. However, application retry logic can create a separate downgrade path if a failed connection silently triggers a new attempt with weaker settings. The security review must include that outer retry policy, load balancers, proxies, and client wrappers — not just the TLS library itself.

9. Larger handshakes and the transport path

Compared with a 32-byte X25519 share, the hybrid client's 1,216-byte share adds 1,184 bytes before framing. That is not the same as adding exactly one network packet. ClientHello already includes extensions, names, signature algorithms, and other data; TCP segmentation and path MTU determine actual packetization, not the payload size alone.

TLS records, TCP segments, IP fragmentation, and QUIC packets are all different boundaries. A large TLS handshake message can span multiple records and segments without requiring IP fragmentation at all. QUIC carries TLS handshake data in its own framing and has different amplification and retransmission considerations entirely. Never transfer a packet-count claim from one transport to another without actually measuring it on that transport.

Middleboxes may mishandle unfamiliar groups, large ClientHello messages, or unusual segmentation despite fully valid protocol behavior. Test enterprise inspection paths, older proxies, VPNs, mobile networks, and origin connections where relevant. An edge endpoint's success does not establish that an internal service mesh or outbound SDK negotiates the same protection — check each hop independently.

10. Executed local TLS lab

This script uses OpenSSL s_server and s_client. It binds only to loopback, generates a disposable ECDSA certificate, explicitly trusts that certificate for this experiment, and verifies the localhost identity. It also checks that a classical-only client cannot negotiate with the hybrid-only server — the negative test matters as much as the positive one.

#!/usr/bin/env bash
set -euo pipefail
OPENSSL=${OPENSSL:-openssl}
LAB=$(mktemp -d)
SERVER_PID=''
cleanup() { if [ -n "$SERVER_PID" ]; then kill "$SERVER_PID" 2>/dev/null || true; wait "$SERVER_PID" 2>/dev/null || true; fi; rm -rf "$LAB"; }
trap cleanup EXIT
umask 077
cd "$LAB"
"$OPENSSL" version
PORT=$(python3 - <<'PY'
import socket
with socket.socket() as s:
 s.bind(('127.0.0.1',0)); print(s.getsockname()[1])
PY
)
"$OPENSSL" req -config /dev/null -x509 -newkey ec -pkeyopt ec_paramgen_curve:P-256 -noenc -keyout key.pem -out cert.pem -days 1 -subj /CN=localhost -addext subjectAltName=DNS:localhost > /dev/null 2>&1
"$OPENSSL" s_server -accept "127.0.0.1:$PORT" -cert cert.pem -key key.pem -tls1_3 -groups X25519MLKEM768 -www > server.log 2>&1 &
SERVER_PID=$!
python3 - "$PORT" <<'PY'
import socket,sys,time
for _ in range(50):
 try:
  with socket.create_connection(('127.0.0.1',int(sys.argv[1])),timeout=.2): break
 except OSError: time.sleep(.1)
else: raise SystemExit('server did not become ready')
PY
printf 'GET / HTTP/1.0\r\n\r\n' | "$OPENSSL" s_client -connect "127.0.0.1:$PORT" -servername localhost -verify_hostname localhost -verify_return_error -CAfile cert.pem -tls1_3 -groups X25519MLKEM768 > client.log 2>&1
python3 - <<'PY'
from pathlib import Path
s=Path('client.log').read_text()
assert 'X25519MLKEM768' in s
assert 'Verify return code: 0 (ok)' in s
for line in s.splitlines():
 if any(x in line for x in ('Negotiated TLS1.3 group','Server Temp Key','Verify return code','SSL handshake has read')): print(line)
PY
if printf '' | "$OPENSSL" s_client -connect "127.0.0.1:$PORT" -servername localhost -CAfile cert.pem -tls1_3 -groups X25519 > negative.log 2>&1; then
 echo 'Unexpected classical-only negotiation' >&2; exit 1
fi
printf '%s\n' 'PASS: authenticated local hybrid TLS and incompatible-group rejection'

The certificate is classical by design here, so this lab demonstrates hybrid key establishment with classical authentication — exactly the intermediate architecture flagged in §6. Part 12 examines signature and certificate migration separately. Using a self-signed certificate here is acceptable because the client explicitly provisions that exact trust anchor for a local experiment and still checks the hostname (never skip that check even in a lab).

11. Extending the experiment responsibly

Repeat the exchange with the P-256 and P-384 hybrid groups if your exact binary supports them. Record the negotiated identifier and verification result rather than accepting process exit alone as proof. Then use a server configuration with multiple groups to observe the effect of client ordering and initial-share choices documented by that library version.

For packet analysis, capture only your own loopback lab traffic and identify the supported_groups and key_share extensions. Packet analyzers may need current dissectors to display new names; a numeric code point can still establish the group even without one. A display filter that recognizes only an obsolete experimental identifier can produce a false "no PQC" conclusion on a perfectly correct connection.

Do not publish TLS key-log files or reuse lab keys in production. Key logs permit decryption of the captured session and belong only in controlled debugging workflows. Many useful measurements — including cleartext ClientHello/ServerHello share sizes — do not require collecting traffic secrets at all.

12. A rollout metric that does not overstate coverage

Define hybrid coverage as the fraction of eligible fresh handshakes that actually negotiate an approved hybrid group. State the denominator explicitly: all connections, capable clients, or a selected canary population produce very different percentages from the same numerator. Record failed attempts separately, since excluding them can make a problematic rollout look artificially more successful.

Track edge and origin connections independently. A browser may use hybrid key establishment to a CDN while the CDN uses classical TLS to the origin server behind it. Both links may have valid encryption and authentication, but the harvest-now-decrypt-later exposure differs between them — a diagram of termination points makes the residual classical path visible instead of hidden inside "TLS enabled everywhere."

A useful rollback threshold combines availability with security policy. For example, a canary may pause expansion when a particular client family experiences an unexplained increase in failures. The response should preserve logs of negotiated identifiers and public error categories, compare against the baseline, and identify whether the issue is capability, framing, or certificate processing — silently removing the hybrid requirement everywhere loses that evidence and changes the protection level for everyone, not just the affected cohort.

Does a hybrid TLS group make a connection "fully post-quantum"?

No (§6). It protects key establishment against harvest-now-decrypt-later. If the server still authenticates with a classical certificate, the connection's authentication remains dependent on classical public-key security against a future active attacker — report the two properties separately.

Can I safely concatenate ECDHE and ML-KEM shares in alphabetical order?

No (§3). X25519MLKEM768 puts ML-KEM first; the two NIST-curve hybrids put ECDHE first. The group name alone is not a safe encoding spec — use a maintained library's implementation rather than rebuilding the combiner.

Does advertising hybrid-group support eliminate the extra round trip from HelloRetryRequest?

Not necessarily (§7). If the client doesn't send its share in the first ClientHello and the server needs it, a retry still happens. The best initial-share strategy depends on server behavior and client population — measure actual retry frequency rather than assuming support alone is enough.

Takeaways

• The three RFC 10024 hybrid groups have different component orderings. X25519MLKEM768 puts ML-KEM first; the P-256/P-384 groups put ECDHE first — never assume a uniform concatenation rule.
• A hybrid group protects key establishment only — a classical certificate still leaves authentication dependent on classical assumptions against a future active attacker (Part 12 covers PKI migration).
• Cipher suite ≠ key-establishment group. Log and measure them as separate fields, or "PQC enabled" telemetry silently misreports the connection.
• Obsolete Kyber-draft code points are not alternate spellings for the final ML-KEM groups — record both human-readable name and numeric code point to catch these in inventory.
• ML-KEM implicit rejection (Part 7 §7) still applies inside TLS — a corrupted-but-same-length ciphertext fails downstream at Finished, not immediately, and shouldn't be conflated with a malformed-input error.
• The reproducible lab confirmed real behavior on real hardware: successful X25519MLKEM768 negotiation with hostname-verified certificate, plus a verified rejection of a classical-only client against the hybrid-only server.

References & further reading

• RFC 10024 — the primary standard: three ML-KEM hybrid groups, §3–5.
• RFC 9954 — hybrid-construction framework, §2.
• RFC 9846 — current TLS 1.3 base specification.
• IANA TLS registry — §4, code points and recommendation flags.
• OpenSSL s_server · s_client — §10, the lab commands.