Performance is a property of an operation in an environment. A signature size comes from a format; signatures per second come from a particular implementation and workload. Keeping those two claims separate is the first step toward a useful comparison — and it's exactly where most "PQC is slow" or "PQC is fast" headlines fall apart. This chapter separates bytes, CPU work, and elapsed time; runs a real OpenSSL 3.5.8 benchmark on an Apple M4 Pro with the actual measured numbers reported below; works through capacity and bandwidth math; and closes with a benchmark protocol built specifically to prevent the comparisons that mislead.
1. The questions worth answering
This chapter turns the algorithms and protocol flows from earlier articles into a measurement plan. You'll separate fixed byte counts from implementation costs, distinguish primitive throughput from service latency, and interpret a small executed OpenSSL benchmark without promoting it into a universal ranking it can't support.
Start with a concrete question: can a firmware verifier fit its stack budget, can a gateway sustain its fresh-handshake rate, or can an offline authority finish a release-signing batch within a maintenance window? Each question emphasizes different measurements. "Which PQC algorithm is fastest?" is incomplete without naming the purpose and the exact operation.
2. Separate bytes, work, and elapsed time
A raw public key, ciphertext, or signature has a defined encoding length for the finalized parameter set. DER and PEM add framing or representation overhead on top. A certificate chain adds signatures, keys, names, extensions, and constraints (Part 12 §5). A TLS handshake adds protocol framing and other messages (Part 11 §9). Network traffic adds retransmissions and transport headers on top of that.
CPU cycles measure processor work under a specified counter model. Elapsed time includes scheduling and waits. Throughput measures completed operations per unit time under a workload. Tail latency describes slow requests, which can matter even when average throughput looks perfectly acceptable. None of these five measurements is interchangeable with another.
| Measurement | Appropriate unit | Main question |
|---|---|---|
| Encoded object | bytes | Does the format fit storage and transport limits? |
| Primitive operation | cycles or microseconds | What does the selected implementation cost? |
| Service capacity | operations/handshakes per second | How much work can the service sustain? |
| User-visible latency | median and tail milliseconds | What does the caller experience? |
| Working memory | peak stack/heap bytes | Does the implementation fit the target? |
| Energy | joules per operation | What does repeated use cost on a battery? |
Memory has several dimensions too: serialized key size, expanded key state, peak stack, heap allocation, working set, and concurrent per-request buffers. A 64-byte private-key encoding does not imply a 64-byte signing workspace (Part 9 §8) — hash-tree construction and polynomial arithmetic can require substantial temporary state well beyond the stored key.
3. A size comparison with purpose preserved
ML-KEM-768 has a 1,184-byte public key and 1,088-byte ciphertext (Part 7 §3). ML-DSA-65 has a 1,952-byte public key and 3,309-byte signature (Part 8 §3). SLH-DSA-SHA2-128s has a 32-byte public key and 7,856-byte signature (Part 9 §8). These are different purposes and categories entirely — placing them in one table does not make them substitutes for each other.
For finalized TLS X25519MLKEM768, the client share is 1,216 bytes and server share 1,120 bytes under RFC 10024 (Part 11 §3). Compared with two 32-byte X25519 shares, the share payload grows by 2,272 bytes across both directions combined — this excludes all other handshake and network overhead.
The Falcon project reports compact submission-era signatures, including 666 bytes for its Falcon-512 figure. The HQC project reports larger KEM key/ciphertext payloads than ML-KEM for its current encodings. Those observations motivate bandwidth studies, but the final FN-DSA and HQC profiles must be checked when available (Part 10 §2). Do not compare a cached server public key with an ephemeral client key sent every connection without accounting for reuse — a large once-provisioned key can be inexpensive per transaction, while a smaller key repeatedly transmitted may dominate aggregate traffic. The protocol's key lifetime determines how the format cost is actually amortized.
4. A reproducible local observation
The course ran OpenSSL 3.5.8 on an Apple M4 Pro with macOS 26.6.2, using the native default provider and a darwin64-arm64 build with the compiler's O3 optimization. The host was an ordinary active workstation — CPU affinity, fixed frequency, thermal state, and background activity were not controlled. The experiment is an exploratory measurement, not a publication-grade benchmark.
The command below uses the official speed interface. The one-second duration is per measured operation, not for the whole command. The elapsed option uses wall-clock time. Three runs were collected; the table below shows the first run explicitly rather than presenting a small sample as a stable population estimate.
OPENSSL=/path/to/openssl-3.5.8/bin/openssl
"$OPENSSL" version -a
for run in 1 2 3; do
"$OPENSSL" speed -elapsed -seconds 1 \
ML-KEM-768 ML-DSA-65 \
SLH-DSA-SHA2-128s SLH-DSA-SHA2-128f \
> "speed-$run.txt" 2>&1
done| Algorithm | Key gen/s | Encaps or sign/s | Decaps or verify/s |
|---|---|---|---|
| ML-KEM-768 | 43,525.0 | 76,567.0 encaps | 52,587.0 decaps |
| ML-DSA-65 | 8,042.0 | 1,633.7 sign | 9,200.0 verify |
| SLH-DSA-SHA2-128s | 38.2 | 5.1 sign | 4,878.0 verify |
| SLH-DSA-SHA2-128f | 2,483.2 | 105.0 sign | 1,691.0 verify |
5. What the benchmark excludes
openssl speed repeatedly exercises library operations inside one process. It does not measure a complete certificate-authority request, a network handshake, an HSM round trip, or application authorization. The reported rate should not be multiplied directly into a service-capacity promise without profiling those additional components separately.
The benchmark also does not establish latency percentiles for individual requests. A one-second aggregate can hide signing retries, scheduler interruptions, or occasional allocations. For latency analysis, use a harness that records per-operation durations, warms up deliberately, and reports a full distribution with enough independent samples to support the desired conclusion.
Message hashing and representation must be specified when comparing signature implementations. A benchmark's built-in message workload is not the same as signing a multi-gigabyte artifact. A system that hashes a canonical manifest before a remote signing request has different boundaries from one that streams the entire file through a signing API.
Inspect the benchmark source or documentation for key parsing, key generation, context setup, validation, and caching behavior. The mlkem-native benchmark page explicitly notes packed-key validation and the difference when applications retain expanded private material — such details can explain a performance difference without any algorithmic change at all. See the mlkem-native benchmark page.
6. A simple capacity calculation
Suppose a hypothetical service spends 0.2 milliseconds of CPU on a new cryptographic operation and receives 5,000 such operations per second. Multiplying gives one CPU-second of work per second, before any other service costs — this is a utilization estimate, not proof that one core can meet latency objectives at that rate.
Fig. 1 — A long-run average CPU-second-per-second calculation can look completely fine while tail latency is already unacceptable. Bursts, uneven scheduling, and slow dependencies matter more near capacity than the average does.
As utilization approaches capacity, queueing can increase sharply — this is standard queueing-theory behavior, not specific to PQC, but it's easy to forget when a benchmark table only shows averages. Bursts, uneven worker scheduling, and slow dependencies produce real tail latency even when long-run averages appear perfectly manageable. Provision headroom based on the complete workload and reliability target, not only the primitive's average execution time.
For bandwidth: an assumed extra 2,272 share bytes across 10,000 fresh handshakes per second gives about 22.72 million additional bytes per second before framing and retransmissions. This is a calculated scenario illustrating scaling — it is not a measured production rate. Resumption and connection reuse can change the number of fresh exchanges substantially, and a performance plan should include connection lifetime, fresh-handshake fraction, request concurrency, and burstiness together, not any one of them in isolation.
7. Network latency and handshake expansion
Larger ClientHello and certificate messages can change segmentation, retransmission cost, and middlebox behavior. The penalty depends on path latency, loss, congestion control, and the placement of handshake bytes in flights. A packet-count estimate based only on Ethernet MTU is insufficient for a heterogeneous internet path (Part 11 §9).
HelloRetryRequest adds a round trip when the initial share doesn't match the selected group (Part 11 §7). If a change improves CPU time but increases retry frequency, user-visible latency can actually worsen — measure group selection and retries together with handshake duration to locate the real cause, not just the CPU number in isolation.
Compare complete handshakes with the same certificate chain, trust checks, resumption configuration, and network conditions. Otherwise, a larger certificate or a disabled session cache can be mistaken for a KEM regression that isn't actually there. Record both client and server versions, since negotiation is inherently a two-party behavior.
8. Memory, caches, and concurrency
Expanded polynomial matrices and intermediate vectors can reduce repeated computation at the cost of memory. A cached key may improve single-operation speed while increasing the service's working set overall. Under high concurrency, cache pressure and memory bandwidth can change the ranking seen in an isolated, warm, single-threaded benchmark.
Measure peak stack on constrained devices, not just heap usage. Interrupt handlers and RTOS task stacks share a finite memory budget, and a cryptographic function that works fine in a desktop process can overflow a small task stack. Include worst-case paths and the compiler configuration actually used in firmware, not a desktop build's defaults.
9. Hardware acceleration and portability
AVX2 and AVX-512 can accelerate suitable operations on supported x86 processors; ARM NEON and RISC-V vector extensions enable other optimized backends. Availability of an instruction set does not prove the chosen library actually uses it, nor that it accelerates every phase of a KEM or signature equally.
Polynomial arithmetic, SHA-2, SHA-3/SHAKE, sampling, and encoding stress different hardware resources. A platform with fast SHA-2 instructions can behave very differently from one with an optimized SHAKE implementation. Wider vectors can also affect frequency, register pressure, or code size — measure the actual dispatch path used and retain a safe fallback for platforms without it.
The pqm4 project provides a research framework for Cortex-M4 measurements, while maintained PQ Code Package implementations expose architecture-specific backends and benchmark tooling. Their results are useful when the platform and measurement conditions match your actual question. Do not translate desktop operations-per-second into microcontroller feasibility by clock-frequency scaling alone — the gap is much larger than a simple ratio suggests. See the pqm4 project.
Hardware accelerators and HSMs add queueing, interface overhead, and key-access policy on top of raw compute. A device may have high bulk throughput but poor latency for small isolated requests. Measure end-to-end request behavior through the intended API, including failures and concurrent clients, rather than reporting only the accelerator's internal operation count.
10. A benchmark protocol for defensible comparisons
First freeze the experiment: algorithm and parameter set, source revision, compiler, flags, provider, CPU model, operating system, power mode, and whether keys are packed or expanded. State exactly which work is inside the timer. Validate outputs before trusting speed results, including negative cases where relevant (Part 13 §10).
Then separate warmup from measurement, collect repeated runs, and randomize or alternate test order to reduce drift from temperature and background activity. Report the full distribution and sample count, not just a mean. If you use cycle counters, document counter access, frequency behavior, virtualization, and whether the counter represents core cycles or another clock entirely.
For regression testing, compare against a baseline built and measured under equivalent conditions. Set thresholds that reflect observed noise and the practical service budget — a one-percent change on a noisy shared runner may be inconclusive, while a large increase in stack usage can be decisive for an embedded target even at a smaller percentage.
Finally, retain raw data and scripts. A chart without the command, revision, and measurement boundary is difficult to reproduce. Distinguish observed results from calculated scenarios and from numbers copied from a primary project source — exactly the discipline §4's table follows above. This makes future upgrades easier to evaluate and prevents accidental marketing claims from creeping into an engineering document.
11. Common misleading comparisons
Comparing ML-KEM encapsulation with signature generation is comparing different tasks entirely. Comparing ML-DSA category 3 with SLH-DSA category 1 without labeling the categories hides a real security-policy difference. Comparing a packed-key API against a preexpanded-key API can measure serialization and setup cost rather than core arithmetic.
Comparing process-per-operation shell commands with in-process library loops overstates the primitive's actual cost. Comparing local verification with remote HSM signing confounds transport and authorization. Comparing only successful requests ignores malformed-input denial-of-service behavior entirely. A useful report names these differences explicitly instead of burying them in a footnote nobody reads.
12. Energy and lifecycle costs deserve their own experiment
A battery-powered device may perform cryptography infrequently but pay a large cost to wake a radio and transmit an expanded object. Another device may remain connected while signing sensor data repeatedly. In the first case, transport energy can dominate; in the second, repeated computation and memory traffic may matter more. A CPU timing table alone cannot resolve that distinction.
Measure energy over a defined operation boundary that includes the relevant wakeup, computation, transmission, and return to idle. Subtract or report the idle baseline consistently, retain the instrument setup, and repeat enough times to distinguish the operation from measurement noise. State clearly whether the result represents a development board or the final hardware — these can differ substantially.
Lifecycle costs also include firmware size, update bandwidth, key provisioning, and verification support over many years. An optimized backend that saves milliseconds but prevents deployment on a large installed fleet may be the wrong system choice entirely. A slower, portable implementation can provide useful compatibility while selected high-value devices receive optimized paths instead.
Does ML-KEM's higher operations/second mean it's "faster than" ML-DSA?
No (§4). They're different operations entirely — key establishment vs. signing — with different security categories and purposes. §11 lists this exact comparison as a common misleading one.
Can I multiply a benchmark's ops/second directly into a service capacity number?
Not safely (§5, §6). The benchmark excludes network, HSM round trips, application authorization, and queueing effects near capacity — use it as a utilization estimate, then provision headroom based on the complete workload and tail latency, not the average alone.
Should I disable constant-time protections to get a faster benchmark number?
No (§11). That changes the product you're actually measuring — a faster number that no longer meets Part 13's implementation-security requirements describes something you can't ship, not a real optimization.
Takeaways
- Bytes, CPU cycles, elapsed time, throughput, tail latency, memory, and energy are seven distinct measurements — a benchmark answering one says nothing about the others.
- The real OpenSSL benchmark showed ML-KEM-768, ML-DSA-65, and both SLH-DSA-128 variants have genuinely different operation-cost profiles — including SLH-DSA-128s being slower to sign but faster to verify than the 128f variant, illustrating the s/f tradeoff concretely.
- A single-process benchmark excludes network, HSM, and authorization overhead — never multiply its ops/second directly into a service-capacity promise.
- Queueing behavior near capacity matters more than the average — tail latency can be unacceptable while long-run CPU utilization still looks fine on paper.
- Category-mismatched, task-mismatched, and setup-vs-arithmetic comparisons are the most common ways PQC benchmarks mislead — label the category, the exact operation, and what's inside the timer every time.
- A faster number achieved by disabling Part 13's security requirements describes a different, unshippable product — optimization has to preserve the security contract to count.
References & further reading
- NIST PQC project · RFC 10024 — §3, standardized sizes.
- Falcon project · HQC project — §3, submission-era figures.
- OpenSSL speed documentation — §4, the lab's interface.
- Course benchmark artifacts — §4, raw outputs and host description.
- mlkem-native benchmark page — §5.
- pqm4 project — §9, Cortex-M4 measurement framework.