Consensus Algorithms · Phase 2

Time and Failure

Article 2.2 of 5

Jul 9, 2026 · devops · 21 min read · 4400 words intermediate

Lamport clocks and logical time.

devops distributed-systems lamport-clocks series-consensus

Article 2.1 established that you can't trust physical clocks to order events across machines — even with excellent NTP or TrueTime-grade hardware, the uncertainty never fully disappears. Leslie Lamport's 1978 insight was to stop trying: instead of measuring when something happened, track what could have influenced what. A Lamport clock is just an integer counter with two rules, yet it captures a precise, provably correct notion of ordering called "happens-before" — using zero physical clock hardware and costing almost nothing to implement. This is the first genuinely different tool in the series, and it's the direct ancestor of the vector clocks in article 2.3 and the version/timestamp mechanisms inside nearly every algorithm from Phase 4 onward.

The reframe: causality instead of time

Here's the question Lamport actually asked, and it's worth sitting with because it's not the obvious one: for most distributed-systems purposes, do you actually need to know the real-world time an event happened? Or do you only need to know whether one event could have caused, or been caused by, another? For an enormous number of practical problems — did this read happen after that write, could this message have been a reply to that one, which of two conflicting updates should a system consider "later" — the second, weaker question is entirely sufficient, and critically, it's a question you can answer exactly, with zero hardware and zero uncertainty, using nothing but the structure of message-passing itself.

The happens-before relation

Lamport defined a relation, written a → b (read "a happens-before b"), using exactly three rules:

  1. Same-process rule. If a and b occur in the same process, and a comes before b in that process's local execution order, then a → b. (A single machine already has a total order for its own events — article 1.1's "one machine gives you a total order for free" — this rule just imports that.)
  2. Message rule. If a is the event of sending a message, and b is the event of receiving that same message, then a → b. A message cannot be received before it's sent — this is the one physically-grounded rule the whole system is built on.
  3. Transitivity. If a → b and b → c, then a → c.

Everything else follows from these three rules, applied exhaustively across a whole system. And critically — this is the detail that trips people up on a first read — happens-before is a partial order, not a total order. If neither a → b nor b → a can be derived from the three rules, the two events are called concurrent (written a || b), meaning as far as causality is concerned, neither could have influenced the other — they might as well have happened simultaneously, even if their real-world wall-clock timestamps (which happens-before doesn't use at all) show a difference.

happens-before: derived purely from local order + message edges Process P p1 p2 (send) p3 Process Q q1 q2 (receive p2's msg) q3 Process R r1 p1 → p2 → q2 → q3 (transitively, p1 → q3). But r1 shares no message edge with anything — r1 is concurrent with every event shown.

Fig 1 — Happens-before as a graph: same-process arrows plus message-send-to-receive arrows, chained by transitivity. r1, with no connecting edge, is concurrent with everything.

The counter mechanism

Deriving happens-before from a full message graph is conceptually clean but not something you want to reconstruct from scratch every time you need an ordering decision. Lamport's actual contribution is a cheap, local mechanism that approximates happens-before well enough to be useful: a single integer counter per process, updated by two rules.

class LamportClock:
    def __init__(self):
        self.counter = 0

    def local_event(self):
        # Rule 1: increment before every local event
        self.counter += 1
        return self.counter

    def send_message(self):
        # sending is itself a local event -- increment, then attach
        self.counter += 1
        return self.counter  # timestamp attached to the outgoing message

    def receive_message(self, msg_timestamp):
        # Rule 2: on receive, jump ahead of whatever the sender had seen
        self.counter = max(self.counter, msg_timestamp) + 1
        return self.counter

Two rules, stated plainly: (1) before every local event (including sending a message), increment the counter. (2) when receiving a message carrying timestamp T, set the local counter to max(local_counter, T) + 1. That second rule is the entire mechanism — it guarantees that a receive event's timestamp is always strictly greater than the send event's timestamp that caused it, which is precisely the property needed to make the counter respect happens-before.

The core guarantee, stated precisely: if a → b (a happens-before b, per the formal relation above), then LC(a) < LC(b) (a's Lamport timestamp is strictly less than b's). This is exactly what you'd want from a clock that's tracking causality instead of physical time. But — and this is the detail every serious treatment of Lamport clocks must state clearly — the converse does not hold. LC(a) < LC(b) does NOT imply a → b. Two concurrent, causally-unrelated events can easily end up with different counter values purely by coincidence of how the counters happened to increment, and a Lamport clock alone cannot tell you whether a given ordering reflects real causality or is just an artifact of the counter values. This single limitation is exactly what motivates vector clocks in the next article.

A worked example, step by step

Three processes, P, Q, R, each starting at counter 0:

StepProcessEventRule appliedResulting counter
1Plocal event p1incrementP=1
2Psends message m1 to Qincrement, attach 2 to m1P=2 (m1 carries timestamp 2)
3Qlocal event q1 (before m1 arrives)incrementQ=1
4Qreceives m1 (timestamp 2)max(1,2)+1 = 3Q=3
5Rlocal event r1 (unrelated, no messages)incrementR=1
6Qsends message m2 to P, carrying timestamp 4incrementQ=4 (m2 carries timestamp 4)
7Preceives m2 (timestamp 4)max(2,4)+1 = 5P=5

Notice step 5: R's event r1 got counter value 1 — the same as Q's q1 — purely because neither has any causal relationship to anything else at that point. If you only looked at the numbers (both "1"), you might be tempted to think they're "simultaneous" or comparable in some meaningful way. They aren't — they're just two independent counters that happened to be in the same state. This is the exact ambiguity the bm-note above warned about, made concrete.

Building a total order: tie-breaking

Many practical uses (a distributed lock queue, a globally ordered event log) want a strict total order — every pair of events comparable, no ties — even though happens-before is only a partial order that leaves concurrent events unordered. The standard fix: break ties using a secondary, arbitrary but consistent key — almost always the process ID. Define: (counter_a, process_a) < (counter_b, process_b) if counter_a < counter_b, or if equal, compare process_a < process_b lexicographically.

This tie-break is arbitrary, and that's the whole point to understand clearly. When two events are truly concurrent (no causal relationship either direction), there is no "objectively correct" answer to which one is "really first" — because, causally speaking, neither is. The process-ID tie-break exists purely to give your system a deterministic, consistent decision it can act on (which write wins, whose lock request is served first) — not to recover some hidden true ordering that the tie-break reveals. Every algorithm that uses this trick (and several in Phase 5 onward do, in spirit) is making the same honest trade: pick a consistent rule, not a "correct" one, because for genuinely concurrent events there is no correct one to find.

What a Lamport clock still can't tell you

Worth stating the limitation plainly, because it's exactly the gap the next article fills:

  • It cannot detect concurrency. Given only two Lamport timestamps, you cannot determine whether the underlying events are causally related or concurrent — you only know one is numerically larger, which (per the bm-note above) doesn't imply causation.
  • It cannot reconstruct "what did this event know about." A Lamport timestamp of 5 tells you this event happened after at least one chain of events reaching counter 4, but not which specific events, or from which processes, contributed to that chain.

These limitations matter concretely in systems like Amazon's Dynamo (referenced in article 1.2) that need to detect and merge concurrent conflicting writes rather than just impose an arbitrary total order on them — for that, you need to actually distinguish "causally related" from "concurrent," which requires tracking more information than a single shared counter can hold. Article 2.3 (Vector Clocks) is the direct fix: instead of one shared counter, keep a whole vector — one counter per process — which is enough information to detect concurrency exactly, at the cost of more bytes per timestamp.

FAQ

Do Lamport clocks require synchronized physical clocks at all?

No — this is the entire point. A Lamport clock is a pure software counter with no relationship whatsoever to wall-clock time or NTP. Two processes can have Lamport counters that are wildly "out of sync" with real elapsed time and the algorithm remains perfectly correct, because it was never trying to track real time in the first place.

Can Lamport timestamps overflow?

In principle yes, for a long-running system with enough events — the standard practical fix is using a sufficiently large integer type (64-bit counters make this a non-issue for essentially any realistic system lifetime) rather than anything clever; it's a non-problem in practice.

Are Lamport clocks used directly in any production system, or are they purely educational?

Both — they're foundational teaching material (as here) but also genuinely used directly in systems needing lightweight causal ordering without full vector-clock overhead, and their core idea (a monotonically-advancing logical counter, bumped on message receipt) shows up, in spirit, inside version numbers and epoch/term counters in several algorithms later in this series, including Raft's "term" concept in Phase 5.

If two events are concurrent, does that mean they happened at literally the same real-world instant?

Not necessarily, and this is a common point of confusion. Concurrent in the happens-before sense means "no causal relationship, in either direction" — it says nothing about real-world timing. Two events could be concurrent in this sense while being seconds apart in wall-clock time, as long as no message chain connects them.

Takeaways

  • Lamport's 1978 reframe: instead of measuring physical time (article 2.1's unsolvable problem), track causality — what could have influenced what — using nothing but message-passing structure.
  • The happens-before relation () is defined by three rules: same-process order, send-before-receive, and transitivity. It's a partial order — events with no causal path either direction are concurrent.
  • The Lamport clock mechanism is one integer counter per process: increment before every local event; on message receive, jump to max(local, received) + 1.
  • Guarantee: a → b implies LC(a) < LC(b). The converse is false — smaller/larger counter values don't imply causal order, since concurrent events can get any relative values.
  • A total order can be built by tie-breaking with process ID — but this is an arbitrary, consistent rule for genuinely concurrent events, not a recovery of some hidden true order.
  • Lamport clocks cannot detect concurrency — they only give you a consistent ordering, not the ability to distinguish "caused by" from "unrelated to." Vector clocks (next article) fix exactly this gap.

References & further reading

← 2.1 Physical Clocks and Why They Lie next: 2.3 Vector Clocks →
© cvam — written in plaintext, served warm