L0/L1/L2 Three-Tier Architecture for High-Throughput Tamper-Evident Audit Logs

A tamper-evident audit log that has to scale past 10^4 events per day, produce per-decision proofs auditors can verify offline years later, and survive a regulatory request to “show this was anchored in jurisdictions you do not control” hits a structural wall the moment it tries to do all three on a single substrate. The wall is not a performance problem that more hardware fixes; it is an impedance mismatch between three operations that run at fundamentally different rates and have fundamentally different trust profiles. The pattern that resolves the mismatch — adopted by 8M Networks for its Evidence Pack tamper-evidence stack, by RFC 9162 Certificate Transparency for the WebPKI, by Bitcoin Lightning for payment channels, and by every credible Ethereum rollup for L1 data posting — is a three-tier separation of storage (L0), settlement (L1), and aggregation (L2). This article makes the layering explicit, walks the worked examples, and argues that the separation is load-bearing rather than incidental.

The Throughput Problem

Imagine a regulated decision engine that emits 10,000 signed Evidence Packs per day. Each pack must, at issuance time, be tied to a cryptographic root that is anchored in at least one trust domain the decision engine does not operate. The naive design timestamps every Evidence Pack directly against an eIDAS-qualified timestamping service: 10,000 timestamp requests per day, each producing a qualified TSA token bound to that single pack. The design works on day one. By day thirty it has produced three problems.

The first is cost. Typical 2025–2026 list pricing for an eIDAS-qualified timestamp is on the order of €0.10 per call, with volume discounts that depend on the QTSP and the contract tier. At 10,000 events per day, 365 days per year, the direct-submit path produces 3.65 million qualified timestamps per year. At list pricing that is €365,000/year for the timestamping line item alone. If the design also requires anchoring in a second independent trust domain (a different jurisdiction’s QTSP, or a public-good transparency log such as Sigstore Rekor) for split-view-attack defence, the bill doubles. For most operators this is the budget item that kills the design before it reaches production.

The second is rate. eIDAS QTSPs are not optimised for high-frequency calls; their internal HSM ceremony, audit logging, and signature serialisation produce per-call latencies of hundreds of milliseconds and per-tenant rate limits in the tens of calls per second. A direct-submit design that bursts past the rate limit gets queued or 429ed. The decision engine’s per-event latency now depends on the QTSP’s queue depth, which is operationally unworkable: the engine cannot return a signed Evidence Pack to its caller within a normal request budget if the timestamping leg of the path is on the critical synchronous path.

The third is failure-mode coupling. If the QTSP has an outage — whether technical, contractual, or geopolitical — every Evidence Pack issued during the outage is degraded. There is no fallback path. Replacing the QTSP after the fact does not retroactively anchor the records that were issued during the gap. The auditor inspecting the gap window sees a discontinuity, and the operator has no clean way to remediate.

These three failures share a common cause: the design has collapsed three operations with very different rate-and-trust profiles onto a single tier. Storage of the events (cheap, fast, single-trust-domain) is happening at the rate of the settlement layer (expensive, slow, multi-trust-domain). The fix is to decouple them.

Naive Single-Tier Alternatives And Why They Fail

Before the three-tier model, there are three single-tier patterns that operators reach for. Each fails distinctly.

One big internal log, where the decision engine stores all events in its own Merkle-tree-backed datastore (immudb, Trillian, an in-house append-only log) and serves inclusion proofs from that store, fails the trust test. The verifier presented with a proof has to trust the operator’s signature on the tree root. There is no second jurisdiction; if the operator is later compelled to produce a different history, no part of the system surfaces the contradiction. The audit log is records, not evidence. (For the structural distinction between records and evidence, see The 5-Question Test for Adding Distributed Ledger Technology.)

One big consensus network, where the operator anchors every event onto a public blockchain (Bitcoin, Ethereum mainnet) directly, fails the cost and rate tests in the same way the QTSP-direct path does, and adds a regulatory and risk-profile complication: the operator now has cryptocurrency exposure, mempool dynamics, fee volatility, and a substrate whose finality model is probabilistic rather than the deterministic synchronous finality the audit consumer expects. The pattern works as a thought experiment but is rejected in any environment where regulatory clarity and predictable cost matter.

One big batch, where the operator collects events for an arbitrarily long window (a day, a week) and then submits a single root, fails the per-decision verifiability test in a subtler way. If the batch is the unit of anchoring, the verifier holding one Evidence Pack cannot, from the pack alone, produce a proof. They need the pack, the inclusion path, and the batch root, and they need the batch root to be itself anchored. The shape is right but the design is incomplete; without an explicit aggregation tier with a defined inclusion-proof contract, “batch occasionally” does not generalise.

The lesson across all three failures is that storage, settlement, and aggregation are operations with different rates, costs, and trust profiles, and any design that puts them on the same tier ends up paying the worst-case bill of all three.

The Three-Tier Model

The pattern that works separates the three operations onto distinct tiers with explicit interfaces between them.

                +--------------------------------------+
                |   L1 — Settlement / Trust Anchors    |
                |   eIDAS QTSP A | QTSP B | Sigstore   |
                |   Rekor (v2) | Sigsum cosigned heads |
                +-------------------+------------------+
                                    ^
                                    | (root only, ~1/day)
                                    |
+----------------------+    +-------+--------+
|                      |    |                |
|  L2 — Aggregation    |--->|  Batch root R  |
|  AnchorAggregator    |    |  (Merkle root  |
|  collects N events,  |    |   over N leaves)|
|  builds Merkle root  |    |                |
|                      |    +----------------+
+----------+-----------+
           |
           | (per-event leaf hash + inclusion proof)
           v
+----------+--------------------------------------+
|                                                 |
|  L0 — Storage Substrate                         |
|  Merkle-tree-backed append-only log             |
|  (immudb / Trillian / Tessera / in-house)       |
|  Single trust domain: the operator              |
|                                                 |
+-------------------------------------------------+

L0 is the storage substrate. It is a single-organisation Merkle-tree-backed append-only log, server-signed, optimised for cheap and fast writes. immudb, Google Trillian, and the newer Tessera library (the explicit Trillian v1 successor; production-ready and the library new transparency-log deployments adopt in 2026) all fit. The trust assumption is one-organisation: the operator vouches for the integrity of L0. L0 alone is not evidence; it is the operator’s word.

L1 is the settlement tier. It consists of multiple independent timestamping or transparency services, each in a different trust domain — typically multiple eIDAS QTSPs in different jurisdictions, plus a public-good transparency log such as Sigstore Rekor (v2, the current public-good instance, tile-backed via the same tlog-tiles API that Tessera and TesseraCT expose) and a witness-cosigned Sigsum log. L1 is regulator-legible, slow per-call, expensive per-call, and produces signed receipts that are checkable offline against widely-distributed root keys. The trust assumption at L1 is multi-domain: defeating the audit requires colluding with the operator and every L1 anchor jurisdiction.

L2 is the aggregation tier. Its job is to take many L0 events, build a Merkle tree over them, and submit only the root to L1. Each L0 event therefore carries an inclusion path leaf-hash → batch-root, and the batch root carries L1 receipts. The verifier presented with one event walks the inclusion path to the batch root, then checks the batch root against the L1 receipts. The interface between L0 and L2 is “L2 reads L0 leaves and produces batch roots”; the interface between L2 and L1 is “L2 submits batch roots and stores L1 receipts alongside the batch metadata.” 8M Networks names its L2 component AnchorAggregator; Bitcoin Lightning calls its equivalent the commitment transaction; Ethereum rollups call their equivalent the batch posting; RFC 9162 Certificate Transparency calls its equivalent the Signed Tree Head. Same shape, different domain.

The verification path that an end-consumer walks is bounded regardless of how much L2 traffic the system carries:

event leaf hash
    | (Merkle inclusion proof, log2(N) hops)
    v
batch root R
    | (cross-reference)
    v
L1 receipt: TSA_A signature on (R, t)
L1 receipt: TSA_B signature on (R, t)
L1 receipt: Sigsum cosigned tree head at offset h, leaf = R
    |
    v
checked offline against well-known L1 root keys
    |
    v
PASS / FAIL

The verifier never queries the live system. They never have to ask the operator “did this happen?” They hold the event, the inclusion path, the batch root, and the L1 receipts; they verify against L1 root keys distributed out-of-band; they get a binary answer.

Worked Example — The Evidence Pack Tamper-Evidence Stack

8M Networks adopted this layering in v0.8.1 (2026-05-05) for its Evidence Pack tamper-evidence stack, and the design choices in each tier illustrate the general pattern.

L0 is an internal Merkle-tree-backed append-only log. The properties needed are write throughput, low per-write latency, and a clean Merkle inclusion-proof API. The shortlist of viable substrates includes immudb (Merkle-tree-per-database, every transaction adds one leaf and rebuilds the path to root, signed ImmutableState), Google Trillian (still maintained as of mid-2025; the canonical reference for personality-on-top transparency-log designs), and Tessera (the explicit Trillian v1 successor, tile-based with CDN-cacheable reads, supporting POSIX, GCP, and AWS storage backends). Of these Tessera is the contemporary recommendation for new deployments because the tile-based design fits modern infrastructure and the API explicitly anticipates the L2 batching model rather than treating each leaf as the unit of settlement.

L1 is two eIDAS QTSPs in different jurisdictions plus Sigstore Rekor (v2). Two QTSPs means a single QTSP outage degrades but does not fail the system; jurisdictional diversity means the system survives a regulatory or contractual issue with one provider. Sigstore Rekor v2 is a public-good transparency log with a 99.5% availability SLO; it is shared across the wider open-source ecosystem and witnessed by independent parties. The combination is regulator-legible (the QTSPs produce qualified timestamps with the legal weight regulators expect) and supply-chain-legible (Rekor anchors place the same root in a different trust ecosystem, gossiped by witnesses described in Witness Networks for Transparency Logs).

L2 is an in-house aggregation component (AnchorAggregator). Its policy is “build one batch per N events or per T seconds, whichever comes first” — with N and T tuned so that the average batch size is large enough to amortise L1 cost and the worst-case latency from L0 admit to L1 receipt is bounded. The batch interval determines two things: how much data the operator loses to a window-end crash before settlement, and the cost-amortisation factor.

The cost-amortisation arithmetic, spelled out, makes the design choice obvious. Direct-submit at 10,000 Evidence Packs per day is 3.65M qualified timestamps per year; at €0.10 per call (typical 2025–2026 list pricing — the exact figure depends on the QTSP and volume tier) that is €365,000/year. L2-aggregated at one batch per day is 365 batch-roots per year, plus a small per-call premium for batch-size, somewhere around €1 per call: €365/year, a 1000x reduction. The 1000x figure is specific to once-per-day batching; per-100-events batching produces a 100x reduction; smaller batches give correspondingly smaller multipliers. The operator picks the batch size that trades window-end exposure against L1 cost. None of these knob choices change the layering, only the parameters.

The auditor inspecting an Evidence Pack three years later receives the pack itself, the L0 leaf hash, the inclusion proof from leaf to batch root, the batch root, the QTSP receipts on the batch root from both jurisdictions, and the Sigsum cosigned tree head. They check the inclusion proof, check the QTSP signatures against the QTSPs’ published keys, check the Sigsum cosignatures against the witness public keys, and either get a clean PASS or see exactly which leg of the verification fails. They never query 8M’s live infrastructure.

Worked Example — Bitcoin And Lightning

The Bitcoin Lightning Network is the same layering applied to a different domain (payments rather than evidence) and is worth walking explicitly because its non-audit-log status sometimes obscures the structural lesson.

The on-chain Bitcoin chain is L1: it is the settlement substrate, expensive per transaction, slow (10-minute average block time, 60-minute deep-confirmation), with a single-trust-domain economic security model (the proof-of-work network and the operators who care about Bitcoin’s value). Direct on-chain payments are the equivalent of direct QTSP timestamps in the audit-log domain: they work, they are expensive, they do not scale to high-throughput consumer use cases.

Lightning is the L2 aggregation tier. A payment channel between two parties is opened with one on-chain L1 transaction (the funding transaction); the parties then exchange thousands of off-chain payments by updating a shared commitment transaction signed under the BOLT 2/3 anchor-commitment pattern (the commit-tx + revocation-tx structure that has remained the BOLT-stable shape since BOLT 2/3 stabilised, even as Taproot Channels based on MuSig2 and “Lightning Liquidity v2” / BLIP-39 dual-funded channels have added material recent improvements that do not change the underlying L1 anchoring shape). Only when the channel is closed does another on-chain L1 transaction fire, settling the net result of those thousands of off-chain payments back to the chain.

The audit-log lesson is in the structure, not in the threat model. Lightning’s threat model is “either party may try to publish a stale commitment to favour themselves,” addressed by the revocation mechanism. The audit log’s threat model is different. But the layering — L1 anchors a small number of expensive transactions, L2 batches a large number of cheap operations against the L1 anchor, the L1-bound result is verifiable from the on-chain artefact alone — is identical. A payment-system architect explaining Lightning and an audit-system architect explaining the Evidence Pack stack are describing the same pattern.

The substrate-level rejection in the audit-log domain is also worth being explicit about: 8M Networks does not use Bitcoin (or any cryptocurrency-economic substrate) at L1 for Evidence Packs. The reasons are regulatory clarity, predictable cost, and avoidance of cryptocurrency exposure for the operating entity. The L0/L2 layering is reused; the L1 substrate is replaced with eIDAS QTSPs and witness-cosigned transparency logs. The pattern survives the substrate change.

Worked Example — Ethereum L2 Rollups

Ethereum L2 rollups — Optimism, Arbitrum, zkSync, Base, and others, with the specific market-share order shifting quarterly — are the same layering at yet another scale.

The Ethereum L1 chain is the settlement substrate. EIP-4844 (Dencun, shipped to mainnet on 2024-03-13) added blob transactions specifically for L2 batch postings: blob fees are materially lower than calldata fees, and L2 operators post their batch summaries as blobs rather than calldata. The amortisation factor depends on the batch design, blob price, and blob-count per posting; the qualitative result is that Ethereum’s per-rollup-batch L1 cost dropped substantially post-Dencun, which made larger throughput tiers economic.

Each L2 is the aggregation tier. The L2 collects user transactions, computes the resulting state transition (or the resulting compressed transaction list, depending on rollup family), and posts a batch root and proof to Ethereum L1 at intervals on the order of minutes to hours. Per-user transactions execute at L2 throughput and L2 cost (cents or sub-cent per transaction); finality at L1 anchoring time (or, for ZK rollups, at proof-acceptance time) is the substrate-level result.

The reason this counts as an instance of the same pattern is that the verification path for an L2 transaction has the same shape as the audit-log verification path: user-transaction → inclusion in L2 batch → batch root → L1 anchoring transaction → checkable against Ethereum’s L1 finality. The verifier of the L2 transaction does not need to trust the L2 operator’s word; they walk the proof to the L1 root. The trust diversity is weaker than the multi-jurisdiction QTSP setup (Ethereum is one trust domain, not multiple), but the layering is the same.

The audit-log architect who looks at Ethereum rollups should take the structural lesson while rejecting the trust-domain choice: rollups demonstrate that the L0/L2/L1 separation works at planet scale and at high throughput, even when L1 is a single substrate. For audit logs, the “single L1” choice is unacceptable, so the recipe is to keep the layering and replace L1 with multi-domain trust anchors.

Worked Example — RFC 9162 Certificate Transparency

The closest analogue to the audit-log domain is RFC 9162 Certificate Transparency (the active CT v2.0 specification, published December 2021 and not superseded as of 2025). CT logs accept TLS certificates from CAs, store them in a Merkle-tree append-only structure, periodically publish Signed Tree Heads, and produce inclusion proofs that any client can check against the STH. The structural mapping to the three-tier model is direct.

L0 is the CT log’s internal Merkle storage — historically Trillian-backed, increasingly TesseraCT-backed in 2026 deployments. The Merkle tree, signing key, and append-only behaviour all live here.

L1 is the witness layer. RFC 9162 §5.1 frames the Maximum Merge Delay (MMD) at which the log promises to merge submitted certificates into the tree; Google’s production CT logs have historically used 24h MMD (subject to per-shard variation; current canonical MMD values are tracked at certificate-transparency.org/known-logs). The witness layer — Sigsum cosigned tree heads, OmniWitness, ArmoredWitness with on the order of 15 custodian locations as of 2026-05 — sits independently of the log operator and gossips STHs across trust domains. The combination of “STH signed by log + STH cosigned by witnesses” is L1: the verifier checks the STH signatures from a set of independent parties.

L2 is the STH publication itself. The STH is exactly the batch root. Each TLS certificate in the log carries an inclusion proof leaf → STH; each STH is signed and cosigned by the L1 witness set. A browser checking a certificate’s CT inclusion path is performing the same walk as an auditor checking an Evidence Pack: walk the inclusion proof to the batch root, check the batch root’s L1 receipts.

The RFC 9162 / CT analogy is the most direct because the domain (regulatory-legible per-certificate verifiability against multiple independent trust domains, scaled to billions of certificates over the WebPKI’s lifetime) is the same domain audit logs occupy. The specification itself is engineering grade, the deployment is mature, and the witness layer is real production infrastructure.

Anti-Patterns — Collapsing Tiers

The most common failure mode in implementations of this pattern is tier collapse: treating an L0 substrate as if it were L1 trust, or treating an L2 aggregation as if it were L1 settlement, or treating L2 consensus as if it produced L1-equivalent receipts.

Storing your own ledger AS your settlement. The decision engine writes events to immudb and serves immudb’s signed ImmutableState to the auditor as proof. The auditor checks the signature, sees it is signed by the operator’s key, and asks the right question: “what makes me trust the operator’s key?” There is no second trust domain; the operator could rotate keys, fork histories, or simply lie. The fix is to take immudb’s tree root and anchor it externally — at which point immudb is L0 and the external anchor is L1. The mistake is conceptual, not technical; it almost always shows up as “we have a Merkle tree, why do we need anything else?”

Trusting an L2 consensus without L1 verification. The operator builds an aggregation tier that produces batch roots and stores them with metadata, but never actually anchors the roots into independent trust domains — relying on the aggregator’s own audit trail or replicated state machine for “trust.” A verifier who distrusts the aggregator has no recourse. This is the same mistake as the previous one, one tier up. Hyperledger Fabric’s ordering service is an instructive case: its batching machinery (MaxMessageCount 500, BatchTimeout 2s) and its consensus options (Raft for production CFT, SmartBFT for production BFT in Fabric v3.0+; Solo removed in v3.0) make it a credible single-organisation L2. As a candidate L1 substitute it fails the trust-diversity test — it is a closed validator set under coordinated governance, not a multi-jurisdictional anchor pool. Operators who want to use Fabric’s ordering service for audit-log batching should treat it as L2 and add real L1 anchors on top, not assume the consensus output is L1-equivalent.

Single-jurisdiction L1. The operator anchors batch roots to one QTSP, in one country, under one contract. The cost goes down (one anchor instead of two), and the design looks like the three-tier pattern. The trust diversity is gone. A regulator in a second jurisdiction asking “show this was witnessed in your jurisdiction too” gets no answer. The whole point of L1 is multi-domain; collapsing to a single domain saves money up front and produces a system that fails the moment a real audit happens.

Skipping L2 entirely. The naive pattern from the Throughput Problem section. Direct-submit per-event L1 anchoring works at low rates and breaks at high rates. The fix is to introduce L2; not to introduce a faster L1 or a cheaper L1.

Operational Concerns

The three-tier model introduces failure modes the single-tier model does not have, and a credible production deployment monitors them explicitly.

Monitoring lag from L2 admit to L1 receipt. In the steady state, an event admitted to L2 should have L1 receipts within the configured batch interval. If the QTSPs are slow, rate-limiting, or unavailable, the L2 backlog grows and the time from admit to receipt drifts. The operator should alert on the 95th percentile admit-to-receipt latency, not the average; tail latency is what the auditor sees on a degraded record.

STH publication regularity. RFC 9162 frames a Maximum Merge Delay; the audit-log analogue is the maximum interval between L2 batch postings. If the system is configured for one batch per day but no batch fires for 36 hours, the operator has either a process bug or an L1 outage. Either way the auditor looking at the gap window will see degraded records. Monitoring is “alert if no L1 receipt has been written in the last 1.5 × configured interval.”

Consistency proof audits. Periodically (RFC 9162 recommends not less than weekly for CT; once per day is reasonable for audit-log L0 substrates), an external auditor should fetch two STHs from L0 separated in time and request a consistency proof that the older tree is a prefix of the newer tree. If the proof does not check, the L0 substrate has been forked or rewritten, and the operator’s signed STHs are no longer trustworthy. This is the audit-log analogue of CT’s split-view-attack defence; without it, the L0 substrate is unmonitored and the attacker’s compromise window is bounded only by external discovery.

Witness gossip. If L1 includes witness-cosigned logs (Sigsum, the OmniWitness/ArmoredWitness ecosystem), the operator should verify that fresh cosignatures are being produced at the configured cadence and that no witness has stopped gossiping. A witness pool that quietly drops below the configured cosignature threshold is a silent degradation; the verifier presented with a record at year-five sees fewer cosignatures than expected and has to decide whether to accept the record. Monitoring is “alert if cosignature count for the latest tree head is below threshold for more than X hours.”

Key rotation. Each tier has its own key lifecycle. L0 server-signing keys, L2 aggregator keys, and L1 anchor public keys all rotate at different cadences. The verifier holding a record from year five may need to check a signature against a key that was retired three years ago; the system has to keep the historical key set discoverable (typically via a TUF-distributed root metadata bundle, as Sigstore does for Rekor). Designs that rely on “current key only” verification break the audit story.

When NOT To Use This Pattern

Three-tier separation is engineering effort the audit-log domain pays back quickly. It is not free, and there are environments where the simpler designs are better.

Low throughput. A system that produces 50 events per day, with predictable bursts and a 24h batch tolerance, can plausibly skip L2 and direct-submit each event to L1. The cost-amortisation argument does not bite at that scale; the engineering simplicity of “one event, one anchor” wins. The threshold at which L2 becomes worthwhile is approximately when the L1 cost line item exceeds the L2 implementation and operations cost — typically at 10^3 events per day and certainly at 10^4.

Single jurisdiction, no trust-diversity requirement. A purely internal compliance log where the auditor is a sister team in the same organisation and trusts the operator’s key directly does not need L1. L0 alone is enough. Adding L2 + L1 is overengineering; the three-tier model exists for cross-organisational and cross-jurisdictional verification, and absent that requirement the pattern is unjustified.

No per-decision verifiability requirement. A telemetry pipeline that ingests events for analytics, produces aggregate metrics, and never needs to answer “show me the verifiable proof for this specific event” is not an audit log. It is a database. Merkle trees and L1 anchoring add complexity for a property the system does not need. (The 5-question test in tech-25 frames this distinction explicitly.)

The structural lesson is to apply the three-tier pattern only when the cost, scale, and trust requirements all justify it. Where they do, the pattern is the dominant design across multiple domains — payments (Bitcoin/Lightning), smart-contract scaling (Ethereum rollups), the WebPKI (RFC 9162 Certificate Transparency), and regulated decision evidence (the Evidence Pack stack) — and the convergence is not coincidence. It is the shape that emerges when storage, settlement, and aggregation are decoupled instead of conflated.