When Missing Privacy Evidence Becomes Zero

At the end of a training run, the privacy measurement existed. The client logged it. Then the return value dropped it.

That small omission changed the meaning of the entire system. Downstream code treated missing evidence as zero privacy cost, an accountant displayed a clean budget it had never been told to advance, and export proceeded because a file existed. Nothing crashed. Every component looked locally reasonable. The false conclusion appeared only when I traced one fact across all of them.

1. The measurement that vanished

The system uses federated learning, so each client trains locally and sends an update to an aggregator. The client applies differential privacy during training through Opacus. At the end of each epoch, data_ingest/fl_client.py asks the privacy engine for epsilon, the numerical privacy-loss bound at a chosen delta:

epsilon = privacy_engine.get_epsilon(DP_TARGET_DELTA)

The value is real enough to appear in the log. It is absent from the value returned to the server:

return self.get_parameters(config={}), n_samples, {"train_loss": avg_loss}

The aggregator in fl_aggregator/strategy.py expects a different contract. It looks for an epsilon metric and supplies a default when the key is missing:

fit_res.metrics.get("epsilon", 0.0)

That default is the decisive line. A measured value of zero and an unobserved value are different facts. The first can support a decision. The second means the decision lacks an input. Converting both to the same floating-point number is an epistemic type error: the program has collapsed what it knows with what it failed to learn.

The neighboring loss metric reveals the same boundary mismatch. The client returns train_loss; the strategy asks for loss. Both look plausible in isolation, so ordinary local review can miss the disagreement. The problem lives between modules, in the meaning of their shared record.

There is a second trap. Even if every client returned epsilon, taking the arithmetic mean of those values would be telemetry, not necessarily a valid global privacy composition. Different clients can have different sampling rates, step counts, and exposure histories. An average can hide the most exposed participant. The server needs the accounting events required by its threat model, not a comforting aggregate of already-composed answers.

2. Four correct components can imply a false system

The server also creates a PrivacyAccountant. It can use a Privacy Loss Distribution backend, called PLD in the code, fall back to a Renyi Differential Privacy accountant, called RDP, record total steps, compute epsilon, and report whether the configured limit has been reached. Its unit tests call step() and verify that the number moves.

The running application does not call that method. A source search finds PrivacyAccountant.step() in the accountant tests, but no production caller advances the server instance created in fl_aggregator/server.py. The status endpoint can therefore report an accountant value of zero even after local training has consumed privacy budget. The display is reading its object correctly. The object was never given the events that make its answer meaningful.

The export path then introduces a third independent truth. When the training server exits, a finally block invokes the post-training pipeline:

finally:
    _update_state(fl_running=False, current_round=num_rounds)
    FL_CURRENT_ROUND.set(num_rounds)
    logger.info("Flower server finished")

    _post_training_pipeline()

That pipeline reconstructs the aggregate model and writes an Open Neural Network Exchange file, called ONNX in the code. The download endpoint serves the latest model if a path exists. The zero-knowledge export endpoint likewise checks for an ONNX file before it starts compilation. Neither route asks whether privacy observations were complete, whether the accountant corresponds to this run, or whether the recorded budget was still valid at the moment of export.

This is why component inventories are weak architecture evidence. The codebase contains private training, an accountant, an export service, and a proof toolchain. Listing those nouns makes the design sound complete. Following one decision from observation to release shows that their authority never converges.

3. A computation proof is not a training-history proof

The exporter in fl_aggregator/zkml/exporter.py is substantial. It can produce model.onnx, calibration input, circuit settings, a compiled circuit, proving and verification keys, and a verifier contract. It uses EZKL, a zero-knowledge proof toolchain for machine-learning models.

Those outputs answer an important question: can a verifier check the statement encoded by this circuit and its public inputs? They do not automatically answer a different question: did the model enter the circuit through an observed training run whose privacy events were complete and within policy?

I call that second property export eligibility. It is deliberately narrower than general provenance. Provenance can tell me where an object came from. Eligibility must decide whether these exact bytes may cross a boundary now.

The distinction matters because proof systems are literal. A valid proof says that the encoded relation held. It does not inherit facts that were never encoded or bound to the relation. If the circuit digest is unrelated to the training round, or the privacy state is unrelated to the exported model digest, the system has several valid facts without a valid conjunction.

The export predicate I want is explicit:

exportable =
    privacy evidence is complete
    AND the authoritative accountant is within its configured limit
    AND the accountant snapshot names the completed training round
    AND the model digest names the bytes sent to circuit generation
    AND every required artifact exists and matches its recorded digest

The first clause is essential. “Not exhausted” is insufficient when the accountant was never advanced. Unknown must fail closed before the numerical comparison is even allowed to run.

4. The missing object is an atomic export manifest

The repair is not another dashboard field. It is a small, immutable manifest emitted at the only place that can see the complete decision. The current code does not implement this object; this is the contract the audit derives.

The write order is part of the contract. First, the server freezes a snapshot containing the round identity and authoritative accounting state. It refuses missing client events instead of substituting zero. It composes those events according to the declared accounting model and stops if the state is unobserved, inconsistent, or over budget.

Only then does it write the ONNX model into a staging directory, compute the content digest, and give those exact bytes to circuit generation. After compilation, it digests the circuit and key material. The manifest is written last. A single rename promotes the staging directory to its final run-specific location. The staging and final directories must share a filesystem if the rename is expected to be atomic.

The serving rules become simple. “Latest” means the newest completed bundle, not the newest loose file. Download rechecks the model digest against the manifest. Proof export accepts a run identity and reads the model named by that bundle. A partial directory, stale key, missing privacy event, or mismatched digest is unavailable by construction.

This design also changes the interface between client and server. A privacy-enabled client must return structured accounting evidence, including the step count and parameters needed by the chosen composition rule. The aggregator must reject an update that claims private training but omits that evidence. Diagnostic client epsilon can still be recorded, but it cannot silently become the server’s authorization rule through an average.

5. The real novelty is the conjunction

The earlier design idea was to preserve context as it crosses a boundary. This failure is different. No individual context object was missing. The client had one truth, the aggregator inferred another, the accountant held a third, and the exporter acted on a fourth. The novel object is the conjunction that none of them could assert alone.

That is also why the bug survived superficially strong evidence. The client log showed epsilon. The accountant endpoint returned a structured report. The model file existed. The proof directory contained cryptographic artifacts. Each observation was true, yet the sentence assembled from them was false: this model is eligible for export under this privacy history.

The correction is a general systems rule. Never let absence inhabit the same value as success. Never let a dashboard object become authoritative unless the events that advance it are part of the production path. Never let file existence stand in for a completed decision. When several subsystems jointly authorize an irreversible boundary crossing, make their conjunction a first-class object and bind it to the exact output bytes.

The hardest code-review findings are often not broken functions. They are false theorems assembled from locally correct premises. Finding one requires reading the gaps between modules as carefully as the modules themselves.