How we would have caught a $100M+ defect that frontier models missed

What happened

ColdCard firmware, commit b18723dd, 1 March 2021, titled “First pass w/ libNgU”. It swapped the wallet’s crypto layer onto a new dependency by moving a git submodule pointer. In the diff, that is one line. Everything the change actually introduces lives in a different repository, at a revision named by that line and nowhere else.

Per Coinkite’s disclosure and the public incident record: between 30 July and 2 August 2026, tracked sweeps moved 1,433.13 BTC from 5,477 addresses across 2,896 transactions, with 1,432.48 BTC arriving after fees. Their own summary of the root cause is one sentence:

A build guard checked whether the hardware RNG macro existed, not whether it was enabled.

That sentence is worth sitting with, because it describes a class of defect rather than an incident. The code was not wrong about cryptography. It was wrong about what a question means.

The guard, and why it reads as correct

Inside the dependency, in ngu/random.c:

#ifdef MICROPY_PY_STM
extern uint32_t rng_get(void);
# define CHIP_TRNG_32()  rng_get()

# ifndef MICROPY_HW_ENABLE_RNG
# error "get a HW TRNG plz"
# endif
#endif

The intent is not ambiguous, and the author wrote it down in the error string: a hardware TRNG is required here. The guard is three lines, it is placed correctly, and it is reachable. It is also, for this macro, the wrong test.

#ifndef asks whether a macro exists. It has nothing to say about what the macro is set to. The C standard draws exactly this line: #ifdef and #ifndef test whether an identifier is currently defined as a macro name, while #if evaluates a constant expression - and in that expression, any identifier that survives macro expansion is replaced with 0. Two different questions, four characters apart.

So #define MICROPY_HW_ENABLE_RNG 0- the ordinary way any build system says “this feature is off” - satisfies the guard completely. The macro exists. The check passes. The build proceeds as though a hardware random number generator had been confirmed present.

Read the highlighted row against the middle column. Of the three states a build can put that macro in, the guard catches the one nobody writes by accident and misses the one every maintainer writes deliberately to mean off. The failure is not that the check is weak. It is that the check is inverted with respect to its own stated purpose.

Why review does not find it

Three properties, each defeating a different kind of reviewer. They compound, which is why this survived both human review and machine review for five years.

There is a fourth property that only appears once you look at the whole port rather than this board. The macro is defaulted to 0in MicroPython’s shared STM32 configuration, for every board, not just this one. Which means the existence test is not merely wrong here - it is vacuous everywhere it is compiled. A guard that can never fire is indistinguishable, from the inside, from a guard that has never needed to.

What a language model does with it

Coinkite report that AI-assisted review ran against this code in the weeks before the exploit and did not catch it, and that frontier models tested since have not caught it either. Our own model-based review missed it too. We think that result is worth publishing rather than hiding, because it is the most useful data point in this entire case study.

Pointed at the file and asked what is wrong with it, a model reasons about the guard and concludes that it is fine - and the reasoning it produces is genuinely good. The idiom is correct. The placement is correct. The error message states a real requirement. Every local fact supports the conclusion. The one fact that overturns it lives in another repository, in a build configuration file that the model was never shown and had no way to ask for.

This is the part that generalises past ColdCard. Fluent review fails on questions whose answer is not in the text being reviewed, and it fails confidently, because nothing in the visible evidence signals that something is missing. The paragraph reads identically whether the macro is undefined, set to 1, or set to 0. Confidence is not calibration.

Stop reading the check. Compile it.

A guard makes a falsifiable claim: put the code in the state I reject, and I will refuse. That claim can be tested rather than assessed. Put the build in the state the guard says it rejects, compile, and observe whether it refuses.

One compilation proves nothing, which is the part that matters. A build that completes silently is equally consistent with “the guard is broken” and “the guard was never compiled in at all”. So the test is two compilations that differ in exactly one setting.

ngu/random.c:29: #error "get a HW TRNG plz"

(no diagnostic)

The control arm establishes that the guard is present, reachable, and does refuse something. The violating arm puts the build in precisely the state the error message names as unacceptable - and the compiler says nothing. That pair is not an opinion about the code. It is a property of the code, and it takes seconds to establish.

One further step settles what the silence costs. The dependency calls an external rng_get(); preprocessing the file that defines that symbol under both configurations shows which implementation the build selects. With the macro enabled, the hardware path. With it at 0, a software pseudo-random generator. The requirement in the error string was not merely unenforced - the thing it demanded was silently replaced.

The chain, end to end

Six steps from a single changed line to the consequence. Four of them are established by running the build rather than reasoning about it - which is what makes the chain checkable by someone who does not trust us.

Note what the chain does notclaim. It does not assert that any specific wallet was drained by this specific path - attribution belongs to Coinkite’s own incident analysis, not to a code reviewer. It establishes that the guard fails to enforce what it states, that the failure selects a different implementation, and that the value reaches material a wallet must keep unpredictable. That is the part a review can prove, and it is enough to stop a merge.

This is not a C problem

It is tempting to file this under preprocessor arcana and move on. That would be a mistake. The confusion between present and enabled has an idiom in every ecosystem, and in each one the mistake reads as diligence.

A Python service that checks hasattr(config, "require_tls") and a firmware build that checks #ifndef MICROPY_HW_ENABLE_RNG have written the same bug. Both look careful. Both pass when the setting is present and switched off. The only difference is that one of them was guarding a Bitcoin wallet.

What this changes about dependency review

Every team bumps dependencies. Most of those bumps arrive as a one-line change, and most review processes treat the size of a diff as a proxy for its risk. This defect is the clearest available demonstration that the proxy is wrong: the smallest possible diff introduced the largest possible consequence, and the code responsible was never on screen.

A reviewer that only reads the diff cannot find this class of defect - not because it is not clever enough, but because the evidence is structurally outside its input. A reviewer that reads the dependency but only reasons about it produces a confident, well-argued, wrong answer, as every model tested against this one has. What separates a finding you can act on from a paragraph you scroll past is whether the claim was executed or merely asserted.

A guard states a falsifiable claim. Put the code in the state it rejects and see whether it refuses. Anything less is a reading of the code, not a test of it.

See the review itself

The full review sits on a public mirror of the original pull request, posted by the same GitHub App a customer installs - the finding, the build receipts, the reachable consequence, and an honest note about which files were and were not reviewed.