Buffer overflows

Buffer overflows are usually described as a memory-safety problem, which is true and skips the useful part. Writing past the end of a buffer is not itself a difficult bug to understand — the interesting questions are why the length was wrong in the first place, and why the consequences range from nothing at all to a machine running an attacker’s code.

The write is the symptom#

MITRE’s weakness catalogue defines CWE-787, Out-of-bounds Write, in a sentence: the product writes data past the end, or before the beginning, of the intended buffer. Both directions belong to it — an index that goes negative is the same weakness as one that runs too far, and it arrives the same way.

The catalogue classifies it as a resultant weakness, meaning it typically follows from an earlier error rather than standing on its own: faulty index manipulation, incorrect pointer arithmetic, improper initialisation. That classification is the practical part. By the time a program writes out of bounds, the mistake has already happened — a size computed from a value that was not validated, an offset scaled by the wrong element width, a loop bound that is right for one buffer and reused for another. Fixing the write means finding the arithmetic, which is usually somewhere else entirely.

The catalogue names the errors that typically come first, and they are worth knowing by name because they are what you actually go looking for. An integer overflow or wraparound, where a length computation exceeds the range of its type and comes back small, so the allocation succeeds and is the wrong size. An off-by-one, where a bound is inclusive that should have been exclusive and a loop writes one element past the end. And an incorrect calculation of buffer size, where the arithmetic is simply wrong about how much room the elements need. None of those is a memory-safety bug in itself. Each of them is an arithmetic bug that becomes one at the moment something writes.

What sits past the end#

A program’s memory does not end where a buffer does. Whatever the compiler and allocator put next is what an overflow lands in, and that is why identical bugs have such different outcomes. CWE-787 lists three: memory corruption, a crash from accessing out-of-range or unauthorised memory, and subsequent write operations producing undefined or unexpected results.

The escalation from corruption to code execution is the one worth being precise about, because it is often described as though an overflow injects a program. It does not. The catalogue puts it as modifying control data such as return addresses in order to execute unexpected code — the overflow overwrites a value the processor is going to trust about where to go next. Nothing new is added; what changes is which existing instructions are reached, and in what order. That is why the layout of the surrounding memory decides everything, and why whether the buffer sat on the stack or the heap changes both what is nearby and what an attacker can do with it.

It is also why a crash is the fortunate case. A crash happens near the fault, points at the code responsible, and cannot be ignored. Corruption that does not crash produces a program still running on memory that is quietly wrong, and the symptom appears later in an unrelated component — the shape of bug that consumes weeks because the place it shows up has nothing to do with the place it happened.

What the defences actually stop#

Three defences are deployed almost everywhere, and each is filed by the catalogue as defence in depth rather than as a fix. Knowing which part of the attack each one removes is what tells you why they are stacked rather than chosen between.

A stack canary is a known value placed between a local buffer and the saved return address, checked before the function returns; the compiler flags that provide it are the familiar ones, Visual Studio’s /GS, FORTIFY_SOURCE, StackGuard and ProPolice. What it detects is a contiguous smash: a write that runs forward from the buffer has to cross the canary to reach the return address. What it does not detect is an indexed write that lands directly on the return address and steps over the canary entirely, and the catalogue is explicit that these mechanisms only detect certain types of overflow. It is also worth being clear about what success looks like: the typical response is to exit the application, so a canary that works converts a possible compromise into a denial of service.

Address space layout randomisation attacks the other half of the problem. If an attacker cannot predict where the executable and its libraries sit in memory, a corrupted return address is a guess rather than a plan. The catalogue rates it defence in depth for a specific reason it states directly: these techniques do not provide a complete solution, because exploits frequently use a separate bug that discloses memory addresses. One information leak elsewhere in the program restores the map, which is why an out-of-bounds read is not the harmless sibling it sounds like.

Data execution prevention, the hardware NX or XD bit, stops the process executing bytes that live in a data region. That closes the oldest version of the attack, in which the overflowing input carried the instructions it wanted run. It leaves the version this page has already described open, and the reason is the sentence the second section rests on: an overflow that overwrites a return address adds no instructions at all. It redirects the processor into code that was already there and already executable. Preventing execution of injected data does nothing about reusing code the program legitimately contains.

Read the three together and the shape is clear. Each removes one step of the chain, none removes the bug, and every one of them is defeated by a different second bug. That is why the mitigations in the next section are the structural ones, and why they are the only kind that ends the class rather than raising its price.

Why the class refuses to close#

The weakness is most prevalent in C and C++, and the reason is a language decision rather than a failing of the programmers using them: indexing memory does not check that the index is in range, because checking costs something and the languages were designed to let you decline to pay it. Managed runtimes make the opposite choice and bounds-check by default, which turns the same programming error into an exception at the point it occurs instead of a corruption that spreads.

That does not make managed code immune, and the qualification matters more than it sounds. Native interfaces reintroduce exactly the exposure the runtime removed, so a service written in a memory-safe language is memory-safe only as far as its dependencies are — and image decoders, compression libraries and cryptographic primitives are commonly native. The safety belongs to the code that was compiled with it, not to the process.

Which is why the mitigations that work are structural rather than vigilant. Bounds-checked containers, lengths carried alongside pointers instead of inferred, validation at the boundary where untrusted input arrives, and compiler and operating-system defences that make a corrupted return address harder to exploit. None of them ask anyone to be more careful, and that is the point — as with any failure that only appears at scale or under a particular input, the reliable fix is one the code enforces rather than one the reviewer is expected to catch.