Congestion control

A TCP sender is never limited by one number. It is limited by two, computed for different reasons by different parties, and it must respect whichever is smaller at any moment. Almost every confused conversation about a slow connection comes from treating those two as one — because the fix for each is in a different place, and tuning the wrong one produces no change at all.

Two windows, two different problems#

RFC 5681 keeps the two limits in separate variables and defines them separately. The receiver window, rwnd, is the most recently advertised receiver window — a number the other end sends you, describing how much buffer it has. The congestion window, cwnd, is a sender-side state variable that limits the amount of data a sender may have in flight. Nobody advertises cwnd and no message carries it: it is the sender’s own running estimate of what the path between the two machines will tolerate. Keeping that estimate is one of the jobs you give up when you choose the protocol that tracks nothing.

The rule joining them is explicit: a sender must not send data beyond the highest acknowledged sequence number plus the minimum of cwnd and rwnd. So the connection runs at whichever limit is tighter, and which one that is decides where a fix belongs. If the receiver’s window is the binding constraint, the receiving application is not draining its socket fast enough and the answer is on that machine. If the congestion window is binding, the network is the constraint and no amount of buffer anywhere changes it.

Slow start is not slow#

A new connection has no evidence about the path, so it does not begin by sending as much as the receiver will allow. It begins with a small congestion window and enlarges it as acknowledgements come back — RFC 5681 permits increasing cwnd by at most one full-sized segment per acknowledgement received. Every returning acknowledgement is a piece of evidence that the path carried what was sent, and the window grows on the strength of it.

That rule compounds. Acknowledging a window’s worth of data grows the window by about a window, so it doubles roughly every round trip — which makes slow start an exponential probe wearing a misleading name. It is slow only in where it begins. The specification hands over to a second algorithm at a threshold called ssthresh: slow start applies while cwnd is below it, congestion avoidance above it, and either may be used when they are equal. Congestion avoidance grows the window by approximately one full-sized segment per round trip instead of doubling — the connection has stopped searching for the capacity and started feeling for its edge.

What happens when a segment is lost#

Everything above describes the window opening. A control loop needs the other half, and the specification gives two different answers depending on how the loss was noticed — because how the sender found out is itself evidence about the path.

Three duplicate acknowledgements are the mild case. RFC 5681 treats the arrival of three duplicate ACKs, with no intervening acknowledgement that moves the left edge of the window, as an indication that a segment has been lost, and the sender retransmits what appears to be the missing segment rather than waiting for a timer to expire. That is fast retransmit, and the reason it can afford to be mild sits in the evidence itself: a duplicate acknowledgement is generated by a segment that did arrive. The path is still delivering. Something went missing in the middle of a stream that is otherwise flowing.

So the window is reduced rather than abandoned. The sender sets ssthresh to max(FlightSize/2, 2*SMSS) — half of what was in flight, floored at two segments — and then, in fast recovery, sets cwnd to that threshold plus three segments, which the specification describes as artificially inflating the congestion window by the number of segments that have left the network. Each further duplicate acknowledgement adds one more segment, on the same reasoning: each one is evidence that another packet has drained out of the path. When the acknowledgement covering the retransmitted data finally arrives, cwnd is set back down to ssthresh, and congestion avoidance carries on from there.

A retransmission timeout is the severe case, and what makes it severe is the absence of evidence. Nothing came back at all — not even duplicate acknowledgements, because no later segment arrived to trigger one. The sender cannot tell a badly congested path from one that has stopped existing, so it assumes the worse of the two: on a timeout cwnd must be set to no more than the loss window, which is one full-sized segment, whatever the initial window happened to be. The connection starts again from slow start.

Put the two halves together and the shape of the algorithm appears. Growth is additive — one segment per round trip, once congestion avoidance is in charge. Reduction is multiplicative — half on a fast retransmit, all the way down to one on a timeout. That asymmetry is deliberate rather than pessimistic: a sender that has just seen loss is a sender whose estimate of the path was wrong, and being wrong about capacity is expensive for everyone else sharing it. So the algorithm gives ground quickly and takes it back slowly — the same instinct an application needs one layer up, where a retry policy that does not back off finishes off the service it was trying to reach.

Where it goes wrong#

The most expensive mistake is treating a short transfer as a bandwidth problem. A connection carrying a small response may finish while still in slow start, having never come close to the capacity of the link — so its speed was set by the number of round trips it needed, not by the bandwidth available. Buying a fatter pipe changes nothing measurable. This is the mechanism behind the advice to reuse connections rather than open new ones: a warm connection has already paid for its probing and starts where the last one left off, while a fresh one begins the climb again.

The second is reading a full receiver window as a healthy connection. The window that matters is the smaller of the two, and only one of them is visible from outside. A receiver advertising plenty of room tells you nothing about whether the sender’s own congestion window is holding it to a fraction of that — which is why the two have to be distinguished before anything is tuned, and why “we increased the buffers and it made no difference” is such a common report.

The third is that the mechanism only reacts to what it can observe. Congestion is inferred from loss, so a path with large buffers in the middle can absorb the excess instead of dropping it — the sender sees no loss, keeps its window open, and the queue simply grows. Nothing is lost and everything waits, which turns a capacity problem into a delay problem, and puts it firmly in the territory where how much arrives and how long each thing takes stop being the same question.