Long polling

You have a page that needs to update when something changes on the server, you already know that asking again and again on a short, fixed schedule is wasteful, and you have probably been told the honest answer is WebSockets and that anything simpler is obsolete. The real question underneath that advice is narrower: is the simple thing — one request, held open, answered late — good enough for what you are building? Most of the time it is, and this article is about the mechanism that makes it work, the comparison that tells you when it stops being enough, and the specific way it fails when it does.

What holding the request actually costs#

The mechanism starts from a limit in HTTP itself: on an ordinary request, a server has no way to hand the client fresh data the client did not ask for. The client has to ask, and the server has to answer that specific request — there is no separate channel it can use to speak first. Long polling does not get around that limit; it works inside it. The client makes an ordinary request. The server does not answer straight away — it holds the request open, unanswered, until there is something worth sending back or until it decides there is not going to be. From the client’s side nothing about the request looks unusual. It is a plain request that is simply slow.

Held requests are not held only by the server, though. Every hop between the client and wherever the answer eventually comes from now has one open request sitting on it — the client’s own connection pool, any proxy or gateway sitting in front of the server, and the server’s own handling of the request — and each of those hops copes with an open connection on its own terms, whether or not anyone building the page thought about them.

A proxy, gateway or client may enforce an idle or total request deadline. Choose the server hold below the relevant deadlines with transit margin; a proxy failure is not the application’s successful empty reply. HTTP/1.1 connection limits can constrain concurrent holds. HTTP/2 multiplexing changes that limit, but stream limits, buffering and server resources still matter. A blocking handler can occupy a thread for the whole hold; an asynchronous handler still retains connection and request state.

Distinguish a successful message response, a successful no-data hold expiry, and a transport or HTTP failure. The first two can lead to another normal poll. Failures need a bounded, cancellable retry policy with backoff. This pseudocode names application helpers rather than implementing an HTTP client:

# Pseudocode: helpers implement the application's HTTP/cursor contract.
def poll_until_stopped(url, on_message, stop, deadline):
    cursor = load_acknowledged_cursor()
    while not stop() and before(deadline):
        try:
            reply = hold_open_request(url, cursor, request_timeout=35)
        except TransportError:
            wait_with_jitter_and_budget(stop, deadline)
            continue
        if reply.is_retryable_http_failure:
            wait_with_jitter_and_budget(stop, deadline, reply.retry_after)
            continue
        reply.raise_if_terminal_error()
        if reply.has_data:
            on_message(reply.data)  # may need duplicate-safe handling
            cursor = reply.cursor
            save_acknowledged_cursor(cursor)
        # An expected empty response can start the next normal hold.

The server might define a successful empty response as 200 with an empty body or 204; that is an application contract. A lost connection or 504 is a different outcome. The cursor must let the server replay retained events after a reconnect gap; deduplicate if processing succeeds but saving the cursor does not. This fragment is not executable network code.

When it is the right answer#

Set the four ways of getting an update from a server next to each other and the choice mostly makes itself.

The infrastructure column compresses more nuance than a few words can carry. Long polling needs the path to tolerate one request held open longer than a proxy’s usual idle allowance, and nothing more exotic than that — it is still plain HTTP throughout. Server-sent events need the same tolerance for an open connection, but only one of them per client instead of a held-then-reopened cycle. WebSockets need every hop on the path to support the protocol upgrade in the first place, which is a stronger requirement than either of the other two ever makes. Plain polling needs nothing unusual from anything, at the cost of never being cheap while nothing is happening.

Long polling wins, plainly, when updates are infrequent, when the traffic really is one-directional — server to client — and when adding a second protocol to the stack costs more than the inefficiency it would remove. It loses when updates come often enough that the overhead of a fresh request for every one of them dominates the exchange, or when the client needs to send data as freely as it receives it, at which point the traffic was never one-directional and no amount of holding a request open changes that.

The timeout nobody configured#

The failure starts with a hold set longer than a proxy’s idle timeout somewhere on the path — usually one nobody on the team configured or even knew was there. The proxy closes the connection once it has sat quiet past its own limit. The client sees a connection close with no response. A failure handler that retries immediately, without backoff or jitter, fires at the same moment as every other client the proxy cut off.

Multiply that by every client on the estate and the shape of the problem appears. Each one held its request for roughly the same length of time before the same proxy cut it off, so each one now retries at roughly the same moment, and the cycle repeats in step across the whole fleet. The load this produces looks exactly like the polling storm long polling was adopted to get away from — except now it is driven by a timeout instead of by genuine traffic. A client that retries because a request was closed is asking the server to handle the same thing twice, which is a narrower version of the same problem that article is about.

A roughly periodic request rate during quiet periods suggests a hold or intermediary clock, but does not prove which one. Normal empty hold expiries are periodic too, and real events can arrive regularly. Compare actual server statuses, proxy errors and observed durations against configured deadlines.

Keep the server’s normal hold shorter than the relevant path deadlines, with margin. Treat its expected empty response as routine. A transport interruption remains a failure: reconnect with bounded, jittered backoff and the last acknowledged cursor, and ensure the server retains events long enough to cover the gap. Cancellation, overall deadlines and terminal authentication errors must stop the loop.

Timeouts, retries, and the assumptions a request makes about the network it crosses are the recurring shape once more than one machine sits on the path — the distributed-systems path is where that shape gets traced in more places than this one request.

When the updates describe a background job, keep the observation separate from the work being observed. Losing the status connection does not establish that the job failed. Follow an accepted export through completion and a lost reply to see why repeating a status lookup and submitting another job need different behaviour.