Orchestrator, or just a pipeline?
A first agent workflow runs on the build system you already have: one event, one job, one agent command, a structured result a person reads. The signals that say you now need a durable engine are specific enough to write down, which means the decision can wait until one of them actually happens.
Why hire a scheduling clerk for a workshop with a single bench? Until jobs start waiting overnight, queueing behind each other and needing someone to sign them off, the clerk has nothing to schedule, and a notebook by the bench does the whole job better — much as it did before anyone suggested the clerk.
The design conversation about a software factory reaches an orchestrator far earlier than it should. Somebody sketches a queue, a lease table, a dead-letter path and a run ledger, and a quarter disappears into a platform that has never executed a single real work item. The first workflow needs none of it — and the point at which it does is specific enough to write down before you get there.
The pipeline you already have#
The entire execution path of a first pilot fits in a paragraph. An event in the ticket tracker or the forge — the service hosting the repository and its pull requests — triggers a step in the continuous integration system you already run. That step starts an isolated container, checks out the repository, and issues one agent command with the work item’s context attached. The agent writes a structured JSON result and a log; both are stored as build artifacts. A person reads the recommendation and decides what to do about it. No automatic writes to the codebase, no message queue, no state machine, no database of runs.
That is genuinely enough, because it answers the two questions that decide whether anything else is worth building. Does one workflow produce output a person would act on? And what does a single run cost — in tokens, in container minutes, and in the reading time it consumes at the other end? Both answers arrive within days, on infrastructure that already holds secrets, emits logs and enforces access control, none of which you have to design.
Building the durable queue, the lease table, the dead-letter handling and the run ledger before any of that has happened is the classic way to lose three months. Each of those components is a real requirement eventually. None of them is a tested requirement yet, so what gets built is shaped by an imagined workload, and first contact with a real one rearranges it. Meanwhile nothing has been validated and no cost per run is known, so the number that decides whether the whole plan is affordable is still missing.
Worth naming the ceiling, so you watch for it rather than discover it. A build job is a process with a time limit, no recollection of anything that happened before it started, and no idea that another job wants the same work item. Every capability an orchestrator sells you is a consequence of removing one of those three constraints.
When an engine earns its place#
Teams tend to treat this as a question of taste, or seniority, or how serious the project is. It is a question of requirements, and the requirements announce themselves. Everything in the left column below is something you observe happening, not something you forecast.
| Situation | Take | Because |
|---|---|---|
| One pilot workflow, one agent, a result a person reads | Pipeline | Nothing here outlives a single job. Adding a queue and a state machine validates no requirement and delays the only evidence that matters — what one run costs and whether the output is any good. |
| Two or three workflows, each finishing inside the hour, nobody waiting | Pipeline | Steps that complete within one job's lifetime are what a build system already does well, on infrastructure that already has secrets, logging and access control. Nothing here is paying back the cost of an engine. |
| A run waits hours or days for a human answer, then must carry on | Engine | A job that sleeps for two days is a job that gets killed. Suspending on an external event and resuming with the accumulated context intact is the thing durable execution exists for. |
| A runner or server restart loses work that was half-finished | Engine | Recovering execution from a recorded history changes where state lives, so it is a foundation rather than a feature. It is not something you retrofit onto a pipeline in an afternoon. |
| Several agents run at once and reach for the same work item | Engine | Two agents opening branches against one ticket is a wasted day and a confusing pull request queue. Leases and a single owner per item are exactly what an orchestrator holds and a build system does not. |
| Retries must tell a timeout apart from a wrong answer | Engine | Transient, deterministic failure and needs-a-human want three different responses. One retry count treats them as one thing and cheerfully re-runs the failures that will never pass. |
| One team's backlog sweep can starve every other team's queue | Engine | Concurrency limits, per-team quotas and spend caps have to be enforced above the individual job, or they are honour-system comments in a configuration file that nobody reads under pressure. |
| You need to stop runs that are already in flight | Engine | Revoking a token stops the next run. Cancelling the twelve already running needs something that knows they exist, and telling an auditor what ran against which instructions needs the same ledger. |
None of those is about volume, which is the part that surprises people. A thousand pipeline runs a day can be fine without an engine, and four runs a week can demand one if two of them wait overnight for an answer. Duration, statefulness and contention move the line; sheer throughput does not.
Once several of the rows are true at the same time, the question stops being whether to adopt an engine and becomes which one. That is not a surrender to fashion. Leases, retry classification, cancellation, quotas and an audit history assembled ad hoc across a handful of pipelines gradually re-create a less mature version of something that already exists — at your own maintenance cost, with your own team as the only people who understand it.
Four shapes are worth telling apart. A durable workflow engine keeps the workflow as ordinary code, records a complete event history, and replays that history to rebuild state when a worker crashes rather than losing the run; Temporal’s documentation describes this as workflows-as-code, and its signals and updates are how a running workflow waits for an external answer and then continues. That is the strongest fit for long-lived, stateful, human-in-the-loop work. It does not supply the coding agent or the sandbox — you pair it with an agent runtime and run each agent as an isolated job — and it is the heaviest of the four to stand up.
A container-native workflow runner makes every step a container, with step-level retry policies and suspend-and-resume steps for approvals; Argo Workflows is the familiar example where Kubernetes is already the substrate. It suits infrastructure that is container-centred and workflows that finish in minutes to hours. It is less natural for long, human-heavy state machines, where the configuration and the glue between steps grow awkward faster than the workflows do. A declarative engine with a visual editor gets you something working fastest and is weakest precisely where a workflow carries substantial domain state, because the logic ends up split between the diagram and the scripts hanging off it. And managed agent steps inside your existing build system are the lowest-effort path of all for a first pilot, on infrastructure that already exists — not a long-lived workflow engine, and not sold as one.
Where it goes wrong#
The custom orchestrator written over a weekend is the most common failure, and it starts out entirely reasonable. A hundred lines, a table of runs, a retry loop. Six months on it holds the lease logic for every agent in the company, one person understands its failure modes, and the retry loop has a defect that only appears when the model provider returns an overload response. Nothing about writing it was a bad decision; the bad decision was never revisiting it once it quietly became load-bearing.
Choosing an engine for the wrong reason fails differently and more expensively. Something arrives because a consultant favoured it, or because it was on a conference slide, and the requirement is reverse-engineered afterwards to fit. The symptom is easy to check for: the team can recite the engine’s features but cannot name the row in that table it was bought to satisfy. It still has to be operated and upgraded, and it collects that cost while validating nothing.
The opposite failure hides better. Five separate pipelines, each with retry logic copy-pasted from the last one, each with a slightly different idea of what counts as a transient error. That is an orchestrator. It has no owner, no tests, no cancellation and no ledger, and because it never got a name, nobody ever has to justify its existence at a planning meeting.
Then the honest part, which is easier to say than to fund. A factory becomes part of your development supply chain, and once it is, it inherits the obligations of one. The run ledger needs backup and restore, because it is the only record of what changed and on whose instructions. The behaviour when the model provider or the tracker or the forge is unavailable has to be chosen deliberately — queue, pause, or fail closed are three different answers, and choosing none of them means you get whichever one the code happens to implement under load. Runs that were half-finished when a dependency dropped have to be recoverable. And there needs to be some answer, however uncomfortable, to what happens if you have to leave the vendor.
None of that is an argument for building the engine first. All of it is an argument for choosing one rather than accreting one. The discipline is the same one that keeps the pilot cheap: do the thing that answers a real question this week, and let the answer decide what gets built next. For a factory that has just started merging changes, that is usually not more orchestration — it is finding out whether the change actually worked once it reached production. The exception worth having early, before any engine, is the ability to stop everything at once, and that belongs with the credentials the agents run under rather than with the scheduler.
IF YOU REMEMBER ONE THING
A first workflow needs a job, not an engine. The signals that change the answer — runs that wait on people, runs that must survive a restart, agents contending for the same work — are things you observe, so the decision can wait until one of them actually happens.
Questions people also ask
4 QUESTIONSDo I need a workflow engine for my first AI agent workflow?
Almost certainly not. A first pilot fits inside one continuous integration job: an event starts a step, the step runs one agent command in an isolated container, and the structured result is stored as a build artifact for a person to read. That is enough to prove one workflow produces output worth acting on and to measure what a run costs. Both answers are what should shape the platform you build afterwards.
When is a CI pipeline no longer enough for agent workflows?
When a run has to outlive the job. The specific triggers are waiting hours or days for a human answer and resuming afterwards, surviving a runner restart, several agents contending for the same work item, retries that must distinguish a timeout from a wrong answer, quotas that stop one team starving another, cancellation of runs already in flight, and a queryable history of what ran against which instructions at what cost.
Should I build my own agent orchestrator?
Only knowingly, and rarely. A hundred lines holding a run table and a retry loop is reasonable on day one and load-bearing by month six, at which point you own leases, cancellation, quotas and an audit trail with one person who understands the failure modes. Building those ad hoc gradually re-creates a less mature version of something that already exists. The honest choice is to name the requirement first, then pick a tool that already meets it.
What does a durable workflow engine give you that a build system does not?
Memory across failures. Temporal records a complete event history for each workflow execution and replays it to rebuild state after a worker crashes, and its signals and updates let a running workflow wait for an external answer and then carry on. A build job has none of that: it is a process with a time limit, no recollection of anything before it started, and no idea that another job wants the same work item.