How Agent Concurrency Affects Elapsed Time and Token Usage to a Tested and Merged Main Branch

At a glance

Workload

Ten additive model and API features in Ridgeline Coffee, a Python web application

Configurations

Fresh sessions: 1, 3, 5, and 10 concurrent slots; Persistent full workload: 1 slot; Three trials per setting; ten fixed tasks per run

Method

Three trials each at 1, 3, 5, and 10 concurrent Codex agents, in randomized blocks. Each run starts from the same commit in an example repository and ends after all ten changes are integrated and final acceptance and regression checks pass. A separate three-trial follow-up gives one persistent session the full workload. Collected November 10, 2026.

Metrics

Elapsed time to verified completion; Successful runs / attempted runs; Reported input, cached input, uncached input, and output tokens

Reproducibility

partially reproducible

Environment

Local Git worktrees on macOS 26.5, arm64, 10 logical CPUs; Python 3.14.5; Codex CLI 0.167.0; gpt-6.1-terra with medium reasoning effort. Implementation and integration share one agent budget.

Principal limitations

• The persistent follow-up changes task batching, branches, handoffs, and validation boundaries together, and ran after the original matrix.
• Three trials per setting on one small, deliberately overlapping workload and one machine.
• Vendor model (OpenAI) behavior may vary between runs.
• Source and summarized results are included, but not full logs due to space.
• Functional acceptance and regression checks on the merged changes do not establish comprehensive code or documentation quality

Experiment design

If I have ten coding tasks, how much sooner can I finish them by running more agents?

I wanted to measure the elapsed time after all ten changes merged into the main branch, with acceptance tests and regression tests passing.

Each agent finishing a task in its own branch is one piece that we can measure, but there are also git integration tasks, merge conflict repairs, and the continuous integration/verification as well.

I ran the same ten coding tasks with 1, 3, 5, and 10 concurrent agents, three times at each setting. All twelve runs completed successfully. The median completion time went from 16.9 minutes with one agent to 7.1 minutes with ten agents, a 2.38× speedup. Five agents were fairly close to the results of ten agents, with a median of 7.4 minutes.

Then I tried a different approach: give just one Codex agent all ten tasks and use that same session for integration. The median completion time was 3.6 minutes, faster than every original run. It also used substantially fewer reported tokens. All three follow-up runs passed the same frozen acceptance and regression checks.

That result certainly makes a compelling argument for using one persistent agent, and not many fresh agents to do the work, if the tasks can be implemented serially like this. One advantage for this particular approach is that the tasks are all similar, working on the same repository, which may not be a realistic scenario. Also, this used three different trials with one set of tasks, so it’s not generalizable - it’s certainly worthy of more experimentation and inquiry.

What the coding agents built

The starting point was Ridgeline Coffee, an example Python web application for a fictional coffee roasting company. Each run began at commit 353c3ccdf3acdaae9acae78b5132241061b5decc in a fresh clone.

The ten tasks were similar - they added computed properties and exposed them through existing authenticated API serializers:

Every task also called for focused Python tests and README documentation. None required a migration or a new dependency.

These tasks overlap on purpose, to better test rework and integration. All ten touch the example application’s app/models.py, app/api.py, and README.md files. Several pairs change the same classes and serializer bodies.

Declaring agent counts, and the definition of done

The original matrix uses fresh sessions for each task. The persistent single agent approach uses a different delivery strategy with the same final completion requirements.

The configured agent count is a cap on simultaneous implementation and integration jobs combined. There are no extra integration agents or coordination agents outside that limit.

A task starts in its own Git worktree based on the latest passing main branch at agent dispatch time. Once the implementation passes validation, it joins an integration queue. Integration is serialized and gets priority when a slot becomes free. Already existing integration jobs do not get interrupted by new tasks.

For every task, a fresh integration agent merges the change into a candidate worktree, resolves conflicts, and repairs regressions. This agent runs even when Git merges cleanly, as tests may fail, or other issues may occur. Main only gets the merged code after the candidate passes verification. The rule for integration is that there are two attempts per task allowed, and failed implementations are not retried.

With only one slot, implementation and integration alternate. With more slots, they can overlap. A successful run through the ten tasks uses at least twenty fresh agent invocations - ten for implementation and ten for integration. Each invocation starts fresh, and nested agents or subagents are disabled to stay within the defined count.

The completion clock starts immediately before task dispatch and stops after final verification of all ten tasks on main. It includes scheduling, implementation, integration, repair, and test execution. Initial cloning, dependency installation, and baseline verification are excluded from this measure, as they are particularly dependent on the initial repo. The report does preserve a separate total time including this setup.

Results for single-task, fresh agents

Each trial block contained all four settings in a seeded random order. Runs executed one at a time, and the ten tasks were dispatched in the same order in every run. The individual completion times:

The largest reduction in median time came from moving from one to three agents: about 7.5 minutes, or 45% less elapsed time. Moving from three to five saved another 1.9 minutes. Doubling the agent count from five to ten saved about 20 seconds, a 4.5% reduction in median time.

There was significant variation, as you might expect with LLMs and their non-deterministic behavior. The three-agent runs ranged from 6.7 to 10.6 minutes, and the five-agent runs from 7.0 to 9.0 minutes. The ten-agent runs ranged from 6.9 to 7.7 minutes. These ranges do overlap. Ten agents had the lowest observed median, but these runs aren’t enough to prove that it would always be lower than five agents across all workloads. The overall pattern matters more: a large drop from one to three agents, then very little change after five.

The one-agent setting also had one slow run. Trial 2 took 22.4 minutes, mostly because its implementation agents took 848 seconds in total, compared with 534 and 495 seconds in the other two trials. The median is not affected much by one slow run, but it explains the wide spread for one agent in the chart.

Follow-up: one persistent agent gets the whole workload

The original one-agent setting still starts twenty separate sessions: ten for implementation and ten for integration. It serializes the work, but each new session has to establish its context again.

For the follow-up, one coding agent session receives the full list of ten tasks at the beginning and then implements them on one workload branch. The harness then resumes that exact session for integration and repair. Each trial starts with a fresh clone and a new session - there are no subagents. All three trials used one logical session across two CLI invocations, compared with twenty sessions and twenty invocations in every original run.

The persistent strategy’s median was 4.70× faster than the fresh-session one-slot setting, and 1.97× faster than the ten-slot setting. The relative percentages are 78.7% and 49.3% less elapsed time, respectively. Every persistent run finished before the fastest original run in this sample.

The baseline commit, task definitions, frozen tests, dependency versions, model selection, reasoning effort, Python, and CLI version matched across the two experiments. The follow-up required a new harness version, preserved in a separate source archive.

There are several changes bundled into this strategy. The agent can plan across all ten tasks, carries context into integration, and avoids per-task handoffs. One branch also removes conflicts between independently developed task branches by construction. Validation and retry boundaries change: the persistent strategy allows up to two integration attempts for the whole workload, while the original allows two per task.

The follow-up also ran after the original randomized matrix. Temporal effects, including service load and caching, could affect the comparison. These results compare two delivery strategies; they do not completely isolate the effect of keeping a session alive.

Token usage alongside completion time

The harness captures the usage fields in Codex’s JSON output. For the original matrix, each invocation starts a separate session, so the twenty final session counters are summed for each run.

Resumed sessions use a different method for measuring token usage. In these three persistent trials, the integration invocation reports the cumulative session counter, including implementation. Using the saved session logs, the final total_token_usage agrees with that counter and with the sum of the individual last_token_usage records. The final session counter gets counted once.

The table shows the median of each token field across the three runs in that configuration:

Cached input is included within input, rather than added to it. For each run, uncached input is input minus cached input. Each column’s median is calculated independently, so the displayed cached and uncached medians do not necessarily add up to the displayed input median. Per-run values and the remaining raw counters are included in the token dataset and comparison report.

The ten-slot fresh strategy used about 27% more median input tokens than the one-slot fresh strategy, while finishing sooner. Most of that increase was cached input: median uncached input rose about 13%, from 289,794 to 327,171 tokens. The persistent strategy used 87.9% fewer median input tokens than the ten-slot fresh strategy, alongside its shorter completion time.

These are reported token measurements, not a billing comparison. The experiment also does not determine which strategy is cheapest, or assign a particular share of tokens to useful work versus rework.

Tests are important for this experiment

The benchmark has 36 public acceptance cases covering the ten feature contracts. They check specified behavior, including rounding, empty data, threshold boundaries, missing values, and API exposure.

The harness keeps these checks and an immutable copy of the original regression tests outside the agents’ worktrees, and checks their hashes before validation. It also runs candidate tests, so tests added by agents participate in the gate. Before a change reaches main, validation includes the original tests, candidate tests, and acceptance contracts for the incoming task and all previously integrated tasks. Final verification checks all ten contracts together.

All fifteen recorded final verification commands exited successfully. The persistent final gates reported 91, 90, and 93 test executions; the original final gates reported 127–139. The frozen checks were identical, but agents added different tests. Passing the same acceptance gate does not establish equal implementation or test quality.

Takeaways

Within the original delivery strategy, adding concurrency reduced the time to a tested main branch substantially: ten concurrent slots delivered a median speedup of 2.38× over one slot. The improvement from five to ten slots was small.

The persistent full-workload strategy produced the more interesting result: one session finished in about half the median time of ten concurrent fresh agents, with much lower reported input usage. For this small, overlapping workload, how tasks were grouped and handed off mattered more than adding slots within the original strategy.

Integration agents spent about 225–420 seconds per run, at every slot count. Integration runs one task at a time, so that time cannot overlap with other integration work no matter how many slots are available. In the ten-agent runs, about 280 seconds of the 424.5-second median was integration, which explains why going from five to ten agents changed so little. The persistent runs spent a median of only 16.5 seconds in integration. With one workload branch, there were no separate task branches to merge. A different integration schedule could be a follow-up experiment.

The one-agent runs had no conflicted merges, because each task started from the latest main branch, which already included the earlier tasks. The three-, five-, and ten-agent runs had 6–9 conflicted merges each, which is expected when parallel tasks change the same files.

The persistent runs also ended with fewer tests: 90–93 test executions in the final gates, compared with 127–139 for the fresh-session runs. All runs passed the same frozen acceptance and regression checks, but writing fewer tests is less work, and may account for part of the persistent strategy’s shorter time and lower token usage.

The scheduler’s single integration lane and the cost of shared-file repairs are reasonable contributors to the original results. Context reuse, whole-workload planning, fewer handoffs, and different validation boundaries are plausible contributors to the persistent result. This experiment does not separate their effects. A useful next comparison would vary these choices independently and interleave the trials.

Limits and reproducibility

This is an exploratory experiment with one application, one machine, one harness (Codex CLI), one model (OpenAI GPT 6.1 Terra), and one narrow set of additive tasks. There are three trials per configuration. The persistent strategy was only tested at one slot, and could possibly be extended to two or three slots.

All invocations used the gpt-6.1-terra model with medium reasoning effort through Codex CLI 0.167.0. The host machine was my MacBook Air M5 with macOS 26.5 and Python 3.14.5. Fluctuations in time could have been from API response time, non-deterministic LLM responses, contention on the host machine, or other environmental issues.

Tasks dispatched later start from a newer passing main, so sequential work benefits from earlier integrated changes. That policy is part of the experiment. The results do not cover agents sharing a checkout, larger backlogs, different integration policies, migration-heavy tasks, or other models. Reported token usage adds another measurement alongside elapsed time, but does not establish the cheapest configuration or equivalent code quality across strategies.

The downloadable artifacts include the original report, the fifteen-run comparison, both experiment configurations, selected final verification evidence, normalized token measurements with counting evidence, and separate source archives for the original and follow-up harnesses. The chart converts the reports’ work_seconds values to minutes without altering the observations. The comparison notes originally left resumed-counter semantics unresolved; the token-only session-log audit above supplies the additional evidence for counting those sessions once. Full command logs, agent output, and trial worktrees remain in the local experiment directory and are not included in these downloads.

The source archive supports rerunning the protocol with the required model access and dependencies. Your reproduced times will certainly vary, and it may also be that the underlying harness or model is unavailable, or behaves differently.