We tested 62 configured OpenCode model entries with the exact prompt Hi. The test measured complete first-turn session latency.
This test does not isolate model inference time. It measures the full OpenCode command path for a new session.
Result summary
- OpenCode listed 62 configured model entries.
- Twenty-two attempts returned text successfully.
- Thirty-nine attempts failed before they returned text.
- One attempt reached the 100-second timeout.
deepseek/deepseek-chathad the lowest successful full elapsed time at 18,582 ms.
The fastest result applies only to this run and this OpenCode setup. It does not prove that this model is always fastest.
Successful attempts ranked by full elapsed time
| Rank | Provider and model | Full elapsed time (ms) | First text (ms) |
|---|---|---|---|
| 1 | deepseek/deepseek-chat | 18,582 | 16,216 |
| 2 | deepseek/deepseek-v4-pro | 19,027 | 16,713 |
| 3 | deepseek/deepseek-reasoner | 19,171 | 16,808 |
| 4 | deepseek/deepseek-v4-flash | 19,406 | 17,105 |
| 5 | openai/gpt-5.6-sol-fast | 19,492 | 16,964 |
| 6 | openai/gpt-5.4 | 19,510 | 17,138 |
| 7 | openai/gpt-5.6-terra | 19,697 | 17,281 |
| 8 | opencode/laguna-s-2.1-free | 20,200 | 17,806 |
| 9 | openai/gpt-5.4-fast | 20,375 | 17,323 |
| 10 | openai/gpt-5.6-terra-fast | 20,568 | 17,873 |
| 11 | opencode/ling-3.0-tiny-free | 20,696 | 18,011 |
| 12 | openai/gpt-5.5 | 21,248 | 18,804 |
| 13 | openai/gpt-5.5-fast | 21,251 | 18,517 |
| 14 | openai/gpt-5.6-luna-fast | 22,127 | 19,222 |
| 15 | openai/gpt-5.4-mini-fast | 22,524 | 20,009 |
| 16 | openai/gpt-5.6-luna | 22,587 | 20,029 |
| 17 | opencode/longcat-2.0-free | 23,627 | 20,759 |
| 18 | openai/gpt-5.6-sol | 24,240 | 21,795 |
| 19 | opencode/big-pickle | 24,302 | 21,782 |
| 20 | openai/gpt-5.4-mini | 24,555 | 20,756 |
| 21 | opencode/mimo-v2.5-free | 27,320 | 24,738 |
| 22 | opencode/deepseek-v4-flash-free | 73,584 | 70,796 |
Failed attempts and timeout
The harness recorded no error details for these attempts. Therefore, the table does not assign a cause.
Some entries target image, audio, video, embedding, live, or research tasks. A text command can be incompatible with such entries.
Elapsed values below show time until failure or timeout. They are not successful response latency values.
| Provider and model | Outcome | Elapsed time (ms) |
|---|---|---|
opencode/nemotron-3-ultra-free | Timeout | 100,208 |
opencode/north-mini-code-free | Failed | 4,215 |
google/deep-research-max-preview-04-2026 | Failed | 12,811 |
google/deep-research-preview-04-2026 | Failed | 12,647 |
google/gemini-2.5-computer-use-preview-10-2025 | Failed | 15,489 |
google/gemini-2.5-flash | Failed | 15,802 |
google/gemini-2.5-flash-image | Failed | 14,722 |
google/gemini-2.5-flash-lite | Failed | 19,886 |
google/gemini-2.5-flash-preview-tts | Failed | 15,370 |
google/gemini-2.5-pro | Failed | 16,168 |
google/gemini-2.5-pro-preview-tts | Failed | 15,493 |
google/gemini-3-flash-preview | Failed | 14,728 |
google/gemini-3-pro-image | Failed | 14,713 |
google/gemini-3-pro-image-preview | Failed | 13,673 |
google/gemini-3.1-flash-image | Failed | 13,278 |
google/gemini-3.1-flash-image-preview | Failed | 12,550 |
google/gemini-3.1-flash-lite | Failed | 14,794 |
google/gemini-3.1-flash-lite-image | Failed | 16,011 |
google/gemini-3.1-flash-live-preview | Failed | 14,023 |
google/gemini-3.1-flash-tts-preview | Failed | 13,187 |
google/gemini-3.1-pro-preview | Failed | 13,222 |
google/gemini-3.1-pro-preview-customtools | Failed | 14,157 |
google/gemini-3.5-flash | Failed | 13,537 |
google/gemini-3.5-flash-lite | Failed | 14,409 |
google/gemini-3.5-live-translate-preview | Failed | 12,127 |
google/gemini-3.6-flash | Failed | 12,225 |
google/gemini-embedding-001 | Failed | 12,522 |
google/gemini-embedding-2 | Failed | 12,109 |
google/gemini-flash-latest | Failed | 12,413 |
google/gemini-flash-lite-latest | Failed | 12,162 |
google/gemini-omni-flash-preview | Failed | 12,314 |
google/gemini-robotics-er-1.6-preview | Failed | 12,166 |
google/gemma-4-26b-a4b-it | Failed | 14,487 |
google/gemma-4-31b-it | Failed | 14,214 |
google/lyria-3-clip-preview | Failed | 12,187 |
google/lyria-3-pro-preview | Failed | 12,212 |
google/veo-3.1-fast-generate-preview | Failed | 12,009 |
google/veo-3.1-generate-preview | Failed | 12,466 |
google/veo-3.1-lite-generate-preview | Failed | 12,350 |
openai/gpt-5.3-codex-spark | Failed | 12,589 |
Method
- OpenCode listed the configured model entries.
- The harness started a new child process for each model.
- Each child process received the exact prompt
Hi. - The harness ran one model entry at a time.
- The harness allowed 100 seconds for each attempt.
- The harness did not retry failed attempts.
The command shape was opencode run --format json --model <model> Hi. Each attempt started a fresh OpenCode session.
The harness started its stopwatch before child process setup and launch. It stopped the watch after output processing and process completion.
Full elapsed time includes local process work, OpenCode startup, provider queues, response generation, streaming, output processing, and shutdown.
First-text time marks the first parsed JSON text event. The ranking uses full elapsed time instead of first-text time.
How to repeat this test
You can run the same benchmark on your own machine. The test uses a PowerShell script and the opencode CLI.
Requirements
- PowerShell 7 or later.
- The
opencodeCLI installed and on your PATH. - At least one model provider configured in your OpenCode credentials.
Step 1: list your configured models
OpenCode can list the models your key providers expose. Use the opencode config command to see which models are available. Copy the model identifiers you want to test.
Step 2: create the benchmark script
Save a PowerShell script that runs each model entry one at a time. The script starts a new opencode run child process for each entry. The script below is the harness we used. Replace the $models array with your own list.
$ErrorActionPreference = "Stop"
# Replace this list with your model identifiers
$models = @(
"deepseek/deepseek-chat",
"openai/gpt-5.4",
"opencode/laguna-s-2.1-free"
)
$opencodeScript = Join-Path $env:USERPROFILE "AppData\Roaming\npm\opencode.ps1"
$resultPath = Join-Path $env:TEMP "opencode-model-benchmark.json"
$timeoutMs = 100000
$results = [System.Collections.Generic.List[object]]::new()
$benchmarkConfig = '{"default_agent":"benchmark","agent":{"benchmark":{"description":"Text-only model speed benchmark","mode":"primary","prompt":"Respond directly to the user message.","permission":{"*":"deny"}}}}'
foreach ($model in $models) {
$stopwatch = [System.Diagnostics.Stopwatch]::StartNew()
$startInfo = [System.Diagnostics.ProcessStartInfo]::new()
$startInfo.FileName = "pwsh.exe"
$startInfo.UseShellExecute = $false
$startInfo.CreateNoWindow = $true
$startInfo.RedirectStandardOutput = $true
$startInfo.RedirectStandardError = $true
foreach ($argument in @("-NoProfile", "-File", $opencodeScript, "run", "--pure", "--format", "json", "--title", "benchmark", "--agent", "benchmark", "--model", $model, "Hi")) {
[void]$startInfo.ArgumentList.Add($argument)
}
$startInfo.Environment["OPENCODE_CONFIG_CONTENT"] = $benchmarkConfig
$process = [System.Diagnostics.Process]::new()
$process.StartInfo = $startInfo
$firstOutputMs = $null
$firstTextMs = $null
$timedOut = $false
$textParts = [System.Collections.Generic.List[string]]::new()
try {
[void]$process.Start()
$stderrTask = $process.StandardError.ReadToEndAsync()
$lineTask = $process.StandardOutput.ReadLineAsync()
$stdoutEnded = $false
while (-not $stdoutEnded) {
if ($lineTask.Wait(20)) {
$line = $lineTask.Result
if ($null -eq $line) { $stdoutEnded = $true; continue }
if ($null -eq $firstOutputMs) { $firstOutputMs = [math]::Round($stopwatch.Elapsed.TotalMilliseconds) }
try {
$event = $line | ConvertFrom-Json
if ($event.type -eq "text" -and $event.part.text) {
if ($null -eq $firstTextMs) { $firstTextMs = [math]::Round($stopwatch.Elapsed.TotalMilliseconds) }
$textParts.Add([string]$event.part.text)
}
} catch {}
$lineTask = $process.StandardOutput.ReadLineAsync()
}
if ($stopwatch.ElapsedMilliseconds -ge $timeoutMs -and -not $process.HasExited) {
$timedOut = $true
$process.Kill($true)
}
}
if (-not $process.HasExited) { [void]$process.WaitForExit(5000) }
} catch {
if (-not $process.HasExited) { $process.Kill($true) }
} finally {
$stopwatch.Stop()
$process.Dispose()
}
$elapsedMs = [math]::Round($stopwatch.Elapsed.TotalMilliseconds)
$response = [string]::Join("", $textParts)
$status = if ($timedOut) { "timeout" } elseif (-not $response) { "no_text_response" } else { "success" }
$results.Add([pscustomobject]@{
model = $model
elapsed_ms = $elapsedMs
first_text_ms = $firstTextMs
status = $status
response = $response
})
Write-Output "MODEL=$model ELAPSED_MS=$elapsedMs STATUS=$status"
}
[System.IO.File]::WriteAllText($resultPath, ($results | ConvertTo-Json -Depth 5), [System.Text.UTF8Encoding]::new($false))
Write-Output "Results saved to $resultPath"
Step 3: run the benchmark
Execute the script in PowerShell 7. It runs one model entry at a time. The total run time depends on your model list and timeouts. Expect several minutes for ten or more entries.
The script sets a custom agent configuration through the OPENCODE_CONFIG_CONTENT environment variable. This agent replies directly to the user message. It does not use tool calls or extra processing.
Step 4: read the results
Open the output JSON file. Each record contains these fields.
model: the provider and model identifier that was tested.elapsed_ms: full-wall-clock time from child-process start to exit or timeout.first_text_ms: time from child-process start until the first parsed text event from the JSON stream.status:success,timeout, orno_text_response.response: the full text the model generated.
Sort successful entries by elapsed_ms to build your own latency ranking.
Entries with status timeout or no_text_response are not successful responses. Their elapsed values show failure detection latency, not usable response speed.
What the test measures
This benchmark measures complete OpenCode session-start latency. The stopwatch covers child-process launch, OpenCode startup, model provider request, text generation, streaming output processing, and process shutdown.
The measurement includes automatic title generation. OpenCode performs title generation during a normal first turn. Title generation adds extra provider traffic and latency.
This is a one-run snapshot. Run the benchmark several times and report percentile values if you want stronger conclusions.
Title generation affects this metric
OpenCode can request an automatic title when it starts a fresh session. This test included that normal first-turn behavior.
Title generation can add provider traffic, queue contention, latency, and cost. It can also use a different model from the tested model.
For these reasons, the results measure OpenCode session-start latency. They do not measure isolated model inference latency.
Sequential and parallel requests
The harness ran model attempts sequentially. This method avoided direct competition between benchmark attempts.
Applications often send parallel requests. Requests to one provider can then share quotas, queues, connections, or service capacity.
Provider bottlenecks can increase latency during parallel work. These sequential results do not predict that increase.
Limits of this test
- Each model entry received one measured run.
- One run cannot show normal variation or tail latency.
- The short prompt does not represent large prompts or long responses.
- Different response lengths affect full elapsed time.
- Provider load and network conditions change over time.
- The test measured latency only. It did not measure response quality, accuracy, or cost.
- Failed entries received no ranking among successful responses.
A stronger test needs warm and cold runs, repeated samples, fixed output limits, and percentile results. It also needs separate parallel-load tests.
Summary
deepseek/deepseek-chatreturned the lowest successful full elapsed time in this run.- The result measures a complete OpenCode first turn, including session-start work.
- Automatic title generation can affect provider traffic, latency, and cost.
- The benchmark used sequential attempts, so it does not measure parallel provider pressure.
- This one-run snapshot cannot establish long-term performance or model quality.
Leave a Reply