OpenCode First-Turn Latency Benchmark: 62 Model Attempts

We tested 62 configured OpenCode model entries with the exact prompt Hi. The test measured complete first-turn session latency.

This test does not isolate model inference time. It measures the full OpenCode command path for a new session.

Result summary

  • OpenCode listed 62 configured model entries.
  • Twenty-two attempts returned text successfully.
  • Thirty-nine attempts failed before they returned text.
  • One attempt reached the 100-second timeout.
  • deepseek/deepseek-chat had the lowest successful full elapsed time at 18,582 ms.

The fastest result applies only to this run and this OpenCode setup. It does not prove that this model is always fastest.

Successful attempts ranked by full elapsed time

RankProvider and modelFull elapsed time (ms)First text (ms)
1deepseek/deepseek-chat18,58216,216
2deepseek/deepseek-v4-pro19,02716,713
3deepseek/deepseek-reasoner19,17116,808
4deepseek/deepseek-v4-flash19,40617,105
5openai/gpt-5.6-sol-fast19,49216,964
6openai/gpt-5.419,51017,138
7openai/gpt-5.6-terra19,69717,281
8opencode/laguna-s-2.1-free20,20017,806
9openai/gpt-5.4-fast20,37517,323
10openai/gpt-5.6-terra-fast20,56817,873
11opencode/ling-3.0-tiny-free20,69618,011
12openai/gpt-5.521,24818,804
13openai/gpt-5.5-fast21,25118,517
14openai/gpt-5.6-luna-fast22,12719,222
15openai/gpt-5.4-mini-fast22,52420,009
16openai/gpt-5.6-luna22,58720,029
17opencode/longcat-2.0-free23,62720,759
18openai/gpt-5.6-sol24,24021,795
19opencode/big-pickle24,30221,782
20openai/gpt-5.4-mini24,55520,756
21opencode/mimo-v2.5-free27,32024,738
22opencode/deepseek-v4-flash-free73,58470,796

Failed attempts and timeout

The harness recorded no error details for these attempts. Therefore, the table does not assign a cause.

Some entries target image, audio, video, embedding, live, or research tasks. A text command can be incompatible with such entries.

Elapsed values below show time until failure or timeout. They are not successful response latency values.

Provider and modelOutcomeElapsed time (ms)
opencode/nemotron-3-ultra-freeTimeout100,208
opencode/north-mini-code-freeFailed4,215
google/deep-research-max-preview-04-2026Failed12,811
google/deep-research-preview-04-2026Failed12,647
google/gemini-2.5-computer-use-preview-10-2025Failed15,489
google/gemini-2.5-flashFailed15,802
google/gemini-2.5-flash-imageFailed14,722
google/gemini-2.5-flash-liteFailed19,886
google/gemini-2.5-flash-preview-ttsFailed15,370
google/gemini-2.5-proFailed16,168
google/gemini-2.5-pro-preview-ttsFailed15,493
google/gemini-3-flash-previewFailed14,728
google/gemini-3-pro-imageFailed14,713
google/gemini-3-pro-image-previewFailed13,673
google/gemini-3.1-flash-imageFailed13,278
google/gemini-3.1-flash-image-previewFailed12,550
google/gemini-3.1-flash-liteFailed14,794
google/gemini-3.1-flash-lite-imageFailed16,011
google/gemini-3.1-flash-live-previewFailed14,023
google/gemini-3.1-flash-tts-previewFailed13,187
google/gemini-3.1-pro-previewFailed13,222
google/gemini-3.1-pro-preview-customtoolsFailed14,157
google/gemini-3.5-flashFailed13,537
google/gemini-3.5-flash-liteFailed14,409
google/gemini-3.5-live-translate-previewFailed12,127
google/gemini-3.6-flashFailed12,225
google/gemini-embedding-001Failed12,522
google/gemini-embedding-2Failed12,109
google/gemini-flash-latestFailed12,413
google/gemini-flash-lite-latestFailed12,162
google/gemini-omni-flash-previewFailed12,314
google/gemini-robotics-er-1.6-previewFailed12,166
google/gemma-4-26b-a4b-itFailed14,487
google/gemma-4-31b-itFailed14,214
google/lyria-3-clip-previewFailed12,187
google/lyria-3-pro-previewFailed12,212
google/veo-3.1-fast-generate-previewFailed12,009
google/veo-3.1-generate-previewFailed12,466
google/veo-3.1-lite-generate-previewFailed12,350
openai/gpt-5.3-codex-sparkFailed12,589

Method

  1. OpenCode listed the configured model entries.
  2. The harness started a new child process for each model.
  3. Each child process received the exact prompt Hi.
  4. The harness ran one model entry at a time.
  5. The harness allowed 100 seconds for each attempt.
  6. The harness did not retry failed attempts.

The command shape was opencode run --format json --model <model> Hi. Each attempt started a fresh OpenCode session.

The harness started its stopwatch before child process setup and launch. It stopped the watch after output processing and process completion.

Full elapsed time includes local process work, OpenCode startup, provider queues, response generation, streaming, output processing, and shutdown.

First-text time marks the first parsed JSON text event. The ranking uses full elapsed time instead of first-text time.

How to repeat this test

You can run the same benchmark on your own machine. The test uses a PowerShell script and the opencode CLI.

Requirements

  • PowerShell 7 or later.
  • The opencode CLI installed and on your PATH.
  • At least one model provider configured in your OpenCode credentials.

Step 1: list your configured models

OpenCode can list the models your key providers expose. Use the opencode config command to see which models are available. Copy the model identifiers you want to test.

Step 2: create the benchmark script

Save a PowerShell script that runs each model entry one at a time. The script starts a new opencode run child process for each entry. The script below is the harness we used. Replace the $models array with your own list.

$ErrorActionPreference = "Stop"

# Replace this list with your model identifiers
$models = @(
    "deepseek/deepseek-chat",
    "openai/gpt-5.4",
    "opencode/laguna-s-2.1-free"
)

$opencodeScript = Join-Path $env:USERPROFILE "AppData\Roaming\npm\opencode.ps1"
$resultPath = Join-Path $env:TEMP "opencode-model-benchmark.json"
$timeoutMs = 100000
$results = [System.Collections.Generic.List[object]]::new()
$benchmarkConfig = '{"default_agent":"benchmark","agent":{"benchmark":{"description":"Text-only model speed benchmark","mode":"primary","prompt":"Respond directly to the user message.","permission":{"*":"deny"}}}}'

foreach ($model in $models) {
    $stopwatch = [System.Diagnostics.Stopwatch]::StartNew()

    $startInfo = [System.Diagnostics.ProcessStartInfo]::new()
    $startInfo.FileName = "pwsh.exe"
    $startInfo.UseShellExecute = $false
    $startInfo.CreateNoWindow = $true
    $startInfo.RedirectStandardOutput = $true
    $startInfo.RedirectStandardError = $true
    foreach ($argument in @("-NoProfile", "-File", $opencodeScript, "run", "--pure", "--format", "json", "--title", "benchmark", "--agent", "benchmark", "--model", $model, "Hi")) {
        [void]$startInfo.ArgumentList.Add($argument)
    }
    $startInfo.Environment["OPENCODE_CONFIG_CONTENT"] = $benchmarkConfig

    $process = [System.Diagnostics.Process]::new()
    $process.StartInfo = $startInfo
    $firstOutputMs = $null
    $firstTextMs = $null
    $timedOut = $false
    $textParts = [System.Collections.Generic.List[string]]::new()

    try {
        [void]$process.Start()
        $stderrTask = $process.StandardError.ReadToEndAsync()
        $lineTask = $process.StandardOutput.ReadLineAsync()
        $stdoutEnded = $false

        while (-not $stdoutEnded) {
            if ($lineTask.Wait(20)) {
                $line = $lineTask.Result
                if ($null -eq $line) { $stdoutEnded = $true; continue }
                if ($null -eq $firstOutputMs) { $firstOutputMs = [math]::Round($stopwatch.Elapsed.TotalMilliseconds) }
                try {
                    $event = $line | ConvertFrom-Json
                    if ($event.type -eq "text" -and $event.part.text) {
                        if ($null -eq $firstTextMs) { $firstTextMs = [math]::Round($stopwatch.Elapsed.TotalMilliseconds) }
                        $textParts.Add([string]$event.part.text)
                    }
                } catch {}
                $lineTask = $process.StandardOutput.ReadLineAsync()
            }
            if ($stopwatch.ElapsedMilliseconds -ge $timeoutMs -and -not $process.HasExited) {
                $timedOut = $true
                $process.Kill($true)
            }
        }
        if (-not $process.HasExited) { [void]$process.WaitForExit(5000) }
    } catch {
        if (-not $process.HasExited) { $process.Kill($true) }
    } finally {
        $stopwatch.Stop()
        $process.Dispose()
    }

    $elapsedMs = [math]::Round($stopwatch.Elapsed.TotalMilliseconds)
    $response = [string]::Join("", $textParts)
    $status = if ($timedOut) { "timeout" } elseif (-not $response) { "no_text_response" } else { "success" }

    $results.Add([pscustomobject]@{
        model = $model
        elapsed_ms = $elapsedMs
        first_text_ms = $firstTextMs
        status = $status
        response = $response
    })
    Write-Output "MODEL=$model ELAPSED_MS=$elapsedMs STATUS=$status"
}

[System.IO.File]::WriteAllText($resultPath, ($results | ConvertTo-Json -Depth 5), [System.Text.UTF8Encoding]::new($false))
Write-Output "Results saved to $resultPath"

Step 3: run the benchmark

Execute the script in PowerShell 7. It runs one model entry at a time. The total run time depends on your model list and timeouts. Expect several minutes for ten or more entries.

The script sets a custom agent configuration through the OPENCODE_CONFIG_CONTENT environment variable. This agent replies directly to the user message. It does not use tool calls or extra processing.

Step 4: read the results

Open the output JSON file. Each record contains these fields.

  • model: the provider and model identifier that was tested.
  • elapsed_ms: full-wall-clock time from child-process start to exit or timeout.
  • first_text_ms: time from child-process start until the first parsed text event from the JSON stream.
  • status: success, timeout, or no_text_response.
  • response: the full text the model generated.

Sort successful entries by elapsed_ms to build your own latency ranking.

Entries with status timeout or no_text_response are not successful responses. Their elapsed values show failure detection latency, not usable response speed.

What the test measures

This benchmark measures complete OpenCode session-start latency. The stopwatch covers child-process launch, OpenCode startup, model provider request, text generation, streaming output processing, and process shutdown.

The measurement includes automatic title generation. OpenCode performs title generation during a normal first turn. Title generation adds extra provider traffic and latency.

This is a one-run snapshot. Run the benchmark several times and report percentile values if you want stronger conclusions.

Title generation affects this metric

OpenCode can request an automatic title when it starts a fresh session. This test included that normal first-turn behavior.

Title generation can add provider traffic, queue contention, latency, and cost. It can also use a different model from the tested model.

For these reasons, the results measure OpenCode session-start latency. They do not measure isolated model inference latency.

Sequential and parallel requests

The harness ran model attempts sequentially. This method avoided direct competition between benchmark attempts.

Applications often send parallel requests. Requests to one provider can then share quotas, queues, connections, or service capacity.

Provider bottlenecks can increase latency during parallel work. These sequential results do not predict that increase.

Limits of this test

  • Each model entry received one measured run.
  • One run cannot show normal variation or tail latency.
  • The short prompt does not represent large prompts or long responses.
  • Different response lengths affect full elapsed time.
  • Provider load and network conditions change over time.
  • The test measured latency only. It did not measure response quality, accuracy, or cost.
  • Failed entries received no ranking among successful responses.

A stronger test needs warm and cold runs, repeated samples, fixed output limits, and percentile results. It also needs separate parallel-load tests.

Summary

  • deepseek/deepseek-chat returned the lowest successful full elapsed time in this run.
  • The result measures a complete OpenCode first turn, including session-start work.
  • Automatic title generation can affect provider traffic, latency, and cost.
  • The benchmark used sequential attempts, so it does not measure parallel provider pressure.
  • This one-run snapshot cannot establish long-term performance or model quality.

Posted

in

by

Tags:

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *