LCP slot reuse segfaults on identical-prompt retry (server_tokens::get_common_prefix) #1

New issue

Closed

opened 2026-08-05 16:35:48 +02:00 by kade · 0 comments

kade commented

2026-08-05 16:35:48 +02:00

Owner

Bug: Identical-prompt LCP reuse segfaults the server

Summary

Submitting the same prompt twice to the server causes a deterministic segfault in server_tokens::get_common_prefix() during LCP slot reuse. This is a production blocker for any agentic workload that retries identical requests.

Reproduction

Start server with --ctx-checkpoints 50 --cache-ram 11500 (or any config that enables the prompt cache)
Send a 300-token request (thinking-ON or thinking-OFF, stream=false)
Send the same prompt again immediately
Server segfaults within seconds

Backtrace (from gdb session, Thread 1)

#0  server_tokens::get_common_prefix(server_tokens const&) const
    (tools/server/server-common.cpp:471)
#1  server_prompt_cache::load(server_prompt&, server_tokens const&, ...)
    (tools/server/server-task.cpp:2379)
#2  server_context_impl::get_available_slot(server_task const&)
#3  server_context_impl::process_single_task(server_task&&)
#4  server_queue::start_loop(long)

Root Cause Hypothesis

index_find_best() returns a prompt_iterator into the states deque. After a prior alloc() or load() erases an entry via states.erase(it_best), the index may still reference the erased entry. On the next call, get_common_prefix() dereferences a dangling iterator -> segfault at address 0xffffffff.

The probe_recent_match_cache mechanism (which caches prompt_iterator values) is the likely culprit -- recorded iterators become invalid after states.erase() without being invalidated.

Severity

Blocker. Any retry logic in agentic workloads (including the r harness loadgen) can trigger this.

Environment

Branch: apply-pr-26323 (HEAD c0ce81f43)
Build: ~/src/llama.cpp/build/bin/llama-server
Model: Ornith-1.0-35B-IQ4_NL
Flags: --ctx-checkpoints 50 --cache-ram 11500 --reasoning-preserve

Reproduction Script

See ~/work/atsinfer_Launcher/experiments/think-ab.sh and lcp-repro.sh for automated repro. The think-ab.sh script reliably crashes on round 2 of thinking-ON concurrent requests.

## Bug: Identical-prompt LCP reuse segfaults the server ### Summary Submitting the same prompt twice to the server causes a deterministic segfault in `server_tokens::get_common_prefix()` during LCP slot reuse. This is a production blocker for any agentic workload that retries identical requests. ### Reproduction 1. Start server with `--ctx-checkpoints 50 --cache-ram 11500` (or any config that enables the prompt cache) 2. Send a 300-token request (thinking-ON or thinking-OFF, stream=false) 3. Send the **same** prompt again immediately 4. Server segfaults within seconds ### Backtrace (from gdb session, Thread 1) ``` #0 server_tokens::get_common_prefix(server_tokens const&) const (tools/server/server-common.cpp:471) #1 server_prompt_cache::load(server_prompt&, server_tokens const&, ...) (tools/server/server-task.cpp:2379) #2 server_context_impl::get_available_slot(server_task const&) #3 server_context_impl::process_single_task(server_task&&) #4 server_queue::start_loop(long) ``` ### Root Cause Hypothesis `index_find_best()` returns a `prompt_iterator` into the `states` deque. After a prior `alloc()` or `load()` erases an entry via `states.erase(it_best)`, the index may still reference the erased entry. On the next call, `get_common_prefix()` dereferences a dangling iterator -> segfault at address `0xffffffff`. The `probe_recent_match_cache` mechanism (which caches `prompt_iterator` values) is the likely culprit -- recorded iterators become invalid after `states.erase()` without being invalidated. ### Severity Blocker. Any retry logic in agentic workloads (including the r harness loadgen) can trigger this. ### Environment - Branch: `apply-pr-26323` (HEAD `c0ce81f43`) - Build: `~/src/llama.cpp/build/bin/llama-server` - Model: Ornith-1.0-35B-IQ4_NL - Flags: `--ctx-checkpoints 50 --cache-ram 11500 --reasoning-preserve` ### Reproduction Script See `~/work/atsinfer_Launcher/experiments/think-ab.sh` and `lcp-repro.sh` for automated repro. The `think-ab.sh` script reliably crashes on round 2 of thinking-ON concurrent requests.

kade referenced this issue

2026-08-05 16:46:43 +02:00

ATSInfer benchmark campaign P0-P3 results and recommendations #2

kade referenced this issue from a commit

2026-08-05 17:35:59 +02:00

fix(server): invalidate LCP recent match cache on state erase

kade referenced this issue from a commit

2026-08-05 19:03:00 +02:00

fix(server): declare server_ws_is_connected in opaque WebSocket bridge header

kade referenced this issue from a commit

2026-08-05 19:03:00 +02:00

fix(server): propagate has_mtmd flag through tokenize_input_subprompt