A local LLM is not slow if you do not cold-start it for every question

Situation

The first request to a local LLM can be much slower because the model is not yet resident in memory. If automation treats that as an ordinary timeout, the system appears randomly unreliable. Requirement extraction also did not need minutes of creative reasoning; it needed short, schema-conformant information extraction.

Approach

I added model warmup at startup and gave health and smoke checks retries that account for real boot time. Requirement extraction used a separate structured no-thinking provider path optimized for precise JSON-like output. Task timeouts were separated from infrastructure unavailability so the recovery behaviour could differ between a slow task and an unavailable provider.

Outcome

Ollama review became faster and more predictable. Cold startup stopped masquerading as a model failure, and simple extraction no longer paid the full cost of a general reasoning loop. Diagnostics clearly distinguished cache hits, timeouts and provider outages.