An idle LLM server unloads the model and loads it again on the next
request. Nothing of the response is sent while that load runs, so a
request that arrives on a cold server can exceed read_timeout and fail --
reliably making the first search after an idle period the one that does
not get a summary. That same request is what starts the load, so asking
again succeeds.
Only a server that failed to answer is asked again. A server that did
answer with an error status is not: a wrong API key or an unknown model
name does not become right on a second attempt.
The status and body of such an error response are now logged. Until now
every upstream failure looked identical from the outside -- HTTP 502 with
no indication of whether the key, the model name or the network was at
fault.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
jasonwitty
4a582c0a15