jason/searxng
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[fix] plugin: ask the LLM server twice before giving up

Browse Source 
An idle LLM server unloads the model and loads it again on the next
request.  Nothing of the response is sent while that load runs, so a
request that arrives on a cold server can exceed read_timeout and fail --
reliably making the first search after an idle period the one that does
not get a summary.  That same request is what starts the load, so asking
again succeeds.

Only a server that failed to answer is asked again.  A server that did
answer with an error status is not: a wrong API key or an unknown model
name does not become right on a second attempt.

The status and body of such an error response are now logged.  Until now
every upstream failure looked identical from the outside -- HTTP 502 with
no indication of whether the key, the model name or the network was at
fault.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
jasonwitty
2026-08-11 11:37:06 -07:00
co-authored by Claude Opus 5
parent ce02bebf7b
commit 4a582c0a15
3 changed files with 132 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/admin/settings/settings_ai_summary.rst
+10 -1
View File
@@ -201,7 +201,16 @@ Troubleshooting
**The summary box shows an error.**
SearXNG could not reach the LLM server, or the server rejected the request.
Check ``base_url`` from the SearXNG machine, confirm the model name exists on
that server (``ollama list``), and check the SearXNG log.
that server (``ollama list``), and check the SearXNG log -- it records the
status and the message the server replied with, which usually names the cause.
**The first search after a while fails, the next one works.**
An idle LLM server unloads the model and has to load it again, and it sends
nothing at all while doing so. If that takes longer than ``read_timeout``
(30 s by default) the request is abandoned. SearXNG repeats the request once,
which covers a normal load, but a large model on slow storage can need more:
raise ``read_timeout``, or keep the model in memory -- with Ollama, set
``OLLAMA_KEEP_ALIVE`` (for example ``-1`` to never unload it).
**The summary starts, then stops mid-sentence.**
The answer exceeded ``stream_timeout`` (120 s by default). Large models on