jason/searxng
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[doc] ai_summary: describe the plugin, not the reasoning behind it

Browse Source 
Both pages carried design rationale and implementation notes that belong
in a review conversation rather than in documentation: why NDJSON is
sent to the browser instead of SSE, why the endpoint is registered in
webapp.py, why the outgoing proxy is bypassed. They are gone, along with
the troubleshooting section.

The section about public instances no longer opens by saying the plugin
is meant for private instances, which contradicted the instructions that
followed it. It now explains why the server, model and key are user
preferences at all -- so that they can be changed at home without
editing settings.yml -- and what that means once other people can reach
the instance.

"AI Summary" is the plugin, "AI summary" is the text it produces, and an
"instance" is always a SearXNG one; a page that also talks about LLM
servers cannot leave that to context.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
jasonwitty
2026-08-13 16:11:49 -07:00
parent 054b3a5c0d
commit 37d7d11959
2 changed files with 85 additions and 175 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/dev/plugins/ai_summary.rst
+38 -77
View File
@@ -11,23 +11,14 @@ AI Summary
- :ref:`result types`
The AI Summary plugin shows a generated answer above the ordinary search
results, unless an engine has already answered the query directly -- with a
Wikipedia infobox, for instance, or an instant answer. It is meant to run
against a local LLM server and speaks the `OpenAI chat completions API`_, so it
works with `Ollama`_, Hugging Face TGI, LiteLLM, vLLM, llama.cpp and anything
else that implements that specification. See :ref:`its configuration <settings
ai_summary>` for how to set one up.
The purpose of the summary is not to tell you what the model memorised during
training, but to summarise the up to date results your query actually returned.
That is what the *grounding* setting does, and why it is enabled by default.
Generating an answer takes seconds, and a search engine that waits seconds
before painting anything is a broken search engine. The summary is therefore
produced asynchronously: the plugin renders an empty box, the result page is
delivered immediately, and the browser fills that box from a second, streaming
request. The results below stay readable and scrollable the whole time.
results. The text comes from an LLM server run by the administrator, which
speaks the `OpenAI chat completions API`_ -- `Ollama`_, Hugging Face TGI,
LiteLLM, vLLM, llama.cpp and anything else implementing that specification.
See :ref:`its configuration <settings ai_summary>` for how to set one up.
The summary is generated asynchronously: the result page is delivered without
delay and carries an empty placeholder, which the browser fills from a second,
streaming request.
Request flow
============
@@ -45,7 +36,7 @@ Request flow
fillcolor="#f4f4f4", color="#999999"];
edge [fontname="sans-serif", fontsize=9, color="#666666"];
browser [label="browser\n(simple theme)"];
browser [label="browser"];
searxng [label="SearXNG"];
engines [label="search engines", fillcolor="#ffffff"];
llm [label="LLM server\nOllama, vLLM, ...", fillcolor="#ffffff"];
@@ -59,84 +50,54 @@ Request flow
searxng -> browser [label=" 7 NDJSON token stream", constraint=false];
}
Steps 1--3 are an ordinary SearXNG search. The plugin's
:py:obj:`post_search <searx.plugins.ai_summary.SXNGPlugin.post_search>` hook
adds an empty :py:obj:`searx.result_types.AiSummary` placeholder to the answer
area and returns immediately, so the page is not delayed.
Steps 1 to 3 are an ordinary SearXNG search. :py:obj:`SXNGPlugin.post_search
<searx.plugins.ai_summary.SXNGPlugin.post_search>` adds an empty
:py:obj:`searx.result_types.AiSummary` placeholder to the answer area and
returns; the result page is not delayed.
Steps 4--7 happen in the browser after the page has painted.
Steps 4 to 7 run in the browser once the page is rendered.
``client/simple/src/js/plugin/AiSummary.ts`` posts to the ``/ai_summary``
endpoint, which opens a streaming request to the LLM server and re-emits the
tokens as they arrive. The user sees the answer being written.
Two format changes happen along the way. The LLM server speaks `SSE`_
(``data: {...}`` lines, terminated by ``data: [DONE]``), because that is what
the OpenAI chat completions API specifies. SearXNG re-emits that to the
browser as `NDJSON`_ -- one JSON object per line, ``{"delta": "..."}`` for each
chunk of text and a final ``{"done": true}``. NDJSON is used because the
browser reads the body with ``fetch`` and a stream reader, where SSE's
``EventSource`` would be the wrong tool: ``EventSource`` cannot issue a POST.
endpoint (registered in :py:obj:`searx.webapp`), which opens a streaming
request to the LLM server and re-emits the tokens as they arrive.
The two streams use different formats. The LLM server sends `SSE`_ --
``data: {...}`` lines terminated by ``data: [DONE]``. SearXNG re-emits them to
the browser as `NDJSON`_: one JSON object per line, ``{"delta": "..."}`` for
each chunk of text and a final ``{"done": true}``.
When no summary is generated
============================
:py:obj:`post_search <searx.plugins.ai_summary.SXNGPlugin.post_search>` skips
the placeholder entirely for:
:py:obj:`SXNGPlugin.post_search
<searx.plugins.ai_summary.SXNGPlugin.post_search>` adds no placeholder for:
- page two and beyond -- a summary belongs with the first impression of a query
- anything but the *general* category
- page two and beyond
- categories other than *general*
- non-HTML output formats (the JSON, CSV and RSS APIs)
- queries where an engine already produced an infobox (wikipedia, wikidata) or
- queries an engine already answered with an infobox (wikipedia, wikidata) or
an instant answer (e.g. ddg definitions)
- an empty query, or no LLM server configured
The infobox rule mirrors what the big engines do: if the query is a lookup of a
well known entity, that entity's own data is a better answer than a generated
paragraph.
API keys
========
Where the API key goes
======================
The administrator's ``api_key`` is only ever sent to the administrator's
``base_url``. This matters because users may set their own server in the
``ai_summary_server`` preference: without the check, any user of the instance
could point that preference at a host they control and collect the instance's
key from the ``Authorization`` header. Users authenticate to their own server
with their own ``ai_summary_api_key`` preference, which is stored in a cookie
and deliberately excluded from the shareable preferences URL.
The administrator's ``api_key`` is sent only to the configured ``base_url``.
Users who point the ``ai_summary_server`` preference at a server of their own
authenticate it with their own ``ai_summary_api_key`` preference, which is
stored in a cookie and excluded from the preferences URL; the administrator's
key is never sent to such a server. A server URL carrying credentials in its
userinfo is ignored, and the administrator's default is used instead.
:py:obj:`_server_api_key <searx.plugins.ai_summary._server_api_key>` implements
the rule; a URL carrying credentials in its userinfo is rejected outright,
because HTTP clients turn that into an ``Authorization`` header of the user's
choosing.
Implementation notes
====================
The ``/ai_summary`` route is registered in :py:obj:`searx.webapp`, next to the
favicon proxy, rather than in the plugin's ``init()``. Flask does not allow
``add_url_rule`` after the first request has been handled, and registering it
from a plugin breaks the test suite.
Requests to the LLM server bypass :py:obj:`searx.network` and are sent with a
plain :py:obj:`httpx.Client`. The outgoing proxy configuration is deliberately
not applied: an LLM server usually sits on localhost or in the local network,
which is exactly what an outgoing proxy is configured to avoid.
The streaming response uses ``direct_passthrough``, so the generator must yield
``bytes`` -- werkzeug asserts on ``str``, and the Flask test client does not
catch it.
.. _Ollama: https://ollama.com/
.. _OpenAI chat completions API: https://platform.openai.com/docs/api-reference/chat
.. _SSE: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events
.. _NDJSON: https://github.com/ndjson/ndjson-spec
the rule.
Reference
=========
.. automodule:: searx.plugins.ai_summary
:members:
.. _Ollama: https://ollama.com/
.. _OpenAI chat completions API: https://platform.openai.com/docs/api-reference/chat
.. _SSE: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events
.. _NDJSON: https://github.com/ndjson/ndjson-spec