jason/searxng
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[doc] ai_summary: rewrite for readers new to the feature

Browse Source 
The documentation was written from the perspective of someone who had
just implemented the plugin: it opened with SSRF caveats and settings
keys, and explained decisions rather than usage.

The admin page now starts from what the feature is, followed by a
four-step quickstart (install Ollama, pull a model, configure, restart)
and a troubleshooting section for the failures that actually occur.  The
quickstart repeats the default plugins because a plugins: block replaces
that list instead of merging into it -- following the short version of
the instructions would otherwise switch every other plugin off.

The developer page gains a rendered data flow diagram.  The two-request
design -- placeholder first, streamed answer second -- is the part of
this plugin that is hard to convey in prose, and the SSE to NDJSON
change is easier to see than to read about.

The module docstring now describes the module and links to both pages,
instead of restating administration guidance.

Add Jason Witty to AUTHORS.rst.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
jasonwitty
2026-08-10 12:30:53 -07:00
parent 8ff85790cf
commit b6eed9a993
4 changed files with 326 additions and 103 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AUTHORS.rst
+1
View File
@@ -179,3 +179,4 @@ features or generally made SearXNG better:
- Tommaso Colella `<https://github.com/gioleppe>`
- @AgentScrubbles
- Filip Mikina `<https://github.com/fiffek>`
- Jason Witty `<https://github.com/jasonwitty>`
docs/admin/settings/settings_ai_summary.rst
+183 -48
View File
@@ -4,80 +4,215 @@
``ai_summary:``
===============
Default configuration of the :ref:`AI summary plugin <ai_summary plugin>`.
Users configure the LLM server URL (any server implementing the OpenAI chat
completions API: Ollama, vLLM, llama.cpp, LM Studio, Hugging Face TGI, ...)
and the model in the *AI Summary* tab of their preferences; the values below
only act as instance wide defaults.
.. sidebar:: Further reading ..
- :ref:`ai_summary plugin`
- :ref:`settings plugins`
- :ref:`settings preferences`
Configuration of the :ref:`AI summary plugin <ai_summary plugin>`, which shows a
short AI generated answer above the search results.
The text is produced by an **LLM server that you run** -- SearXNG does not ship
a model and does not talk to any AI provider on its own. Anything that speaks
the `OpenAI chat completions API`_ works: `Ollama`_, vLLM, llama.cpp, LM Studio,
Hugging Face TGI and others. The plugin is not activated by default.
.. contents::
:depth: 2
:local:
:backlinks: entry
.. _ai_summary quickstart:
Quickstart
==========
A local setup on the same machine as SearXNG, in four steps.
**1. Install Ollama**
.. code:: sh
curl -fsSL https://ollama.com/install.sh | sh
**2. Download a model**
.. code:: sh
ollama pull gemma3:4b
``gemma3:4b`` needs roughly 4 GB of memory and runs on CPU if you have no GPU.
On a small machine use ``gemma3:1b`` instead; any model in the `Ollama
library`_ works.
**3. Configure SearXNG**
Add this to your ``settings.yml``:
.. code:: yaml
ai_summary:
base_url: "http://127.0.0.1:11434"
model: "llama3.2:3b"
model: "gemma3:4b"
An LLM server that requires authentication -- e.g. vLLM or llama.cpp started
with ``--api-key``, or a server behind an authenticating reverse proxy -- is
configured with an ``api_key``:
plugins:
searx.plugins.ai_summary.SXNGPlugin:
active: true
# keep the plugins you already had, see the warning below
searx.plugins.calculator.SXNGPlugin: {active: true}
searx.plugins.hash_plugin.SXNGPlugin: {active: true}
searx.plugins.self_info.SXNGPlugin: {active: true}
searx.plugins.unit_converter.SXNGPlugin: {active: true}
searx.plugins.ahmia_filter.SXNGPlugin: {active: true}
searx.plugins.hostnames.SXNGPlugin: {active: true}
searx.plugins.time_zone.SXNGPlugin: {active: true}
searx.plugins.tracker_url_remover.SXNGPlugin: {active: true}
searx.plugins.infinite_scroll.SXNGPlugin: {active: false}
searx.plugins.oa_doi_rewrite.SXNGPlugin: {active: false}
searx.plugins.tor_check.SXNGPlugin: {active: false}
.. warning::
A ``plugins:`` block **replaces** the default list, it is not merged into it
(:ref:`settings plugins`). If you list only the AI summary plugin, every
other plugin is switched off. That is why the block above repeats the
defaults -- drop the lines for plugins you do not want.
**4. Restart SearXNG** and search for something.
The summary appears above the results while it is still being written. If
nothing appears, see :ref:`ai_summary troubleshooting`.
Options
=======
Only ``base_url`` is required; a model is needed too, but if you leave ``model``
empty the first entry of ``models`` is used.
.. code:: yaml
ai_summary:
base_url: "http://127.0.0.1:11434"
model: "gemma3:4b"
grounding: true
.. autoclass:: searx.ai_summary.SettingsAISummary
:members:
Servers that need a password
============================
Some servers require authentication -- vLLM and llama.cpp when started with
``--api-key``, a gateway such as LiteLLM, or any LLM server placed behind an
authenticating reverse proxy. Give SearXNG the key with ``api_key``:
.. code:: yaml
ai_summary:
base_url: "http://127.0.0.1:8000"
api_key: "sk-..."
model: "llama3.2:3b"
model: "gemma3:4b"
The key is sent in an ``Authorization: Bearer`` header and only to the
``base_url`` above. A user who points the ``ai_summary_server`` preference at
a server of their own never gets the administrator's key; for such a server
the user configures their own key in the ``ai_summary_api_key`` preference.
SearXNG has no indirection for secrets in ``settings.yml``, so the file
holding the key should be readable by the SearXNG process only.
The key is sent as an ``Authorization: Bearer`` header, and only to the server
in ``base_url``. Users who point the ``ai_summary_server`` preference at a
server of their own never receive it; they set their own key in the
``ai_summary_api_key`` preference instead.
SearXNG has no separate secret store, so the key sits in ``settings.yml`` --
make sure that file is readable only by the user SearXNG runs as.
The *AI Summary* tab of the preferences is only shown when the plugin is
activated in ``settings.yml`` (``active: true``); an instance that does not
offer AI summaries does not show the tab at all.
.. _ai_summary grounding:
Grounding
=========
With ``grounding`` (the default) the query **and the top search results**
(title, URL and snippet of at most ``max_context_items`` results) are sent to
the LLM server; without it only the query is sent and the model answers from
its own knowledge. Grounded answers are more accurate and more current, at
the cost of a longer prompt.
Grounding decides *what the model is told*:
Which data leaves the SearXNG instance therefore depends on where the LLM
server runs: with a server on localhost or in the local network nothing leaves
the network, with a hosted server the query and the result snippets are sent
to that provider. This is why the plugin is not activated by default and why
its documentation recommends a local LLM server. Users can opt out of
grounding in their preferences (``ai_summary_grounding``).
``grounding: true`` (the default)
The query **and the top search results** (title, URL and snippet, at most
``max_context_items`` of them) are sent to the LLM server. Answers reflect
what the search actually found, so they are more accurate and more current.
``grounding: false``
Only the query is sent, and the model answers from its own training data.
Faster and cheaper, but the answer can be outdated or invented.
Users can switch this in their preferences (``ai_summary_grounding``).
How much this matters for privacy depends entirely on where the LLM server runs.
With a server on localhost or in your own network, nothing leaves that network.
With a hosted server, the query and the result snippets are sent to the
provider. This is the main reason a local server is recommended.
Public instances
================
.. attention::
A user configurable server URL allows any user of the instance to make the
SearXNG server send requests to a URL of their choice (`SSRF`_), and each
summary is real LLM work. This plugin is intended for private instances --
on a public instance lock the related preferences (:ref:`settings
preferences`):
The plugin is designed for private instances. Read this before enabling it
on a public one.
.. code:: yaml
Two things change on a public instance:
preferences:
lock:
- ai_summary_server
- ai_summary_api_key
- ai_summary_model
- ai_summary_grounding
**Every search costs real work.** A summary is a full LLM inference. Traffic
that was cheap to serve becomes expensive, on hardware you pay for.
Locking ``ai_summary_server`` and ``ai_summary_api_key`` matters most: an
unlocked pair lets any user of the instance make SearXNG send an
``Authorization`` header of their choosing to a host of their choosing.
**Users can choose the server SearXNG talks to.** The ``ai_summary_server``
preference makes your instance send requests to an address of the user's
choosing, which is a `SSRF`_ vector: it can be pointed at services inside your
network that are not reachable from outside.
Lock the preferences so that only your configuration is used
(:ref:`settings preferences`):
.. code:: yaml
preferences:
lock:
- ai_summary_server
- ai_summary_api_key
- ai_summary_model
- ai_summary_grounding
Locking ``ai_summary_server`` closes the SSRF vector; locking
``ai_summary_api_key`` stops users making your instance send an
``Authorization`` header of their choosing to a host of their choosing.
.. _ai_summary troubleshooting:
Troubleshooting
===============
**No summary appears at all.**
The plugin deliberately stays quiet in several cases: on page two and beyond,
outside the *general* category, and when an engine already answered the query
with an infobox or an instant answer. Try a question-like query on the first
page. Check that the plugin is enabled both in ``settings.yml`` and in your
own preferences.
**The model list in the preferences is empty.**
SearXNG asks the server for its models (``GET /v1/models``) once at startup.
An empty list means that request failed -- the server was not running yet,
the URL is wrong, or it needs an ``api_key``. The log records the reason;
restart SearXNG after fixing it.
**The summary box shows an error.**
SearXNG could not reach the LLM server, or the server rejected the request.
Check ``base_url`` from the SearXNG machine, confirm the model name exists on
that server (``ollama list``), and check the SearXNG log.
**The summary starts, then stops mid-sentence.**
The answer exceeded ``stream_timeout`` (120 s by default). Large models on
CPU are slow; either raise the limit or use a smaller model.
.. _Ollama: https://ollama.com/
.. _Ollama library: https://ollama.com/library
.. _OpenAI chat completions API: https://platform.openai.com/docs/api-reference/chat
.. _SSRF: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery
.. autoclass:: searx.ai_summary.SettingsAISummary
:members:
docs/dev/plugins/ai_summary.rst
+129
View File
@@ -4,5 +4,134 @@
AI summary
==========
.. sidebar:: Further reading ..
- :ref:`settings ai_summary`
- :ref:`dev plugin`
- :ref:`result types`
.. contents::
:depth: 2
:local:
:backlinks: entry
The AI summary plugin shows a generated answer above the search results. The
text comes from an LLM server the administrator runs; see :ref:`settings
ai_summary` for how to set one up.
The interesting part of this plugin is *when* things happen. Generating an
answer takes seconds, and a search engine that waits seconds before painting
anything is a broken search engine. So the plugin never blocks the result
page: it renders an empty box, and the browser fills that box afterwards from a
second, streaming request.
Request flow
============
.. _ai_summary dataflow:
.. kernel-render:: DOT
:alt: Data flow between browser, SearXNG and the LLM server
:caption: A search that produces a summary: two requests, not one
digraph ai_summary {
rankdir=LR;
graph [fontname="sans-serif", ranksep=1.1, nodesep=0.4];
node [fontname="sans-serif", fontsize=11, shape=box, style="rounded,filled",
fillcolor="#f4f4f4", color="#999999"];
edge [fontname="sans-serif", fontsize=9, color="#666666"];
browser [label="browser\n(simple theme)"];
searxng [label="SearXNG"];
engines [label="search engines", fillcolor="#ffffff"];
llm [label="LLM server\nOllama, vLLM, ...", fillcolor="#ffffff"];
browser -> searxng [label=" 1 GET /search"];
searxng -> engines [label=" 2 query"];
searxng -> browser [label=" 3 result page,\l empty summary box\l", constraint=false];
browser -> searxng [label=" 4 POST /ai_summary\l (query + results)\l"];
searxng -> llm [label=" 5 POST /v1/chat/completions"];
llm -> searxng [label=" 6 SSE token stream", constraint=false];
searxng -> browser [label=" 7 NDJSON token stream", constraint=false];
}
Steps 1--3 are an ordinary SearXNG search. The plugin's
:py:obj:`post_search <searx.plugins.ai_summary.SXNGPlugin.post_search>` hook
adds an empty :py:obj:`searx.result_types.AiSummary` placeholder to the answer
area and returns immediately, so the page is not delayed.
Steps 4--7 happen in the browser after the page has painted.
``client/simple/src/js/plugin/AiSummary.ts`` posts to the ``/ai_summary``
endpoint, which opens a streaming request to the LLM server and re-emits the
tokens as they arrive. The user sees the answer being written.
Two format changes happen along the way. The LLM server speaks `SSE`_
(``data: {...}`` lines, terminated by ``data: [DONE]``), because that is what
the OpenAI chat completions API specifies. SearXNG re-emits that to the
browser as `NDJSON`_ -- one JSON object per line, ``{"delta": "..."}`` for each
chunk of text and a final ``{"done": true}``. NDJSON is used because the
browser reads the body with ``fetch`` and a stream reader, where SSE's
``EventSource`` would be the wrong tool: ``EventSource`` cannot issue a POST.
When no summary is generated
============================
:py:obj:`post_search <searx.plugins.ai_summary.SXNGPlugin.post_search>` skips
the placeholder entirely for:
- page two and beyond -- a summary belongs with the first impression of a query
- anything but the *general* category
- non-HTML output formats (the JSON, CSV and RSS APIs)
- queries where an engine already produced an infobox (wikipedia, wikidata) or
an instant answer (e.g. ddg definitions)
- an empty query, or no LLM server configured
The infobox rule mirrors what the big engines do: if the query is a lookup of a
well known entity, that entity's own data is a better answer than a generated
paragraph.
Where the API key goes
======================
The administrator's ``api_key`` is only ever sent to the administrator's
``base_url``. This matters because users may set their own server in the
``ai_summary_server`` preference: without the check, any user of the instance
could point that preference at a host they control and collect the instance's
key from the ``Authorization`` header. Users authenticate to their own server
with their own ``ai_summary_api_key`` preference, which is stored in a cookie
and deliberately excluded from the shareable preferences URL.
:py:obj:`_server_api_key <searx.plugins.ai_summary._server_api_key>` implements
the rule; a URL carrying credentials in its userinfo is rejected outright,
because HTTP clients turn that into an ``Authorization`` header of the user's
choosing.
Implementation notes
====================
The ``/ai_summary`` route is registered in :py:obj:`searx.webapp`, next to the
favicon proxy, rather than in the plugin's ``init()``. Flask does not allow
``add_url_rule`` after the first request has been handled, and registering it
from a plugin breaks the test suite.
Requests to the LLM server bypass :py:obj:`searx.network` and are sent with a
plain :py:obj:`httpx.Client`. The outgoing proxy configuration is deliberately
not applied: an LLM server usually sits on localhost or in the local network,
which is exactly what an outgoing proxy is configured to avoid.
The streaming response uses ``direct_passthrough``, so the generator must yield
``bytes`` -- werkzeug asserts on ``str``, and the Flask test client does not
catch it.
.. _SSE: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events
.. _NDJSON: https://github.com/ndjson/ndjson-spec
Reference
=========
.. automodule:: searx.plugins.ai_summary
:members:
searx/plugins/ai_summary.py
+13 -55
View File
@@ -1,64 +1,22 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Plugin that displays an AI generated summary of the search query at the top
of the result page. The summary is generated by a (local) LLM server that
implements the `OpenAI chat completions API`_ -- e.g. `Ollama`_, vLLM,
llama.cpp, LM Studio or Hugging Face TGI.
"""Implementation of the AI summary plugin, which shows a generated answer above
the search results. The answer comes from an LLM server that implements the
`OpenAI chat completions API`_ (Ollama, vLLM, llama.cpp, LM Studio, Hugging Face
TGI, ...) and that the administrator runs.
The LLM server URL and the model are configured by the user in the *AI
Summary* tab of the preferences (``ai_summary_server``, ``ai_summary_model``);
the administrator can configure instance wide defaults in the ``ai_summary:``
section and lock the preferences via :ref:`settings preferences`.
- :ref:`ai_summary plugin` describes the design and the request flow.
- :ref:`settings ai_summary` describes how to configure it.
.. attention::
This module holds the plugin itself and the ``/ai_summary`` endpoint
(:py:obj:`ai_summary_view`, registered in :py:obj:`searx.webapp`). The endpoint
streams the answer to the browser, so that the result page is never delayed by
the LLM; :py:obj:`SXNGPlugin.post_search` only adds an empty
:py:obj:`searx.result_types.AiSummary` placeholder for the client to fill.
A user configurable server URL allows any user of the instance to make the
SearXNG server send requests to a URL of their choice (`SSRF`_), and each
summary is real LLM work. This plugin is intended for private instances --
on a public instance, lock the ``ai_summary_server``, ``ai_summary_model``
and ``ai_summary_grounding`` preferences and configure the ``ai_summary:``
section instead.
Settings of the ``ai_summary:`` section are defined in
:py:obj:`searx.ai_summary.SettingsAISummary`.
The result page is never delayed by this plugin: it only places an empty
placeholder (:py:obj:`searx.result_types.AiSummary`) in the answer area, which
is filled asynchronously by the client (``client/simple/src/js/plugin/
AiSummary.ts``) from the ``/ai_summary`` endpoint (registered in
:py:obj:`searx.webapp`). The endpoint re-emits the SSE token stream of the
LLM server's ``/v1/chat/completions`` to the client as `NDJSON`_.
A summary is only generated on the first page of a *general* search and only
if no engine has contributed an infobox (e.g. wikipedia / wikidata) or an
instant answer (e.g. ddg definitions) -- in these cases the query is most
likely a lookup of a well known term that is already answered.
The requests to the LLM server are sent directly (not via
:py:obj:`searx.network`), an outgoing proxy configuration is deliberately not
applied to reach an LLM server in the local network.
A server that requires authentication (e.g. vLLM or llama.cpp started with
``--api-key``, or an LLM server behind an authenticating reverse proxy) is
configured with an ``api_key``. The administrator's key is only sent to the
server in ``base_url``, never to a server a user configured; for their own
server users configure their own key in the ``ai_summary_api_key`` preference
(:py:obj:`_server_api_key`).
Configuration of the defaults (:py:obj:`searx.ai_summary.SettingsAISummary`):
.. code:: yaml
ai_summary:
base_url: "http://127.0.0.1:11434"
model: "llama3.2:3b"
.. code:: yaml
plugins:
searx.plugins.ai_summary.SXNGPlugin:
active: false
.. _Ollama: https://ollama.com/
.. _OpenAI chat completions API: https://platform.openai.com/docs/api-reference/chat
.. _NDJSON: https://github.com/ndjson/ndjson-spec
.. _SSRF: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery
"""
import typing as t