jasonwitty
8ff85790cf
Grounding is on by default, and it sends the top search results along with the query. Which data leaves the instance depends on where the LLM server runs -- per the privacy-by-design directive the reader has to be able to see that from the documentation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
84 lines
3.1 KiB
ReStructuredText
84 lines
3.1 KiB
ReStructuredText
.. _settings ai_summary:
|
|
|
|
===============
|
|
``ai_summary:``
|
|
===============
|
|
|
|
Default configuration of the :ref:`AI summary plugin <ai_summary plugin>`.
|
|
Users configure the LLM server URL (any server implementing the OpenAI chat
|
|
completions API: Ollama, vLLM, llama.cpp, LM Studio, Hugging Face TGI, ...)
|
|
and the model in the *AI Summary* tab of their preferences; the values below
|
|
only act as instance wide defaults.
|
|
|
|
.. code:: yaml
|
|
|
|
ai_summary:
|
|
base_url: "http://127.0.0.1:11434"
|
|
model: "llama3.2:3b"
|
|
|
|
An LLM server that requires authentication -- e.g. vLLM or llama.cpp started
|
|
with ``--api-key``, or a server behind an authenticating reverse proxy -- is
|
|
configured with an ``api_key``:
|
|
|
|
.. code:: yaml
|
|
|
|
ai_summary:
|
|
base_url: "http://127.0.0.1:8000"
|
|
api_key: "sk-..."
|
|
model: "llama3.2:3b"
|
|
|
|
The key is sent in an ``Authorization: Bearer`` header and only to the
|
|
``base_url`` above. A user who points the ``ai_summary_server`` preference at
|
|
a server of their own never gets the administrator's key; for such a server
|
|
the user configures their own key in the ``ai_summary_api_key`` preference.
|
|
SearXNG has no indirection for secrets in ``settings.yml``, so the file
|
|
holding the key should be readable by the SearXNG process only.
|
|
|
|
The *AI Summary* tab of the preferences is only shown when the plugin is
|
|
activated in ``settings.yml`` (``active: true``); an instance that does not
|
|
offer AI summaries does not show the tab at all.
|
|
|
|
.. _ai_summary grounding:
|
|
|
|
Grounding
|
|
=========
|
|
|
|
With ``grounding`` (the default) the query **and the top search results**
|
|
(title, URL and snippet of at most ``max_context_items`` results) are sent to
|
|
the LLM server; without it only the query is sent and the model answers from
|
|
its own knowledge. Grounded answers are more accurate and more current, at
|
|
the cost of a longer prompt.
|
|
|
|
Which data leaves the SearXNG instance therefore depends on where the LLM
|
|
server runs: with a server on localhost or in the local network nothing leaves
|
|
the network, with a hosted server the query and the result snippets are sent
|
|
to that provider. This is why the plugin is not activated by default and why
|
|
its documentation recommends a local LLM server. Users can opt out of
|
|
grounding in their preferences (``ai_summary_grounding``).
|
|
|
|
.. attention::
|
|
|
|
A user configurable server URL allows any user of the instance to make the
|
|
SearXNG server send requests to a URL of their choice (`SSRF`_), and each
|
|
summary is real LLM work. This plugin is intended for private instances --
|
|
on a public instance lock the related preferences (:ref:`settings
|
|
preferences`):
|
|
|
|
.. code:: yaml
|
|
|
|
preferences:
|
|
lock:
|
|
- ai_summary_server
|
|
- ai_summary_api_key
|
|
- ai_summary_model
|
|
- ai_summary_grounding
|
|
|
|
Locking ``ai_summary_server`` and ``ai_summary_api_key`` matters most: an
|
|
unlocked pair lets any user of the instance make SearXNG send an
|
|
``Authorization`` header of their choosing to a host of their choosing.
|
|
|
|
.. _SSRF: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery
|
|
|
|
.. autoclass:: searx.ai_summary.SettingsAISummary
|
|
:members:
|