searxng/docs/admin/settings/settings_ai_summary.rst

.. _settings ai_summary:

===============
``ai_summary:``
===============

Default configuration of the :ref:`AI summary plugin <ai_summary plugin>`.
Users configure the LLM server URL (any server implementing the OpenAI chat
completions API: Ollama, vLLM, llama.cpp, LM Studio, Hugging Face TGI, ...)
and the model in the *AI Summary* tab of their preferences; the values below
only act as instance wide defaults.

.. code:: yaml

   ai_summary:
     base_url: "http://127.0.0.1:11434"
     model: "llama3.2:3b"

An LLM server that requires authentication -- e.g. vLLM or llama.cpp started
with ``--api-key``, or a server behind an authenticating reverse proxy -- is
configured with an ``api_key``:

.. code:: yaml

   ai_summary:
     base_url: "http://127.0.0.1:8000"
     api_key: "sk-..."
     model: "llama3.2:3b"

The key is sent in an ``Authorization: Bearer`` header and only to the
``base_url`` above.  A user who points the ``ai_summary_server`` preference at
a server of their own never gets the administrator's key; for such a server
the user configures their own key in the ``ai_summary_api_key`` preference.
SearXNG has no indirection for secrets in ``settings.yml``, so the file
holding the key should be readable by the SearXNG process only.

The *AI Summary* tab of the preferences is only shown when the plugin is
activated in ``settings.yml`` (``active: true``); an instance that does not
offer AI summaries does not show the tab at all.

.. _ai_summary grounding:

Grounding
=========

With ``grounding`` (the default) the query **and the top search results**
(title, URL and snippet of at most ``max_context_items`` results) are sent to
the LLM server; without it only the query is sent and the model answers from
its own knowledge.  Grounded answers are more accurate and more current, at
the cost of a longer prompt.

Which data leaves the SearXNG instance therefore depends on where the LLM
server runs: with a server on localhost or in the local network nothing leaves
the network, with a hosted server the query and the result snippets are sent
to that provider.  This is why the plugin is not activated by default and why
its documentation recommends a local LLM server.  Users can opt out of
grounding in their preferences (``ai_summary_grounding``).

.. attention::

   A user configurable server URL allows any user of the instance to make the
   SearXNG server send requests to a URL of their choice (`SSRF`_), and each
   summary is real LLM work.  This plugin is intended for private instances --
   on a public instance lock the related preferences (:ref:`settings
   preferences`):

   .. code:: yaml

      preferences:
        lock:
          - ai_summary_server
          - ai_summary_api_key
          - ai_summary_model
          - ai_summary_grounding

   Locking ``ai_summary_server`` and ``ai_summary_api_key`` matters most: an
   unlocked pair lets any user of the instance make SearXNG send an
   ``Authorization`` header of their choosing to a host of their choosing.

.. _SSRF: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery

.. autoclass:: searx.ai_summary.SettingsAISummary
   :members: