searxng/docs/admin/settings/settings_ai_summary.rst

.. _settings ai_summary:

===============
``ai_summary:``
===============

.. sidebar:: Further reading ..

   - :ref:`ai_summary plugin`
   - :ref:`settings plugins`
   - :ref:`settings preferences`

Configuration of the :ref:`AI Summary plugin <ai_summary plugin>`, which shows a
short AI generated answer above the search results.

The text is produced by an **LLM server that you run** -- SearXNG does not ship
a model and does not talk to any AI provider on its own.  Anything that speaks
the `OpenAI chat completions API`_ works: `Ollama`_, vLLM, llama.cpp, LM Studio,
Hugging Face TGI and others.  The plugin is not activated by default.


.. _ai_summary quickstart:

Quickstart
==========

A local setup on the same machine as SearXNG, in four steps.

**1. Install Ollama**

.. code:: sh

   curl -fsSL https://ollama.com/install.sh | sh

**2. Download a model**

.. code:: sh

   ollama pull gemma3:4b

``gemma3:4b`` needs roughly 4 GB of memory and runs on CPU if you have no GPU.
On a small machine use ``gemma3:1b`` instead; any model in the `Ollama
library`_ works.

**3. Configure SearXNG**

Add this to your ``settings.yml``:

.. code:: yaml

   ai_summary:
     base_url: "http://127.0.0.1:11434"
     model: "gemma3:4b"

   plugins:
     searx.plugins.ai_summary.SXNGPlugin:
       active: true
     # keep the plugins you already had, see the warning below
     searx.plugins.calculator.SXNGPlugin: {active: true}
     searx.plugins.hash_plugin.SXNGPlugin: {active: true}
     searx.plugins.self_info.SXNGPlugin: {active: true}
     searx.plugins.unit_converter.SXNGPlugin: {active: true}
     searx.plugins.ahmia_filter.SXNGPlugin: {active: true}
     searx.plugins.hostnames.SXNGPlugin: {active: true}
     searx.plugins.time_zone.SXNGPlugin: {active: true}
     searx.plugins.tracker_url_remover.SXNGPlugin: {active: true}
     searx.plugins.infinite_scroll.SXNGPlugin: {active: false}
     searx.plugins.oa_doi_rewrite.SXNGPlugin: {active: false}
     searx.plugins.tor_check.SXNGPlugin: {active: false}

.. warning::

   A ``plugins:`` block **replaces** the default list, it is not merged into it
   (:ref:`settings plugins`).  If you list only the AI Summary plugin, every
   other plugin is switched off.  That is why the block above repeats the
   defaults -- drop the lines for plugins you do not want.

**4. Restart SearXNG** and search for something.

The summary appears above the results while it is still being written.  If
nothing appears, see :ref:`ai_summary troubleshooting`.


Options
=======

Only ``base_url`` is required; a model is needed too, but if you leave ``model``
empty the first entry of ``models`` is used.

.. code:: yaml

   ai_summary:
     base_url: "http://127.0.0.1:11434"
     model: "gemma3:4b"
     grounding: true

.. autoclass:: searx.ai_summary.SettingsAISummary
   :members:


Servers that need a password
============================

Some servers require authentication -- vLLM and llama.cpp when started with
``--api-key``, a gateway such as LiteLLM, or any LLM server placed behind an
authenticating reverse proxy.  Give SearXNG the key with ``api_key``:

.. code:: yaml

   ai_summary:
     base_url: "http://127.0.0.1:8000"
     api_key: "sk-..."
     model: "gemma3:4b"

The key is sent as an ``Authorization: Bearer`` header, and only to the server
in ``base_url``.  Users who point the ``ai_summary_server`` preference at a
server of their own never receive it; they set their own key in the
``ai_summary_api_key`` preference instead.

SearXNG has no separate secret store, so the key sits in ``settings.yml`` --
make sure that file is readable only by the user SearXNG runs as.


.. _ai_summary grounding:

Grounding
=========

Grounding decides *what the model is told*:

``grounding: true`` (the default)
   The query **and the top search results** (title, URL and snippet, at most
   ``max_context_items`` of them) are sent to the LLM server.  Answers reflect
   what the search actually found, so they are more accurate and more current.

``grounding: false``
   Only the query is sent, and the model answers from its own training data.
   Faster and cheaper, but the answer can be outdated or invented.

Users can switch this in their preferences (``ai_summary_grounding``).

How much this matters for privacy depends entirely on where the LLM server runs.
With a server on localhost or in your own network, nothing leaves that network.
With a hosted server, the query and the result snippets are sent to the
provider.  This is the main reason a local server is recommended.


Public instances
================

.. attention::

   The plugin is designed for private instances.  Read this before enabling it
   on a public one.

Two things change on a public instance:

**Every search costs real work.** A summary is a full LLM inference.  Traffic
that was cheap to serve becomes expensive, on hardware you pay for.

**Users can choose the server SearXNG talks to.** The ``ai_summary_server``
preference makes your instance send requests to an address of the user's
choosing, which is a `SSRF`_ vector: it can be pointed at services inside your
network that are not reachable from outside.

Lock the preferences so that only your configuration is used
(:ref:`settings preferences`):

.. code:: yaml

   preferences:
     lock:
       - ai_summary_server
       - ai_summary_api_key
       - ai_summary_model
       - ai_summary_grounding

Locking ``ai_summary_server`` closes the SSRF vector; locking
``ai_summary_api_key`` stops users making your instance send an
``Authorization`` header of their choosing to a host of their choosing.


.. _ai_summary troubleshooting:

Troubleshooting
===============

**No summary appears at all.**
  The plugin deliberately stays quiet in several cases: on page two and beyond,
  outside the *general* category, and when an engine already answered the query
  with an infobox or an instant answer.  Try a question-like query on the first
  page.  Check that the plugin is enabled both in ``settings.yml`` and in your
  own preferences.

**The model list in the preferences is empty.**
  SearXNG asks the server for its models (``GET /v1/models``) once at startup.
  An empty list means that request failed -- the server was not running yet,
  the URL is wrong, or it needs an ``api_key``.  The log records the reason;
  restart SearXNG after fixing it.

**The summary box shows an error.**
  SearXNG could not reach the LLM server, or the server rejected the request.
  Check ``base_url`` from the SearXNG machine, confirm the model name exists on
  that server (``ollama list``), and check the SearXNG log -- it records the
  status and the message the server replied with, which usually names the cause.

**The first search after a while fails, the next one works.**
  An idle LLM server unloads the model and has to load it again, and it sends
  nothing at all while doing so.  If that takes longer than ``read_timeout``
  (30 s by default) the request is abandoned.  SearXNG repeats the request once,
  which covers a normal load, but a large model on slow storage can need more:
  raise ``read_timeout``, or keep the model in memory -- with Ollama, set
  ``OLLAMA_KEEP_ALIVE`` (for example ``-1`` to never unload it).

**The summary starts, then stops mid-sentence.**
  The answer exceeded ``stream_timeout`` (120 s by default).  Large models on
  CPU are slow; either raise the limit or use a smaller model.

.. _Ollama: https://ollama.com/
.. _Ollama library: https://ollama.com/library
.. _OpenAI chat completions API: https://platform.openai.com/docs/api-reference/chat
.. _SSRF: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery