jasonwitty
19cc7a6f9f
Servers that require authentication (e.g. vLLM or llama.cpp started with --api-key, or an LLM server behind an authenticating reverse proxy) can now be configured with an ai_summary.api_key, sent as "Authorization: Bearer". The key is administrator configuration only: there is no preference for it, and it is only sent to the configured base_url. Users can point the ai_summary_server preference at a server of their own, and such a server must not be handed the instance API key -- otherwise every user of the instance could capture it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
57 lines
1.8 KiB
ReStructuredText
57 lines
1.8 KiB
ReStructuredText
.. _settings ai_summary:
|
|
|
|
===============
|
|
``ai_summary:``
|
|
===============
|
|
|
|
Default configuration of the :ref:`AI summary plugin <ai_summary plugin>`.
|
|
Users configure the LLM server URL (any server implementing the OpenAI chat
|
|
completions API: Ollama, vLLM, llama.cpp, LM Studio, Hugging Face TGI, ...)
|
|
and the model in the *AI Summary* tab of their preferences; the values below
|
|
only act as instance wide defaults.
|
|
|
|
.. code:: yaml
|
|
|
|
ai_summary:
|
|
base_url: "http://127.0.0.1:11434"
|
|
model: "llama3.2:3b"
|
|
|
|
An LLM server that requires authentication -- e.g. vLLM or llama.cpp started
|
|
with ``--api-key``, or a server behind an authenticating reverse proxy -- is
|
|
configured with an ``api_key``:
|
|
|
|
.. code:: yaml
|
|
|
|
ai_summary:
|
|
base_url: "http://127.0.0.1:8000"
|
|
api_key: "sk-..."
|
|
model: "llama3.2:3b"
|
|
|
|
The key is sent in an ``Authorization: Bearer`` header. There is no user
|
|
preference for it, and it is only sent to the ``base_url`` above: a user who
|
|
points the ``ai_summary_server`` preference at a server of their own gets no
|
|
``Authorization`` header. SearXNG has no indirection for secrets in
|
|
``settings.yml``, so the file holding the key should be readable by the
|
|
SearXNG process only.
|
|
|
|
.. attention::
|
|
|
|
A user configurable server URL allows any user of the instance to make the
|
|
SearXNG server send requests to a URL of their choice (`SSRF`_), and each
|
|
summary is real LLM work. This plugin is intended for private instances --
|
|
on a public instance lock the related preferences (:ref:`settings
|
|
preferences`):
|
|
|
|
.. code:: yaml
|
|
|
|
preferences:
|
|
lock:
|
|
- ai_summary_server
|
|
- ai_summary_model
|
|
- ai_summary_grounding
|
|
|
|
.. _SSRF: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery
|
|
|
|
.. autoclass:: searx.ai_summary.SettingsAISummary
|
|
:members:
|