jasonwitty
4abb7dba67
Talk to the LLM server via GET /v1/models and POST /v1/chat/completions (SSE) instead of Ollama's native API. Any OpenAI compatible server now works (Ollama, vLLM, llama.cpp, LM Studio, Hugging Face TGI, ...); Ollama serves this API natively, existing setups keep working unchanged. The Ollama specific keep_alive option is dropped, the ai_summary.grounding setting is added as instance wide default of the grounding preference. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
39 lines
1.1 KiB
ReStructuredText
39 lines
1.1 KiB
ReStructuredText
.. _settings ai_summary:
|
|
|
|
===============
|
|
``ai_summary:``
|
|
===============
|
|
|
|
Default configuration of the :ref:`AI summary plugin <ai_summary plugin>`.
|
|
Users configure the LLM server URL (any server implementing the OpenAI chat
|
|
completions API: Ollama, vLLM, llama.cpp, LM Studio, Hugging Face TGI, ...)
|
|
and the model in the *AI Summary* tab of their preferences; the values below
|
|
only act as instance wide defaults.
|
|
|
|
.. code:: yaml
|
|
|
|
ai_summary:
|
|
base_url: "http://127.0.0.1:11434"
|
|
model: "llama3.2:3b"
|
|
|
|
.. attention::
|
|
|
|
A user configurable server URL allows any user of the instance to make the
|
|
SearXNG server send requests to a URL of their choice (`SSRF`_), and each
|
|
summary is real LLM work. This plugin is intended for private instances --
|
|
on a public instance lock the related preferences (:ref:`settings
|
|
preferences`):
|
|
|
|
.. code:: yaml
|
|
|
|
preferences:
|
|
lock:
|
|
- ai_summary_server
|
|
- ai_summary_model
|
|
- ai_summary_grounding
|
|
|
|
.. _SSRF: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery
|
|
|
|
.. autoclass:: searx.ai_summary.SettingsAISummary
|
|
:members:
|