[mod] ai_summary plugin: switch to the OpenAI chat completions API

Talk to the LLM server via GET /v1/models and POST /v1/chat/completions
(SSE) instead of Ollama's native API.  Any OpenAI compatible server now
works (Ollama, vLLM, llama.cpp, LM Studio, Hugging Face TGI, ...);
Ollama serves this API natively, existing setups keep working unchanged.

The Ollama specific keep_alive option is dropped, the ai_summary.grounding
setting is added as instance wide default of the grounding preference.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
jasonwitty
2026-07-28 13:01:24 -07:00
co-authored by Claude Fable 5
parent 112541db28
commit 4abb7dba67
6 changed files with 70 additions and 57 deletions
+11 -13
View File
@@ -7,7 +7,7 @@
# - https://github.com/searxng/searxng/issues/5284
from __future__ import annotations
__all__ = ["SettingsAISummary", "MODELS", "model_choices", "build_ollama_messages"]
__all__ = ["SettingsAISummary", "MODELS", "model_choices", "build_chat_messages"]
import msgspec
@@ -43,9 +43,11 @@ class SettingsAISummary(msgspec.Struct, kw_only=True, forbid_unknown_fields=True
"""
base_url: str = ""
"""Default base URL of the Ollama server (e.g. ``http://127.0.0.1:11434``).
Users can set their own server URL in the preferences
(``ai_summary_server``) unless that preference is locked."""
"""Default base URL of the LLM server (e.g. ``http://127.0.0.1:11434``
for Ollama). Any server that implements the OpenAI chat completions API
works (Ollama, vLLM, llama.cpp, LM Studio, Hugging Face TGI, ...). Users
can set their own server URL in the preferences (``ai_summary_server``)
unless that preference is locked."""
model: str = ""
"""Name of the default model (e.g. ``llama3.2:3b``). If empty, the first
@@ -56,7 +58,7 @@ class SettingsAISummary(msgspec.Struct, kw_only=True, forbid_unknown_fields=True
models: list[str] = []
"""List of model names suggested to the user in the preferences. If empty
and :py:obj:`SettingsAISummary.base_url` is set, the list is requested
once at application setup from the Ollama server (``GET /api/tags``)."""
once at application setup from the LLM server (``GET /v1/models``)."""
grounding: bool = False
"""Default of the ``ai_summary_grounding`` user preference: ground the
@@ -64,7 +66,7 @@ class SettingsAISummary(msgspec.Struct, kw_only=True, forbid_unknown_fields=True
preferences unless that preference is locked."""
connect_timeout: float = 5.0
"""Timeout (seconds) to establish a TCP connection to the Ollama server."""
"""Timeout (seconds) to establish a TCP connection to the LLM server."""
read_timeout: float = 30.0
"""Maximum gap (seconds) between two chunks of the token stream."""
@@ -72,10 +74,6 @@ class SettingsAISummary(msgspec.Struct, kw_only=True, forbid_unknown_fields=True
stream_timeout: float = 120.0
"""Wall clock limit (seconds) for one completion."""
keep_alive: str = "5m"
"""How long the model stays loaded in memory after the request (passed
through to Ollama's ``keep_alive`` option)."""
max_context_items: int = 5
"""Maximum number of search results accepted as grounding context."""
@@ -99,13 +97,13 @@ def model_choices() -> list[str]:
return list(MODELS)
def build_ollama_messages(
def build_chat_messages(
cfg: SettingsAISummary,
messages: list[dict[str, str]],
context: list[dict[str, str]] | None = None,
) -> list[dict[str, str]]:
"""Build the message list for Ollama's ``/api/chat`` from the (already
validated) request ``messages``, prepending a system prompt. When
"""Build the message list for the chat completions request from the
(already validated) request ``messages``, prepending a system prompt. When
``context`` items are given, the grounded system prompt is used and the
context items are serialized into its ``{context}`` placeholder."""