From ce02bebf7b06be03efbb5d940fdaa644026bd07f Mon Sep 17 00:00:00 2001 From: jasonwitty Date: Mon, 10 Aug 2026 16:31:01 -0700 Subject: [PATCH] [doc] ai_summary: say what the summary is for in the introduction The introduction described the mechanism before the purpose. It now states what the plugin does, that it is meant for a local LLM server speaking the OpenAI chat completions API, and -- the part that was missing -- that the summary exists to condense the results the query actually returned rather than to recite training data. Co-Authored-By: Claude Opus 5 --- docs/dev/plugins/ai_summary.rst | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/docs/dev/plugins/ai_summary.rst b/docs/dev/plugins/ai_summary.rst index f965f1ff4..f8830bea5 100644 --- a/docs/dev/plugins/ai_summary.rst +++ b/docs/dev/plugins/ai_summary.rst @@ -10,15 +10,23 @@ AI Summary - :ref:`dev plugin` - :ref:`result types` -The AI Summary plugin shows a generated answer above the search results. The -text comes from an LLM server the administrator runs; see :ref:`its -configuration ` for how to set one up. +The AI Summary plugin shows a generated answer above the ordinary search +results, unless an engine has already answered the query directly -- with a +Wikipedia infobox, for instance, or an instant answer. It is meant to run +against a local LLM server and speaks the `OpenAI chat completions API`_, so it +works with `Ollama`_, Hugging Face TGI, LiteLLM, vLLM, llama.cpp and anything +else that implements that specification. See :ref:`its configuration ` for how to set one up. -The interesting part of this plugin is *when* things happen. Generating an -answer takes seconds, and a search engine that waits seconds before painting -anything is a broken search engine. So the plugin never blocks the result -page: it renders an empty box, and the browser fills that box afterwards from a -second, streaming request. +The purpose of the summary is not to tell you what the model memorised during +training, but to summarise the up to date results your query actually returned. +That is what the *grounding* setting does, and why it is enabled by default. + +Generating an answer takes seconds, and a search engine that waits seconds +before painting anything is a broken search engine. The summary is therefore +produced asynchronously: the plugin renders an empty box, the result page is +delivered immediately, and the browser fills that box from a second, streaming +request. The results below stay readable and scrollable the whole time. Request flow @@ -122,6 +130,8 @@ The streaming response uses ``direct_passthrough``, so the generator must yield ``bytes`` -- werkzeug asserts on ``str``, and the Flask test client does not catch it. +.. _Ollama: https://ollama.com/ +.. _OpenAI chat completions API: https://platform.openai.com/docs/api-reference/chat .. _SSE: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events .. _NDJSON: https://github.com/ndjson/ndjson-spec