[doc] ai_summary: say what the summary is for in the introduction

The introduction described the mechanism before the purpose.  It now
states what the plugin does, that it is meant for a local LLM server
speaking the OpenAI chat completions API, and -- the part that was
missing -- that the summary exists to condense the results the query
actually returned rather than to recite training data.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
jasonwitty
2026-08-10 16:31:01 -07:00
parent 6b50b23467
commit ce02bebf7b
+18 -8
View File
@@ -10,15 +10,23 @@ AI Summary
- :ref:`dev plugin`
- :ref:`result types`
The AI Summary plugin shows a generated answer above the search results. The
text comes from an LLM server the administrator runs; see :ref:`its
configuration <settings ai_summary>` for how to set one up.
The AI Summary plugin shows a generated answer above the ordinary search
results, unless an engine has already answered the query directly -- with a
Wikipedia infobox, for instance, or an instant answer. It is meant to run
against a local LLM server and speaks the `OpenAI chat completions API`_, so it
works with `Ollama`_, Hugging Face TGI, LiteLLM, vLLM, llama.cpp and anything
else that implements that specification. See :ref:`its configuration <settings
ai_summary>` for how to set one up.
The interesting part of this plugin is *when* things happen. Generating an
answer takes seconds, and a search engine that waits seconds before painting
anything is a broken search engine. So the plugin never blocks the result
page: it renders an empty box, and the browser fills that box afterwards from a
second, streaming request.
The purpose of the summary is not to tell you what the model memorised during
training, but to summarise the up to date results your query actually returned.
That is what the *grounding* setting does, and why it is enabled by default.
Generating an answer takes seconds, and a search engine that waits seconds
before painting anything is a broken search engine. The summary is therefore
produced asynchronously: the plugin renders an empty box, the result page is
delivered immediately, and the browser fills that box from a second, streaming
request. The results below stay readable and scrollable the whole time.
Request flow
@@ -122,6 +130,8 @@ The streaming response uses ``direct_passthrough``, so the generator must yield
``bytes`` -- werkzeug asserts on ``str``, and the Flask test client does not
catch it.
.. _Ollama: https://ollama.com/
.. _OpenAI chat completions API: https://platform.openai.com/docs/api-reference/chat
.. _SSE: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events
.. _NDJSON: https://github.com/ndjson/ndjson-spec