[doc] ai_summary: say what the summary is for in the introduction
The introduction described the mechanism before the purpose. It now states what the plugin does, that it is meant for a local LLM server speaking the OpenAI chat completions API, and -- the part that was missing -- that the summary exists to condense the results the query actually returned rather than to recite training data. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -10,15 +10,23 @@ AI Summary
|
|||||||
- :ref:`dev plugin`
|
- :ref:`dev plugin`
|
||||||
- :ref:`result types`
|
- :ref:`result types`
|
||||||
|
|
||||||
The AI Summary plugin shows a generated answer above the search results. The
|
The AI Summary plugin shows a generated answer above the ordinary search
|
||||||
text comes from an LLM server the administrator runs; see :ref:`its
|
results, unless an engine has already answered the query directly -- with a
|
||||||
configuration <settings ai_summary>` for how to set one up.
|
Wikipedia infobox, for instance, or an instant answer. It is meant to run
|
||||||
|
against a local LLM server and speaks the `OpenAI chat completions API`_, so it
|
||||||
|
works with `Ollama`_, Hugging Face TGI, LiteLLM, vLLM, llama.cpp and anything
|
||||||
|
else that implements that specification. See :ref:`its configuration <settings
|
||||||
|
ai_summary>` for how to set one up.
|
||||||
|
|
||||||
The interesting part of this plugin is *when* things happen. Generating an
|
The purpose of the summary is not to tell you what the model memorised during
|
||||||
answer takes seconds, and a search engine that waits seconds before painting
|
training, but to summarise the up to date results your query actually returned.
|
||||||
anything is a broken search engine. So the plugin never blocks the result
|
That is what the *grounding* setting does, and why it is enabled by default.
|
||||||
page: it renders an empty box, and the browser fills that box afterwards from a
|
|
||||||
second, streaming request.
|
Generating an answer takes seconds, and a search engine that waits seconds
|
||||||
|
before painting anything is a broken search engine. The summary is therefore
|
||||||
|
produced asynchronously: the plugin renders an empty box, the result page is
|
||||||
|
delivered immediately, and the browser fills that box from a second, streaming
|
||||||
|
request. The results below stay readable and scrollable the whole time.
|
||||||
|
|
||||||
|
|
||||||
Request flow
|
Request flow
|
||||||
@@ -122,6 +130,8 @@ The streaming response uses ``direct_passthrough``, so the generator must yield
|
|||||||
``bytes`` -- werkzeug asserts on ``str``, and the Flask test client does not
|
``bytes`` -- werkzeug asserts on ``str``, and the Flask test client does not
|
||||||
catch it.
|
catch it.
|
||||||
|
|
||||||
|
.. _Ollama: https://ollama.com/
|
||||||
|
.. _OpenAI chat completions API: https://platform.openai.com/docs/api-reference/chat
|
||||||
.. _SSE: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events
|
.. _SSE: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events
|
||||||
.. _NDJSON: https://github.com/ndjson/ndjson-spec
|
.. _NDJSON: https://github.com/ndjson/ndjson-spec
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user