|
OpenAnima 0.2.0
Middleware for conversational NPCs that interact with the player, the world and each other
|
Inference backends and their settings. More...
Classes | |
| struct | Call |
| class | IProvider |
| class | RestProvider |
| struct | RestProviderConfig |
Typedefs | |
| using | DeltaSink = std::function< void(std::string_view text)> |
Inference backends and their settings.
One interface to implement, one configuration type to fill, and the REST implementation for the chat-completions contract.
| using OpenAnima::Provider::DeltaSink = typedef std::function<void(std::string_view text)> |
Receives the answer as a provider produces it.
An empty sink means the caller wants the whole answer at once, and a provider is then free to ask for it in a single response. That is not only an optimisation: some servers report the token usage only when the answer is not streamed.