Add minimal python client example for the server, streaming callback

[chrismrutherford]

When integrating llama.cpp with web services I use the server for inference. I find the server is fast and efficient using this method as the client is more or less pass-through.

There doesn't seem to be a good set of python examples for the server, possibly because most people use the openai client library? I was using this, but found it difficult to pass llama.cpp specific parameters such as cache_prompt, also it required that it required the model parameter to be present, even though it is not used by the server.

Here is a minimal python client that supports streaming. There is a streaming callback function that receives the data, here you can do things like check for "." for building sentences and passing them to a voice server, minimizing latency.

[scottmudge]

Cool, was looking for something exactly like this.

[shigins]

Thank you for this, the streaming example helped me out.

[teleprint-me]

Looks good.