Python: Add llmman local model sample using the Ollama connector - #14373
Python: Add llmman local model sample using the Ollama connector#14373Eric Curtin (ericcurtin) wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
The new sample includes a curl ... | sh install instruction, which is a risky pattern to recommend in repository samples and should be replaced with a safer pointer to upstream install docs.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds a new Python concept sample demonstrating how to run chat completion against a local llmman server by reusing the existing OllamaChatCompletion connector with a different host, and updates the concepts README to surface the new sample alongside other local-model examples.
Changes:
- Add
llmman_chat_completion.pysample usingOllamaChatCompletion(host="http://localhost:17434"). - Update the Local Models section header to mention the Ollama connector and llmman.
- Add the new llmman sample link under Local Models in the concepts README.
File summaries
| File | Description |
|---|---|
| python/samples/concepts/README.md | Updates Local Models section header and adds a link to the new llmman chat sample. |
| python/samples/concepts/local_models/llmman_chat_completion.py | New concept sample showing how to use the Ollama connector against llmman’s Ollama-compatible API host. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| # Install and start the server, then pull a model: | ||
| # curl -fsSL https://raw.githubusercontent.com/llmmanorg/llmman/main/install.sh | sh | ||
| # llmman serve | ||
| # llmman pull gemma4 |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
MAF Automated Review — Iteration 1
Result: Findings reported
Scope: full PR (1 commit(s)): 7b6d991d5c78
Model: claude-opus-4.8
Overview
This is a documentation/sample-only change: a new local_models concept sample that points the existing OllamaChatCompletion connector at llmman's http://localhost:17434 host, plus a README index entry. No library or public-API code is touched, so there is no compatibility, lifecycle, serialization, or security-boundary risk; the connector constructor args, settings.options usage, and service_url() behavior are all correct and match the vetted sibling samples. The one substantive gap is that the sample documents the llmman server setup but omits the semantic-kernel[ollama] extra it requires, so it fails at import in a standard install.
Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
1 verified finding remained after source verification (1 medium) across 1 file. Details are attached to the affected lines below.
Affected areas: python/samples/concepts/local_models/llmman_chat_completion.py
| # llmman is a local model runner that serves the Ollama API (alongside | ||
| # OpenAI- and Anthropic-compatible ones) on port 17434, so the existing | ||
| # Ollama connector works unchanged; only the host differs. | ||
| # Install and start the server, then pull a model: |
There was a problem hiding this comment.
The setup instructions cover the llmman server but omit the Python dependency this sample needs. Unlike the existing ollama_chat_completion.py (which uses the OpenAI connector), this sample imports OllamaChatCompletion, which imports from ollama import AsyncClient at module load. The ollama package is an optional extra (pyproject.toml:115, ollama = ["ollama ~= 0.4"]), so a user running this sample after a standard pip install semantic-kernel hits ModuleNotFoundError: No module named 'ollama' at line 6 before anything executes. Please document the extra in the setup block, matching the convention in foundry_local_chatbot.py (which documents pip install foundry-local-sdk), e.g.:
# pip install "semantic-kernel[ollama]"
llmman (https://github.com/llmmanorg/llmman) serves the Ollama API on port 17434, so OllamaChatCompletion works against it unchanged; only the host differs. Add a concept sample mirroring the existing local model samples and list it in the concepts README. No library code or public API is changed.
7b6d991 to
503893e
Compare
Motivation and Context
Adds a sample for llmman, a local model runner that serves the Ollama API on port 17434. No library code or public API changes, so no issue was filed (trivial change per CONTRIBUTING).
Description
python/samples/concepts/local_models/llmman_chat_completion.py: new sample usingOllamaChatCompletionwithhost="http://localhost:17434", mirroring the Ollama / LM Studio samples. Reusing the Ollama connector keeps function calling, images andoptionsworking unchanged.python/samples/concepts/README.md: list the sample under "Local Models" and mention the Ollama connector/llmman in the section header.Testing:
ruff check,ruff format --checkandpy_compilepass on the new file;kernel.get_service("local-gpt")resolves to anOllamaChatCompletionwithservice_url()http://localhost:17434. Not run against a live llmman server.Contribution Checklist