Ollama

Ollama is the local server twinny supports best. It runs as a background service, pulls models with one command, and exposes native endpoints for completion (/api/generate) and embeddings (/api/embed) as well as an OpenAI-compatible chat API (/v1). It is also what Devices shares between your machines.

Install and pull

Install from ollama.com/download (macOS, Windows, Linux). Then:

ollama pull qwen2.5-coder:7b-instruct   # chat, edits, review
ollama pull qwen2.5-coder:1.5b-base     # completion
ollama pull nomic-embed-text            # workspace index (optional)

ollama list shows what you have; ollama ps shows what is loaded and whether it is on the GPU.

Let twinny set it up

With Ollama running, twinny’s first-run discovery finds it on port 11434 and creates providers for each job it has a model for. Later, Add provider → Ollama does the same for one job, listing the models from the server so you can pick.

Defaults twinny uses:

Job	Path	Model name
Chat	/v1 (twinny appends /chat/completions)	e.g. qwen2.5-coder:7b-instruct
Autocomplete	/api/generate	e.g. qwen2.5-coder:1.5b-base, FIM template Automatic
Embeddings	/api/embed	e.g. nomic-embed-text

Model names are Ollama tags, exactly as ollama list prints them.

Keeping models loaded

Ollama unloads a model after a period of inactivity, and the next request pays the load time. twinny.keepAlive (default 5m) is sent with each completion request to control how long the completion model stays loaded: 1h, 24h, or -1 to keep it resident. Chat requests do not carry it, so the chat model follows Ollama’s own default.

When VS Code starts or regains focus, twinny asks Ollama to load the completion model with a one-token request, so the first suggestion does not wait for it. Nothing is sent if the model was used in the last four minutes. Turn this off with twinny.warmUpModel.

If chat and completion use different models and your GPU cannot hold both, they swap each other out; either use the same model for both, use a smaller completion model, or accept the swap.

Context size

Ollama’s default context window is small. For chat and review with long inputs, make a variant with a bigger context:

# Modelfile
FROM qwen2.5-coder:7b-instruct
PARAMETER num_ctx 16384

ollama create qwen2.5-coder:7b-instruct-16k -f Modelfile

and use that tag in the chat provider. Larger context uses more memory. Completion does not need a large window; twinny keeps FIM prompts short.

Reaching Ollama from elsewhere

Ollama listens on 127.0.0.1 only. To let another machine or a container call it, start it with:

OLLAMA_HOST=0.0.0.0 ollama serve

(or set OLLAMA_HOST in the service’s environment), then point the provider at that machine’s LAN address. Anyone on the network can then use it, so do this only on networks you trust.

For your own machines, Devices is the better option: it needs no change to Ollama, works across networks, and only paired devices can connect.

Ollama in Docker or WSL

• Ollama in a container, VS Code on the host: publish the port (-p 11434:11434) and use localhost.
• VS Code in a container (Dev Containers), Ollama on the host: use host.docker.internal as the hostname, or the host’s LAN address.
• WSL: Ollama installed in Windows is reachable from WSL at the Windows host address; Ollama installed inside WSL is localhost from a WSL-based VS Code.

twinny’s Ollama settings

These are used beyond a single provider, for listing models and for the Ollama that this machine shares with paired devices:

Setting	Default
twinny.ollamaHostname	0.0.0.0	Treated as localhost when calling
twinny.ollamaApiPort	11434
twinny.ollamaUseTls	false
twinny.keepAlive	5m	Sent with each completion request

Tips

• Speed: ollama ps should show 100% GPU. If it shows CPU, the model is too big for the VRAM; pick a smaller one.
• Quantisation: the default tags are 4-bit. :7b-instruct-q8_0 is higher quality and twice the memory; :7b-instruct-q3_K_M is smaller and slightly worse.
• Several models at once: recent Ollama versions can keep more than one loaded when memory allows (OLLAMA_MAX_LOADED_MODELS).
• Logs: journalctl -u ollama on Linux, or the Ollama app’s log on macOS and Windows, show what the server did with a request.