> For the complete documentation index, see [llms.txt](/llms.txt).
> Markdown versions of each page are available by appending .md to any URL.

# Set up Ollama

Install Ollama, run LLMs locally, compare model performance, and integrate local models into your apps using Warp.

Run AI models locally — with no API costs, no data leaving your machine, and your choice of model. This guide walks through installing Ollama, running your first model, integrating it into an existing app, and customizing model behavior using Warp.

![Setting up Ollama in Warp video](https://i.ytimg.com/vi/Aq8vDxUg4VE/sddefault.jpg)

## Prerequisites

**Hardware requirements**

You'll need enough memory to load the model weights. The rule of thumb: roughly 1GB of memory per billion parameters.

| Model size | Example models | Minimum RAM/VRAM |
| --- | --- | --- |
| 3B | Llama 3.2 3B, Phi-3 Mini | 4GB |
| 7–8B | Mistral 7B, Llama 3.1 8B | 8GB |
| 13B | Llama 2 13B | 16GB |
| 30–34B | Mixtral, Codestral | 24GB |
| 70B | Llama 3.1 70B | 48GB |

On Apple Silicon Macs, unified memory is shared between CPU and GPU — an M2 Mac with 16GB unified memory can run 7–8B models comfortably.

**Software requirements**

-   macOS 12+, Windows 10+, or Ubuntu 20.04+
-   No account required — Ollama is free and open source

## 1\. Install Ollama

**macOS**

Download the macOS app from [ollama.com/download](https://ollama.com/download). Once installed, Ollama runs as a background service and appears in your menu bar.

Alternatively, install via Homebrew:

```
brew install ollama
```

If you installed via Homebrew, start the server manually before using it:

```
ollama serve
```

**Linux**

```
curl -fsSL https://ollama.com/install.sh | sh
```

The installer sets up Ollama as a `systemd` service that starts automatically on boot.

**Windows**

Download the Windows installer from [ollama.com/download](https://ollama.com/download). Ollama runs as a background service after installation.

## 2\. Run your first model

Pull a model to download it locally. This example uses Llama 3.2 (3B), a lightweight model that runs on most hardware:

```
ollama pull llama3.2
```

Then start an interactive chat session:

```
ollama run llama3.2
```

Type your prompt and press Enter. Type `/bye` to exit the session.

To try a larger model with stronger reasoning:

```
ollama run llama3.1:8b
```

Use `ollama list` to see all models you've downloaded, and `ollama ps` to see which models are currently loaded in memory.

## 3\. Understand model types

When browsing [ollama.com/library](https://ollama.com/library), you'll see labels on models that indicate their capabilities:

| Term | Meaning |
| --- | --- |
| **Thinking** | The model "thinks" before answering; better for complex reasoning. |
| **Tools** | Models can call external functions or utilities (e.g., web search). |
| **Vision** | Can process and respond to images. |
| **Embedding** | Converts text to numeric vectors for search or RAG pipelines. |
| **Quantization** | Reduces memory use by lowering weight precision (e.g., 4-bit, 8-bit). |

Quantized models (e.g., `q4_0`, `q8_0` variants) use significantly less memory than full-precision originals. Start with a quantized version if you're memory-constrained.

## 4\. Compare model performance

To benchmark models side by side, Warp makes it easy to run prompts in parallel tabs and compare results. Use `ollama run` with a specific prompt to get inline timing stats:

```
ollama run llama3.2 "Summarize the Rust ownership model in one paragraph"
```

Ollama prints timing information at the end of each non-interactive request, including tokens generated and throughput (`eval rate: X tokens/s`). Run the same prompt on two different models to compare speed and output quality before committing to one.

To monitor memory usage while a model runs:

```
# See which models are loaded and how much memory they're usingollama ps
```

## 5\. Integrate Ollama into your app

Ollama exposes an OpenAI-compatible REST API at `http://localhost:11434/v1`. You can drop it in wherever you're currently using the OpenAI API — just update three values:

1.  **Base URL** → `http://localhost:11434/v1`
2.  **API key** → any non-empty string (e.g., `"ollama"`)
3.  **Model name** → the name of your local model (e.g., `"llama3.2"`)

**Python (OpenAI SDK)**

```
from openai import OpenAI
client = OpenAI(    base_url="http://localhost:11434/v1",    api_key="ollama",  # required by the SDK but not validated by Ollama)
response = client.chat.completions.create(    model="llama3.2",    messages=[        {"role": "user", "content": "Explain async/await in Python"}    ])
print(response.choices[0].message.content)
```

**Node.js (OpenAI SDK)**

```
import OpenAI from "openai";
const client = new OpenAI({  baseURL: "http://localhost:11434/v1",  apiKey: "ollama", // required by the SDK but not validated by Ollama});
const response = await client.chat.completions.create({  model: "llama3.2",  messages: [{ role: "user", content: "Explain async/await in Python" }],});
console.log(response.choices[0].message.content);
```

Open your app's code in Warp and use the agent to locate the OpenAI client initialization, swap in the new values, and test the connection — all from the same terminal session.

## 6\. Customize model behavior

You can create a custom model variant by writing a `Modelfile` — a configuration file that sets a system prompt, adjusts generation parameters, or builds on any existing Ollama model.

Create a file called `Modelfile`:

```
FROM llama3.2
SYSTEM """You are a focused code review assistant.Always identify security issues first, then logic errors, then style.Keep feedback concise and actionable."""
PARAMETER temperature 0.3PARAMETER num_ctx 8192
```

Then build and run your custom model:

```
ollama create code-reviewer -f ./Modelfileollama run code-reviewer
```

Your custom model is saved locally and available any time you run `ollama run code-reviewer`. Ask Warp's agent to generate a Modelfile for a specific use case — describe what you want the model to do, and let the agent write the configuration.

## Productivity tips

-   **Preload a model** — Run `ollama run <model> ""` to load a model into memory before your first real prompt, so the initial response isn't delayed by model loading time.
-   **Unload to free memory** — Use `ollama stop <model>` to unload a model from memory before loading a larger one.
-   **Script with the REST API** — For quick testing without the SDK: `curl http://localhost:11434/api/generate -d '{"model": "llama3.2", "prompt": "Hello", "stream": false}'`
-   **Keep a model inventory** — `ollama list` shows all locally cached models with their sizes and last-used dates. Use `ollama rm <model>` to delete models you no longer need.
-   **Compare models in parallel** — Open side-by-side Warp tabs and run the same prompt against two different models to compare quality and speed before deciding which to use in your app.

## Next steps

You have Ollama installed, a model running locally, and a path to integrate it into any app using the OpenAI-compatible API.

Explore related guides and features:

-   [Set up Claude Code](/guides/external-tools/how-to-set-up-claude-code/) or [Set up Codex CLI](/guides/external-tools/how-to-set-up-codex-cli/) — use cloud-based agents alongside your local Ollama setup
-   [Run multiple agents at once](/guides/agent-workflows/how-to-run-multiple-ai-coding-agents/) — run Ollama and a cloud agent in parallel to compare outputs on the same task
-   [Ollama model library](https://ollama.com/library) — browse all available models with size and capability details
-   [Ollama documentation](https://github.com/ollama/ollama/blob/main/README.md) — advanced configuration, GPU setup, and environment variables