Running AI Locally in 2026: What's Changed with Ollama?

AI-generated illustration inspired by the artwork from my original local LLM guide.

Updated: November 2026

In January 2025, I wrote a beginner's guide to running a Large Language Model locally using Ollama. At the time, it was surprisingly easy to download a model such as Meta's Llama 3.2 and start chatting with it directly from a terminal.

Almost two years later, the local AI landscape has changed dramatically.

The models have become considerably more capable, Ollama has gained a proper desktop application, hardware support and performance have improved, and local AI is no longer limited to answering questions in a terminal window.

You can now use local models to understand images, analyse documents, generate code, power AI agents and even generate images and speech.

So, if I were starting from scratch today, how would I go about it?

What is Ollama?

Ollama is a free application that makes it easy to download, manage and run AI models on your own computer.

Instead of sending every question to a cloud service such as ChatGPT or Claude, you can download an AI model and run it directly on your own hardware.

Once the model has been downloaded, you can use it without an internet connection.

There are several advantages to this:

• Privacy: Your conversations and documents can remain on your own computer.
• No subscription required: Running downloaded models locally doesn't require an ongoing subscription or per-message charges.
• Offline operation: Once installed, local models continue working without internet access.
• Experimentation: You can try different models and compare their capabilities.
• Development: You can integrate AI into your own applications through a local API.

Ollama runs on macOS, Windows and Linux, and supports models from organisations including Google, Meta, Alibaba, Microsoft and OpenAI.

It's worth understanding that not every model is fully open-source. Some are described as open-weight, meaning the model weights are available to download but the licence and access to training information can vary.

What's improved since 2025?

1. You no longer need to use the terminal

One of the biggest improvements is the introduction of Ollama's desktop chat application for macOS and Windows.

Previously, getting started involved opening a terminal, downloading a model using a command and then typing prompts into a command-line interface.

That still works, and it's something I use regularly, but it is no longer necessary for beginners.

The desktop application lets you choose and download models, have conversations and attach files through a graphical interface.

You can even drag documents such as PDFs into a conversation and ask questions about their contents.

This makes Ollama much more accessible to anyone who simply wants to try running AI locally.

2. The AI models are significantly better

When I wrote the original article, I used Meta's Llama 3.2, a relatively small model that was ideal for demonstrating the technology.

It was impressive at the time, but today's models are considerably more sophisticated.

Some of the models I'd look at now include:

The model you choose will depend on your hardware and what you're trying to achieve.

Smaller models are generally faster and require less memory. Larger models can offer improved reasoning and capabilities but need considerably more resources.

3. Better performance and hardware support

Ollama has also improved considerably under the bonnet.

On Apple Silicon Macs, Ollama now supports an engine based on Apple's MLX machine learning framework, which can take better advantage of the Mac's GPU and unified memory.

There have also been improvements to NVIDIA GPU performance, support for a wider range of graphics hardware and memory management.

That last point is particularly important.

Previously, trying to load a model that was too large for the available memory could cause problems. Ollama now has improved model scheduling and memory allocation to reduce these issues.

These improvements don't mean that a huge AI model will suddenly run quickly on an ordinary laptop, but they do make better use of the hardware you already have.

4. AI can now understand more than text

The original article concentrated entirely on text-based language models.

Ollama now supports multimodal models that can process different types of information.

For example, with a compatible model you can provide a photograph and ask the AI to describe it, identify objects or explain what is happening.

You can also analyse screenshots, charts and documents.

Ollama has introduced experimental local image generation on macOS, and more recently added text-to-speech models. Availability still varies by operating system and model.

It's a significant step beyond the simple text conversations I demonstrated in 2025.

5. AI coding agents

This is one of the developments I find most interesting.

Ollama can now work with AI coding tools such as OpenAI Codex, Claude Code and OpenCode.

Instead of simply asking an AI to generate a piece of code, these tools can work through larger software development tasks, inspect project files, propose changes and run commands, subject to the permissions you give them.

Ollama provides a way to use compatible local models as the AI engine behind these tools.

There's even an ollama launch command to help configure supported coding applications.

This opens up the possibility of experimenting with AI-assisted software development without sending your entire project to a cloud-hosted model.

However, it's important to remember that local models are not necessarily as capable as the largest cloud models, particularly when working on complex projects.

Installing Ollama in 2026

If I were starting again today, I'd keep the initial setup simple.

Step 1: Download Ollama

Visit the official website:

https://ollama.com

Download the version for your operating system and follow the installation instructions.

For Windows and macOS, the graphical application provides the easiest starting point.

Linux users can follow the installation instructions on the Ollama website.

Step 2: Choose your first model

Rather than immediately downloading the largest model available, I'd start with something relatively small.

My recommendation would be Qwen 3.5 9B.

It's a modern, capable model and a good introduction to what local AI can now achieve.

Using the Ollama desktop application, select the model, download it and start a conversation.

Try asking it to explain a technical concept, summarise some text or help write a short piece of code.

You can also experiment with attaching an image or document.

Step 3: Try the terminal

Although the graphical application makes things much easier, it's worth understanding how to use Ollama from the command line.

Open Terminal on macOS or PowerShell on Windows and enter:

ollama run qwen3.5:9b

Ollama will download the model automatically if it isn't already installed, then open an interactive conversation.

Enter a prompt such as:

Explain how a home computer network works,
including the roles of routers, switches
and IP addresses.

You should see the model generate a response.

To leave the conversation, enter:

/bye

Some other useful Ollama commands are:

# Show installed models
ollama list

# Download a model
ollama pull gemma4:12b

# Run a model
ollama run gemma4:12b

# Show models currently loaded
ollama ps

# Remove a model
ollama rm gemma4:12b

You can experiment with different models by downloading them and comparing their answers.

What hardware do you need?

Running AI locally requires more computing resources than most everyday applications.

The most important consideration is memory, particularly GPU video memory (VRAM) on a Windows or Linux PC, or unified memory on an Apple Silicon Mac.

As a rough guide:

These are approximate starting points, not guaranteed requirements. The model's quantisation, context length, operating system and available GPU memory all affect what will run successfully.

A model that fits in memory isn't necessarily going to run quickly. If it cannot fit entirely into GPU memory, performance may be considerably slower.

Apple Silicon Macs are particularly interesting because the CPU and GPU share a pool of unified memory. This can make it possible to run larger models than you might expect from a conventional PC with a modest graphics card.

However, you certainly don't need an expensive workstation to get started.

Trying a more powerful model

Once you're comfortable with a smaller model, I'd suggest trying OpenAI's GPT-OSS 20B.

It's an open-weight reasoning model designed for tasks including programming, problem-solving and more complex questions.

To download and run it:

ollama run gpt-oss:20b

The model download is approximately 14GB, and it generally requires at least 16GB of usable memory, with more memory potentially needed for longer conversations.

Compare its answers with your smaller model.

Try giving both models the same question that requires several stages of reasoning, or ask them to explain some unfamiliar code.

You'll soon discover that different models have different strengths.

Running AI locally versus in the cloud

Another interesting development is that Ollama now supports cloud-hosted models.

This means you can continue using the Ollama application and tools while accessing models that are far too large to run on your own computer.

For example:

ollama run gpt-oss:120b-cloud

This runs a hosted model rather than downloading the full model weights onto your machine.

It's a useful option if you need more advanced capabilities without investing in expensive hardware.

However, there is an important difference.

When you use a cloud model, your prompts are sent to an external service.

Ollama states that its hosted models operate under a zero-data-retention policy, but this is still not the same as processing everything locally.

Cloud models also require an internet connection and are subject to account requirements and usage limits or charges.

For maximum local control and offline operation, stick with downloaded models.

Taking things further: AI-powered applications

For developers, one of Ollama's most useful features is its local API.

By default, Ollama exposes an API on your own computer at:

http://localhost:11434

This allows other applications to send prompts to your local models and receive generated responses.

For example, you can make a request using curl:

curl http://localhost:11434/api/generate \
  -d '{
    "model": "qwen3.5:9b",
    "prompt": "Explain what an API is in simple terms.",
    "stream": false
  }'

The response is returned as JSON.

From here, you can start integrating local AI into Python scripts, websites, developer tools or home automation systems.

You can also experiment with software such as Open WebUI, which provides a more extensive browser-based interface for working with Ollama and other AI services.

For coding, I'd also explore Ollama's integration with AI agents:

ollama launch codex

This helps you configure Codex to work with an appropriate model. For more demanding coding tasks, you'll generally want a capable model and a much larger context window than you'd need for ordinary chatting.

One final point: Ollama's local API is designed for trusted local use. I wouldn't expose it directly to the public internet without appropriate authentication and network security.

Is running AI locally worth it in 2026?

Absolutely, especially if you're interested in technology, software development or simply understanding how AI works.

There are still limitations.

Cloud-hosted frontier models will often outperform smaller local models, particularly on complex reasoning, research and extended coding tasks.

And while the models themselves can be free to download and run, your hardware still consumes electricity and costs money.

But I think the biggest attraction of local AI is having control over the technology.

You decide which models to use, where your data goes, how your applications interact with AI and whether you want to depend on an external provider.

Compared with my first experiment in January 2025, getting started is easier, the available models are more capable, and the potential applications are considerably more interesting.

If I were beginning today, I'd install Ollama, start with Qwen 3.5, experiment with Gemma 4 and GPT-OSS, and then explore what I could build using the local API.

The most exciting part of local AI isn't simply chatting with a model. It's being able to incorporate AI into your own projects, running on hardware that you control.

---

For more information, visit the Ollama website, browse its model library, or read the official documentation.