ollama

Ollama is an open-source application for running large language models locally on Linux, macOS and Windows. It combines a command-line interface with a model server, letting you chat with AI models on your computer or connect them to other applications. Developers can also deploy Ollama using its official Docker image and build integrations with the Python and JavaScript libraries.

Getting started with a local model takes a single command: ollama run <model>. Ollama downloads the selected model and opens an interactive conversation in your terminal. Its model catalog includes Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other families. Check the individual model entry before downloading: available sizes, capabilities and local or cloud execution vary between models.

For application development, Ollama provides a REST API for running and managing models. You can use it as the model backend for chat interfaces, automation scripts and compatible coding tools. Its supported OpenAI API endpoints allow applications built around OpenAI clients to connect to an Ollama server, some OpenAI features may not be supported.

Local performance depends on the model, available RAM, GPU memory and context length. Ollama can load models onto supported GPUs, use system memory for CPU inference, or distribute a model across both. Run ollama ps and inspect the PROCESSOR column to check whether a loaded model is using your GPU. Larger models and longer conversations generally require more memory.

When you run a model locally, Ollama does not send your prompts and responses to its cloud service. Optional cloud models process requests remotely and provide access to models beyond your computer’s practical hardware limits. You can disable Ollama’s cloud features for a setup dedicated to local inference.

Main features

• Run local language models on Linux, macOS and Windows, with a terminal interface for interactive conversations.
• Download and run models through a consistent command-line workflow, then connect them to applications through the API.
• Integrate Python and JavaScript applications using the official libraries, or deploy the model server in Docker.
• Generate text embeddings with dedicated embedding models for semantic search, document retrieval and retrieval-augmented generation (RAG).
• Use tool calling with compatible models so applications can execute functions and return their results to the conversation.
• Request structured JSON responses using a JSON schema for local applications that need predictable data formats. Ollama Cloud currently does not support structured outputs.
• Inspect loaded models and CPU or GPU usage with ollama ps, and disable cloud features when you want local operation.