2023-07-07 09:34:05 -07:00
2023-07-07 09:34:05 -07:00
2023-07-07 10:27:33 -04:00
2023-07-07 10:22:37 -04:00
2023-07-01 17:54:29 -04:00
2023-07-07 09:34:05 -07:00
2023-07-07 09:34:05 -07:00
2023-07-06 16:34:44 -04:00
2023-07-06 16:34:44 -04:00
2023-07-02 17:34:46 -04:00
2023-07-06 16:34:44 -04:00
2023-07-06 17:07:40 -07:00
2023-07-06 17:07:40 -07:00
2023-06-26 15:57:13 -04:00
2023-07-06 16:34:44 -04:00
2023-07-06 16:34:44 -04:00
2023-07-06 16:34:44 -04:00
2023-07-06 16:46:05 -04:00

ollama

Ollama

Run large language models with llama.cpp.

Note: certain models that can be run with this project are intended for research and/or non-commercial use only.

Features

  • Download and run popular large language models
  • Switch between multiple models on the fly
  • Hardware acceleration where available (Metal, CUDA)
  • Fast inference server written in Go, powered by llama.cpp
  • REST API to use with your application (python, typescript SDKs coming soon)

Install

  • Download for macOS
  • Download for Windows (coming soon)
  • Docker: docker run -p 11434:11434 ollama/ollama

You can also build the binary from source.

Quickstart

Run the model that started it all.

ollama run llama

Example models

💬 Chat

Have a conversation.

ollama run vicuna "Why is the sky blue?"

🗺️ Instructions

Ask questions. Get answers.

ollama run orca "Write an email to my boss."

👩‍💻 Code completion

Sometimes you just need a little help writing code.

ollama run replit "Give me react code to render a button"

📖 Storytelling

Venture into the unknown.

ollama run nous-hermes "Once upon a time"

Advanced usage

Run a local model

ollama run ~/Downloads/vicuna-7b-v1.3.ggmlv3.q4_1.bin

Building

make

To run it start the server:

./ollama server &

Finally, run a model!

./ollama run ~/Downloads/vicuna-7b-v1.3.ggmlv3.q4_1.bin

API Reference

POST /api/pull

Download a model

curl -X POST http://localhost:11343/api/pull -d '{"model": "orca"}'

POST /api/generate

Complete a prompt

curl -X POST http://localhost:11434/api/generate -d '{"model": "orca", "prompt": "hello!", "stream": true}'
Description
Get up and running with Llama 2, Mistral, and other large language models.
Readme MIT 170 MiB
Languages
Go 93.3%
C 2.6%
Shell 1.2%
TypeScript 1%
PowerShell 0.8%
Other 1%