2023-07-16 17:00:09 -07:00
2023-07-16 16:30:07 -07:00
2023-07-16 16:43:11 -07:00
2023-07-14 21:41:46 -07:00
2023-07-14 20:11:44 -07:00
2023-07-12 19:16:39 -07:00
2023-07-16 17:00:09 -07:00
2023-07-06 16:34:44 -04:00
2023-07-02 17:34:46 -04:00
2023-07-11 14:58:10 -07:00
2023-07-11 14:58:10 -07:00
2023-06-26 15:57:13 -04:00
2023-07-13 17:13:00 -07:00
2023-07-12 09:42:26 -07:00
2023-07-12 19:16:28 -07:00

ollama

Ollama

Run large language models with llama.cpp.

Note: certain models that can be run with Ollama are intended for research and/or non-commercial use only.

Features

  • Download and run popular large language models
  • Switch between multiple models on the fly
  • Hardware acceleration where available (Metal, CUDA)
  • Fast inference server written in Go, powered by llama.cpp
  • REST API to use with your application (python, typescript SDKs coming soon)

Install

  • Download for macOS
  • Download for Windows (coming soon)

You can also build the binary from source.

Quickstart

Run a fast and simple model.

ollama run orca

Example models

💬 Chat

Have a conversation.

ollama run vicuna "Why is the sky blue?"

🗺️ Instructions

Get a helping hand.

ollama run orca "Write an email to my boss."

🔎 Ask questions about documents

Send the contents of a document and ask questions about it.

ollama run nous-hermes "$(cat input.txt)", please summarize this story

📖 Storytelling

Venture into the unknown.

ollama run nous-hermes "Once upon a time"

Advanced usage

Run a local model

ollama run ~/Downloads/vicuna-7b-v1.3.ggmlv3.q4_1.bin

Building

go build .

To run it start the server:

./ollama server &

Finally, run a model!

./ollama run ~/Downloads/vicuna-7b-v1.3.ggmlv3.q4_1.bin

API Reference

POST /api/pull

Download a model

curl -X POST http://localhost:11343/api/pull -d '{"model": "orca"}'

POST /api/generate

Complete a prompt

curl -X POST http://localhost:11434/api/generate -d '{"model": "orca", "prompt": "hello!"}'
Description
Get up and running with Llama 2, Mistral, and other large language models.
Readme MIT 170 MiB
Languages
Go 93.3%
C 2.6%
Shell 1.2%
TypeScript 1%
PowerShell 0.8%
Other 1%