Ollama

CLI

• Pull a model

ollama pull <model_name>

• Show model Info

# List all models
ollama list
 
# Show model info
ollama show <model_name>
 
# Show model template
ollama show <model_name> --template

• Chat with model in CLI

ollama run <model_name>

• See running models

ollama ps

• See Debug logs

cat ~/.ollama/logs/server.log
 
# Continous monitor new logs
tail -f ~/.ollama/logs/server.log

• Enable Debug mode
  • https://github.com/ollama/ollama/pull/10650
  • Env variable: OLLAMA_DEBUG
    • 0, false: INFO level
    • 1, true: DEBUG level
    • 2: TRACE level

# Quit Ollama and start from terminal
OLLAMA_DEBUG=2 ollama serve

API

• https://docs.ollama.com/api
• /api/generate
  • Generates a response for the provided prompt
• /api/chat
  • Generate the next chat message in a conversation between a user and an assistant
• /api/embed
  • Creates vector embeddings representing the input text
• List and Show Models
  • /api/tags
  • /api/ps
  • /api/show
• Create a new Model
  • /api/create
    • Create Model with following params
      • Prompt template
      • System prompt
      • Parameters
      • Message History
      • Quantization Level
  • /api/copy
    • Copy a Model
• Managing Models
  • /api/pull
    • Pull a Model
  • /api/push
    • Push a Model
  • api/delete
    • Delete a Model
• /api/version
  • Ollama Version

Generate vs Chat API

• https://github.com/ollama/ollama/issues/2774

Example

• Conversation

[
    {
        "role": "system",
        "content": "You are a bored assistant. Provide short answers.",
    },
    {
        "role": "user",
        "content": "Why is the sky blue?",
    },
    {
        "role": "assistant",
        "content": "Because the gods wanted it that way.",
    },
    {
        "role": "user",
        "content": "Why did the gods want it that way?",
    }
]

• Llama3 Template

{{ if .System }}<|start_header_id|>system<|end_header_id|>
 
{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>
 
{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>
 
{{ .Response }}<|eot_id|>

• Chat Raw Text

<|start_header_id|>system<|end_header_id|>

You are a bored assistant. Provide short answers.<|eot_id|><|start_header_id|>user<|end_header_id|>

Why is the sky blue?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Because the gods wanted it that way.<|eot_id|><|start_header_id|>user<|end_header_id|>

Why did the gods want it that way?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

• Generate Raw Text

<|start_header_id|>user<|end_header_id|>

[{"role": "system", "content": "You are a bored assistant. Provide short answers."}, {"role": "user", "content": "Why is the sky blue?"}, {"role": "assistant", "content": "Because the gods wanted it that way."}, {"role": "user", "content": "Why did the gods want it that way?"}]<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Example

• Qwen2.5:7b
• Chat Template

{{- if .Messages }}
{{- if or .System .Tools }}<|im_start|>system
{{- if .System }}
{{ .System }}
{{- end }}
{{- if .Tools }}
 
# Tools
 
You may call one or more functions to assist with the user query.
 
You are provided with function signatures within <tools></tools> XML tags:
<tools>
{{- range .Tools }}
{"type": "function", "function": {{ .Function }}}
{{- end }}
</tools>
 
For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
<tool_call>
{"name": <function-name>, "arguments": <args-json-object>}
</tool_call>
{{- end }}<|im_end|>
{{ end }}
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 -}}
{{- if eq .Role "user" }}<|im_start|>user
{{ .Content }}<|im_end|>
{{ else if eq .Role "assistant" }}<|im_start|>assistant
{{ if .Content }}{{ .Content }}
{{- else if .ToolCalls }}<tool_call>
{{ range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{ end }}</tool_call>
{{- end }}{{ if not $last }}<|im_end|>
{{ end }}
{{- else if eq .Role "tool" }}<|im_start|>user
<tool_response>
{{ .Content }}
</tool_response><|im_end|>
{{ end }}
{{- if and (ne .Role "assistant") $last }}<|im_start|>assistant
{{ end }}
{{- end }}
{{- else }}
{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ end }}{{ .Response }}{{ if .Response }}<|im_end|>{{ end }}%