Security camera descriptions and search with Frigate GenAI

What they built

Frigate, the open-source NVR, can hand camera snapshots and frame sequences to a generative AI provider. That gives you object descriptions, review summaries, semantic search embeddings and chat.

Local providers (llama.cpp, Ollama) keep all processing on your own hardware.

Setup

• Provider: ollama or llamacpp (llama.cpp also supports transcription)
• Recommended vision models: qwen3-vl (frames), qwen3.6/qwen3.8, gemma4 (annotated_frames)
• Roles per provider: descriptions, chat, embeddings, transcribe

Gotchas

• The docs warn that running GenAI on CPU is impractical because of inference times
• Embeddings need a multimodal embedding model; a chat model returns vectors that silently give poor search

“Local providers run on your own hardware and keep all data processing private.”— Frigate docs

Do it yourself

1. Run Ollama on a GPU machine and pull qwen3-vl:4b
2. Add a genai provider block to the Frigate config
3. Assign roles (descriptions, chat, embeddings) as needed
4. Enable descriptions per camera for the objects you care about

genai:
  my_provider:
    provider: ollama
    base_url: http://localhost:11434
    model: qwen3-vl:4b
    roles:
      - descriptions
      - embeddings
      - chat