OpenNVR benchmarks every model its camera agent uses on a 4-core CPU

What they built

Varun Pratap Singh maintains OpenNVR, an open source video recorder with a camera agent built in: you can ask it what's happening on a camera, arm an alarm or search old footage, and it will soon look into events on its own.

Most people who run it have a small 4-core PC and no GPU, so the team tested every model the agent depends on (tool-calling language model, vision, speech to text, text to speech and image search) on a 4 vCPU machine, instead of picking models by reputation. The test scripts and raw numbers are in the repo so anyone can rerun them.

Four defaults changed, and several results were not what they expected, starting with a 4B language model that scored worse than a 1.7B one.

Setup

• A 4 vCPU machine with no GPU; all timings on 4 CPU threads
• Language models served through Ollama (granite4:micro, granite4:tiny-h, qwen3:1.7b, LFM2-1.2B-Tool, qwen3:4b)
• Vision: qwen3-vl:2b-instruct, gemma3:4b, Moondream 0.5B
• Speech: Parakeet TDT 0.6B v3 via onnx-asr and faster-whisper; TTS: Piper and Kokoro-82M
• Search: SigLIP 2 B/16 vs CLIP ViT-B/32; detector YOLO26n vs YOLOv8n

How it works

1. Language model: 20 investigation cases such as 'person at the loading bay at 01:14', each run 3 times; the model must check the camera, history and site rules before deciding.
2. Vision: 128 labelled images with 3 questions each, including one about something that isn't there, where a 'yes' counts as a hallucination, plus a people count.
3. Speech to text measured word error rate on the phrases people actually say to the agent; TTS checked whether it runs faster than real time.
4. Search tested finding the right image from a text description like 'red truck at the dock'.

Results & numbers

• granite4:micro passed 44 of 60 cases; qwen3:4b passed 9 of 60 because it thought for a long time and almost never called a tool. Granite 4 micro is now the default, at roughly twice the time per turn.
• qwen3-vl:2b-instruct gave 0 false 'yes' answers out of 126 but needed 18 s per frame on CPU, so Moondream 0.5B (7.6 s, 22 false yeses) stays the CPU default and Qwen3-VL is used when a GPU is present.
• Parakeet TDT 0.6B v3 hit 1.4% word error rate at 0.31 s per sentence, versus 3.3% and 1.42 s for faster-whisper base.en.
• Kokoro-82M ran at 1.9x real time on 4 cores, so Piper (0.23x) was kept.
• SigLIP 2 B/16 scored 0.672 mAP against CLIP's 0.571 for footage search; the detector moved from YOLOv8n (75 ms) to YOLO26n (62 ms).

Gotchas

• In Ollama, the plain qwen3-vl:2b tag is the thinking version and ignores think: false; use the -instruct tag.
• Telling the model to always look before describing made Granite check a camera when a user said 'thanks, that's all'. Small talk now goes to the model without tools, which helped every model.
• Parakeet can't translate and doesn't do Hindi, so Whisper stays for sites that need that.
• Rankings flip on a GPU, so test on the hardware your users actually have.

“Whether the model checks with its tools before deciding mattered more than parameter count.”— Varun Pratap Singh

Do it yourself

1. List every model your local agent depends on (LLM, vision, speech, TTS, search).
2. Write one small test per job with known answers, including a 'thing that isn't there' question to catch vision hallucinations.
3. Run every candidate on the hardware your users have, with the thread count fixed, and repeat each case a few times.
4. Read the traces, not just scores: check whether the agent actually calls tools before deciding.
5. Pick defaults per hardware class, for example a fast small vision model on CPU and a more accurate one when a GPU is present.
6. Reuse or compare against OpenNVR's published harness in examples/camera-agent/MODELS_AND_LATENCY.md.