Paperless AI on an 8 GB RTX 3060 Ti shared with Frigate
What they built
Replying to a user whose Paperless AI wasn't triggering, u/dpy22101 shared a working setup: Paperless-ngx 3.0.2's built-in AI running locally through Ollama on a modest, shared GPU.
Setup
- Paperless-ngx v3.0.2, built-in AI via Ollama
- GPU: NVIDIA RTX 3060 Ti 8 GB (shared with Frigate and occasionally Plex)
- LLM: Gemma 3 4B (gemma3:latest); embeddings: embeddinggemma
- Context size 8,192 tokens
Results & numbers
- Library: 1,318 documents / 7,892 pages
- Response time ~5.4 s median, ~6.1 s average
- LLM VRAM use ~2.7 GB, fully GPU-resident
Gotchas
- Qwen3 8B was partly offloaded to CPU on the limited VRAM and was significantly slower
Do it yourself
- Pick a small model that fits entirely in your free VRAM (here Gemma 3 4B)
- Pull an embedding model too (embeddinggemma)
- Set the Paperless AI backend to Ollama with an 8k context
- Check that the model stays 100% on GPU when other services share the card
Related
Home RAG assistant for embedded-systems docs on Proxmox
Ollama, llama.cpp, Open WebUI, BGE-M3 and SearXNG in a GPU-passthrough VM. Moving from 6 GB to 16 GB of VRAM changed which models could run.
Auto-tagging documents in Paperless-ngx with Ollama
Since version 3.0, Paperless-ngx suggests titles, senders, document types and tags, and can chat with your documents, using a local Ollama model.
Self-hosted bookmarks that tag themselves with local AI (Karakeep)
Karakeep (formerly Hoarder) runs on a Debian LXC, and gemma3:4b in Ollama tags each bookmark. Getting reliable tags from a small model takes one key setting.
Self-hosted AI email assistant on Linux (n8n + Ollama)
Classifies IMAP mail with llama3.1:8b, routes it, saves reply drafts and extracts action items as JSON. Reports 85-95% classification accuracy.