AegisVision
A multi-camera surveillance and behavioral monitoring platform that runs on ordinary hardware. It does real-time face recognition with anti-spoofing liveness checks, tracks occupancy and gate traffic, remembers where tracked assets were last seen, sends alerts, and includes an AI copilot that answers questions about the live database in plain English.
- stars
- 7
- forks
- 2
- watchers
- 1
- tests passing
- 51
- detection latency ms
- <10 on CPU (SCRFD-500M)
AegisVision turns any CCTV camera, webcam, or RTSP stream into a monitoring system for offices, schools, colleges, and work sites.
What it does
- Enrollment is plug and play: drop a folder of photos and an info.json into
data/known_persons/and the person is recognized within minutes. - Recognition runs SCRFD + ArcFace through ONNX, with an automatic dlib fallback if the primary backend isn't available.
- Unknown faces are tagged on-frame but not written to disk unless you explicitly turn that on.
- Anti-spoofing: every match is gated by a liveness score checked against face texture.
- Behavioral analytics: gate in/out tracking, live occupancy, idle vs. active time, phone-distraction detection, drowsiness and fatigue signals, and a spatial memory of where tracked assets were last seen.
- Alerts escalate through rules and go out over WebSocket, Telegram, or a JSON webhook.
- An AI copilot answers natural-language questions against the live database, with a rule-based fallback when no API key is configured.
Architecture
Each camera has its own background capture worker feeding one shared pipeline: detect, track, anti-spoof, embed, match. SQLite stores relational data and ChromaDB stores the vector embeddings, kept in sync with SQLite as the source of truth. FastAPI serves typed REST routes plus MJPEG and WebSocket streams. The whole stack deploys as a single Docker Compose host behind NGINX.
The hard part
- Swappable recognition backends: supporting ONNX/insightface with a dlib fallback meant the vector index had to rebuild itself when embedding dimensions changed between backends (128 to 512).
- Privacy vs. usefulness: the system doesn't persist unknown-face snapshots by default, so keeping alerts useful required careful gating rather than just switching the feature off.
- Real-time load across cameras: running detect, track, anti-spoof, embed, match per camera without a frame backlog needed a per-camera worker model with back-pressure and auto-reconnect for flaky RTSP streams.
- Alert fatigue: raw detections go through a rule engine (for example UNKNOWN three times, or repeated drowsiness) so operators don't get flooded.
Outcome
- An end-to-end pipeline that works: camera ingestion, recognition, behavioral analytics, alerting, and a queryable copilot in one deployable stack.
- 51 passing tests covering API smoke tests, plug-and-play ingestion, vector semantics including backend-swap dimension rebuilds, and recognition backend selection.
- Detection under 10ms on CPU with the SCRFD-500M model, so a GPU isn't required.
- Deployable as a single Docker Compose stack with no per-site configuration.
What I'd do differently
- Putting every recognition threshold (
FACE_DISTANCE_THRESHOLD,ANTI_SPOOF_THRESHOLD, and so on) in one config file made tuning and debugging far easier than spreading them across services. - Designing for a backend-agnostic recognizer early (ONNX first, dlib fallback) avoided a rewrite later and kept the system working when native dependencies were missing.
- Making never-write-unknown-faces-to-disk the default forced privacy into the design instead of leaving it as something to add later.
- Using SQLite as the source of truth with Chroma as a mirror, rather than the other way around, simplified consistency handling.
Architecture & screenshots
More in AI / ML
OpenGrammar
Privacy-first, self-hosted writing assistant: a browser extension with a 156,000-word offline engine and more than 40 rule patterns, an AI router that only sees what local rules miss, and a Hono backend that deploys to five runtimes.
contexa
Versioned memory for LLM agents, built on Git's branching model: observation-thought-action logs, commits, branches and merges for an agent's context, implemented in seven languages that share one .GCC/ on-disk format.