AegisVision
A plug-and-play, multi-camera AI surveillance and behavioral-monitoring platform. Combines real-time face recognition with anti-spoofing liveness checks, occupancy/gate analytics, spatial anti-theft memory, instant alerting, and a generative-AI copilot for querying the live database in plain English.
- stars
- 7
- forks
- 2
- watchers
- 1
- tests passing
- 51
- detection latency ms
- <10 on CPU (SCRFD-500M)
AegisVision turns any CCTV camera, webcam, or RTSP stream into an intelligent command center for offices, schools, colleges, and work sites.
Core Capabilities
- Plug-and-play enrollment — drop a folder of a person's photos + info.json
into
data/known_persons/and they're recognized within minutes. - Real-time recognition — SCRFD + ArcFace via ONNX, with an automatic dlib fallback if the primary backend is unavailable.
- Privacy-first by default — unknown faces are tagged on-frame but never written to disk unless explicitly enabled.
- Anti-spoofing — liveness scoring gates every match against face texture.
- Behavioral analytics — gate in/out tracking, live occupancy, idle vs. active time, phone-distraction detection, drowsiness/fatigue signals, and spatial anti-theft memory for tracked assets.
- Real-time alerting — rule-based escalation pushed via WebSocket, Telegram, or JSON webhook.
- Generative-AI copilot — natural-language querying over the live database, with a rule-based fallback when no API key is configured.
Architecture
One background capture worker per camera feeds a shared pipeline: detect → track → anti-spoof → embed → match. SQLite holds relational data while ChromaDB stores vector embeddings, kept in sync as the single source of truth. FastAPI serves typed REST routes plus MJPEG/WebSocket streams, with the whole stack deployable as a single Docker Compose host behind NGINX.
The hard part
- Swappable recognition backends — supporting both ONNX/insightface and a dlib fallback meant the vector index had to self-heal when embedding dimensions changed (128 → 512) between backends.
- Privacy vs. usefulness tradeoff — designing the system to never persist unknown-face snapshots by default, while still keeping alerting useful, required careful gating logic rather than just disabling the feature outright.
- Real-time multi-camera load — running detect → track → anti-spoof → embed → match per camera without frame backlog needed a per-camera worker model with back-pressure and auto-reconnect for flaky RTSP streams.
- Alert fatigue — raw detections needed a rule-engine layer (e.g. UNKNOWN×3, repeated drowsiness) to avoid flooding operators with noise.
Outcome
- Shipped a working end-to-end pipeline: camera ingestion, recognition, behavioral analytics, alerting, and a queryable copilot, all in one deployable stack.
- 51 passing tests covering API smoke tests, plug-and-play ingestion, vector semantics (including backend-swap dimension rebuilds), and recognition backend selection.
- Sub-10ms CPU detection latency using the SCRFD-500M model, keeping the system usable without a GPU.
- Deployable as a single Docker Compose stack with no per-site configuration.
What I'd do differently
- Centralizing all recognition thresholds (
FACE_DISTANCE_THRESHOLD,ANTI_SPOOF_THRESHOLD, etc.) in one config file made tuning and debugging far easier than scattering them across services. - Designing for backend-agnostic recognition (ONNX first, dlib fallback) early on avoided a rewrite later and made the system resilient to missing native dependencies.
- Defaulting to "never write unknown faces to disk" forced privacy to be a first-class design constraint rather than an afterthought.
- Keeping a relational store (SQLite) as the source of truth with a vector index (Chroma) as a mirror — rather than vice versa — simplified consistency handling significantly.