Home Hub
Moved into a new place. Chronic procrastination set in. For a techie, there's really only one acceptable form of procrastination — build a smart home from scratch.
Two months later: a self-hosted stack on a Beelink mini-PC, a local AI agent named Nova, a 3D Three.js dashboard, an iPhone-as-house-remote, and a small physical robot being assembled to give Nova a body.
how it snowballed
The Brain
Local · regex-fast · increasingly opinionated
Meet Nova
The local AI agent that ties the apartment together. FastAPI + Ollama (qwen3:8b), running entirely on the Beelink. No cloud LLM in the hot path.
Telegram iPhone App Samsung TV / Mac
│ │ │
▼ ▼ ▼
┌────────┐ ┌────────┐ ┌──────────┐
│ bot │ │ voice │ │ http │
│polling │ │ ws │ │ /chat │
└────┬───┘ └────┬───┘ └────┬─────┘
└──────────────────┼──────────────────┘
▼
┌────────────────────┐
│ NovaDispatcher │ ← single entry point
└─────────┬──────────┘
▼
┌───────────┴───────────┐
▼ ▼
┌─────────────┐ ┌──────────────┐
│ regex │ no │ ollama │
│ pattern │ ──────► │ qwen3:8b │
│ matcher │ match │ (local LLM) │
└──────┬──────┘ └──────┬───────┘
│ match │ classify
▼ ▼
┌──────────────────────────────────────┐
│ 18 intent handlers │
│ HOME_CONTROL · TV · ROUTINE · ... │
└──────────────────────────────────────┘Two-layer classifier
Most messages hit a regex pattern in under a millisecond — zero LLM tokens. Only the ambiguous ones fall through to qwen3:8b for classification. Keeps response times under 1ms for ~80% of commands.
Single dispatcher
Telegram, voice WebSocket, and HTTP all funnel into one NovaDispatcher.dispatch(). Channels handle their own UI quirks (keyboards, TTS), routing logic is shared.
Voice & wake word
Self-trained OpenWakeWord model recognizes “Hey Nova”. Voice replies via Piper TTS using a cloned Jarvis voice from Iron Man. (Iron Man tax: paid.)
Three channels, one dispatcher
Different transports, same brain. Channel-specific UI (inline keyboards, TTS audio) handled at the edge — routing logic shared.
Telegram
bot pollingText messages, /commands, receipt photos (→ Gemini OCR → Notion)
Voice WebSocket
/ws/nova/voice16kHz PCM audio from the iPhone app. Self-trained wake word → Whisper STT → dispatch → Piper (Jarvis voice) TTS
HTTP
POST /api/nova/chatPlain JSON for the dashboard and any web client
NovaDispatcher.dispatch(text, chat_id, channel)What Nova does
18 intents across 8 domains. Plus pronoun resolution ('turn it off'), confirmation state machine ('Turn off all 6 lights? [Yes][No]'), and a tool-calling PlannerAgent for compound commands.
HOME_CONTROLLights, switches, brightness, fans
HOME_VACUUMRoborock — start, pause, dock, room
HOME_STATUS"Is the kitchen light on?"
PRESENCE_QUERYWho's home — by MAC
TV_CONTROLPower, apps, volume, d-pad
MEDIA_PLAY"play X on spotify/youtube"
MEDIA_PLAYBACKPause, skip, what's playing
WEATHERWeather, forecast, temperature
TIMER_ALARMNatural-language timers
CALENDARSchedule, events
MEMORYReminders, shopping list, notes
ROUTINEGood morning, bedtime, leaving
SCENEMovie mode, party, date night
COMPOUNDPlannerAgent — tool-calling
WIFI_QRShow WiFi QR on TV
REMOTE_QRShow TV remote link QR
CONVERSATIONOpen-ended chat (qwen3 /think)
INFO_QUERYFactual questions (/nothink)
Scene Editor
Visual programming for the home — drag-and-drop blocks with trigger / condition / action types, branching flowcharts, and live execution tracking. Build a routine without writing code.
scene · movie_modescene · wine_modescene · welcome_homeThe Body
Screens · lights · and, soon, limbs
Surfaces
What you actually see and touch. The iframes below are real demo builds of the apps — click around, toggle lights, trigger scenes, chat with Nova. Mock state lives in your browser and syncs across iframes.
3D Apartment Dashboard
Three.js + React + Vite · entry: mac.htmlA 3D scene of the actual apartment — rooms, lights, the TV. Click a lamp, the real lamp turns on. First version was for the Mac browser.
Then I wanted it on the Samsung TV. Turns out Samsung doesn't exactly want you uploading custom apps to their TVs — but if you find the right magic incantation in dev mode, you can sideload one anyway. I did. Same Vite build, three entry points: /app (mobile), mac.html, tv.html.
iPhone Dashboard
React + Capacitor · tabbed bottom navThree tabs: Lights (5 rooms with toggles + brightness sliders + “2 of 3 on” counter), Remote (Spotify / YouTube / TV — try the rickroll), Nova (real chat with chip shortcuts).
About Netflix and Prime: their integration is, quote, painfully thin. You can open the app. That is the entire API surface they expose.
Network Monitor (v3 → v4)
Pi-hole DNS · FRITZ!Box TR-064 · InfluxDB · Grafana“People-first” presence dashboard. Detects who's home by MAC (FRITZ!Box TR-064 + Pi-hole DNS queries), tracks usage in InfluxDB, visualizes in Grafana. Drives the morning briefing and Welcome-Home routine.
A body for Nova
The current procrastination project — moving Nova out of a kitchen phone and into a real robot.
Three ESP32 codebases, one robot
First version of always-listening Nova ran on a 10-year-old phone in the kitchen — voice app on a permanent charger, microphone open. It worked. It was also, undeniably, a phone on a charger.
The replacement: a small physical robot with an animated face, body effects, and a microphone. The web preview above is the actual implementation — same p5.js eye styles (roboeyes / cozmo / kawaii / statusring), same canvas body renderers (waveform / lavalamp / horror / matrix / gauges / status), same scene system. Switch the buttons.
The hardest part: GIF streaming over MQTT — RGB565 little-endian frame decoding took longer to debug than I'd like to admit. On the actual ESP32 hardware, GIFs play directly on the body screen via the AnimatedGIF library off SPIFFS. Here's what the GREETING intent renders:
esp-head/Animated eye display — blink, look, expressions, GIF playback
esp-body/Body controller — movement, scene orchestration
esp-body-effects/Visual effects library — lava lamp, horror, wave standby (15+ board configs)
The Craft
Details · surprises · process · dead ends
The little things
Side quests, surprises, and 'wait, it does that too?' moments.
Dead ends
Mature projects have graveyards. Three paths sketched, started, or experimented with — then cut. Knowing what to remove matters as much as knowing what to ship.
The full stack
Layered, mostly local, mostly open-source.
Built one design doc at a time
Every meaningful feature gets a written design plan before any code. 21 documents, ordered by build date. Peak velocity: 35 commits on Apr 8, 30 on Apr 2, 29 on Apr 9.
Personal project · Andrei Zitti · started Feb 2026, still going