Blog

Notes from building NTPSense.

Architecture decisions, feature deep-dives, and the occasional root-cause writeup — as they happen, not polished after the fact.

Multi-WAN that knows the difference between "dedicated" and "shared"

Most gateway failover logic treats every WAN link the same — pick the highest-priority one that's up, move to the next tier if it isn't. That's fine until your topology has a genuine asymmetry: a dedicated fiber line with your own public IP, and a shared/NAT wireless backup that's really only there for emergencies.

The traffic that matters most for something like our Site Mesh VPN — the mesh coordination itself, NTP sync, DNS — is router-originated, not client transit traffic. It already follows whatever the system default gateway is set to. The gap wasn't the failover mechanism; it was that nothing in the data model distinguished a dedicated uplink from a shared one, so nothing could warn an admin who'd accidentally put the shared link at a higher priority tier than the dedicated one.

The fix was small on purpose: a link_type field on each gateway, and a validator that warns — not blocks — when the tier ordering doesn't match. Validated live by deliberately mis-ordering a real topology and watching the warning appear, then disappear once corrected.

Site Mesh VPN: self-hosted, no third-party relay in the trust path

Full-mesh site-to-site connectivity is table stakes for a branch-office gateway. Where we made a deliberate choice: the coordination server (Headscale) runs on hardware you control, not a SaaS backend. Your keys never leave your infrastructure.

Getting this production-solid meant working through real coordination-layer issues that don't show up in a demo — node deregistration when a site is deleted, a race condition where the mesh interface silently comes up without an IP address, field-name changes between Headscale versions. Each one confirmed against real VM logs before we called it done, not assumed fixed because the code looked right.

Why we built NTPSense from scratch on FreeBSD instead of forking pfSense

The honest answer: forking gets you to a demo faster, but you inherit someone else's architecture decisions along with their technical debt. We wanted a control plane we actually understood end-to-end — a single Rust daemon (ntpsense-configd) governing pf, Kea DHCP, WireGuard, and Squid through one Unix-socket interface, not a collection of shell scripts calling each other.

FreeBSD itself wasn't a hard requirement so much as a consequence — native pf, native wg(4), a base system that doesn't fight you. The tradeoff is real: less prior art to lean on, more root causes to chase down ourselves. We think it's worth it for a product whose entire job is being trustworthy at the network edge.

Want to see it running?

Pick a tier, get an ISO, and be filtering traffic in under an hour.

Get Started →