Encrypted DNS + NTP = Deadlock

Photo by Uzoma Ozurumba, CC BY-SA 4.0, no changes

I’m generally a big fan of encrypted DNS for the security it provides with negligible performance impact. But I recently found out the hard way that DNS over TLS/HTTPS can deadlock with Network Time Protocol (NTP) on embedded devices without a battery-backed real-time clock.

Many low-cost routers lack a battery for keeping time during power loss. After these devices lose mains power, they start with their clock initialized to some early time, usually the Unix epoch, January 1st, 1970.

The device then uses NTP to retrieve the current time over the internet. The NTP source is specified as a set of domain names like pool.ntp.org, so the device begins DNS resolution of the chosen domain to begin NTP.

As part of the encrypted DNS process, the TLS certificate of the upstream provider is validated. But TLS certificates are only valid within a certain time frame, which is specified by the ‘Not Before’ and ‘Not After’ conditions on the certificate. So validating them requires local time to be correct!

And so a deadlock occurs — we can’t update local time because we can’t resolve domain names because we don’t have correct local time!

To address this, I suggest using a few IP addresses for NTP servers so the device can bootstrap. Almost no one provides IP addresses for NTP servers, NIST is the only one I can find. It would be great to see Google or Cloudflare use their infrastructure to provide anycasted NTP IP addresses.

I scanned the relevant RFCs and couldn’t find any awareness of this circular dependency.

If you use DNS over TLS/HTTPS on your router, add one of the NIST IP addresses to your NTP configuration before you lose power and it deadlocks on startup like mine did.

Notes

Great discussion on Hacker News. Others report this issue can occur with other security tools, including DNSSEC, Wireguard, and 802.1X.

Thank you to David Orr for reviewing this post.

--

--

Hacker, entrepreneur, and quantified self nerd. cyounkins at gmail.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Craig Younkins

Hacker, entrepreneur, and quantified self nerd. cyounkins at gmail.