Failover Time Calculator

You build a highly available stack to power your home lab. When you need it most, it falls over. Primary goes down. Secondary comes back online, but does nothing. Packets gets lost. You look at the loading screen. Reality bites and the recovery time isn’t what your hardware dictates. It’s usually someone waiting for their coffee or a DNS cache.

You don’t need a calculator to do the maths, but knowing how those numbers arrive is far more important then the total. The first problem is detection. Health check are not an on/off switch; we imagine it should be. Instead, its running a probe every couple seconds, and if it miss three times, then only at that point does it declare something has failed. This means that by the time it wants to do something, it has been six second since its last probe. Make that lower and you’re faster…but now you risk noise. You don’t want your cluster to promote a new leader when your network hiccuped for a heartbeat, just because there was a dropped packet. So here’s the tension between speed and stability in most HA designs.

Why Your Backup Server Is Slower Than You Think

Finally, there’s the promotion itself. Moving a VIP is fast, but booting a VM is not. That’s why the calculator allows you to set the promotion time, taking into account how your architecture work. For example, with Patroni running PostgreSQL, an election will take 15 seconds. But if you’re rebooting a guest in a hypervisor, that can be many minutes. The input makes you confess what you actualy do when you restart something. I see a lot of people thinking that their backup server is hot and running. In reality, it’s often asleep and cold.

This second layer of complexity is DNS and no one sees it coming. Promote a database in ten seconds, but your users will still get an error because their client resolver has cached the old IP address for three hundred seconds. By comparing DNS TTL against your traffic patterns, the tool brings this to light. DNS doesn’t matter much if you’re moving between backends behind a load balancer. But it controls the whole recovery time if you’re cutting over domain records. Most people miss this when planning their RTO.

The slowest variable in the room? Human intervention. A manual runbook step adds minutes to your outage. An assisted approval process also add delay. Automation factors are applied to decrease this delay and reflect how much of the decision you’ve pre-approved for yourself. Your need for a thumbs up from an engineer at 2 AM, is part of your recovery time. Often the most useful change you can make is automating that approval.

But how servers behave is only half the story: clients also matter. Load balancers don’t immediately kill connections (that would lose data!), but instead drain them. Browsers will hang on stale TCP sessions. Apps will keep trying connections with exponential backoff until they work. And these behaviors are there to protect data integrity, but at the expense of speed. Because nobody’s connected again, a perfectly healthy backend could still make a frontend look broken. This tool has inputs for those kinds of things too: reconnect time and retry backoff, which are very real obstacles.

This is all laid out pretty clearly in the reference table on the page which lists typical ranges for various technologies. It serves as a good reminder that DNS cutover is measured in minutes and VRRP failover is measured in seconds. Knowing where your architecture fits based off that spectrum will help you set realistic expectations. You can’t force a DNS record to update faster than its TTL without breaking caching entirely. But you can tune your health check intervals or warm up your standby nodes.

Buying all that fancy equipment isn’t really planning for failure. Planning for failure is mostly managing your expectations. The seconds spent recovering originate somewhere. These are the seconds spent in boot processes or stale caches. These are the seconds it takes to detect something. If you know where the biggest waste of your time is, you can optimise well. Spending 1% less every place results in diminishing returns. Go after the bottleneck.

A number isn’t enough. Pressing calculate should tell you more than a number. It should give you a map of where you are weak and where you are strong. You can tweak your approval thresholds or automate them. Or even where your actual recovery bottlenecks is. Because that’s what matters, the goal is not zero downtime, because that is a myth. What matters is knowing exactly how long the lights are going to be out when they do go out.