Key concepts: RTO, RPO and the difference between DR and HA

RTO (Recovery Time Objective) is the maximum downtime the business can tolerate from when a disaster occurs until the system is operational again. If the business says "we cannot be down for more than 2 hours," the RTO is 2 hours.

RPO (Recovery Point Objective) is the maximum amount of data the business can lose. If the RPO is 1 hour, the system must be able to recover data up to less than 1 hour before the disaster.

The difference between RTO and RPO defines the technology needed: an RTO of 30 minutes and RPO of 5 minutes requires an active real-time replica, not daily backups.

HA vs DR: High Availability handles individual node failures within the same datacenter (seconds of interruption, automatic). Disaster Recovery handles the complete loss of a datacenter or region (minutes or hours, generally manual). They are complementary and must be planned together.

Signs that DR is not ready

  • There is no written runbook with numbered steps the team can execute without improvising
  • RTO and RPO have never been formally defined or approved by the business
  • The production backup is on the same server or datacenter as the database
  • Nobody has tested that backup restoration works in the DR environment
  • Only one person knows the failover procedure
  • The secrets (passwords, keys) needed to execute DR are stored only on the primary server

Business risks

  • A real disaster with an untested DR can result in weeks of recovery instead of hours
  • Data loss may exceed the approved RPO if backups are not updated at the correct frequency
  • Regulated companies (financial sector, healthcare) may face sanctions if they cannot demonstrate recovery capability
  • Dependence on a single person is an operational risk that can collapse DR if that person is unavailable

Components of a DR plan for databases

  • 1. RTO and RPO defined and approved by the business Without these numbers signed by a business owner, the technical team cannot design the correct architecture. Each critical database must have its own RTO and RPO.
  • 2. Backup strategy aligned with the RPO If the RPO is 1 hour, backups must be frequent (transaction logs every 15 minutes, full backup daily). If the RPO is 24 hours, a daily backup may be sufficient.
  • 3. DR replica in a different zone/region from the primary A backup in the same datacenter as the server is not DR; it's a local backup. The DR environment must be in a different physical location that can survive the same disaster that affected the primary.
  • 4. DR runbook with numbered and verifiable steps The runbook must be executable by someone who did not design the system. Include: access to the DR environment, verification that backups/replica are available, failover steps, post-failover validation, and success criteria.
  • 5. Secrets and credentials available outside the primary site DR database passwords, backup encryption keys and API tokens needed for failover must be in a secrets manager accessible without depending on the primary site.
  • 6. Monitoring replication lag to DR The DR replica lag must be continuously monitored. A lag exceeding the RPO means you could not meet the objective if a disaster occurs at that moment.
  • 7. Automation of backup verification Verify daily that the last backup is complete, its size is reasonable and it can at least be listed or restored in a test environment.
  • 8. DNS failover or load balancer process documented If applications connect by hostname or VIP, the process to redirect that name or IP to the DR server must be documented and tested.
  • 9. DR activation criteria defined Who decides to activate DR? How much downtime is needed to escalate to DR? What is the approval process? Having these decisions made in advance avoids losing time during an emergency.
  • 10. Drill documented and executed at least once a year A real drill involves: cutting access to the primary, running the failover according to the runbook, measuring the actual time, validating that applications work in DR, and documenting findings.

Post-failover validation: what to verify before declaring DR operational

  • The DR database responds to queries from critical applications
  • Recent transactions are present and record counts are reasonable
  • Database jobs (ETL, reports, backups) are active in the new environment
  • External integrations (APIs, connectors) are pointing to the new endpoint
  • Monitoring is configured for the new primary (DR) — not still pointing at the failed server
  • The process to recover the primary site to prepare for failback has been initiated
TERMINAL · GENERIC EXAMPLE 
# Measure the REAL time of a restore (actual RTO, not an estimate)
time pg_restore -d dr_test_db backup.dump

When to escalate to a specialist DBA

  • The DR plan has never been tested and a real disaster is in progress
  • The DR replica lag exceeds the RPO before the disaster occurs
  • The runbook contains steps the team cannot execute without the original author
  • The company has no defined RTO and RPO and needs to design the DR strategy from scratch
  • A drill reveals that the actual recovery time significantly exceeds the approved RTO

Frequently asked questions

What is the difference between HA and DR?

HA (High Availability) prevents downtime due to individual node failures within the same datacenter. An automatic failover (Patroni, Always On, Data Guard) occurs in seconds and is transparent to the application. DR (Disaster Recovery) handles the complete loss of a datacenter or region. Recovery can take minutes or hours and generally requires manual intervention. They are complementary: you can have HA within a datacenter and DR to another datacenter, and you need both for a robust architecture.

How often should a DR drill be done?

At least once a year for critical systems; ideally every quarter for high-criticality systems. A drill must reproduce real conditions: cut access to the primary, execute the complete runbook, measure the actual RTO and RPO obtained, and document findings. A plan that exists only in an untested document is not a reliable DR plan.

Is a cloud backup sufficient as DR?

It depends on the business RTO and RPO. If the RTO is 4 hours and the RPO is 24 hours, daily backups in S3 or Azure Blob may be sufficient if you can restore to a cloud instance in that time. If the RTO is 15 minutes and the RPO is 1 hour, you need an active synchronized real-time replica to the cloud, not just backups. The decision must be based on how much downtime costs the business vs how much DR costs.

What critical information must be available outside the primary site?

The DR runbook with detailed steps, DR environment access credentials (database passwords, backup encryption keys, API tokens), the escalation contact list with direct phone numbers, the DR environment architecture documentation, and post-failover validation scripts. If any of these pieces is only on the failed server, DR becomes much slower and riskier.

Related resources