Cloud engineering

SaaS restore drills: measure RPO and RTO separately

Test service usability and recoverable data against separate objectives. This planning analysis shows what to measure before a restore exercise can support a workload-level conclusion.

Synthetic baseline
51 minutes to usable service and an 8-minute recoverable data gap
Example targets
RTO 45 minutes and RPO 5 minutes, chosen only for this calculation
Evidence supplied
Executed arithmetic plus an unexecuted drill template, without operational measurements
A blue service model being assembled beside a stack of preserved data layers, with two separate amber clock-like rings.
Conceptual illustration of restoring usable service and preserving a recoverable data point.

A completed restore is only one timestamp

A restore drill passes its recovery objectives only when the required service is usable within the time target and its validated recoverable data falls within the allowed loss window. Finishing the database restore command answers a narrower question. Authentication, files, background work and traffic routing may still be unavailable or inconsistent.

In the synthetic planning example here, data restoration takes 15 minutes but the full service sequence takes 51 minutes. The recoverable point is eight minutes before the interruption. Against example targets of 45 minutes for RTO and five minutes for RPO, both tests fail. These are calculated scenarios, not measurements from an operational restore drill or a cloud benchmark.

The article supplies a calculator and an unexecuted drill record. Use the arithmetic to check a proposed timeline, then replace assumptions with evidence from an authorized exercise of the actual workload.

Define the two clocks before the exercise

Recovery time objective, or RTO, expresses the maximum acceptable interruption before the defined service returns. Recovery point objective, or RPO, expresses the tolerated age of recoverable data relative to the disruption. The objectives are targets. Observed recovery duration and the validated data gap are the outcomes compared with them. AWS's guidance on recovery objectives also calls for considering business impact and dependent workloads when choosing targets.

For this example, the interruption begins at 12:00, usable service returns at 12:51 and the latest validated consistent point is 11:52. Recovery duration is 51 minutes. The recoverable gap is eight minutes. All times belong to one illustrative timeline, with no time-zone conversion involved.

Write down what usable service means before starting. It might require a tenant administrator to sign in, open an existing record and create a new one with the expected permissions. A read-only landing page is not sufficient evidence for that acceptance condition. If degraded service has a separate objective, record it separately instead of silently changing the finish line.

Explicit clock boundaries in the synthetic planning example
MeasurementStart or referenceEnd or recovered pointExample result
Service recovery durationInterruption at 12:00Required journey works at 12:5151 minutes against a 45-minute target
Recoverable data gapInterruption at 12:00Consistent point at 11:528 minutes against a 5-minute target
Data restore phaseRestore begins at 12:18Restore command finishes at 12:3315 minutes, one phase of service recovery

Account for the whole recovery path

The baseline assumes six sequential phases: four minutes to detect, six to decide, eight to provision, fifteen to restore data, twelve to validate and six to cut over. Their sum is 51 minutes. The durations are editable assumptions, not recommended targets for those activities.

Six sequential synthetic phases total 51 minutes: detect 4, decide 6, provision 8, restore data 15, validate 12 and cut over 6.
Figure 1. Assumed sequential recovery path. The data restore occupies 15 of the modeled 51 minutes. View full-size figure.

Real tasks can overlap. If provisioning and access preparation run concurrently, sum the sequential intervals on the critical path rather than adding every person's task duration. Keep raw start and finish timestamps so another reviewer can reconstruct that path. Include delays caused by missing access, unavailable configuration or an approval the runbook actually requires.

A backup schedule does not establish the recoverable point by itself. Validate which committed data and related objects are present. For a database plus object storage, a recent database snapshot may reference a file version that was never copied. Report the latest point at which the required business state is consistent, along with any known missing records or uncertain interval. The eight-minute gap here is a time window, not a claim that every write in that window was lost.

Change one assumption at a time

Download the planning calculator, save it as calculate.py and run it with Python 3. It performs arithmetic only and writes JSON calculations and a scenario CSV. It creates no infrastructure and executes no restore.

The sensitivity check changes data restoration from 15 to eight minutes, the recoverable gap from eight to three minutes, or both. All other phase durations stay fixed. Treat these as independent planning changes for this exercise; a real architecture may couple restore speed, replication freshness and validation effort.

Independent changes to restore duration and recoverable point, with other phases fixed
Synthetic caseTotal recovery durationRecoverable gapTime target met?Point target met?
Baseline51 minutes8 minutesNoNo
Faster restore only44 minutes8 minutesYesNo
Fresher recoverable point only51 minutes3 minutesNoYes
Both changes44 minutes3 minutesYesYes
Four calculated cases show that faster restoration can meet the 45-minute time target while missing the 5-minute point target, and fresher data can do the reverse.
Figure 2. Separate pass conditions. Dashed lines mark example targets, with lower values meeting each objective. View full-size figure.

A seven-minute reduction in one phase gets the modeled duration below 45 minutes. It does not improve the data point. Conversely, fresher data does not shorten this modeled recovery path. There is no probability distribution or repeated sample behind the table, so it cannot establish a percentile, reliability rate or production guarantee.

Collect evidence at the application boundary

The drill record template has empty timestamps and evidence fields. Its checks are marked not executed. Record the failure scenario, workload scope, owners and required customer journey, then capture the recoverable point and each phase with evidence references. Keep source logs and configuration versions in an access-controlled operational record rather than placing secrets in the template.

The template asks for database consistency and object-reference checks, access to configuration and keys, tenant authentication and authorization, queued-work reconciliation, customer read/write behavior, recovery-site capacity and isolation of the original writer. These are proposed checks to adapt to the workload. A green database health check does not demonstrate that a restored tenant has the correct permissions or that a worker will avoid repeating an external effect.

Use the tenant placement cutover checklist when routing and write ownership are part of the scenario. For jobs, review fencing-token limits before assuming that an expired worker lease prevents old work from writing. AWS's disaster recovery testing guidance emphasizes exercising the recovery path and documenting runbook problems. Record what the exercise actually covered and what remained simulated.

Turn a failed objective into a scoped change

Assign the missed objective to the phase or dependency that explains it. An unavailable identity path needs a different change from a slow data transfer. A missing object version needs a different change from delayed traffic cutover. Give each follow-up an owner, acceptance evidence and a repeatable scenario that could show whether it worked.

When those gaps require changes to the recovery environment, connect the drill to the cloud architecture. Pharos Production's cloud architecture and disaster recovery services cover cloud migration, multi-region deployments and disaster recovery, with a recovery drill included in the described production-readiness process. Use the measured gaps to scope infrastructure, access, data consistency and cutover work, and ask which recovery evidence the engagement will deliver.

Keep the two outcomes visible in the final drill record. Record the service recovery duration beside its RTO target and the validated data gap beside its RPO target. A single successful phase should not erase a failed workload objective.

Sources

Documentation checked .

  1. AWS: define recovery objectives
  2. AWS: test disaster recovery