01 The big picture
Two fully isolated VPCs, one per region. us-east-1 is the active primary; ca-central-1 is the warm standby. Global services — Route 53, CloudFront, and the Step Functions recovery workflow — sit above both regions and steer traffic between them.
flowchart TB
user([End user / browser])
subgraph GLOBAL["Global services"]
r53["Route 53
DNS + failover records"]
cf["CloudFront
2 origin groups w/ auto-failover"]
end
user --> r53 --> cf
subgraph PRIMARY["Primary region — us-east-1 (ACTIVE)"]
direction TB
albP["ALB (primary)"]
ecsP["ECS Fargate
WordPress · 2 tasks"]
rdsP["RDS MySQL
writer"]
s3P["S3 media bucket
(primary)"]
albP --> ecsP
ecsP --> rdsP
ecsP --> s3P
end
subgraph DR["DR region — ca-central-1 (WARM STANDBY)"]
direction TB
albD["ALB (DR)"]
ecsD["ECS Fargate
WordPress · 0 tasks"]
rdsD["RDS MySQL
read replica"]
s3D["S3 media bucket
(DR)"]
albD --> ecsD
ecsD -.-> rdsD
ecsD --> s3D
end
cf -->|"app origin group
primary → DR"| albP
cf -.->|failover| albD
cf -->|"media origin group
/wp-content/uploads/*
primary → DR"| s3P
cf -.->|failover| s3D
rdsP ==>|"async cross-region
replication"| rdsD
s3P ==>|"cross-region
replication (CRR)"| s3D
classDef active fill:#12351f,stroke:#3fb950,color:#e6edf3;
classDef standby fill:#3a1d1d,stroke:#f85149,color:#e6edf3;
class albP,ecsP,rdsP,s3P active;
class albD,ecsD,rdsD,s3D standby;
02 Why admin traffic bypasses the CDN
A subtle but important design decision: public/frontend traffic goes through CloudFront, but the WordPress admin dashboard is routed directly to the ALB via Route 53. CloudFront origin groups do not support the POST requests that WordPress login and content writes require, so sending admin traffic through the CDN would break the dashboard.
flowchart LR
subgraph pub["Public traffic"]
d1["rqays.com
www.rqays.com"] --> cf1["CloudFront"] --> a1["ALB
(primary → DR failover)"]
end
subgraph adm["Admin traffic"]
d2["admin.rqays.com"] --> r53f["Route 53
failover routing"] --> a2["Primary ALB → DR ALB"]
end
classDef n fill:#1c2330,stroke:#ff9900,color:#e6edf3;
class d1,cf1,a1,d2,r53f,a2 n;
03 How a region failover actually runs
Read availability fails over automatically at the CloudFront origin-group layer (the origin groups
fail over on HTTP 403, 404, 500, 502, 503, 504, plus connection timeouts and origin
unreachability). But full recovery — promoting the database and
bringing compute online — is deliberately operator-triggered, then executed
automatically by a Step Functions workflow. This avoids flapping between regions on transient errors.
sequenceDiagram
actor Op as Operator
participant CF as CloudFront
participant SFN as Step Functions
participant RDS as RDS (DR replica)
participant ECS as ECS (DR service)
participant App as App health check
Note over CF: Primary origin returns 403/404/5xx or times out
CF->>CF: Auto-failover reads to DR ALB & DR S3
Note over Op: Operator confirms incident, starts workflow
Op->>SFN: Start DR failover execution
SFN->>RDS: ReplicaFailoverHandler — promote replica
SFN->>RDS: ValidateDBWritable — confirm writable
RDS-->>SFN: DB is now a writer
SFN->>ECS: ServiceRecoveryHandler — scale 0 → 2 tasks
ECS-->>SFN: Tasks healthy, registered to DR target group
SFN->>App: ValidateApplication — HTTP health check
App-->>SFN: 200 OK
SFN-->>Op: Failover complete (auditable execution log)
| Layer | Failover behavior | Trigger |
|---|---|---|
| Application reads (ALB) | Automatic via CloudFront origin group | HTTP 403/404/500/502/503/504, timeout, unreachable |
Media reads (S3, /wp-content/uploads/*) | Automatic via CloudFront S3 origin group | Same origin-failover codes |
| Database (RDS) | Operator-triggered, then automated | Step Functions execution |
| Compute (ECS) | Scaled 0 → 2 by the workflow | Step Functions execution |
04 How the Terraform is organized
The code is split into reusable modules and environment stacks (bootstrap, global, primary, dr, operations). Each region deploys independently, which prevents cross-region dependency cycles and keeps the two regions truly isolated. State is remote (S3) per stack.
flowchart TB boot["bootstrap
OIDC provider + GitHub Actions IAM role"] subgraph modules["Reusable modules"] m["vpc · sg · alb · ecs · rds · s3
acm · cdn_dns · endpoint · iam · lambda"] end subgraph global["environments/global"] g1["iam"]:::g g2["oac"]:::g g3["cdn_dns"]:::g end subgraph prim["environments/primary"] p1["network → rds → s3 → alb → ecs"]:::p p2["failover-alarms"]:::p end subgraph drenv["environments/dr"] d1["network → read_replica_rds → s3 → alb → ecs"]:::d end ops["operations/dr_orchestration
Step Functions + Lambda handlers"] boot --> global modules -.consumed by.-> global modules -.consumed by.-> prim modules -.consumed by.-> drenv prim --> ops drenv --> ops global --> ops classDef g fill:#22303f,stroke:#4493f8,color:#e6edf3; classDef p fill:#12351f,stroke:#3fb950,color:#e6edf3; classDef d fill:#3a1d1d,stroke:#f85149,color:#e6edf3;
05 Design decisions & tradeoffs
The most interview-relevant part of this project is that the tradeoffs are deliberate, not accidental.
Well-Architected mapping
| Pillar | Implementation |
|---|---|
| Reliability | Multi-region, CDN auto-failover, RDS replica, orchestrated recovery |
| Security | TLS everywhere, IAM roles (no keys), Secrets Manager, least-privilege SGs, private subnets |
| Performance | CloudFront CDN, S3 media offload, Fargate |
| Cost optimization | Warm standby, VPC endpoints to avoid NAT charges |
| Operational excellence | Full IaC, OIDC CI/CD, auditable Step Functions runbook |