Introduction
Every digital transformation project starts with the same question from the solution architect:
"We'll have 1 frontend, 20 backend microservices, and 2 databases. What infrastructure do we need?"
And every DevOps architect knows this question is deceptively simple. The answer isn't just "pick some EC2 instances." It's a multi-dimensional decision involving:
- 6 environments with fundamentally different purposes (Dev, QA, UAT, Staging, Pre-Prod, Prod)
- Cost constraints that demand right-sizing without over-provisioning
- Security boundaries that get stricter as you move toward production
- Team dynamics — 50 developers need parallel workstreams without stepping on each other
- Compliance requirements (SOC2, GDPR) that dictate data handling and access controls
- Disaster recovery expectations that vary by environment criticality
This guide is the reference document you hand to your solution architect, your CTO, and your finance team.
---
The Scenario
Throughout this guide, we'll use a concrete project:
| Dimension | Details |
|---|---|
| <strong>Frontend</strong> | 1 React SPA served via CDN |
| <strong>Backend</strong> | 20 microservices (Java Spring Boot + Node.js mix) |
| <strong>Databases</strong> | 1 PostgreSQL (relational) + 1 MongoDB (document store) |
| <strong>Team size</strong> | 50 developers across 8 squads |
| <strong>Traffic (Prod)</strong> | 50,000 concurrent users, 500 req/sec sustained, 2,000 req/sec peak |
| <strong>Data volume</strong> | 500GB relational + 200GB document store (Year 1) |
| <strong>Compliance</strong> | SOC2 Type II + GDPR |
| <strong>Availability target</strong> | 99.95% (Prod), 99.5% (Pre-Prod/Staging), best-effort (Dev/QA) |
---
The DevOps Architect's Decision Framework
Before jumping into instance sizes, a DevOps architect evaluates these dimensions for each environment:
1. Purpose and Usage Pattern
Each environment exists for a specific reason. Understanding its purpose dictates everything else:
| Environment | Purpose | Who Uses It | Usage Pattern |
|---|---|---|---|
| <strong>Dev</strong> | Active development, feature building, debugging | Developers (50 people) | 8-12 hours/day, weekdays. Bursty, unpredictable |
| <strong>QA</strong> | Automated testing, regression suites, exploratory testing | QA engineers (10 people) + CI pipelines | Pipeline-triggered, batch workloads |
| <strong>UAT</strong> | Business validation, stakeholder demos, acceptance testing | Business analysts, product owners (15 people) | Scheduled sessions, low continuous load |
| <strong>Staging</strong> | Pre-production validation, performance testing, integration testing | DevOps team, senior engineers (10 people) | Mirrors Prod traffic patterns at lower scale |
| <strong>Pre-Prod</strong> | Final validation, data migration rehearsals, disaster recovery drills | Entire engineering + ops team | Identical to Prod configuration, intermittent use |
| <strong>Production</strong> | Live customer-facing traffic | End users (50,000 concurrent) | 24/7, auto-scaling, high availability |
2. The Sizing Multiplier Principle
A practical framework that DevOps architects use:
Dev = Prod × 0.15 (minimum viable for development)
QA = Prod × 0.25 (enough to run test suites meaningfully)
UAT = Prod × 0.20 (business users don't generate high load)
Staging = Prod × 0.50 (must simulate production patterns)
Pre-Prod = Prod × 1.0 (identical to Prod, runs intermittently)
Prod = Baseline (fully sized for peak + 30% headroom)
This isn't a rigid formula — it's a starting point that gets refined based on actual usage data after the first 3 months.
3. Key Decision Inputs
The DevOps architect collects these inputs before sizing:
From the Solution Architect:
- Number of services and their communication patterns (sync vs async)
- Expected request/response sizes (affects memory and network)
- Stateful vs stateless services (affects storage and scaling approach)
- Background job requirements (batch processing, scheduled tasks)
From the Product Manager:
- Expected user growth curve (Year 1 → Year 3 projections)
- Peak traffic scenarios (Black Friday, campaign launches, seasonal patterns)
- Geographic distribution of users (single region vs multi-region)
- Data retention requirements (compliance-driven)
From the Development Team:
- Technology stack specifics (JVM memory requirements, Node.js event loop constraints)
- Build and test pipeline requirements (CI/CD compute needs)
- Local development patterns (do they need personal namespaces?)
- Third-party integrations (external API dependencies, message queues)
From Finance:
- Total infrastructure budget envelope
- CapEx vs OpEx preference
- Reserved instance commitment appetite (1-year vs 3-year)
---
Container Orchestration: EKS vs ECS Fargate Decision
Before sizing individual environments, the DevOps architect makes a fundamental platform decision.
Decision Matrix
| Factor | EKS (Kubernetes) | ECS Fargate (Serverless Containers) |
|---|---|---|
| <strong>Operational overhead</strong> | High — manage node groups, upgrades, add-ons | Low — AWS manages compute layer |
| <strong>Cost at scale (20 services)</strong> | Lower at sustained load (reserved nodes) | Higher at sustained load (per-vCPU/GB pricing) |
| <strong>Developer experience</strong> | Steeper learning curve, powerful debugging | Simpler deployment model, limited debugging |
| <strong>Portability</strong> | Multi-cloud (K8s is standard) | AWS-locked |
| <strong>Service mesh</strong> | Istio, Linkerd (mature) | App Mesh (limited) |
| <strong>Team skill requirement</strong> | CKA-level Kubernetes expertise needed | AWS ECS/Fargate familiarity sufficient |
| <strong>Best for Dev/QA</strong> | Overkill unless team already uses K8s | Ideal — no cluster management overhead |
| <strong>Best for Production</strong> | Superior for 20+ services at scale | Good for fewer than 10 services or variable traffic |
Our Recommendation for This Scenario
For a 20-microservice digital transformation project with 50 developers:
- Dev and QA: ECS Fargate — developers focus on code, not cluster management. Fast iteration, no node scaling concerns.
- UAT through Production: EKS — the operational investment pays off at 20 services. Service mesh, advanced scheduling, namespace isolation per team, and cost optimization via reserved nodes.
Azure equivalent: AKS (Azure Kubernetes Service) for all environments. AKS has no control plane cost, making it viable even for Dev.
---
Environment-by-Environment Configuration
Environment 1: Development (Dev)
Philosophy: Fast feedback loops. Developers must deploy and test independently without waiting for shared resources. Cost-optimized, not performance-optimized.
Compute Configuration
| Component | AWS (ECS Fargate) | Azure (AKS) |
|---|---|---|
| <strong>Orchestration</strong> | ECS Fargate cluster | AKS cluster (B-series burstable nodes) |
| <strong>Frontend</strong> | 1 task: 0.25 vCPU, 512MB | 1 pod: 250m CPU, 512Mi memory |
| <strong>Each microservice</strong> | 1 task: 0.5 vCPU, 1GB | 1 pod: 500m CPU, 1Gi memory |
| <strong>Total compute</strong> | 20 services x 0.5 vCPU = 10 vCPU, 20GB RAM | 3 nodes x Standard_B4ms (4 vCPU, 16GB) |
| <strong>Scaling</strong> | Fixed — no auto-scaling in Dev | Fixed — manual scaling only |
| <strong>Availability</strong> | Single-AZ | Single-AZ |
| <strong>Uptime target</strong> | Best effort (no SLA) | Best effort (no SLA) |
Why These Specs for Dev
- 0.5 vCPU per service: Enough to start the JVM/Node process and handle a single developer's test requests. Not enough for concurrent users — that's intentional.
- Single-AZ: No reason to pay for cross-AZ redundancy when downtime does not affect customers.
- No auto-scaling: Developers do not generate traffic spikes. Fixed allocation is predictable and cheaper.
- Burstable instances (Azure B-series): Dev workloads are idle 80% of the time. Burstable credits give burst capacity when a developer actively tests.
Database Configuration (Dev)
| Component | AWS | Azure |
|---|---|---|
| <strong>PostgreSQL</strong> | RDS db.t4g.medium (2 vCPU, 4GB), 50GB gp3 | Azure DB for PostgreSQL Flexible, Burstable B2ms (2 vCPU, 8GB), 64GB |
| <strong>MongoDB</strong> | DocumentDB t3.medium (2 vCPU, 4GB) | Cosmos DB (MongoDB API), 400 RU/s provisioned |
| <strong>High Availability</strong> | Single-AZ, no read replicas | Single-AZ, no replicas |
| <strong>Backups</strong> | Automated daily, 7-day retention | Automated daily, 7-day retention |
| <strong>Data</strong> | Synthetic/anonymized — NEVER production data | Same |
Networking (Dev)
| Component | AWS | Azure |
|---|---|---|
| <strong>VPC/VNet</strong> | Dedicated VPC (10.0.0.0/16) | Dedicated VNet (10.0.0.0/16) |
| <strong>Subnets</strong> | 2 private + 1 public (single AZ) | 2 subnets (1 for nodes, 1 for DB) |
| <strong>NAT</strong> | NAT Instance (t3.micro) — NOT NAT Gateway ($32/mo savings) | Azure NAT Gateway |
| <strong>Load Balancer</strong> | ALB (shared across all services) | NGINX Ingress Controller on AKS |
| <strong>DNS</strong> | Route53 private zone: *.dev.internal | Azure Private DNS zone |
| <strong>VPN/Access</strong> | AWS Client VPN or Tailscale | Azure VPN Gateway or Tailscale |
Dev Environment Special Considerations
---
Environment 2: QA (Quality Assurance)
Philosophy: Reliability for automated pipelines. QA runs 500+ automated tests across 20 services simultaneously. The environment must handle parallel test execution without flakiness caused by resource starvation.
Compute Configuration
| Component | AWS (ECS Fargate) | Azure (AKS) |
|---|---|---|
| <strong>Orchestration</strong> | ECS Fargate cluster | AKS cluster (D-series compute-optimized) |
| <strong>Frontend</strong> | 1 task: 0.5 vCPU, 1GB | 1 pod: 500m CPU, 1Gi memory |
| <strong>Each microservice</strong> | 1 task: 1 vCPU, 2GB | 1 pod: 1000m CPU, 2Gi memory |
| <strong>Total compute</strong> | 20 services x 1 vCPU = 20 vCPU, 40GB RAM | 4 nodes x Standard_D4s_v5 (4 vCPU, 16GB) |
| <strong>Test runners</strong> | 4 Fargate tasks: 2 vCPU, 4GB each (parallel test execution) | 2 dedicated test-runner pods: 2 vCPU, 4Gi each |
| <strong>Scaling</strong> | Fixed during test runs, scale-to-zero between | Fixed node count, pods scale to zero between runs |
| <strong>Availability</strong> | Single-AZ | Single-AZ |
Why QA Needs More Than Dev
- 1 vCPU per service (2x Dev): Automated tests hit all 20 services simultaneously. Each service handles 10-50 concurrent test requests — enough to expose race conditions and concurrency bugs that 0.5 vCPU would mask.
- Dedicated test runner compute: Integration test suites (Selenium, Playwright, API tests) need their own compute so they do not compete with the services they are testing.
- Scale-to-zero between runs: QA is batch-oriented. Tests run for 30-45 minutes, then nothing for hours. Pay only for active test windows.
Database Configuration (QA)
| Component | AWS | Azure |
|---|---|---|
| <strong>PostgreSQL</strong> | RDS db.t4g.large (2 vCPU, 8GB), 100GB gp3 | Azure DB for PostgreSQL Flexible, GP D2ds_v4 (2 vCPU, 8GB), 128GB |
| <strong>MongoDB</strong> | DocumentDB t3.medium (2 vCPU, 4GB) | Cosmos DB (MongoDB API), 800 RU/s |
| <strong>Special requirement</strong> | Database reset automation between test suites | Same — clean state for every test run |
| <strong>Data</strong> | Curated test datasets (covers edge cases) | Same |
QA Environment Special Considerations
---
Environment 3: UAT (User Acceptance Testing)
Philosophy: Stability and predictability. Business stakeholders perform manual testing and demos here. It must feel production-like without production costs. No surprise outages during a demo to the VP.
Compute Configuration
| Component | AWS (EKS) | Azure (AKS) |
|---|---|---|
| <strong>Orchestration</strong> | EKS cluster (starts here for production parity) | AKS cluster (D-series standard) |
| <strong>Node group</strong> | 3 x t3.xlarge (4 vCPU, 16GB) | 3 x Standard_D4s_v5 (4 vCPU, 16GB) |
| <strong>Frontend</strong> | 2 replicas: 0.5 vCPU, 1GB each | 2 replicas: 500m CPU, 1Gi each |
| <strong>Each microservice</strong> | 1 replica: 0.5 vCPU, 1GB | 1 replica: 500m CPU, 1Gi |
| <strong>Total compute</strong> | 12 vCPU, 48GB across 3 nodes | Same |
| <strong>Scaling</strong> | HPA enabled (min 1, max 2 per service) | Same |
| <strong>Availability</strong> | Multi-AZ (2 AZs) | 2 availability zones |
Why UAT Gets Multi-AZ but Lower Compute
- Multi-AZ (2 zones): A UAT crash during a stakeholder demo destroys confidence. Multi-AZ prevents single-point failures from ruining a business-critical review session.
- Lower compute per service (0.5 vCPU): Business users generate minimal concurrent load (15 people clicking through flows). The compute is for stability, not throughput.
- EKS starts here: UAT is the first environment where we mirror production orchestration. This catches Kubernetes-specific issues before they reach production.
Database Configuration (UAT)
| Component | AWS | Azure |
|---|---|---|
| <strong>PostgreSQL</strong> | RDS db.t4g.large (2 vCPU, 8GB), 100GB gp3, Multi-AZ standby | Azure DB for PostgreSQL Flexible, GP D2ds_v4, Zone Redundant HA |
| <strong>MongoDB</strong> | DocumentDB r5.large (2 vCPU, 16GB), 2-node cluster | Cosmos DB (MongoDB API), 1000 RU/s |
| <strong>High Availability</strong> | Multi-AZ enabled | Zone redundant |
| <strong>Data</strong> | Production-like volume with anonymized/masked PII | Same |
UAT Environment Special Considerations
---
Environment 4: Staging
Philosophy: Production mirror at reduced scale. Performance testing, integration testing with external systems, and deployment rehearsals happen here. This is the last stop where engineers have full debugging access.
Compute Configuration
| Component | AWS (EKS) | Azure (AKS) |
|---|---|---|
| <strong>Orchestration</strong> | EKS cluster (same version as Prod) | AKS cluster (same version as Prod) |
| <strong>Node group</strong> | 4 x c5.xlarge (4 vCPU, 8GB) — compute-optimized | 4 x Standard_D4s_v5 (4 vCPU, 16GB) |
| <strong>Frontend</strong> | 2 replicas: 1 vCPU, 2GB each | 2 replicas: 1000m CPU, 2Gi each |
| <strong>Each microservice</strong> | 2 replicas: 1 vCPU, 2GB each | 2 replicas: 1000m CPU, 2Gi each |
| <strong>Total compute</strong> | 16 vCPU, 32GB across 4 nodes | 16 vCPU, 64GB across 4 nodes |
| <strong>Scaling</strong> | HPA enabled (min 2, max 4 per service) | Same |
| <strong>Availability</strong> | Multi-AZ (3 AZs — same as Prod) | 3 availability zones |
Why Staging Gets Production-Level Topology
- 3 AZs (same as Prod): Staging must validate that your pod anti-affinity rules, cross-AZ traffic patterns, and zone-failure handling work correctly. If Staging is 2 AZ and Prod is 3 AZ, you will miss topology-specific bugs.
- 2 replicas per service (Prod minimum): Tests Kubernetes rolling updates, pod disruption budgets, and load balancing behavior. Single-replica services mask deployment failures.
- Compute-optimized nodes (c5): Staging runs performance tests. CPU-bound workloads need consistent compute — burstable instances give misleading benchmarks.
- 50% of Prod scale: Enough to catch performance regressions without paying for full production capacity.
Database Configuration (Staging)
| Component | AWS | Azure |
|---|---|---|
| <strong>PostgreSQL</strong> | RDS db.r6g.large (2 vCPU, 16GB), 200GB gp3, Multi-AZ | Azure DB for PostgreSQL Flexible, MO E2ds_v4 (2 vCPU, 16GB), Zone Redundant |
| <strong>MongoDB</strong> | DocumentDB r5.large (2 vCPU, 16GB), 3-node cluster | Cosmos DB, 2000 RU/s, Strong consistency |
| <strong>Read replicas</strong> | 1 read replica (validates read/write splitting logic) | 1 read replica |
| <strong>Data volume</strong> | 50% of production data volume (anonymized) | Same |
| <strong>Connection pooling</strong> | PgBouncer sidecar (validates connection pool behavior) | Same |
Staging Environment Special Considerations
---
Environment 5: Pre-Production (Pre-Prod)
Philosophy: Production-identical. Exists solely to validate that the exact production configuration works with the latest code. Runs intermittently (not 24/7) to control costs.
Compute Configuration
| Component | AWS (EKS) | Azure (AKS) |
|---|---|---|
| <strong>Orchestration</strong> | EKS cluster (IDENTICAL version, add-ons, config to Prod) | AKS cluster (identical to Prod) |
| <strong>Node group</strong> | 6 x c5.2xlarge (8 vCPU, 16GB) — same instance type as Prod | 6 x Standard_D8s_v5 (8 vCPU, 32GB) |
| <strong>Frontend</strong> | 3 replicas: 1 vCPU, 2GB each (same as Prod) | Same as Prod |
| <strong>Each microservice</strong> | 2-3 replicas: 1-2 vCPU, 2-4GB each (same as Prod) | Same as Prod |
| <strong>Total compute</strong> | 48 vCPU, 96GB across 6 nodes | Same |
| <strong>Scaling</strong> | HPA with same thresholds as Prod | Same |
| <strong>Availability</strong> | Multi-AZ (3 AZs) — identical to Prod | 3 availability zones |
Why Pre-Prod Mirrors Production Exactly
- Configuration validation: The number one cause of "works in Staging, fails in Prod" is configuration drift. Pre-Prod eliminates this by being a 1:1 clone.
- Data migration rehearsal: Execute migrations on Pre-Prod with production-equivalent data volumes. Measure exact timing, validate rollback procedures.
- Disaster recovery drills: Practice full environment recovery from backups. Measure RTO and validate it meets your SLA.
- Security audit target: Penetration testing runs against Pre-Prod to avoid impacting real users.
Database Configuration (Pre-Prod)
| Component | AWS | Azure |
|---|---|---|
| <strong>PostgreSQL</strong> | RDS db.r6g.xlarge (4 vCPU, 32GB), 500GB gp3, Multi-AZ, 2 read replicas | Azure DB for PostgreSQL Flexible, MO E4ds_v4, Zone Redundant, 2 read replicas |
| <strong>MongoDB</strong> | DocumentDB r5.xlarge (4 vCPU, 32GB), 3-node cluster | Cosmos DB, 4000 RU/s, Strong consistency |
| <strong>Data volume</strong> | 100% of production data volume (anonymized) | Same |
| <strong>Backup testing</strong> | Weekly restore-from-backup validation | Same |
Pre-Prod Environment Special Considerations
---
Environment 6: Production (Prod)
Philosophy: Reliability, performance, and security above all else. Every decision optimizes for uptime, response time, and data integrity. Cost is important but secondary to customer experience.
Compute Configuration
| Component | AWS (EKS) | Azure (AKS) |
|---|---|---|
| <strong>Orchestration</strong> | EKS cluster, managed node groups, Kubernetes 1.29 | AKS cluster, system + user node pools |
| <strong>System node group</strong> | 3 x t3.large (2 vCPU, 8GB) — cluster system pods | System pool: 3 x Standard_D2s_v5 (2 vCPU, 8GB) |
| <strong>Application node group</strong> | 6 x c5.2xlarge (8 vCPU, 16GB) — microservices | User pool: 6 x Standard_D8s_v5 (8 vCPU, 32GB) |
| <strong>Frontend</strong> | 3 replicas: 1 vCPU, 2GB each, spread across 3 AZs | Same |
| <strong>Each microservice</strong> | 2-4 replicas: 1-2 vCPU, 2-4GB each (varies by service) | Same |
| <strong>Total compute</strong> | 48 vCPU, 96GB (app) + 6 vCPU, 24GB (system) | Same |
| <strong>Scaling</strong> | HPA (CPU 70%, custom metrics), Cluster Autoscaler (max 12 nodes) | HPA + Cluster Autoscaler |
| <strong>Availability</strong> | Multi-AZ (3 AZs), pod anti-affinity, PDB (minAvailable: 50%) | 3 zones, same policies |
Per-Service Sizing Strategy (Production)
Not all 20 microservices need the same resources. Categorize them:
| Service Category | Examples | CPU | Memory | Replicas |
|---|---|---|---|---|
| <strong>API Gateway / BFF</strong> | api-gateway, bff-web | 2 vCPU | 4GB | 3-4 |
| <strong>Core business</strong> | order-service, payment-service, user-service | 1 vCPU | 2GB | 3 |
| <strong>Data-intensive</strong> | analytics-service, reporting-service | 2 vCPU | 4GB | 2 |
| <strong>Background workers</strong> | notification-service, queue-processor | 0.5 vCPU | 1GB | 2-4 |
| <strong>Lightweight utilities</strong> | config-service, feature-flags | 0.25 vCPU | 512MB | 2 |
Database Configuration (Production)
| Component | AWS | Azure |
|---|---|---|
| <strong>PostgreSQL</strong> | RDS db.r6g.2xlarge (8 vCPU, 64GB), 500GB gp3 (3000 IOPS), Multi-AZ | Azure DB for PostgreSQL Flexible, MO E8ds_v4 (8 vCPU, 64GB), Zone Redundant |
| <strong>Read replicas</strong> | 2 read replicas (db.r6g.xlarge) | 2 read replicas |
| <strong>MongoDB</strong> | DocumentDB r5.2xlarge (8 vCPU, 64GB), 3-node cluster | Cosmos DB, 10000 RU/s autoscale (max 40000) |
| <strong>Connection pooling</strong> | PgBouncer (transaction mode, max 200 connections) | Built-in PgBouncer mode |
| <strong>Backups</strong> | Snapshots every 1 hour, 35-day retention, cross-region copy | 35-day retention, geo-redundant storage |
| <strong>Encryption</strong> | AES-256 at rest (KMS CMK), TLS 1.3 in transit | AES-256 (Customer-managed key), TLS 1.3 |
Production Networking
| Component | AWS | Azure |
|---|---|---|
| <strong>VPC/VNet</strong> | Dedicated VPC (10.100.0.0/16) | Dedicated VNet (10.100.0.0/16) |
| <strong>Subnets</strong> | 3 public + 3 private + 3 database (per AZ) | 3 subnets per tier per AZ |
| <strong>Load Balancer</strong> | ALB + NLB (for gRPC/TCP) | Application Gateway (WAF v2) + internal LB |
| <strong>WAF</strong> | AWS WAF (OWASP rules, rate limiting, geo-blocking) | Azure WAF on Application Gateway |
| <strong>CDN</strong> | CloudFront (frontend SPA + API caching) | Azure Front Door (global LB + CDN + WAF) |
| <strong>Service mesh</strong> | Istio (mTLS, traffic management) | Istio on AKS |
| <strong>Private connectivity</strong> | VPC endpoints for S3, SQS, ECR | Private endpoints for Azure services |
Production Security Controls
| Domain | Implementation |
|---|---|
| <strong>Network</strong> | No public IPs on pods. Traffic through ALB/AppGW only. Security groups restrict east-west to declared dependencies. |
| <strong>Identity</strong> | IRSA (AWS) / Workload Identity (Azure). No static credentials. Per-service IAM roles. |
| <strong>Secrets</strong> | Secrets Manager / Key Vault with 90-day rotation. External Secrets Operator for K8s sync. |
| <strong>Containers</strong> | Read-only root FS, non-root, no privileged. Image scanning on push. OPA Gatekeeper admission control. |
| <strong>Data</strong> | Encryption everywhere. PII envelope-encrypted at app layer. GDPR erasure pipeline. |
| <strong>Audit</strong> | CloudTrail/Activity Log. K8s audit logs to SIEM. All access logged. |
Production Observability
| Layer | Tools | Key Metrics |
|---|---|---|
| <strong>Infrastructure</strong> | CloudWatch/Azure Monitor + Prometheus | Node CPU/memory, pod restarts, OOMKills |
| <strong>Application</strong> | Prometheus + Grafana + Jaeger | RED metrics (Rate, Error, Duration). p50/p95/p99 per service |
| <strong>Business</strong> | Custom metrics | Orders/min, conversion rate, active sessions |
| <strong>Alerting</strong> | PagerDuty/OpsGenie | P1: page on-call. P2: Slack. P3: ticket |
| <strong>SLO</strong> | Grafana SLO dashboards | 99.95% = 21.6 min/month error budget. Alert at 50% consumed |
---
CI/CD Infrastructure (Shared Across Environments)
The CI/CD pipeline itself needs dedicated compute — separate from application environments.
Pipeline Infrastructure
| Component | AWS | Azure |
|---|---|---|
| <strong>Source control</strong> | GitHub Enterprise | Azure DevOps or GitHub |
| <strong>CI runners</strong> | GitHub Actions (self-hosted on EKS, 8 x c5.xlarge spot) | Azure DevOps agents (VMSS-based, Standard_D4s_v5, 8 agents) |
| <strong>Container registry</strong> | ECR (one repo per service, lifecycle policies) | ACR Premium (geo-replication, content trust) |
| <strong>Artifact storage</strong> | S3 (Helm charts, test reports, build artifacts) | Azure Blob Storage |
| <strong>Security scanning</strong> | Trivy (container), Semgrep (SAST), OWASP ZAP (DAST) | Same tools or Microsoft Defender for Cloud |
CI/CD Sizing for 50 Developers
| Metric | Requirement |
|---|---|
| <strong>Concurrent builds</strong> | 8-12 (one per squad, peak hours) |
| <strong>Build time target</strong> | Under 10 minutes (unit test + build + push) |
| <strong>Full pipeline (to Staging)</strong> | Under 20 minutes |
| <strong>Deployment frequency</strong> | 5-10/day to Dev, 2-3/day to QA, 1/day to Staging |
| <strong>Runner scaling</strong> | 2 always-on + scale to 12 during peak hours |
---
Cost Estimation: The Complete Picture
Monthly Cost Summary (AWS)
| Environment | Compute | Database | Networking | Other | Monthly Total |
|---|---|---|---|---|---|
| <strong>Dev</strong> | $450 | $280 | $80 | $50 | <strong>$860</strong> |
| <strong>QA</strong> | $350 | $320 | $80 | $50 | <strong>$800</strong> |
| <strong>UAT</strong> | $650 | $450 | $150 | $80 | <strong>$1,330</strong> |
| <strong>Staging</strong> | $1,100 | $750 | $200 | $120 | <strong>$2,170</strong> |
| <strong>Pre-Prod</strong> | $800 | $600 | $150 | $100 | <strong>$1,650</strong> |
| <strong>Production</strong> | $3,200 | $2,800 | $600 | $400 | <strong>$7,000</strong> |
| <strong>CI/CD</strong> | $500 | — | $50 | $100 | <strong>$650</strong> |
| <strong>TOTAL</strong> | <strong>$14,460/month</strong> |
Monthly Cost Summary (Azure)
| Environment | Compute | Database | Networking | Other | Monthly Total |
|---|---|---|---|---|---|
| <strong>Dev</strong> | $380 | $250 | $60 | $40 | <strong>$730</strong> |
| <strong>QA</strong> | $420 | $350 | $60 | $40 | <strong>$870</strong> |
| <strong>UAT</strong> | $580 | $500 | $120 | $70 | <strong>$1,270</strong> |
| <strong>Staging</strong> | $950 | $800 | $180 | $100 | <strong>$2,030</strong> |
| <strong>Pre-Prod</strong> | $700 | $550 | $130 | $80 | <strong>$1,460</strong> |
| <strong>Production</strong> | $2,800 | $2,500 | $500 | $350 | <strong>$6,150</strong> |
| <strong>CI/CD</strong> | $450 | — | $40 | $80 | <strong>$570</strong> |
| <strong>TOTAL</strong> | <strong>$13,080/month</strong> |
Cost Optimization Strategies
| Strategy | Savings | Applies To |
|---|---|---|
| <strong>Scheduled shutdown (Dev, QA)</strong> | 55-65% on compute | Dev (12h/day), QA (scale-to-zero) |
| <strong>Reserved Instances / Savings Plans</strong> | 30-40% on compute | Prod, Staging (always-on) |
| <strong>Spot instances (CI runners)</strong> | 60-70% on CI compute | CI/CD runners |
| <strong>Pre-Prod on-demand scheduling</strong> | 70-80% vs always-on | Pre-Prod (runs ~8 days/month) |
| <strong>Right-sizing after 3 months</strong> | 15-25% across all | Based on actual usage data |
Year-1 Total Cost Projection
| Scenario | AWS (Annual) | Azure (Annual) |
|---|---|---|
| <strong>On-demand (no optimization)</strong> | $210,000 | $190,000 |
| <strong>With reserved + scheduling</strong> | $145,000 | $130,000 |
| <strong>Fully optimized (post 3-month tuning)</strong> | $125,000 | $112,000 |
---
Network Architecture Across Environments
Account/Subscription Strategy
| Approach | AWS | Azure |
|---|---|---|
| <strong>Recommended</strong> | Separate AWS accounts per environment (Organizations) | Separate subscriptions per environment (Management Group) |
| <strong>Why</strong> | Blast radius containment. Billing isolation. Service limit isolation. | Same. Resource limit isolation. Cost attribution. RBAC boundaries. |
| <strong>VPC CIDR plan</strong> | Dev: 10.0.0.0/16, QA: 10.1.0.0/16, UAT: 10.2.0.0/16, Staging: 10.3.0.0/16, Pre-Prod: 10.4.0.0/16, Prod: 10.100.0.0/16 | Same CIDR scheme in VNets |
| <strong>Cross-env connectivity</strong> | Transit Gateway (hub-and-spoke) | Azure Virtual WAN or Hub-Spoke peering |
DNS Strategy
Production: api.yourapp.com → Prod ALB/AppGW
Pre-Prod: api.preprod.yourapp.com → Pre-Prod ALB
Staging: api.staging.yourapp.com → Staging ALB
UAT: api.uat.yourapp.com → UAT ALB
QA: api.qa.internal → Private DNS (no public access)
Dev: api.dev.internal → Private DNS (no public access)
Dev and QA should NOT be publicly accessible. Access via VPN only. This reduces attack surface and prevents accidental external traffic.
---
Security Posture by Environment
Security scales with environment criticality:
| Control | Dev | QA | UAT | Staging | Pre-Prod | Prod |
|---|---|---|---|---|---|---|
| <strong>Network</strong> | VPN-only | VPN-only | VPN + allowlist | VPN + allowlist | Same as Prod | WAF + geo-block + rate limit |
| <strong>IAM</strong> | Broad developer access | CI roles only | Read-only for business | Time-boxed engineer access | JIT access only | Break-glass only, audited |
| <strong>Data</strong> | Synthetic | Synthetic | Anonymized prod-like | Anonymized full volume | Full volume anonymized | Real customer data (encrypted) |
| <strong>Secrets</strong> | Env vars (acceptable) | Parameter Store | Secrets Manager | Same as Prod | Same as Prod | Auto-rotation + audit |
| <strong>Scanning</strong> | Warn only | Warn only | Block CRITICAL | Block HIGH+ | Block HIGH+ | Block HIGH+ plus runtime (Falco) |
| <strong>Pen testing</strong> | None | None | Annual | Quarterly DAST | Before release | Continuous (bug bounty) |
| <strong>Backup/DR</strong> | Not required | Not required | Daily | Daily + tested | Same as Prod | Hourly + cross-region + tested |
---
Disaster Recovery by Environment
| Environment | RPO | RTO | Strategy |
|---|---|---|---|
| <strong>Dev</strong> | 24 hours | 4-8 hours | Recreate from IaC |
| <strong>QA</strong> | 24 hours | 2-4 hours | Recreate from IaC |
| <strong>UAT</strong> | 12 hours | 2-4 hours | Restore from daily backup |
| <strong>Staging</strong> | 4 hours | 1-2 hours | Backup + auto-recovery |
| <strong>Pre-Prod</strong> | 1 hour | 30 minutes | Same as Prod (validates DR) |
| <strong>Production</strong> | 5 minutes | 15 minutes | Multi-AZ failover + cross-region backup |
---
Environment Promotion Pipeline
Developer branch → Dev (auto-deploy on push)
↓
PR merge → QA (auto-deploy, test suites run)
↓
Tests pass → UAT (manual trigger, stakeholder sign-off)
↓
Sign-off → Staging (auto-deploy, performance tests)
↓
All green → Pre-Prod (manual gate, release candidate)
↓
Release approval → Production (canary → full rollout)
Promotion Gates
| Gate | Automated? | Criteria |
|---|---|---|
| Dev → QA | Yes | PR merged, image built |
| QA → UAT | Yes | All tests pass |
| UAT → Staging | Manual | Product owner sign-off |
| Staging → Pre-Prod | Manual | Perf tests pass, no P1/P2 bugs |
| Pre-Prod → Prod | Manual | Release manager + CAB approval |
---
Common Mistakes DevOps Architects Avoid
1. Making Dev identical to Prod
The mistake: "Let's make all environments identical to avoid surprises."
Why it fails: Six identical production environments costs 6x. For our scenario, $42,000/month instead of $14,460/month.
The right approach: Only Pre-Prod is identical to Prod. Other environments are purposefully right-sized. Dev will occasionally behave differently — Staging and Pre-Prod catch those differences.
2. Using production data in lower environments
The mistake: "Developers need real data to test properly."
Why it fails: GDPR Article 5, SOC2 CC6.1, and common sense. If Dev gets breached, customer PII is exposed.
The right approach: Automated data masking pipeline. Copy production schema and volume patterns, replace PII with faker-generated data. Maintain referential integrity.
3. Skipping Pre-Prod to save money
The mistake: "Staging is good enough."
Why it fails: Staging runs at 50% capacity with different instance types. It will NOT catch memory issues at production limits, throughput bottlenecks at full volume, or configuration drift.
The right approach: Run Pre-Prod intermittently (8 days/month). Costs 25% of always-on but catches what Staging misses.
4. Shared databases across environments
The mistake: "QA and UAT can share a database."
Why it fails: QA resets data during test suites while UAT stakeholders are running demos. Career-ending moment.
The right approach: Each environment gets its own database. Size differently, never share.
5. No scheduled shutdown for lower environments
The mistake: Running Dev and QA 24/7.
Why it fails: Dev sits idle 65% of the month. That is wasted money.
The right approach: Dev shuts down at 8 PM, starts at 8 AM weekdays. QA scales to zero between CI runs. Saves $3,000-4,000/month.
6. Over-sizing from Day 1
The mistake: Sizing for Year-2 aspirational traffic from Month 1.
Why it fails: Paying Year-2 prices with Month-1 traffic. Auto-scaling exists.
The right approach: Size for Month 1 reality. Auto-scale with generous max. Buy reserved capacity after 3 months of data.
---
The DevOps Architect's Checklist
Week 1: Discovery
- [ ] Service count, tech stack, communication patterns from Solution Architect
- [ ] Traffic projections (Day 1, Month 6, Year 1, Year 3) from Product Manager
- [ ] Compliance requirements (SOC2, GDPR, HIPAA, PCI-DSS)
- [ ] Third-party integrations and their sandbox availability
- [ ] Team size and squad structure
- [ ] Budget envelope from Finance
- [ ] Existing infrastructure and portability requirements
Week 2: Architecture Design
- [ ] Container orchestration platform choice
- [ ] Account/subscription isolation strategy
- [ ] VPC/VNet CIDR and connectivity planning
- [ ] Compute sizing per environment (multiplier principle)
- [ ] Database topology per environment
- [ ] CI/CD infrastructure design
- [ ] Observability stack design
Week 3: Security and Compliance
- [ ] IAM strategy per environment
- [ ] Secrets management lifecycle
- [ ] Network security design (SGs, NACLs, WAF)
- [ ] Data masking pipeline design
- [ ] Backup and DR strategy per environment
- [ ] Compliance evidence collection points
Week 4: Cost and Approval
- [ ] Detailed cost model
- [ ] Optimization roadmap
- [ ] Year-1 projection presentation
- [ ] Budget approval
- [ ] Terraform/Bicep modules creation
- [ ] Promotion pipeline documentation
---
Infrastructure-as-Code: Reusability Pattern
Same modules, different variable values per environment:
# environments/prod/terraform.tfvars
environment = "prod"
node_instance_type = "c5.2xlarge"
node_count_min = 6
node_count_max = 12
rds_instance_class = "db.r6g.2xlarge"
rds_multi_az = true
rds_read_replicas = 2
enable_waf = true
enable_cdn = true
scheduled_shutdown = false
# environments/dev/terraform.tfvars
environment = "dev"
node_instance_type = "t3.medium"
node_count_min = 2
node_count_max = 3
rds_instance_class = "db.t4g.medium"
rds_multi_az = false
rds_read_replicas = 0
enable_waf = false
enable_cdn = false
scheduled_shutdown = true
One codebase manages all environments. Changes apply uniformly. No configuration drift.
---
Conclusion
Infrastructure sizing is not a one-time decision — it is a living architecture that evolves with your project.
Three principles that guide every decision:
When the solution architect asks "what configuration do we need?" — the answer is this document.
---
Quick Reference: Complete Environment Comparison
| Dimension | Dev | QA | UAT | Staging | Pre-Prod | Prod |
|---|---|---|---|---|---|---|
| <strong>Platform</strong> | Fargate/AKS burstable | Fargate/AKS | EKS/AKS | EKS/AKS | EKS/AKS (=Prod) | EKS/AKS |
| <strong>CPU per service</strong> | 0.5 vCPU | 1 vCPU | 0.5 vCPU | 1 vCPU | Same as Prod | 1-2 vCPU |
| <strong>Replicas</strong> | 1 | 1 | 1 | 2 | Same as Prod | 2-4 |
| <strong>AZs</strong> | 1 | 1 | 2 | 3 | 3 | 3 |
| <strong>DB HA</strong> | None | None | Multi-AZ | Multi-AZ + replica | Same as Prod | Multi-AZ + 2 replicas |
| <strong>Auto-scaling</strong> | None | None | Min HPA | Full HPA | Same as Prod | HPA + Cluster Autoscaler |
| <strong>WAF/CDN</strong> | None | None | None | Optional | Same as Prod | Full |
| <strong>Uptime</strong> | 12h weekdays | On-demand | 24/7 | 24/7 | ~8d/month | 24/7 (99.95%) |
| <strong>AWS cost/month</strong> | $860 | $800 | $1,330 | $2,170 | $1,650 | $7,000 |
| <strong>Azure cost/month</strong> | $730 | $870 | $1,270 | $2,030 | $1,460 | $6,150 |
---
Frequently Asked Questions
How do I size infrastructure for a new application?
Start with baseline metrics: expected requests per second, average response time target, and data storage growth rate. Use load testing to determine per-instance throughput, then add 40-50% headroom for traffic spikes. Begin with smaller instances and scale up based on actual metrics rather than over-provisioning from day one.
What is the difference between vertical and horizontal scaling?
Vertical scaling (scaling up) means adding more CPU/RAM to existing servers, which is simpler but has hardware limits and requires downtime. Horizontal scaling (scaling out) means adding more servers behind a load balancer, which is more complex but offers near-unlimited growth and fault tolerance. Design for horizontal scaling from the start for production workloads.
How do I calculate the right number of Kubernetes nodes?
Sum the CPU and memory requests of all pods you need to run, add 20-30% for system daemons and overhead, then divide by your chosen instance type's allocatable resources. Factor in N+1 redundancy so losing one node doesn't cause resource pressure. Use the Kubernetes Cluster Autoscaler to dynamically adjust node count based on actual demand.
When should I use reserved instances versus on-demand?
Use reserved instances (or savings plans) for baseline steady-state workloads that run 24/7 — typically databases, core services, and minimum web tier capacity. Use on-demand for variable workloads and development environments. The sweet spot is committing to 1-year terms for 40% savings on your predictable baseline while keeping 30-40% of capacity on-demand for flexibility.