TL;DR — Quick Fix
Switch to autoscaling self-hosted runners on Kubernetes to cut CI costs by 60-80%:
# Install actions-runner-controller (ARC) on your K8s cluster
helm repo add actions-runner-controller https://actions-runner-controller.github.io/actions-runner-controller
helm install arc actions-runner-controller/actions-runner-controller \
--namespace actions-runner-system \
--create-namespace \
--set authSecret.create=true \
--set authSecret.github_app_id="12345" \
--set authSecret.github_app_installation_id="67890" \
--set authSecret.github_app_private_key="$(cat private-key.pem)"
# Deploy an autoscaling runner set
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
name: org-runners
spec:
replicas: 1
template:
spec:
organization: my-org
ephemeral: true
labels:
- self-hosted
- linux
- x64
---
apiVersion: actions.summerwind.dev/v1alpha1
kind: HorizontalRunnerAutoscaler
metadata:
name: org-runners-autoscaler
spec:
scaleTargetRef:
name: org-runners
minReplicas: 0
maxReplicas: 20
scaleUpTriggers:
- githubEvent:
workflowJob: {}
duration: "30m"
---
Architecture — Self-Hosted Runner Autoscaling
---
Step 1 — Cost Comparison: GitHub-Hosted vs Self-Hosted
| Metric | GitHub-Hosted | Self-Hosted (Spot) | Savings |
|---|---|---|---|
| 2-core Linux | $0.008/min | ~$0.001/min | 87% |
| 8-core Linux | $0.032/min | ~$0.003/min | 91% |
| 100K mins/month | $800-$3,200 | $100-$300 | 70-90% |
| Idle cost | Minutes still tick | $0 (scale to zero) | 100% |
| Storage | 14GB included | Custom (unlimited) | - |
| Docker layer cache | Cold start each run | Persistent cache | 3-5x faster |
---
Step 2 — Deploy Actions Runner Controller (ARC)
# arc-values.yaml — production configuration
replicaCount: 2
authSecret:
create: true
github_app_id: "12345"
github_app_installation_id: "67890"
github_app_private_key: |
-----BEGIN RSA PRIVATE KEY-----
(your key here)
-----END RSA PRIVATE KEY-----
metrics:
serviceMonitor:
enabled: true
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 100m
memory: 256Mi
# runner-deployment.yaml — ephemeral runners
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
name: linux-runners
namespace: actions-runner-system
spec:
replicas: 1
template:
spec:
organization: my-org
ephemeral: true
dockerEnabled: true
dockerMTU: 1400
labels:
- self-hosted
- linux
- x64
- large
resources:
limits:
cpu: "4"
memory: "8Gi"
requests:
cpu: "2"
memory: "4Gi"
volumeMounts:
- name: work
mountPath: /runner/_work
volumes:
- name: work
emptyDir:
sizeLimit: 50Gi
---
Step 3 — KEDA-Based Autoscaling
For more fine-grained scaling based on workflow queue depth:
# keda-scaler.yaml
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: runner-scaler
namespace: actions-runner-system
spec:
scaleTargetRef:
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
name: linux-runners
minReplicaCount: 0
maxReplicaCount: 30
pollingInterval: 10
cooldownPeriod: 300
triggers:
- type: github-runner
metadata:
owner: "my-org"
runnerScope: "org"
labels: "self-hosted,linux,x64,large"
targetWorkflowQueueLength: "1"
authenticationRef:
name: github-trigger-auth
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: github-trigger-auth
namespace: actions-runner-system
spec:
secretTargetRef:
- parameter: personalAccessToken
name: github-auth
key: pat
---
Step 4 — Spot Instances for Runners
# Karpenter NodePool for CI runners
apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
name: ci-runners
spec:
template:
spec:
requirements:
- key: karpenter.sh/capacity-type
operator: In
values: ["spot"]
- key: karpenter.k8s.aws/instance-category
operator: In
values: ["m", "c"]
- key: karpenter.k8s.aws/instance-size
operator: In
values: ["large", "xlarge", "2xlarge"]
- key: kubernetes.io/arch
operator: In
values: ["amd64"]
nodeClassRef:
name: ci-node-class
taints:
- key: runner
value: "true"
effect: NoSchedule
disruption:
consolidationPolicy: WhenEmpty
consolidateAfter: 60s
limits:
cpu: "200"
memory: 400Gi
---
Step 5 — Cost Monitoring and Alerts
#!/bin/bash
# weekly-ci-cost-report.sh
MINUTES_USED=$(gh api /orgs/my-org/settings/billing/actions \
--jq '.total_minutes_used')
PAID_MINUTES=$(gh api /orgs/my-org/settings/billing/actions \
--jq '.total_paid_minutes_used')
COST=$(echo "$PAID_MINUTES * 0.008" | bc)
echo "Weekly CI Cost Report:"
echo " Total minutes: $MINUTES_USED"
echo " Paid minutes: $PAID_MINUTES"
echo " Estimated cost: \$$COST"
# Prometheus alerting rules for runner costs
groups:
- name: runner-cost-alerts
rules:
- alert: RunnerCostSpike
expr: |
sum(rate(runner_job_duration_seconds_total[1h])) * 0.008 > 10
for: 15m
labels:
severity: warning
annotations:
summary: "CI runner cost spike detected"
- alert: IdleRunnersRunning
expr: |
count(runner_status{status="idle"}) > 5
for: 30m
labels:
severity: warning
annotations:
summary: "Idle runners not scaling down"
---
Frequently Asked Questions
How fast do self-hosted runners scale up?
With webhook-based scaling (ARC), runners start in 20-40 seconds when Kubernetes nodes are available. If Karpenter needs to provision a new spot node, add 60-90 seconds. This is faster than GitHub's own larger runner startup for 8+ core machines.
Are ephemeral runners safe for security?
Yes, ephemeral runners are more secure than persistent ones. Each job gets a fresh container with no state from previous jobs. This prevents credential leakage, supply chain attacks, and crypto-mining on idle runners.
What if spot instances get reclaimed during a build?
CI jobs are retryable by nature. Configure runs-on with a retry strategy in your workflow, or use job.if: failure() to trigger reruns. With diversified spot pools, interruption rates stay under 5%.
Should I cache Docker layers on self-hosted runners?
Yes, this is one of the biggest speed advantages. Use a persistent volume for Docker build cache or a distributed cache like BuildKit with S3 backend. Teams report 3-5x faster Docker builds compared to GitHub-hosted runners.
How do I handle multiple architectures (ARM + x86)?
Deploy separate RunnerDeployments with different labels: linux-x64 and linux-arm64. Use Karpenter NodePools targeting arm64 instances (Graviton). ARM spot instances are 40% cheaper than equivalent x86.
---