GitLab supports several advanced pipeline architectures beyond simple sequential stage execution. DAG pipelines let jobs start as soon as their dependencies complete,

GitLab CI/CD Pipelines — Fixing Timeouts, Slow Runners & Failed Deployments

Q: 1. The .gitlab-ci.yml Structure

Every GitLab CI pipeline starts with a .gitlab-ci.yml file at the root of your repository. This YAML file defines stages, jobs, scripts, artifacts, and caching.

Q: 10. GitLab CI vs GitHub Actions

| Feature | GitLab CI | GitHub Actions | |---------|-----------|----------------| | Config file | .gitlab-ci.yml | .github/workflows/.yml |

# GitLab CI/CD Complete Guide

GitLab CI/CD is one of the most powerful integrated CI/CD platforms available today.

Unlike standalone CI tools, GitLab CI lives inside your source control platform —

giving you pipelines, container registries, environments, security scanning, and

deployment tracking in a single interface. This guide covers everything from writing

your first .gitlab-ci.yml to deploying production workloads on Kubernetes.

---

1. The .gitlab-ci.yml Structure

Every GitLab CI pipeline starts with a .gitlab-ci.yml file at the root of your

repository. This YAML file defines stages, jobs, scripts, artifacts, and caching.

Basic Structure

stages: - build - test - deploy default: image: node:20-alpine before_script: - echo "Pipeline starting..." variables: NODE_ENV: "production" DOCKER_REGISTRY: "registry.gitlab.com" build: stage: build script: - npm ci - npm run build artifacts: paths: - dist/

expire_in: 1 hour

Jobs, Scripts, and Artifacts

Jobs are the fundamental units of execution. Each job runs in its own container

(or shell session) and must belong to a stage.

test:unit: stage: test script: - npm ci - npm run test:unit artifacts: reports: junit: junit-report.xml when: always test:integration: stage: test script: - npm ci - npm run test:integration services: - postgres:15 variables: POSTGRES_DB: test_db POSTGRES_USER: runner

POSTGRES_PASSWORD: secret

Cache vs Artifacts

Understanding the difference between cache and artifacts is critical:

Artifacts pass data between jobs within the same pipeline (build outputs, test reports).
Cache persists data across pipelines for the same branch (node_modules, pip packages).

build: stage: build cache: key: files: - package-lock.json paths: - node_modules/ policy: pull-push script: - npm ci - npm run build artifacts: paths: - dist/

expire_in: 30 minutes

---

2. GitLab Runners

Runners are the agents that execute your CI/CD jobs. GitLab offers shared runners

on GitLab.com, but most production teams register their own specific runners for

performance and security.

Shared vs Specific Runners

Feature	Shared Runners	Specific Runners
Setup	Zero config	Requires registration
Cost	Included in plan (with limits)	Self-hosted infrastructure
Performance	Variable (shared queue)	Predictable
Security	Multi-tenant	Isolated to your projects
Customization	Limited	Full control

Docker Executor

The Docker executor is the most common choice. Each job gets a fresh container:

# Runner config (config.toml)
[[runners]]
  name = "docker-runner-01"
  url = "https://gitlab.com/"
  executor = "docker"
  [runners.docker]
    image = "alpine:latest"
    privileged = false
    volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
    allowed_images = ["node:", "python:", "golang:*"]
    pull_policy = ["if-not-present"]

Kubernetes Executor

For dynamic scaling, the Kubernetes executor spins up pods per job:

[[runners]]
  name = "k8s-runner"
  url = "https://gitlab.com/"
  executor = "kubernetes"
  [runners.kubernetes]
    namespace = "gitlab-runners"
    image = "alpine:latest"
    service_account = "gitlab-runner"
    [runners.kubernetes.pod_labels]
      "ci/managed-by" = "gitlab"
    [runners.kubernetes.node_selector]
      "node-type" = "ci"
    [runners.kubernetes.resources]
      [runners.kubernetes.resources.requests]
        cpu = "500m"
        memory = "1Gi"
      [runners.kubernetes.resources.limits]
        cpu = "2"
        memory = "4Gi"

Tagging Runners

Use tags to route jobs to specific runners:

deploy:production: stage: deploy tags: - production - aws script:

- ./deploy.sh production

---

3. Variables and Secrets Management

GitLab CI provides multiple layers of variable management with built-in

secret masking and environment scoping.

Variable Hierarchy

Variables are resolved in this order (last wins):

Runner-level variables

Instance-level CI/CD variables

Group-level CI/CD variables

Project-level CI/CD variables

.gitlab-ci.yml variables

Job-level variables

Trigger/API variables

Defining Variables

variables: APP_NAME: "my-service" DEPLOY_REGION: "us-east-1" deploy: stage: deploy variables: ENVIRONMENT: "production" script: - echo "Deploying $APP_NAME to $DEPLOY_REGION"

- echo "Using secret from CI/CD settings (masked)"

Protected and Masked Variables

Configure sensitive variables in Settings > CI/CD > Variables:

Protected: Only available on protected branches/tags
Masked: Hidden from job logs (must meet masking requirements)
Environment scope: Limit to specific environments

Using External Secret Managers

deploy: stage: deploy id_tokens: VAULT_TOKEN: aud: https://vault.example.com secrets: DATABASE_PASSWORD: vault: production/db/password@secrets API_KEY: vault: production/api/key@secrets script:

- ./deploy.sh

---

4. Pipeline Patterns

GitLab supports several advanced pipeline architectures beyond simple

sequential stage execution.

Directed Acyclic Graph (DAG)

DAG pipelines let jobs start as soon as their dependencies complete,

rather than waiting for the entire previous stage:

stages: - build - test - deploy build:frontend: stage: build script: npm run build:frontend artifacts: paths: [frontend/dist/] build:backend: stage: build script: go build -o server ./cmd/server artifacts: paths: [server] test:frontend: stage: test needs: [build:frontend] script: npm run test:frontend test:backend: stage: test needs: [build:backend] script: go test ./... deploy: stage: deploy needs: [test:frontend, test:backend]

script: ./deploy.sh

Parent-Child Pipelines

Split complex pipelines into smaller, maintainable configurations:

# .gitlab-ci.yml (parent) stages: - triggers trigger:frontend: stage: triggers trigger: include: frontend/.gitlab-ci.yml strategy: depend rules: - changes: - frontend/*/ trigger:backend: stage: triggers trigger: include: backend/.gitlab-ci.yml strategy: depend rules: - changes:

- backend/*/

# frontend/.gitlab-ci.yml (child) stages: - build - test build: stage: build image: node:20 script: - npm ci - npm run build test: stage: test image: node:20 script:

- npm run test

Multi-Project Pipelines

Trigger pipelines in other repositories:

deploy:infrastructure: stage: deploy trigger: project: devops/infrastructure branch: main strategy: depend variables: APP_VERSION: $CI_COMMIT_TAG

DEPLOY_ENV: production

---

5. Environments and Review Apps

Environments let you track deployments and create dynamic preview

instances for every merge request.

Static Environments

deploy:staging: stage: deploy script: - kubectl apply -f k8s/staging/ environment: name: staging url: https://staging.myapp.com rules: - if: $CI_COMMIT_BRANCH == "develop" deploy:production: stage: deploy script: - kubectl apply -f k8s/production/ environment: name: production url: https://myapp.com rules: - if: $CI_COMMIT_BRANCH == "main"

when: manual

Dynamic Review Apps

Create an environment per merge request branch:

review:deploy:
  stage: deploy
  script:
    - helm upgrade --install review-$CI_MERGE_REQUEST_IID ./chart
        --set image.tag=$CI_COMMIT_SHA
        --set ingress.host=$CI_ENVIRONMENT_SLUG.review.myapp.com
        --namespace review-apps
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    url: https://$CI_ENVIRONMENT_SLUG.review.myapp.com
    on_stop: review:stop
    auto_stop_in: 3 days
  rules:
    - if: $CI_MERGE_REQUEST_IID

review:stop:
  stage: deploy
  script:
    - helm uninstall review-$CI_MERGE_REQUEST_IID --namespace review-apps
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    action: stop
  rules:
    - if: $CI_MERGE_REQUEST_IID
      when: manual
  allow_failure: true

---

6. Security Scanning

GitLab provides built-in security scanners that integrate directly into your

merge request workflow. Results appear in the MR widget and the Security Dashboard.

SAST (Static Application Security Testing)

include: - template: Security/SAST.gitlab-ci.yml sast: stage: test variables: SAST_EXCLUDED_PATHS: "spec,test,tests,tmp"

SEARCH_MAX_DEPTH: 4

DAST (Dynamic Application Security Testing)

include: - template: Security/DAST.gitlab-ci.yml dast: stage: dast variables: DAST_WEBSITE: https://$CI_ENVIRONMENT_SLUG.review.myapp.com DAST_FULL_SCAN_ENABLED: "true" DAST_BROWSER_SCAN: "true"

needs: ["review:deploy"]

Container Scanning

include: - template: Security/Container-Scanning.gitlab-ci.yml container_scanning: stage: test variables: CS_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA

CS_SEVERITY_THRESHOLD: "HIGH"

Dependency Scanning

include: - template: Security/Dependency-Scanning.gitlab-ci.yml dependency_scanning: stage: test variables:

DS_EXCLUDED_ANALYZERS: "gemnasium-python"

Complete Security Pipeline

include: - template: Security/SAST.gitlab-ci.yml - template: Security/Secret-Detection.gitlab-ci.yml - template: Security/Dependency-Scanning.gitlab-ci.yml - template: Security/Container-Scanning.gitlab-ci.yml - template: Security/License-Scanning.gitlab-ci.yml stages: - build - test - security

- deploy

---

7. Kubernetes Deployment with GitLab CI

GitLab CI integrates tightly with Kubernetes for container orchestration

and progressive delivery.

Building and Pushing Images

variables: DOCKER_IMAGE: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG DOCKER_TAG: $CI_COMMIT_SHA build:image: stage: build image: docker:24 services: - docker:24-dind variables: DOCKER_TLS_CERTDIR: "/certs" before_script: - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY script: - docker build --cache-from $DOCKER_IMAGE:latest --tag $DOCKER_IMAGE:$DOCKER_TAG --tag $DOCKER_IMAGE:latest . - docker push $DOCKER_IMAGE:$DOCKER_TAG

- docker push $DOCKER_IMAGE:latest

Deploying with Helm

.deploy_template: &deploy_template
  stage: deploy
  image: alpine/helm:3.14
  before_script:
    - kubectl config use-context $KUBE_CONTEXT

deploy:staging:
  <<: *deploy_template
  script:
    - helm upgrade --install $APP_NAME ./helm/chart
        --namespace staging
        --set image.repository=$DOCKER_IMAGE
        --set image.tag=$DOCKER_TAG
        --set replicas=2
        --values ./helm/values-staging.yaml
        --wait --timeout 300s
  environment:
    name: staging
    url: https://staging.myapp.com
  rules:
    - if: $CI_COMMIT_BRANCH == "develop"

deploy:production:
  <<: *deploy_template
  script:
    - helm upgrade --install $APP_NAME ./helm/chart
        --namespace production
        --set image.repository=$DOCKER_IMAGE
        --set image.tag=$DOCKER_TAG
        --set replicas=5
        --values ./helm/values-production.yaml
        --wait --timeout 600s
  environment:
    name: production
    url: https://myapp.com
  rules:
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/
      when: manual

Canary Deployments

deploy:canary:
  stage: deploy
  script:
    - helm upgrade --install $APP_NAME-canary ./helm/chart
        --namespace production
        --set image.tag=$DOCKER_TAG
        --set replicas=1
        --set canary.enabled=true
        --set canary.weight=10
  environment:
    name: production/canary
  rules:
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/
      when: manual

deploy:promote:
  stage: deploy
  script:
    - helm upgrade --install $APP_NAME ./helm/chart
        --namespace production
        --set image.tag=$DOCKER_TAG
        --set replicas=5
    - helm uninstall $APP_NAME-canary --namespace production
  environment:
    name: production
  needs: [deploy:canary]
  when: manual

---

8. Caching and Optimization Strategies

Pipeline speed directly impacts developer productivity. Here are proven

strategies to reduce execution time.

Layered Caching

variables: PIP_CACHE_DIR: "$CI_PROJECT_DIR/.pip-cache" NPM_CONFIG_CACHE: "$CI_PROJECT_DIR/.npm-cache" .node_cache: cache: - key: files: - package-lock.json paths: - node_modules/ policy: pull - key: npm-global-$CI_COMMIT_REF_SLUG paths: - .npm-cache/ policy: pull-push .python_cache: cache: key: files: - requirements.txt - poetry.lock paths: - .pip-cache/

- .venv/

Conditional Job Execution

Only run jobs when relevant files change:

test:frontend: stage: test extends: .node_cache script: - npm run test:frontend rules: - changes: - frontend/*/ - package.json - package-lock.json test:backend: stage: test script: - go test ./... rules: - changes: - cmd/*/ - internal/*/ - go.mod

- go.sum

Parallel Test Execution

Split test suites across multiple jobs:

test: stage: test parallel: 4 script:

- npm run test -- --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL

Docker Layer Caching

build:image: stage: build image: docker:24 services: - docker:24-dind variables: DOCKER_BUILDKIT: "1" script: - docker pull $CI_REGISTRY_IMAGE:latest || true - docker build --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from $CI_REGISTRY_IMAGE:latest --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA --tag $CI_REGISTRY_IMAGE:latest . - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA

- docker push $CI_REGISTRY_IMAGE:latest

Interruptible Jobs

Cancel redundant pipelines when new commits arrive:

default: interruptible: true workflow: auto_cancel: on_new_commit: interruptible deploy:production:

interruptible: false

---

9. Production .gitlab-ci.yml Examples

Node.js Application

stages:
  - install
  - quality
  - build
  - deploy

variables:
  NODE_IMAGE: node:20-alpine

install:
  stage: install
  image: $NODE_IMAGE
  script:
    - npm ci --prefer-offline
  cache:
    key:
      files: [package-lock.json]
    paths: [node_modules/]
    policy: pull-push
  artifacts:
    paths: [node_modules/]
    expire_in: 30 minutes

lint:
  stage: quality
  image: $NODE_IMAGE
  needs: [install]
  script:
    - npm run lint
    - npm run type-check

test:
  stage: quality
  image: $NODE_IMAGE
  needs: [install]
  script:
    - npm run test:ci
  coverage: '/Statements\s:\s(\d+\.?\d*)%/'
  artifacts:
    reports:
      junit: reports/junit.xml
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml

build:
  stage: build
  image: docker:24
  services: [docker:24-dind]
  script:
    - docker login -u $CI_REGISTRY_USER
        -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
  rules:
    - if: $CI_COMMIT_BRANCH == "main"

deploy:
  stage: deploy
  image: bitnami/kubectl:latest
  script:
    - kubectl set image deployment/app
        app=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
        --namespace production
    - kubectl rollout status deployment/app
        --namespace production --timeout=300s
  environment:
    name: production
    url: https://myapp.com
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
      when: manual

Python Application

stages: - test - build - deploy variables: PIP_CACHE_DIR: "$CI_PROJECT_DIR/.pip-cache" PYTHON_IMAGE: python:3.12-slim .python_setup: image: $PYTHON_IMAGE cache: key: files: [poetry.lock] paths: - .pip-cache/ - .venv/ before_script: - pip install poetry - poetry config virtualenvs.in-project true - poetry install --no-interaction lint: stage: test extends: .python_setup script: - poetry run ruff check . - poetry run mypy src/ test: stage: test extends: .python_setup services: - postgres:15 - redis:7 variables: DATABASE_URL: "postgresql://runner:secret@postgres/test" REDIS_URL: "redis://redis:6379" POSTGRES_DB: test POSTGRES_USER: runner POSTGRES_PASSWORD: secret script: - poetry run pytest --junitxml=report.xml --cov=src/ artifacts: reports: junit: report.xml build: stage: build image: docker:24 services: [docker:24-dind] script: - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA . - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA rules: - if: $CI_COMMIT_BRANCH == "main" deploy: stage: deploy image: alpine/helm:3.14 script: - helm upgrade --install api ./helm --set image.tag=$CI_COMMIT_SHA --namespace production environment: name: production rules: - if: $CI_COMMIT_BRANCH == "main"

when: manual

Go Application

stages: - test - build - deploy variables: GOPATH: "$CI_PROJECT_DIR/.go" GO_IMAGE: golang:1.22-alpine .go_setup: image: $GO_IMAGE cache: key: files: [go.sum] paths: - .go/pkg/mod/ before_script: - go mod download lint: stage: test extends: .go_setup script: - go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest - golangci-lint run ./... test: stage: test extends: .go_setup script: - go test -race -coverprofile=coverage.out ./... - go tool cover -func=coverage.out coverage: '/total:\s+$statements$\s+(\d+\.\d+)%/' build: stage: build image: docker:24 services: [docker:24-dind] script: - docker build --build-arg VERSION=$CI_COMMIT_TAG -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA . - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA rules: - if: $CI_COMMIT_TAG =~ /^v\d+/ deploy: stage: deploy image: bitnami/kubectl:latest script: - kubectl set image deployment/api api=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA -n production - kubectl rollout status deployment/api -n production --timeout=300s environment: name: production rules: - if: $CI_COMMIT_TAG =~ /^v\d+/

when: manual

---

10. GitLab CI vs GitHub Actions

Feature	GitLab CI	GitHub Actions
Config file	<code class="inline-code">.gitlab-ci.yml</code>	<code class="inline-code">.github/workflows/*.yml</code>
Execution model	Stages + jobs	Workflows + jobs + steps
Runner hosting	Self-hosted + shared	Self-hosted + GitHub-hosted
Container registry	Built-in	GitHub Packages (GHCR)
DAG support	<code class="inline-code">needs</code> keyword	<code class="inline-code">needs</code> on jobs
Matrix builds	<code class="inline-code">parallel:matrix</code>	<code class="inline-code">strategy.matrix</code>
Environments	Built-in with tracking	Environments with protection
Review apps	Native support	Manual setup required
Security scanning	Built-in (SAST, DAST, etc.)	Marketplace actions
Secrets	CI/CD variables + Vault	Repository/org secrets
Artifact sharing	<code class="inline-code">artifacts</code> + <code class="inline-code">dependencies</code>	<code class="inline-code">upload/download-artifact</code>
Caching	File-key based cache	<code class="inline-code">actions/cache</code>
Auto DevOps	Yes (zero-config)	No equivalent
Parent-child	Native trigger pipelines	Reusable workflows
Pipeline editor	Built-in visual editor	No native editor
Merge trains	Built-in	Not available
Pricing model	Minutes-based (self-host free)	Minutes-based

When to Choose GitLab CI

You want an all-in-one platform (SCM + CI + registry + security)
Your team uses self-hosted infrastructure
You need built-in security scanning without marketplace dependencies
You want merge trains and advanced merge request workflows
Review apps are a core part of your development process

When to Choose GitHub Actions

Your code already lives on GitHub
You prefer a marketplace ecosystem of community actions
You need integration with the broader GitHub ecosystem
Your team prefers a modular, composable workflow approach
You want GitHub-hosted runners without infrastructure management

---

Summary

GitLab CI/CD provides a comprehensive, integrated pipeline platform that

scales from single-developer projects to enterprise deployments. Key

takeaways:

Structure pipelines with stages and DAG for optimal parallelism

Use specific runners with appropriate executors for production

Layer your caching strategy with file-key caches and conditional execution

Leverage parent-child pipelines for monorepo architectures

Enable security scanning early — it integrates into merge requests

Use environments and review apps for deployment tracking and previews

Optimize aggressively with parallel jobs and interruptible pipelines

Start with a simple pipeline that builds, tests, and deploys. Layer in

security scanning, review apps, and advanced patterns as your team matures.

The best CI/CD pipeline is one your team trusts and maintains consistently.

---

Frequently Asked Questions

What is the difference between GitHub Actions and GitLab CI?

GitLab CI uses a single .gitlab-ci.yml file with stages that run sequentially, while GitHub Actions uses multiple workflow files with jobs that run in parallel by default. GitLab CI is tightly integrated with GitLab's DevOps platform (registry, security scanning, environments), while GitHub Actions has a larger marketplace of community actions. GitLab CI includes built-in container registry and Auto DevOps.

How do I speed up GitLab CI pipelines?

Use cache: to persist dependencies between runs, leverage rules:changes to skip jobs when irrelevant files change, and run independent jobs in the same stage for parallelism. Use smaller Docker images for jobs, enable shallow cloning with GIT_DEPTH: 1, and consider GitLab's parent-child pipelines to split large configurations.

What are GitLab CI stages and how do they work?

Stages define the execution order of jobs — all jobs in a stage run in parallel, and the next stage starts only when all jobs in the previous stage complete. Default stages are build, test, deploy. Jobs are assigned to stages with the stage: keyword. A failed job stops subsequent stages unless marked with allow_failure: true.

How do I use GitLab CI/CD variables securely?

Define sensitive values in Settings > CI/CD > Variables with "Masked" and "Protected" options enabled. Masked variables are hidden in job logs, and protected variables are only available in protected branches/tags. Never echo secrets in scripts, and use file-type variables for multi-line secrets like SSH keys or certificates.