Ansible Dynamic Inventory — Stop Hardcoding Cloud Hosts

The Problem

A static inventory file lists hosts by hand. That works until your infrastructure lives in the cloud, where an autoscaling group spins instances up and down, IPs change on reboot, and nobody wants to edit a text file every time a server appears. The moment someone forgets to update the inventory, a playbook runs against the wrong set of hosts — or misses new ones entirely. Dynamic inventory fixes this by asking the cloud provider "which hosts exist right now?" at runtime, so the inventory is never stale.

Instead of a file you maintain, dynamic inventory is a plugin that calls the provider's API every time you run a playbook and builds the host list on the spot.

Why Static Inventory Breaks in the Cloud

Static inventory assumes hosts are long-lived and known in advance. In the cloud, neither holds:

• IPs are ephemeral. An instance reboot or replacement changes its address.
• Hosts are created and destroyed constantly. Autoscaling means the host count moves on its own.
• Humans fall behind. Any manual list drifts from reality within hours.
• Metadata is lost. A flat list can't express "all web instances in ap-south-1 tagged env=prod."

Dynamic inventory treats the cloud provider as the source of truth. The host list is generated fresh on every run, so there is nothing to keep in sync.

How Inventory Plugins Work

Ansible ships inventory plugins for major providers — aws_ec2, azure_rm, gcp_compute, and more. A plugin is configured with a YAML file whose name ends in a recognized suffix (.aws_ec2.yml or an inventory_*.yml referenced explicitly). At runtime Ansible:

Reads the plugin config file.

Authenticates to the provider (for AWS, via standard credentials/role).

Queries the API for matching instances.

Builds hosts and groups them according to your rules.

You point ansible-playbook at the config file exactly like a static one:

ansible-playbook -i inventory.aws_ec2.yml site.yml

# Preview what the plugin returns, as a tree
ansible-inventory -i inventory.aws_ec2.yml --graph

A Sample aws_ec2 Inventory Config

Here is a realistic aws_ec2 plugin configuration that discovers running instances in two regions and groups them automatically:

# inventory.aws_ec2.yml
plugin: amazon.aws.aws_ec2

regions:
  - ap-south-1
  - us-east-1

# Only consider running instances
filters:
  instance-state-name: running
  tag:Project: devopskit

# Use the private IP as the connection address
hostnames:
  - private-ip-address

# Create groups automatically from instance tags
keyed_groups:
  - key: tags.Role          # tag Role=web  -> group tag_Role_web
    prefix: tag_Role
  - key: placement.region   # region        -> group aws_region_ap_south_1
    prefix: aws_region
  - key: tags.Environment   # tag Environment=prod -> group tag_Environment_prod
    prefix: tag_Environment

# Attach extra host variables from instance metadata
compose:
  ansible_host: private_ip_address

# Cache results so repeated runs don't re-hit the API
cache: true
cache_plugin: jsonfile
cache_timeout: 300
cache_connection: /tmp/ansible_aws_ec2_cache

Grouping by Tags with keyed_groups

keyed_groups is the feature that makes dynamic inventory powerful. Each entry turns an instance attribute into a group. With the config above, an instance tagged Role=web lands in a group named tag_Role_web. Your playbook then targets that group as if you had written it by hand:

---
name: Configure all web instances, wherever they live
  hosts: tag_Role_web

  become: true
  roles:
    - nginx

Because the group is rebuilt from live tags every run, adding a new instance with Role=web means the next playbook run picks it up automatically. No file edit, no missed host.

Caching: Balancing Freshness and API Load

Querying the EC2 API on every single run adds latency and can hit rate limits in large accounts. The cache settings above store the result in a local file for cache_timeout seconds (300 here) so back-to-back runs reuse it.

The trade-off is directness: with caching on, a brand-new instance might not appear until the cache expires. When you need the absolute latest state, bust the cache:

ansible-inventory -i inventory.aws_ec2.yml --graph --flush-cache

Keep caching on for routine runs and flush it when you know infrastructure just changed.

Common Mistakes to Avoid

• Wrong file name or missing plugin: key. Ansible won't recognize the file as an inventory plugin and silently treats it as a static host named after the file.
• Over-broad filters. Omitting instance-state-name: running pulls in terminated or stopped hosts you can't connect to.
• Forgetting the collection. aws_ec2 lives in the amazon.aws collection — install it with ansible-galaxy collection install amazon.aws and the boto3 Python library.
• Stale cache confusion. When a new instance "won't show up," it's usually the cache. Flush it before assuming the config is wrong.
• Hardcoding IPs anyway. Mixing a static host list with dynamic inventory reintroduces exactly the drift you were trying to eliminate.

FAQ

How is dynamic inventory different from a static file?

A static inventory is a hand-maintained list of hosts, while dynamic inventory is a plugin that queries your cloud provider's API at runtime and builds the host list on the fly. Dynamic inventory never goes stale because it reflects whatever instances actually exist each time you run a playbook.

What does keyed_groups do?

keyed_groups turns instance attributes — most often tags — into Ansible groups automatically. For example, keying on the Role tag produces groups like tag_Role_web, so your playbooks can target hosts by tag without ever listing them manually.

Does caching make my inventory out of date?

It can, briefly. Caching stores API results for a set timeout to reduce latency and API calls, so an instance created within that window may not appear until the cache expires. Flush the cache with --flush-cache when you need the current state immediately.

What do I need to use the aws_ec2 plugin?

You need the amazon.aws collection (ansible-galaxy collection install amazon.aws), the boto3 Python library, and valid AWS credentials available through the standard chain — environment variables, a shared credentials file, or an instance role.