DevOps / Ansible Interview questions

Ansible is an open-source, agentless automation tool used for configuration management, application deployment, and orchestration. It connects to managed nodes over standard SSH (or WinRM for Windows) and pushes changes out, rather than requiring a persistent agent installed on every target machine.

Automation is described declaratively in YAML files called playbooks, which list the desired end state of a system rather than a sequence of imperative shell commands. Ansible reads the playbook, connects to the target hosts listed in its inventory, and applies the necessary changes using reusable units of work called modules. Because there's no agent to install, patch, or keep running, and the connection uses infrastructure most environments already have (SSH), Ansible has a comparatively low barrier to adoption compared to agent-based tools.

• Agentless - connects over SSH/WinRM, no software to install on managed nodes.
• Declarative YAML playbooks - describe desired state, not step-by-step commands.
• Idempotent modules - re-running a playbook only changes what's actually out of state.
• Reusable roles and collections - packaged, shareable units of automation.
• Extensive module library - covers cloud providers, networking gear, containers, and OS-level tasks.
• Human-readable output and dry-run (check mode) - preview changes before applying them.

These features combine to make Ansible approachable for teams that want automation without a steep learning curve or heavy new infrastructure, since existing SSH access is often all that's required to get started.

A playbook is a YAML file that defines one or more plays, each of which maps a group of hosts to an ordered list of tasks to run against them. It's the primary unit of automation in Ansible, describing what should happen and where.

---
- name: Configure web servers
  hosts: webservers
  become: true
  tasks:
    - name: Install nginx
      ansible.builtin.package:
        name: nginx
        state: present
    - name: Start nginx
      ansible.builtin.service:
        name: nginx
        state: started

Each task in a play calls a module (like package or service above) with specific arguments, and Ansible executes tasks in the order they're written, top to bottom, against every host in the play. Playbooks can include variables, conditionals, loops, and roles to handle more complex logic beyond a simple linear task list.

An inventory file lists the managed nodes Ansible can operate on, organized into groups so playbooks can target them by name. It can be as simple as a static INI or YAML file, or generated dynamically by querying a cloud provider or CMDB.

# INI-style static inventory
[webservers]
web1.example.com
web2.example.com

[dbservers]
db1.example.com ansible_user=admin

[production:children]
webservers
dbservers

Groups can nest other groups (as shown with production:children), and per-host or per-group variables can be attached directly in the inventory or via separate group_vars/host_vars directories. Playbooks then reference these groups in their hosts: field, so the same playbook can be pointed at different environments just by swapping the inventory file.

A module is a self-contained, reusable unit of code that performs one specific piece of work, such as installing a package, copying a file, or creating a cloud resource. Tasks in a playbook are really just calls to a module with a set of arguments.

- name: Ensure a config file is present
  ansible.builtin.copy:
    src: app.conf
    dest: /etc/app/app.conf
    mode: '0644'

Modules are designed to be idempotent: the copy module above checks whether the destination file already matches the source before making any change, so running the same task repeatedly only reports a change the first time. Ansible ships hundreds of built-in modules under the ansible.builtin collection, and thousands more are available through community and vendor-published collections for things like cloud infrastructure, network devices, and container platforms.

Facts are pieces of information Ansible automatically discovers about a managed host at the start of a play, such as its IP addresses, OS distribution, kernel version, mounted filesystems, and available memory. They're gathered by the built-in setup module, which runs implicitly unless gather_facts: false is set.

- name: Show the OS family
  ansible.builtin.debug:
    msg: "This host runs {{ ansible_facts['os_family'] }}"

Facts let playbooks make conditional decisions based on a host's actual state, for example installing a different package name depending on distribution, without the playbook author needing to hardcode assumptions about every target. Because gathering facts takes time on every host, it can be disabled for plays that don't need them, or supplemented/replaced with fact caching for large inventories.

A role is a standardized way of packaging a set of tasks, handlers, variables, templates, and files around a single purpose, like "configure nginx" or "install a database," so that logic can be reused across multiple playbooks and projects.

roles/
  nginx/
    tasks/main.yml
    handlers/main.yml
    templates/nginx.conf.j2
    defaults/main.yml
    vars/main.yml
    files/
    meta/main.yml

Ansible expects this fixed directory structure so it can automatically find a role's tasks, handlers, and variables without extra configuration. A playbook then simply references the role by name, and Ansible pulls in everything the role provides.

- hosts: webservers
  roles:
    - nginx

This structure encourages breaking automation into composable, testable units rather than one large, monolithic playbook.

Ansible pulls variables from many possible sources, each with a different scope and precedence.

Understanding which source wins matters because the same variable name can be set in several places at once; Ansible resolves this through a well-defined precedence order rather than "last one wins" by file position alone.

A standard role follows a fixed layout that Ansible auto-discovers, with each subdirectory holding a specific kind of content:

• tasks/main.yml - the main list of tasks the role performs.
• handlers/main.yml - handlers the role's tasks can notify.
• defaults/main.yml - lowest-precedence default variable values.
• vars/main.yml - higher-precedence, role-specific variables.
• templates/ - Jinja2 template files rendered onto target hosts.
• files/ - static files copied as-is to target hosts.
• meta/main.yml - role metadata and dependencies on other roles.

Any directory that isn't needed can simply be omitted; Ansible only looks for what's present, which keeps small roles lightweight while still supporting the full structure when a role grows more complex.

A handler is a task that only runs when explicitly triggered by another task's notify directive, and typically only when that task actually reports a change. Handlers are the standard way to express "restart this service, but only if its configuration actually changed."

tasks:
  - name: Update nginx config
    ansible.builtin.template:
      src: nginx.conf.j2
      dest: /etc/nginx/nginx.conf
    notify: Restart nginx

handlers:
  - name: Restart nginx
    ansible.builtin.service:
      name: nginx
      state: restarted

Handlers are defined much like normal tasks but live in a separate handlers section (or a role's handlers/main.yml), and by default they run once, at the end of the play, after all regular tasks have completed - even if multiple tasks notify the same handler during the run, it still only fires a single time.

Ansible Vault is a built-in feature for encrypting sensitive content, like passwords, API keys, or entire variable files, so secrets can be safely committed to version control alongside the rest of a playbook.

ansible-vault encrypt secrets.yml
ansible-vault view secrets.yml
ansible-playbook site.yml --ask-vault-pass

Vault can encrypt a whole file or just a single string value with ansible-vault encrypt_string, letting a sensitive value sit inline in an otherwise plaintext file. At runtime, ansible-playbook needs the vault password, supplied interactively, from a password file, or via a vault ID for setups using multiple different passwords, to decrypt the content in memory before use. This keeps secrets out of plaintext in the repository while still letting playbooks reference them like any other variable.

Tags let you label individual tasks, roles, or blocks so you can selectively run or skip parts of a playbook at execution time, instead of always running everything from top to bottom.

tasks:
  - name: Install packages
    ansible.builtin.package:
      name: nginx
    tags: [install]
  - name: Deploy configuration
    ansible.builtin.template:
      src: nginx.conf.j2
      dest: /etc/nginx/nginx.conf
    tags: [config]

ansible-playbook site.yml --tags config
ansible-playbook site.yml --skip-tags install

This is useful during iterative development or troubleshooting, where re-running an entire playbook (including slow steps like package installation) just to test a configuration change wastes time. Two special tags, always and never, give extra control: tasks tagged always run regardless of which tags were requested, and tasks tagged never are skipped unless explicitly requested by name.

ansible.cfg is Ansible's main configuration file, controlling default behavior like which inventory file to use, connection settings, privilege escalation defaults, and plugin paths, so these don't need to be repeated as command-line flags on every run.

[defaults]
inventory = ./inventory/production
remote_user = deploy
host_key_checking = False
forks = 20

[privilege_escalation]
become = True
become_method = sudo

Ansible looks for this file in a defined order of precedence: an ANSIBLE_CONFIG environment variable path, then ansible.cfg in the current directory, then ~/.ansible.cfg in the user's home directory, then a system-wide /etc/ansible/ansible.cfg. Only the first one found is used (settings aren't merged across multiple files), which is why teams typically keep one project-local ansible.cfg checked into version control alongside the playbooks it configures.

become tells Ansible to execute a task (or an entire play) as a different user, typically root, using a privilege escalation method like sudo, rather than requiring the SSH connection itself to log in as that user.

- name: Install a system package
  hosts: webservers
  become: true
  become_user: root
  become_method: sudo
  tasks:
    - name: Install nginx
      ansible.builtin.package:
        name: nginx
        state: present

become can be set at the play level (applying to every task), at an individual task level (for one-off privilege changes), or globally via ansible.cfg. By default it escalates to root, but become_user can target any account, and become_method supports alternatives like su or platform-specific mechanisms beyond sudo. This lets a playbook connect over SSH as a low-privilege deployment user while still performing root-level system changes only where explicitly needed.

Ansible Galaxy is both a public hub for sharing reusable roles and collections, and the ansible-galaxy command-line tool for installing them into a local project.

ansible-galaxy install geerlingguy.nginx
ansible-galaxy collection install community.docker

# requirements.yml
roles:
  - name: geerlingguy.nginx
collections:
  - name: community.docker
    version: ">=3.0.0"

Rather than writing a role for common tasks like installing nginx or PostgreSQL from scratch, teams can pull a well-maintained community role or vendor-published collection from Galaxy and layer their own configuration on top. A requirements.yml file lets a project declare exactly which roles/collections and versions it depends on, so ansible-galaxy install -r requirements.yml can reproducibly set up a fresh environment's dependencies.

Ansible was deliberately designed to avoid the operational overhead that agent-based tools carry: installing, upgrading, and keeping a background service running and secure on every single managed node, which itself becomes something that needs monitoring and patching over time.

Instead, Ansible pushes a small amount of Python code over an existing SSH (or WinRM) connection at the moment a task runs, executes it on the target, and then cleans up, leaving nothing persistent behind. Since SSH access is already how most Linux infrastructure is administered, there's no new listening service or network port to open, and no separate credential/agent lifecycle to manage. The trade-off is that Ansible depends on Python being present on managed nodes (for most modules) and on a reachable SSH connection, so extremely locked-down or agent-only environments (or very large fleets with strict connection-per-second limits) sometimes still favor a persistent-agent model, but for the majority of infrastructure the agentless approach reduces both attack surface and operational burden.

Idempotency in Ansible comes from module design, not from the playbook or engine automatically detecting "nothing to do." Each module is written to first check the current state of the target (a file's contents, whether a package is installed, a service's running state) and only performs an action if that state doesn't already match what was requested.

- name: Ensure nginx is installed
  ansible.builtin.package:
    name: nginx
    state: present   # module checks first: is it already installed?

If the desired state is already true, the module reports ok (no change); if it had to act, it reports changed. This is why re-running the exact same playbook against a system already in the desired state produces no changes the second time. Idempotency depends on the module author having implemented this check correctly - a poorly written custom module or a raw shell command run via shell/command won't automatically be idempotent unless the playbook author adds explicit conditions (like creates: or a when guard) to replicate that check-before-act behavior.

Use import_tasks/import_playbook when the structure is fixed and known ahead of time, since static processing lets tools like --list-tasks show the full task list without running anything. Use include_tasks/include_role when the set of tasks to run genuinely depends on a runtime condition or needs to loop over a dynamic list, since only dynamic inclusion can evaluate that condition as the play unfolds rather than upfront.

Whenever the same module is applied repeatedly with only the argument values changing, a loop keeps the playbook shorter and easier to maintain than writing out a near-identical task for each item.

- name: Install several packages
  ansible.builtin.package:
    name: "{{ item }}"
    state: present
  loop:
    - nginx
    - git
    - curl

This is the right tool when the list of items is either fixed and short, or comes from a variable (including one populated dynamically from facts or a registered result), and each iteration performs conceptually the same action. Duplicating the task manually instead means every future change (adding a package, adjusting an option) has to be repeated in multiple places, which is exactly the kind of copy-paste maintenance burden loops exist to eliminate. For more complex iteration, such as looping over a list of dictionaries with multiple related values, loop with structured data or the older with_items/with_dict family handles that cleanly as well.

By default, when a task fails on a given host, Ansible immediately stops running further tasks against that specific host for the remainder of the play, while continuing normally on any other hosts that haven't failed. The host is marked as failed and, unless later corrected, is excluded from any remaining tasks in that play.

- name: A task that might fail but shouldn't stop the play
  ansible.builtin.command: /opt/scripts/check.sh
  ignore_errors: true

- name: Custom failure condition
  ansible.builtin.command: /opt/scripts/deploy.sh
  register: result
  failed_when: "'ERROR' in result.stdout"

ignore_errors: true lets the play continue on that host even after a failure, while failed_when lets you redefine what "failure" even means for a task, useful when a script's exit code doesn't perfectly line up with success/failure. Failed hosts are recorded in a .retry file by default, which can be fed back into --limit @site.retry to re-run the playbook only against the hosts that failed last time, rather than the whole inventory.

Because the same variable name can be set in role defaults, inventory, play vars, facts, and command-line extra vars simultaneously, Ansible defines a strict precedence order so the outcome is deterministic rather than depending on file-read order.

From lowest to highest precedence (simplified): role defaults, inventory file vars, inventory group_vars/host_vars, playbook vars, role/include vars, block/task vars, registered variables and facts, and finally command-line -e extra vars, which always win over everything else. This ordering exists so that a role can ship sensible fallback defaults (lowest precedence) that a consuming playbook or inventory can override, while still allowing an operator to force a specific value at runtime via -e regardless of what's set anywhere else in the codebase - useful for one-off overrides during troubleshooting without editing any files.

A hardcoded password or API key in a playbook or variable file gets committed to version control history permanently - even deleting it in a later commit doesn't remove it from earlier history that anyone with repo access (or a leaked clone) can still read. Secrets in plain YAML also tend to leak into console output, CI logs, and backups far more easily than a value that's deliberately protected.

- name: Bad - visible in logs and version control
  ansible.builtin.debug:
    msg: "Password is hunter2"

- name: Better - vault-encrypted and hidden from output
  ansible.builtin.mysql_user:
    password: "{{ vault_db_password }}"
  no_log: true

Ansible Vault exists specifically to solve this: it lets secrets live encrypted in the same repository as everything else, decrypted only in memory at runtime with a password the playbook itself never contains. Pairing vault-encrypted variables with no_log: true on tasks that touch sensitive values also prevents those values from appearing in Ansible's own console or log output during a run.

Ansible's push, agentless model suits teams that want to get started quickly without standing up new server infrastructure, and its YAML syntax is generally considered more approachable for people without a programming background. Chef and Puppet's pull-based, agent-driven model can be a better fit for very large, continuously self-healing fleets where nodes need to enforce their desired state on their own schedule rather than waiting for a push, though at the cost of more infrastructure and a steeper learning curve to operate.

When a task or play sets become: true, Ansible connects to the managed node with the normal login user first, then invokes the configured become_method (most commonly sudo) to re-execute the module as the target user, usually root, only for that specific task.

- hosts: dbservers
  become: true
  become_method: sudo
  become_user: root
  vars:
    ansible_become_password: "{{ vault_sudo_pass }}"

Credentials for privilege escalation (a sudo password, if required) can be supplied via --ask-become-pass interactively, a vault-encrypted variable, or configured passwordless sudo on the target hosts, which is the most common production setup since it avoids needing to store or prompt for a second password at all. Because become is scoped per task or play rather than for the whole SSH session, a playbook can run most tasks as an unprivileged user and only escalate for the specific steps that genuinely need elevated permissions, limiting the blast radius of any single task going wrong.

A static inventory file is fine when the set of managed hosts is small and changes rarely, since manually editing a text file is simple in that case. Dynamic inventory becomes valuable once hosts are created and destroyed frequently, such as in an auto-scaling cloud environment, where a static list would constantly go stale.

# aws_ec2 dynamic inventory plugin example
plugin: amazon.aws.aws_ec2
regions:
  - us-east-1
filters:
  tag:Environment: production

Dynamic inventory plugins query a live source, cloud provider APIs (AWS, Azure, GCP), a CMDB, or container orchestration platforms, at run time, and can group hosts automatically by attributes like tags, region, or instance type. This is the right choice whenever the "truth" about which hosts exist and their properties already lives somewhere else in the infrastructure, since keeping a static file manually in sync with that source is both extra work and a source of drift the moment someone forgets to update it.

• Increase forks (forks = 20 or higher in ansible.cfg) so more hosts are processed in parallel instead of the default 5.
• Disable unnecessary fact gathering (gather_facts: false) for plays that don't need host facts.
• Enable fact caching (e.g. to Redis or a JSON file) so repeated runs don't re-gather facts from scratch every time.
• Use pipelining (pipelining = True) to reduce the number of SSH round trips per task.
• Prefer the free strategy for independent hosts, so faster hosts aren't held up waiting for slower ones on every task.
• Batch related changes into fewer, well-designed modules rather than many small shell/command tasks, since built-in modules are generally more efficient than shelling out.

Most large gains come from parallelism (forks) and cutting unnecessary work (facts, redundant SSH connections) rather than from micro-optimizing individual task logic, so it's worth checking these settings before assuming a slow playbook needs a rewrite.

- name: Static file, unchanged everywhere
  ansible.builtin.copy:
    src: files/motd
    dest: /etc/motd

- name: Config that varies per host
  ansible.builtin.template:
    src: templates/app.conf.j2
    dest: /etc/app/app.conf

Use copy for content that's identical across every target, like a license file or a static binary. Use template whenever the destination file needs to embed values that differ by host or environment, such as a database hostname, port number, or feature flag, since Jinja2 templating ({{ variable }}, {% if %}, {% for %}) inside a .j2 file is only processed by the template module, not copy.

A service restart is disruptive - it causes a brief outage - so it should only happen when something that actually requires it has changed, like an updated configuration file. If a restart were written as a regular task, it would run every single time the playbook executes, regardless of whether anything actually changed, causing unnecessary service interruptions on every run.

By putting the restart in a handler and having the configuration-writing task notify it, Ansible only fires the restart when that specific task reports changed. If the config was already correct and nothing changed, the notify never fires and the handler never runs, keeping the service untouched. Handlers also naturally coalesce: even if several different tasks in a play all notify the same handler, it still runs only once, at the end of the play, rather than restarting the same service multiple times in a row for each individual change that triggered it.

Jinja2 is the templating engine Ansible uses to evaluate expressions like {{ variable }}, conditionals, and loops, both directly inside playbook YAML values and inside .j2 template files processed by the template module.

# Inline in a playbook
- name: Show a computed value
  ansible.builtin.debug:
    msg: "{{ ansible_facts['memtotal_mb'] // 1024 }} GB of RAM"

# Inside a .j2 template file
server_name {{ inventory_hostname }};
{% if enable_ssl %}
listen 443 ssl;
{% endif %}

Inside {{ }} expressions, Ansible supports the full range of Jinja2 features: arithmetic, string manipulation, and filters (like {{ my_list | length }} or {{ name | upper }}) that transform a value inline. Template files use the block-style {% %} syntax for control flow such as if/else and for loops, letting a single template generate different output per host based on that host's variables and facts, which is what makes the template module powerful for generating per-host configuration files from one shared source file.

A role is the older, simpler unit of reuse focused purely on tasks/handlers/templates for one job. A collection is the modern distribution format that can contain many roles plus custom modules, plugins, and inventory sources, all versioned and namespaced together (like community.docker or amazon.aws). In practice they're complementary rather than competing: a collection often contains one or more roles inside it, and most of Ansible's own built-in functionality now ships as the ansible.builtin collection.

A block groups related tasks together, and pairing it with rescue gives Ansible try/catch-like error handling: if any task inside the block fails, execution jumps to the rescue section instead of immediately failing the whole host.

- block:
    - name: Deploy new application version
      ansible.builtin.command: /opt/deploy.sh
  rescue:
    - name: Roll back to previous version
      ansible.builtin.command: /opt/rollback.sh
  always:
    - name: Notify deployment status
      ansible.builtin.debug:
        msg: "Deployment attempt finished"

Use this pattern whenever a sequence of steps needs a defined fallback if something partway through fails, such as rolling back a deployment, cleaning up a partially-created resource, or sending an alert before giving up on that host. The optional always section runs regardless of whether the block succeeded or the rescue was triggered, making it a natural place for cleanup or notification logic that must happen either way.

Check mode (--check) runs a playbook in a "dry run" fashion: Ansible connects to real hosts and asks each module to report what change would be made, without actually making that change on the target system.

ansible-playbook site.yml --check --diff

This works because well-written modules implement supports_check_mode logic internally: when check mode is active, the module performs its normal "what's the current state versus desired state" comparison, but stops short of the step that actually writes the change, instead reporting whether it would have reported changed. Adding --diff alongside --check shows a line-level diff for file-based changes, like what a config file would look like after templating. Not every module supports check mode equally well, particularly custom or older modules and raw shell/command tasks, which may report inaccurate results or need explicit check_mode: false/changed_when handling to behave sensibly during a dry run.

Ansible's core design point is that the machine running ansible-playbook, the control node, connects out to managed nodes on demand over SSH, executes what's needed, and then disconnects - there's no persistent server process that managed nodes must check in with or that the control node needs to keep running between playbook executions.

This is different from tools that require a central server managed nodes periodically poll (a Puppet Master or Chef Server), which itself becomes infrastructure that needs deploying, scaling, and keeping highly available. Because Ansible's control node can be practically any machine with SSH access and the ansible package installed, even a laptop, teams can start automating without provisioning new dedicated infrastructure first. For larger, team-oriented setups needing scheduling, RBAC, and a web UI, Ansible Automation Platform (AWX/Tower) adds an optional central server on top of core Ansible - but that's an added convenience layer, not a requirement for Ansible itself to function.

# Ad-hoc: quick one-off check
ansible webservers -m ansible.builtin.ping

# Playbook: repeatable, structured automation
ansible-playbook site.yml

Ad-hoc commands are ideal for quick, throwaway operations, like checking connectivity, restarting a service on a handful of hosts, or grabbing a fact, where writing a whole playbook would be overkill. ansible-playbook is the tool for anything that should be repeatable, reviewable, and version-controlled, since a playbook can express multi-step logic, handlers, conditionals, and roles that a single ad-hoc command simply can't.

1. Increase verbosity (-v, -vvv, or -vvvv) to see the exact module arguments and, at higher levels, the raw SSH/connection details.
2. Check the error message and module return values - most modules return a descriptive msg field explaining exactly what went wrong.
3. Register the task's output and debug it to inspect intermediate values when a downstream task behaves unexpectedly.
4. Run with --check --diff to see what change was expected, isolating whether the issue is in the logic or the actual execution.
5. Test connectivity separately with an ad-hoc ansible host -m ping to rule out SSH/auth issues before assuming the playbook logic is at fault.
6. Isolate with --limit and --tags to re-run just the failing host and task rather than the whole inventory and playbook.

ansible-playbook site.yml --limit web1.example.com --tags deploy -vvv

Most task failures fall into one of a few buckets - a connectivity/auth problem, a variable that resolved to something unexpected, or a module receiving arguments it doesn't accept in the current version - and narrowing which bucket applies with verbosity and targeted re-runs is usually faster than guessing from the top-level error alone.

Ansible's control node does the heavy lifting of assembling exactly what needs to run, then ships a minimal, self-contained payload to each target rather than relying on anything pre-installed there beyond Python.

For each task, the control node resolves all variables and Jinja2 expressions locally, then packages the relevant module's Python code together with its resolved arguments into a single file, copies that file to a temporary directory on the managed node over the existing SSH connection, and executes it there. The module runs, performs its idempotent check-then-act logic, and prints a JSON result back to the control node over the same channel, which the control node parses to decide whether the task succeeded, failed, or reported a change. The temporary payload is then removed from the managed node, leaving no persistent footprint - this repeats independently for every host (up to the configured fork limit) and every task in the play.

Running ansible-playbook moves through parsing, per-play host resolution, and then a repeated per-task cycle across every play in the file.

Ansible first parses the whole playbook and inventory, then processes each play in order: it resolves which hosts match the play's hosts: pattern, optionally gathers facts from them, and then executes each task in sequence against all matched hosts (in parallel, up to the fork limit) before moving to the next task. Once every task in a play has run, any handlers that were notified during that play fire, and only then does Ansible move on to the next play in the file. At the very end, it prints the recap - a per-host summary of ok/changed/unreachable/failed counts - giving a final overview of what happened across the entire run.

A single task moves through templating, module dispatch, execution, and result handling before Ansible considers it complete for a given host.

Before anything runs, Ansible evaluates the task's when condition (if any); if false, the task is marked skipped and nothing further happens for that host. Otherwise, module arguments are templated with Jinja2 using currently known variables and facts, and the module is dispatched to the host - respecting any become or delegate_to directives that change who executes it or where. The module runs its check-then-act logic and returns a result; if register is set, that result is stored in a variable for later tasks to reference, and if the task both notify-ed a handler and reported changed, that handler is queued to run at the end of the play. This entire sequence happens independently, per host, which is why a task can succeed on one host and fail on another within the same play.

Idempotency isn't an engine-level guarantee that Ansible enforces automatically on arbitrary code - it's a contract that well-written modules implement internally, typically structured as a "state comparison" pattern shared across most built-in modules.

# Pseudocode of typical idempotent module internals
current_state = inspect_target()
desired_state = module_arguments

if current_state == desired_state:
    return {"changed": False}
else:
    apply_changes(desired_state)
    return {"changed": True}

Internally, a module like package or file first queries the actual state of the target (is the package installed? does the file have these permissions?), compares that against the arguments it was given, and only calls the underlying system action (installing, chmod-ing, etc.) if there's an actual difference. This is why the Ansible documentation explicitly marks which modules are idempotent and which aren't - the guarantee lives in each module's implementation, not in the Ansible engine that calls it, so a custom or third-party module that skips this comparison step and always performs its action isn't idempotent no matter how it's invoked.

Fact gathering runs as an implicit task at the start of a play (unless disabled), dispatching the built-in setup module to every host in the play before any of the play's own tasks execute.

The setup module inspects the host directly, reading things like network interfaces, mounted filesystems, distribution and kernel version, and environment variables, and packages all of it into a nested ansible_facts dictionary. This happens once per host per play run and can be relatively slow on hosts with many network interfaces or filesystems, since it's doing real inspection work, not just returning a cached label. If fact caching is configured (to a JSON file, Redis, or another supported backend), gathered facts are also persisted there with a configurable expiration, so a subsequent playbook run - even a completely separate one - can reuse recently-gathered facts instead of re-running setup from scratch on every single execution.

• Raise forks well above the default to match available control-node CPU/network capacity, so more hosts run truly in parallel.
• Enable persistent fact caching (Redis or similar) so a large inventory doesn't re-gather facts on every single run.
• Use the free strategy for independent hosts so faster hosts aren't bottlenecked waiting for the slowest host on every task.
• Enable SSH pipelining and connection reuse (ControlPersist) to cut per-task SSH connection overhead significantly at scale.
• Split very large inventories into batches with serial, so a single run doesn't try to open thousands of simultaneous connections at once.
• Move to Execution Environments / Ansible Automation Platform for scheduling, job isolation, and horizontal scaling of control-node capacity itself when a single control node becomes the bottleneck.

At large scale, the bottleneck is usually connection and fact-gathering overhead multiplied across thousands of hosts rather than task logic itself, so most of these optimizations target reducing per-host overhead and increasing safe parallelism rather than rewriting playbooks.

Neither is universally better; each optimizes for a different failure mode when hosts vary in speed or reliability.

Choose linear when task ordering across the whole batch matters, for example a coordinated rolling deploy where you need every host at the same step before proceeding, or when predictable, synchronized output is more valuable than raw speed. Choose free when hosts are truly independent of each other and speed varies a lot, for example a large fleet of similar web servers, since a single slow or high-latency host under linear would otherwise stall every other host waiting at each task boundary - free lets fast hosts complete without waiting.

By default, Ansible processes only 5 hosts at a time (forks = 5); for each task, it must finish running against all currently-active hosts (up to that limit) before starting the next batch, so with a large inventory, execution time scales roughly with (number of hosts / forks) rather than shrinking automatically as inventory grows.

# ansible.cfg
[defaults]
forks = 50

Increasing forks raises how many hosts the control node processes concurrently per task, which directly cuts total wall-clock time for large inventories, since more hosts are being worked on in parallel rather than queued waiting for a slot to free up. The practical ceiling is the control node's own resources: each fork consumes CPU, memory, and a network connection, so forks should be raised in proportion to what the control node (and target network/SSH capacity) can actually sustain, rather than set arbitrarily high, since an overloaded control node can end up slower overall from resource contention than a more conservative fork count.

Automation that isn't idempotent is dangerous to re-run: if a task blindly creates a user, appends a line to a config file, or restarts a service every single time it executes regardless of current state, running the same playbook twice can create duplicate users, duplicate config lines, or unnecessary service restarts, turning routine re-runs into a source of drift or outages rather than a safe, predictable operation.

Idempotency is what makes Ansible's core workflow - "just run the playbook again" - safe as a default response to almost any situation, whether that's recovering from a partial failure, applying a playbook to a fleet that's in a mixed, unknown state, or simply re-verifying that a system still matches its intended configuration. Because idempotent tasks only report changed when something genuinely needed to happen, playbook output over time also becomes a meaningful signal: a run that reports many changes on a system that should already be configured correctly is itself useful information, suggesting drift, whereas a run reporting all ok confirms the system matches the desired state exactly. Without that guarantee, every re-run would carry real risk instead of being routine.

1. Time the gather_facts step specifically using -vvv or the profile_tasks callback to confirm it's actually the bottleneck versus other tasks.
2. Disable gathering for plays that don't need facts (gather_facts: false), which is often the single biggest win if facts aren't actually used.
3. Narrow gathered subsets with gather_subset to skip expensive categories (like all network interfaces) when only a few specific facts are needed.
4. Enable fact caching so repeated runs within the cache's expiration window skip live gathering entirely and reuse the last known values.
5. Check for slow or high-latency hosts specifically - a handful of consistently slow hosts (not the whole inventory) often points to a host-level issue, like DNS resolution delays during setup, rather than an Ansible-wide problem.

- hosts: all
  gather_facts: true
  vars:
    ansible_facts_modules: setup
  tasks:
    - name: Only gather what's needed
      ansible.builtin.setup:
        gather_subset:
          - network

Fact gathering is one of the more overlooked sources of playbook slowness precisely because it's implicit - it's worth explicitly timing it before assuming a slow run is due to the playbook's own task logic.

Internally, Ansible builds up a variable's final value by layering every source that defines it, in a fixed precedence order, so the value that "wins" is whichever source sits highest in that order, regardless of the order files happen to be read from disk.

As Ansible processes a playbook, it maintains an internal merged variable namespace per host; each layer above simply overwrites the same key if it's also defined at a lower layer, rather than merging values together (with the notable exception of hash/dictionary merging, which can be configured via hash_behaviour, though the default is still overwrite, not merge). Because -e extra vars sit at the very top, they always win regardless of what's set anywhere else in roles, inventory, or playbooks - this is deliberate, since it gives an operator a reliable way to force a value at runtime without needing to trace through every file that might otherwise define it.

Even if several different tasks across a play all call notify on the same handler name, and each of those tasks independently reports changed, Ansible still only executes that handler once, at the point handlers run (by default, at the end of the play).

tasks:
  - name: Update main config
    ansible.builtin.template:
      src: app.conf.j2
      dest: /etc/app/app.conf
    notify: Restart app

  - name: Update feature flags
    ansible.builtin.template:
      src: flags.conf.j2
      dest: /etc/app/flags.conf
    notify: Restart app

handlers:
  - name: Restart app
    ansible.builtin.service:
      name: app
      state: restarted

Internally, Ansible tracks notified handlers as a de-duplicated set per host rather than a queue that fires once per notification, which is exactly the behavior you want: a service that needed to pick up two separate config changes only needs one restart to reflect both, not two back-to-back restarts. This also means handler order is determined by where the handler is defined (in handlers/main.yml or the playbook's handlers section), not by which task happened to notify it first or last.

Fact caching stores the ansible_facts dictionary gathered for each host in an external cache backend, keyed by hostname, with a configurable time-to-live, so a future run can read cached facts instead of re-running the setup module.

# ansible.cfg
[defaults]
fact_caching = redis
fact_caching_connection = localhost:6379:0
fact_caching_timeout = 86400

Supported backends include a simple jsonfile (facts written to local disk as JSON), redis, and other plugin-based options, chosen based on whether facts need to be shared across multiple control nodes (favoring a networked backend like Redis) or just persisted locally between runs (where jsonfile is simpler). On each play, Ansible checks whether a valid, non-expired cache entry exists for a given host; if so, it loads facts from there instead of dispatching setup, and if not (or if the entry expired past fact_caching_timeout), it gathers fresh facts and writes the result back to the cache for next time.

serial splits a play's target hosts into smaller batches and runs the entire play (all its tasks and handlers) to completion for one batch before starting the next, instead of running each task against every host at once.

- hosts: webservers
  serial: 2
  max_fail_percentage: 20
  tasks:
    - name: Deploy new app version
      ansible.builtin.command: /opt/deploy.sh
    - name: Health check
      ansible.builtin.uri:
        url: "http://{{ inventory_hostname }}/health"
      register: health
      until: health.status == 200
      retries: 5

With serial: 2, only 2 hosts are taken through the whole play at a time - deploy, health check, and any handlers - before the next 2 hosts start, keeping the majority of the fleet serving traffic throughout the rollout. max_fail_percentage adds a safety valve: if failures across completed batches exceed that threshold, Ansible stops starting new batches rather than continuing to roll a broken change out to the entire fleet. serial can also take a list (e.g. [1, 5, "100%"]) to start with a small canary batch and progressively widen, which is a common pattern for cautious production rollouts.

• Use run_once for cluster-wide actions - a task that only needs to happen once across the whole play (like registering a load balancer entry) shouldn't repeat per host.
• Cache expensive lookups - if a role queries an external API or database for shared data, fetch it once (e.g. via delegate_to on a single host, or run_once) and share the result via a fact or registered variable, rather than every host repeating the same external call.
• Structure role dependencies carefully in meta/main.yml - without care, the same dependency role can run multiple times if several roles in a play each declare it as a dependency; set allow_duplicates: false where repetition isn't needed.
• Use group_vars/host_vars instead of per-role redundant variable definitions so shared configuration isn't copy-pasted into every role that needs it.
• Leverage fact caching so roles across many plays in the same run (or repeated runs) aren't each triggering their own full fact gathering pass.

The common thread is identifying work that's logically "once per play/run" rather than "once per host" and making that explicit with the right directive, instead of letting Ansible's default per-host execution model repeat genuinely shared work unnecessarily across every target.