Ansible Core Wiki

Overview

Ansible is a radically simple IT automation system that handles configuration management, application deployment, cloud provisioning, ad-hoc task execution, network automation, and multi-node orchestration. This repository contains ansible-core, the foundation of the Ansible automation platform.

What is Ansible?

Ansible enables infrastructure automation through a simple, agentless architecture. It uses SSH for communication with remote systems, requires no additional software installation on target machines, and describes infrastructure in human-readable YAML playbooks. The current development version is 2.21.0.dev0, targeting Python 3.12 and later.

Core Design Principles

Ansible is built on these foundational principles:

• Agentless architecture - Leverages existing SSH daemon, no custom agents or open ports
• Simple setup - Minimal learning curve with straightforward installation
• Parallel execution - Manages machines quickly and concurrently
• Human-friendly syntax - Infrastructure described in readable YAML
• Security-focused - Easy auditability and review of automation content
• Language-agnostic - Module development in any dynamic language

Repository Structure

lib/ansible/
├── cli/              # Command-line interfaces (ansible, ansible-playbook, etc.)
├── executor/         # Task execution engine and strategies
├── inventory/        # Inventory management and host/group definitions
├── modules/          # Core built-in automation modules
├── module_utils/     # Shared utilities for modules
├── plugins/          # Plugin framework (filters, tests, lookups, etc.)
├── playbook/         # Playbook parsing and execution logic
├── parsing/          # YAML and data parsing
├── config/           # Configuration management
├── galaxy/           # Collections and role management
├── vars/             # Variable management
└── utils/            # Utility functions and helpers

Key Components

CLI Layer (lib/ansible/cli/) provides entry points for all Ansible commands: ansible, ansible-playbook, ansible-galaxy, ansible-vault, and others. Each CLI tool handles argument parsing and dispatches to the appropriate execution engine.

Executor (lib/ansible/executor/) contains the core execution engine that runs tasks and plays. It manages task queues, handles parallel execution, and coordinates with remote systems through connection plugins.

Modules (lib/ansible/modules/) are the units of work in Ansible. They perform specific actions on target systems (e.g., package installation, file management, service control). Modules run on remote hosts and return structured JSON results.

Plugins (lib/ansible/plugins/) provide extensibility through multiple plugin types: action plugins, connection plugins, filters, lookups, strategies, and more. Plugins execute locally in the Ansible process.

Inventory (lib/ansible/inventory/) manages host and group definitions from various sources (static files, dynamic scripts, cloud providers).

Development Requirements

• Python version: 3.12 or later (controller code)
• License: GPLv3 compatible for ansible-core; BSD-2-Clause for lib/ansible/module_utils/
• Testing: Uses ansible-test for sanity, unit, and integration tests
• Branch model: devel branch for active development; stable-2.X branches for releases

Getting Started

Install ansible-core from PyPI:

pip install ansible-core

Or run the development version directly:

pip install -e .

For development, set up your environment and run tests:

ansible-test sanity -v --docker default
ansible-test units -v --docker default

Architecture & Execution Flow

Overview

Ansible's execution flow follows a hierarchical pipeline: playbooks are parsed into plays, plays are decomposed into tasks, and tasks are executed across inventory hosts using a pluggable strategy system. The architecture emphasizes parallelism through worker processes while maintaining flexibility through strategy plugins that control task scheduling.

Execution Pipeline

Loading diagram...

Key Components

PlaybookExecutor (lib/ansible/executor/playbook_executor.py) is the entry point for ansible-playbook. It loads playbook files, creates a TaskQueueManager, and iterates through plays, handling serialization (batching hosts) and managing overall execution state.

TaskQueueManager (lib/ansible/executor/task_queue_manager.py) orchestrates play execution. It initializes worker processes (up to the fork limit), loads the strategy plugin, creates a PlayIterator, and manages the FinalQueue for collecting results and callbacks from worker processes.

PlayIterator (lib/ansible/executor/play_iterator.py) maintains per-host execution state through HostState objects. It tracks which block, task, and phase (setup, tasks, rescue, always, handlers) each host is in, enabling the strategy to determine the next task for each host. States include SETUP, TASKS, RESCUE, ALWAYS, HANDLERS, and COMPLETE.

Strategy Plugins (lib/ansible/plugins/strategy/) control task scheduling. The linear strategy (default) keeps hosts in lockstep—all hosts execute the same task before moving to the next. The free strategy allows hosts to progress independently, maximizing parallelism. Both inherit from StrategyBase and implement a run() method that loops through tasks, queuing them via _queue_task().

WorkerProcess (lib/ansible/executor/process/worker.py) is a multiprocessing worker that executes a single task on a single host. It instantiates a TaskExecutor, runs it, and sends results back through the FinalQueue.

TaskExecutor (lib/ansible/executor/task_executor.py) handles task execution logic: evaluating loops, templating variables, loading action plugins, and executing the task. It returns a result dictionary that includes status, changed flag, and output.

Data Flow

1. Parsing: PlaybookExecutor loads YAML playbooks into Playbook objects containing Play objects.
2. Iteration Setup: TaskQueueManager.run() creates a PlayIterator initialized with all blocks (tasks, rescue, always, handlers) for the play.
3. Task Scheduling: The strategy plugin calls iterator.get_next_task_for_host() to determine the next task for each host, respecting the host's current state.
4. Task Queuing: _queue_task() spawns a WorkerProcess for each (host, task) pair, up to the fork limit.
5. Execution: WorkerProcess._run() creates a TaskExecutor and executes the task, sending results to FinalQueue.
6. Result Processing: The strategy reads from FinalQueue, updates host states in the iterator, and triggers callbacks (e.g., v2_runner_on_ok, v2_runner_on_failed).
7. State Advancement: After processing results, the strategy loops back to step 3 until all hosts reach IteratingStates.COMPLETE.

Parallelism & Throttling

Worker processes are pooled and reused. The number of workers equals min(forks, batch_size). Tasks can be throttled via the throttle keyword, which limits concurrent execution. The run_once keyword executes a task on a single host only. Handlers are flushed at the end of each block or when explicitly triggered via meta: flush_handlers.

CLI Interface & Entry Points

Ansible's CLI system provides a unified framework for all command-line tools (ansible, ansible-playbook, ansible-vault, etc.). The architecture uses an abstract base class with specialized subclasses for each tool.

Base CLI Architecture

The CLI abstract base class in lib/ansible/cli/__init__.py defines the core interface that all CLI tools implement. Each CLI tool must override three key methods:

• init_parser() - Creates and configures the argument parser with tool-specific options
• post_process_args() - Validates and transforms parsed arguments before execution
• run() - Executes the actual command logic

The base class handles common concerns: Python version checking, locale initialization, blocking I/O validation, and error handling.

Execution Flow

Loading diagram...

CLI Subclasses

Each Ansible tool is implemented as a CLI subclass:

• PlaybookCLI (ansible-playbook) - Executes playbooks via PlaybookExecutor
• AdHocCLI (ansible) - Runs single tasks via TaskQueueManager
• VaultCLI (ansible-vault) - Encrypts/decrypts files with subcommands (encrypt, decrypt, view, etc.)
• GalaxyCLI (ansible-galaxy) - Manages roles and collections with subcommands
• InventoryCLI (ansible-inventory) - Displays inventory in various formats
• ConfigCLI (ansible-config) - Views and manages configuration

Entry Points

Each tool in bin/ is a simple script that imports its corresponding CLI class and calls cli_executor():

from ansible.cli.playbook import PlaybookCLI

if __name__ == '__main__':
    PlaybookCLI.cli_executor()

The cli_executor() class method handles initialization, error handling, and exit codes. It creates the CLI instance, calls run(), and catches exceptions to provide user-friendly error messages.

Argument Processing

Arguments flow through three stages:

1. Parsing - init_parser() defines options using option_helpers utilities
2. Post-processing - post_process_args() validates and transforms options
3. Context - Parsed arguments stored in context.CLIARGS for global access

Common options are added via helper functions like add_inventory_options(), add_vault_options(), and add_connect_options().

Playbook & Task System

The playbook system is the core abstraction layer that transforms YAML playbook definitions into executable task sequences. It provides a hierarchical object model with validation, inheritance, and composition patterns.

Core Hierarchy

The system follows a strict parent-child hierarchy:

• Play – Top-level container representing a set of hosts and their tasks/roles
• Block – Groups tasks with optional error handling (block/rescue/always sections)
• Task – Individual module invocation with arguments and control flow
• Handler – Special task subclass triggered by task notifications
• Role – Reusable task collection with metadata and dependencies

Loading diagram...

FieldAttributeBase System

All playbook objects inherit from FieldAttributeBase, which provides declarative attribute management through FieldAttribute and NonInheritableFieldAttribute descriptors. This system handles:

• Type validation and coercion
• Default value assignment
• Inheritance from parent objects
• Post-validation templating
• Priority-based loading order

Attributes are defined as class-level descriptors and automatically processed during load_data().

Block Structure

Blocks implement Ansible’s error handling model with three task lists:

• block – Main tasks (executed first)
• rescue – Tasks run if any block task fails
• always – Tasks run regardless of success/failure

Blocks inherit from multiple mixins: Conditional, Taggable, Notifiable, and Delegatable, enabling conditional execution, tag filtering, handler notifications, and host delegation.

Task Execution Flow

Tasks support advanced control mechanisms:

• Loops – Iterate via loop attribute with LoopControl configuration
• Retries – Automatic retry with retries and until conditions
• Async – Non-blocking execution with polling
• Conditionals – when clauses for skipping
• Registration – Store results in variables via register

Play Compilation

The Play.compile() method transforms the declarative structure into an executable block list:

1. Compiles role dependencies recursively
2. Merges pre-tasks, roles, tasks, and post-tasks
3. Injects flush_handlers meta-tasks between sections
4. Handles force_handlers mode for guaranteed handler execution

This compiled list is then iterated by the executor, which manages host batching, strategy selection, and result aggregation.

Plugin Architecture & Extensibility

Ansible's plugin system is the foundation for extensibility, allowing developers to customize behavior across execution, connectivity, output, and data transformation. All plugins inherit from AnsiblePlugin, a base class that provides configuration management, option handling, and metadata tracking.

Plugin Types & Architecture

Ansible provides 12+ plugin types, each serving a distinct purpose:

• Action plugins (lib/ansible/plugins/action/) - Execute task logic on the controller before delegating to modules
• Connection plugins (lib/ansible/plugins/connection/) - Manage communication with remote hosts (SSH, WinRM, local, etc.)
• Strategy plugins (lib/ansible/plugins/strategy/) - Control playbook execution flow (linear, free, debug, etc.)
• Callback plugins (lib/ansible/plugins/callback/) - Process and display task results
• Filter plugins (lib/ansible/plugins/filter/) - Transform data in Jinja2 templates
• Lookup plugins (lib/ansible/plugins/lookup/) - Retrieve data from external sources
• Inventory plugins (lib/ansible/plugins/inventory/) - Parse inventory sources
• Become plugins - Privilege escalation (sudo, su, runas)
• Cache plugins - Store facts and variables
• Shell plugins - Execute commands on remote systems
• Test plugins - Conditional tests in templates
• Vars plugins - Load variables dynamically

Plugin Loading & Discovery

The PluginLoader class (loader.py) manages plugin discovery and instantiation. It searches plugin paths in order:

1. Playbook-relative directories
2. Configured paths (from ansible.cfg)
3. Python path and built-in plugins

class PluginLoader:
    def __init__(self, class_name, package, config, subdir, aliases=None):
        self.class_name = class_name
        self.package = package
        self.subdir = subdir
        self._module_cache = {}
        self._plugin_path_cache = {}

Plugins are cached after first load to avoid repeated filesystem searches. The loader resolves fully-qualified collection names (FQCNs) like community.general.ssh to actual plugin files.

Base Plugin Class & Configuration

All plugins inherit from AnsiblePlugin, which provides:

• Option management - get_option(), set_options() for plugin configuration
• Plugin metadata - _load_name, ansible_name, _original_path for tracking
• Configuration origin tracking - Know whether options came from config files, variables, or direct assignment

class AnsiblePlugin(metaclass=abc.ABCMeta):
    def __init__(self):
        self._options = {}
        self._origins = {}
    
    def get_option(self, option, hostvars=None):
        # Retrieve plugin configuration with fallback to defaults
        pass

Specialized Plugin Base Classes

Each plugin type extends AnsiblePlugin with type-specific functionality:

• ActionBase - Handles file transfer, module execution, connection management
• ConnectionBase - Manages host connectivity, authentication, command execution
• StrategyBase - Controls task iteration, queuing, and result processing
• CallbackBase - Processes task results, handles display formatting
• LookupBase - Provides data retrieval with template and loader context
• InventoryBase - Parses inventory sources and populates host/group data

Plugin Discovery & Resolution

Loading diagram...

The loader resolves plugin names through multiple strategies: exact FQCN matches, collection searches, and legacy name resolution. Redirects and deprecations are tracked to guide users toward current plugin names.

Extending Ansible

To create a custom plugin:

1. Inherit from the appropriate base class (e.g., ActionBase, ConnectionBase)
2. Implement required abstract methods
3. Add plugin metadata via DOCUMENTATION and EXAMPLES YAML blocks
4. Place in a plugin directory: plugins/action/, plugins/connection/, etc.
5. Reference via FQCN: namespace.collection.plugin_name

Plugins support configuration through ansible.cfg, environment variables, and playbook keywords, making them highly customizable for different environments and use cases.

Inventory & Variables Management

Ansible's inventory and variables system forms the foundation for host management and dynamic configuration. The inventory defines which hosts to manage, while the variables system provides the data that drives playbook execution.

Inventory Architecture

The InventoryManager (lib/ansible/inventory/manager.py) orchestrates inventory loading and parsing. It accepts multiple sources (files, directories, or dynamic scripts) and delegates parsing to specialized inventory plugins. The manager maintains an InventoryData object containing all hosts and groups.

# Core inventory structure
class InventoryManager:
    def __init__(self, loader, sources=None, parse=True, cache=True):
        self._inventory = InventoryData()  # Holds hosts and groups
        self._sources = sources  # Inventory sources to parse
        if parse:
            self.parse_sources(cache=cache)

The InventoryData class stores hosts and groups in dictionaries, with special handling for the implicit all and ungrouped groups. It provides methods to add/remove hosts and groups, and maintains group hierarchies through parent-child relationships.

Host and Group Objects

Host objects represent individual managed nodes. Each host maintains:

• A unique name and address
• A dictionary of variables (host.vars)
• Group memberships (host.groups)
• Magic variables like inventory_hostname and group_names

Group objects represent collections of hosts with:

• Child and parent group relationships (supporting nested hierarchies)
• Group-level variables
• Depth tracking to detect circular dependencies

# Host magic variables
def get_magic_vars(self):
    return {
        'inventory_hostname': self.name,
        'inventory_hostname_short': self.name.split('.')[0],
        'group_names': sorted([g.name for g in self.get_groups()])
    }

Variable Resolution Pipeline

The VariableManager (lib/ansible/vars/manager.py) implements Ansible's variable precedence system. When get_vars() is called with a host context, it combines variables in this order:

1. Role defaults (if play context exists)
2. Group variables from group_vars/ files
3. Host variables from host_vars/ files
4. Fact cache (gathered facts from previous tasks)
5. Play variables
6. Task variables
7. Extra variables (command-line --extra-vars)
8. Magic variables (computed at runtime)

# Simplified precedence flow
all_vars = {}
all_vars.update(role_defaults)
all_vars.update(group_vars)
all_vars.update(host_vars)
all_vars.update(facts)
all_vars.update(play_vars)
all_vars.update(task_vars)
all_vars.update(extra_vars)
all_vars.update(magic_vars)

Variable Sources and Plugins

Variables come from multiple sources:

• Inventory plugins parse inventory files and set host/group variables
• Vars plugins load variables from group_vars/ and host_vars/ directories adjacent to inventory sources
• DataLoader handles YAML/JSON parsing with vault decryption support
• Templar (template engine) processes Jinja2 expressions in variable values

The HostVars class provides on-demand templating of host variables, allowing playbooks to access other hosts' variables via hostvars[hostname].

Data Flow Diagram

Loading diagram...

Key Concepts

Reconciliation: After parsing all inventory sources, reconcile_inventory() ensures consistency—all groups inherit from all, circular dependencies are detected, and group hierarchies are validated.

Caching: The InventoryManager caches parsed patterns and host lookups. Vars are cached per host to avoid redundant resolution during task execution.

Trust and Templating: Variables from inventory plugins are inspected for template expressions. The trusted_by_default flag on plugins determines whether values are treated as safe templates or literal strings.

Modules & Module Utilities

Modules are the fundamental units of work in Ansible. They are self-contained programs (typically Python) that perform specific tasks on managed nodes. Module utilities provide shared code that modules import to avoid duplication and ensure consistency.

Module Structure

Every Ansible module follows a standard structure:

1. Documentation Block - YAML metadata describing the module, its options, and return values
2. Examples Block - Usage examples for users
3. Return Block - Documentation of output values
4. Main Function - The actual implementation logic
5. Entry Point - if __name__ == '__main__': main()

DOCUMENTATION = """
---
module: command
short_description: Execute commands on targets
options:
  cmd:
    description: The command to run
    type: str
"""

EXAMPLES = """
- name: Run a command
  ansible.builtin.command:
    cmd: /usr/bin/whoami
"""

RETURN = """
stdout:
  description: Standard output
  type: str
"""

from ansible.module_utils.basic import AnsibleModule

def main():
    module = AnsibleModule(
        argument_spec=dict(cmd=dict(type='str', required=True)),
        supports_check_mode=True
    )
    # Implementation here
    module.exit_json(changed=False, stdout='result')

if __name__ == '__main__':
    main()

AnsibleModule Class

The AnsibleModule class in lib/ansible/module_utils/basic.py is the core interface for Python modules. It handles:

• Argument Parsing - Validates and coerces module parameters against an argument_spec
• Check Mode - Allows modules to preview changes without applying them
• Logging & Output - Manages module result serialization and error reporting
• File Operations - Provides utilities for safe file manipulation
• Subprocess Execution - Wraps subprocess calls with proper error handling

Key methods include exit_json() (success), fail_json() (failure), and run_command() (execute shell commands).

Module Utilities Organization

Module utilities are organized by function:

• basic.py - Core AnsibleModule and common functions
• common/arg_spec.py - Argument validation and coercion
• common/parameters.py - Parameter handling (aliases, defaults, validation)
• common/validation.py - Type checking and constraint validation
• facts/ - System fact collectors for hardware, network, packages
• urls.py - HTTP request utilities
• service.py - Service management helpers
• connection.py - Network device connection management

Module Execution Pipeline

When a module is executed, Ansible performs these steps:

1. Discovery - Locate the module file in lib/ansible/modules/ or collections
2. Compilation - Parse module code and identify dependencies on module_utils
3. Dependency Resolution - Recursively collect all imported module_utils
4. AnsiballZ Packaging - Bundle module code and utilities into a ZIP archive
5. Transfer - Send the ZIP to the remote host
6. Execution - Remote Python unpacks and runs the module
7. Result Collection - Module outputs JSON result back to controller

The modify_module() function in lib/ansible/executor/module_common.py orchestrates this process, using ModuleDepFinder to analyze imports and build the final payload.

Collections and Module Discovery

Modules can be located in:

• Built-in - lib/ansible/modules/ (ansible-core)
• Collections - ansible_collections/namespace/collection/plugins/modules/

Collection modules can import from collection-specific module_utils:

from ansible.module_utils.basic import AnsibleModule
from ..module_utils.my_util import helper_function

Relative imports allow collections to maintain their own utility libraries separate from core.

Testing & Quality Assurance

Ansible uses a comprehensive three-tier testing framework managed by the unified ansible-test tool. This infrastructure ensures code quality, correctness, and compatibility across multiple Python versions and platforms.

Testing Tiers

Sanity Tests perform static analysis and linting on the codebase. These include:

• Code-smell tests: Custom Python scripts checking for common issues (boilerplate, deprecated patterns, unwanted files)
• Linting: pylint, pep8, black, mypy for type checking
• Module validation: Checks module documentation, argument specs, and metadata
• YAML validation: pymarkdown and yamllint for documentation and config files

Run with: ansible-test sanity -v --docker default

Unit Tests use pytest to test individual components in isolation. Located in test/units/, they mirror the lib/ansible/ structure. Key features:

• Pytest configuration in test/lib/ansible_test/_data/pytest/config/default.ini with xfail_strict=true
• Fixtures in test/units/conftest.py provide collection loaders and context mocking
• Tests run across multiple Python versions (3.9 through 3.14)
• Parallel execution with pytest-xdist (-n auto)
• Coverage reporting and JUnit XML output

Run with: ansible-test units -v --docker default

Integration Tests validate functionality end-to-end using playbooks and scripts. Organized in test/integration/targets/:

• Each target is a self-contained test directory with aliases, tasks/, runme.sh, or runme.yml
• Tests run on actual or containerized systems (Ubuntu 2404, Windows, etc.)
• Supports role-based tests, playbook-based tests, and custom shell scripts
• Modules execute on target hosts; plugins execute locally in the controller

Run with: ansible-test integration -v --docker ubuntu2404

CI Pipeline

Azure Pipelines orchestrates testing across stages:

• Sanity Stage: Split into two groups (heavy tests like pylint in group 2, others in group 1)
• Units Stage: Runs across Python 3.9–3.14
• Windows Stage: Tests WinRM and PSRP connections on Windows Server 2016–2025
• Integration Stage: Platform-specific containers for comprehensive testing

Coverage data aggregates across all stages for reporting.

Key Testing Patterns

Use --docker default for sanity and unit tests (isolated, reliable). Use --docker ubuntu2404 (or other distros) for integration tests, never --docker default. The --venv option provides a fallback when containers are unavailable but is less reliable for unit tests.

Tests can target specific files or use --changed to run only tests affected by recent changes. The --coverage flag enables code coverage tracking across test runs.