*.pyc
*.retry
/playbooks/host-keys
ansible.log

This is a suggested minimal ansible.cfg, though you might want to start from an ansible example for the included comments:

[defaults]
inventory      = site.yaml
gather_timeout = 30
roles_path    = local_roles:external_roles:roles
inventory_plugins     = roles/data-utilities/inventory_plugins
filter_plugins     = roles/ansible-data-utilities/filter_plugins

[inventory]
enable_plugins = site_yaml
unparsed_is_failed = True

[ssh_connection]
pipelining = True
scp_if_ssh = smart

[site_yaml]
dynamic_groups = True

The following settings for the default section often also make sense:

remote_tmp     = ~/.ansible/tmp
local_tmp      = ~/.ansible/tmp
forks          = 20
gathering      = smart

A list of files containing group variables, which typically at least should contain all.yml. See Ansible inventory introduction for more details.

While files without file extension work using either .yml or .yaml is strongly recommended.

A symlink to a directory of checked out roles or unpacked roles. If strict version checking is not required this is an easy approach for simple setups as well as for development. For controlled setups the local_roles approach described in the next section typically is better.

A placeholder directory for temporarily storing or linking roles to for development. It can be created empty with a placeholder README similar to the following:

This directory is for holding roles during development, before importing a new
release into the roles subfolder.

This directory can either hold subdirectories of direct git clones, or symlinks
to role directories. Typically having a clone of the upstream role somewhere,
and just linking to it is the easier way:

  ln -s /path/to/roles/sample-role .

or from the ansible root:

  ln -s /path/to/roles/sample-role local_roles/

Don't forget to remove links/directories when importing a newly released role!

This directory contains the infrastructure playbooks and related files.

This directory holds roles versioned together with the Ansible configuration. There are two approaches for managing this:

• synchronising from a git source tree or a release tarball
• adding roles as git submodules

The submodule approach is recommended.

A README in the preferred markup format for the project with at least basic information on how to run Ansible and update roles is recommended.

The site configuration. See the inventory plugin documentation for details.

On macOS images without custom provisioning remote login needs to be enabled. For modern macOS versions this typically also requires granting full disk access privileges:

% sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist
% sudo systemsetup -setremotelogin on

Ansible will use python3, which is not installed per default, and comes with the Command Line Developer Tools. If not installed a first Ansible run may fail with an interactive prompt on the desktop to install those. After finishing the installer the Ansible run should succeed.

The easiest way for managing Windows via Ansible is by just making sure SSH is available on the Windows server. This can be done by a custom scripted Windows installation, or by following this section.

The deploy-ssh powershell script can be used both for installing SSH during an unattended Windows installation (with the SSH installer provided on the media), as well as enable it later on.

It will search for the installer OpenSSH-Win64.zip in c:\ci, the current directory and in the current users Download directory, in that order. If not found it will try to download it from the Win32-OpenSSH release page. It should be downloaded and executed in a powershell session with elevated privileges:

> Invoke-WebRequest -Uri https://raw.githubusercontent.com/aardsoft/ansible-data-utilities/master/doc/deploy-ssh.ps1 -OutFile deploy-ssh.ps1
> ./deploy-ssh.ps1

If the execution policy for scripts has not been changed from the default it may be necessary to bypass it for running the script:

> powershell -executionpolicy bypass -File ./deploy-ssh.ps1

On old Windows versions it may be required to force Powershell to use recent TLS mechanisms for the download - set the following if above throws SSL errors:

> [Net.ServicePointManager]::SecurityProtocol = "tls12, tls11, tls"

If using RDP to connect to the server the connection may get terminated while running deploy-ssh.ps1, though SSH access should be reachable from that point:

$ ssh -o PubkeyAuthentication=no Administrator@windows-system

Depending on the Windows version it may now be possible to also run an access playbook, or manual setup of SSH keys may be required. Running the playbook will handle setting up authorized keys as on other platforms - when manually installing an authorized_keys file for access setup note that the default configuration uses ProgramData\ssh\administrators_authorized_keys for any users in the administrators group. For other users the location is in .ssh\authorized_keys in the root of their home directory.

Creating a custom image allows adding or removing components to the regular Windows installation, and pre-load scripts required for taking over a host by Ansible.

The windows-image powershell script can be used to mostly automate image creation. While it will ask for any missing parameters it is recommended to create an answer file. The following example contains all possible variables:

$iso_path="c:\Users\example\Documents\Windows.iso"
$new_iso_path="c:\Users\example\Documents\Windows_unattended.iso"
$virtio_iso_path="c:\Users\example\Documents\virtio.iso"
$image_path="c:\unattended_image"
$ci_source_path="c:\ci"
$ci_target_dir="\ci"
$autounattend_path="c:\Autounattend.xml"
$os_index=6
$wait_for_manual=$true
$remove_apps=@(
  "Microsoft.BingWeather"
  "Microsoft.Getstarted"
)

It should be saved as powershell script (with ending .ps1), and can be passed to the windows-image script either as first argument, or through the environment variable CI_ISO_PARAMETERS.

autounattend_path is the path to a that can be used as Autounattend.xml for an unattended installation. If this variable is not set, but the CI script directory described below contains Autounattend.xml this will be used without a need for configuration.

os_index is the index of the Windows variant to be used for multi variant source images - if unspecified the script will list all available variants before prompting. The specified index will be converted to a single variant install.wim. This setting is only relevant on Windows media containing an install.esd file - for newer media shipping a multi variant install.wim the file will be used directly, and variant selection happens at installation time.

remove_apps is a list of preinstalled Windows applications that should be removed from the image. The list of applications in mount_path can be obtained with Get-AppxProvisionedPackage -path mount-path|Format-Table.

wait_for_manual makes the script pause before unmounting boot and install images, allowing manual injection of components.

ci_source_path is the path to a directory containing the files to be injected into the image. A detailed suggestion on how to use this directory is in the section below.

ci_target_dir is the path to a directory relative to the mountpoint of the install media to place the files. \ci would place them at the root of the install drive after installation (typically c:\ci)

virtio_iso_path is the path to an ISO image containing virtio drivers. If this is specified (and exists) all drivers on this ISO will be injected into the new image. Newer Windows versions require signed drivers - it is recommended to use WHQL signed drivers. One option for this is to use the drivers extracted from rocky linux.

The script expects a Windows ISO file, which can be created by

• downloading a Wndows 11 ISO
• using the Windows 11 media creation tool
• downloading Windows 11 images through the Windows insider program
• download Windows images through uupdump

At the time of writing Arm64 images can only be obtained via the insider program or uupdump. The same four options exists for Windows 10, links should be trivially obtainable by searching the microsoft site.

Creating an unattended Windows image requires following the stops for creating a custom Windows image in the section above first.

Next the Windows ADK needs to be installed for creating an answer file. Note that creating Windows 10 or 32-bit images may require using legacy ADK versions. Selecting Deployment Tools in the features list should be sufficient.

This include handles management of SSL keys and certificates. It can pull keys or certificates from a password store, or generate missing files.

It takes the following variables:

• ssl_size, the size of the SSL key. Defaults to 2048
• ssl_key, the path to the key file
• ssl_certificate, the path to the certificate file
• ssl_key_passdb, a passdb entry containing the SSL key
• ssl_certificate_passdb, a list of passdb entries containing SSL certificates. This allows easy inclusion of intermediaries as well.

If ssl_certificate is set without a passdb source a self signed certificate will be generated, if it is missing on disk. This is mostly useful when using services like letsencrypt - this allows bringing up the services with temporary certificates, to be replaced as soon as a system is operable enough to request proper certificates.

Example usage:

- include_role:
    name: data-utilities
    tasks_from: add_ssl_files
  vars:
    ssl_key: /etc/ssl/private/test.key
    ssl_certificate: /etc/ssl/private/test.pem
    ssl_size: 4096

When using passdb entries for key and certificates it also is possible to have keys and certificates combined in a single file, specified by ssl_key:

- include_role:
    name: data-utilities
    tasks_from: add_ssl_files
  vars:
    ssl_key: /etc/ssl/private/test.pem
    ssl_key_passdb: test-key
    ssl_certificate_passdb:
      - test-cert
      - intermediary

Using a single file without passdb entries will result in certificate generation being skipped, which quite likely will lead to failures.

This include configures gpg for one or more users, with optional pkcs11-scd support, expecting configuration in a variable named gpg_users. The basic-host role will automatically set this up if the variable is defined.

Sample configuration for user accounts user1 and user2:

gpg_users:
  user1:
    gnupg-pkcs11-scd:
    gpg-agent.conf.extra:
      - allow-emacs-pinentry
      - debug-level advanced
  user2:
    gpg-agent:
      debug: advanced
      verbose:
    gpg.conf:
      - no-secmem-warning

This include configures gpg for a single user, and expects to be run as that user. The include for multiple users just uses this include in a loop. The data structure expected as gpg is therefore a single user entry of the structure described there.

This include handles package installation for the provided packages on the following distribution/package manager combinations:

• SuSE with zypper
• RedHat/CentOS/Fedora with yum
• RedHat/CentOS/Fedora with dnf
• Debian/Ubuntu with apt

It takes the following variables:

• packages, a list of packages to install.
• install_retries, the number of retries if package installation fails. Defaults to 3.

Example usage:

- include_role:
    name: data-utilities
    tasks_from: install_packages
  vars:
    install_retries: 10
    packages:
      - nmap

This include abstracts commonly used service settings for systemd services and launchd services. The setting names are modeled after systemd services, lowercased with underscores between words.

Additionally state can be set to one of the values supported by the Ansible systemd module. It defaults to restarted, though the state change is only triggered if the service file changes on disk. Due to shortcomings of the OS X launchd every value not abesnt will lead to the service being started according to the service file.

For OS X and Linux the following keys are supported:

Linux additionally supports:

• exec_start_pre
• exec_start_post
• restart_sec
• description
• wanted_by, defaulting to multi-user.target
• type

- include_role:
    name: data-utilities
    tasks_from: manage_service
  vars:
    service:
      name: test
      state: absent

- include_role:
    name: data-utilities
    tasks_from: manage_service
  vars:
    service:
      name: test
      exec_start: /path/to/binary
      environment:
        foo: bar

This include checks if the running Ansible version matches the version range this role has been tested with. Additionally it also exports the variable data_utilities_version, and sets up some default variables. This also implicitely registers filter paths with Ansible.

The basic-host role includes this file, so unless another role needs to use a higher data-utilities version than basic-host was tested for including this file should not be necessary. When not using basic-host, or using multiple roles before basic-host this include should be included early on in the play.

For checking if data utilities is available in the correct version the data_utilities_minver variable can be set:

- include_role:
    name: data-utilities
    tasks_from: check_versions
  vars:
    data_utilities_minver: 0.1
  tags:
    - base_config
    - access_setup

This include tries to set the system hostname. On Windows this may require a reboot - execution continues once the system is reachable again.

For setting the hostname either the hostname variable is used, or - if missing - the hostname is generated from the inventory hostname.

This adds or removes files for motd. Variables controlling this should be set in group or host variables.

motd_templates controls templates for all hosts. motd_templates_<hostname> controls templates for the specific host only. motd_templates_<groupname> are added if the host is in that group.

The variables should be a dict, with optional keys filename and state. If state is absent the file will be removed, added for all other values. filename controls the filename in the motd directory - if unset the key will be used. The template is always looked up as the key with .j2 appended.

The basic-host role includes this file for all Linux systems - so as long as this role is used there should be no need to include this file.

motd_templates:
  motd_default:

motd_templates_dummy:
  motd_dummy:
  motd_gone:
    state: absent

This example would add motd_default.j2 to all hosts, and motd_dummy.j2 to a host named or in a group called dummy, and remove motd_gone from that host.

This include configures name services for a system - this mainly, but not only, covers DNS. Windows currently is not supported.

Nameservers from a list in the variable nameservers are used, if available. It usually makes sense to set a default in groupvars for all, and override it for other systems/groups.

There also is some legacy support for copying in prefilled resolv.conf templates based on resolv_location and site_region settings - this was implemented due to legacy Ansible restriction, and should not be used in new deployments.

This include configures timezone and other time related settings. On Windows this also sets NTP servers. For Linux this is handled in a separate ntp-client role.

NTP servers are read from a list in the ntp_servers variable. This typically should be set in the default groupvars for all, with overrides as needed.

For Windows the timezone is configured in host_timezone_win, using Microsofts time zone names. For other system standard TZ database names in the host_timezone variable are used.

For all systems the hardware clock is set to UTC.

This initialises the default password store. Variables controlling the setup should be set in group or host variables. the basic-host role includes this file - so as long as this role is used there should be no need to include this file.

Available variables are:

• passdb, defaulting to passwordstore
• passdb_password_length, default 20. This is used when creating passwords from within Ansible.
• passdb_password_create, bool, default True. Configures if Ansible is allowed to create missing passwords.
• passdb_extra_arg, default = create={{passdb_password_create}} length={{passdb_password_length}}=
• passdb_check_entry, default empty. Allows setting a record to check if passdb is working. This is useful for writing playbooks executable by both full access admins and people without access to some passwords. A sensible value typically is the entry for the default root password.

The variable default_passdb_available will be set by this include:

• True if no passdb_check_entry has been configured. This can lead to errors when executed without correct passdb access permissions.
• True if passdb_check_entry is accessible.
• False if passdb_check_entry is inaccessible.

Tasks/roles should use this variable to guard sections prompting for credentials, or skip execution completely.

While optional it is sensible to pre-define groups with host filters to avoid mistakes

For infrastructure spanning multiple physical locations hosts or host groups can be allocated to sites.

This will generate multiple helper variables for easier consumption in roles:

• vlans
• dhcp_networks