Bootstrapping with Ansible

Sun, Jan 24, 2021 6-minute read

I’m using Ansible to manage my home-lab environment because it’s easy to learn, has many built-in and contributed tasks, and is agent-less.

Despite being agent-less, some bootstrapping is needed to seamlessly manage hosts with Ansible, and in this post I’ll show how to use Ansible to bootstrap hosts to be managed with Ansible.

TLDR: The inventory and playbook examples are in this gist.

I manage two types of hosts with Ansible and they require subtly different bootstrapping processes:

  • On-premise VM: These are provisioned with a non-privileged user (ansible) that can log in via SSH using a password.
  • Cloud VM: These are provisioned with a privileged user (root) which can log in via SSH using a password.

So what state do we want these hosts in to manage them seamlessly with Ansible?

  • Non-privileged user (ansible) can log in via SSH using only public key authentication.
  • Non-privileged user (ansible) can run privileged commands using password-less sudo.
  • Privileged user (root) cannot log in via SSH.

The following scenario was tested with Ansible 2.9 running against Debian 10 (Buster) hosts.

Start by creating an inventory file listing the hosts to manage. Mine is simply called inventory:

[onprem_vm]
host1.internal.domain
host2.internal.domain
host3.internal.domain

[cloud_vm]
host4.external.domain
host5.external.domain
host6.external.domain

[all:vars]
ansible_user=ansible
ansible_python_interpreter=/usr/bin/python3

This example defines two host groups (onprem_vm and cloud_vm) and in each lists the fully-qualified domain names (FQDN) of the hosts to be managed.

The last part of the inventory file assigns two variables to the virtual all host group which, as you’d expect, includes all the hosts in the inventory. The ansible_user variable specifies the user Ansible connects to the host as and ansible_python_interpreter specifies the Python executable to use on the host.

You may have noticed a problem already - some of my hosts don’t have an ansible user. Create a new Ansible playbook called bootstrap.yaml and add the following playbook that provisions the ansible group and user:

- hosts: all
  vars_files:
    - vault.yaml
  tasks:
    - group:
        name: ansible
        gid: 1000
        state: present
    - user:
        name: ansible
        uid: 1000
        group: ansible
        groups: cdrom,floppy,audio,dip,video,plugdev,netdev
        password: "{{ ansible_password_crypted }}"
        shell: /bin/bash
        state: present

Although the account’s password is crypted I keep all secrets in an Ansible Vault file, so instead of including it in the playbook I reference the variable ansible_password_crypted imported from an Ansible Vault vault.yaml using the vars_files property.

We create vault.yaml by running the following in the same directory as the bootstrap.yaml:

ansible-vault create vault.yaml

Running this prompts for a password to protect the file with and then opens it in your default editor. We just have a single secret so we’ll add that, save, and close the editor:

ansible_password_crypted: $6$YZlrM35Vyi.L6PKX$7u8dHLR82O4VSObPtjJzmA4cqUtmZGncyhAPjNze9LdwqXcax00Fe3FhQY4HqLXadl/XMDkUIfE8dMt8U.pOY0

How did I get the crypted password (btw this is not my real crypted password)? A quick Google search came up with this one-line suggestion using Python 3:

python3 -c 'import crypt,getpass;pw=getpass.getpass();print(crypt.crypt(pw) if (pw==getpass.getpass("Confirm: ")) else exit())'

We want the ansible user to be able to run privileged commands using sudo without an additional password prompt. The sudo command isn’t installed by default on Debian 10 so we add a task to the playbook to install it (we also install curl because the next task will need it):

    - apt:
        update_cache: yes
        name:
          - curl
          - sudo
        state: present

We then another task that creates a sudo policy file that allows the ansible user to run all commands, as all users, without a password prompt:

    - copy:
        content: "ansible	ALL = (ALL) NOPASSWD:ALL"
        dest: /etc/sudoers.d/ansible

To force public key authentication for the ansible user it needs at least one public key in its authorized_keys file. We’ll use the ansible.posix.authorized_key task to add the public keys so we need to install the ansible.posix collection from Ansible Galaxy by running:

ansible-galaxy collection install ansible.posix

Now we add the following task to the playbook:

    - ansible.posix.authorized_key:
        user: ansible
        key: https://github.com/iamwillbar.keys
        exclusive: yes
        state: present

This example pulls public keys for a GitHub user but you can include the public key directly if you prefer.

Finally, we configure the SSH server to start automatically and to allow only public key authentication (make sure you have console access to the host in case you lock yourself out).

We add a task to replace sshd_config, we store the output of this in the sshd_config variable using register: sshd_config, which is used to restart the SSH server only if it changed:

    - copy:
        content: |
          PermitRootLogin no
          PasswordAuthentication no
          ChallengeResponseAuthentication no
          AcceptEnv LANG LC_*
          Subsystem sftp /usr/lib/openssh/sftp-server          
        dest: /etc/ssh/sshd_config
      register: sshd_config
    - service:
        name: sshd
        enabled: yes
        state: started
    - service:
        name: sshd
        state: restarted
      when: sshd_config.changed

We’ve added the required functionality to the playbook but there’s a problem - it only works if we run it as a privileged user and some of the hosts can only be connected to as a non-privileged user.

To solve this we use a feature built into Ansible that allows the playbook to become a different user after connecting (using either su or sudo). Because some hosts connect as a privileged user and some don’t we need to do this only when we’re connecting as a user other than the ansible user.

Go to the beginning of the playbook and change:

- hosts: all
  vars_files:
    - vault.yaml
  tasks:

To:

- hosts: all
  vars_files:
    - vault.yaml
  become: "{{ ansible_ssh_user is undefined or ansible_user == ansible_ssh_user | ternary('yes', 'no') }}"
  become_method: su
  tasks:

This checks if we’re connecting to the host as the same user that we configured as the ansible_user (ansible in our case) and if so sets become: yes since we know we need to become a privileged user. Otherwise, it sets become: no. We use su instead of sudo because when the host is being bootstrapped it may not have sudo installed yet.

This leaves one outstanding question, how do we connect as a user other than the ansible_user? For the answer to that we turn to an old GitHub issue and we’ll use two different commands to run the bootstrap playbook depending on whether we’re connecting as the priviledged user root or the non-privileged user ansible.

If we’re bootstrapping a host where we connect with a non-privileged account we’ll run:

ansible-playbook bootstrap.yaml -i inventory -l host1.internal.domain --ask-pass --ask-become-pass --ask-vault-pass

We are prompted for three passwords:

  • SSH password: This is the password for the non-privileged user (ansible in our case).
  • Become password: This is the password for the privileged user we’ll become (root).
  • Vault password: This is the password you used when creating the Ansible Vault (vault.yaml) earlier.

If we’re bootstrapping a host where we connect with a privileged account we’ll run:

ansible-playbook bootstrap.yaml -i inventory -l host1.internal.domain --ask-pass --ask-vault-pass -e ansible_ssh_user=root

We remove --ask-become-pass because we’re connecting as a privileged user and don’t need to become another user and we pass -e ansible_ssh_user=root to make the SSH connection as that user instead of ansible_user.

In this case we’re prompted for two passwords:

  • SSH password: This time this is the password for the privileged user (root).
  • Vault password: This is the password you used when creating vault.yaml earlier.

Now you can use Ansible to bootstrap any host so you can manage it with Ansible, securely using public key authentication as a dedicated account. Once the host has been bootstrapped you can run any playbook like this:

ansible-playbook other_playbook.yaml -i inventory --ask-vault-pass

The inventory and playbook examples are available in this gist.