The Wizard: Ansible, Molecule and Test Driven Development

- 33 mins
infrastructure as code software craftsmanship

Magic has existed since the dawn of time.

It has always been there, hidden in plain sight, making amazing things possible for those willing to open their eyes and harness its power.

You can’t remember the first time you used it, and yet it feels like you’ve never existed without it. You’ve obliterated endless armies of enemies by virtue of spells and enchantments, and you’ve also constructed awe-inspiring marvels from the ground up. As a result, you’re renowned for your overwhelming powers in every single one of your incarnations through time. Everyone knows about the Wizard.

This affinity with magic grants you the ability to see the real nature of things. Time for example, is not linear (as it was believed to be up until the late 21st century). Sometimes, it is hard to keep up with it…

Suddenly you wake up. Or you come into consciousness. What year is it?

You look to your left. N is sitting right there. He looks at you with mild concern in his eyes.

Wizard

You look at the spell he is trying to cast. His intention is clear: he wants to share the work of his clan with all the sentient entities in this reality by means of a portal.

---
# task file for nginx

- name: Install nginx
  yum:
    name: nginx
    state: present

It is clear why he is failing.

He is not taking into account the constraints of his clan’s environment. He must first gather the requirements of his spell, and he’s not looking at the right places. As a matter of fact, he is not looking at all. The answer is simple. Nevertheless…

You reflect for a second, wondering if someone even taught N how to properly cast a spell.

You feel betrayed by the sound of his words. This is unacceptable. A whole stack of death spells appear in your head. Breathe, you repeat to yourself. In. Out. You keep going.

Ah, a familiar name. You like L. You fought together at Desaix, before he started mentoring his own apprentices on the arts of war. He must have taught N a thing or two. You wonder if he will be capable of surviving the trial.

You harness your energy as you get ready to cast your first spell. You feel alive again. You’re in familiar grounds. Before you start working, you need an empty canvas, a playground to construct the structure and robustness of your magic. An isolated enclosure which will allow you to try and cast your spells, and which also follows the same rules as the target reality for the spell you’re trying to create.

He’s fast. Time to make a choice. The Wanderer, or the Whale? The Whale is indeed faster.

The Whale it is. You concentrate and cast your first incantation.

$ molecule init role --role-name nginx --driver-name docker

You feel the energy flowing through. A whirlpool of blue water forms in the sky, and The Whale appears from inside of it. It comes swimming down through the air, looking at you both, and sings its song loudly. An empty enclosure materializes out of thin air, trapping you and N inside of it. The Whale disappears from the whirlpool where it came, right before the whirlpool itself vanishes into nothingness, as fast as it had appeared.

--> Initializing new role nginx…

nginx/
├── README.md
├── defaults
│   └── main.yml
├── handlers
│   └── main.yml
├── meta
│   └── main.yml
├── molecule
│   └── default
│       ├── Dockerfile.j2
│       ├── INSTALL.rst
│       ├── create.yml
│       ├── destroy.yml
│       ├── molecule.yml
│       ├── playbook.yml
│       └── tests
│           └── test_default.py
├── tasks
│   └── main.yml
└── vars
    └── main.yml

You feel the need to test N.

He’s looking at what you’ve produced, his eyebrows frowning.

Good. He realises there’s a difference. How can you easily explain the fact that, in order to control such an enclosure, you need intermediate spells to both create it and destroy it at will? You decide that it is not important right now, you’ll come back later to it.

You concentrate and you cast the spell on the enclosure/playground.

$ molecule test

--> Test matrix
└── default
    ├── destroy
    ├── dependency
    ├── syntax
    ├── create
    ├── converge
    ├── idempotence
    ├── lint
    ├── side_effect
    ├── verify
    └── destroy

N is impressed by the fireworks. He has so many questions. You feel overjoyed, but you give him a second so that he can get his ideas together.

Snarky.

He wants you to know he’s not a child. He contemplates the enclosure’s structure. You wait for a while until he starts talking again.

You decide now is the time to go down the rabbit hole.

You raise your hand, and you snap your fingers. You hear warping sounds as the enclosure somehow shifts. Everything looks the same, yet everything feels different.

# molecule.yml
---
dependency:
  name: galaxy
driver:
  name: docker
lint:
  name: yamllint
platforms:
  - name: instance
    image: centos:7
provisioner:
  name: ansible
  lint:
    name: ansible-lint
scenario:
  name: default
  test_sequence:
    - lint
verifier:
  name: testinfra
  lint:
    name: flake8

You take a deep breath, and you cast the spell again.

$  molecule test
--> Test matrix
└── default
    └── lint
--> Scenario: 'default'
--> Action: 'lint'
--> Executing Yamllint on files found in /private/tmp/test/nginx/...
Lint completed successfully.
--> Executing Flake8 on files found in /private/tmp/test/nginx/molecule/default/tests/...
Lint completed successfully.
--> Executing Ansible Lint on /private/tmp/test/nginx/molecule/default/playbook.yml...
Lint completed successfully.

Once you’re finished, you look back at N. He gets where you’re going.

He studies the playground spells thoroughly.

Good. He is connecting the dots, seeing outside of the box. You decide that you like N a little better now. Enough to keep the game going.

You notice something is missing, but you decide to keep going to take advantage of the synergy. You will get back to it, eventually. You snap your fingers once again, altering the enclosure to match his words.

# molecule.yml
...
scenario:
  name: default
  test_sequence:
    - destroy
    - syntax
    - lint
    - create
    - converge
    - verify

You cast the spell once again, using the modified reality of the enclosure. The fireworks appear one more time, this time slightly smaller.

$  molecule test
--> Test matrix
└── default
    ├── destroy
    ├── syntax
    ├── lint
    ├── create
    ├── converge
    └── verify

N seems happy.

He understands how it works. You take advantage of his motivation to feed him more knowledge.

Knowledge is power. Everyone knows that. If you’re able to acquire knowledge faster, you’re able to acquire power faster as well.

You let that sink in for a while, hoping that he doesn’t give up. You look outside the window. It’s raining. You like rain. At least on Earth. It’s better than on Venus, where it melts the skin off of your bones if you don’t cast a protection enchantment around you. That is if the temperature or the pressure don’t get you first. N doesn’t give up. He looks at you.

You start summoning The Whale again, but you only hear its song this time. It is as loud as it was before. The enclosure disappears and reappears around you.

$ molecule create
--> Test matrix
└── default
    └── create
--> Scenario: 'default'
--> Action: 'create'

You look at N, and you think about the next steps. This is going to sound familiar to him.

You clap your hands once, loudly. As you separate them a stream of shiny green light emanates from them.

 $molecule verify
--> Test matrix
└── default
    └── verify
--> Scenario: 'default'
--> Action: 'verify'
--> Executing Testinfra tests found in /private/tmp/test/nginx/molecule/default/tests/...
    Verifier completed successfully.

Everything is okay, for now. Back to N.

L had done his job right.

You spawn a glob of energy with your hands and use it to craft a rule within the enclosure’s reality:

import os

import testinfra.utils.ansible_runner

testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
    os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('all')


def test_nginx_is_installed(host):
    # Given
    nginx = host.package('nginx')

    # Then
    assert nginx.is_installed

You clap your hands together, just as you did a minute ago. Nevertheless, red shiny light appears this time as you separate them.

$ molecule verify

=================================== FAILURES ===================================
_________________ test_nginx_is_installed[ansible://instance] __________________

host = <testinfra.host.Host object at 0x10a9a7350>

    def test_nginx_is_installed(host):
        # Given
        nginx = host.package('nginx')

        # Then
>       assert nginx.is_installed
E       assert False
E        +  where False = <package nginx>.is_installed

tests/test_default.py:14: AssertionError

You close your eyes and concentrate. The structure of the spell N had in the first place is recreated from the ground.

---
# tasks file for nginx

- name: Install nginx
  yum:
    name: nginx
    state: present

Back to square one. It is now the same spell N had when he asked you for help. You cast the spell, knowing what will happen beforehand.

 $molecule converge
--> Test matrix
└── default
    ├── create
    └── converge
--> Scenario: 'default'
--> Action: 'create'
Skipping, instances already created.
--> Scenario: 'default'
--> Action: 'converge'

    TASK [nginx : Install nginx] ***************************************************
    fatal: [instance]: FAILED! => {"changed": false, "failed": true, "msg": "No package matching 'nginx' found available, installed or updated", "rc": 126, "results": ["No package matching 'nginx' found available, installed or updated"]}

No fireworks. N seems discouraged. You think about giving him a little push. He deserves it. It’s not about the magic itself, it’s about context. He’s not getting the sources for his portal from the right index. He needs the Epel codex.

He smiles. He’s getting in the game, getting the feel of the mindset. He likes it.

You modify the structure of the spell once again, specifying that you want your portal to be created following the instructions on the Epel codex.

---
# tasks file for nginx

- name: Install epel release
  yum:
    name: epel-release
    state: present

- name: Install nginx
  yum:
    name: nginx
    state: present

You cast the spell, with the new structure.

$ molecule converge
--> Test matrix
└── default
    ├── create
    └── converge
--> Scenario: 'default'
--> Action: 'create'
Skipping, instances already created.
--> Scenario: 'default'
--> Action: 'converge'
…
    PLAY RECAP *********************************************************************
    instance                   : ok=3    changed=2    unreachable=0    failed=0

The spell works, or at least it seems to work. The fireworks make the air vibrate within the enclosure. N seems happy. He doesn’t have the right reflexes yet. Can you blame him? It’s his first existence, and he’s just starting to play with magic. He won’t realize that his portal is not yet active, he’s just dazzled with the fireworks. It’s time for a new trial.

[root@instance /]# curl localhost
curl: (7) Failed to connect to ::1: Cannot assign requested address

He is about to modify the spell’s structure, when he suddenly stops. He looks at you.

He passed the test. He might make a good disciple. You start to understand why L started teaching in the first place.

def test_nginx_is_running(host):
    # Given
    nginx = host.service('nginx')

    # Then
    assert nginx.is_running

You know the Whale won’t let N get away with that, but you want to see him try. Failure is part of the learning process. It is almost more important than success. It’s the only way he can learn to get up and keep trying.

$ molecule verify
    E       AssertionError: Unexpected exit code 1 for CommandResult(command=u'systemctl is-active nginx', exit_status=1, stdout=u'', stderr=u'Failed to get D-Bus connection: Operation not permitted')

N claps his hands like you did before, but no light appears as he separated his hands. He looks as you, confused.

platforms:
  - name: instance
    image: centos/systemd
    privileged: True
    command: /usr/sbin/init

The Whale will surely accept this offering. You know it will not reject your summoning anymore. N verifies the spell again.

$ molecule test
=================================== FAILURES ===================================
____________ test_nginx_is_running_and_enabled[ansible://instance] _____________

host = <testinfra.host.Host object at 0x110adff50>

    def test_nginx_is_running(host):
        # Given
        nginx = host.service('nginx')

        # Then
>       assert nginx.is_running
E       assert False
E        +  where False = <service nginx>.is_running

tests/test_default.py:20: AssertionError

-- Docs: http://doc.pytest.org/en/latest/warnings.html
================ 1 failed, 1 passed, 1 warnings in 6.74 seconds ================

You smile as you see N craft. He is not ready yet, but he is well under way. You feel proud of him.

He shows you the structure of his spell.

---
# tasks file for nginx

- name: Install epel release
  yum:
    name: epel-release
    state: present

- name: Install nginx
  yum:
    name: nginx
    state: present

- name: Start nginx
  command: systemctl start nginx

All of your previously found pride turns into disappointment. This is preposterous. It can’t be serious. He can’t be serious. He should understand that you can’t be spawning portals indefinitely. The results could be catastrophic.

But then…you contemplate him. You realize he’s tired. Idempotence is a hard concept to grasp too. Benjamin Peirce taught it for over 50 years at Harvard. You remember the first time Merlin brought you to an enclosure like this one. Just the memory of it makes you feel goosebumps on the back of your neck.

You decide that you’re going to let him continue, for education purposes only. He’ll realize the problem with his proposal.

He casts the spell himself.

$ molecule converge
--> Test matrix
└── default
    ├── create
    └── converge
--> Scenario: 'default'
--> Action: 'create'
Skipping, instances already created.
--> Scenario: 'default'
--> Action: 'converge'

    PLAY RECAP *********************************************************************
    instance                   : ok=4    changed=1    unreachable=0    failed=0

Right after the fireworks, a green portal appears right in front of both of you. It seems to work.

$ molecule verify
--> Test matrix
└── default
    └── verify
--> Scenario: 'default'
--> Action: 'verify'
--> Executing Testinfra tests found in /Users/sebiwi/stuff/test/nginx/molecule/default/tests/...
    ============================= test session starts ==============================
    ===================== 2 passed, 1 warnings in 6.70 seconds =====================
Verifier completed successfully.

Green light everywhere.

Or is it? You decide to give him a little push in the right direction.

$molecule test
--> Test matrix
└── default
    ├── destroy
    ├── syntax
    ├── lint
    ├── create
    ├── converge
    └── verify

---> Scenario: 'default'
--> Action: 'lint'
--> Executing Yamllint on files found in /Users/sebiwi/stuff/test/nginx/...
Lint completed successfully.
--> Executing Flake8 on files found in /Users/sebiwi/stuff/test/nginx/molecule/default/tests/...
Lint completed successfully.
--> Executing Ansible Lint on /Users/sebiwi/stuff/test/nginx/molecule/default/playbook.yml...
    [ANSIBLE0012] Commands should not change things if nothing needs doing
    /Users/sebiwi/stuff/test/nginx/tasks/main.yml:14
    Task/Handler: Start nginx

No fireworks, no lights.

You stare in disbelief as N “corrects” the structure of the spell.

---
# tasks file for nginx

- name: Install epel release
  yum:
    name: epel-release
    state: present

- name: Install nginx
  yum:
    name: nginx
    state: present

- name: Start nginx
  command: systemctl start nginx
  tags:
    - skip_ansible_lint

He finishes the modifications and he casts the spell once again.

$  molecule test
--> Test matrix
└── default
    ├── destroy
    ├── syntax
    ├── lint
    ├── create
    ├── converge
    └── verify

All steps completed successfully.

He looks at you proudly. You’re dismayed. He’s being naive and reckless. You realize that it is time to handle the situation.

You close your eyes as you change the reality of the enclosure around you once again.

scenario:
  name: default
  test_sequence:
    - destroy
    - syntax
    - lint
    - create
    - converge
    - idempotence
    - verify

You cast the spell again.

$  molecule test
--> Test matrix
└── default
    ├── destroy
    ├── syntax
    ├── lint
    ├── create
    ├── converge
    ├── idempotence
    └── verify

--> Scenario: 'default'
--> Action: 'idempotence'
ERROR: Idempotence test failed because of the following tasks:
* [instance] => nginx : Start nginx

N looks at you.

You hand over the control of the enclosure to N. He modifies the spell structure.

---
# tasks file for nginx

- name: Install epel release
  yum:
    name: epel-release
    state: present

- name: Install nginx
  yum:
    name: nginx
    state: present

- name: Start nginx
  service:
    name: nginx
    state: started

You smile with relief.

$  molecule test
--> Test matrix
└── default
    ├── destroy
    ├── syntax
    ├── lint
    ├── create
    ├── converge
    ├── idempotence
    └── verify

All steps completed successfully.

---
# tasks file for nginx

- name: Install epel release
  yum:
    name: epel-release
    state: present

- name: Install nginx
  yum:
    name: nginx
    state: present
  notify: Start nginx

---
# handlers file for nginx

- name: Start nginx
  service:
    name: nginx
    state: started

$  molecule test
--> Test matrix
└── default
    ├── destroy
    ├── syntax
    ├── lint
    ├── create
    ├── converge
    ├── idempotence
    └── verify

All steps completed successfully.

N looks euphoric.

He’s applying his well-earned war experience to a whole different situation, and it is working like a charm. There’s a slight catch within your proposal, but you decide that you will let him discover it by himself.

---
# tasks file for nginx

- name: Install epel-release and nginx
  yum:
    name: ""
    state: present
  with_items:
    - epel-release
    - nginx
  notify:
    - Start nginx

$ molecule test
...
--> Scenario: 'default'
--> Action: 'converge'

    PLAY [Converge] ****************************************************************

    TASK [Gathering Facts] *********************************************************
    ok: [instance]

    TASK [nginx : Install epel-release and nginx] **********************************
    failed: [instance] (item=[u'epel-release', u'nginx']) => {"changed": false, "failed": true, "item": ["epel-release", "nginx"], "msg": "No package matching 'nginx' found available, installed or updated", "rc": 126, "results": ["No package matching 'nginx' found available, installed or updated"]}

    PLAY RECAP *********************************************************************
    instance                   : ok=1    changed=0    unreachable=0    failed=1

N transforms the spell structure back to its previous form. You feel relaxed. You always feel this way after creating something impressive. You feel the same way you felt when you structured the pyramids, only to a minor extent. This time, however, the actual creator is N. You only acted as a catalyzer. The knowledge was always inside of N, he just needed to hear the right questions.

N looks tired, but he wants to keep going.

N creates a new rule within the enclosure’s reality.

def test_nginx_is_enabled(host):
    # Given
    nginx = host.service('nginx')

    # Then
    assert nginx.is_enabled

He claps his hands together. Red light appears everywhere.

$ molecule verify
--> Test matrix
└── default
    └── verify
--> Scenario: 'default'
--> Action: 'verify'
--> Executing Testinfra tests found in /Users/sebiwi/stuff/test/nginx/molecule/default/tests/…

    =================================== FAILURES ===================================
    __________________ test_nginx_is_enabled[ansible://instance] ___________________

    host = <testinfra.host.Host object at 0x103b20fd0>

       def test_nginx_is_enabled(host):
            # Given
            nginx = host.service('nginx')

            # Then
    >       assert nginx.is_enabled
    E       assert False
    E        +  where False = <service nginx>.is_enabled

    tests/test_default.py:28: AssertionError
    -- Docs: http://doc.pytest.org/en/latest/warnings.html
    ================ 1 failed, 2 passed, 1 warnings in 7.29 seconds ================

N modifies the structure of the spell to match the new reality constraints. He’s got a clear head now, he knows what is missing.

---
# handlers file for nginx

- name: Start nginx
  service:
    name: nginx
    state: started
    enabled: yes

$  molecule test
--> Test matrix
└── default
    ├── destroy
    ├── syntax
    ├── lint
    ├── create
    ├── converge
    ├── idempotence
    └── verify

All steps completed successfully.

The green portal appears in front of you both. You know it is sustainable, you both tested it thoroughly. It’s a proper wizard spell. People would say it is one of your own. N looks at you and smiles.

You do agree.

N seems in charge. He needs more time to develop himself, to learn, to become comfortable with this whole new world. You won’t be as useful during that period as you were now. It’s time to go.

N looks at where you used to be 10 seconds ago. You are no longer there.

Related Posts

Sebiwi

Sebiwi

Perfectionist

comments powered by Disqus
rss facebook twitter github youtube mail spotify instagram linkedin google google-plus pinterest medium vimeo stackoverflow reddit quora