Testing

Molecule has an extensive set of unit and functional tests. Molecule uses `Tox`_ Factors to generate a matrix of python x Ansible x unit/functional tests. Manual setup required as of this time.

Dependencies

Tests will be skipped when the driver’s binary is not present.

Install the test framework `Tox`_.

$ pip install tox

Install tox-tags.

Install the remaining requirements in a venv (optional).

$ pip install -r test-requirements.txt -r requirements.txt

Full

Run all tests, including linting and coverage reports. This should be run prior to merging or submitting a pull request.

$ tox

Unit

Run all unit tests with coverage.

$ tox -t unit

Functional

Run all functional tests.

Note

The functional tests are a work in progress. They need better structure and reuse.

$ tox -t functional

Run all functional tests targeting the docker driver.

$ tox -t functional -- -v -k docker

Delegated

Run all the functional delegated tests.

$ ansible-playbook -i test/resources/playbooks/delegated/inventory \
  test/resources/playbooks/delegated/create.yml
$ tox -t functional -- --delegated -v -k delegated
$ ansible-playbook -i test/resources/playbooks/delegated/inventory \
  test/resources/playbooks/delegated/destroy.yml

Formatting

The formatting is done using YAPF.

$ tox -e format

Linting

Linting is performed by Flake8.

$ tox -e $(tox -l | grep lint | paste -d, -s -)

LXC

Follow the steps detailed in the Vagrantfile below.

$ cd test/functional/lxc
$ vagrant up

LXD

Follow the steps detailed in the Vagrantfile below.

$ cd test/functional/lxd
$ vagrant up

Continuous integration

Travis CI

Travis is a CI platform, which can be used to test Ansible roles.

A .travis.yml testing a role named foo1 with the Docker driver.

---
sudo: required
language: python
services:
  - docker
before_install:
  - sudo apt-get -qq update
install:
  - pip install molecule
  # - pip install required driver (e.g. docker, python-vagrant, shade, boto, apache-libcloud)
script:
  - molecule test

A .travis.yml using `Tox`_ as described below.

---
sudo: required
language: python
services:
  - docker
before_install:
  - sudo apt-get -qq update
install:
  - pip install tox-travis
script:
  - tox

Tox

`Tox`_ is a generic virtualenv management, and test command line tool. `Tox`_ can be used in conjunction with Factors and Molecule, to perform scenario tests.

To test the role against multiple versions of Ansible.

[tox]
minversion = 1.8
envlist = py{27}-ansible{20,21,22}
skipsdist = true

[testenv]
passenv = *
deps =
    -rrequirements.txt
    ansible20: ansible==2.0.2.0
    ansible21: ansible==2.1.2.0
    ansible22: ansible==2.2.0.0
commands =
    molecule test

To view the factor generated tox environments.

$ tox -l
py27-ansible20
py27-ansible21
py27-ansible22

Detox

Detox is a distributed version of Tox which can be used to make efficient use of multiple CPUs. Detox will run tox environment tests in parallel.

Detox takes the same arguments and configuration as tox, however Molecule must be made aware of the parallel testing by setting a MOLECULE_EPHEMERAL_DIRECTORY environment variable per environment.

[tox]
minversion = 1.8
envlist = py{27}_ansible{23,24}
skipsdist = true

[testenv]
deps =
    -rrequirements.txt
    ansible23: ansible==2.3
    ansible24: ansible==2.4
commands =
    molecule test
setenv =
    MOLECULE_EPHEMERAL_DIRECTORY={envname}

If you are utilizing the Openstack driver you will have to make sure that your envname variable does not contain any invalid characters, particularly -.

You also must include the MOLECULE_EPHEMERAL_DIRECTORY variable in the molecule.yml configuration file.

---
dependency:
  name: galaxy
driver:
  name: docker
lint:
  name: yamllint
platforms:
  - name: instance1-$MOLECULE_EPHEMERAL_DIRECTORY
    image: mariadb
  - name: instance2-$MOLECULE_EPHEMERAL_DIRECTORY
    image: retr0h/centos7-systemd-ansible:latest
    privileged: True
    command: /usr/sbin/init
provisioner:
  name: ansible
  lint:
    name: ansible-lint
scenario:
  name: default
verifier:
  name: testinfra
  lint:
    name: flake8