Testing

Molecule has an extensive set of unit and functional tests. Molecule uses Tox Factors to generate a matrix of python x Ansible x unit/functional tests. Manual setup required as of this time.

Dependencies

Tests will be skipped when the driver’s binary is not present.

Install the test framework Tox.

$ pip install tox

For some tests RuboCop is required.

$ apt-get install ruby ruby-dev
$ gem install rubocop

Full

Run all tests, including linting and coverage reports. This should be run prior to merging or submitting a pull request.

$ tox

List available scenarios

List all available scenarios. This is useful to target specific Python and Ansible version for the functional and unit tests.

$ tox -l

Unit

Run all unit tests with coverage.

$ tox -e 'py{27,35,36,37}-ansible{25,26,27}-unit'

Run all unit tests for a specific version of Python and Ansible (here Python 3.7 and Ansible 2.7).

$ tox -e py37-ansible27-unit

Functional

Run all functional tests for all supported platforms.

Note

The functional tests are a work in progress. They need better structure and reuse. They are also very slow and costly in terms of system resources.

$ tox -e 'py{27,35,36,37}-ansible{25,26,27}-functional'

Run all functional tests for a specific version of Python and Ansible (here Python 3.7 and Ansible 2.7).

$ tox -e py37-ansible27-functional

Run all functional tests targeting the docker driver.

$ tox -e 'py{27,35,36,37}-ansible{25,26,27}-functional' -- -v -k docker

Delegated

Run all the functional delegated tests.

$ ansible-playbook -i test/resources/playbooks/delegated/inventory \
  test/resources/playbooks/delegated/create.yml
$ tox -t functional -- --delegated -v -k delegated
$ ansible-playbook -i test/resources/playbooks/delegated/inventory \
  test/resources/playbooks/delegated/destroy.yml

Linting

Linting is performed by a combination of linters. Run all the linters (some perform changes to conform the code to the style rules).

$ tox -e lint

Documentation

Generate the documentation, using sphinx.

$ tox -e doc

Build docker

Build the docker container.

$ tox -e build-docker

Continuous integration

Molecule output will use ANSI colors if stdout is an interactive TTY and TERM value seems to support it. You can define PY_COLORS=1 to force use of ANSI colors, which can be handly for some CI systems.

Travis CI

Travis is a CI platform, which can be used to test Ansible roles.

A .travis.yml testing a role named foo1 with the Docker driver.

---
sudo: required
language: python
services:
  - docker
before_install:
  - sudo apt-get -qq update
install:
  - pip install molecule
  # - pip install required driver (e.g. docker, python-vagrant, shade, boto, apache-libcloud)
script:
  - molecule test

A .travis.yml using Tox as described below.

---
sudo: required
language: python
services:
  - docker
before_install:
  - sudo apt-get -qq update
install:
  - pip install tox-travis
script:
  - tox

Gitlab CI

Gitlab includes its own CI. Pipelines are usually defined in a .gitlab-ci.yml file in the top folder of a repository, to be ran on Gitlab Runners.

Here is an example setting up a virtualenv and testing an Ansible role via Molecule. User-level pip is cached and so is the virtual environment to save time. And this is run over a runner tagged pip36 and docker, because its a minimal CentOS 7 VM installed with pip36 from IUS repository and docker.

---
image: docker:git

services:
  - docker:dind

before_script:
  - apk update && apk add --no-cache docker
    python3-dev py3-pip docker gcc git curl build-base
    autoconf automake py3-cryptography linux-headers
    musl-dev libffi-dev openssl-dev openssh
  - docker info
  - python3 --version

molecule:
stage: test
script:
  - pip3 install ansible molecule docker
  - ansible --version
  - cd roles/testrole && molecule test

Jenkins Pipeline

Jenkins projects can also be defined in a file, by default named Jenkinsfile in the top folder of a repository. Two syntax are available, Declarative and Scripted. Here is an example using the declarative syntax, setting up a virtualenv and testing an Ansible role via Molecule.

pipeline {

  agent {
    // Node setup : minimal centos7, plugged into Jenkins, and
    // git config --global http.sslVerify false
    // sudo yum -y install https://centos7.iuscommunity.org/ius-release.rpm
    // sudo yum -y install python36u python36u-pip python36u-devel git curl gcc
    // git config --global http.sslVerify false
    // sudo curl -fsSL get.docker.com | bash
    label 'Molecule_Slave'
  }

  stages {

    stage ('Get latest code') {
      steps {
        checkout scm
      }
    }

    stage ('Setup Python virtual environment') {
      steps {
        sh '''
          export HTTP_PROXY=http://10.123.123.123:8080
          export HTTPS_PROXY=http://10.123.123.123:8080
          pip3.6 install virtualenv
          virtualenv virtenv
          source virtenv/bin/activate
          pip install --upgrade ansible molecule docker
        '''
      }
    }

    stage ('Display versions') {
      steps {
        sh '''
          source virtenv/bin/activate
          docker -v
          python -V
          ansible --version
          molecule --version
        '''
      }
    }

    stage ('Molecule test') {
      steps {
        sh '''
          source virtenv/bin/activate
          molecule test
        '''
      }
    }

  }

}

The following Jenkinsfile uses the official ‘quay.io/ansible/molecule’ image.

pipeline {
  agent {
    docker {
      image 'quay.io/ansible/molecule'
      args '-v /var/run/docker.sock:/var/run/docker.sock'
    }
  }

  stages {

    stage ('Display versions') {
      steps {
        sh '''
          docker -v
          python -V
          ansible --version
          molecule --version
        '''
      }
    }

    stage ('Molecule test') {
      steps {
        sh 'sudo molecule test --all'
      }
    }

  } // close stages
}   // close pipeline

Note

For Jenkins to work properly using a Multibranch Pipeline or a GitHub Organisation - as used by Blue Ocean, the role name in the scenario/playbook.yml should be changed to perform a lookup of the role root directory. For example :

---
- name: Converge
  hosts: all
  roles:
    - role: "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') | basename }}"

This is the cleaner of the current choices. See issue1567_comment for additional detail.

Tox

Tox is a generic virtualenv management, and test command line tool. Tox can be used in conjunction with Factors and Molecule, to perform scenario tests.

To test the role against multiple versions of Ansible.

[tox]
minversion = 1.8
envlist = py{27}-ansible{20,21,22}
skipsdist = true

[testenv]
passenv = *
deps =
    -rrequirements.txt
    ansible20: ansible==2.0.2.0
    ansible21: ansible==2.1.2.0
    ansible22: ansible==2.2.0.0
commands =
    molecule test

To view the factor generated tox environments.

$ tox -l
py27-ansible20
py27-ansible21
py27-ansible22

If using the –parallel functionality of Tox (version 3.7 onwards), Molecule must be made aware of the parallel testing by setting a MOLECULE_EPHEMERAL_DIRECTORY environment variable per environment. In addition, we export a TOX_ENVNAME environment variable, it’s the name of our tox env.

[tox]
minversion = 3.7
envlist = py{27}_ansible{23,24}
skipsdist = true

[testenv]
deps =
    -rrequirements.txt
    ansible23: ansible==2.3
    ansible24: ansible==2.4
commands =
    molecule test
setenv =
    TOX_ENVNAME={envname}
    MOLECULE_EPHEMERAL_DIRECTORY=/tmp/{envname}

If you are utilizing the Openstack driver you will have to make sure that your envname variable does not contain any invalid characters, particularly -.

You also must include the TOX_ENVNAME variable in name of each platform in molecule.yml configuration file. This way, ther names won’t create any conflict.

---
dependency:
  name: galaxy
driver:
  name: docker
lint:
  name: yamllint
platforms:
  - name: instance1-$TOX_ENVNAME
    image: mariadb
  - name: instance2-$TOX_ENVNAME
    image: retr0h/centos7-systemd-ansible:latest
    privileged: True
    command: /usr/sbin/init
provisioner:
  name: ansible
  lint:
    name: ansible-lint
verifier:
  name: testinfra
  lint:
    name: flake8