Automated Testing of GitHub Actions for Docker Image Deployment

Unit and Integration Testing Strategies for Robust CI/CD Pipelines

In any tech team, CI/CD pipelines are indispensable. They essentially automate the process of building, testing, and deploying application/software/services. Testing these pipelines is equally as important for ensuring that the code is robust and behaves as expected in a production environment.

One common task in CI/CD pipelines is building and pushing Docker images to container registries like Google Artifact Registry (GAR), Amazon Elastic Container Registry (ECR) or DockerHub. In this blog, we will explore how to test a GitHub Actions (GAs) pipeline for pushing an image to DockerHub. Specifically, we'll look at two types of testing: unit testing and integration testing.

Testing CI/CD Github Actions Pipelines

In this blog, you'll learn:

1. Unit Testing to validate individual components like Dockerfiles and scripts.

2. Integration Testing using tools like act to ensure your GitHub Actions workflows function as expected.

1. Unit Testing the Docker Image

Unit testing in the context of a Docker image involves running tests against small testable parts of the script in isolation from the rest of the code. This can help ensure that individual actions work as expected. So, without having to run the entire workflow, we can test the core logic that the action is performing.

Test the Dockerfile

Before we even get to the GA, we have to make sure the Dockerfile builds as expected. We can write a script that tries to build the Docker image and fails if the docker build command fails.

Test the script

If the GA is running a shell or Python script to push the Docker image, we can test that script in isolation. For example, if we have a Python function that constructs docker push command, we can write a unit test for that function.

Mock external calls

If the action makes calls to external services (like in our case, DockerHub), we can mock those calls.

For example, if we’re using Python’s subprocess module to execute the docker push command, we can mock subprocess.run() to test that it gets called with expected arguments.

• actions.py contains the actual logic for pushing a Docker image.

• test_actions.py contains unit tests to verify that the Docker image can be built and pushed correctly.

The tests are designed to catch any issues in the Docker build and push process, ensuring that the actual shell commands are being called with the correct arguments.

actions.py

import subprocess

def construct_docker_push_command(image_name, tag):
    return f"docker push {image_name}:{tag}"

def function_that_calls_docker_push():
    try:
        subprocess.run(["docker", "push", "samtools:latest"], check=True)
    except subprocess.CalledProcessError as e:
        print(f"An error occurred while pushing the Docker image: {e}")

In actions.py, you have a function called function_that_calls_docker_push() that uses Python's subprocess module to run a shell command for pushing a Docker image.

test_actions.py

import unittest
from unittest.mock import patch
from action import function_that_calls_docker_push, construct_docker_push_command

class TestDockerActions(unittest.TestCase):

    def test_construct_docker_push_command(self):
        self.assertEqual(construct_docker_push_command("samtools", "latest"), "docker push samtools:latest")

    @patch("action.subprocess.run")
    def test_docker_push(self, mock_run):
        function_that_calls_docker_push()
        mock_run.assert_called_with(["docker", "push", "samtools:latest"], check=True)
    
if __name__ == '__main__':
    unittest.main()

In test_actions.py, you have several unit tests:

1. test_construct_docker_push_command: This test checks if the function construct_docker_push_command() returns the correct Docker push command string.

2. test_docker_push: This test mocks the subprocess.run() function and checks if function_that_calls_docker_push() calls it with the correct arguments.

3. @patch("subprocess.run") is using Python's unittest.mock to replace subprocess.run with a mock object for the duration of the test.

4. mock_run.assert_called_with(["docker", "push", "samtools:latest"]) checks if the mock was called with the specified arguments.

If you run python3 -m unittest test_action.py, you should see something like this, if all tests pass:

2. Integration Testing GitHub Actions

Integration testing involves testing the interaction between multiple units of an application. They help ensure the action works as expected when it interacts with the GitHub environment or other services.

We will use act, which will let us run GA workflows locally, and help speed up the development and testing process. Make sure it’s installed and ready to run:

brew install act

act

It's worth noting that act doesn't perfectly emulate GitHub Actions. Some third-party actions may not work as expected, so always double-check your results.

Ok, let’s use act to run the entire workflow and ensure all steps are executed correctly. First we will define a few environment variables in a .secrets file (placed in your root directory). This is useful for setting secrets or other environment-specific variables that your GitHub Actions might need.

DOCKERHUB_USERNAME=yourusername
DOCKERHUB_TOKEN=yourpassword

Here's a sample GitHub Actions YAML configuration for a Docker build and push workflow:

name: Docker Push

on:
  push:
    branches:
      - main

jobs:
  docker:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v2

    - name: Build Docker image
      run: docker build -t samtools:latest .

    - name: Login to DockerHub
      uses: docker/login-action@v1
      with:
        username: ${{ secrets.DOCKERHUB_USERNAME }}
        password: ${{ secrets.DOCKERHUB_TOKEN }}
    - name: Tag image
      run: docker tag samtools:latest ${{ secrets.DOCKERHUB_USERNAME }}/samtools:latest
    - name: Push Docker image
      run: docker push ${{ secrets.DOCKERHUB_USERNAME }}/samtools:latest

If everything runs smoothly, you should see something like this:

By running these integration tests, you can catch issues that unit tests might miss, such as problems that only appear when multiple steps are run together.

---

That's all for now! If you want the full code from this blog, check out this repo.

Have you implemented similar testing strategies in your CI/CD pipelines? 👀 If so, I would love to hear your thoughts on this!