Skip to main content

Accelerate Your Docker Compose in GitHub Actions with Buildx Cache

· 3 min read
Ryo Koike

My ToDo app repo has a GitHub Action that runs npm run test for both frontend and backend using Docker Compose. It used to take 4 minutes and 48 seconds to run tests after building images, but by using buildx cache backends, it was shortened to 2 minutes and 18 seconds. The latency reduces by 2 minutes and 30 seconds, and the performance improvement is over 2x (see the related pull request). This article guides you simple steps you can add to benefit from caches.

TL;DR​

Just add these two steps:

--- snip ---
jobs:
  test_api:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

+      - uses: docker/setup-buildx-action@v2
+
+      - name: Build the backend image with cache
+        uses: docker/build-push-action@v4
+        with:
+          load: true
+          context: backend
+          file: backend/Dockerfile.dev
+          cache-from: type=gha,scope=$GITHUB_REF_NAME-backend-dev
+          cache-to: type=gha,scope=$GITHUB_REF_NAME-backend-dev,mode=max

      - name: Start backend
        run: docker compose -f docker-compose-dev.yml -f docker-compose-test-single.yml up -d backend

      - name: Run test
        run: docker compose -f docker-compose-dev.yml exec -T backend npm test -- --watchAll=false
  test_frontend:
    --- snip ---

Description​

docker/setup-buildx-action sets up the buildx builder instance and uses it in the following docker command.

In docker/build-push-action, three options are important: load, cache-from, and cache-to.

  • load loads the built image from "buildx world" to "docker world" so the subsequent docker command can use the built image. Without this option, the next docker compose up command builds the image again because the build result is contained in buildx, not exposed to docker.
  • cache-from option accepts multiple types. Here, we use the dedicated type for GitHub Actions (gha). Other types that will be useful elsewhere includes registry (loads cache from container registry) and local (loads cache from local filesystem).
  • cache-to option is similar to cache-from, but it also accepts an important option: mode. If mode is set to max, each layer of the built image is cached, while min mode only caches the end result.

In my case, I build both frontend and backend in a single GitHub Action. That is why I add scope option to cache-from and cache-to to distingush those caches.

Full YAML file

name: Run tests with Docker-Compose and Makefile
on:
  workflow_dispatch:
  pull_request:
    branches:
      - main
  push:
    branches:
      - main
jobs:
  test_frontend:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - uses: docker/setup-buildx-action@v2
        with:
          driver: docker-container

      - name: Build the [frontend] image with cache
        uses: docker/build-push-action@v4
        with:
          load: true
          context: frontend
          file: frontend/Dockerfile.dev
          tags: |
            frontend-test
          cache-from: type=gha,scope=$GITHUB_REF_NAME-frontend-dev
          cache-to: type=gha,scope=$GITHUB_REF_NAME-frontend-dev,mode=max

      - name: Run test
        run: docker run --rm frontend-test npm test -- --watchAll=false

  test_api:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - uses: docker/setup-buildx-action@v2
        with:
          driver: docker-container

      - name: Build the [api] image with cache
        uses: docker/build-push-action@v4
        with:
          load: true
          context: api
          file: api/Dockerfile.dev
          cache-from: type=gha,scope=$GITHUB_REF_NAME-api-dev
          cache-to: type=gha,scope=$GITHUB_REF_NAME-api-dev,mode=max

      - name: Start api
        run: docker compose -f docker-compose-dev.yml -f docker-compose-test-single.yml up -d --remove-orphans api

      - name: Run test
        run: docker compose -f docker-compose-dev.yml exec -T api npm test -- --watchAll=false

Summary​

It was just a small addition to a GitHub Action, but the performance improvemnet was impressive. I hope this post helps your daily development.

References​