AWS CodeBuild

info

What you'll learn

How to set up Cypress tests in AWS CodeBuild
How to cache dependencies and build artifacts
How to run Cypress tests in parallel with AWS CodeBuild
How to use Cypress Cloud with AWS CodeBuild

Basic Setup

The example below is a basic CI setup and job using the AWS CodeBuild to run Cypress tests within the Electron browser. This AWS CodeBuild configuration is placed within buildspec.yml.

Detailed documentation is available in the AWS CodeBuild Documentation.

buildspec.yml
version: 0.2

phases:
  install:
    runtime-versions:
      nodejs: latest
    commands:
      # Set COMMIT_INFO variables to send Git specifics to Cypress Cloud when recording
      # https://docs.cypress.io/app/continuous-integration/overview#Git-information
      - export COMMIT_INFO_BRANCH="$(git rev-parse HEAD | xargs git name-rev |
        cut -d' ' -f2 | sed 's/remotes\/origin\///g')"
      - export COMMIT_INFO_MESSAGE="$(git log -1 --pretty=%B)"
      - export COMMIT_INFO_EMAIL="$(git log -1 --pretty=%ae)"
      - export COMMIT_INFO_AUTHOR="$(git log -1 --pretty=%an)"
      - export COMMIT_INFO_SHA="$(git log -1 --pretty=%H)"
      - export COMMIT_INFO_REMOTE="$(git config --get remote.origin.url)"
      - npm ci
  pre_build:
    commands:
      - npm run cy:verify
      - npm run cy:info
  build:
    commands:
      - npm run start:ci &
      - npx cypress run --record

How this buildspec works:

On push to this repository, this job will provision and start AWS-hosted Amazon Linux instance with Node.js for running the outlined pre_build and build for the declared commands within the commands section of the configuration.
AWS CodeBuild will checkout our code from our GitHub repository.
Finally, our buildspec.yml configuration will:
- Install npm dependencies
- Start the project web server (npm start:ci)
- Run the Cypress tests within our GitHub repository within Electron.

tip

Try it out

To try out the example above yourself, fork the Cypress Kitchen Sink example project and place the above AWS CodeBuild configuration in buildspec.yml.

Testing with Cypress Docker Images

As of version 0.2, CodeBuild does not provide a way to specify a custom image for single build configurations. One way to solve this is using an AWS CodeBuild batch build-list strategy.

Enabling batch builds

Per the AWS CodeBuild batch build documentation, AWS CodeBuild creates a separate build for each possible configuration combination for a batch build-list strategy. Therefore, AWS CodeBuild projects must be created or updated to run batch configuration.

Follow these steps to enable batch configuration for existing AWS CodeBuild projects:

Navigate to the AWS CodeBuild Console
Select the project
Click "Edit" --> "Batch Configuration"
Create "New service Role" and enter the name of the role
Leave all other options as optional
Click "Update batch configuration"
Start the Build

Cypress Amazon Public ECR

AWS CodeBuild offers a build-list strategy of different job configurations for a single job definition.

The build-list strategy offers a way to specify an image hosted on Docker Hub or the Amazon Elastic Container Registry (ECR).

Docker Images for running Cypress locally and in CI are published to the Amazon ECR Public Gallery.:

Read about Cypress docker variants to decide which image is best for your project.

buildspec.yml
version: 0.2

## AWS CodeBuild Batch configuration
## https://docs.aws.amazon.com/codebuild/latest/userguide/batch-build-buildspec.html

## Define build to run using the
## "cypress/browsers:22.12.0" image
## from the Cypress Amazon ECR Public Gallery
batch:
  fast-fail: false
  build-list:
    - identifier: cypress-e2e-tests
      env:
        image: public.ecr.aws/cypress-io/cypress/browsers:22.12.0

phases:
  install:
    runtime-versions:
      nodejs: latest
    commands:
      # Set COMMIT_INFO variables to send Git specifics to Cypress Cloud when recording
      # https://docs.cypress.io/app/continuous-integration/overview#Git-information
      - export COMMIT_INFO_BRANCH="$(git rev-parse HEAD | xargs git name-rev |
        cut -d' ' -f2 | sed 's/remotes\/origin\///g')"
      - export COMMIT_INFO_MESSAGE="$(git log -1 --pretty=%B)"
      - export COMMIT_INFO_EMAIL="$(git log -1 --pretty=%ae)"
      - export COMMIT_INFO_AUTHOR="$(git log -1 --pretty=%an)"
      - export COMMIT_INFO_SHA="$(git log -1 --pretty=%H)"
      - export COMMIT_INFO_REMOTE="$(git config --get remote.origin.url)"
      - npm ci
  pre_build:
    commands:
      - npm run cy:verify
      - npm run cy:info
  build:
    commands:
      - npm run start:ci &
      - npx cypress run --record --browser firefox

Caching Dependencies and Build Artifacts

Caching with AWS CodeBuild directly can be challenging. The Build caching in AWS CodeBuild document offers details on local or Amazon S3 caching.

Per the documentation, "Local caching stores a cache locally on a build host that is available to that build host only". This will not be useful during parallel test runs.

The "Amazon S3 caching stores the cache in an Amazon S3 bucket that is available across multiple build hosts". While this may sound useful, in practice the upload of cached dependencies can take some time. Furthermore, each worker will attempt to save it's dependency cache to Amazon S3, which increases build time significantly.

Beyond the scope of this guide, but AWS CodePipeline may be of use to cache the initial source, dependencies and build output for use in AWS CodeBuild jobs using AWS CodePipeline Input and Output Artifacts.

Reference the AWS CodePipeline integration with CodeBuild and multiple input sources and output artifacts sample example for details on how to configure a CodePipeline with an output artifact.

Parallelization

Cypress Cloud offers the ability to parallelize and group test runs along with additional insights and analytics for Cypress tests.

AWS CodeBuild offers a batch build-matrix strategy for declaring different job configurations for a single job definition. The batch build-matrix strategy provides an option to specify a container image for the job. Jobs declared within a build-matrix strategy can run in parallel which enables us run multiples instances of Cypress at same time as we will see later in this section.

See Enabling batch builds for instructions on how to enable batch builds.

info

The following configuration using the --parallel and --record flags to cypress run requires setting up recording test results to Cypress Cloud.

Parallelizing the build

To setup multiple containers to run in parallel, the build-matrix configuration uses a set of variables (CY_GROUP_SPEC and WORKERS) with a list of items specific to each group for the build.

The fields are delimited by a pipe (|) character as follows:

## Group Name | Browser | Specs | Cypress Configuration options (optional)

'UI-Chrome-Mobile|chrome|cypress/tests/ui/*|viewportWidth=375,viewportHeight=667'

The build-matrix will run all permutations delimited items.

buildspec.yml
batch:
  fast-fail: false
  build-matrix:
    # ...
    dynamic:
      env:
        # ...
        variables:
          CY_GROUP_SPEC:
            - 'UI-Chrome|chrome|cypress/tests/ui/*'
            - 'UI-Chrome-Mobile|chrome|cypress/tests/ui/*|viewportWidth=375,viewportHeight=667'
            - 'API|chrome|cypress/tests/api/*'
            - 'UI-Firefox|firefox|cypress/tests/ui/*'
            - 'UI-Firefox-Mobile|firefox|cypress/tests/ui/*|viewportWidth=375,viewportHeight=667'

During the install phase, we utilize shell scripting with the cut command to assign values from the delimited CY_GROUP_SPEC passed to the worker into shell variables that will be used in the build phase when running cypress run.

buildspec.yml
batch:
  # ...

phases:
  install:
    commands:
      # Set COMMIT_INFO variables to send Git specifics to Cypress Cloud when recording
      # https://docs.cypress.io//app/continuous-integration/overview#Git-information
      - export COMMIT_INFO_BRANCH="$(git rev-parse HEAD | xargs git name-rev |
        cut -d' ' -f2 | sed 's/remotes\/origin\///g')"
      - export COMMIT_INFO_MESSAGE="$(git log -1 --pretty=%B)"
      - export COMMIT_INFO_EMAIL="$(git log -1 --pretty=%ae)"
      - export COMMIT_INFO_AUTHOR="$(git log -1 --pretty=%an)"
      - export COMMIT_INFO_SHA="$(git log -1 --pretty=%H)"
      - export COMMIT_INFO_REMOTE="$(git config --get remote.origin.url)"
      - CY_GROUP=$(echo $CY_GROUP_SPEC | cut -d'|' -f1)
      - CY_BROWSER=$(echo $CY_GROUP_SPEC | cut -d'|' -f2)
      - CY_SPEC=$(echo $CY_GROUP_SPEC | cut -d'|' -f3)
      - CY_CONFIG=$(echo $CY_GROUP_SPEC | cut -d'|' -f4)
      - npm ci
## ...

To parallelize the runs, we need to add an additional variable to the build-matrix strategy, WORKERS.

buildspec.yml
batch:
  fast-fail: false
  build-matrix:
    # ...
    dynamic:
      env:
        # ...
        variables:
          CY_GROUP_SPEC:
            # ...
          WORKERS:
            - 1
            - 2
            - 3
            - 4
            - 5

info

Note

The WORKERS array is filled with filler (or dummy) items to provision the desired number of CI machine instances within the batch build-matrix strategy and will provide 5 workers to each group defined in the CY_GROUP_SPEC.

Finally, the script variables are passed to the call to cypress run.

buildspec.yml
phases:
  install:
    # ...
  build:
    commands:
      - npm start:ci &
      - npx cypress run --record --parallel --browser $CY_BROWSER --ci-build-id
        $CODEBUILD_INITIATOR --group "$CY_GROUP" --spec "$CY_SPEC" --config
        "$CY_CONFIG"

Using Cypress Cloud with AWS CodeBuild

caution

Dashboard analytics are dependent on your CI environment reliably providing commit SHA data (typically via an environment variable) which is not always present by default. This is not a problem for most users, but if you are facing integration issues with your CodeBuild setup, please make sure the git information is being sent properly by following these guidelines, or just see the example codebuild.yml file at the top of this page. If you are still facing issues after this, please contact us.

In the AWS CodeBuild configuration we have defined in the previous section, we are leveraging three useful features of Cypress Cloud:

Recording test results with the --record flag to Cypress Cloud.
- In-depth and shareable test reports.
- Visibility into test failures via quick access to Test Replay, error messages, stack traces, screenshots, videos, and contextual details.
- Integrating testing with the pull-request (PR) process via commit status check guards and convenient test report comments.
- Detecting flaky tests and surfacing them via Slack alerts or GitHub PR status checks.
Parallelizing test runs and optimizing their execution via intelligent load-balancing of test specs across CI machines with the --parallel flag.
Organizing and consolidating multiple cypress run calls by labeled groups into a single report within Cypress Cloud. In the example above we use the --group UI-Chrome flag to organize all UI tests for the Chrome browser into a group labeled "UI-Chrome" inside the Cypress Cloud report.

What you'll learn​

Basic Setup​

Testing with Cypress Docker Images​

Enabling batch builds​

Cypress Amazon Public ECR​

Caching Dependencies and Build Artifacts​

Parallelization​

Parallelizing the build​

Using Cypress Cloud with AWS CodeBuild​

Contents

What you'll learn

Basic Setup

Testing with Cypress Docker Images

Enabling batch builds

Cypress Amazon Public ECR

Caching Dependencies and Build Artifacts

Parallelization

Parallelizing the build

Using Cypress Cloud with AWS CodeBuild