Configuring CI Using GitHub Actions and Nx
There are two general approaches to setting up CI with Nx - using a single job or distributing tasks across multiple jobs. For smaller repositories, a single job is faster and cheaper, but once a full CI run starts taking 10 to 15 minutes, using multiple jobs becomes the better option. Nx Cloud's distributed task execution allows you to keep the CI pipeline fast as you scale. As the repository grows, all you need to do is add more agents.
Process Only Affected Projects With One Job on GitHub Actions
Below is an example of an GitHub Actions setup that runs on a single job, building and testing only what is affected. This uses the nx affected command to run the tasks only for the projects that were affected by that PR.
1name: CI
2on:
3 push:
4 branches:
5 # Change this if your primary branch is not main
6 - main
7 pull_request:
8
9# Needed for nx-set-shas when run on the main branch
10permissions:
11 actions: read
12 contents: read
13
14jobs:
15 main:
16 runs-on: ubuntu-latest
17 steps:
18 - uses: actions/checkout@v4
19 with:
20 fetch-depth: 0
21 # Cache node_modules
22 - uses: actions/setup-node@v3
23 with:
24 node-version: 20
25 cache: 'npm'
26 - run: npm ci
27 - uses: nrwl/nx-set-shas@v3
28 # This line is needed for nx affected to work when CI is running on a PR
29 - run: git branch --track main origin/main
30
31 - run: npx nx format:check
32 - run: npx nx affected -t lint,test,build --parallel=3
33Get the Commit of the Last Successful Build
GitHub can track the last successful run on the main branch and use this as a reference point for the BASE. The nrwl/nx-set-shas provides a convenient implementation of this functionality which you can drop into your existing CI config. To understand why knowing the last successful build is important for the affected command, check out the in-depth explanation in Actions's docs.
Distribute Tasks Across Agents on GitHub Actions
To set up Distributed Task Execution (DTE), you can run this generator:
❯
npx nx g ci-workflow --ci=github
Or you can copy and paste the workflow below:
1name: CI
2on:
3 push:
4 branches:
5 - main
6 pull_request:
7
8# Needed for nx-set-shas when run on the main branch
9permissions:
10 actions: read
11 contents: read
12
13jobs:
14 main:
15 name: Nx Cloud - Main Job
16 uses: nrwl/ci/.github/workflows/nx-cloud-main.yml@v0.13.0
17 with:
18 number-of-agents: 3
19 parallel-commands: |
20 npx nx-cloud record -- npx nx format:check
21 parallel-commands-on-agents: |
22 npx nx affected -t lint,test,build --parallel=2
23
24 agents:
25 name: Nx Cloud - Agents
26 uses: nrwl/ci/.github/workflows/nx-cloud-agents.yml@v0.13.0
27 with:
28 number-of-agents: 3
29This configuration is using two reusable workflows from the nrwl/ci repository. You can check out the full API for those workflows.
The first workflow is for the main job:
1 uses: nrwl/ci/.github/workflows/nx-cloud-main.yml@v0.13.0
2The parallel-commands script will be run on the main job. The parallel-commands-on-agents script will be distributed across the available agents.
The second workflow is for the agents:
1 uses: nrwl/ci/.github/workflows/nx-cloud-agents.yml@v0.13.0
2The number-of-agents property controls how many agent jobs are created. Note that this property should be the same number for each workflow.
The number-of-agents property and the --parallel flag both parallelize tasks, but in different ways. The way this workflow is written, there will be 3 agents running tasks and each agent will try to run 2 tasks at once. If a particular CI run only has 2 tasks, only one agent will be used.
Custom Distributed CI with Nx Cloud on GitHub Actions
Our reusable GitHub workflow represents a good set of defaults that works for a large number of our users. However, reusable GitHub workflows come with their limitations.
If the reusable workflow above doesn't satisfy your needs you should create a custom workflow. If you were to rewrite the reusable workflow yourself, it would look something like this:
1name: CI
2on:
3 push:
4 branches:
5 - main
6 pull_request:
7
8# Needed for nx-set-shas when run on the main branch
9permissions:
10 actions: read
11 contents: read
12
13env:
14 NX_CLOUD_DISTRIBUTED_EXECUTION: true # this enables DTE
15 NX_CLOUD_DISTRIBUTED_EXECUTION_AGENT_COUNT: 3 # expected number of agents
16 NX_BRANCH: ${{ github.event.number || github.ref_name }}
17 NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
18 NPM_TOKEN: ${{ secrets.NPM_TOKEN }} # this is needed if our pipeline publishes to npm
19
20jobs:
21 main:
22 name: Nx Cloud - Main Job
23 runs-on: ubuntu-latest
24 steps:
25 - uses: actions/checkout@v4
26 name: Checkout [Pull Request]
27 if: ${{ github.event_name == 'pull_request' }}
28 with:
29 # By default, PRs will be checked-out based on the Merge Commit, but we want the actual branch HEAD.
30 ref: ${{ github.event.pull_request.head.sha }}
31 # We need to fetch all branches and commits so that Nx affected has a base to compare against.
32 fetch-depth: 0
33
34 - uses: actions/checkout@v4
35 name: Checkout [Default Branch]
36 if: ${{ github.event_name != 'pull_request' }}
37 with:
38 # We need to fetch all branches and commits so that Nx affected has a base to compare against.
39 fetch-depth: 0
40
41 # Set node/npm/yarn versions using volta
42 - uses: volta-cli/action@v4
43 with:
44 package-json-path: '${{ github.workspace }}/package.json'
45
46 - name: Use the package manager cache if available
47 uses: actions/setup-node@v3
48 with:
49 node-version: 20
50 cache: 'npm'
51
52 - name: Install dependencies
53 run: npm ci
54
55 - name: Check out the default branch
56 run: git branch --track main origin/main
57
58 - name: Initialize the Nx Cloud distributed CI run and stop agents when the build tasks are done
59 run: npx nx-cloud start-ci-run --stop-agents-after=build
60
61 - name: Run commands in parallel
62 run: |
63 # initialize an array to store process IDs (PIDs)
64 pids=()
65
66 # function to run commands and store the PID
67 function run_command() {
68 local command=$1
69 $command & # run the command in the background
70 pids+=($!) # store the PID of the background process
71 }
72
73 # list of commands to be run on main has env flag NX_CLOUD_DISTRIBUTED_EXECUTION set to false
74 run_command "NX_CLOUD_DISTRIBUTED_EXECUTION=false npx nx-cloud record -- npx nx format:check"
75
76 # list of commands to be run on agents
77 run_command "npx nx affected -t lint,test,build --parallel=3"
78
79 # wait for all background processes to finish
80 for pid in ${pids[*]}; do
81 if ! wait $pid; then
82 exit 1 # exit with an error status if any process fails
83 fi
84 done
85
86 exit 0 # exits with success status if a all processes complete successfully
87
88 agents:
89 name: Agent ${{ matrix.agent }}
90 runs-on: ubuntu-latest
91 strategy:
92 matrix:
93 # Add more agents here as your repository expands
94 agent: [1, 2, 3]
95 steps:
96 - name: Checkout
97 uses: actions/checkout@v4
98
99 # Set node/npm/yarn versions using volta
100 - uses: volta-cli/action@v4
101 with:
102 package-json-path: '${{ github.workspace }}/package.json'
103
104 - name: Use the package manager cache if available
105 uses: actions/setup-node@v3
106 with:
107 node-version: 20
108 cache: 'npm'
109
110 - name: Install dependencies
111 run: npm ci
112
113 - name: Start Nx Agent ${{ matrix.agent }}
114 run: npx nx-cloud start-agent
115 env:
116 NX_AGENT_NAME: ${{ matrix.agent }}
117There are comments throughout the workflow to help you understand what is happening in each section.