A workflow for managing private submodules in a private repository without personal access tokens for Github actions

Background

Ever since Travis CI decided to drink corporate kool-aid, the search for a new CI has been brought to the fore again. Github Actions seem uniquely suited for private repositories, since most CIs bill for private repos. However, the basic authentication setup for the checkout action involves using one SSH key, effectively a personal access token, for both the main project and all submodules. This is untenable for anyone working with a team.

Solution

The fix, as it were, is in two steps. We will first require a deploy key to be set for the private submodule, and then store the private portion in the private repo. We will begin with a concrete setup.

Setup

Consider a standard C++ build setup:

 1name: Fake secret project
 2
 3on:
 4  push:
 5    branches: [master, development]
 6  pull_request:
 7    branches: [master]
 8
 9jobs:
10  build:
11    runs-on: ${{ matrix.os }}
12
13    strategy:
14      max-parallel: 4
15      matrix:
16        os: [ubuntu-18.04]
17        cpp-version: [g++-7, g++-9, clang++]
18
19    steps:
20      - uses: actions/checkout@v2
21      - name: build
22        env:
23          CXX: ${{ matrix.cpp-version }}
24        run: |
25          mkdir build && cd build
26          cmake -DCMAKE_CXX_COMPILER="$CXX" -DCMAKE_CXX_FLAGS="-std=c++11" ../
27          make -j$(nproc)          
28      - name: run
29        run: |
30          ./super_secret

Key Generation

This section is standard. We will generate a deploy key essentially. Note that it isn’t necessary to set a password, using one would only minimally improve security and bring in some annoying script modifications.

1# Anywhere safe
2ssh-keygen -t rsa -b 4096 -C "Fake Deployment Key" -f 'priv_sub_a' -N ''

Private Submodule Repo

We will store the public portion of the key as a deploy key in the private submodule repository.

1# Copy contents
2cat priv_sub_a.pub | xclip -selection clipboard

Note that, as shown in Fig. 1 we do not need to give write access to this key.

Private Project Repo

Now we will need the private portion of the key as a secret in the private project repository (see Fig. 2).

1# Copy contents of private key
2cat priv_sub_a | xclip -selection clipboard
Figure 1: Secret setup in the private project

Figure 1: Secret setup in the private project

Workflow Modifications

Now we will simply update our workflow 1. We will simply add the following step:

1- name: get_subm
2  env:
3    SSHK: ${{ secrets.SUB_SSHK_A }}
4  run: |
5    mkdir -p $HOME/.ssh
6    echo "$SSHK" > $HOME/.ssh/ssh.key
7    chmod 600 $HOME/.ssh/ssh.key
8    export GIT_SSH_COMMAND="ssh -i $HOME/.ssh/ssh.key"
9    git submodule update --init --recursive

This will work for a single private submodule and multiple public submodules. For multiple private submodules, we would not initialize them recursively, but instead use a separate key for each.

 1- name: get_subm_a
 2  env:
 3    SSHK: ${{ secrets.SUB_SSHK_A }}
 4  run: |
 5    mkdir -p $HOME/.ssh
 6    echo "$SSHK" > $HOME/.ssh/ssh.key
 7    chmod 600 $HOME/.ssh/ssh.key
 8    export GIT_SSH_COMMAND="ssh -i $HOME/.ssh/ssh.key"
 9    git submodule update --init -- <specific relative path to submodule A>    
10- name: get_subm_b
11  env:
12    SSHK: ${{ secrets.SUB_SSHK_B }}
13  run: |
14    mkdir -p $HOME/.ssh
15    echo "$SSHK" > $HOME/.ssh/ssh.key
16    chmod 600 $HOME/.ssh/ssh.key
17    export GIT_SSH_COMMAND="ssh -i $HOME/.ssh/ssh.key"
18    git submodule update --init -- <specific relative path to submodule B>

Note that it is not possible to use the same SSH key in multiple submodule repositories, as each deploy key can only be associated with one repository.

Complete Workflow

Putting the above steps together, for the case of a single private submodule and multiple public submodules, we have:

 1name: Fake secret project
 2
 3on:
 4  push:
 5    branches: [master, development]
 6  pull_request:
 7    branches: [master]
 8
 9jobs:
10  build:
11    runs-on: ${{ matrix.os }}
12
13    strategy:
14      max-parallel: 4
15      matrix:
16        os: [ubuntu-18.04]
17        cpp-version: [g++-7, g++-9, clang++]
18
19    steps:
20      - uses: actions/checkout@v2
21      - name: get_subm
22        env:
23          SSHK: ${{ secrets.SUB_SSHK_A }}
24        run: |
25          mkdir -p $HOME/.ssh
26          echo "$SSHK" > $HOME/.ssh/ssh.key
27          chmod 600 $HOME/.ssh/ssh.key
28          export GIT_SSH_COMMAND="ssh -i $HOME/.ssh/ssh.key"
29          git submodule update --init --recursive          
30      - name: build
31        env:
32          CXX: ${{ matrix.cpp-version }}
33        run: |
34          mkdir build && cd build
35          cmake -DCMAKE_CXX_COMPILER="$CXX" -DCMAKE_CXX_FLAGS="-std=c++11" ../
36          make -j$(nproc)          
37      - name: run
38        run: |
39          ./super_secret

Conclusions

We have demonstrated a minimally invasive setup for working with a private submodule, which is trivially extensible to multiple such submodules. With this, it appears that GH actions might a viable option (as opposed to say, Wercker), at least for private teams. A more full comparative post might be warranted at a later date.

  1. submodule-checkout uses a similar concept but unfortunately does not extend to multiple submodules and the submodules are checked out as root ↩︎