Automating documenation deployment with Travis, rake and nix


In the previous post we generated documentation using Doxygen with Exhale to handle Sphinx. Now we will clean up the earlier workflow with rake and ensure the environment is reproducible with nix while deploying to Travis CI.


  1. Documenting C++ with Doxygen and Sphinx - Exhale
  2. Publishing Doxygen and Sphinx with Nix and Rake <– You are here
  3. Documenting C++ with Doxygen and Sphinx - doxyrest
  4. Adding Tutorials to Sphinx Projects


A quick reminder of the setup we generated in the last post:

1tree -d $prj/ -L 2
│  ├──Doxygen
│  └──Sphinx
│  └──pkgs
│  └──symengine

We had further setup files to enable documentation generation with a manual two stage process (handling doxygen and sphinx separately).

1cd docs/Doxygen
2doxygen Doxyfile-prj.cfg
3cd ../Sphinx
4make html
5mv build/html ../../public

This might be extracted into a simple script, and then we might decide to have a script and then we might try to replicate all the functionality of a good build system with scripts.

Thankfully, we will instead start with a build script defined as above to transition to nix, before using an actual build tool for our dirty work.

Adding Nix

It wouldn’t make sense for me to not stick nix into this. I recall the dark days of setting up Dockerfiles to ensure reproducible environments on Travis.

At this point one might assume we will leverage the requirements.txt based workflow described earlier in Niv and Mach-Nix for Nix Python. While this would make sense, there are two barriers to its usage:

  • It is slower than a poetry build, as dependency resolution is performed
  • It does not play well with existing projects
    • Most python projects do not rely solely on requirements.txt 1


Recall that as sphinx is originally meant for and most often used for Python projects, we will need to consider the possibility (remote though it is) that there might be users who would like to test the documentation without setting up nix.

Thus we will look to the poetry2nix project instead. We note the following:

  • The poetry2nix setup is faster (as it consumes a lockfile instead of solving dependencies from requirements.txt)
    • mach-nix however, is more flexible and can make use of the poetry2nix overrides
  • In a strange chicken and egg problem, we will have to manually generate the lockfile, thereby creating an impure poetry project for every update, though the nix setup will not need it later
    • This is one of the major reasons to prefer mach-nix for newer projects

Shell Environment

We prep our sources in the usual way, by running niv init in the project root to generate the nix/ folder and the sources therein. With all that in mind, the shell.nix file at this point is fairly standard, keeping the general niv setup in mind (described in a previous Nix tutorial):

1# -*- mode: nix-mode -*-
3  sources = import ./nix/sources.nix;
4  pkgs = import sources.nixpkgs { };
5  customPython = pkgs.poetry2nix.mkPoetryEnv { projectDir = ./.; };
6in pkgs.mkShell {
7  buildInputs = with pkgs; [ doxygen customPython rake darkhttpd ];

Where the most interesting aspect is that the projectDir is to be the location of the project root, though both poetrylock and pyproject variables are supported.


We consider the problem of refactoring the script:

1#!/usr/bin/env bash
2cd docs/Doxygen
3doxygen Doxyfile-prj.cfg
4cd ../Sphinx
5make html
6mv build/html ../../public

Without resorting to methods such as nix-shell --run --pure.

Nix Bash

Script in hand, we would like to be able to run it directly in the nix environment. We modify the script as follows:

 1#! /usr/bin/env nix-shell
 2#! nix-shell deps.nix -i bash
 4# Build Doxygen
 5cd docs/Doxygen
 6doxygen Doxyfile-syme.cfg
 8# Build Sphinx
 9cd ../Sphinx
10make html
11mv build/html ../../public
13# Local Variables:
14# mode: shell-script
15# End:

This calls on a deps.nix2 which we shall generate in a manner very reminiscent of the shell.nix 3 as follows:

2  sources = import ./../nix/sources.nix;
3  pkgs = import sources.nixpkgs { };
4  customPython = pkgs.poetry2nix.mkPoetryEnv { projectDir = ./../.; };
5in pkgs.runCommand "dummy" {
6  buildInputs = with pkgs; [ doxygen customPython ];
7} ""

Only the paths have changed, and instead of creating and returning a shell environment with mkShell we instead “run” a derivation instead. At this point we can run this simply as:


This is reasonably ready (as a first draft) for being incorporated into a continuous integration workflow.

Travis CI

Seeing as Travis provides first class nix support, as well as excellent integration with GitHub, we will prefer it.


A minor but necessary evil is setting up a PAP (personal access token) from here. Depending on what repositories are being used, the scope should encompass repo permissions (minimally public_repo), and admin:org permissions might be required.

Having obtained the token, we will need to navigate to the Settings section on the Travis web-UI and add the token as an environment variable, we might be partial to a name like GH_TOKEN.

Figure 1: Settings at

Figure 1: Settings at

Build Configuration

We will leverage the following configuration:

 1language: nix
 4  - sudo mkdir -p /etc/nix
 5  - echo "substituters = file://$HOME/" | sudo tee -a /etc/nix/nix.conf > /dev/null
 6  - echo 'require-sigs = false' | sudo tee -a /etc/nix/nix.conf > /dev/null
 9  - sudo mkdir -p /etc/nix && echo 'sandbox = true' | sudo tee /etc/nix/nix.conf
12  - scripts/
15  - mkdir -p $HOME/
16  - nix copy --to file://$HOME/ -f shell.nix buildInputs
19  nix: true
20  directories:
21    - $HOME/
24  provider: pages
25  local_dir: ./public/
26  skip_cleanup: true
27  github_token: $GH_TOKEN # Set in the settings page of your repository, as a secure variable
28  keep_history: true
29  target_branch: master # Required for user pages
30  on:
31    branch: src

Where all the action is essentially in script and deploy. Note however, that the before_cache step should change if there is a default.nix instead. We will in this case, consider the situation of having an organization or user page being the deploy target.


Usable though the preceding setting is, it is still rather unwieldy in that:

  • there are a bunch of artifacts which need to be cleaned manually
  • it is fragile and tied to the folder names

We can fix this with any of the popular build systems, however here we will focus on the excellent rake 4. We shall commit to our course of action by removing make.

1cd docs/Sphinx
2rm Makefile make.bat # other make cruft



We will begin by requiring rake and setting basic variables.

1require 'rake'
3CWD = File.expand_path(__dir__)
4DOXYFILE = "Doxyfile-prj.cfg"
5OUTDIR = File.join(CWD,"public")
6SPHINXDIR = File.join(CWD,"docs/Sphinx")

This section should give a fairly clear idea of how the Rakefile itself is essentially pure ruby code. We are now beginning to have more holistic control of how our project is structured.


The general form of a task is simply:

1desc "Blah blah"
2task :name do
3# Something

Some variations of this will be considered when appropriate.


A clean task is a good first task, being as it is almost trivial in all build systems.

1desc "Clean the generated content"
2task :clean do
3  rm_rf "public"
4  rm_rf "docs/Doxygen/gen_docs"
5  rm_rf "docs/Sphinx/build"


We will use the lightweight darkhttpd server for our generated documentation.

1desc "Serve site with darkhttpd"
2task :darkServe, [:port] do |task, args|
3  args.with_defaults(:port => "1337")
4  sh "darkhttpd #{OUTDIR} --port #{args.port}"

Note that we have leveraged the args system in this case, and also used the top-level OUTDIR variable.


Since the doxygen output is a pre-requisite, it makes sense to set it up early on.

1desc "Build doxygen"
2task :mkDoxy do
3  Dir.chdir(to = File.join(CWD,"docs/Doxygen"))
4  system('doxygen', DOXYFILE)


This task will depend on having the doxygen output, so we will express this idiomatically by making the doxygen task run early on.

1desc "Build Sphinx"
2task :mkSphinx, [:builder] => ["mkDoxy"] do |task, args|
3  args.with_defaults(:builder => "html")
4  Dir.chdir(to = File.join(CWD,"docs/Sphinx"))
5  sh "poetry install"
6  sh "poetry run sphinx-build source #{OUTDIR} -b #{args.builder}"

There are some subtleties here, notably:

  • The task is meant to run without nix
  • We use the args setup as before

No Nix Meta

With this we can now set up a task to build the documentation without having nix.

1desc "Build site without Nix"
2task :noNixBuild => "mkSphinx" do
3  Rake::Task["darkServe"].execute

The main take-away here is that we finally call the Rake library itself, but within the task, which means the dependency tree is respected and we get doxygen->sphinx->darkhttpd as required.

Nix Builder

For nix use we note that we are unable to enter the nix environment from within the Rakefile itself. We work around this by being more descriptive.

1desc "Build Nix Sphinx, use as nix-shell --run 'rake mkNixDoc' --pure"
2task :mkNixDoc, [:builder] => "mkDoxy" do |task, args|
3  args.with_defaults(:builder => "html")
4  Dir.chdir(to = SPHINXDIR)
5  sh "sphinx-build source #{OUTDIR} -b #{args.builder}"

Final Form

The final Rakefile shall be (with a default task defined):

 1require 'rake'
 3# Variables
 4CWD = File.expand_path(__dir__)
 5DOXYFILE = "Doxyfile-prj.cfg"
 6OUTDIR = File.join(CWD,"public")
 7SPHINXDIR = File.join(CWD,"docs/Sphinx")
 9# Tasks
10task :default => :darkServe
12desc "Clean the generated content"
13task :clean do
14  rm_rf "public"
15  rm_rf "docs/Doxygen/gen_docs"
16  rm_rf "docs/Sphinx/build"
19desc "Serve site with darkhttpd"
20task :darkServe, [:port] do |task, args|
21  args.with_defaults(:port => "1337")
22  sh "darkhttpd #{OUTDIR} --port #{args.port}"
25desc "Build Nix Sphinx, use as nix-shell --run 'rake mkNixDoc' --pure"
26task :mkNixDoc, [:builder] => "mkDoxy" do |task, args|
27  args.with_defaults(:builder => "html")
28  Dir.chdir(to = SPHINXDIR)
29  sh "sphinx-build source #{OUTDIR} -b #{args.builder}"
32desc "Build site without Nix"
33task :noNixBuild => "mkSphinx" do
34  Rake::Task["darkServe"].execute
37desc "Build doxygen"
38task :mkDoxy do
39  Dir.chdir(to = File.join(CWD,"docs/Doxygen"))
40  system('doxygen', DOXYFILE)
43desc "Build Sphinx"
44task :mkSphinx, [:builder] => ["mkDoxyRest"] do |task, args|
45  args.with_defaults(:builder => "html")
46  Dir.chdir(to = File.join(CWD,"docs/Sphinx"))
47  sh "poetry install"
48  sh "poetry run sphinx-build source #{OUTDIR} -b #{args.builder}"


We are now in a position to fix our travis build configuration. Simply replace the old and fragile script section with the following:

2  - nix-shell --run "rake mkNixDoc" --show-trace --verbose --pure


As a bonus section, consider the addition of the following .envrc for those who keep multiple ruby versions:

1eval "$(rbenv init -)"
2rbenv shell 2.6.2
3rake -T

Activate this with the usual direnv allow. This has the added benefit of listing the defined tasks when cd‘ing into the project directory.


A lot has happened on the tooling end, even though the documentation itself has not been updated further. We have managed to setup a robust environment which is both reproducible and also amenable to users who do not have nix. We have also setup a build system, which can help us in many more ways as well (asset optimization through the rails pipeline). In the next post, we will return to the documentation itself for further tinkering.

  1. Poetry and Pipenv come to mind ↩︎

  2. Chris Warbo has a good introduction to the nix shebang ↩︎

  3. In this instance, we could have simply called on shell.nix instead, but it illustrates a more general concept ↩︎

  4. Avdi’s blog has a fantastic introduction to rake and Rakefiles ↩︎