Background

My move away from the powerful, but unimaginatively named HPC clusters of IITK 1 brought me in close contact with the Lua based 2 lmod module system. Rather than fall into the rabbit hole of brew we will leverage the existing system to add our new libraries. Not finding any good collections of these composable environments, and having failed once before to install Nix as a user without admin access, I decided to start my own collection of Lmod recipies. The rest of this post details the installation proceedure to be carried out in conjunction with the hzHPC_lmod repo.

Setting Up

These are reproduced from the repo for completeness.

git clone https://github.com/kobus-v-schoor/dotgit.git
mkdir -p ~/.local/bin
cp -r dotgit/bin/dotgit* ~/.local/bin
cat dotgit/bin/bash_completion >> ~/.bash_completion
rm -rf dotgit

I actually strongly suggest using a target from my Dotfiles in conjunction with this, but it isn’t really required, so:

~/.local/bin/dotgit restore hzhpc

Note that because of the suggested separation, I have not opted to setup a shell or even ensure that there are scripts here to help keep module in your path. Those are in my Dotfiles. If, you opt to not use these dotfiles, then do not run the ml load commands.

LMod Libraries

Note that: garpur already has lmod and a module for GNU gcc 9.2.0

The scripts in this post will also be part of the repo, but keep in mind that these are not meant to be robust ways to install anything, and every command should be run by hand because things will probably break badly.

GMP

myprefix=$HOME/.hpc/gcc/gmp/6.2.0
export PATH
wget https://gmplib.org/download/gmp/gmp-6.2.0.tar.xz
tar xfv gmp-6.2.0.tar.xz
cd gmp-6.2.0
./configure --prefix=$myprefix    \
            --enable-cxx     \
            --docdir=$myprefix/doc/gmp-6.1.2
make -j$(nproc)
make install

MPFR

myprefix=$HOME/.hpc/gcc/mpfr/4.1.0
export PATH
wget https://www.mpfr.org/mpfr-current/mpfr-4.1.0.tar.xz
tar xfv mpfr-4.1.0.tar.xz
cd mpfr-4.1.0
./configure --prefix=$myprefix    \
            --enable-thread-safe     \
            --with-gmp=$HOME/.hpc/gcc/gmp/6.2.0 \
            --docdir=$myprefix/doc/mpfr-4.1.0
make -j$(nproc)
make install

MPC

myprefix=$HOME/.hpc/gcc/mpc/1.2.0
export PATH
wget https://ftp.gnu.org/gnu/mpc/mpc-1.2.0.tar.gz
tar xfv mpc-1.2.0.tar.gz
cd mpc-1.2.0
./configure --prefix=$myprefix    \
            --with-gmp=$HOME/.hpc/gcc/gmp/6.2.0 \
            --with-mpfr=$HOME/.hpc/gcc/mpfr/4.1.0 \
            --docdir=$myprefix/doc/mpc-1.2.0
make -j$(nproc)
make install

GCC 9.2.0

mkdir -p ~/tmpHPC
cd $HOME/tmpHPC
myprefix=$HOME/.hpc/gcc/9.2.0
export PATH
export LIBRARY_PATH=/usr/lib64/:$LIBRARY_PATH
wget https://ftp.gnu.org/gnu/gcc/gcc-9.2.0/gcc-9.2.0.tar.xz
tar xfv gcc-9.2.0.tar.xz
cd gcc-9.2.0
case $(uname -m) in
  x86_64)
    sed -e '/m64=/s/lib64/lib/' \
        -i.orig gcc/config/i386/t-linux64
  ;;
esac
mkdir -p build                                         &&
cd    build                                            &&

SED=sed                               \
../configure --prefix=$myprefix            \
             --enable-languages=c,c++,fortran \
             --disable-multilib       \
             --with-gmp=$HOME/.hpc/gcc/gmp/6.2.0 \
             --with-mpfr=$HOME/.hpc/gcc/mpfr/4.1.0 \
             --with-mpc=$HOME/.hpc/gcc/mpc/1.2.0 \
             --disable-bootstrap      \
             --with-system-zlib
make -j$(nproc)
ml load gcc/9.2.0

Autotools

Following the standard approach outlined in the GNU Autotools FAQ:

mkdir -p ~/tmpHPC
cd $HOME/tmpHPC
myprefix=$HOME/.hpc/autotools
export PATH
wget http://ftp.gnu.org/gnu/m4/m4-1.4.18.tar.gz
wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz
wget http://ftp.gnu.org/gnu/automake/automake-1.16.2.tar.gz
wget http://ftp.gnu.org/gnu/libtool/libtool-2.4.6.tar.gz
gzip -dc m4-1.4.18.tar.gz | tar xvf -
gzip -dc autoconf-2.69.tar.gz | tar xvf -
gzip -dc automake-1.16.2.tar.gz | tar xvf -
gzip -dc libtool-2.4.6.tar.gz | tar xvf -
cd m4-1.4.18
./configure -C --prefix=$myprefix/m4/1.4.18 && make -j$(nproc) && make install
cd ../autoconf-2.69
./configure -C --prefix=$myprefix/autoconf/2.69 && make -j$(nproc) && make install
cd ../automake-1.16.2
./configure -C --prefix=$myprefix/automake/1.16.2 && make -j$(nproc) && make install
cd ../libtool-2.4.6
./configure -C --prefix=$myprefix/libtool/2.4.6 && make -j$(nproc) && make install
ml load autotools/autotools

We also need the archive.

myprefix=$HOME/.hpc/autotools
git clone git://git.sv.gnu.org/autoconf-archive.git
cd autoconf-archive

Combined with the lmod manual gives rise to the following definiton (roughly the same for each one):

local home    = os.getenv("HOME")
local version = myModuleVersion()
local pkgName = myModuleName()
local pkg     = pathJoin(home,".hpc",pkgName,version,"bin")
prepend_path("PATH", pkg)

We will no longer bother with the module definitions for the rest of this post, as they are handled and documented in the repo.

Perl

This is essentially the setup from the main docs.

# Get Perl
curl -L http://xrl.us/installperlnix | bash
# Use Perl
ml use perl/5.28.0
cpanm ExtUtils::MakeMaker # For git
ml load perl/5.28.0

Git

This is very similar to the previous approach. However, since by default the system perl was being picked up, some slight changes have been made.

myprefix=$HOME/.hpc/git/2.9.5
PATH=$myprefix/bin:$PATH
export PATH
wget https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.9.5.tar.gz
gzip -dc git-2.9.5.tar.gz | tar xvf -
cd git-2.9.5
./configure --with-perl=$(which perl) -C --prefix=$myprefix
make -j $(nproc)
make install
ml load git/2.9.5

Caveat

Also, for TRAMP, we would prefer having a more constant path, so we can set up a symlink:

mkdir ~/.hpc/bin
ln ~/.hpc/git/2.9.5/bin/git ~/.hpc/bin/git

Boost

The boost website is utterly incomprehensible. As is the documentation. Also, fun fact, the move from svn makes things worse. Thankfully, a quick dive into the slightly better Github wiki led to this nugget:

git clone --recursive https://github.com/boostorg/boost.git
cd boost
git checkout tags/boost-1.73.0 # or whatever branch you want to use
./bootstrap.sh
./b2 headers

This means we’re almost done!

./b2
./b2 install --prefix=$HOME/.hpc/boost/boost-1.73.0
ml load boost/boost-1.73.0

Pkg-Config

myprefix=$HOME/.hpc/pkg-config/0.29.2
wget https://pkg-config.freedesktop.org/releases/pkg-config-0.29.2.tar.gz
gzip -dc pkg-config-0.29.2.tar.gz | tar xvf -
cd pkg-config-0.29.2
./configure --prefix=$myprefix --with-internal-glib --disable-host-tool --docdir=$myprefix/share/doc/pkg-config-0.29.2
mkdir $myprefix/lib
make -j $(nproc)
make install
ml load pkg-config/0.29.2

Zlib

myprefix=$HOME/.hpc/zlib/1.2.11
wget http://zlib.net/zlib-1.2.11.tar.gz
gzip -dc zlib-1.2.11.tar.gz | tar xvf -
cd zlib-1.2.11
./configure --prefix=$myprefix
make -j $(nproc)
make install
ml load zlib/1.2.11

XZ Utils

myprefix=$HOME/.hpc/xz/5.2.5
wget https://tukaani.org/xz/xz-5.2.5.tar.gz
gzip -dc xz-5.2.5.tar.gz | tar xvf -
cd xz-5.2.5
./configure --prefix=$myprefix --enable-threads=yes
make -j $(nproc)
make install
ml load xz/5.2.5

OpenSSL

myprefix=$HOME/.hpc/openssl/1.1.1d
wget https://www.openssl.org/source/openssl-1.1.1d.tar.gz
gzip -dc openssl-1.1.1d.tar.gz | tar xvf -
cd openssl-1.1.1d
./config --prefix=$myprefix --openssldir=$myprefix/etc/ssl shared zlib-dynamic
make -j $(nproc)
make install
ml load openssl/1.1.1d

Cmake

myprefix=$HOME/.hpc/cmake/3.18.1
wget https://github.com/Kitware/CMake/releases/download/v3.18.1/cmake-3.18.1.tar.gz
gzip -dc cmake-3.18.1.tar.gz | tar xvf -
cd cmake-3.18.1
./bootstrap --prefix=$myprefix
make -j $(nproc)
make install
ml load cmake/3.18.1

GNU-Make

myprefix=$HOME/.hpc/make/4.3
wget http://ftp.gnu.org/gnu/make/make-4.3.tar.gz
gzip -dc make-4.3.tar.gz | tar xvf -
cd make-4.3
./configure --prefix=$myprefix
make -j $(nproc)
make install
ml load make/4.3

Brotli

myprefix=$HOME/.hpc/brotli/1.0.1
git clone https://github.com/bagder/libbrotli
cd libbrotli
./autogen.sh
./configure --prefix=$myprefix
make -j $(nproc)
make install
ml load brotli/1.0.1

ncurses

We will need to manually ensure the paths for pkg-config are in a feasible location.

myprefix=$HOME/.hpc/ncurses/6.2
wget https://invisible-mirror.net/archives/ncurses/ncurses-6.2.tar.gz
gzip -dc ncurses-6.2.tar.gz | tar xvf -
cd ncurses-6.2
./configure --prefix=$myprefix --enable-widec --enable-pc-files --with-shared
make -j $(nproc)
make install
mkdir pkgconfig
cp misc/formw.pc misc/menuw.pc misc/ncurses++w.pc misc/ncursesw.pc misc/panelw.pc pkgconfig/
mv pkgconfig $myprefix/lib/
ml load ncurses/6.2

texinfo

myprefix=$HOME/.hpc/texinfo/6.7
wget http://ftp.gnu.org/gnu/texinfo/texinfo-6.7.tar.gz
gzip -dc texinfo-6.7.tar.gz | tar xvf -
cd texinfo-6.7
./configure --prefix=$myprefix
make -j $(nproc)
make install
ml load texinfo/6.7

gperf

myprefix=$HOME/.hpc/gperf/3.1
wget http://ftp.gnu.org/gnu/gperf/gperf-3.1.tar.gz
gzip -dc gperf-3.1.tar.gz | tar xvf -
cd gperf-3.1
./configure --prefix=$myprefix
make -j $(nproc)
make install
ml load gperf/3.1

libseccomp

There is a bug, which requires modifying src/system.c to change __NR_seccomp to _nr_seccomp.

myprefix=$HOME/.hpc/libseccomp/2.5.0
git clone https://github.com/seccomp/libseccomp
cd libseccomp
git checkout tags/v2.5.0
./autogen.sh
./configure --prefix=$myprefix
make -j $(nproc)
make install
ml load libseccomp/2.5.0

Alternatively, it is easier to work with an older version.

myprefix=$HOME/.hpc/libseccomp/2.4.4
wget https://github.com/seccomp/libseccomp/releases/download/v2.4.4/libseccomp-2.4.4.tar.gz
tar xfv libseccomp-2.4.4.tar.gz
cd libseccomp-2.4.4
./configure --prefix=$myprefix
make -j $(nproc)
make install
ml load libseccomp/2.4.4

BDWGC

myprefix=$HOME/.hpc/bdwgc/8.0.4
wget https://github.com/ivmai/bdwgc/releases/download/v8.0.4/gc-8.0.4.tar.gz
gzip -dc gc-8.0.4.tar.gz | tar xvf -
cd gc-8.0.4
./configure --prefix=$myprefix --enable-cplusplus
make -j $(nproc)
make install
ml load bdwgc/8.0.4

pcre

We will prep both pcre2 and pcre.

myprefix=$HOME/.hpc/pcre2/10.35 wget https://ftp.pcre.org/pub/pcre/pcre2-10.35.tar.gz gzip -dc pcre2-10.35.tar.gz | tar xvf - cd pcre2-10.35 ./configure –prefix=$myprefix \
–enable-pcre2-16 \
–enable-pcre2-32 \
–enable-pcre2grep-libz make -j $(nproc) make install ml load pcre2/10.35

myprefix=$HOME/.hpc/pcre/8.44 wget https://ftp.pcre.org/pub/pcre/pcre-8.44.tar.gz gzip -dc pcre-8.44.tar.gz | tar xvf - cd pcre-8.44 ./configure –prefix=$myprefix \
–enable-pcre-16 \
–enable-pcre-32 \
–enable-pcregrep-libz make -j $(nproc) make install ml load pcre/8.44

bison

myprefix=$HOME/.hpc/bison/3.7.1
wget http://ftp.gnu.org/gnu/bison/bison-3.7.1.tar.gz
gzip -dc bison-3.7.1.tar.gz | tar xvf -
cd bison-3.7.1
./configure --prefix=$myprefix
make -j $(nproc)
make install
ml load bison/3.7.1

flex

myprefix=$HOME/.hpc/flex/2.6.4
wget https://github.com/westes/flex/releases/download/v2.6.4/flex-2.6.4.tar.gz
gzip -dc flex-2.6.4.tar.gz | tar xvf -
cd flex-2.6.4
./configure --prefix=$myprefix
make -j $(nproc)
make install
ml load flex/2.6.4

jq

myprefix=$HOME/.hpc/jq/1.6
git clone https://github.com/stedolan/jq.git
cd jq
git submodule update --init
git checkout tags/jq-1.6
autoreconf -fi
./configure --prefix=$myprefix --with-oniguruma=builtin
make -j $(nproc)
make install
ml load jq/1.6

bzip2

Needed to manually configure it as shown here

myprefix=$HOME/.hpc/bzip2/1.0.8
wget https://www.sourceware.org/pub/bzip2/bzip2-1.0.8.tar.gz
gzip -dc bzip2-1.0.8.tar.gz | tar xvf -
cd bzip2-1.0.8
make -f Makefile-libbz2_so
ln -sf libbz2.so.1.0 libbz2.so
mkdir -p $myprefix/include
mkdir -p $myprefix/lib
cp -avf bzlib.h $myprefix/include
cp -avf libbz2.so* $myprefix/lib
make install PREFIX=$myprefix
ml load bzip2/1.0.8

sqlite

myprefix=$HOME/.hpc/sqlite/3.32.3
wget https://www.sqlite.org/2020/sqlite-autoconf-3320300.tar.gz
gzip -dc sqlite-autoconf-3320300.tar.gz | tar xvf -
cd sqlite-autoconf-3320300
./configure --prefix=$myprefix
make -j $(nproc)
make install
ml load sqlite/3.32.3

editline

myprefix=$HOME/.hpc/editline/1.17.1
wget https://github.com/troglobit/editline/releases/download/1.17.1/editline-1.17.1.tar.gz
gzip -dc editline-1.17.1.tar.gz | tar xvf -
cd editline-1.17.1
./configure --prefix=$myprefix
make -j $(nproc)
make install
ml load editline/1.17.1

Miniconda

We don’t need this very much, but it is still useful for some edge cases, mainly revolving around jupyter infrastructure.

cd $HOME
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
chmod +x Miniconda3-latest-Linux-x86_64.sh
./Miniconda3-latest-Linux-x86_64.sh
# Do not allow it to mess up the shell rc files
eval "$($HOME/miniconda3/bin/conda shell.zsh hook)"

Note that we will prefer the manual evaluation since it can be handled in the lmod file.

Applications

Libraries and git aside, there are some tools we might want to have.

ag

The silver searcher, along with rg is very useful to have.

myprefix=$HOME/.hpc/the_silver_searcher/2.2.0
wget https://geoff.greer.fm/ag/releases/the_silver_searcher-2.2.0.tar.gz
gzip -dc the_silver_searcher-2.2.0.tar.gz | tar xvf -
cd the_silver_searcher-2.2.0
./configure --prefix=$myprefix
make -j $(nproc)
make install
ml load the_silver_searcher/2.2.0

Neovim

myprefix=$HOME/.hpc/nvim/0.5.0
wget https://github.com/neovim/neovim/releases/download/nightly/nvim.appimage
chmod +x nvim.appimage
./nvim.appimage --appimage-extract
mkdir -p $myprefix
mv squashfs-root/usr/* $myprefix
ml load nvim/0.5.0

Tmux

myprefix=$HOME/.hpc/tmux/3.1b
wget https://github.com/tmux/tmux/releases/download/3.1b/tmux-3.1b-x86_64.AppImage
chmod +x tmux-3.1b-x86_64.AppImage
rm -rf squashfs-root
./tmux-3.1b-x86_64.AppImage --appimage-extract
mkdir -p $myprefix
mv squashfs-root/usr/bin squashfs-root/usr/lib squashfs-root/usr/share $myprefix
ml load tmux/3.1b

Zsh

More of an update than a requirement.

myprefix=$HOME/.hpc/zsh/5.8
wget https://github.com/zsh-users/zsh/archive/zsh-5.8.tar.gz
gzip -dc zsh-5.8.tar.gz | tar xvf -
cd zsh-zsh-5.8
./configure --prefix=$myprefix
make -j $(nproc)
make install
ml load zsh/5.8

Conclusion

Having composed a bunch of these, I will of course try to somehow get nix up and running so it can bootstrap itself and allow me to work in peace. I might also eventually create shell scripts to automate updating these, but hopefully I can set up nix and not re-create package manager logic in lua.

  1. They were called hpc2013 and hpc2010 respectively ↩︎

  2. I really like Lua, enough to embed it in d-SEAMS ↩︎