Installing NSuite

The first stage of the NSuite workflow is to install the simulation engine(s) to benchmark or validate. This page describes how to obtain NSuite and then perform this step so that benchmarks and validation tests can be run.

Obtaining NSuite

Before installing, first get a copy of NSuite. The simplest way to do this is to clone the repository using git:

git clone
cd nsuite
git checkout v1.0

In the example above, git checkout v1.0 is used to pick a tagged version of NSuite. If omitted, the latest development version in the master branch will be used.

Installing Simulation Engines

NSuite provides a script that performs the following operations:

  • Obtain the source code for simulation engines.
  • Compile and install the simulation engines.
  • Compile and install benchmark and validation test drivers.

Basic usage of is best illustrated with some examples:

# download and install Arbor
./ arbor

# download and install NEURON and CoreNEURON
./ neuron coreneuron

# download install all three of Arbor, NEURON and CoreNEURON
./ all

# download install NEURON in relative path install
./ neuron --prefix=install

# download install NEURON in relative path that includes time stamp
# e.g. install-2019-03-22
./ neuron --prefix=install-$(date +%F)

# download install NEURON in absolute path
./ neuron --prefix=/home/uname/install

The simulation engines to install are provided as arguments. Further options for installing the simulation engines in a user-specified path and customising the build environment can be provided:

Flag Default value Explanation
simulator none Which simulation engines to download and install. Any number of the following: {arbor, neuron, coreneuron, all}.
--prefix current path Path for downloading, compiling, installing simulation engines. Also used to store inputs and outputs from benchmarks and validation tests. Can be either a relative or absolute path.
--env none Optional script for configuring the environment and build steps. See Directory Structure.

Directory Structure

The following directory structure will be generated when is run:

├── build
│   ├── arbor
│   ├── coreneuron
│   ├── neuron
│   └── ...
├── install
│   ├── bin
│   ├── include
│   ├── lib
│   └── share
├── config
├── input
│   └── benchmarks
├── output
│   ├── benchmarks
│   └── validation
└── cache

If no prefix is provided, the directory structure is created in the nsuite path. The contents of each sub-directory are summarised:

build Source code for simulation engines is checked out, and compiled here.
install Installation target for the simulation engine libraries, executables, headers, etc.
config The environment used to build each simulation engine is stored here, to load per-simulator when running benchmarks and validation tests.
cache Validation data sets are stored here when generated during the installation phase.
input generated by running benchmarks Input files for benchmark runs in sub-directories for each benchmark configuration.
output generated by running benchmarks/validation Benchmark and validation outputs in sub-directories for each benchmark/validation configuration.

Customizing the environment

NSuite attempts to detect features of the environment that will influence how simulation engines are compiled and run, including compilers, MPI support and CPU core counts. HPC systems have multiple compilers, MPI implementations and hardware resources available, which are typically configured using modules. It isn’t possible for NSuite to detect which options to choose on such systems, so user can customise the compilation and execution of simulation engines. To do this, a user provides an environment configuration script that will sourced after NSuite has performed automatic environment detection and configuration.

The script is specified with the --env flag:

./install-local arbor
./install-local neuron

In the example above, different configurations are used for Arbor and NEURON. This can be used, for example, to choose compilers that produce optimal results on each respective simulator, or when different simulators require different versions of a library.

Examples of scripts for two HPC systems, Piz Daint and JUWELS, can be found in the scripts sub-directory in NSuite.

General Variables

The following variables are universal to all of the simulation engines.

Variable Default value Explanation
ns_cc mpicc if available, else gcc/clang on Linux/OS X The C compiler for compiling simulation engines.
ns_cxx mpicxx if available, else g++/clang++ on Linux/OS X The C++ compiler for compiling simulation engines.
ns_with_mpi ON iff MPI is detected ON/OFF to compile simulation engines with MPI enabled. Also controls whether mpirun is used to launch benchmarks.
ns_makej 4 Number of parallel jobs to use when compiling.
ns_python which python3 The Python interpreter to use. Must be Python 3.
ns_threads_per_core automatic The number of threads per core for parallel benchmarks.
ns_cores_per_socket automatic The number of cores per socket for parallel benchmarks.
ns_sockets 1 The number of sockets for parallel benchmarks. One MPI rank is used per socket if MPI support is enabled.
run_with_mpi Bash function for OpenMPI A bash function for launching an executable and flags with multithreading and optionally MPI, based on the ns_threads_per_core, ns_cores_per_socket, ns_sockets variables.

Simulator-Specific Variables

There are Arbor-specific options for checking out Arbor from a Git repository, and for configuring target-specific optimizations.

Variable Default value Explanation
ns_arb_git_repo URL or directory for the Git repository to check out Arbor source from.
ns_arb_branch v0.5 The branch/tag/SHA to check out. Master will be used if empty.
ns_arb_arch native The CPU architecture target for Arbor. Must be set when cross compiling. Default native targets the architecture used to configure NSuite.
ns_arb_gpu none Build Arbor with/without GPU support. Available options are none, cuda, clang, hip-clang
ns_arb_vectorize ON Whether to use explicit vectorization for Arbor.

The NEURON-specific options are for configuring where to get NEURON’s source from. NEURON can be downloaded from a tar ball for a specific version, or cloned from a Git repository.

The official versions of NEURON’s source code available to download are inconsistently packaged, so it is not possible to automatically determine how to download and install from a version string alone, e.g. “7.6.2”. This is why three variables must be set if downloading a NEURON tarball.

Variable Default value Explanation
ns_nrn_tarball nrn-7.6.5.tar.gz The name of the tar ball file (caution: not named consistently between versions).
ns_nrn_url versions/v7.6/7.6.5/${ns_nrn_tarball} The URL of the tar ball (caution: not name consistently between versions).
ns_nrn_path nrn-7.6 The name of the path after expanding the tar ball (caution: not name consistently between versions).
ns_nrn_git_repo empty URL or path of Git repository. If set it will be used instead of downloading a tarball.
ns_nrn_branch master Branch or commit SHA to use if sourcing from Git.

CoreNEURON has more support than NEURON for targeting different hardware, either via automatic vectorization, or using OpenACC for GPUs. However, it is quite difficult to build, particularly as part of an automated pipeline: users have to directly provide architecture- and compiler-specific flags to CMake. As soon as we are able to build CoreNEURON this way ourselves, we will add more flags for targeting different architectures.

Variable Default value Explanation
ns_cnrn_git_repo URL or path of Git repository.
ns_cnrn_sha 0.14 Branch, tag or commit SHA of Git repository.

Example custom environment

Below is a custom configuration script for a Cray cluster with Intel KNL processors. It configures all platform-specific details that can’t be automatically detected by

  • loading and swapping required modules;
  • setting a platform-specific magic variable CRAYPE_LINK_TYPE required to make CMake play nice;
  • configuring MPI with the Cray MPI wrapper;
  • configuring Arbor to compile with KNL support;
  • configuring the number of threads and MPI ranks with which to run benchmarks.
# set up Cray Programming environment to use GNU toolchain
[ "$PE_ENV" = "CRAY" ] && module swap PrgEnv-cray PrgEnv-gnu

# load python, gcc version and CMake
module load cray-python/
module swap gcc/7.3.0   # load after cray-python
module load CMake

# set for CMake to correctly configure Arbor and CoreNEURON
export CRAYPE_LINK_TYPE=dynamic

# Python, MPI and build options for this system
ns_python=$(which python3)
ns_cc=$(which cc)
ns_cxx=$(which CC)

# simulator-specific options

# cluster-specific options

run_with_mpi() {
    # this system uses Slurm's srun to launch MPI jobs on compute nodes
    srun -n $ns_sockets -c $ns_threads_per_socket $*