Singularity enables users to have full control of their environment. This means that a non-privileged user can “swap out” the operating system on the host for one they control. So if the host system is running RHEL6 but your application runs in Ubuntu, you can create an Ubuntu image, install your applications into that image, copy the image to another host, and run your application on that host in it’s native Ubuntu environment!

Singularity containers let users run applications in a Linux environment of their choosing.

Possible uses for Singularity on Lumiere cluster:

• Run an application that was built for a different distribution of Linux (Ubuntu 16, CentoOS 7, …) than the host OS(CentOS 6).
• Reproduce an environment to run a workflow created by someone else. For example, a workflow developed at a different site on RedHat 7.0
• Run a series of applications (a 'pipeline') that includes applications built on different platforms
• Run GPU applications like tensorFlow
• Run an application compiled with newer version of GLIBC
• Some difficult Applications to install from source can be embedded inside a container

Singularity workflow consists of:

1. Creating containers images : this operation needs root access on the host.
2. Using these images to run applications.

First we need to load singularity:

$ module load tools/singularity

We can show available containers installed on lumiere cluster:

$ ls $SING_SPOOL

For the moment, we have installed:

• Singularity sets up the container environment and creates the necessary files directories (HOME and WORK)
• When the application(s) finish their foreground execution process, the container and namespaces collapse and vanish cleanly

The shell Singularity sub-command will automatically spawn a shell within a container. The default and the only requirement of any Singularity container is that /bin/sh must be present, and thus /bin/sh is also used as the default shell.

The usage is as follows:

$ singularity shell
USAGE: singularity (options) shell [container image] (options)

Example, Printing the OS release inside the container:

$ singularity shell $SING_SPOOL/ubuntu-16.img 
Singularity: Invoking an interactive shell within container...
 
Singularity.ubuntu-16.img> cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"

The exec Singularity sub-command allows to spawn an arbitrary command within container image as if it were running directly on the host system. All standard IO, pipes, and file systems are accessible via the command being exec'ed within the container.

The usage is as follows:

$ singularity exec
USAGE: singularity (options) exec [container image] [command] (options)
The command that you want to exec will follow the container image along with any additional arguments will all be passed directly to the program being executed within the container.

Example, printing the OS release inside the container:

$ singularity exec $SING_SPOOL/ubuntu-16.img cat /etc/lsb-release
 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"

Executing Singularity inside SGE is transparent, it's like executing native application.

Let's start with a simple python example hello.py located in the current directory with the contents of:

#!/usr/bin/python
 
import sys
print("Hello World: The Python version is %s.%s.%s" % sys.version_info[:3])

Because WORK directory is automatically bound into the container, and we are running this from WORK directory, we can easily execute that script using the Python within the container:

$ module load tools/singularity 
 
$ singularity exec $SING_SPOOL/ubuntu-61 /usr/bin/python hello.py 
 
Hello World: The Python version is 2.7.5

SGE script looks like:

#!/bin/bash
 
#$ -q all.q
 
module load tools/singularity
 
singularity exec $SING_SPOOL/ubuntu-61 /usr/bin/python hello.py

And what about multi-threaded applications like OpenMP ?

Simply request openmp SGE environment by adding this option #$ -pe openmp N where N the number of cores.

and run applications like:

$ singularity exec $SING_SPOOL/ubuntu-61 OMP_NUM_THREADS=$NSLOTS /path/to/myappli

…and MPI applications ?

Singularity only support OpenMPI 2.x series. We only tested MPI Java based applications.

See this example on how to use OpenMPI with Singularity.

• MBT
• TensorFlow
• Hex

• Singularity home
• Singularity on GitHub
• https://hpc.nih.gov/apps/singularity.html