In this section will show how to build caffe container using singularity. The goal is to show you how to build images on your machine, and copy them to the Mesocentre for execution.

First, download install Singularity from here: http://singularity.lbl.gov.

Once installed, follow the steps below:

The image can be created with this commande:

$ sudo create -size 2048 caffe-cpu.img

This command will create a file named caffe-cpu.img of size 2Go.

Next, we need install system (ubuntu), all required softwares and finally caffe.

The Bootsrap file is given bellow:

Singularity

BootStrap: docker
From: ubuntu:latest
 
%setup
# commands to be executed on host outside container during bootstrap
 
%post
# commands to be executed inside container during bootstrap
 
# create bind points for MesoFC HPC environment
mkdir -p /Work
mkdir -p /Home
 
### Install depencies applications
apt-get update && apt-get install -y --no-install-recommends \
        build-essential \
        cmake \
        git \
        wget \
        libatlas-base-dev \
        libboost-all-dev \
        libgflags-dev \
        libgoogle-glog-dev \
        libhdf5-serial-dev \
        libleveldb-dev \
        liblmdb-dev \
        libopencv-dev \
        libprotobuf-dev \
        libsnappy-dev \
        protobuf-compiler \
        python-dev \
        python-numpy \
        python-pip \
        python-setuptools \
        python-scipy && \
#rm -rf /var/lib/apt/lists/*
 
### Install caffe in /opt
cd /opt
 
git clone  https://github.com/BVLC/caffe.git  && \
    cd caffe && pip install --upgrade pip && \
    cd python && pip install -r requirements.txt&& \
    cd ..&&mkdir build && cd build && \
    cmake -DCPU_ONLY=1 .. && \
    make -j"$(nproc)"&&make install
 
 
### update environement variable
cd 
echo "PATH=/opt/caffe/build/install/bin:\$PATH" >> /environment
echo "PYTHONPATH=/opt/caffe/build/install/python:\$PYTHONPATH" >> /environment 
echo "LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:/opt/caffe/build/install/lib" >> /environment 
echo "LIBRARY_PATH=\$LIBRARY_PATH:/opt/caffe/build/install/lib" >> /environment 
echo "CPATH=\$CPATH:/opt/caffe/build/install/include" >> /environment  
echo "C_INCLUDE_PATH=\$C_INCLUDE_PATH:/opt/caffe/build/install/include" >>  /environment 
echo "CPLUS_INCLUDE_PATH=\$CPLUS_INCLUDE_PATH:/opt/caffe/build/install/include" >> /environment 
 
echo "export PATH PYTHONPATH LD_LIBRARY_PATH LIBRARY_PATH CPATH C_INCLUDE_PATH CPLUS_INCLUDE_PATH" >> /environment 
ln -s /environment /.singularity.d/env/90-env.sh
 
%runscript
# commands to be executed when the container runs
 
%test
# commands to be executed within container at close of bootstrap process

The command to use is:

$ sudo singularity bootstrap caffe-cpu.img Singularity 

Where

• caffe-cpu.img: the image created before
• Singularity: the bootstrap file

This will take some minutes (take a coffee).

If all goes well, caffe is installed by success inside the container caffe-cpu.img in /opt/caffee folder.

You can run caffe, in your host, inside the container:

$ singularity exec caffe-cpu.img caffe

Just copy this file to your cluster account (WORK folder) and run singularity as usual.

For example for interactive usage:

$ qlogin -lh_vmem=6G

$ cd WORK

$ module load tools/singularity

$ singularity exec caffe-cpu.img caffe 

• Caffe Optimized for GPU