Difference between revisions of "HPC:Singularity"

Revision as of 19:40, 22 March 2021

Singularity is an open-source application for creating and running software containers, designed primarily for scientific computing on Linux-based computing clusters like the Penn Medicine HPC system. Singularity containers enable self-contained, stable, portable, and reproducible computing environments and software stacks that can be shared and used across different machines and computing clusters, such as for research collaborations spanning multiple institutions.

Usage

Begin by launching an interactive/non-interactive session (bsub -Is bash). Singularity is installed on all HPC nodes outside of the environment module system, so there is no need to load a module in order to use it. Instead, you can directly use the singularity commands.

Note: commands that require sudo will not be available to you on HPC systems. The current version of Singularity installed is 3.7.1.

For detailed help information run 'singularity help' within an interactive session.

Obtaining Singularity Images

Container images are executable files that define the software environment for the container and run the container. A single container image can be used to run multiple instances of the same container concurrently for different jobs.

To use container images on the HPC, images need to be pulled (i.e downloaded) from an external resource like the Container Library or Docker Hub or Singularity Hub. Container images may also be built externally from a definition file and then transferred to HPC system where it can be run.

Pulling pre-built images

If a pre-built container image containing all the necessary software packages has been identified in Container Library or Docker Hub or Singularity Hub, it can easily be pulled down using the 'singularity pull' command.

Example below shows how to pull down the "ubuntu" image from the Singularity Container Library:

singularity pull library://ubuntu

The above command will create an image file called "ubuntu_latest.sif". To use an alternate naming convention for image files:

singularity pull ubuntu.sif library://ubuntu

Example below shows how to pull down the "ubuntu" image from Docker Hub:

singularity pull docker://ubuntu

Building images from a definition file

You can also build a custom container image by using a definition file that instructs the build process. For security reasons we do not allow users to build directly on our systems; you must build externally and then transfer the image to the cluster.

Running Singularity container images

Use one of the following commands to interact with Singularity containers:

* singularity shell — for an interactive shell within the container
* singularity exec  — for executing commands within the container
* singularity run   — for executing a pre-defined runscript within the container

@@ Line 7: / Line 7: @@
 Begin by launching an interactive/non-interactive session (bsub -Is bash). Singularity is installed on all HPC nodes outside of the environment module system, so there is no need to load a module in order to use it. Instead, you can directly use the singularity commands.
-'''Note:''' commands that require sudo will not be available to you on HPC systems. The current version of Singularity installed is 3.7.1.
+'''Note: commands that require sudo will not be available to you on HPC systems. The current version of Singularity installed is 3.7.1.'''
 For detailed help information run 'singularity help' within an interactive session.