HPC:Environment Modules

The PMACS cluster utilizes modules to enable users to alter their environment. These modules alter your execution path ($PATH) and add or modify the user's environment variables. User loadable modules are available if the system default packages don't meet your requirements. Environment modules are used in place of hard-coding paths in your $HOME/.bashrc file.

To know what modules are available, you'll need to run the "module avail" command from an interactive session:

[asrini@consign ~]$ bsub -Is bash
Job <9990024> is submitted to default queue <interactive>.
<<Waiting for dispatch ...>>
<<Starting on node063.hpc.local>>
    
[asrini@node063 ~]$ module avail

------------------------------------------------------------------- /usr/share/Modules/modulefiles -------------------------------------------------------------------
NAMD-2.9-Linux-x86_64-multicore dot                             module-info                     picard-1.96                     rum-2.0.5_05
STAR-2.3.0e                     java-sdk-1.6.0                  modules                         pkg-config-path                 samtools-0.1.19
STAR-hg19                       java-sdk-1.7.0                  mpich2-x86_64                   python-2.7.5                    use.own
STAR-mm9                        ld-library-path                 null                            r-libs-user
bowtie2-2.1.0                   manpath                         openmpi-1.5.4-x86_64            ruby-1.8.7-p374
devtoolset-2                    module-cvs                      perl5lib                        ruby-1.9.3-p448 

The module names should be pretty self-explainatory, but some are not. To see information about a module you can issue a module show [module name]:

[asrini@node063 ~]$ module show null
-------------------------------------------------------------------
/usr/share/Modules/modulefiles/null:

module-whatis	 does absolutely nothing
-------------------------------------------------------------------

[asrini@node063 ~]$ module show r-libs-user
-------------------------------------------------------------------
/usr/share/Modules/modulefiles/r-libs-user:

module-whatis	 Sets R_LIBS_USER=$HOME/R/library
setenv		 R_LIBS_USER ~/R/library
-------------------------------------------------------------------

[asrini@node063 ~]$ module show devtoolset-2
-------------------------------------------------------------------
/usr/share/Modules/modulefiles/devtoolset-2:

module-whatis	 Devtoolset-2 packages include the newer versions of gcc
prepend-path	 PATH /opt/rh/devtoolset-2/root/usr/bin
prepend-path	 MANPATH /opt/rh/devtoolset-2/root/usr/share/man
prepend-path	 INFOPATH /opt/rh/devtoolset-2/root/usr/share/info
-------------------------------------------------------------------

Example use of modules:

[asrini@node063 ~]$ python -V
Python 2.6.6

[asrini@node063 ~]$ which python
/usr/bin/python

[asrini@node063 ~]$ module load python-2.7.5

[asrini@node063 ~]$ python -V
Python 2.7.5

[asrini@node063 ~]$ which python
/opt/software/python/python-2.7.5/bin/python

After running the above commands, you will be able to use python v2.7.5 till you exit out of the interactive session or till you unload the module:

[asrini@node063 ~]$ module unload python-2.7.5

[asrini@node063 ~]$ which python
/usr/bin/python

Modules may also be included in your job scripts and submitted as a batch job.

Using Modules at Login

In order to have modules automatically load into your environment, you would add the module commands to your $HOME/.bashrc file. Note that modules are not available on the PMACS head node, hence, you'll need to ensure that your login script attempts to load a module only if you are on a compute node:

[asrini@consign ~]$ more .bashrc
# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi
#
#
# Modules to load
if [ $HOSTNAME != "consign.hpc.local" ] && [ $HOSTNAME != "mercury.pmacs.upenn.edu" ]; then
	module load python-2.7.5
fi

# more stuff below .....

[asrini@consign ~]$ which python
/usr/bin/python
[asrini@consign ~]$ bsub -Is bash
Job <172129> is submitted to default queue <interactive>.
<<Waiting for dispatch ...>>
<<Starting on node063.hpc.local>>
[asrini@node063 ~]$ which python
/opt/software/python/python-2.7.5/bin/python

Using Modules in BSUB

In order to use modules in your BSUB processes, you will need to source the module initiation profile script.

Here is an example bash script that loads samtools version 0.1.19 into your PATH and then issues a command to output the help information from samtools to STDOUT:

$ cat mod.sh
#!/bin/bash
if [ -f /etc/profile.d/modules.sh ]; then
   source /etc/profile.d/modules.sh
fi
module load samtools-0.1.19
samtools

When we run the above script:

$ bsub -e mod.e -o mod.o sh mod.sh

The following message is captured in the .e file:

$ cat mod.e

Program: samtools (Tools for alignments in the SAM format)
Version: 0.1.19-44428cd

Usage:   samtools <command> [options]

Command: view        SAM<->BAM conversion
         sort        sort alignment file
         mpileup     multi-way pileup
         depth       compute the depth
         faidx       index/extract FASTA
         tview       text alignment viewer
         index       index alignment
         idxstats    BAM index stats (r595 or later)
         fixmate     fix mate information
         flagstat    simple stats
         calmd       recalculate MD/NM tags and '=' bases
         merge       merge sorted alignments
         rmdup       remove PCR duplicates
         reheader    replace BAM header
         cat         concatenate BAMs
         bedcov      read depth per BED region
         targetcut   cut fosmid regions (for fosmid pool only)
         phase       phase heterozygotes
         bamshuf     shuffle and group alignments by name