HPC:Environment Modules
Environment Modules
The PMACS cluster utilizes modules to enable users to alter their environment. These modules alter your execution path ($PATH) and add or modify the user's environment variables. User loadable modules are available if the system default packages don't meet your requirements. Environment modules are used in place of hard-coding paths in your $HOME/.bashrc file.
To know what modules are available, you'll need to run the "module avail" command from an interactive session:
[asrini@consign ~]$ bsub -Is bash
Job <9990024> is submitted to default queue <interactive>.
<<Waiting for dispatch ...>>
<<Starting on node063.hpc.local>>
[asrini@node063 ~]$ module avail
------------------------------------------------------------------- /usr/share/Modules/modulefiles -------------------------------------------------------------------
NAMD-2.9-Linux-x86_64-multicore dot module-info picard-1.96 rum-2.0.5_05
STAR-2.3.0e java-sdk-1.6.0 modules pkg-config-path samtools-0.1.19
STAR-hg19 java-sdk-1.7.0 mpich2-x86_64 python-2.7.5 use.own
STAR-mm9 ld-library-path null r-libs-user
bowtie2-2.1.0 manpath openmpi-1.5.4-x86_64 ruby-1.8.7-p374
devtoolset-2 module-cvs perl5lib ruby-1.9.3-p448
The module names should be pretty self-explainatory, but some are not. To see information about a module
you can issue a module show [module name]:
[asrini@node063 ~]$ module show null ------------------------------------------------------------------- /usr/share/Modules/modulefiles/null: module-whatis does absolutely nothing ------------------------------------------------------------------- [asrini@node063 ~]$ module show r-libs-user ------------------------------------------------------------------- /usr/share/Modules/modulefiles/r-libs-user: module-whatis Sets R_LIBS_USER=$HOME/R/library setenv R_LIBS_USER ~/R/library ------------------------------------------------------------------- [asrini@node063 ~]$ module show devtoolset-2 ------------------------------------------------------------------- /usr/share/Modules/modulefiles/devtoolset-2: module-whatis Devtoolset-2 packages include the newer versions of gcc prepend-path PATH /opt/rh/devtoolset-2/root/usr/bin prepend-path MANPATH /opt/rh/devtoolset-2/root/usr/share/man prepend-path INFOPATH /opt/rh/devtoolset-2/root/usr/share/info -------------------------------------------------------------------
Example use of modules:
[asrini@node063 ~]$ python -V Python 2.6.6 [asrini@node063 ~]$ which python /usr/bin/python [asrini@node063 ~]$ module load python-2.7.5 [asrini@node063 ~]$ python -V Python 2.7.5 [asrini@node063 ~]$ which python /opt/software/python/python-2.7.5/bin/python
After running the above commands, you will be able to use python v2.7.5 till you exit out of the interactive session or till you unload the module:
[asrini@node063 ~]$ module unload python-2.7.5 [asrini@node063 ~]$ which python /usr/bin/python
Modules may also be included in your job scripts and submitted as a batch job.
Using Modules at Login
In order to have modules automatically load into your environment, you would add the module commands to your $HOME/.bashrc file. Note that modules are not available on the PMACS head node, hence, you'll need to ensure that your login script attempts to load a module only if you are on a compute node:
[asrini@consign ~]$ more .bashrc # .bashrc # Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc fi # # # Modules to load if [ $HOSTNAME != "consign.hpc.local" ] && [ $HOSTNAME != "mercury.pmacs.upenn.edu" ]; then module load python-2.7.5 fi # more stuff below ..... [asrini@consign ~]$ which python /usr/bin/python [asrini@consign ~]$ bsub -Is bash Job <172129> is submitted to default queue <interactive>. <<Waiting for dispatch ...>> <<Starting on node063.hpc.local>> [asrini@node063 ~]$ which python /opt/software/python/python-2.7.5/bin/python
