Difference between revisions of "GCB 537:Main Page"

From HPC wiki
 
(21 intermediate revisions by the same user not shown)
Line 1: Line 1:
This page is intended to provide students enrolled in GCB537 with information about how to access the dedicated compute cluster for this course. For detailed information about the PMACS HPC go to [[HPC:Main_Page|our main page]]
+
This page is intended to provide students enrolled in GCB537 with information about how to access the dedicated compute cluster for this course. The course website is located [https://canvas.upenn.edu/courses/1266569 here].  For detailed information about the PMACS HPC go to [[HPC:Main_Page|our main page]]
  
 +
 +
=== Other Pages ===
 +
*[[HPC:FAQ|HPC FAQ ]]
 +
*[[HPC:User_Guide|User Guide]]
 +
*[[HPC:Software|Available Software]]
  
 
== '''Important information''' ==
 
== '''Important information''' ==
Line 6: Line 11:
 
=== Setup ===
 
=== Setup ===
 
In the sections that follow, information is provided on how student laptops/desktops can be configured to access the compute environment dedicated for the course: GCB537. '''You may find references to a server called demohpc.pmacs.upenn.edu, please remember to replace this with gcb537.pmacs.upenn.edu'''
 
In the sections that follow, information is provided on how student laptops/desktops can be configured to access the compute environment dedicated for the course: GCB537. '''You may find references to a server called demohpc.pmacs.upenn.edu, please remember to replace this with gcb537.pmacs.upenn.edu'''
 +
 +
If you are looking for the course website, it is located [https://canvas.upenn.edu/courses/1266569 here].
  
 
=== The Environment ===  
 
=== The Environment ===  
Line 15: Line 22:
 
* SSH is the only protocol allowed through the firewall to the course's compute environment. File transfer to/from this environment is allowed only through SCP/SFTP. Students are expected to use client programs that support SSH (for terminal/command line access) and SCP/SFTP (for file transfer operations). SSH/SCP/SFTP connections to this environment is allowed only by means of PublicKey authentication and not password based authentication. Instructions to setup Public-Private keypairs is provided in the sections below.  
 
* SSH is the only protocol allowed through the firewall to the course's compute environment. File transfer to/from this environment is allowed only through SCP/SFTP. Students are expected to use client programs that support SSH (for terminal/command line access) and SCP/SFTP (for file transfer operations). SSH/SCP/SFTP connections to this environment is allowed only by means of PublicKey authentication and not password based authentication. Instructions to setup Public-Private keypairs is provided in the sections below.  
  
==== List of Important Servers ====
+
=== Backup ===
 +
 
 +
Data/code stored in any of the accounts on the class cluster is <strong>NOT BACKED UP</strong>. It is the <strong>student's responsibility</strong> to make sure data/code for this course is either version controlled (using Git, for example) or stored elsewhere.
 +
 
 +
=== Temporary Password ===
 +
 
 +
While the class cluster environment itself is configured for Public Key based authentication only, passwords are necessary to login into the PMACS VPN, should students wish to access this environment from off-campus/non-PennNet networks. A temporary password will be provided to all students who don't already have PMACS accounts. Students must change this temporary password and enroll into the PMACS password reset system prior to the start of the class. All PMACS passwords expire every 180 days, therefore this password change should be necessary only once this semester (provided you don't forget your password!).
 +
 
 +
Students who already have valid PMACS accounts will not be provided a temporary password and can continue to use their existing PMACS account passwords to login into the VPN.
 +
 
 +
=== List of Important Servers ===
 
* <strong>gcb537.pmacs.upenn.edu</strong> : login/job submission server ; Do '''NOT''' run jobs on this server; Always use one of the compute nodes
 
* <strong>gcb537.pmacs.upenn.edu</strong> : login/job submission server ; Do '''NOT''' run jobs on this server; Always use one of the compute nodes
 
* [https://juneau.med.upenn.edu/ <strong>VPN For Off-Campus Access</strong>]
 
  
 
* [https://reset.pmacs.upenn.edu/ <strong>Password reset application</strong>]
 
* [https://reset.pmacs.upenn.edu/ <strong>Password reset application</strong>]
  
 
==== VPN (for off-campus access) ====
 
==== VPN (for off-campus access) ====
All students enrolled in the GCB537 course can use this VPN to establish secure connections to the class cluster when trying to SSH from off-campus. This is a web based VPN that can be accessed [https://juneau.med.upenn.edu/ <strong>here</strong>]. Once a VPN tunnel has been setup, normal SSH connections to the cluster head node: consign.pmacs.upenn.edu can be established.  
+
Students enrolled in the GCB537 course will need to use our VPN to establish secure connections to the class cluster when trying to SSH from off-campus. Once the VPN tunnel has been setup, normal SSH connections to the GCB537 class cluster head node: gcb537.pmacs.upenn.edu can be established.
 +
 
 +
There are separate stand-alone VPN applications for different Operating systems.
 +
 
 +
Download the appropriate installer for
 +
 +
[https://upenn.box.com/s/o7yl6jh35o6j3xuu8q6bj6mixt4x4xwh Mac OSX]
 +
 
 +
OR
  
'''Note 1''' There may be other VPNs managed by PMACS/UPHS. Please ensure that while connecting to the class cluster, you are using this VPN only.
+
[https://upenn.box.com/s/zknl5nlneve6upn5nyyngkpe7w20n0rv Windows]
  
'''Note 2''' The VPN link above may present you with a self-signed certificate ("Untrusted connection") message. It is good practice to always verify that you are connecting to the right server. The SHA1 fingerprint for the certificate is:
+
Once the appropriate installer has been downloaded and installed, a reboot may be required to reboot. Subsequently, the stand-alone client console
<pre> F9:60:55:46:7E:B4:35:55:A7:2D:06:F7:D8:0D:94:E4:3F:7D:D2:A1 </pre>
+
must be used, without needing to use a web browser (as described on our wiki).
  
'''Note 3''' The web-based VPN is known to work with browsers like Firefox, Internet Explorer (IE) and Safari. There are known issues with using Chrome.
 
  
=== Backup ===
+
See pictures [[HPC:Login#More_VPN_INFO| below]] for details on how to configure the Standalone VPN client (works for Windows and MacOS). Separate information for GNU/Linux VPN clients is also [[HPC:Login#More_VPN_INFO| below]].
  
Data/code stored in any of the accounts on the class cluster is <strong>NOT BACKED UP</strong>. Always make sure your data/code is either version controlled (using Git) or stored elsewhere.
+
==== VPN Client for GNU/Linux ====
 +
The FortiClient application for GNU/Linux systems needs to downloaded directly from the FortiNet website.
  
 
== Connecting to the class cluster environment ==
 
== Connecting to the class cluster environment ==
Secure shell (ssh) is the only supported method of connecting to the class cluster environment. The login machine name is gcb537.pmacs.upenn.edu.  Below are instructions on how to create Public-Private keys on GNU/Linux, Mac OSX and Windows systems.  
+
Secure shell (ssh) is the only supported method of connecting to the class cluster environment. '''The login machine name is gcb537.pmacs.upenn.edu.''' Below are instructions on how to create Public-Private keys on GNU/Linux, Mac OSX and Windows systems.  
  
 
'''Note:''' While there are several SSH clients available for all platforms. We only show instructions for and are able to support the usage of the following tools.  
 
'''Note:''' While there are several SSH clients available for all platforms. We only show instructions for and are able to support the usage of the following tools.  
Line 63: Line 86:
  
 
''' Note 1: While following the instructions below, replace the server name demohpc.pmacs.upenn.edu with gcb537.pmacs.upenn.edu '''  
 
''' Note 1: While following the instructions below, replace the server name demohpc.pmacs.upenn.edu with gcb537.pmacs.upenn.edu '''  
 +
 
''' Note 2: Your username is not "bob"! ''''
 
''' Note 2: Your username is not "bob"! ''''
  
  
 
[[image:Ssh-gnu linux-mac.png|center|1000px]]
 
[[image:Ssh-gnu linux-mac.png|center|1000px]]
 
  
 
==== FileZilla setup for copying files to/from class cluster environment ====
 
==== FileZilla setup for copying files to/from class cluster environment ====
Line 73: Line 96:
  
 
''' Note 1: While following the instructions below, replace the server name demohpc.pmacs.upenn.edu with gcb537.pmacs.upenn.edu '''  
 
''' Note 1: While following the instructions below, replace the server name demohpc.pmacs.upenn.edu with gcb537.pmacs.upenn.edu '''  
 +
 
''' Note 2: Your username is not "bob"! ''''
 
''' Note 2: Your username is not "bob"! ''''
  
Line 120: Line 144:
  
 
''' Note 1: While following the instructions below, replace the server name demohpc.pmacs.upenn.edu with gcb537.pmacs.upenn.edu '''  
 
''' Note 1: While following the instructions below, replace the server name demohpc.pmacs.upenn.edu with gcb537.pmacs.upenn.edu '''  
 +
 
''' Note 2: Your username is not "bob"! ''''
 
''' Note 2: Your username is not "bob"! ''''
  
Line 135: Line 160:
  
 
''' Note 1: While following the instructions below, replace the server name demohpc.pmacs.upenn.edu with gcb537.pmacs.upenn.edu '''  
 
''' Note 1: While following the instructions below, replace the server name demohpc.pmacs.upenn.edu with gcb537.pmacs.upenn.edu '''  
 +
 
''' Note 2: Your username is not "bob"! ''''
 
''' Note 2: Your username is not "bob"! ''''
  
Line 145: Line 171:
 
[[image:WinSCP6.png|center]] <br>
 
[[image:WinSCP6.png|center]] <br>
 
[[image:WinSCP7.png|center]] <br>
 
[[image:WinSCP7.png|center]] <br>
 +
 +
== '''More VPN INFO''' ==
 +
Below are screenshots that describe how to use the PMACS VPN on Windows or MacOS systems:
 +
 +
[[image:PMACS_VPN_OSX_setup.png|center|500px]]
 +
 +
Below are screenshots that describe how to use the PMACS VPN on GNU/Linux systems:
 +
 +
 +
[[image:Forticlient1-1.png|center|500px]]
 +
 +
 +
 +
[[image:Forticlient2.png|center|500px]]
 +
 +
 +
 +
[[image:Forticlient3.png|center|500px]]
 +
 +
 +
 +
[[image:Forticlient4-1.png|center|500px]]

Latest revision as of 15:58, 31 January 2019

This page is intended to provide students enrolled in GCB537 with information about how to access the dedicated compute cluster for this course. The course website is located here. For detailed information about the PMACS HPC go to our main page


Other Pages

Important information

Setup

In the sections that follow, information is provided on how student laptops/desktops can be configured to access the compute environment dedicated for the course: GCB537. You may find references to a server called demohpc.pmacs.upenn.edu, please remember to replace this with gcb537.pmacs.upenn.edu

If you are looking for the course website, it is located here.

The Environment

  • The compute cluster environment for GCB537 course closely resembles the PMACS HPC environment. Some of the basic job submission commands taught in this course may be used on the PMACS cluster.

However, some changes to the commands/scripts may be necessary before attempting to run these on the PMACS cluster.

  • The course environment is behind a firewall and students who wish to access this environment from off-campus/non-Penn networks must use the VPN. See below for VPN instructions.
  • SSH is the only protocol allowed through the firewall to the course's compute environment. File transfer to/from this environment is allowed only through SCP/SFTP. Students are expected to use client programs that support SSH (for terminal/command line access) and SCP/SFTP (for file transfer operations). SSH/SCP/SFTP connections to this environment is allowed only by means of PublicKey authentication and not password based authentication. Instructions to setup Public-Private keypairs is provided in the sections below.

Backup

Data/code stored in any of the accounts on the class cluster is NOT BACKED UP. It is the student's responsibility to make sure data/code for this course is either version controlled (using Git, for example) or stored elsewhere.

Temporary Password

While the class cluster environment itself is configured for Public Key based authentication only, passwords are necessary to login into the PMACS VPN, should students wish to access this environment from off-campus/non-PennNet networks. A temporary password will be provided to all students who don't already have PMACS accounts. Students must change this temporary password and enroll into the PMACS password reset system prior to the start of the class. All PMACS passwords expire every 180 days, therefore this password change should be necessary only once this semester (provided you don't forget your password!).

Students who already have valid PMACS accounts will not be provided a temporary password and can continue to use their existing PMACS account passwords to login into the VPN.

List of Important Servers

  • gcb537.pmacs.upenn.edu : login/job submission server ; Do NOT run jobs on this server; Always use one of the compute nodes

VPN (for off-campus access)

Students enrolled in the GCB537 course will need to use our VPN to establish secure connections to the class cluster when trying to SSH from off-campus. Once the VPN tunnel has been setup, normal SSH connections to the GCB537 class cluster head node: gcb537.pmacs.upenn.edu can be established.

There are separate stand-alone VPN applications for different Operating systems.

Download the appropriate installer for

Mac OSX

OR

Windows

Once the appropriate installer has been downloaded and installed, a reboot may be required to reboot. Subsequently, the stand-alone client console must be used, without needing to use a web browser (as described on our wiki).


See pictures below for details on how to configure the Standalone VPN client (works for Windows and MacOS). Separate information for GNU/Linux VPN clients is also below.

VPN Client for GNU/Linux

The FortiClient application for GNU/Linux systems needs to downloaded directly from the FortiNet website.

Connecting to the class cluster environment

Secure shell (ssh) is the only supported method of connecting to the class cluster environment. The login machine name is gcb537.pmacs.upenn.edu. Below are instructions on how to create Public-Private keys on GNU/Linux, Mac OSX and Windows systems.

Note: While there are several SSH clients available for all platforms. We only show instructions for and are able to support the usage of the following tools.

GNU/Linux or Mac OSX:

  • ssh via built-in terminal emulator program
  • File transfer via either command line (scp) or FileZilla

Windows:

  • ssh via PuTTY
  • File transfer (SCP/SFTP) via WinSCP

If you have a preference for a different ssh client or file transfer program, please ensure that you know how to configure it to use Public Key Authenticaion before the start of the semester.

Setup information for GNU/Linux and Mac OSX Users

Summary of tasks:

  • Verify that you haven't already generated a Public-Priviate keypair (look inside the .ssh directory in your home area)
  • If don't have keys, open a terminal and run the ssh-keygen command and follow the on-screen instructions refer picture below.
  • If you do have keys, copy the id_rsa.pub key to your desktop as username-id_rsa.pub where username is your PennKey username
  • Email this file as an attachment to pmacshpc@med.upenn.edu (please don't paste the contents of the file in the email). Ensure that emails have your full name, PennKey and the subject line "Public keys for GCB537"
  • Configure FileZilla/Cyberduck to use the newly generated keys
  • Do not delete either the Public or Private key till after the semester

SSH key generation and testing connection to the class compute environment

Note 1: While following the instructions below, replace the server name demohpc.pmacs.upenn.edu with gcb537.pmacs.upenn.edu

Note 2: Your username is not "bob"! '


Ssh-gnu linux-mac.png

FileZilla setup for copying files to/from class cluster environment

Note 1: While following the instructions below, replace the server name demohpc.pmacs.upenn.edu with gcb537.pmacs.upenn.edu

Note 2: Your username is not "bob"! '

FileZilla0.png


FileZilla1.png


FileZilla2.png


FileZilla3.png


FileZilla4.png


FileZilla5.png


FileZilla6.png


FileZilla7.png


FileZilla8.png


FileZilla9.png


Setup information for Windows Users

Summary of steps

  • Verify that you haven't already generated a Public-Priviate keypair (if you haven't used keys before, its safe to assume you don't have a key pair already)
  • If you haven't installed PuTTY (the full installer), you can download and install it from here
  • Once installed, PuttyGen should be used to generate and save a PuTTy compatible Public-Private keypair
  • PuTTY will be configured to use the generated keypair
  • Install WinSCP for file transfer. WinSCP can be downloaded from here
  • Import the PuTTY compatible keys and profile into WinSCP
  • Test the connection
  • Email only the Public key as an attachment to pmacshpc@med.upenn.edu (please don't paste the contents of the file in the email). Ensure that emails have your full name, PennKey and the subject line "Public keys for GCB537"
  • Do not delete either the Public or the Private key till after the semester.

Step 1: Download and install the PuTTY full installer (if you don't already have it installed)

PuTTY download.png

Step 2: Generate Keypair and save a copy of both the Public and Private keys

PuTTYgen1.png


PuTTYgen2.png


PuTTYgen3.png


PuTTYgen4.png


PuTTYgen5.png


PuTTYgen6.png


PuTTYgen7.png


PuTTYgen8.png


PuTTYgen9.png



Step 3: Configure PuTTY to use the keys that were generated above

Note 1: While following the instructions below, replace the server name demohpc.pmacs.upenn.edu with gcb537.pmacs.upenn.edu

Note 2: Your username is not "bob"! '

PuTTY1.png


PuTTY2.png


PuTTY3.png


PuTTY4.png



PuTTY6.png


PuTTY7.png


PuTTY8.png


PuTTY9.png


Step 4: Install and configure WinSCP to use the PuTTY profile created above

Note 1: While following the instructions below, replace the server name demohpc.pmacs.upenn.edu with gcb537.pmacs.upenn.edu

Note 2: Your username is not "bob"! '

WinSCP0.png


WinSCP1.png


WinSCP2.png


WinSCP3.png


WinSCP4.png


WinSCP5.png


WinSCP6.png


WinSCP7.png


More VPN INFO

Below are screenshots that describe how to use the PMACS VPN on Windows or MacOS systems:

PMACS VPN OSX setup.png

Below are screenshots that describe how to use the PMACS VPN on GNU/Linux systems:


Forticlient1-1.png


Forticlient2.png


Forticlient3.png


Forticlient4-1.png