ECPS Cluster : SSH and SFTP ACCESS

SSH is the basic tool for remote access in UNIX/Linux systems.

In this page you will learn:

 

Your operating system Windows™ Mac OS X Linux (RedHat, CentOS) Linux (Debian, Ubuntu)
Start by installing... MobaXTerm XQuartz
yum install \
  openssh-clients
apt-get install \
  openssh-client
Then...

Set up your SSH client

To access your home directory through SFTP using a graphical interface... Use MobaXTerm's SFTP feature (instructions) Install and configure Cyberduck
yum install \
    filezilla

Usage instructions

apt-get install \
   filezilla

Usage instructions

To access your files directly...   Use sshfs    

 

First steps with SSH and the command line

In this section, you will be getting familiar with the UNIX command line, both locally (on your workstation or laptop) and remotely (on the cluster).

Get familiar with the command line on your workstation

  1. Start a terminal on your workstation:
    Mac OS X Run the Terminal application
    Linux

    Run gnome-terminal, Konsole, xterm or equivalent

    Windows

    Start MobaXTerm

     

  2. Get familiar with the command line on your workstation.
    On all operating systems (including Windows and MobaXTerm), the commands will look the same:
    Command What it does Examples
    ls
    List directory ls
    ls /drives/c   # (Windows)
    ls /etc           # (Mac OS X, Linux)
    cd
    Change directory cd /drives/c   # (Windows)
    cd /etc           # (Mac OS X, Linux)
        # Then try again
    ls
        # Return to your home directory with just
    cd
    pwd
    Print Working Directory
    pwd

     

  3. Get familiar with the following features and specificities, which are all hallmarks of UNIX and your shell (the program that is accepting the commands):
    • command-line completion (TAB key)
    • history (Up arrow key, Ctrl+R etc.)
    • the prompt (the short piece of text displayed before every command)
    • single file system tree: there are no drive letters – Even under Windows, the C: drive is represented as the /drives/c directory
    💡If your workstation runs Linux or Mac OS X, the shell is a deeply embedded component of your operating system; for instance, a number of automated tasks are written as scripts run by the shell. On the other hand, if you are using Windows, a compatibility layer known as Cygwin (installed as part and parcel of MobaXTerm) is at work behind the scenes.
  4. Exercise: using the cd and ls commands, find and list your files on your workstation's drive.

If you would like to learn more:

Access ECPS over SSH

Now that you are familiar with how the command line works on your local workstation, let's connect to ECPS and take a look.

  1. Type
    ssh -X ecpsinf01.epfl.ch -l mygaspar
    ⚠ Be sure to replace mygaspar with your GASPAR user name.
    This should prompt you for a password (later, we'll look into making the access password-less). Type in your GASPAR password, and you should be granted access
  2. Look around: you will find that while the shell looks and feels the same as your workstation, the paths and files are different.

  3. The node you are logged into right now is called the front-end node. There are other nodes; you can reach them over ssh as well from the front-end node, for instance like this
      ssh ecpsc10
    💡This time around, you shouldn't have to type in your password, remember your GASPAR username or pass the -X command line option; this is all automatic. (We'll learn later how to achieve the same for the initial ssh command.)
    💡You will find that the /home directory (where your personal files are, as well as your colleagues') is indeed the same on all compute nodes and the front-end node. The reason is that they are mounted over NFS from a shared file server node (that you do not have ssh acess to; only administrators have it).
  4. To close the various ssh sessions, just type
    exit
    or close the terminal window they are running in.

X11 basics

If set up properly, the SSH protocol provides a number of additional services besides the command line; among which is the ability to run graphical programs on a remote server, and have the results displayed on your workstation.

  1. ssh into the front-end node as per the above §
  2. from there, ssh into one of the compute nodes
  3. type
    matlab
    After a while, the Matlab display should appear on your workstation.

    💡Note how the window decoration, ugly fonts, and overall sluggishness indicate that you are running over a network connection. Plus, you need to keep your workstation up and running and connected to the network at all times, lest Matlab crash! In actual production use, you would want to use a different way to use Matlab in the cluster. This is just to get the point across regarding X11.

Automation

Logging In Without a Password

In order to log in to ECPS without a password, you need to set up a private / public key pair on your workstation, and then upload the public key with ECPS to make it trusted for your home account.

  1. Open a terminal on your workstation
  2. Type
      ssh-keygen
    When prompted for a pass phrase, just press Enter.
  3. Type
    ssh-copy-id mygaspar@ecpsinf01.epfl.ch
    You should be prompted for your password one last time. Be sure to replace mygaspar with your GASPAR user name.
  4. Control that you can now ssh (like you did previously) without having to enter a password.

More information about SSH public keys

💡 As we saw previously, this is not required for password-less SSH inside the cluster. This is because te latter uses a different authentication mechanism.

Forgetting About All the Command-Line Options

If you edit your .ssh/config file inside your home directory on your workstation, you can put in there all the flags, GASPAR login names etc. that we've been using. Even better, adding a few tricks to your ~/.ssh/config file will let you access individual nodes inside the cluster in a single command.

  1. Copy and paste the following text into your .ssh/config – Be sure to replace mygaspar with your GASPAR login name, as usual.
    Host ecpsinf01.epfl.ch ecpsinf01 ecps
         Hostname ecpsinf01.epfl.ch
         User mygaspar
         ForwardAgent yes
         ForwardX11 yes
         ForwardX11Timeout 596h
         DynamicForward 3333
    
    Host *
        XAuthLocation /opt/X11/bin/xauth
      
    💡 The dot (.) at the beginning of .ssh/config means that the .ssh directory is hidden under UNIX and Mac OS X (it won't show up with ls by default). It is also somewhat hidden in Windows, but for a different reason. You will find it under My Documents → MobaXterm → home → .ssh
  2. You should now be able to access any node directly (as if by a single hop) without any additional arguments or flags on the command line, i.e.
      ssh ecpsinf01
    and all the aforementioned features should Just Work™ (X11 forwarding and SOCKS proxy)

Using sshfs to access your files (Mac only)

  1. Install homebrew
  2. Type
    brew install sshfs
  3. Profit!!