Grus user guide

PREREQUISITES

If you have received a NGI delivery and only want to transfer the data to your SNIC-SENS project on Bianca, please follow our Deliver guide instead.

In order to access the Grus delivery service you need:

You need to be a member of the delivery project on SUPR.
An accepted SNIC common user agreement in SUPR.
An UPPMAX user account

Finally, note that it will take some time before changes in SUPR are reflected everywhere. For example if the PI adds you as a member of the delivery project, it may take an hour until you are able to log in.

NGI has published a set of demonstration videos about downloading data from Grus.

1. Set up 2 factor authentication

Follow the instuctions in Setting up two factor authetication.

Note that this is a second factor specific for UPPMAX. You may have set up two factor authentication for other sites (e.g. SUPR), but you can not use codes from that second factor for authenticating to UPPMAX.

Make sure you enter the correct code! The code you use on Grus is the one labeled UPPMAX, highlighted in green below.

2. Login

Note that Grus only supports connections with sftp. If you connect with regular ssh you will be disconnected directly after authentication.

For small amounts of data: Using a graphical connection:

Possibly the easiest way to get your data is to start a graphical Filezilla instance. To do this on Rackham, connect to the cluster using -Y to enable graphical transfer over the ssh connection.

# connect with graphics enabled
ssh -Y username@rackham.uppmax.uu.se 
    
# start filezilla once you are connected to the cluster
filezilla

Note that you have to be using a terminal program that is able to receive the graphics for this to work. If you are running Windows you can use MobaXterm and on a Mac you can install XQuartz. Please see our Getting started-guide for more information

To connect to Grus from FileZilla you need to click on File and then Site Manager. Inside the Site Manager you can add a New Site, call it Grus and then make sure you enter the connection details as outlined below.

Host: grus.uppmax.uu.se
Port: 22
Protocol: SFTP
Logon Type: Ask for password
User name: <username>-<projid>

Note that you do not have to enter the password at this stage. That comes later. You also have to make a change to a more "hidden" setting called "Limit number of simultaneous connections" in the Transfer settings:

NB! Make sure User is your normal UPPMAX username followed by a hyphen '-' and then the name of your delivery project. For example, 'user-delivery123456'.

In the end it should look like the following:

When you click on connect, you will be prompted for a password. The password is your normal UPPMAX password directly followed by the six digit code from your Authenticator application.

For example, if my password is "secret" and my six digit code at this moment is "287463" the password becomes "secret287463".

Once connected you should see the files on Grus on the right hand side of the Filezilla windows and the files on the computer you are running Filezilla on on the left side. You can now "drag and drop" to download files.

CLI example 1: Using the standard sftp client

Another way to transfer files involve running sftp on the command line and type all the commands yourself. This is recommended for advanced users or if X11-forwarding for some reason is unavailable. You start off by connecting to Grus using the sftp program.

$ sftp <username>-<projid>@grus.uppmax.uu.se
Ex.
$ sftp myuser-delivery999999@grus.uppmax.uu.se

As password you use your normal UPPMAX password directly followed by
the six digits from the second factor application from step 1.

Ex. if your password is "VerySecret" and the second factor code is 123 456 you would type VerySecret123456 as the password in this step.

Once connected you will have to type the sftp commands to look around and download files. Have a look at the Basic SFTP commands guide to get started with it.

CLI example 2: Using lftp
Another possibility is to use lftp (a file transfer client that supports different protocol, sftp among them).

$ lftp sftp://<username>-<projid>@grus.uppmax.uu.se/
Password:
cd ok, cwd=/
lftp <username>-<projid>@grus.uppmax.uu.se:/>

lftp has a mirror command which can be used to download an entire directory tree,

mirror datadir local_copy_of_datadir

will download datadir into local_copy_of_datadir.

NB: You may also need to configure lftp to not open additional connections. This can be done with the command

set net:connection_limnit 1

that you can either execute in your session or stick in ~/.lftp/rc or /etc/lftp.conf.

Mounting the sftp-server with sshfs

This is only possible on your own system. UPPMAX doesn't have sshfs client package installed (security reasons). sshfs exists on Linux -- packages sshf in Ubuntu and fuse-sshfs (from EPEL) on RedHat-ish systems. It also exist on Windows and Mac OS X. When you have sshfs installed, you can mount the wharf. Here is only example with Linux client (as normal user):

mkdir ~/grus_mnt

sshfs <username>-<projname>@grus.uppmax.uu.se: ~/grus_mnt

After that you can use grus_mnt exactly as a local directory. For example with rsync.

To unmount it do:

fusermount -u ~/grus_mnt