Data Transfer
=============

Files can be moved to and from the cluster in a variety of ways. For transferring large amounts of data, Globus is recommended.

| Prerequisite: 
| Duo Connect
| Global Protect VPN


SCP(Linux and Mac users)
------------------------
Secure copy protocol is used to transfer data between a local host and a remote host or between two remote host.
The format for using scp is the following:

.. hpc-prompt:: hpcterm >
 
   scp <src> <dest>

Transfer local to remote(upload)
................................
From your local machine

.. hpc-prompt:: hpcterm local@local-desktop:

   scp -r /path/to/localfiles  <username>@easley.auburn.edu:/home/username/location


Transfer remote to local(download)
..................................
From your local machine

.. hpc-prompt:: hpcterm local@local-desktop:

   scp -r <username>@easley.auburn.edu:/home/username/location  /path/to/localfiles/
 


Transfer remote to remote(Transfer Hopper files to Easley)
..........................................................
From Easley console,

.. hpc-prompt:: hpcterm >

   scp -r username@hopper.auburn.edu:/home/username/path/to/files  /home/username/path/to/files/

Transfer script
...............
Migration scripts are available for users on the Easley cluster that use scp:

| /tools/scripts/migrate_hopper_home.sh 
| /tools/scripts/migrate_hopper_scratch.sh 


When to use SCP?
................
When transferring small amounts of data from a Linux or Mac machine.


WINSCP(Windows Users)
---------------------
If you want to move files from Hopper/Easley to your local windows machine we suggest using winscp. Winscp offers the ease and convenience of drag and drop capabilities.

When to use Winscp?
...................
When transferring small amounts of data.

GLOBUS
------

If it’s a large amount of files you are moving between Hopper and Easley we suggest using Globus. Globus uses gridftp and in most cases will 
dramatically increase transfer speed.
If you want to transfer to your local machine, choose the Globus Personal Connect option from Collections.

The following instructions focus on using Globus to transfer between Easley and Hopper but it can just as easily transfer to/from your own machine.
Open a browser tab to globus.org and click Log In.


.. image:: images/step1.png
  :width: 800
  :align: center


Choose Auburn University from 'Use your existing organization login' dropdown then click the Continue.

.. image:: images/step2.PNG
  :width: 800
  :align: center

Login using your AU credentials. ( you should see the standard AUthenticate screen )

.. image:: images/step3.PNG
  :width: 800
  :align: center

Choose FILE MANAGER from the ribbon on the left.

.. image:: images/step4.PNG
  :width: 800
  :align: center

Choose the to/from locations.
Click inside the either of the 'Collection' boxes and type auburn ( it should present matching selections as you type )
Easley: auburn#easley
Hopper: auburn#hopper
Once you choose you will be prompted to login.

.. image:: images/step5.PNG
  :width: 800
  :align: center


After you've logged in you will see your $HOME directory on that server.
There is also a 'Path' box where you can type ( ie: /scratch )select either files or folders to transfer and click the start button that corresponds to the direction you want to transfer.
Next time your Collection choices should be remembered ( though you may need to authenticate again ).

See also https://docs.globus.org/how-to/get-started/.


When to use Globus?
...................
When transferring large amounts of data that will take a significant amount of time, frequent data transfer, or when  using an unreliable connection. Data transfer will typically resume after temporary network disruption.