datatransferwhere

Where to Initiate Large File Transfers

A GridFTP server is configured to automatically start GridFTP server processes. It supports the Globus GridFTP protocol, performs authentication, and performs very fast data transfers.

See the GridFTP Server Table below for client and server connection info for each XSEDE site. You can also view current end-to-end speed measurements for many XSEDE resources on the PSC GridFTP Metrics page.

Dedicated GridFTP Servers

Many sites have nodes that are dedicated GridFTP servers. These nodes utilize the full XSEDE backbone bandwidth (10-30 Gb/sec), so the Wide Area Network is rarely a bottleneck. Because fewer tasks are running, they generally have better transfer rates. GSI authentication and proxy certificates provide security for such transfers.

While all XSEDE sites have at least one GridFTP server, not all nodes on XSEDE resources are GridFTP servers. When moving data, be sure to request a server listed in the table below to achieve the fastest rate. Transfer nodes are selected via round-robin assignment.

Use the syntax shown on the Transfer Examples page when integrating transfer requests into job execution scripts.

Third Party Transfers

A GridFTP transfer between two servers, rather than between a server and a client, is called a third-party transfer. This occurs when the transfer is initiated by a GridFTP client that is neither the source nor the destination of the operation. This allows use of dedicated transfer nodes without having to log in to the transfer node, simplifying the workflow. See the Transfer Examples page for details.

graphic needed from: https://www.teragrid.org/web/user-support/transfer_location

The table below provides a list of GridFTP resources by site. It provides information on where to start a transfer and where to send the transfer. Any of the servers listed in the GridFTP Server column can be used as the "from" or "to" location.

table/portlet needed from: https://www.teragrid.org/web/user-support/transfer_location The table below provides a list of RFT resources by site. It provides information on where to start a transfer and where to send the transfer. Any of the servers listed in the RFT Server column can be used as the "from" or "to" location.

table/portlet needed from: https://www.teragrid.org/web/user-support/transfer_location

PSC GridFTP Server Deployment

Pople

The Pople GridFTP server is single-striped with a 1Gb/s interface.

PSC Archiver

GridFTP service on the PSC archiver runs on six archiver cache nodes. These are not striped, and each has a 1Gb network interface. Servers are assigned on a round-robin basis as transfers are initiated to/from:

gsiftp://gridftp.archiver.psc.teragrid.org or gsiftp://gridftp.psc.teragrid.org

To transfer a file from the PSC archiver to Pople, login to Pople and use the far get command.

To transfer a file from Pople to the PSC archiver, login to Pople and use the far store command.

SDSC HPSS Archival Storage

HPSS at SDSC does not have a GridFTP front end. HPSS requires the use of HSI interface to move data in and out of HPSS. Therefore, an extra step is required to transfer between compute resource file systems and another XSEDE site using the GridFTP server.

• Move data onto SDSC TG cluster parallel filesystem with hsi.
• Use a grid-ftp tool to move data elsewhere [to and from SDSC]