Making a Tough Call

XSEDE Allocates National-scale Resources With Dedicated Staff, Peer Review, Advanced Online Tools

By Ken Chiacchia

November 30, 2016

 

For Megan Justice, an undergraduate working in Marshall University's Genomics Core Facility, opportunity came in the form of frustration.

"I started working at the facility in August 2014," she says. "About a year later, I was helping with the work on the Sumatran rhinoceros and Narcissus flycatcher genome assemblies."

The two assembly projects, led by Herman Mays, an evolutionary biologist at the Huntington, WV, university; and Jim Denvir, co-director of Marshall's Genomics Core Facility, promised critical advances in our understanding of evolutionary processes. The rhino and flycatcher genomes, the scientists expected, could shed light on how species respond to climate change and other environmental stresses. The rhino, a tropical species, was nonetheless more closely related to the ice-age wooly rhinoceros than to other modern species. The flycatchers are a rare example of a species that have populations that are migratory and some that are not.

The actual wet-lab sequencing of the species' DNA fragments went well, Justice says. But the assembly step, using the computer to combine the sequence fragments into the two full genomes kept crashing their campus computing cluster. That's when their Campus Champion, Jack Smith, told them about XSEDE.

As an undergrad, Justice didn't expect to play a leading role in the XSEDE process. But when it came time to put in for an allocation request, Denvir was swamped with teaching responsibilities.

"He asked me if I'd be willing to work on [the allocation submission]," she explains. "I said, ‘Yes.'"

Online Tools

The XSEDE allocations process begins, as do most XSEDE interactions, with the user portal at portal.xsede.org. (This site requires an XSEDE account, so new users should first check out www.xsede.org/user-portal to get an account.) The user portal allows users to enter the XSEDE Resource Allocation Service (XRAS), an online tool that helps users to apply for XSEDE-allocated resources and helps XSEDE staff to manage the process.

XRAS gives users "one stop shopping" for requesting XSEDE-allocated resources, says Dave Hart, director of XSEDE's Resource Allocations Service and user services manager at the Computation and Information Systems Laboratory of the National Center for Atmospheric Research (NCAR).

These resources are as diverse as the Texas Advanced Computing Center's (TACC's) massively parallel Stampede; the San Diego Supercomputer Center's (SDSC's) Comet cluster; Indiana University's/TACC's Jetstream cloud computing environment; and the Pittsburgh Supercomputing Center's (PSC's) heterogeneous Bridges system, featuring extreme memory nodes. XSEDE also includes valuable human resources: an XSEDE-supported corps of computational experts available through the Extended Collaborative Support Service (ECSS).

"XRAS provides an easy-to-use interface to request all XSEDE-allocated resources," says Amy Schuele, XSEDE manager of allocations, accounting and account management systems; and technical program manager at the National Center for Supercomputing Applications (NCSA). "There is also quite a bit of documentation on the user portal about all the resources that are allocated through XSEDE," which can be a big help in gauging which XSEDE resources are the best fit for a given project; ECSS can also help determine which machines to request.

Once users have evaluated their computational plans and looked through the user guides, they can submit a request. Startup allocations can be requested any time; lager-scale research allocation requests are accepted quarterly. XSEDE personnel recommend a startup as a first step even for users experienced with high performance computing resources elsewhere.

"We always encourage people to have some initial experience [with XSEDE resources], whether that be a startup allocation or through a Campus Champion's allocation," says Ken Hackworth, XSEDE allocations manager and coordinator of allocations at PSC. "What we might call benchmarking: they're testing their code performance … so that they're ready to submit for a larger allocation."

Justice found the process to be straightforward.

"The first thing I did was get on the XSEDE user portal and read a lot of examples of allocations," she says. "I researched the different centers to see which was the best fit for us … The examples online were very specific, and included a genomics assembly example. It's a step-by-step process that shows you what you'll need for the next step before you even get there."

Weighing Carefully

Total research allocation requests each quarter can substantially exceed the available resources. Because of that, the next step for the larger proposals is peer review. For that process, XSEDE convenes the XSEDE Resource Allocations Committee (XRAC) each quarter to review and discuss the merits of the research requests. XRAC is composed of computational experts from universities and research organizations across the country.

"We receive the requests every quarter in a submission window that lasts for a month," Hackworth says. "Then we pull together all the proposals—we're seeing an average of 200-plus requests per quarter … In total, in a quarter we'll see over a billion compute hours requested."

"The reviewers are my heroes," Hart adds. "They are faced with a huge volume of requests every quarter; they volunteer their time, and we couldn't do this without them." The reviewers tend to be doubly experienced: In addition to being subject matter experts in the field they're judging, many are also XSEDE users themselves.

Unfortunately, the over-subscription of the resources means that XSEDE usually has to scale back the review panel's recommendations. For this task, they use an algorithm that weighs available compute time, the size of each request, and to some extent the awardee's source of funding — NSF-funded awards receive a greater weight, though any funding source for non-proprietary work is eligible. Of course, all XSEDE allocations are supported by the NSF and users are not charged for them. The full policy can be found at portal.xsede.org/allocation-policies#80.

XSEDE is interested in helping users succeed in their allocation requests. Training courses and webinars as well as unofficial help such as pre-submission reviews of proposals are all available on request.

Continued Refinement

XSEDE continues to streamline and refine the allocations policies and processes, Hart says. One innovation planned for 2017 is a series of submission templates for users. These should have the dual benefit of clarifying what the reviewers are looking for and splitting material in submissions into fields that can be searched and analyzed more effectively.

"This will give [users] somewhere to start," Hart says. "It won't be mandatory at first and will roll out over time as we get feedback from XRAC members. And for experienced users it will be OK to still use their successful prior submissions as a starting point." The aim, he explains, will be to give first-time users in particular a greater chance at success.

For Justice, her experience with the XSEDE allocations process provided more than just the experience of participating in the successful assembly of two biologically important species.

"Marshall is not a very large university," she explains. "When it was time to apply to grad school, I wanted to take a step to a larger institution. At the University of North Carolina Chapel Hill, they were so impressed I had worked on a project like this, that I had worked independently on writing this application. I think it really gave me an edge … When I accepted their offer of admission, one of the researchers who interviewed me even called to ask if I would be interested in doing a rotation in their lab."