Optimal Job Size for the SLURM Scheduler?

2014-03-21T15:12:38Z

I developed my experiment setup on Kraken and ported to Stampede a month and a half ago, and for the first month it was great. In the last week or so, however, my queue throughput has tanked pretty severely. On Kraken, I think the scheduler favored large jobs, except if you could make them small enough to run as "backfill," and I managed to split my experiments up into small enough chunks that it was frequently running as backfill. This brings me to my question: what is the optimal job size? Is it set up the same as Kraken to where it favors larger jobs, but will backfill? If so, how small do jobs need to be to run as backfill? Waiting 4 hours for 32-core-for-20-minute jobs to go through is getting a little old.

Thanks!