Jobs running abnormally slow on Comet Joseph Andrew Barranco https://conferences.xsede.org/c/message_boards/find_message?p_l_id=&messageId=1585876 2017-06-25T18:40:07Z 2017-06-25T18:38:47Z

Recently, some jobs on Comet seem to be running a factor of 4 slower than usual (that is, they run slow from the start and throughout the entire calculation). I have cancelled such jobs and and started them over, and then they run at the expected speed. This has only occurred since I started running on 32 nodes (768 cores). What could cause the exact same code to behave like this?<br /><br />I am going to start to keep track of which nodes are being used... Could it be a problem with one faulty slow node? My code runs at the pace of the slowest node.<br /><br />Any advice on how I might diagnose this?<br /><br />For the record, this code has run successfully on Stampede (TACC) and Pleiades (NASA) and not had this problem.

Joseph Andrew Barranco 2017-06-25T18:38:47Z