how to run MPI + OpenMP batch Lester Ingber https://conferences.xsede.org/c/message_boards/find_message?p_l_id=&messageId=657097 2014-02-03T19:33:18Z 2014-02-03T19:32:23Z

I just had a chance to test my new scripts. There are few corrections in the attached zipfile (containing three scripts) from my previous posting.<br /><br />Lester

Lester Ingber 2014-02-03T19:32:23Z Lester Ingber https://conferences.xsede.org/c/message_boards/find_message?p_l_id=&messageId=655579 2014-01-31T02:24:16Z 2014-01-31T02:24:16Z

Jeff:<br /> <br />Thanks for your reply. I got some concrete answers from Glenn Lockwood in XSEDE Support. He recommends splitting the total jobs into pieces. The final draft I have is attached in three short files {bundler.pl, xxsede_mpi_openmp_master.csh, xxsede_mpi_openmp_template.csh} contained in mpi_open.zip.<br /><br />Lester

Lester Ingber 2014-01-31T02:24:16Z Lester Ingber https://conferences.xsede.org/c/message_boards/find_message?p_l_id=&messageId=652531 2014-01-26T23:49:17Z 2014-01-25T16:05:52Z

Without complete docs on mpirun and ibrun, I still am not sure how to submit new jobs on trestle?<br /><br />I have successfully run jobs under MPI with a script:<br />#!/bin/tcsh -xv<br /># qsub xxsede_multiple.csh &gt;&amp;! tp.log_all<br />#PBS -q normal<br />#PBS -A TG-PHY130022<br />#PBS -l nodes=4:ppn=30<br />#PBS -l walltime=25:00:00<br />#PBS -o xsede_output<br />#PBS -N cmi_eeg<br />#PBS -V<br />cd $PBS_O_WORKDIR<br />...<br />mpirun_rsh -np 120 -hostfile $PBS_NODEFILE MV2_ENABLE_AFFINITY=0 {$PWD_WORK}/CMI_EEG/bundler.pl {$PWD_WORK}/CMI_EEG/tasks<br /><br /><br />However, now, each of the 120 MPI processes/runs/fits (using my ASA optimization code) ALSO can run independently in parallel under OpenMP. I understand that each ASA run can spawn of up to 16 threads, e.g., 120x16 MPI+OpenMP processses could run together. How do I submit this? What changes need to be made in the above script? Is 512 the maximum that can be spawned at any time; if so, then I likely would need 4 separate aggregate runs of 30x16 each, etc.<br /><br />Alternatively, I was advised to run batch scripts using ibrun. I assume that this means I would use ibrun INSTEAD of mpirun? If so, how do I pass the same parameters + #PBS settings as I have been using under mpirun?<br /><br />Thanks.<br /><br />Lester<br /><br /><br />P.S.:<br />I do see quite a bit of info on such hybrid runs on<br />https://www.cac.cornell.edu/ranger/Hybrid/smpnodes.aspx<br />but I would like some definitive answers on the use of my previous mpi script to expand into the hyrbrid runs on Tresles. So far, the info I have received does not seem to jive with the kinds of codes/examples given in the cornell tutorial.<br /><br />I&#039;d be just as happy to run a top-level MPI-C code that would call ASA-C codes, each ASA-C run set up as now to use OpenMP. I&#039;d just like to get an example that will run on Trestles.

Lester Ingber 2014-01-25T16:05:52Z