Using Persistent Communication in Fortran MPI Julio Cesar Mendez https://conferences.xsede.org/c/message_boards/find_message?p_l_id=&messageId=1562708 2017-05-31T13:34:06Z 2017-05-31T13:33:19Z

Dear community; <br />I attended the summer bootcamp on 2015. We discussed OpemMP, MPI and OpenACC. <br />In the boot camp we solved the famous problem of laplace equation. However, I am working with a more complex and bigger problem, a CFD real code. <br />In this code, I need to update the ghost cells at each time step. Therefore the overhead is quite high. As a result, I have decided to use persistent communication, but my ghost cells are populated with zeroes. <br />The code runs, but the solution is not right. As I said before, all my ghost cells are populated with zeroes, I have tried different things but nothing has worked out. <br />I wanted to know if someone has used the persistent communication with the Laplace equation. If so, I will greatly appreciate if you can share the code with me to compare my syntax with a wroking version. Below I show a small part of the code. <br /><br />The code, calls the MPI_Subroutine where I set the communication characteristics. <br /> <br />!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!<br />!Starting up MPI<br />call MPI_INIT(ierr)<br />call MPI_COMM_SIZE(MPI_COMM_WORLD,npes,ierr)<br />call MPI_COMM_RANK(MPI_COMM_WORLD,MyRank,ierr)<br /><br />!Compute the size of local block (1D Decomposition)<br />Jmax = JmaxGlobal<br />Imax = ImaxGlobal/npes<br />if (MyRank.lt.(ImaxGlobal - npes*Imax)) then<br /> Imax = Imax + 1<br />end if<br />if (MyRank.ne.0.and.MyRank.ne.(npes-1)) then<br /> Imax = Imax + 2<br />Else<br /> Imax = Imax + 1<br />endif<br /><br />! Computing neighboars<br />if (MyRank.eq.0) then<br /> Left = MPI_PROC_NULL<br />else<br /> Left = MyRank - 1<br />end if<br /><br />if (MyRank.eq.(npes -1)) then<br /> Right = MPI_PROC_NULL<br />else<br /> Right = MyRank + 1<br />end if<br /><br /><br />! Initializing the Arrays in each processor, according to the number of local nodes<br />Call InitializeArrays<br /><br />!Creating the channel of communication for this computation,<br />!Sending and receiving the u_old (Ghost cells)<br />Call MPI_SEND_INIT(u_old(2,: ),Jmax,MPI_DOUBLE_PRECISION,Left,tag,MPI_COMM_WORLD,req(1),ierr)<br />Call MPI_RECV_INIT(u_old(Imax,: ),jmax,MPI_DOUBLE_PRECISION,Right,tag,MPI_COMM_WORLD,req(2),ierr)<br />Call MPI_SEND_INIT(u_old(Imax-1,: ),Jmax,MPI_DOUBLE_PRECISION,Right,tag,MPI_COMM_WORLD,req(3),ierr)<br />Call MPI_RECV_INIT(u_old(1,: ),jmax,MPI_DOUBLE_PRECISION,Left,tag,MPI_COMM_WORLD,req(4),ierr)<br /><br />End Subroutine MPI_Subroutine<br /><br />!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!<br /><br />From the main code, where the do loop is, I call the MPI_STARTALL and WaitALL in each time step. <br /><br />Call MPI_STARTALL(4,req,ierr) <br />Call MPI_WAITALL(4,req,status,ierr) <br /><br />Req is an array of dimension (4) the same status. <br /><br />I am using Fortran 90... Any suggestions and comments? <br />Thanks before hand

Julio Cesar Mendez 2017-05-31T13:33:19Z