MPI job submission using gcc
- Details Hits: 8300
To run mpi programs using gcc, you must first load the right compiler and the right mpi lib. This is done by using:
module load compilers/gcc-4.1.2
module load mpi/openmpi-1.3.2_gcc-4.1.2
A list of all available modules is found by using
module avail
Quicksteps:
- compile program
- decide what queue you're runnning, what paralell environment, how many slots
- wait program to finish, view output
[alexandru.herisanu@fep-53-3 mpi]$ qsub -q ibm-quad.q -pe openmpi 4 -cwd
module load compilers/gcc-4.1.2
module load mpi/openmpi-1.3.2_gcc-4.1.2
mpirun -np $NSLOTS ./mpi_scatter
Your job 12 ("STDIN") has been submitted
[alexandru.herisanu@fep-53-3 mpi]$ watch qstat
Last step, view output:
[alexandru.herisanu@fep-53-3 mpi]$ cat STDIN.*12
Warning: no access to tty (Bad file descriptor).
Thus no job control in this shell.
rank= 1 Results: 5.000000 6.000000 7.000000 8.000000
rank= 0 Results: 1.000000 2.000000 3.000000 4.000000
rank= 3 Results: 13.000000 14.000000 15.000000 16.000000
rank= 2 Results: 9.000000 10.000000 11.000000 12.000000
Let me explain the qsub options needed:
-q selects the queue, the group of machines to be used (TBD: add link to all available queues)
ibm-quad.q, ibm-opteron.q, ibm-nehalem.q, cell-qs22.q
-pe requests multiple job slots (paralell environment). This environment can use a tight or loose integration with sge. This basically specifies what mpi flavor to use and how to schedule them. *1 means maximum one mpi process per machine. I want 4 slots using openmpi means
ex: -pe openmpi 4
hpmpi, intelmpi, intelmpi*1, openmpi, openmpi*1
-cwd change current working directory to the one from the job was submitted. This means you can use ./executable to run the program from the script, instead of using the whole absolute path
mpirun -np $NSLOTS ./mpi_scatter
- the $NSLOTS variable here is the one from qsub (-pe openmpi 4).
- we can use ./mpi_scatter because we used -cwd