MPI job submission using gcc
- Details
- Hits: 8300
To run mpi programs using gcc, you must first load the right compiler and the right mpi lib. This is done by using:
module load compilers/gcc-4.1.2
module load mpi/openmpi-1.3.2_gcc-4.1.2
A list of all available modules is found by using
module avail
Quicksteps:
- compile program
- decide what queue you're runnning, what paralell environment, how many slots
- wait program to finish, view output
[alexandru.herisanu@fep-53-3 mpi]$ qsub -q ibm-quad.q -pe openmpi 4 -cwd
module load compilers/gcc-4.1.2
module load mpi/openmpi-1.3.2_gcc-4.1.2
mpirun -np $NSLOTS ./mpi_scatter
Your job 12 ("STDIN") has been submitted
[alexandru.herisanu@fep-53-3 mpi]$ watch qstat
Last step, view output:
[alexandru.herisanu@fep-53-3 mpi]$ cat STDIN.*12
Warning: no access to tty (Bad file descriptor).
Thus no job control in this shell.
rank= 1 Results: 5.000000 6.000000 7.000000 8.000000
rank= 0 Results: 1.000000 2.000000 3.000000 4.000000
rank= 3 Results: 13.000000 14.000000 15.000000 16.000000
rank= 2 Results: 9.000000 10.000000 11.000000 12.000000
Let me explain the qsub options needed:
-q selects the queue, the group of machines to be used (TBD: add link to all available queues)
ibm-quad.q, ibm-opteron.q, ibm-nehalem.q, cell-qs22.q
-pe requests multiple job slots (paralell environment). This environment can use a tight or loose integration with sge. This basically specifies what mpi flavor to use and how to schedule them. *1 means maximum one mpi process per machine. I want 4 slots using openmpi means
ex: -pe openmpi 4
hpmpi, intelmpi, intelmpi*1, openmpi, openmpi*1
-cwd change current working directory to the one from the job was submitted. This means you can use ./executable to run the program from the script, instead of using the whole absolute path
mpirun -np $NSLOTS ./mpi_scatter
- the $NSLOTS variable here is the one from qsub (-pe openmpi 4).
- we can use ./mpi_scatter because we used -cwd
Intel MPI Howto pe clusterul NCIT
- Details
- Hits: 5531
In cadrul clusterului NCIT, avem la dispozitie compilatorul Intel 11.1 - 0.38 si libraria IntelMPI 3.2.1 - 009.
Pentru a folosi IntelMPI trebuie sa selectam compilatorul si libraria mpi. Acest lucru se face fie prin comanda module:
[heri@fep-53-2 ~]$ module load compilers/intel-11.1_038
[heri@fep-53-2 ~]$ module load mpi/intelmpi-3.2.1_009
sau direct, prin folosirea scripturilor de mediu venite cu compiler-ul. In bash:
[heri@fep-53-2 ~]$ . /opt/intel/Compiler/11.1/038/bin/iccvars.sh intel64 (pt. C/C++)
[heri@fep-53-2 ~]$ . /opt/intel/Compiler/11.1/038/bin/ifortvars.sh intel64 (pt. Fortran)
[heri@fep-53-2 ~]$ . /opt/intel/impi/3.2.1/bin64/mpivars.sh (pt. 64bit arhitecture - default)
[heri@fep-53-2 ~]$ . /opt/intel/impi/3.2.1/bin/mpivars.sh (pt. 32biti, nesuportat pe clusterul nostru)
In cadrul SGE, shell-ul default este csh, iar scripturile de mediu sunt:
[heri@fep-53-2 ~]$ . /opt/intel/Compiler/11.1/038/bin/iccvars.csh intel64 (pt. C/C++)
[heri@fep-53-2 ~]$ . /opt/intel/Compiler/11.1/038/bin/ifortvars.csh intel64 (pt. Fortran)
[heri@fep-53-2 ~]$ . /opt/intel/impi/3.2.1/bin64/mpivars.csh (pt. 64bit arhitecture - default)
[heri@fep-53-2 ~]$ . /opt/intel/impi/3.2.1/bin/mpivars.csh (pt. 32biti, nesuportat pe clusterul nostru)
Environment-ul paralel pentru IntelMPI este intelmpi si intelmpi*1.
#!/bin/csh
source /opt/intel/Compiler/11.1/038/bin/iccvars.csh intel64
source /opt/intel/impi/3.2.1/bin64/mpivars.csh
setenv MPD_CON_EXT "sge_$JOB_ID.$SGE_TASK_ID"
mpiexec -np $NSLOTS ./hello_world.exe
Acest script il salvam sub numele de myscript.sh si se ruleaza cu comanda
qsub -q [coada] -pe [paralell_environment] [nr_sloturi] -cwd myscript.sh
Ex: qsub -q ibm-quad.q -pe openmpi 4 -cwd myscript.sh
(Mai mult detalii despre setup in acest articol).
Job-ul se urmareste cu comanda qstat, sau dupa preferinta watch qstat, iar output-ul se afla in fisierele myscript.*
MPI job submission using SunStudio 12
- Details
- Hits: 6035
The default compiler, Sun Studio 12, is installed in /opt/sun/sunstudio12.
[heri@fep ~]$ cc -V
cc: Sun C 5.9 Linux_i386 Patch 124871-01 2007/07/31
usage: cc [ options] files. Use 'cc -flags' for details
For this example we will compile a basic hello_world example.
Multihomed MPI communication using OpenMPI
- Details
- Hits: 6696
Today we tested something funny. We have a HS21 Blade Center with two Cisco Switches as I/O. The problem we faces was that the uplink speeds between the Blade Centers were maximum 4Gb/s because each cisco switch only had 4 external ports and there was no way to make a trunk between the two I/Os.