HPL on Intel MPI+MKL

来源：互联网发布：mac分区后怎么恢复编辑：程序博客网时间：2024/05/14 02:51

Disable CUDA

1. Compiling Make.CUDA: delete-DCUDA

2. SetCUDA_DGEMM_SPLIT, CUDA_DTRSM_SPLIT to 0.00 (run_linpack script)

Modify code so that the codes could run directly on CPU nodes
/src/cuda/cuda_dgemm.c
Line:690 split < 0.01
testing/ptest/HPL_pdtest.c

Load Libries
module load intel
module load cuda/5.0rc
source /home/limin/intel/impi/4.0.3.008/bin64/mpivars.sh
export LD_LIBRARY_PATH=/opt/intel/composer_xe_2013.0.079/mkl/lib/intel64:$LD_LIBRARY_PATH

Supose we have two nodes in hosts

Intel MPI

export OMP_NUM_THREADS=8
/home/limin/intel/impi/4.0.3.008/intel64/bin/mpirun -n 8 -f hosts -perhost 4 -genv I_MPI_PIN_DOMAIN node ./xhpl_hybrid_intel64
export OMP_NUM_THREADS=4
/home/limin/intel/impi/4.0.3.008/intel64/bin/mpirun -n 8 -f hosts -perhost 2 -genv I_MPI_PIN_DOMAIN=omp:scatter ./xhpl_hybrid_intel64

If use MV2 (compiled with MV2 MPI)
mpirun_rsh -np 1 -hostfile hosts MV2_CPU_MAPPING=0-3:4-7 ./xhpl_hybrid_intel64
mpirun_rsh -np 2 -hostfile hosts MV2_ENABLE_AFFINITY=0 ./xhpl_hybrid_intel64