Friday, November 29, 2019

Installing mpi4py & Voro++

I. Install mpi4py

1. on Window

Note: 
- install Visual Studio Build Tools at this link: http://go.microsoft.com/fwlink/?LinkId=691126&fixForIE=.exe
- download Microsoft MPI v10.1, install both msmpisdk.msi and msmpisetup.exe
https://docs.microsoft.com/en-us/message-passing-interface/microsoft-mpi
#check: mpiexec -help

1.1a. install mpi4py with conda: 
# for intel MPI 
pip install mpi4py impi

1.1b.  install mpi4py on Python without Conda: (being used)
install another python3.7 independent to Conda
https://www.python.org/downloads/windows/
pip install mpi4py numpy scipy 
pip search mpi 
pip install impi

# register mpiexec
mpiexec -register
windows user & password
test:       mpiexec -np 2 python -m mpi4py.bench helloworld                       

1.2 Run:

- with anaconda: open Anaconda_Prompt
 $ mpiexec -np 8 python script.py

- without anaconda: open cmd as administrator

 $ mpiexec -np 8 python script.py


2. on Linux

make sure mpi4py link to right MPI lib (OpenMPI or MPICH,  or Impi), then use right command to run: 
mpirun -np 5 python
# or
mpiexec -np 5 python

2.1. Install with installing conda-mpi:  
to chose right version mpi4py, have to attach option [-c CHENNAL] in conda install,
    conda-forge
    intel
    bioconda
    anaconda
Note: conda-forge contains both version of mpi4py for openmpi and mpich
conda search -c conda-forge mpi4py     # find package
conda search -c intel       mpi4py   
conda search -c anaconda    mpi4py   

conda install [-c chennal] <package_name>=<version>=<build_string>

conda create --name new_name --clone old_name
conda remove --name old_name --all
# for mpi4py with OpenMPI(being used)
module load conda/conda3
conda create  -n  py37ompi python=3.7 scipy numpy scikit-learn

source activate   py37ompi
pip install tess ovito                  # voro++ 

conda install -c conda-forge mpi4py=3.0.3=py37hd0bea5a_0


for mpich 
conda create --name py37mpich --clone py37ompi
source activate    py37mpich
conda uninstall mpi4py mpi

conda install -c conda-forge mpi4py=3.0.3=py37hcf07815_0  

# for intel MPI (just support python 3.6)
conda create  -n  py36impi python=3.6 scipy numpy scikit-learn
source activate    py36impi 
conda install -c intel mpi4py=3.0.0=py36_intel_0 
pip install tess ovito 

TEST:
mpirun -np 5 python -m mpi4py.bench helloworld
## ------
Hello, World! I am process 0 of 5 on leopard.
Hello, World! I am process 1 of 5 on leopard.
Hello, World! I am process 2 of 5 on leopard.
Hello, World! I am process 3 of 5 on leopard.

Hello, World! I am process 4 of 5 on leopard.

NOTE: there are 3 env for python with include conda-mpi: py36mpi, py37mpi, py27mpi
 - but load conda-mpi maybe cause unexpected conflict with other MPI. So consider to install mpi4py only, without include conda-mpi --> use pip install

2.2. Install without installing conda-mpi:  (being used)
Note: using conda to install mpi4py, it also installsconda-mpi which cannot control and may conflict with other MPI. To use openMPI we want, must use pip install: (but this normally fail to link MPI compiler with python36, work with python37)
module load conda/conda3

conda create  -n  py37 python=3.7 scipy numpy 


source activate   py37
pip install mpi4py tess

TEST: (this word on Centos7)
module load mpi/openmpi4.0.2-Intel2019xeU4
module load conda/py37

mpirun -np 5 python -m mpi4py.bench helloworld
## ------

TEST: ( on Centos6 --> glibc error)
module load mpi/openmpi4.0.2-Intel2019xe     
module load conda/py37

mpirun -np 5 python -m mpi4py.bench helloworld

## ------


II. Install Voro++

module load conda3
source activate py37 
pip install     tess                                           #  voro++ library


Ref : https://github.com/abria/TeraStitcher/wiki/Multi-CPU-parallelization-using-MPI-and-Python-scripts
https://oncomputingwell.princeton.edu/2018/11/installing-and-running-mpi4py-on-the-cluster/

No comments:

Post a Comment