I. Install mpi4py

1. on Window

Note:

- install Visual Studio Build Tools at this link: http://go.microsoft.com/fwlink/?LinkId=691126&fixForIE=.exe

- download Microsoft MPI v10.1, install both msmpisdk.msi and msmpisetup.exe

https://docs.microsoft.com/en-us/message-passing-interface/microsoft-mpi

#check: mpiexec -help

1.1a. install mpi4py with conda:

# for intel MPI

pip install mpi4py impi

1.1b. install mpi4py on Python without Conda: (being used)
install another python3.7 independent to Conda
https://www.python.org/downloads/windows/
pip install mpi4py numpy scipy

pip search mpi
pip install impi

# register mpiexec
mpiexec -register
windows user & password

test: mpiexec -np 2 python -m mpi4py.bench helloworld

1.2 Run:

- with anaconda: open Anaconda_Prompt
$ mpiexec -np 8 python script.py

- without anaconda: open cmd as administrator
$ mpiexec -np 8 python script.py

2. on Linux

Install Conda

make sure mpi4py link to right MPI lib (OpenMPI or MPICH, or Impi), then use right command to run:

mpirun -np 5 python
# or
mpiexec -np 5 python

2.1. Install with installing conda-mpi:

to chose right version mpi4py, have to attach option [-c CHENNAL] in conda install,
conda-forge
intel
bioconda
anaconda

Note: conda-forge contains both version of mpi4py for openmpi and mpich
conda search -c conda-forge mpi4py # find package
conda search -c intel mpi4py
conda search -c anaconda mpi4py

conda install [-c chennal] <package_name>=<version>=<build_string>

conda create --name new_name --clone old_name
conda remove --name old_name --all

# for mpi4py with OpenMPI: (being used)
module load conda/conda3

conda create -n py37ompi python=3.7 scipy numpy scikit-learn

source activate py37ompi
pip install tess ovito # voro++

conda install -c conda-forge mpi4py=3.0.3=py37hd0bea5a_0

# for mpich

conda create --name py37mpich --clone py37ompi

source activate py37mpich
conda uninstall mpi4py mpi

conda install -c conda-forge mpi4py=3.0.3=py37hcf07815_0

# for intel MPI (just support python 3.6)
conda create -n py36impi python=3.6 scipy numpy scikit-learn
source activate py36impi
conda install -c intel mpi4py=3.0.0=py36_intel_0
pip install tess ovito

TEST:
mpirun -np 5 python -m mpi4py.bench helloworld
## ------
Hello, World! I am process 0 of 5 on leopard.
Hello, World! I am process 1 of 5 on leopard.
Hello, World! I am process 2 of 5 on leopard.
Hello, World! I am process 3 of 5 on leopard.

Hello, World! I am process 4 of 5 on leopard.

NOTE: there are 3 env for python with include conda-mpi: py36mpi, py37mpi, py27mpi
- but load conda-mpi maybe cause unexpected conflict with other MPI. So consider to install mpi4py only, without include conda-mpi --> use pip install

2.2. Install without installing conda-mpi: (being used)

Note: using conda to install mpi4py, it also installsconda-mpi which cannot control and may conflict with other MPI. To use openMPI we want, must use pip install: (but this normally fail to link MPI compiler with python36, work with python37)
module load conda/conda3

conda create -n py37 python=3.7 scipy numpy

source activate py37

pip install mpi4py tess

TEST: (this word on Centos7)
module load mpi/openmpi4.0.2-Intel2019xeU4
module load conda/py37

mpirun -np 5 python -m mpi4py.bench helloworld
## ------

TEST: ( on Centos6 --> glibc error)
module load mpi/openmpi4.0.2-Intel2019xe
module load conda/py37

mpirun -np 5 python -m mpi4py.bench helloworld

## ------

II. Install Voro++

module load conda3

source activate py37

pip install tess # voro++ library

Ref : https://github.com/abria/TeraStitcher/wiki/Multi-CPU-parallelization-using-MPI-and-Python-scripts
https://oncomputingwell.princeton.edu/2018/11/installing-and-running-mpi4py-on-the-cluster/

https://www.tegakari.net/en/2020/08/windows-subsystem-for-linux-2wsl2_vol1/

I. Install WSL on Windows

https://docs.microsoft.com/en-us/windows/wsl/install-win10

There are 2 versions: Comparing WSL 2 and WSL 1. Should use WSL 2 for better performance.

a. Prepare
# Enable both the Windows Subsystem for Linux and the Virtual Machine Platform optional components.
Open PowerShell as an admin and run the following script: (After the script is complete, you need to reboot your machine, since this enables new Windows features.)

dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart

dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart

# Download & Install the Linux kernel update package

WSL2 Linux kernel update package for x64 machines

# Set WSL 2 as your default version: in Powershell

wsl --set-default-version 2

b. Installing WSL:

# using Microsoft Store
go to: https://aka.ms/wslstore
and intsall a Linux shell

# not use Microsoft Store

https://docs.microsoft.com/en-us/windows/wsl/install-manual

download: Ubuntu_2004.2020.424.0_x64.appx

in powerShell: Add-AppxPackage .\Ubuntu_2004.2020.424.0_x64.appx

c. Create a Unix Account

Open up a Command Prompt and run the following command: > bash

d. Location

on Window 10, "~home/user" directory locates at :

## for WSL1

%localappdata%\lxss\home\{username}
C:\Users\{user}\AppData\Local\lxss\{username}

## for WSL2

\\wsl$\Ubuntu-20.04\home\{username}

## Hard drives: cd /mnt/d/work/

II. Work on WSL

# Login root as default
open cmd: ubuntu config --default-user root

# Update Ubuntu: sudo apt update && apt upgrade

# delete folder/file

sudo -r -f \folder

# install GCC

sudo apt-get install gcc-10 g++-10 gfortran-10

# install python

https://phoenixnap.com/kb/how-to-install-python-3-ubuntu

# check: python --version

## set isntall python3 as default (similar for python2, )

sudo apt install python-is-python3 python-dev python3-dev

1. Insall Modules
sudo apt-get install tcl environment-modules

2. install cuda

2a. Install NVIDIA driver for WSL2

https://docs.nvidia.com/cuda/wsl-user-guide/index.html

download: https://developer.nvidia.com/cuda/wsl/download

460.20_gameready_win10-dch_64bit_international.exe

Note: Do not install any Linux display driver in WSL. The Windows Display Driver will install both the regular driver components for native Windows and for WSL support.

2b. Install cuda-toolkit for WSL2

https://docs.nvidia.com/cuda/wsl-user-guide/index.html

# Launch Ubuntu terminal

# First, set up the CUDA network repository.

sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub
sudo sh -c 'echo "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64 /" > /etc/apt/sources.list.d/cuda.list'
sudo apt-get update

# Now install CUDA. Note that for WSL 2, you should use the cuda-toolkit-<version>
#(Do not choose the cuda, cuda-11-0, or cuda-drivers meta-packages under WSL 2)

sudo apt-get install -y cuda-toolkit-11-0

#test  (path: \\wsl$\Ubuntu-20.04\usr\local\cuda-11.1)
nvcc --version               # cuda version

NOTE: gpu works with Window Insider Prebuild 20236 -->

3. OpenMPI + gcc +gpu

cd wSourceCode
tar xvf openmpi-4.1.0rc3.tar.gz
cd openmpi-4.1.0rc3
mkdir buildGCC-cuda && cd buildGCC-cuda
##
export myCUDA=/usr/local/cuda-11.0
../configure CC=gcc CXX=g++ FC=gfortran F77=gfortran \
--with-sge --without-verbs --without-ucx --with-cuda=${myCUDA} \
--prefix=/opt/app/openmpi/4.1.0-gcc9.3-cuda11
make
sudo make install

## use, 
export PATH=/opt/app/openmpi/4.1.0-gcc9.3-cuda11/bin:$PATH
export LD_LIBRARY_PATH=/opt/app/openmpi/4.1.0-gcc9.3-cuda11/lib:$LD_LIBRARY_PATH

## make Module files (\\wsl$\Ubuntu-20.04\home\tha\1moduleFiles)
# module use /home/tha/1moduleFiles
set     topdir          /opt/app/openmpi/4.1.0-gcc9.3-cuda11

prepend-path   PATH                $topdir/bin
prepend-path   LD_LIBRARY_PATH     $topdir/lib
prepend-path   INCLUDE             $topdir/include 

prepend-path   PKG_CONFIG_PATH 	   $topdir/lib/pkgconfig          # this is required

4. Compile lammps
### we have GTX 1060 with Pascal architecture
sudo apt install zlib1g-dev ocl-icd-opencl-dev  pkg-config
sudo apt-get install libblas-dev liblapack-dev libgsl-dev    #for plumed
##
git clone --branch master https://github.com/lammps/lammps.git lammps_master 
cd lammps_master 
git pull origin master
mkdir build-gpu && cd build-gpu

##
module load cmake-3.18.3 
module load ompi/4.1.0-gcc9.3-cuda11
module load cuda-11.2
#--
export PATH=$PATH:/opt/app/openmpi/4.1.0-gcc9.3-cuda11/bin
export CC=mpicc
export CXX=mpic++
export FORTRAN=mpifort
cmake ../cmake -C ../cmake/presets/all_on.cmake \
-DLAMMPS_EXCEPTIONS=yes -DBUILD_MPI=yes -DBUILD_OMP=yes -DLAMMPS_MACHINE=mpi \
-DPKG_USER-OMP=yes -DPKG_USER-INTEL=no -DPKG_KOKKOS=yes \
 -DPKG_GPU=yes -DGPU_API=cuda -D GPU_ARCH=sm_61 \
-DPKG_USER-SMD=yes -DDOWNLOAD_EIGEN3=yes -DDOWNLOAD_VORO=yes \
-DPKG_KIM=no -DDOWNLOAD_KIM=no -DPKG_LATTE=no -DPKG_MSCG=no -DPKG_USER-ATC=no -DPKG_USER-MESONT=no  \
-DPKG_USER-ADIOS=no -DPKG_USER-NETCDF=no -DPKG_USER-QUIP=no -DPKG_USER-SCAFACOS=no \
-DPKG_USER-VTK=no -DPKG_USER-H5MD=no \
-DPKG_USER-PLUMED=yes -DDOWNLOAD_PLUMED=yes \
-DCMAKE_C_COMPILER=mpicc -DCMAKE_CXX_COMPILER=mpic++ -DCMAKE_Fortran_COMPILER=mpifort \
-DCMAKE_INSTALL_PREFIX=/opt/app/lammps/master-gpu

make -j 8
sudo make install

####################

-DGPU_API=cuda -D GPU_ARCH=sm_61

III. Install Anaconda on WSL:

https://gist.github.com/kauffmanes/5e74916617f9993bc3479f401dfec7da

Go to https://repo.continuum.io/archive to find the list of Anaconda releases
From the terminal run wget https://repo.continuum.io/archive/[YOUR VERSION].

wget https://repo.continuum.io/archive/Anaconda3-2019.07-Linux-x86_64.sh

Run installation : bash Anaconda3-2019.07-Linux-x86_64.sh

Update conda (optional): conda update conda

Manually add the Anaconda bin folder to your PATH:

1. open file "~/.bashrc" (note: use Vim in Linux, donot open using Windows explorer)

(C:\Users\thang\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\home\tha)

2. added : export PATH=/home/tha/anaconda3/bin:$PATH

Test the installed python:

open Sub_Linux_Prompt, and type: which python

/home/tha/anaconda3/bin/python

II. Install Xeus-cling Kernel for Jupyter notebook on Window's subLinux (WSL)

3. Install Xeus-cling

- Assume Conda is installed
- open Sub_Linux_Prompt, and type: conda install xeus-cling notebook -c QuantStack -c conda-forge
- Update xeus-cling (optional): conda upate xeus-cling

4. set the BROWSER for WSL:

add the following command to the bottom of my ~/.bashrc file.

vim .bashrc

export BROWSER='/mnt/c/Program Files (x86)/Google/Chrome/Application/chrome.exe'

NOTE: open .bashrc file using Windows may cause "permission error" on next launch. To solve it, type:
sudo chmod -R 777 /home/thang/.bashrc

5. Using:

open "Sub_Linux_Prompt" from an "arbitrary folder": Shift + Right Click and selecting the Open Linux shell here
type: jupyter notebook
copy the last URL in Sub_Linux_Prompt to IE

Ref.:
https://github.com/QuantStack/xeus-cling/blob/master/README.md
https://libinruan.github.io/2018/11/06/Install-Jupyter-s-C-kernel-on-Windows-Subsystem-for-Linux/
https://gist.github.com/kauffmanes/5e74916617f9993bc3479f401dfec7da
https://www.howtoforge.com/vim-basics

Thang's Notes

menu

Friday, November 29, 2019

Installing mpi4py & Voro++