LLVM-Clang

I. Source Code and Building LLVM

https://llvm.org/docs/GettingStarted.html#getting-the-source-code-and-building-llvm

https://clang.llvm.org/get_started.html

1. Software requirement:

Package	Version	Notes
CMake	>=3.4.3	Makefile/workspace generator
GCC	>=5.1.0	C/C++ compiler1
python	>=2.7	Automated test suite2

# Supporting: 

# libxml2: (can use conda instead)

http://jianghao.wang/post/2017-11-07-install-packages-on-hpc/#install-libxml2

wget ftp://xmlsoft.org/libxml2/libxml2-2.9.9.tar.gz tar zxf libxml2-2.9.9.tar.gz cd libxml2-2.9.9 ./configure --prefix=/home1/p001cao/local/app/libxml2-2.9.9 --without-python

make make install

#--

prepend-path PKG_CONFIG_PATH $topdir/lib/pkgconfig

2. Check out the LLVM project

tar xvf llvm-project-llvmorg-10.0.0.tar.gz
cd llvm-project-llvmorg-10.0.0
#-- or
git clone --branch release/10.x https://github.com/llvm/llvm-project.git llvm-10
cd llvm-10
git checkout release/10.x

git pull origin release/10.x

mkdir build
cd build

3. Build LLVM and Clang:

https://llvm.org/docs/CMake.html
http://btorpey.github.io/blog/2015/01/02/building-clang/

CMAKE_C_COMPILER	The location of the C compiler to use.
CMAKE_CXX_COMPILER	The location of the C++ compiler to use.
CMAKE_INSTALL_PREFIX	The location where the compiler should be installed.
CMAKE_CXX_LINK_FLAGS	Additional flags to be passed to the linker for C++ programs.  See below for more information.
GCC_INSTALL_PREFIX	Setting this parameter when building clang is equivalent to specifying the --gcc-toolchain parameter when invoking clang. See below for more information.

Load compilers:
module load cmake-3.15.1
module load compiler/gcc-9.2.0
export myGCC=/home1/p001cao/local/app/compiler/gcc-9.2.0

cmake ../llvm -G "Unix Makefiles" -DCMAKE_BUILD_TYPE="Release \
-DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_OPTIMIZED_TABLEGEN=ON \
-DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;compiler-rt;libc;libclc;libcxx;libcxxabi;libunwind;lld;openmp;parallel-libs;polly;pstl;mlir" \
-DCMAKE_CXX_COMPILER=${myGCC}/bin/g++ \
-DCMAKE_C_COMPILER=${myGCC}/bin/gcc \
-DGCC_INSTALL_PREFIX=${myGCC} \
-DCMAKE_CXX_LINK_FLAGS="-L${myGCC}/lib64 -Wl,-rpath,${myGCC}/lib64" \
-DCMAKE_INSTALL_PREFIX=/home1/p001cao/local/app/compiler/clang-10

make -j 8

make install

NOTE:

* a dot (.) in cmake mean current dir (cmake --build . -j 8 = make -j 8)
* must use -DGCC_INSTALL_PREFIX=/home1/p001cao/local/app/compiler/gcc-9.1.0
if not, clang will use default system GCC
* CMAKE_CXX_LINK_FLAGS

In the clang build script this is set to "-L${HOST_GCC}/lib64 -Wl,-rpath,${HOST_GCC}/lib64". What this does is two-fold:
* The -L parameter adds the following directory to the search path for the linker. This is needed so the linker can locate the libraries installed with gcc 4.8.2.
* The -Wl,-rpath, parameter installs a “run path” into any executables (including shared libraries) created during the build. This is needed so any executables created can find their dependent libraries at run-time.

https://stackoverflow.com/questions/50510626/manually-built-gcc-and-clang-clang-looks-at-system-c-paths-not-my-gccs

http://clang-developers.42468.n3.nabble.com/Trouble-building-Clang-on-CentOS-6-td4058013.html

I.b Using Ninja

-G "Unix Makefiles": usually get error, so consider to use -GNinja

https://libcxx.llvm.org/docs/BuildingLibcxx.html

## build Ninja with Cmake:
git clone git://github.com/ninja-build/ninja.git

cd ninja
#--

module load cmake-3.15.1

module load compiler/gcc-9.2.0

### CMake
cmake -Bbuild-cmake -H.
cmake --build build-cmake

## this will build ninja in dir:  ../build-cmake/ninja

## build clang 10

module load cmake-3.15.1
module load conda/py37clangSupp
module load compiler/gcc-9.2.0
export myGCC=/home1/p001cao/local/app/compiler/gcc-9.2.0

cmake ../llvm -GNinja -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_MAKE_PROGRAM=/home1/p001cao/local/wSourceCode/ninja/build-cmake/ninja \
-DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_OPTIMIZED_TABLEGEN=ON \
-DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;compiler-rt;libc;libclc;libcxx;libcxxabi;libunwind;lld;openmp;parallel-libs;polly;pstl;mlir" \
-DCMAKE_CXX_COMPILER=${myGCC}/bin/g++ \
-DCMAKE_C_COMPILER=${myGCC}/bin/gcc \
-DGCC_INSTALL_PREFIX=${myGCC} \
-DCMAKE_CXX_LINK_FLAGS="-L${myGCC}/lib64 -Wl,-rpath,${myGCC}/lib64" \
-DCMAKE_INSTALL_PREFIX=/home1/p001cao/local/app/compiler/clang-10

cmake --build . -j 8                                           # cannot use make -j 8 with ninja
cmake --build . --target install

-DLIBCXX_ENABLE_SHARED=YES -DLIBCXX_ENABLE_STATIC=NO \

## build clang 11 
(support Fortran compiler: flang)
git clone --branch master https://github.com/llvm/llvm-project.git llvm-11
cd llvm-11

git pull origin master 

mkdir build
cd build

#(use conda, consider to use cmake, Ninja in conda)
# install: LibEdit, libxml2, binutils... into conda/py37clangSupp
module load compiler/gcc-9.2.0
module load conda/conda3
conda create -n py37clangSupp python=3.7
prepend-path PKG_CONFIG_PATH $topdir/lib/pkgconfig

source activate py37clangSupp
conda install cmake make ninja libedit git lua libxml2 binutils

#-- (use Cmake, Ninja from conda)
module load conda/py37clangSupp
module load compiler/gcc-9.2.0
export myGCC=/home1/p001cao/local/app/compiler/gcc-9.2.0

cmake ../llvm -GNinja -DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_OPTIMIZED_TABLEGEN=ON \
-DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;compiler-rt;libc;libclc;libcxx;libcxxabi;libunwind;lld;openmp;parallel-libs;polly;pstl;mlir;flang" \
-DCMAKE_CXX_COMPILER=${myGCC}/bin/g++ \
-DCMAKE_C_COMPILER=${myGCC}/bin/gcc \
-DCMAKE_FC_COMPILER=${myGCC}/bin/gfortran \
-DGCC_INSTALL_PREFIX=${myGCC} \
-DCMAKE_CXX_LINK_FLAGS="-L${myGCC}/lib64 -Wl,-rpath,${myGCC}/lib64" \
-DCMAKE_INSTALL_PREFIX=/home1/p001cao/local/app/compiler/clang-11

cmake --build . -j 8

cmake --build . --target install

II. install OpenMPI-4.0.3 with Clang (USC2)

Using LLD & link-time Optimization

LLD is installed as ld.lld. On Unix, linkers are invoked by compiler drivers, so you are not expected to use that command directly. There are a few ways to tell compiler drivers to use ld.lld instead of the default linker.

The easiest way to do that is to overwrite the default linker. After installing LLD to somewhere on your disk, you can create a symbolic link by doing ln -s /path/to/ld.lld /usr/bin/ld so that /usr/bin/ld is resolved to LLD.

If you don’t want to change the system setting, you can use clang’s -fuse-ld option. In this way, you want to set -fuse-ld=lld to LDFLAGS when building your programs.

LLD leaves its name and version number to a .comment section in an output. If you are in doubt whether you are successfully using LLD or not, run readelf --string-dump .comment <output-file> and examine the output. If the string “Linker: LLD” is included in the output, you are using LLD.

https://lld.llvm.org/
https://llvm.org/docs/LinkTimeOptimization.html

Configure OpenMPI-clang-10 
module load compiler/clang-10

../configure  CC=clang CXX=clang++ LDFLAGS=-fuse-ld=lld \
--with-sge --with-verbs --without-ucx --without-mca \
--prefix=/home1/p001cao/local/app/openmpi/4.0.3-clang10-ib-noUCX

--with-ucx
* without ucx may have better performance
Configure OpenMPI-clang-11
module load compiler/clang-11

../configure  CC=clang CXX=clang++ FC=flang F77=flang LDFLAGS=-fuse-ld=lld \
--with-sge --with-verbs --without-ucx --without-mca \
--prefix=/home1/p001cao/local/app/openmpi/4.0.3-clang11-ib-noUCX

make -j8

make all install


B. USC1:

## build clang

module load cmake-3.15.1
module load conda/py37clangSupp         # include cmake +ninja
module load compiler/gcc-9.2.0
export myGCC=/uhome/p001cao/local/app/compiler/gcc-9.2.0

cmake ../llvm -GNinja -DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_OPTIMIZED_TABLEGEN=ON \
-DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;compiler-rt;libc;libclc;libcxx;libcxxabi;libunwind;lld;openmp;parallel-libs;polly;pstl;mlir" \
-DCMAKE_CXX_COMPILER=${myGCC}/bin/g++ \
-DCMAKE_C_COMPILER=${myGCC}/bin/gcc \
-DGCC_INSTALL_PREFIX=${myGCC} \
-DCMAKE_CXX_LINK_FLAGS="-L${myGCC}/lib64 -Wl,-rpath,${myGCC}/lib64" \
-DCMAKE_INSTALL_PREFIX=/uhome/p001cao/local/app/compiler/clang-10

cmake --build . -j 8                                       
cmake --build . --target install

## OpenMPI-4.0.3 with Clang
module load compiler/clang-10

#-- lion

../configure CC=clang CXX=clang++ LDFLAGS=-fuse-ld=lld \
--with-sge --without-verbs --without-ucx \
--prefix=/uhome/p001cao/local/app/openmpi/4.0.3-clang10-noIB

#-- eagle

../configure CC=clang CXX=clang++ LDFLAGS=-fuse-ld=lld \
--with-sge --with-verbs --without-ucx \
--prefix=/uhome/p001cao/local/app/openmpi/4.0.3-clang10-IB

make -j8

make all install