Docker

使用Docker安装和运行WRF


安装步骤:官网|本地

进入容器:

检查三个编译器:gfortran、gcc、cpp

1、查看位置:使用命令which查看编译器位置:

可见gfortran编译在容器里面是没有的!

2、查看版本:

3、安装gfortran

使用yum install -y gfortran安装报错:

改用:yum install -y gcc-gfortran

安装成功后检查版本:

安装官网的测试程序测试环境:

还要测试:

删除./TEST_csh.csh前面的注释后可以:



设置环境变量


1、编辑环境变量命令vi ~/.bashrc

然后按i插入以下环境变量设置:

export DIR="/home/wwwroot/byp/Build_WRF/LIBRARIES"
export CC=gcc
export CXX=g++
export FC=gfortran
export FCFLAGS=-m64
export F77=gfortran
export FFLAGS=-m64
export JASPERLIB=$DIR/grib2/lib
export JASPERINC=$DIR/grib2/include
export LDFLAGS=-L$DIR/grib2/lib
export CPPFLAGS=-I$DIR/grib2/include

export PATH=$DIR/netcdf/bin:$PATH
export NETCDF=$DIR/netcdf
export PATH=$DIR/mpich/bin:$PATH

按Esc;按冒号:;wq,保存推出;

2、激活设置source ~/.bashrc



下载:

wget -c -b https://github.com/wrf-model/WRF/releases/download/v4.4.2/v4.4.2.tar.gz
wget -c -b https://github.com/wrf-model/WPS/archive/refs/tags/v4.4.tar.gz

需要注册一下,然后跳转到以下下载WRF页面(本地

git安装:

不过未见编译就退出了


执行编译: ./compile em_real>&log.compile 使用vi log.compile查看结果:

make[3]: Leaving directory `/home/wwwroot/byp/WRF/WRF/phys'
ar: module_cam_support.o: No such file or directory
make[2]: [physics] Error 1 (ignored)
make[2]: Leaving directory `/home/wwwroot/byp/WRF/WRF/phys'
/bin/sh: line 2: [: -eq: unary operator expected
make[1]: Leaving directory `/home/wwwroot/byp/WRF/WRF'

------------------------------------------------------------------------------
Error Error Error NoahMP submodule files not populating WRF directories
------------------------------------------------------------------------------

make: *** [wrf] Error 31



Docker安装

1、重新创建一个容器

原来看到images里面有centos:latest的,但是使用命令:
docker run -itd --name centos_byp -e TZ=Asia/Shanghai --restart=always --privileged=true -v /share/users_root/hqx_bai/byp/web/systems/Docker:/home/byp/docker centos:latest
发现提示没有latest,需要重新拉镜像:

2、下载安装包

然后进入WRF官网下载WRF和WPS(4.0):

wget -c -b https://www2.mmm.ucar.edu/wrf/src/WRFV4.0.TAR.gz
wget -c -b https://www2.mmm.ucar.edu/wrf/src/WPSV4.0.TAR.gz

3、下载编译器参考):

wget -c -b https://registrationcenter-download.intel.com/akdlm/irc_nas/19079/l_BaseKit_p_2023.0.0.25537.sh
wget -c -b https://registrationcenter-download.intel.com/akdlm/irc_nas/19084/l_HPCKit_p_2023.0.0.25400.sh

首先确保有没有安装好编译语言gfortrangccg++以及cpp

使用命令测试是否存在:

which gfortran
which cpp
which gcc
下图可见,在宿主机上gfortran是存在的。但是进入容器内,就未发现:

由于centos8已经停止服务,因此会报错(参考01):

需要修改软件源mirrors 地址,具体操作如下:
cd /etc/yum.repos.d/
sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*

使用命令:yum install -y gcc-gfortran安装。然后用以下命令查询版本:

gcc --version
gfortran --version
cpp --version
发现都安装了:

4、测试环境:

按照官网介绍,下载测试程序:

wget -c -b https://www2.mmm.ucar.edu/wrf/OnLineTutorial/compile_tutorial/tar_files/Fortran_C_tests.tar

在新建立的test文件夹内解压测试程序压缩包:

tar -xvf Fortran_C_tests.tar

测试程序:

gfortran TEST_1_fortran_only_fixed.f
./a.out
gfortran TEST_2_fortran_only_free.f90
./a.out
gcc TEST_3_c_only.c
./a.out
gcc -c -m64 TEST_4_fortran+c_c.c
gfortran -c -m64 TEST_4_fortran+c_f.f90
gfortran -m64 TEST_4_fortran+c_f.o TEST_4_fortran+c_c.o
./a.out

然后以下三个程序测试,有两个不行:

./TEST_csh.csh
./TEST_perl.pl
./TEST_sh.sh

./TEST_csh.csh内容:
#!/bin/csh
echo "SUCCESS csh test"


./TEST_perl.pl内容:
#!/usr/bin/perl
print "SUCCESS perl test ";

./TEST_sh.sh内容:
#!/bin/sh
echo "SUCCESS sh test"

由于csh和perl测试失败(见上图),因此安装:

yum install -y csh
yum install -y perl
注意:perl语言的安装的时间需要比较长!

5、安装依赖库:

mpich-3.0.4
netcdf-4.1.3
Jasper-1.900.1
libpng-1.2.50
zlib-1.2.7
Libraries使用命令:

wget -c -b https://www2.mmm.ucar.edu/wrf/OnLineTutorial/compile_tutorial/tar_files/mpich-3.0.4.tar.gz
wget -c -b https://www2.mmm.ucar.edu/wrf/OnLineTutorial/compile_tutorial/tar_files/netcdf-4.1.3.tar.gz
wget -c -b https://www2.mmm.ucar.edu/wrf/OnLineTutorial/compile_tutorial/tar_files/jasper-1.900.1.tar.gz
wget -c -b https://www2.mmm.ucar.edu/wrf/OnLineTutorial/compile_tutorial/tar_files/libpng-1.2.50.tar.gz
wget -c -b https://www2.mmm.ucar.edu/wrf/OnLineTutorial/compile_tutorial/tar_files/zlib-1.2.7.tar.gz

Libraries拷贝到WRF的文件夹内:

cp -r Libraries/ ../WRF_WorkStation/

先设置一下环境变量:

vi ~/.bashrc
插入一下环境设置:
export DIR=/home/byp/WRF_WorkStation/Libraries
export CC=gcc
export CXX=g++
export FC=gfortran
export FCFLAGS=-m64
export F77=gfortran
export FFLAGS=-m64
export JASPERLIB=$DIR/grib2/lib
export JASPERINC=$DIR/grib2/include
export LDFLAGS=-L$DIR/grib2/lib
export CPPFLAGS=-I$DIR/grib2/include

source ~/.bashrc

可以通过echo $DIR来查看你设置的安装路径:

安装:

(1)netcdf

tar xzvf netcdf-4.1.3.tar.gz
cd netcdf-4.1.3
./configure --prefix=$DIR/netcdf --disable-dap  --disable-netcdf-4 --disable-shared
make
make install
似乎有报错:

make: *** [Makefile:551: install-recursive] Error 1

然后配置环境变量:

vi ~/.bashrc

###--->netcdf<---###
export NETCDF=$DIR/netcdf
export PATH=$NETCDF/bin:$PATH

source ~/.bashrc

提示错误:

应该是没有安装g++,不过直接安装g++报错:

yum -y install gcc+ gcc-c++

应该是:

yum install gcc-c++ libstdc++-devel

使用命令查看版本:

g++ -v

使用命令可以查看到g++:

ls /usr/bin/

然后再次:

./configure --prefix=$DIR/netcdf --disable-dap  --disable-netcdf-4 --disable-shared
make
make install

终于成功:

然后设置环境变量:

vi ~/.bashrc

###--->netcdf<---###
export NETCDF=$DIR/netcdf
export PATH=$NETCDF/bin:$PATH

source ~/.bashrc
输入命令:

nc-config

(2)mpich

tar xzvf mpich-3.0.4.tar.gz     #or just .tar if no .gz present
cd mpich-3.0.4
./configure --prefix=$DIR/mpich
make
make install
make check #测试安装(结果见下图)

环境变量配置:

vi ~/.bashrc

###--->mpich<---###
export MPICH=$DIR/mpich
export PATH=$MPICH/bin:$PATH

source ~/.bashrc

tar xzvf zlib-1.2.7.tar.gz     #or just .tar if no .gz present
cd zlib-1.2.7
./configure --prefix=$DIR/grib2
make
make install


tar xzvf libpng-1.2.50.tar.gz     #or just .tar if no .gz present
cd libpng-1.2.50
./configure --prefix=$DIR/grib2
make
make install


tar xzvf jasper-1.900.1.tar.gz     #or just .tar if no .gz present
cd jasper-1.900.1
./configure --prefix=$DIR/grib2
make
make install
然后配置环境变量:

vi ~/.bashrc

###--->zlib<---###
export ZLIB=$DIR/grib2
export PATH=$ZLIB/bin:$PATH

###--->libpng<---###
export LIBPNG=$DIR/grib2
export PATH=$LIBPNG/bin:$PATH

###--->jasper<---###
export JASPER=$DIR/grib2
export PATH=$JASPER/bin:$PATH

###--->WPS Compiler<---###
export WRFIO_NCD_LARGE_FILE_SUPPORT=1
export LD_LIBRARY_PATH=$DIR/grib2/lib:$LD_LIBRARY_PATH
source ~/.bashrc

总的环境配置:见文件

库的兼容性测试

进入test文件夹:

wget https://www2.mmm.ucar.edu/wrf/OnLineTutorial/compile_tutorial/tar_files/Fortran_C_NETCDF_MPI_tests.tar

首先下载Fortran_C_NETCDF_MPI_tests.tar并解压,然后开始进行两个测试,一个是netcdf的测试,另一个是mpich的测试。

这里注意把设置的NETCDF路径下的netcdf.inc复制到test文件夹下。

cp ${NETCDF}/include/netcdf.inc .

然后测试:

tar -xf Fortran_C_NETCDF_MPI_tests.tar

gfortran -c 01_fortran+c+netcdf_f.f
gcc -c 01_fortran+c+netcdf_c.c
gfortran 01_fortran+c+netcdf_f.o 01_fortran+c+netcdf_c.o
-L${NETCDF}/lib -lnetcdff -lnetcdf
./a.out

mpif90 -c 02_fortran+c+netcdf+mpi_f.f
mpicc -c 02_fortran+c+netcdf+mpi_c.c
mpif90 02_fortran+c+netcdf+mpi_f.o
02_fortran+c+netcdf+mpi_c.o
-L${NETCDF}/lib -lnetcdff -lnetcdf
mpirun ./a.out

将WRF和WPS安装包拷贝到同一个目录下,然后解压:

cp ../docker/WRFV4.0.TAR.gz ./
cp ../docker/WPSV4.0.TAR.gz ./
tar -xf WRFV4.0.TAR.gz
tar -xf WPSV4.0.TAR.gz
然后先进入WRF文件夹:

./configure
15
1
./compile em_real >& log.compile
ls -ls main/*.exe
但是出现报错:

根据提示,其实perl是已经安装了的:

所以应该只是需要安装m4:

yum install m4

然后再执行:

./compile em_real >& log.compile

但是一下之就跳出来了,说明有问题,于是查看:

vi log.compile

还是ifort的问题:

然后将ifort安装文件拷贝过来:

cp ../docker/l_HPCKit_p_2023.0.0.25400.sh ./
cp ../docker/l_BaseKit_p_2023.0.0.25537.sh ./

sh ./l_BaseKit_p_2023.0.0.25537.sh
sh ./l_HPCKit_p_2023.0.0.25400.sh

开始需要的时间就比较长:

然后开始安装BaseKit:

然后开始安装HPCKit:

注意一定要设置环境变量:

find / -name setvars.sh

环境变量:

vi ~/.bashrc

# intel compiller
source /opt/intel/oneapi/setvars.sh --force

source ~/.bashrc



依然出错:

which SFC
/bin/sh: which: command not found

安装which:

yum install -y which

但是依然经过2分钟又跳出来了:

报错:

文章说:之前运行过WRF了,需要 ./clean -a 指令清除之前的过程, 然后重新编译./configure ./compile,结果就出来了

不过,还是不成功:

查看日志,发现time没有安装,于是
yum install -y time

根据文章介绍:

注意:如果重新编译需要先执行./clean -aa,不要使用./clean -a,./configure之后都需要重新修改configure.wrf文件。


继续分析log文件:

icc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.
landread.c(68): catastrophic error: cannot open source file "rpc/types.h"
#include
^
根据文章介绍:

yum install -y libtirpc-dev
ln -s /usr/include/tirpc/rpc/types.h /usr/include/rpc


因为安装libtirpc-dev失败,因此升级yum:

yum update -y

其实是漏了一个字母:l !!!

yum install libstdc++-devel libtirpc-devel

问题分析:
rpc/types.h是libtirpc-devel软件包里的文件,在fedora 7版本以后就从glibc单独拿出来了。升级安装libtirpc软件包。并添加头文件,链接到库。
yum install -y libtirpc-dev
ln -s /usr/include/tirpc/rpc/types.h /usr/include/rpc

依然编译失败。

报错:

landread.c(69): catastrophic error: cannot open source file "rpc/xdr.h"
#include

根据文章介绍rpc问题:


cannot open source file "netconfig.h"
ln -s /usr/include/tirpc/rpc/netconfig.h /usr/include/rpc

cd /usr/include/tirpc/rpc/


ln -s /usr/include/tirpc/netconfig.h /usr/include

进入rpc文件夹:

cd /usr/include/tirpc/rpc/

ln -s /usr/include/tirpc/rpc/xdr.h /usr/include/rpc


然后返回:

cd /home/byp/WRF_WorkStation/WRF

现在不出现rpc问题了,也就是需要连接:

ln -s /usr/include/tirpc/rpc/types.h /usr/include/rpc
ln -s /usr/include/tirpc/netconfig.h /usr/include
ln -s /usr/include/tirpc/rpc/xdr.h /usr/include/rpc

解决rpc问题了,又出现:

module_physics_init.f90(4481): error #7013: This module file was not generated by any release of this compiler.   [NETCDF]
use netcdf
------^
根据文章介绍,需要修改configure.wrf:

DM_FC           =       mpiifort
DM_CC           =       mpiicc
CC_TOOLS        =      icc
#注意:CC_TOOLS需要按照上面的方式修改。否则源码中的一个standard.c无法正常编译

./compile -j 4 em_real 2>&1 | tee compile.log
./compile em_real>&log.compile

在第9次编译后,依然出现问题:

registry.c(63): warning #266: function "sym_forget" declared implicitly
sym_forget() ;
^

问了专家:

安装wrf用的intel编译器,netcdf安装时不是用intel的,,要用同一套编译器安装,才可以

网上说:

新版的netcdf吧c和fortran代码拆分出来了,但是WRF模式编译依赖于netcdf-fortran,而netcdf-fortran安装依赖于netcdf-c,因此这两个库都要装


make clean
./configure --prefix=$DIR/netcdf --disable-dap  --disable-netcdf-4 --enable-shared
make
make install





WRF Docker Container Tutorial Information
容器操作实践
WRF安装
WRF模式安装+详细运行教程
WRF模式安装记录
地球科学社|WRF数值模型快速安装





BypInformation