From ea63fdd02f7da49fbd7235d381883e0a4f5b3f93 Mon Sep 17 00:00:00 2001 From: David A Minton Date: Fri, 28 Jul 2023 11:23:09 -0400 Subject: [PATCH 1/5] Updated README.md with new directions on compiling --- README.md | 60 +++++++++++++++++++++++++++++++------------------------ 1 file changed, 34 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index f45a812c3..4ffe2667c 100644 --- a/README.md +++ b/README.md @@ -63,55 +63,63 @@ $ git pull You now have a Swiftest repository on your personal machine that you may compile, edit, and run as you see fit. -**Compiling Swiftest** +**Compiling the Swiftest driver program** -Swiftest is written in modern Fortran and must be compiled before it can be run. After compilation, an executable, called the Swiftest driver, will have been created in the ```/swiftest/bin/``` directory. - -Swiftest is compiled through [CMake](https://cmake.org/). Compiling with CMake has a number of benefits that provide a streamlined experience for the Swiftest user and developer. At compilation, CMake will automatically select the set of flags that are compatible with the local compiler. CMake also allows a Swiftest developer to re-compile only the files that have been edited, instead of requiring the developer to re-compile the entire Swiftest program. Please visit the CMake website for more information on how to install CMake. - -Once CMake is installed, navigate to the topmost directory in your Swiftest repository. It is best practice to create a ```build``` directory in your topmost directory from which you will compile Swiftest. This way, temporary CMake files will not clutter up the ```swiftest/src/``` sub-directories. To create a new directory and then navigate into that directory, type the following: +***Compiling `swiftest_driver` using Docker*** +By far the simplest, most reliable way of compiling the driver program is via a Docker container. The Swiftest project contains a Dockerfile that may be used to generate an executable without needing to provide any external dependencies, other than the Docker engine itself (see [here](https://docs.docker.com/get-docker/) for instructions on obtaining Docker). Once Docker is installed and the Docker engine is running, execute: ``` -$ mkdir build -$ cd build +$ docker build --target=export_driver --output=bin --build-arg MACHINE_CODE_VALUE="Host" [--build-arg EXTRA_CMAKE_OPTIONS=["-D"] ``` +***Compiling `swiftest_driver` using CMake*** + +The Swiftest driver program is written in modern Fortran and must be compiled before it can be run. After compilation, an executable, called the `swiftest_driver``, will have been created in the ```bin/``` directory. + +Swiftest is compiled through [CMake](https://cmake.org/). Compiling with CMake has a number of benefits that provide a streamlined experience for the Swiftest user and developer. At compilation, CMake will automatically select the set of flags that are compatible with the local compiler. CMake also allows a Swiftest developer to re-compile only the files that have been edited, instead of requiring the developer to re-compile the entire Swiftest program. Please visit the CMake website for more information on how to install CMake. + As mentioned in the **System Requirements** section, Swiftest requires the NetCDF and NetCDF Fortran libraries to be installed prior to compilation. If the libraries are installed in the standard library location on your machine, CMake should be able to find the libraries without specifying the path. However, if CMake struggles to find the NetCDF libraries, there are two ways to set the path to these libraries. 1. Create an environment variable called ```NETCDF_FORTRAN_HOME``` that contains the path to the location where the libraries are installed -2. Set the path at the time of compilation using ```-CMAKE_PREFIX_PATH=/path/to/netcdf/``` +2. Set the path at the build step using ```-CMAKE_PREFIX_PATH=/path/to/netcdf/``` CMake allows the user to specify a set of compiler flags to use during compilation. We define five sets of compiler flags: release, testing, profile, math, and debug. To view and/or edit the flags included in each set, see ```swiftest/cmake/Modules/SetFortranFlags.cmake```. As a general rule, the release flags are fully optimized and best used when running Swiftest with the goal of generating results. This is the default set of flags. When making changes to the Swiftest source code, it is best to compile Swiftest using the debug set of flags. You may also define your own set of compiler flags. -To build Swiftest with the release flags (default) using the Intel fortran compiler (ifort), type the following: +Navigate to the topmost directory in your Swiftest repository. It is best practice to create a ```build``` directory in your topmost directory from which you will compile Swiftest. This way, temporary CMake files will not clutter up the ```swiftest/src/``` sub-directories. The commands to build the source code into a ```build``` directory and compile Swiftest are: + ``` -$ cmake .. +$ cmake -B build -S . +$ cmake --build build ``` -To build with the debug flags, type: +The [CMake Fortran template](https://github.com/SethMMorton/cmake_fortran_template) comes with a script that can be used to clean out any build artifacts and start from scratch: + ``` -$ cmake .. -DCMAKE_BUILD_TYPE=DEBUG +$ cmake -P distclean.cmake ``` -To build with another set of flags, simply replace ```DEBUG``` in the above line with the name of the flags you wish to use. -Add ```-CMAKE_PREFIX_PATH=/path/to/netcdf/``` to these commands as needed. +The Swiftest CMake configuration comes with several customization options: + +| Option | CMake command | +| --------------------------------|------------------------------------------------------------| +| Build type | \-DCMAKE_BUILD_TYPE=[**RELEASE**\|DEBUG\|TESTING\|PROFILE] | +| Enable/Disable OpenMP support | \-DUSE_OPENMP=[**ON**\|OFF] | +| Enable/Disable SIMD directives | \-DUSE_SIMD=[**ON**\|OFF] | +| Enable/Disable Coarray support (experimental) | \-DUSE_COARRAY=[ON\|**OFF**] | +| Set Fortran compiler path | \-DCMAKE_Fortran_COMPILER=/path/to/fortran/compiler | +| Set path to make program | \-DCMAKE_MAKE_PROGRAM=/path/to/make | +| Enable/Disable shared libraries (Intel only) | \-DBUILD_SHARED_LIBS=[**ON\|OFF] | -If using the GCC fortran compiler (gfortran), add the following flags: -``` --DCMAKE_Fortran_FLAGS="-I/usr/lib64/gfortran/modules/ -ffree-line-length-512" -``` -You can manually specify the compiler you wish to use with the following flag: -``` -c-DCMAKE_Fortran_COMPILER=$(which ifort) -``` -After building Swiftest, make the executable using: +To see a list of all possible options available to CMake: ``` -$ make +$ cmake -B build -S . -LA ``` -The Swiftest executable, called ```swiftest_driver```, should now be created in the ```/swiftest/bin/``` directory. +The Swiftest executable, called ```swiftest_driver```, should now be created in the ```bin/``` directory. + + **Download the `swiftest_driver` as a Docker or Singularity container.** From 2975351a3f5aeb004428aaa112e422432c15aede Mon Sep 17 00:00:00 2001 From: MintoDA1 <51412913+MintoDA1@users.noreply.github.com> Date: Fri, 28 Jul 2023 11:41:58 -0400 Subject: [PATCH 2/5] Finished changes to the README to improve the compiling instructions --- README.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 4ffe2667c..246ff5b9c 100644 --- a/README.md +++ b/README.md @@ -69,9 +69,20 @@ You now have a Swiftest repository on your personal machine that you may compile By far the simplest, most reliable way of compiling the driver program is via a Docker container. The Swiftest project contains a Dockerfile that may be used to generate an executable without needing to provide any external dependencies, other than the Docker engine itself (see [here](https://docs.docker.com/get-docker/) for instructions on obtaining Docker). Once Docker is installed and the Docker engine is running, execute: ``` -$ docker build --target=export_driver --output=bin --build-arg MACHINE_CODE_VALUE="Host" [--build-arg EXTRA_CMAKE_OPTIONS=["-D"] +$ docker build --target=export_driver \ + --output=bin \ + --build-arg MACHINE_CODE_VALUE="Host" \ + [ --build-arg BUILD_TYPE="*RELEASE*|DEBUG|TESTING|PROFILE" ] \ + [ --build-arg EXTRA_CMAKE_OPTIONS="-D" ] ``` +The Docker build will download and compile all of the library dependencies (HDF5, NetCDF-C, and NetCDF-Fortran) as static libraries and the Swiftest driver using Intel compilers. Once completed, the Swiftest executable, called ```swiftest_driver```, should now be created in the ```bin/``` directory. + +> Note: The Dockerfile is designed to build an executable that is compatible with a broad range of CPU architectures by specifying the SSE2 instruction as a target for SIMD instructions using the `-x` compiler option. When compiling on the same CPU archictecture you plan to execute the driver program, for the highest possible SIMD performance, use `--build-arg MACHINE_CODE_VALUE="Host" to override the default `MACHINE_CODE_VALUE="SSE2"`. For additional options see [here](https://www.intel.com/content/www/us/en/docs/fortran-compiler/developer-guide-reference/2023-1/x-qx.html). + +The optional Docker argument `EXTRA_CMAKE_OPTIONS` is provided to pass any additional CMake arguments (see below). + + ***Compiling `swiftest_driver` using CMake*** The Swiftest driver program is written in modern Fortran and must be compiled before it can be run. After compilation, an executable, called the `swiftest_driver``, will have been created in the ```bin/``` directory. @@ -120,7 +131,6 @@ $ cmake -B build -S . -LA The Swiftest executable, called ```swiftest_driver```, should now be created in the ```bin/``` directory. - **Download the `swiftest_driver` as a Docker or Singularity container.** The Swiftest driver is available as a Docker container on DockerHub in two versions: Intel and GNU. The Intel version was compiled for the x86_64 CPU using the Intel classic Fortran compiler. The GNU version was compliled for the x86_64 CPU using gfortran. The Intel version is faster than the GNU version (though not as fast as a native compile to the target CPU that you wish to run it on due to vectorization optimizations that Swiftest takes advantage of), however it is much larger: The Intel version is ~2.7GB while the GNU version is ~300MB. The Singularity container pulls from the same DockerHub container. From 1d0fa8d0bd6fbf05e2461cf563a84a8037f74ee3 Mon Sep 17 00:00:00 2001 From: David Minton Date: Fri, 28 Jul 2023 11:53:19 -0400 Subject: [PATCH 3/5] Updated the line length flag for gfortran --- cmake/Modules/SetFortranFlags.cmake | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmake/Modules/SetFortranFlags.cmake b/cmake/Modules/SetFortranFlags.cmake index 4c8cc9b85..b95be5575 100644 --- a/cmake/Modules/SetFortranFlags.cmake +++ b/cmake/Modules/SetFortranFlags.cmake @@ -93,7 +93,7 @@ SET_COMPILE_FLAG(CMAKE_Fortran_FLAGS "${CMAKE_Fortran_FLAGS}" # Allows for lines longer than 80 characters without truncation SET_COMPILE_FLAG(CMAKE_Fortran_FLAGS "${CMAKE_Fortran_FLAGS}" - Fortran "-ffree-line-length-none" # GNU (gfortran) + Fortran "-ffree-line-length-512" # GNU (gfortran) ) # Disables right margin wrapping in list-directed output From fa594430f7a9fafae0da9952ca70cce0dd9008e7 Mon Sep 17 00:00:00 2001 From: MintoDA1 <51412913+MintoDA1@users.noreply.github.com> Date: Fri, 28 Jul 2023 12:08:28 -0400 Subject: [PATCH 4/5] Added additional row in the CMake options table to show how to add aditional flags, such as include paths --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 246ff5b9c..1f8c2e75f 100644 --- a/README.md +++ b/README.md @@ -120,7 +120,8 @@ The Swiftest CMake configuration comes with several customization options: | Enable/Disable Coarray support (experimental) | \-DUSE_COARRAY=[ON\|**OFF**] | | Set Fortran compiler path | \-DCMAKE_Fortran_COMPILER=/path/to/fortran/compiler | | Set path to make program | \-DCMAKE_MAKE_PROGRAM=/path/to/make | -| Enable/Disable shared libraries (Intel only) | \-DBUILD_SHARED_LIBS=[**ON\|OFF] | +| Enable/Disable shared libraries (Intel only) | \-DBUILD_SHARED_LIBS=[**ON\|OFF] | +| Add additional include path | \-DCMAKE_Fortran_FLAGS="-I/path/to/libries | To see a list of all possible options available to CMake: From 5970e5e6db6f6dc5cc199f2d84b3632dc4050b5d Mon Sep 17 00:00:00 2001 From: MintoDA1 <51412913+MintoDA1@users.noreply.github.com> Date: Fri, 28 Jul 2023 12:24:55 -0400 Subject: [PATCH 5/5] Added sanity check to ensure tstep_out <= tstop --- python/swiftest/swiftest/simulation_class.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/python/swiftest/swiftest/simulation_class.py b/python/swiftest/swiftest/simulation_class.py index 139bec27d..809ca5615 100644 --- a/python/swiftest/swiftest/simulation_class.py +++ b/python/swiftest/swiftest/simulation_class.py @@ -113,7 +113,7 @@ def __init__(self,read_param: bool = False, Parameter input file equivalent: `ISTEP_OUT` tstep_out : float, optional The approximate time between when outputs are written to file. Passing this computes - `istep_out = floor(tstep_out/dt)`. *Note*: only `istep_out` or `tstep_out` can be set. + `istep_out = floor(tstep_out/dt)`. *Note*: only `istep_out` or `tstep_out` can be set. `tstep_out` must be less than `tstop` Parameter input file equivalent: None nstep_out : int, optional The total number of times that outputs are written to file. Passing this allows for a geometric progression of output steps: @@ -676,6 +676,10 @@ def set_simulation_time(self, return {} else: update_list.append("istep_out") + + if tstep_out is not None and tstep_out > tstop: + warnings.warn("tstep_out must be less than tstop. Setting tstep_out=tstop",stacklevel=2) + tstep_out = tstop if tstep_out is not None and dt is not None: istep_out = int(tstep_out / dt) @@ -683,6 +687,7 @@ def set_simulation_time(self, if istep_out is not None: self.param['ISTEP_OUT'] = int(istep_out) + if nstep_out is not None: if istep_out is None: warnings.warn("nstep_out requires either istep_out or tstep_out to also be set", stacklevel=2)