Compiling and Running TOUGHREACT v3.32 using Ubuntu/Linux
I've been able to get the newest version of TOUGHREACT working with my workstation, which has Linux/Ubuntu (v 17.04) as its OS. Setting things up didn't go without hitches, though, so I thought I'd point out a few things, in case anyone else is trying to do the same. There was also one issue I've yet to resolve, and I'd love to hear feedback about it, if possible. It's written at the bottom of the post.
I compiled all executables using gfortran with up to 1e6 elements and 3e6 connections. In order to do so, I used the following compiler/linker options:
"FFLAGS = -O2 -mcmodel=large -fstack-arrays -fdefault-integer-8 -fdefault-real-8 -fdefault-double-8 -malign-double -fopenmp"
You may notice there's no mention of declaring the stack size here. For Ubuntu, you declare this separately in the terminal using the command: "ulimit -s unlimited" or "ulimit -s <hard_stack_limit>", where <hard_stack_limit> is the size (in MB) you wish to dedicate to your stack. Ultimately, you're limited on how large of a problem you can simulate by the amount of memory you have.
I also had some issues running the executables that were provided in the installation, which were created using the ifort (Intel Fotran) compiler. When I attempted to run it at first, it shot back an error stating that it couldn't find the library file "libifport.so.5". After digging into the documentation provided with TOUGHREACT, I came across the following link, listed at the bottom of the file "TOUGHREACT_V332_InstallationUsage.pdf":
From this link, you choose the version of ifort whose libraries you'd like to download. I ended up choosing "Parallel Studio XE 2017 (All Editions)" in the Linux column and choosing Update 4 under the column titled "Intel Fotran Compiler for Linux."
This link provides a .tgz file, which can be unpacked and installed using the directions provided by Intel. From there, I only needed to perform one more step before I could use some of the provided executables. This was to tell Ubuntu where to find the library files when running the executable. To do this, you need to find where the appropriate library files are located (call it <ifort_library_filepath>). Once you do, run the following command in the terminal:
Once this was done, I was able to run through all of the example problems using the executables with the suffix "_linux", which where created for smaller problems. I wasn't able to run all of the example problems with the executables with suffix "_linux_1m", which were created for problems up to 999999 elements and 3e6 connections. In particular, when I ran problem P3, the file runlog.out issued a bunch of warnings that looked like this:
"Warning: chemistry did not converge at node Fza18 (routine NEWTONEQ), Non-convergence type: relative Node temperature (C): 27.827 Liq.sat.: 1.9614E-01
Species: cl- Relative error = 1.130+172 Tolerance= 1.000E-06 Program execution was not aborted. Check results!"
So it appears there are some numerical errors for this case. Any help on figuring out the root of this problem would be much appreciated.
Mikey, thanks for giving detailed info on Linux installation from the user's perspective! That information is in the new User's Manual I am working on now, but some was not in the InstallationUsage document that came with the distribution. I'll update that document, too, since it is easier to find than reading through a long User's Manual.
Regarding the lack of convergence, that happens at specific grid blocks in some problems. Usually, the transport at the next time step changes the water composition enough to allow convergence. These convergence problems can often be avoided with very small time steps, however, then the problems take much longer to run, and usually the end results are nearly the same. There is always a trade-off between getting things to run relatively fast and maintaining accuracy vs. trying to get the most accurate solution by taking very small time steps. Setting the Courant number (RCOUR in solute.inp) to ~0.5 is the safest way to maintain accuracy for advection-dominated problems. However, if many elements undergo non-convergence, and the time step is decreasing as a result, then likely the maximum time step is too large. If the time step is too large, then errors in mineral abundances and porosity/permeability changes can be in error because they are cumulative.
There are no options in TOUGHREACT to limit the time step for chemical diffusion. One must find a time step large enough that the results stay about the same as a smaller time step. The timestep cannot be too small or else the chemical changes may be too small and numerical cutoffs can take place if the mineral changes are extremely small. I always suggest one runs multiple simulations with differing time stepping to make sure the results are reasonable. TOUGH2 will allow the time step to keep increasing, which may be reasonable for flow simulations, but completely erroneous reactive transport results or nonconvergence will occur. So limit the maximum time step in flow.inp and invoke the Courant condition for all simulations. Then check the results by plotting the chemical profiles and mineral abundance profiles.