In this PR, we removed the dependency of UVM when building with CUDA for the SStruct solvers, and added a "memory tracker" to help debugging memory leak or misuse.
This PR adds GPU-support to hypre's examples. A new `Makefile_gpu` is provided in `src/examples` as an sample makefile, so one can compile the examples with make -f Makefile_gpu if hypre has been built with GPUs.
This pull request adds a -save <ext> feature to the runtest.sh script to allow testing against different saved files on different platforms such as GPU machines. See Issue #255. A few additional things were done:
- All of the checks against the saved files were moved out of the individual tests and into runtest.sh.
- The output-file sanity checks that are in many of the tests were modified so they no longer depend on the saved files. Several issues were also uncovered and fixed.
This PR contains the support of UMPIRE pooling allocators for host and GPU memory. Configure hypre with --with-umpire, device and uvm allocations and deallocations are done with umpire, whereas host pool is not enabled by default. This PR also includes some other minor changes:
Adding .gitignore to the repo
Removing all malloc/calloc/realloc/free and regression testing on finding them
No longer compile ij.c with C++ compiler. It goes back to a C code now.
Introducing HYPRE_USING_GPU, which is equivalent to HYPRE_USING_CUDA || HYPRE_USING_DEVICE_OPENMP
Adding a few user-level interfaces: HYPRE_SetMemoryLocation, HYPRE_SetExecutionPolicy, HYPRE_SetGPUMemoryPoolSize and HYPRE_CSRMatrixSetSpGemmUseCusparse
Co-authored-by: li50@llnl.gov <liruipengblue@gmail.com>
Co-authored-by: Rob Falgout <rfalgout@llnl.gov>
Co-authored-by: Ruipeng Li <li50@llnl.gov>
This PR adds HYPRE_WARP_BITSHIFT macro, which will allow us to hide instances of '>> 5' for forthcoming HIP changes that will need a bit shift of 6 rather than 5. This PR was copied from #265 by @pbauman
Co-authored-by: Paul T. Bauman <ptbauman@gmail.com>
* This commit has CUDA based smoothers for AMG based on the triangular parts of sparse matrices. This includes an Gauss-Seidel (relax_type==3), which uses CUSPARSE triangular solvers to invert L. Symmetric Gauss Seidel is implemented in relax_type==6 also via CUSPARSE. Finally, 2 new smoothers are added. THe first is a 2 stage approximation to Gauss Seidel using a parallel MatVec and L (relax_type==11). The second (relax_type==12) is a less effective version of 11. It uses A_diag instead of L for the smoothing. CPU implementations of these new smoothers are also provided. For the two stage algorithms, L and U are NOT explicitly created. This seems faster and saves memory. In the two stage preconditioner, multiply by invdiag rather than divide by diagonal reduces register pressure and yields full occupancy.
Co-authored-by: Paul Mullowney <pmullown@nrel.gov>
Co-authored-by: PaulMullowney <60452402+PaulMullowney@users.noreply.github.com>
* Fix bug in communication for Schur offD element map and update comments on communication pattern options.
* Fix bug to avoid double deletion of solution array.
This merge introduces new features to the parallel ILU solvers in hypre. In particular we have GPU support for BJ-ILU(0) and GMRES-ILU(0). In addition, this merge includes a new option for GMRES-ILU(0) using MILU(0) to build restriction/interpolation operators used to construct the Schur complement matrix by a Galerkin product. This option is also available on the GPU. Key commits include:
* ILU updates with bug fixes for compiling the cuda version
* Update local RCM ordering option to support nonsymmetric matrices
* Update regression tests to test new features
* Reference manual updates, Code cleanup and bug fixes
Co-authored-by: Tianshi Xu <xu16@ray59.coralea.llnl.gov>
Co-authored-by: Tianshi Xu <xu16@lassen708.coral.llnl.gov>
Co-authored-by: Xu <xu16@bellsofireland.llnl.gov>
Co-authored-by: Kote Hitenze <hitenze@jotenshis-MacBook-Pro.local>
Co-authored-by: Tianshi Xu <xuxx1180@umn.edu>
Co-authored-by: Ruipeng Li <li50@llnl.gov>
Modified update-release.sh script to improve portability (for LINUX, UNIX and macOS).
* The use of the 'date' command has been modified to use GNUs 'date' command (if installed).
* In addition, single quotes for sed commands have been replaced by double quotes to allow the use of single quotes
around internal variables. This appears to be more portable than the use of '\x27'. Note that this means shell meta-
characters need to be escaped if they need to be treated as string literals. Other said lines are also modified
accordingly for consistency.
This sets the default convergence tolerance to 1.0e-6 uniformly in hypre. Only three solvers had to be updated (AMG, ILU, and MGR), along with the corresponding documentation.
This PR fixes some memory leaks of BoomerAMGDD. Additional topics covered are:
* Perform code refactoring to meet the style used in hypre.
* Add GetIterations and FinalResidual interfaces for BoomerAMGDD.
* Add new regression tests.
* Make FAC Relaxation dependent on the execution policy.
* added extended+e interpolation, CUDA implementation
* added two regression tests for extended+e interpolation
* renamed some routines to better reflect type of interpolation
Co-authored-by: Ulrike M. Yang <ulrikey@ray53.coralea.llnl.gov>
Co-authored-by: Ruipeng Li <li50@llnl.gov>
This includes the implementation of the AMG-DD algorithm, a variant of BoomerAMG designed to limit communication.
AMG-DD may be used as a standalone solver or a preconditioner for Krylov methods (note that AMG-DD is a non-symmetric preconditioner). For an example of how to set up and use AMG-DD, see the IJ driver (src/test/ij.c).
A list with the parameters of AMG-DD is given below:
Padding (recommended default 1): HYPRE_BoomerAMGDDSetPadding(...)
Number of ghost layers (recommended default 1): HYPRE_BoomerAMGDDSetNumGhostLayers(...)
Number of inner FAC cycles per AMG-DD iteration (default 2): HYPRE_BoomerAMGDDSetFACNumCycles(...)
FAC cycle type: HYPRE_BoomerAMGDDSetFACCycleType(...)
1 = V-cycle (default)
2 = W-cycle
3 = F-cycle
Number of relaxations on each level during FAC cycle: HYPRE_BoomerAMGDDSetFACNumRelax(...)
Type of local relaxation during FAC cycle: HYPRE_BoomerAMGDDSetFACRelaxType(...)
0 = Jacobi
1 = Gauss-Seidel
2 = ordered Gauss-Seidel
3 = C/F L1-scaled Jacobi (default)
For more details of the algorithm, see Mitchell W.B., R. Strzodka, and R.D. Falgout (2020), Parallel Performance of Algebraic Multigrid Domain Decomposition (AMG-DD).
