> > > > 12:46 master= ~/petsc/src/snes/tutorials$ make > PETSC_DIR=/ccs/home/adams/petsc PETSC_ARCH=arch-summit-opt-gnu-cuda-omp -f > mymake run NP=4 EXTRA='-dm_refine 2 -dm_view' > jsrun -n 1 -a 4 -c 4 -g 1 -r 1 --smpiargs "-gpu" ./ex13 -dm_plex_box_dim 3 > -dm_plex_box_simplex 0 -potential_petscspace_degree 1 -dm_refine 1 > -ksp_type cg -ksp_rtol 1.e-11 -ksp_norm_type unpreconditioned -pc_type gamg > -pc_gamg_type agg -pc_gamg_agg_nsmooths 1 -pc_gamg_coarse_eq_limit 1000 > -pc_gamg_square_graph 1 -pc_gamg_threshold 0.05 -pc_gamg_threshold_scale .0 > -mg_levels_ksp_type chebyshev -mg_levels_ksp_max_it 1 > -mg_levels_esteig_ksp_type cg -mg_levels_esteig_ksp_max_it 10 > -mg_levels_ksp_chebyshev_esteig 0,0.05,0,1.05 -mg_levels_pc_type jacobi > -ksp_monitor -dm_refine 2 -dm_view -vec_type cuda -mat_type aijcusparse > DM Object: Mesh 4 MPI processes > type: plex > Mesh in 3 dimensions: > > > > * 0-cells: 125 0 0 0 1-cells: 300 0 0 0 2-cells: 240 0 0 0 3-cells: 64 > 0 0 0* > Labels: > celltype: 4 strata with value/size (0 (125), 1 (300), 4 (240), 7 (64)) > depth: 4 strata with value/size (0 (125), 1 (300), 2 (240), 3 (64)) > marker: 1 strata with value/size (1 (378)) > Face Sets: 6 strata with value/size (6 (49), 5 (49), 3 (49), 4 (49), 1 > (49), 2 (49) >
This is not getting distributed. 14:24 master= ~/petsc/src/snes/tutorials$ make PETSC_DIR=/ccs/home/adams/petsc PETSC_ARCH=arch-summit-opt-gnu-cuda-omp -f mymake run NP=4 EXTRA='-dm_plex_box_faces 4,4,4 -dm_distribute -dm_view' jsrun -n 1 -a 4 -c 4 -g 1 -r 1 --smpiargs "-gpu" ./ex13 -dm_plex_box_dim 3 -dm_plex_box_simplex 0 -potential_petscspace_degree 1 -dm_refine 1 -ksp_type cg -ksp_rtol 1.e-11 -ksp_norm_type unpreconditioned -pc_type gamg -pc_gamg_type agg -pc_gamg_agg_nsmooths 1 -pc_gamg_coarse_eq_limit 1000 -pc_gamg_square_graph 1 -pc_gamg_threshold 0.05 -pc_gamg_threshold_scale .0 -mg_levels_ksp_type chebyshev -mg_levels_ksp_max_it 1 -mg_levels_esteig_ksp_type cg -mg_levels_esteig_ksp_max_it 10 -mg_levels_ksp_chebyshev_esteig 0,0.05,0,1.05 -mg_levels_pc_type jacobi -ksp_monitor *-dm_plex_box_faces 4,4,4 -dm_distribute *-dm_view -vec_type cuda -mat_type aijcusparse DM Object: Mesh 4 MPI processes type: plex Mesh in 3 dimensions: * 0-cells: 225 225 225 225 1-cells: 560 560 560 560 2-cells: 464 464 464 464 3-cells: 128 128 128 128* Labels: celltype: 4 strata with value/size (0 (225), 1 (560), 4 (464), 7 (128)) depth: 4 strata with value/size (0 (225), 1 (560), 2 (464), 3 (128)) marker: 1 strata with value/size (1 (384)) Face Sets: 4 strata with value/size (1 (72), 3 (72), 5 (36), 6 (36)) 0 KSP Residual norm 3.405958130078e+00 1 KSP Residual norm 3.963373201766e-01 2 KSP Residual norm 1.282320967427e-01 3 KSP Residual norm 2.824310691528e-02 4 KSP Residual norm 7.599375477471e-03 5 KSP Residual norm 2.083060123161e-03 6 KSP Residual norm 3.808511520034e-04 7 KSP Residual norm 8.175282902639e-05 8 KSP Residual norm 2.175564241206e-05 9 KSP Residual norm 5.595617772296e-06 10 KSP Residual norm 1.577629043326e-06 11 KSP Residual norm 4.401090414293e-07 12 KSP Residual norm 9.285166456648e-08 13 KSP Residual norm 1.606071620113e-08 14 KSP Residual norm 3.371039707655e-09 15 KSP Residual norm 7.260259461122e-10 16 KSP Residual norm 1.511146527931e-10 17 KSP Residual norm 3.804907486855e-11 18 KSP Residual norm 1.062507039176e-11