[petsc-users] CPU utilization during GPU solver

On Sat, Nov 17, 2012 at 3:05 PM, David Fuentes wrote: > Thanks Jed. > I was trying to run it in dbg mode to verify if all significant parts of the > solver were running on the GPU and not on the CPU by mistake. > I cant pinpoint what part of the solver is running on the CPU. When I run > top while

[petsc-users] CPU utilization during GPU solver

t; -I/opt/apps/PETSC/petsc-3.3-p4/gcc-4.4.3-mpich2-1.2-epd-sm_20-dbg/forbind/include > >>> -I/usr/include/mpich2 > >>> - > >>> > >>> Using C linker: /usr/bin/mpicxx > >>> Using Fortran linker: /usr/bin/mpif90 > >>> Using libraries: > >>> > -Wl,-rpath,/opt/apps/PETSC/petsc-3.3-p4/gcc-4.4.3-mpich2-1.2-epd-sm_20-dbg/lib > >>> -L/opt/apps/PETSC/petsc-3.3-p4/gcc-4.4.3-mpich2-1.2-epd-sm_20-dbg/lib > >>> -lpetsc > >>> > -Wl,-rpath,/opt/apps/PETSC/petsc-3.3-p4/gcc-4.4.3-mpich2-1.2-epd-sm_20-dbg/lib > >>> -L/opt/apps/PETSC/petsc-3.3-p4/gcc-4.4.3-mpich2-1.2-epd-sm_20-dbg/lib > >>> -ltriangle -lX11 -lpthread -lsuperlu_dist_3.1 -lcmumps -ldmumps > -lsmumps > >>> -lzmumps -lmumps_common -lpord -lparmetis -lmetis -lscalapack -lblacs > >>> -Wl,-rpath,/opt/apps/cuda/4.2//cuda/lib64 > -L/opt/apps/cuda/4.2//cuda/lib64 > >>> -lcufft -lcublas -lcudart -lcusparse > >>> > -Wl,-rpath,/opt/MATLAB/R2011a/sys/os/glnxa64:/opt/MATLAB/R2011a/bin/glnxa64:/opt/MATLAB/R2011a/extern/lib/glnxa64 > >>> -L/opt/MATLAB/R2011a/bin/glnxa64 > -L/opt/MATLAB/R2011a/extern/lib/glnxa64 > >>> -leng -lmex -lmx -lmat -lut -licudata -licui18n -licuuc > >>> -Wl,-rpath,/opt/apps/EPD/epd-7.3-1-rh5-x86_64/lib > >>> -L/opt/apps/EPD/epd-7.3-1-rh5-x86_64/lib -lmkl_rt -lmkl_intel_thread > >>> -lmkl_core -liomp5 -lexoIIv2for -lexodus -lnetcdf_c++ -lnetcdf > >>> -Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/4.4.3 > >>> -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -lmpichf90 -lgfortran -lm -lm > >>> -lmpichcxx -lstdc++ -lmpichcxx -lstdc++ -ldl -lmpich -lopa -lpthread > -lrt > >>> -lgcc_s -ldl > >>> - > >>> > >>> > >>> > >>> On Sat, Nov 17, 2012 at 11:02 AM, Matthew Knepley > >>> wrote: > >>>> > >>>> On Sat, Nov 17, 2012 at 10:50 AM, David Fuentes > >>>> wrote: > >>>> > Hi, > >>>> > > >>>> > I'm using petsc 3.3p4 > >>>> > I'm trying to run a nonlinear SNES solver on GPU with gmres and > jacobi > >>>> > PC > >>>> > using VECSEQCUSP and MATSEQAIJCUSP datatypes for the rhs and > jacobian > >>>> > matrix > >>>> > respectively. > >>>> > When running top I still see significant CPU utilization (800-900 > >>>> > %CPU) > >>>> > during the solve ? possibly from some multithreaded operations ? > >>>> > > >>>> > Is this expected ? > >>>> > I was thinking that since I input everything into the solver as a > CUSP > >>>> > datatype, all linear algebra operations would be on the GPU device > >>>> > from > >>>> > there and wasn't expecting to see such CPU utilization during the > >>>> > solve ? > >>>> > Do I probably have an error in my code somewhere ? > >>>> > >>>> We cannot answer performance questions without -log_summary > >>>> > >>>>Matt > >>>> > >>>> > Thanks, > >>>> > David > >>>> > >>>> > >>>> > >>>> -- > >>>> What most experimenters take for granted before they begin their > >>>> experiments is infinitely more interesting than any results to which > >>>> their experiments lead. > >>>> -- Norbert Wiener > >>> > >>> > > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > -- next part -- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20121117/6a2088e4/attachment-0001.html>

[petsc-users] CPU utilization during GPU solver

;> > I'm trying to run a nonlinear SNES solver on GPU with gmres and jacobi >> PC >> > using VECSEQCUSP and MATSEQAIJCUSP datatypes for the rhs and jacobian >> matrix >> > respectively. >> > When running top I still see significant CPU utilization (800-900 %CPU) >> > during the solve ? possibly from some multithreaded operations ? >> > >> > Is this expected ? >> > I was thinking that since I input everything into the solver as a CUSP >> > datatype, all linear algebra operations would be on the GPU device from >> > there and wasn't expecting to see such CPU utilization during the solve >> ? >> > Do I probably have an error in my code somewhere ? >> >> We cannot answer performance questions without -log_summary >> >>Matt >> >> > Thanks, >> > David >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which >> their experiments lead. >> -- Norbert Wiener >> > > -- next part -- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20121117/1c4c3454/attachment-0001.html>

[petsc-users] CPU utilization during GPU solver

Hi David, the cuda-runtime might spawn threads in addition to PETSc. How many GPUs do you have on your system? You might also want to compare with the CUDA examples. If they also run at ~800% CPU utilization, then it's definitely due to the CUDA runtime. Note that there might full CPU utilizat

[petsc-users] CPU utilization during GPU solver

-lmkl_intel_thread >> -lmkl_core -liomp5 -lexoIIv2for -lexodus -lnetcdf_c++ -lnetcdf >> -Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/4.4.3 >> -L/usr/lib/gcc/x86_64-linux-gnu/4.4.3 -lmpichf90 -lgfortran -lm -lm >> -lmpichcxx -lstdc++ -lmpichcxx -lstdc++ -ldl -lmpich -lopa -lpthread -lrt >> -lgcc_s -ldl >> - >> >> >> >> On Sat, Nov 17, 2012 at 11:02 AM, Matthew Knepley > gmail.com>wrote: >> >>> On Sat, Nov 17, 2012 at 10:50 AM, David Fuentes >>> wrote: >>> > Hi, >>> > >>> > I'm using petsc 3.3p4 >>> > I'm trying to run a nonlinear SNES solver on GPU with gmres and jacobi >>> PC >>> > using VECSEQCUSP and MATSEQAIJCUSP datatypes for the rhs and jacobian >>> matrix >>> > respectively. >>> > When running top I still see significant CPU utilization (800-900 %CPU) >>> > during the solve ? possibly from some multithreaded operations ? >>> > >>> > Is this expected ? >>> > I was thinking that since I input everything into the solver as a CUSP >>> > datatype, all linear algebra operations would be on the GPU device from >>> > there and wasn't expecting to see such CPU utilization during the >>> solve ? >>> > Do I probably have an error in my code somewhere ? >>> >>> We cannot answer performance questions without -log_summary >>> >>>Matt >>> >>> > Thanks, >>> > David >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which >>> their experiments lead. >>> -- Norbert Wiener >>> >> >> -- next part -- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20121117/a8c748af/attachment.html>

[petsc-users] CPU utilization during GPU solver

st experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > -- next part -- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20121117/6521a3ad/attachment.html>

[petsc-users] CPU utilization during GPU solver

On Sat, Nov 17, 2012 at 10:50 AM, David Fuentes wrote: > Hi, > > I'm using petsc 3.3p4 > I'm trying to run a nonlinear SNES solver on GPU with gmres and jacobi PC > using VECSEQCUSP and MATSEQAIJCUSP datatypes for the rhs and jacobian matrix > respectively. > When running top I still see significa

[petsc-users] Which IDE you use to develop petsc application

On 11/16/12 2:03 AM, Barry Smith wrote: > Because we are developing PETSc as a library that must be portable to > many users most people who develop the PETSc libraries do not use a IDE. > > For someone who is writing AN APPLICATION that uses PETSc (especially an > application with guis)

[petsc-users] CPU utilization during GPU solver

ave an error in my code somewhere ? Thanks, David -- next part -- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20121117/80af73f9/attachment-0001.html>