Hi Dave I should just say that we still have not finished a code review with the petsc-dev team so any faults should be assumed to be ours rather than theirs!
Michele just put a preprint of our paper covering the preliminary work on arXiv, which you might found useful: http://arxiv.org/abs/1205.2005 As we have not yet merged with the truck so you'd need to pull a branch from: https://bitbucket.org/wence/petsc-dev-omp and configure using --with-openmp A key issue when running is to set thread/core affinity. Unfortunately there is no general way of doing this - it depends on your compiler. But likwid-pin can make your life a little easier: http://code.google.com/p/likwid/wiki/LikwidPin Cheers Gerard Dave Nystrom emailed the following on 10/05/12 05:55: > Hi Gerard, > > Thanks for the info. Is there any documentation on how to use the petsc > OpenMP support? I would be interested in trying it out. > > Thanks, > > Dave > > Gerard Gorman writes: > > Hi Dave > > > > OpenMP support exists for vec and mat (only AIJ so far). There is a big > > difference in performance depending on available memory bandwidth and > > the compiler OpenMP implementation. In application codes (such as > > Fluidity which is our main target code for this work) there are other > > significant costs such as matrix assembly. So in general you have to > > consider how easy it will be to thread to the other computationally > > expensive sections of your code, otherwise the overall speed-up of your > > application will be modest. > > > > Cheers > > Gerard > > > > Dave Nystrom emailed the following on 09/05/12 04:29: > > > Is the pthreads support further along than the OpenMP support? I have > not > > > tried the pthreads support yet. Does either the pthreads support or the > > > OpenMP support implement the matvec or do they just do vector type > > > operations? > > > > > > Jed Brown writes: > > > > On Tue, May 8, 2012 at 9:23 PM, Dave Nystrom <dnystrom1 at > comcast.net> wrote: > > > > > > > > > I see that petsc-dev now has some OpenMP support. Would a serial, > non-mpi > > > > > code that uses petsc-dev be able to gain much performance > improvement from > > > > > it > > > > > now for the case of doing sparse linear solve with cg and jacobi > > > > > preconditioning? > > > > > > > > > > > > > The kernels are being transitioned to use the threadcomm, which > enables > > > > OpenMP and other threading models. > > > > > > > > We anticipate that pthreads will provide the best performance because > > > > operations can be less synchronous than with OpenMP (for which a > parallel > > > > region implies barrier semantics). But if other parts of an > application > > > > are using OpenMP, it would be preferable for PETSc to also use OpenMP > so > > > > that it can share the same thread pool. The same applies to TBB. > >
