On Thu 2009-04-09 08:12, Tim Kroeger wrote: > Dear Jed, > > On Wed, 8 Apr 2009, Jed Brown wrote: > >> I would be interested to see profiling output from Tim's code with >> ghosted vectors. > > What exactly do you mean? In other words, what do I have to do to > produce the profiling output that you would like to see?
Run with -log_summary so that we can see how much time is being spent in PETSc scatters (this should be the only part that is different between the two cases). Also, if you have access to Oprofile or similar tools, this could tell us where in libmesh the time is being spent. Your timing results show a huge difference that really shouldn't be there. Also, there must be some way for you to adjust the problem size or the integration period so that it takes a few minutes rather than many hours. I realize there was some correctness issue that only manifested itself on large problem sizes after a long integration period, but this shouldn't matter for profiling. >> Also, note that there is a bit of insanity in petsc_vector.C, consider >> this from PetscVector<T>::add(const T) > > Actually, I noticed some of these things when I implemented the ghosted > vectors, but I decided to change as little as possible, assuming that the > hitherto existing implementation was "sensible". What do you think about > just preparing a patch yourself that removes these inefficiencies and > contributing that to the library? I guess that everybody would > appreciate that. I can do this for this simple case, but to be honest, I only occasionally use libmesh (mostly as a reference to compare my own work to). The way ghosted vectors are being used is not quite how they were intended and I don't want to make any deep changes because I don't feel confident that I understand their effects. Basically, there is a distinction between the global vector and the local vector which libmesh is trying to hide by keeping the ghosted entries up to date (essentially regarding the local vectors as parallel vectors and defining collective operations on them). A cleaner model is that the local vectors really are just serial vectors and operations on them are not collective, the global vectors have no ghosted entries. Mostly one works with the global vectors, but during function evaluation and Jacobian assembly, the ghosted values are updated, and only the local form is worked with, assembly is via a local-to-global map. The fact that when using VecGhost, the local and global vectors share memory is just an optimization, they should generally be thought of as separate objects. I would be happy to describe this in more detail if that would be useful, but it's sufficiently different that I have no place making these changes. Jed
pgpp3fUhQfCzE.pgp
Description: PGP signature
------------------------------------------------------------------------------ This SF.net email is sponsored by: High Quality Requirements in a Collaborative Environment. Download a free trial of Rational Requirements Composer Now! http://p.sf.net/sfu/www-ibm-com
_______________________________________________ Libmesh-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/libmesh-devel
