[petsc-dev] Commit collision/corruption?

[Sean Farley]

Jed Brown writes:

> On Wed, Mar 6, 2013 at 7:08 PM, Sean Farley  wrote:
>
>> Jed Brown writes:
>>
>> > On Wed, Mar 6, 2013 at 6:34 PM, Sean Farley  wrote:
>> >
>> >> > Run 'hg export de73c9a7d341d846b5e16a8d61a48'. Note how they do not
>> exist
>> >> > in the formatted patch. Why are they written into the commit?
>> >>
>> >> I don't see this. I see contents for both of Barry's merges. You'll have
>> >> to provide a test case where this happens to rule out gitifyhg.
>> >
>> >
>> > Look for changes to src/mat/impls/aij/seq/ftn-kernels/*.h.
>> >
>> > This all happened before I was involved.
>>
>> You'll have to provide a test case. Also, all the petsc devs might as
>> well use this time to upgrade their mercurial to the latest (2.5.2 at
>> the time of this writing).
>
>
> I gave you instructions above, but here it is more verbosely.
>
> $ hg log -vr de73c9a7d341d846b5e16a8d61a48  |grep fsolvebaij.h
> files:   src/contrib/style/checks/PetscFunctionBegin.sh
> src/contrib/style/checks/PetscFunctionBegin2.sh
> src/contrib/style/checks/PetscFunctionReturn.sh
> src/dm/impls/composite/pack.c src/dm/impls/plex/f90-custom/zplexf90.c
> src/dm/impls/plex/ftn-custom/zplex.c src/dm/impls/plex/plexcreate.c
> src/ksp/pc/impls/bjacobi/bjacobi.c src/ksp/pc/impls/fieldsplit/fieldsplit.c
> src/ksp/pc/impls/gamg/gamg.c src/ksp/pc/impls/openmp/hpc.c
> src/mat/impls/aij/mpi/mpiaij.c src/mat/impls/aij/mpi/mpiaij.h
> src/mat/impls/aij/seq/aij.h
> src/mat/impls/aij/seq/crl/ftn-kernels/fmultcrl.h
> src/mat/impls/aij/seq/ftn-kernels/fmult.h
> src/mat/impls/aij/seq/ftn-kernels/fmultadd.h
> src/mat/impls/aij/seq/ftn-kernels/frelax.h
> src/mat/impls/aij/seq/ftn-kernels/fsolve.h src/mat/impls/baij/seq/baij.h
> src/mat/impls/baij/seq/ftn-kernels/fsolvebaij.h
> src/mat/impls/dense/seq/dense.h src/mat/impls/fft/fft.h
> src/mat/impls/maij/maij.h src/mat/impls/mffd/mffd.c
> src/mat/impls/sbaij/seq/sbaij.h src/mat/utils/zerodiag.c
> src/snes/mf/snesmfj.c src/vec/vec/impls/dvecimpl.h
> src/vec/vec/impls/seq/seqcusp/cuspvecimpl.h
> $ hg export de73c9a7d341d846b5e16a8d61a48  |grep fsolvebaij.h
> $
>
> So fsolvebaij.h (and 9 other files) are not affected by the commit, but are
> named anyway.
>
> Recall that this is a commit that Barry made on his own, before I had
> anything to do with that branch. There was no rebase and no gitifyhg
> involved. If you strip that commit and 'hg import' what you just exported,
> you will get a different hash because those files that are named in the
> commit, but not affected by the commit, are no longer present.
>
> Please show me in the hg specification for commits where it is allowed for
> a commit to name files that are unrelated to the commit and do not appear
> in 'hg export', but whose file names are part of the SHA1.
>
>
>
>
> Editorializing, I believe that diffs are a cruddy way to archive state
> because it relies on the diff being normalized. The above conflict would be
> impossible with git because the sha1 represents content (not a diff). The
> sha1 cannot be different without the content being different. With hg, we
> have multiple ways to represent semantically identical commits because the
> binary diff representation is not normalized (it's possible for it to name
> files that are not affected by the commit). That is, a bug in Hg can create
> a commit like this (based on non-normalized diff), but a bug in Git cannot
> create a commit based on a non-normalized representation because the only
> representation is the content itself.

Responding to this paragraph more in depth now; your information above
is at best misinformed and at worse complete bs. Both mercurial and git
store the sha1 representation with a binary-safe "diff" algorithm. Git's
seems to be like xdiff and mercurial's is mdiff.

Let's look again at your original request, 'hg export de73c9a | grep
fsolvebaij' shows no output. Well, that's because you're diffing against
the wrong parent. Running 'hg export --switch-parent de73c9a | grep
fsolvebaij' gives:

diff --git a/src/mat/impls/baij/seq/ftn-kernels/fsolvebaij.h 
b/src/mat/impls/baij/seq/ftn-kernels/fsolvebaij.h
--- a/src/mat/impls/baij/seq/ftn-kernels/fsolvebaij.h
+++ b/src/mat/impls/baij/seq/ftn-kernels/fsolvebaij.h

I seem to remember someone once saying, "This is a visualization issue."
[1]

Editorializing, I believe that you are wasting everyone's time with your
constant (and mostly vacuous) complaining about mercurial. No one has a
problem with this merge except for you. And, judging from your earlier
emails, it was a problem for you after you ran it through gitifyhg. So,
again, I say that this is a bug with your extension. If you could not
muck up the petsc repository with these erroneous commits, I would much
appreciate it.

[1] - http://lists.mcs.anl.gov/pipermail/petsc-dev/2012-February/007490.html

[petsc-dev] Commit collision/corruption?

[Jed Brown]

On Wed, Mar 6, 2013 at 8:37 PM, Sean Farley  wrote:

> > This *is* pure mercurial. All relevant commits were made by Barry. (He is
> > not to blame, but I want to make it exceedingly clear that I was not
> > involved in the creation of this "bad" commit.)
>
> This is what I'm skeptical about. Starting from an empty repo, you'll
> need to show the commands of what generates this bad merge.


So bugs do not exist until they have a minimal reproducible test case? Must
be nice.
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/d1ad3bb1/attachment.html>

[petsc-dev] Commit collision/corruption?

[Sean Farley]

Jed Brown writes:

> On Wed, Mar 6, 2013 at 8:15 PM, Sean Farley  wrote:
>
>> No, I got that. I, and no other dev, can reproduce. What I was implying
>> is an error between the chair and the keyboard. Perhaps you didn't
>> realize you did a rebase or some other such human mistake.
>>
>> > $ hg log -vr de73c9a7d341d846b5e16a8d61a48  |grep fsolvebaij.h
>>
>
> If you say you cannot reproduce, please show the output of the above
> command.
>
> It's obvious that it names files with zero changes in the web interface
> (see attached pic). Bear in mind that Barry made this commit. Barry uses
> vanilla Hg so it was produced by Hg core.
>
>
>> > files:   src/contrib/style/checks/PetscFunctionBegin.sh
>> > src/contrib/style/checks/PetscFunctionBegin2.sh
>> > src/contrib/style/checks/PetscFunctionReturn.sh
>> > src/dm/impls/composite/pack.c src/dm/impls/plex/f90-custom/zplexf90.c
>> > src/dm/impls/plex/ftn-custom/zplex.c src/dm/impls/plex/plexcreate.c
>> > src/ksp/pc/impls/bjacobi/bjacobi.c
>> src/ksp/pc/impls/fieldsplit/fieldsplit.c
>> > src/ksp/pc/impls/gamg/gamg.c src/ksp/pc/impls/openmp/hpc.c
>> > src/mat/impls/aij/mpi/mpiaij.c src/mat/impls/aij/mpi/mpiaij.h
>> > src/mat/impls/aij/seq/aij.h
>> > src/mat/impls/aij/seq/crl/ftn-kernels/fmultcrl.h
>> > src/mat/impls/aij/seq/ftn-kernels/fmult.h
>> > src/mat/impls/aij/seq/ftn-kernels/fmultadd.h
>> > src/mat/impls/aij/seq/ftn-kernels/frelax.h
>> > src/mat/impls/aij/seq/ftn-kernels/fsolve.h src/mat/impls/baij/seq/baij.h
>> > src/mat/impls/baij/seq/ftn-kernels/fsolvebaij.h
>> > src/mat/impls/dense/seq/dense.h src/mat/impls/fft/fft.h
>> > src/mat/impls/maij/maij.h src/mat/impls/mffd/mffd.c
>> > src/mat/impls/sbaij/seq/sbaij.h src/mat/utils/zerodiag.c
>> > src/snes/mf/snesmfj.c src/vec/vec/impls/dvecimpl.h
>> > src/vec/vec/impls/seq/seqcusp/cuspvecimpl.h
>> > $ hg export de73c9a7d341d846b5e16a8d61a48  |grep fsolvebaij.h
>> > $
>> >
>> > So fsolvebaij.h (and 9 other files) are not affected by the commit, but
>> are
>> > named anyway.
>> >
>> > Recall that this is a commit that Barry made on his own, before I had
>> > anything to do with that branch. There was no rebase and no gitifyhg
>> > involved. If you strip that commit and 'hg import' what you just
>> exported,
>> > you will get a different hash because those files that are named in the
>> > commit, but not affected by the commit, are no longer present.
>> >
>> > Please show me in the hg specification for commits where it is allowed
>> for
>> > a commit to name files that are unrelated to the commit and do not appear
>> > in 'hg export', but whose file names are part of the SHA1.
>>
>> Since I cannot reproduce this, I am skeptical of this being pure
>> mercurial. I posit that you, perhaps unknowingly, infected the tree with
>> gitifyhg (or rebase?).
>>
>> We could be in some corner case of case-collision with a
>> case-insensitive file system but we'd need a pure mercurial test case.
>>
>
> This *is* pure mercurial. All relevant commits were made by Barry. (He is
> not to blame, but I want to make it exceedingly clear that I was not
> involved in the creation of this "bad" commit.)

This is what I'm skeptical about. Starting from an empty repo, you'll
need to show the commands of what generates this bad merge.

[petsc-dev] Commit collision/corruption?

[Jed Brown]

On Wed, Mar 6, 2013 at 8:15 PM, Sean Farley  wrote:

> No, I got that. I, and no other dev, can reproduce. What I was implying
> is an error between the chair and the keyboard. Perhaps you didn't
> realize you did a rebase or some other such human mistake.
>
> > $ hg log -vr de73c9a7d341d846b5e16a8d61a48  |grep fsolvebaij.h
>

If you say you cannot reproduce, please show the output of the above
command.

It's obvious that it names files with zero changes in the web interface
(see attached pic). Bear in mind that Barry made this commit. Barry uses
vanilla Hg so it was produced by Hg core.


> > files:   src/contrib/style/checks/PetscFunctionBegin.sh
> > src/contrib/style/checks/PetscFunctionBegin2.sh
> > src/contrib/style/checks/PetscFunctionReturn.sh
> > src/dm/impls/composite/pack.c src/dm/impls/plex/f90-custom/zplexf90.c
> > src/dm/impls/plex/ftn-custom/zplex.c src/dm/impls/plex/plexcreate.c
> > src/ksp/pc/impls/bjacobi/bjacobi.c
> src/ksp/pc/impls/fieldsplit/fieldsplit.c
> > src/ksp/pc/impls/gamg/gamg.c src/ksp/pc/impls/openmp/hpc.c
> > src/mat/impls/aij/mpi/mpiaij.c src/mat/impls/aij/mpi/mpiaij.h
> > src/mat/impls/aij/seq/aij.h
> > src/mat/impls/aij/seq/crl/ftn-kernels/fmultcrl.h
> > src/mat/impls/aij/seq/ftn-kernels/fmult.h
> > src/mat/impls/aij/seq/ftn-kernels/fmultadd.h
> > src/mat/impls/aij/seq/ftn-kernels/frelax.h
> > src/mat/impls/aij/seq/ftn-kernels/fsolve.h src/mat/impls/baij/seq/baij.h
> > src/mat/impls/baij/seq/ftn-kernels/fsolvebaij.h
> > src/mat/impls/dense/seq/dense.h src/mat/impls/fft/fft.h
> > src/mat/impls/maij/maij.h src/mat/impls/mffd/mffd.c
> > src/mat/impls/sbaij/seq/sbaij.h src/mat/utils/zerodiag.c
> > src/snes/mf/snesmfj.c src/vec/vec/impls/dvecimpl.h
> > src/vec/vec/impls/seq/seqcusp/cuspvecimpl.h
> > $ hg export de73c9a7d341d846b5e16a8d61a48  |grep fsolvebaij.h
> > $
> >
> > So fsolvebaij.h (and 9 other files) are not affected by the commit, but
> are
> > named anyway.
> >
> > Recall that this is a commit that Barry made on his own, before I had
> > anything to do with that branch. There was no rebase and no gitifyhg
> > involved. If you strip that commit and 'hg import' what you just
> exported,
> > you will get a different hash because those files that are named in the
> > commit, but not affected by the commit, are no longer present.
> >
> > Please show me in the hg specification for commits where it is allowed
> for
> > a commit to name files that are unrelated to the commit and do not appear
> > in 'hg export', but whose file names are part of the SHA1.
>
> Since I cannot reproduce this, I am skeptical of this being pure
> mercurial. I posit that you, perhaps unknowingly, infected the tree with
> gitifyhg (or rebase?).
>
> We could be in some corner case of case-collision with a
> case-insensitive file system but we'd need a pure mercurial test case.
>

This *is* pure mercurial. All relevant commits were made by Barry. (He is
not to blame, but I want to make it exceedingly clear that I was not
involved in the creation of this "bad" commit.)
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/61931dd6/attachment-0001.html>
-- next part --
A non-text attachment was scrubbed...
Name: zero-change-files.png
Type: image/png
Size: 68603 bytes
Desc: not available
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/61931dd6/attachment-0001.png>

[petsc-dev] Commit collision/corruption?

[Sean Farley]

Jed Brown writes:

> On Wed, Mar 6, 2013 at 7:08 PM, Sean Farley  wrote:
>
>> Jed Brown writes:
>>
>> > On Wed, Mar 6, 2013 at 6:34 PM, Sean Farley  wrote:
>> >
>> >> > Run 'hg export de73c9a7d341d846b5e16a8d61a48'. Note how they do not
>> exist
>> >> > in the formatted patch. Why are they written into the commit?
>> >>
>> >> I don't see this. I see contents for both of Barry's merges. You'll have
>> >> to provide a test case where this happens to rule out gitifyhg.
>> >
>> >
>> > Look for changes to src/mat/impls/aij/seq/ftn-kernels/*.h.
>> >
>> > This all happened before I was involved.
>>
>> You'll have to provide a test case. Also, all the petsc devs might as
>> well use this time to upgrade their mercurial to the latest (2.5.2 at
>> the time of this writing).
>
>
> I gave you instructions above, but here it is more verbosely.

No, I got that. I, and no other dev, can reproduce. What I was implying
is an error between the chair and the keyboard. Perhaps you didn't
realize you did a rebase or some other such human mistake.

> $ hg log -vr de73c9a7d341d846b5e16a8d61a48  |grep fsolvebaij.h
> files:   src/contrib/style/checks/PetscFunctionBegin.sh
> src/contrib/style/checks/PetscFunctionBegin2.sh
> src/contrib/style/checks/PetscFunctionReturn.sh
> src/dm/impls/composite/pack.c src/dm/impls/plex/f90-custom/zplexf90.c
> src/dm/impls/plex/ftn-custom/zplex.c src/dm/impls/plex/plexcreate.c
> src/ksp/pc/impls/bjacobi/bjacobi.c src/ksp/pc/impls/fieldsplit/fieldsplit.c
> src/ksp/pc/impls/gamg/gamg.c src/ksp/pc/impls/openmp/hpc.c
> src/mat/impls/aij/mpi/mpiaij.c src/mat/impls/aij/mpi/mpiaij.h
> src/mat/impls/aij/seq/aij.h
> src/mat/impls/aij/seq/crl/ftn-kernels/fmultcrl.h
> src/mat/impls/aij/seq/ftn-kernels/fmult.h
> src/mat/impls/aij/seq/ftn-kernels/fmultadd.h
> src/mat/impls/aij/seq/ftn-kernels/frelax.h
> src/mat/impls/aij/seq/ftn-kernels/fsolve.h src/mat/impls/baij/seq/baij.h
> src/mat/impls/baij/seq/ftn-kernels/fsolvebaij.h
> src/mat/impls/dense/seq/dense.h src/mat/impls/fft/fft.h
> src/mat/impls/maij/maij.h src/mat/impls/mffd/mffd.c
> src/mat/impls/sbaij/seq/sbaij.h src/mat/utils/zerodiag.c
> src/snes/mf/snesmfj.c src/vec/vec/impls/dvecimpl.h
> src/vec/vec/impls/seq/seqcusp/cuspvecimpl.h
> $ hg export de73c9a7d341d846b5e16a8d61a48  |grep fsolvebaij.h
> $
>
> So fsolvebaij.h (and 9 other files) are not affected by the commit, but are
> named anyway.
>
> Recall that this is a commit that Barry made on his own, before I had
> anything to do with that branch. There was no rebase and no gitifyhg
> involved. If you strip that commit and 'hg import' what you just exported,
> you will get a different hash because those files that are named in the
> commit, but not affected by the commit, are no longer present.
>
> Please show me in the hg specification for commits where it is allowed for
> a commit to name files that are unrelated to the commit and do not appear
> in 'hg export', but whose file names are part of the SHA1.

Since I cannot reproduce this, I am skeptical of this being pure
mercurial. I posit that you, perhaps unknowingly, infected the tree with
gitifyhg (or rebase?).

We could be in some corner case of case-collision with a
case-insensitive file system but we'd need a pure mercurial test case.

> Editorializing, I believe that diffs are a cruddy way to archive state
> because it relies on the diff being normalized. The above conflict would be
> impossible with git because the sha1 represents content (not a diff). The
> sha1 cannot be different without the content being different. With hg, we
> have multiple ways to represent semantically identical commits because the
> binary diff representation is not normalized (it's possible for it to name
> files that are not affected by the commit). That is, a bug in Hg can create
> a commit like this (based on non-normalized diff), but a bug in Git cannot
> create a commit based on a non-normalized representation because the only
> representation is the content itself.

You have yet to prove this is a bug in mercurial and not from you. Hence
my earlier request for a complete test case.

[petsc-dev] Commit collision/corruption?

[Jed Brown]

On Wed, Mar 6, 2013 at 7:08 PM, Sean Farley  wrote:

> Jed Brown writes:
>
> > On Wed, Mar 6, 2013 at 6:34 PM, Sean Farley  wrote:
> >
> >> > Run 'hg export de73c9a7d341d846b5e16a8d61a48'. Note how they do not
> exist
> >> > in the formatted patch. Why are they written into the commit?
> >>
> >> I don't see this. I see contents for both of Barry's merges. You'll have
> >> to provide a test case where this happens to rule out gitifyhg.
> >
> >
> > Look for changes to src/mat/impls/aij/seq/ftn-kernels/*.h.
> >
> > This all happened before I was involved.
>
> You'll have to provide a test case. Also, all the petsc devs might as
> well use this time to upgrade their mercurial to the latest (2.5.2 at
> the time of this writing).


I gave you instructions above, but here it is more verbosely.

$ hg log -vr de73c9a7d341d846b5e16a8d61a48  |grep fsolvebaij.h
files:   src/contrib/style/checks/PetscFunctionBegin.sh
src/contrib/style/checks/PetscFunctionBegin2.sh
src/contrib/style/checks/PetscFunctionReturn.sh
src/dm/impls/composite/pack.c src/dm/impls/plex/f90-custom/zplexf90.c
src/dm/impls/plex/ftn-custom/zplex.c src/dm/impls/plex/plexcreate.c
src/ksp/pc/impls/bjacobi/bjacobi.c src/ksp/pc/impls/fieldsplit/fieldsplit.c
src/ksp/pc/impls/gamg/gamg.c src/ksp/pc/impls/openmp/hpc.c
src/mat/impls/aij/mpi/mpiaij.c src/mat/impls/aij/mpi/mpiaij.h
src/mat/impls/aij/seq/aij.h
src/mat/impls/aij/seq/crl/ftn-kernels/fmultcrl.h
src/mat/impls/aij/seq/ftn-kernels/fmult.h
src/mat/impls/aij/seq/ftn-kernels/fmultadd.h
src/mat/impls/aij/seq/ftn-kernels/frelax.h
src/mat/impls/aij/seq/ftn-kernels/fsolve.h src/mat/impls/baij/seq/baij.h
src/mat/impls/baij/seq/ftn-kernels/fsolvebaij.h
src/mat/impls/dense/seq/dense.h src/mat/impls/fft/fft.h
src/mat/impls/maij/maij.h src/mat/impls/mffd/mffd.c
src/mat/impls/sbaij/seq/sbaij.h src/mat/utils/zerodiag.c
src/snes/mf/snesmfj.c src/vec/vec/impls/dvecimpl.h
src/vec/vec/impls/seq/seqcusp/cuspvecimpl.h
$ hg export de73c9a7d341d846b5e16a8d61a48  |grep fsolvebaij.h
$

So fsolvebaij.h (and 9 other files) are not affected by the commit, but are
named anyway.

Recall that this is a commit that Barry made on his own, before I had
anything to do with that branch. There was no rebase and no gitifyhg
involved. If you strip that commit and 'hg import' what you just exported,
you will get a different hash because those files that are named in the
commit, but not affected by the commit, are no longer present.

Please show me in the hg specification for commits where it is allowed for
a commit to name files that are unrelated to the commit and do not appear
in 'hg export', but whose file names are part of the SHA1.




Editorializing, I believe that diffs are a cruddy way to archive state
because it relies on the diff being normalized. The above conflict would be
impossible with git because the sha1 represents content (not a diff). The
sha1 cannot be different without the content being different. With hg, we
have multiple ways to represent semantically identical commits because the
binary diff representation is not normalized (it's possible for it to name
files that are not affected by the commit). That is, a bug in Hg can create
a commit like this (based on non-normalized diff), but a bug in Git cannot
create a commit based on a non-normalized representation because the only
representation is the content itself.
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/0e7c1816/attachment-0001.html>

[petsc-dev] Commit collision/corruption?

[Sean Farley]

Jed Brown writes:

> On Wed, Mar 6, 2013 at 6:34 PM, Sean Farley  wrote:
>
>> > Run 'hg export de73c9a7d341d846b5e16a8d61a48'. Note how they do not exist
>> > in the formatted patch. Why are they written into the commit?
>>
>> I don't see this. I see contents for both of Barry's merges. You'll have
>> to provide a test case where this happens to rule out gitifyhg.
>
>
> Look for changes to src/mat/impls/aij/seq/ftn-kernels/*.h.
>
> This all happened before I was involved.

You'll have to provide a test case. Also, all the petsc devs might as
well use this time to upgrade their mercurial to the latest (2.5.2 at
the time of this writing).

[petsc-dev] Commit collision/corruption?

[Jed Brown]

On Wed, Mar 6, 2013 at 6:34 PM, Sean Farley  wrote:

> > Run 'hg export de73c9a7d341d846b5e16a8d61a48'. Note how they do not exist
> > in the formatted patch. Why are they written into the commit?
>
> I don't see this. I see contents for both of Barry's merges. You'll have
> to provide a test case where this happens to rule out gitifyhg.


Look for changes to src/mat/impls/aij/seq/ftn-kernels/*.h.

This all happened before I was involved.
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/10b3313f/attachment.html>

[petsc-dev] Commit collision/corruption?

[Sean Farley]

Jed Brown writes:

> On Wed, Mar 6, 2013 at 5:25 PM, Sean Farley  wrote:
>
>> This right here is the most suspect part. The minute you start monkeying
>> around with private functions and whatnot, you are in the danger zone. I
>> will have a hard time believing this isn't a bug in gitifyhg. In fact, I
>> am willing to bet on it.
>>
>
> This commit took place before anything else. Can you explain the
> zero-content files?
>
> https://bitbucket.org/BarryFSmith/petsc-dev-simp/commits/de73c9a7d341d846b5e16a8d61a48242f0521c02
>
> Run 'hg export de73c9a7d341d846b5e16a8d61a48'. Note how they do not exist
> in the formatted patch. Why are they written into the commit?

I don't see this. I see contents for both of Barry's merges. You'll have
to provide a test case where this happens to rule out gitifyhg.

[petsc-dev] Some problems with builds --with-matlab

[Barry Smith]

  Fixed both 767347ff803d   I believe

On Mar 6, 2013, at 11:00 AM, "Jose E. Roman"  wrote:

> Hi.
> 
> Today I pushed some fixes for compilation --with-matlab --with-matlab-engine. 
> However, there is still a compilation problem when Fortran in enabled: the 
> automatic Fortran bindings in src/sys/classes/matlabengine need the 
> definition of PetscMatlabEngine. I tried adding /*I "petscmatlab.h" I*/ in 
> matlab.c but that does not work.

   This is the right fix and worked for me. Did you rerun make allfortranstubs?

   Barry

> 
> The other problem is related to the visibility topic discussed in another 
> thread. In SLEPc we have SlepcInitializeNoPointers() to be called from 
> Matlab. There I set PetscBeganMPI = PETSC_FALSE (as is done in 
> PetscInitializeNoPointers()) because otherwise I get PetscFinalize() errors 
> the second time I run a Matlab script containing SLEPc calls. So the symbol 
> PetscBeganMPI should be exported (unless you can suggest a better solution).
> 
> Jose
> 

[petsc-dev] Some problems with builds --with-matlab

[Jose E. Roman]

Hi.

Today I pushed some fixes for compilation --with-matlab --with-matlab-engine. 
However, there is still a compilation problem when Fortran in enabled: the 
automatic Fortran bindings in src/sys/classes/matlabengine need the definition 
of PetscMatlabEngine. I tried adding /*I "petscmatlab.h" I*/ in matlab.c but 
that does not work.

The other problem is related to the visibility topic discussed in another 
thread. In SLEPc we have SlepcInitializeNoPointers() to be called from Matlab. 
There I set PetscBeganMPI = PETSC_FALSE (as is done in 
PetscInitializeNoPointers()) because otherwise I get PetscFinalize() errors the 
second time I run a Matlab script containing SLEPc calls. So the symbol 
PetscBeganMPI should be exported (unless you can suggest a better solution).

Jose

[petsc-dev] Commit collision/corruption?

[Jed Brown]

On Wed, Mar 6, 2013 at 5:44 PM, Satish Balay  wrote:

> This is the clue that points to "commit" massaging/altering tools
> [like "rebase" - that I refer to and "gitifyhg" that Sean is pointing
> to] as the instigator. They don't do 'hg operation' on files - but do
> internal commit metadata manipulation - that can result in such
> "empty" diffs in commits.
>

Barry made that commit. He does not use rebase and does not use gitifyhg.

We may not have noticed it if I hadn't imported it through gitifyhg later,
but the original commit still appears to be messed up. There is no way for
gitifyhg to preserve sha1s for corrupt hg commits, we can only expoct to do
it for valid commits.
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/88bf9900/attachment.html>

[petsc-dev] Commit collision/corruption?

[Satish Balay]

On Wed, 6 Mar 2013, Jed Brown wrote:

> On Wed, Mar 6, 2013 at 5:25 PM, Sean Farley  wrote:
> 
> > This right here is the most suspect part. The minute you start monkeying
> > around with private functions and whatnot, you are in the danger zone. I
> > will have a hard time believing this isn't a bug in gitifyhg. In fact, I
> > am willing to bet on it.
> >
> 
> This commit took place before anything else. Can you explain the
> zero-content files?

This is the clue that points to "commit" massaging/altering tools
[like "rebase" - that I refer to and "gitifyhg" that Sean is pointing
to] as the instigator. They don't do 'hg operation' on files - but do
internal commit metadata manipulation - that can result in such
"empty" diffs in commits.

Satish

> https://bitbucket.org/BarryFSmith/petsc-dev-simp/commits/de73c9a7d341d846b5e16a8d61a48242f0521c02
> 
> Run 'hg export de73c9a7d341d846b5e16a8d61a48'. Note how they do not exist
> in the formatted patch. Why are they written into the commit?
> 

[petsc-dev] Commit collision/corruption?

[Jed Brown]

On Wed, Mar 6, 2013 at 5:25 PM, Sean Farley  wrote:

> This right here is the most suspect part. The minute you start monkeying
> around with private functions and whatnot, you are in the danger zone. I
> will have a hard time believing this isn't a bug in gitifyhg. In fact, I
> am willing to bet on it.
>

This commit took place before anything else. Can you explain the
zero-content files?

https://bitbucket.org/BarryFSmith/petsc-dev-simp/commits/de73c9a7d341d846b5e16a8d61a48242f0521c02

Run 'hg export de73c9a7d341d846b5e16a8d61a48'. Note how they do not exist
in the formatted patch. Why are they written into the commit?
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/30f8e402/attachment.html>

[petsc-dev] Commit collision/corruption?

[Sean Farley]

Jed Brown writes:

> On Wed, Mar 6, 2013 at 5:10 PM, Sean Farley  wrote:
>
>> Who did the rebasing? Which version of mercurial? Jed, was this
>> through your gitifyhg extension?
>>
>
> There was never any rebasing. Barry made the messed up commit over here.
>
> https://bitbucket.org/BarryFSmith/petsc-dev-simp/commits/de73c9a7d341d846b5e16a8d61a48242f0521c02
>
> It was rewritten on subsequent push (when gitifyhg applied the commit to my
> local hg) to only name the files that changed.

This right here is the most suspect part. The minute you start monkeying
around with private functions and whatnot, you are in the danger zone. I
will have a hard time believing this isn't a bug in gitifyhg. In fact, I
am willing to bet on it.

[petsc-dev] Commit collision/corruption?

[Jed Brown]

On Wed, Mar 6, 2013 at 5:05 PM, Satish Balay  wrote:

> The empty 'diffs' is interesting. It happens with 'rebase of a merge'
> [for some hg versions or all hg version?]  But then this is a merge
> commit [not a rebase commit]. And then Barry doesn't use rebase..
>
> So my suspession is - BitBucket doing someting funky for a commit that
> got pushed via 2 diffrent routes.
>

Bitbucket cannot rewrite SHA1s. It was the hg running on Barry's machine
that made the funky commit.
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/8de81a16/attachment.html>

[petsc-dev] Commit collision/corruption?

[Jed Brown]

On Wed, Mar 6, 2013 at 5:10 PM, Sean Farley  wrote:

> Who did the rebasing? Which version of mercurial? Jed, was this
> through your gitifyhg extension?
>

There was never any rebasing. Barry made the messed up commit over here.

https://bitbucket.org/BarryFSmith/petsc-dev-simp/commits/de73c9a7d341d846b5e16a8d61a48242f0521c02

It was rewritten on subsequent push (when gitifyhg applied the commit to my
local hg) to only name the files that changed.
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/79da1db8/attachment.html>

[petsc-dev] Commit collision/corruption?

[Sean Farley]

On Wed, Mar 6, 2013 at 5:05 PM, Satish Balay  wrote:
> On Wed, 6 Mar 2013, Jed Brown wrote:
>
>> On Wed, Mar 6, 2013 at 4:54 PM, Karl Rupp  wrote:
>>
>> > Hmm, maybe this is just the same commit applied to different branches?
>> > (You probably checked
>> > https://bitbucket.org/petsc/**petsc-dev/commits/all/already?)
>> > One seems to be applied to the 'yellow branch', the other to the 'blue
>> > branch'.
>> >
>>
>> The color is not significant. Neither commit has a named branch associated
>> with it.
>>
>> (Hg branches are like labels in the sense that you can have several heads
>> that are all part of the same named "branch".)
>
> The empty 'diffs' is interesting. It happens with 'rebase of a merge'
> [for some hg versions or all hg version?]  But then this is a merge
> commit [not a rebase commit]. And then Barry doesn't use rebase..
>
> So my suspession is - BitBucket doing someting funky for a commit that
> got pushed via 2 diffrent routes.

Who did the rebasing? Which version of mercurial? Jed, was this
through your gitifyhg extension?

[petsc-dev] Commit collision/corruption?

[Satish Balay]

On Wed, 6 Mar 2013, Jed Brown wrote:

> On Wed, Mar 6, 2013 at 4:54 PM, Karl Rupp  wrote:
> 
> > Hmm, maybe this is just the same commit applied to different branches?
> > (You probably checked
> > https://bitbucket.org/petsc/**petsc-dev/commits/all/already?)
> > One seems to be applied to the 'yellow branch', the other to the 'blue
> > branch'.
> >
> 
> The color is not significant. Neither commit has a named branch associated
> with it.
> 
> (Hg branches are like labels in the sense that you can have several heads
> that are all part of the same named "branch".)

The empty 'diffs' is interesting. It happens with 'rebase of a merge'
[for some hg versions or all hg version?]  But then this is a merge
commit [not a rebase commit]. And then Barry doesn't use rebase..

So my suspession is - BitBucket doing someting funky for a commit that
got pushed via 2 diffrent routes.

Satish

[petsc-dev] Commit collision/corruption?

[Jed Brown]

On Wed, Mar 6, 2013 at 4:54 PM, Karl Rupp  wrote:

> Hmm, maybe this is just the same commit applied to different branches?
> (You probably checked
> https://bitbucket.org/petsc/**petsc-dev/commits/all/<https://bitbucket.org/petsc/petsc-dev/commits/all/>already?)
> One seems to be applied to the 'yellow branch', the other to the 'blue
> branch'.
>

The color is not significant. Neither commit has a named branch associated
with it.

(Hg branches are like labels in the sense that you can have several heads
that are all part of the same named "branch".)
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/139f91a4/attachment.html>

[petsc-dev] Commit collision/corruption?

[Jed Brown]

On Wed, Mar 6, 2013 at 4:53 PM, Barry Smith  wrote:

>
>   No idea. Presumably related to my "branch" stuff but I don't remember
> anything odd happening
>

Hg _should_ make it impossible to create a commit like that. Neither of
those commits has a named branch.


>   Barry
>
> On Mar 6, 2013, at 4:34 PM, Jed Brown  wrote:
>
> > Can anyone determine the difference between these two commits?
> >
> > https://bitbucket.org/petsc/petsc-dev/commits/de73c9a7d341
> > https://bitbucket.org/petsc/petsc-dev/commits/84df07d03c6e
> >
> > They have the same parents and exactly the same content. The first one
> shows a few files with 0 lines changed, but those file names are elided in
> the second because they really did not change. I think the first is a
> corrupt commit, but I don't know how it was created. Can anyone tell what
> happened here?
>
>
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/0d58f155/attachment.html>

[petsc-dev] Commit collision/corruption?

[Karl Rupp]

Hi Jed,

On 03/06/2013 04:34 PM, Jed Brown wrote:
> Can anyone determine the difference between these two commits?
>
> https://bitbucket.org/petsc/petsc-dev/commits/de73c9a7d341
> https://bitbucket.org/petsc/petsc-dev/commits/84df07d03c6e
>
> They have the same parents and exactly the same content. The first one
> shows a few files with 0 lines changed, but those file names are elided
> in the second because they really did not change. I think the first is a
> corrupt commit, but I don't know how it was created. Can anyone tell
> what happened here?

Hmm, maybe this is just the same commit applied to different branches?
(You probably checked
https://bitbucket.org/petsc/petsc-dev/commits/all/ already?)
One seems to be applied to the 'yellow branch', the other to the 'blue 
branch'.

Best regards,
Karli

[petsc-dev] Commit collision/corruption?

[Barry Smith]

  No idea. Presumably related to my "branch" stuff but I don't remember 
anything odd happening

  Barry

On Mar 6, 2013, at 4:34 PM, Jed Brown  wrote:

> Can anyone determine the difference between these two commits?
> 
> https://bitbucket.org/petsc/petsc-dev/commits/de73c9a7d341
> https://bitbucket.org/petsc/petsc-dev/commits/84df07d03c6e
> 
> They have the same parents and exactly the same content. The first one shows 
> a few files with 0 lines changed, but those file names are elided in the 
> second because they really did not change. I think the first is a corrupt 
> commit, but I don't know how it was created. Can anyone tell what happened 
> here?

[petsc-dev] Commit collision/corruption?

[Jed Brown]

Can anyone determine the difference between these two commits?

https://bitbucket.org/petsc/petsc-dev/commits/de73c9a7d341
https://bitbucket.org/petsc/petsc-dev/commits/84df07d03c6e

They have the same parents and exactly the same content. The first one
shows a few files with 0 lines changed, but those file names are elided in
the second because they really did not change. I think the first is a
corrupt commit, but I don't know how it was created. Can anyone tell what
happened here?
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/fe6be827/attachment-0001.html>

[petsc-dev] OpenMP in PETSc when calling from Fortran?

[Barry Smith]

   I don't see any options for turning on the threads here?

  #PETSc Option Table entries:
-ksp_type bcgs
-log_summary
-pc_type lu
#End of PETSc Option Table entries

  From http://www.mcs.anl.gov/petsc/features/threads.html

? The three important run-time options for using threads are:
? -threadcomm_nthreads : Sets the number of threads
? -threadcomm_affinities : Sets the core 
affinities of threads
? -threadcomm_type : Threading model 
(OpenMP, pthread, nothread)
? Run with -help to see the avialable options with threads.
? A few tutorial examples are located at 
$PETSC_DIR/src/sys/threadcomm/examples/tutorials

  Also LU is a direct solver that is not threaded so using threads for this 
exact run will not help (much) at all. The threads will only show useful speed 
up for iterative methods.

   Barry

  As time goes by we hope to have more extensive support in more routines for 
threads but things like factorization and solve are difficult so out side help 
would be very useful.

On Mar 6, 2013, at 3:39 AM, ?smund Ervik  wrote:

> Hi again,
> 
> On 01. mars 2013 20:06, Jed Brown wrote:
>> 
>> Matrix and vector operations are probably running in parallel, but probably
>> not the operations that are taking time. Always send -log_summary if you
>> have a performance question.
>> 
> 
> I don't think they are running in parallel. When I analyze my code in
> Intel Vtune Amplifier, the only routines running in parallel are my own
> OpenMP ones. Indeed, if I comment out my OpenMP pragmas and recompile my
> code, it never uses more than one thread.
> 
> -log_summary is shown below; this is using -pc_type lu -ksp_type bcgs.
> The fastest PC for my cases is usually BoomerAMG from HYPRE, so i used
> LU instead here in order to limit the test to PETSc only. The summary
> agrees with Vtune that MatLUFactorNumeric is the most time-consuming
> routine; in general it seems that the PC is always the most time-consuming.
> 
> Any advice on how to get OpenMP working?
> 
> Regards,
> ?smund
> 
> 
> 
> -- PETSc Performance
> Summary: --
> 
> ./run on a arch-linux2-c-opt named vsl161 with 1 processor, by asmunder
> Wed Mar  6 10:14:55 2013
> Using Petsc Development HG revision:
> 58cc6199509f1642f637843f1ca468283bf5ced9  HG Date: Wed Jan 30 00:39:35
> 2013 -0600
> 
> Max   Max/MinAvg  Total
> Time (sec):   4.446e+02  1.0   4.446e+02
> Objects:  2.017e+03  1.0   2.017e+03
> Flops:3.919e+11  1.0   3.919e+11  3.919e+11
> Flops/sec:8.815e+08  1.0   8.815e+08  8.815e+08
> MPI Messages: 0.000e+00  0.0   0.000e+00  0.000e+00
> MPI Message Lengths:  0.000e+00  0.0   0.000e+00  0.000e+00
> MPI Reductions:   2.818e+03  1.0
> 
> Flop counting convention: 1 flop = 1 real number operation of type
> (multiply/divide/add/subtract)
>e.g., VecAXPY() for real vectors of length N
> --> 2N flops
>and VecAXPY() for complex vectors of length
> N --> 8N flops
> 
> Summary of Stages:   - Time --  - Flops -  --- Messages
> ---  -- Message Lengths --  -- Reductions --
>Avg %Total Avg %Total   counts
> %Total Avg %Total   counts   %Total
> 0:  Main Stage: 4.4460e+02 100.0%  3.9191e+11 100.0%  0.000e+00
> 0.0%  0.000e+000.0%  2.817e+03 100.0%
> 
> 
> See the 'Profiling' chapter of the users' manual for details on
> interpreting output.
> Phase summary info:
>   Count: number of times phase was executed
>   Time and Flops: Max - maximum over all processors
>   Ratio - ratio of maximum to minimum over all processors
>   Mess: number of messages sent
>   Avg. len: average message length (bytes)
>   Reduct: number of global reductions
>   Global: entire computation
>   Stage: stages of a computation. Set stages with PetscLogStagePush()
> and PetscLogStagePop().
>  %T - percent time in this phase %f - percent flops in this
> phase
>  %M - percent messages in this phase %L - percent message
> lengths in this phase
>  %R - percent reductions in this phase
>   Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time
> over all processors)
> 
> EventCount  Time (sec) Flops
> --- Global ---  --- Stage ---   Total
>   Max Ratio  Max Ratio   Max  Ratio  Mess   Avg len
> Reduct  %T %f %M %L %R  %T %f %M %L %R Mflop/s
> --

[petsc-dev] Some questions about adding default global to local, etc., functions for DMShell

[Barry Smith]

On Mar 6, 2013, at 2:18 PM, Matthew Knepley  wrote:

> On Wed, Mar 6, 2013 at 2:54 PM, Richard Tran Mills  
> wrote:
> Hi Folks,
> 
> I am adding some pointers to scatter contexts within the internal "data" 
> struct for DMShell, e.g. the 'gtol' and 'ltog' members in
> 
> typedef struct  {
>   Vec Xglobal;
>   Vec Xlocal;
>   Mat A;
>   VecScatter *gtol;
>   VecScatter *ltog;
> } DM_Shell;
> 
> First question: When in DMCreate_Shell we do
> 
> ierr = PetscNewLog(dm,DM_Shell,&shell);CHKERRQ(ierr);
> 
> will this guarantee that gtol and ltog are set to NULL, or do I need to 
> explicitly set these somewhere?  I see that the allocated memory is zeroed 
> but I'm not sure how NULL might actually be represented in memory.
> 
> Yes, they will be NULL.
>  
> I have also added some new functions like 
> DMShellDefaultGlobalToLocalBegin()/End(), and in DMCreate_Shell I do 
> 
>   dm->ops->globaltolocalbegin = DMShellDefaultGlobalToLocalBegin;
>   dm->ops->globaltolocalend   = DMShellDefaultGlobalToLocalEnd;
> 
   Bad names. Since they are not user functions they should have an underscore 
in them and specification of implementation after the underscore. Perhaps  
DMGlobalToLocalBegin_Shell_Default?

Check out the PETSc developers manual style guide (which is now chapter one 
so people actually read it :-)) Section 1.1 and 1.2 
http://www.mcs.anl.gov/petsc/developers/developers.pdf


   Barry



> The user can set different routines via DMShellSetGlobalToLocalBegin()/End(), 
> but if the user only sets the gtol and ltog scatter contexts, then the 
> default routines will perform the appropriate scatter operations.
> 
> Second question: What is the proper "PETSc way" to throw an error if gtol is 
> NULL and tell the user that DMShellDefaultGlobalToLocalBegin() cannot be used 
> without first setting the scatter context via 
> DMShellSetGlobalToLocalVecScatter()?
> 
> SETERRQ(comm, PETSC_ERR_ARG_WRONGSTATE, "DMShellDefaultGlobalToLocalBegin() 
> cannot be used without first setting the scatter context via 
> DMShellSetGlobalToLocalVecScatter()");
> 
>Matt
>  
> Thanks,
> Richard
> 
> 
> 
> -- 
> Richard Tran Mills, Ph.D.
> Computational Earth Scientist  | Joint Assistant Professor
> Hydrogeochemical Dynamics Team | EECS and Earth & Planetary Sciences
> Oak Ridge National Laboratory  | University of Tennessee, Knoxville
> E-mail: 
> rmills at ornl.gov  V: 865-241-3198 http://climate.ornl.gov/~rmills
> 
> 
> 
> -- 
> What most experimenters take for granted before they begin their experiments 
> is infinitely more interesting than any results to which their experiments 
> lead.
> -- Norbert Wiener

[petsc-dev] Some questions about adding default global to local, etc., functions for DMShell

[Matthew Knepley]

On Wed, Mar 6, 2013 at 2:54 PM, Richard Tran Mills  wrote:

>  Hi Folks,
>
> I am adding some pointers to scatter contexts within the internal "data"
> struct for DMShell, e.g. the 'gtol' and 'ltog' members in
>
> typedef struct  {
>   Vec Xglobal;
>   Vec Xlocal;
>   Mat A;
>   VecScatter *gtol;
>   VecScatter *ltog;
> } DM_Shell;
>
> First question: When in DMCreate_Shell we do
>
> ierr = PetscNewLog(dm,DM_Shell,&shell);CHKERRQ(ierr);
>
> will this guarantee that gtol and ltog are set to NULL, or do I need to
> explicitly set these somewhere?  I see that the allocated memory is zeroed
> but I'm not sure how NULL might actually be represented in memory.
>

Yes, they will be NULL.


> I have also added some new functions like
> DMShellDefaultGlobalToLocalBegin()/End(), and in DMCreate_Shell I do
>
>   dm->ops->globaltolocalbegin = DMShellDefaultGlobalToLocalBegin;
>   dm->ops->globaltolocalend   = DMShellDefaultGlobalToLocalEnd;
>
> The user can set different routines via
> DMShellSetGlobalToLocalBegin()/End(), but if the user only sets the gtol
> and ltog scatter contexts, then the default routines will perform the
> appropriate scatter operations.
>
> Second question: What is the proper "PETSc way" to throw an error if gtol
> is NULL and tell the user that DMShellDefaultGlobalToLocalBegin() cannot be
> used without first setting the scatter context via
> DMShellSetGlobalToLocalVecScatter()?
>

SETERRQ(comm, PETSC_ERR_ARG_WRONGSTATE, "DMShellDefaultGlobalToLocalBegin()
cannot be used without first setting the scatter context via
DMShellSetGlobalToLocalVecScatter()");

   Matt


> Thanks,
> Richard
>
>
>
> --
> Richard Tran Mills, Ph.D.
> Computational Earth Scientist  | Joint Assistant Professor
> Hydrogeochemical Dynamics Team | EECS and Earth & Planetary Sciences
> Oak Ridge National Laboratory  | University of Tennessee, Knoxville
> E-mail: rmills at ornl.gov  V: 865-241-3198 http://climate.ornl.gov/~rmills
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/32951c50/attachment.html>

[petsc-dev] Some questions about adding default global to local, etc., functions for DMShell

[Richard Tran Mills]

Hi Folks,

I am adding some pointers to scatter contexts within the internal "data" 
struct for DMShell, e.g. the 'gtol' and 'ltog' members in

typedef struct  {
   Vec Xglobal;
   Vec Xlocal;
   Mat A;
   VecScatter *gtol;
   VecScatter *ltog;
} DM_Shell;

First question: When in DMCreate_Shell we do

ierr = PetscNewLog(dm,DM_Shell,&shell);CHKERRQ(ierr);

will this guarantee that gtol and ltog are set to NULL, or do I need to 
explicitly set these somewhere?  I see that the allocated memory is 
zeroed but I'm not sure how NULL might actually be represented in memory.

I have also added some new functions like 
DMShellDefaultGlobalToLocalBegin()/End(), and in DMCreate_Shell I do

   dm->ops->globaltolocalbegin = DMShellDefaultGlobalToLocalBegin;
   dm->ops->globaltolocalend   = DMShellDefaultGlobalToLocalEnd;

The user can set different routines via 
DMShellSetGlobalToLocalBegin()/End(), but if the user only sets the gtol 
and ltog scatter contexts, then the default routines will perform the 
appropriate scatter operations.

Second question: What is the proper "PETSc way" to throw an error if 
gtol is NULL and tell the user that DMShellDefaultGlobalToLocalBegin() 
cannot be used without first setting the scatter context via 
DMShellSetGlobalToLocalVecScatter()?

Thanks,
Richard



-- 
Richard Tran Mills, Ph.D.
Computational Earth Scientist  | Joint Assistant Professor
Hydrogeochemical Dynamics Team | EECS and Earth & Planetary Sciences
Oak Ridge National Laboratory  | University of Tennessee, Knoxville
E-mail: rmills at ornl.gov  V: 865-241-3198 http://climate.ornl.gov/~rmills

-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/3b8c48c8/attachment.html>

[petsc-dev] meaning of PETSC_USE_EXTERN_CXX?

[Jed Brown]

On Wed, Mar 6, 2013 at 10:33 AM, Barry Smith  wrote:

>Are we ready to move this "branch" back into petsc-dev?   (Aside from
> the visibility stuff which is not yet mainstream and won't affect users, I
> think it is ready to go).


Yeah, I just merged it.
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/9e982283/attachment.html>

[petsc-dev] meaning of PETSC_USE_EXTERN_CXX?

[Jed Brown]

On Tue, Mar 5, 2013 at 10:14 PM, Jed Brown  wrote:

> Okay, ready for testing.
>
>
> https://bitbucket.org/BarryFSmith/petsc-dev-simp/commits/8d2ebbb193fb583bccc64015e35640c4e08c3426


Merged to petsc-dev now.

https://bitbucket.org/petsc/petsc-dev/commits/f848d02318cae92d7b32037c7ee88f92dbe46347
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/f986053b/attachment-0001.html>

[petsc-dev] meaning of PETSC_USE_EXTERN_CXX?

[Barry Smith]

   Are we ready to move this "branch" back into petsc-dev?   (Aside from the 
visibility stuff which is not yet mainstream and won't affect users, I think it 
is ready to go).

Barry

On Mar 6, 2013, at 12:10 AM, Jed Brown  wrote:

> 
> On Tue, Mar 5, 2013 at 11:57 PM, Barry Smith  wrote:
> This is very nice, much better than before, no confusing macros and 
> complicated #if conditions, simpler for the users to understand also.
> 
>   Does this mean we shouldn't/don't need to/should continue to mark functions 
> used in a single file as static?
> 
> If we use --with-visibility, unadorned functions behave like PETSC_INTERN 
> except that their names will be mangled for clanguage=C++. (The symbols 
> aren't visible outside the library so that doesn't matter, except to language 
> purists and possibly C++ exception propagation.) I would still prefer to use 
> 'static' whenever possible because it makes it easier to determine how much 
> code could depend on that function.
>  
>   I found a few problems so far
> 
>0) Do these really need PETSC_EXTERN? Note that they are not declared 
> PETSC_EXTERN when defined, only in snesimpl.h
> 
> PETSC_EXTERN PetscErrorCode SNESReset_VI(SNES);
> PETSC_EXTERN PetscErrorCode SNESDestroy_VI(SNES);
> PETSC_EXTERN PetscErrorCode SNESView_VI(SNES,PetscViewer);
> PETSC_EXTERN PetscErrorCode SNESSetFromOptions_VI(SNES);
> PETSC_EXTERN PetscErrorCode SNESSetUp_VI(SNES);
> PETSC_EXTERN_TYPEDEF typedef PetscErrorCode 
> (*SNESVIComputeVariableBoundsFunction)(SNES,Vec,Vec);
> PETSC_EXTERN PetscErrorCode 
> SNESVISetComputeVariableBounds_VI(SNES,SNESVIComputeVariableBoundsFunction);
> PETSC_EXTERN PetscErrorCode SNESVISetVariableBounds_VI(SNES,Vec,Vec);
> PETSC_EXTERN PetscErrorCode 
> SNESDefaultConverged_VI(SNES,PetscInt,PetscReal,PetscReal,PetscReal,SNESConvergedReason*,void*);
> 
> They can all become PETSC_INTERN unless we have reason to support plugin 
> authors or petscts calling those directly instead of using an interface.
> 
> 

[petsc-dev] question about Krylov solves, symmetric matrices, and ICC

[Jed Brown]

On Wed, Mar 6, 2013 at 10:05 AM, Paul Mullowney  wrote:

> Understood. But, the petsc ILU storage for L and U is non traditional
>
> In ILU, all of L is stored and then U is stored. For U, the diagonal
> element of a row in U is placed at the end of each row (to get better cache
> performance in the solve).
>

This optimization is a relationship between L and U.

For Cholesky, there is only U.
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/d754350c/attachment.html>

[petsc-dev] question about Krylov solves, symmetric matrices, and ICC

[Jed Brown]

On Wed, Mar 6, 2013 at 9:57 AM, Paul Mullowney  wrote:

> This works. Thanks.
>
> Does the Cholesky/ICC factorization have a specialized storage i.e. like
> the ILU storage?
>

There is only one triangular factor so it's not "special". You could write
an ICC that duplicated storage by copying the factor to optimize
back-solves for streaming, but it's of very little value compared to just
using ILU.
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/9553f77f/attachment.html>

[petsc-dev] question about Krylov solves, symmetric matrices, and ICC

[Hong Zhang]

Paul :

>
> Does the Cholesky/ICC factorization have a specialized storage i.e. like the
> ILU storage?

Cholesky/ICC only stores upper half (U) matrix, not L.
ILU stores L and U.

Hong
>
> -Paul
>
>> Insert it as the upper triangular part in a MATSBAIJ, then everything you
>> want will work.
>
>

[petsc-dev] OpenMP in PETSc when calling from Fortran?

[Åsmund Ervik]

Hi again,

On 01. mars 2013 20:06, Jed Brown wrote:
> 
> Matrix and vector operations are probably running in parallel, but probably
> not the operations that are taking time. Always send -log_summary if you
> have a performance question.
> 

I don't think they are running in parallel. When I analyze my code in
Intel Vtune Amplifier, the only routines running in parallel are my own
OpenMP ones. Indeed, if I comment out my OpenMP pragmas and recompile my
code, it never uses more than one thread.

-log_summary is shown below; this is using -pc_type lu -ksp_type bcgs.
The fastest PC for my cases is usually BoomerAMG from HYPRE, so i used
LU instead here in order to limit the test to PETSc only. The summary
agrees with Vtune that MatLUFactorNumeric is the most time-consuming
routine; in general it seems that the PC is always the most time-consuming.

Any advice on how to get OpenMP working?

Regards,
?smund



-- PETSc Performance
Summary: --

./run on a arch-linux2-c-opt named vsl161 with 1 processor, by asmunder
Wed Mar  6 10:14:55 2013
Using Petsc Development HG revision:
58cc6199509f1642f637843f1ca468283bf5ced9  HG Date: Wed Jan 30 00:39:35
2013 -0600

 Max   Max/MinAvg  Total
Time (sec):   4.446e+02  1.0   4.446e+02
Objects:  2.017e+03  1.0   2.017e+03
Flops:3.919e+11  1.0   3.919e+11  3.919e+11
Flops/sec:8.815e+08  1.0   8.815e+08  8.815e+08
MPI Messages: 0.000e+00  0.0   0.000e+00  0.000e+00
MPI Message Lengths:  0.000e+00  0.0   0.000e+00  0.000e+00
MPI Reductions:   2.818e+03  1.0

Flop counting convention: 1 flop = 1 real number operation of type
(multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N
--> 2N flops
and VecAXPY() for complex vectors of length
N --> 8N flops

Summary of Stages:   - Time --  - Flops -  --- Messages
---  -- Message Lengths --  -- Reductions --
Avg %Total Avg %Total   counts
%Total Avg %Total   counts   %Total
 0:  Main Stage: 4.4460e+02 100.0%  3.9191e+11 100.0%  0.000e+00
0.0%  0.000e+000.0%  2.817e+03 100.0%


See the 'Profiling' chapter of the users' manual for details on
interpreting output.
Phase summary info:
   Count: number of times phase was executed
   Time and Flops: Max - maximum over all processors
   Ratio - ratio of maximum to minimum over all processors
   Mess: number of messages sent
   Avg. len: average message length (bytes)
   Reduct: number of global reductions
   Global: entire computation
   Stage: stages of a computation. Set stages with PetscLogStagePush()
and PetscLogStagePop().
  %T - percent time in this phase %f - percent flops in this
phase
  %M - percent messages in this phase %L - percent message
lengths in this phase
  %R - percent reductions in this phase
   Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time
over all processors)

EventCount  Time (sec) Flops
 --- Global ---  --- Stage ---   Total
   Max Ratio  Max Ratio   Max  Ratio  Mess   Avg len
Reduct  %T %f %M %L %R  %T %f %M %L %R Mflop/s


--- Event Stage 0: Main Stage

VecDot   802 1.0 9.2811e-02 1.0 1.96e+08 1.0 0.0e+00 0.0e+00
0.0e+00  0  0  0  0  0   0  0  0  0  0  2117
VecDotNorm2  401 1.0 7.1333e-02 1.0 1.96e+08 1.0 0.0e+00 0.0e+00
4.0e+02  0  0  0  0 14   0  0  0  0 14  2755
VecNorm 1203 1.0 7.8265e-02 1.0 2.95e+08 1.0 0.0e+00 0.0e+00
0.0e+00  0  0  0  0  0   0  0  0  0  0  3766
VecCopy  802 1.0 1.1754e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00
0.0e+00  0  0  0  0  0   0  0  0  0  0 0
VecSet  1211 1.0 9.9961e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00
0.0e+00  0  0  0  0  0   0  0  0  0  0 0
VecAXPY  401 1.0 4.5847e-02 1.0 9.82e+07 1.0 0.0e+00 0.0e+00
0.0e+00  0  0  0  0  0   0  0  0  0  0  2143
VecAXPBYCZ   802 1.0 1.3489e-01 1.0 3.93e+08 1.0 0.0e+00 0.0e+00
0.0e+00  0  0  0  0  0   0  0  0  0  0  2913
VecWAXPY 802 1.0 1.2292e-01 1.0 1.96e+08 1.0 0.0e+00 0.0e+00
0.0e+00  0  0  0  0  0   0  0  0  0  0  1599
VecAssemblyBegin 802 1.0 2.4509e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00
0.0e+00  0  0  0  0  0   0  0  0  0  0 0
VecAssemblyEnd   802 1.0 6.7234e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00
0.0e+00  0  0  0  0  0   0  0  0  0  0 0
MatMu

[petsc-dev] question about Krylov solves, symmetric matrices, and ICC

[Paul Mullowney]

On 03/06/2013 09:09 AM, Jed Brown wrote:
>
> On Wed, Mar 6, 2013 at 9:57 AM, Paul Mullowney  <mailto:paulm at txcorp.com>> wrote:
>
> This works. Thanks.
>
> Does the Cholesky/ICC factorization have a specialized storage
> i.e. like the ILU storage?
>
>

This answers my question. Thanks. Agreed that ILU is the way to go.

> There is only one triangular factor so it's not "special". You could 
> write an ICC that duplicated storage by copying the factor to optimize 
> back-solves for streaming, but it's of very little value compared to 
> just using ILU.

-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/b69820a3/attachment.html>

[petsc-dev] question about Krylov solves, symmetric matrices, and ICC

[Paul Mullowney]

Understood. But, the petsc ILU storage for L and U is non traditional

In ILU, all of L is stored and then U is stored. For U, the diagonal 
element of a row in U is placed at the end of each row (to get better 
cache performance in the solve).

I'm wondering if anything similar happens in Cholesky where the 
diagonals are placed at the end of each row.

-Paul

> Cholesky/ICC only stores upper half (U) matrix, not L.
> ILU stores L and U.
>
> Hong
>> -Paul
>>
>>> Insert it as the upper triangular part in a MATSBAIJ, then everything you
>>> want will work.
>>

[petsc-dev] question about Krylov solves, symmetric matrices, and ICC

[Paul Mullowney]

This works. Thanks.

Does the Cholesky/ICC factorization have a specialized storage i.e. like 
the ILU storage?

-Paul
> Insert it as the upper triangular part in a MATSBAIJ, then everything 
> you want will work.

[petsc-dev] meaning of PETSC_USE_EXTERN_CXX?

[Matthew Knepley]

On Wed, Mar 6, 2013 at 7:49 AM, Jed Brown  wrote:

>
> On Wed, Mar 6, 2013 at 5:58 AM, Matthew Knepley  wrote:
>
>> The whole idea of compiler flags is that the compiler understands them. I
>> think its a bad model to have
>> flags that might work on one file and not another. That just seems like a
>> nightmare.
>>
>
> -fvisibility=hidden causes functions to be hidden by default unless (a)
> they are explicitly 'extern' or (b) their visibility is set explicitly.
> This is like the Windows default so it's not exactly esoteric, but most
> packages will not build shared libraries correctly with this flag.
>
> Note that Open MPI sets visibility=hidden by default on systems that
> support it. Now that we have PETSC_INTERN, we have an explicit way of
> stating when symbols are meant to be internal to PETSc. If every function
> in PETSc uses either PETSC_EXTERN, PETSC_INTERN, or static, then
> -fvisibility=hidden will not change anything so we could skip it.
>
>
>> If there really are compiler flags that can break projects, they should
>> be removed from compilerOptions.py, and
>> put directly in PETSc/Configure.py and added into the output in Dump().
>>
>
> There is no way to test it from BuildSystem without affecting the way all
> packages are built?
>

This is not about testing at all, its about using. You explicitly stick it
in CFLAGS.

   Matt


> The visibility macros also should not be used when building user code
> (like examples) because we don't want to enforce the same discipline on
> them. Maybe we should test for visibility so that
> __attribute__((visibility("hidden"))) can be used, but leave off the
> compiler flag (except for developer builds to enforce that all exports are
> explicit---anything that fails will also fail on Windows).
>



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/1ab943cc/attachment-0001.html>

[petsc-dev] meaning of PETSC_USE_EXTERN_CXX?

[Jed Brown]

On Wed, Mar 6, 2013 at 7:24 AM, Matthew Knepley  wrote:

> This is not about testing at all, its about using. You explicitly stick it
> in CFLAGS.
>

I wanted to keep the code that tested whether the compiler understands
__attribute__((visibility("default"))) and -fvisibility=hidden in
BuildSystem so that it could be reused from other projects.

The option should probably be split into --with-visibility which defaults
to true when available and --with-visibility-hidden which compiles with
-fvisibility=hidden and might only be used for our testing.
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/f150b9e4/attachment.html>

[petsc-dev] meaning of PETSC_USE_EXTERN_CXX?

[Matthew Knepley]

On Wed, Mar 6, 2013 at 12:57 AM, Barry Smith  wrote:

>
>   This is very nice, much better than before, no confusing macros and
> complicated #if conditions, simpler for the users to understand also.
>
>   Does this mean we shouldn't/don't need to/should continue to mark
> functions used in a single file as static?
>
>   I found a few problems so far
>
>0) Do these really need PETSC_EXTERN? Note that they are not declared
> PETSC_EXTERN when defined, only in snesimpl.h
>
> PETSC_EXTERN PetscErrorCode SNESReset_VI(SNES);
> PETSC_EXTERN PetscErrorCode SNESDestroy_VI(SNES);
> PETSC_EXTERN PetscErrorCode SNESView_VI(SNES,PetscViewer);
> PETSC_EXTERN PetscErrorCode SNESSetFromOptions_VI(SNES);
> PETSC_EXTERN PetscErrorCode SNESSetUp_VI(SNES);
> PETSC_EXTERN_TYPEDEF typedef PetscErrorCode
> (*SNESVIComputeVariableBoundsFunction)(SNES,Vec,Vec);
> PETSC_EXTERN PetscErrorCode
> SNESVISetComputeVariableBounds_VI(SNES,SNESVIComputeVariableBoundsFunction);
> PETSC_EXTERN PetscErrorCode SNESVISetVariableBounds_VI(SNES,Vec,Vec);
> PETSC_EXTERN PetscErrorCode
> SNESDefaultConverged_VI(SNES,PetscInt,PetscReal,PetscReal,PetscReal,SNESConvergedReason*,void*);
>
>
>1) arch-dynamic-cxx-split  (you can guess what these mean, split means
> split libraries) fails with gfortran on Mac. This may have been this way
> for a while, I'm guess it is related to how common blocks in shared
> libraries are not shared by default and nothing to do with recent changes.
>
>--with-visibility
>
>2) Fortran auto generated functions are not visible to user code
>
> Key in Matt's grumbling about sowing and how it should be
> rewritten in Python.  We need to modify bfort to insert visibility macro
> for the functions. Satish, can you look at that?
>
>3) The -fvisibility=hidden is blindly passed down to all
> --download-packages and will break them all. I fixed MPI.py to crudely
> remove it but we should have it auto-removed for all package builds. Matt,
> can you look at that?


The whole idea of compiler flags is that the compiler understands them. I
think its a bad model to have
flags that might work on one file and not another. That just seems like a
nightmare.

If there really are compiler flags that can break projects, they should be
removed from compilerOptions.py, and
put directly in PETSc/Configure.py and added into the output in Dump().

Matt


>
>Barry
>
>
>
> On Mar 5, 2013, at 10:14 PM, Jed Brown  wrote:
>
> >
> > On Tue, Mar 5, 2013 at 8:28 PM, Jed Brown  wrote:
> > On Tue, Mar 5, 2013 at 7:19 PM, Jed Brown  wrote:
> > I'm testing it now.
> >
> > I pushed two minor patches and I'm going through configure now.
> >
> >
> https://bitbucket.org/BarryFSmith/petsc-dev-simp/commits/41ad8d3b80c7a9edf4d80bf2e09f7fb8c0405f2b
> >
> > Okay, ready for testing.
> >
> >
> https://bitbucket.org/BarryFSmith/petsc-dev-simp/commits/8d2ebbb193fb583bccc64015e35640c4e08c3426
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/4f62f498/attachment.html>

[petsc-dev] meaning of PETSC_USE_EXTERN_CXX?

[Jed Brown]

On Wed, Mar 6, 2013 at 5:58 AM, Matthew Knepley  wrote:

> The whole idea of compiler flags is that the compiler understands them. I
> think its a bad model to have
> flags that might work on one file and not another. That just seems like a
> nightmare.
>

-fvisibility=hidden causes functions to be hidden by default unless (a)
they are explicitly 'extern' or (b) their visibility is set explicitly.
This is like the Windows default so it's not exactly esoteric, but most
packages will not build shared libraries correctly with this flag.

Note that Open MPI sets visibility=hidden by default on systems that
support it. Now that we have PETSC_INTERN, we have an explicit way of
stating when symbols are meant to be internal to PETSc. If every function
in PETSc uses either PETSC_EXTERN, PETSC_INTERN, or static, then
-fvisibility=hidden will not change anything so we could skip it.


> If there really are compiler flags that can break projects, they should be
> removed from compilerOptions.py, and
> put directly in PETSc/Configure.py and added into the output in Dump().
>

There is no way to test it from BuildSystem without affecting the way all
packages are built?

The visibility macros also should not be used when building user code (like
examples) because we don't want to enforce the same discipline on them.
Maybe we should test for visibility so that
__attribute__((visibility("hidden"))) can be used, but leave off the
compiler flag (except for developer builds to enforce that all exports are
explicit---anything that fails will also fail on Windows).
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/d99f3f42/attachment.html>

[petsc-dev] meaning of PETSC_USE_EXTERN_CXX?

[Jed Brown]

On Wed, Mar 6, 2013 at 12:09 AM, Sean Farley  wrote:

> Two places in a file? The following
>
> [paths]
> barry = bb://BarryFSmith/petsc-dev
>
> adds 'barry/default' (and other branches) as a local tag(s).
>

I was looking at this

http://mercurial.selenic.com/wiki/RemoteBranchesExtension

[paths]
default=http://hg.intevation.org/mercurial/crew
crew=http://hg.intevation.org/mercurial/crew
mpm=http://selenic.com/hg

[remotebranches]
upstream=crew,mpm


which looks like two places to me.


> Yes, that is why I forked remote-branches. So that I could remotely add
> bookmarks. But that's besides the point. You are muddying the waters
> here with talk of divergent bookmarks and Mercurial's internals. I'm
> trying to stick with the simpler model of pull-requests for now.
>

Pull requests are orthogonal to having multiple branches/bookmarks in a
repository. I'm saying that the latter still appears to be painful with hg.


>
> To me, having 300 branches to look at is a crazy amount of information
> overload.
>

That's why keeps them in the remote namespace and does not show them to you
unless you look for them. It is, however, much simpler than having 300
"remotebranches" coming from different repositories.


> Granted, what I'm suggesting is for you and other devs that want to work
> on people's branches to manually add it. I can understand that's
> cumbersome for you but it is pretty straight-forward and easy to explain
> to new devs.
>

It's less scalable, more error-prone, and when debugging remotely, it's
seems like we have to ask people to send the contents of their .hg/hgrc to
figure out how they are set up. We can try it, but I think it will not work
well.


> I'll argue that most of what you're mentioning is useless without
> context; i.e. if I want to be hacking on someone else's branch then I
> have a reason to be looking at their work and discussing it with
> them.
>

This is a cumbersome way to get a summary of what's going on.


> How is having http://bitbucket.org/petsc/petsc-next breaking this model?
> You'll have the remote forks of the devs you want to keep track of, then
> you can merge / bake things in petsc-next and edit the history when
> done.
>

How would I go about finding all feature branches that have been cooking in
next for more than a week, but are not yet merged to master? If someone
else merged it, I won't even be able to figure out from the history where
the merge commit came from. Note that I _never_ merge 'next' into 'master',
I only merge stable features.

If I'm rebuilding 'next' after a release, how do I make sure I have all the
branches that did not graduate for the release?


> You are doing a disservice here by sweeping under the rug the difficulty of
> switching an entire team to a new tool. Especially since what you are
> describing is achievable with mercurial: (a) create fork, (b) hack away,
>

1. Note that this "recommendation" is still broken:

https://bitbucket.org/petsc/petsc-dev/commits/featurebranches

2. If we are going to change workflow significantly from the status quo, I
think we should consider all the options. Sure, we may end up making only a
small incremental change now, but if it's going to be a long progression,
it may ultimately be better to do it all now.
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/13ceae95/attachment-0001.html>

[petsc-dev] meaning of PETSC_USE_EXTERN_CXX?

[Jed Brown]

On Tue, Mar 5, 2013 at 11:57 PM, Barry Smith  wrote:

> This is very nice, much better than before, no confusing macros and
> complicated #if conditions, simpler for the users to understand also.
>
>   Does this mean we shouldn't/don't need to/should continue to mark
> functions used in a single file as static?
>

If we use --with-visibility, unadorned functions behave like PETSC_INTERN
except that their names will be mangled for clanguage=C++. (The symbols
aren't visible outside the library so that doesn't matter, except to
language purists and possibly C++ exception propagation.) I would still
prefer to use 'static' whenever possible because it makes it easier to
determine how much code could depend on that function.


>   I found a few problems so far
>
>0) Do these really need PETSC_EXTERN? Note that they are not declared
> PETSC_EXTERN when defined, only in snesimpl.h
>
> PETSC_EXTERN PetscErrorCode SNESReset_VI(SNES);
> PETSC_EXTERN PetscErrorCode SNESDestroy_VI(SNES);
> PETSC_EXTERN PetscErrorCode SNESView_VI(SNES,PetscViewer);
> PETSC_EXTERN PetscErrorCode SNESSetFromOptions_VI(SNES);
> PETSC_EXTERN PetscErrorCode SNESSetUp_VI(SNES);
> PETSC_EXTERN_TYPEDEF typedef PetscErrorCode
> (*SNESVIComputeVariableBoundsFunction)(SNES,Vec,Vec);
> PETSC_EXTERN PetscErrorCode
> SNESVISetComputeVariableBounds_VI(SNES,SNESVIComputeVariableBoundsFunction);
> PETSC_EXTERN PetscErrorCode SNESVISetVariableBounds_VI(SNES,Vec,Vec);
> PETSC_EXTERN PetscErrorCode
> SNESDefaultConverged_VI(SNES,PetscInt,PetscReal,PetscReal,PetscReal,SNESConvergedReason*,void*);
>

They can all become PETSC_INTERN unless we have reason to support plugin
authors or petscts calling those directly instead of using an interface.
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130306/9b9f0e38/attachment.html>

[petsc-dev] meaning of PETSC_USE_EXTERN_CXX?

[Sean Farley]

Jed Brown writes:

> On Tue, Mar 5, 2013 at 10:19 PM, Sean Farley  wrote:
>
>> > This does not appear to namespace the branches and it looks like you have
>> > to edit .hg/hgrc for each remote that you want to track. It is quite
>> > different from namespaced branches/bookmarks.
>>
>> Sure, but only the person wanting to pull has to add a path to their
>> .hgrc which is extremely easy to understand and do.
>>
>
> It's two places in a file, compared to 'git remote add my-remote-name
> the.url:repo'.

Two places in a file? The following

[paths]
barry = bb://BarryFSmith/petsc-dev

adds 'barry/default' (and other branches) as a local tag(s).

I could implement 'hg pull bb://seanfarley/petsc-experiment --add
foo' (I made up those options just now; haven't decided how to present
them to the user yet) to save a command, if you really want, but that's
on the back-burner for now.

> The remote bookmarks are still not namespaced so I still can't safely fetch
> from all remotes before going offline (and even if I'm online, many
> operations are relatively slow). After 'hg pull somewhere' (rather than
> pulling a specific branch/bookmark), I don't have a way to keep track of
> which bookmarks came from where. At least divergent conflicts give warnings
> "divergent bookmark yyy stored as yyy at default", but other bookmarks are
> automatically fast-forwarded and new bookmarks are added (I can't find an
> option to treat everything as a "divergent" bookmark even when it could
> fast-forward. I'd much prefer that.)
>
> This is much "too global" management of bookmarks if you ask me.

Yes, that is why I forked remote-branches. So that I could remotely add
bookmarks. But that's besides the point. You are muddying the waters
here with talk of divergent bookmarks and Mercurial's internals. I'm
trying to stick with the simpler model of pull-requests for now.

>> You misunderstand me. I think branches/bookmarks are pretty cool and
>> useful. I also think using changeset evolution (to achieve the shared
>> garbage-collection of git's unrefed heads) is cool, too. But those
>> workflows
>> are advanced and require each user of the repo to conform, without
>> buying all *that* much more than the fork / pull-request model.
>>
>
> With a git repository, those branches really do not affect the user. Try
> this

[snip]

> So I can easily browse ongoing projects ('git log gitster/ap/log-mailmap')
> and I can work on them ('git checkout ap/log-mailmap', hack away...) but
> unless I'm directly interacting with a branch, I'm totally unaffected by
> those ongoing projects and never see them in my normal workflow ('git log',
> 'git branch', etc).

To me, having 300 branches to look at is a crazy amount of information
overload. Plus, it's a huge change in workflow. Hence, I am trying to
nudge people incrementally into the pull-request model. It won't shatter
people's current workflow (even those that use mercurial as a
local-subversion-type of repo). Not breaking people's current workflow
(no matter how much you dislike it) seems like a huge win to me.

Granted, what I'm suggesting is for you and other devs that want to work
on people's branches to manually add it. I can understand that's
cumbersome for you but it is pretty straight-forward and easy to explain
to new devs.

I'll argue that most of what you're mentioning is useless without
context; i.e. if I want to be hacking on someone else's branch then I
have a reason to be looking at their work and discussing it with
them. Or if I'm assigned to review their patches, then I have to pull
them from somewhere. That's either their own fork or some bookmark or
some email thread.

> If we are going to consider allowing topic branches to "cook in next" for a
> while (to get tested in interaction with the rest of the system) before
> being merged to a stable branch, then we need to keep these bookmarks
> around somehow, yet it's unacceptable for them to intrude on normal
> workflow. I still haven't heard of a viable plan in Hg's roadmap.

How is having http://bitbucket.org/petsc/petsc-next breaking this model?
You'll have the remote forks of the devs you want to keep track of, then
you can merge / bake things in petsc-next and edit the history when
done.

You can nitpick about this workflow, but I think it achieves quite a
bit; and as a bonus it's an opt-in model (i.e. only you and the devs
that want this workflow have to change; others can keep chugging along).

>> Instead, what I am suggesting is taking incremental steps in that
>> direction. The fork / pull-request model is pretty darn simple and
>> allows each developer to work with either workflow (or even some
>> hybrid). If Barry doesn't want to ever change his habits, then it's
>> fine. Others could pull, clean-up, and merge his work.
>>
>> On top of that, new developers (including people more comfortable with
>> subversion / cvs / centralized development) won't be bombarded with too
>> much and can work in their own fork as they please

[petsc-dev] meaning of PETSC_USE_EXTERN_CXX?

[Barry Smith]

  nm -o arch-dynamic-cxx-split-visibility/lib/libpetsc*.dylib  | grep " T " | 
grep "[A-Za-z]_[A-Za-z]" | grep -v Create

   finds a few other oddball public functions that maybe don't need to be 
public. Do we need to just go through and replace the PETSC_EXTERN for those 
that don't need it?

   I had made things like _MatConvert_MPIMAIJ_MPIAIJ PETSC_EXTERN_C because we 
use dlsym on a built string to find them.  But can we actually make them 
PETSC_INTERN now?

   Barry



On Mar 5, 2013, at 11:57 PM, Barry Smith  wrote:

> 
>  This is very nice, much better than before, no confusing macros and 
> complicated #if conditions, simpler for the users to understand also.
> 
>  Does this mean we shouldn't/don't need to/should continue to mark functions 
> used in a single file as static?
> 
>  I found a few problems so far 
> 
>   0) Do these really need PETSC_EXTERN? Note that they are not declared 
> PETSC_EXTERN when defined, only in snesimpl.h
> 
> PETSC_EXTERN PetscErrorCode SNESReset_VI(SNES);
> PETSC_EXTERN PetscErrorCode SNESDestroy_VI(SNES);
> PETSC_EXTERN PetscErrorCode SNESView_VI(SNES,PetscViewer);
> PETSC_EXTERN PetscErrorCode SNESSetFromOptions_VI(SNES);
> PETSC_EXTERN PetscErrorCode SNESSetUp_VI(SNES);
> PETSC_EXTERN_TYPEDEF typedef PetscErrorCode 
> (*SNESVIComputeVariableBoundsFunction)(SNES,Vec,Vec);
> PETSC_EXTERN PetscErrorCode 
> SNESVISetComputeVariableBounds_VI(SNES,SNESVIComputeVariableBoundsFunction);
> PETSC_EXTERN PetscErrorCode SNESVISetVariableBounds_VI(SNES,Vec,Vec);
> PETSC_EXTERN PetscErrorCode 
> SNESDefaultConverged_VI(SNES,PetscInt,PetscReal,PetscReal,PetscReal,SNESConvergedReason*,void*);
> 
> 
>   1) arch-dynamic-cxx-split  (you can guess what these mean, split means 
> split libraries) fails with gfortran on Mac. This may have been this way for 
> a while, I'm guess it is related to how common blocks in shared libraries are 
> not shared by default and nothing to do with recent changes.
> 
>   --with-visibility
> 
>   2) Fortran auto generated functions are not visible to user code 
> 
>Key in Matt's grumbling about sowing and how it should be rewritten in 
> Python.  We need to modify bfort to insert visibility macro for the 
> functions. Satish, can you look at that?
> 
>   3) The -fvisibility=hidden is blindly passed down to all 
> --download-packages and will break them all. I fixed MPI.py to crudely remove 
> it but we should have it auto-removed for all package builds. Matt, can you 
> look at that?
> 
>   Barry
> 
> 
> 
> On Mar 5, 2013, at 10:14 PM, Jed Brown  wrote:
> 
>> 
>> On Tue, Mar 5, 2013 at 8:28 PM, Jed Brown  wrote:
>> On Tue, Mar 5, 2013 at 7:19 PM, Jed Brown  wrote:
>> I'm testing it now.
>> 
>> I pushed two minor patches and I'm going through configure now.
>> 
>> https://bitbucket.org/BarryFSmith/petsc-dev-simp/commits/41ad8d3b80c7a9edf4d80bf2e09f7fb8c0405f2b
>> 
>> Okay, ready for testing.
>> 
>> https://bitbucket.org/BarryFSmith/petsc-dev-simp/commits/8d2ebbb193fb583bccc64015e35640c4e08c3426
>