On Thu, 22 Oct 2015, Barry Smith wrote: > > > On Oct 22, 2015, at 7:28 PM, Satish Balay <ba...@mcs.anl.gov> wrote: > > > > The current infrastruture tests --download-petsc4py in petsc 'next' > > and 'master' branches. > > > > Wrt next - usually a change in a feature branch (when it gets merged > > to next) triggers this breakage. And usual thing to do is to fix it > > in the feature branch (and merge to next again) > > > > Wrt to petsc4py - this would mean updating petsc4py.py [in petsc] with > > the petsc4py repo gitcommit [optionally also tarball ] that > > corresponds to the fix. [one way to handle this on petsc4py repo is to > > create a feature branch for this fix] > > Yeah this is nuts, having to maintaining matching branches between petsc > and petsc4py and merge them at the same time into next and then master. > Clearly no one will ever do this and hence petsc4py will always be out of > step from PETSc. I do really believe that incorporating petsc4py into PETSc > would just make all these difficulties go away. For example maintaining Tao > is a breeze now since it is updated in the same branches as PETSc. >
Perhaps I should rephrase the issue in the following way: We should decide on what level petsc4py and petsc should be synced. [and the testsuite should be modified appropriately for that] Currently its done at next, master branchs. For this to work - the above - previously mentioned workflow is required. [and for this workflow - a single repo does makes thing easier] We could say sync should happen only between petsc4py-master & petsc-master. [so issues - if any in feature branches/next wont't be dealt with - and its ok for the master branches to be out of sync for a brief time for the fix to propergate]. For this - we would have a different workflow [wrt syncing petsc4py wrt petsc - and a slightly different test suite - testing only master. [We could continue having the curent test suite - but ignore errors in next]. And e-mails to Lisandro will trigger only on petsc4py errors on master. Lisandro would then fix petsc4py (master) - and then update petsc4py.py in petsc-master. Or perhaps some other mode is preferable.. Satish > Barry > > > > > When this (petsc) feature branch is merged into master - the > > corresponding feature branch on the petsc4py side can be merged into > > petsc4py master [if petsc4py-master is to be in sync with > > petsc-master]. > > > > [if there are multiple feature branches on the petsc side requiring > > corresponding petsc4py changes - somehow that has to be handled] > > > > Perhaps testing petsc4py via petsc infrastructure [using > > --download-petsc4py] is not the appropriate approach? > > > > Or we should do this only on master - not next? [or just have a > > single --download-petsc4py test - thats not mixed with other tests? - > > so its breakage is not critical for other things] > > > > Also we don't currently run any petsc4py tests. [we just test if it > > compiles or not] > > > > Satish > > > > On Thu, 22 Oct 2015, Barry Smith wrote: > > > >> > >> Lisandro, > >> > >> It is tested nightly but there is apparently currently no email sent on > >> failure. See the bottom of, for example, > >> http://ftp.mcs.anl.gov/pub/petsc/nightlylogs/archive/2015/10/22/make_next_arch-linux-pkgs-opt_crank.log > >> > >> I think what should happen is the petsc4py log file should be published > >> to the web (like the other log files) and a script look through it and > >> decide to send email if it detects a problem. Like we currently have for > >> the other log files. Someone has to dig into the nightly test > >> infrastructure to figure out where the new logic needs to go. Currently > >> Satish and Karl understand that infrastructure the best. > >> > >> Barry > >> > >> > >> > >> > >> > >>> On Oct 22, 2015, at 6:44 PM, Lisandro Dalcin <dalc...@gmail.com> wrote: > >>> > >>> On 22 October 2015 at 23:20, Barry Smith <bsm...@mcs.anl.gov> wrote: > >>>> > >>>> Lisandro, > >>>> > >>>> You never responded to this. I assume it was because you did not like > >>>> the idea? This is not an attempt to take control of petsc4py or to take > >>>> credit away from petsc4py from you. Petsc4py is your package and you > >>>> will always be the one who deserves the credit. > >>>> > >>> > >>> Oh, sorry! I missed this email. > >>> > >>> > >>>> The problem is that mutual efficient development of very intertwined > >>>> packages is difficult if they are in different respositories. For > >>>> example petsc4py has been out of date with petsc master for many weeks > >>>> now because it is too much effort to update the petsc4py repository each > >>>> time we make some change in PETSc. Thus it becomes a burden on you to go > >>>> in every once in a while and fix up petsc4py. With one repository > >>>> changes would happen in both PETSc source and petsc4py source at the > >>>> same time and tests would detect problems immediately. > >>>> > >>>> Thoughts? > >>>> > >>> > >>> I general terms, I prefer to keep things separate as much a possible. > >>> IMHO, a much important step would be to incorporate petsc4py to the > >>> nightly tests, clearly separating the petsc and petsc4py tests runs. I > >>> would love to get an email every time petsc4py nightly breaks, this > >>> way I would be able to push fixes within a day or two, say. > >>> Maintaining petsc4py requires knowledge of PETSc+Python+Cython, so > >>> ultimately I'm the one in best position to do such work, if you guys > >>> start adding commits to petsc4py withing the main petsc repo, I'm > >>> likely going to miss these changes. For example, today Michael Lange > >>> made some changes, I got an email from Bitbucket, then did a review > >>> and spotted a few issues. > >>> > >>> My hesitation to incorporate petsc4py within petsc repo is, in > >>> Python's [and linear algebra :-)] Zen : "Sparse is better than dense". > >>> If we continue adding stuff to core PETSc, at same point we will endup > >>> like Trilinos ( DON'T MISS the sencond entry in the FAQ: > >>> https://trilinos.org/download/public-git-repository/). > >>> > >>> All that being said, I don't have any problem at all adding petsc4py > >>> to the main PETSc repository. But I still think this is not a good > >>> idea, it will not solve all the problems. Nightly tests with automatic > >>> notifications to me about breakages is perhaps is all what we need to > >>> keep petsc4py in close sync with petsc/master. What about trying this > >>> first to see how it goes? > >>> > >>> > >>> PS: Please understand my position has nothing to do with credits about > >>> the project or ego-related crap. Actually, your proposal would mean > >>> less responsibility and maintenance work for me. But I don't think it > >>> is a move in the right direction. > >>> > >>> > >>> -- > >>> Lisandro Dalcin > >>> ============ > >>> Research Scientist > >>> Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) > >>> Numerical Porous Media Center (NumPor) > >>> King Abdullah University of Science and Technology (KAUST) > >>> http://numpor.kaust.edu.sa/ > >>> > >>> 4700 King Abdullah University of Science and Technology > >>> al-Khawarizmi Bldg (Bldg 1), Office # 4332 > >>> Thuwal 23955-6900, Kingdom of Saudi Arabia > >>> http://www.kaust.edu.sa > >>> > >>> Office Phone: +966 12 808-0459 > >> > >> > > > >
