Re: Lets maintain libbpp-phyl-omics in Debian Med team
Hi Julien, some of the libbpp packages now arrived in unstable. It would be great if you could try to reprocude the failures in the test cases. It might be sensible to rebuild libbpp-seq in a clean Debian unstable chroot. The build shows: # For the moment ignore: # 3/7 Test #3: test_io ..***Exception: Other 0.01 sec # terminate called after throwing an instance of 'bpp::IOException' # what(): Fasta::appendFromStream: can't read from istream input # # 86% tests passed, 1 tests failed out of 7 I'll push the missing libbpp packages to new meanwhile. Kind regards Andreas. On Mon, Apr 11, 2016 at 01:09:55PM +0200, Andreas Tille wrote: > Hi Julien, > > On Mon, Apr 11, 2016 at 12:19:29PM +0200, Julien Yann Dutheil wrote: > > > On Mon, Apr 11, 2016 at 09:10:00AM +0200, Julien Yann Dutheil wrote: > > > > I will look at the bug (and try to compile with gcc6 for a check of all > > > > libs. As for bpp-seq, I think it is because it needs some input files > > > that > > > > should be in the test folder too (eg example.fasta, example.aln and > > > > example.ph). > > > > > > I have verified that these files are existing in the very same folder as > > > the test code. I'm not sure whether this is the correct location for > > > these files since may be the test code will be started from a different > > > folder (I tried a patch adding "test/" in front of the file names but > > > this did not helped. I also tried deactivating single tests inside > > > test_io - also with no success. > > > > > > > They should be in the same directory as the program is launched (which is > > "test/" in my CMake script). > > OK, so it might be worth investigating why it might fail. Are you able > to reproduce this issue? > > > My plan is ultimately to deactivate unit > > testing by default as several users have complained about that, but in the > > mean time this can be done by adding BUILD_TESTING=OFF to the cmake command > > line. > > I would consider switching of the tests a way backward. Debian is > striving to get as most as possible package delivered with build time > tests as well as test of the packages on the Debian mirror in regular > intervals. Specifically for scientific software I'd regard this of high > relevance and thus the Debian Med team is running a Google Summer of > Code project to add more tests. > > I think the best way to cope with non passing tests is to give more > detailed output on failing tests to let users check what might be > wrong and make the tests more robust. > > There is no need to switch of the tests in general in the packaging as > you can see in the according debian/rules file[1] I have worked around > the issue and have provided an according documentation[2]. Hiding eyes > from non-passing tests should not be the solution here. > > Kind regards > > Andreas. > > > [1] > https://anonscm.debian.org/cgit/debian-med/libbpp-seq.git/tree/debian/rules > [2] > https://anonscm.debian.org/cgit/debian-med/libbpp-seq.git/tree/debian/README.source > > -- > http://fam-tille.de > > -- http://fam-tille.de
Re: Lets maintain libbpp-phyl-omics in Debian Med team
Hi Julien, On Wed, Apr 13, 2016 at 11:55:26AM +0200, Julien Yann Dutheil wrote: > Thanks a lot. Just need to know what I should include upstream to > facilitate the packaging of the next versions. I guess the more I can > include the easier it should be for you, right? No, please include no debian/ dir at all. That's the best "service" upstream can do for Debian packaging. (If you want more resoning about this simple statement I could seek for several links on our mailing lists - but it boils down to this simple statement.) Kind regards Andreas. -- http://fam-tille.de
Re: Lets maintain libbpp-phyl-omics in Debian Med team
Hi Julien, On Wed, Apr 13, 2016 at 10:00:19AM +0200, Julien Yann Dutheil wrote: > > I have started to compile the libs with gcc 6. So far I get no issue if I > add (-std=c++98) to the compiler arguments (-std=gnu14 begin the default in > g++6, as far as I understood). We're currently porting the libs to c++11 ad > the next release should be able to compile with C++14 without warning. Sounds good. > I can see that there was a suggestion for a rename of the libraries, why is > that so? (I always had difficulties understanding library numbering, > already from libtool :s) Where did you see such a rename - I don't understand what you mean. Or do you mean the rename of the *package*? This is due to libg++ ABI change and is documented in bug #791093 which will be closed by my upload. It has nothing to do with libtools and is part of a general transition in Debian. Since you have not dealt with these bugs all libbpp* packages have RC critical bugs and would not have been released with the next Debian release. > I'm not sure how to see the changes you made for fixing the conversion > issue (issue that I do not have)? Its basically the name change of the package, conflicts with the old name and a rebuild. You do not have this issue since you are not creating a distribution of several software packages needing to cope with an ABI change in a central library. In other words: You do not need to care about this (if you do not care about the packaging but only developing the upstream library code). Kind regards Andreas. -- http://fam-tille.de
Re: Lets maintain libbpp-phyl-omics in Debian Med team
Hi Julien, On Mon, Apr 11, 2016 at 12:19:29PM +0200, Julien Yann Dutheil wrote: > > On Mon, Apr 11, 2016 at 09:10:00AM +0200, Julien Yann Dutheil wrote: > > > I will look at the bug (and try to compile with gcc6 for a check of all > > > libs. As for bpp-seq, I think it is because it needs some input files > > that > > > should be in the test folder too (eg example.fasta, example.aln and > > > example.ph). > > > > I have verified that these files are existing in the very same folder as > > the test code. I'm not sure whether this is the correct location for > > these files since may be the test code will be started from a different > > folder (I tried a patch adding "test/" in front of the file names but > > this did not helped. I also tried deactivating single tests inside > > test_io - also with no success. > > > > They should be in the same directory as the program is launched (which is > "test/" in my CMake script). OK, so it might be worth investigating why it might fail. Are you able to reproduce this issue? > My plan is ultimately to deactivate unit > testing by default as several users have complained about that, but in the > mean time this can be done by adding BUILD_TESTING=OFF to the cmake command > line. I would consider switching of the tests a way backward. Debian is striving to get as most as possible package delivered with build time tests as well as test of the packages on the Debian mirror in regular intervals. Specifically for scientific software I'd regard this of high relevance and thus the Debian Med team is running a Google Summer of Code project to add more tests. I think the best way to cope with non passing tests is to give more detailed output on failing tests to let users check what might be wrong and make the tests more robust. There is no need to switch of the tests in general in the packaging as you can see in the according debian/rules file[1] I have worked around the issue and have provided an according documentation[2]. Hiding eyes from non-passing tests should not be the solution here. Kind regards Andreas. [1] https://anonscm.debian.org/cgit/debian-med/libbpp-seq.git/tree/debian/rules [2] https://anonscm.debian.org/cgit/debian-med/libbpp-seq.git/tree/debian/README.source -- http://fam-tille.de
GIt tags on Github (Was: Lets maintain libbpp-phyl-omics in Debian Med team)
Hi Julien, On Mon, Apr 11, 2016 at 12:16:54PM +0200, Julien Yann Dutheil wrote: > Dear Andreas, > > On Mon, Apr 11, 2016 at 10:50 AM, Andreas Tille wrote: > > > Hi Julien, > > > > > > > Just a pragmatic question: If I would like to include these in the 2.2.0 > > > version, how can I do that with tags? I mean, we created a dedicated > > branch > > > for each stable version so that it's easy to include such fixes and not > > all > > > the further modification that might have happened on the master branch. > > > With tags, that would be like inserting a commit back in time, just > > before > > > the tagged commit... is that possible? (Sorry for my still poor knowledge > > > of git, I just know the few commands that I'm using on a daily basis :s) > > > > I admit I do not work with Github and if there are any specifics. But > > tagging something back in time should be no problem at all. As far as > > I know the branch also does not matter. You go to the commit that was > > released as version 2.2.0 (and reflects the content of the release > > tarball) and than you add a tag to this. > > [This mail goes in CC to the Debian Med list - others might correct me > > if I'm wrong.] > > > > > Tagging back in time is no pb, my question was more to included a commit > back in time, so that it is inlcuded in the "tagged" version. I think I cna > only do that in git via a new branch... I'm not sure whether I understand your problem correctly. As far as I understood git the most common practice is to have a master branch and there you are tagging your releases. If you create additional branches for development these will be merged into master once a release is done and than you tag the release. Anyway, for the moment I can check out the final release tarballs for 2.2.0 from http://biopp.univ-montp2.fr/Repositories/sources/ which is perfectly sufficient. It would be great if you start wit tagging released at Github from your next version on. Kind regards Andreas. -- http://fam-tille.de
Re: Lets maintain libbpp-phyl-omics in Debian Med team
Dear Andreas, On Mon, Apr 11, 2016 at 10:50 AM, Andreas Tille wrote: > Hi Julien, > > > > Just a pragmatic question: If I would like to include these in the 2.2.0 > > version, how can I do that with tags? I mean, we created a dedicated > branch > > for each stable version so that it's easy to include such fixes and not > all > > the further modification that might have happened on the master branch. > > With tags, that would be like inserting a commit back in time, just > before > > the tagged commit... is that possible? (Sorry for my still poor knowledge > > of git, I just know the few commands that I'm using on a daily basis :s) > > I admit I do not work with Github and if there are any specifics. But > tagging something back in time should be no problem at all. As far as > I know the branch also does not matter. You go to the commit that was > released as version 2.2.0 (and reflects the content of the release > tarball) and than you add a tag to this. > [This mail goes in CC to the Debian Med list - others might correct me > if I'm wrong.] > > Tagging back in time is no pb, my question was more to included a commit back in time, so that it is inlcuded in the "tagged" version. I think I cna only do that in git via a new branch... J. > Kind regards > > Andreas. > > -- > http://fam-tille.de > -- Julien Y. Dutheil, Ph-D 0 (+49) 6421 178 986 § Max Planck Institute for Evolutionary Biology Molecular Systems Evolution Department of Evolutionary Genetics Plön -- GERMANY § Institute of Evolutionary Sciences - Montpellier University of Montpellier 2 -- FRANCE
Re: Lets maintain libbpp-phyl-omics in Debian Med team
Hi Julien, On Mon, Apr 11, 2016 at 09:10:00AM +0200, Julien Yann Dutheil wrote: > I will look at the bug (and try to compile with gcc6 for a check of all > libs. As for bpp-seq, I think it is because it needs some input files that > should be in the test folder too (eg example.fasta, example.aln and > example.ph). I have verified that these files are existing in the very same folder as the test code. I'm not sure whether this is the correct location for these files since may be the test code will be started from a different folder (I tried a patch adding "test/" in front of the file names but this did not helped. I also tried deactivating single tests inside test_io - also with no success. > Just a pragmatic question: If I would like to include these in the 2.2.0 > version, how can I do that with tags? I mean, we created a dedicated branch > for each stable version so that it's easy to include such fixes and not all > the further modification that might have happened on the master branch. > With tags, that would be like inserting a commit back in time, just before > the tagged commit... is that possible? (Sorry for my still poor knowledge > of git, I just know the few commands that I'm using on a daily basis :s) I admit I do not work with Github and if there are any specifics. But tagging something back in time should be no problem at all. As far as I know the branch also does not matter. You go to the commit that was released as version 2.2.0 (and reflects the content of the release tarball) and than you add a tag to this. [This mail goes in CC to the Debian Med list - others might correct me if I'm wrong.] Kind regards Andreas. -- http://fam-tille.de
Re: Lets maintain libbpp-phyl-omics in Debian Med team
Hi Julien, On Sat, Apr 09, 2016 at 12:19:07AM +0200, Andreas Tille wrote: > > I've started with libbpp-core[1] which is now conform to Debian Med > policy and fixes two open bugs. I have not yet closed "FTBFS with GCC > 6: cannot convert x to y" [2]. It would be great if you as upstream > could have a look and make sure it will at least be fixed in the next > release (or tell me if it is fixed even in the current release). > > I'll continue with the other packages in this manner. I continued with libbpp-seq[3] but this build shows a problem when running the test suite: dh_auto_test make -j1 test ARGS\+=-j1 make[1]: Verzeichnis „/home/andreas/debian-maintain/alioth/debian-med_git/build-area/libbpp-seq-2.2.0/obj-x86_64-linux-gnu“ wird betreten Running tests... /usr/bin/ctest --force-new-ctest-process -j1 Test project /home/andreas/debian-maintain/alioth/debian-med_git/build-area/libbpp-seq-2.2.0/obj-x86_64-linux-gnu Start 1: test_alphabets 1/7 Test #1: test_alphabets ... Passed0.01 sec Start 2: test_sequences 2/7 Test #2: test_sequences ... Passed0.01 sec Start 3: test_io 3/7 Test #3: test_io ..***Exception: Other 0.01 sec terminate called after throwing an instance of 'bpp::IOException' what(): Fasta::appendFromStream: can't read from istream input Start 4: test_containers 4/7 Test #4: test_containers .. Passed0.01 sec Start 5: test_alignment_scores 5/7 Test #5: test_alignment_scores Passed0.01 sec Start 6: test_walker 6/7 Test #6: test_walker .. Passed0.02 sec Start 7: test_bowker 7/7 Test #7: test_bowker .. Passed 76.56 sec 86% tests passed, 1 tests failed out of 7 Total Test time (real) = 76.62 sec The following tests FAILED: 3 - test_io (OTHER_FAULT) Errors while running CTest Makefile:130: die Regel für Ziel „test“ scheiterte make[1]: *** [test] Fehler 8 (Sorry for German locale). Since libcpp-core needed to pass the new queue due to the name change forced by the g++-5 transition you need to use your own libbpp-core-dev package to reproduce this build which hopefully will express the same problem. Could you please comment on this? Kind regards Andreas. > [1] https://anonscm.debian.org/git/debian-med/libbpp-core.git > [2] https://bugs.debian.org/811657 [3] https://anonscm.debian.org/git/debian-med/libbpp-seq.git -- http://fam-tille.de
Re: Lets maintain libbpp-phyl-omics in Debian Med team
Hi Julien, On Thu, Apr 07, 2016 at 08:57:48AM +0200, Julien Yann Dutheil wrote: > > Sorry, no. You push a tag to github (or whatever repository you might > > use) and uscan will recognise a new version. It will download the > > according tarball and this tarball will be used as source tarball for > > the next package. > > I'm still a bit unclear about that, but let's just get the mentoring and > once I will have done it once, I will know how it works :) I've started with libbpp-core[1] which is now conform to Debian Med policy and fixes two open bugs. I have not yet closed "FTBFS with GCC 6: cannot convert x to y" [2]. It would be great if you as upstream could have a look and make sure it will at least be fixed in the next release (or tell me if it is fixed even in the current release). I'll continue with the other packages in this manner. Kind regards Andreas. [1] https://anonscm.debian.org/git/debian-med/libbpp-core.git [2] https://bugs.debian.org/811657 -- http://fam-tille.de
Re: Lets maintain libbpp-phyl-omics in Debian Med team
Dear Julien, On Thu, Apr 07, 2016 at 08:57:48AM +0200, Julien Yann Dutheil wrote: > > Sorry, no. You push a tag to github (or whatever repository you might > > use) and uscan will recognise a new version. It will download the > > according tarball and this tarball will be used as source tarball for > > the next package. > > I'm still a bit unclear about that, but let's just get the mentoring and > once I will have done it once, I will know how it works :) I will start by > tagging the release (on biopp.univ-montp2.fr/git/bpp-core). I checked this repository but it seems there is no way to download a tarball matching a tag as it can be done at Github. However, at your download page http://biopp.univ-montp2.fr/repos/sources/ I can find versioned source tarballs. If you confirm that these are OK and up to date that would be sufficient. > Many thanks again for your consideration, You are welcome Andreas. -- http://fam-tille.de
Re: Lets maintain libbpp-phyl-omics in Debian Med team
Dear Andreas, On Thu, Apr 7, 2016 at 6:18 AM, Andreas Tille wrote: > Hi Julien, > > > > > but I understand that you do that from the debian git > > repositories, right? So if I push a commit tagged 2.3.0 to > git.debian.org, > > then you will automatically create a new version of the package, is that > > correct? > > Sorry, no. You push a tag to github (or whatever repository you might > use) and uscan will recognise a new version. It will download the > according tarball and this tarball will be used as source tarball for > the next package. > I'm still a bit unclear about that, but let's just get the mentoring and once I will have done it once, I will know how it works :) I will start by tagging the release (on biopp.univ-montp2.fr/git/bpp-core). Many thanks again for your consideration, Best, Julien. -- Julien Y. Dutheil, Ph-D 0 (+49) 6421 178 986 § Max Planck Institute for Evolutionary Biology Molecular Systems Evolution Department of Evolutionary Genetics Plön -- GERMANY § Institute of Evolutionary Sciences - Montpellier University of Montpellier 2 -- FRANCE
Re: Lets maintain libbpp-phyl-omics in Debian Med team
Hi Julien, On Wed, Apr 06, 2016 at 10:00:03PM +0200, Julien Yann Dutheil wrote: > > Sorry, that's not team maintenance. The only sensible place to do the > > maintenance of the Debian packaging is on git.debian.org. We have > > developed a complex system of tools working with this repository and > > there is no advantage to use Github. > > > > What I meant is for creating the first repository on git.debian.org, that > it should be initiated from our repository at > http://biopp.univ-montp2.fr/git/ and not from the github one. I realize I > misunderstood your previous email: I thought you had a tool to scan from > release tags, Yes, that is true. It is called uscan and you should have written a file debian/watch that is processed by uscan. > but I understand that you do that from the debian git > repositories, right? So if I push a commit tagged 2.3.0 to git.debian.org, > then you will automatically create a new version of the package, is that > correct? Sorry, no. You push a tag to github (or whatever repository you might use) and uscan will recognise a new version. It will download the according tarball and this tarball will be used as source tarball for the next package. > If so it does not matter indeed where we push the commit from. Just work with your upstream code and tag the releases where the source is maintained (as you would do if there would be no Debian packaging attempt and several others projects are doing). > > > - We start with the latest stable version (2.2.0), for which we have > > > debian/ubuntu packages on our local repository but which are not yet on > > > debian (I think the latest there is 2.1.0) and the same for MafFilter. > > > > I'd suggest to base the first commit on the latest Debian release and > > inject your latest packaging on top of this. Where exactly do you have > > this packaging? > > We have everything here: http://biopp.univ-montp2.fr/repos/apt/ Ok, I'll have a look. > > > - I will tag the commits on the master branch of each lib / software with > > > version numbers. Is there a dedicated nomenclature for that or is sthg > > like > > > rel-2-2-0 good enough? > > > > I'd suggest simply 2.2.0. > > Ok. > > > > - While we're at it, may I ask if we could also package my software CoMap > > > (detection of coevolving positions in a molecule) which also depends on > > the > > > bpp libraries? > > > > Sure we can. I'm not sure whether you are interested in a Mentoring > > of the Month project[1]. > > I would love that! It depends however on the time requested for completing > the tasks... my time at the computer is getting more and more reduced > (unfortunately). You will spend the time you have. Since you seem to have some packaging skills this might not last that long. Kind regards Andreas. -- http://fam-tille.de
Re: Lets maintain libbpp-phyl-omics in Debian Med team
Hi Julien, On Wed, Apr 06, 2016 at 08:37:28PM +0200, Julien Yann Dutheil wrote: > > - We use our "central" git repository biopp.univ-montp2.fr/git/ to create > the debian ones. As I am still new to github, I would feel more comfortable > in using our good old repos (which I know to be up to date). Sorry, that's not team maintenance. The only sensible place to do the maintenance of the Debian packaging is on git.debian.org. We have developed a complex system of tools working with this repository and there is no advantage to use Github. > - We start with the latest stable version (2.2.0), for which we have > debian/ubuntu packages on our local repository but which are not yet on > debian (I think the latest there is 2.1.0) and the same for MafFilter. I'd suggest to base the first commit on the latest Debian release and inject your latest packaging on top of this. Where exactly do you have this packaging? > - I will tag the commits on the master branch of each lib / software with > version numbers. Is there a dedicated nomenclature for that or is sthg like > rel-2-2-0 good enough? I'd suggest simply 2.2.0. > - While we're at it, may I ask if we could also package my software CoMap > (detection of coevolving positions in a molecule) which also depends on the > bpp libraries? Sure we can. I'm not sure whether you are interested in a Mentoring of the Month project[1]. I'd suggest to update all the libs and than continue with the applications you consider worth packaging. > I'll be of course very happy to help where I can (I just need to know what > to do!). I'd suggest to sneak into the Debian Med team policy[2] which gives a sensible introduction about the things to do. The first would be to get a login on alioth.debian.org which will grant you commit permissions to the team repository (once we've added your id to the team). > Many thanks for your consideration, Thanks for your cooperation Andreas. [1] https://wiki.debian.org/DebianMed/MoM [2] https://debian-med.alioth.debian.org/docs/policy.html -- http://fam-tille.de
Re: Lets maintain libbpp-phyl-omics in Debian Med team
On 06/04/16 15:46, Andreas Tille wrote: As for the git repository, may I ask why you need to create one? Would that be a local debian copy of the one on github? Makes me think that it would be time to synchronize our local repos and the github one... The Debian *packaging* repository is different from your development repository - at least in the most frequently used workflow for Debian packaging. This repository stores only *release tarballs* (thus my request for tagging releases) and the debian packaging is what is kept track of (which you do not want inside your upstream code). +1 for tagging your releases. It defines from which commit the official release tarball originated, which helps for sending our patches for inclusion upstream. There are other workflows where the Debian repository is using a clone from Github (or whereever the repository can be found) but as I said this is an unusual case which is used very less in our team. One such example is this one: https://anonscm.debian.org/cgit/debian-science/packages/python-arrayfire.git Regardless of the choice of packaging repository layout, the important thing is that the upstream branch (containing the upstream source) and the packaging branch (containing the debianized source) are clearly separated. Hope this helps, Ghis
Re: Lets maintain libbpp-phyl-omics in Debian Med team
Hi Julien, On Wed, Apr 06, 2016 at 03:46:57PM +0200, Julien Yann Dutheil wrote: > Dear Andreas, Dear Loïc, > > I do not know how we did that, but we certainly did not do it on purpose :) You fast and helpful responses are a sign that it was not on purpose. ;-) > As far as I'm concerned, I'm fine with your suggestion. Good. > As for the git repository, may I ask why you need to create one? Would that > be a local debian copy of the one on github? > Makes me think that it would be time to synchronize our local repos and the > github one... The Debian *packaging* repository is different from your development repository - at least in the most frequently used workflow for Debian packaging. This repository stores only *release tarballs* (thus my request for tagging releases) and the debian packaging is what is kept track of (which you do not want inside your upstream code). There are other workflows where the Debian repository is using a clone from Github (or whereever the repository can be found) but as I said this is an unusual case which is used very less in our team. Kind regards Andreas. -- http://fam-tille.de
Lets maintain libbpp-phyl-omics in Debian Med team
Hi Loic and Julien, congratulations - you were the first who was flying below my radar about bioinformatics software inside Debian. You managed to inject a set of library packages (libbpp-core-dev, libbpp-phyl-dev, libbpp-phyl-omics-dev, libbpp-popgen-dev, libbpp-qt-dev, libbpp-raa-dev, libbpp-seq-dev, libbpp-seq-omics-dev) without beeing noticed by anybody in the Debian Med team which is staffed by several people packaging software in bioinformatics. How did you managed this and who is your sponsor who uploaded these packages? On the other hand it seems that you are not aware that the Debian Med team exists - otherwise you would have contacted us most probably. Since I'd consider this an unfortunate situation I'd suggest, that we move all the packages under team maintenance in Debian Med Git. Since the packages have open RC bugs it seems you might seek for help in this anyway and I think we can provide it. So my first step would be to start with libbpp-core-dev, create a Git repository for the packaging, polish the outdated packaging to Debian Med team policy standards, add you as Uploaders and try to fix the open bugs. Once I'm ready I would ask you again for confirmation (in case you might not have answered yet). If I do not hear from you I'll assume agreement with the above procedure and go for a team upload. BTW, I have added the development packages to the bio-dev task of Debian Med[1] where you could find them soon. Kind regards and thanks for your work on libbpp-* in any case Andreas. [1] http://blends.debian.org/med/tasks/bio-dev -- http://fam-tille.de
