Re: [Debian-med-packaging] Bug#879886: libhts2: libhts2 needs to handle ABI changes
Hi Diane and everybody, Le Tue, Nov 07, 2017 at 05:09:34PM -0800, Diane Trout a écrit : > > I do think we should bring back the symbols file I think so too. Symbols file are strange to work with because their update usually goes through a build failure that outputs a patch, which is not very intuitive. And then the patched symbols file has to be edited to remove the Debian minor version, otherwise it complicates backports etc. Perhaps it can be simplified, better explained and streamlined. In any case, I think that for the htslib it is worth the effort. > I was wondering if we should split the cram headers into a > libhts-private-dev so we can at least track what is depending on the > non-public api. An ideal solution, and I understand that it may not be easy, would be to make the upstream users of htslib talk with the htslib developers, so that they can implement what they want to without needing to access private functions. I think that it would fit the aims of both sides. > I did realize that my thought about updating the SOVERSION might be > wrong because I was just looking in the source tree for the removed > functions but I should have been checking the public header files. Indeed, packages using private functions need to have a tight dependency on the htslib (unless we are very confident that there are regression tests that cover this area of the code). Packages that are more well-behaved can infer their dependency through the (to be re-added) symbols file. Have a nice day, Charles -- Charles Plessy Debian Med packaging team, http://www.debian.org/devel/debian-med Tsurumi, Kanagawa, Japan
Re: [Debian-med-packaging] Bug#879886: libhts2: libhts2 needs to handle ABI changes
One of the htslib developers filed a new bug, https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=881170 asking us to not make their private libraries public. His suggestions are fairly similar to whats Charles proposed. What I'm thinking is: - TODO Recommit symbols file - TODO Split private cram headers off into a new libhts-private-dev package - WAITING Try to talk htslib & SeqLib developers to agree on a public cram api so we can drop the private-dev package. > > Symbols file are strange to work with because their update usually > goes > through a build failure that outputs a patch, which is not very > intuitive. And then the patched symbols file has to be edited to > remove > the Debian minor version, otherwise it complicates backports etc. > Perhaps it can be simplified, better explained and streamlined. In > any > case, I think that for the htslib it is worth the effort. The KDE team had some nice utilities that downloaded the symbols files for all the architectures and could do batch patches. Unfortunately I think they're KDE specific. I'll commit my rebuilt symbols files the next time I'm not spending my day writing emails to everyone else. I need a chance to look more carefully if the missing symbols were actually not part of the private api. > > > I was wondering if we should split the cram headers into a > > libhts-private-dev so we can at least track what is depending on > > the > > non-public api. > > An ideal solution, and I understand that it may not be easy, would be > to > make the upstream users of htslib talk with the htslib developers, so > that they can implement what they want to without needing to access > private functions. I think that it would fit the aims of both sides. I wonder if I can forward one Debian bug to multiple upstreams But I tried to get SeqLb & htslib to talk to each other. SeqLib issue: https://github.com/walaj/SeqLib/issues/21 htslib issue: https://github.com/samtools/htslib/issues/619 > > > I did realize that my thought about updating the SOVERSION might be > > wrong because I was just looking in the source tree for the removed > > functions but I should have been checking the public header files. > > Indeed, packages using private functions need to have a tight > dependency > on the htslib (unless we are very confident that there are regression > tests that cover this area of the code). Packages that are more > well-behaved can infer their dependency through the (to be re-added) > symbols file. So that implies packaging that uses -private-dev that implies they have an == dependency on a specific binary version of libhts? That should probably probably be documented in a README.Debian for the private-dev package. Should I push these proposed changes to a clone of the htslib packaging repository? A branch of the alioth repository, or just push it to the alioth master? Diane >
Re: [Debian-med-packaging] Bug#879886: libhts2: libhts2 needs to handle ABI changes
Dear Diane, On Wed, Nov 08, 2017 at 04:58:49PM -0800, Diane Trout wrote: > > Should I push these proposed changes to a clone of the htslib packaging > repository? A branch of the alioth repository, or just push it to the > alioth master? Thanks for your sane considerations and pleas push to alioth master. Kind regards Andreas. -- http://fam-tille.de
Re: [Debian-med-packaging] Bug#879886: libhts2: libhts2 needs to handle ABI changes
Hi, Diane, Thanks for working on this. On November 8, 2017 7:58:49 PM EST, Diane Trout wrote: >One of the htslib developers filed a new bug, > >https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=881170 > >asking us to not make their private libraries public. His suggestions >are fairly similar to whats Charles proposed. > >What I'm thinking is: > >- TODO Recommit symbols file +1 >- TODO Split private cram headers off into a new libhts-private-dev >package I'd rather be in favor of restoring the bundled htslib to seqlib as the short term solution. Putting a private package in the archive may exacerbate the problem and is odd nevertheless. And there is another action item-- TODO update the htslib package to the latest release. >- WAITING Try to talk htslib & SeqLib developers to agree on a public >cram api so we can drop the private-dev package. > > > >[...] > >> >> > I was wondering if we should split the cram headers into a >> > libhts-private-dev so we can at least track what is depending on >> > the >> > non-public api. We would find this out anyway because the packages woukd break until either a dependency on such a package had to be added (most likely by our team anyway), or the library had to be rebundled. >> >[...] > >> >> > I did realize that my thought about updating the SOVERSION might be >> > wrong because I was just looking in the source tree for the removed >> > functions but I should have been checking the public header files. >> >> Indeed, packages using private functions need to have a tight >> dependency >> on the htslib (unless we are very confident that there are regression >> tests that cover this area of the code). Packages that are more >> well-behaved can infer their dependency through the (to be re-added) >> symbols file. > > > >So that implies packaging that uses -private-dev that implies they have >an == dependency on a specific binary version of libhts? > >That should probably probably be documented in a README.Debian for the >private-dev package. > Does this imply a transition for each htslib update? Many thanks and regards Afif
Re: [Debian-med-packaging] Bug#879886: libhts2: libhts2 needs to handle ABI changes
On Thu, 2017-11-09 at 02:03 -0500, Afif Elghraoui wrote: > > - TODO Split private cram headers off into a new libhts-private-dev > > package > > I'd rather be in favor of restoring the bundled htslib to seqlib as > the short term solution. Putting a private package in the archive may > exacerbate the problem and is odd nevertheless. The no convenience copies of libraries is a pretty strong rule of Debian, and there are good maintenance reasons for it. Although I'm not opposed to it I'd like several people to agree that its the best option first. On the plus side overriding it would allow us to drop the patch that is making the cram symbols public, on the downside we'd have to remember that bugs involving htslib also impact libseqlib. I think we'd need to use the Built-Using tag? I haven't used that before. On the other hand upstream did suggest that the private-dev library was a viable temporary solution. (Though doing that would push htslib into NEW). > > And there is another action item-- > TODO update the htslib package to the latest release. Very true I did try building 1.6 and there was a problem with running tests that I haven't investigated yet. Diane
Re: [Debian-med-packaging] Bug#879886: libhts2: libhts2 needs to handle ABI changes
On November 9, 2017 2:32:56 AM EST, Diane Trout wrote: >On Thu, 2017-11-09 at 02:03 -0500, Afif Elghraoui wrote: >> > - TODO Split private cram headers off into a new libhts-private-dev >> > package >> >> I'd rather be in favor of restoring the bundled htslib to seqlib as >> the short term solution. Putting a private package in the archive may >> exacerbate the problem and is odd nevertheless. > >The no convenience copies of libraries is a pretty strong rule of >Debian, and there are good maintenance reasons for it. Yes, I understand that, but we don't have good options right now. I don't see the maintenance advantages for seqlib as worth requiring a transition for every htslib update to make sure it doesn't break--in addition to putting a package in the archive and telling users/developers not to install it. > Although I'm not >opposed to it I'd like several people to agree that its the best option >first. > >On the plus side overriding it would allow us to drop the patch that is >making the cram symbols public, on the downside we'd have to remember >that bugs involving htslib also impact libseqlib. If this whole situation is primarily seqlib's problem, I think it's only fair for it to bear the kludges required for its approach. Otherwise, we have the kludge in htslib and the need for a htslib transition with every update, right? In fact, lofreq never entered Debian because it needs to use samtools as a library and we were not going to bundle it [1]. I felt that would be an RC bug. > >I think we'd need to use the Built-Using tag? I haven't used that >before. > >On the other hand upstream did suggest that the private-dev library was >a viable temporary solution. (Though doing that would push htslib into >NEW). Well, I think they were saying that if we were going to go so far as to misrepresent htslib, we should at the very least make a division and distribute htslib proper as such. I read that as saying anything would be better than the current situation--not necessarily that they're equally better. > >> >> And there is another action item-- >> TODO update the htslib package to the latest release. > >Very true I did try building 1.6 and there was a problem with >running tests that I haven't investigated yet. > Regards Afif 1. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=808895
Re: [Debian-med-packaging] Bug#879886: libhts2: libhts2 needs to handle ABI changes
On Wed, Nov 08, 2017 at 11:32:56PM -0800, Diane Trout wrote: > I think we'd need to use the Built-Using tag? I haven't used that > before. No, that's needed when doing static linking for GPL compliance (and other kind of things, but all related to static linking that thanks god is not a topic here... > (Though doing that would push htslib into NEW). Please worry not about NEW, I've got contacts with ftp-master and can get stuff out fairly quick (after a respectful time passed, like a week at least). > > And there is another action item-- > > TODO update the htslib package to the latest release. > > Very true I did try building 1.6 and there was a problem with > running tests that I haven't investigated yet. Let me recommend adding the symbols file and getting it right before doing 1.6, so symbols new in 1.6 are correctly versioned. -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. more about me: https://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `- signature.asc Description: PGP signature
Re: [Debian-med-packaging] Bug#879886: libhts2: libhts2 needs to handle ABI changes
On November 9, 2017 6:20:03 PM EST, Diane Trout wrote: > [...] > >Anyone want to review? or should I go ahead and release? > I think you can release. However, it appears the changelog wasn't updated in git since the last upload [1] you'll need to go for a 1.5-3. regards Afif 1. http://metadata.ftp-master.debian.org/changelogs/main/h/htslib/htslib_1.5-2_changelog
Re: [Debian-med-packaging] Bug#879886: libhts2: libhts2 needs to handle ABI changes
On Thu, Nov 09, 2017 at 10:11:30PM -0500, Afif Elghraoui wrote: > On November 9, 2017 6:20:03 PM EST, Diane Trout wrote: > > > [...] > > > >Anyone want to review? or should I go ahead and release? > > I think you can release. Yes please release it, and IMHO close this bug with that upload. I've tested building python-pysam against it, and it gets a dependency on 'libhts2 (>= 1.5)' instead of just 'libhts2' :) > However, it appears the changelog wasn't updated in git since the last upload > [1] you'll need to go for a 1.5-3. I've fixed it. I force pushed a tag change for debian/1.5-2, as it pointed to something that was not what was uploaded (bad tille!) -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. more about me: https://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `- signature.asc Description: PGP signature