Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi Martin, On Mon, Jan 11, 2016 at 11:06:30AM +0100, Martin Uecker wrote: > > It was meant for system-wide add-ons installed by > other packages or compiled locally. But you are right, > another package could create it itself and locally compiled > stuff should go in /usr/local/... ... so it is good that I asked. :-) > So I removed it by having a debian patch. But it isn't > clear to me whether the change should also be reflected > directly in the master branch or only as a commit which adds > the patch in debian/patches/. The tools do not seem to > care... I reverted the change in Makefile in master branch since this would lead to confusion when trying to apply the patch. I guess the conflict did not occure when you have your local .pc dir with quilt patches but this is not in the Git repository. I have uploaded the package to unstable which now needs to pass the new queue (currently this needs about three monthes patience, sorry :-(). Thanks for your work on this and the successful MoM project Andreas. -- http://fam-tille.de
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi Andreas, > For me the package is close to ready if I would understand why you > insist on the empty dir usr/lib/bart/commands/. I do not see any reason > for this since in /usr only packages can / should write and a package > that writes to this location would create the package itself. Users > should not write to /usr - so why leaving the empty dir there? > > Is your intention rather to create > > /var/lib/bart/commands/ > > and set apropriate permissions in a postinst script? It was meant for system-wide add-ons installed by other packages or compiled locally. But you are right, another package could create it itself and locally compiled stuff should go in /usr/local/... So I removed it by having a debian patch. But it isn't clear to me whether the change should also be reflected directly in the master branch or only as a commit which adds the patch in debian/patches/. The tools do not seem to care... Martin
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi Martin, On Sun, Jan 10, 2016 at 02:56:27PM +0100, Martin Uecker wrote: > > There are three remaining and easy to fix lintian issues: > > > > I: bart source: duplicate-short-description bart libbart-dev > > W: bart: package-installs-into-obsolete-dir etc/bash_completion.d/ : > > ^etc/bash_completion.d/ -> usr/share/bash-completion/completions (see also > > https://bugs.debian.org/776954) > > I: bart: package-contains-empty-directory usr/lib/bart/commands/ > > > > If these are fixed I could upload. > > Fixed. I wonder why I didn't get these warning. Probably because > I called lintian on the binaries and not on the .dsc file. Not really. I think the .changes file is the best target to fetch all lintian issues. The last two items are definitely a *.deb issue. I guess it is since you are not using the latest lintian and may be not asking for the info severity. I'd recommend two things: 1. Always use lintian from unstable which can be easily dony by the following: $ cat /etc/apt/preferences.d/01-lintian.pref Package: lintian Pin: release a=unstable Pin-Priority: 601 (provided you have set your prefered distribution with some pin priority to say 500 - for me that is testing) 2. Make sure you also get lintian info issues which can be done that way: $ cat ~/.lintianrc color=always display-experimental=no display-info=yes pedantic=no ## show-overrides=yes For me the package is close to ready if I would understand why you insist on the empty dir usr/lib/bart/commands/. I do not see any reason for this since in /usr only packages can / should write and a package that writes to this location would create the package itself. Users should not write to /usr - so why leaving the empty dir there? Is your intention rather to create /var/lib/bart/commands/ and set apropriate permissions in a postinst script? Kind regards Andreas. -- http://fam-tille.de
Re: Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
> There are three remaining and easy to fix lintian issues: > > I: bart source: duplicate-short-description bart libbart-dev > W: bart: package-installs-into-obsolete-dir etc/bash_completion.d/ : > ^etc/bash_completion.d/ -> usr/share/bash-completion/completions (see also > https://bugs.debian.org/776954) > I: bart: package-contains-empty-directory usr/lib/bart/commands/ > > If these are fixed I could upload. Fixed. I wonder why I didn't get these warning. Probably because I called lintian on the binaries and not on the .dsc file. Martin
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi, On Sat, Jan 09, 2016 at 10:46:45AM +, Ghislain Vaillant wrote: > On 09/01/16 08:03, Martin Uecker wrote: > >What are the next steps? > > Submit an RFS bug [1] and upload the package to mentors [2]. In the Debian Med team we simplify this. A mail here on the list where you announce that you think the state in Git is ready is sufficient. Meanwhile I'm able to build with gbp and I will check. Kind regards Andreas. -- http://fam-tille.de
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi, no further comments to the OpenCL question which is IMHO fully answered (way better than I could since I have no experience in this field). There are three remaining and easy to fix lintian issues: I: bart source: duplicate-short-description bart libbart-dev N: N:The listed binary packages all share the same short description (the N:first line of the Description control field). The package names may N:provide enough additional information to distinguish between the N:packages, but it's common to also add a word or two to the short N:description to clarify the difference. N: N:Severity: wishlist, Certainty: possible N: N:Check: control-file, Type: source N: W: bart: package-installs-into-obsolete-dir etc/bash_completion.d/ : ^etc/bash_completion.d/ -> usr/share/bash-completion/completions (see also https://bugs.debian.org/776954) N: N:The package installs files to an obsolete directory. Please use a newer N:path. N: N:Severity: normal, Certainty: certain N: N:Check: files, Type: binary, udeb N: W: bart: package-installs-into-obsolete-dir etc/bash_completion.d/bart_completion.sh : ^etc/bash_completion.d/ -> usr/share/bash-completion/completions (see also https://bugs.debian.org/776954) I: bart: package-contains-empty-directory usr/lib/bart/commands/ N: N:This package installs an empty directory. This might be intentional but N:it's normally a mistake. If it is intentional, add a lintian override. N: N:If a package ships with or installs empty directories, you can remove N:them in debian/rules by calling: N: N: $ find path/to/base/dir -type d -empty -delete N: N:Severity: wishlist, Certainty: possible N: N:Check: files, Type: binary, udeb N: If these are fixed I could upload. Kind regards Andreas. -- http://fam-tille.de
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
On 09/01/16 08:03, Martin Uecker wrote: What are the next steps? Submit an RFS bug [1] and upload the package to mentors [2]. That's the usual route. Maybe the MoM process is different though. To be checked with Andreas. [1] http://mentors.debian.net/sponsor/rfs-howto [2] http://mentors.debian.net/ [3] https://wiki.debian.org/DebianMed/MoM Best, Ghis
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
On 09/01/16 08:35, Martin Uecker wrote: Ghislain Vaillant : FYI, the same trick is used for other dependencies with multiple backends such as OpenCL. What is the state of OpenCL on Debian? The OpenCL headers are regularly updated, so are the main open-source drivers like ocl-icd (reference) and beignet (Intel). There is also pocl (CPU) but the packaging is lagging and its maintainer does not look very active. I am personally using the Nvidia one on my laptop and it works fine too. We only support CUDA in BART, but I always wanted to add OpenCL support and it should be fairly easy... I personally maintain a few packages which requires OpenCL. You only need to build-depend on ocl-icd-opencl-dev | opencl-dev (which would also pull opencl-headers) and you are good to go. Best regards, Ghis
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
On 09/01/16 08:35, Martin Uecker wrote: Ghislain Vaillant : FYI, the same trick is used for other dependencies with multiple backends such as OpenCL. What is the state of OpenCL on Debian? The OpenCL headers are regularly updated, so are the main open-source drivers like ocl-icd (reference) and beignet (Intel). There is also pocl (CPU) but the packaging is lagging and its maintainer does not look very active. I am personally using the Nvidia one on my laptop and it works fine too. We only support CUDA in BART, but I always wanted to add OpenCL support and it should be fairly easy... I personally maintain a few packages which requires OpenCL. You only need to build-depend on ocl-icd-opencl-dev | opencl-dev (which would also pull opencl-headers) and you are good to go. Best regards, Ghis
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Ghislain Vaillant : > > FYI, the same trick is used for other dependencies with multiple > backends such as OpenCL. What is the state of OpenCL on Debian? We only support CUDA in BART, but I always wanted to add OpenCL support and it should be fairly easy... Martin
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi Ghislain, Ghislain Vaillant : > On 08/01/16 19:29, Martin Uecker wrote: > Good. Then, you might want to use libbart-dev as the package name, which > is usually the naming convention for packages containing any libraries > (in shared or static form). Done. > > Also one can replace liblapack.so using update-alternatives. > > Why would it matter what I compile against as all alternatives > > should be ABI compatible? > > Absolutely. It is just that if I use OpenBLAS locally, I don't want to > have to install lapack-dev to build bart, which would also mess up with > my alternatives in the process. Using liblapack-dev | liblapack.so is > just a bit friendlier. This makes sense. This is just a virtual package where liblapack-dev, ATLAS, etc. have a "Provides: liblapack.so". Somehow the filename-like name of the virtual package got me confused. What are the next steps? From my side, everything looks good. Ofcourse, there might be some issues when compiling for all the different architectures. Martin
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
On 08/01/16 19:29, Martin Uecker wrote: Hi Ghisvail! - Binary packages split-up. I am not quite sure about the usefulness of the bart-dev package. The final compilation line shows: ... i.e. the final output is one executable (bart) with private libraries (libbox, libgrecon...) linked statically. So it looks to me that these libraries are not intended to be used publicly? There are meant to be used and there are people who are using them for internal projects. Ofcourse, they currently use the source distribution of BART, but I think a -dev package would be useful in the long run. I also have a small image viewer build on top of the libraries, which I would like to package eventually: https://github.com/mrirecon/view Good. Then, you might want to use libbart-dev as the package name, which is usually the naming convention for packages containing any libraries (in shared or static form). - lapack-dev -> lapack-dev | lapack.so in your list of Build-Depends in d/control. That way your package can be built with optimized libraries like atlas or openblas. Sounds like a good idea, but I do not understand it. What does it mean to depend on "lapack.so"? Using Build-Depends on liblapack-dev alone forces local source package build to have liblapack-dev installed. Using Build-Depends on liblapack-dev | liblapack.so, a local source package build can be done with other BLAS providers than Netlib, such as libopenblas-dev or libatlas-dev. Also one can replace liblapack.so using update-alternatives. Why would it matter what I compile against as all alternatives should be ABI compatible? Absolutely. It is just that if I use OpenBLAS locally, I don't want to have to install lapack-dev to build bart, which would also mess up with my alternatives in the process. Using liblapack-dev | liblapack.so is just a bit friendlier. FYI, the same trick is used for other dependencies with multiple backends such as OpenCL. Hope this helped, Ghis
Re: Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi Ghisvail! > - Binary packages split-up. > > I am not quite sure about the usefulness of the bart-dev package. The final > compilation line shows: ... > i.e. the final output is one executable (bart) with private libraries > (libbox, libgrecon...) linked statically. So it looks to me that these > libraries are not intended to be used publicly? There are meant to be used and there are people who are using them for internal projects. Ofcourse, they currently use the source distribution of BART, but I think a -dev package would be useful in the long run. I also have a small image viewer build on top of the libraries, which I would like to package eventually: https://github.com/mrirecon/view > > - Licensing of the bart Debian packages. > > Since the software is compiled with GSL and FFTW support, the resulting > binary should be distributed under a compatible GPL-like license. This may > need to acknowledged somewhere, maybe in debian/README.Debian. For an example > of such acknowledgement, have a look at how this is done with ITK [1]. I added a README.Debian. > - Copyright listing. > > The output of licensecheck shows that some files have different copyright > attributions than yours. For instance: > > > ./matlab/imshow3.m: UNKNOWN > [Copyright: Michael Lustig 2012 [sx,sy,nc] = size(img);] > > Please be super thorough on these, since a non-exhaustive copyright listing > is a strong motive for package rejection. We have permission to re-distribute everything under the BSD license. I reworded the copyright file to Copyright: 2013-2016 The Regents of the University of California 2013-2016 BART Developer Team and Contributors to make clear that there are other contributors who own copyright themselves. > > - lapack-dev -> lapack-dev | lapack.so in your list of Build-Depends in > d/control. > > > That way your package can be built with optimized libraries like atlas or > openblas. Sounds like a good idea, but I do not understand it. What does it mean to depend on "lapack.so"? Also one can replace liblapack.so using update-alternatives. Why would it matter what I compile against as all alternatives should be ABI compatible? > - Potential overlinkage. > > You might want to check these warnings out: > > dpkg-shlibdeps: warning: package could avoid a useless dependency if > debian/bart/usr/bin/bart was not linked against libgslcblas.so.0 (it uses > none of the library's symbols) dpkg-shlibdeps: warning: package could avoid a > useless dependency if debian/bart/usr/bin/bart was not linked against > libz.so.1 (it uses none of the library's symbols) Those should be fixed with the new upsteam release. >Apart from these comments. Looks good on my side. Thank you for your help! Martin G
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
On Fri, Jan 01, 2016 at 11:00:20AM +0100, Martin Uecker wrote: > > >dh_clean > > dpkg-source -i.git -I.git -b bart-0.2.09d > > dpkg-source: info: using options from bart-0.2.09d/debian/source/options: > > --extend-diff-ignore=(^|/)(version.(inc|txt)|commands.txt)$ > > dpkg-source: info: using source format '3.0 (quilt)' > > dpkg-source: info: building bart using existing ./bart_0.2.09d.orig.tar.gz > > dpkg-source: info: local changes detected, the modified files are: > > bart-0.2.09d/version.txt > > dpkg-source: error: aborting due to unexpected upstream changes, see > > /tmp/bart_0.2.09d-1.diff.anXVz0 > > dpkg-source: info: you can integrate the local changes with dpkg-source > > --commit > > dpkg-buildpackage: error: dpkg-source -i.git -I.git -b bart-0.2.09d gave > > error exit status 2 > > > > > > which is caused by the following diff: > > > > --- bart-0.2.09d.orig/version.txt > > +++ bart-0.2.09d/version.txt > > @@ -1 +1 @@ > > -v0.2.09 > > +v0.2.09d > > > > I don't understand why. Shouldn't the '--extend-diff-ignore' option prevent > this? > For me it works without errors using I admit I was not aware about this option. > dpkg-buildpackage > > and als using > > gbp buildpackage --git-pbuilder Strange - I cloned from scratch and no success. The only thing I could imaginge might be in ~/.gbp.conf: $ cat ~/.gbp.conf [DEFAULT] builder = ~/bin/git-pbuilder # Might lead to problems because it tries to use non-patched makefiles # cleaner = fakeroot debian/rules clean cleaner = /bin/true pristine-tar = True export=WC [buildpackage] # use this for more svn-buildpackage like behaviour: export-dir = ../build-area/ tarball-dir = ../tarballs/ Any idea whether some option is wrong here. > Happy new year! Same to you and the readers of the list Andreas. -- http://fam-tille.de
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi Andreas, > I get: > > ... >dh_clean > dpkg-source -i.git -I.git -b bart-0.2.09d > dpkg-source: info: using options from bart-0.2.09d/debian/source/options: > --extend-diff-ignore=(^|/)(version.(inc|txt)|commands.txt)$ > dpkg-source: info: using source format '3.0 (quilt)' > dpkg-source: info: building bart using existing ./bart_0.2.09d.orig.tar.gz > dpkg-source: info: local changes detected, the modified files are: > bart-0.2.09d/version.txt > dpkg-source: error: aborting due to unexpected upstream changes, see > /tmp/bart_0.2.09d-1.diff.anXVz0 > dpkg-source: info: you can integrate the local changes with dpkg-source > --commit > dpkg-buildpackage: error: dpkg-source -i.git -I.git -b bart-0.2.09d gave > error exit status 2 > > > which is caused by the following diff: > > --- bart-0.2.09d.orig/version.txt > +++ bart-0.2.09d/version.txt > @@ -1 +1 @@ > -v0.2.09 > +v0.2.09d > I don't understand why. Shouldn't the '--extend-diff-ignore' option prevent this? For me it works without errors using dpkg-buildpackage and als using gbp buildpackage --git-pbuilder Happy new year! Martin > > Also I wonder whether there is a way to build and > > test the package on other architectures than amd64? > > The usual procedure is to upload and see whether autobuilders will cope > with it - otherwise you can review the logfiles. > > Kind regards > > Andreas. > > -- > http://fam-tille.de >
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi Martin, Great to see that your packaging work is progressing. Here are a few remarks on my side: - Binary packages split-up. I am not quite sure about the usefulness of the bart-dev package. The final compilation line shows: gcc -O3 -ffast-math -std=c99 -Wmissing-prototypes -I/home/ghislain/debian/packages/bart/src/ -fopenmp -Dmain_real=main_bart -o bart src/main.c /home/ghislain/debian/packages/bart/src/bart.o lib/libbox.a lib/libbox2.a lib/libgrecon.a lib/libsense.a lib/libnoir.a lib/libwavelet2.a lib/libiter.a lib/liblinops.a lib/libwavelet3.a lib/liblowrank.a lib/libnoncart.a lib/libcalib.a lib/libsimu.a lib/libsake.a lib/libdfwavelet.a lib/libnum.a lib/libmisc.a lib/libnum.a lib/libmisc.a -L/usr//lib -lfftw3f_threads -lfftw3f -llapack -lblas -lgsl -lgslcblas -lpng -lz -lm i.e. the final output is one executable (bart) with private libraries (libbox, libgrecon...) linked statically. So it looks to me that these libraries are not intended to be used publicly? - Licensing of the bart Debian packages. Since the software is compiled with GSL and FFTW support, the resulting binary should be distributed under a compatible GPL-like license. This may need to acknowledged somewhere, maybe in debian/README.Debian. For an example of such acknowledgement, have a look at how this is done with ITK [1]. [1] http://anonscm.debian.org/viewvc/debian-med/trunk/packages/insighttoolkit/trunk/debian/copyright?view=markup - Copyright listing. The output of licensecheck shows that some files have different copyright attributions than yours. For instance: ./matlab/imshow3.m: UNKNOWN [Copyright: Michael Lustig 2012 [sx,sy,nc] = size(img);] Please be super thorough on these, since a non-exhaustive copyright listing is a strong motive for package rejection. - lapack-dev -> lapack-dev | lapack.so in your list of Build-Depends in d/control. That way your package can be built with optimized libraries like atlas or openblas. - Potential overlinkage. You might want to check these warnings out: dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/bart/usr/bin/bart was not linked against libgslcblas.so.0 (it uses none of the library's symbols) dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/bart/usr/bin/bart was not linked against libz.so.1 (it uses none of the library's symbols) Which could be solved with some hardening flags, if you don't want to touch your Makefiles too much: https://wiki.debian.org/HardeningWalkthrough Apart from these comments. Looks good on my side. Ghislain
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi Martin, On Mon, Dec 28, 2015 at 11:30:33AM +, Uecker, Martin wrote: > the bart package now produces a -dev package and a > octave-bart package. It also includes a bash completion > script. pristine-tar produces the correct upstream > tar ball. Sounds good. > I would appreciate if you could take a look and let > me know what else could be improved... I get: ... dh_clean dpkg-source -i.git -I.git -b bart-0.2.09d dpkg-source: info: using options from bart-0.2.09d/debian/source/options: --extend-diff-ignore=(^|/)(version.(inc|txt)|commands.txt)$ dpkg-source: info: using source format '3.0 (quilt)' dpkg-source: info: building bart using existing ./bart_0.2.09d.orig.tar.gz dpkg-source: info: local changes detected, the modified files are: bart-0.2.09d/version.txt dpkg-source: error: aborting due to unexpected upstream changes, see /tmp/bart_0.2.09d-1.diff.anXVz0 dpkg-source: info: you can integrate the local changes with dpkg-source --commit dpkg-buildpackage: error: dpkg-source -i.git -I.git -b bart-0.2.09d gave error exit status 2 which is caused by the following diff: --- bart-0.2.09d.orig/version.txt +++ bart-0.2.09d/version.txt @@ -1 +1 @@ -v0.2.09 +v0.2.09d > Also I wonder whether there is a way to build and > test the package on other architectures than amd64? The usual procedure is to upload and see whether autobuilders will cope with it - otherwise you can review the logfiles. Kind regards Andreas. -- http://fam-tille.de
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi Martin, some pieces of advice below: On 07/12/15 11:26, Martin Uecker wrote: The only issue is that I have to specify the tag with 'gdb' because it looks for 'upstream/v0.2.09' which does not currently exist. Instead of adding these tags, I would prefer to reconfigure 'gdb' to use the existing upstream tags without the prefix 'upstream/'. Then I could just push the regular upstream tags. Would this be ok? Just add a debian/gbp.conf [1] file with the following content: [DEFAULT] upstream-branch = upstream debian-branch = master upstream-tag = v%(version)s debian-tag = debian/%(version)s pristine-tar = True That way you are documenting how your packaging repository is layed out explicitly (always a good thing), plus gbp will pick up these options over the default ones under $HOME/.gbp.conf. That way, you won't need to explicitly specify the tag via --git-upstream-tag anymore. [1] http://honk.sigxcpu.org/projects/git-buildpackage/manual-html/man.gbp.conf.html I also created a newer upstream tag v0.2.09d specifically for the initial packaging work because I made some upstream changes to make packaging easier. The bart_0.2.09.orig.tar.gz which is now in the pristine-tar branch actually corresonds to v0.2.09d. I hope that is not too messed up, but with the next upstream version (to be released soon) this hack would gone. Since you have yet to submit a first Debian package publicly, you are kind of free to do whatever you like with the packaging specific branches (pristine-tar, master) of your repository (force push, rebase...). Just make sure everything is in order once your package is ready for its first submission. Ghis
Re: Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi Andreas, > On Mon, Dec 07, 2015 at 08:18:59AM +, Uecker, Martin wrote: > > > On Sun, Dec 06, 2015 at 11:59:02PM +, Uecker, Martin wrote: > > > > > Put simply, pristine-tar is our way to encapsulate access to the > > > > > source > > > > > tarball used for packaging. Someone who checks out a d-science > > > > > repository does not need to know where the tarball comes from > > > > > (github, > > > > > bitbucket, PyPI...), he or she can just check it out using > > > > > pristine-tar > > > > > on the packaging repository. > > > > > > > > Ok, I created a tar ball using a git archive (which matches what > > > > github does) and then used pristine-tar to check it in. > > > > > > I think this is a misunderstanding. You should write a debian/watch file > > > (line 22 of this template > > > > > > https://anonscm.debian.org/viewvc/debian-med/trunk/package_template/watch?revision=20511&view=markup > > > is your friend) and use the downloaded tarball when importing > > > pristine-tar. > > > > Ok, done. > > > > Please note that there is no difference between downloading > > tar balls from github which uses 'git archive' to create them > > or creating them locally using 'git archive' (with the > > right arguments). This already produces bit-identical results > > (with the same hash)! So there is really no point in downloading > > upstream tarballs from Github when one has a local copy of the git > > repository. > > I have no doubt that *Github* can create bit-identical results. > However, I have doubt that you can create from a local clone the > very same tarball. I agree that it is a bit surprising, but it does work: wget https://github.com/mrirecon/bart/archive/v0.2.09d.tar.gz git archive --prefix=bart-0.2.09d/ -o bart_0.2.09d.tar.gz v0.2.09d sha256sum v0.2.09d.tar.gz bart_0.2.09d.tar.gz 35879d41dcdefedc9fbca40267490d055e3a4840d7731a0f221c6976035dfff0 v0.2.09d.tar.gz 35879d41dcdefedc9fbca40267490d055e3a4840d7731a0f221c6976035dfff0 bart_0.2.09d.tar.gz > This is the role of the pristine-tar branch > and currently I can not build bart by the following procedure: > > >ssh://git.debian.org/git/debian-med/bart.git >cd bart >gbp buildpackage The following works for me: git clone ssh://uecker-gu...@git.debian.org/git/debian-med/bart.git cd bart gbp buildpackage --git-upstream-tag=v0.2.09d --git-pbuilder Also the following should also work correctly: git clone ssh://uecker-gu...@git.debian.org/git/debian-med/bart.git cd bart pristine-tar checkout ../bart_0.2.09.orig.tar.gz dpkg-buildpackage The only issue is that I have to specify the tag with 'gdb' because it looks for 'upstream/v0.2.09' which does not currently exist. Instead of adding these tags, I would prefer to reconfigure 'gdb' to use the existing upstream tags without the prefix 'upstream/'. Then I could just push the regular upstream tags. Would this be ok? I also created a newer upstream tag v0.2.09d specifically for the initial packaging work because I made some upstream changes to make packaging easier. The bart_0.2.09.orig.tar.gz which is now in the pristine-tar branch actually corresonds to v0.2.09d. I hope that is not too messed up, but with the next upstream version (to be released soon) this hack would gone. Martin
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
On Mon, 2015-12-07 at 08:58 +, Ghislain Vaillant wrote: > You could use something like libgsl0-dev | libgsl-dev to stay > compatible with earlier versions? This should probably be the other way around, i.e. libgsl-dev | libgsl0-dev so that the new package takes precedence. Best, Gert
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi Martin, On Mon, Dec 07, 2015 at 08:18:59AM +, Uecker, Martin wrote: > > Hi Andreas, > > > On Sun, Dec 06, 2015 at 11:59:02PM +, Uecker, Martin wrote: > > > > Put simply, pristine-tar is our way to encapsulate access to the source > > > > tarball used for packaging. Someone who checks out a d-science > > > > repository does not need to know where the tarball comes from (github, > > > > bitbucket, PyPI...), he or she can just check it out using pristine-tar > > > > on the packaging repository. > > > > > > Ok, I created a tar ball using a git archive (which matches what > > > github does) and then used pristine-tar to check it in. > > > > I think this is a misunderstanding. You should write a debian/watch file > > (line 22 of this template > > > > https://anonscm.debian.org/viewvc/debian-med/trunk/package_template/watch?revision=20511&view=markup > > is your friend) and use the downloaded tarball when importing pristine-tar. > > Ok, done. > > Please note that there is no difference between downloading > tar balls from github which uses 'git archive' to create them > or creating them locally using 'git archive' (with the > right arguments). This already produces bit-identical results > (with the same hash)! So there is really no point in downloading > upstream tarballs from Github when one has a local copy of the git > repository. I have no doubt that *Github* can create bit-identical results. However, I have doubt that you can create from a local clone the very same tarball. This is the role of the pristine-tar branch and currently I can not build bart by the following procedure: ssh://git.debian.org/git/debian-med/bart.git cd bart gbp buildpackage I simply get diffs between the upstream branch and the orig.tar.gz. The above should be possible and it becomes possible if you do cd bart uscan --verbose --force-download gbp import-orig --pristine-tar ../bart*.orig.tar.gz > > You could add these in additional python-bart octave-bart binary > > packages (sorry, matlab can not be provided as official Debian package). > > You should read the according pages at wiki.debian.org where to put > > Python modules (or you just check your local system where these are > > stored) and Octave files (I never dealt with these but I guess there is > > a wiki paga as well). Feel free to ask me if you are struck in the > > jungle of documentation and I'll provide more specific pointers. > > Ok, I have to look at it. There are only very few small scripts, > so I would rather put it in the same package. OK, you are the expert here. :-) > > Another remark to the packaging: Currently there is a libgsl migration > > ongoing and you should use libgsl-dev instead of libgsl0-dev. > > Done. Although now it doesn't build locally on my Ubuntu machine > anymore (only using pbuilder in a sid change root). That's an unfortunate consequence of the transition. Kind regards Andreas. -- http://fam-tille.de
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
On 07/12/15 08:18, Uecker, Martin wrote: Hi Andreas, On Sun, Dec 06, 2015 at 11:59:02PM +, Uecker, Martin wrote: Put simply, pristine-tar is our way to encapsulate access to the source tarball used for packaging. Someone who checks out a d-science repository does not need to know where the tarball comes from (github, bitbucket, PyPI...), he or she can just check it out using pristine-tar on the packaging repository. Ok, I created a tar ball using a git archive (which matches what github does) and then used pristine-tar to check it in. I think this is a misunderstanding. You should write a debian/watch file (line 22 of this template https://anonscm.debian.org/viewvc/debian-med/trunk/package_template/watch?revision=20511&view=markup is your friend) and use the downloaded tarball when importing pristine-tar. Ok, done. Please note that there is no difference between downloading tar balls from github which uses 'git archive' to create them or creating them locally using 'git archive' (with the right arguments). This already produces bit-identical results (with the same hash)! So there is really no point in downloading upstream tarballs from Github when one has a local copy of the git repository. Actually, you can do it with `gbp buildpackage` by passing the --git-no-pristine-tar and --git-pristine-tar-commit, which effectively will produce the pristine tarball from the tag you based your packaging off. Andreas gave you the general guideline which works for any source. The --git*pristine-tar* options only work if you are using the upstream git repository. FYI, make sure you have a valid watch file (although you are not using uscan), because that is also what is used by the Debian Package Tracking System to signal when new upstream releases are available. You could add these in additional python-bart octave-bart binary packages (sorry, matlab can not be provided as official Debian package). You should read the according pages at wiki.debian.org where to put Python modules (or you just check your local system where these are stored) and Octave files (I never dealt with these but I guess there is a wiki paga as well). Feel free to ask me if you are struck in the jungle of documentation and I'll provide more specific pointers. Ok, I have to look at it. There are only very few small scripts, so I would rather put it in the same package. Another remark to the packaging: Currently there is a libgsl migration ongoing and you should use libgsl-dev instead of libgsl0-dev. Done. Although now it doesn't build locally on my Ubuntu machine anymore (only using pbuilder in a sid change root). You could use something like libgsl0-dev | libgsl-dev to stay compatible with earlier versions? Ghis
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi Andreas, > On Sun, Dec 06, 2015 at 11:59:02PM +, Uecker, Martin wrote: > > > Put simply, pristine-tar is our way to encapsulate access to the source > > > tarball used for packaging. Someone who checks out a d-science > > > repository does not need to know where the tarball comes from (github, > > > bitbucket, PyPI...), he or she can just check it out using pristine-tar > > > on the packaging repository. > > > > Ok, I created a tar ball using a git archive (which matches what > > github does) and then used pristine-tar to check it in. > > I think this is a misunderstanding. You should write a debian/watch file > (line 22 of this template > > https://anonscm.debian.org/viewvc/debian-med/trunk/package_template/watch?revision=20511&view=markup > is your friend) and use the downloaded tarball when importing pristine-tar. Ok, done. Please note that there is no difference between downloading tar balls from github which uses 'git archive' to create them or creating them locally using 'git archive' (with the right arguments). This already produces bit-identical results (with the same hash)! So there is really no point in downloading upstream tarballs from Github when one has a local copy of the git repository. > > gbp can also create tar balls from the same tag and check > > in ione step, but somehow the hash does not seem to match > > exactly (the content does). I wonder why... > > You stumbled upon the very problem pristine-tar is solving: Tarballs > simply have different checksums even if featuring identical content. > This is for instance due to different time stamps, user ids etc after > unpackaging on a target system. Feel free to seek Debian lists for > several discussions explaining the problem. I already know ;-) I am (in)famous for starting a flamewar on debian-devel about reproducible builds in 2007... > You could add these in additional python-bart octave-bart binary > packages (sorry, matlab can not be provided as official Debian package). > You should read the according pages at wiki.debian.org where to put > Python modules (or you just check your local system where these are > stored) and Octave files (I never dealt with these but I guess there is > a wiki paga as well). Feel free to ask me if you are struck in the > jungle of documentation and I'll provide more specific pointers. Ok, I have to look at it. There are only very few small scripts, so I would rather put it in the same package. > Another remark to the packaging: Currently there is a libgsl migration > ongoing and you should use libgsl-dev instead of libgsl0-dev. Done. Although now it doesn't build locally on my Ubuntu machine anymore (only using pbuilder in a sid change root). Martin
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi, > > I also want to add octave/matlab/python scripts. But I am not > > sure where to put them. I would be nice if there was a way to > > add them to the default search path for octave/matlab, for > > example. > > You could add these in additional python-bart octave-bart binary > packages (sorry, matlab can not be provided as official Debian package). Take a look at the matlab-support package. Not a perfect solution, but maybe good enough Michael > You should raead the according pages at wiki.debian.org where to put > Python modules (or you just check your local system where these are > stored) and Octave files (I never dealt with these but I guess there is > a wiki paga as well). Feel free to ask me if you are struck in the > jungle of documentation and I'll provide more specific pointers. > > Another remark to the packaging: Currently there is a libgsl migration > ongoing and you should use libgsl-dev instead of libgsl0-dev. > > Kind regards > >Andreas. > > -- > http://fam-tille.de >
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi Martin, On Sun, Dec 06, 2015 at 11:59:02PM +, Uecker, Martin wrote: > > Put simply, pristine-tar is our way to encapsulate access to the source > > tarball used for packaging. Someone who checks out a d-science > > repository does not need to know where the tarball comes from (github, > > bitbucket, PyPI...), he or she can just check it out using pristine-tar > > on the packaging repository. > > Ok, I created a tar ball using a git archive (which matches what > github does) and then used pristine-tar to check it in. I think this is a misunderstanding. You should write a debian/watch file (line 22 of this template https://anonscm.debian.org/viewvc/debian-med/trunk/package_template/watch?revision=20511&view=markup is your friend) and use the downloaded tarball when importing pristine-tar. > gbp can also create tar balls from the same tag and check > in ione step, but somehow the hash does not seem to match > exactly (the content does). I wonder why... You stumbled upon the very problem pristine-tar is solving: Tarballs simply have different checksums even if featuring identical content. This is for instance due to different time stamps, user ids etc after unpackaging on a target system. Feel free to seek Debian lists for several discussions explaining the problem. > > Quick question, is the package supposed to install just the bart > > executable and its accompanying documentation or something else in > > addition ? > > I also want to add octave/matlab/python scripts. But I am not > sure where to put them. I would be nice if there was a way to > add them to the default search path for octave/matlab, for > example. You could add these in additional python-bart octave-bart binary packages (sorry, matlab can not be provided as official Debian package). You should read the according pages at wiki.debian.org where to put Python modules (or you just check your local system where these are stored) and Octave files (I never dealt with these but I guess there is a wiki paga as well). Feel free to ask me if you are struck in the jungle of documentation and I'll provide more specific pointers. Another remark to the packaging: Currently there is a libgsl migration ongoing and you should use libgsl-dev instead of libgsl0-dev. Kind regards Andreas. -- http://fam-tille.de
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Am Sonntag, den 06.12.2015, 13:57 + schrieb Ghislain Vaillant: > > On 06/12/15 12:32, Uecker, Martin wrote: > > Hi Andreas, > > > >>> Ok, I put it in /git/debian-med/bart.git as described in the > >>> debian-med policy. > >> I checked this out and added Vcs fields and Homepage to Debian control > >> (please pull). I noticed that the pristine-tar branch is missing in the > >> git repository. You can get this easily by redoing > >> > >> gbp import-orig --pristine-tar your_orig_tar_gz > >> > >> (just make sure you use the --pristine-tar option) and push the > >> pristine-tar branch. The rationale is that we can both easily work > >> on a byte identical tarball right from the Git archive. > > Ok, what is the deal with pristine-tar? Our upstream tar balls are > > generated by github using git-archive. So there does not seem to > > be any point to create tar balls just to re-import them using > > pristine-tar, or is there? > > Put simply, pristine-tar is our way to encapsulate access to the source > tarball used for packaging. Someone who checks out a d-science > repository does not need to know where the tarball comes from (github, > bitbucket, PyPI...), he or she can just check it out using pristine-tar > on the packaging repository. Ok, I created a tar ball using a git archive (which matches what github does) and then used pristine-tar to check it in. gbp can also create tar balls from the same tag and check in ione step, but somehow the hash does not seem to match exactly (the content does). I wonder why... ... > Running licensecheck on the source tree revealed a few files which have > missing copyright headers: ... > Which you may or may not want to act upon. Since the other source files > have one it might be picked up during the review process anyway. I don't see a reason to change this if it isn't necessary. > Otherwise, looks good. > > Quick question, is the package supposed to install just the bart > executable and its accompanying documentation or something else in > addition ? I also want to add octave/matlab/python scripts. But I am not sure where to put them. I would be nice if there was a way to add them to the default search path for octave/matlab, for example. Martin
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
On 06/12/15 12:32, Uecker, Martin wrote: Hi Andreas, Ok, I put it in /git/debian-med/bart.git as described in the debian-med policy. I checked this out and added Vcs fields and Homepage to Debian control (please pull). I noticed that the pristine-tar branch is missing in the git repository. You can get this easily by redoing gbp import-orig --pristine-tar your_orig_tar_gz (just make sure you use the --pristine-tar option) and push the pristine-tar branch. The rationale is that we can both easily work on a byte identical tarball right from the Git archive. Ok, what is the deal with pristine-tar? Our upstream tar balls are generated by github using git-archive. So there does not seem to be any point to create tar balls just to re-import them using pristine-tar, or is there? Put simply, pristine-tar is our way to encapsulate access to the source tarball used for packaging. Someone who checks out a d-science repository does not need to know where the tarball comes from (github, bitbucket, PyPI...), he or she can just check it out using pristine-tar on the packaging repository. I assume that the debian/control file is missing other Build-Depends than just debhelper. Yep. I added some. It builds using git-pbuilder. FYI, it should also be possible to build your source package without gbp, by checking out the source tarball with pristine-tar and calling dpkg-buildpackage on master. It is also a good way to test whether your clean target works as intended, by running dpkg-buildpackage twice and see whether it succeeds. If the clean target is insufficient, dpkg-buildpackage will complain of a mismatch between the source tarball and the content of the packaging tree. I'd also recommend to use cme fix dpkg-control which would bump your Standards-Version to 3.9.6 (which you can also do manually but cme is a nifty tool I'd like to introduce to newcomers). My experience so far: It pulls in a million of perl packages, then I had to figure out that I still have to install an extra package for dpkg-control to work, then it complains that it doesn't know libpng-dev, but if I replace it with libpng12-dev fixes it to be libpng-dev again. It also spits out a lot of incomprehensible warnings about the VCS-* lines. It didn't bump the standards version, but decides to do some arbitrary reformatting ;-) Yes cme can be a bit picky sometimes. Anyway, Standards-Version should be set to 3.9.6 indeed, which Lintian would complain about later when you build the package. Running licensecheck on the source tree revealed a few files which have missing copyright headers: licensecheck -r --copyright . | grep "No copyright" ./vars.m: *No copyright* UNKNOWN ./update-version.sh: *No copyright* UNKNOWN ./makedoc.sh: *No copyright* UNKNOWN ./src/simu/phantom.h: *No copyright* UNKNOWN ./src/simu/sens.h: *No copyright* UNKNOWN ./src/mat2cfl.c: *No copyright* UNKNOWN ./src/wavelet3/wl3-cuda.h: *No copyright* UNKNOWN ./src/wavelet3/wavthresh.h: *No copyright* UNKNOWN ./src/linops/sum.h: *No copyright* UNKNOWN ./src/linops/ufft.h: *No copyright* UNKNOWN ./src/linops/sampling.h: *No copyright* UNKNOWN ./src/linops/rvc.h: *No copyright* UNKNOWN ./src/dfwavelet/prox_dfwavelet.h: *No copyright* UNKNOWN ./src/sake/sake.h: *No copyright* UNKNOWN ./src/calib/calmat.h: *No copyright* UNKNOWN ./src/calib/cc.h: *No copyright* UNKNOWN ./src/num/simplex.h: *No copyright* UNKNOWN ./src/num/convoaa.h: *No copyright* UNKNOWN ./src/num/shuffle.h: *No copyright* UNKNOWN ./src/num/shuffle.c: *No copyright* UNKNOWN ./src/num/filter.h: *No copyright* UNKNOWN ./src/iter/vec.h: *No copyright* UNKNOWN ./src/iter/vec.c: *No copyright* UNKNOWN ./src/ismrm/read.h: *No copyright* UNKNOWN ./src/misc/version.h: *No copyright* UNKNOWN ./src/misc/cppwrap.h: *No copyright* UNKNOWN ./src/misc/pcaa.h: *No copyright* UNKNOWN ./src/misc/version.c: *No copyright* UNKNOWN ./src/misc/utils.h: *No copyright* UNKNOWN ./src/misc/cppmap.h: *No copyright* UNKNOWN ./src/misc/png.h: *No copyright* UNKNOWN ./src/misc/dicom.h: *No copyright* UNKNOWN ./src/bbox.c: *No copyright* UNKNOWN ./ar_lock.sh: *No copyright* UNKNOWN ./update-if-changed.sh: *No copyright* UNKNOWN ./git-version.sh: *No copyright* UNKNOWN ./matlab/bart.m: *No copyright* UNKNOWN ./matlab/writecfl.m: *No copyright* UNKNOWN ./matlab/readcfl.m: *No copyright* UNKNOWN ./scripts/rtnlinv.m: *No copyright* UNKNOWN ./vars.sh: *No copyright* UNKNOWN ./octview.m: *No copyright* UNKNOWN Which you may or may not want to act upon. Since the other source files have one it might be picked up during the review process anyway. Otherwise, looks good. Quick question, is the package supposed to install just the bart executable and its accompanying documentation or something else in addition ? Cheers, Ghis
Re: [MoM] Re: bart - tools for computational magnetic resonance imaging
Hi Andreas, > > > > Ok, I put it in /git/debian-med/bart.git as described in the > > debian-med policy. > > I checked this out and added Vcs fields and Homepage to Debian control > (please pull). I noticed that the pristine-tar branch is missing in the > git repository. You can get this easily by redoing > > gbp import-orig --pristine-tar your_orig_tar_gz > > (just make sure you use the --pristine-tar option) and push the > pristine-tar branch. The rationale is that we can both easily work > on a byte identical tarball right from the Git archive. Ok, what is the deal with pristine-tar? Our upstream tar balls are generated by github using git-archive. So there does not seem to be any point to create tar balls just to re-import them using pristine-tar, or is there? > I assume that the debian/control file is missing other Build-Depends > than just debhelper. Yep. I added some. It builds using git-pbuilder. > I'd also recommend to use > > cme fix dpkg-control > > which would bump your Standards-Version to 3.9.6 (which you can also do > manually but cme is a nifty tool I'd like to introduce to newcomers). My experience so far: It pulls in a million of perl packages, then I had to figure out that I still have to install an extra package for dpkg-control to work, then it complains that it doesn't know libpng-dev, but if I replace it with libpng12-dev fixes it to be libpng-dev again. It also spits out a lot of incomprehensible warnings about the VCS-* lines. It didn't bump the standards version, but decides to do some arbitrary reformatting ;-) Martin
