from:"Yaroslav Halchenko"

On Fri, 04 Sep 2020, Steffen Möller wrote:

>  * sharing data between colleagues - can you have two different versions
> at the same time?

sure, similar to git... well -- it is git ;)  so multiple versions
across collaborators, multiple versions on your own box  etc -- all
possible.  When you get into it really, you might even like to start
using BTRFS as your filesystem -- provides awesome CoW feature so you
could breed your huge datasets without wasting too much space.

Re versions: especially mind blowing is the ability to quickly switch
between versions -- "large" files are just symlinks.  The only gotcha
remaining -- switching between dataset with subdatasets versions
is not yet "convenienced", but it is possible to have multiple dataset
hierarchy clones of different versions.

>  * I see this mostly orthogonal to the question how we organize our data
> relative to whatever "dataRoot" we define

well -- you could have disjoint datasets, it is not required to bring
them all up into a superdataset, although that could have benefits.

>  * we still have a community-effort to collect the data from somewhere
> (which likely is not a git repository) and post-process it (like some
> indexing for a variety of tools) and to finally prepare the data somewhere

for "processing" checkout "datalad run" and datalad-container extension
providing "datalad container-run".  Then you could you have your
preprocessing entirely reproducible and simple provenance recorded
within git commits. 

handbook on that:
http://handbook.datalad.org/en/latest/basics/basics-run.html

And Michael ATM is actively looking into making snakemake to
tollerate datalad (well, git-annex), so you might like to define your
snakemake workflows

>  * with some agreement between us on how to formulate the metadata in a
> machine-readable manner so we know what tool needs to check out what
> files for which workflows

unfortunately cannot recommend anything specific ATM since not familiar
with bioinformatics metadata and its use within workflows.

> I should now read a bit in your handbook. And think a bit more about it
> over the weekend.

I hope you like it.  Adina and Michael did (well -- still doing) awesome
job with it.

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
WWW:   http://www.linkedin.com/in/yarik

Re: Datalad (Was: How to package human, mouse and viral genomes?)



On Thu, 03 Sep 2020, Andreas Tille wrote:

> On Thu, Sep 03, 2020 at 08:30:54PM +0200, Steffen M�ller wrote:
> > I looked at datasets.datalad.org.

> I'd strongly recommend the very entertaining talk from DebConf

> 
> https://saimei.ftp.acc.umu.se/pub/debian-meetings/2020/DebConf20/52-datalad-decentralized-management-of-digital-objects-for-open-science.webm

> Kind regards

>  Andreas. 

> PS: Yaroslav, I have not seen yet any commit of yours to add datalad
> to the science tasks.

guilty as charged!  Where are they now?

https://salsa.debian.org/science-team?utf8=%E2%9C%93=hpc 
https://salsa.debian.org/search?utf8=%E2%9C%93=false=_ref==science+tasks

all no good

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
WWW:   http://www.linkedin.com/in/yarik

Re: How to package human, mouse and viral genomes?

On Thu, 03 Sep 2020, Steffen Möller wrote:

> The name "datalad" (https://www.thesaurus.com/browse/lad) I definitely like.

;-)  The history goes:  it was called datagit, but naive Yarik decided
to asks FSF for a permission... so we had to come up with a new name, I
flew into Germany, we drank (khe khe -- discussed not yet taken names)
for the entire weekend. One of the requirements was: the mascot for the
project must be capable of wearing the orange fury coat (and that name
should be short and still available across social media etc).  The
best we came up with was FTF (Faster than Floppy, forget about mascot),
and I flew back... when I landed, Michael sent me "he got it" and it was
"DataLad" which made total sense if you think about what "git" is ;)

Some kick back though we were told about:
https://en.wikipedia.org/wiki/Lad_culture

> I suggest to collect more input/use cases over the weekend and then see
> how this goes. My immediate thought is that we describe our alternatives
> and try them all on a machine that we share.

We have a public matrix board, welcome to join
https://app.element.io/#/room/#datalad:matrix.org
if you have quick questions to address

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
WWW:   http://www.linkedin.com/in/yarik

signature.asc
Description: PGP signature

Re: How to package human, mouse and viral genomes?

On Thu, 03 Sep 2020, Steffen Möller wrote:

> I looked at datasets.datalad.org. I could well imagine to use your
> technology for other (larger) databases like Pfam or UniProt or PDB. For
> cute little genomes my initial reaction was that I felt overwhelmed.
> Your pointer will certainly help to define what we want. Many thanks!

FWIW, a few more notes since you seems to be interested ;): we do
have an elderly https://github.com/datalad/datalad-crawler/ "origin of
it all" but now just an extension to datalad which allows for efficient
"updates" and crawling of external resources. See e.g. this
asciinema/script: https://www.datalad.org/for/data-consumers

But in many use cases a straight "datalad addurls" command
(http://docs.datalad.org/en/stable/generated/man/datalad-addurls.html
part of handbook with an example:
http://handbook.datalad.org/en/latest/usecases/HCP_dataset.html?highlight=addurls#dataset-creation-with-datalad-addurls)
could be sufficient to "quickly" (depending on bandwidth and/or either
you use --fast option) populate a datalad dataset with files specified
in a spreadsheet/structured records.

So if you have some kind of .json or .csv/.tsv with records -- you could
try it quickly. addurls also automagically adds columns as git-annex
metadata per each file so someone could "toy around" (so far I
underused the feature) with "git annex views":
https://git-annex.branchable.com/git-annex-view/
or later to facilitate metadata extraction/aggregation/search.

A sample dataset (original cause for addurls to be written) is available
from http://datasets.datalad.org/?dir=/labs/openneurolab/metasearch if
you decide to explore (that data is open so no authorization for
access would be needed).

The problem you might encounter in your cases is (not that great)
scalability of git/git-annex to contain hundreds of files in a
single repo. So you might like splitting them into subdatasets (git
submodules) or providing custom views as I had mentioned before.

addurls makes it easy by establishing a subdataset whenever it
encounters // (instead of /) for path separation in the provided
filename.

PS I shut up now ;) Sorry for the flood of info. We are just very
excited for DataLad even though we had been working on it for over 6
years and should be sick of it and git-annex by now ;-) but we do
not!

--
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
WWW: http://www.linkedin.com/in/yarik

Re: How to package human, mouse and viral genomes?

You might like to listen to debconf20 talk on DataLad ;-)

At some point I have started even to establish some kind of dh-datalad
helper so that  .deb package would contain a datalad dataset
(git/git-annex repo), and would just `get`  data files upon
installation...   So -- yes, they would not be "self contained" but it
is infeasible for any sizeable data packages on debian.  But they could
be versioned, point to specific git state of corresponding datasets,
provide lightweight and efficient upgrades (only changed/new files would
need to be fetched), etc.  They could be partitioned into smaller
subdatasets or custom views to be provided, like we have

https://github.com/datalad-datasets/hcp-structural-preprocessed
which is a selection from a larger
https://github.com/datalad-datasets/human-connectome-project-openaccess

Never finished that helper though -- we just (develop and) use datalad
directly and had no debian packages which would need strict dependency
on the datasets.  More of sample  datasets could be found on
https://datasets.datalad.org/ -- data primarily comes from original
repositories, and covers now > 200TB

We had started to collect resources someone might like to datalad'ify
relevant to bioinformatics:
https://github.com/datalad/datalad/milestone/14?closed=1
but since we are not in bioinformatics field, never actually addressed
them.

I also know that https://github.com/notestaff is actively using
git-annex (not sure if datalad -- but he did submit some issues, so he
might) for bioinformatics.  Might be worth checking with him
if git-annex/datalad would be decided to be used.

On Thu, 03 Sep 2020, Steffen Möller wrote:

> Hello,

> We are closing in on the workflows. What is kind of missing are the
> mostly invariant inputs like the genomes of pathogens and very much so
> the reference genomes of the human, mouse, rat, worm, fly,  you name
> them.

> Other than a few years ago, hard drives are now big enough to
> accommodate the one or other genome and derivative indexes. Just - I
> don't think we want to organize in our regular Debian infrastructure
> something as variant as public genome (yes, they are still regularly
> updated, very much so) and that is so very security-irrelevant (just
> some data). Also, different sites will vary a lot in where this data
> shall be organized and all those scripts should likely be
> executed/initiated as/by non-root. There are public sites for this from
> where this data can be downloaded. Any redundancy to these sites imho
> mostly hurts us. The other side is that to just get something up quickly
> and for reproducibility tests, our infrastructure is difficult to beat.

> Please kindly throw your ideas at me how you would like whole genomes to
> be presented by Debian to the average user and to professionals. Just
> reply to this thread and/or send me "+1"s a PM and I summarize this up
> in a document which I suggest we then talk about in a jitsi meeting.

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
WWW:   http://www.linkedin.com/in/yarik

Re: dcm2niix -- I have an intent to take over (still under Debian Med team), objections?

2018-12-04 Thread Yaroslav Halchenko

On Tue, 04 Dec 2018, Andreas Tille wrote:

> On Tue, Dec 04, 2018 at 06:39:15AM +0100, Ghislain Vaillant wrote:
> > > I would be happy to move it to salsa.  I just pointed to it as the one
> > > we currently use

> > > I guess I would then move it to dcm2niix-epoch1 smth like that (on salsa
> > > i.e.) so previous versions could be found from the original
> > > dcm2niix repo (to which I would also add a commit obliterating
> > > everything and stating to look into that -epoch1 repo)

> > Please use the original packaging work on salsa and keep the history intact.

> > I don't see what would prevent you from rebasing the work done on
> > Neurodebian on top of mine. Epoch bumps don't justify history rewrites or
> > new repositories, imo.

> I agree with Ghislain.  Feel free to simply rebase your packaging but
> please keep the same repository name.  We have the policy that the
> package resides in a repository with the same name as the source package
> (and some of our tools are relying on this) and I do not see any reason
> to diverge from it.  There is also no point in keeping an unused
> repository of the old packaging around.

I am still wondering what I should do about dcm_qa* submodules.  I keep
thinking about making a dedicated package (dcm2niix-qa) shipping them.

Without that, I could of cause bring (merge) upstream git tree into your
packaging repo, with all the submodules, but that would "ruin" the clean
history it has now.  I would not be able just to "rebase" my packaging
on top without a merge if I am to bring submodules.

Meanwhile, pushed my adjusted neurodebian packaging to
http://github.com/neurodebian/dcm2niix
debian branch, which AFAIK should be good to go as long as we figure
out dcm_qa* situation. Will do a full sweep of builds and if all is
good, upload to NeuroDebian interim

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: dcm2niix -- I have an intent to take over (still under Debian Med team), objections?

2018-12-03 Thread Yaroslav Halchenko



On Mon, 03 Dec 2018, Andreas Tille wrote:

> Hi Yaroslav,

> On Mon, Dec 03, 2018 at 03:36:19PM -0500, Yaroslav Halchenko wrote:

> > Chris asked me to take over maintenance in Debian as well, and Ghislain
> > blessed me as well.

> Fine for me.

> > To minimize amount of work for myself, I would like
> > to continue with NeuroDebian packaging, just polishing it up (copyright
> > file and may be some cleanup).  How NeuroDebian packaging is
> > different (dropping historical perspective) ATM it is at
> > http://github.com/neurodebian/dcm2niix 

> Please do not do this.  The development platform for Debian packages is
> salsa.debian.org and *lots* of QA tools are relying to find the packaging
> code here.  I do not mind about the team but I mind a lot about the host
> any Blends package can be found.

I would be happy to move it to salsa.  I just pointed to it as the one
we currently use

I guess I would then move it to dcm2niix-epoch1 smth like that (on salsa
i.e.) so previous versions could be found from the original
dcm2niix repo (to which I would also add a commit obliterating
everything and stating to look into that -epoch1 repo)

> > - main difference is our git repo sitting on top of the upstream so we
> >   also can produce an .orig. tarball with dcm_qa submodule which
> >   provides data for testing of correct operation.  That increases the
> >   tarball size but IMHO it is worth it!

> I do not mind about the tarball size.

I also didn't but current one (280MB) made me think... not sure yet what
thoughts it would give ;)

> > ... removed all agreeed upon ...

> > Please let me know what you think

> All those packaging details sound sensible but please, pretty
> please stick to salsa.d.o.

sure

> Thank you for your work on this package

Cheers!
-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

dcm2niix -- I have an intent to take over (still under Debian Med team), objections?

2018-12-03 Thread Yaroslav Halchenko

Dear Team Comrades,

Upon blessing from Ghislain Antony Vaillant (previous team maintainer)
and Chris Rorden (upstream, CCed) I would like to take over the
maintenance of dcm2niix.  And I would like to do it largely by taking
the dcm2niix packaging setup we have in NeuroDebian.

bits of history

- we have initially packaged dcm2niix long ago (Sep 2015) but 
  forgot to file an ITP because there were a number of outstanding
  license issues to be resolved before considering for "Debian
  proper".

- Rightfully so, without seeing an ITP, Ghislain provided separate
  packaging in Dec 2016.

- I don't remember at which stage licensing issues got resolved ;)
  and that is when I realized that there is already a package in Debian.
  so we started to talk with Ghislain in Jan 2017 about possibly merging
  the effort, but never converged.

Chris asked me to take over maintenance in Debian as well, and Ghislain
blessed me as well.  To minimize amount of work for myself, I would like
to continue with NeuroDebian packaging, just polishing it up (copyright
file and may be some cleanup).  How NeuroDebian packaging is
different (dropping historical perspective) ATM it is at
http://github.com/neurodebian/dcm2niix 

- main difference is our git repo sitting on top of the upstream so we
  also can produce an .orig. tarball with dcm_qa submodule which
  provides data for testing of correct operation.  That increases the
  tarball size but IMHO it is worth it!

- we run (build-time only ATM) tests using that dcm_qa/ data.  That
  already allowed to iron out differences in behavior between i386 and
  amd64.

  I expect though that testing for all the other platforms would open a
  huge can of worms.  But I think it would be beneficial in the long
  run to name them all ;)  I bet Chris (the upstream) would be
  "thrilled" to help nailing them down

  any objections on relying on gbp to produce orig source tarballs?

- in a rush toward "let's converge packaging" I have added needed then
  epoch 1: to the versioning.It would need to "propagate" into
  Debian.  I hope that is ok

- we still use debhelper 9 for maximal ease of backportability.

  any objections?

- we do carry a few patches 
  https://github.com/neurodebian/dcm2niix/tree/debian/debian/patches
  including patches for the packaging backports on elderly jessie (and equally 
old ubuntus)
  
https://github.com/neurodebian/dcm2niix/blob/debian/debian/patches/jessie-dsc-patch
  probably some symlinks for really old /EOLed ubuntus could be removed,
  will do now

  But any objections against carrying backport patches?

Please let me know what you think
-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: mrtrix3 is "coming". Howto tests?

2018-09-07 Thread Yaroslav Halchenko



On Fri, 07 Sep 2018, Andreas Tille wrote:

> > to recreate it from git tree.  Without awareness of submodules, it would 
> > need
> > to keep the delta for the entire submodule tree.  Not sure if that is worth 
> > to
> > breed across commits, since it would just keep adding those 16MB with every
> > pristine-tar commit

> Well, if you have good reasons to derive from policy (I think you gave
> somehow good reasons considering your workflow ... ) please at least mention
> this in debian/README.source to inform other team members.

Done, Sir!

> Kind regards and thanks for working on this

Regards, Sir! ;)

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: mrtrix3 is "coming". Howto tests?

2018-09-06 Thread Yaroslav Halchenko



On Thu, 06 Sep 2018, Andreas Tille wrote:

> On Wed, Sep 05, 2018 at 04:12:55PM -0400, Yaroslav Halchenko wrote:

> > > I know there is some way to create pristine-tar from plain Git packaging.
> > > I'd be really happy if you would consider this since it enables other
> > > team members to rebuild the package with the workflow described in team
> > > policy.

> > I do not think pristine-tar would work with git submodules, thus
> > requiring a delta of the size of the testing/data .

> Pristine-tar has nothing to do with the structure of the git repository.

$> grep ls-tree /usr/bin/pristine-tar
"git ls-tree -r --full-name $branch | git update-index --index-info");
  open(LIST, "git ls-tree $b --name-only | sort -Vr |");

...

(git)hopa:~exppsy/mrtrix3[debian]git
$> du -scm .git/objects 
26  .git/objects
26  total

$> pristine-tar commit 
../build-area/mrtrix3_3.0\~rc3+git86-g4b523b413.orig.tar.gz master
warning: pristine-gz cannot reproduce build of 
../build-area/mrtrix3_3.0~rc3+git86-g4b523b413.orig.tar.gz; storing 96% size 
diff in delta (16653927 bytes)
(Please consider filing a bug report so the delta size can be improved.)
pristine-tar: committed mrtrix3_3.0~rc3+git86-g4b523b413.orig.tar.gz.delta to 
branch pristine-tar
pristine-tar commit  master  59.63s user 1.09s system 102% cpu 59.043 total

$> du -scm .git/objects 
 
56  .git/objects
56  total


> It just stores metainformation of the orig.tar.gz you produced in a
> separate branch to enable gbp recreating a byte-identical tarball as you
> used to upload.

to recreate it from git tree.  Without awareness of submodules, it would need
to keep the delta for the entire submodule tree.  Not sure if that is worth to
breed across commits, since it would just keep adding those 16MB with every
pristine-tar commit

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: mrtrix3 is "coming". Howto tests?

2018-09-05 Thread Yaroslav Halchenko



On Wed, 05 Sep 2018, Andreas Tille wrote:

> Hi Yaroslav,

> On Tue, Sep 04, 2018 at 07:51:52PM -0400, Yaroslav Halchenko wrote:

> > The interesting aspect is that upstream repo provides git submodule
> > testing/data .  It is not that large - just 15 MB compressed but it
> > doesn't change frequently.  There is two ways to handle it:

> > - "Lazy" me would love just to wrap it in the mrtrix3 source
> >   package (not yet sure what will be "upstream source releases", I
> >   am packaging straight from git).  It would be super easy thanks to gbp
> >   ability to include submodules (I do it for dcm2niix neurodebian pkg)

> > - Debian soul in  me says "spend more time and generate one more
> >   source/binary package which would be useless of the mrtrix3 context.

> > What do you think?

> I know there is some way to create pristine-tar from plain Git packaging.
> I'd be really happy if you would consider this since it enables other
> team members to rebuild the package with the workflow described in team
> policy.

I do not think pristine-tar would work with git submodules, thus
requiring a delta of the size of the testing/data .

gbp  can be used to produce .orig.tar.gz and I have added 
https://salsa.debian.org/med-team/mrtrix3/blob/debian/debian/gbp.conf
which instructs to package submodules as well, so anyone could generate
a new .orig.tgz for the new upload, and fetch from packages.d.o for an
existing version.

Is there a better way?

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

mrtrix3 is "coming". Howto tests?

2018-09-04 Thread Yaroslav Halchenko

Hello Team,

We have mrtrix 0.2.x series in Debian which is still used by some.
Upstream worked hard and soon (hopefully) will push out 3.0 release as
mrtrix3.  I've migrated and tuned up packaging for it (for
3.0~rc3+git86-g4b523b413) which is now at
https://salsa.debian.org/med-team/mrtrix3

The interesting aspect is that upstream repo provides git submodule
testing/data .  It is not that large - just 15 MB compressed but it
doesn't change frequently.  There is two ways to handle it:

- "Lazy" me would love just to wrap it in the mrtrix3 source
  package (not yet sure what will be "upstream source releases", I
  am packaging straight from git).  It would be super easy thanks to gbp
  ability to include submodules (I do it for dcm2niix neurodebian pkg)

- Debian soul in  me says "spend more time and generate one more
  source/binary package which would be useless of the mrtrix3 context.

What do you think?

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: Using recent packages on stable systems (Was: Depends available in other PPA's)

2018-03-09 Thread Yaroslav Halchenko


On Fri, 09 Mar 2018, Andreas Tille wrote:
> > > > - a lot of people are going to show up running ubuntu.� What might you 
> > > > all
> > > > recommend?� In theory ubuntu people could add debian testing to
> > > > `apt/sources.list.d`.
> > > I do not think that it is a good idea to mix Debian testing with Ubuntu
> > > randomversion.  IMHO the most sensible way to go would be to use the
> > > backporting system the NeuroDebian team has developed.  They are doing
> > > automatic backports to Debian stable / oldstable and several Ubuntu
> > > versions.  Its a real pitty that this nifty system is only used by
> > > NeuroDebian and nobody took the time to make this a bit more generic to
> > > make it useful for all Blends (NeuroDebian team are basically two very
> > > kind and helpful but totally overworked people who will not spent their
> > > time to port their scripting system for any other use - so we are on our
> > > own if we want to use it).
> > Hmm... the 'Get NeuroDebian' section at neuro.debian.net does look nice.

> +1

> > I don't suppose the code for the backporting system is hosted at a
> > particular place that I could take a look at?� Since I am new I probably
> > couldn't/shouldn't do much, but I would be interested in taking a look.

> NeuroDebian members usually are reading this list but I'm CCing their
> list explicitly hereby.  I guess it will be in some public Git repository
> which you can clone and enhance.

moreover it is just an

   apt-get install neurodebian-dev

away from you :-)   Also in git at:
http://git.debian.org/?p=pkg-exppsy/neurodebian.git
http://github.com/neurodebian/neurodebian
so choose your poison

A few bullet points relating to it

- our distributed and still used by me system is aged but still works.
  Michael has/is using a newer version
  he came up with for scheduling builds using condor etc.

- the core backporting helper (backport-dsc) is aged  but still
  working.  It was "offered" to be included to devscripts
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=660208
  but it never happened.

  That one allows to automate creating backport source package, possibly
  while also applying distro specific patches (quite often that is what
  keeps me remaining benevolent maintainer for some packages since
  others prefer to not have any additional non-debian patches within
  Debian source package)

- nd_backport  is just a wrapper around backport-dsc to provide
  neurodebian specifics

- nd_build  is to build a (backported) source package on any cowbuilder

- nd_build4allnd  wrapper/looper around nd_backport and nd_build to
  build on all "supported" Debian/Ubuntus given an original .dsc file

here is an elderly howto which might still work 90%-100% of the way
http://neuro.debian.net/blog/2012/2012-04-14_ndtools.html

hope this helps
-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: Sprint-derived paper now out:

2017-11-17 Thread Yaroslav Halchenko

On Fri, 17 Nov 2017, Steffen Möller wrote:

> Dear all,

> this paper has now surfaced

> Möller, Steffen; Prescott, Stuart W.; Wirzenius, Lars; Reinholdtsen,
> Petter; Chapman, Brad; Prins, Pjotr; Soiland-Reyes, Stian; Klötzl,
> Fabian; Bagnacani, Andrea; Kalaš, Matúš; Tille, Andreas; Crusoe, Michael
> R. (2017): Robust cross-platform workflows: how technical and scientific
> communities collaborate to develop, test and share best practices for
> data analysis. Data Science and Engineering.
> https://doi.org/10.1007/s41019-017-0050-4

Wow -- it is a really nice one, very informative and integrative.
Congrats!  Thanks for sharing the link as well!

To give a little bit of food for thought for the next sprints etc.  In
the paper you already make a good accent on standards, as the driving
force for more efficient and reliable work and resultant environments.
But as data are actually the primary object of operation on, and the
artifact produced are typically data files, it might be nice to
also give additional special attention to data (and meta-data)
standardization.  Even though Debian etc projects are working on
software/tools integration, and are agnostic of the data formats used by
those, we are in a good position to promote through demonstration the
benefits of  data standardization.

As for the background -- in neuroimaging we are pushing toward data
files standardization at the level of the entire experiments now (not
just per file): http://bids.neuroimaging.io .  Adherence to the standard
allows for more efficient collaboration, meta-data manipulation
(integration, querying etc), cross-tools integration etc.  CWL, as far
as I see it (I have no working experience with it), seems to provide
some level of data description, if not standardizing types/layout of
input, just again emphasizing on the benefit from such standardizations.

Once again, congrats!
Cheers,
-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: Auto creation of docker images from Debian (Med) packages?

2017-10-12 Thread Yaroslav Halchenko


On Thu, 12 Oct 2017, Andreas Tille wrote:
> > we auto generate docker images using their stock utility (stockbrew or what 
> > it became), see https://github.com/neurodebian/dockerfiles

> > For singularity, we auto generate just one "ultimate one" - 
> > https://github.com/neurodebian/neurodebian/blob/master/Singularity on 
> > singularity hub

> thanks for your always helpful hints.

> I'd like to repeat myself again:  We definitely should think wider and
> try to generalise what those NeuroDebian guys are doing for Blend *in*
> general.  The effort to adapt it specifically to Debian Med is most
> probably close to the same but we open the chance to attract developers
> of other Blends to contribute to this idea.

> Again: Please think widely.  This will finally help also NeuroDebian
> once they might realise that all their stuff will be provided by a
> common Blends framework. ;-)

If I think just a bit wider might thought will become transparent...

meanwhile -- might be a good time to realize the blend-wide thoughts
into action -- new singularity (I will look into updating the package
for experimental for now) is out with new "toys":
checkout http://singularity.lbl.gov/release-2-4
in particular:

Singularity Registry If you want to host your own Registry, we listened and you 
are in luck! Singularity Registry is ready for your use, along with a portal 
for you to “register your registry” to make it easier to share your images. 
Singularity Hub, along with improved building and updated builders, will follow 
this release.

and it is still lonely there in the sregistry:
https://singularityhub.github.io/containers/ so blends could shine there!

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: Bug#862361: RFS: dcm2niix/1.0.20170429-1 [experimental]

2017-05-11 Thread Yaroslav Halchenko


On Thu, 11 May 2017, Ghislain Vaillant wrote:

> Package: sponsorship-requests
> Severity: normal

> Dear mentors,

> I am looking for a sponsor for the following package:

> * Package name: dcm2niix
>   Version : 1.0.20170429-1
>   Upstream Author : Chris Rorden
> * URL : https://github.com/rordenlab/dcm2niix
> * License : BSD
>   Section: science

> One can check out the package by visiting the following URL
> (debian/experimental branch):

>   https://anonscm.debian.org/git/debian-med/dcm2niix.git

> Changes since the last upload:

>   * New upstream version 1.0.20170429
>   * Update copyright information
>   * Drop the patch queue, no longer required
>   * Specify the source directory and build system
>   * Use the system zlib and openjpeg2 libraries
>   * Build and install the manpages with Sphinx

> Best regards,

uploaded, thanks!

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: [Neurodebian-devel] Help needed for pandas bug: Could anybody verify the suspicion that tzdata might have some influence?

2017-03-28 Thread Yaroslav Halchenko


On Tue, 28 Mar 2017, Andreas Tille wrote:

> > - I have already stated many times that if you want to move any of the
> >   core packages under bigger (-med, -science, etc) maintenance -- I
> >   don't mind.  But it shouldn't complicate my own work on those
> >   packages.  Someone's "mess" might as well be my "consistency" and I
> >   often can't afford spending time figuring out how this are now. And
> >   even worse time "fixing" up packaging (e.g. re-enabling testing,
> >   re-introducing dropped patches, etc) to make it as "messy" as it
> >   was before ;)

> It might be interesting to know what part of the "mess" is important to
> you.  

itemize "mess"y aspects (I was not the one to provide this
qualification) and I will let you know.  Could as well happen that
the answer will be "none"

> For instance it might help to know in advance if you are fine with
> the gbp layout specified by the packaging policy of the target team. :-)

yes -- it is fine.  I use gbp as well

> I fully agree that a migration of packaging to a team VCS should not
> include changing / dropping any patches in the first place.

good

> > - so far I still see only myself as the one who took care about pandas
> >   even after I cried out loud for help.  So helping instead of
> >   complaining (again) would be more helpful (I started reading
> >   this with an idea that we are talking about tzdata issue, but
> >   apparently we are just making water harder)

> While I know that you always was cooperating when we started discussing
> about some issue this discussion used to start only after rdepends of
> one of the Neurodebian packages were affected by removal from testing
> warnings.  When checking the bug log no response to those RC bugs was
> logged.  This is simply frustrating since this means a time delay of at
> least 10 days until somebody starts reacting while beeing totally
> unclear whether some work was done or not.  I fully understand if you
> are swamped with more important things but responding to an RC bug by
> tagging it help and forwarding the mail to interested parties - in this
> case Debian Science for instance - would help a lot.

would be great indeed

> May be you consider all this making water harder, but I consider not
> responding to RC bugs (without an appropriate VAC message) in times of
> freeze not responsible maintenance.  

yeah, agree.  In the long run, I may indeed need to shorten the list of
packages I maintain.  I am NOT on VAC virtually ever but indeed
seems getting short of time to provide adequate oversight at times.
Again -- actual help is welcome.

> I'd like to repeat here that I
> would not say so if there would be *any* message crying for help.  Since
> I also personally have no idea about Pandas and this issue I simply took
> some action and involved tzdata maintainers.  This could have happened
> 10 days ago and this is my main point.  (And sorry if I'm to German
> here. :-P )

it is ok -- I like Germans in general ;-)

> > - regarding original tzdata issue -- since we are just expressing
> >   sentiments here -- it would have been cool if maintainers of
> >   core packages would do 'reverse depends testing' before uploading (as
> >   I am trying to do in such cases) so we stop running after a gone train
> >   [see e.g. our ad-hoc messy helper
> >   
> > https://github.com/neurodebian/neurodebian/blob/master/tools/nd_build_testrdepends
> >   which I usually use successfully when preparing next uploads for
> >   cython, nibabel, etc]

> Perfectly valid argument, yes.  This again shows that you are doing
> really cool stuff in NeuroDebian which I totally appreciate.
> Unfortunately this does not make its way where it IMHO belongs like for
> instance at debian...@lists.debian.org.  Well, you wrote some helpful
> tool and have hidden it nicely out of reach for those people who are
> obviously in need of this.

apt-get install neurodebian-dev

I have pointed to it multiple times.
I think there is a (possibly better) alternative now

$> apt-cache show ratt
Package: ratt
...
Description-en: Rebuild All The Things!
 ratt (“Rebuild All The Things!”) operates on a Debian .changes file of 
a
 just-built package, identifies all reverse-build-dependencies and 
rebuilds
 them with the .debs from the .changes file.
 .
 The intended use-case is, for example, to package a new snapshot of
 a Go library and verify that the new version does not break any other
 Go libraries/binaries.

but I haven't tried it so YMMV.

You can help leading the initiative to promote it/them to debian-qa@ or
-devel@ or any other appropriate in your opinion channel.  I would
really appreciate!

> > - outdated (not pushed) git on github or alioth is all the same -- just
> >   me forgetting to push.  "Fixed now" (thanks) - and present on
> >   both github and alioth git://git.debian.org/git/pkg-exppsy/pandas.git
> Thanks.
> > Thanks

Re: Help needed for pandas bug: Could anybody verify the suspicion that tzdata might have some influence?

2017-03-28 Thread Yaroslav Halchenko


On Tue, 28 Mar 2017, Ghislain Vaillant wrote:

> On 28/03/17 10:37, Andreas Tille wrote:
> > PS: Yaroslav, you know my opinion about using Vcs outside of debian.org and
> > deriving from policies that are widely established.  Currently
> >   git://github.com/neurodebian/pandas.git
> > is not even featuring the latest uploads - last changelog entry is

> > pandas (0.19.2-2) unstable; urgency=medium

> >   * Exclude a number of tests while running on non-amd64 platforms
> > due to bugs in numpy/pandas

> >  -- Yaroslav Halchenko <deb...@onerussian.com>  Wed, 11 Jan 2017 12:13:05 
> > -0500


> > I'm sorry to repeat myself but you are not creating a welcoming
> > environment for people who intend to help.

> This sentiment is shared on my side too.

> It was very disappointing to discover that nipype will not be part of
> Stretch due to an RC, which looks like many of those the team fixed during
> the Numpy 1.12 migration [1]. Besides, the packaging repository is quite
> frankly a mess [2] and is outdated.

> Looking at the other packages maintained by NeuroDebian and affected by RCs
> [3] (which include nipy, dipy and pandas), the repository layout is
> inconsistent from one package to the next [4, 5, 6]. IMO, as an experienced
> packager who has contributed to many different teams, this completely
> cancels out the benefits of having packages team-maintained in the first
> place.

> I am wondering whether NeuroDebian should instead be focusing on maintaining
> high-quality backports of neuroimaging software for supported Debian and
> Ubuntu releases (a goal it currently fulfills very well), and leave the main
> packaging effort to other Debian packaging teams (Science, Med, Python...).

I will try to be short

- I have already stated many times that if you want to move any of the
  core packages under bigger (-med, -science, etc) maintenance -- I
  don't mind.  But it shouldn't complicate my own work on those
  packages.  Someone's "mess" might as well be my "consistency" and I
  often can't afford spending time figuring out how this are now. And
  even worse time "fixing" up packaging (e.g. re-enabling testing,
  re-introducing dropped patches, etc) to make it as "messy" as it
  was before ;)

- so far I still see only myself as the one who took care about pandas
  even after I cried out loud for help.  So helping instead of
  complaining (again) would be more helpful (I started reading
  this with an idea that we are talking about tzdata issue, but
  apparently we are just making water harder)

- regarding original tzdata issue -- since we are just expressing
  sentiments here -- it would have been cool if maintainers of
  core packages would do 'reverse depends testing' before uploading (as
  I am trying to do in such cases) so we stop running after a gone train
  [see e.g. our ad-hoc messy helper
  
https://github.com/neurodebian/neurodebian/blob/master/tools/nd_build_testrdepends
  which I usually use successfully when preparing next uploads for
  cython, nibabel, etc]

- outdated (not pushed) git on github or alioth is all the same -- just
  me forgetting to push.  "Fixed now" (thanks) - and present on
  both github and alioth git://git.debian.org/git/pkg-exppsy/pandas.git

Thanks in advance for your help!

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: dcm2niix

2017-01-11 Thread Yaroslav Halchenko


On Wed, 11 Jan 2017, Ghislain Vaillant wrote:

> On Wed, 2017-01-11 at 11:22 -0500, Yaroslav Halchenko wrote:
> > Hi Ghislain,

> > Thanks for packaging and uploading to dcm2niix!

> You're welcome.

> > It is a pity though that we duplicated the effort somewhat since
> > we maintained dcm2niix within NeuroDebian for a while and didn't upload
> > primarily due to some licensing issues we brought up with upstream and
> > which were later resolved (e.g. of console/ujpeg.*)

> Before packaging a piece of software for Debian, I systematically check
> whether an ITP has already been filed for it on the Debian BTS. There
> was none, so I went for it. The duplication is a pity, but I can't be
> blamed for following the standard procedure for contributing a new
> package to the archive.

;) oh -- it wasn't my intend to blame anyone.  Indeed, if to blame we
could blame us (NeuroDebian) since indeed I don't think we ever filed an
ITP for this one

> > It would be nice to converge and co-maintain a single package, may be
> > under some team, e.g our pkg-exppsy (neurodebian) team or  debian-med --
> > whatever you would prefer

> I believe the package should be maintained by the Debian Med Team, and
> subsequently backported to Debian Stable and Ubuntu LTS by NeuroDebian,
> if desirable. The primary source for derivative projects should remain
> Debian.

sure

> > our packaging is present on alioth and github:
> > git://git.debian.org/git/pkg-exppsy/dcm2niix.git
> > https://github.com/neurodebian/dcm2niix
> Ack.

> > unfortunately there is a stumbling stone on our end also -- versioning
> > since upstream was inconsistent and I was naive to switch from our
> > custom 0.0.date to their 'date' scheme they took for earlier
> > releases, and now they have switched to 1.0.date ... heh

> Ack.
> You are referring to the following issue [1], right?
> [1] https://github.com/rordenlab/dcm2niix/issues/28

I guess ;)


> > correct resolution, to account for poor us NeuroDebianoids would be to
> > introduce an epoch making it 1:1.0.20161101  so currently present
> > version in neurodebian (20160921+git16-g0339407-1~nd+1) would upgrade to
> > it.


> > What do you think? ;)

> Well, do I really have a choice?

;) now that you are the authority over the Debian's dcm2niix package --
more than ever ;)

> An alternative solution could be to convince Chris to revert to the old
> MMDD versioning scheme. This way both the Debian and NeuroDebian
> packages can be upgraded without such hack. He is about to make a new
> release so we should act fast.

indeed!  I will gently follow up on the
https://github.com/rordenlab/dcm2niix/issues/28
now

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: upload aghermann-1.1.0, fix #824574 and #828008

2017-01-04 Thread Yaroslav Halchenko

uploaded but note that it fails to build on anything but recent debians
aghermann_1.1.2-1_amd64.build   OK  4:59.08 real, 194.54 user, 37.90 sys, 
2278480 out
aghermann_1.1.2-1~nd70+1_i386.build FAILED  0:38.75 real, 6.05 user, 6.84 
sys, 82376 out
aghermann_1.1.2-1~nd70+1_amd64.buildFAILED  0:49.63 real, 6.35 user, 8.34 
sys, 87336 out
aghermann_1.1.2-1~nd80+1_i386.build FAILED  0:52.50 real, 8.14 user, 8.94 
sys, 96672 out
aghermann_1.1.2-1~nd80+1_amd64.buildFAILED  0:51.23 real, 7.49 user, 8.68 
sys, 102624 out
aghermann_1.1.2-1~nd90+1_i386.build OK  5:17.99 real, 216.12 user, 
33.92 sys, 2133400 out
aghermann_1.1.2-1~nd90+1_amd64.buildOK  5:01.61 real, 196.50 user, 
39.55 sys, 2225304 out
aghermann_1.1.2-1~nd+1_i386.build   OK  5:14.28 real, 215.76 user, 
34.07 sys, 2131576 out
aghermann_1.1.2-1~nd+1_amd64.build  OK  4:55.06 real, 193.71 user, 
37.91 sys, 2235328 out
aghermann_1.1.2-1~nd12.04+1_i386.build  FAILED  0:41.61 real, 5.88 user, 7.11 
sys, 83944 out
aghermann_1.1.2-1~nd12.04+1_amd64.build FAILED  0:40.71 real, 7.69 user, 6.76 
sys, 139376 out
aghermann_1.1.2-1~nd14.04+1_i386.build  FAILED  0:27.23 real, 5.44 user, 4.94 
sys, 97296 out
aghermann_1.1.2-1~nd14.04+1_amd64.build FAILED  0:34.88 real, 5.45 user, 5.63 
sys, 103488 out
aghermann_1.1.2-1~nd15.04+1_i386.build  FAILED  0:41.47 real, 6.92 user, 6.92 
sys, 105496 out
aghermann_1.1.2-1~nd15.04+1_amd64.build FAILED  0:41.93 real, 6.72 user, 7.00 
sys, 112208 out
aghermann_1.1.2-1~nd15.10+1_i386.build  FAILED  0:41.57 real, 6.78 user, 6.95 
sys, 109416 out
aghermann_1.1.2-1~nd15.10+1_amd64.build FAILED  0:41.99 real, 6.50 user, 7.20 
sys, 116392 out
aghermann_1.1.2-1~nd16.04+1_i386.build  FAILED  2:46.26 real, 97.94 user, 23.00 
sys, 1269032 out
aghermann_1.1.2-1~nd16.04+1_amd64.build FAILED  2:48.69 real, 93.90 user, 26.01 
sys, 1289888 out
aghermann_1.1.2-1~nd16.10+1_i386.build  FAILED  2:46.31 real, 103.70 user, 
19.59 sys, 1476272 out
aghermann_1.1.2-1~nd16.10+1_amd64.build FAILED  2:50.67 real, 98.16 user, 24.29 
sys, 1546360 out


http://neuro.debian.net/_files/_buildlogs/aghermann/1.1.2

On Tue, 03 Jan 2017, andrei zavada wrote:

> Hi Yaroslav,

> Here's 
> http://alfinston.dlinkddns.com/johnhommer.com/code/aghermann/source/deb/aghermann_1.1.2-1.dsc
> for a new upstream version of aghermann, mostly with bugfixes but also
> with a more cme-compliant debian/control.

> Cheers,
> Andrei

> On 26 August 2016 at 01:29, andrei zavada <johnhom...@gmail.com> wrote:
> > Thanks muchly!

> > On 26 August 2016 at 01:26, Yaroslav Halchenko <deb...@onerussian.com> 
> > wrote:
> >> they were built and just waiting upload


> >> On Fri, 26 Aug 2016, andrei zavada wrote:

> >>> That was quick!

> >>> On 26 August 2016 at 01:15, Yaroslav Halchenko <deb...@onerussian.com> 
> >>> wrote:
> >>> > thanks ... built but then there was power outage.. my screen was "reset"
> >>> > so forgot the "TODOs"  for uploads



-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: Shall we develop a Debian Med Tutorial? Fwd: [BC2-conference] Call for workshop and tutorial proposals open now.

2016-11-30 Thread Yaroslav Halchenko


On Wed, 30 Nov 2016, Steffen Möller wrote:

> Hello,

> much like many of you I just received this invitation to come up with a
> tutorial and/or workshop for the Basel (Switzerland) conference. The
> ISMB (next time in Prague (CZ) if I recall correctly) I expect to send
> the same around any minute or I already missed it as they typically have
> deadlines in December.

> Question: Shall we? I am not so much after explaining how to prepare a
> Debian package but we should demonstrate the amazing wealth of software
> packages that are readily available to be installed and workflows that
> can be addressed with them. A key message could be that the principles
> explained in this tutorial one would be platform independent as
> VirtualBox does a decent job and also ready for clouds with Amazon and
> others. With respect to workflows I would then go and explain just any
> that is also covered by the Common Workflow Language development.

> Just ping me whoever is up for it. It is quite a bit of work and not so
> very rewarding, scientifically. We shall expect to be offered to write a
> book, which I would very much like to be something towards a
> "bioinformatics recipies" book. This kind of avoids the term "workflow".

YES!  you can also note that anyone using Debian, Ubuntu or any other
derivative, directly or in the cloud or via virtualbox/docker/whatnot
pretty much already uses "Debian Med" ;)

to accent on that you could link to other educational materials you
could find online which talk about installing anything Debian Med
related, e.g. like our http://neuro.debian.net/vm.html which
includes pointers to few youtube videos on how to get started using it.

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: RRID: a namespace for research software

2016-06-28 Thread Yaroslav Halchenko

Related (although we yet to "support" RRIDs) - have a look at 
http://duecredit.org , might be of interest if you are looking into sitting 
surviving methods and contributions, not only entire package

On June 28, 2016 5:57:00 PM GMT+02:00, Michael Crusoe 
 wrote:
>On Tue, Jun 28, 2016 at 9:10 AM, Steffen Möller
>
>wrote:
>
>>
>> On 27/06/16 16:22, Andreas Tille wrote:
>> > On Mon, Jun 27, 2016 at 10:04:40AM -0400, Michael Crusoe wrote:
>> >> http://identifiers.org/rrid/
>> >>
>> >> Example: http://identifiers.org/rrid/RRID:SCR_001156
>> >>
>> >> Where would be the best place to incorporate this into Debian? We
>are
>> >> adding support for this into the CWL spec so it would be nice to
>have a
>> >> resolver that give a Debian package name for a given RRID.
>> > Add a new field to debian/upstream/metadata ?
>> >
>> This is also what came to my mind. As it happens, we just proposed
>>
>> Registrations:
>>   - Registry: bio.tools
>> Entry: http://bio.tools/tool/DebianMed/bowtie/1.1.1
>>   - Registry: SEQwiki
>> Entry: http://seqanswers.com/wiki/Bowtie
>>
>> so you may want to add
>>
>>   - Registry: RRID
>> Entry: MIR:0558
>>
>> Well, actually, there does not seem to be an entry for Bowtie, yet
>>
>
>Search is at https://scicrunch.org/resolver?query=bowtie=
>
>Bowtie is http://identifiers.org/rrid/RRID:SCR_005476
>
>
>> I am uncertain if one wants to have a complete URL or just the ID.
>Matúš
>> preferred the complete URL but I do not see any such for MIRIAM.
>>
>
>Complete URL please. For khmer it is
>http://identifiers.org/rrid/RRID:SCR_001156
>
>
>>
>> For the resolver that you mentioned, this can at some point be
>implemented
>> via the unified Debian database.
>>
>
>Great!
>
>
>>
>> Best,
>>
>> Steffen
>>
>>
>
>
>-- 
>Michael R. Crusoe
>Community Engineer & Co-founder
>Common Workflow Language project
>https://impactstory.org/u/-0002-2961-9670
>michael.cru...@gmail.com
>+32 (0) 2 808 25 58
>+1 480 627 9108

-- 
Sent from a phone which beats iPhone.

Re: Open-source MRI hardware initiative project

2016-05-07 Thread Yaroslav Halchenko

I just want to complement (inlined with original email below) to
already good responses, and hope that you Broche would continue the
discussion by providing your feedback to our feedback ;)

On Sat, 09 Apr 2016, Broche, Lionel wrote:
> Hello Debian-Med team,

> I am a researcher in MRI hardware at the University of Aberdeen,
> Scotland. I am currently working on the development of a completely new
> type of MRI system (see ffc-mri.org), but I would like to avoid the
> traditional route of commercialisation as I see many problems with it.
> Instead, I have been thinking for a while of preparing an initiative for
> the development of open-source hardware in MRI.

> The aim of this initiative would be, in a first stage, to pool the
> technical solutions already in the public domain together so as to help
> small research labs like mine and, in a second stage, to create a rally
> point for these labs to share knowledge, resources and to organise
> collective work. If this proves successful, it may expand into a
> complete open source MRI hardware platform but that would be in the far
> future. I already approached several research groups who expressed their
> enthusiasm about this idea so there would be several academic
> participants to start with (at least 3, probably 5 to 8), and some of
> them are already willing to provide some designs.

> I would like to get some advices from the Debian Med community regarding
> several aspects:
> - What solution would you think is the most appropriate to organise a
> community portal? I do not have any IT help from my University on this
> project but I am willing to put a bit of my own money to get a server
> somewhere if necessary. Bear in mind that I have little training on how
> to maintain a website, though I can take some time to learn.

since no IT/$ support ATM, the best way IMHO would be to effectively use
available "Free" infrastructure:  

- github.com  -- create project organization, push your code there,
  provide source downloads for releases (so don't forget to tag them etc),
  use bug tracker system, ...

- travis-ci.org -- to the degree applicable, enable automatic
  testing/building/upload of your content (code, docs, ...)

- http://readthedocs.org -- if you develop documentation in rest/sphinx
  or even markdown -- could provide you automatic docs builds etc
  Very easy and efficient way to bootstrap documentation-oriented
  website

- if you need mailing list or forum, you could create a project on
  http://nitrc.org/, which is although specific to neuroinformatics,
  could still be useful.

> - I know there is an active part of Debian Med that works on MRI
> software (and make great things, actually!), would any of them be
> interested in this initiative? If yes, what would you expect from it, or
> what would you be willing to provide at this stage? Also, would you have
> some recommendations so that open-sourced MRI hardware would easily
> interface with the already existing open-source software?

other recommendations were already expressed, but I guess

- you might still want it to interface with existing PAC system be it
  open-source (orthanc, xnat, ...) or proprietary

- depending on your target "consumer" you might want to provide "out of
  box" options for export of data in data formats which neuroimaging
  analysis toolkits can consume, e.g. nifti

- in the long run, providing some kind of programmable interface to
  control the beast and obtain the data "online" for real-time
  acquisition "closed loop" systems could be a neat feature

- would be nice if there was a simulator... have you looked at 
  http://od1n.sourceforge.net
  Then interested folks could assess some kind of "applicability" I
  guess.


> - Would you have any suggestions regarding the conduct of such a
> project? I have no experience in the management of open source projects
> and I am actively looking for documentation about it. In particular, how
> can I organise this project so as to avoid bottlenecks in the future?

shameless ad: as per http://www.gigasciencejournal.com/content/4/1/31
make sure you asap clear out copyright/license for your work and choose
a DFSG-compatible free and open source license for your code, data,
documentation.  That could help to guarantee that you yourself
would have access to your work in the future, happen you change your
employer ;)

> - Can you see any funding bodies that could be interested in this
> initiative, in the short to medium term?

may be  https://www.incf.org/resources/funding-support, depending on
what kind of support you are looking for...  But according to
http://www.ffc-mri.org/funding.html you have already gotten various
grants in the past... so pursuing with the same agencies could be an
option.  So it all again depends I guess on what funds you are
looking for?

> - Do you know of organisations that would be interested to know of this
> project or to provide guidance? I already plan to contact OSHWA and the
> CERN

Re: Bug#814000: ITP: cwltool -- Common workflow language reference implementation

2016-04-20 Thread Yaroslav Halchenko


On Wed, 20 Apr 2016, Andreas Tille wrote:

> Hi Afif,

> On Tue, Apr 19, 2016 at 10:12:40PM -0700, Afif Elghraoui wrote:
> > > and the reason is given in the description of the task:

> > >Note that there is no according metapackage created since the packages
> > >might be to different to make sense to install all on one machine.

> > > I think this reason has a point - so we come back to the discussion how
> > > to get a sensible dependency between cwltool and our tasks ...

> > With the current setup, it seems like the answer should be to create
> > another metapackage for workflow management systems. Of the entries
> > currently in science-tools, at least pegasus-wms, snakemake, and cwltool
> > could go directly in there. Then both our tasks and science-tools can
> > refer to this new metapackage.

> I like this idea.  Feel free to do exactly this. :-)

FWIW +1 on that

additional candidates for inclusion if it would be a generic umbrella
for all workflow management systems:

python-nipype
coop-computing-tools

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: [MoM] Fwd: Outreachy project (CI for all biological applications inside Debian)

2016-03-22 Thread Yaroslav Halchenko

Fwiw, for all packages I maintain, wherever upstream provides tests, I am 
executing them at build time (scikit-learn, stats models, pymvpa2 are few 
random ones for eg). For many, I (or contributors) also started to enable 
autopkgtests, see eg pandas. Depending on where to draw the line for bio, those 
might be good examples.

On March 21, 2016 5:31:13 PM GMT+01:00, merlettaia  wrote:
>Hi Andreas,
>I've decided to start with Concavity.
>Is there a sample of a well-test-covered package?
>
>Tanya
>
>2016-03-21 10:27 GMT+03:00 Andreas Tille :
>
>> Hi Tanya,
>>
>> On Mon, Mar 21, 2016 at 08:26:17AM +0300, merlettaia wrote:
>> > My name is Tatiana Malygina, I'm another girl from Russia, who
>wants to
>> > help with writing tests for biological applications in Debian =)
>>
>> I admit I'm really happy to get several competent students for this
>> project since when I was registering it I was afraid that nobody
>would
>> sign on.
>>
>> > For Andreas: me and Ann were in the same study group in university,
>and
>> > actually we are friends. But out experience in bioinformatics and
>our
>> > primary research areas differ - that's why I want something
>structural
>> > bioinformatics-related as starting packaging project.
>> > For example, project from this list would be great:  dssp,
>Concavity,
>> > bio3d, ncols, norsnet, visualization (PyMol, garlic, ...) or
>>
>> Regarding PyMol:  We also need to get the new upstream version of
>PyMol
>> (1.8) build for Debian.  I remember there were some issues in
>building
>> it.
>>
>> > docking-related (autodock, autodock-vina, racoon, etc.). If I'm not
>a
>> > perfect candidate for outreachy internship, I would like to help
>with
>> > packaging for these tools as a volunteer - with some of them (I
>mean
>> > docking software) I had no prior experience, and for me it is a
>great
>> > chance to obtain it (and a motivation to finally read
>docking-related
>> > articles).
>>
>> I have the hope that we might get even two students since there is
>> outreachy program and Google summer of code.  No idea whether this
>works
>> - but from my perspective we have tasks even for two students. ;-)
>>
>> For sure any volunteer is welcome as well and the Mentoring of Month
>> project is open for anybody.  So getting some packaging training
>should
>> be granted.
>>
>> > I've created account on alioth (username - latticetower-guest), I
>also
>> like
>> > git.
>>
>> latticetower-guest is now member of the Debian Med team and has
>commit
>> permissions.  I'm also busy to migrate those packages from SVN to Git
>> which I consider sensible candidates for creating a test suite
>(recently
>> I moved stacks and exonerate - feel free to ask me for moving
>others).
>>
>> If you are interested in Garlic I noticed that the packaging is in a
>> quite old fashioned state.  We could negotiate with DebiChem team
>what
>> might be the best way to enhance the packaging (either you will get
>> commit permission to their VCS and we move it into their Git or we
>take
>> it over into Debian Med team.  Its also lacking a test suite.  Just
>let
>> us know where you would like to start.
>>
>> Kind regards
>>
>>   Andreas.
>>
>> --
>> http://fam-tille.de
>>

-- 
Sent from a phone which beats iPhone.

Re: bart - tools for computational magnetic resonance imaging

2015-12-01 Thread Yaroslav Halchenko


On Tue, 01 Dec 2015, Ghislain Vaillant wrote:
> >>I can set the packaging repository up on d-med and push your initial
> >>work to it if you like. That would make collaborative work with members
> >>of d-med much easier, myself included.

> >So I tried to do it myself. See my mail to Andreas.

> Perfect. Don't hesitate to ping me once you are happy with the
> current state of your packaging work. I am happy to help you on the
> final touch before review.

> Good luck and thanks for your contribution to Debian.

Indeed thanks in advance Martin.

And thanks Ghislain -- since of relevance, I have uploaded backports of
ismrmrd to NeuroDebian, happen you need them on some older
debian/ubuntu.  Whenever bart package gets into Debian -- can do the
same to it ;)

Cheers!
-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: Debian Med usage or download stats

2015-11-17 Thread Yaroslav Halchenko

On Tue, 17 Nov 2015, ti...@debian.org wrote:

>http://blends.debian.org/med/tasks/bio#emboss

> Emboss seems to be the most frequently used package maintained by the
> Debian Med team.  We have no number of those users who are disabling
> popcon and also no numbers who are using derivatives like Ubuntu or
> BioLinux.  So the numbers listed at the tasks page are *minimum* numbers
> that might be significantly higher but we can not know.

On a related topic, since might be of interest.  For NeuroDebian we are
providing neurodebian-popularity-contest package (which evil us made all
packages in neurodebian archive dependent on, so would not fly as is for
debian proper), which installs

$> cat /etc/popularity-contest.d/neurodebian.conf 
KEYRING="$KEYRING --keyring 
/usr/share/popularity-contest/neurodebian-popcon.gpg"
POPCONKEY="$POPCONKEY -r 0x544665CB8B48DE1D"
SUBMITURLS="$SUBMITURLS http://neuro.debian.net/cgi-bin/popcon-submit.cgi;

making popcon (if enabled on that box) report also to our popcon server
so we can collect stats from both (recent) debian and ubuntus:
http://neuro.debian.net/popularity.html#popularity-contest

so, similar setup I guess could be achieved for debian-med so you could
possibly get stats from biolinux and other derivatives if they encourage
installation of debian-med-popularity-contest or alike.

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: debian-med (possibly + -science) hackathon 2016 dates?

2015-10-15 Thread Yaroslav Halchenko


On Fri, 16 Oct 2015, Steffen Möller wrote:

> Hello,

> On 15.10.2015 23:52, Yaroslav Halchenko wrote:
> > Hi Andreas,

> > Sorry for me forgetting, when will it happen?  May be time to start
> > planing ... ;)


> Indeed. I put up
> https://wiki.debian.org/Sprints/2015/DebianMed2016
> so we can all start dumping our thoughts/expectations.

shouldn't it be 

https://wiki.debian.org/Sprints/2016/DebianMed2016

?
-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

debian-med (possibly + -science) hackathon 2016 dates?

2015-10-15 Thread Yaroslav Halchenko

Hi Andreas,

Sorry for me forgetting, when will it happen?  May be time to start
planing ... ;)

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Re: About backports.

2015-08-09 Thread Yaroslav Halchenko

On Mon, 10 Aug 2015, Charles Plessy wrote:
Like you indicated, that's not the only concern. On behalf of stable
release users, I think making a recent version of samtools available in
stable-backports is important-- I think the backports repository is a
huge reason why Debian stable is usable for everyday purposes (and it's
the main reason why I switched to Debian from Ubuntu for my own machines).

Backports are definitely great, and the requirement that a backported package
must have been in Testing is both an asset for the quality and a limitation
for
the logistics. In some cases, Testing migration is completely out of our
control, for instance with the GCC5 migrations currently.

I sometimes wonder if it would be better to have a separate repository for
providing Debian Med packages to Stable users. Ideally it would be a Debian
PPA, but these are not developed yet.

The big advantage that a separate repository would have over the traditional
backports, is that it would allow us to also distribute some packages for
Stable without rebuidling them, when they pass their regression tests on
Stable. This would make such a backport archive much more comprehensive.

FWIW, this is our approach in NeuroDebian: whenever possible enable build-time
testing, then build and upload at the same time to both main Debian archive
sid, and NeuroDebian -- backports for anything supported (Debians and Ubuntus).
Everything is streamlined with some elderly scripts (now in neurodebian-dev
package) around cowbuilder:

sudo nd_build4debianmain *dsc --hookdir=$HOME/neurodebian/tools/hooks { sudo
nd_updateall; sudo nd_build4allnd *dsc; }

So if it FTBFS for debian sid, I fall into that environment to troubleshoot it
etc. If passes -- all build envs get updated, and I build backports for all of
them. Above generates summary.build which tells which builds succeeded, and
where failed I look into corresponding .build file to see what needs to be
fixed.

This is of cause straightforward for leaf packages... if package is used
by other packages, and I suspect that it might cause various FTBFS and/or tests
failures, I usually also run downstream regression testing by rebuilding all
its dependees against old and new versions of the package [1] on Debian sid and
stable if I aim to upload backports.

[1]
http://anonscm.debian.org/cgit/pkg-exppsy/neurodebian.git/tree/tools/nd_build_testrdepends

--
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik

--
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150810013519.gh28...@onerussian.com

Re: sponsor upload aghermann-1.0.4 please?

2015-05-12 Thread Yaroslav Halchenko

I am traveling but I believe it was build and I just forgot about uploading 
it.. unfortunately laptop refuses to connect atm so if not uploaded by tomorrow 
I will do that in the morning
Cheers

On May 11, 2015 10:50:02 PM GMT+01:00, andrei zavada johnhom...@gmail.com 
wrote:
Hi Andreas,

As my trusty sponsor Yaroslav seems to have been preoccupied by other
matters since some weeks ago, could you please stand in his stead and
push
the button for my package?  There's a fix for some unfortunate
regression on
GTK3's part which screws the font sizes (they were rendered at 1pt),
and it
makes me feel for those 20-ish users who did install it.  The link to
the .dsc file is below:

 Using this opportunity, I would like to ever so slightly poke
Yaroslav to
 push the upload button on my own package, aghermann, where I fix a
FTBFS
 with gcc-5 to close #69

(http://johnhommer.com/academic/code/aghermann/source/deb/aghermann_1.0.4-1.dsc).
 
Cheers,
Andrei


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
https://lists.debian.org/87725925-8350-4d6b-b021-27819ab8b...@email.android.com

Re: sponsor upload aghermann-1.0.4 please?

2015-05-12 Thread Yaroslav Halchenko


On Tue, 12 May 2015, Andreas Tille wrote:

 Hi,

 On Tue, May 12, 2015 at 12:17:21AM +0100, Yaroslav Halchenko wrote:
  I am traveling but I believe it was build and I just forgot about uploading 
  it.. unfortunately laptop refuses to connect atm so if not uploaded by 
  tomorrow I will do that in the morning

 Since I just cloned the Git repository you would have lost in uploading. :-)

would indeed ;)  in particular because I also build for all
neurodebian supported debian/ubuntu releases -- so takes longer ;)

apparently I didn't build 1.0.4 yet anyways -- you won! ;)

 However, I have a question:  Is there any reason to keep on uploading
 to experimental?  Unstable should be fine, IMHO.

yeah -- Andrei, please adjust for upload to unstable and we will upload.

Cheers,
-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150512081803.gg17...@onerussian.com

Re: Debian Med Sprint in Montreal @ PyCon 2015?

2015-02-12 Thread Yaroslav Halchenko


On Thu, 12 Feb 2015, Michael Crusoe wrote:

Is there interest in having a Debian Med Sprint at PyCon 2015? The dates
are Monday, April 13th 2015 - Thursday, April 16th 2015.
I'll be there running a Sprint for the khmer project but I would be happy
to have Debian folk (both experienced and new contributors) there as well.
Is this a good idea? Who would be interested in co-hosting with me?

THAT WOULD BE GREAT!  My problem though is that I will be there
only for the duration of the conference, so wouldn't be able to stay
longer for the sprint.  But is there anything I could help with -- let
me know.

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150212165858.gc7...@onerussian.com

Re: aghermann-1.0.3, reproducible builds

2015-02-03 Thread Yaroslav Halchenko


On Tue, 03 Feb 2015, andrei zavada wrote:

 Hi Yaroslav,

 I see some red ink appearing on my QA page, this time suggesting to avoid
 using __DATE__ and __TIME__ macros in order to ensure reproducible builds:
 https://wiki.debian.org/ReproducibleBuilds/TimestampsFromCPPMacros.

 To address this, I have made appropriate changes to aghermann, resulting in
 version 1.0.3-1, which is right here for you to kindly sponsor upload:
 http://johnhommer.com/academic/code/aghermann/source/deb/aghermann_1.0.3-1.dsc.

well -- if you want to upload to unstable with purpose of propagating to
jessie -- clarify with debian-release if they would allow this change
in.

Otherwise change changelog release to experimental

 (The other package, cnrun, in the current version (2.0.1-1) awaiting admission
 to unstable, is good wrt this issue, so there's nothing to be done there.)

neurodebian package has been awaiting in NEW queue already for nearly a
record -- 6 months, and that is after addressing ftpmaster's
concerns... but what kind of admission are you talking about for cnrun?
I see nothing in NEW and for new upload -- should go to experimental as
well
-- 
-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150204000158.gv7...@onerussian.com

Re: preparing insighttookit 4.7

2015-01-30 Thread Yaroslav Halchenko


On Tue, 27 Jan 2015, Steve M. Robbins wrote:

 Ha!  Good point.  

 That reminds me: some years ago I considered creating a new source package 
 for 
 the ITK docs.  Never got further than considering it, but that would solve 
 the 
 builder problem, anyway.


  BTW, the i386 pbuilder build also went fine, so hopefully you shouldn't
  have any problems.

 Great!  I'm building now for the upload.

THANK YOU guys for the update and the upload!  Would you mind sharing
the source package somehow (while it is stuck in the NEW queue) -- I
might try build backports for NeuroDebian ;)

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150130180410.gw7...@onerussian.com

Re: Task for MRI (Magnetic Resonance Imaging) development

2014-12-08 Thread Yaroslav Halchenko


On Mon, 08 Dec 2014, Ghislain Vaillant wrote:

Dear all,

I originally started to contribute to the Debian ecosystem to help make
available some of the development library, such as [1], necessary to work
with MRI data. In the field of MRI processing, we are currently witnessing
a growing interest towards the release of opensource tools / framework
specific to MRI [2, 3]. Since most reconstruction clusters I know run on
Linux, I can see Debian playing a great part in providing a centralized
repository for distributing and releasing MRI related software. Before
that, It would be nice to have a centralized place to discuss and
coordinate packaging jobs for MRI software, which I thought the creation
of dedicated task [4] would officially acknowledge.

Hi Ghislain,

First of, does it sound silly ?

not all

Is that enough to justify the creation of a new task ?

in addition to
http://blends.debian.org/med/tasks/imaging
and
http://blends.debian.org/med/tasks/imaging-devel
?

There is already a bulk of MRI related software packages
contributed to listings in those two tasks

What would you call the task?  what software would you like to list
there?  If to make just MRI specific task -- not sure.

Would there be any interest from other people besides me ?
How much work would it represent to organize such task ?

not much ;)

Just testing the waters.

FWIW you might like also to know about the NeuroDebian project:
http://neuro.debian.net/ , which while concentrating on brains also is
quite MRI-centric ;)  We do package/maintain some software under
debian-med and debian-science blends umbrellas and list them in those
tasks pages.

As for the packaging discussion  place -- debian-med mailing list would
indeed be a good venue ;)

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20141208173652.gr14...@onerussian.com

Re: Big data needed for unit test

2014-12-01 Thread Yaroslav Halchenko


On Mon, 01 Dec 2014, Corentin Desfarges wrote:
I'm still working on the packaging of fw4spl, and I'm faced with a new
problematic : One of the unit tests needs to load an important data file,
which has a big size (~200 Mo).

such large data arrays are worth their own packages... then you could
have build-depend on it... before that -- indeed just disable that test

Should I simply remove this test, or can I include the data file in the
package ?

IMHO too big/too static (I bet) to be included  and then shipped again
and again and again with any new upstream software release.

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20141201143935.gw32...@onerussian.com

Re: please upload cnrun2

2014-11-25 Thread Yaroslav Halchenko


On Tue, 25 Nov 2014, andrei zavada wrote:

 On Mon, 24 Nov 2014 15:23:20 -0500
 Yaroslav Halchenko deb...@onerussian.com wrote:


  On Mon, 24 Nov 2014, andrei zavada wrote:

   On Mon, 24 Nov 2014 14:59:24 -0500
   Yaroslav Halchenko deb...@onerussian.com wrote:


On Sun, 23 Nov 2014, andrei zavada wrote:

 I am investigating this, and hopefully I'll find a fix soon.
 Supporting ancient distributions is always full of surprises like
 this.

so finally my upload was processed properly and cnrun is in the NEW ( I
guess you got the note)


   Indeed, I was notified in due course, earlier today.  It came right in the
   middle of my fidgeting with ax_lua.m4, which eventually helped.

   I think I can nail down any outstanding issues and... Now that 2.0.0 gets
   accepted, I suppose I could release a 2.0.1, then?

  yeah -- sure ;)


 There indeed were issues with ax_lua.m4 a couple of years ago, it seems; I
 have incorporated a recent, working version of that aclocal file into the
 distribution tarball, and now cnrun builds on all nd-supported arches
 (barring really old ones where g++ is pre-4.7).

 One notable exception is raring: nd_buildall reports the following:

 $ grep -C5 404 cnrun_2.0.1-1~nd13.04+1_i386.build
   man-db{a} pkg-config{a} po-debconf{a} texinfo{a} 
 0 packages upgraded, 40 newly installed, 0 to remove and 0 not upgraded.
 Need to get 1842 kB/12.3 MB of archives. After unpacking 41.4 MB will be used.
 Writing extended state information...
 Err http://ubuntu.media.mit.edu/ubuntu/ raring/main liblua5.1-0 i386 5.1.5-4
   404  Not Found
 Err http://ubuntu.media.mit.edu/ubuntu/ raring/main liblua5.1-0-dev i386
 5.1.5-4
   404  Not Found
 Err http://ubuntu.media.mit.edu/ubuntu/ raring-updates/main libxml2-dev i386
 2.9.0+dfsg1-4ubuntu4.3
   404  Not Found
 Err http://ubuntu.media.mit.edu/ubuntu/ raring/main lua5.1 i386 5.1.5-4
   404  Not Found
 Err http://ubuntu.media.mit.edu/ubuntu/ raring/main texinfo i386
 4.13a.dfsg.1-10ubuntu4
   404  Not Found
 E: Failed to fetch
 http://ubuntu.media.mit.edu/ubuntu/pool/main/l/lua5.1/liblua5.1-0_5.1.5-4_i386.deb:
  404  Not Found
 E: Unable to correct for unavailable packages

this one (raring) has EOL'ed already , so just remove from listing in 
/etc/neurodebian/cmdsettings.sh


 Of which I tend to think it is down to some temporary problem only appearing
 on ubuntu.medi.mit.edu.

 Kindly push the button again?  The .dsc URL is
 http://johnhommer.com/academic/code/cnrun/source/deb/cnrun_2.0.1-1.dsc

building

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20141125140945.ga32...@onerussian.com

upstream metadata bib

2014-10-30 Thread Yaroslav Halchenko

I remember seeing somewhere a compiled bibtex from the upstream metadata
we have been feeding our packages with... could someone remind me a url?

do we have already a tool to collate a .bib file given a list of
software in question?

Thank in advance!
-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20141031025059.gq30...@onerussian.com

Re: Compiling binaries on package installation


On Tue, 14 Oct 2014, Michael Banck wrote:

  Or should I point out how to compile your own, like it is done in the
  Building an optimized OpenBLAS packages on your architecture in the
  README.Debian file of openblas-base?

 Not sure what's in those files, but I suggest to support
 DEB_BUILD_OPTIONS=custom in your Debian packaging, which would build an
 optimized package for a the user.  That's what the ATLAS package are
 (were?) doing, and if we have enough packages like that, doing the m-a
 like tool as mentioned above would be a matter of knowing which packages
 do support it.

+1 BUT from a user perspective it would have been much more
convenient if there was e.g. 'atlas-custom-installer' package which
would do that automagically at the package installation time.  With
'build yourself' approach necessary manual interaction is also inferior
for delivering urgent functionality or security updates, since there
would be no automatic update.

You might also look at matlab-support-dev package and its rdepends
-- since Matlab is not distributable, with this package we help to
automate building of the Matlab extensions at the installation time,
using registered within matlab-support alternatives instance of
user-installed Matlab.  So far seemed to work quite neat.

Cheers,
-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20141014124033.gk18...@onerussian.com

Re: Compiling binaries on package installation


On Tue, 14 Oct 2014, Michael Banck wrote:
Or should I point out how to compile your own, like it is done in the
Building an optimized OpenBLAS packages on your architecture in the
README.Debian file of openblas-base?

   Not sure what's in those files, but I suggest to support
   DEB_BUILD_OPTIONS=custom in your Debian packaging, which would build an
   optimized package for a the user.  That's what the ATLAS package are
   (were?) doing, and if we have enough packages like that, doing the m-a
   like tool as mentioned above would be a matter of knowing which packages
   do support it.

  +1 BUT from a user perspective it would have been much more
  convenient if there was e.g. 'atlas-custom-installer' package which
  would do that automagically at the package installation time.

 Well, one downside of that would be that you'd need a development
 environment on every compute node you install it on, no?

yeap -- but Debian makes it so easy to get any kind of such an
environment, that for me use of a few more megs of disk space is not
really a cost to even worth mentioning ;)

 If you bundle a please rebuild my packages with optimization script
 with a local .deb archive, you'd only need to do it once on a frontend
 box.

and initially expose those as an apt repository, adjust apt configuration on
every node...   Then have 2 stage upgrade procedure (in first wave it
would rebuild locally build beasts, and place them into APT repo, then
apt-get update and {dist-,}upgrade to pick them up).

I am not saying that it all couldn't be done, I am just saying
that for a user it is not available out of the box without manual
configuration etc.

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20141014130016.gn18...@onerussian.com

Re: Compiling binaries on package installation


On Tue, 14 Oct 2014, Michael Banck wrote:
   If you bundle a please rebuild my packages with optimization script
   with a local .deb archive, you'd only need to do it once on a frontend
   box.

  and initially expose those as an apt repository, adjust apt configuration on
  every node...   Then have 2 stage upgrade procedure (in first wave it
  would rebuild locally build beasts, and place them into APT repo, then
  apt-get update and {dist-,}upgrade to pick them up).

 Sure, Einen Tod muss man sterben, as the germans say.

 I think it might be a potential GSoC project to come up with a
 mostly-integrated solution (at least on the build host).  The

+100! and not only on the build host ;)  meanwhile it would be
nice even to collect use-cases/examples when we would need such a beast

 configuration-management side of the APT repos on the client are of
 course an issue and cumbersome, but maybe out-of-scope.

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20141014132528.go18...@onerussian.com

Re: sponsor upload aghermann-1.0.2 fixing #764820


On Wed, 15 Oct 2014, andrei zavada wrote:

 Hi Yaroslav, Laurent,

 There was a bug (#764820) reported last week for aghermann-1.0.1, about menu
 entries missing icons.  So here's 1.0.2 with a fix (also for some lesser
 issues raised by current lintian), which I am humbly asking you to sponsor
 upload.

 The dsc link is
 http://johnhommer.com/academic/code/aghermann/source/deb/aghermann_1.0.2-1.dsc.

looks good
building now
upload -- tomorrow

cheers

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20141015004405.gd18...@onerussian.com

Re: insighttoolkit4 backport on wheezy 32bit -- 2 tests failures

2014-05-30 Thread Yaroslav Halchenko


On Fri, 30 May 2014, Gert Wollny wrote:
 About the failing tests: The test

itkTIFFImageIOCompression_RGBTestImageJPEG_JPEG 

 also fails on wheezy amd64. The error

   TIFFImageIO: error out of disk space 

 is nonsense though, this is the default ITK response to a failing
 TIFFWriteScanline. The real error message is sent to TIFFError and is
 printed out to stderr. I guess the relevant output is 

   JPEGLib: Bogus input colorspace.

 which would support the assumption that this is a problem with the
 libtiff version. 


 Considering this, one could indeed disable this test but then one should
 probably also disable the corresponding functionality.

indeed.  What would be a preferable way to disable it?

a. throw exception when trying to access it?
b. just remove from the API?

seems indeed to happen for me too on amd64, this test was not there in
4.5.0-3 so that one built fine on wheezy

 The other error message I didn't see on amd64, and I will see if I can
 do a recompile on i386.

   would you mind adding a direct dependency to insighttoolkit4 package's 
   control?

 Actually, it is currently not possible - libvtk5-dev is required
 depends on libtiff4-dev. 

I must be too slow on Friday -- could you elaborate?  even if
libvtk5-dev bdepends on libtiff4-dev, that doesn't forbid itk4 bdepend
on it...

 Considering that the transition to libvtk6-dev is pending in sid, it
 might become necessary to also backport this one, and since libvtk6-dev
 also depends on libtiff-dev, it should be possible to forced it to use
 libtiff5-dev on wheezy. I'm not sure whether itk is already vtk6 ready
 though. 

For now I would prefer to not care about vtk6 as for backport of current
itk4 to wheezy goes 

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20140530125741.gg19...@onerussian.com

Re: insighttoolkit4 backport on wheezy 32bit -- 2 tests failures

2014-05-30 Thread Yaroslav Halchenko


On Fri, 30 May 2014, Gert Wollny wrote:

 On Fri, 2014-05-30 at 08:57 -0400, Yaroslav Halchenko wrote:

   Considering this, one could indeed disable this test but then one should
   probably also disable the corresponding functionality.

  indeed.  What would be a preferable way to disable it?

  a. throw exception when trying to access it?
  b. just remove from the API?
 I've dug a bit deeper and what comes out is the following: 
 The test image is a four channel image, and now I assume, that the old
 version of libtiff doesn't support the alpha channel when writing to
 JPEG, because COMPRESSION_JPEG is already defined in the according
 tiff.h header file.

 I'd suggest to fallback to COMPRESSION_PACKBITS in itkTIFFImageIO.cxx
 for case TIFFImageIO::JPEG, old libtiff and more than three color
 components. (Patch attached) This way calling code doesn't break, and I
 doubt that the compression type is crucial for storing an image. 

cool -- thanks -- I will try it out
(lazy me just running now
sudo nd_build nd+debian wheezy i386 *~nd70*dsc 
--hookdir=$HOME/neurodebian/tools/hooks)
and will wait for it to fail ;)

 would you mind adding a direct dependency to insighttoolkit4 
 package's control?

   Actually, it is currently not possible - libvtk5-dev is required
   depends on libtiff4-dev. 

  I must be too slow on Friday -- could you elaborate?  even if
  libvtk5-dev bdepends on libtiff4-dev, that doesn't forbid itk4 bdepend
  on it...
 libvtk5-dev does not bdepend on libtiff4-dev, it *depends* on it, i.e. 
 when you install libtiff5-dev on wheezy, libtiff4-dev and libvtk5-dev
 will be removed (that's how I realized that there is a problem). 

gotcha

 The other question is of course, when tk is linked against libtiff4 and 
 itk against libtiff5 and both are loaded by program X then you can not
 be sure what code is actually called when a TIFF function is invoked
 that is available in both libraries.

indeed
-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20140530143319.gk19...@onerussian.com

insighttoolkit4 backport on wheezy 32bit -- 2 tests failures

2014-05-29 Thread Yaroslav Halchenko

Hi Guys,

I have (again) tried to provide a backport build of itk4 for wheezy (to
be shipped from NeuroDebian) and got those 2 tests to fail.

926 - itkTIFFImageIOCompression_RGBTestImageJPEG_JPEG (Failed)
2168 - itkOtsuThresholdImageFilterTestShort (Failed)

Would you be so kind to share your expertise on

1. either I could consider such a failure minor thus probably just
excluding those tests from being run

2. or may be you see how I could easily mitigate/fix underlying problem?

Details: actually looking at 926 -- it complains out of disk space but I
consider it VERY unlikely since that box has it in abundance :-/

926/2518 Test #926: itkTIFFImageIOCompression_RGBTestImageJPEG_JPEG 
...***Failed0.10 sec
Input Filename: 
/tmp/buildd/insighttoolkit4-4.5.2/BUILD/ExternalData/Testing/Data/Input/RGBTestImageJPEG.tif
Output Filename: 
/tmp/buildd/insighttoolkit4-4.5.2/BUILD/Testing/Temporary/TestTIFFCompression_RGBTestImageJPEG_JPEG.tif
Compression: JPEG
  Pixel type (string): rgba
  Component Type is unsigned_char
  Component size: 1
  Number of dimensions: 2
JPEGLib: Bogus input colorspace.
Unexpected exception in ImageFileWriter

itk::ExceptionObject (0x87da1b8)
Location: unknown
File: /tmp/buildd/insighttoolkit4-4.5.2/Modules/IO/TIFF/src/itkTIFFImageIO.cxx
Line: 1947
Description: itk::ERROR: TIFFImageIO(0x87d49f8): TIFFImageIO: error out of disk 
space


Exception detected while reading 
/tmp/buildd/insighttoolkit4-4.5.2/BUILD/Testing/Temporary/TestTIFFCompression_RGBTestImageJPEG_JPEG.tif
 :  Could not create IO object for file 
/tmp/buildd/insighttoolkit4-4.5.2/BUILD/Testing/Temporary/TestTIFFCompression_RGBTestImageJPEG_JPEG.tif
  Tried to create one of the following:
MetaImageIO
GDCMImageIO
JPEGImageIO
VTKImageIO
PNGImageIO
TIFFImageIO
BMPImageIO
NrrdImageIO
GiplImageIO
NiftiImageIO
  You probably failed to set a file suffix, or
set the suffix to an unsupported type.

DartMeasurement name=BaselineImageName 
type=text/stringINVALID_BASELINE_GIVEN/DartMeasurement


2168/2518 Test #2168: itkOtsuThresholdImageFilterTestShort 
..***Failed0.13 sec
 Start OtsuThresholdImageFilter  OtsuThresholdImageFilter (0x93c27b0)
  RTTI typeinfo:   itk::OtsuThresholdImageFilteritk::Imageshort, 2u, 
itk::Imageunsigned char, 2u, itk::Imageunsigned char, 2u 
  Reference Count: 3
  Modified Time: 66
  Debug: Off
  Object Name:
  Observers:  
StartEvent(SimpleMemberCommand)
EndEvent(SimpleMemberCommand)
ProgressEvent(SimpleMemberCommand)
IterationEvent(SimpleMemberCommand)
AbortEvent(SimpleMemberCommand)
  Inputs:
Primary: (0x93c2578) *
  Indexed Inputs:
0: Primary (0x93c2578)
  Required Input Names: Primary
  NumberOfRequiredInputs: 1
  Outputs:   
Primary: (0x93c4f50)
  Indexed Outputs:
0: Primary (0x93c4f50)
  NumberOfRequiredOutputs: 1
  Number Of Threads: 8
  ReleaseDataFlag: Off
  ReleaseDataBeforeUpdateFlag: Off
  AbortGenerateData: Off
  Progress: 0
  Multithreader:
RTTI typeinfo:   itk::MultiThreader
Reference Count: 1
Modified Time: 35
Debug: Off
Object Name:
Observers:
  none
Thread Count: 8
Global Maximum Number Of Threads: 128
Global Default Number Of Threads: 8
  CoordinateTolerance: 1e-06
  DirectionTolerance: 1e-06
  OutsideValue: 0
  InsideValue: 255
  Calculator: OtsuThresholdCalculator (0x93c5098)
  RTTI typeinfo:   
itk::OtsuThresholdCalculatoritk::Statistics::Histogramdouble, 
itk::Statistics::DenseFrequencyContainer2, short
  Reference Count: 1
  Modified Time: 51
  Debug: Off
  Object Name:
  Observers:
none
  Inputs:
Primary: (0)
  Indexed Inputs:
0: Primary (0)
  No Required Input Names
  NumberOfRequiredInputs: 0
  Outputs:
Primary: (0x93c77f8)
  Indexed Outputs:
0: Primary (0x93c77f8)
  NumberOfRequiredOutputs: 1
  Number Of Threads: 8
  ReleaseDataFlag: Off
  ReleaseDataBeforeUpdateFlag: On
  AbortGenerateData: Off
  Progress: 0
  Multithreader:
RTTI typeinfo:   itk::MultiThreader
Reference Count: 1
Modified Time: 46
Debug: Off
Object Name:
Observers:
  none
Thread Count: 8
Global Maximum Number Of Threads: 128
Global Default Number Of Threads: 8
  Threshold (computed): 0
  Mask image in use: 0
  Masking of output: 1
Progress  | 0 | 0.00390625 | 0.0078125 | 0.0117188 | 0.015625 | 0.0195312 | 
0.0234375 | 0.0273438 | 0.03125 | 0.0351562
 | 0.0390625 | 0.0429688 | 0.046875 | 0.0507812 | 0.0546875 | 0.0585938 | 
0.0625 | 0.0664062 | 0.0703125 | 0.0742188
 | 0.078125 | 0.0820312 | 0.0859375 | 0.0898438 | 0.09375 | 0.0976562 | 
0.101562 | 0.105469 | 0.109375 | 0.113281
 | 0.117188 | 0.121094 | 0.125 | 0.128906 | 0.132812 | 0.136719 | 0.140625 | 
0.144531 | 0.148438

Re: insighttoolkit4 backport on wheezy 32bit -- 2 tests failures

2014-05-29 Thread Yaroslav Halchenko


On Thu, 29 May 2014, Gert Wollny wrote:

 Hello Yaroslav, 


 On Thu, 2014-05-29 at 10:33 -0400, Yaroslav Halchenko wrote:
  I have (again) tried to provide a backport build of itk4 for wheezy (to
  be shipped from NeuroDebian) and got those 2 tests to fail.

  926 - itkTIFFImageIOCompression_RGBTestImageJPEG_JPEG (Failed)
  2168 - itkOtsuThresholdImageFilterTestShort (Failed)

  Would you be so kind to share your expertise on

  1. either I could consider such a failure minor thus probably just
  excluding those tests from being run
 I don't think we should disable tests without also disabling the
 according functionality. 

  2. or may be you see how I could easily mitigate/fix underlying problem?
 I will try to compile it myself to debug the errors (if I can reproduce
 them). Actually, I already tried a 

  nd_build nd+debian wheezy i386 ... 

 but the dependencies could not be resolved. Could it be that you
 backported a gccxml but didn't upload it to the neurodebian repo yet?

That is strange since if you used that command, it should have resulted
in identical environment (given that you nd_updatedist recently):

regarding the gccxml:

~/deb/builds/insighttoolkit4/4.5.2-1$ grep gccxml 
insighttoolkit4_4.5.2-1~nd70+1_i386.build | head
Depends: debhelper (= 9), cmake, swig (= 2.0), gccxml (= 0.9.0+cvs20120420), 
zlib1g-dev (= 1.2.2), libpng12-dev, libtiff-dev, libfftw3-dev, libdcmtk2-dev, 
libgdcm2-dev, libdouble-conversion-dev, uuid-dev, libminc-dev, libhdf5-dev, 
python-all-dev, libvtk5-dev, python-vtk
 pbuilder-satisfydepends-dummy depends on gccxml (= 0.9.0+cvs20120420); 
however:
  Package gccxml is not installed.
  gccxml{a} gettext{a} gettext-base{a} groff-base{a} hdf5-helpers{a}
Selecting previously unselected package gccxml.
Unpacking gccxml (from .../gccxml_0.9.0+cvs20120420-4_i386.deb) ...
Setting up gccxml (0.9.0+cvs20120420-4) ...

so it comes from the stock wheezy:

https://packages.debian.org/search?keywords=gccxml
wheezy (stable) (devel): XML output extension to GCC
0.9.0+cvs20120420-4: amd64 armel armhf i386 ia64 kfreebsd-amd64 kfreebsd-i386 
mips mipsel powerpc s390 s390x sparc 

Just in case -- uploaded my currently completed logs to 
http://neuro.debian.net/_files/_buildlogs/insighttoolkit4/4.5.2

from NeuroDebian it just gets
Unpacking neurodebian-popularity-contest (from 
.../neurodebian-popularity-contest_0.32~nd70+1_all.deb) ...
Unpacking libdouble-conversion1:i386 (from 
.../libdouble-conversion1_2.0.1-1~nd70+1_i386.deb) ...
Unpacking libdouble-conversion-dev (from 
.../libdouble-conversion-dev_2.0.1-1~nd70+1_i386.deb) ...


  Details: actually looking at 926 -- it complains out of disk space but I
  consider it VERY unlikely since that box has it in abundance :-/
 That smells like a integer conversion gone wrong. 

ah -- could well be... I had somewhat similar symptoms from apt while trying to
update some very elderly 32bit ubuntu chroot on this box...

 It could be that for
 the backport  one has to enforce libtiff5-dev.  

rright -- that one was not pulled in, while in sid it does it

$ wget -q -O- 
'https://buildd.debian.org/status/fetch.php?pkg=insighttoolkit4arch=i386ver=4.5.2-2stamp=1401264590'
 | grep libtiff5-dev 
  libtiff5 libtiff5-dev libtiffxx5 libtk8.5 libtorque2 libudev1 libunistring0
  libtiff5-dev libtiffxx5 libtk8.5 libtorque2 libudev1 libunistring0 libva1
Get:257 http://mirror.bm.debian.org/debian/ unstable/main libtiff5-dev i386 
4.0.3-8 [335 kB]
Selecting previously unselected package libtiff5-dev:i386.
Preparing to unpack .../libtiff5-dev_4.0.3-8_i386.deb ...
Unpacking libtiff5-dev:i386 (4.0.3-8) ...
Setting up libtiff5-dev:i386 (4.0.3-8) ...

would you mind adding a direct dependency to insighttoolkit4 package's control?

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


--
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20140529205536.gd19...@onerussian.com

Re: Fwd: paper: Custom software development for use in a clinical laboratory

2014-05-20 Thread Yaroslav Halchenko

very interesting -- thanks for sharing.  I wondered though, if such an
in-lab software releases under FOSS license could then it be re-used or
better built-upon in another lab without violating FDA/etc regulations?

In-house developments resting on the work of FOSS projects are
nice but is there a way those could contribute back to the ecosystem?

What if such are exposed as 'services' and not 'software/devices' --
could then they be re-used within current legal framework(s)?

On Tue, 20 May 2014, Andreas Tille wrote:

 Hi,

 I think this paper might also be of interest for readers of Debian Med
 list ...

 Kind regards

Andreas.

 - Forwarded message from Ian Malone ibmal...@gmail.com -

 Date: Tue, 20 May 2014 12:22:21 +0100
 From: Ian Malone ibmal...@gmail.com
 To: FedoraMedical medical-...@lists.fedorahosted.org
 Subject: paper: Custom software development for use in a clinical laboratory

 Hi, been a bit quiet here of late.
 Anyway, was reading this this morning and thought it might be of
 interest to some:
 J Pathol Inform. 2012; 3: 44.
 Published online Dec 20, 2012. doi:  10.4103/2153-3539.104906
 PMCID: PMC3551490

 Custom software development for use in a clinical laboratory

 John H. Sinard* and Peter Gershkovich
 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3551490/
 (Open Access)
-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist,Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20140520164623.gg19...@onerussian.com

Re: Q-Leap Networks contributes to Debian Med


On Thu, 24 Apr 2014, r...@q-leap.de wrote:
 Tony We appreciated the pizza, of course, but your interest in
 Tony supporting Debian Med is very welcome indeed. I've just moved
 Tony house, and I don't yet have a broadband connection. However,
 Tony when I get reconnected, I'll build a new Qlustar Beowulf and
 Tony try to get Bio-Linux running on it.

 Sounds good. Feedback appreciated. We're just finishing up release 8.1.1
 which will already have a small sign of Debian Med support. In the
 QluMan GUI, we've added chroot management in which you will be able to

oh that sounds interesting ;)  is any part of Qlustar, such as QluMan
GUI open-sourced?

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20140424125834.gs8...@onerussian.com

Re: Q-Leap Networks contributes to Debian Med


On Thu, 24 Apr 2014, r...@q-leap.de wrote:

  Yaroslav == Yaroslav Halchenko deb...@onerussian.com writes:

 Yaroslav On Thu, 24 Apr 2014, r...@q-leap.de wrote:
 Tony We appreciated the pizza, of course, but your interest in
 Tony supporting Debian Med is very welcome indeed. I've just moved
 Tony house, and I don't yet have a broadband connection. However,
 Tony when I get reconnected, I'll build a new Qlustar Beowulf and
 Tony try to get Bio-Linux running on it.

  Sounds good. Feedback appreciated. We're just finishing up
  release 8.1.1 which will already have a small sign of Debian Med
  support. In the QluMan GUI, we've added chroot management in
  which you will be able to

 Yaroslav oh that sounds interesting ;) is any part of Qlustar, such
 Yaroslav as QluMan GUI open-sourced?

 Everything in Qlustar is open-sourced apart from QluMan ( = cluster
 management framework including GUI).

would you be so kind to point to the sources?
https://www.google.com/search?q=open+source+site%3Aqlustar.com
returns empty and there is no obvious link on the website (or those are
only available as .dsc source packages from the repository?)

Pity that QluMan is not FOSS -- it is the one which got me intrigued ;)

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20140424141103.gw8...@onerussian.com

Re: Q-Leap Networks contributes to Debian Med


On Thu, 24 Apr 2014, r...@q-leap.de wrote:

  Yaroslav == Yaroslav Halchenko deb...@onerussian.com writes:

 Yaroslav On Thu, 24 Apr 2014, r...@q-leap.de wrote:

   Yaroslav == Yaroslav Halchenko deb...@onerussian.com
   writes:

 Yaroslav On Thu, 24 Apr 2014, r...@q-leap.de wrote:
 Tony We appreciated the pizza, of course, but your interest in
 Tony supporting Debian Med is very welcome indeed. I've just moved
 Tony house, and I don't yet have a broadband connection. However,
 Tony when I get reconnected, I'll build a new Qlustar Beowulf and
 Tony try to get Bio-Linux running on it.

   Sounds good. Feedback appreciated. We're just finishing up
   release 8.1.1 which will already have a small sign of Debian
   Med support. In the QluMan GUI, we've added chroot
   management in which you will be able to

 Yaroslav oh that sounds interesting ;) is any part of Qlustar, such
 Yaroslav as QluMan GUI open-sourced?

  Everything in Qlustar is open-sourced apart from QluMan ( =
  cluster management framework including GUI).

 Yaroslav would you be so kind to point to the sources?
 Yaroslav https://www.google.com/search?q=open+source+site%3Aqlustar.com
 Yaroslav returns empty and there is no obvious link on the website
 Yaroslav (or those are only available as .dsc source packages from
 Yaroslav the repository?)

 Exactly. Like Debian/Ubuntu everything is packaged as debs.

.debs are binary packages, I am inquiring about source packages (i.e.
dsc).  Would you be so kind to point me to the Debian repository
containing those?

NB https://qlustar.com/download provides a binary download, which is
   probably containing some GPL-licensed software, for which you should 
   inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d. (GPL v3).  So you might like to add a
   note/instructions on that page.


 Yaroslav Pity that QluMan is not FOSS -- it is the one which got me
 Yaroslav intrigued ;)

 Oh well. Maybe one day  ... :)

today is a nice day for doing great things -- and it might be better
than tomorrow ;)

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20140424143726.gy8...@onerussian.com

Re: Q-Leap Networks contributes to Debian Med


On Thu, 24 Apr 2014, r...@q-leap.de wrote:

 You're right. How about the added note on:

 https://www.qlustar.com/download

much better -- thanks!

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/2014042411.gg8...@onerussian.com

Re: Presentation of Debian Med on local Next Generation Sequencing workshop (April)

2014-03-24 Thread Yaroslav Halchenko


On Fri, 21 Mar 2014, Charles Plessy wrote:
 For packages without upstream tests, in the meantime, even the most simple
 tests like running the command with the --help option, are potentially useful.

+10 ;)  I also try to run some Demo or Example script if such is
provided (often via xvfb) to indeed catch the most broken beasts

 My favorite example is T-COFFEE, where we were distributing a 100 % broken
 package on arm for years.  With autopkgtest, we will have a better overview on
 what package works where (just a successful build is not enough), and this 
 will
 help people to evaluate the feasiblity of projects based on Debian on amd64,
 and also on more rarely used platforms (such as the Raspberry Pi as we read on
 this list).

well -- the best one IMHO here is the s390x -- big-endian 64bit.  It
helped to detect/iron out bugs for cases where unit-tests were not
developed to test those corner cases, but which naturally appear
when e.g. bytes get swapped ;)  armel is my favorite to detect race
conditions, since it is somewhat slower than regular x86, thus those
race conditions do not come that obvious ;)

 Slightly more complex tests can often be built from the examples in the
 program's README.  On my side, I am still on a learning phase with 
 autopkgtest,
 but I hope that collectively we will have a clear vision on what to recommend
 Upstream so that they can write or accept user-contributed tests are trivially
 run by autopkgtest.

well -- autopkgtest is indeed nice BUT for basic QA I do prefer
build-time testing, since that guarantees that package with known issues
doesn't even get to the users before issues get resolved.  And it goes
through testing on all supported architectures (if not architecture all
of cause).

My main reluctance with autopkgtest was the need to duplicate definition
of 'running tests' twice in debian/rules and then under debian/tests...
sadt seems might be of help to avoid this duplication now -- I will look
later to see if that would be possible to just call it (with
corresponding tune up of PATH/PYTHONPATH) with debian/rules...


 I also like a lot the “litteral programming” side of the “reproducible
 research” trend.  I am developing some tutorials based on real data analyses
 from my work, which are currently implemented in knitr
 (http://yihui.name/knitr/), and which I hope to iron out so that they can be
 used to test a Debian image in an integrated way.  However, they currently
 still use programs that are not yet packaged.

ha -- being not R user much -- haven't heard about knitr.  Looks
interesting indeed.

Python folks were blessed recently with IPython which now goes even
stronger forward (thanks to support from Sloan foundation and others).
If you managed to not hear about that: have a look at sample ipython
notebooks at http://nbviewer.ipython.org/ .

 Work in progress but comments welcome !  
 https://github.com/charles-plessy/tutorial

it is pity there is no some kind of 'knitrviewer.org' ;)

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20140324144419.gb28...@onerussian.com

Re: Presentation of Debian Med on local Next Generation Sequencing workshop (April)


On Wed, 19 Mar 2014, Carlos Borroto wrote:

 I lost some of my initial excitement about contributing to Debian Med.
 If I cannot use the results of my effort in the system where I
 actually do my job

quick side question: which are running ?

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20140319165030.gv28...@onerussian.com

Re: Presentation of Debian Med on local Next Generation Sequencing workshop (April)

On Wed, 19 Mar 2014, Diane Trout wrote:
  I think the more effort we spent in autopkgtest suites the more we will
  be able to come closer to reproducibility.  I hope that this effort will
  be rewarded by such projects who for whatever reason do not (yet) trust
  our work.

 There's also the issue of upgrades changing your environment.

 We've had issues with newer versions of cufflinks and tophat introducing new 
 bugs. 

and possibly fixing some of the existing ones ;)

 As a result the scientists our group tend to want to have consistent 
 versions installed.

this project of Michael Hanke could be of interest for you if you are
keen to join:  https://testkraut.readthedocs.org   The idea is to make
it easy to create regression tests of arbitrary (user) processing
pipelines.  I bet he would welcome contributions!

 I'm not sure if there's a good way of having multiple versions of a debian 
 package available on a single system. 

Theoretically it is possible and done for some packages.  But in general
it is too much burden and all boils down to man power.

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20140319232435.gy28...@onerussian.com

Re: Presentation of Debian Med on local Next Generation Sequencing workshop (April)


On Wed, 19 Mar 2014, Carlos Borroto wrote:
   I lost some of my initial excitement about contributing to Debian Med.
   If I cannot use the results of my effort in the system where I
   actually do my job

  quick side question: which are running ?

Are you asking about systems using a lot of software already packaged by
Debian but not using APT(at least not with the oficial repository) to
install such software?

;-) something like that.  I just was aiming at more specific pointer,
e.g. distribution/release, since that would also add to the picture.
E.g. using some stable but old(er) release of Debian or Ubuntu might
indeed demand users to get fresh versions installed manually (or from
non-official repositories, such as the ones you use or neuro.debian.net)

Beside the two big ones I already mentioned, Galaxy and CloudBioLinux, I
don't know any other good example of a popular system for computational
biology not using official packages. I also don't know of an example of a
system that does. BioLinux, from where CloudBioLinux comes, is a close,
but my experience is that the collaboration is not as close as you would
expected.
I have also used several institutional HPC clusters, some that are mainly
used for computational biology and run on Debian or Ubuntu. None relied on
oficial packages or even APT.

run on Debian or Ubuntu but not using packages or APT... oh well -- at
least it keeps some useful admins on a payroll ;)  joking aside though,
situation with institutional HPC clusters is indeed peculiar, since
they need to address many users from different disciplines, and
even if all the software would be provided by stock distribution, some
users would still demand custom versions and/or builds.

But when we would look at smaller deployments (labs/departments),
starting off with stock distribution/packages could save lots of
human power in the short and long run.  Then custom installations could
still be done, using the same 'environment modules' system as they use
across many HPCs.

Finally, let me repeat myself. I think Debian Med is doing a great job. I
hope more people could see the benefits of using the official packages. My
limited involvement allowed me to see how much you can get by following
good practices for packaging free software. My point was it seems there is
a need to highlight the importance of these benefits in a meeting like the
one mentioned in this thread. 

+1  ;)

Also a need for information on how to get
reproducibility but still use the official packages.

actually that could be the easiest form to achieve kind of
reproducibility: e.g.  if someone uses stock release of Debian, as I
have demonstrated, it is very easy to recreate very similar (if
not identical) environment in 1 (or few) commands (such as debootstrap).

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-med-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20140320020204.gb28...@onerussian.com

Re: Presentation of Debian Med on local Next Generation Sequencing workshop (April)