Re: CI for mapdamage (Was: Outreachy 2017 (Round 14))
Hi Nadiya, On Wed, Mar 29, 2017 at 06:36:41PM -0700, Nadiya Sitdykova wrote: > I uncommented all tests in run-unit-test and now they work perfectly fine > so I closed the bug #848326 inside changelog > Also I added some info to README.test, which I think could be useful for > users Thanks for your fine work. I've uploaded the package and opened a bug against release.debian.org to unblock the testing migration to make sure users of next stable release will profit from these changes. I'm really happy that we have spotted and fixed a package that was basically non-functional before. Kind regards Andreas. -- http://fam-tille.de
Re: CI for mapdamage (Was: Outreachy 2017 (Round 14))
Hello Andreas,
I uncommented all tests in run-unit-test and now they work perfectly fine
so I closed the bug #848326 inside changelog
Also I added some info to README.test, which I think could be useful for
users
Regards, Nadiya
On Wed, Mar 29, 2017 at 2:48 PM, Nadiya Sitdykova
wrote:
> Hello Andreas,
>
> >Did you do some investigation what dependency providing liblapack.so and
> >libblas.so are needed (see my mail from yesterday)?
>
> Yes, I tried to run the test with different packages installed, and here
> are results:
>
> packages| result| user
> time
>
> --
> libatlas-base-dev | Successful run | 11m24.204s
> libopenblas-dev | Successful run | 10m31.860s
> libblas-dev+liblapack-dev | Successful run | 10m34.368s
>
> It seems that libatlas-base-dev is less effective, but I'm not sure if we
> should exclude it from Depends, so I left if for now
>
> On Tue, Mar 28, 2017 at 11:29 PM, Andreas Tille wrote:
>
>> [Changed subject to split up to topics in one thread]
>>
>> Hi Nadiya,
>>
>> On Tue, Mar 28, 2017 at 10:23:00PM -0700, Nadiya Sitdykova wrote:
>> > 1) I read the paper about mapdamage and figured out that with given
>> pe.sam
>> > file the reference TGCGA wouldn't make sense as the particular type
>> of
>> > substitutions is specific for ancient DNA damages. So further for tests
>> I
>> > used reference CGATACGA -- which contains all types of nucleotides and
>> make
>> > sense regarding specifics of DNA damage.
>> >
>> > 2) But even after fixing (1) I still got an error (log attached). The
>> > problem is that on given dataset from pe_test mapdamage produces data
>> with
>> > NaNs and then tries to compute quantiles on this data. I think that on
>> real
>> > date NaNs also could be produced, so I treated this issue as a bug and
>> made
>> > a patch which changes the way how quantiles are computed --- now it
>> removes
>> > NaNs [1]
>> >
>> > 3) I wrote about all this to upstream as a comments to earlier created
>> > issue and asked if it's really was a bug in their opinion
>> >
>> > 4) FInally, move forward to fixing mapdamage test suite
>>
>> Pretty good job. That's exactly the contribution I'm envisioning for the
>> Outreachy project. I've just added the URL of the upstream issue to the
>> patch.
>>
>> > 11: stop("missing values and NaN's not allowed if 'na.rm' is FALSE")
>> > 10: quantile.default(newX[, i], ...)
>> > 9: FUN(newX[, i], ...)
>> > 8: apply(X, 1, quantile, c(0.025))
>> > 7: data.frame(x = 1:nrow(da), pos = da[, "Pos"], mea = apply(X,
>> >1, mean), med = apply(X, 1, median), loCI = apply(X, 1, quantile,
>> >c(0.025)), hiCI = apply(X, 1, quantile, c(0.975)))
>> > 6: calcSampleStats(da, CTs)
>> > 5: postPredCheck(dat, mcmcOut)
>> > 4: eval(expr, envir, enclos)
>> > 3: eval(ei, envir)
>> > 2: withVisible(eval(ei, envir))
>> > 1: source(paste(path_to_mapDamage_stats, "main.R", sep = ""))
>> > The Bayesian statistics program failed to finish
>> > Traceback (most recent call last):
>> > File "/usr/local/bin/mapDamage", line 382, in
>> > sys.exit(main())
>> > File "/usr/local/bin/mapDamage", line 365, in main
>> > mapdamage.rscript.run_stats(options)
>> > File "/usr/local/lib/python2.7/dist-packages/mapdamage/rscript.py",
>> line 144, in run_stats
>> > raise e
>> > subprocess.CalledProcessError: Command '['Rscript',
>> '/usr/local/lib/python2.7/dist-packages/mapdamage/Rscripts/stats/runGeneral.R',
>> '--args', '30', '1', '10', '5', '0', '0', '1', '1', '0', '0', '1',
>> '12', 'results_pe/', '/usr/local/lib/python2.7/dist
>> -packages/mapdamage/Rscripts/stats/', 'results_pe/Stats_out', '0', '0',
>> '0', 'results_pe/acgt_ratio.csv', '0', '0']' returned non-zero exit status 1
>>
>> I'd recommend to manually run this script.
>>
>> Did you do some investigation what dependency providing liblapack.so and
>> libblas.so are needed (see my mail from yesterday)?
>>
>> Kind regards
>>
>> Andreas.
>>
>> --
>> http://fam-tille.de
>>
>>
>
Re: CI for mapdamage (Was: Outreachy 2017 (Round 14))
Hello Andreas,
>Did you do some investigation what dependency providing liblapack.so and
>libblas.so are needed (see my mail from yesterday)?
Yes, I tried to run the test with different packages installed, and here
are results:
packages| result| user time
--
libatlas-base-dev | Successful run | 11m24.204s
libopenblas-dev | Successful run | 10m31.860s
libblas-dev+liblapack-dev | Successful run | 10m34.368s
It seems that libatlas-base-dev is less effective, but I'm not sure if we
should exclude it from Depends, so I left if for now
On Tue, Mar 28, 2017 at 11:29 PM, Andreas Tille wrote:
> [Changed subject to split up to topics in one thread]
>
> Hi Nadiya,
>
> On Tue, Mar 28, 2017 at 10:23:00PM -0700, Nadiya Sitdykova wrote:
> > 1) I read the paper about mapdamage and figured out that with given
> pe.sam
> > file the reference TGCGA wouldn't make sense as the particular type
> of
> > substitutions is specific for ancient DNA damages. So further for tests I
> > used reference CGATACGA -- which contains all types of nucleotides and
> make
> > sense regarding specifics of DNA damage.
> >
> > 2) But even after fixing (1) I still got an error (log attached). The
> > problem is that on given dataset from pe_test mapdamage produces data
> with
> > NaNs and then tries to compute quantiles on this data. I think that on
> real
> > date NaNs also could be produced, so I treated this issue as a bug and
> made
> > a patch which changes the way how quantiles are computed --- now it
> removes
> > NaNs [1]
> >
> > 3) I wrote about all this to upstream as a comments to earlier created
> > issue and asked if it's really was a bug in their opinion
> >
> > 4) FInally, move forward to fixing mapdamage test suite
>
> Pretty good job. That's exactly the contribution I'm envisioning for the
> Outreachy project. I've just added the URL of the upstream issue to the
> patch.
>
> > 11: stop("missing values and NaN's not allowed if 'na.rm' is FALSE")
> > 10: quantile.default(newX[, i], ...)
> > 9: FUN(newX[, i], ...)
> > 8: apply(X, 1, quantile, c(0.025))
> > 7: data.frame(x = 1:nrow(da), pos = da[, "Pos"], mea = apply(X,
> >1, mean), med = apply(X, 1, median), loCI = apply(X, 1, quantile,
> >c(0.025)), hiCI = apply(X, 1, quantile, c(0.975)))
> > 6: calcSampleStats(da, CTs)
> > 5: postPredCheck(dat, mcmcOut)
> > 4: eval(expr, envir, enclos)
> > 3: eval(ei, envir)
> > 2: withVisible(eval(ei, envir))
> > 1: source(paste(path_to_mapDamage_stats, "main.R", sep = ""))
> > The Bayesian statistics program failed to finish
> > Traceback (most recent call last):
> > File "/usr/local/bin/mapDamage", line 382, in
> > sys.exit(main())
> > File "/usr/local/bin/mapDamage", line 365, in main
> > mapdamage.rscript.run_stats(options)
> > File "/usr/local/lib/python2.7/dist-packages/mapdamage/rscript.py",
> line 144, in run_stats
> > raise e
> > subprocess.CalledProcessError: Command '['Rscript',
> '/usr/local/lib/python2.7/dist-packages/mapdamage/Rscripts/stats/runGeneral.R',
> '--args', '30', '1', '10', '5', '0', '0', '1', '1', '0', '0', '1',
> '12', 'results_pe/',
> '/usr/local/lib/python2.7/dist-packages/mapdamage/Rscripts/stats/',
> 'results_pe/Stats_out', '0', '0', '0', 'results_pe/acgt_ratio.csv', '0',
> '0']' returned non-zero exit status 1
>
> I'd recommend to manually run this script.
>
> Did you do some investigation what dependency providing liblapack.so and
> libblas.so are needed (see my mail from yesterday)?
>
> Kind regards
>
> Andreas.
>
> --
> http://fam-tille.de
>
>
