Bug#877670: [Debian-med-packaging] Bug#877670: Bug#877670: bcftools FTBFS on armel armhf and ppc64el

[Graham Inggs]

Control: reassign -1 htslib 1.4-1
Control: tags -1 + patch
Control: affects -1 bcftools

It seems this code appeared in htslib 1.4, but was only tested in bcftools 1.5.
Description: Fix calculation of PLs on ARM and POWER
Bug: https://github.com/samtools/bcftools/issues/702
Bug-Debian: https://bugs.debian.org/877670
Forwarded: https://github.com/samtools/htslib/pull/617
Author: Graham Inggs 
Last-Update: 2017-11-08
--- a/errmod.c
+++ b/errmod.c
@@ -82,10 +82,11 @@
 double le1 = log(1.0 - e);
 for (n = 1; n <= 255; ++n) {
 double *beta = em->beta + (q<<16|n<<8);
-sum1 = sum = 0.0;
-for (k = n; k >= 0; --k, sum1 = sum) {
-sum = sum1 + expl(lC[n<<8|k] + k*le + (n-k)*le1);
-beta[k] = -10. / M_LN10 * logl(sum1 / sum);
+sum1 = lC[n<<8|n] + n*le;
+beta[n] = INFINITY;
+for (k = n - 1; k >= 0; --k, sum1 = sum) {
+sum = sum1 + log1pl(expl(lC[n<<8|k] + k*le + (n-k)*le1 - sum1));
+beta[k] = -10. / M_LN10 * (sum1 - sum);
 }
 }
 }

Bug#877670: [Debian-med-packaging] Bug#877670: Bug#877670: bcftools FTBFS on armel armhf and ppc64el

[Graham Inggs]

FWIW, I tried building htslib 1.6 and bcftools 1.6, but the issue is
still present.
I also tried building htslib 1.5 and becftools 1.5 with -fsigned-char
but that didn't help either.

Looking closer at test/test.pl, I noticed the failing test is new, so
this is not a regression.  Simply dropping the line below was enough
to get the build to succeed on armhf:
test_mpileup($opts,in=>[qw(indel-AD.1)],out=>'mpileup/indel-AD.1.out',ref=>'indel-AD.1.fa',args=>q[-a
AD]);

I'm inclined to upload with a patch skipping this still, unless there
are objections.

Bug#877670: [Debian-med-packaging] Bug#877670: bcftools FTBFS on armel armhf and ppc64el

[Graham Inggs]

On 14 October 2017 at 03:12, peter green  wrote:
> Here is the diff I got on my armhf system.
>
> --- test/mpileup/indel-AD.1.out 2017-06-20 11:49:44.0 +
> +++ test/mpileup/indel-AD.1.out.new 2017-10-14 01:06:23.687285852 +
> @@ -154,7 +154,7 @@
>  00F524 .   T   G,<*>   0   .
> DP=119;I16=64,44,1,0,4253,180647,27,729,6480,388800,60,3600,2460,59296,25,625;QS=0.993531,0.00646862,0;SGB=-0.379885;RPB=1;MQB=1;MQSB=1;BQB=1;MQ0F=0
> PL:AD   0,255,255,255,255,255:108,1,0
>  00F525 .   A   <*> 0   .
> DP=119;I16=63,39,0,0,4126,179608,0,0,6120,367200,0,0,2330,56198,0,0;QS=1,0;MQSB=1;MQ0F=0
> PL:AD   0,255,255:102,0
>  00F526 .   T   <*> 0   .
> DP=120;I16=62,45,0,0,4234,181460,0,0,6420,385200,0,0,2450,58910,0,0;QS=1,0;MQSB=1;MQ0F=0
> PL:AD   0,255,255:107,0
> -00F527 .   C   <*> 0   .
> DP=120;I16=64,40,0,0,4275,186993,0,0,6240,374400,0,0,2382,57234,0,0;QS=1,0;MQSB=1;MQ0F=0
> PL:AD   0,255,255:104,0
> +00F527 .   C   <*> 0   .
> DP=120;I16=64,40,0,0,4275,186993,0,0,6240,374400,0,0,2382,57234,0,0;QS=1,0;MQSB=1;MQ0F=0
> PL:AD   0,255,0:104,0
>  00F528 .   A   <*> 0   .
> DP=121;I16=62,45,0,0,4346,189994,0,0,6420,385200,0,0,2426,58030,0,0;QS=1,0;MQSB=1;MQ0F=0
> PL:AD   0,255,255:107,0
>  00F529 .   T   <*> 0   .
> DP=123;I16=64,43,0,0,4397,193617,0,0,6420,385200,0,0,2439,58361,0,0;QS=1,0;MQSB=1;MQ0F=0
> PL:AD   0,255,255:107,0
>  00F530 .   G   <*> 0   .
> DP=123;I16=66,42,0,0,4392,192268,0,0,6480,388800,0,0,2486,59400,0,0;QS=1,0;MQSB=1;MQ0F=0
> PL:AD   0,255,255:108,0

Thanks!

This failure reminds me of #865060, which failed on architectures
where char is unsigned by default.
Reverting upstream's commit 'Fix mpileup's incorrect FMT/AD indel
depths' [1] avoids the FTBFS, but that doesn't necessarily mean the
bug is in that commit.  I'll dig further when I have a chance.

[1] 
https://github.com/samtools/bcftools/commit/3c1205c12b3854912c9a6dfe4b214524ea07f12a

Bug#877670: bcftools FTBFS on armel armhf and ppc64el

[peter green]

On 14/10/17 01:16, peter green wrote:


I'm going to see if I can reproduce this bug locally and find out what the 
output difference is.


Here is the diff I got on my armhf system.

--- test/mpileup/indel-AD.1.out 2017-06-20 11:49:44.0 +
+++ test/mpileup/indel-AD.1.out.new 2017-10-14 01:06:23.687285852 +
@@ -154,7 +154,7 @@
 00F524 .   T   G,<*>   0   .   
DP=119;I16=64,44,1,0,4253,180647,27,729,6480,388800,60,3600,2460,59296,25,625;QS=0.993531,0.00646862,0;SGB=-0.379885;RPB=1;MQB=1;MQSB=1;BQB=1;MQ0F=0
   PL:AD   0,255,255,255,255,255:108,1,0
 00F525 .   A   <*> 0   .   
DP=119;I16=63,39,0,0,4126,179608,0,0,6120,367200,0,0,2330,56198,0,0;QS=1,0;MQSB=1;MQ0F=0
PL:AD   0,255,255:102,0
 00F526 .   T   <*> 0   .   
DP=120;I16=62,45,0,0,4234,181460,0,0,6420,385200,0,0,2450,58910,0,0;QS=1,0;MQSB=1;MQ0F=0
PL:AD   0,255,255:107,0
-00F527 .   C   <*> 0   .   
DP=120;I16=64,40,0,0,4275,186993,0,0,6240,374400,0,0,2382,57234,0,0;QS=1,0;MQSB=1;MQ0F=0
PL:AD   0,255,255:104,0
+00F527 .   C   <*> 0   .   
DP=120;I16=64,40,0,0,4275,186993,0,0,6240,374400,0,0,2382,57234,0,0;QS=1,0;MQSB=1;MQ0F=0
PL:AD   0,255,0:104,0
 00F528 .   A   <*> 0   .   
DP=121;I16=62,45,0,0,4346,189994,0,0,6420,385200,0,0,2426,58030,0,0;QS=1,0;MQSB=1;MQ0F=0
PL:AD   0,255,255:107,0
 00F529 .   T   <*> 0   .   
DP=123;I16=64,43,0,0,4397,193617,0,0,6420,385200,0,0,2439,58361,0,0;QS=1,0;MQSB=1;MQ0F=0
PL:AD   0,255,255:107,0
 00F530 .   G   <*> 0   .   
DP=123;I16=66,42,0,0,4392,192268,0,0,6480,388800,0,0,2486,59400,0,0;QS=1,0;MQSB=1;MQ0F=0
PL:AD   0,255,255:108,0

Bug#877670: bcftools FTBFS on armel armhf and ppc64el

[peter green]

retitle 877670 bcftools FTBFS on armel, armhf and ppc64el
found 877670 1.5-2
thanks

The good news is that bcftools 1.5-2 successfully built on mipsel.

The bad news is that still leaves three architectures where this package has 
build regressions. ppc64el, armel and armhf. All three now seem to suffer from 
the same set of four test failures so i'm using this bug to cover all three. 
Interestingly all four tests seem to use the same test log file.

I'm going to see if I can reproduce this bug locally and find out what the 
output difference is.