On Tue, 9 Jul 2002, Abramo Bagnara wrote:
> > > > > And the summing/normalization code in pcm_route.c treats the sample as
> > > > > if it were unsigned.
> > >
> > > Be careful here. Use of signed vs. unsigned was an hard decision.
> > > Can you point me exactly where you see a problem?
> >
> > In the following code snippet
> >
> > add_int64_att:
> > sum.as_sint64 += (u_int64_t) sample * ttp->as_int;
> >
> > each of the three variables has a signed integer type. Converting
> > the sample from int32_t to u_int64_t would give wrong results for
> > negative values, e.g., the sample -1 (0xffffffff) would be converted
> > to 4294967295 (0x00000000ffffffff).
> >
> > The following code
> >
> > norm_int:
> > if (sum.as_sint64 > (u_int32_t)0xffffffff)
> > sample = (u_int32_t)0xffffffff;
> > else
> > sample = sum.as_sint64;
> >
> > doesn't check for negative values of sum.as_sint64. Additionally,
> > using 0xffffffff in case of overflow results in sample == -1.
> >
> > These code snippets would be correct if the sample would be unsigned,
> > but my point was that this isn't the case.
>
> I've just checked: this has been broken by Jaroslav with CVS version
> 1.55. I'd like to restore the original behaviour I wrote (that used
> unsigned for efficiency reason), but I don't understand what Jaroslav
> tried to solve.
>
> Jaroslav?
You cannot use unsigned values, because we need to sum samples (it means:
add positive and substract negative ones). If we are in positive range, we
can only add samples, so the final sum result is wrong.
I've fixed the normalization (hopefully) for int64 and float values. Could
you, guys, verify new code?
Jaroslav
-----
Jaroslav Kysela <[EMAIL PROTECTED]>
Linux Kernel Sound Maintainer
ALSA Project http://www.alsa-project.org
SuSE Linux http://www.suse.com
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Two, two, TWO treats in one.
http://thinkgeek.com/sf
_______________________________________________
Alsa-devel mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/alsa-devel
