Abramo Bagnara wrote: > min & max are interchanged indeed. My bad...
Oh, there's no problem with using _max instead of _min -- I just noticed that the code for _max writes 0 for unsigned 16/24/32-bit values ... ;o) > Use of 32/64 bits for 24/32 bit is wanted. Take in account that this > function is called on mix results (where I need to avoid wrap before > normalization). Then it should use bigger types for 8/16 bits, too. Checking for a type to overflow its own valid range seems to be rather pointless to me. > > > And the summing/normalization code in pcm_route.c treats the sample as > > > if it were unsigned. > > Be careful here. Use of signed vs. unsigned was an hard decision. > Can you point me exactly where you see a problem? In the following code snippet add_int64_att: sum.as_sint64 += (u_int64_t) sample * ttp->as_int; each of the three variables has a signed integer type. Converting the sample from int32_t to u_int64_t would give wrong results for negative values, e.g., the sample -1 (0xffffffff) would be converted to 4294967295 (0x00000000ffffffff). The following code norm_int: if (sum.as_sint64 > (u_int32_t)0xffffffff) sample = (u_int32_t)0xffffffff; else sample = sum.as_sint64; doesn't check for negative values of sum.as_sint64. Additionally, using 0xffffffff in case of overflow results in sample == -1. These code snippets would be correct if the sample would be unsigned, but my point was that this isn't the case. Clemens ------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Stuff, things, and much much more. http://thinkgeek.com/sf _______________________________________________ Alsa-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/alsa-devel