Hello, > This is the relative sample quantization error that we make when using > > single precision. Let's just ceil this value, i.e. for every sample > computed in single precision we may have an integer quantization error > of +-1 . >
BTW, it should be noted that since dithering is applied when writing 16bit samples, the LSB of each sample is not significant because it moves in a pseudo-random way. So, probably this quantization error is a "do-not-care" error when emitting with 16bit samples, indipendently by the math resolution. Sincerely.
_______________________________________________ fluid-dev mailing list fluid-dev@nongnu.org https://lists.nongnu.org/mailman/listinfo/fluid-dev
