On Monday, 16 May 2016 at 09:00:49 UTC, Marco Leise wrote:
Ola's example could be some X-ray imaging format. Although it is still a vague example,
I don't think quantization is a particularly vague example! I think that is rather common.
Doing something like floor(x*(1<<24))*(1.0/(1<<24)) to quantize to 24 bit is just silly and inefficient.
Error diffusion is useful in scenarios where you want to reduce the accumulation-error or avoid banding. It can be used in audio in many other settings.
Anyway, irregular higher precision is almost always worse than regular lower precision when computing time series, comparing results or doing unit testing.
