Can anyone point to cogent analysis of the representational error in twiddle factors due to cos(PI / 2^N) ?
ie: representational error = cos(PI/2^N) - CPU_representation( cos(PI/2^N) ) For example, when represented as single-precision FP, cos(PI / 2^N) only has 5 or 6 significant bits when N=20, and drops to 3 significant bits when N=22. Clearly the problem is less acute with double-precision, since at N=20, there will still be 37 significant bits in "cos" It seems that the representational error would create a phase error in the transform, and make accurate convolution increasingly difficult in proportion to N. looking forward to your insights, Paul _______________________________________________ Prime mailing list [email protected] http://hogranch.com/mailman/listinfo/prime
