On 3 October 2012 19:26, Sebastian Kuzminsky <[email protected]> wrote:

> That looks great!

Glad you think so.
I have a supplementary question.

I can support an arbitrary precision and width by using a lot of
floating-point, i.e. a floating-point / or * 10 for every digit in the
floats.
The example display takes 17uS (max) per thread to generate in that
case. That is the cost when generating 9 digits into the buffer. (more
digits, more cost).
I have re-coded to first copy the offset value into a uint (so, for
example, 4 digits of pi becomes 31415). This saves a lot of time, the
max execution time for the same sample display drops to 4.8uS.
The drawback is that this only allows 9 digits of decimal precision.
(999,999,999 is the biggest that fits in a uint).
Using a U64 would allow me to support 19 digits, but there is no
64-bit division or modulus in the kernel which makes it more tricky.

I can imagine coding extra runs through, taking 9 digit slices each
time. That adds a fair bit to the complexity.

Does is seem reasonable to limit the display to 9 digits + sign +
decimal point?
That supports numbers like 99999.1234 which allows display to a
ten-thousandth of an inch on a machine bed 1.3 miles long, or to 100nm
on a typical 100mm cross-slide.
I suspect that for the intended use it is not worth the overhead to
support higher precision, but I would appreciate opinions.

For reference, a bit of fiddling has Axis displaying to 16 digits of
precision (12 places of decimals before rounding error appear on G0
X[1/3])

-- 
atp
If you can't fix it, you don't own it.
http://www.ifixit.com/Manifesto

------------------------------------------------------------------------------
Don't let slow site performance ruin your business. Deploy New Relic APM
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
http://p.sf.net/sfu/newrelic-dev2dev
_______________________________________________
Emc-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/emc-developers

Reply via email to