Re: [PATCH] staging: comedi: usbduxsigma: usbduxfast_ai_cmdtest rounding error

2019-11-18 Thread Ian Abbott
On 18/11/2019 17:21, Bernd Porr wrote: 'get_cmd_generic_timed' fills the cmd structure with an informed guess and then calls 'usbduxfast_ai_cmdtest' repeatedly while 'usbduxfast_ai_cmdtest' is modifying the cmd struct until it no longer changes. However, because of rounding errors this never

[PATCH] staging: comedi: usbduxsigma: usbduxfast_ai_cmdtest rounding error

2019-11-18 Thread Bernd Porr
'get_cmd_generic_timed' fills the cmd structure with an informed guess and then calls 'usbduxfast_ai_cmdtest' repeatedly while 'usbduxfast_ai_cmdtest' is modifying the cmd struct until it no longer changes. However, because of rounding errors this never converged because 'steps = (cmd->convert_arg