Re: [Numpy-discussion] Guidelines for floating point comparison

2021-02-28 Thread Bernard Knaepen
X/ARM64 will be another difficult one). > > > > This advice is moot if you have a precise bound for the error. > > > > Kevin > > > > > > From: Ralf Gommers <mailto:ralf.gomm...@gmail.com> > Sent: Wednesday, February 24, 2021 12:25 PM

[Numpy-discussion] Guidelines for floating point comparison

2021-02-24 Thread Bernard Knaepen
Hi all, We are developing a code that heavily relies on NumPy. Some of our regression tests rely on floating point number comparisons and we are a bit lost in determining how to choose atol and rtol (we are trying to do all operations in double precision). We would like to set atol and rtol as