Hi,
calculating mean on int matrix fails to produce expected results - see
below. I noticed
https://eigen.tuxfamily.org/bz/show_bug.cgi?id=426
which is a bit similar, but different.
Obviously all similar reduxes (sum, prod) fails as well. Is there a way
how to change type used as the internal storage and result for sum.
mean calculations?
Thanks
--
Petr Kubánek
Software Engineer - Telescope Control System
Large Binocular Telescope Observatory
580D Steward Observatory
933 N Cherry Avenue
Tucson, AZ 85721
This results in:
-4095 12288.6
12288.5 12288.6
#include <Eigen/Eigen>
#include <iostream>
int main(int, char **)
{
Eigen::Matrix<int16_t, 2, 2> m;
m <<
INT16_MAX / 2.0, INT16_MAX,
2, 2;
double mean = (INT16_MAX / 2.0 + INT16_MAX + 2 + 2) / 4.0;
std::cout << m.mean() << " " << mean << std::endl;
Eigen::Matrix<double, 2, 2> md = m.cast<double>();
std::cout << md.mean() << " " << mean << std::endl;
}
