Now you guys come up with samples of floating point rounding hehehehe.... A couple months ago I was looking for such rounding demos for my tutorials I am writing. Note that this is just a part of programming. Many real numbers do not have a direct representation in base 2 (binary). So to represent those numbers the floating point library estimates the numbers. The way to deal with this issue is to always define the precision needed and use the most appropriate data type then limit the precision to something less. This is a common issue with floating point math and why you will find many specialized math libraries for science and engineering with use much more precise methods to represent real numbers.
Thanks for the demos I may use them in my tutorials if that's ok with you? Randall On Fri, Dec 23, 2011 at 10:10 AM, <gam...@googlecode.com> wrote: > > Comment #6 on issue 165 by jussi.la...@gmail.com: Arithmetic problem with > float > http://code.google.com/p/gambas/issues/detail?id=165 > > That is kind of expected result, > since with doubles (float in Gambas) the actual value is (5.025 - 5): > 0.0250000000000003552713678800500929355621337890625000 > And Gambas rounds it to: > 0.0250000000000004 > > See this for further details: > http://en.wikipedia.org/wiki/Floating_point > > > > ------------------------------------------------------------------------------ > Write once. Port to many. > Get the SDK and tools to simplify cross-platform app development. Create > new or port existing apps to sell to consumers worldwide. Explore the > Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join > http://p.sf.net/sfu/intel-appdev > _______________________________________________ > Gambas-user mailing list > Gambas-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/gambas-user > -- If you ask me if it can be done. The answer is YES, it can always be done. The correct questions however are... What will it cost, and how long will it take? ------------------------------------------------------------------------------ Write once. Port to many. Get the SDK and tools to simplify cross-platform app development. Create new or port existing apps to sell to consumers worldwide. Explore the Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join http://p.sf.net/sfu/intel-appdev _______________________________________________ Gambas-user mailing list Gambas-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/gambas-user
