elbriga wrote:
> Thanks for the detailed answer!
>
> Changing the function sinature seams to have solved the problem:
> CREATE OR REPLACE FUNCTION ceilDecimal(num numeric) RETURNS float AS
> $BODY$
> BEGIN
> RETURN CEIL(num * 100) / 100;
> END
> $BODY$
> LANGUAGE 'plpgsql';
>
>
> SELECT ce
Thanks for the detailed answer!
Changing the function sinature seams to have solved the problem:
CREATE OR REPLACE FUNCTION ceilDecimal(num numeric) RETURNS float AS
$BODY$
BEGIN
RETURN CEIL(num * 100) / 100;
END
$BODY$
LANGUAGE 'plpgsql';
SELECT ceilDecimal(0.07);
ceildecimal
elbriga wrote:
> Hi,
> I have this pl function:
> CREATE OR REPLACE FUNCTION ceilDecimal(num float) RETURNS float AS
> $BODY$
> BEGIN
> RETURN CEIL(num * 100) / 100;
> END
> $BODY$
> LANGUAGE 'plpgsql';
>
> It is supposed to do a "decimail ceil" for 2 decimal places.
> But when I do "S
Hi,
I have this pl function:
CREATE OR REPLACE FUNCTION ceilDecimal(num float) RETURNS float AS
$BODY$
BEGIN
RETURN CEIL(num * 100) / 100;
END
$BODY$
LANGUAGE 'plpgsql';
It is supposed to do a "decimail ceil" for 2 decimal places.
But when I do "SELECT ceilDecimal(0.07)"
It will retu
>>"Paolo Saudin" writes:
>> I have a problem with a query wich simple aggregate values. In the sample
>> below I have two values, 1.3 and 1.4. Rounding their average with one
>> decimals, should give 1.4.
>
>You seem way overoptimistic about float4 values being exact. They are
>not. The actual c
>Paolo Saudin wrote:
>Hi,
>
>I have a problem with a query wich simple aggregate values. In the sample
below I have two values, 1.3 and 1.4. Rounding their average with one
decimals, should give 1.4.
>The first query with - cast( tables_seb.tbl_arvier_chamencon.id_1 AS
numeric) AS value -
"Paolo Saudin" writes:
> I have a problem with a query wich simple aggregate values. In the sample
> below I have two values, 1.3 and 1.4. Rounding their average with one
> decimals, should give 1.4.
You seem way overoptimistic about float4 values being exact. They are
not. The actual computati
Paolo Saudin wrote:
Hi,
I have a problem with a query wich simple
aggregate values.
In the sample below I have two values, 1.3 and 1.4. Rounding their
average with
one decimals, should give 1.4.
The first query with - cast(
tables_seb.tbl_arvier_chamencon.id_1 AS nu
Hi,
I have a problem with a query wich simple aggregate values. In the sample
below I have two values, 1.3 and 1.4. Rounding their average with one
decimals, should give 1.4.
The first query with - cast( tables_seb.tbl_arvier_chamencon.id_1 AS
numeric) AS value - give the expected result,
glene77is wrote:
On May 14, 3:27 pm, [EMAIL PROTECTED] (Sam Mason) wrote:
On Wed, May 14, 2008 at 02:08:47PM -0400, Justin wrote:
Sam Mason wrote:
What doesfoxprouse for storing numbers? or is it just that you never
pushed it hard enough for the abstractions to show through.
On May 14, 3:27 pm, [EMAIL PROTECTED] (Sam Mason) wrote:
> On Wed, May 14, 2008 at 02:08:47PM -0400, Justin wrote:
> > Sam Mason wrote:
> > >What doesfoxprouse for storing numbers? or is it just that you never
> > >pushed it hard enough for the abstractions to show through.
>
> > I know i pushed it
Andy Anderson wrote:
Sam Mason wrote:
If you mean FoxPro, I think this is another case of MS screwing up.
On May 14, 2008, at 4:08 PM, Justin wrote:
Foxpro normally did not suffer form other MS screw ups.
That's because MS bought it from a third-party developer.
(And so, of course, they co
Sam Mason wrote:
If you mean FoxPro, I think this is another case of MS screwing up.
On May 14, 2008, at 4:08 PM, Justin wrote:
Foxpro normally did not suffer form other MS screw ups.
That's because MS bought it from a third-party developer.
(And so, of course, they couldn't allow that to s
Andy Anderson wrote:
On Wed, May 14, 2008 at 02:08:47PM -0400, Justin wrote:
My problem is we calculate resistance of parts in a Foxpro app that we
want to move because we want to bring all the custom apps into one
framework and single database.
Take this calculation (0.05/3* 1.0025) whi
Sam Mason wrote:
On Wed, May 14, 2008 at 02:08:47PM -0400, Justin wrote:
Sam Mason wrote:
What does foxpro use for storing numbers? or is it just that you never
pushed it hard enough for the abstractions to show through.
I know i pushed it. Foxpro for the most has only 4 bas
On Wed, May 14, 2008 at 02:08:47PM -0400, Justin wrote:
My problem is we calculate resistance of parts in a Foxpro app
that we
want to move because we want to bring all the custom apps into one
framework and single database.
Take this calculation (0.05/3* 1.0025) which is used to
calcul
On Wed, May 14, 2008 at 02:08:47PM -0400, Justin wrote:
> Sam Mason wrote:
> >What does foxpro use for storing numbers? or is it just that you never
> >pushed it hard enough for the abstractions to show through.
>
> I know i pushed it. Foxpro for the most has only 4 basic data types
> Numeri
Sam Mason wrote:
On Wed, May 14, 2008 at 11:47:52AM -0400, Justin wrote:
I have forgotten how much i hate C++
What we're talking about doesn't have much to do with C++, it's floating
point maths in general.
Its not doing what you say it would but it did do other odd ball
things.
On Wed, May 14, 2008 at 11:47:52AM -0400, Justin wrote:
> I have forgotten how much i hate C++
What we're talking about doesn't have much to do with C++, it's floating
point maths in general.
> Its not doing what you say it would but it did do other odd ball
> things. I miss my foxpro :-(.
Wh
I have forgotten how much i hate C++
Its not doing what you say it would but it did do other odd ball
things. I miss my foxpro :-(.
Plus its not holding 15 precision points
#include
#include
int main()
{
double a = 0.1;
//double b = 1000;
double c = 1;
On May 13, 2008, at 1:36 PM, Justin wrote:
Is is limit less no, but what is?
numeric is limitless, unless you specifically bound it. Or you run
out of space...
--
Decibel!, aka Jim C. Nasby, Database Architect [EMAIL PROTECTED]
Give your computer some brain candy! www.distributed.net Team
On May 12, 2008, at 10:42 PM, Craig Ringer wrote:
Personally I'd be tempted to use a `double precision' (float8) for
things like materials consumption.
Or you could just use an un-bounded numeric...
--
Decibel!, aka Jim C. Nasby, Database Architect [EMAIL PROTECTED]
Give your computer some br
On Tue, May 13, 2008 at 02:36:18PM -0400, Justin wrote:
> Double holds 15 places which is the highest value of precision it can
> maintain before rounding occurs.
>
> Is is limit less no, but what is?
>
> Practically speaking taking a vale 0.000,000,000,000,001 aka
> 1 trillionth of anything,
Double holds 15 places which is the highest value of precision it can
maintain before rounding occurs.
Is is limit less no, but what is?
Practically speaking taking a vale 0.000,000,000,000,001 aka
1 trillionth of anything, i view the problem solved for 98% of problems.
Does it hide? Yes,
Tomasz Ostrowski wrote:
On 2008-05-13 17:29, Justin wrote:
Tomasz Ostrowski wrote:
On 2008-05-12 20:49, Justin wrote:
We take (List Price * discount Percent) * Number of Pieces = net
price.
This is wrong. You should do in Excel:
( price * amount ) * discount
Thanks to Andy's C code here is the pl/pgSQL function which does the
same thing.
I added a tiny bit so instead of returning a numeric value with 20
trailing zeros it returns a value with the desired precision.
-Begin Code -
create or replace function ro
2008/5/13 Justin <[EMAIL PROTECTED]>:
> Your saying in Excel, Multiplication is not Commutativity??? that sends
> shudders down my back
The word you want in this case is "associative". Since floating point
math is not exact, sometimes the associativity (and other) properties
of some operati
Tomasz Ostrowski wrote:
On 2008-05-12 20:49, Justin wrote:
We take (List Price * discount Percent) * Number of Pieces = net
price.
This is wrong. You should do in Excel:
( price * amount ) * discount
As otherwise any small error in representation of price*discount would
be multiplied
On 2008-05-12 20:49, Justin wrote:
> We take (List Price * discount Percent) * Number of Pieces = net
> price.
This is wrong. You should do in Excel:
( price * amount ) * discount
As otherwise any small error in representation of price*discount would
be multiplied by usually high amount.
I'd do
Craig Ringer wrote:
Justin wrote:
WE have several columns in table defined with numeric (20,10) thats is
just insanity.
Not necessarily. I have a few places where a monetary value is
mulitiplied by a ratio quantity. For some of the historical data
imported from another system the ra
Justin wrote:
> WE have several columns in table defined with numeric (20,10) thats is
> just insanity.
Not necessarily. I have a few places where a monetary value is
mulitiplied by a ratio quantity. For some of the historical data
imported from another system the ratio can be irrational or at le
thats what i'm trying to get a grasp on, what postgres is doing with
calculation as it truncates or rounds the number when committing the
records to the physical table.
I just start digging into this as we are having problems where some
fields in the database are precision of 2 and other go
On Mon, May 12, 2008 at 06:37:02PM -0400, Justin wrote:
> I guess i have not been very clear.
>
> lets take this
>select (9/10), (9/10)::numeric, (9::numeric/10::numeric), (9./10),
> (9*.1)
>
> With the given select statement i expected the results all to be same,
> especially sense it cast
On May 12, 2008, at 6:37 PM, Justin wrote:
lets take this
select (9/10), (9/10)::numeric, (9::numeric/10::numeric), (9./10),
(9*.1)
With the given select statement i expected the results all to be
same,
especially sense it cast 4 of the 5 to numeric either with explicit
cast
or by conta
Yet another option, of course, is to simply not do any calculations
in PostgreSQL, and accept the results from Excel as definitive...
which seems to be what is desired, anyway.
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://w
I guess i have not been very clear.
lets take this
select (9/10), (9/10)::numeric, (9::numeric/10::numeric), (9./10),
(9*.1)
With the given select statement i expected the results all to be same,
especially sense it cast 4 of the 5 to numeric either with explicit cast
or by containing a deci
Justin wrote:
I tried casting them to numeric and it was still wrong
How do the results differ from what you expect? You've posted a bunch of
code, but haven't explained what you think is wrong with the results.
Can you post a couple of SMALL examples and explain how the results are
differe
Justin wrote:
Craig Ringer wrote:
`double' in C++ refers to double precision floating point. `double' is
subject to all the usual fun with rational decimals being irrational
binary floats (and vice versa).
Not according to MS specific if i'm reading it correctly
*Microsoft Specific >*
T
I tried casting them to numeric and it was still wrong
OK i just added decimal point after the 9 and 1 it work at that point.
Thats an odd result i would not have expected it to do that.
This prompts another question how does postgres figure out the data
types passed in an SQL string???
Andy
As i'm playing around with rounding and the numeric field precision ran
into a odd set of results i don't understand
here is the sql i wrote the first four inserts are calculations we run
everyday and they make sense but if division is used the results are
not right or am i missing something
Craig Ringer wrote:
Justin wrote:
No floating point is being used every variable is declared as numeric
on the Postgresql side and in the C++ which is the UI side
everything is double.
`double' in C++ refers to double precision floating point. `double' is
subject to all the usual fun wit
Justin wrote:
No floating point is being used every variable is declared as numeric on
the Postgresql side and in the C++ which is the UI side everything is
double.
`double' in C++ refers to double precision floating point. `double' is
subject to all the usual fun with rational decimals bei
Andy Anderson wrote:
Can you be more explicit about the rounding that's wrong in Excel?
Are you talking about the n5 round-up to n+1 that Excel uses
vs. n5 round-to-even n (sometimes called Banker's Rounding)?
On May 12, 2008, at 2:38 PM, Justin wrote:
Yes i'm taking about difference
thats how i loaded the price list to start with. The problems with
sales orders are entered and the automatic pricing kicks in ( the
discounts are calculated * the number or pieces ordered) it goes to
down the tubes.
I could just rewrite the pricing stored procedures to call a rounding
procedur
Lincoln Yeoh wrote:
At 01:48 AM 5/13/2008, Justin wrote:
I have very annoying problem that i would like to get a work around
in place so the data entry people stop trying to kill me.
Normally people give quotes out of the price book which was done in
Excel like 15 years ago and just has be
Andy Anderson wrote:
Can you be more explicit about the rounding that's wrong in Excel? Are
you talking about the n5 round-up to n+1 that Excel uses vs.
n5 round-to-even n (sometimes called Banker's Rounding)?
Yes i'm talking about difference between bankers rounding verse Excels
cra
Can you be more explicit about the rounding that's wrong in Excel?
Are you talking about the n5 round-up to n+1 that Excel uses
vs. n5 round-to-even n (sometimes called Banker's Rounding)?
-- Andy
On May 12, 2008, at 1:48 PM, Justin wrote:
I have very annoying problem that i would l
At 01:48 AM 5/13/2008, Justin wrote:
I have very annoying problem that i would like to get a work
around in place so the data entry people stop trying to kill me.
Normally people give quotes out of the price book which was done in
Excel like 15 years ago and just has been updated over the
ye
I have very annoying problem that i would like to get a work around in
place so the data entry people stop trying to kill me.
Normally people give quotes out of the price book which was done in
Excel like 15 years ago and just has been updated over the years. the
problem is excel is rounding
49 matches
Mail list logo