Re: [sqlalchemy] why MySQL DOUBLE is asdecimal=True by default

[Mike Bayer]

still looks like DOUBLE should be dealing with Python floats.
DOUBLE_PRECISION in the Oracle dialect is also Float and doesn't
default to decimal coercion.

We can fix this problem, it's just one of those changes that you have
to warn users like crazy on and then just do the change on a major
release.

On Mon, Apr 29, 2019 at 2:53 PM Jonathan Vanasco  wrote:
>
> FWIW, that PR would have been during MySQL5.0 - not 5.5.
>
> 5.0 was released in October 2005; 5.1 was not released until November 2008
>
> assuming the 5.0 docs...
>>
>> https://docs.oracle.com/cd/E19078-01/mysql/mysql-refman-5.0/data-types.html#numeric-types
>
>
> looking deeper at their docs, this bit may have influenced someone:
>
> 
>
> 10.7. Choosing the Right Type for a Column
>
> For optimum storage, you should try to use the most precise type in all 
> cases. For example, if an integer column is used for values in the range from 
> 1 to 9, MEDIUMINT UNSIGNED is the best type. Of the types that represent 
> all the required values, this type uses the least amount of storage.
>
> Tables created in MySQL 5.0.3 and above use a new storage format for DECIMAL 
> columns. All basic calculations (+, -, *, and /) with DECIMAL columns are 
> done with precision of 65 decimal (base 10) digits. See Section 10.1.1, 
> “Overview of Numeric Types”.
>
> Prior to MySQL 5.0.3, calculations on DECIMAL values are performed using 
> double-precision operations. If accuracy is not too important or if speed is 
> the highest priority, the DOUBLE type may be good enough. For high precision, 
> you can always convert to a fixed-point type stored in a BIGINT. This enables 
> you to do all calculations with 64-bit integers and then convert results back 
> to floating-point values as necessary.
>
>
>
> all that being said, updating mysql was a chore and often scary then, so most 
> linux distributions and dbas held things back and did not upgrade as 
> aggressively as people have the past 5 years. 3.23/4.1 was likely a core 
> target for SqlAlachemy - not 5.x
>
>
> http://ftp.nchu.edu.tw/MySQL/doc/refman/4.1/en/choosing-types.html
>
> http://ftp.nchu.edu.tw/MySQL/doc/refman/4.1/en/numeric-types.html
>
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] why MySQL DOUBLE is asdecimal=True by default

[Jonathan Vanasco]

FWIW, that PR would have been during MySQL5.0 - not 5.5. 

5.0 was released in October 2005; 5.1 was not released until November 2008

assuming the 5.0 docs...

>
> https://docs.oracle.com/cd/E19078-01/mysql/mysql-refman-5.0/data-types.html#numeric-types


looking deeper at their docs, this bit may have influenced someone:



10.7. Choosing the Right Type for a Column

For optimum storage, you should try to use the most precise type in all 
cases. For example, if an integer column is used for values in the range 
from 1 to 9, MEDIUMINT UNSIGNED is the best type. Of the types that 
represent all the required values, this type uses the least amount of 
storage.

Tables created in MySQL 5.0.3 and above use a new storage format for DECIMAL 

 columns. 
All basic calculations (+, -, *, and /) with DECIMAL 

 columns 
are done with precision of 65 decimal (base 10) digits. See Section 10.1.1, 
“Overview of Numeric Types” 

.

Prior to MySQL 5.0.3, calculations on DECIMAL 

 values 
are performed using double-precision operations. If accuracy is not too 
important or if speed is the highest priority, the DOUBLE 

 type 
may be good enough. For high precision, you can always convert to a 
fixed-point type stored in a BIGINT 
.
 
This enables you to do all calculations with 64-bit integers and then 
convert results back to floating-point values as necessary.



all that being said, updating mysql was a chore and often scary then, so 
most linux distributions and dbas held things back and did not upgrade as 
aggressively as people have the past 5 years. 3.23/4.1 was likely a core 
target for SqlAlachemy - not 5.x


http://ftp.nchu.edu.tw/MySQL/doc/refman/4.1/en/choosing-types.html

http://ftp.nchu.edu.tw/MySQL/doc/refman/4.1/en/numeric-types.html



-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] why MySQL DOUBLE is asdecimal=True by default

[Michał Dobrzański]

Interesting. Thank you for the extensive explanation.

pon., 29 kwi 2019 o 19:49 Mike Bayer  napisał(a):

> On Mon, Apr 29, 2019 at 1:13 PM mdob  wrote:
> >
> > Just out of curiosity, why it was decided that MySQL DOUBLE, which is
> approximation, will be presented in ORM as Decimal by default instead of
> float?
> >
> > MySQL DOUBLE -
> https://dev.mysql.com/doc/refman/5.5/en/floating-point-types.html
> > SQLAlchemy DOUBLE -
> https://docs.sqlalchemy.org/en/13/dialects/mysql.html#sqlalchemy.dialects.mysql.DOUBLE
>
> no idea, lets do a git blame
>
> it looks like the issue to first add Decimal support was:
>
> https://github.com/sqlalchemy/sqlalchemy/issues/646
>
> it appears to refer to a no longer tracked SVN changeset, the merge
> can be seen at
> https://github.com/sqlalchemy/sqlalchemy/commit/ed4fc64bb0ac61c27bc4af32962fb129e74a36bf
> ,
> where quite arbitrarily the asdecimal flag is True for DOUBLE even
> though the superclass is Float.  That's all 12 years ago.
>
> Let's try to blame for the ".. note" that was added, since someone
> seems to have noticed this was wrong about six years ago, seems like
> it was in
> https://github.com/sqlalchemy/sqlalchemy/commit/6b79d2ea7951abc2bb6083b541db0fbf71590dd3
> where we made it a lot more explicit how the float coercion occurs.
>
> All we have is a guess.   It would have been that I either observed or
> guessed that Python's float, at least the "ten digits" that was coming
> out by default back on my cPython 2.3 interpreter on my PowerPC mac at
> the time, was not accurate enough to represent the values coming back
> from a double-precision floating point value.
> Trying to see what the likely floating point representation is for
> cPython is ...a ha we can do this:
>
> >>> import sys
> >>> sys.float_info
> sys.float_info(max=1.7976931348623157e+308, max_exp=1024,
> max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021,
> min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16,
> radix=2, rounds=1)
>
> I have no idea what would have been happening at that time, it was
> likely to preserve round trips for the additional digits, but this was
> shortsighted because the driver at the time MySQLdb was likely
> returning floats anyway, current branch of it does:
>
> https://github.com/PyMySQL/mysqlclient-python/blob/master/MySQLdb/converters.py#L107
> so seems like a careless oversight.  Sorry!
>
>
>
>
>
> >
> > Thanks,
> > Michal
> >
> > --
> > SQLAlchemy -
> > The Python SQL Toolkit and Object Relational Mapper
> >
> > http://www.sqlalchemy.org/
> >
> > To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> > ---
> > You received this message because you are subscribed to the Google
> Groups "sqlalchemy" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to sqlalchemy+unsubscr...@googlegroups.com.
> > To post to this group, send email to sqlalchemy@googlegroups.com.
> > Visit this group at https://groups.google.com/group/sqlalchemy.
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example.  See  http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] why MySQL DOUBLE is asdecimal=True by default

[Mike Bayer]

On Mon, Apr 29, 2019 at 1:13 PM mdob  wrote:
>
> Just out of curiosity, why it was decided that MySQL DOUBLE, which is 
> approximation, will be presented in ORM as Decimal by default instead of 
> float?
>
> MySQL DOUBLE - 
> https://dev.mysql.com/doc/refman/5.5/en/floating-point-types.html
> SQLAlchemy DOUBLE - 
> https://docs.sqlalchemy.org/en/13/dialects/mysql.html#sqlalchemy.dialects.mysql.DOUBLE

no idea, lets do a git blame

it looks like the issue to first add Decimal support was:

https://github.com/sqlalchemy/sqlalchemy/issues/646

it appears to refer to a no longer tracked SVN changeset, the merge
can be seen at 
https://github.com/sqlalchemy/sqlalchemy/commit/ed4fc64bb0ac61c27bc4af32962fb129e74a36bf,
where quite arbitrarily the asdecimal flag is True for DOUBLE even
though the superclass is Float.  That's all 12 years ago.

Let's try to blame for the ".. note" that was added, since someone
seems to have noticed this was wrong about six years ago, seems like
it was in 
https://github.com/sqlalchemy/sqlalchemy/commit/6b79d2ea7951abc2bb6083b541db0fbf71590dd3
where we made it a lot more explicit how the float coercion occurs.

All we have is a guess.   It would have been that I either observed or
guessed that Python's float, at least the "ten digits" that was coming
out by default back on my cPython 2.3 interpreter on my PowerPC mac at
the time, was not accurate enough to represent the values coming back
from a double-precision floating point value.
Trying to see what the likely floating point representation is for
cPython is ...a ha we can do this:

>>> import sys
>>> sys.float_info
sys.float_info(max=1.7976931348623157e+308, max_exp=1024,
max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021,
min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16,
radix=2, rounds=1)

I have no idea what would have been happening at that time, it was
likely to preserve round trips for the additional digits, but this was
shortsighted because the driver at the time MySQLdb was likely
returning floats anyway, current branch of it does:
https://github.com/PyMySQL/mysqlclient-python/blob/master/MySQLdb/converters.py#L107
so seems like a careless oversight.  Sorry!





>
> Thanks,
> Michal
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> ---
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

[sqlalchemy] why MySQL DOUBLE is asdecimal=True by default

[mdob]

Just out of curiosity, why it was decided that MySQL DOUBLE, which is 
approximation, will be presented in ORM as Decimal by default instead of 
float?

MySQL DOUBLE - 
https://dev.mysql.com/doc/refman/5.5/en/floating-point-types.html
SQLAlchemy DOUBLE - 
https://docs.sqlalchemy.org/en/13/dialects/mysql.html#sqlalchemy.dialects.mysql.DOUBLE

Thanks, 
Michal

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.