[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374128#comment-15374128 ] Vineet Goel commented on HAWQ-19: - [~liming01] and [~ftian] - does this JIRA need to be targeted for "2.0.0.0-incubating" release? If not, we should change the Fix Version to "backlog" or another release. Please update the JIRA Fix Version field soon so "2.0.0.0-incubating" is not blocked. > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Ming LI > Fix For: 2.0.0 > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168841#comment-15168841 ] ASF GitHub Bot commented on HAWQ-19: Github user 0x0FFF closed the pull request at: https://github.com/apache/incubator-hawq/pull/323 > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Ming LI > Fix For: backlog > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168340#comment-15168340 ] ASF GitHub Bot commented on HAWQ-19: Github user changleicn commented on the pull request: https://github.com/apache/incubator-hawq/pull/323#issuecomment-189090648 @0x0FFF the code is in, can you close this pull request? Thanks! > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Ming LI > Fix For: backlog > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15158648#comment-15158648 ] ASF GitHub Bot commented on HAWQ-19: Github user 0x0FFF commented on the pull request: https://github.com/apache/incubator-hawq/pull/323#issuecomment-187631667 Sorry, I was in a trip so didn't have time to work on this. Thank you for fixing it > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Ming LI > Fix For: backlog > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156415#comment-15156415 ] ASF GitHub Bot commented on HAWQ-19: Github user liming01 commented on the pull request: https://github.com/apache/incubator-hawq/pull/323#issuecomment-186979419 I have fixed the issue I pointed out above, and merge this change into master branch. So I close this pull request. > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Ming LI > Fix For: backlog > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151584#comment-15151584 ] ASF GitHub Bot commented on HAWQ-19: Github user liming01 commented on the pull request: https://github.com/apache/incubator-hawq/pull/323#issuecomment-185507606 Hi Alexey, If you need my help to move on, please let me know. Thanks. > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Ming LI > Fix For: backlog > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147240#comment-15147240 ] ASF GitHub Bot commented on HAWQ-19: Github user liming01 commented on the pull request: https://github.com/apache/incubator-hawq/pull/323#issuecomment-184176240 Cool! BTW, There are 2 points here: (1) It seems we need to add meta info for functions you added(e.g. int8_mul_cash() ) into below files: apache-hawq/src/backend/utils/fmgrtab.c apache-hawq/src/include/catalog/pg_operator.h apache-hawq/src/include/catalog/pg_proc.h (2) I saw you changed argument type to int64 for function cash_div_int4(), but didn't change for fucntion cash_div_int2(). I guess you may want to fix the problem caused by (1) point above. So please revert this change. It may cause the result precision. > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Ming LI > Fix For: backlog > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15146966#comment-15146966 ] ASF GitHub Bot commented on HAWQ-19: Github user changleicn commented on the pull request: https://github.com/apache/incubator-hawq/pull/323#issuecomment-184081420 nice. LGTM. > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Lei Chang > Fix For: backlog > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140573#comment-15140573 ] ASF GitHub Bot commented on HAWQ-19: Github user 0x0FFF commented on the pull request: https://github.com/apache/incubator-hawq/pull/323#issuecomment-182280320 Added installcheck to cover "money" data type (both to good, serial and parallel checks) > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Lei Chang > Fix For: backlog > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132137#comment-15132137 ] ASF GitHub Bot commented on HAWQ-19: Github user 0x0FFF commented on the pull request: https://github.com/apache/incubator-hawq/pull/323#issuecomment-179770190 Thanks for mentioning, fixed and tested Parquet compatibility > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Lei Chang > Fix For: backlog > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132231#comment-15132231 ] ASF GitHub Bot commented on HAWQ-19: Github user 0x0FFF commented on the pull request: https://github.com/apache/incubator-hawq/pull/323#issuecomment-179796519 As of HAWQ inputformat, it never [supported "money" datatype](https://github.com/apache/incubator-hawq/blob/master/contrib/hawq-hadoop/hawq-mapreduce-common/src/main/java/com/pivotal/hawq/mapreduce/schema/HAWQPrimitiveField.java#L33): ```java /** * Enumeration of all supported primitive type. * Note that CHAR and BPCHAR are the same, we include BPCHAR for internal usage. */ public static enum PrimitiveType { BOOL, BIT, VARBIT, BYTEA, INT2, INT4, INT8, FLOAT4, FLOAT8, NUMERIC, CHAR, BPCHAR, VARCHAR, TEXT, DATE, TIME, TIMETZ, TIMESTAMP, TIMESTAMPTZ, INTERVAL, POINT, LSEG, BOX, CIRCLE, PATH, POLYGON, MACADDR, INET, CIDR, XML } ``` We can add it, but I think it should be a separate PR Also it would be nice to port other "cash" functions from postgres, allowing convertation of numeric, int and float to money > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Lei Chang > Fix For: backlog > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133482#comment-15133482 ] ASF GitHub Bot commented on HAWQ-19: Github user changleicn commented on the pull request: https://github.com/apache/incubator-hawq/pull/323#issuecomment-180145944 A PR to support "money" type and porting other "cash" functions are great. And another suggestion is to add a installcheck-good test to cover this fix. > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Lei Chang > Fix For: backlog > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15130012#comment-15130012 ] ASF GitHub Bot commented on HAWQ-19: Github user 0x0FFF commented on the pull request: https://github.com/apache/incubator-hawq/pull/323#issuecomment-179084818 Speaking about this specific case: do we plan an upgrade path from HAWQ 1.x to 2.0? If no, then this PR can be easily merged into 2.0. If yes, we should really care about the upgrade path, and maybe the better solution here would be the following: instead of changing existing data type introduce new one, like "money8" for 8-byte money data type. This would again be a catalog change (for example, in pg_type), but still won't affect the customers using old data type "money" and layout of their existing tables Generally speaking, I agree that this is a tough question. Postgres is not a very good example here - you can easily do pg_dump and pg_restore there as the data volumes it operates with are usually quite small. As of HAWQ, making dump and restore to accommodate storage format change can't be an option as we might have hundreds of TB there. > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Lei Chang > Fix For: backlog > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131645#comment-15131645 ] ASF GitHub Bot commented on HAWQ-19: Github user changleicn commented on the pull request: https://github.com/apache/incubator-hawq/pull/323#issuecomment-179593353 First of all, good fix. In 2.0GA, upgrade path from hawq 1.x to 2.0 is via data reloading. Have we tested parquet format, and mapreduce inputformat? The change might have impact on these aspects? > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Lei Chang > Fix For: backlog > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128686#comment-15128686 ] ASF GitHub Bot commented on HAWQ-19: Github user cwelton commented on the pull request: https://github.com/apache/incubator-hawq/pull/323#issuecomment-178726045 This change as on disk format implications so we must also consider the impact on upgrade. > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Lei Chang > Fix For: backlog > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128729#comment-15128729 ] ASF GitHub Bot commented on HAWQ-19: Github user cwelton commented on the pull request: https://github.com/apache/incubator-hawq/pull/323#issuecomment-178738333 Good topic for the community to discuss. As an example the Postgres community used to take a stance of tolerance around binary version changes, but has since adopted a stronger stance around binary compatibility. Its ultimately a question of innovation vs maintainability and strongly effects how quickly people will move from one version of the product to the next. > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Lei Chang > Fix For: backlog > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128446#comment-15128446 ] ASF GitHub Bot commented on HAWQ-19: GitHub user 0x0FFF opened a pull request: https://github.com/apache/incubator-hawq/pull/323 [HAWQ-19] Money type overflow fixed by back porting int64 support from Postgres commit 74a4019 This commit fixes the issue [HAWQ-19](https://issues.apache.org/jira/browse/HAWQ-19) "Money type overflow" by back porting commit [74a4019](https://github.com/postgres/postgres/commit/74a40190aabd27b052ea7a63e93f7ed47743755d#diff-7ddd78dd264f8cf2a62cf572baf90b84) to HAWQ. Changes internal representation of "money" datatype, input function, cash_words function, adds function to multiply/divide cash by int8. Also changes pg_type to accommodate new internal representation Before this commit: ``` test=# select '$3000'::money; money - -$12,949,672.96 (1 row) ``` After this commit: ``` test=# select '$3000'::money; money $30,000,000.00 (1 row) ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/0x0FFF/incubator-hawq HAWQ-19 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/323.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #323 commit d69b81699bd5cd90d7adb83429b033692f6823a7 Author: Alexey GrishchenkoDate: 2016-02-02T15:24:16Z [HAWQ-19] Money type overflow fixed by back porting int64 support from Postgres commit 74a4019 > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Lei Chang > Fix For: backlog > > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943929#comment-14943929 ] Caleb Welton commented on HAWQ-19: -- This was the commit in Postgres that widened money from 4 to 8 bytes. https://github.com/postgres/postgres/commit/74a40190aabd27b052ea7a63e93f7ed47743755d#diff-7ddd78dd264f8cf2a62cf572baf90b84 > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: Catalog >Reporter: Feng Tian >Assignee: Lei Chang > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936755#comment-14936755 ] Lei Chang commented on HAWQ-19: --- Looks Ming's account is not ready yet. Assign it later. > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: core >Reporter: Feng Tian >Assignee: Lei Chang > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-19) Money type overflow
[ https://issues.apache.org/jira/browse/HAWQ-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936748#comment-14936748 ] Lei Chang commented on HAWQ-19: --- Hi Feng, I think you might have a patch already. It would be nice for you to attaching the patch or start a pull request. > Money type overflow > --- > > Key: HAWQ-19 > URL: https://issues.apache.org/jira/browse/HAWQ-19 > Project: Apache HAWQ > Issue Type: Bug > Components: core >Reporter: Feng Tian >Assignee: Lei Chang > > Use tpch schema, but change l_extendedprice to use MONEY type, run Q1, you > should see negative amounts. > I believe this is due to overflow. > Side mark, postgres 9 money type use 8 bytes and will return correct result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)