[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13548200#comment-13548200 ] Hudson commented on HIVE-3678: -- Integrated in Hive-trunk-hadoop2 #54 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/54/]) HIVE-3678 : Add metastore upgrade scripts for column stats schema changes (Shreepadma via Ashutosh Chauhan) (Revision 1414509) Result = ABORTED hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1414509 Files : * /hive/trunk/metastore/scripts/upgrade/derby/012-HIVE-1362.derby.sql * /hive/trunk/metastore/scripts/upgrade/derby/hive-schema-0.10.0.derby.sql * /hive/trunk/metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql * /hive/trunk/metastore/scripts/upgrade/mysql/012-HIVE-1362.mysql.sql * /hive/trunk/metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql * /hive/trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql * /hive/trunk/metastore/scripts/upgrade/oracle/012-HIVE-1362.oracle.sql * /hive/trunk/metastore/scripts/upgrade/oracle/hive-schema-0.10.0.oracle.sql * /hive/trunk/metastore/scripts/upgrade/postgres/012-HIVE-1362.postgres.sql * /hive/trunk/metastore/scripts/upgrade/postgres/hive-schema-0.10.0.postgres.sql * /hive/trunk/metastore/scripts/upgrade/postgres/upgrade-0.9.0-to-0.10.0.postgres.sql * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java * /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MPartitionColumnStatistics.java * /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MTableColumnStatistics.java * /hive/trunk/metastore/src/model/package.jdo Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt, HIVE-3678.2.patch.txt, HIVE-3678.3.patch.txt, HIVE-3678.4.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13505430#comment-13505430 ] Hudson commented on HIVE-3678: -- Integrated in Hive-trunk-h0.21 #1822 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1822/]) HIVE-3678 : Add metastore upgrade scripts for column stats schema changes (Shreepadma via Ashutosh Chauhan) (Revision 1414509) Result = ABORTED hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1414509 Files : * /hive/trunk/metastore/scripts/upgrade/derby/012-HIVE-1362.derby.sql * /hive/trunk/metastore/scripts/upgrade/derby/hive-schema-0.10.0.derby.sql * /hive/trunk/metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql * /hive/trunk/metastore/scripts/upgrade/mysql/012-HIVE-1362.mysql.sql * /hive/trunk/metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql * /hive/trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql * /hive/trunk/metastore/scripts/upgrade/oracle/012-HIVE-1362.oracle.sql * /hive/trunk/metastore/scripts/upgrade/oracle/hive-schema-0.10.0.oracle.sql * /hive/trunk/metastore/scripts/upgrade/postgres/012-HIVE-1362.postgres.sql * /hive/trunk/metastore/scripts/upgrade/postgres/hive-schema-0.10.0.postgres.sql * /hive/trunk/metastore/scripts/upgrade/postgres/upgrade-0.9.0-to-0.10.0.postgres.sql * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java * /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MPartitionColumnStatistics.java * /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MTableColumnStatistics.java * /hive/trunk/metastore/src/model/package.jdo Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt, HIVE-3678.2.patch.txt, HIVE-3678.3.patch.txt, HIVE-3678.4.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13504852#comment-13504852 ] Shreepadma Venugopalan commented on HIVE-3678: -- Uploaded patch rebased off tip of trunk. Thanks. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt, HIVE-3678.2.patch.txt, HIVE-3678.3.patch.txt, HIVE-3678.4.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13504869#comment-13504869 ] Ashutosh Chauhan commented on HIVE-3678: Thanks, Shreepadma for updating patch. Running tests now. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt, HIVE-3678.2.patch.txt, HIVE-3678.3.patch.txt, HIVE-3678.4.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13504337#comment-13504337 ] Shreepadma Venugopalan commented on HIVE-3678: -- @Ashutosh: I've uploaded a new patch which adds 2 varchar columns for storing BigDecimal low and high values. Thanks. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt, HIVE-3678.2.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13504338#comment-13504338 ] Shreepadma Venugopalan commented on HIVE-3678: -- Updated patch is available on both JIRA and RB. Thanks. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt, HIVE-3678.2.patch.txt, HIVE-3678.3.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13504416#comment-13504416 ] Ashutosh Chauhan commented on HIVE-3678: +1 patch looks good, though fails to apply cleanly. Can you update the patch on latest trunk? Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt, HIVE-3678.2.patch.txt, HIVE-3678.3.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13503531#comment-13503531 ] Shreepadma Venugopalan commented on HIVE-3678: -- @Ashutosh: If store long/double types as a varchar instead of storing it as a numeric type, we can avoid evolving the schema when we add a BigDecimal type. That's the only benefit I see for storing long/double as a varchar. However, I agree with you that we should avoid untyping data when possible. Let me know your thoughts. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt, HIVE-3678.2.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13503535#comment-13503535 ] Ashutosh Chauhan commented on HIVE-3678: Lets store numeric types with long/double. Can you add varchar column for BigDecimal in the patch. Lets finish the work while we are at it and get it in. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt, HIVE-3678.2.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13503420#comment-13503420 ] Ashutosh Chauhan commented on HIVE-3678: @Shreepadma, Seems like you are suggesting to store BigDecimal statistics as plain strings in CLOBs. I think its a fair choice. As I am thinking more about choice between one json blob for all stats Vs different column for diff stat, I am inclining towards different columns. Apart from performance argument for (de)serialization, extra code that we have to write to do (de)serialization will present more opportunities to introduce bugs, its always good to avoid writing code unless its useful in substantial ways. In nutshell, I am +1 on the patch as it is. Unless, Carl has some other suggestions, lets go with it. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt, HIVE-3678.2.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13503425#comment-13503425 ] Shreepadma Venugopalan commented on HIVE-3678: -- @Ashutosh: Thanks for your comments. Do you think it makes sense to store numeric long/double/bigdecimal values in a varchar column? I don't see consistent BLOB/CLOB support across DB vendors and versions. If you agree, I'll make the change to store these numeric values in a varchar column and post a new patch. Thanks. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt, HIVE-3678.2.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13503438#comment-13503438 ] Ashutosh Chauhan commented on HIVE-3678: Lets stick with long and double for numeric types (as it is at the moment), we shouldn't untype the data unless there is a strong reason for it. For BigDecimal, lets go with varchar column. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt, HIVE-3678.2.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502440#comment-13502440 ] Shreepadma Venugopalan commented on HIVE-3678: -- @Ashutosh: My answers are inline. We can add two more column in M*ColumnStatistics table of type BigDecimal: BigDecimalLowValue and BigDecimalHighValue. But is BigDecimal type supported consistently across different DBs? Agreed, BigDecimal is not consistently supported across DBs. Hence we can't add a BigDecimal column consistently across DB vendors and versions easily. We can have these two columns of type Double, but then we loose precision. Yes, we can store BigDecimal and Long as Double but we will lose precision. We can store as plain strings in column of type varchar. The maximum number of digits after the decimal point in a BigDecimal number is unlimited for all practical purpose. If we stored it in a varchar, it could result in truncation of some digits following the decimal point in some cases, but this seems to be the only practical solution. We can store in json format in column of type varchar. The maximum number of digits in a BigDecimal number after the decimal point is unlimited for all practical purposes (Java allows nearly 2 billion digits after the decimal point). At this time, we collect MIN, MAX column values for numeric columns. If we stored BigDecimal value, we may exceed the varchar size limit and as a result truncate the JSON blob. This would result in a malformed JSON object. Additionally we will also lose some of the column statistics. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt, HIVE-3678.2.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501392#comment-13501392 ] Ashutosh Chauhan commented on HIVE-3678: Shreepadma, Any thoughts on how we are going to store low and high values for BigDecimal column? Seems like there are four possibilites: * We can add two more column in M*ColumnStatistics table of type BigDecimal: BigDecimalLowValue and BigDecimalHighValue. But is BigDecimal type supported consistently across different DBs? * We can have these two columns of type Double, but then we loose precision. * We can store as plain strings in column of type varchar. * We can store in json format in column of type varchar. What do you think will be the best way? I imagine there are other numeric types too for which neither of long or double will be a good choice. This question will come up again if we choose to add support for these later. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt, HIVE-3678.2.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500500#comment-13500500 ] Carl Steinbach commented on HIVE-3678: -- The upgrade scripts look good to me. As for HIVE-3712 which is included in this patch, I have started to wonder if it would be better for the metastore DB to store the column stats values (e.g. min/max value, num trues/falses, min/max/avg length, etc) as a JSON text blob. This approach would make the code more portable by eliminating dependencies on specific DBs and will also make it easier to add new fields in the future. The big downside of this approach is that we won't be able to push down column stats filters on these fields, but I'm not convinced that this is a practical use case in the first place. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500526#comment-13500526 ] Shreepadma Venugopalan commented on HIVE-3678: -- With the changes from HIVE-3712, the column schema has *no* dependency on any specific db. The column schema, with the changes from HIVE-3712, uses simple data types, which are supported across DBs. The primary motivation for making the change to the schema in HIVE-3712 was to avoid storing column statistics fields as a BLOB. The problem with using a BLOB is a) BLOBs are designed to store large volumes of data in the order of GBs and are hence stored outside the row. A consequence of this design is BLOBs don't perform well for storing small amounts of data. While some DBs such as Oracle inline small BLOBs, all DBs don't. While BLOBs are the only practical choice for storing data whose size is not known in advance, it is an overkill for storing around 100 bytes of data, and b) there is no uniform support across DB vendors and versions. Hence I don't really see the value in storing this as a JSON BLOB. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500570#comment-13500570 ] Carl Steinbach commented on HIVE-3678: -- Sorry for the confusion. When I wrote blob I was trying to convey only that the field will be opaque to the DB (since it's a JSON struct), not that it will actually be stored in a BLOB column. If we store the JSON struct in a VARCHAR we have at least 4000 bytes to work with. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[
https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500599#comment-13500599
]
Ashutosh Chauhan commented on HIVE-3678:
I agree with Carl, making it easier to evolve such that its independent of
exact type will be a win. We already have one such use-case with BigDecimal
support being added over on HIVE-2693.
Also, following looks unintentional change.
{code}
-- Constraints for table PARTITION_KEYS
-ALTER TABLE PARTITION_KEYS ADD CONSTRAINT PARTITION_KEYS_FK1 FOREIGN KEY
(TBL_ID) REFERENCES TBLS (TBL_ID) INITIALLY DEFERRED ;
+ALTER TABLE PARTITION_KEYS ADD CONSTRAINT PARTITION_KEYS_FK1 FOREIGN KEY
(TBTB_ID) REFERENCES TBLS (TBL_ID) INITIALLY DEFERRED ;
{code}
Add metastore upgrade scripts for column stats schema changes
-
Key: HIVE-3678
URL: https://issues.apache.org/jira/browse/HIVE-3678
Project: Hive
Issue Type: Bug
Components: Metastore
Reporter: Shreepadma Venugopalan
Assignee: Shreepadma Venugopalan
Fix For: 0.10.0
Attachments: HIVE-3678.1.patch.txt
Add upgrade script for column statistics schema changes for
Postgres/MySQL/Oracle/Derby
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499930#comment-13499930 ] Shreepadma Venugopalan commented on HIVE-3678: -- Review board link: https://reviews.apache.org/r/8119/ Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-3678.1.patch.txt Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493392#comment-13493392 ] Gang Tim Liu commented on HIVE-3678: [~shreepadma] thank you for working on it. Do you have plan when you submit a patch? I know it takes efforts to code and test in 4 different types of databases. Would you please release mysql version first? Then file a jira to follow up Postgres/Oracle/Derby. Just a suggestion for fast release but up to you to decide. thanks a lot again Tim Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493587#comment-13493587 ] Shreepadma Venugopalan commented on HIVE-3678: -- @Tim: I'm currently working on providing the upgrade scripts for different databases. Since there is a plan to release Hive 0.10 soon, we have to provide upgrade scripts for all of them. Thanks. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493589#comment-13493589 ] Gang Tim Liu commented on HIVE-3678: sure. no problem. Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
