[jira] [Commented] (HIVE-19057) Query result caching cannot be disabled by client
[
https://issues.apache.org/jira/browse/HIVE-19057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419324#comment-16419324
]
Deepesh Khandelwal commented on HIVE-19057:
---
Thanks [~jdere] for review and commit!
> Query result caching cannot be disabled by client
> -
>
> Key: HIVE-19057
> URL: https://issues.apache.org/jira/browse/HIVE-19057
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19057.1.patch
>
>
> HIVE-18513 introduced query results caching along with some toggles to
> control enabling/disabling it. We should whiltelist the following configs so
> that the end user can dynamically control it in their session.
> {noformat}
> hive.query.results.cache.enabled
> hive.query.results.cache.wait.for.pending.results
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19057) Query result caching cannot be disabled by client
[
https://issues.apache.org/jira/browse/HIVE-19057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal reassigned HIVE-19057:
-
Assignee: Deepesh Khandelwal
> Query result caching cannot be disabled by client
> -
>
> Key: HIVE-19057
> URL: https://issues.apache.org/jira/browse/HIVE-19057
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Major
> Attachments: HIVE-19057.1.patch
>
>
> HIVE-18513 introduced query results caching along with some toggles to
> control enabling/disabling it. We should whiltelist the following configs so
> that the end user can dynamically control it in their session.
> {noformat}
> hive.query.results.cache.enabled
> hive.query.results.cache.wait.for.pending.results
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19057) Query result caching cannot be disabled by client
[
https://issues.apache.org/jira/browse/HIVE-19057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414798#comment-16414798
]
Deepesh Khandelwal commented on HIVE-19057:
---
Attaching a patch for whitelisting the two properties. [~jdere] can you please
review?
> Query result caching cannot be disabled by client
> -
>
> Key: HIVE-19057
> URL: https://issues.apache.org/jira/browse/HIVE-19057
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Deepesh Khandelwal
>Priority: Major
> Attachments: HIVE-19057.1.patch
>
>
> HIVE-18513 introduced query results caching along with some toggles to
> control enabling/disabling it. We should whiltelist the following configs so
> that the end user can dynamically control it in their session.
> {noformat}
> hive.query.results.cache.enabled
> hive.query.results.cache.wait.for.pending.results
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19057) Query result caching cannot be disabled by client
[
https://issues.apache.org/jira/browse/HIVE-19057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-19057:
--
Attachment: HIVE-19057.1.patch
> Query result caching cannot be disabled by client
> -
>
> Key: HIVE-19057
> URL: https://issues.apache.org/jira/browse/HIVE-19057
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Deepesh Khandelwal
>Priority: Major
> Attachments: HIVE-19057.1.patch
>
>
> HIVE-18513 introduced query results caching along with some toggles to
> control enabling/disabling it. We should whiltelist the following configs so
> that the end user can dynamically control it in their session.
> {noformat}
> hive.query.results.cache.enabled
> hive.query.results.cache.wait.for.pending.results
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18631) Hive metastore schema initialization failing on mysql
[
https://issues.apache.org/jira/browse/HIVE-18631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353373#comment-16353373
]
Deepesh Khandelwal commented on HIVE-18631:
---
[~jcamachorodriguez] can you review the patch?
> Hive metastore schema initialization failing on mysql
> -
>
> Key: HIVE-18631
> URL: https://issues.apache.org/jira/browse/HIVE-18631
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
> Attachments: HIVE-18631.patch
>
>
> Hive metastore schema on mysql is broken after the commit for HIVE-18546.
> Following error is seen during schema initialization:
> {noformat}
> 0: jdbc:mysql://localhost.localdomain> CREATE TABLE IF NOT EXISTS `TBLS`
> ( `TBL_ID` bigint(20) NOT NULL, `CREATE_TIME` int(11) NOT NULL, `DB_ID`
> bigint(
> 20) DEFAULT NULL, `LAST_ACCESS_TIME` int(11) NOT NULL, `OWNER` varchar(767)
> CHAR
> ACTER SET latin1 COLLATE latin1_bin DEFAULT NULL, `RETENTION` int(11) NOT
> NULL,
> `SD_ID` bigint(20) DEFAULT NULL, `TBL_NAME` varchar(256) CHARACTER SET latin1
> CO
> LLATE latin1_bin DEFAULT NULL, `TBL_TYPE` varchar(128) CHARACTER SET latin1
> COLL
> ATE latin1_bin DEFAULT NULL, `VIEW_EXPANDED_TEXT` mediumtext,
> `VIEW_ORIGINAL_TEX
> T` mediumtext, `IS_REWRITE_ENABLED` bit(1) NOT NULL DEFAULT 0 PRIMARY KEY
> (`TBL_
> ID`), UNIQUE KEY `UNIQUETABLE` (`TBL_NAME`,`DB_ID`), KEY `TBLS_N50`
> (`SD_ID`), K
> EY `TBLS_N49` (`DB_ID`), CONSTRAINT `TBLS_FK1` FOREIGN KEY (`SD_ID`)
> REFERENCES
> `SDS` (`SD_ID`), CONSTRAINT `TBLS_FK2` FOREIGN KEY (`DB_ID`) REFERENCES `DBS`
> (`
> DB_ID`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1
> Error: You have an error in your SQL syntax; check the manual that
> corresponds to your MySQL server version for the right syntax to use near
> '(`TBL_ID`), UNIQUE KEY `UNIQUETABLE` (`TBL_NAME`,`DB_ID`), KEY `TBLS_N50`
> (`SD_I' at line 1 (state=42000,code=1064)
> Closing: 0:
> jdbc:mysql://localhost.localdomain/hivedb?createDatabaseIfNotExist=true
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> Underlying cause: java.io.IOException : Schema script failed, errorcode 2
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:586)
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:559)
> at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:304)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:218)
> Caused by: java.io.IOException: Schema script failed, errorcode 2
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:957)
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:935)
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:582)
> ... 8 more
> *** schemaTool failed ***{noformat}
> In the file metastore/scripts/upgrade/mysql/hive-schema-3.0.0.mysql.sql one
> of the column definitions in the `TBLS` table is missing a comma at the end
> {code:java}
> `IS_REWRITE_ENABLED` bit(1) NOT NULL DEFAULT 0{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18631) Hive metastore schema initialization failing on mysql
[
https://issues.apache.org/jira/browse/HIVE-18631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-18631:
--
Attachment: HIVE-18631.patch
> Hive metastore schema initialization failing on mysql
> -
>
> Key: HIVE-18631
> URL: https://issues.apache.org/jira/browse/HIVE-18631
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
> Attachments: HIVE-18631.patch
>
>
> Hive metastore schema on mysql is broken after the commit for HIVE-18546.
> Following error is seen during schema initialization:
> {noformat}
> 0: jdbc:mysql://localhost.localdomain> CREATE TABLE IF NOT EXISTS `TBLS`
> ( `TBL_ID` bigint(20) NOT NULL, `CREATE_TIME` int(11) NOT NULL, `DB_ID`
> bigint(
> 20) DEFAULT NULL, `LAST_ACCESS_TIME` int(11) NOT NULL, `OWNER` varchar(767)
> CHAR
> ACTER SET latin1 COLLATE latin1_bin DEFAULT NULL, `RETENTION` int(11) NOT
> NULL,
> `SD_ID` bigint(20) DEFAULT NULL, `TBL_NAME` varchar(256) CHARACTER SET latin1
> CO
> LLATE latin1_bin DEFAULT NULL, `TBL_TYPE` varchar(128) CHARACTER SET latin1
> COLL
> ATE latin1_bin DEFAULT NULL, `VIEW_EXPANDED_TEXT` mediumtext,
> `VIEW_ORIGINAL_TEX
> T` mediumtext, `IS_REWRITE_ENABLED` bit(1) NOT NULL DEFAULT 0 PRIMARY KEY
> (`TBL_
> ID`), UNIQUE KEY `UNIQUETABLE` (`TBL_NAME`,`DB_ID`), KEY `TBLS_N50`
> (`SD_ID`), K
> EY `TBLS_N49` (`DB_ID`), CONSTRAINT `TBLS_FK1` FOREIGN KEY (`SD_ID`)
> REFERENCES
> `SDS` (`SD_ID`), CONSTRAINT `TBLS_FK2` FOREIGN KEY (`DB_ID`) REFERENCES `DBS`
> (`
> DB_ID`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1
> Error: You have an error in your SQL syntax; check the manual that
> corresponds to your MySQL server version for the right syntax to use near
> '(`TBL_ID`), UNIQUE KEY `UNIQUETABLE` (`TBL_NAME`,`DB_ID`), KEY `TBLS_N50`
> (`SD_I' at line 1 (state=42000,code=1064)
> Closing: 0:
> jdbc:mysql://localhost.localdomain/hivedb?createDatabaseIfNotExist=true
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> Underlying cause: java.io.IOException : Schema script failed, errorcode 2
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:586)
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:559)
> at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:304)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:218)
> Caused by: java.io.IOException: Schema script failed, errorcode 2
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:957)
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:935)
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:582)
> ... 8 more
> *** schemaTool failed ***{noformat}
> In the file metastore/scripts/upgrade/mysql/hive-schema-3.0.0.mysql.sql one
> of the column definitions in the `TBLS` table is missing a comma at the end
> {code:java}
> `IS_REWRITE_ENABLED` bit(1) NOT NULL DEFAULT 0{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18631) Hive metastore schema initialization failing on mysql
[
https://issues.apache.org/jira/browse/HIVE-18631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal reassigned HIVE-18631:
-
> Hive metastore schema initialization failing on mysql
> -
>
> Key: HIVE-18631
> URL: https://issues.apache.org/jira/browse/HIVE-18631
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
>
> Hive metastore schema on mysql is broken after the commit for HIVE-18546.
> Following error is seen during schema initialization:
> {noformat}
> 0: jdbc:mysql://localhost.localdomain> CREATE TABLE IF NOT EXISTS `TBLS`
> ( `TBL_ID` bigint(20) NOT NULL, `CREATE_TIME` int(11) NOT NULL, `DB_ID`
> bigint(
> 20) DEFAULT NULL, `LAST_ACCESS_TIME` int(11) NOT NULL, `OWNER` varchar(767)
> CHAR
> ACTER SET latin1 COLLATE latin1_bin DEFAULT NULL, `RETENTION` int(11) NOT
> NULL,
> `SD_ID` bigint(20) DEFAULT NULL, `TBL_NAME` varchar(256) CHARACTER SET latin1
> CO
> LLATE latin1_bin DEFAULT NULL, `TBL_TYPE` varchar(128) CHARACTER SET latin1
> COLL
> ATE latin1_bin DEFAULT NULL, `VIEW_EXPANDED_TEXT` mediumtext,
> `VIEW_ORIGINAL_TEX
> T` mediumtext, `IS_REWRITE_ENABLED` bit(1) NOT NULL DEFAULT 0 PRIMARY KEY
> (`TBL_
> ID`), UNIQUE KEY `UNIQUETABLE` (`TBL_NAME`,`DB_ID`), KEY `TBLS_N50`
> (`SD_ID`), K
> EY `TBLS_N49` (`DB_ID`), CONSTRAINT `TBLS_FK1` FOREIGN KEY (`SD_ID`)
> REFERENCES
> `SDS` (`SD_ID`), CONSTRAINT `TBLS_FK2` FOREIGN KEY (`DB_ID`) REFERENCES `DBS`
> (`
> DB_ID`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1
> Error: You have an error in your SQL syntax; check the manual that
> corresponds to your MySQL server version for the right syntax to use near
> '(`TBL_ID`), UNIQUE KEY `UNIQUETABLE` (`TBL_NAME`,`DB_ID`), KEY `TBLS_N50`
> (`SD_I' at line 1 (state=42000,code=1064)
> Closing: 0:
> jdbc:mysql://localhost.localdomain/hivedb?createDatabaseIfNotExist=true
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> Underlying cause: java.io.IOException : Schema script failed, errorcode 2
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:586)
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:559)
> at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:304)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:218)
> Caused by: java.io.IOException: Schema script failed, errorcode 2
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:957)
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:935)
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:582)
> ... 8 more
> *** schemaTool failed ***{noformat}
> In the file metastore/scripts/upgrade/mysql/hive-schema-3.0.0.mysql.sql one
> of the column definitions in the `TBLS` table is missing a comma at the end
> {code:java}
> `IS_REWRITE_ENABLED` bit(1) NOT NULL DEFAULT 0{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18579) Changes from HIVE-18495 introduced import paths from shaded jars
[
https://issues.apache.org/jira/browse/HIVE-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal reassigned HIVE-18579:
-
> Changes from HIVE-18495 introduced import paths from shaded jars
>
>
> Key: HIVE-18579
> URL: https://issues.apache.org/jira/browse/HIVE-18579
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Zoltan Haindrich
>Priority: Critical
>
> When compiling the latest code after HIVE-18495 seeing the following issue:
> {noformat}
> -
> [ERROR] COMPILATION ERROR :
> -
> [ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/HiveTestEnvSetup.java:[29,64]
> package org.apache.hadoop.hbase.shaded.com.google.common.collect does not
> exist
> [ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/MiniZooKeeperCluster.java:[43,68]
> package org.apache.hadoop.hbase.shaded.com.google.common.annotations does
> not exist
> ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/MiniZooKeeperCluster.java:[100,4]
> cannot find symbol
> symbol: class VisibleForTesting
> location: class org.apache.hive.testutils.MiniZooKeeperCluster
> [INFO] 3 errors
> {noformat}
> Seems like org.apache.hadoop.hbase.shaded.com.google.* is being used, I am
> guessing the idea was to use com.google.*. Not sure why we didn't see this
> failing in Apache Hive build system.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18465) Hive metastore schema initialization failing on postgres
[
https://issues.apache.org/jira/browse/HIVE-18465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328395#comment-16328395
]
Deepesh Khandelwal commented on HIVE-18465:
---
[~jcamachorodriguez] can you review the patch?
> Hive metastore schema initialization failing on postgres
>
>
> Key: HIVE-18465
> URL: https://issues.apache.org/jira/browse/HIVE-18465
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18465.patch
>
>
> Hive metastore schema on postgres is broken after the commit for HIVE-14498.
> Following error is seen during schema initialization:
> {noformat}
> 0: jdbc:postgresql://localhost.localdomain:54> ALTER TABLE ONLY
> "MV_CREATION_MET
> ADATA" ADD CONSTRAINT "MV_CREATION_METADATA_FK" FOREIGN KEY ("TBL_ID")
> REFERENCE
> S "TBLS"("TBL_ID") DEFERRABLE
> Error: ERROR: there is no unique constraint matching given keys for
> referenced table "TBLS" (state=42830,code=0)
> Closing: 0: jdbc:postgresql://localhost.localdomain:5432/hive
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> Underlying cause: java.io.IOException : Schema script failed, errorcode 2
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:586)
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:559)
> at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.io.IOException: Schema script failed, errorcode 2
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:957)
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:935)
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:582)
> ... 8 more
> *** schemaTool failed ***{noformat}
> In the file metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql
> the ordering of statement
> {noformat}
> ALTER TABLE ONLY "MV_CREATION_METADATA"
> ADD CONSTRAINT "MV_CREATION_METADATA_FK" FOREIGN KEY ("TBL_ID") REFERENCES
> "TBLS"("TBL_ID") DEFERRABLE;{noformat}
> is before the definition of unique constraints for TBLS which is causing the
> issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18465) Hive metastore schema initialization failing on postgres
[
https://issues.apache.org/jira/browse/HIVE-18465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-18465:
--
Fix Version/s: 3.0.0
Status: Patch Available (was: Open)
> Hive metastore schema initialization failing on postgres
>
>
> Key: HIVE-18465
> URL: https://issues.apache.org/jira/browse/HIVE-18465
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18465.patch
>
>
> Hive metastore schema on postgres is broken after the commit for HIVE-14498.
> Following error is seen during schema initialization:
> {noformat}
> 0: jdbc:postgresql://localhost.localdomain:54> ALTER TABLE ONLY
> "MV_CREATION_MET
> ADATA" ADD CONSTRAINT "MV_CREATION_METADATA_FK" FOREIGN KEY ("TBL_ID")
> REFERENCE
> S "TBLS"("TBL_ID") DEFERRABLE
> Error: ERROR: there is no unique constraint matching given keys for
> referenced table "TBLS" (state=42830,code=0)
> Closing: 0: jdbc:postgresql://localhost.localdomain:5432/hive
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> Underlying cause: java.io.IOException : Schema script failed, errorcode 2
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:586)
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:559)
> at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.io.IOException: Schema script failed, errorcode 2
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:957)
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:935)
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:582)
> ... 8 more
> *** schemaTool failed ***{noformat}
> In the file metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql
> the ordering of statement
> {noformat}
> ALTER TABLE ONLY "MV_CREATION_METADATA"
> ADD CONSTRAINT "MV_CREATION_METADATA_FK" FOREIGN KEY ("TBL_ID") REFERENCES
> "TBLS"("TBL_ID") DEFERRABLE;{noformat}
> is before the definition of unique constraints for TBLS which is causing the
> issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18465) Hive metastore schema initialization failing on postgres
[
https://issues.apache.org/jira/browse/HIVE-18465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-18465:
--
Attachment: HIVE-18465.patch
> Hive metastore schema initialization failing on postgres
>
>
> Key: HIVE-18465
> URL: https://issues.apache.org/jira/browse/HIVE-18465
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Major
> Attachments: HIVE-18465.patch
>
>
> Hive metastore schema on postgres is broken after the commit for HIVE-14498.
> Following error is seen during schema initialization:
> {noformat}
> 0: jdbc:postgresql://localhost.localdomain:54> ALTER TABLE ONLY
> "MV_CREATION_MET
> ADATA" ADD CONSTRAINT "MV_CREATION_METADATA_FK" FOREIGN KEY ("TBL_ID")
> REFERENCE
> S "TBLS"("TBL_ID") DEFERRABLE
> Error: ERROR: there is no unique constraint matching given keys for
> referenced table "TBLS" (state=42830,code=0)
> Closing: 0: jdbc:postgresql://localhost.localdomain:5432/hive
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> Underlying cause: java.io.IOException : Schema script failed, errorcode 2
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:586)
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:559)
> at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.io.IOException: Schema script failed, errorcode 2
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:957)
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:935)
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:582)
> ... 8 more
> *** schemaTool failed ***{noformat}
> In the file metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql
> the ordering of statement
> {noformat}
> ALTER TABLE ONLY "MV_CREATION_METADATA"
> ADD CONSTRAINT "MV_CREATION_METADATA_FK" FOREIGN KEY ("TBL_ID") REFERENCES
> "TBLS"("TBL_ID") DEFERRABLE;{noformat}
> is before the definition of unique constraints for TBLS which is causing the
> issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18465) Hive metastore schema initialization failing on postgres
[
https://issues.apache.org/jira/browse/HIVE-18465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal reassigned HIVE-18465:
-
> Hive metastore schema initialization failing on postgres
>
>
> Key: HIVE-18465
> URL: https://issues.apache.org/jira/browse/HIVE-18465
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Major
>
> Hive metastore schema on postgres is broken after the commit for HIVE-14498.
> Following error is seen during schema initialization:
> {noformat}
> 0: jdbc:postgresql://localhost.localdomain:54> ALTER TABLE ONLY
> "MV_CREATION_MET
> ADATA" ADD CONSTRAINT "MV_CREATION_METADATA_FK" FOREIGN KEY ("TBL_ID")
> REFERENCE
> S "TBLS"("TBL_ID") DEFERRABLE
> Error: ERROR: there is no unique constraint matching given keys for
> referenced table "TBLS" (state=42830,code=0)
> Closing: 0: jdbc:postgresql://localhost.localdomain:5432/hive
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> Underlying cause: java.io.IOException : Schema script failed, errorcode 2
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:586)
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:559)
> at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.io.IOException: Schema script failed, errorcode 2
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:957)
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:935)
> at
> org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:582)
> ... 8 more
> *** schemaTool failed ***{noformat}
> In the file metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql
> the ordering of statement
> {noformat}
> ALTER TABLE ONLY "MV_CREATION_METADATA"
> ADD CONSTRAINT "MV_CREATION_METADATA_FK" FOREIGN KEY ("TBL_ID") REFERENCES
> "TBLS"("TBL_ID") DEFERRABLE;{noformat}
> is before the definition of unique constraints for TBLS which is causing the
> issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18220) Workload Management tables have broken constraints defined on postgres schema
[
https://issues.apache.org/jira/browse/HIVE-18220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-18220:
--
Status: Patch Available (was: Open)
> Workload Management tables have broken constraints defined on postgres schema
> -
>
> Key: HIVE-18220
> URL: https://issues.apache.org/jira/browse/HIVE-18220
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
> Attachments: HIVE-18220.patch
>
>
> Schema initialization on Postgres fails with the following error:
> {noformat}
> 0: jdbc:postgresql://localhost.localdomain:54> ALTER TABLE ONLY "WM_POOL" ADD
> CO
> NSTRAINT "UNIQUE_WM_RESOURCEPLAN" UNIQUE ("NAME")
> Error: ERROR: column "NAME" named in key does not exist (state=42703,code=0)
> Closing: 0: jdbc:postgresql://localhost.localdomain:5432/hive
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> Underlying cause: java.io.IOException : Schema script failed, errorcode 2
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:586)
> at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:559)
> at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.io.IOException: Schema script failed, errorcode 2
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:957)
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:935)
> at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:582)
> ... 8 more
> {noformat}
> It is due to couple on incorrect constraint definitions in the schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18220) Workload Management tables have broken constraints defined on postgres schema
[
https://issues.apache.org/jira/browse/HIVE-18220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-18220:
--
Attachment: HIVE-18220.patch
Attaching a patch with the fix. Verified in my local postgres db that applying
this patch causes schema initialization to proceed. [~sershe] can you review?
> Workload Management tables have broken constraints defined on postgres schema
> -
>
> Key: HIVE-18220
> URL: https://issues.apache.org/jira/browse/HIVE-18220
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
> Attachments: HIVE-18220.patch
>
>
> Schema initialization on Postgres fails with the following error:
> {noformat}
> 0: jdbc:postgresql://localhost.localdomain:54> ALTER TABLE ONLY "WM_POOL" ADD
> CO
> NSTRAINT "UNIQUE_WM_RESOURCEPLAN" UNIQUE ("NAME")
> Error: ERROR: column "NAME" named in key does not exist (state=42703,code=0)
> Closing: 0: jdbc:postgresql://localhost.localdomain:5432/hive
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> Underlying cause: java.io.IOException : Schema script failed, errorcode 2
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:586)
> at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:559)
> at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.io.IOException: Schema script failed, errorcode 2
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:957)
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:935)
> at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:582)
> ... 8 more
> {noformat}
> It is due to couple on incorrect constraint definitions in the schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18220) Workload Management tables have broken constraints defined on postgres schema
[
https://issues.apache.org/jira/browse/HIVE-18220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal reassigned HIVE-18220:
-
> Workload Management tables have broken constraints defined on postgres schema
> -
>
> Key: HIVE-18220
> URL: https://issues.apache.org/jira/browse/HIVE-18220
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
>
> Schema initialization on Postgres fails with the following error:
> {noformat}
> 0: jdbc:postgresql://localhost.localdomain:54> ALTER TABLE ONLY "WM_POOL" ADD
> CO
> NSTRAINT "UNIQUE_WM_RESOURCEPLAN" UNIQUE ("NAME")
> Error: ERROR: column "NAME" named in key does not exist (state=42703,code=0)
> Closing: 0: jdbc:postgresql://localhost.localdomain:5432/hive
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> Underlying cause: java.io.IOException : Schema script failed, errorcode 2
> org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization
> FAILED! Metastore state would be inconsistent !!
> at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:586)
> at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:559)
> at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.io.IOException: Schema script failed, errorcode 2
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:957)
> at
> org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:935)
> at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:582)
> ... 8 more
> {noformat}
> It is due to couple on incorrect constraint definitions in the schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18195) Hive schema broken on postgres
[
https://issues.apache.org/jira/browse/HIVE-18195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16274949#comment-16274949
]
Deepesh Khandelwal commented on HIVE-18195:
---
The Apache Yetus warning is due to missing Apache header in
ql/src/java/org/apache/hadoop/hive/ql/optimizer/TablePropertyEnrichmentOptimizer.java
that got committed as part of HIVE-14792, created a separate bug HIVE-18198 to
track that one. [~sershe] can you review that one as well?
> Hive schema broken on postgres
> --
>
> Key: HIVE-18195
> URL: https://issues.apache.org/jira/browse/HIVE-18195
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
> Attachments: HIVE-18195.patch
>
>
> Hive metastore schema on postgres is broken after the commit for HIVE-17954.
> Basicaly the following file
> metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql incorrectly
> defines WM_POOL with column ALLOC_FRACTION with DOUBLE data type, it should
> be double precision.
> {noformat}
> CREATE TABLE "WM_POOL" (
> "POOL_ID" bigint NOT NULL,
> "RP_ID" bigint NOT NULL,
> "PATH" character varying(1024) NOT NULL,
> "ALLOC_FRACTION" DOUBLE,
> "QUERY_PARALLELISM" integer,
> "SCHEDULING_POLICY" character varying(1024)
> );
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18198) TablePropertyEnrichmentOptimizer.java is missing the Apache license header
[
https://issues.apache.org/jira/browse/HIVE-18198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-18198:
--
Attachment: HIVE-18198.patch
> TablePropertyEnrichmentOptimizer.java is missing the Apache license header
> --
>
> Key: HIVE-18198
> URL: https://issues.apache.org/jira/browse/HIVE-18198
> Project: Hive
> Issue Type: Bug
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Attachments: HIVE-18198.patch
>
>
> This causes warnings in the yetus check:
> {quote}
> Lines that start with ? in the ASF License report indicate files that do
> not have an Apache license header:
> !?
> /data/hiveptest/working/yetus/ql/src/java/org/apache/hadoop/hive/ql/optimizer/TablePropertyEnrichmentOptimizer.java
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18198) TablePropertyEnrichmentOptimizer.java is missing the Apache license header
[
https://issues.apache.org/jira/browse/HIVE-18198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal reassigned HIVE-18198:
-
> TablePropertyEnrichmentOptimizer.java is missing the Apache license header
> --
>
> Key: HIVE-18198
> URL: https://issues.apache.org/jira/browse/HIVE-18198
> Project: Hive
> Issue Type: Bug
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>
> This causes warnings in the yetus check:
> {quote}
> Lines that start with ? in the ASF License report indicate files that do
> not have an Apache license header:
> !?
> /data/hiveptest/working/yetus/ql/src/java/org/apache/hadoop/hive/ql/optimizer/TablePropertyEnrichmentOptimizer.java
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18195) Hive schema broken on postgres
[
https://issues.apache.org/jira/browse/HIVE-18195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-18195:
--
Attachment: HIVE-18195.patch
Attaching patch with the fix. Please review.
cc. [~sershe]
> Hive schema broken on postgres
> --
>
> Key: HIVE-18195
> URL: https://issues.apache.org/jira/browse/HIVE-18195
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
> Attachments: HIVE-18195.patch
>
>
> Hive metastore schema on postgres is broken after the commit for HIVE-17954.
> Basicaly the following file
> metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql incorrectly
> defines WM_POOL with column ALLOC_FRACTION with DOUBLE data type, it should
> be double precision.
> {noformat}
> CREATE TABLE "WM_POOL" (
> "POOL_ID" bigint NOT NULL,
> "RP_ID" bigint NOT NULL,
> "PATH" character varying(1024) NOT NULL,
> "ALLOC_FRACTION" DOUBLE,
> "QUERY_PARALLELISM" integer,
> "SCHEDULING_POLICY" character varying(1024)
> );
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18195) Hive schema broken on postgres
[
https://issues.apache.org/jira/browse/HIVE-18195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal reassigned HIVE-18195:
-
> Hive schema broken on postgres
> --
>
> Key: HIVE-18195
> URL: https://issues.apache.org/jira/browse/HIVE-18195
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
>
> Hive metastore schema on postgres is broken after the commit for HIVE-17954.
> Basicaly the following file
> metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql incorrectly
> defines WM_POOL with column ALLOC_FRACTION with DOUBLE data type, it should
> be double precision.
> {noformat}
> CREATE TABLE "WM_POOL" (
> "POOL_ID" bigint NOT NULL,
> "RP_ID" bigint NOT NULL,
> "PATH" character varying(1024) NOT NULL,
> "ALLOC_FRACTION" DOUBLE,
> "QUERY_PARALLELISM" integer,
> "SCHEDULING_POLICY" character varying(1024)
> );
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-16234) Add support for quarter in trunc udf
[
https://issues.apache.org/jira/browse/HIVE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930931#comment-15930931
]
Deepesh Khandelwal commented on HIVE-16234:
---
Thanks [~ashutoshc] for the review and commit!
[~leftylev] for the doc part, is that something I need to do once 2.2.0 is
released?
> Add support for quarter in trunc udf
>
>
> Key: HIVE-16234
> URL: https://issues.apache.org/jira/browse/HIVE-16234
> Project: Hive
> Issue Type: Improvement
> Components: UDF
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Labels: TODOC2.2
> Fix For: 2.2.0
>
> Attachments: HIVE-16234.1.patch
>
>
> Hive has a Date function trunc(string date, string format) that returns date
> truncated to the unit specified by the format. Supported formats:
> MONTH/MON/MM, YEAR//YY.
> Goal here is to extend support to QUARTER/Q.
> Example:
> SELECT trunc('2017-03-15', 'Q');
> '2017-01-01'
> SELECT trunc('2017-12-31', 'Q');
> '2017-10-01'
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-16234) Add support for quarter in trunc udf
[
https://issues.apache.org/jira/browse/HIVE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-16234:
--
Attachment: HIVE-16234.1.patch
Attaching patch for review. Also created a review board, see
https://reviews.apache.org/r/57701/
cc. [~ashutoshc] [~jdere]
> Add support for quarter in trunc udf
>
>
> Key: HIVE-16234
> URL: https://issues.apache.org/jira/browse/HIVE-16234
> Project: Hive
> Issue Type: Improvement
> Components: UDF
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Attachments: HIVE-16234.1.patch
>
>
> Hive has a Date function trunc(string date, string format) that returns date
> truncated to the unit specified by the format. Supported formats:
> MONTH/MON/MM, YEAR//YY.
> Goal here is to extend support to QUARTER/Q.
> Example:
> SELECT trunc('2017-03-15', 'Q');
> '2017-01-01'
> SELECT trunc('2017-12-31', 'Q');
> '2017-10-01'
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (HIVE-16234) Add support for quarter in trunc udf
[
https://issues.apache.org/jira/browse/HIVE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal reassigned HIVE-16234:
-
> Add support for quarter in trunc udf
>
>
> Key: HIVE-16234
> URL: https://issues.apache.org/jira/browse/HIVE-16234
> Project: Hive
> Issue Type: Improvement
> Components: UDF
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>
> Hive has a Date function trunc(string date, string format) that returns date
> truncated to the unit specified by the format. Supported formats:
> MONTH/MON/MM, YEAR//YY.
> Goal here is to extend support to QUARTER/Q.
> Example:
> SELECT trunc('2017-03-15', 'Q');
> '2017-01-01'
> SELECT trunc('2017-12-31', 'Q');
> '2017-10-01'
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15930) Syntax error in the MySQL metastore schema definition
[
https://issues.apache.org/jira/browse/HIVE-15930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868780#comment-15868780
]
Deepesh Khandelwal commented on HIVE-15930:
---
Thanks [~sershe] for the review and commit!
> Syntax error in the MySQL metastore schema definition
> -
>
> Key: HIVE-15930
> URL: https://issues.apache.org/jira/browse/HIVE-15930
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Critical
> Fix For: 2.2.0
>
> Attachments: HIVE-15930.patch
>
>
> The MySQL metastore definition
> metastore/scripts/upgrade/mysql/hive-schema-2.2.0.mysql.sql contains a syntax
> error for the table NOTIFICATION_LOG:
> {noformat}
> CREATE TABLE IF NOT EXISTS `NOTIFICATION_LOG`
> (
> `NL_ID` BIGINT(20) NOT NULL,
> `EVENT_ID` BIGINT(20) NOT NULL,
> `EVENT_TIME` INT(11) NOT NULL,
> `EVENT_TYPE` varchar(32) NOT NULL,
> `DB_NAME` varchar(128),
> `TBL_NAME` varchar(128),
> `MESSAGE` longtext,
> `MESSAGE_FORMAT` varchar(16)
> PRIMARY KEY (`NL_ID`)
> ) ENGINE=InnoDB DEFAULT CHARSET=latin1;
> {noformat}
> Notice a comma is missing after {{`MESSAGE_FORMAT` varchar(16)}}.
> This got introduced through HIVE-10562.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15930) Syntax error in the MySQL metastore schema definition
[
https://issues.apache.org/jira/browse/HIVE-15930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868505#comment-15868505
]
Deepesh Khandelwal commented on HIVE-15930:
---
Failures look unrelated to my change.
> Syntax error in the MySQL metastore schema definition
> -
>
> Key: HIVE-15930
> URL: https://issues.apache.org/jira/browse/HIVE-15930
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Critical
> Fix For: 2.2.0
>
> Attachments: HIVE-15930.patch
>
>
> The MySQL metastore definition
> metastore/scripts/upgrade/mysql/hive-schema-2.2.0.mysql.sql contains a syntax
> error for the table NOTIFICATION_LOG:
> {noformat}
> CREATE TABLE IF NOT EXISTS `NOTIFICATION_LOG`
> (
> `NL_ID` BIGINT(20) NOT NULL,
> `EVENT_ID` BIGINT(20) NOT NULL,
> `EVENT_TIME` INT(11) NOT NULL,
> `EVENT_TYPE` varchar(32) NOT NULL,
> `DB_NAME` varchar(128),
> `TBL_NAME` varchar(128),
> `MESSAGE` longtext,
> `MESSAGE_FORMAT` varchar(16)
> PRIMARY KEY (`NL_ID`)
> ) ENGINE=InnoDB DEFAULT CHARSET=latin1;
> {noformat}
> Notice a comma is missing after {{`MESSAGE_FORMAT` varchar(16)}}.
> This got introduced through HIVE-10562.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-15930) Syntax error in the MySQL metastore schema definition
[
https://issues.apache.org/jira/browse/HIVE-15930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-15930:
--
Fix Version/s: 2.2.0
Status: Patch Available (was: Open)
> Syntax error in the MySQL metastore schema definition
> -
>
> Key: HIVE-15930
> URL: https://issues.apache.org/jira/browse/HIVE-15930
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Critical
> Fix For: 2.2.0
>
> Attachments: HIVE-15930.patch
>
>
> The MySQL metastore definition
> metastore/scripts/upgrade/mysql/hive-schema-2.2.0.mysql.sql contains a syntax
> error for the table NOTIFICATION_LOG:
> {noformat}
> CREATE TABLE IF NOT EXISTS `NOTIFICATION_LOG`
> (
> `NL_ID` BIGINT(20) NOT NULL,
> `EVENT_ID` BIGINT(20) NOT NULL,
> `EVENT_TIME` INT(11) NOT NULL,
> `EVENT_TYPE` varchar(32) NOT NULL,
> `DB_NAME` varchar(128),
> `TBL_NAME` varchar(128),
> `MESSAGE` longtext,
> `MESSAGE_FORMAT` varchar(16)
> PRIMARY KEY (`NL_ID`)
> ) ENGINE=InnoDB DEFAULT CHARSET=latin1;
> {noformat}
> Notice a comma is missing after {{`MESSAGE_FORMAT` varchar(16)}}.
> This got introduced through HIVE-10562.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-15930) Syntax error in the MySQL metastore schema definition
[
https://issues.apache.org/jira/browse/HIVE-15930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-15930:
--
Attachment: HIVE-15930.patch
Attaching a patch with the fix.
cc. [~sushanth] [~thejas]
> Syntax error in the MySQL metastore schema definition
> -
>
> Key: HIVE-15930
> URL: https://issues.apache.org/jira/browse/HIVE-15930
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Critical
> Attachments: HIVE-15930.patch
>
>
> The MySQL metastore definition
> metastore/scripts/upgrade/mysql/hive-schema-2.2.0.mysql.sql contains a syntax
> error for the table NOTIFICATION_LOG:
> {noformat}
> CREATE TABLE IF NOT EXISTS `NOTIFICATION_LOG`
> (
> `NL_ID` BIGINT(20) NOT NULL,
> `EVENT_ID` BIGINT(20) NOT NULL,
> `EVENT_TIME` INT(11) NOT NULL,
> `EVENT_TYPE` varchar(32) NOT NULL,
> `DB_NAME` varchar(128),
> `TBL_NAME` varchar(128),
> `MESSAGE` longtext,
> `MESSAGE_FORMAT` varchar(16)
> PRIMARY KEY (`NL_ID`)
> ) ENGINE=InnoDB DEFAULT CHARSET=latin1;
> {noformat}
> Notice a comma is missing after {{`MESSAGE_FORMAT` varchar(16)}}.
> This got introduced through HIVE-10562.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (HIVE-15930) Syntax error in the MySQL metastore schema definition
[
https://issues.apache.org/jira/browse/HIVE-15930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal reassigned HIVE-15930:
-
> Syntax error in the MySQL metastore schema definition
> -
>
> Key: HIVE-15930
> URL: https://issues.apache.org/jira/browse/HIVE-15930
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Critical
>
> The MySQL metastore definition
> metastore/scripts/upgrade/mysql/hive-schema-2.2.0.mysql.sql contains a syntax
> error for the table NOTIFICATION_LOG:
> {noformat}
> CREATE TABLE IF NOT EXISTS `NOTIFICATION_LOG`
> (
> `NL_ID` BIGINT(20) NOT NULL,
> `EVENT_ID` BIGINT(20) NOT NULL,
> `EVENT_TIME` INT(11) NOT NULL,
> `EVENT_TYPE` varchar(32) NOT NULL,
> `DB_NAME` varchar(128),
> `TBL_NAME` varchar(128),
> `MESSAGE` longtext,
> `MESSAGE_FORMAT` varchar(16)
> PRIMARY KEY (`NL_ID`)
> ) ENGINE=InnoDB DEFAULT CHARSET=latin1;
> {noformat}
> Notice a comma is missing after {{`MESSAGE_FORMAT` varchar(16)}}.
> This got introduced through HIVE-10562.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15220) WebHCat test driver not capturing end time of test accurately
[
https://issues.apache.org/jira/browse/HIVE-15220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671380#comment-15671380
]
Deepesh Khandelwal commented on HIVE-15220:
---
Thanks [~thejas] for the review and commit!
> WebHCat test driver not capturing end time of test accurately
> -
>
> Key: HIVE-15220
> URL: https://issues.apache.org/jira/browse/HIVE-15220
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Trivial
> Fix For: 2.2.0
>
> Attachments: HIVE-15220.1.patch
>
>
> Webhcat e2e testsuite prints message while ending test run:
> {noformat}
> Ending test at 1479264720
> {noformat}
> Currently it is not capturing the end time correctly.
> NO PRECOMMIT TESTS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15220) WebHCat test driver not capturing end time of test accurately
[
https://issues.apache.org/jira/browse/HIVE-15220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-15220:
--
Status: Patch Available (was: Open)
> WebHCat test driver not capturing end time of test accurately
> -
>
> Key: HIVE-15220
> URL: https://issues.apache.org/jira/browse/HIVE-15220
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Trivial
> Attachments: HIVE-15220.1.patch
>
>
> Webhcat e2e testsuite prints message while ending test run:
> {noformat}
> Ending test at 1479264720
> {noformat}
> Currently it is not capturing the end time correctly.
> NO PRECOMMIT TESTS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15220) WebHCat test driver not capturing end time of test accurately
[
https://issues.apache.org/jira/browse/HIVE-15220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-15220:
--
Attachment: HIVE-15220.1.patch
Attaching the patch with the change.
[~thejas] [~daijy] can you please review?
> WebHCat test driver not capturing end time of test accurately
> -
>
> Key: HIVE-15220
> URL: https://issues.apache.org/jira/browse/HIVE-15220
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Trivial
> Attachments: HIVE-15220.1.patch
>
>
> Webhcat e2e testsuite prints message while ending test run:
> {noformat}
> Ending test at 1479264720
> {noformat}
> Currently it is not capturing the end time correctly.
> NO PRECOMMIT TESTS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15220) WebHCat test driver not capturing end time of test accurately
[
https://issues.apache.org/jira/browse/HIVE-15220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-15220:
--
Summary: WebHCat test driver not capturing end time of test accurately
(was: WebHCat test not capturing end time of test accurately)
> WebHCat test driver not capturing end time of test accurately
> -
>
> Key: HIVE-15220
> URL: https://issues.apache.org/jira/browse/HIVE-15220
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Trivial
>
> Webhcat e2e testsuite prints message while ending test run:
> {noformat}
> Ending test at 1479264720
> {noformat}
> Currently it is not capturing the end time correctly.
> NO PRECOMMIT TESTS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13002) metastore call timing is not threadsafe
[
https://issues.apache.org/jira/browse/HIVE-13002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169642#comment-15169642
]
Deepesh Khandelwal commented on HIVE-13002:
---
[~sershe] any reason why you replaced all the Map<...> with
ConcurrentHashMap<...>? I would think that just changing the implementation at
the initialization time would be enough and better.
> metastore call timing is not threadsafe
> ---
>
> Key: HIVE-13002
> URL: https://issues.apache.org/jira/browse/HIVE-13002
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-13002.01.patch, HIVE-13002.patch
>
>
> Discovered in some q test run:
> {noformat}
> TestCliDriver.testCliDriver_insert_values_orig_table:123->runTest:199
> Unexpected exception java.util.ConcurrentModificationException
> at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926)
> at java.util.HashMap$EntryIterator.next(HashMap.java:966)
> at java.util.HashMap$EntryIterator.next(HashMap.java:964)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.dumpAndClearMetaCallTiming(Hive.java:3412)
> at
> org.apache.hadoop.hive.ql.Driver.dumpMetaCallTimingWithoutEx(Driver.java:574)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1722)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1342)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1113)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1101)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-12419) hive.log.trace.id needs to be whitelisted
[ https://issues.apache.org/jira/browse/HIVE-12419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepesh Khandelwal resolved HIVE-12419. --- Resolution: Not A Problem Looks like common/src/java/org/apache/hadoop/hive/conf/HiveConf.java already has safe regex for hive.log.* so should work without any issues. > hive.log.trace.id needs to be whitelisted > - > > Key: HIVE-12419 > URL: https://issues.apache.org/jira/browse/HIVE-12419 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Deepesh Khandelwal >Assignee: Deepesh Khandelwal > Fix For: 2.0.0 > > > HIVE-12249 introduces hive.log.trace.id as part of improving logging for hive > queries. The property needs to be added to SQL Std Auth whitelisted > properties list to be usable with HiveServer2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11902) Abort txn cleanup thread throws SyntaxErrorException
[
https://issues.apache.org/jira/browse/HIVE-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14907015#comment-14907015
]
Deepesh Khandelwal commented on HIVE-11902:
---
Thanks [~ekoifman] for the review and commit!
> Abort txn cleanup thread throws SyntaxErrorException
>
>
> Key: HIVE-11902
> URL: https://issues.apache.org/jira/browse/HIVE-11902
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Fix For: 1.3.0
>
> Attachments: HIVE-11902.patch
>
>
> When cleaning left over transactions we see the DeadTxnReaper code threw the
> following exception:
> {noformat}
> 2015-09-21 05:23:38,148 WARN [DeadTxnReaper-0]: txn.TxnHandler
> (TxnHandler.java:performTimeOuts(1876)) - Aborting timedout transactions
> failed due to You have an error in your SQL syntax; check the manual that
> corresponds to your MySQL server version for the right syntax to use near ')'
> at line 1(SQLState=42000,ErrorCode=1064)
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error
> in your SQL syntax; check the manual that corresponds to your MySQL server
> version for the right syntax to use near ')' at line 1
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
> at com.mysql.jdbc.Util.getInstance(Util.java:360)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:978)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2526)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1618)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1549)
> at
> com.jolbox.bonecp.StatementHandle.executeUpdate(StatementHandle.java:497)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.abortTxns(TxnHandler.java:1275)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.performTimeOuts(TxnHandler.java:1866)
> at
> org.apache.hadoop.hive.ql.txn.AcidHouseKeeperService$TimedoutTxnReaper.run(AcidHouseKeeperService.java:87)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The problem here is that the method {{abortTxns(Connection dbConn, List
> txnids)}} in
> metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
> creates the following bad query when txnids list is empty.
> {code}
> delete from HIVE_LOCKS where hl_txnid in ();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11902) Abort txn cleanup thread throws SyntaxErrorException
[
https://issues.apache.org/jira/browse/HIVE-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903017#comment-14903017
]
Deepesh Khandelwal commented on HIVE-11902:
---
The failing hcat test
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation also
seems unrelated.
> Abort txn cleanup thread throws SyntaxErrorException
>
>
> Key: HIVE-11902
> URL: https://issues.apache.org/jira/browse/HIVE-11902
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Attachments: HIVE-11902.patch
>
>
> When cleaning left over transactions we see the DeadTxnReaper code threw the
> following exception:
> {noformat}
> 2015-09-21 05:23:38,148 WARN [DeadTxnReaper-0]: txn.TxnHandler
> (TxnHandler.java:performTimeOuts(1876)) - Aborting timedout transactions
> failed due to You have an error in your SQL syntax; check the manual that
> corresponds to your MySQL server version for the right syntax to use near ')'
> at line 1(SQLState=42000,ErrorCode=1064)
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error
> in your SQL syntax; check the manual that corresponds to your MySQL server
> version for the right syntax to use near ')' at line 1
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
> at com.mysql.jdbc.Util.getInstance(Util.java:360)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:978)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2526)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1618)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1549)
> at
> com.jolbox.bonecp.StatementHandle.executeUpdate(StatementHandle.java:497)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.abortTxns(TxnHandler.java:1275)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.performTimeOuts(TxnHandler.java:1866)
> at
> org.apache.hadoop.hive.ql.txn.AcidHouseKeeperService$TimedoutTxnReaper.run(AcidHouseKeeperService.java:87)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The problem here is that the method {{abortTxns(Connection dbConn, List
> txnids)}} in
> metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
> creates the following bad query when txnids list is empty.
> {code}
> delete from HIVE_LOCKS where hl_txnid in ();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11902) Abort txn cleanup thread throws SyntaxErrorException
[
https://issues.apache.org/jira/browse/HIVE-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-11902:
--
Attachment: (was: HIVE-11902.patch)
> Abort txn cleanup thread throws SyntaxErrorException
>
>
> Key: HIVE-11902
> URL: https://issues.apache.org/jira/browse/HIVE-11902
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Attachments: HIVE-11902.patch
>
>
> When cleaning left over transactions we see the DeadTxnReaper code threw the
> following exception:
> {noformat}
> 2015-09-21 05:23:38,148 WARN [DeadTxnReaper-0]: txn.TxnHandler
> (TxnHandler.java:performTimeOuts(1876)) - Aborting timedout transactions
> failed due to You have an error in your SQL syntax; check the manual that
> corresponds to your MySQL server version for the right syntax to use near ')'
> at line 1(SQLState=42000,ErrorCode=1064)
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error
> in your SQL syntax; check the manual that corresponds to your MySQL server
> version for the right syntax to use near ')' at line 1
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
> at com.mysql.jdbc.Util.getInstance(Util.java:360)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:978)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2526)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1618)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1549)
> at
> com.jolbox.bonecp.StatementHandle.executeUpdate(StatementHandle.java:497)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.abortTxns(TxnHandler.java:1275)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.performTimeOuts(TxnHandler.java:1866)
> at
> org.apache.hadoop.hive.ql.txn.AcidHouseKeeperService$TimedoutTxnReaper.run(AcidHouseKeeperService.java:87)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The problem here is that the method {{abortTxns(Connection dbConn, List
> txnids)}} in
> metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
> creates the following bad query when txnids list is empty.
> {code}
> delete from HIVE_LOCKS where hl_txnid in ();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11902) Abort txn cleanup thread throws SyntaxErrorException
[
https://issues.apache.org/jira/browse/HIVE-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-11902:
--
Attachment: HIVE-11902.patch
> Abort txn cleanup thread throws SyntaxErrorException
>
>
> Key: HIVE-11902
> URL: https://issues.apache.org/jira/browse/HIVE-11902
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Attachments: HIVE-11902.patch
>
>
> When cleaning left over transactions we see the DeadTxnReaper code threw the
> following exception:
> {noformat}
> 2015-09-21 05:23:38,148 WARN [DeadTxnReaper-0]: txn.TxnHandler
> (TxnHandler.java:performTimeOuts(1876)) - Aborting timedout transactions
> failed due to You have an error in your SQL syntax; check the manual that
> corresponds to your MySQL server version for the right syntax to use near ')'
> at line 1(SQLState=42000,ErrorCode=1064)
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error
> in your SQL syntax; check the manual that corresponds to your MySQL server
> version for the right syntax to use near ')' at line 1
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
> at com.mysql.jdbc.Util.getInstance(Util.java:360)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:978)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2526)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1618)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1549)
> at
> com.jolbox.bonecp.StatementHandle.executeUpdate(StatementHandle.java:497)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.abortTxns(TxnHandler.java:1275)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.performTimeOuts(TxnHandler.java:1866)
> at
> org.apache.hadoop.hive.ql.txn.AcidHouseKeeperService$TimedoutTxnReaper.run(AcidHouseKeeperService.java:87)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The problem here is that the method {{abortTxns(Connection dbConn, List
> txnids)}} in
> metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
> creates the following bad query when txnids list is empty.
> {code}
> delete from HIVE_LOCKS where hl_txnid in ();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11902) Abort txn cleanup thread throws SyntaxErrorException
[
https://issues.apache.org/jira/browse/HIVE-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-11902:
--
Attachment: HIVE-11902.patch
Attaching a patch for review. Also minor change in the logging message. cc
[~ekoifman]
> Abort txn cleanup thread throws SyntaxErrorException
>
>
> Key: HIVE-11902
> URL: https://issues.apache.org/jira/browse/HIVE-11902
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Attachments: HIVE-11902.patch
>
>
> When cleaning left over transactions we see the DeadTxnReaper code threw the
> following exception:
> {noformat}
> 2015-09-21 05:23:38,148 WARN [DeadTxnReaper-0]: txn.TxnHandler
> (TxnHandler.java:performTimeOuts(1876)) - Aborting timedout transactions
> failed due to You have an error in your SQL syntax; check the manual that
> corresponds to your MySQL server version for the right syntax to use near ')'
> at line 1(SQLState=42000,ErrorCode=1064)
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error
> in your SQL syntax; check the manual that corresponds to your MySQL server
> version for the right syntax to use near ')' at line 1
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
> at com.mysql.jdbc.Util.getInstance(Util.java:360)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:978)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2526)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1618)
> at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1549)
> at
> com.jolbox.bonecp.StatementHandle.executeUpdate(StatementHandle.java:497)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.abortTxns(TxnHandler.java:1275)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.performTimeOuts(TxnHandler.java:1866)
> at
> org.apache.hadoop.hive.ql.txn.AcidHouseKeeperService$TimedoutTxnReaper.run(AcidHouseKeeperService.java:87)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The problem here is that the method {{abortTxns(Connection dbConn, List
> txnids)}} in
> metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
> creates the following bad query when txnids list is empty.
> {code}
> delete from HIVE_LOCKS where hl_txnid in ();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HIVE-11628) DB type detection code is failing on Oracle 12
[
https://issues.apache.org/jira/browse/HIVE-11628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal resolved HIVE-11628.
---
Resolution: Won't Fix
DB type detection code is failing on Oracle 12
--
Key: HIVE-11628
URL: https://issues.apache.org/jira/browse/HIVE-11628
Project: Hive
Issue Type: Bug
Components: Metastore
Environment: Oracle 12
Reporter: Deepesh Khandelwal
Assignee: Deepesh Khandelwal
Fix For: 2.0.0
Attachments: HIVE-11628.patch
DB type detection code is failing when using Oracle 12 as backing store.
When determining qualification for direct SQL, in the logs following message
is seen:
{noformat}
2015-08-14 01:15:16,020 INFO [pool-6-thread-109]:
metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:init(131)) - Using
direct SQL, underlying DB is OTHER
{noformat}
Currently in org/apache/hadoop/hive/metastore/MetaStoreDirectSql, there is a
code snippet:
{code}
private DB determineDbType() {
DB dbType = DB.OTHER;
if (runDbCheck(SET @@session.sql_mode=ANSI_QUOTES, MySql)) {
dbType = DB.MYSQL;
} else if (runDbCheck(SELECT version from v$instance, Oracle)) {
dbType = DB.ORACLE;
} else if (runDbCheck(SELECT @@version, MSSQL)) {
dbType = DB.MSSQL;
} else {
// TODO: maybe we should use getProductName to identify all the DBs
String productName = getProductName();
if (productName != null productName.toLowerCase().contains(derby))
{
dbType = DB.DERBY;
}
}
return dbType;
}
{code}
The code relies on access to v$instance in order to identify the backend DB
as Oracle, but this can fail if users are not granted select privileges on v$
tables. An alternate way is specified on [Oracle Database Reference
pages|http://docs.oracle.com/cd/B19306_01/server.102/b14237/statviews_4224.htm]
works.
I will attach a potential patch that should work.
Without the patch the workaround here would be to grant select privileges on
v$ tables.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11628) DB type detection code is failing on Oracle 12
[
https://issues.apache.org/jira/browse/HIVE-11628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717133#comment-14717133
]
Deepesh Khandelwal commented on HIVE-11628:
---
I would like to abandon this one in favor of HIVE-11123.
DB type detection code is failing on Oracle 12
--
Key: HIVE-11628
URL: https://issues.apache.org/jira/browse/HIVE-11628
Project: Hive
Issue Type: Bug
Components: Metastore
Environment: Oracle 12
Reporter: Deepesh Khandelwal
Assignee: Deepesh Khandelwal
Fix For: 2.0.0
Attachments: HIVE-11628.patch
DB type detection code is failing when using Oracle 12 as backing store.
When determining qualification for direct SQL, in the logs following message
is seen:
{noformat}
2015-08-14 01:15:16,020 INFO [pool-6-thread-109]:
metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:init(131)) - Using
direct SQL, underlying DB is OTHER
{noformat}
Currently in org/apache/hadoop/hive/metastore/MetaStoreDirectSql, there is a
code snippet:
{code}
private DB determineDbType() {
DB dbType = DB.OTHER;
if (runDbCheck(SET @@session.sql_mode=ANSI_QUOTES, MySql)) {
dbType = DB.MYSQL;
} else if (runDbCheck(SELECT version from v$instance, Oracle)) {
dbType = DB.ORACLE;
} else if (runDbCheck(SELECT @@version, MSSQL)) {
dbType = DB.MSSQL;
} else {
// TODO: maybe we should use getProductName to identify all the DBs
String productName = getProductName();
if (productName != null productName.toLowerCase().contains(derby))
{
dbType = DB.DERBY;
}
}
return dbType;
}
{code}
The code relies on access to v$instance in order to identify the backend DB
as Oracle, but this can fail if users are not granted select privileges on v$
tables. An alternate way is specified on [Oracle Database Reference
pages|http://docs.oracle.com/cd/B19306_01/server.102/b14237/statviews_4224.htm]
works.
I will attach a potential patch that should work.
Without the patch the workaround here would be to grant select privileges on
v$ tables.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11123) Fix how to confirm the RDBMS product name at Metastore.
[
https://issues.apache.org/jira/browse/HIVE-11123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710109#comment-14710109
]
Deepesh Khandelwal commented on HIVE-11123:
---
[~sinchii] thanks for the patch! What version of Oracle did you use for testing
this? We saw some issues with existing code (without your patch) on Oracle 12
whereas it worked fine on Oracle 11. Just want to make sure that it works as
expected between different Oracle versions.
Fix how to confirm the RDBMS product name at Metastore.
---
Key: HIVE-11123
URL: https://issues.apache.org/jira/browse/HIVE-11123
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 1.2.0
Environment: PostgreSQL
Reporter: Shinichi Yamashita
Assignee: Shinichi Yamashita
Attachments: HIVE-11123.1.patch, HIVE-11123.2.patch,
HIVE-11123.3.patch, HIVE-11123.4.patch
I use PostgreSQL to Hive Metastore. And I saw the following message at
PostgreSQL log.
{code}
2015-06-26 10:58:15.488 JST ERROR: syntax error at or near @@ at
character 5
2015-06-26 10:58:15.488 JST STATEMENT: SET @@session.sql_mode=ANSI_QUOTES
2015-06-26 10:58:15.489 JST ERROR: relation v$instance does not exist
at character 21
2015-06-26 10:58:15.489 JST STATEMENT: SELECT version FROM v$instance
2015-06-26 10:58:15.490 JST ERROR: column version does not exist at
character 10
2015-06-26 10:58:15.490 JST STATEMENT: SELECT @@version
{code}
When Hive CLI and Beeline embedded mode are carried out, this message is
output to PostgreSQL log.
These queries are called from MetaStoreDirectSql#determineDbType. And if we
use MetaStoreDirectSql#getProductName, we need not to call these queries.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11628) DB type detection code is failing on Oracle 12
[
https://issues.apache.org/jira/browse/HIVE-11628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710111#comment-14710111
]
Deepesh Khandelwal commented on HIVE-11628:
---
Yes, I agree, HIVE-11123 is a more robust fix. If that works fine between
Oracle versions (11, 12), then we don't need this. I have posted a question for
[~sinchii] about his test environment.
DB type detection code is failing on Oracle 12
--
Key: HIVE-11628
URL: https://issues.apache.org/jira/browse/HIVE-11628
Project: Hive
Issue Type: Bug
Components: Metastore
Environment: Oracle 12
Reporter: Deepesh Khandelwal
Assignee: Deepesh Khandelwal
Fix For: 2.0.0
Attachments: HIVE-11628.patch
DB type detection code is failing when using Oracle 12 as backing store.
When determining qualification for direct SQL, in the logs following message
is seen:
{noformat}
2015-08-14 01:15:16,020 INFO [pool-6-thread-109]:
metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:init(131)) - Using
direct SQL, underlying DB is OTHER
{noformat}
Currently in org/apache/hadoop/hive/metastore/MetaStoreDirectSql, there is a
code snippet:
{code}
private DB determineDbType() {
DB dbType = DB.OTHER;
if (runDbCheck(SET @@session.sql_mode=ANSI_QUOTES, MySql)) {
dbType = DB.MYSQL;
} else if (runDbCheck(SELECT version from v$instance, Oracle)) {
dbType = DB.ORACLE;
} else if (runDbCheck(SELECT @@version, MSSQL)) {
dbType = DB.MSSQL;
} else {
// TODO: maybe we should use getProductName to identify all the DBs
String productName = getProductName();
if (productName != null productName.toLowerCase().contains(derby))
{
dbType = DB.DERBY;
}
}
return dbType;
}
{code}
The code relies on access to v$instance in order to identify the backend DB
as Oracle, but this can fail if users are not granted select privileges on v$
tables. An alternate way is specified on [Oracle Database Reference
pages|http://docs.oracle.com/cd/B19306_01/server.102/b14237/statviews_4224.htm]
works.
I will attach a potential patch that should work.
Without the patch the workaround here would be to grant select privileges on
v$ tables.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11628) DB type detection code is failing on Oracle 12
[
https://issues.apache.org/jira/browse/HIVE-11628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-11628:
--
Attachment: HIVE-11628.patch
Attaching the patch for review.
DB type detection code is failing on Oracle 12
--
Key: HIVE-11628
URL: https://issues.apache.org/jira/browse/HIVE-11628
Project: Hive
Issue Type: Bug
Components: Metastore
Environment: Oracle 12
Reporter: Deepesh Khandelwal
Assignee: Deepesh Khandelwal
Attachments: HIVE-11628.patch
DB type detection code is failing when using Oracle 12 as backing store.
When determining qualification for direct SQL, in the logs following message
is seen:
{noformat}
2015-08-14 01:15:16,020 INFO [pool-6-thread-109]:
metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:init(131)) - Using
direct SQL, underlying DB is OTHER
{noformat}
Currently in org/apache/hadoop/hive/metastore/MetaStoreDirectSql, there is a
code snippet:
{code}
private DB determineDbType() {
DB dbType = DB.OTHER;
if (runDbCheck(SET @@session.sql_mode=ANSI_QUOTES, MySql)) {
dbType = DB.MYSQL;
} else if (runDbCheck(SELECT version from v$instance, Oracle)) {
dbType = DB.ORACLE;
} else if (runDbCheck(SELECT @@version, MSSQL)) {
dbType = DB.MSSQL;
} else {
// TODO: maybe we should use getProductName to identify all the DBs
String productName = getProductName();
if (productName != null productName.toLowerCase().contains(derby))
{
dbType = DB.DERBY;
}
}
return dbType;
}
{code}
The code relies on access to v$instance in order to identify the backend DB
as Oracle, but this can fail if users are not granted select privileges on v$
tables. An alternate way is specified on [Oracle Database Reference
pages|http://docs.oracle.com/cd/B19306_01/server.102/b14237/statviews_4224.htm]
works.
I will attach a potential patch that should work.
Without the patch the workaround here would be to grant select privileges on
v$ tables.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HIVE-10863) Merge trunk to Spark branch 5/28/2015 [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepesh Khandelwal reassigned HIVE-10863: - Assignee: Deepesh Khandelwal (was: Xuefu Zhang) Merge trunk to Spark branch 5/28/2015 [Spark Branch] Key: HIVE-10863 URL: https://issues.apache.org/jira/browse/HIVE-10863 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Xuefu Zhang Assignee: Deepesh Khandelwal Attachments: HIVE-10863.0-spark.patch, mj.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10724) WebHCat e2e test TestStreaming_5 fails on Windows
[
https://issues.apache.org/jira/browse/HIVE-10724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561375#comment-14561375
]
Deepesh Khandelwal commented on HIVE-10724:
---
Thanks [~ekoifman] for the review and commit!
WebHCat e2e test TestStreaming_5 fails on Windows
-
Key: HIVE-10724
URL: https://issues.apache.org/jira/browse/HIVE-10724
Project: Hive
Issue Type: Bug
Components: Tests
Affects Versions: 1.2.0
Reporter: Deepesh Khandelwal
Assignee: Deepesh Khandelwal
Fix For: 1.3.0
Attachments: HIVE-10724.patch
The test TestStreaming_5 fails with the following error on Windows:
{noformat}
Passed in parameter is incorrectly quoted:
\\StreamXmlRecordReader,begin=xml,end=/xml\\
{noformat}
The problem is the extra quotes in the post_options in the test
{{'inputreader=StreamXmlRecordReader,begin=xml,end=/xml'}}
Removing the double quotes
{{'inputreader=StreamXmlRecordReader,begin=xml,end=/xml'}} makes the test
happy on both Linux and Windows.
NO PRECOMMIT TESTS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10724) WebHCat e2e test TestStreaming_5 fails on Windows
[
https://issues.apache.org/jira/browse/HIVE-10724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-10724:
--
Description:
The test TestStreaming_5 fails with the following error on Windows:
{noformat}
Passed in parameter is incorrectly quoted:
\\StreamXmlRecordReader,begin=xml,end=/xml\\
{noformat}
The problem is the extra quotes in the post_options in the test
{{'inputreader=StreamXmlRecordReader,begin=xml,end=/xml'}}
Removing the double quotes
{{'inputreader=StreamXmlRecordReader,begin=xml,end=/xml'}} makes the test happy
on both Linux and Windows.
NO PRECOMMIT TESTS
was:
The test TestStreaming_5 fails with the following error on Windows:
{noformat}
Passed in parameter is incorrectly quoted:
\\StreamXmlRecordReader,begin=xml,end=/xml\\
{noformat}
The problem is the extra quotes in the post_options in the test
{{'inputreader=StreamXmlRecordReader,begin=xml,end=/xml'}}
Removing the double quotes
{{'inputreader=StreamXmlRecordReader,begin=xml,end=/xml'}} makes the test happy
on both Linux and Windows.
WebHCat e2e test TestStreaming_5 fails on Windows
-
Key: HIVE-10724
URL: https://issues.apache.org/jira/browse/HIVE-10724
Project: Hive
Issue Type: Bug
Components: Tests
Affects Versions: 1.2.0
Reporter: Deepesh Khandelwal
Assignee: Deepesh Khandelwal
Attachments: HIVE-10724.patch
The test TestStreaming_5 fails with the following error on Windows:
{noformat}
Passed in parameter is incorrectly quoted:
\\StreamXmlRecordReader,begin=xml,end=/xml\\
{noformat}
The problem is the extra quotes in the post_options in the test
{{'inputreader=StreamXmlRecordReader,begin=xml,end=/xml'}}
Removing the double quotes
{{'inputreader=StreamXmlRecordReader,begin=xml,end=/xml'}} makes the test
happy on both Linux and Windows.
NO PRECOMMIT TESTS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10724) WebHCat e2e test TestStreaming_5 fails on Windows
[
https://issues.apache.org/jira/browse/HIVE-10724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-10724:
--
Attachment: HIVE-10724.patch
Attaching a patch with the described fix.
WebHCat e2e test TestStreaming_5 fails on Windows
-
Key: HIVE-10724
URL: https://issues.apache.org/jira/browse/HIVE-10724
Project: Hive
Issue Type: Bug
Components: Tests
Affects Versions: 1.2.0
Reporter: Deepesh Khandelwal
Assignee: Deepesh Khandelwal
Attachments: HIVE-10724.patch
The test TestStreaming_5 fails with the following error on Windows:
{noformat}
Passed in parameter is incorrectly quoted:
\\StreamXmlRecordReader,begin=xml,end=/xml\\
{noformat}
The problem is the extra quotes in the post_options in the test
{{'inputreader=StreamXmlRecordReader,begin=xml,end=/xml'}}
Removing the double quotes
{{'inputreader=StreamXmlRecordReader,begin=xml,end=/xml'}} makes the test
happy on both Linux and Windows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10074) Ability to run HCat Client Unit tests in a system test setting
[
https://issues.apache.org/jira/browse/HIVE-10074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384666#comment-14384666
]
Deepesh Khandelwal commented on HIVE-10074:
---
[~sushanth] the error still seems unrelated. Can you suggest what the next
steps here should be?
Ability to run HCat Client Unit tests in a system test setting
--
Key: HIVE-10074
URL: https://issues.apache.org/jira/browse/HIVE-10074
Project: Hive
Issue Type: Bug
Components: Tests
Reporter: Deepesh Khandelwal
Assignee: Deepesh Khandelwal
Attachments: HIVE-10074.1.patch, HIVE-10074.patch
Following testsuite
{{hcatalog/webhcat/java-client/src/test/java/org/apache/hive/hcatalog/api/TestHCatClient.java}}
is a JUnit testsuite to test some basic HCat client API. During setup it
brings up a Hive Metastore with embedded Derby. The testsuite however will be
even more useful if it can be run against a running Hive Metastore
(transparent to whatever backing DB its running against).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
