[jira] [Commented] (HIVE-22433) Hive JDBC Storage Handler: Incorrect results fetched from BOOLEAN and TIMESTAMP DataType From JDBC Data Source
[ https://issues.apache.org/jira/browse/HIVE-22433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965263#comment-16965263 ] Hive QA commented on HIVE-22433: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12984671/HIVE-22433.03.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 17546 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/19261/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19261/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19261/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12984671 - PreCommit-HIVE-Build > Hive JDBC Storage Handler: Incorrect results fetched from BOOLEAN and > TIMESTAMP DataType From JDBC Data Source > -- > > Key: HIVE-22433 > URL: https://issues.apache.org/jira/browse/HIVE-22433 > Project: Hive > Issue Type: Bug > Components: StorageHandler >Reporter: Syed Shameerur Rahman >Assignee: Syed Shameerur Rahman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22433.01.patch, HIVE-22433.02.patch, > HIVE-22433.03.patch > > > Steps to Reproduce: > {code:java} > //Derby table: > create table testtbl(a BOOLEAN, b TIMESTAMP); > // Insert to table via mysql connector > // data in db > true 2019-11-11 12:00:00 > //Hive table: > CREATE EXTERNAL TABLE `hive_table`( > a BOOLEAN, b TIMESTAMP > ) > STORED BY > 'org.apache.hive.storage.jdbc.JdbcStorageHandler' > TBLPROPERTIES ( > > 'hive.sql.database.type'='DERBY', > 'hive.sql.dbcp.password'='', > 'hive.sql.dbcp.username'='', > 'hive.sql.jdbc.driver'='', > 'hive.sql.jdbc.url'='', > 'hive.sql.table'='testtbl'); > //Hive query: > select * from hive_table; > // result from select query > false 2019-11-11 20:00:00 > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22433) Hive JDBC Storage Handler: Incorrect results fetched from BOOLEAN and TIMESTAMP DataType From JDBC Data Source
[ https://issues.apache.org/jira/browse/HIVE-22433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964917#comment-16964917 ] Syed Shameerur Rahman edited comment on HIVE-22433 at 11/2/19 7:22 AM: --- Updated PR and HIVE-22433.02.patch to fix the failing qtest and changed the logic to deserialize timestamp data type. was (Author: srahman): Updated PR and HIVE-22433.02.patch to fix the failing qtest and changed the logic to deserialize timestamp data type. Note: The input timetstamp in external_jdbc_table_typeconversion.q, external_jdbc_table_partition.q is different from the output timestamp recorded in external_jdbc_table_typeconversion.q.out, external_jdbc_table_partition.q.out, Hence changing the output qtest to match with the input > Hive JDBC Storage Handler: Incorrect results fetched from BOOLEAN and > TIMESTAMP DataType From JDBC Data Source > -- > > Key: HIVE-22433 > URL: https://issues.apache.org/jira/browse/HIVE-22433 > Project: Hive > Issue Type: Bug > Components: StorageHandler >Reporter: Syed Shameerur Rahman >Assignee: Syed Shameerur Rahman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22433.01.patch, HIVE-22433.02.patch, > HIVE-22433.03.patch > > > Steps to Reproduce: > {code:java} > //Derby table: > create table testtbl(a BOOLEAN, b TIMESTAMP); > // Insert to table via mysql connector > // data in db > true 2019-11-11 12:00:00 > //Hive table: > CREATE EXTERNAL TABLE `hive_table`( > a BOOLEAN, b TIMESTAMP > ) > STORED BY > 'org.apache.hive.storage.jdbc.JdbcStorageHandler' > TBLPROPERTIES ( > > 'hive.sql.database.type'='DERBY', > 'hive.sql.dbcp.password'='', > 'hive.sql.dbcp.username'='', > 'hive.sql.jdbc.driver'='', > 'hive.sql.jdbc.url'='', > 'hive.sql.table'='testtbl'); > //Hive query: > select * from hive_table; > // result from select query > false 2019-11-11 20:00:00 > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22433) Hive JDBC Storage Handler: Incorrect results fetched from BOOLEAN and TIMESTAMP DataType From JDBC Data Source
[ https://issues.apache.org/jira/browse/HIVE-22433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965264#comment-16965264 ] Syed Shameerur Rahman commented on HIVE-22433: -- [~jcamachorodriguez] Can you please review? *Note*: The input timetstamp in external_jdbc_table_typeconversion.q and external_jdbc_table_partition.q is different from the output timestamp recorded in external_jdbc_table_typeconversion.q.out and external_jdbc_table_partition.q.out, Hence changing the output qtest to match with the input > Hive JDBC Storage Handler: Incorrect results fetched from BOOLEAN and > TIMESTAMP DataType From JDBC Data Source > -- > > Key: HIVE-22433 > URL: https://issues.apache.org/jira/browse/HIVE-22433 > Project: Hive > Issue Type: Bug > Components: StorageHandler >Reporter: Syed Shameerur Rahman >Assignee: Syed Shameerur Rahman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22433.01.patch, HIVE-22433.02.patch, > HIVE-22433.03.patch > > > Steps to Reproduce: > {code:java} > //Derby table: > create table testtbl(a BOOLEAN, b TIMESTAMP); > // Insert to table via mysql connector > // data in db > true 2019-11-11 12:00:00 > //Hive table: > CREATE EXTERNAL TABLE `hive_table`( > a BOOLEAN, b TIMESTAMP > ) > STORED BY > 'org.apache.hive.storage.jdbc.JdbcStorageHandler' > TBLPROPERTIES ( > > 'hive.sql.database.type'='DERBY', > 'hive.sql.dbcp.password'='', > 'hive.sql.dbcp.username'='', > 'hive.sql.jdbc.driver'='', > 'hive.sql.jdbc.url'='', > 'hive.sql.table'='testtbl'); > //Hive query: > select * from hive_table; > // result from select query > false 2019-11-11 20:00:00 > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22448) CBO: Expand the multiple count distinct with a group-by key
[ https://issues.apache.org/jira/browse/HIVE-22448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965268#comment-16965268 ] Hive QA commented on HIVE-22448: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m 24s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 11s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 40s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 12s{color} | {color:blue} ql in master has 1545 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 8s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 8s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 40s{color} | {color:red} ql: The patch generated 5 new + 4 unchanged - 1 fixed = 9 total (was 5) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 22s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 16s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 25m 56s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19262/dev-support/hive-personality.sh | | git revision | master / 1c2b7fa | | Default Java | 1.8.0_111 | | findbugs | v3.0.1 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-19262/yetus/diff-checkstyle-ql.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-19262/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > CBO: Expand the multiple count distinct with a group-by key > --- > > Key: HIVE-22448 > URL: https://issues.apache.org/jira/browse/HIVE-22448 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Gopal Vijayaraghavan >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-22448.patch > > > {code} > create temporary table mytable1 (x integer, y integer, z integer, a integer); > explain cbo > select z, x, count(distinct y), count(distinct a) > from mytable1 > group by z, x; > explain cbo > select count(distinct y), count(distinct a) > from mytable1 > {code} > The first is not vectorized, the second one is because of the grouping-set > based rewrite for count distinct. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22448) CBO: Expand the multiple count distinct with a group-by key
[ https://issues.apache.org/jira/browse/HIVE-22448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965280#comment-16965280 ] Hive QA commented on HIVE-22448: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12984672/HIVE-22448.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 17547 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[multigroupbydistinct] (batchId=11) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[reduce_deduplicate_distinct] (batchId=160) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[limit_pushdown3] (batchId=169) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[limit_pushdown] (batchId=181) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[offset_limit_ppd_optimizer] (batchId=180) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join18_multi_distinct] (batchId=125) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join18_multi_distinct] (batchId=126) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[limit_pushdown] (batchId=145) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/19262/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19262/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19262/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 8 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12984672 - PreCommit-HIVE-Build > CBO: Expand the multiple count distinct with a group-by key > --- > > Key: HIVE-22448 > URL: https://issues.apache.org/jira/browse/HIVE-22448 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Gopal Vijayaraghavan >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-22448.patch > > > {code} > create temporary table mytable1 (x integer, y integer, z integer, a integer); > explain cbo > select z, x, count(distinct y), count(distinct a) > from mytable1 > group by z, x; > explain cbo > select count(distinct y), count(distinct a) > from mytable1 > {code} > The first is not vectorized, the second one is because of the grouping-set > based rewrite for count distinct. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[ https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-21954: Attachment: HIVE-21954.08.patch > QTest: support for running qtests on various metastore DBs > -- > > Key: HIVE-21954 > URL: https://issues.apache.org/jira/browse/HIVE-21954 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore, Testing Infrastructure >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch, > HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch, > HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch, > HIVE-21954.07.patch, HIVE-21954.08.patch > > > In HIVE-21940, a postgres metastore related issue has been fixed, and a local > reproduction has been provided. > {code} > export QTEST_LEAVE_FILES=true > docker kill metastore-test-postgres-install > docker rm metastore-test-postgres-install > cd standalone-metastore > mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch > -Dmetastore.itest.no.stop.container=true > cd .. > mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest > -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q > -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore > {code} > The problem with this solution is that data/conf/hive-site.xml has to be > edited manually. My proposal is to introduce a property > (-Dmetastore.db=postgres), which can take care of the parameters on the fly. > 2 supported solutions could be: > 1. simple parameters: -Dmetastore.db=postgres > In this case, tests depend on settings from ITestPostgres class (password, > db, etc.) > 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro > patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch] > > In the first implementation, I would not start metastore db automatically > (which is done be 'mvn verify ...'), but it's still under planning. > In the long term, we should consider running this kind of tests in precommit > phase, so maybe -Dmetastore.db=postgres could start metastore db > automatically. Also we should consider running some qtests on various > metastores. I would not pick randomly, but choose some "metastore-heavy" ones > instead. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22427) PersistenceManagerProvider Logs a Warning About datanucleus.autoStartMechanismMode
[ https://issues.apache.org/jira/browse/HIVE-22427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965293#comment-16965293 ] László Bodor commented on HIVE-22427: - so with the patch it WARNs only if a user sets this property to an invalid value (actually, to any non-null value) according to the original state, the property should not have any non-null value, so this make sense to me +1 > PersistenceManagerProvider Logs a Warning About > datanucleus.autoStartMechanismMode > -- > > Key: HIVE-22427 > URL: https://issues.apache.org/jira/browse/HIVE-22427 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Affects Versions: 3.2.0 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22427.1.patch > > > {code:none} > WARN [pool-6-thread-2] metastore.PersistenceManagerProvider: > datanucleus.autoStartMechanismMode is set to unsupported value null . Setting > it to value: ignored > {code} > This does not need to be a WARN level logging for this scenario. Perhaps if > user configures the value to some non-null value, then emit a warning, > otherwise, simply emit an INFO level stating that the configuration is not > set and that a reasonable default value will be used. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22444) Clean up Project POM Files
[ https://issues.apache.org/jira/browse/HIVE-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965299#comment-16965299 ] Hive QA commented on HIVE-22444: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m 0s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 55s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 23s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 58s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 27s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 18s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 14s{color} | {color:red} hive-blobstore in the patch failed. {color} | | {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 21s{color} | {color:red} hive-minikdc in the patch failed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 11s{color} | {color:green} storage-api generated 0 new + 0 unchanged - 2 fixed = 0 total (was 2) {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 20s{color} | {color:green} serde in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 14s{color} | {color:green} upgrade-acid in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 13s{color} | {color:green} custom-serde in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 17s{color} | {color:green} custom-udfs in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 13s{color} | {color:green} udf-classloader-udf1 in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 13s{color} | {color:green} udf-classloader-udf2 in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 13s{color} | {color:green} udf-vectorized-badexample in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 23s{color} | {color:green} hcatalog-unit in the patch passed. {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 14s{color} | {color:red} hive-blobstore in the patch failed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 21s{color} | {color:green} hive-jmh in the patch passed. {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 21s{color} | {color:red} hive-minikdc in the patch failed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s{color} | {color:green} hive-unit in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 23s{color} | {color:green} hive-unit-hadoop2 in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 19s{color} | {color:green} qtest-accumulo in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 14s{color} | {color:green} qtest-druid in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s{color} | {color:green} qtest-kudu in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 19s{color} | {color:green} qtest-spark in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 13s{color} | {color:green} test-serde in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 31s{color} | {color:green} util in the patch passed. {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 20s{color} | {color:green} The patch has no
[jira] [Commented] (HIVE-22444) Clean up Project POM Files
[ https://issues.apache.org/jira/browse/HIVE-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965306#comment-16965306 ] Hive QA commented on HIVE-22444: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12984676/HIVE-22444.1.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17541 tests executed *Failed tests:* {noformat} TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=299) TestMiniHiveKafkaCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=299) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/19263/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19263/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19263/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12984676 - PreCommit-HIVE-Build > Clean up Project POM Files > -- > > Key: HIVE-22444 > URL: https://issues.apache.org/jira/browse/HIVE-22444 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22444.1.patch, HIVE-22444.1.patch > > > # Address warnings in the build process > # Use DependencyManagement in Root POM for ITest (see HIVE-22426) > # General POM cleanup -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22274) Upgrade Calcite version to 1.21.0
[ https://issues.apache.org/jira/browse/HIVE-22274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965320#comment-16965320 ] Hive QA commented on HIVE-22274: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12984673/HIVE-22274.19.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17535 tests executed *Failed tests:* {noformat} TestJdbcWithMiniLlapArrow - did not produce a TEST-*.xml file (likely timed out) (batchId=284) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/19264/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19264/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19264/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12984673 - PreCommit-HIVE-Build > Upgrade Calcite version to 1.21.0 > - > > Key: HIVE-22274 > URL: https://issues.apache.org/jira/browse/HIVE-22274 > Project: Hive > Issue Type: Task >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22274.1.patch, HIVE-22274.10.patch, > HIVE-22274.10.patch, HIVE-22274.11.patch, HIVE-22274.12.patch, > HIVE-22274.13.patch, HIVE-22274.15.patch, HIVE-22274.16.patch, > HIVE-22274.17.patch, HIVE-22274.18.patch, HIVE-22274.19.patch, > HIVE-22274.19.patch, HIVE-22274.2.patch, HIVE-22274.3.patch, > HIVE-22274.4.patch, HIVE-22274.5.patch, HIVE-22274.6.patch, > HIVE-22274.7.patch, HIVE-22274.8.patch, HIVE-22274.9.patch, HIVE-22274.patch > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22274) Upgrade Calcite version to 1.21.0
[ https://issues.apache.org/jira/browse/HIVE-22274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965323#comment-16965323 ] Hive QA commented on HIVE-22274: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 45s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 18s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m 3s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 45s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 16s{color} | {color:blue} ql in master has 1545 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 32s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 35s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 9m 16s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 26s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m 39s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 39s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m 39s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 47s{color} | {color:red} ql: The patch generated 37 new + 329 unchanged - 12 fixed = 366 total (was 341) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 12s{color} | {color:red} root: The patch generated 38 new + 381 unchanged - 12 fixed = 419 total (was 393) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 20s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 52 unchanged - 0 fixed = 53 total (was 52) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m 28s{color} | {color:red} ql generated 8 new + 1542 unchanged - 3 fixed = 1550 total (was 1545) {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m 2s{color} | {color:red} ql generated 4 new + 96 unchanged - 4 fixed = 100 total (was 100) {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 7m 50s{color} | {color:red} root generated 4 new + 333 unchanged - 4 fixed = 337 total (was 337) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 28s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 82m 29s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | Dead store to joinInfo in org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveJoinFactoryImpl.createJoin(RelNode, RelNode, RexNode, Set, JoinRelType, boolean) At HiveRelFactories.java:org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveJoinFactoryImpl.createJoin(RelNode, RelNode, RexNode, Set, JoinRelType, boolean) At HiveRelFactories.java:[line 170] | | | Dead store to joinInfo in org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveJoinFactoryImpl.createJoin(RelNode, RelNode, RexNode, JoinRelType, Set, boolean) At HiveRelFactories.java:org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveJo
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[ https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965350#comment-16965350 ] Hive QA commented on HIVE-21954: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12984680/HIVE-21954.08.patch {color:green}SUCCESS:{color} +1 due to 12 test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 17547 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/19265/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19265/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19265/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12984680 - PreCommit-HIVE-Build > QTest: support for running qtests on various metastore DBs > -- > > Key: HIVE-21954 > URL: https://issues.apache.org/jira/browse/HIVE-21954 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore, Testing Infrastructure >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch, > HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch, > HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch, > HIVE-21954.07.patch, HIVE-21954.08.patch > > > In HIVE-21940, a postgres metastore related issue has been fixed, and a local > reproduction has been provided. > {code} > export QTEST_LEAVE_FILES=true > docker kill metastore-test-postgres-install > docker rm metastore-test-postgres-install > cd standalone-metastore > mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch > -Dmetastore.itest.no.stop.container=true > cd .. > mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest > -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q > -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore > {code} > The problem with this solution is that data/conf/hive-site.xml has to be > edited manually. My proposal is to introduce a property > (-Dmetastore.db=postgres), which can take care of the parameters on the fly. > 2 supported solutions could be: > 1. simple parameters: -Dmetastore.db=postgres > In this case, tests depend on settings from ITestPostgres class (password, > db, etc.) > 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro > patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch] > > In the first implementation, I would not start metastore db automatically > (which is done be 'mvn verify ...'), but it's still under planning. > In the long term, we should consider running this kind of tests in precommit > phase, so maybe -Dmetastore.db=postgres could start metastore db > automatically. Also we should consider running some qtests on various > metastores. I would not pick randomly, but choose some "metastore-heavy" ones > instead. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[ https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965362#comment-16965362 ] Hive QA commented on HIVE-21954: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 48s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 18s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m 56s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m 38s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m 17s{color} | {color:blue} standalone-metastore/metastore-server in master has 171 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 11s{color} | {color:blue} ql in master has 1545 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 46s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 50s{color} | {color:blue} itests/util in master has 53 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 11m 17s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 27s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m 51s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 25s{color} | {color:red} standalone-metastore: The patch generated 1 new + 104 unchanged - 6 fixed = 105 total (was 110) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 18s{color} | {color:red} standalone-metastore/metastore-server: The patch generated 1 new + 104 unchanged - 6 fixed = 105 total (was 110) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 24s{color} | {color:red} root: The patch generated 1 new + 801 unchanged - 6 fixed = 802 total (was 807) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 4s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 11m 18s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 95m 34s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense xml javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19265/dev-support/hive-personality.sh | | git revision | master / 1c2b7fa | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-19265/yetus/diff-checkstyle-standalone-metastore.txt | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-19265/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-19265/yetus/diff-checkstyle-root.txt | | modules | C: standalone-metastore standalone-metastore/metastore-server ql . itests/hive-unit
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[ https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965365#comment-16965365 ] László Bodor commented on HIVE-21954: - [~kgyrtkirk], [~pvary]: could you please take a look? the patch contains the following: 1. Database rule refactoring in standalone-metastore project 2. Qtest infrastructure: preparing qtest infra for being able to run any qtest against any kind of metastores 3. A bunch of cleanups in the current qtest mess With the patch, you can run the following (if your docker is set up properly, meaning that you can run the metastore integration tests, already described in standalone-metastore/DEV-README): {code} mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q -Dtest.metastore.db=mssql mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q -Dtest.metastore.db=mysql mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q -Dtest.metastore.db=postgres mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q -Dtest.metastore.db=oracle -Ditest.jdbc.jars=/path/to/your/god/damn/oracle/jdbc/driver/ojdbc6.jar {code} > QTest: support for running qtests on various metastore DBs > -- > > Key: HIVE-21954 > URL: https://issues.apache.org/jira/browse/HIVE-21954 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore, Testing Infrastructure >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch, > HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch, > HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch, > HIVE-21954.07.patch, HIVE-21954.08.patch > > > In HIVE-21940, a postgres metastore related issue has been fixed, and a local > reproduction has been provided. > {code} > export QTEST_LEAVE_FILES=true > docker kill metastore-test-postgres-install > docker rm metastore-test-postgres-install > cd standalone-metastore > mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch > -Dmetastore.itest.no.stop.container=true > cd .. > mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest > -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q > -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore > {code} > The problem with this solution is that data/conf/hive-site.xml has to be > edited manually. My proposal is to introduce a property > (-Dmetastore.db=postgres), which can take care of the parameters on the fly. > 2 supported solutions could be: > 1. simple parameters: -Dmetastore.db=postgres > In this case, tests depend on settings from ITestPostgres class (password, > db, etc.) > 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro > patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch] > > In the first implementation, I would not start metastore db automatically > (which is done be 'mvn verify ...'), but it's still under planning. > In the long term, we should consider running this kind of tests in precommit > phase, so maybe -Dmetastore.db=postgres could start metastore db > automatically. Also we should consider running some qtests on various > metastores. I would not pick randomly, but choose some "metastore-heavy" ones > instead. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[ https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965365#comment-16965365 ] László Bodor edited comment on HIVE-21954 at 11/2/19 1:34 PM: -- [~kgyrtkirk], [~pvary]: could you please take a look? the patch contains the following: 1. Database rule refactoring in standalone-metastore project 2. Qtest infrastructure: preparing qtest infra for being able to run any qtest against any kind of metastores 3. A bunch of cleanups in the current qtest mess With the patch, you can run the following (if your docker is set up properly, meaning that you can run the metastore integration tests, already described in standalone-metastore/DEV-README): {code} mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q -Dtest.metastore.db=mssql mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q -Dtest.metastore.db=mysql mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q -Dtest.metastore.db=postgres mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q -Dtest.metastore.db=oracle -Ditest.jdbc.jars=/path/to/your/god/damn/oracle/jdbc/driver/ojdbc6.jar {code} looking forward for comments: https://reviews.apache.org/r/71711/ was (Author: abstractdog): [~kgyrtkirk], [~pvary]: could you please take a look? the patch contains the following: 1. Database rule refactoring in standalone-metastore project 2. Qtest infrastructure: preparing qtest infra for being able to run any qtest against any kind of metastores 3. A bunch of cleanups in the current qtest mess With the patch, you can run the following (if your docker is set up properly, meaning that you can run the metastore integration tests, already described in standalone-metastore/DEV-README): {code} mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q -Dtest.metastore.db=mssql mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q -Dtest.metastore.db=mysql mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q -Dtest.metastore.db=postgres mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q -Dtest.metastore.db=oracle -Ditest.jdbc.jars=/path/to/your/god/damn/oracle/jdbc/driver/ojdbc6.jar {code} > QTest: support for running qtests on various metastore DBs > -- > > Key: HIVE-21954 > URL: https://issues.apache.org/jira/browse/HIVE-21954 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore, Testing Infrastructure >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch, > HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch, > HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch, > HIVE-21954.07.patch, HIVE-21954.08.patch > > > In HIVE-21940, a postgres metastore related issue has been fixed, and a local > reproduction has been provided. > {code} > export QTEST_LEAVE_FILES=true > docker kill metastore-test-postgres-install > docker rm metastore-test-postgres-install > cd standalone-metastore > mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch > -Dmetastore.itest.no.stop.container=true > cd .. > mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest > -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q > -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore > {code} > The problem with this solution is that data/conf/hive-site.xml has to be > edited manually. My proposal is to introduce a property > (-Dmetastore.db=postgres), which can take care of the parameters on the fly. > 2 supported solutions could be: > 1. simple parameters: -Dmetastore.db=postgres > In this case, tests depend on settings from ITestPostgres class (password, > db, etc.) > 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro > patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch] > > In the first implementation, I would not start metastore db automatically > (which is done be 'mvn verify ...'), but it's still under planning. > In the long term, we should consider running this kind of tests in precommit > phase, so maybe -Dmetastore.db=postgres could s
[jira] [Updated] (HIVE-22448) CBO: Expand the multiple count distinct with a group-by key
[ https://issues.apache.org/jira/browse/HIVE-22448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22448: --- Attachment: HIVE-22448.01.patch > CBO: Expand the multiple count distinct with a group-by key > --- > > Key: HIVE-22448 > URL: https://issues.apache.org/jira/browse/HIVE-22448 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Gopal Vijayaraghavan >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-22448.01.patch, HIVE-22448.patch > > > {code} > create temporary table mytable1 (x integer, y integer, z integer, a integer); > explain cbo > select z, x, count(distinct y), count(distinct a) > from mytable1 > group by z, x; > explain cbo > select count(distinct y), count(distinct a) > from mytable1 > {code} > The first is not vectorized, the second one is because of the grouping-set > based rewrite for count distinct. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22448) CBO: Expand the multiple count distinct with a group-by key
[ https://issues.apache.org/jira/browse/HIVE-22448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22448: -- Labels: pull-request-available (was: ) > CBO: Expand the multiple count distinct with a group-by key > --- > > Key: HIVE-22448 > URL: https://issues.apache.org/jira/browse/HIVE-22448 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Gopal Vijayaraghavan >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22448.01.patch, HIVE-22448.patch > > > {code} > create temporary table mytable1 (x integer, y integer, z integer, a integer); > explain cbo > select z, x, count(distinct y), count(distinct a) > from mytable1 > group by z, x; > explain cbo > select count(distinct y), count(distinct a) > from mytable1 > {code} > The first is not vectorized, the second one is because of the grouping-set > based rewrite for count distinct. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22448) CBO: Expand the multiple count distinct with a group-by key
[ https://issues.apache.org/jira/browse/HIVE-22448?focusedWorklogId=337738&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-337738 ] ASF GitHub Bot logged work on HIVE-22448: - Author: ASF GitHub Bot Created on: 02/Nov/19 18:31 Start Date: 02/Nov/19 18:31 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #838: HIVE-22448 URL: https://github.com/apache/hive/pull/838 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 337738) Remaining Estimate: 0h Time Spent: 10m > CBO: Expand the multiple count distinct with a group-by key > --- > > Key: HIVE-22448 > URL: https://issues.apache.org/jira/browse/HIVE-22448 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Gopal Vijayaraghavan >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22448.01.patch, HIVE-22448.patch > > Time Spent: 10m > Remaining Estimate: 0h > > {code} > create temporary table mytable1 (x integer, y integer, z integer, a integer); > explain cbo > select z, x, count(distinct y), count(distinct a) > from mytable1 > group by z, x; > explain cbo > select count(distinct y), count(distinct a) > from mytable1 > {code} > The first is not vectorized, the second one is because of the grouping-set > based rewrite for count distinct. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22274) Upgrade Calcite version to 1.21.0
[ https://issues.apache.org/jira/browse/HIVE-22274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965457#comment-16965457 ] Jesus Camacho Rodriguez commented on HIVE-22274: +1 (pending tests) > Upgrade Calcite version to 1.21.0 > - > > Key: HIVE-22274 > URL: https://issues.apache.org/jira/browse/HIVE-22274 > Project: Hive > Issue Type: Task >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22274.1.patch, HIVE-22274.10.patch, > HIVE-22274.10.patch, HIVE-22274.11.patch, HIVE-22274.12.patch, > HIVE-22274.13.patch, HIVE-22274.15.patch, HIVE-22274.16.patch, > HIVE-22274.17.patch, HIVE-22274.18.patch, HIVE-22274.19.patch, > HIVE-22274.19.patch, HIVE-22274.2.patch, HIVE-22274.3.patch, > HIVE-22274.4.patch, HIVE-22274.5.patch, HIVE-22274.6.patch, > HIVE-22274.7.patch, HIVE-22274.8.patch, HIVE-22274.9.patch, HIVE-22274.patch > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22448) CBO: Expand the multiple count distinct with a group-by key
[ https://issues.apache.org/jira/browse/HIVE-22448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965459#comment-16965459 ] Hive QA commented on HIVE-22448: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 56s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m 7s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 41s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 19s{color} | {color:blue} ql in master has 1545 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 29s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 8s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 8s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 41s{color} | {color:red} ql: The patch generated 5 new + 4 unchanged - 1 fixed = 9 total (was 5) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 21s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 16s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 27m 14s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19266/dev-support/hive-personality.sh | | git revision | master / 1c2b7fa | | Default Java | 1.8.0_111 | | findbugs | v3.0.1 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-19266/yetus/diff-checkstyle-ql.txt | | modules | C: ql itests U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-19266/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > CBO: Expand the multiple count distinct with a group-by key > --- > > Key: HIVE-22448 > URL: https://issues.apache.org/jira/browse/HIVE-22448 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Gopal Vijayaraghavan >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22448.01.patch, HIVE-22448.patch > > Time Spent: 10m > Remaining Estimate: 0h > > {code} > create temporary table mytable1 (x integer, y integer, z integer, a integer); > explain cbo > select z, x, count(distinct y), count(distinct a) > from mytable1 > group by z, x; > explain cbo > select count(distinct y), count(distinct a) > from mytable1 > {code} > The first is not vectorized, the second one is because of the grouping-set > based rewrite for count distinct. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22448) CBO: Expand the multiple count distinct with a group-by key
[ https://issues.apache.org/jira/browse/HIVE-22448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965477#comment-16965477 ] Hive QA commented on HIVE-22448: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12984694/HIVE-22448.01.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 17547 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/19266/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19266/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19266/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12984694 - PreCommit-HIVE-Build > CBO: Expand the multiple count distinct with a group-by key > --- > > Key: HIVE-22448 > URL: https://issues.apache.org/jira/browse/HIVE-22448 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Gopal Vijayaraghavan >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22448.01.patch, HIVE-22448.patch > > Time Spent: 10m > Remaining Estimate: 0h > > {code} > create temporary table mytable1 (x integer, y integer, z integer, a integer); > explain cbo > select z, x, count(distinct y), count(distinct a) > from mytable1 > group by z, x; > explain cbo > select count(distinct y), count(distinct a) > from mytable1 > {code} > The first is not vectorized, the second one is because of the grouping-set > based rewrite for count distinct. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22312) MapJoinCounterHook doesnot work for tez
[ https://issues.apache.org/jira/browse/HIVE-22312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965495#comment-16965495 ] Jesus Camacho Rodriguez commented on HIVE-22312: [~pulkits], can you create a PR? I will take a look at this shortly. Thanks > MapJoinCounterHook doesnot work for tez > --- > > Key: HIVE-22312 > URL: https://issues.apache.org/jira/browse/HIVE-22312 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: All Versions >Reporter: Pulkit Sharma >Assignee: Pulkit Sharma >Priority: Major > Attachments: HIVE-22312.01.patch, HIVE-22312.02.patch, > HIVE-22312.03.patch > > > In [HIVE-1792|https://issues.apache.org/jira/browse/HIVE-1792], > MapJoinCounterHook hook was added to track joins that get converted to map > join. This hook gets List of Tasks from hookContext and check Tag associated > with each task. For mr, we create Conditional tasks in case of joins and add > tags for respective join conversions. This does not work in case of tez as we > only create TezTask(no Conditional Task is created) which can handle multiple > joins in contrast to one Conditional Task per join in mr. > The current approach will fail even if we add tag to TezTask as it can have > multiple joins of the same type which will require counter. > One possible solution for tez, is to parse query-plan after query completion > which we get from hookContext to get workGraph. Using workGraph, we can walk > through Operator Tree to find join conversions. > If this approach looks good, I can raise Pull Request > cc [~ashutoshc] [~jcamachorodriguez] [~pxiong] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-17350) metrics errors when retrying HS2 startup
[ https://issues.apache.org/jira/browse/HIVE-17350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965502#comment-16965502 ] t oo edited comment on HIVE-17350 at 11/2/19 9:19 PM: -- [~sershe] [~khwunchai] Can u give more details? Are u saying with metrics enabled that give process won't start at all? was (Author: toopt4): [~sershe] [~khwunchai] Can I give more details? Are u saying with metrics enabled that give process won't start at all? > metrics errors when retrying HS2 startup > > > Key: HIVE-17350 > URL: https://issues.apache.org/jira/browse/HIVE-17350 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Priority: Major > > Looks like there are some sort of retries that happen when HS2 init fails. > When HS2 startup fails for an unrelated reason and is retried, the metrics > source initialization fails on subsequent attempts. > {noformat} > 2017-08-15T23:31:47,650 WARN [main]: impl.MetricsSystemImpl > (MetricsSystemImpl.java:init(152)) - hiveserver2 metrics system already > initialized! > 2017-08-15T23:31:47,650 ERROR [main]: metastore.HiveMetaStore > (HiveMetaStore.java:init(438)) - error in Metrics init: > java.lang.reflect.InvocationTargetException null > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.hadoop.hive.common.metrics.common.MetricsFactory.init(MetricsFactory.java:42) > at > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:435) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148) > at > org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107) > at > org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:79) > at > org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:92) > at > org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6892) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:140) > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1653) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:83) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) > at > org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3612) > at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3664) > at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3644) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:582) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:545) > at > org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:128) > at org.apache.hive.service.cli.CLIService.init(CLIService.java:113) > at > org.apache.hive.service.CompositeService.init(CompositeService.java:59) > at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:139) > at > org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:595) > at > org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:97) > at > org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:843) > at org.apache.
[jira] [Commented] (HIVE-17350) metrics errors when retrying HS2 startup
[ https://issues.apache.org/jira/browse/HIVE-17350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965502#comment-16965502 ] t oo commented on HIVE-17350: - [~sershe] [~khwunchai] Can I give more details? Are u saying with metrics enabled that give process won't start at all? > metrics errors when retrying HS2 startup > > > Key: HIVE-17350 > URL: https://issues.apache.org/jira/browse/HIVE-17350 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Priority: Major > > Looks like there are some sort of retries that happen when HS2 init fails. > When HS2 startup fails for an unrelated reason and is retried, the metrics > source initialization fails on subsequent attempts. > {noformat} > 2017-08-15T23:31:47,650 WARN [main]: impl.MetricsSystemImpl > (MetricsSystemImpl.java:init(152)) - hiveserver2 metrics system already > initialized! > 2017-08-15T23:31:47,650 ERROR [main]: metastore.HiveMetaStore > (HiveMetaStore.java:init(438)) - error in Metrics init: > java.lang.reflect.InvocationTargetException null > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.hadoop.hive.common.metrics.common.MetricsFactory.init(MetricsFactory.java:42) > at > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:435) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148) > at > org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107) > at > org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:79) > at > org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:92) > at > org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6892) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:140) > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1653) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:83) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) > at > org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3612) > at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3664) > at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3644) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:582) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:545) > at > org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:128) > at org.apache.hive.service.cli.CLIService.init(CLIService.java:113) > at > org.apache.hive.service.CompositeService.init(CompositeService.java:59) > at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:139) > at > org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:595) > at > org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:97) > at > org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:843) > at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:712) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(Native
[jira] [Comment Edited] (HIVE-17350) metrics errors when retrying HS2 startup
[ https://issues.apache.org/jira/browse/HIVE-17350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965502#comment-16965502 ] t oo edited comment on HIVE-17350 at 11/2/19 9:20 PM: -- [~sershe] [~khwunchai] Can u give more details? Are u saying with metrics enabled that hive process won't start at all? was (Author: toopt4): [~sershe] [~khwunchai] Can u give more details? Are u saying with metrics enabled that give process won't start at all? > metrics errors when retrying HS2 startup > > > Key: HIVE-17350 > URL: https://issues.apache.org/jira/browse/HIVE-17350 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Priority: Major > > Looks like there are some sort of retries that happen when HS2 init fails. > When HS2 startup fails for an unrelated reason and is retried, the metrics > source initialization fails on subsequent attempts. > {noformat} > 2017-08-15T23:31:47,650 WARN [main]: impl.MetricsSystemImpl > (MetricsSystemImpl.java:init(152)) - hiveserver2 metrics system already > initialized! > 2017-08-15T23:31:47,650 ERROR [main]: metastore.HiveMetaStore > (HiveMetaStore.java:init(438)) - error in Metrics init: > java.lang.reflect.InvocationTargetException null > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.hadoop.hive.common.metrics.common.MetricsFactory.init(MetricsFactory.java:42) > at > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:435) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148) > at > org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107) > at > org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:79) > at > org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:92) > at > org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6892) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:140) > at > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1653) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:83) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) > at > org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3612) > at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3664) > at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3644) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:582) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:545) > at > org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:128) > at org.apache.hive.service.cli.CLIService.init(CLIService.java:113) > at > org.apache.hive.service.CompositeService.init(CompositeService.java:59) > at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:139) > at > org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:595) > at > org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:97) > at > org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:843) > at org.apache.
[jira] [Commented] (HIVE-15141) metrics reporter using HADOOP2 is not able to re-initialize - and prevents hiveserver2 recovery
[ https://issues.apache.org/jira/browse/HIVE-15141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965504#comment-16965504 ] t oo commented on HIVE-15141: - Are metrics enabled causing process to crash? > metrics reporter using HADOOP2 is not able to re-initialize - and prevents > hiveserver2 recovery > --- > > Key: HIVE-15141 > URL: https://issues.apache.org/jira/browse/HIVE-15141 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > > * hiveserver2 initializes {{MetricsFactory}} => CodahaleMetrics created => > registers HADOOP2 source > * exception from somewhere...possibly recoverable > * MetricsFactory deinitializes the backend with close() > * retries failing because the metrics system cant continue -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22444) Clean up Project POM Files
[ https://issues.apache.org/jira/browse/HIVE-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22444: -- Attachment: (was: HIVE-22444.1.patch) > Clean up Project POM Files > -- > > Key: HIVE-22444 > URL: https://issues.apache.org/jira/browse/HIVE-22444 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22444.1.patch > > > # Address warnings in the build process > # Use DependencyManagement in Root POM for ITest (see HIVE-22426) > # General POM cleanup -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22444) Clean up Project POM Files
[ https://issues.apache.org/jira/browse/HIVE-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22444: -- Status: Patch Available (was: Open) > Clean up Project POM Files > -- > > Key: HIVE-22444 > URL: https://issues.apache.org/jira/browse/HIVE-22444 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22444.1.patch, HIVE-22444.2.patch > > > # Address warnings in the build process > # Use DependencyManagement in Root POM for ITest (see HIVE-22426) > # General POM cleanup -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22444) Clean up Project POM Files
[ https://issues.apache.org/jira/browse/HIVE-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22444: -- Attachment: HIVE-22444.2.patch > Clean up Project POM Files > -- > > Key: HIVE-22444 > URL: https://issues.apache.org/jira/browse/HIVE-22444 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22444.1.patch, HIVE-22444.2.patch > > > # Address warnings in the build process > # Use DependencyManagement in Root POM for ITest (see HIVE-22426) > # General POM cleanup -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22444) Clean up Project POM Files
[ https://issues.apache.org/jira/browse/HIVE-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22444: -- Status: Open (was: Patch Available) > Clean up Project POM Files > -- > > Key: HIVE-22444 > URL: https://issues.apache.org/jira/browse/HIVE-22444 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22444.1.patch, HIVE-22444.2.patch > > > # Address warnings in the build process > # Use DependencyManagement in Root POM for ITest (see HIVE-22426) > # General POM cleanup -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22274) Upgrade Calcite version to 1.21.0
[ https://issues.apache.org/jira/browse/HIVE-22274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin updated HIVE-22274: Attachment: HIVE-22274.19.patch > Upgrade Calcite version to 1.21.0 > - > > Key: HIVE-22274 > URL: https://issues.apache.org/jira/browse/HIVE-22274 > Project: Hive > Issue Type: Task >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22274.1.patch, HIVE-22274.10.patch, > HIVE-22274.10.patch, HIVE-22274.11.patch, HIVE-22274.12.patch, > HIVE-22274.13.patch, HIVE-22274.15.patch, HIVE-22274.16.patch, > HIVE-22274.17.patch, HIVE-22274.18.patch, HIVE-22274.19.patch, > HIVE-22274.19.patch, HIVE-22274.19.patch, HIVE-22274.2.patch, > HIVE-22274.3.patch, HIVE-22274.4.patch, HIVE-22274.5.patch, > HIVE-22274.6.patch, HIVE-22274.7.patch, HIVE-22274.8.patch, > HIVE-22274.9.patch, HIVE-22274.patch > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22444) Clean up Project POM Files
[ https://issues.apache.org/jira/browse/HIVE-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965525#comment-16965525 ] Hive QA commented on HIVE-22444: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 37s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 58s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 17s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 55s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 29s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 7s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 10s{color} | {color:green} storage-api generated 0 new + 0 unchanged - 2 fixed = 0 total (was 2) {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 20s{color} | {color:green} serde in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 13s{color} | {color:green} upgrade-acid in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 13s{color} | {color:green} custom-serde in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s{color} | {color:green} custom-udfs in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 13s{color} | {color:green} udf-classloader-udf1 in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 14s{color} | {color:green} udf-classloader-udf2 in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 13s{color} | {color:green} udf-vectorized-badexample in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 24s{color} | {color:green} hcatalog-unit in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s{color} | {color:green} hive-blobstore in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 21s{color} | {color:green} hive-jmh in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 23s{color} | {color:green} hive-minikdc in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s{color} | {color:green} hive-unit in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 24s{color} | {color:green} hive-unit-hadoop2 in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 18s{color} | {color:green} qtest-accumulo in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 14s{color} | {color:green} qtest-druid in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s{color} | {color:green} qtest-kudu in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 18s{color} | {color:green} qtest-spark in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 14s{color} | {color:green} test-serde in the patch passed. {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s{color} | {color:green} util in the patch passed. {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 22s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 26s{color} | {color:gree
[jira] [Commented] (HIVE-22444) Clean up Project POM Files
[ https://issues.apache.org/jira/browse/HIVE-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965533#comment-16965533 ] Hive QA commented on HIVE-22444: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12984696/HIVE-22444.2.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 73 failed/errored test(s), 17488 tests executed *Failed tests:* {noformat} TestJdbcWithMiniLlapArrow - did not produce a TEST-*.xml file (likely timed out) (batchId=284) TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=299) TestMiniHiveKafkaCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=299) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[buckets] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[create_database] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[create_like] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ctas_blobstore_to_blobstore] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ctas_blobstore_to_hdfs] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ctas_hdfs_to_blobstore] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[explain] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[having] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_blobstore] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_local] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_warehouse] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_local_to_blobstore] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_blobstore] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_blobstore_nonpart] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_local] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_warehouse] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_warehouse_nonpart] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_local_to_blobstore] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_blobstore_to_blobstore] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_empty_into_blobstore] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_dynamic_partitions] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_table] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_directory] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_table] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[join2] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[join] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[load_data] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[map_join] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[map_join_on_filter] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[multiple_agg] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[multiple_db] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[nested_outer_join] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[orc_buckets] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[orc_format_nonpart] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[orc_format_part] (batchId=302) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[orc_nonstd_par
[jira] [Commented] (HIVE-22274) Upgrade Calcite version to 1.21.0
[ https://issues.apache.org/jira/browse/HIVE-22274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965538#comment-16965538 ] Hive QA commented on HIVE-22274: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 56s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 41s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 3s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 29s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 17s{color} | {color:blue} ql in master has 1545 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 31s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 33s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 43s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m 46s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 26s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m 0s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 43s{color} | {color:red} ql: The patch generated 37 new + 329 unchanged - 12 fixed = 366 total (was 341) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 6s{color} | {color:red} root: The patch generated 38 new + 381 unchanged - 12 fixed = 419 total (was 393) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 18s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 52 unchanged - 0 fixed = 53 total (was 52) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m 15s{color} | {color:red} ql generated 8 new + 1542 unchanged - 3 fixed = 1550 total (was 1545) {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m 1s{color} | {color:red} ql generated 4 new + 96 unchanged - 4 fixed = 100 total (was 100) {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 6m 45s{color} | {color:red} root generated 4 new + 333 unchanged - 4 fixed = 337 total (was 337) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 76m 59s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | Dead store to joinInfo in org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveJoinFactoryImpl.createJoin(RelNode, RelNode, RexNode, Set, JoinRelType, boolean) At HiveRelFactories.java:org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveJoinFactoryImpl.createJoin(RelNode, RelNode, RexNode, Set, JoinRelType, boolean) At HiveRelFactories.java:[line 170] | | | Dead store to joinInfo in org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveJoinFactoryImpl.createJoin(RelNode, RelNode, RexNode, JoinRelType, Set, boolean) At HiveRelFactories.java:org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveJo
[jira] [Commented] (HIVE-22274) Upgrade Calcite version to 1.21.0
[ https://issues.apache.org/jira/browse/HIVE-22274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965537#comment-16965537 ] Hive QA commented on HIVE-22274: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12984697/HIVE-22274.19.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17546 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning (batchId=364) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/19268/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19268/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19268/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12984697 - PreCommit-HIVE-Build > Upgrade Calcite version to 1.21.0 > - > > Key: HIVE-22274 > URL: https://issues.apache.org/jira/browse/HIVE-22274 > Project: Hive > Issue Type: Task >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22274.1.patch, HIVE-22274.10.patch, > HIVE-22274.10.patch, HIVE-22274.11.patch, HIVE-22274.12.patch, > HIVE-22274.13.patch, HIVE-22274.15.patch, HIVE-22274.16.patch, > HIVE-22274.17.patch, HIVE-22274.18.patch, HIVE-22274.19.patch, > HIVE-22274.19.patch, HIVE-22274.19.patch, HIVE-22274.2.patch, > HIVE-22274.3.patch, HIVE-22274.4.patch, HIVE-22274.5.patch, > HIVE-22274.6.patch, HIVE-22274.7.patch, HIVE-22274.8.patch, > HIVE-22274.9.patch, HIVE-22274.patch > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22274) Upgrade Calcite version to 1.21.0
[ https://issues.apache.org/jira/browse/HIVE-22274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Carlin updated HIVE-22274: Attachment: HIVE-22274.19.patch > Upgrade Calcite version to 1.21.0 > - > > Key: HIVE-22274 > URL: https://issues.apache.org/jira/browse/HIVE-22274 > Project: Hive > Issue Type: Task >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22274.1.patch, HIVE-22274.10.patch, > HIVE-22274.10.patch, HIVE-22274.11.patch, HIVE-22274.12.patch, > HIVE-22274.13.patch, HIVE-22274.15.patch, HIVE-22274.16.patch, > HIVE-22274.17.patch, HIVE-22274.18.patch, HIVE-22274.19.patch, > HIVE-22274.19.patch, HIVE-22274.19.patch, HIVE-22274.19.patch, > HIVE-22274.2.patch, HIVE-22274.3.patch, HIVE-22274.4.patch, > HIVE-22274.5.patch, HIVE-22274.6.patch, HIVE-22274.7.patch, > HIVE-22274.8.patch, HIVE-22274.9.patch, HIVE-22274.patch > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22274) Upgrade Calcite version to 1.21.0
[ https://issues.apache.org/jira/browse/HIVE-22274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965552#comment-16965552 ] Hive QA commented on HIVE-22274: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 49s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 32s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 10s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 28s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 8s{color} | {color:blue} ql in master has 1545 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 31s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 32s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 43s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m 42s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 25s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m 13s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m 7s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m 7s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 45s{color} | {color:red} ql: The patch generated 37 new + 329 unchanged - 12 fixed = 366 total (was 341) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 8s{color} | {color:red} root: The patch generated 38 new + 381 unchanged - 12 fixed = 419 total (was 393) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 19s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 52 unchanged - 0 fixed = 53 total (was 52) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m 15s{color} | {color:red} ql generated 8 new + 1542 unchanged - 3 fixed = 1550 total (was 1545) {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m 2s{color} | {color:red} ql generated 4 new + 96 unchanged - 4 fixed = 100 total (was 100) {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 6m 46s{color} | {color:red} root generated 4 new + 333 unchanged - 4 fixed = 337 total (was 337) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 76m 44s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | Dead store to joinInfo in org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveJoinFactoryImpl.createJoin(RelNode, RelNode, RexNode, Set, JoinRelType, boolean) At HiveRelFactories.java:org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveJoinFactoryImpl.createJoin(RelNode, RelNode, RexNode, Set, JoinRelType, boolean) At HiveRelFactories.java:[line 170] | | | Dead store to joinInfo in org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveJoinFactoryImpl.createJoin(RelNode, RelNode, RexNode, JoinRelType, Set, boolean) At HiveRelFactories.java:org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveJo
[jira] [Commented] (HIVE-22274) Upgrade Calcite version to 1.21.0
[ https://issues.apache.org/jira/browse/HIVE-22274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965554#comment-16965554 ] Hive QA commented on HIVE-22274: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12984698/HIVE-22274.19.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 17546 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/19269/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19269/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19269/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12984698 - PreCommit-HIVE-Build > Upgrade Calcite version to 1.21.0 > - > > Key: HIVE-22274 > URL: https://issues.apache.org/jira/browse/HIVE-22274 > Project: Hive > Issue Type: Task >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22274.1.patch, HIVE-22274.10.patch, > HIVE-22274.10.patch, HIVE-22274.11.patch, HIVE-22274.12.patch, > HIVE-22274.13.patch, HIVE-22274.15.patch, HIVE-22274.16.patch, > HIVE-22274.17.patch, HIVE-22274.18.patch, HIVE-22274.19.patch, > HIVE-22274.19.patch, HIVE-22274.19.patch, HIVE-22274.19.patch, > HIVE-22274.2.patch, HIVE-22274.3.patch, HIVE-22274.4.patch, > HIVE-22274.5.patch, HIVE-22274.6.patch, HIVE-22274.7.patch, > HIVE-22274.8.patch, HIVE-22274.9.patch, HIVE-22274.patch > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22274) Upgrade Calcite version to 1.21.0
[ https://issues.apache.org/jira/browse/HIVE-22274?focusedWorklogId=337767&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-337767 ] ASF GitHub Bot logged work on HIVE-22274: - Author: ASF GitHub Bot Created on: 03/Nov/19 05:18 Start Date: 03/Nov/19 05:18 Worklog Time Spent: 10m Work Description: asfgit commented on pull request #809: HIVE-22274: upgrade Calcite to 1.21 URL: https://github.com/apache/hive/pull/809 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 337767) Time Spent: 1h 10m (was: 1h) > Upgrade Calcite version to 1.21.0 > - > > Key: HIVE-22274 > URL: https://issues.apache.org/jira/browse/HIVE-22274 > Project: Hive > Issue Type: Task >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22274.1.patch, HIVE-22274.10.patch, > HIVE-22274.10.patch, HIVE-22274.11.patch, HIVE-22274.12.patch, > HIVE-22274.13.patch, HIVE-22274.15.patch, HIVE-22274.16.patch, > HIVE-22274.17.patch, HIVE-22274.18.patch, HIVE-22274.19.patch, > HIVE-22274.19.patch, HIVE-22274.19.patch, HIVE-22274.19.patch, > HIVE-22274.2.patch, HIVE-22274.3.patch, HIVE-22274.4.patch, > HIVE-22274.5.patch, HIVE-22274.6.patch, HIVE-22274.7.patch, > HIVE-22274.8.patch, HIVE-22274.9.patch, HIVE-22274.patch > > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22274) Upgrade Calcite version to 1.21.0
[ https://issues.apache.org/jira/browse/HIVE-22274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22274: --- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master, thanks [~scarlin]! > Upgrade Calcite version to 1.21.0 > - > > Key: HIVE-22274 > URL: https://issues.apache.org/jira/browse/HIVE-22274 > Project: Hive > Issue Type: Task >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Steve Carlin >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22274.1.patch, HIVE-22274.10.patch, > HIVE-22274.10.patch, HIVE-22274.11.patch, HIVE-22274.12.patch, > HIVE-22274.13.patch, HIVE-22274.15.patch, HIVE-22274.16.patch, > HIVE-22274.17.patch, HIVE-22274.18.patch, HIVE-22274.19.patch, > HIVE-22274.19.patch, HIVE-22274.19.patch, HIVE-22274.19.patch, > HIVE-22274.2.patch, HIVE-22274.3.patch, HIVE-22274.4.patch, > HIVE-22274.5.patch, HIVE-22274.6.patch, HIVE-22274.7.patch, > HIVE-22274.8.patch, HIVE-22274.9.patch, HIVE-22274.patch > > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)