[jira] [Comment Edited] (HIVE-19084) Test case in Hive Query Language fails with a java.lang.AssertionError.
[ https://issues.apache.org/jira/browse/HIVE-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473832#comment-16473832 ] Alisha Prabhu edited comment on HIVE-19084 at 5/14/18 6:56 AM: --- Hi [~steveyeom2017], Could you please review the above patch ? There's a small change that I have introduced in the patch i.e. have sorted the FileStatus array object as, the listStatus function of the FileSystem does not guarantee to return the List of files/directories status in a sorted order. was (Author: alishap): Hi [~steveyeom2017], could you please review the above patch ? There's a small change that I have introduced in the patch i.e. have sorted the FileStatus array object as, the listStatus function of the FileSystem does not guarantee to return the List of files/directories status in a sorted order. > Test case in Hive Query Language fails with a java.lang.AssertionError. > --- > > Key: HIVE-19084 > URL: https://issues.apache.org/jira/browse/HIVE-19084 > Project: Hive > Issue Type: Bug > Components: Test, Transactions > Environment: uname -a > Linux pts00607-vm3 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:46 UTC > 2018 ppc64le ppc64le ppc64le GNU/Linux >Reporter: Alisha Prabhu >Assignee: Steve Yeom >Priority: Major > Labels: patch-available > Attachments: HIVE-19084.1.patch > > > The test case testInsertOverwriteForPartitionedMmTable in > TestTxnCommandsForMmTable.java and TestTxnCommandsForOrcMmTable.java fails > with a java.lang.AssertionError. > Maven command used is mvn > -Dtest=TestTxnCommandsForMmTable#testInsertOverwriteForPartitionedMmTable test > The test case fails as the listStatus function of the FileSystem does not > guarantee to return the List of files/directories status in a sorted order. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19084) Test case in Hive Query Language fails with a java.lang.AssertionError.
[ https://issues.apache.org/jira/browse/HIVE-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473832#comment-16473832 ] Alisha Prabhu commented on HIVE-19084: -- Hi [~steveyeom2017], could you please review the above patch ? There's a small change that I have introduced in the patch i.e. have sorted the FileStatus array object as, the listStatus function of the FileSystem does not guarantee to return the List of files/directories status in a sorted order. > Test case in Hive Query Language fails with a java.lang.AssertionError. > --- > > Key: HIVE-19084 > URL: https://issues.apache.org/jira/browse/HIVE-19084 > Project: Hive > Issue Type: Bug > Components: Test, Transactions > Environment: uname -a > Linux pts00607-vm3 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:46 UTC > 2018 ppc64le ppc64le ppc64le GNU/Linux >Reporter: Alisha Prabhu >Assignee: Steve Yeom >Priority: Major > Labels: patch-available > Attachments: HIVE-19084.1.patch > > > The test case testInsertOverwriteForPartitionedMmTable in > TestTxnCommandsForMmTable.java and TestTxnCommandsForOrcMmTable.java fails > with a java.lang.AssertionError. > Maven command used is mvn > -Dtest=TestTxnCommandsForMmTable#testInsertOverwriteForPartitionedMmTable test > The test case fails as the listStatus function of the FileSystem does not > guarantee to return the List of files/directories status in a sorted order. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19384) Vectorization: IfExprTimestamp* do not handle NULLs correctly
[ https://issues.apache.org/jira/browse/HIVE-19384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19384: Attachment: HIVE-19384.06.patch > Vectorization: IfExprTimestamp* do not handle NULLs correctly > - > > Key: HIVE-19384 > URL: https://issues.apache.org/jira/browse/HIVE-19384 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Blocker > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19384.01.patch, HIVE-19384.02.patch, > HIVE-19384.04.patch, HIVE-19384.05.patch, HIVE-19384.06.patch > > > HIVE-18622: "Vectorization: IF Statements, Comparisons, and more do not > handle NULLs correctly" didn't quite fix the IfExprTimestamp* classes > right > {noformat} > // Carefully handle NULLs... > outputColVector.noNulls = false;{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[ https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473828#comment-16473828 ] Laszlo Bodor commented on HIVE-17852: - list_bucket_dml_10 failures are related, I'm going to check > remove support for list bucketing "stored as directories" in 3.0 > > > Key: HIVE-17852 > URL: https://issues.apache.org/jira/browse/HIVE-17852 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-17852.01.patch > > > From the email thread: > 1) LB, when stored as directories, adds a lot of low-level complexity to Hive > tables that has to be accounted for in many places in the code where the > files are written or modified - from FSOP to ACID/replication/export. > 2) While working on some FSOP code I noticed that some of that logic is > broken - e.g. the duplicate file removal from tasks, a pretty fundamental > correctness feature in Hive, may be broken. LB also doesn’t appear to be > compatible with e.g. regular bucketing. > 3) The feature hasn’t seen development activity in a while; it also doesn’t > appear to be used a lot. > Keeping with the theme of cleaning up “legacy” code for 3.0, I was proposing > we remove it. > (2) also suggested that, if needed, it might be easier to implement similar > functionality by adding some flexibility to partitions (which LB directories > look like anyway); that would also keep the logic on a higher level of > abstraction (split generation, partition pruning) as opposed to many > low-level places like FSOP, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19384) Vectorization: IfExprTimestamp* do not handle NULLs correctly
[ https://issues.apache.org/jira/browse/HIVE-19384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473825#comment-16473825 ] Hive QA commented on HIVE-19384: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12923224/HIVE-19384.05.patch {color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 14409 tests executed *Failed tests:* {noformat} TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed out) (batchId=217) TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) (batchId=286) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fouter_join_ppr] (batchId=33) org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic] (batchId=253) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1] (batchId=175) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_bhif] (batchId=157) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acidvec_part] (batchId=170) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_part_llap_io] (batchId=170) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[special_character_in_tabnames_1] (batchId=165) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=163) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_dynpart_hashjoin_1] (batchId=174) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] (batchId=172) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_vector_dynpart_hashjoin_1] (batchId=172) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats] (batchId=167) org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut (batchId=225) org.apache.hadoop.hive.metastore.TestOldSchema.testPartitionOps (batchId=217) org.apache.hadoop.hive.metastore.TestStats.partitionedTableDeprecatedCalls (batchId=211) org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228) org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01 (batchId=228) org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232) org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=236) org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel (batchId=236) org.apache.hive.jdbc.TestSSL.testSSLFetchHttp (batchId=239) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill (batchId=240) org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=304) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/10921/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10921/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10921/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 25 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12923224 - PreCommit-HIVE-Build > Vectorization: IfExprTimestamp* do not handle NULLs correctly > - > > Key: HIVE-19384 > URL: https://issues.apache.org/jira/browse/HIVE-19384 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Blocker > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19384.01.patch, HIVE-19384.02.patch, > HIVE-19384.04.patch, HIVE-19384.05.patch > > > HIVE-18622: "Vectorization: IF Statements, Comparisons, and more do not > handle NULLs correctly" didn't quite fix the IfExprTimestamp* classes > right > {noformat} > // Carefully handle NULLs... > outputColVector.noNulls = false;{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19370) Issue: ADD Months function on timestamp datatype fields in hive
[ https://issues.apache.org/jira/browse/HIVE-19370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473811#comment-16473811 ] Amit Chauhan commented on HIVE-19370: - [~bharos92] I think this should be fine, IMO if string [correct]value is provided user should cast it as timestamp and the Default TS format and provide as input for adding months. for TS format you can document that udf expects in default TS format and that should be sufficient. Thanks for taking this up. > Issue: ADD Months function on timestamp datatype fields in hive > --- > > Key: HIVE-19370 > URL: https://issues.apache.org/jira/browse/HIVE-19370 > Project: Hive > Issue Type: Bug >Reporter: Amit Chauhan >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-19370.01.patch > > > *Issue:* > while using ADD_Months function on a timestamp datatype column the output > omits the time part[HH:MM:SS] part from output. > which should not be the case. > *query:* EMAIL_FAILURE_DTMZ is of datatype timestamp in hive. > hive> select CUSTOMER_ID,EMAIL_FAILURE_DTMZ,ADD_MONTHS (EMAIL_FAILURE_DTMZ , > 1) from TABLE1 where CUSTOMER_ID=125674937; > OK > 125674937 2015-12-09 12:25:53 2016-01-09 > *hiver version :* > hive> !hive --version; > Hive 1.2.1000.2.5.6.0-40 > > can you please help if somehow I can get below as output: > > 125674937 2015-12-09 12:25:53 2016-01-09 12:25:53 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19084) Test case in Hive Query Language fails with a java.lang.AssertionError.
[ https://issues.apache.org/jira/browse/HIVE-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alisha Prabhu updated HIVE-19084: - Labels: patch-available (was: ) Status: Patch Available (was: Open) > Test case in Hive Query Language fails with a java.lang.AssertionError. > --- > > Key: HIVE-19084 > URL: https://issues.apache.org/jira/browse/HIVE-19084 > Project: Hive > Issue Type: Bug > Components: Test, Transactions > Environment: uname -a > Linux pts00607-vm3 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:46 UTC > 2018 ppc64le ppc64le ppc64le GNU/Linux >Reporter: Alisha Prabhu >Assignee: Steve Yeom >Priority: Major > Labels: patch-available > Attachments: HIVE-19084.1.patch > > > The test case testInsertOverwriteForPartitionedMmTable in > TestTxnCommandsForMmTable.java and TestTxnCommandsForOrcMmTable.java fails > with a java.lang.AssertionError. > Maven command used is mvn > -Dtest=TestTxnCommandsForMmTable#testInsertOverwriteForPartitionedMmTable test > The test case fails as the listStatus function of the FileSystem does not > guarantee to return the List of files/directories status in a sorted order. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19084) Test case in Hive Query Language fails with a java.lang.AssertionError.
[ https://issues.apache.org/jira/browse/HIVE-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alisha Prabhu updated HIVE-19084: - Attachment: HIVE-19084.1.patch > Test case in Hive Query Language fails with a java.lang.AssertionError. > --- > > Key: HIVE-19084 > URL: https://issues.apache.org/jira/browse/HIVE-19084 > Project: Hive > Issue Type: Bug > Components: Test, Transactions > Environment: uname -a > Linux pts00607-vm3 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:46 UTC > 2018 ppc64le ppc64le ppc64le GNU/Linux >Reporter: Alisha Prabhu >Assignee: Steve Yeom >Priority: Major > Attachments: HIVE-19084.1.patch > > > The test case testInsertOverwriteForPartitionedMmTable in > TestTxnCommandsForMmTable.java and TestTxnCommandsForOrcMmTable.java fails > with a java.lang.AssertionError. > Maven command used is mvn > -Dtest=TestTxnCommandsForMmTable#testInsertOverwriteForPartitionedMmTable test > The test case fails as the listStatus function of the FileSystem does not > guarantee to return the List of files/directories status in a sorted order. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19084) Test case in Hive Query Language fails with a java.lang.AssertionError.
[ https://issues.apache.org/jira/browse/HIVE-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alisha Prabhu updated HIVE-19084: - Attachment: (was: HIVE-19084.1.patch) > Test case in Hive Query Language fails with a java.lang.AssertionError. > --- > > Key: HIVE-19084 > URL: https://issues.apache.org/jira/browse/HIVE-19084 > Project: Hive > Issue Type: Bug > Components: Test, Transactions > Environment: uname -a > Linux pts00607-vm3 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:46 UTC > 2018 ppc64le ppc64le ppc64le GNU/Linux >Reporter: Alisha Prabhu >Assignee: Steve Yeom >Priority: Major > > The test case testInsertOverwriteForPartitionedMmTable in > TestTxnCommandsForMmTable.java and TestTxnCommandsForOrcMmTable.java fails > with a java.lang.AssertionError. > Maven command used is mvn > -Dtest=TestTxnCommandsForMmTable#testInsertOverwriteForPartitionedMmTable test > The test case fails as the listStatus function of the FileSystem does not > guarantee to return the List of files/directories status in a sorted order. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19384) Vectorization: IfExprTimestamp* do not handle NULLs correctly
[ https://issues.apache.org/jira/browse/HIVE-19384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473793#comment-16473793 ] Hive QA commented on HIVE-19384: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 33s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 51s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 17s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 2s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 23s{color} | {color:blue} storage-api in master has 48 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 51s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 12s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 30s{color} | {color:red} ql in the patch failed. {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 18s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 10s{color} | {color:red} storage-api: The patch generated 2 new + 12 unchanged - 2 fixed = 14 total (was 14) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 44s{color} | {color:red} ql: The patch generated 74 new + 608 unchanged - 16 fixed = 682 total (was 624) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 11s{color} | {color:red} vector-code-gen: The patch generated 3 new + 316 unchanged - 0 fixed = 319 total (was 316) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m 1s{color} | {color:red} ql generated 8 new + 2320 unchanged - 1 fixed = 2328 total (was 2321) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 24m 5s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | Class org.apache.hadoop.hive.ql.exec.vector.expressions.gen.IfExprIntervalDayTimeColumnScalar defines non-transient non-serializable instance field arg3Scalar In IfExprIntervalDayTimeColumnScalar.java:instance field arg3Scalar In IfExprIntervalDayTimeColumnScalar.java | | | Class org.apache.hadoop.hive.ql.exec.vector.expressions.gen.IfExprIntervalDayTimeScalarColumn defines non-transient non-serializable instance field arg2Scalar In IfExprIntervalDayTimeScalarColumn.java:instance field arg2Scalar In IfExprIntervalDayTimeScalarColumn.java | | | Class org.apache.hadoop.hive.ql.exec.vector.expressions.gen.IfExprIntervalDayTimeScalarScalar defines non-transient non-serializable instance field arg2Scalar In IfExprIntervalDayTimeScalarScalar.java:instance field arg2Scalar In IfExprIntervalDayTimeScalarScalar.java | | | Class org.apache.hadoop.hive.ql.exec.vector.expressions.gen.IfExprIntervalDayTimeScalarScalar defines non-transient non-serializable instance field arg3Scalar In IfExprIntervalDayTimeScalarScalar.java:instance field arg3Scalar In IfExprIntervalDayTimeScalarScalar.java | | | new org.apache.hadoop.hive.ql.exec.vector.expressions.gen.IfExprTimestampColumnScalar(int, int, Timestamp, int) may expose internal representation by storing an externally mutable object into IfExprTimestampColumnScalar.arg3Scalar At If
[jira] [Updated] (HIVE-19488) enable CM root based on db parameter, identifying a db as source of replication.
[ https://issues.apache.org/jira/browse/HIVE-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-19488: --- Attachment: HIVE-19488.03.patch > enable CM root based on db parameter, identifying a db as source of > replication. > > > Key: HIVE-19488 > URL: https://issues.apache.org/jira/browse/HIVE-19488 > Project: Hive > Issue Type: Task > Components: Hive, HiveServer2, repl >Affects Versions: 3.1.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Fix For: 3.1.0 > > Attachments: HIVE-19488.01.patch, HIVE-19488.02.patch, > HIVE-19488.03.patch > > > * add a parameter at db level to identify if its a source of replication. > beacon will set this. > * Enable CM root only for databases that are a source of a replication > policy, for other db's skip the CM root functionality. > * prevent database drop if the parameter indicating its source of a > replication, is set. > * as an upgrade to this version, beacon should set the property on all > existing database policies, in affect. > * the parameter should be of the form . – repl.source.for : List < policy > ids > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19184) Hive 3.0.0 release branch preparation
[ https://issues.apache.org/jira/browse/HIVE-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473782#comment-16473782 ] Hive QA commented on HIVE-19184: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12919517/HIVE-19184.01-branch-3.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 28 failed/errored test(s), 14191 tests executed *Failed tests:* {noformat} TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=253) TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253) TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=253) TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=253) TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed out) (batchId=217) TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=253) TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) (batchId=286) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[orc_format_part] (batchId=256) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1] (batchId=175) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=163) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] (batchId=172) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_vector_dynpart_hashjoin_1] (batchId=172) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats] (batchId=167) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insertsel_fail] (batchId=95) org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut (batchId=225) org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228) org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01 (batchId=228) org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232) org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2 (batchId=241) org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles (batchId=241) org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent (batchId=241) org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps (batchId=241) org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead (batchId=241) org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite (batchId=241) org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes (batchId=241) org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime (batchId=241) org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime (batchId=241) org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill (batchId=241) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/10920/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10920/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10920/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 28 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12919517 - PreCommit-HIVE-Build > Hive 3.0.0 release branch preparation > - > > Key: HIVE-19184 > URL: https://issues.apache.org/jira/browse/HIVE-19184 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19184.01-branch-3.patch > > > Need to do bunch of things to prepare branch-3 for release e.g. > * Update pom.xml to delete SNAPSHOT > * Update .reviewboardrc > * Remove storage-api module to build > * Change storage-api depdency etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19108) Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q causes Wrong Query Results
[ https://issues.apache.org/jira/browse/HIVE-19108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473773#comment-16473773 ] Haifeng Chen commented on HIVE-19108: - Thanks [~mmccline], [~vihangk1] and [~vgarg] for reviewing and moving this work forward. > Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q > causes Wrong Query Results > --- > > Key: HIVE-19108 > URL: https://issues.apache.org/jira/browse/HIVE-19108 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Matt McCline >Assignee: Haifeng Chen >Priority: Critical > Attachments: HIVE-19108.01.patch, HIVE-19108.02.patch, > HIVE-19108.03.patch, HIVE-19108.04.patch, HIVE-19108.05-branch-3.patch > > > Found in vectorization enable by default experiment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19184) Hive 3.0.0 release branch preparation
[ https://issues.apache.org/jira/browse/HIVE-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473762#comment-16473762 ] Hive QA commented on HIVE-19184: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 24s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 0m 53s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-10920/dev-support/hive-personality.sh | | git revision | master / f327624 | | modules | C: . U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-10920/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Hive 3.0.0 release branch preparation > - > > Key: HIVE-19184 > URL: https://issues.apache.org/jira/browse/HIVE-19184 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-19184.01-branch-3.patch > > > Need to do bunch of things to prepare branch-3 for release e.g. > * Update pom.xml to delete SNAPSHOT > * Update .reviewboardrc > * Remove storage-api module to build > * Change storage-api depdency etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19454) Test failure : org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails with java.lang.AssertionError
[ https://issues.apache.org/jira/browse/HIVE-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473761#comment-16473761 ] Alisha Prabhu commented on HIVE-19454: -- Thanks for the update [~djaiswal]. > Test failure : > org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails > with java.lang.AssertionError > -- > > Key: HIVE-19454 > URL: https://issues.apache.org/jira/browse/HIVE-19454 > Project: Hive > Issue Type: Bug > Environment: uname -a > Linux pts00433-vm40 3.10.0-862.el7.ppc64le #1 SMP Wed Mar 21 22:20:19 UTC > 2018 ppc64le ppc64le ppc64le GNU/Linux >Reporter: Alisha Prabhu >Priority: Major > Attachments: HIVE-19454.1.patch, HIVE-19454.2.patch > > > org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion1 > refers to the same test case. > Maven command used is : mvn -Dtest=TestTxnCommands2 test > Error: > {code:java} > [INFO] Running org.apache.hadoop.hive.ql.TestTxnCommands2 > [ERROR] Tests run: 44, Failures: 1, Errors: 0, Skipped: 3, Time elapsed: > 618.215 s <<< FAILURE! - in org.apache.hadoop.hive.ql.TestTxnCommands2 > [ERROR] > testNonAcidToAcidConversion1(org.apache.hadoop.hive.ql.TestTxnCommands2) > Time elapsed: 17.557 s <<< FAILURE! > java.lang.AssertionError > at org.junit.Assert.fail(Assert.java:86) > at org.junit.Assert.assertTrue(Assert.java:41) > at org.junit.Assert.assertTrue(Assert.java:52) > at > org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1(TestTxnCommands2.java:499) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) > at > org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:168) > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) > at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) > at > org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379) > at > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340) > at > org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125) > at > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413) > [INFO] > [INFO] Results: > [INFO] > [ERROR] Failures: > [ERROR] TestTxnCommands2.testNonAcidToAcidConversion1:499 > [INFO] > [ERROR] Tests run: 44, Failures: 1, Errors: 0, Skipped: 3 > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[ https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473759#comment-16473759 ] Hive QA commented on HIVE-17852: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 48s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 42s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 28s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 83m 12s{color} | {color:green} master passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 32s{color} | {color:red} branch/itests/hive-unit cannot run setBugDatabaseInfo from findbugs {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 21s{color} | {color:red} branch/metastore cannot run setBugDatabaseInfo from findbugs {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 51s{color} | {color:red} branch/ql cannot run setBugDatabaseInfo from findbugs {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 6s{color} | {color:red} branch/standalone-metastore cannot run setBugDatabaseInfo from findbugs {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m 38s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 7s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 25s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 25s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 52m 49s{color} | {color:red} root: The patch generated 1398 new + 245048 unchanged - 1515 fixed = 246446 total (was 246563) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 39s{color} | {color:red} itests/hive-unit: The patch generated 150 new + 11808 unchanged - 157 fixed = 11958 total (was 11965) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 24m 56s{color} | {color:red} ql: The patch generated 773 new + 126044 unchanged - 868 fixed = 126817 total (was 126912) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 4m 7s{color} | {color:red} standalone-metastore: The patch generated 473 new + 19451 unchanged - 488 fixed = 19924 total (was 19939) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 2s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 31s{color} | {color:red} patch/itests/hive-unit cannot run setBugDatabaseInfo from findbugs {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 21s{color} | {color:red} patch/metastore cannot run setBugDatabaseInfo from findbugs {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 43s{color} | {color:red} patch/ql cannot run setBugDatabaseInfo from findbugs {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 3s{color} | {color:red} patch/standalone-metastore cannot run setBugDatabaseInfo from findbugs {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 5m 12s{color} | {color:red} root generated 2 new + 366 unchanged - 2 fixed = 368 total (was 368) {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 51s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100 total (was 100) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 16s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color}
[jira] [Commented] (HIVE-19515) TestRpc.testServerPort is consistently failing
[ https://issues.apache.org/jira/browse/HIVE-19515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473755#comment-16473755 ] Hive QA commented on HIVE-19515: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12923221/HIVE-19515.1.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 24 failed/errored test(s), 14395 tests executed *Failed tests:* {noformat} TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed out) (batchId=217) TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) (batchId=286) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fouter_join_ppr] (batchId=33) org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic] (batchId=253) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1] (batchId=175) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_bhif] (batchId=157) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acidvec_part] (batchId=170) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_part_llap_io] (batchId=170) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[special_character_in_tabnames_1] (batchId=165) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=163) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] (batchId=172) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_vector_dynpart_hashjoin_1] (batchId=172) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats] (batchId=167) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat] (batchId=183) org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut (batchId=225) org.apache.hadoop.hive.metastore.TestOldSchema.testPartitionOps (batchId=217) org.apache.hadoop.hive.metastore.TestStats.partitionedTableDeprecatedCalls (batchId=211) org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228) org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01 (batchId=228) org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232) org.apache.hadoop.hive.ql.plan.mapping.TestReOptimization.testStatCachingMetaStore (batchId=299) org.apache.hive.hcatalog.pig.TestRCFileHCatStorer.testWriteDecimalXY (batchId=196) org.apache.hive.jdbc.TestSSL.testSSLFetchHttp (batchId=239) org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.org.apache.hive.service.TestHS2ImpersonationWithRemoteMS (batchId=238) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/10919/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10919/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10919/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 24 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12923221 - PreCommit-HIVE-Build > TestRpc.testServerPort is consistently failing > -- > > Key: HIVE-19515 > URL: https://issues.apache.org/jira/browse/HIVE-19515 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-19515.1.patch > > > {{TestRpc.testServerPort}} is consistently failing due to HIVE-17838 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19485) dump directory for non native tables should not be created
[ https://issues.apache.org/jira/browse/HIVE-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-19485: --- Attachment: HIVE-19485.1.patch > dump directory for non native tables should not be created > -- > > Key: HIVE-19485 > URL: https://issues.apache.org/jira/browse/HIVE-19485 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: anishek >Assignee: anishek >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19485.0.patch, HIVE-19485.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19514) Successful test logs should be copied
[ https://issues.apache.org/jira/browse/HIVE-19514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473752#comment-16473752 ] Vihang Karajgaonkar commented on HIVE-19514: Ptest server now only copies successful logs. This inprovement shaves off 10 min from the total execution time. {noformat} 2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase took 64 minutes 2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase.TotalRsyncElapsedTime took 8 minutes 2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: PrepPhase took 5 minutes 2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: ReportingPhase took 0 minutes 2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: TestCheckPhase took 0 minutes 2018-05-14 01:29:45,702 INFO [TestExecutor] PTest.run:237 PERF: YetusPhase took 0 minutes {noformat} > Successful test logs should be copied > - > > Key: HIVE-19514 > URL: https://issues.apache.org/jira/browse/HIVE-19514 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > > Ptest tries to copy ~20G worth of logs from the worker nodes to the server. > The total aggregate time spent by all the Ptest server in copying these files > is close to 15 min (although much of that is in parallel threads). Most of > the times we don't check for the logs. We seldom check for logs of successful > tests. We should ignore copying successful test logs by default to improve > runtime. > {noformat} > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:235 Executed 13568 > tests > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: > ExecutionPhase took 71 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: > ExecutionPhase.TotalRsyncElapsedTime took 15 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: PrepPhase > took 5 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: > ReportingPhase took 0 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: > TestCheckPhase took 0 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: YetusPhase > took 0 minutes > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-19514) Successful test logs should be copied
[ https://issues.apache.org/jira/browse/HIVE-19514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar resolved HIVE-19514. Resolution: Fixed > Successful test logs should be copied > - > > Key: HIVE-19514 > URL: https://issues.apache.org/jira/browse/HIVE-19514 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > > Ptest tries to copy ~20G worth of logs from the worker nodes to the server. > The total aggregate time spent by all the Ptest server in copying these files > is close to 15 min (although much of that is in parallel threads). Most of > the times we don't check for the logs. We seldom check for logs of successful > tests. We should ignore copying successful test logs by default to improve > runtime. > {noformat} > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:235 Executed 13568 > tests > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: > ExecutionPhase took 71 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: > ExecutionPhase.TotalRsyncElapsedTime took 15 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: PrepPhase > took 5 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: > ReportingPhase took 0 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: > TestCheckPhase took 0 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: YetusPhase > took 0 minutes > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19479) encoded stream seek is incorrect for 0-length RGs in LLAP IO
[ https://issues.apache.org/jira/browse/HIVE-19479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19479: --- Fix Version/s: 3.0.0 > encoded stream seek is incorrect for 0-length RGs in LLAP IO > > > Key: HIVE-19479 > URL: https://issues.apache.org/jira/browse/HIVE-19479 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19479.01.patch, HIVE-19479.patch > > > The PositionProvider offset is not updated correctly and an error like this > may happen: > {noformat} > Caused by: java.lang.IllegalArgumentException: Seek in LENGTH to 541 is > outside of the data > at > org.apache.orc.impl.InStream$UncompressedStream.seek(InStream.java:161) > at > org.apache.orc.impl.InStream$UncompressedStream.seek(InStream.java:123) > at > org.apache.orc.impl.RunLengthIntegerReaderV2.seek(RunLengthIntegerReaderV2.java:331) > at > org.apache.hadoop.hive.ql.io.orc.encoded.EncodedTreeReaderFactory$StringStreamReader.seek(EncodedTreeReaderFactory.java:298) > at > org.apache.hadoop.hive.ql.io.orc.encoded.EncodedTreeReaderFactory$StringStreamReader.seek(EncodedTreeReaderFactory.java:258) > at > org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.repositionInStreams(OrcEncodedDataConsumer.java:250) > at > org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:134) > at > org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:62) > {noformat} > We found this happens when ORC writes a strange stream combination - data > stream for a RG has no values (the rows all have nulls), but there are values > (0-s) in length stream for the same rows. That is technically a valid ORC > file, although writing the 0s is completely useless. > This may be fixed separately in ORC, but since these files now exist in the > wild we should handle them correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19479) encoded stream seek is incorrect for 0-length RGs in LLAP IO
[ https://issues.apache.org/jira/browse/HIVE-19479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473740#comment-16473740 ] Vineet Garg commented on HIVE-19479: Never mind I am able to cherry-pick this. I have pushed this to branch-3 > encoded stream seek is incorrect for 0-length RGs in LLAP IO > > > Key: HIVE-19479 > URL: https://issues.apache.org/jira/browse/HIVE-19479 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19479.01.patch, HIVE-19479.patch > > > The PositionProvider offset is not updated correctly and an error like this > may happen: > {noformat} > Caused by: java.lang.IllegalArgumentException: Seek in LENGTH to 541 is > outside of the data > at > org.apache.orc.impl.InStream$UncompressedStream.seek(InStream.java:161) > at > org.apache.orc.impl.InStream$UncompressedStream.seek(InStream.java:123) > at > org.apache.orc.impl.RunLengthIntegerReaderV2.seek(RunLengthIntegerReaderV2.java:331) > at > org.apache.hadoop.hive.ql.io.orc.encoded.EncodedTreeReaderFactory$StringStreamReader.seek(EncodedTreeReaderFactory.java:298) > at > org.apache.hadoop.hive.ql.io.orc.encoded.EncodedTreeReaderFactory$StringStreamReader.seek(EncodedTreeReaderFactory.java:258) > at > org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.repositionInStreams(OrcEncodedDataConsumer.java:250) > at > org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:134) > at > org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:62) > {noformat} > We found this happens when ORC writes a strange stream combination - data > stream for a RG has no values (the rows all have nulls), but there are values > (0-s) in length stream for the same rows. That is technically a valid ORC > file, although writing the 0s is completely useless. > This may be fixed separately in ORC, but since these files now exist in the > wild we should handle them correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19479) encoded stream seek is incorrect for 0-length RGs in LLAP IO
[ https://issues.apache.org/jira/browse/HIVE-19479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473739#comment-16473739 ] Vineet Garg commented on HIVE-19479: [~sershe] Can you upload patch for branch-3? > encoded stream seek is incorrect for 0-length RGs in LLAP IO > > > Key: HIVE-19479 > URL: https://issues.apache.org/jira/browse/HIVE-19479 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19479.01.patch, HIVE-19479.patch > > > The PositionProvider offset is not updated correctly and an error like this > may happen: > {noformat} > Caused by: java.lang.IllegalArgumentException: Seek in LENGTH to 541 is > outside of the data > at > org.apache.orc.impl.InStream$UncompressedStream.seek(InStream.java:161) > at > org.apache.orc.impl.InStream$UncompressedStream.seek(InStream.java:123) > at > org.apache.orc.impl.RunLengthIntegerReaderV2.seek(RunLengthIntegerReaderV2.java:331) > at > org.apache.hadoop.hive.ql.io.orc.encoded.EncodedTreeReaderFactory$StringStreamReader.seek(EncodedTreeReaderFactory.java:298) > at > org.apache.hadoop.hive.ql.io.orc.encoded.EncodedTreeReaderFactory$StringStreamReader.seek(EncodedTreeReaderFactory.java:258) > at > org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.repositionInStreams(OrcEncodedDataConsumer.java:250) > at > org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:134) > at > org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:62) > {noformat} > We found this happens when ORC writes a strange stream combination - data > stream for a RG has no values (the rows all have nulls), but there are values > (0-s) in length stream for the same rows. That is technically a valid ORC > file, although writing the 0s is completely useless. > This may be fixed separately in ORC, but since these files now exist in the > wild we should handle them correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19522) Add SchemaRegistry docs
[ https://issues.apache.org/jira/browse/HIVE-19522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani reassigned HIVE-19522: - > Add SchemaRegistry docs > --- > > Key: HIVE-19522 > URL: https://issues.apache.org/jira/browse/HIVE-19522 > Project: Hive > Issue Type: Sub-task >Reporter: Sriharsha Chintalapani >Assignee: Sriharsha Chintalapani >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19521) Add docker support for standalone-metastore
[ https://issues.apache.org/jira/browse/HIVE-19521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani reassigned HIVE-19521: - > Add docker support for standalone-metastore > --- > > Key: HIVE-19521 > URL: https://issues.apache.org/jira/browse/HIVE-19521 > Project: Hive > Issue Type: Sub-task >Reporter: Sriharsha Chintalapani >Assignee: Sriharsha Chintalapani >Priority: Major > > Build docker module to run hive metastore & registry servers -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19520) Add SchemaRegistry Client to Hive Meta Store
[ https://issues.apache.org/jira/browse/HIVE-19520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473730#comment-16473730 ] ASF GitHub Bot commented on HIVE-19520: --- GitHub user harshach opened a pull request: https://github.com/apache/hive/pull/349 [WIP] HIVE-19520: Add SchemaRegistry Client to Hive Meta Store WIP patchp, requires merging HIVE-17989 You can merge this pull request into a Git repository by running: $ git pull https://github.com/harshach/hive HIVE-19520 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/349.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #349 commit 1ba9b62d9ef488355e1a97dbc7237c1472349a24 Author: Alan Gates Date: 2017-10-19T23:49:38Z HIVE-17983 Make the standalone metastore generate tarballs etc. commit b623000140605778b1173c0d9772ce051b70cbb8 Author: Alan Gates Date: 2017-10-19T23:49:38Z HIVE-17990 Add Thrift and DB storage for Schema Registry objects commit 6032eb627ede54c9ca3d79341d3fd95b6bc33863 Author: Sriharsha Chintalapani Date: 2018-02-25T05:02:06Z HIVE-18799: Add SchemaRegistry APIs to Hive MetaStore. commit bfb5b2a48204c7c43f72948fa8bf27b64ed62ad1 Author: Sriharsha Chintalapani Date: 2018-02-25T06:51:01Z Merge remote-tracking branch 'apache/standalone-metastore' into standalone-metastore commit 539a14da4027fc270f036747844f64b8f3a06d3d Author: Sriharsha Chintalapani Date: 2018-03-15T16:57:23Z removed storage module for registry commit 50951ed34304a0cb2999227adbb2e51104cdf517 Author: Sriharsha Chintalapani Date: 2018-04-19T21:41:20Z Registry code changes to use HiveMetaStore thrift api to store objects. commit 5627175f19ce22bc2e5d9fa61ddeb7dbc671ca82 Author: Sriharsha Chintalapani Date: 2018-04-25T22:14:10Z Follow-up patch for Registry APIs commit 35243aae9d638b8320dabdb7a84fece394c1d8e4 Author: Sriharsha Chintalapani Date: 2018-05-14T01:51:21Z HIVE-18799: Add SchemaRegistry APIs to Hive MetaStore commit b6844e3d563c55752db8b95976cb5d8c8c8b2fe9 Author: Sriharsha Chintalapani Date: 2018-05-14T03:14:57Z HIVE-19520: Add SchemaRegistry Client to Hive Meta Store > Add SchemaRegistry Client to Hive Meta Store > > > Key: HIVE-19520 > URL: https://issues.apache.org/jira/browse/HIVE-19520 > Project: Hive > Issue Type: Sub-task >Reporter: Sriharsha Chintalapani >Assignee: Sriharsha Chintalapani >Priority: Major > Labels: pull-request-available > > Schema registry client allows users to embed a HTTP client to access the > schemas during the runtime for streaming clients. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19520) Add SchemaRegistry Client to Hive Meta Store
[ https://issues.apache.org/jira/browse/HIVE-19520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-19520: -- Labels: pull-request-available (was: ) > Add SchemaRegistry Client to Hive Meta Store > > > Key: HIVE-19520 > URL: https://issues.apache.org/jira/browse/HIVE-19520 > Project: Hive > Issue Type: Sub-task >Reporter: Sriharsha Chintalapani >Assignee: Sriharsha Chintalapani >Priority: Major > Labels: pull-request-available > > Schema registry client allows users to embed a HTTP client to access the > schemas during the runtime for streaming clients. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19515) TestRpc.testServerPort is consistently failing
[ https://issues.apache.org/jira/browse/HIVE-19515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473723#comment-16473723 ] Hive QA commented on HIVE-19515: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 3s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 11s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 26s{color} | {color:blue} spark-client in master has 15 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 11s{color} | {color:red} spark-client: The patch generated 1 new + 19 unchanged - 0 fixed = 20 total (was 19) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 11m 15s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-10919/dev-support/hive-personality.sh | | git revision | master / f327624 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-10919/yetus/diff-checkstyle-spark-client.txt | | modules | C: spark-client U: spark-client | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-10919/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > TestRpc.testServerPort is consistently failing > -- > > Key: HIVE-19515 > URL: https://issues.apache.org/jira/browse/HIVE-19515 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-19515.1.patch > > > {{TestRpc.testServerPort}} is consistently failing due to HIVE-17838 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-6980) Drop table by using direct sql
[ https://issues.apache.org/jira/browse/HIVE-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473711#comment-16473711 ] Hive QA commented on HIVE-6980: --- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12923219/HIVE-6980.4.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 31 failed/errored test(s), 14395 tests executed *Failed tests:* {noformat} TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed out) (batchId=217) TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) (batchId=286) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fouter_join_ppr] (batchId=33) org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic] (batchId=253) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1] (batchId=175) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_4] (batchId=164) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_bhif] (batchId=157) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acidvec_part] (batchId=170) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_part_llap_io] (batchId=170) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[special_character_in_tabnames_1] (batchId=165) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=163) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_dynpart_hashjoin_1] (batchId=174) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] (batchId=172) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats] (batchId=167) org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.testCliDriver[spark_job_max_tasks] (batchId=255) org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.testCliDriver[spark_stage_max_tasks] (batchId=255) org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.testCliDriver[spark_task_failure] (batchId=255) org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query39] (batchId=255) org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut (batchId=225) org.apache.hadoop.hive.metastore.TestOldSchema.testPartitionOps (batchId=217) org.apache.hadoop.hive.metastore.TestStats.partitionedTableDeprecatedCalls (batchId=211) org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228) org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01 (batchId=228) org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232) org.apache.hadoop.hive.ql.TestTxnCommands.testMergeUpdateDelete (batchId=301) org.apache.hadoop.hive.ql.TestTxnCommands2.testAlterTable (batchId=286) org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=236) org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel (batchId=236) org.apache.hive.jdbc.TestSSL.testSSLFetchHttp (batchId=239) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveBackKill (batchId=240) org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=304) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/10918/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10918/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10918/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 31 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12923219 - PreCommit-HIVE-Build > Drop table by using direct sql > -- > > Key: HIVE-6980 > URL: https://issues.apache.org/jira/browse/HIVE-6980 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 0.12.0 >Reporter: Selina Zhang >Assignee: Peter Vary >Priority: Major > Attachments: HIVE-6980.2.patch, HIVE-6980.3.patch, HIVE-6980.4.patch, > HIVE-6980.patch, drop_table_after.png, drop_table_before.png > > > Dropping table which has lots of partitions is slow. Even after applying the > patch of HIVE-6265, the drop table still takes hours (100K+ partitions). > The fixes come with two parts: > 1. use directSQL to query the partitions protect mode; > the current implementation needs to transfer the Partition object to client > and check the protect mode for each partition. I'd like t
[jira] [Updated] (HIVE-19519) Add SchemaRegistry UI to Hive Metastore
[ https://issues.apache.org/jira/browse/HIVE-19519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-19519: -- Labels: pull-request-available (was: ) > Add SchemaRegistry UI to Hive Metastore > > > Key: HIVE-19519 > URL: https://issues.apache.org/jira/browse/HIVE-19519 > Project: Hive > Issue Type: Sub-task >Reporter: Sriharsha Chintalapani >Assignee: Sriharsha Chintalapani >Priority: Major > Labels: pull-request-available > > Schema registry has a web UI to browse and add schemas and manage them. This > Jira is to add that UI into Hive MetaStore -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19519) Add SchemaRegistry UI to Hive Metastore
[ https://issues.apache.org/jira/browse/HIVE-19519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473705#comment-16473705 ] ASF GitHub Bot commented on HIVE-19519: --- GitHub user harshach opened a pull request: https://github.com/apache/hive/pull/348 HIVE-19519: Add Schema Registry UI to Hive MetaStore You can merge this pull request into a Git repository by running: $ git pull https://github.com/harshach/hive HIVE-19519 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/348.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #348 commit eeae927eaf38e6747c9002902149cc908ec539d5 Author: Sriharsha Chintalapani Date: 2018-05-14T02:27:59Z HIVE-19519: Add Schema Registry UI to Hive MetaStore > Add SchemaRegistry UI to Hive Metastore > > > Key: HIVE-19519 > URL: https://issues.apache.org/jira/browse/HIVE-19519 > Project: Hive > Issue Type: Sub-task >Reporter: Sriharsha Chintalapani >Assignee: Sriharsha Chintalapani >Priority: Major > Labels: pull-request-available > > Schema registry has a web UI to browse and add schemas and manage them. This > Jira is to add that UI into Hive MetaStore -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19424) NPE In MetaDataFormatters
[ https://issues.apache.org/jira/browse/HIVE-19424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alice Fan updated HIVE-19424: - Attachment: HIVE-19424.2.patch > NPE In MetaDataFormatters > - > > Key: HIVE-19424 > URL: https://issues.apache.org/jira/browse/HIVE-19424 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Metastore, Standalone Metastore >Affects Versions: 3.0.0, 2.4.0 >Reporter: BELUGA BEHR >Assignee: Alice Fan >Priority: Minor > Attachments: HIVE-19424.1.patch, HIVE-19424.2.patch > > > h2. Overview > According to the Hive Schema definition, a table's {{INPUT_FORMAT}} class can > be set to NULL. However, there are places in the code where we do not > account for this NULL value, in particular the {{MetaDataFormatters}} classes > {{TextMetaDataFormatter}} and {{JsonMetaDataFormatter}}. In addition, there > is no debug level logging in the {{MetaDataFormatters}} classes to tell me > which table in particular is causing the problem. > {code:sql|title=hive-schema-2.2.0.mysql.sql} > CREATE TABLE IF NOT EXISTS `SDS` ( > `SD_ID` bigint(20) NOT NULL, > `CD_ID` bigint(20) DEFAULT NULL, > `INPUT_FORMAT` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin > DEFAULT NULL, > `IS_COMPRESSED` bit(1) NOT NULL, > ... > {code} > {code:java|title=TextMetaDataFormatter.java} > // Not checking for a null return from getInputFormatClass > inputFormattCls = par.getInputFormatClass().getName(); > outputFormattCls = par.getOutputFormatClass().getName(); > {code} > h2. Reproduction > {code:sql} > -- MySQL Backend > update SDS SET INPUT_FORMAT=NULL WHERE SD_ID=XXX; > {code} > {code} > // Hive > SHOW TABLE EXTENDED FROM default LIKE '*'; > // HS2 Logs > [HiveServer2-Background-Pool: Thread-464]: Error running hive query: > org.apache.hive.service.cli.HiveSQLException: Error while processing > statement: FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. Exception while processing show table > status > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:400) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:238) > at > org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:89) > at > org.apache.hive.service.cli.operation.SQLOperation$3$1.run(SQLOperation.java:301) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920) > at > org.apache.hive.service.cli.operation.SQLOperation$3.run(SQLOperation.java:314) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Exception while > processing show table status > at > org.apache.hadoop.hive.ql.exec.DDLTask.showTableStatus(DDLTask.java:3025) > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:405) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:99) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2052) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1748) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1501) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1285) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1280) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:236) > ... 11 more > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.metadata.formatting.TextMetaDataFormatter.showTableStatus(TextMetaDataFormatter.java:202) > at > org.apache.hadoop.hive.ql.exec.DDLTask.showTableStatus(DDLTask.java:3020) > ... 20 more > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17989) Adding support for Schema Registry objects to standalone
[ https://issues.apache.org/jira/browse/HIVE-17989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473689#comment-16473689 ] ASF GitHub Bot commented on HIVE-17989: --- GitHub user harshach opened a pull request: https://github.com/apache/hive/pull/347 HIVE-17989: Adding support for Schema Registry objects to standalone You can merge this pull request into a Git repository by running: $ git pull https://github.com/harshach/hive standalone-metastore-v1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/347.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #347 commit 1ba9b62d9ef488355e1a97dbc7237c1472349a24 Author: Alan Gates Date: 2017-10-19T23:49:38Z HIVE-17983 Make the standalone metastore generate tarballs etc. commit b623000140605778b1173c0d9772ce051b70cbb8 Author: Alan Gates Date: 2017-10-19T23:49:38Z HIVE-17990 Add Thrift and DB storage for Schema Registry objects commit 6032eb627ede54c9ca3d79341d3fd95b6bc33863 Author: Sriharsha Chintalapani Date: 2018-02-25T05:02:06Z HIVE-18799: Add SchemaRegistry APIs to Hive MetaStore. commit bfb5b2a48204c7c43f72948fa8bf27b64ed62ad1 Author: Sriharsha Chintalapani Date: 2018-02-25T06:51:01Z Merge remote-tracking branch 'apache/standalone-metastore' into standalone-metastore commit 539a14da4027fc270f036747844f64b8f3a06d3d Author: Sriharsha Chintalapani Date: 2018-03-15T16:57:23Z removed storage module for registry commit 50951ed34304a0cb2999227adbb2e51104cdf517 Author: Sriharsha Chintalapani Date: 2018-04-19T21:41:20Z Registry code changes to use HiveMetaStore thrift api to store objects. commit 5627175f19ce22bc2e5d9fa61ddeb7dbc671ca82 Author: Sriharsha Chintalapani Date: 2018-04-25T22:14:10Z Follow-up patch for Registry APIs commit 35243aae9d638b8320dabdb7a84fece394c1d8e4 Author: Sriharsha Chintalapani Date: 2018-05-14T01:51:21Z HIVE-18799: Add SchemaRegistry APIs to Hive MetaStore > Adding support for Schema Registry objects to standalone > > > Key: HIVE-17989 > URL: https://issues.apache.org/jira/browse/HIVE-17989 > Project: Hive > Issue Type: New Feature > Components: Standalone Metastore >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Labels: pull-request-available > > The team building the [Hortonworks Schema > Registry|https://github.com/hortonworks/registry/] has expressed interest in > integrating it into the standalone metastore. This will allow users to have > one data catalog for table based and stream based data. This JIRA will serve > as an umbrella JIRA for tracking related issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17989) Adding support for Schema Registry objects to standalone
[ https://issues.apache.org/jira/browse/HIVE-17989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-17989: -- Labels: pull-request-available (was: ) > Adding support for Schema Registry objects to standalone > > > Key: HIVE-17989 > URL: https://issues.apache.org/jira/browse/HIVE-17989 > Project: Hive > Issue Type: New Feature > Components: Standalone Metastore >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Labels: pull-request-available > > The team building the [Hortonworks Schema > Registry|https://github.com/hortonworks/registry/] has expressed interest in > integrating it into the standalone metastore. This will allow users to have > one data catalog for table based and stream based data. This JIRA will serve > as an umbrella JIRA for tracking related issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HIVE-19493) VectorUDFDateDiffColCol copySelected does not handle nulls correctly
[ https://issues.apache.org/jira/browse/HIVE-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19493: Comment: was deleted (was: I'm creating a new TestVectorDateAddSub UT that generates random data/NULLs and batches with specific isRepeating patterns. When I run it without your patch I see failures which is encouraging. It is based on changes in HIVE-19384 which Teddy is code reading, so I need to get that one in first.) > VectorUDFDateDiffColCol copySelected does not handle nulls correctly > > > Key: HIVE-19493 > URL: https://issues.apache.org/jira/browse/HIVE-19493 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Attachments: HIVE-19493.01.patch, HIVE-19493.02.patch > > > The {{copySelected}} method in {{VectorUDFDateDiffColCol}} class was missed > during HIVE-18622 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19520) Add SchemaRegistry Client to Hive Meta Store
[ https://issues.apache.org/jira/browse/HIVE-19520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani reassigned HIVE-19520: - > Add SchemaRegistry Client to Hive Meta Store > > > Key: HIVE-19520 > URL: https://issues.apache.org/jira/browse/HIVE-19520 > Project: Hive > Issue Type: Sub-task >Reporter: Sriharsha Chintalapani >Assignee: Sriharsha Chintalapani >Priority: Major > > Schema registry client allows users to embed a HTTP client to access the > schemas during the runtime for streaming clients. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-6980) Drop table by using direct sql
[ https://issues.apache.org/jira/browse/HIVE-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473688#comment-16473688 ] Hive QA commented on HIVE-6980: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 8s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 25s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 29s{color} | {color:blue} standalone-metastore in master has 215 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 45s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 24s{color} | {color:red} standalone-metastore: The patch generated 6 new + 675 unchanged - 5 fixed = 681 total (was 680) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 38s{color} | {color:red} standalone-metastore generated 1 new + 214 unchanged - 1 fixed = 215 total (was 215) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 21m 24s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:standalone-metastore | | | org.apache.hadoop.hive.metastore.ObjectStore$12.run(List) concatenates strings using + in a loop At ObjectStore.java:in a loop At ObjectStore.java:[line 8161] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-10918/dev-support/hive-personality.sh | | git revision | master / f327624 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-10918/yetus/diff-checkstyle-standalone-metastore.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-10918/yetus/new-findbugs-standalone-metastore.html | | modules | C: standalone-metastore U: standalone-metastore | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-10918/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Drop table by using direct sql > -- > > Key: HIVE-6980 > URL: https://issues.apache.org/jira/browse/HIVE-6980 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 0.12.0 >Reporter: Selina Zhang >Assignee: Peter Vary >Priority: Major > Attachments: HIVE-6980.2.patch, HIVE-6980.3.patch, HIVE-6980.4.patch, > HIVE-6980.patch, drop_table_after.png, drop_table_before.png > > > Dropping table which has lots of partitions is slow. Even after applying the > patch of HIVE-6265, the drop table still takes hours (100K+ partitions). > The fixes come with two parts: > 1. use directSQL to query the partitions protect mode; > the current implementation needs to transfer the Partition object to client > and check the protect mode for each partition. I'd like to move this part of > logic to metastore. The check will be done by direct sql (if direct sql is > disabled, execut
[jira] [Assigned] (HIVE-19519) Add SchemaRegistry UI to Hive Metastore
[ https://issues.apache.org/jira/browse/HIVE-19519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani reassigned HIVE-19519: - Assignee: Sriharsha Chintalapani > Add SchemaRegistry UI to Hive Metastore > > > Key: HIVE-19519 > URL: https://issues.apache.org/jira/browse/HIVE-19519 > Project: Hive > Issue Type: Sub-task >Reporter: Sriharsha Chintalapani >Assignee: Sriharsha Chintalapani >Priority: Major > > Schema registry has a web UI to browse and add schemas and manage them. This > Jira is to add that UI into Hive MetaStore -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473684#comment-16473684 ] Sahil Takiar commented on HIVE-18533: - [~lirui] good catch. Updated. > Add option to use InProcessLauncher to submit spark jobs > > > Key: HIVE-18533 > URL: https://issues.apache.org/jira/browse/HIVE-18533 > Project: Hive > Issue Type: Improvement > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-18533.1.patch, HIVE-18533.2.patch, > HIVE-18533.3.patch, HIVE-18533.4.patch, HIVE-18533.5.patch, > HIVE-18533.6.patch, HIVE-18533.7.patch, HIVE-18533.8.patch, > HIVE-18533.9.patch, HIVE-18533.91.patch, HIVE-18533.94.patch, > HIVE-18533.95.patch, HIVE-18831.93.patch > > > See discussion in HIVE-16484 for details. > I think this will help with reducing the amount of time it takes to open a > HoS session + debuggability (no need launch a separate process to run a Spark > app). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18533: Attachment: HIVE-18533.95.patch > Add option to use InProcessLauncher to submit spark jobs > > > Key: HIVE-18533 > URL: https://issues.apache.org/jira/browse/HIVE-18533 > Project: Hive > Issue Type: Improvement > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-18533.1.patch, HIVE-18533.2.patch, > HIVE-18533.3.patch, HIVE-18533.4.patch, HIVE-18533.5.patch, > HIVE-18533.6.patch, HIVE-18533.7.patch, HIVE-18533.8.patch, > HIVE-18533.9.patch, HIVE-18533.91.patch, HIVE-18533.94.patch, > HIVE-18533.95.patch, HIVE-18831.93.patch > > > See discussion in HIVE-16484 for details. > I think this will help with reducing the amount of time it takes to open a > HoS session + debuggability (no need launch a separate process to run a Spark > app). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[ https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473683#comment-16473683 ] Hive QA commented on HIVE-17852: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12923212/HIVE-17852.01.patch {color:green}SUCCESS:{color} +1 due to 34 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 31 failed/errored test(s), 14392 tests executed *Failed tests:* {noformat} TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed out) (batchId=217) TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) (batchId=286) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join32] (batchId=88) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fouter_join_ppr] (batchId=33) org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic] (batchId=253) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1] (batchId=175) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_4] (batchId=164) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[list_bucket_dml_10] (batchId=161) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_bhif] (batchId=157) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan] (batchId=167) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acidvec_part] (batchId=170) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_part_llap_io] (batchId=170) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[special_character_in_tabnames_1] (batchId=165) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=163) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] (batchId=172) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_vector_dynpart_hashjoin_1] (batchId=172) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats] (batchId=167) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat] (batchId=183) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[list_bucket_dml_10] (batchId=182) org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut (batchId=225) org.apache.hadoop.hive.metastore.TestOldSchema.testPartitionOps (batchId=217) org.apache.hadoop.hive.metastore.TestStats.partitionedTableDeprecatedCalls (batchId=211) org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228) org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01 (batchId=228) org.apache.hadoop.hive.ql.TestAutoPurgeTables.testExternalNoAutoPurge (batchId=233) org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232) org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=236) org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel (batchId=236) org.apache.hive.jdbc.TestSSL.testSSLFetchHttp (batchId=239) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveBackKill (batchId=240) org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=304) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/10917/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10917/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10917/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 31 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12923212 - PreCommit-HIVE-Build > remove support for list bucketing "stored as directories" in 3.0 > > > Key: HIVE-17852 > URL: https://issues.apache.org/jira/browse/HIVE-17852 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-17852.01.patch > > > From the email thread: > 1) LB, when stored as directories, adds a lot of low-level complexity to Hive > tables that has to be accounted for in many places in the code where the > files are written or modified - from FSOP to ACID/replication/export. > 2) While working on some FSOP code I noticed that some of that logic is > broken - e.g. the duplicate file removal from tasks, a pretty fundamental > correctness feature in H
[jira] [Commented] (HIVE-19488) enable CM root based on db parameter, identifying a db as source of replication.
[ https://issues.apache.org/jira/browse/HIVE-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473674#comment-16473674 ] Hive QA commented on HIVE-19488: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 45s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 12s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 17s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 22s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 34s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 53s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m 59s{color} | {color:blue} standalone-metastore in master has 215 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 15s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 34s{color} | {color:red} hive-unit in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 45s{color} | {color:red} ql in the patch failed. {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 20s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 20s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 21s{color} | {color:red} itests/hive-unit: The patch generated 9 new + 592 unchanged - 0 fixed = 601 total (was 592) {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 41s{color} | {color:green} The patch ql passed checkstyle {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 21s{color} | {color:green} standalone-metastore: The patch generated 0 new + 462 unchanged - 1 fixed = 462 total (was 463) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 15s{color} | {color:red} standalone-metastore generated 1 new + 215 unchanged - 0 fixed = 216 total (was 215) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 7s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 38m 31s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:standalone-metastore | | | Possible null pointer dereference of db in org.apache.hadoop.hive.metastore.ReplChangeManager.getReplPolicyIdString(Database) Dereferenced at ReplChangeManager.java:db in org.apache.hadoop.hive.metastore.ReplChangeManager.getReplPolicyIdString(Database) Dereferenced at ReplChangeManager.java:[line 495] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-10916/dev-support/hive-personality.sh | | git revision | master / 78b4d80 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | mvninstall | http://104.198.109.242/logs//PreCommit-HIVE-Build-10916/yetus/patch-mvninstall-itests_hive-unit.txt | | mvninstall | http://104.198.109.242/logs//PreCommit-HIVE-Build-10916/yetus/patch-mvninstall-ql.txt | | checkstyle | http://104.198.109.242/logs/
[jira] [Updated] (HIVE-19517) Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate
[ https://issues.apache.org/jira/browse/HIVE-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19517: --- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to branch-3 and master. > Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate > > > Key: HIVE-19517 > URL: https://issues.apache.org/jira/browse/HIVE-19517 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-19517.1.patch, HIVE-19517.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19516) TestNegative merge_negative_5 and mm_concatenate are causing timeouts
[ https://issues.apache.org/jira/browse/HIVE-19516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473672#comment-16473672 ] Vineet Garg commented on HIVE-19516: cc [~sershe] [~ekoifman] > TestNegative merge_negative_5 and mm_concatenate are causing timeouts > - > > Key: HIVE-19516 > URL: https://issues.apache.org/jira/browse/HIVE-19516 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Vineet Garg >Priority: Major > > I haven't tried to reproduce this in isolation but it is reproducible if you > run in batch on local system > {noformat} > mvn -B test -Dtest.groups= -Dtest=TestNegativeCliDriver > -Dqfile=udf_invalid.q,authorization_uri_export.q,default_constraint_complex_default_value.q,druid_datasource2.q,check_constraint_max_length.q,view_update.q,default_partition_name.q,authorization_public_create.q,load_wrong_fileformat_rc_seq.q,default_constraint_invalid_type.q,altern1.q,describe_xpath1.q,drop_view_failure2.q,temp_table_rename.q,invalid_select_column_with_subquery.q,udf_trunc_error1.q,insert_view_failure.q,dbtxnmgr_nodbunlock.q,authorization_show_columns.q,cte_recursion.q,merge_constraint_notnull.q,clusterbyorderby.q,orc_type_promotion2.q,ctas_noperm_loc.q,udf_min.q,udf_instr_wrong_args_len.q,invalid_create_tbl2.q,part_col_complex_type.q,authorization_drop_db_empty.q,smb_mapjoin_14.q,subquery_scalar_multi_rows.q,alter_partition_coltype_2columns.q,subquery_corr_in_agg.q,insert_overwrite_notnull_constraint.q,authorization_show_grant_otheruser_wtab.q,regex_col_groupby.q,ptf_negative_DuplicateWindowAlias.q,exim_22_export_authfail.q,authorization_insert_noinspriv.q,udf_likeany_wrong1.q,groupby_key.q,ambiguous_col.q,groupby3_multi_distinct.q,authorization_alter_drop_ptn.q,invalid_cast_from_binary_5.q,show_create_table_does_not_exist.q,invalid_select_column.q,exim_20_managed_location_over_existing.q,interval_3.q,authorization_compile.q,join35.q,udf_concat_ws_wrong3.q,create_or_replace_view8.q,create_external_with_notnull_constraint.q,split_sample_out_of_range.q,materialized_view_no_transactional_rewrite.q,authorization_show_grant_otherrole.q,create_with_constraints_duplicate_name.q,invalid_stddev_samp_syntax.q,authorization_view_disable_cbo_7.q,autolocal1.q,avro_non_nullable_union.q,load_orc_negative_part.q,drop_view_failure1.q,columnstats_partlvl_invalid_values_autogather.q,exim_13_nonnative_import.q,alter_table_wrong_regex.q,udf_next_day_error_2.q,authorization_select.q,udf_trunc_error2.q,authorization_view_7.q,udf_format_number_wrong5.q,touch2.q,orc_type_promotion1.q,lateral_view_alias.q,show_tables_bad_db1.q,unset_table_property.q,alter_non_native.q,nvl_mismatch_type.q,load_orc_negative3.q,authorization_create_role_no_admin.q,invalid_distinct1.q,authorization_grant_server.q,orc_type_promotion3_acid.q,hms_using_serde_alter_table_update_columns.q,show_tables_bad1.q,macro_unused_parameter.q,drop_invalid_constraint3.q,drop_partition_filter_failure.q,char_pad_convert_fail3.q,exim_23_import_exist_authfail.q,drop_invalid_constraint4.q,authorization_create_macro1.q,archive1.q,subquery_multiple_cols_in_select.q,change_hive_hdfs_session_path.q,udf_trunc_error3.q,invalid_variance_syntax.q,authorization_truncate_2.q,invalid_avg_syntax.q,invalid_select_column_with_tablename.q,mm_truncate_cols.q,groupby_grouping_sets1.q,druid_location.q,groupby2_multi_distinct.q,authorization_sba_drop_table.q,dynamic_partitions_with_whitelist.q,compare_string_bigint_2.q,udf_greatest_error_2.q,authorization_view_6.q,show_tablestatus.q,duplicate_alias_in_transform_schema.q,create_with_fk_uk_same_tab.q,udtf_not_supported3.q,alter_table_constraint_invalid_fk_col2.q,udtf_not_supported1.q,dbtxnmgr_notableunlock.q,ptf_negative_InvalidValueBoundary.q,alter_table_constraint_duplicate_pk.q,udf_printf_wrong4.q,create_view_failure9.q,udf_elt_wrong_type.q,selectDistinctStarNeg_1.q,invalid_mapjoin1.q,load_stored_as_dirs.q,input1.q,udf_sort_array_wrong1.q,invalid_distinct2.q,invalid_select_fn.q,authorization_role_grant_otherrole.q,archive4.q,load_nonpart_authfail.q,recursive_view.q,authorization_view_disable_cbo_1.q,desc_failure4.q,create_not_acid.q,udf_sort_array_wrong3.q,char_pad_convert_fail0.q,udf_map_values_arg_type.q,alter_view_failure6_2.q,alter_partition_change_col_nonexist.q,update_non_acid_table.q,authorization_view_disable_cbo_5.q,ct_noperm_loc.q,interval_1.q,authorization_show_grant_otheruser_all.q,authorization_view_2.q,show_tables_bad2.q,groupby_rollup2.q,truncate_column_seqfile.q,create_view_failure5.q,authorization_create_view.q,ptf_window_boundaries.q,ctasnullcol.q,input_part0_neg_2.q,create_or_replace_view1.q,udf_max.q,exim_01_nonpart_over_loaded.q,msck_repair_1.q,orc_change_fileformat_acid.q,udf_nonexistent_resource.q,msck_repair_3.q,exim_19_external_over_e
[jira] [Commented] (HIVE-19517) Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate
[ https://issues.apache.org/jira/browse/HIVE-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473671#comment-16473671 ] Vihang Karajgaonkar commented on HIVE-19517: +1 > Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate > > > Key: HIVE-19517 > URL: https://issues.apache.org/jira/browse/HIVE-19517 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-19517.1.patch, HIVE-19517.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HIVE-19513) ptest version in pom.xml should be 1.0
[ https://issues.apache.org/jira/browse/HIVE-19513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar reopened HIVE-19513: turns out this patch broke the jenkins job because its classpath hardcodes 3.0 version of the jar :( > ptest version in pom.xml should be 1.0 > -- > > Key: HIVE-19513 > URL: https://issues.apache.org/jira/browse/HIVE-19513 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Minor > Attachments: HIVE-19513.01.patch > > > Jenkins job has the hardcoded API point as > {{http://:8080/hive-ptest-1.0}}. Currently the pom.xml has version as 3.0 > which creates a hive-ptest-3.0.war file which is not correct. Changing back > the version to 1.0 helps updating the ptest code without changing the jenkins > job API endpoint. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19493) VectorUDFDateDiffColCol copySelected does not handle nulls correctly
[ https://issues.apache.org/jira/browse/HIVE-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473666#comment-16473666 ] Matt McCline commented on HIVE-19493: - I'm creating a new TestVectorDateAddSub UT that generates random data/NULLs and batches with specific isRepeating patterns. When I run it without your patch I see failures which is encouraging. It is based on changes in HIVE-19384 which Teddy is code reading, so I need to get that one in first. > VectorUDFDateDiffColCol copySelected does not handle nulls correctly > > > Key: HIVE-19493 > URL: https://issues.apache.org/jira/browse/HIVE-19493 > Project: Hive > Issue Type: Bug > Components: Vectorization >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Attachments: HIVE-19493.01.patch, HIVE-19493.02.patch > > > The {{copySelected}} method in {{VectorUDFDateDiffColCol}} class was missed > during HIVE-18622 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19384) Vectorization: IfExprTimestamp* do not handle NULLs correctly
[ https://issues.apache.org/jira/browse/HIVE-19384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473665#comment-16473665 ] Teddy Choi commented on HIVE-19384: --- +1 LGTM. Thanks! > Vectorization: IfExprTimestamp* do not handle NULLs correctly > - > > Key: HIVE-19384 > URL: https://issues.apache.org/jira/browse/HIVE-19384 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Blocker > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19384.01.patch, HIVE-19384.02.patch, > HIVE-19384.04.patch, HIVE-19384.05.patch > > > HIVE-18622: "Vectorization: IF Statements, Comparisons, and more do not > handle NULLs correctly" didn't quite fix the IfExprTimestamp* classes > right > {noformat} > // Carefully handle NULLs... > outputColVector.noNulls = false;{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[ https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter updated HIVE-19307: Status: Patch Available (was: Open) > Support ArrowOutputStream in LlapOutputFormatService > > > Key: HIVE-19307 > URL: https://issues.apache.org/jira/browse/HIVE-19307 > Project: Hive > Issue Type: Task > Components: llap >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch, > HIVE-19307.5.patch, HIVE-19307.6.patch, HIVE-19307.7.patch > > > Support pushing arrow batches through > org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[ https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter updated HIVE-19307: Attachment: HIVE-19307.7.patch > Support ArrowOutputStream in LlapOutputFormatService > > > Key: HIVE-19307 > URL: https://issues.apache.org/jira/browse/HIVE-19307 > Project: Hive > Issue Type: Task > Components: llap >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch, > HIVE-19307.5.patch, HIVE-19307.6.patch, HIVE-19307.7.patch > > > Support pushing arrow batches through > org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[ https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter updated HIVE-19307: Status: Open (was: Patch Available) > Support ArrowOutputStream in LlapOutputFormatService > > > Key: HIVE-19307 > URL: https://issues.apache.org/jira/browse/HIVE-19307 > Project: Hive > Issue Type: Task > Components: llap >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch, > HIVE-19307.5.patch, HIVE-19307.6.patch > > > Support pushing arrow batches through > org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[ https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473655#comment-16473655 ] Hive QA commented on HIVE-19307: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12923187/HIVE-19307.6.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 33 failed/errored test(s), 13595 tests executed *Failed tests:* {noformat} TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=94) [infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,scriptfile1.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q] TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=95) [nopart_insert.q,insert_into_with_schema.q,input41.q,having1.q,create_table_failure3.q,default_constraint_invalid_default_value.q,database_drop_not_empty_restrict.q,windowing_after_orderby.q,orderbysortby.q,subquery_select_distinct2.q,authorization_uri_alterpart_loc.q,udf_last_day_error_1.q,constraint_duplicate_name.q,create_table_failure4.q,alter_tableprops_external_with_notnull_constraint.q,semijoin5.q,udf_format_number_wrong4.q,deletejar.q,exim_11_nonpart_noncompat_sorting.q,show_tables_bad_db2.q,drop_func_nonexistent.q,alter_table_non_partitioned_table_cascade.q,check_constraint_subquery.q,load_wrong_fileformat.q,check_constraint_udtf.q,lockneg_try_db_lock_conflict.q,udf_field_wrong_args_len.q,create_table_failure2.q,create_with_fk_constraints_enforced.q,groupby2_map_skew_multi_distinct.q,mm_update.q,authorization_update_noupdatepriv.q,show_columns2.q,authorization_insert_noselectpriv.q,orc_replace_columns3_acid.q,compare_double_bigint.q,authorization_set_nonexistent_conf.q,alter_rename_partition_failure3.q,split_sample_wrong_format2.q,create_with_fk_pk_same_tab.q,compare_double_bigint_2.q,authorization_show_roles_no_admin.q,materialized_view_authorization_rebuild_no_grant.q,unionLimit.q,authorization_revoke_table_fail2.q,duplicate_insert3.q,authorization_desc_table_nosel.q,stats_noscan_non_native.q,orc_change_serde_acid.q,create_or_replace_view7.q,exim_07_nonpart_noncompat_ifof.q,create_with_unique_constraints_enforced.q,udf_concat_ws_wrong2.q,fileformat_bad_class.q,merge_negative_2.q,exim_15_part_nonpart.q,authorization_not_owner_drop_view.q,external1.q,authorization_uri_insert.q,create_with_fk_wrong_ref.q,columnstats_tbllvl_incorrect_column.q,authorization_show_parts_nosel.q,authorization_not_owner_drop_tab.q,external2.q,authorization_deletejar.q,temp_table_create_like_partitions.q,udf_greatest_error_1.q,ptf_negative_AggrFuncsWithNoGBYNoPartDef.q,alter_view_as_select_not_exist.q,touch1.q,groupby3_map_skew_multi_distinct.q,insert_into_notnull_constraint.q,exchange_partition_neg_partition_missing.q,groupby_cube_multi_gby.q,columnstats_tbllvl.q,drop_invalid_constraint2.q,alter_table_add_partition.q,update_not_acid.q,archive5.q,alter_table_constraint_invalid_pk_col.q,ivyDownload.q,udf_instr_wrong_type.q,bad_sample_clause.q,authorization_not_owner_drop_tab2.q,authorization_alter_db_owner.q,show_columns1.q,orc_type_promotion3.q,create_view_failure8.q,strict_join.q,udf_add_months_error_1.q,groupby_cube2.q,groupby_cube1.q,groupby_rollup1.q,genericFileFormat.q,invalid_cast_from_binary_4.q,drop_invalid_constraint1.q,serde_regex.q,show_partitions1.q,check_constraint_nonboolean_expr.q,invalid_cast_from_binary_6.q,create_with_multi_pk_constraint.q,udf_field_wrong_type.q,groupby_grouping_sets4.q,groupby_grouping_sets3.q,insertsel_fail.q,udf_locate_wrong_type.q,orc_type_promotion1_acid.q,set_table_property.q,create_or_replace_view2.q,groupby_grouping_sets2.q,alter_view_failure.q,distinct_windowing_failure1.q,invalid_t_alter2.q,alter_table_constraint_invalid_fk_col1.q,invalid_varchar_length_2.q,authorization_show_grant_otheruser_alltabs.q,subquery_windowing_corr.q,compact_non_acid_table.q,authorization_view_4.q,authorization_disallow_transform.q,materialized_view_authorization_rebuild_other.q,authorization_fail_4.q,dbtxnmgr_nodblock.q,set_hiveconf_internal_variable1.q,input_part0_neg.q,udf_printf_wrong3.q,load_orc_negative2.q,druid_buckets.q,archive2.q,authorization_addjar.q,invalid_sum_syntax.q,insert_into_with_schema1.q,udf_add_months_error_2.q,dyn_part_max_per_node.q,authorization_revoke_table_fail1.q,udf_printf_wrong2.q,archive_multi3.q,udf_printf_wrong1.q,subquery_subquery_chain.q,authorization_view_disable_cbo_4.q,no_matching_udf.q,create_view_failure7.q,drop_native_udf.q,truncate_column_list_bucketing.q,authorization_uri_add_partition.q,authorization_view_disable_cbo_3.q,bad_exec_hooks.q,authorization_view_disable_cbo_2.q,fetchtask_ioexception.q,char_pad_convert_fail2.q,authorization_set_role_neg1.q,serde_rege
[jira] [Commented] (HIVE-19517) Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate
[ https://issues.apache.org/jira/browse/HIVE-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473652#comment-16473652 ] Vineet Garg commented on HIVE-19517: Thanks [~vihangk1]. I have updated the patch. It indeed skips the test. > Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate > > > Key: HIVE-19517 > URL: https://issues.apache.org/jira/browse/HIVE-19517 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-19517.1.patch, HIVE-19517.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19517) Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate
[ https://issues.apache.org/jira/browse/HIVE-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19517: --- Status: Patch Available (was: Open) > Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate > > > Key: HIVE-19517 > URL: https://issues.apache.org/jira/browse/HIVE-19517 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-19517.1.patch, HIVE-19517.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19517) Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate
[ https://issues.apache.org/jira/browse/HIVE-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19517: --- Attachment: HIVE-19517.2.patch > Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate > > > Key: HIVE-19517 > URL: https://issues.apache.org/jira/browse/HIVE-19517 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-19517.1.patch, HIVE-19517.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19517) Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate
[ https://issues.apache.org/jira/browse/HIVE-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473649#comment-16473649 ] Vihang Karajgaonkar edited comment on HIVE-19517 at 5/13/18 10:33 PM: -- You can exclude q-file by adding them to the exclude list like seen here https://github.com/apache/hive/blob/master/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CliConfigs.java#L362 Quick way to test would be to try to run them locally using {{TestNegativeCliDriver}} and you see them not being run. The {{Tests run}} shown by maven should show 0 and the {{TEST*.xml}} file will not list them. was (Author: vihangk1): You can exclude q-file by adding them to the exclude list like seen here https://github.com/apache/hive/blob/master/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CliConfigs.java#L362 > Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate > > > Key: HIVE-19517 > URL: https://issues.apache.org/jira/browse/HIVE-19517 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-19517.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19517) Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate
[ https://issues.apache.org/jira/browse/HIVE-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473649#comment-16473649 ] Vihang Karajgaonkar commented on HIVE-19517: You can exclude q-file by adding them to the exclude list like seen here https://github.com/apache/hive/blob/master/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CliConfigs.java#L362 > Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate > > > Key: HIVE-19517 > URL: https://issues.apache.org/jira/browse/HIVE-19517 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-19517.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-19513) ptest version in pom.xml should be 1.0
[ https://issues.apache.org/jira/browse/HIVE-19513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar resolved HIVE-19513. Resolution: Fixed merged into master. Thanks [~stakiar] > ptest version in pom.xml should be 1.0 > -- > > Key: HIVE-19513 > URL: https://issues.apache.org/jira/browse/HIVE-19513 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Minor > Attachments: HIVE-19513.01.patch > > > Jenkins job has the hardcoded API point as > {{http://:8080/hive-ptest-1.0}}. Currently the pom.xml has version as 3.0 > which creates a hive-ptest-3.0.war file which is not correct. Changing back > the version to 1.0 helps updating the ptest code without changing the jenkins > job API endpoint. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19384) Vectorization: IfExprTimestamp* do not handle NULLs correctly
[ https://issues.apache.org/jira/browse/HIVE-19384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19384: Status: Patch Available (was: In Progress) > Vectorization: IfExprTimestamp* do not handle NULLs correctly > - > > Key: HIVE-19384 > URL: https://issues.apache.org/jira/browse/HIVE-19384 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Blocker > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19384.01.patch, HIVE-19384.02.patch, > HIVE-19384.04.patch, HIVE-19384.05.patch > > > HIVE-18622: "Vectorization: IF Statements, Comparisons, and more do not > handle NULLs correctly" didn't quite fix the IfExprTimestamp* classes > right > {noformat} > // Carefully handle NULLs... > outputColVector.noNulls = false;{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[ https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473645#comment-16473645 ] Hive QA commented on HIVE-19307: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 42s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 16s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 23s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 1s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 33s{color} | {color:blue} common in master has 62 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 5s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 43s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 20s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 20s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 46s{color} | {color:red} ql: The patch generated 3 new + 565 unchanged - 14 fixed = 568 total (was 579) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 57s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 27m 32s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-10904/dev-support/hive-personality.sh | | git revision | master / d71032e | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-10904/yetus/diff-checkstyle-ql.txt | | modules | C: common ql U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-10904/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Support ArrowOutputStream in LlapOutputFormatService > > > Key: HIVE-19307 > URL: https://issues.apache.org/jira/browse/HIVE-19307 > Project: Hive > Issue Type: Task > Components: llap >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch, > HIVE-19307.5.patch, HIVE-19307.6.patch > > > Support pushing arrow batches through > org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19384) Vectorization: IfExprTimestamp* do not handle NULLs correctly
[ https://issues.apache.org/jira/browse/HIVE-19384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19384: Status: In Progress (was: Patch Available) > Vectorization: IfExprTimestamp* do not handle NULLs correctly > - > > Key: HIVE-19384 > URL: https://issues.apache.org/jira/browse/HIVE-19384 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Blocker > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19384.01.patch, HIVE-19384.02.patch, > HIVE-19384.04.patch, HIVE-19384.05.patch > > > HIVE-18622: "Vectorization: IF Statements, Comparisons, and more do not > handle NULLs correctly" didn't quite fix the IfExprTimestamp* classes > right > {noformat} > // Carefully handle NULLs... > outputColVector.noNulls = false;{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19384) Vectorization: IfExprTimestamp* do not handle NULLs correctly
[ https://issues.apache.org/jira/browse/HIVE-19384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-19384: Attachment: HIVE-19384.05.patch > Vectorization: IfExprTimestamp* do not handle NULLs correctly > - > > Key: HIVE-19384 > URL: https://issues.apache.org/jira/browse/HIVE-19384 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Blocker > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19384.01.patch, HIVE-19384.02.patch, > HIVE-19384.04.patch, HIVE-19384.05.patch > > > HIVE-18622: "Vectorization: IF Statements, Comparisons, and more do not > handle NULLs correctly" didn't quite fix the IfExprTimestamp* classes > right > {noformat} > // Carefully handle NULLs... > outputColVector.noNulls = false;{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19506) Test suites timing out
[ https://issues.apache.org/jira/browse/HIVE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19506: --- Fix Version/s: 3.0.0 > Test suites timing out > -- > > Key: HIVE-19506 > URL: https://issues.apache.org/jira/browse/HIVE-19506 > Project: Hive > Issue Type: Bug >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Blocker > Fix For: 3.0.0 > > Attachments: HIVE-19506.1.patch, HIVE-19506.2.patch, > HIVE-19506.3.patch > > > Currently a lot of test suites are timing out consistently: > TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed > out) > TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed > out) > TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) > TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed > out) > TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed > out) > TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-19506) Test suites timing out
[ https://issues.apache.org/jira/browse/HIVE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg resolved HIVE-19506. Resolution: Fixed > Test suites timing out > -- > > Key: HIVE-19506 > URL: https://issues.apache.org/jira/browse/HIVE-19506 > Project: Hive > Issue Type: Bug >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Blocker > Fix For: 3.0.0 > > Attachments: HIVE-19506.1.patch, HIVE-19506.2.patch, > HIVE-19506.3.patch > > > Currently a lot of test suites are timing out consistently: > TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed > out) > TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed > out) > TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) > TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed > out) > TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed > out) > TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19495) Arrow SerDe itest failure
[ https://issues.apache.org/jira/browse/HIVE-19495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473642#comment-16473642 ] Hive QA commented on HIVE-19495: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12923183/HIVE-19495.1.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 24 failed/errored test(s), 13606 tests executed *Failed tests:* {noformat} TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=95) [nopart_insert.q,insert_into_with_schema.q,input41.q,having1.q,create_table_failure3.q,default_constraint_invalid_default_value.q,database_drop_not_empty_restrict.q,windowing_after_orderby.q,orderbysortby.q,subquery_select_distinct2.q,authorization_uri_alterpart_loc.q,udf_last_day_error_1.q,constraint_duplicate_name.q,create_table_failure4.q,alter_tableprops_external_with_notnull_constraint.q,semijoin5.q,udf_format_number_wrong4.q,deletejar.q,exim_11_nonpart_noncompat_sorting.q,show_tables_bad_db2.q,drop_func_nonexistent.q,alter_table_non_partitioned_table_cascade.q,check_constraint_subquery.q,load_wrong_fileformat.q,check_constraint_udtf.q,lockneg_try_db_lock_conflict.q,udf_field_wrong_args_len.q,create_table_failure2.q,create_with_fk_constraints_enforced.q,groupby2_map_skew_multi_distinct.q,mm_update.q,authorization_update_noupdatepriv.q,show_columns2.q,authorization_insert_noselectpriv.q,orc_replace_columns3_acid.q,compare_double_bigint.q,authorization_set_nonexistent_conf.q,alter_rename_partition_failure3.q,split_sample_wrong_format2.q,create_with_fk_pk_same_tab.q,compare_double_bigint_2.q,authorization_show_roles_no_admin.q,materialized_view_authorization_rebuild_no_grant.q,unionLimit.q,authorization_revoke_table_fail2.q,duplicate_insert3.q,authorization_desc_table_nosel.q,stats_noscan_non_native.q,orc_change_serde_acid.q,create_or_replace_view7.q,exim_07_nonpart_noncompat_ifof.q,create_with_unique_constraints_enforced.q,udf_concat_ws_wrong2.q,fileformat_bad_class.q,merge_negative_2.q,exim_15_part_nonpart.q,authorization_not_owner_drop_view.q,external1.q,authorization_uri_insert.q,create_with_fk_wrong_ref.q,columnstats_tbllvl_incorrect_column.q,authorization_show_parts_nosel.q,authorization_not_owner_drop_tab.q,external2.q,authorization_deletejar.q,temp_table_create_like_partitions.q,udf_greatest_error_1.q,ptf_negative_AggrFuncsWithNoGBYNoPartDef.q,alter_view_as_select_not_exist.q,touch1.q,groupby3_map_skew_multi_distinct.q,insert_into_notnull_constraint.q,exchange_partition_neg_partition_missing.q,groupby_cube_multi_gby.q,columnstats_tbllvl.q,drop_invalid_constraint2.q,alter_table_add_partition.q,update_not_acid.q,archive5.q,alter_table_constraint_invalid_pk_col.q,ivyDownload.q,udf_instr_wrong_type.q,bad_sample_clause.q,authorization_not_owner_drop_tab2.q,authorization_alter_db_owner.q,show_columns1.q,orc_type_promotion3.q,create_view_failure8.q,strict_join.q,udf_add_months_error_1.q,groupby_cube2.q,groupby_cube1.q,groupby_rollup1.q,genericFileFormat.q,invalid_cast_from_binary_4.q,drop_invalid_constraint1.q,serde_regex.q,show_partitions1.q,check_constraint_nonboolean_expr.q,invalid_cast_from_binary_6.q,create_with_multi_pk_constraint.q,udf_field_wrong_type.q,groupby_grouping_sets4.q,groupby_grouping_sets3.q,insertsel_fail.q,udf_locate_wrong_type.q,orc_type_promotion1_acid.q,set_table_property.q,create_or_replace_view2.q,groupby_grouping_sets2.q,alter_view_failure.q,distinct_windowing_failure1.q,invalid_t_alter2.q,alter_table_constraint_invalid_fk_col1.q,invalid_varchar_length_2.q,authorization_show_grant_otheruser_alltabs.q,subquery_windowing_corr.q,compact_non_acid_table.q,authorization_view_4.q,authorization_disallow_transform.q,materialized_view_authorization_rebuild_other.q,authorization_fail_4.q,dbtxnmgr_nodblock.q,set_hiveconf_internal_variable1.q,input_part0_neg.q,udf_printf_wrong3.q,load_orc_negative2.q,druid_buckets.q,archive2.q,authorization_addjar.q,invalid_sum_syntax.q,insert_into_with_schema1.q,udf_add_months_error_2.q,dyn_part_max_per_node.q,authorization_revoke_table_fail1.q,udf_printf_wrong2.q,archive_multi3.q,udf_printf_wrong1.q,subquery_subquery_chain.q,authorization_view_disable_cbo_4.q,no_matching_udf.q,create_view_failure7.q,drop_native_udf.q,truncate_column_list_bucketing.q,authorization_uri_add_partition.q,authorization_view_disable_cbo_3.q,bad_exec_hooks.q,authorization_view_disable_cbo_2.q,fetchtask_ioexception.q,char_pad_convert_fail2.q,authorization_set_role_neg1.q,serde_regex3.q,authorization_delete_nodeletepriv.q,materialized_view_delete.q,create_or_replace_view6.q,bucket_mapjoin_wrong_table_metadata_2.q,udf_sort_array_by_wrong2.q,local_mapred_error_cache.q,alter_external_acid.q,mm_concatenate.q,authorization_fail_3.q,set_hiveconf_internal_variable0.q,udf_last_day_error_2.q,alter_table_constraint_invalid_ref.q,create_tabl
[jira] [Commented] (HIVE-19517) Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate
[ https://issues.apache.org/jira/browse/HIVE-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473639#comment-16473639 ] Vineet Garg commented on HIVE-19517: [~jcamachorodriguez] Can you take a quick look? I couldn't figure out how to disable these negative tests so i have deleted them for now. > Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate > > > Key: HIVE-19517 > URL: https://issues.apache.org/jira/browse/HIVE-19517 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-19517.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19517) Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate
[ https://issues.apache.org/jira/browse/HIVE-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19517: --- Attachment: HIVE-19517.1.patch > Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate > > > Key: HIVE-19517 > URL: https://issues.apache.org/jira/browse/HIVE-19517 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-19517.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19517) Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate
[ https://issues.apache.org/jira/browse/HIVE-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-19517: -- > Disable/delete TestNegativeCliDriver merge_negative_5 and mm_concatenate > > > Key: HIVE-19517 > URL: https://issues.apache.org/jira/browse/HIVE-19517 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19506) Test suites timing out
[ https://issues.apache.org/jira/browse/HIVE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473636#comment-16473636 ] Vineet Garg commented on HIVE-19506: Found negative tests which are causing time out (HIVE-19516) > Test suites timing out > -- > > Key: HIVE-19506 > URL: https://issues.apache.org/jira/browse/HIVE-19506 > Project: Hive > Issue Type: Bug >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Blocker > Attachments: HIVE-19506.1.patch, HIVE-19506.2.patch, > HIVE-19506.3.patch > > > Currently a lot of test suites are timing out consistently: > TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed > out) > TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed > out) > TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) > TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed > out) > TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed > out) > TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19509) Disable tests that are failing continuously
[ https://issues.apache.org/jira/browse/HIVE-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473635#comment-16473635 ] Sahil Takiar commented on HIVE-19509: - Have a fix for {{TestRpc.testServerPort}} HIVE-19515 > Disable tests that are failing continuously > --- > > Key: HIVE-19509 > URL: https://issues.apache.org/jira/browse/HIVE-19509 > Project: Hive > Issue Type: Task > Components: Testing Infrastructure, Tests >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19509.patch > > > As per discussion in mailing list. We will bring the failure count to zero > and be less tolerant with ptest runs that are not clean. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19515) TestRpc.testServerPort is consistently failing
[ https://issues.apache.org/jira/browse/HIVE-19515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-19515: Status: Patch Available (was: Open) > TestRpc.testServerPort is consistently failing > -- > > Key: HIVE-19515 > URL: https://issues.apache.org/jira/browse/HIVE-19515 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-19515.1.patch > > > {{TestRpc.testServerPort}} is consistently failing due to HIVE-17838 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19515) TestRpc.testServerPort is consistently failing
[ https://issues.apache.org/jira/browse/HIVE-19515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-19515: Attachment: HIVE-19515.1.patch > TestRpc.testServerPort is consistently failing > -- > > Key: HIVE-19515 > URL: https://issues.apache.org/jira/browse/HIVE-19515 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-19515.1.patch > > > {{TestRpc.testServerPort}} is consistently failing due to HIVE-17838 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19515) TestRpc.testServerPort is consistently failing
[ https://issues.apache.org/jira/browse/HIVE-19515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473633#comment-16473633 ] Sahil Takiar commented on HIVE-19515: - [~vihangk1] could you take a look? > TestRpc.testServerPort is consistently failing > -- > > Key: HIVE-19515 > URL: https://issues.apache.org/jira/browse/HIVE-19515 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-19515.1.patch > > > {{TestRpc.testServerPort}} is consistently failing due to HIVE-17838 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19515) TestRpc.testServerPort is consistently failing
[ https://issues.apache.org/jira/browse/HIVE-19515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar reassigned HIVE-19515: --- > TestRpc.testServerPort is consistently failing > -- > > Key: HIVE-19515 > URL: https://issues.apache.org/jira/browse/HIVE-19515 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > > {{TestRpc.testServerPort}} is consistently failing due to HIVE-17838 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19513) ptest version in pom.xml should be 1.0
[ https://issues.apache.org/jira/browse/HIVE-19513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473631#comment-16473631 ] Sahil Takiar commented on HIVE-19513: - +1 LGTM. Noticed the same thing when deploying some ptest changes in the past, but never got around to filing a JIRA. Thanks for doing this! > ptest version in pom.xml should be 1.0 > -- > > Key: HIVE-19513 > URL: https://issues.apache.org/jira/browse/HIVE-19513 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Minor > Attachments: HIVE-19513.01.patch > > > Jenkins job has the hardcoded API point as > {{http://:8080/hive-ptest-1.0}}. Currently the pom.xml has version as 3.0 > which creates a hive-ptest-3.0.war file which is not correct. Changing back > the version to 1.0 helps updating the ptest code without changing the jenkins > job API endpoint. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19495) Arrow SerDe itest failure
[ https://issues.apache.org/jira/browse/HIVE-19495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473620#comment-16473620 ] Hive QA commented on HIVE-19495: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 10s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 6s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 46s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 10s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 25s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 5s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 49s{color} | {color:red} ql: The patch generated 555 new + 123 unchanged - 460 fixed = 678 total (was 583) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 23m 37s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-10903/dev-support/hive-personality.sh | | git revision | master / d71032e | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-10903/yetus/diff-checkstyle-ql.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-10903/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Arrow SerDe itest failure > - > > Key: HIVE-19495 > URL: https://issues.apache.org/jira/browse/HIVE-19495 > Project: Hive > Issue Type: Sub-task > Components: Serializers/Deserializers >Reporter: Eric Wohlstadter >Assignee: Teddy Choi >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19495.1.patch > > > "You tried to write a Bit type when you are using a ValueWriter of type > NullableMapWriter." -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-6980) Drop table by using direct sql
[ https://issues.apache.org/jira/browse/HIVE-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-6980: - Attachment: HIVE-6980.4.patch > Drop table by using direct sql > -- > > Key: HIVE-6980 > URL: https://issues.apache.org/jira/browse/HIVE-6980 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 0.12.0 >Reporter: Selina Zhang >Assignee: Peter Vary >Priority: Major > Attachments: HIVE-6980.2.patch, HIVE-6980.3.patch, HIVE-6980.4.patch, > HIVE-6980.patch, drop_table_after.png, drop_table_before.png > > > Dropping table which has lots of partitions is slow. Even after applying the > patch of HIVE-6265, the drop table still takes hours (100K+ partitions). > The fixes come with two parts: > 1. use directSQL to query the partitions protect mode; > the current implementation needs to transfer the Partition object to client > and check the protect mode for each partition. I'd like to move this part of > logic to metastore. The check will be done by direct sql (if direct sql is > disabled, execute the same logic in the ObjectStore); > 2. use directSQL to drop partitions for table; > there maybe two solutions here: > 1. add "DELETE CASCADE" in the schema. In this way we only need to delete > entries from partitions table use direct sql. May need to change > datanucleus.deletionPolicy = DataNucleus. > 2. clean up the dependent tables by issue DELETE statement. This also needs > to turn on datanucleus.query.sql.allowAll > Both of above solutions should be able to fix the problem. The DELETE CASCADE > has to change schemas and prepare upgrade scripts. The second solutions added > maintenance cost if new tables added in the future releases. > Please advice. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-13745) UDF current_date、current_timestamp、unix_timestamp NPE
[ https://issues.apache.org/jira/browse/HIVE-13745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473615#comment-16473615 ] Hive QA commented on HIVE-13745: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12923177/HIVE-13745.2-branch-2.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 10683 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries] (batchId=227) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_tableproperty_optimize] (batchId=22) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explaindenpendencydiffengs] (batchId=38) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=142) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic] (batchId=139) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[table_nonprintable] (batchId=140) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid] (batchId=158) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats] (batchId=153) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_parquet_types] (batchId=155) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[merge_negative_5] (batchId=88) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs] (batchId=115) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_input_format_excludes] (batchId=117) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_ptf] (batchId=125) org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=176) org.apache.hive.jdbc.TestMultiSessionsHS2WithLocalClusterSpark.org.apache.hive.jdbc.TestMultiSessionsHS2WithLocalClusterSpark (batchId=224) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/10901/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10901/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10901/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 15 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12923177 - PreCommit-HIVE-Build > UDF current_date、current_timestamp、unix_timestamp NPE > - > > Key: HIVE-13745 > URL: https://issues.apache.org/jira/browse/HIVE-13745 > Project: Hive > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Biao Wu >Assignee: Yongzhi Chen >Priority: Major > Attachments: HIVE-13745.1.patch, HIVE-13745.2-branch-2.patch, > HIVE-13745.patch > > > NullPointerException when current_date is used in mapreduce -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19506) Test suites timing out
[ https://issues.apache.org/jira/browse/HIVE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473613#comment-16473613 ] Vineet Garg commented on HIVE-19506: Pushed this to master. > Test suites timing out > -- > > Key: HIVE-19506 > URL: https://issues.apache.org/jira/browse/HIVE-19506 > Project: Hive > Issue Type: Bug >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Blocker > Attachments: HIVE-19506.1.patch, HIVE-19506.2.patch, > HIVE-19506.3.patch > > > Currently a lot of test suites are timing out consistently: > TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed > out) > TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed > out) > TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) > TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed > out) > TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed > out) > TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19506) Test suites timing out
[ https://issues.apache.org/jira/browse/HIVE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-19506: -- Assignee: Vineet Garg > Test suites timing out > -- > > Key: HIVE-19506 > URL: https://issues.apache.org/jira/browse/HIVE-19506 > Project: Hive > Issue Type: Bug >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Blocker > Attachments: HIVE-19506.1.patch, HIVE-19506.2.patch, > HIVE-19506.3.patch > > > Currently a lot of test suites are timing out consistently: > TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed > out) > TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed > out) > TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) > TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed > out) > TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed > out) > TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19435) Incremental replication cause data loss if a table is dropped followed by create and insert-into with different partition type.
[ https://issues.apache.org/jira/browse/HIVE-19435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473612#comment-16473612 ] ASF GitHub Bot commented on HIVE-19435: --- Github user sankarh closed the pull request at: https://github.com/apache/hive/pull/343 > Incremental replication cause data loss if a table is dropped followed by > create and insert-into with different partition type. > --- > > Key: HIVE-19435 > URL: https://issues.apache.org/jira/browse/HIVE-19435 > Project: Hive > Issue Type: Bug > Components: HiveServer2, repl >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, pull-request-available, replication > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19435.01-branch-3.patch, HIVE-19435.01.patch, > HIVE-19435.02.patch, HIVE-19435.03.patch > > > If the incremental dump have drop of partitioned table followed by > create/insert on non-partitioned table with same name, doesn't replicate the > data. Explained below. > Let's say we have a partitioned table T1 which was already replicated to > target. > DROP_TABLE(T1)->CREATE_TABLE(T1) (Non-partitioned) -> INSERT(T1)(10) > After REPL LOAD, T1 doesn't have any data. > Same is valid for non-partitioned to partitioned and partition spec mismatch > case as well. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19476) Fix failures in TestReplicationScenariosAcidTables, TestReplicationOnHDFSEncryptedZones and TestCopyUtils
[ https://issues.apache.org/jira/browse/HIVE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473611#comment-16473611 ] ASF GitHub Bot commented on HIVE-19476: --- Github user sankarh closed the pull request at: https://github.com/apache/hive/pull/344 > Fix failures in TestReplicationScenariosAcidTables, > TestReplicationOnHDFSEncryptedZones and TestCopyUtils > - > > Key: HIVE-19476 > URL: https://issues.apache.org/jira/browse/HIVE-19476 > Project: Hive > Issue Type: Bug > Components: HiveServer2, repl >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: Tests, pull-request-available > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19476.01-branch-3.patch, HIVE-19476.01.patch, > HIVE-19476.02.patch > > > TestReplicationScenariosAcidTables broken due to HIVE-17657. > cc [~sershe] > TestReplicationOnHDFSEncryptedZones and TestCopyUtils are failing due to > auto-stats gathered which thrown exception irrelevant to theses tests. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19476) Fix failures in TestReplicationScenariosAcidTables, TestReplicationOnHDFSEncryptedZones and TestCopyUtils
[ https://issues.apache.org/jira/browse/HIVE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-19476: Resolution: Fixed Status: Resolved (was: Patch Available) > Fix failures in TestReplicationScenariosAcidTables, > TestReplicationOnHDFSEncryptedZones and TestCopyUtils > - > > Key: HIVE-19476 > URL: https://issues.apache.org/jira/browse/HIVE-19476 > Project: Hive > Issue Type: Bug > Components: HiveServer2, repl >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: Tests, pull-request-available > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19476.01-branch-3.patch, HIVE-19476.01.patch, > HIVE-19476.02.patch > > > TestReplicationScenariosAcidTables broken due to HIVE-17657. > cc [~sershe] > TestReplicationOnHDFSEncryptedZones and TestCopyUtils are failing due to > auto-stats gathered which thrown exception irrelevant to theses tests. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19476) Fix failures in TestReplicationScenariosAcidTables, TestReplicationOnHDFSEncryptedZones and TestCopyUtils
[ https://issues.apache.org/jira/browse/HIVE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473610#comment-16473610 ] Sankar Hariappan commented on HIVE-19476: - Test failures as seen in [http://104.198.109.242/logs/PreCommit-HIVE-Build-10900/failed/] is irrelevant to the patch. Patch committed to branch-3. > Fix failures in TestReplicationScenariosAcidTables, > TestReplicationOnHDFSEncryptedZones and TestCopyUtils > - > > Key: HIVE-19476 > URL: https://issues.apache.org/jira/browse/HIVE-19476 > Project: Hive > Issue Type: Bug > Components: HiveServer2, repl >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: Tests, pull-request-available > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19476.01-branch-3.patch, HIVE-19476.01.patch, > HIVE-19476.02.patch > > > TestReplicationScenariosAcidTables broken due to HIVE-17657. > cc [~sershe] > TestReplicationOnHDFSEncryptedZones and TestCopyUtils are failing due to > auto-stats gathered which thrown exception irrelevant to theses tests. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19506) Test suites timing out
[ https://issues.apache.org/jira/browse/HIVE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19506: --- Status: Open (was: Patch Available) > Test suites timing out > -- > > Key: HIVE-19506 > URL: https://issues.apache.org/jira/browse/HIVE-19506 > Project: Hive > Issue Type: Bug >Reporter: Vineet Garg >Priority: Blocker > Attachments: HIVE-19506.1.patch, HIVE-19506.2.patch, > HIVE-19506.3.patch > > > Currently a lot of test suites are timing out consistently: > TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed > out) > TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed > out) > TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) > TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed > out) > TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed > out) > TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19506) Test suites timing out
[ https://issues.apache.org/jira/browse/HIVE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19506: --- Attachment: HIVE-19506.3.patch > Test suites timing out > -- > > Key: HIVE-19506 > URL: https://issues.apache.org/jira/browse/HIVE-19506 > Project: Hive > Issue Type: Bug >Reporter: Vineet Garg >Priority: Blocker > Attachments: HIVE-19506.1.patch, HIVE-19506.2.patch, > HIVE-19506.3.patch > > > Currently a lot of test suites are timing out consistently: > TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed > out) > TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed > out) > TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) > TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed > out) > TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed > out) > TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[ https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-17852: Attachment: HIVE-17852.01.patch > remove support for list bucketing "stored as directories" in 3.0 > > > Key: HIVE-17852 > URL: https://issues.apache.org/jira/browse/HIVE-17852 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-17852.01.patch > > > From the email thread: > 1) LB, when stored as directories, adds a lot of low-level complexity to Hive > tables that has to be accounted for in many places in the code where the > files are written or modified - from FSOP to ACID/replication/export. > 2) While working on some FSOP code I noticed that some of that logic is > broken - e.g. the duplicate file removal from tasks, a pretty fundamental > correctness feature in Hive, may be broken. LB also doesn’t appear to be > compatible with e.g. regular bucketing. > 3) The feature hasn’t seen development activity in a while; it also doesn’t > appear to be used a lot. > Keeping with the theme of cleaning up “legacy” code for 3.0, I was proposing > we remove it. > (2) also suggested that, if needed, it might be easier to implement similar > functionality by adding some flexibility to partitions (which LB directories > look like anyway); that would also keep the logic on a higher level of > abstraction (split generation, partition pruning) as opposed to many > low-level places like FSOP, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[ https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-17852: Attachment: (was: HIVE-17852.01.patch) > remove support for list bucketing "stored as directories" in 3.0 > > > Key: HIVE-17852 > URL: https://issues.apache.org/jira/browse/HIVE-17852 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-17852.01.patch > > > From the email thread: > 1) LB, when stored as directories, adds a lot of low-level complexity to Hive > tables that has to be accounted for in many places in the code where the > files are written or modified - from FSOP to ACID/replication/export. > 2) While working on some FSOP code I noticed that some of that logic is > broken - e.g. the duplicate file removal from tasks, a pretty fundamental > correctness feature in Hive, may be broken. LB also doesn’t appear to be > compatible with e.g. regular bucketing. > 3) The feature hasn’t seen development activity in a while; it also doesn’t > appear to be used a lot. > Keeping with the theme of cleaning up “legacy” code for 3.0, I was proposing > we remove it. > (2) also suggested that, if needed, it might be easier to implement similar > functionality by adding some flexibility to partitions (which LB directories > look like anyway); that would also keep the logic on a higher level of > abstraction (split generation, partition pruning) as opposed to many > low-level places like FSOP, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19476) Fix failures in TestReplicationScenariosAcidTables, TestReplicationOnHDFSEncryptedZones and TestCopyUtils
[ https://issues.apache.org/jira/browse/HIVE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473581#comment-16473581 ] Hive QA commented on HIVE-19476: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 10s{color} | {color:red} /data/hiveptest/logs/PreCommit-HIVE-Build-10900/patches/PreCommit-HIVE-Build-10900.patch does not apply to master. Rebase required? Wrong Branch? See http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-10900/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Fix failures in TestReplicationScenariosAcidTables, > TestReplicationOnHDFSEncryptedZones and TestCopyUtils > - > > Key: HIVE-19476 > URL: https://issues.apache.org/jira/browse/HIVE-19476 > Project: Hive > Issue Type: Bug > Components: HiveServer2, repl >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: Tests, pull-request-available > Fix For: 3.0.0, 3.1.0 > > Attachments: HIVE-19476.01-branch-3.patch, HIVE-19476.01.patch, > HIVE-19476.02.patch > > > TestReplicationScenariosAcidTables broken due to HIVE-17657. > cc [~sershe] > TestReplicationOnHDFSEncryptedZones and TestCopyUtils are failing due to > auto-stats gathered which thrown exception irrelevant to theses tests. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18079) Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator bit-size
[ https://issues.apache.org/jira/browse/HIVE-18079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473576#comment-16473576 ] Hive QA commented on HIVE-18079: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12923137/HIVE-18079.12.patch {color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 37 failed/errored test(s), 13606 tests executed *Failed tests:* {noformat} TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed out) (batchId=247) TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed out) (batchId=247) TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=95) [nopart_insert.q,insert_into_with_schema.q,input41.q,having1.q,create_table_failure3.q,default_constraint_invalid_default_value.q,database_drop_not_empty_restrict.q,windowing_after_orderby.q,orderbysortby.q,subquery_select_distinct2.q,authorization_uri_alterpart_loc.q,udf_last_day_error_1.q,constraint_duplicate_name.q,create_table_failure4.q,alter_tableprops_external_with_notnull_constraint.q,semijoin5.q,udf_format_number_wrong4.q,deletejar.q,exim_11_nonpart_noncompat_sorting.q,show_tables_bad_db2.q,drop_func_nonexistent.q,alter_table_non_partitioned_table_cascade.q,check_constraint_subquery.q,load_wrong_fileformat.q,check_constraint_udtf.q,lockneg_try_db_lock_conflict.q,udf_field_wrong_args_len.q,create_table_failure2.q,create_with_fk_constraints_enforced.q,groupby2_map_skew_multi_distinct.q,mm_update.q,authorization_update_noupdatepriv.q,show_columns2.q,authorization_insert_noselectpriv.q,orc_replace_columns3_acid.q,compare_double_bigint.q,authorization_set_nonexistent_conf.q,alter_rename_partition_failure3.q,split_sample_wrong_format2.q,create_with_fk_pk_same_tab.q,compare_double_bigint_2.q,authorization_show_roles_no_admin.q,materialized_view_authorization_rebuild_no_grant.q,unionLimit.q,authorization_revoke_table_fail2.q,duplicate_insert3.q,authorization_desc_table_nosel.q,stats_noscan_non_native.q,orc_change_serde_acid.q,create_or_replace_view7.q,exim_07_nonpart_noncompat_ifof.q,create_with_unique_constraints_enforced.q,udf_concat_ws_wrong2.q,fileformat_bad_class.q,merge_negative_2.q,exim_15_part_nonpart.q,authorization_not_owner_drop_view.q,external1.q,authorization_uri_insert.q,create_with_fk_wrong_ref.q,columnstats_tbllvl_incorrect_column.q,authorization_show_parts_nosel.q,authorization_not_owner_drop_tab.q,external2.q,authorization_deletejar.q,temp_table_create_like_partitions.q,udf_greatest_error_1.q,ptf_negative_AggrFuncsWithNoGBYNoPartDef.q,alter_view_as_select_not_exist.q,touch1.q,groupby3_map_skew_multi_distinct.q,insert_into_notnull_constraint.q,exchange_partition_neg_partition_missing.q,groupby_cube_multi_gby.q,columnstats_tbllvl.q,drop_invalid_constraint2.q,alter_table_add_partition.q,update_not_acid.q,archive5.q,alter_table_constraint_invalid_pk_col.q,ivyDownload.q,udf_instr_wrong_type.q,bad_sample_clause.q,authorization_not_owner_drop_tab2.q,authorization_alter_db_owner.q,show_columns1.q,orc_type_promotion3.q,create_view_failure8.q,strict_join.q,udf_add_months_error_1.q,groupby_cube2.q,groupby_cube1.q,groupby_rollup1.q,genericFileFormat.q,invalid_cast_from_binary_4.q,drop_invalid_constraint1.q,serde_regex.q,show_partitions1.q,check_constraint_nonboolean_expr.q,invalid_cast_from_binary_6.q,create_with_multi_pk_constraint.q,udf_field_wrong_type.q,groupby_grouping_sets4.q,groupby_grouping_sets3.q,insertsel_fail.q,udf_locate_wrong_type.q,orc_type_promotion1_acid.q,set_table_property.q,create_or_replace_view2.q,groupby_grouping_sets2.q,alter_view_failure.q,distinct_windowing_failure1.q,invalid_t_alter2.q,alter_table_constraint_invalid_fk_col1.q,invalid_varchar_length_2.q,authorization_show_grant_otheruser_alltabs.q,subquery_windowing_corr.q,compact_non_acid_table.q,authorization_view_4.q,authorization_disallow_transform.q,materialized_view_authorization_rebuild_other.q,authorization_fail_4.q,dbtxnmgr_nodblock.q,set_hiveconf_internal_variable1.q,input_part0_neg.q,udf_printf_wrong3.q,load_orc_negative2.q,druid_buckets.q,archive2.q,authorization_addjar.q,invalid_sum_syntax.q,insert_into_with_schema1.q,udf_add_months_error_2.q,dyn_part_max_per_node.q,authorization_revoke_table_fail1.q,udf_printf_wrong2.q,archive_multi3.q,udf_printf_wrong1.q,subquery_subquery_chain.q,authorization_view_disable_cbo_4.q,no_matching_udf.q,create_view_failure7.q,drop_native_udf.q,truncate_column_list_bucketing.q,authorization_uri_add_partition.q,authorization_view_disable_cbo_3.q,bad_exec_hooks.q,authorization_view_disable_cbo_2.q,fetchtask_ioexception.q,char_pad_convert_fail2.q,authorization_set_role_neg1.q,serde_regex3.q,authorization_delete_nodeletepriv.q,materialized_view_delete.q,create_or_replace_view6.q,bucket_mapjoin_wrong_table_metadata_2.q,udf_sort_array_by_wrong2.q,
[jira] [Updated] (HIVE-19514) Successful test logs should be copied
[ https://issues.apache.org/jira/browse/HIVE-19514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-19514: --- Description: Ptest tries to copy ~20G worth of logs from the worker nodes to the server. The total aggregate time spent by all the Ptest server in copying these files is close to 15 min (although much of that is in parallel threads). Most of the times we don't check for the logs. We seldom check for logs of successful tests. We should ignore copying successful test logs by default to improve runtime. {noformat} 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:235 Executed 13568 tests 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase took 71 minutes 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase.TotalRsyncElapsedTime took 15 minutes 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: PrepPhase took 5 minutes 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: ReportingPhase took 0 minutes 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: TestCheckPhase took 0 minutes 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: YetusPhase took 0 minutes {noformat} was: Ptest tries to copy ~20G worth of logs from the worker nodes to the server. The total aggregate time spent by all the hosts in copying these files is close to 15 min in copying files. Most of the times we don't check for the logs. We seldom check for logs of successful tests. We should ignore copying successful test logs by default to improve runtime. {noformat} 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:235 Executed 13568 tests 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase took 71 minutes 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase.TotalRsyncElapsedTime took 15 minutes 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: PrepPhase took 5 minutes 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: ReportingPhase took 0 minutes 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: TestCheckPhase took 0 minutes 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: YetusPhase took 0 minutes {noformat} > Successful test logs should be copied > - > > Key: HIVE-19514 > URL: https://issues.apache.org/jira/browse/HIVE-19514 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > > Ptest tries to copy ~20G worth of logs from the worker nodes to the server. > The total aggregate time spent by all the Ptest server in copying these files > is close to 15 min (although much of that is in parallel threads). Most of > the times we don't check for the logs. We seldom check for logs of successful > tests. We should ignore copying successful test logs by default to improve > runtime. > {noformat} > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:235 Executed 13568 > tests > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: > ExecutionPhase took 71 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: > ExecutionPhase.TotalRsyncElapsedTime took 15 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: PrepPhase > took 5 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: > ReportingPhase took 0 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: > TestCheckPhase took 0 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: YetusPhase > took 0 minutes > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19514) Successful test logs should be copied
[ https://issues.apache.org/jira/browse/HIVE-19514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar reassigned HIVE-19514: -- > Successful test logs should be copied > - > > Key: HIVE-19514 > URL: https://issues.apache.org/jira/browse/HIVE-19514 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > > Ptest tries to copy ~20G worth of logs from the worker nodes to the server. > The total aggregate time spent by all the hosts in copying these files is > close to 15 min in copying files. Most of the times we don't check for the > logs. We seldom check for logs of successful tests. We should ignore copying > successful test logs by default to improve runtime. > {noformat} > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:235 Executed 13568 > tests > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: > ExecutionPhase took 71 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: > ExecutionPhase.TotalRsyncElapsedTime took 15 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: PrepPhase > took 5 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: > ReportingPhase took 0 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: > TestCheckPhase took 0 minutes > 2018-05-13 16:10:15,243 INFO [TestExecutor] PTest.run:237 PERF: YetusPhase > took 0 minutes > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18079) Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator bit-size
[ https://issues.apache.org/jira/browse/HIVE-18079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473561#comment-16473561 ] Hive QA commented on HIVE-18079: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 43s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 30s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 47s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 2s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 13s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 16s{color} | {color:blue} standalone-metastore in master has 215 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 57s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 44s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 44s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 19s{color} | {color:red} standalone-metastore: The patch generated 12 new + 38 unchanged - 1 fixed = 50 total (was 39) {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m 56s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 7s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 37m 12s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-10899/dev-support/hive-personality.sh | | git revision | master / 6f9368b | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-10899/yetus/diff-checkstyle-standalone-metastore.txt | | whitespace | http://104.198.109.242/logs//PreCommit-HIVE-Build-10899/yetus/whitespace-eol.txt | | modules | C: ql standalone-metastore U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-10899/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator > bit-size > > > Key: HIVE-18079 > URL: https://issues.apache.org/jira/browse/HIVE-18079 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore, Statistics >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-18079.1.patch, HIVE-18079.10.patch, > HIVE-18079.11.patch, HIVE-18079.12.patch, HIVE-18079.2.patch, > HIVE-18079.4.patch, HIVE-18079.5.patch, HIVE-18079.6.patch, > HIVE-18079.7.patch, HIVE-18079.8.patch, HIVE-18079.9.patch > > > HyperLogLog can merge a 14 bit HLL into a 10 bit HLL bitset, beca
[jira] [Commented] (HIVE-19506) Test suites timing out
[ https://issues.apache.org/jira/browse/HIVE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473555#comment-16473555 ] Jesus Camacho Rodriguez commented on HIVE-19506: [~vgarg], I think we do not need to wait for ptest run to merge this one since the tests are just timing out. I believe {{TestNegativeCliDriver}} may have another test timing out, but you can commit in different batches and just add the commit IDs to this issue to speed up the resolution. > Test suites timing out > -- > > Key: HIVE-19506 > URL: https://issues.apache.org/jira/browse/HIVE-19506 > Project: Hive > Issue Type: Bug >Reporter: Vineet Garg >Priority: Blocker > Attachments: HIVE-19506.1.patch, HIVE-19506.2.patch > > > Currently a lot of test suites are timing out consistently: > TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed > out) > TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed > out) > TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) > TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed > out) > TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed > out) > TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[ https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-17852: Attachment: HIVE-17852.01.patch > remove support for list bucketing "stored as directories" in 3.0 > > > Key: HIVE-17852 > URL: https://issues.apache.org/jira/browse/HIVE-17852 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-17852.01.patch > > > From the email thread: > 1) LB, when stored as directories, adds a lot of low-level complexity to Hive > tables that has to be accounted for in many places in the code where the > files are written or modified - from FSOP to ACID/replication/export. > 2) While working on some FSOP code I noticed that some of that logic is > broken - e.g. the duplicate file removal from tasks, a pretty fundamental > correctness feature in Hive, may be broken. LB also doesn’t appear to be > compatible with e.g. regular bucketing. > 3) The feature hasn’t seen development activity in a while; it also doesn’t > appear to be used a lot. > Keeping with the theme of cleaning up “legacy” code for 3.0, I was proposing > we remove it. > (2) also suggested that, if needed, it might be easier to implement similar > functionality by adding some flexibility to partitions (which LB directories > look like anyway); that would also keep the logic on a higher level of > abstraction (split generation, partition pruning) as opposed to many > low-level places like FSOP, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[ https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-17852: Status: Patch Available (was: Open) > remove support for list bucketing "stored as directories" in 3.0 > > > Key: HIVE-17852 > URL: https://issues.apache.org/jira/browse/HIVE-17852 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Laszlo Bodor >Priority: Major > > From the email thread: > 1) LB, when stored as directories, adds a lot of low-level complexity to Hive > tables that has to be accounted for in many places in the code where the > files are written or modified - from FSOP to ACID/replication/export. > 2) While working on some FSOP code I noticed that some of that logic is > broken - e.g. the duplicate file removal from tasks, a pretty fundamental > correctness feature in Hive, may be broken. LB also doesn’t appear to be > compatible with e.g. regular bucketing. > 3) The feature hasn’t seen development activity in a while; it also doesn’t > appear to be used a lot. > Keeping with the theme of cleaning up “legacy” code for 3.0, I was proposing > we remove it. > (2) also suggested that, if needed, it might be easier to implement similar > functionality by adding some flexibility to partitions (which LB directories > look like anyway); that would also keep the logic on a higher level of > abstraction (split generation, partition pruning) as opposed to many > low-level places like FSOP, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19424) NPE In MetaDataFormatters
[ https://issues.apache.org/jira/browse/HIVE-19424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473542#comment-16473542 ] Hive QA commented on HIVE-19424: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12923121/HIVE-19424.1.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 31 failed/errored test(s), 13568 tests executed *Failed tests:* {noformat} TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed out) (batchId=247) TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed out) (batchId=247) TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=94) [infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,scriptfile1.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q] TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=95) [nopart_insert.q,insert_into_with_schema.q,input41.q,having1.q,create_table_failure3.q,default_constraint_invalid_default_value.q,database_drop_not_empty_restrict.q,windowing_after_orderby.q,orderbysortby.q,subquery_select_distinct2.q,authorization_uri_alterpart_loc.q,udf_last_day_error_1.q,constraint_duplicate_name.q,create_table_failure4.q,alter_tableprops_external_with_notnull_constraint.q,semijoin5.q,udf_format_number_wrong4.q,deletejar.q,exim_11_nonpart_noncompat_sorting.q,show_tables_bad_db2.q,drop_func_nonexistent.q,alter_table_non_partitioned_table_cascade.q,check_constraint_subquery.q,load_wrong_fileformat.q,check_constraint_udtf.q,lockneg_try_db_lock_conflict.q,udf_field_wrong_args_len.q,create_table_failure2.q,create_with_fk_constraints_enforced.q,groupby2_map_skew_multi_distinct.q,mm_update.q,authorization_update_noupdatepriv.q,show_columns2.q,authorization_insert_noselectpriv.q,orc_replace_columns3_acid.q,compare_double_bigint.q,authorization_set_nonexistent_conf.q,alter_rename_partition_failure3.q,split_sample_wrong_format2.q,create_with_fk_pk_same_tab.q,compare_double_bigint_2.q,authorization_show_roles_no_admin.q,materialized_view_authorization_rebuild_no_grant.q,unionLimit.q,authorization_revoke_table_fail2.q,duplicate_insert3.q,authorization_desc_table_nosel.q,stats_noscan_non_native.q,orc_change_serde_acid.q,create_or_replace_view7.q,exim_07_nonpart_noncompat_ifof.q,create_with_unique_constraints_enforced.q,udf_concat_ws_wrong2.q,fileformat_bad_class.q,merge_negative_2.q,exim_15_part_nonpart.q,authorization_not_owner_drop_view.q,external1.q,authorization_uri_insert.q,create_with_fk_wrong_ref.q,columnstats_tbllvl_incorrect_column.q,authorization_show_parts_nosel.q,authorization_not_owner_drop_tab.q,external2.q,authorization_deletejar.q,temp_table_create_like_partitions.q,udf_greatest_error_1.q,ptf_negative_AggrFuncsWithNoGBYNoPartDef.q,alter_view_as_select_not_exist.q,touch1.q,groupby3_map_skew_multi_distinct.q,insert_into_notnull_constraint.q,exchange_partition_neg_partition_missing.q,groupby_cube_multi_gby.q,columnstats_tbllvl.q,drop_invalid_constraint2.q,alter_table_add_partition.q,update_not_acid.q,archive5.q,alter_table_constraint_invalid_pk_col.q,ivyDownload.q,udf_instr_wrong_type.q,bad_sample_clause.q,authorization_not_owner_drop_tab2.q,authorization_alter_db_owner.q,show_columns1.q,orc_type_promotion3.q,create_view_failure8.q,strict_join.q,udf_add_months_error_1.q,groupby_cube2.q,groupby_cube1.q,groupby_rollup1.q,genericFileFormat.q,invalid_cast_from_binary_4.q,drop_invalid_constraint1.q,serde_regex.q,show_partitions1.q,check_constraint_nonboolean_expr.q,invalid_cast_from_binary_6.q,create_with_multi_pk_constraint.q,udf_field_wrong_type.q,groupby_grouping_sets4.q,groupby_grouping_sets3.q,insertsel_fail.q,udf_locate_wrong_type.q,orc_type_promotion1_acid.q,set_table_property.q,create_or_replace_view2.q,groupby_grouping_sets2.q,alter_view_failure.q,distinct_windowing_failure1.q,invalid_t_alter2.q,alter_table_constraint_invalid_fk_col1.q,invalid_varchar_length_2.q,authorization_show_grant_otheruser_alltabs.q,subquery_windowing_corr.q,compact_non_acid_table.q,authorization_view_4.q,authorization_disallow_transform.q,materialized_view_authorization_rebuild_other.q,authorization_fail_4.q,dbtxnmgr_nodblock.q,set_hiveconf_internal_variable1.q,input_part0_neg.q,udf_printf_wrong3.q,load_orc_negative2.q,druid_buckets.q,archive2.q,authorization_addjar.q,invalid_sum_syntax.q,insert_into_with_schema1.q,udf_add_months_error_2.q,dyn_part_max_per_node.q,authorization_revoke_table_fail1.q,udf_printf_wrong2.q,archive_multi3.q,udf_printf_wrong1.q,subquery_subquery_chain.q,authorization_view_disable_cbo_4.q,no_matching_udf.q,create_view_failure7.q,drop_native_udf.q,truncate_column_list_bucketing.q,authorization_uri
[jira] [Commented] (HIVE-19424) NPE In MetaDataFormatters
[ https://issues.apache.org/jira/browse/HIVE-19424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473523#comment-16473523 ] Hive QA commented on HIVE-19424: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 9s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 39s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 12s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 59s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 3s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 39s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 15s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 24m 10s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-10898/dev-support/hive-personality.sh | | git revision | master / 6f9368b | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | whitespace | http://104.198.109.242/logs//PreCommit-HIVE-Build-10898/yetus/whitespace-eol.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-10898/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > NPE In MetaDataFormatters > - > > Key: HIVE-19424 > URL: https://issues.apache.org/jira/browse/HIVE-19424 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Metastore, Standalone Metastore >Affects Versions: 3.0.0, 2.4.0 >Reporter: BELUGA BEHR >Assignee: Alice Fan >Priority: Minor > Attachments: HIVE-19424.1.patch > > > h2. Overview > According to the Hive Schema definition, a table's {{INPUT_FORMAT}} class can > be set to NULL. However, there are places in the code where we do not > account for this NULL value, in particular the {{MetaDataFormatters}} classes > {{TextMetaDataFormatter}} and {{JsonMetaDataFormatter}}. In addition, there > is no debug level logging in the {{MetaDataFormatters}} classes to tell me > which table in particular is causing the problem. > {code:sql|title=hive-schema-2.2.0.mysql.sql} > CREATE TABLE IF NOT EXISTS `SDS` ( > `SD_ID` bigint(20) NOT NULL, > `CD_ID` bigint(20) DEFAULT NULL, > `INPUT_FORMAT` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin > DEFAULT NULL, > `IS_COMPRESSED` bit(1) NOT NULL, > ... > {code} > {code:java|title=TextMetaDataFormatter.java} > // Not checking for a null return from getInputFormatClass > inputFormattCls = par.getInputFormatClass().getName(); > outputFormattCls = par.getOutputFormatClass().getName(); > {code} > h2. Reproduction > {code:sql} > -- MySQL Backend > update SDS SET INPUT
[jira] [Commented] (HIVE-19258) add originals support to MM tables (and make the conversion a metadata only operation)
[ https://issues.apache.org/jira/browse/HIVE-19258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473515#comment-16473515 ] Hive QA commented on HIVE-19258: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12923114/HIVE-19258.06.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/10897/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10897/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10897/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2018-05-13 14:50:03.492 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-10897/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2018-05-13 14:50:03.495 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 6f9368b HIVE-19510 : Add performance metrics in Ptest to find the total time spend in rsync (Vihang Karajgaonkar) + git clean -f -d + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 6f9368b HIVE-19510 : Add performance metrics in Ptest to find the total time spend in rsync (Vihang Karajgaonkar) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2018-05-13 14:50:08.816 + rm -rf ../yetus_PreCommit-HIVE-Build-10897 + mkdir ../yetus_PreCommit-HIVE-Build-10897 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-10897 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-10897/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/io/FileOperations.java:101 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/io/FileOperations.java' with conflicts. Going to apply patch with: git apply -p0 /data/hiveptest/working/scratch/build.patch:887: trailing whitespace. error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/io/FileOperations.java:101 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/io/FileOperations.java' with conflicts. U ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/io/FileOperations.java warning: 1 line adds whitespace errors. + exit 1 ' {noformat} This message is automatically generated. ATTACHMENT ID: 12923114 - PreCommit-HIVE-Build > add originals support to MM tables (and make the conversion a metadata only > operation) > -- > > Key: HIVE-19258 > URL: https://issues.apache.org/jira/browse/HIVE-19258 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19258.01.patch, HIVE-19258.02.patch, > HIVE-19258.03.patch, HIVE-19258.04.patch, HIVE-19258.05.patch, > HIVE-19258.06.patch, HIVE-19258.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19332) Disable compute.query.using.stats for external table
[ https://issues.apache.org/jira/browse/HIVE-19332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473514#comment-16473514 ] Hive QA commented on HIVE-19332: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12923109/HIVE-19332.6.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 32 failed/errored test(s), 13569 tests executed *Failed tests:* {noformat} TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed out) (batchId=247) TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed out) (batchId=247) TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=94) [infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,scriptfile1.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q] TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=95) [nopart_insert.q,insert_into_with_schema.q,input41.q,having1.q,create_table_failure3.q,default_constraint_invalid_default_value.q,database_drop_not_empty_restrict.q,windowing_after_orderby.q,orderbysortby.q,subquery_select_distinct2.q,authorization_uri_alterpart_loc.q,udf_last_day_error_1.q,constraint_duplicate_name.q,create_table_failure4.q,alter_tableprops_external_with_notnull_constraint.q,semijoin5.q,udf_format_number_wrong4.q,deletejar.q,exim_11_nonpart_noncompat_sorting.q,show_tables_bad_db2.q,drop_func_nonexistent.q,alter_table_non_partitioned_table_cascade.q,check_constraint_subquery.q,load_wrong_fileformat.q,check_constraint_udtf.q,lockneg_try_db_lock_conflict.q,udf_field_wrong_args_len.q,create_table_failure2.q,create_with_fk_constraints_enforced.q,groupby2_map_skew_multi_distinct.q,mm_update.q,authorization_update_noupdatepriv.q,show_columns2.q,authorization_insert_noselectpriv.q,orc_replace_columns3_acid.q,compare_double_bigint.q,authorization_set_nonexistent_conf.q,alter_rename_partition_failure3.q,split_sample_wrong_format2.q,create_with_fk_pk_same_tab.q,compare_double_bigint_2.q,authorization_show_roles_no_admin.q,materialized_view_authorization_rebuild_no_grant.q,unionLimit.q,authorization_revoke_table_fail2.q,duplicate_insert3.q,authorization_desc_table_nosel.q,stats_noscan_non_native.q,orc_change_serde_acid.q,create_or_replace_view7.q,exim_07_nonpart_noncompat_ifof.q,create_with_unique_constraints_enforced.q,udf_concat_ws_wrong2.q,fileformat_bad_class.q,merge_negative_2.q,exim_15_part_nonpart.q,authorization_not_owner_drop_view.q,external1.q,authorization_uri_insert.q,create_with_fk_wrong_ref.q,columnstats_tbllvl_incorrect_column.q,authorization_show_parts_nosel.q,authorization_not_owner_drop_tab.q,external2.q,authorization_deletejar.q,temp_table_create_like_partitions.q,udf_greatest_error_1.q,ptf_negative_AggrFuncsWithNoGBYNoPartDef.q,alter_view_as_select_not_exist.q,touch1.q,groupby3_map_skew_multi_distinct.q,insert_into_notnull_constraint.q,exchange_partition_neg_partition_missing.q,groupby_cube_multi_gby.q,columnstats_tbllvl.q,drop_invalid_constraint2.q,alter_table_add_partition.q,update_not_acid.q,archive5.q,alter_table_constraint_invalid_pk_col.q,ivyDownload.q,udf_instr_wrong_type.q,bad_sample_clause.q,authorization_not_owner_drop_tab2.q,authorization_alter_db_owner.q,show_columns1.q,orc_type_promotion3.q,create_view_failure8.q,strict_join.q,udf_add_months_error_1.q,groupby_cube2.q,groupby_cube1.q,groupby_rollup1.q,genericFileFormat.q,invalid_cast_from_binary_4.q,drop_invalid_constraint1.q,serde_regex.q,show_partitions1.q,check_constraint_nonboolean_expr.q,invalid_cast_from_binary_6.q,create_with_multi_pk_constraint.q,udf_field_wrong_type.q,groupby_grouping_sets4.q,groupby_grouping_sets3.q,insertsel_fail.q,udf_locate_wrong_type.q,orc_type_promotion1_acid.q,set_table_property.q,create_or_replace_view2.q,groupby_grouping_sets2.q,alter_view_failure.q,distinct_windowing_failure1.q,invalid_t_alter2.q,alter_table_constraint_invalid_fk_col1.q,invalid_varchar_length_2.q,authorization_show_grant_otheruser_alltabs.q,subquery_windowing_corr.q,compact_non_acid_table.q,authorization_view_4.q,authorization_disallow_transform.q,materialized_view_authorization_rebuild_other.q,authorization_fail_4.q,dbtxnmgr_nodblock.q,set_hiveconf_internal_variable1.q,input_part0_neg.q,udf_printf_wrong3.q,load_orc_negative2.q,druid_buckets.q,archive2.q,authorization_addjar.q,invalid_sum_syntax.q,insert_into_with_schema1.q,udf_add_months_error_2.q,dyn_part_max_per_node.q,authorization_revoke_table_fail1.q,udf_printf_wrong2.q,archive_multi3.q,udf_printf_wrong1.q,subquery_subquery_chain.q,authorization_view_disable_cbo_4.q,no_matching_udf.q,create_view_failure7.q,drop_native_udf.q,truncate_column_list_bucketing.q,authorization_