[jira] [Commented] (HIVE-20091) Tez: Add security credentials for FileSinkOperator output
[ https://issues.apache.org/jira/browse/HIVE-20091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535651#comment-16535651 ] Hive QA commented on HIVE-20091: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 25s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 38s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 57s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 22s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 38s{color} | {color:red} ql: The patch generated 2 new + 40 unchanged - 0 fixed = 42 total (was 40) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m 15s{color} | {color:red} ql generated 1 new + 2287 unchanged - 0 fixed = 2288 total (was 2287) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 22m 55s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | Should org.apache.hadoop.hive.ql.exec.tez.DagUtils$CollectFileSinkUrisNodeProcessor be a _static_ inner class? At DagUtils.java:inner class? At DagUtils.java:[lines 192-214] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12438/dev-support/hive-personality.sh | | git revision | master / 8494522 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12438/yetus/diff-checkstyle-ql.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-12438/yetus/new-findbugs-ql.html | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12438/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Tez: Add security credentials for FileSinkOperator output > - > > Key: HIVE-20091 > URL: https://issues.apache.org/jira/browse/HIVE-20091 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20091.01.patch, HIVE-20091.02.patch, > HIVE-20091.03.patch > > > DagUtils needs to add security credentials for the output for the > FileSinkOperator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20118) SessionStateUserAuthenticator.getGroupNames() is always empty
[ https://issues.apache.org/jira/browse/HIVE-20118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai reassigned HIVE-20118: - > SessionStateUserAuthenticator.getGroupNames() is always empty > - > > Key: HIVE-20118 > URL: https://issues.apache.org/jira/browse/HIVE-20118 > Project: Hive > Issue Type: Improvement > Components: Authorization >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20118) SessionStateUserAuthenticator.getGroupNames() is always empty
[ https://issues.apache.org/jira/browse/HIVE-20118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-20118: -- Attachment: HIVE-20118.1.patch > SessionStateUserAuthenticator.getGroupNames() is always empty > - > > Key: HIVE-20118 > URL: https://issues.apache.org/jira/browse/HIVE-20118 > Project: Hive > Issue Type: Improvement > Components: Authorization >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-20118.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20118) SessionStateUserAuthenticator.getGroupNames() is always empty
[ https://issues.apache.org/jira/browse/HIVE-20118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-20118: -- Status: Patch Available (was: Open) > SessionStateUserAuthenticator.getGroupNames() is always empty > - > > Key: HIVE-20118 > URL: https://issues.apache.org/jira/browse/HIVE-20118 > Project: Hive > Issue Type: Improvement > Components: Authorization >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-20118.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20091) Tez: Add security credentials for FileSinkOperator output
[ https://issues.apache.org/jira/browse/HIVE-20091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535686#comment-16535686 ] Hive QA commented on HIVE-20091: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930504/HIVE-20091.03.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 14629 tests executed *Failed tests:* {noformat} TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestLocationQueries - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out) (batchId=240) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12438/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12438/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12438/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12930504 - PreCommit-HIVE-Build > Tez: Add security credentials for FileSinkOperator output > - > > Key: HIVE-20091 > URL: https://issues.apache.org/jira/browse/HIVE-20091 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20091.01.patch, HIVE-20091.02.patch, > HIVE-20091.03.patch > > > DagUtils needs to add security credentials for the output for the > FileSinkOperator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-12847) ORC file footer cache should be memory sensitive
[ https://issues.apache.org/jira/browse/HIVE-12847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535685#comment-16535685 ] Wang Haihua commented on HIVE-12847: Any progress now? We encounter this also for data size of many tables could explan to 10TB. Thanks! cc [~nemon] > ORC file footer cache should be memory sensitive > > > Key: HIVE-12847 > URL: https://issues.apache.org/jira/browse/HIVE-12847 > Project: Hive > Issue Type: Improvement > Components: File Formats, ORC >Affects Versions: 1.2.1 >Reporter: Nemon Lou >Assignee: Nemon Lou >Priority: Major > Attachments: HIVE-12847.patch > > > The size based footer cache can not control memory usage properly. > Having seen a HiveServer2 hang (full GC all the time) due to ORC file footer > cache taking up too much heap memory. > A simple query like "select * from orc_table limit 1" can make HiveServer2 > hang. > The input table has about 1000 ORC files and each ORC file owns about 2500 > stripes. > {noformat} > num #instances #bytes class name > -- >1: 21465360125758432120 > org.apache.hadoop.hive.ql.io.orc.OrcProto$ColumnStatistics >3: 122233301 8800797672 > org.apache.hadoop.hive.ql.io.orc.OrcProto$StringStatistics >5: 89439001 6439608072 > org.apache.hadoop.hive.ql.io.orc.OrcProto$IntegerStatistics >7: 2981300 262354400 > org.apache.hadoop.hive.ql.io.orc.OrcProto$StripeInformation >9: 2981300 143102400 > org.apache.hadoop.hive.ql.io.orc.OrcProto$StripeStatistics > 12: 2983691 71608584 > org.apache.hadoop.hive.ql.io.orc.ReaderImpl$StripeInformationImpl > 15: 809297121752 > org.apache.hadoop.hive.ql.io.orc.OrcProto$Type > 17:1032825783792 > org.apache.hadoop.mapreduce.lib.input.FileSplit > 20: 516413305024 > org.apache.hadoop.hive.ql.exec.FetchOperator$FetchInputFormatSplit > 21: 516413305024 org.apache.hadoop.hive.ql.io.orc.OrcSplit > 31: 1 413152 > [Lorg.apache.hadoop.hive.ql.exec.FetchOperator$FetchInputFormatSplit; > 100: 1122 26928 org.apache.hadoop.hive.ql.io.orc.Metadata > > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20091) Tez: Add security credentials for FileSinkOperator output
[ https://issues.apache.org/jira/browse/HIVE-20091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535689#comment-16535689 ] Matt McCline commented on HIVE-20091: - Again. > Tez: Add security credentials for FileSinkOperator output > - > > Key: HIVE-20091 > URL: https://issues.apache.org/jira/browse/HIVE-20091 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20091.01.patch, HIVE-20091.02.patch, > HIVE-20091.03.patch > > > DagUtils needs to add security credentials for the output for the > FileSinkOperator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20091) Tez: Add security credentials for FileSinkOperator output
[ https://issues.apache.org/jira/browse/HIVE-20091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20091: Status: Patch Available (was: In Progress) > Tez: Add security credentials for FileSinkOperator output > - > > Key: HIVE-20091 > URL: https://issues.apache.org/jira/browse/HIVE-20091 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20091.01.patch, HIVE-20091.02.patch, > HIVE-20091.03.patch, HIVE-20091.04.patch > > > DagUtils needs to add security credentials for the output for the > FileSinkOperator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20091) Tez: Add security credentials for FileSinkOperator output
[ https://issues.apache.org/jira/browse/HIVE-20091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20091: Status: In Progress (was: Patch Available) > Tez: Add security credentials for FileSinkOperator output > - > > Key: HIVE-20091 > URL: https://issues.apache.org/jira/browse/HIVE-20091 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20091.01.patch, HIVE-20091.02.patch, > HIVE-20091.03.patch, HIVE-20091.04.patch > > > DagUtils needs to add security credentials for the output for the > FileSinkOperator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20091) Tez: Add security credentials for FileSinkOperator output
[ https://issues.apache.org/jira/browse/HIVE-20091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20091: Attachment: HIVE-20091.04.patch > Tez: Add security credentials for FileSinkOperator output > - > > Key: HIVE-20091 > URL: https://issues.apache.org/jira/browse/HIVE-20091 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20091.01.patch, HIVE-20091.02.patch, > HIVE-20091.03.patch, HIVE-20091.04.patch > > > DagUtils needs to add security credentials for the output for the > FileSinkOperator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20043) HiveServer2: SessionState has a static sync block around an AtomicBoolean
[ https://issues.apache.org/jira/browse/HIVE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535695#comment-16535695 ] Hive QA commented on HIVE-20043: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 28s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 38s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 47s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 38s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 7s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 22m 40s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12439/dev-support/hive-personality.sh | | git revision | master / 8494522 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12439/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > HiveServer2: SessionState has a static sync block around an AtomicBoolean > - > > Key: HIVE-20043 > URL: https://issues.apache.org/jira/browse/HIVE-20043 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Laszlo Bodor >Priority: Major > Labels: Branch3Candidate, Concurrency > Attachments: HIVE-20043.01.patch, HIVE-20043.02.patch > > > {code} > private static void start(SessionState startSs, boolean isAsync, LogHelper > console) { > ... > synchronized(SessionState.class) { > if (!startSs.isStarted.compareAndSet(false, true)) { > return; > } > } > {code} > startSs.isStarted is an AtomicBoolean, which makes it hard to know why this > code is locked with a static lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20043) HiveServer2: SessionState has a static sync block around an AtomicBoolean
[ https://issues.apache.org/jira/browse/HIVE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535714#comment-16535714 ] Hive QA commented on HIVE-20043: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930506/HIVE-20043.02.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 14616 tests executed *Failed tests:* {noformat} TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestLocationQueries - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestReplicationScenariosAcidTables - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out) (batchId=240) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_dynamic_partition] (batchId=190) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_expressions] (batchId=190) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test1] (batchId=190) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_alter] (batchId=190) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_insert] (batchId=190) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12439/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12439/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12439/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 11 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12930506 - PreCommit-HIVE-Build > HiveServer2: SessionState has a static sync block around an AtomicBoolean > - > > Key: HIVE-20043 > URL: https://issues.apache.org/jira/browse/HIVE-20043 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Laszlo Bodor >Priority: Major > Labels: Branch3Candidate, Concurrency > Attachments: HIVE-20043.01.patch, HIVE-20043.02.patch > > > {code} > private static void start(SessionState startSs, boolean isAsync, LogHelper > console) { > ... > synchronized(SessionState.class) { > if (!startSs.isStarted.compareAndSet(false, true)) { > return; > } > } > {code} > startSs.isStarted is an AtomicBoolean, which makes it hard to know why this > code is locked with a static lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18545) Add UDF to parse complex types from json
[ https://issues.apache.org/jira/browse/HIVE-18545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535728#comment-16535728 ] Hive QA commented on HIVE-18545: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 39s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 10s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 14s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 27s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 36s{color} | {color:blue} serde in master has 193 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 52s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 35s{color} | {color:blue} hcatalog/core in master has 33 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 23s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 12m 31s{color} | {color:red} branch/itests/hive-jmh cannot run convertXmlToText from findbugs {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 40s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 16s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 16s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 12s{color} | {color:red} serde: The patch generated 23 new + 6 unchanged - 382 fixed = 29 total (was 388) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 38s{color} | {color:red} ql: The patch generated 11 new + 71 unchanged - 0 fixed = 82 total (was 71) {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 11s{color} | {color:green} hcatalog/core: The patch generated 0 new + 10 unchanged - 31 fixed = 10 total (was 41) {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 10s{color} | {color:green} The patch streaming passed checkstyle {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 9s{color} | {color:red} itests/hive-jmh: The patch generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0) {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch 3 line(s) with tabs. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 47s{color} | {color:red} serde generated 4 new + 190 unchanged - 3 fixed = 194 total (was 193) {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 3s{color} | {color:green} ql in the patch passed. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 43s{color} | {color:green} hcatalog/core generated 0 new + 31 unchanged - 2 fixed = 31 total (was 33) {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 29s{color} | {color:green} streaming in the patch passed. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 9m 20s{color} | {color:red} patch/itests/hive-jmh cannot run convertXmlToText from findbugs {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 45s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color
[jira] [Commented] (HIVE-18545) Add UDF to parse complex types from json
[ https://issues.apache.org/jira/browse/HIVE-18545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535730#comment-16535730 ] Hive QA commented on HIVE-18545: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930512/HIVE-18545.07.patch {color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 14636 tests executed *Failed tests:* {noformat} TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestLocationQueries - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out) (batchId=240) org.apache.hive.hcatalog.api.repl.commands.TestCommands.org.apache.hive.hcatalog.api.repl.commands.TestCommands (batchId=200) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12440/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12440/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12440/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12930512 - PreCommit-HIVE-Build > Add UDF to parse complex types from json > > > Key: HIVE-18545 > URL: https://issues.apache.org/jira/browse/HIVE-18545 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18545.02.patch, HIVE-18545.03.patch, > HIVE-18545.04.patch, HIVE-18545.05.patch, HIVE-18545.06.patch, > HIVE-18545.06.patch, HIVE-18545.06.patch, HIVE-18545.07.patch, > HIVE-18545.07.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20001) With doas set to true, running select query as hrt_qa user on external table fails due to permission denied to read /warehouse/tablespace/managed directory.
[ https://issues.apache.org/jira/browse/HIVE-20001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535737#comment-16535737 ] Hive QA commented on HIVE-20001: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 34s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 58s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 43s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 58s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m 55s{color} | {color:blue} standalone-metastore in master has 217 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 57s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 46s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 8s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 42s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 21s{color} | {color:red} standalone-metastore: The patch generated 2 new + 581 unchanged - 0 fixed = 583 total (was 581) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m 1s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 50s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 33m 50s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile xml | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12441/dev-support/hive-personality.sh | | git revision | master / 8494522 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12441/yetus/diff-checkstyle-standalone-metastore.txt | | modules | C: standalone-metastore ql U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12441/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > With doas set to true, running select query as hrt_qa user on external table > fails due to permission denied to read /warehouse/tablespace/managed > directory. > > > Key: HIVE-20001 > URL: https://issues.apache.org/jira/browse/HIVE-20001 > Project: Hive > Issue Type: Bug >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20001.1.patch, HIVE-20001.1.patch, > HIVE-20001.2.patch, HIVE-20001.3.patch, HIVE-20001.4.patch, HIVE-20001.5.patch > > > Hive: With doas set to true, running select query as hrt_qa user on external > table fails due to
[jira] [Commented] (HIVE-20001) With doas set to true, running select query as hrt_qa user on external table fails due to permission denied to read /warehouse/tablespace/managed directory.
[ https://issues.apache.org/jira/browse/HIVE-20001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535744#comment-16535744 ] Hive QA commented on HIVE-20001: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930536/HIVE-20001.5.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 14635 tests executed *Failed tests:* {noformat} TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out) (batchId=240) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timestamptz] (batchId=191) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_joins] (batchId=191) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_masking] (batchId=191) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12441/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12441/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12441/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 5 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12930536 - PreCommit-HIVE-Build > With doas set to true, running select query as hrt_qa user on external table > fails due to permission denied to read /warehouse/tablespace/managed > directory. > > > Key: HIVE-20001 > URL: https://issues.apache.org/jira/browse/HIVE-20001 > Project: Hive > Issue Type: Bug >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20001.1.patch, HIVE-20001.1.patch, > HIVE-20001.2.patch, HIVE-20001.3.patch, HIVE-20001.4.patch, HIVE-20001.5.patch > > > Hive: With doas set to true, running select query as hrt_qa user on external > table fails due to permission denied to read /warehouse/tablespace/managed > directory. > Steps: > 1. Create a external table. > 2. Set doas to true. > 3. run select count(*) using user hrt_qa. > Table creation query. > {code} > beeline -n hrt_qa -p pwd -u > "jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit" > --outputformat=tsv -e "drop table if exists test_table purge; > create external table test_table(id int, age int) row format delimited fields > terminated by '|' stored as textfile; > load data inpath '/tmp/table1.dat' overwrite into table test_table; > {code} > select count(*) query execution fails > {code} > beeline -n hrt_qa -p pwd -u > "jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit" > --outputformat=tsv -e "select count(*) from test_table where age>30 and > id<10100;" > 2018-06-22 10:22:29,328|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: Class path contains > multiple SLF4J bindings. > 2018-06-22 10:22:29,330|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: See > http://www.slf4j.org/codes.html#multiple_bindings for an explanation. > 2018-06-22 10:22:29,335|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: Actual binding is of > type [org.apache.logging.slf4j.Log4jLoggerFactory] > 2018-06-22 10:22:31,408|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|Format tsv is deprecated, > please use tsv2 > 2018-06-22 10:22:31,529|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|Connecting to > jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.sit
[jira] [Commented] (HIVE-15974) Support real, double precision and numeric data types
[ https://issues.apache.org/jira/browse/HIVE-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535751#comment-16535751 ] Hive QA commented on HIVE-15974: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 46s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 45s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 57s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 20s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 45s{color} | {color:red} ql: The patch generated 1 new + 1018 unchanged - 0 fixed = 1019 total (was 1018) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 6s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 23m 9s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12442/dev-support/hive-personality.sh | | git revision | master / 8494522 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12442/yetus/diff-checkstyle-ql.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12442/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Support real, double precision and numeric data types > - > > Key: HIVE-15974 > URL: https://issues.apache.org/jira/browse/HIVE-15974 > Project: Hive > Issue Type: Sub-task > Components: SQL >Reporter: Carter Shanklin >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-15974.01.patch > > > Support the standard names for these datatypes, which map to existing Hive > datatypes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[ https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-17852: Attachment: HIVE-17852.15.patch > remove support for list bucketing "stored as directories" in 3.0 > > > Key: HIVE-17852 > URL: https://issues.apache.org/jira/browse/HIVE-17852 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Laszlo Bodor >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-17852.01.patch, HIVE-17852.02.patch, > HIVE-17852.03.patch, HIVE-17852.04.patch, HIVE-17852.05.patch, > HIVE-17852.06.patch, HIVE-17852.07.patch, HIVE-17852.08.patch, > HIVE-17852.09.patch, HIVE-17852.10.patch, HIVE-17852.11.patch, > HIVE-17852.12.patch, HIVE-17852.13.patch, HIVE-17852.14.patch, > HIVE-17852.15.patch > > > From the email thread: > 1) LB, when stored as directories, adds a lot of low-level complexity to Hive > tables that has to be accounted for in many places in the code where the > files are written or modified - from FSOP to ACID/replication/export. > 2) While working on some FSOP code I noticed that some of that logic is > broken - e.g. the duplicate file removal from tasks, a pretty fundamental > correctness feature in Hive, may be broken. LB also doesn’t appear to be > compatible with e.g. regular bucketing. > 3) The feature hasn’t seen development activity in a while; it also doesn’t > appear to be used a lot. > Keeping with the theme of cleaning up “legacy” code for 3.0, I was proposing > we remove it. > (2) also suggested that, if needed, it might be easier to implement similar > functionality by adding some flexibility to partitions (which LB directories > look like anyway); that would also keep the logic on a higher level of > abstraction (split generation, partition pruning) as opposed to many > low-level places like FSOP, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-15974) Support real, double precision and numeric data types
[ https://issues.apache.org/jira/browse/HIVE-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535764#comment-16535764 ] Hive QA commented on HIVE-15974: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930539/HIVE-15974.01.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 14632 tests executed *Failed tests:* {noformat} TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out) (batchId=240) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timestamptz] (batchId=191) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_joins] (batchId=191) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_masking] (batchId=191) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12442/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12442/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12442/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 4 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12930539 - PreCommit-HIVE-Build > Support real, double precision and numeric data types > - > > Key: HIVE-15974 > URL: https://issues.apache.org/jira/browse/HIVE-15974 > Project: Hive > Issue Type: Sub-task > Components: SQL >Reporter: Carter Shanklin >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-15974.01.patch > > > Support the standard names for these datatypes, which map to existing Hive > datatypes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20105) Druid-Hive: tpcds query on timestamp throws java.lang.IllegalArgumentException: Cannot create timestamp, parsing error
[ https://issues.apache.org/jira/browse/HIVE-20105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535765#comment-16535765 ] Hive QA commented on HIVE-20105: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930544/HIVE-20105.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12443/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12443/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12443/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2018-07-07 14:03:41.867 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-12443/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2018-07-07 14:03:41.871 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 8494522 HIVE-20098 : Statistics: NPE when getting Date column partition statistics (Andrii via Gopal V) + git clean -f -d + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 8494522 HIVE-20098 : Statistics: NPE when getting Date column partition statistics (Andrii via Gopal V) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2018-07-07 14:03:43.334 + rm -rf ../yetus_PreCommit-HIVE-Build-12443 + mkdir ../yetus_PreCommit-HIVE-Build-12443 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-12443 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-12443/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: a/druid-handler/src/java/org/apache/hadoop/hive/druid/serde/DruidSerDe.java: does not exist in index error: a/ql/src/test/queries/clientpositive/druid_timestamptz.q: does not exist in index error: a/ql/src/test/results/clientpositive/druid/druid_timestamptz.q.out: does not exist in index error: patch failed: druid-handler/src/java/org/apache/hadoop/hive/druid/serde/DruidSerDe.java:393 error: repository lacks the necessary blob to fall back on 3-way merge. error: druid-handler/src/java/org/apache/hadoop/hive/druid/serde/DruidSerDe.java: patch does not apply error: src/java/org/apache/hadoop/hive/druid/serde/DruidSerDe.java: does not exist in index error: src/test/queries/clientpositive/druid_timestamptz.q: does not exist in index error: src/test/results/clientpositive/druid/druid_timestamptz.q.out: does not exist in index The patch does not appear to apply with p0, p1, or p2 + result=1 + '[' 1 -ne 0 ']' + rm -rf yetus_PreCommit-HIVE-Build-12443 + exit 1 ' {noformat} This message is automatically generated. ATTACHMENT ID: 12930544 - PreCommit-HIVE-Build > Druid-Hive: tpcds query on timestamp throws > java.lang.IllegalArgumentException: Cannot create timestamp, parsing error > -- > > Key: HIVE-20105 > URL: https://issues.apache.org/jira/browse/HIVE-20105 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Dileep Kumar Chiguruvada >Assignee: Nishant Bangarwa >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-20105.patch > > > Druid-Hive: tpcds query on timestamp trows > java.lang.IllegalArgumentException: Cannot create timestamp, parsing error. > {code} > SELECT `__time`, max(ss_quantity), sum(ss_wholesale_cost) > FROM misc_store_sales_denormal
[jira] [Commented] (HIVE-20085) Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional
[ https://issues.apache.org/jira/browse/HIVE-20085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535775#comment-16535775 ] Hive QA commented on HIVE-20085: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 45s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 19s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 16s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 25s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 31s{color} | {color:blue} common in master has 64 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 10s{color} | {color:blue} standalone-metastore in master has 217 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 55s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 26s{color} | {color:blue} druid-handler in master has 13 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 12s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 24s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 15s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 10s{color} | {color:red} druid-handler: The patch generated 4 new + 64 unchanged - 1 fixed = 68 total (was 65) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 10s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 41m 1s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12444/dev-support/hive-personality.sh | | git revision | master / 8494522 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12444/yetus/diff-checkstyle-druid-handler.txt | | modules | C: common standalone-metastore ql druid-handler U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12444/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Druid-Hive (managed) table creation fails with strict managed table checks: > Table is marked as a managed table but is not transactional > --- > > Key: HIVE-20085 > URL: https://issues.apache.org/jira/browse/HIVE-20085 > Project: Hive > Issue Type: Bug > Components: Hive, StorageHandler >Affects Versions: 3.0.0 >Reporter: Dileep Kumar Chiguruvada >Assignee: Nishant Bangarwa >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-20
[jira] [Commented] (HIVE-20096) dynamic partition pruning results in java.lang.UnsupportedOperationException: Not expecting to handle any events
[ https://issues.apache.org/jira/browse/HIVE-20096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535795#comment-16535795 ] Karthik commented on HIVE-20096: Thank you Gopal for explaining the issue. Have updated hive configuration as below: hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat; hive.tez.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat; hive.compute.splits.in.am=true; Same error occurs with above configuration. > dynamic partition pruning results in java.lang.UnsupportedOperationException: > Not expecting to handle any events > > > Key: HIVE-20096 > URL: https://issues.apache.org/jira/browse/HIVE-20096 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 > Environment: Hive 3.0.0 and Tez 0.91 . >Reporter: Karthik >Priority: Major > > * Setting hive.tez.dynamic.partition.pruning=true partition results in error > "java.lang.UnsupportedOperationException: Not expecting to handle any events" > for some queries. > * Below is stack trace as shown in Hive CLI and JDBC. > -- > VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED > -- > Map 3 llap INITED 1 0 0 1 0 0 > Map 4 .. llap SUCCEEDED 1 1 0 0 0 0 > Map 1 llap INITIALIZING -1 0 0 -1 0 0 > Reducer 2 llap INITED 3 0 0 3 0 0 > -- > VERTICES: 01/04 [=>>-] 20% ELAPSED TIME: 0.56 s > -- > 2018-07-05 06:41:22,813 INFO [32a75ad3-f0de-4255-b1d5-0c919e3bc672 main] > SessionState (SessionState.java:logInfo(1163)) - Map 1: -/- Map 3: 0/1 Map 4: > 1/1 Reducer 2: 0/3 > Status: Failed > 2018-07-05 06:41:22,814 ERROR [32a75ad3-f0de-4255-b1d5-0c919e3bc672 main] > SessionState (SessionState.java:printError(1237)) - Status: Failed > Vertex failed, vertexName=Map 1, vertexId=vertex_1530776596475_0020_4_02, > diagnostics=[Vertex vertex_1530776596475_0020_4_02 [Map 1] killed/failed due > to:ROOT_INPUT_INIT_FAILURE, Vertex Input: my_fact_table initializer failed, > vertex=vertex_1530776596475_0020_4_02 [Map 1], > java.lang.UnsupportedOperationException: Not expecting to handle any events > at > org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.handleInputInitializerEvent(MRInputAMSplitGenerator.java:170) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InitializerWrapper.sendEvents(RootInputInitializerManager.java:518) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InitializerWrapper.handleInputInitializerEvents(RootInputInitializerManager.java:511) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager.handleInitializerEvents(RootInputInitializerManager.java:201) > at > org.apache.tez.dag.app.dag.impl.VertexImpl.handleRoutedTezEvents(VertexImpl.java:4009) > at > org.apache.tez.dag.app.dag.impl.VertexImpl.access$4000(VertexImpl.java:205) > at > org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:3738) > at > org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:3731) > at > org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385) > at > org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302) > at > org.apache.hadoop.yarn.state.StateMachineFactory.access$500(StateMachineFactory.java:46) > at > org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:487) > at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:59) > at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1939) > at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:204) > at > org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2317) > at > org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2303) > at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:180) > at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:115) > at java.lang.Thread.run(Thread.java:748) > ] > 2018-07-05 06:41:22,814 ERROR [32a75ad3-f0de-4255-b1d5-0c919e3bc672 main] > SessionState (SessionState.java:printError(1237)) - Vertex failed, > vertexName=Map 1, vertexId=vertex_1530776596475_0020_4_02, > diagnostics=[Vertex vertex_1530776596475_0020_4_02 [Map 1] killed/failed due
[jira] [Commented] (HIVE-20085) Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional
[ https://issues.apache.org/jira/browse/HIVE-20085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535796#comment-16535796 ] Hive QA commented on HIVE-20085: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930549/HIVE-20085.5.patch {color:green}SUCCESS:{color} +1 due to 15 test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 14631 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12444/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12444/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12444/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12930549 - PreCommit-HIVE-Build > Druid-Hive (managed) table creation fails with strict managed table checks: > Table is marked as a managed table but is not transactional > --- > > Key: HIVE-20085 > URL: https://issues.apache.org/jira/browse/HIVE-20085 > Project: Hive > Issue Type: Bug > Components: Hive, StorageHandler >Affects Versions: 3.0.0 >Reporter: Dileep Kumar Chiguruvada >Assignee: Nishant Bangarwa >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-20085.1.patch, HIVE-20085.2.patch, > HIVE-20085.3.patch, HIVE-20085.4.patch, HIVE-20085.5.patch, HIVE-20085.patch > > > Druid-Hive (managed) table creation fails with strict managed table checks: > Table is marked as a managed table but is not transactional > {code} > drop table if exists calcs; > create table calcs > STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' > TBLPROPERTIES ( > "druid.segment.granularity" = "MONTH", > "druid.query.granularity" = "DAY") > AS SELECT > cast(datetime0 as timestamp with local time zone) `__time`, > key, > str0, str1, str2, str3, > date0, date1, date2, date3, > time0, time1, > datetime0, datetime1, > zzz, > cast(bool0 as string) bool0, > cast(bool1 as string) bool1, > cast(bool2 as string) bool2, > cast(bool3 as string) bool3, > int0, int1, int2, int3, > num0, num1, num2, num3, num4 > from tableau_orc.calcs; > 2018-07-03 04:57:31,911|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Status: Running > (Executing on YARN cluster with App id application_1530592209763_0009) > ... > ... > 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SHUFFLE_BYTES_TO_MEM: > 0 > 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SHUFFLE_PHASE_TIME: > 330 > 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SPILLED_RECORDS: 17 > 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : > TaskCounter_Reducer_2_OUTPUT_out_Reducer_2: > 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : OUTPUT_RECORDS: 0 > 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : > org.apache.hadoop.hive.llap.counters.LlapWmCounters: > 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : GUARANTEED_QUEUED_NS: > 0 > 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : > GUARANTEED_RUNNING_NS: 0 > 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : > SPECULATIVE_QUEUED_NS: 2162643606 > 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : > SPECULATIVE_RUNNING_NS: 12151664909 > 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task > [Stage-2:DEPENDENCY_COLLECTION] in serial mode > 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task > [Stage-0:MOVE] in serial mode > 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Moving data to > directory > hdfs://mycluster/warehouse/tablespace/managed/hive/druid_tableau.db/calcs
[jira] [Assigned] (HIVE-20119) permissions on files in transactional tables
[ https://issues.apache.org/jira/browse/HIVE-20119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman reassigned HIVE-20119: - > permissions on files in transactional tables > > > Key: HIVE-20119 > URL: https://issues.apache.org/jira/browse/HIVE-20119 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Major > > What should these be? With doAs they end up being owned by the user and then > depending on umask cleaner may not be able to delete them - thus compaction > is marked as failed -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20085) Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional
[ https://issues.apache.org/jira/browse/HIVE-20085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-20085: Resolution: Fixed Fix Version/s: (was: 3.0.0) 4.0.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks, Nishant! > Druid-Hive (managed) table creation fails with strict managed table checks: > Table is marked as a managed table but is not transactional > --- > > Key: HIVE-20085 > URL: https://issues.apache.org/jira/browse/HIVE-20085 > Project: Hive > Issue Type: Bug > Components: Hive, StorageHandler >Affects Versions: 3.0.0 >Reporter: Dileep Kumar Chiguruvada >Assignee: Nishant Bangarwa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20085.1.patch, HIVE-20085.2.patch, > HIVE-20085.3.patch, HIVE-20085.4.patch, HIVE-20085.5.patch, HIVE-20085.patch > > > Druid-Hive (managed) table creation fails with strict managed table checks: > Table is marked as a managed table but is not transactional > {code} > drop table if exists calcs; > create table calcs > STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' > TBLPROPERTIES ( > "druid.segment.granularity" = "MONTH", > "druid.query.granularity" = "DAY") > AS SELECT > cast(datetime0 as timestamp with local time zone) `__time`, > key, > str0, str1, str2, str3, > date0, date1, date2, date3, > time0, time1, > datetime0, datetime1, > zzz, > cast(bool0 as string) bool0, > cast(bool1 as string) bool1, > cast(bool2 as string) bool2, > cast(bool3 as string) bool3, > int0, int1, int2, int3, > num0, num1, num2, num3, num4 > from tableau_orc.calcs; > 2018-07-03 04:57:31,911|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Status: Running > (Executing on YARN cluster with App id application_1530592209763_0009) > ... > ... > 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SHUFFLE_BYTES_TO_MEM: > 0 > 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SHUFFLE_PHASE_TIME: > 330 > 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SPILLED_RECORDS: 17 > 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : > TaskCounter_Reducer_2_OUTPUT_out_Reducer_2: > 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : OUTPUT_RECORDS: 0 > 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : > org.apache.hadoop.hive.llap.counters.LlapWmCounters: > 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : GUARANTEED_QUEUED_NS: > 0 > 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : > GUARANTEED_RUNNING_NS: 0 > 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : > SPECULATIVE_QUEUED_NS: 2162643606 > 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : > SPECULATIVE_RUNNING_NS: 12151664909 > 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task > [Stage-2:DEPENDENCY_COLLECTION] in serial mode > 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task > [Stage-0:MOVE] in serial mode > 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Moving data to > directory > hdfs://mycluster/warehouse/tablespace/managed/hive/druid_tableau.db/calcs > from > hdfs://mycluster/warehouse/tablespace/managed/hive/druid_tableau.db/.hive-staging_hive_2018-07-03_04-57-27_351_7124633902209008283-3/-ext-10002 > 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task > [Stage-4:DDL] in serial mode > 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 - > tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|ERROR : FAILED: Execution > Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. > MetaException(message:Table druid_tableau.calcs failed strict managed table > checks due to the following reason: Table is marked as a managed table but is > not transactional.) > 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 -
[jira] [Commented] (HIVE-19806) Several tests do not properly sort their output
[ https://issues.apache.org/jira/browse/HIVE-19806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535826#comment-16535826 ] Hive QA commented on HIVE-19806: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 43s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 30s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 31s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 15s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 55s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m 30s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m 40s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 11s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 64m 14s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12445/dev-support/hive-personality.sh | | git revision | master / 8494522 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | modules | C: ql . itests itests/hive-blobstore itests/hive-unit U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12445/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Several tests do not properly sort their output > --- > > Key: HIVE-19806 > URL: https://issues.apache.org/jira/browse/HIVE-19806 > Project: Hive > Issue Type: Bug > Components: Test >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Attachments: HIVE-19806.2.patch, HIVE-19806.3.patch, > HIVE-19806.4.patch, HIVE-19806.5.patch, HIVE-19806.patch > > > A number of the tests produce unsorted output that happens to come out the > same on people's laptops and the ptest infrastructure. But when run on a > separate linux box the sort differences show up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19806) Several tests do not properly sort their output
[ https://issues.apache.org/jira/browse/HIVE-19806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535849#comment-16535849 ] Hive QA commented on HIVE-19806: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930551/HIVE-19806.5.patch {color:green}SUCCESS:{color} +1 due to 16 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 14616 tests executed *Failed tests:* {noformat} TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestLocationQueries - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestReplicationScenariosAcidTables - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out) (batchId=240) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testCancelRenewTokenFlow (batchId=262) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testConnection (batchId=262) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testIsValid (batchId=262) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testIsValidNeg (batchId=262) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeProxyAuth (batchId=262) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeTokenAuth (batchId=262) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testProxyAuth (batchId=262) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testRenewDelegationToken (batchId=262) org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testTokenAuth (batchId=262) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12445/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12445/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12445/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 15 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12930551 - PreCommit-HIVE-Build > Several tests do not properly sort their output > --- > > Key: HIVE-19806 > URL: https://issues.apache.org/jira/browse/HIVE-19806 > Project: Hive > Issue Type: Bug > Components: Test >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Attachments: HIVE-19806.2.patch, HIVE-19806.3.patch, > HIVE-19806.4.patch, HIVE-19806.5.patch, HIVE-19806.patch > > > A number of the tests produce unsorted output that happens to come out the > same on people's laptops and the ptest infrastructure. But when run on a > separate linux box the sort differences show up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19812) Disable external table replication by default via a configuration property
[ https://issues.apache.org/jira/browse/HIVE-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535852#comment-16535852 ] Hive QA commented on HIVE-19812: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 9s{color} | {color:red} /data/hiveptest/logs/PreCommit-HIVE-Build-12447/patches/PreCommit-HIVE-Build-12447.patch does not apply to master. Rebase required? Wrong Branch? See http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12447/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Disable external table replication by default via a configuration property > -- > > Key: HIVE-19812 > URL: https://issues.apache.org/jira/browse/HIVE-19812 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-19812.01.patch, HIVE-19812.02.patch, > HIVE-19812.03.patch, HIVE-19812.04.patch, HIVE-19812.05.patch, > HIVE-19812.06-branch-3.patch, HIVE-19812.06.patch, HIVE-19812.07.patch, > HIVE-19812.08.patch, HIVE-19812.09.patch, HIVE-19812.10-branch-3.patch, > HIVE-19812.10.patch, HIVE-19812.11.patch > > > use a hive config property to allow external table replication. set this > property by default to prevent external table replication. > for metadata only hive repl always export metadata for external tables. > > REPL_DUMP_EXTERNAL_TABLES("hive.repl.dump.include.external.tables", false, > "Indicates if repl dump should include information about external tables. It > should be \n" > + "used in conjunction with 'hive.repl.dump.metadata.only' set to false. if > 'hive.repl.dump.metadata.only' \n" > + " is set to true then this config parameter has no effect as external table > meta data is flushed \n" > + " always by default.") > This should be done for only replication dump and not for export -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19812) Disable external table replication by default via a configuration property
[ https://issues.apache.org/jira/browse/HIVE-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535864#comment-16535864 ] Hive QA commented on HIVE-19812: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930562/HIVE-19812.10-branch-3.patch {color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 14396 tests executed *Failed tests:* {noformat} TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=257) TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=257) TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=257) TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=257) TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=257) org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testManagedPaths (batchId=234) org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation (batchId=242) org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=309) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12447/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12447/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12447/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 8 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12930562 - PreCommit-HIVE-Build > Disable external table replication by default via a configuration property > -- > > Key: HIVE-19812 > URL: https://issues.apache.org/jira/browse/HIVE-19812 > Project: Hive > Issue Type: Task > Components: repl >Affects Versions: 3.1.0, 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-19812.01.patch, HIVE-19812.02.patch, > HIVE-19812.03.patch, HIVE-19812.04.patch, HIVE-19812.05.patch, > HIVE-19812.06-branch-3.patch, HIVE-19812.06.patch, HIVE-19812.07.patch, > HIVE-19812.08.patch, HIVE-19812.09.patch, HIVE-19812.10-branch-3.patch, > HIVE-19812.10.patch, HIVE-19812.11.patch > > > use a hive config property to allow external table replication. set this > property by default to prevent external table replication. > for metadata only hive repl always export metadata for external tables. > > REPL_DUMP_EXTERNAL_TABLES("hive.repl.dump.include.external.tables", false, > "Indicates if repl dump should include information about external tables. It > should be \n" > + "used in conjunction with 'hive.repl.dump.metadata.only' set to false. if > 'hive.repl.dump.metadata.only' \n" > + " is set to true then this config parameter has no effect as external table > meta data is flushed \n" > + " always by default.") > This should be done for only replication dump and not for export -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20097) Convert standalone-metastore to a submodule
[ https://issues.apache.org/jira/browse/HIVE-20097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535886#comment-16535886 ] Hive QA commented on HIVE-20097: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930577/HIVE-20097.02.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 14627 tests executed *Failed tests:* {noformat} TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestLocationQueries - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out) (batchId=240) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12448/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12448/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12448/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 5 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12930577 - PreCommit-HIVE-Build > Convert standalone-metastore to a submodule > --- > > Key: HIVE-20097 > URL: https://issues.apache.org/jira/browse/HIVE-20097 > Project: Hive > Issue Type: Sub-task > Components: Hive, Metastore, Standalone Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-20097.01.patch, HIVE-20097.02.patch > > > This is a subtask to stage HIVE-17751 changes into several smaller phases. > The first part is moving existing code in hive-standalone-metastore to a > sub-module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535887#comment-16535887 ] Hive QA commented on HIVE-17751: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930593/HIVE-17751.13.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12449/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12449/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12449/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2018-07-07 20:04:55.518 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-12449/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2018-07-07 20:04:55.524 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 406cde9 HIVE-20085 : Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional (Nishant Bangarwa via Ashutosh Chauhan) + git clean -f -d Removing standalone-metastore/metastore-common/ + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 406cde9 HIVE-20085 : Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional (Nishant Bangarwa via Ashutosh Chauhan) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2018-07-07 20:04:57.231 + rm -rf ../yetus_PreCommit-HIVE-Build-12449 + mkdir ../yetus_PreCommit-HIVE-Build-12449 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-12449 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-12449/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch fatal: git apply: bad git-diff - inconsistent old filename on line 2817 error: patch failed: standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java:455 Falling back to three-way merge... Applied patch to 'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java' with conflicts. Going to apply patch with: git apply -p1 /data/hiveptest/working/scratch/build.patch:6932: new blank line at EOF. + error: patch failed: standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java:455 Falling back to three-way merge... Applied patch to 'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java' with conflicts. U standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java warning: 1 line adds whitespace errors. + result=1 + '[' 1 -ne 0 ']' + rm -rf yetus_PreCommit-HIVE-Build-12449 + exit 1 ' {noformat} This message is automatically generated. ATTACHMENT ID: 12930593 - PreCommit-HIVE-Build > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.01.patch, HIVE-17751.02.patch, > HIVE-17751.03.patch, HIVE-17751.04.patch, > HIVE-17751.06-standalone-metastore.patc
[jira] [Commented] (HIVE-20032) Don't serialize hashCode when groupByShuffle and RDD cacheing is disabled
[ https://issues.apache.org/jira/browse/HIVE-20032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535902#comment-16535902 ] Hive QA commented on HIVE-20032: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m 5s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m 32s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 28s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 52s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 56s{color} | {color:blue} common in master has 64 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 40s{color} | {color:blue} spark-client in master has 10 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 6m 13s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 9s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 29s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 7s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 35s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 10s{color} | {color:red} kryo-registrator: The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m 44s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 39s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 44m 10s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12450/dev-support/hive-personality.sh | | git revision | master / 406cde9 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12450/yetus/diff-checkstyle-kryo-registrator.txt | | modules | C: common spark-client ql kryo-registrator U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12450/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Don't serialize hashCode when groupByShuffle and RDD cacheing is disabled > - > > Key: HIVE-20032 > URL: https://issues.apache.org/jira/browse/HIVE-20032 > Project: Hive > Issue Type: Improvement > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-20032.1.patch, HIVE-20032.2.patch > > > Follow up on HIVE-15104, if we don't enable RDD cacheing or groupByShuffles, > then we don't need to serialize the hashCode when shuffling data in HoS. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20097) Convert standalone-metastore to a submodule
[ https://issues.apache.org/jira/browse/HIVE-20097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535910#comment-16535910 ] Hive QA commented on HIVE-20097: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 20s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 14s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m 43s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 11m 11s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 24s{color} | {color:blue} upgrade-acid in master has 1 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 2s{color} | {color:blue} standalone-metastore in master has 217 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 52s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 39s{color} | {color:blue} llap-server in master has 84 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 38s{color} | {color:blue} service in master has 48 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 35s{color} | {color:blue} hcatalog/core in master has 33 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 28s{color} | {color:blue} hcatalog/webhcat/java-client in master has 3 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 32s{color} | {color:blue} hcatalog/webhcat/svr in master has 96 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 44s{color} | {color:blue} itests/util in master has 52 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 11m 41s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m 22s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 20m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 20m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 19s{color} | {color:green} The patch upgrade-acid passed checkstyle {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 12s{color} | {color:green} standalone-metastore: The patch generated 0 new + 0 unchanged - 5505 fixed = 0 total (was 5505) {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 11s{color} | {color:green} The patch metastore-common passed checkstyle {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 46s{color} | {color:green} root: The patch generated 0 new + 0 unchanged - 5505 fixed = 0 total (was 5505) {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 21s{color} | {color:green} The patch core passed checkstyle {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 17s{color} | {color:green} The patch java-client passed checkstyle {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 17s{color} | {color:green} The patch svr passed checkstyle {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 19s{color} | {color:green} The patch hcatalog-unit passed checkstyle {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}
[jira] [Commented] (HIVE-20032) Don't serialize hashCode when groupByShuffle and RDD cacheing is disabled
[ https://issues.apache.org/jira/browse/HIVE-20032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535918#comment-16535918 ] Hive QA commented on HIVE-20032: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930601/HIVE-20032.2.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 361 failed/errored test(s), 14628 tests executed *Failed tests:* {noformat} TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestLocationQueries - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out) (batchId=240) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_dynamic_partition] (batchId=190) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_expressions] (batchId=190) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test1] (batchId=190) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_alter] (batchId=190) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_insert] (batchId=190) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[dynamic_rdd_cache] (batchId=185) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[infer_bucket_sort_map_operators] (batchId=186) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[infer_bucket_sort_num_buckets] (batchId=185) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_constprog_dpp] (batchId=186) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning] (batchId=184) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_2] (batchId=186) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_3] (batchId=186) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_4] (batchId=186) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_6] (batchId=185) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_7] (batchId=185) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1] (batchId=185) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_in_process_launcher] (batchId=184) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_multi_insert_parallel_orderby] (batchId=186) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_opt_shuffle_serde] (batchId=185) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_use_op_stats] (batchId=184) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=184) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[vector_outer_join1] (batchId=185) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[vector_outer_join2] (batchId=184) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join0] (batchId=147) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join10] (batchId=123) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join11] (batchId=111) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join12] (batchId=118) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join13] (batchId=144) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join15] (batchId=114) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join16] (batchId=125) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join18] (batchId=112) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join18_multi_distinct] (batchId=119) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join20] (batchId=148) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join22] (batchId=132) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join24] (batchId=141) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join26] (batchId=113) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join27] (batchId=148) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join30] (batchId=121) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join31] (batchId=128) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_smb_mapjoi
[jira] [Commented] (HIVE-20032) Don't serialize hashCode when groupByShuffle and RDD cacheing is disabled
[ https://issues.apache.org/jira/browse/HIVE-20032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535919#comment-16535919 ] Hive QA commented on HIVE-20032: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930601/HIVE-20032.2.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12451/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12451/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12451/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Tests exited with: Exception: Patch URL https://issues.apache.org/jira/secure/attachment/12930601/HIVE-20032.2.patch was found in seen patch url's cache and a test was probably run already on it. Aborting... {noformat} This message is automatically generated. ATTACHMENT ID: 12930601 - PreCommit-HIVE-Build > Don't serialize hashCode when groupByShuffle and RDD cacheing is disabled > - > > Key: HIVE-20032 > URL: https://issues.apache.org/jira/browse/HIVE-20032 > Project: Hive > Issue Type: Improvement > Components: Spark >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-20032.1.patch, HIVE-20032.2.patch > > > Follow up on HIVE-15104, if we don't enable RDD cacheing or groupByShuffles, > then we don't need to serialize the hashCode when shuffling data in HoS. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20111) HBase-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional
[ https://issues.apache.org/jira/browse/HIVE-20111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535920#comment-16535920 ] Hive QA commented on HIVE-20111: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930615/HIVE-20111.01.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12452/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12452/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12452/ Messages: {noformat} This message was trimmed, see log for full details + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2018-07-07 22:21:24.536 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 406cde9 HIVE-20085 : Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional (Nishant Bangarwa via Ashutosh Chauhan) + git clean -f -d + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 406cde9 HIVE-20085 : Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional (Nishant Bangarwa via Ashutosh Chauhan) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2018-07-07 22:21:26.260 + rm -rf ../yetus_PreCommit-HIVE-Build-12452 + mkdir ../yetus_PreCommit-HIVE-Build-12452 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-12452 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-12452/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: a/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseMetaHook.java: does not exist in index error: a/hbase-handler/src/test/queries/negative/cascade_dbdrop.q: does not exist in index error: a/hbase-handler/src/test/queries/negative/cascade_dbdrop_hadoop20.q: does not exist in index error: a/hbase-handler/src/test/queries/negative/generatehfiles_require_family_path.q: does not exist in index error: a/hbase-handler/src/test/queries/negative/hbase_ddl.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/external_table_ppd.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_binary_binary.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_binary_map_queries.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_binary_map_queries_prefix.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_binary_storage_queries.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_custom_key.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_custom_key2.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_custom_key3.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_ddl.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_decimal_decimal.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_handler_bulk.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_joins.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_null_first_col.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_ppd_join.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_ppd_key_range.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_pushdown.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_queries.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_scan_params.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_single_sourced_multi_insert.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_timestamp.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_timestamp_format.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbase_viewjoins.q: does not exist in index error: a/hbase-handler/src/test/queries/positive/hbasestats.q: does not exist in index error: a/hbase-handler
[jira] [Updated] (HIVE-20097) Convert standalone-metastore to a submodule
[ https://issues.apache.org/jira/browse/HIVE-20097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-20097: -- Attachment: HIVE-20097.03.patch > Convert standalone-metastore to a submodule > --- > > Key: HIVE-20097 > URL: https://issues.apache.org/jira/browse/HIVE-20097 > Project: Hive > Issue Type: Sub-task > Components: Hive, Metastore, Standalone Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-20097.01.patch, HIVE-20097.02.patch, > HIVE-20097.03.patch > > > This is a subtask to stage HIVE-17751 changes into several smaller phases. > The first part is moving existing code in hive-standalone-metastore to a > sub-module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20097) Convert standalone-metastore to a submodule
[ https://issues.apache.org/jira/browse/HIVE-20097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535925#comment-16535925 ] Alexander Kolbasov commented on HIVE-20097: --- Patch 03 based on commit {code} * commit 406cde9883a0a9f2d740c8c9e0104b5c06a4fbff (origin/master, origin/HEAD) | Author: Nishant Bangarwa | Date: Sat Jul 7 09:22:17 2018 -0700 | | HIVE-20085 : Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional (Nishant Bangarwa via Ashutosh Chauhan) {code} > Convert standalone-metastore to a submodule > --- > > Key: HIVE-20097 > URL: https://issues.apache.org/jira/browse/HIVE-20097 > Project: Hive > Issue Type: Sub-task > Components: Hive, Metastore, Standalone Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-20097.01.patch, HIVE-20097.02.patch, > HIVE-20097.03.patch > > > This is a subtask to stage HIVE-17751 changes into several smaller phases. > The first part is moving existing code in hive-standalone-metastore to a > sub-module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20100) OpTraits : Select Optraits should stop when a mismatch is detected
[ https://issues.apache.org/jira/browse/HIVE-20100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535927#comment-16535927 ] Hive QA commented on HIVE-20100: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 57s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 38s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 1s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 22s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 8s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 23m 21s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12453/dev-support/hive-personality.sh | | git revision | master / 406cde9 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12453/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > OpTraits : Select Optraits should stop when a mismatch is detected > -- > > Key: HIVE-20100 > URL: https://issues.apache.org/jira/browse/HIVE-20100 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20100.1.patch, HIVE-20100.2.patch, > HIVE-20100.3.patch > > > The select operator's optraits logic as stated in the comment is, > // For bucket columns > // If all the columns match to the parent, put them in the bucket cols > // else, add empty list. > // For sort columns > // Keep the subset of all the columns as long as order is maintained. > > However, this is not happening due to a bug. The bool found is never reset, > so if a single match is found, the value remains true and allows the optraits > get populated with partial list of columns for bucket col which is incorrect. > This may lead to creation of SMB join which should not happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20100) OpTraits : Select Optraits should stop when a mismatch is detected
[ https://issues.apache.org/jira/browse/HIVE-20100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535938#comment-16535938 ] Hive QA commented on HIVE-20100: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930629/HIVE-20100.3.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14631 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMmConversionLocks (batchId=310) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12453/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12453/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12453/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12930629 - PreCommit-HIVE-Build > OpTraits : Select Optraits should stop when a mismatch is detected > -- > > Key: HIVE-20100 > URL: https://issues.apache.org/jira/browse/HIVE-20100 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20100.1.patch, HIVE-20100.2.patch, > HIVE-20100.3.patch > > > The select operator's optraits logic as stated in the comment is, > // For bucket columns > // If all the columns match to the parent, put them in the bucket cols > // else, add empty list. > // For sort columns > // Keep the subset of all the columns as long as order is maintained. > > However, this is not happening due to a bug. The bool found is never reset, > so if a single match is found, the value remains true and allows the optraits > get populated with partial list of columns for bucket col which is incorrect. > This may lead to creation of SMB join which should not happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20112) Accumulo-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional
[ https://issues.apache.org/jira/browse/HIVE-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535939#comment-16535939 ] Hive QA commented on HIVE-20112: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930616/HIVE-20112.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12454/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12454/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12454/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2018-07-07 23:38:04.365 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-12454/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2018-07-07 23:38:04.369 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 406cde9 HIVE-20085 : Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional (Nishant Bangarwa via Ashutosh Chauhan) + git clean -f -d + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 406cde9 HIVE-20085 : Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional (Nishant Bangarwa via Ashutosh Chauhan) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2018-07-07 23:38:05.586 + rm -rf ../yetus_PreCommit-HIVE-Build-12454 + mkdir ../yetus_PreCommit-HIVE-Build-12454 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-12454 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-12454/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: a/accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/AccumuloStorageHandler.java: does not exist in index error: a/accumulo-handler/src/test/org/apache/hadoop/hive/accumulo/TestAccumuloStorageHandler.java: does not exist in index error: a/accumulo-handler/src/test/queries/positive/accumulo_custom_key.q: does not exist in index error: a/accumulo-handler/src/test/queries/positive/accumulo_custom_key2.q: does not exist in index error: a/accumulo-handler/src/test/queries/positive/accumulo_index.q: does not exist in index error: a/accumulo-handler/src/test/queries/positive/accumulo_joins.q: does not exist in index error: a/accumulo-handler/src/test/queries/positive/accumulo_predicate_pushdown.q: does not exist in index error: a/accumulo-handler/src/test/queries/positive/accumulo_queries.q: does not exist in index error: a/accumulo-handler/src/test/queries/positive/accumulo_single_sourced_multi_insert.q: does not exist in index error: a/accumulo-handler/src/test/results/positive/accumulo_custom_key.q.out: does not exist in index error: a/accumulo-handler/src/test/results/positive/accumulo_custom_key2.q.out: does not exist in index error: a/accumulo-handler/src/test/results/positive/accumulo_index.q.out: does not exist in index error: a/accumulo-handler/src/test/results/positive/accumulo_joins.q.out: does not exist in index error: a/accumulo-handler/src/test/results/positive/accumulo_predicate_pushdown.q.out: does not exist in index error: a/accumulo-handler/src/test/results/positive/accumulo_queries.q.out: does not exist in index error: a/accumulo-handler/src/test/results/positive/accumulo_single_sourced_multi_insert.q.out: does not exist in index error: patch failed: accumulo-handler/src/test/queries/positive/accumulo_custom_key.q:1 Falling back
[jira] [Commented] (HIVE-20111) HBase-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional
[ https://issues.apache.org/jira/browse/HIVE-20111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535941#comment-16535941 ] Hive QA commented on HIVE-20111: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930615/HIVE-20111.01.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12455/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12455/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12455/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Tests exited with: Exception: Patch URL https://issues.apache.org/jira/secure/attachment/12930615/HIVE-20111.01.patch was found in seen patch url's cache and a test was probably run already on it. Aborting... {noformat} This message is automatically generated. ATTACHMENT ID: 12930615 - PreCommit-HIVE-Build > HBase-Hive (managed) table creation fails with strict managed table checks: > Table is marked as a managed table but is not transactional > --- > > Key: HIVE-20111 > URL: https://issues.apache.org/jira/browse/HIVE-20111 > Project: Hive > Issue Type: Bug > Components: Hive, StorageHandler >Affects Versions: 3.0.0 >Reporter: Romil Choksi >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-20111.01.patch > > > Similar to HIVE-20085. HBase-Hive (managed) table creation fails with strict > managed table checks: Table is marked as a managed table but is not > transactional -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-17751: -- Attachment: HIVE-17751.14.patch > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.01.patch, HIVE-17751.02.patch, > HIVE-17751.03.patch, HIVE-17751.04.patch, > HIVE-17751.06-standalone-metastore.patch, HIVE-17751.07.patch, > HIVE-17751.08.patch, HIVE-17751.09.patch, HIVE-17751.10.patch, > HIVE-17751.11.patch, HIVE-17751.13.patch, HIVE-17751.14.patch > > > external applications which are interfacing with HMS should ideally only > include HMSClient library instead of one big library containing server as > well. We should ideally have a thin client library so that cross version > support for external applications is easier. We should sub-divide the > standalone module into possibly 3 modules (one for common classes, one for > client classes and one for server) or 2 sub-modules (one for client and one > for server) so that we can generate separate jars for HMS client and server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535943#comment-16535943 ] Alexander Kolbasov commented on HIVE-17751: --- Patch 14 merged with {code} * commit 406cde9883a0a9f2d740c8c9e0104b5c06a4fbff (origin/master, origin/HEAD, master) | Author: Nishant Bangarwa | Date: Sat Jul 7 09:22:17 2018 -0700 | | HIVE-20085 : Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional (Nishant Bangarwa via Ashutosh Chauhan) {code} > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.01.patch, HIVE-17751.02.patch, > HIVE-17751.03.patch, HIVE-17751.04.patch, > HIVE-17751.06-standalone-metastore.patch, HIVE-17751.07.patch, > HIVE-17751.08.patch, HIVE-17751.09.patch, HIVE-17751.10.patch, > HIVE-17751.11.patch, HIVE-17751.13.patch, HIVE-17751.14.patch > > > external applications which are interfacing with HMS should ideally only > include HMSClient library instead of one big library containing server as > well. We should ideally have a thin client library so that cross version > support for external applications is easier. We should sub-divide the > standalone module into possibly 3 modules (one for common classes, one for > client classes and one for server) or 2 sub-modules (one for client and one > for server) so that we can generate separate jars for HMS client and server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20001) With doas set to true, running select query as hrt_qa user on external table fails due to permission denied to read /warehouse/tablespace/managed directory.
[ https://issues.apache.org/jira/browse/HIVE-20001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535945#comment-16535945 ] Dongjoon Hyun commented on HIVE-20001: -- This patch has a bug for `non-hive` catalog. For `non-hive` catalog, warehouse/database/tables are owned by non-hive users previously. 1) the warehouse is owned by non-hive users. 2) the tables are created as the table owners (also non-hive users). 3) this patch enforces CREATE DATABASE as a admin, `hive`, and breaks table creations in (2). Could you add a test case for that and fix it? > With doas set to true, running select query as hrt_qa user on external table > fails due to permission denied to read /warehouse/tablespace/managed > directory. > > > Key: HIVE-20001 > URL: https://issues.apache.org/jira/browse/HIVE-20001 > Project: Hive > Issue Type: Bug >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20001.1.patch, HIVE-20001.1.patch, > HIVE-20001.2.patch, HIVE-20001.3.patch, HIVE-20001.4.patch, HIVE-20001.5.patch > > > Hive: With doas set to true, running select query as hrt_qa user on external > table fails due to permission denied to read /warehouse/tablespace/managed > directory. > Steps: > 1. Create a external table. > 2. Set doas to true. > 3. run select count(*) using user hrt_qa. > Table creation query. > {code} > beeline -n hrt_qa -p pwd -u > "jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit" > --outputformat=tsv -e "drop table if exists test_table purge; > create external table test_table(id int, age int) row format delimited fields > terminated by '|' stored as textfile; > load data inpath '/tmp/table1.dat' overwrite into table test_table; > {code} > select count(*) query execution fails > {code} > beeline -n hrt_qa -p pwd -u > "jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit" > --outputformat=tsv -e "select count(*) from test_table where age>30 and > id<10100;" > 2018-06-22 10:22:29,328|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: Class path contains > multiple SLF4J bindings. > 2018-06-22 10:22:29,330|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: See > http://www.slf4j.org/codes.html#multiple_bindings for an explanation. > 2018-06-22 10:22:29,335|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: Actual binding is of > type [org.apache.logging.slf4j.Log4jLoggerFactory] > 2018-06-22 10:22:31,408|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|Format tsv is deprecated, > please use tsv2 > 2018-06-22 10:22:31,529|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|Connecting to > jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit > 2018-06-22 10:22:32,031|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22 10:22:32 [main]: > INFO jdbc.HiveConnection: Connected to > ctr-e138-1518143905142-375925-01-04.hwx.site:10001 > 2018-06-22 10:22:34,130|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22 10:22:34 [main]: > WARN jdbc.HiveConnection: Failed to connect to > ctr-e138-1518143905142-375925-01-04.hwx.site:10001 > 2018-06-22 10:22:34,244|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22 10:22:34 [main]: > WARN jdbc.HiveConnection: Could not open client transport with JDBC Uri: > jdbc:hive2://ctr-e138-1518143905142-37592
[jira] [Updated] (HIVE-20001) With doas set to true, running select query as hrt_qa user on external table fails due to permission denied to read /warehouse/tablespace/managed directory.
[ https://issues.apache.org/jira/browse/HIVE-20001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated HIVE-20001: - Priority: Critical (was: Major) > With doas set to true, running select query as hrt_qa user on external table > fails due to permission denied to read /warehouse/tablespace/managed > directory. > > > Key: HIVE-20001 > URL: https://issues.apache.org/jira/browse/HIVE-20001 > Project: Hive > Issue Type: Bug >Reporter: Jaume M >Assignee: Jaume M >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-20001.1.patch, HIVE-20001.1.patch, > HIVE-20001.2.patch, HIVE-20001.3.patch, HIVE-20001.4.patch, HIVE-20001.5.patch > > > Hive: With doas set to true, running select query as hrt_qa user on external > table fails due to permission denied to read /warehouse/tablespace/managed > directory. > Steps: > 1. Create a external table. > 2. Set doas to true. > 3. run select count(*) using user hrt_qa. > Table creation query. > {code} > beeline -n hrt_qa -p pwd -u > "jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit" > --outputformat=tsv -e "drop table if exists test_table purge; > create external table test_table(id int, age int) row format delimited fields > terminated by '|' stored as textfile; > load data inpath '/tmp/table1.dat' overwrite into table test_table; > {code} > select count(*) query execution fails > {code} > beeline -n hrt_qa -p pwd -u > "jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit" > --outputformat=tsv -e "select count(*) from test_table where age>30 and > id<10100;" > 2018-06-22 10:22:29,328|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: Class path contains > multiple SLF4J bindings. > 2018-06-22 10:22:29,330|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: See > http://www.slf4j.org/codes.html#multiple_bindings for an explanation. > 2018-06-22 10:22:29,335|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: Actual binding is of > type [org.apache.logging.slf4j.Log4jLoggerFactory] > 2018-06-22 10:22:31,408|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|Format tsv is deprecated, > please use tsv2 > 2018-06-22 10:22:31,529|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|Connecting to > jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit > 2018-06-22 10:22:32,031|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22 10:22:32 [main]: > INFO jdbc.HiveConnection: Connected to > ctr-e138-1518143905142-375925-01-04.hwx.site:10001 > 2018-06-22 10:22:34,130|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22 10:22:34 [main]: > WARN jdbc.HiveConnection: Failed to connect to > ctr-e138-1518143905142-375925-01-04.hwx.site:10001 > 2018-06-22 10:22:34,244|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22 10:22:34 [main]: > WARN jdbc.HiveConnection: Could not open client transport with JDBC Uri: > jdbc:hive2://ctr-e138-1518143905142-375925-01-04.hwx.site:10001/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit: > Failed to open new session: > org.apache.hadoop.hive.ql.metadata.HiveException: > MetaException(message:java.security.AccessControlException: Permiss
[jira] [Updated] (HIVE-20001) With doas set to true, running select query as hrt_qa user on external table fails due to permission denied to read /warehouse/tablespace/managed directory.
[ https://issues.apache.org/jira/browse/HIVE-20001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated HIVE-20001: - Affects Version/s: 3.0.0 > With doas set to true, running select query as hrt_qa user on external table > fails due to permission denied to read /warehouse/tablespace/managed > directory. > > > Key: HIVE-20001 > URL: https://issues.apache.org/jira/browse/HIVE-20001 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Jaume M >Assignee: Jaume M >Priority: Critical > Labels: pull-request-available > Attachments: HIVE-20001.1.patch, HIVE-20001.1.patch, > HIVE-20001.2.patch, HIVE-20001.3.patch, HIVE-20001.4.patch, HIVE-20001.5.patch > > > Hive: With doas set to true, running select query as hrt_qa user on external > table fails due to permission denied to read /warehouse/tablespace/managed > directory. > Steps: > 1. Create a external table. > 2. Set doas to true. > 3. run select count(*) using user hrt_qa. > Table creation query. > {code} > beeline -n hrt_qa -p pwd -u > "jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit" > --outputformat=tsv -e "drop table if exists test_table purge; > create external table test_table(id int, age int) row format delimited fields > terminated by '|' stored as textfile; > load data inpath '/tmp/table1.dat' overwrite into table test_table; > {code} > select count(*) query execution fails > {code} > beeline -n hrt_qa -p pwd -u > "jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit" > --outputformat=tsv -e "select count(*) from test_table where age>30 and > id<10100;" > 2018-06-22 10:22:29,328|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: Class path contains > multiple SLF4J bindings. > 2018-06-22 10:22:29,330|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: See > http://www.slf4j.org/codes.html#multiple_bindings for an explanation. > 2018-06-22 10:22:29,335|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: Actual binding is of > type [org.apache.logging.slf4j.Log4jLoggerFactory] > 2018-06-22 10:22:31,408|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|Format tsv is deprecated, > please use tsv2 > 2018-06-22 10:22:31,529|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|Connecting to > jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit > 2018-06-22 10:22:32,031|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22 10:22:32 [main]: > INFO jdbc.HiveConnection: Connected to > ctr-e138-1518143905142-375925-01-04.hwx.site:10001 > 2018-06-22 10:22:34,130|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22 10:22:34 [main]: > WARN jdbc.HiveConnection: Failed to connect to > ctr-e138-1518143905142-375925-01-04.hwx.site:10001 > 2018-06-22 10:22:34,244|INFO|Thread-126|machine.py:111 - > tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22 10:22:34 [main]: > WARN jdbc.HiveConnection: Could not open client transport with JDBC Uri: > jdbc:hive2://ctr-e138-1518143905142-375925-01-04.hwx.site:10001/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit: > Failed to open new session: > org.apache.hadoop.hive.ql.metadata.HiveException: > MetaException(message:java.security.AccessCont
[jira] [Commented] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[ https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535946#comment-16535946 ] Hive QA commented on HIVE-20113: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 48s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 37s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 53s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 32s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 38s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 23m 13s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12456/dev-support/hive-personality.sh | | git revision | master / 406cde9 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | whitespace | http://104.198.109.242/logs//PreCommit-HIVE-Build-12456/yetus/whitespace-eol.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12456/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Shuffle avoidance: Disable 1-1 edges for sorted shuffle > > > Key: HIVE-20113 > URL: https://issues.apache.org/jira/browse/HIVE-20113 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate > Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch > > > The sorted shuffle avoidance can have some issues when the shuffle data gets > broken up into multiple chunks on disk. > The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to > have a final merge at all, it should open a single compressed file and write > a single index entry. > Until the shuffle issue is resolved & a lot more testing, it is prudent to > disable the optimization for sorted shuffle edges and stop rewriting the > RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20100) OpTraits : Select Optraits should stop when a mismatch is detected
[ https://issues.apache.org/jira/browse/HIVE-20100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-20100: -- Attachment: HIVE-20100.4.patch > OpTraits : Select Optraits should stop when a mismatch is detected > -- > > Key: HIVE-20100 > URL: https://issues.apache.org/jira/browse/HIVE-20100 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20100.1.patch, HIVE-20100.2.patch, > HIVE-20100.3.patch, HIVE-20100.4.patch > > > The select operator's optraits logic as stated in the comment is, > // For bucket columns > // If all the columns match to the parent, put them in the bucket cols > // else, add empty list. > // For sort columns > // Keep the subset of all the columns as long as order is maintained. > > However, this is not happening due to a bug. The bool found is never reset, > so if a single match is found, the value remains true and allows the optraits > get populated with partial list of columns for bucket col which is incorrect. > This may lead to creation of SMB join which should not happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[ https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535949#comment-16535949 ] Hive QA commented on HIVE-20113: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930622/HIVE-20113.2.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 33 failed/errored test(s), 14627 tests executed *Failed tests:* {noformat} TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestLocationQueries - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out) (batchId=240) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timestamptz] (batchId=191) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_joins] (batchId=191) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_masking] (batchId=191) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[parallel_colstats] (batchId=151) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_no_buckets] (batchId=172) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_4] (batchId=158) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query10] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query2] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query31] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query32] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query33] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query35] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query45] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query56] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query58] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query60] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query64] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query69] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query6] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query77] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query78] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query83] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query92] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query95] (batchId=261) org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query97] (batchId=261) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12456/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12456/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12456/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 33 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12930622 - PreCommit-HIVE-Build > Shuffle avoidance: Disable 1-1 edges for sorted shuffle > > > Key: HIVE-20113 > URL: https://issues.apache.org/jira/browse/HIVE-20113 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate > Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch > > > The sorted shuffle avoidance can have some issues when the shuffle data gets > broken up into multiple chunks on disk. > The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to > have a final merge at all, it should open a single compressed file
[jira] [Commented] (HIVE-20116) TezTask is using parent logger
[ https://issues.apache.org/jira/browse/HIVE-20116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535953#comment-16535953 ] Hive QA commented on HIVE-20116: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 12s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 38s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 50s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 38s{color} | {color:red} ql: The patch generated 1 new + 21 unchanged - 0 fixed = 22 total (was 21) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m 12s{color} | {color:red} ql generated 2 new + 2287 unchanged - 0 fixed = 2289 total (was 2287) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 23m 35s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | Class org.apache.hadoop.hive.ql.exec.tez.TezTask defines non-transient non-serializable instance field dagClient In TezTask.java:instance field dagClient In TezTask.java | | | org.apache.hadoop.hive.ql.exec.tez.TezTask is Serializable; consider declaring a serialVersionUID At TezTask.java:a serialVersionUID At TezTask.java:[lines 98-684] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12457/dev-support/hive-personality.sh | | git revision | master / 406cde9 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12457/yetus/diff-checkstyle-ql.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-12457/yetus/new-findbugs-ql.html | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12457/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > TezTask is using parent logger > -- > > Key: HIVE-20116 > URL: https://issues.apache.org/jira/browse/HIVE-20116 > Project: Hive > Issue Type: Bug > Components: Logging >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20116.1.patch > > > TezTask is using parent's logger (Task). It should instead use its own class > name. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20116) TezTask is using parent logger
[ https://issues.apache.org/jira/browse/HIVE-20116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535963#comment-16535963 ] Hive QA commented on HIVE-20116: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930620/HIVE-20116.1.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 14616 tests executed *Failed tests:* {noformat} TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestLocationQueries - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestReplicationScenariosAcidTables - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out) (batchId=240) org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithTez.testFetchResultsOfLogWithExecutionMode (batchId=245) org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithTez.testFetchResultsOfLogWithPerformanceMode (batchId=245) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12457/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12457/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12457/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 8 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12930620 - PreCommit-HIVE-Build > TezTask is using parent logger > -- > > Key: HIVE-20116 > URL: https://issues.apache.org/jira/browse/HIVE-20116 > Project: Hive > Issue Type: Bug > Components: Logging >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20116.1.patch > > > TezTask is using parent's logger (Task). It should instead use its own class > name. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[ https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535967#comment-16535967 ] Hive QA commented on HIVE-20113: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930622/HIVE-20113.2.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12459/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12459/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12459/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Tests exited with: Exception: Patch URL https://issues.apache.org/jira/secure/attachment/12930622/HIVE-20113.2.patch was found in seen patch url's cache and a test was probably run already on it. Aborting... {noformat} This message is automatically generated. ATTACHMENT ID: 12930622 - PreCommit-HIVE-Build > Shuffle avoidance: Disable 1-1 edges for sorted shuffle > > > Key: HIVE-20113 > URL: https://issues.apache.org/jira/browse/HIVE-20113 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate > Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch > > > The sorted shuffle avoidance can have some issues when the shuffle data gets > broken up into multiple chunks on disk. > The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to > have a final merge at all, it should open a single compressed file and write > a single index entry. > Until the shuffle issue is resolved & a lot more testing, it is prudent to > disable the optimization for sorted shuffle edges and stop rewriting the > RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-12316) Improved integration test for Hive
[ https://issues.apache.org/jira/browse/HIVE-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535964#comment-16535964 ] Hive QA commented on HIVE-12316: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12776619/HIVE-12316.5.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12458/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12458/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12458/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2018-07-08 02:14:21.668 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-12458/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2018-07-08 02:14:21.671 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 406cde9 HIVE-20085 : Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional (Nishant Bangarwa via Ashutosh Chauhan) + git clean -f -d Removing itests/${project.basedir}/ + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 406cde9 HIVE-20085 : Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional (Nishant Bangarwa via Ashutosh Chauhan) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2018-07-08 02:14:23.173 + rm -rf ../yetus_PreCommit-HIVE-Build-12458 + mkdir ../yetus_PreCommit-HIVE-Build-12458 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-12458 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-12458/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: patch failed: hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java:387 Falling back to three-way merge... Applied patch to 'hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java' with conflicts. error: itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java: does not exist in index error: patch failed: itests/pom.xml:42 Falling back to three-way merge... Applied patch to 'itests/pom.xml' with conflicts. error: streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java: does not exist in index error: hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java: does not exist in index error: patch failed: pom.xml:42 Falling back to three-way merge... Applied patch to 'pom.xml' with conflicts. fatal: git diff header lacks filename information when removing 2 leading pathname components (line 16188) The patch does not appear to apply with p0, p1, or p2 + result=1 + '[' 1 -ne 0 ']' + rm -rf yetus_PreCommit-HIVE-Build-12458 + exit 1 ' {noformat} This message is automatically generated. ATTACHMENT ID: 12776619 - PreCommit-HIVE-Build > Improved integration test for Hive > -- > > Key: HIVE-12316 > URL: https://issues.apache.org/jira/browse/HIVE-12316 > Project: Hive > Issue Type: New Feature > Components: Testing Infrastructure >Affects Versions: 2.0.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Attachments: HIVE-12316.2.patch, HIVE-12316.5.patch, HIVE-12316.patch > > > In working with Hive testing I have found there are several issues that are > causing problems for developers, test
[jira] [Commented] (HIVE-20117) schema changes for txn stats
[ https://issues.apache.org/jira/browse/HIVE-20117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535970#comment-16535970 ] Hive QA commented on HIVE-20117: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 55s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 11m 2s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12460/dev-support/hive-personality.sh | | git revision | master / 406cde9 | | Default Java | 1.8.0_111 | | modules | C: standalone-metastore U: standalone-metastore | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12460/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > schema changes for txn stats > > > Key: HIVE-20117 > URL: https://issues.apache.org/jira/browse/HIVE-20117 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20117.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20117) schema changes for txn stats
[ https://issues.apache.org/jira/browse/HIVE-20117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535981#comment-16535981 ] Hive QA commented on HIVE-20117: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930625/HIVE-20117.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 14616 tests executed *Failed tests:* {noformat} TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestLocationQueries - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestReplicationScenariosAcidTables - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out) (batchId=240) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12460/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12460/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12460/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12930625 - PreCommit-HIVE-Build > schema changes for txn stats > > > Key: HIVE-20117 > URL: https://issues.apache.org/jira/browse/HIVE-20117 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20117.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20100) OpTraits : Select Optraits should stop when a mismatch is detected
[ https://issues.apache.org/jira/browse/HIVE-20100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535985#comment-16535985 ] Hive QA commented on HIVE-20100: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 56s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 37s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 5s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 3s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 10s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 23m 27s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12461/dev-support/hive-personality.sh | | git revision | master / 406cde9 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12461/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > OpTraits : Select Optraits should stop when a mismatch is detected > -- > > Key: HIVE-20100 > URL: https://issues.apache.org/jira/browse/HIVE-20100 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20100.1.patch, HIVE-20100.2.patch, > HIVE-20100.3.patch, HIVE-20100.4.patch > > > The select operator's optraits logic as stated in the comment is, > // For bucket columns > // If all the columns match to the parent, put them in the bucket cols > // else, add empty list. > // For sort columns > // Keep the subset of all the columns as long as order is maintained. > > However, this is not happening due to a bug. The bool found is never reset, > so if a single match is found, the value remains true and allows the optraits > get populated with partial list of columns for bucket col which is incorrect. > This may lead to creation of SMB join which should not happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20100) OpTraits : Select Optraits should stop when a mismatch is detected
[ https://issues.apache.org/jira/browse/HIVE-20100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535989#comment-16535989 ] Hive QA commented on HIVE-20100: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930655/HIVE-20100.4.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 14628 tests executed *Failed tests:* {noformat} TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestLocationQueries - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely timed out) (batchId=240) TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out) (batchId=240) org.apache.hadoop.hive.metastore.client.TestNotNullConstraint.addNoSuchTable[Remote] (batchId=218) org.apache.hadoop.hive.metastore.client.TestNotNullConstraint.createGetDrop[Remote] (batchId=218) org.apache.hadoop.hive.metastore.client.TestNotNullConstraint.createTableWithConstraintsPkInOtherCatalog[Remote] (batchId=218) org.apache.hadoop.hive.metastore.client.TestNotNullConstraint.createTableWithConstraintsPk[Remote] (batchId=218) org.apache.hadoop.hive.metastore.client.TestNotNullConstraint.doubleAddNotNullConstraint[Remote] (batchId=218) org.apache.hadoop.hive.metastore.client.TestNotNullConstraint.getNoSuchCatalog[Remote] (batchId=218) org.apache.hadoop.hive.metastore.client.TestNotNullConstraint.getNoSuchDb[Remote] (batchId=218) org.apache.hadoop.hive.metastore.client.TestNotNullConstraint.getNoSuchTable[Remote] (batchId=218) org.apache.hadoop.hive.metastore.client.TestNotNullConstraint.inOtherCatalog[Remote] (batchId=218) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12461/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12461/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12461/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 13 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12930655 - PreCommit-HIVE-Build > OpTraits : Select Optraits should stop when a mismatch is detected > -- > > Key: HIVE-20100 > URL: https://issues.apache.org/jira/browse/HIVE-20100 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20100.1.patch, HIVE-20100.2.patch, > HIVE-20100.3.patch, HIVE-20100.4.patch > > > The select operator's optraits logic as stated in the comment is, > // For bucket columns > // If all the columns match to the parent, put them in the bucket cols > // else, add empty list. > // For sort columns > // Keep the subset of all the columns as long as order is maintained. > > However, this is not happening due to a bug. The bool found is never reset, > so if a single match is found, the value remains true and allows the optraits > get populated with partial list of columns for bucket col which is incorrect. > This may lead to creation of SMB join which should not happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20100) OpTraits : Select Optraits should stop when a mismatch is detected
[ https://issues.apache.org/jira/browse/HIVE-20100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-20100: -- Attachment: HIVE-20100.5.patch > OpTraits : Select Optraits should stop when a mismatch is detected > -- > > Key: HIVE-20100 > URL: https://issues.apache.org/jira/browse/HIVE-20100 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-20100.1.patch, HIVE-20100.2.patch, > HIVE-20100.3.patch, HIVE-20100.4.patch, HIVE-20100.5.patch > > > The select operator's optraits logic as stated in the comment is, > // For bucket columns > // If all the columns match to the parent, put them in the bucket cols > // else, add empty list. > // For sort columns > // Keep the subset of all the columns as long as order is maintained. > > However, this is not happening due to a bug. The bool found is never reset, > so if a single match is found, the value remains true and allows the optraits > get populated with partial list of columns for bucket col which is incorrect. > This may lead to creation of SMB join which should not happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20006) Make materializations invalidation cache work with multiple active remote metastores
[ https://issues.apache.org/jira/browse/HIVE-20006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16536000#comment-16536000 ] Hive QA commented on HIVE-20006: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12930635/HIVE-20006.03.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 14629 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timestamptz] (batchId=191) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_joins] (batchId=191) org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_masking] (batchId=191) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12462/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12462/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12462/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12930635 - PreCommit-HIVE-Build > Make materializations invalidation cache work with multiple active remote > metastores > > > Key: HIVE-20006 > URL: https://issues.apache.org/jira/browse/HIVE-20006 > Project: Hive > Issue Type: Improvement > Components: Materialized views >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Critical > Attachments: HIVE-19027.01.patch, HIVE-19027.02.patch, > HIVE-19027.03.patch, HIVE-19027.04.patch, HIVE-20006.01.patch, > HIVE-20006.02.patch, HIVE-20006.03.patch, HIVE-20006.patch > > > The main points: > - Only MVs stored in transactional tables can have a time window value of 0. > Those are the only MVs that can be guaranteed to not be outdated when a query > is executed, if we use custom storage handlers to store the materialized > view, we cannot make any promises. > - For MVs that +cannot be outdated+, we do not check the metastore. Instead, > comparison is based on valid write id lists. > - For MVs that +can be outdated+, we still rely on the invalidation cache. > ** The window for valid outdated MVs can be specified in intervals of 1 > minute (less than that, it is difficult to have any guarantees about whether > the MV is actually outdated by less than a minute or not). > ** The async loading is done every interval / 2 (or probably better, we can > make it configurable). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20006) Make materializations invalidation cache work with multiple active remote metastores
[ https://issues.apache.org/jira/browse/HIVE-20006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16536001#comment-16536001 ] Hive QA commented on HIVE-20006: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 31s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 16s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 12s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m 4s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 32s{color} | {color:blue} common in master has 64 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 7s{color} | {color:blue} standalone-metastore in master has 217 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 55s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m 49s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 22s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 0s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 14s{color} | {color:red} common: The patch generated 1 new + 422 unchanged - 5 fixed = 423 total (was 427) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 38s{color} | {color:red} standalone-metastore: The patch generated 9 new + 1959 unchanged - 13 fixed = 1968 total (was 1972) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 42s{color} | {color:red} ql: The patch generated 4 new + 380 unchanged - 4 fixed = 384 total (was 384) {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 30s{color} | {color:red} root: The patch generated 14 new + 2761 unchanged - 22 fixed = 2775 total (was 2783) {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch has 6 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 19s{color} | {color:red} standalone-metastore generated 3 new + 215 unchanged - 2 fixed = 218 total (was 217) {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m 1s{color} | {color:red} ql generated 1 new + 2286 unchanged - 1 fixed = 2287 total (was 2287) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s{color} | {color:green} common in the patch passed. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s{color} | {color:green} standalone-metastore generated 0 new + 53 unchanged - 1 fixed = 53 total (was 54) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s{color} | {color:green} ql in the patch passed. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m 19s{color} | {color:green} root generated 0 new + 370 unchanged - 1 fixed = 370 total (was 371) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 22s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 72m 27s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:standalone-metastore | | | org.apache.hadoop.hive.metastore.txn.TxnHandler.lockMaterializationRebuild(String, String, long) passes a nonconstant String to an execute or addBatch method on an SQL
[jira] [Commented] (HIVE-20118) SessionStateUserAuthenticator.getGroupNames() is always empty
[ https://issues.apache.org/jira/browse/HIVE-20118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16536003#comment-16536003 ] Hive QA commented on HIVE-20118: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 44s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 37s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 54s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 22m 47s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12463/dev-support/hive-personality.sh | | git revision | master / 406cde9 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12463/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > SessionStateUserAuthenticator.getGroupNames() is always empty > - > > Key: HIVE-20118 > URL: https://issues.apache.org/jira/browse/HIVE-20118 > Project: Hive > Issue Type: Improvement > Components: Authorization >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-20118.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)