[jira] [Updated] (HIVE-20360) QTest: ignore driver/qtest exclusions if -Dqfile param is set
[
https://issues.apache.org/jira/browse/HIVE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-20360:
Attachment: HIVE-20360.07.patch
> QTest: ignore driver/qtest exclusions if -Dqfile param is set
> -
>
> Key: HIVE-20360
> URL: https://issues.apache.org/jira/browse/HIVE-20360
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Affects Versions: 3.1.0
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-20360.01.patch, HIVE-20360.02.patch,
> HIVE-20360.03.patch, HIVE-20360.04.patch, HIVE-20360.05.patch,
> HIVE-20360.06.patch, HIVE-20360.07.patch
>
>
> Sometimes I need to run qtests with another driver for testing purposes. In
> this case I have to edit testconfiguration.properties which seems a bit
> hacky, even if it's temporary.
> In this case, no tests will run (however there's a log message):
> {code:java}
> mvn test -Pitests -pl itests/qtest -pl itests/util -Dtest=TestCliDriver
> -Dqfile=bucketizedhiveinputformat.q
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20360) QTest: ignore driver/qtest exclusions if -Dqfile param is set
[
https://issues.apache.org/jira/browse/HIVE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689131#comment-16689131
]
Laszlo Bodor commented on HIVE-20360:
-
unrelated failures, uploading 07.patch
> QTest: ignore driver/qtest exclusions if -Dqfile param is set
> -
>
> Key: HIVE-20360
> URL: https://issues.apache.org/jira/browse/HIVE-20360
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Affects Versions: 3.1.0
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-20360.01.patch, HIVE-20360.02.patch,
> HIVE-20360.03.patch, HIVE-20360.04.patch, HIVE-20360.05.patch,
> HIVE-20360.06.patch, HIVE-20360.07.patch
>
>
> Sometimes I need to run qtests with another driver for testing purposes. In
> this case I have to edit testconfiguration.properties which seems a bit
> hacky, even if it's temporary.
> In this case, no tests will run (however there's a log message):
> {code:java}
> mvn test -Pitests -pl itests/qtest -pl itests/util -Dtest=TestCliDriver
> -Dqfile=bucketizedhiveinputformat.q
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20928) NPE in StatsUtils for complex type
[
https://issues.apache.org/jira/browse/HIVE-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689130#comment-16689130
]
Hive QA commented on HIVE-20928:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948458/HIVE-20928.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15543 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=197)
[druidmini_masking.q,druidmini_joins.q,druid_timestamptz.q]
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testParallelCompilation4 (batchId=262)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14958/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14958/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14958/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948458 - PreCommit-HIVE-Build
> NPE in StatsUtils for complex type
> --
>
> Key: HIVE-20928
> URL: https://issues.apache.org/jira/browse/HIVE-20928
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.3.4
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-20928.1.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getWritableSize(StatsUtils.java:1147)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfMap(StatsUtils.java:1108)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfComplexTypes(StatsUtils.java:978)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getAvgColLenOf(StatsUtils.java:916)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getColStatisticsFromExpression(StatsUtils.java:1374)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getColStatisticsFromExprMap(StatsUtils.java:1197)
> at
> org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$GroupByStatsRule.process(StatsRulesProcFactory.java:1009)
> at
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
> at
> org.apache.hadoop.hive.ql.lib.LevelOrderWalker.walk(LevelOrderWalker.java:143)
> at
> org.apache.hadoop.hive.ql.lib.LevelOrderWalker.startWalking(LevelOrderWalker.java:122)
> at
> org.apache.hadoop.hive.ql.optimizer.stats.annotation.AnnotateWithStatistics.transform(AnnotateWithStatistics.java:78)
> at
> org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.runStatsAnnotation(SparkCompiler.java:240)
> {noformat}
> Issue should be there in master as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20929) The method HiveMetaStoreClient.getValidWriteIds(String fullTableName) failed to execute.
[ https://issues.apache.org/jira/browse/HIVE-20929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weizheng reassigned HIVE-20929: --- > The method HiveMetaStoreClient.getValidWriteIds(String fullTableName) failed > to execute. > > > Key: HIVE-20929 > URL: https://issues.apache.org/jira/browse/HIVE-20929 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.1.0 > Environment: hadoop3.1.0 > hive3.1.0 >Reporter: weizheng >Assignee: weizheng >Priority: Major > > When I use the getValidWriteIds(String fullTableName) method of > HiveMetaStoreClient to get writeIds, I get the errors: > Exception in thread "main" org.apache.thrift.protocol.TProtocolException: > Required field 'validTxnList' is unset! > Struct:GetValidWriteIdsRequest(fullTableNames:[default.w2], validTxnList:null) > at > org.apache.hadoop.hive.metastore.api.GetValidWriteIdsRequest.validate(GetValidWriteIdsRequest.java:396) > at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_valid_write_ids_args.validate(ThriftHiveMetastore.java) > at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_valid_write_ids_args$get_valid_write_ids_argsStandardScheme.write(ThriftHiveMetastore.java) > at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_valid_write_ids_args$get_valid_write_ids_argsStandardScheme.write(ThriftHiveMetastore.java) > at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_valid_write_ids_args.write(ThriftHiveMetastore.java) > at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:71) > at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62) > at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_get_valid_write_ids(ThriftHiveMetastore.java:5443) > at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_valid_write_ids(ThriftHiveMetastore.java:5435) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getValidWriteIds(HiveMetaStoreClient.java:2581) > at com.inspur.mutation.test.TableOperate.getRowsMap(TableOperate.java:521) > at com.inspur.procuder.GenerateData.main(GenerateData.java:65) > > So I can only use the following methods instead： > ValidWriteIdList writeIds; > tablesList.add(AcidUtils.getFullTableName(table.getDbName(), > table.getTableName())); > List writeIdList = > metaStoreClient.getValidWriteIds(tablesList,txns.toString()); > writeIds = TxnUtils.createValidReaderWriteIdList(writeIdList.get(0)); -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20852) Compaction Initiator ignores datas inserted by Stream Data Ingest
[ https://issues.apache.org/jira/browse/HIVE-20852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689079#comment-16689079 ] Kei Miyauchi commented on HIVE-20852: - # Create stream_table(id int, body string) # Insert CSV data via NiFi 1.7 "GenerateFlowFile" and "PutHive3Streaming" into table stream_table. # At first, major compaction runs on stream_table. # After keeping flow running some more, there's enough files on stream_table to run minor compaction, but it isn't even checked by Initiator. # Create Another table normal_table(id int, body string) # Insert datas into normal_table manually # It is checked, and get compacted. > Compaction Initiator ignores datas inserted by Stream Data Ingest > - > > Key: HIVE-20852 > URL: https://issues.apache.org/jira/browse/HIVE-20852 > Project: Hive > Issue Type: Bug > Components: API, Transactions >Affects Versions: 3.1.0 >Reporter: Kei Miyauchi >Priority: Major > > HI, > Before compaction, Initiator decides whether a table/partition is > potentialCompaction by querying COMPLETE_TXN_COMPONENT. > But I found the transactions which committed by Stream Data Ingest is not > stored on COMPLETE_TXN_COMPONENT. This is because the statement "insert into > COMPLETED_TXN_COMPONENTS (ctc_txnid, ctc_database, ctc_table, ctc_partition, > ctc_writeid, ctc_update_delete) select tc_txnid, tc_database, tc_table, > tc_partition, tc_writeid, 'N' from TXN_COMPONENTS where tc_txnid = (id)" > fails. > I found INSERT statement to TXN_COMPONENTS isn't fired. select subquery above > returns 0 row. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Attachment: HIVE-20919.06.patch > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch, > HIVE-20919.03.patch, HIVE-20919.04.patch, HIVE-20919.05.patch, > HIVE-20919.06.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Status: Open (was: Patch Available) > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch, > HIVE-20919.03.patch, HIVE-20919.04.patch, HIVE-20919.05.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Status: Patch Available (was: Open) > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch, > HIVE-20919.03.patch, HIVE-20919.04.patch, HIVE-20919.05.patch, > HIVE-20919.06.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20928) NPE in StatsUtils for complex type
[
https://issues.apache.org/jira/browse/HIVE-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated HIVE-20928:
Assignee: (was: Rajesh Balamohan)
Status: Patch Available (was: Open)
> NPE in StatsUtils for complex type
> --
>
> Key: HIVE-20928
> URL: https://issues.apache.org/jira/browse/HIVE-20928
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.3.4
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-20928.1.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getWritableSize(StatsUtils.java:1147)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfMap(StatsUtils.java:1108)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfComplexTypes(StatsUtils.java:978)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getAvgColLenOf(StatsUtils.java:916)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getColStatisticsFromExpression(StatsUtils.java:1374)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getColStatisticsFromExprMap(StatsUtils.java:1197)
> at
> org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$GroupByStatsRule.process(StatsRulesProcFactory.java:1009)
> at
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
> at
> org.apache.hadoop.hive.ql.lib.LevelOrderWalker.walk(LevelOrderWalker.java:143)
> at
> org.apache.hadoop.hive.ql.lib.LevelOrderWalker.startWalking(LevelOrderWalker.java:122)
> at
> org.apache.hadoop.hive.ql.optimizer.stats.annotation.AnnotateWithStatistics.transform(AnnotateWithStatistics.java:78)
> at
> org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.runStatsAnnotation(SparkCompiler.java:240)
> {noformat}
> Issue should be there in master as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20928) NPE in StatsUtils for complex type
[
https://issues.apache.org/jira/browse/HIVE-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan reassigned HIVE-20928:
---
Assignee: Rajesh Balamohan
> NPE in StatsUtils for complex type
> --
>
> Key: HIVE-20928
> URL: https://issues.apache.org/jira/browse/HIVE-20928
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.3.4
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-20928.1.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getWritableSize(StatsUtils.java:1147)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfMap(StatsUtils.java:1108)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfComplexTypes(StatsUtils.java:978)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getAvgColLenOf(StatsUtils.java:916)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getColStatisticsFromExpression(StatsUtils.java:1374)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getColStatisticsFromExprMap(StatsUtils.java:1197)
> at
> org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$GroupByStatsRule.process(StatsRulesProcFactory.java:1009)
> at
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
> at
> org.apache.hadoop.hive.ql.lib.LevelOrderWalker.walk(LevelOrderWalker.java:143)
> at
> org.apache.hadoop.hive.ql.lib.LevelOrderWalker.startWalking(LevelOrderWalker.java:122)
> at
> org.apache.hadoop.hive.ql.optimizer.stats.annotation.AnnotateWithStatistics.transform(AnnotateWithStatistics.java:78)
> at
> org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.runStatsAnnotation(SparkCompiler.java:240)
> {noformat}
> Issue should be there in master as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20928) NPE in StatsUtils for complex type
[
https://issues.apache.org/jira/browse/HIVE-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated HIVE-20928:
Attachment: HIVE-20928.1.patch
> NPE in StatsUtils for complex type
> --
>
> Key: HIVE-20928
> URL: https://issues.apache.org/jira/browse/HIVE-20928
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.3.4
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-20928.1.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getWritableSize(StatsUtils.java:1147)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfMap(StatsUtils.java:1108)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfComplexTypes(StatsUtils.java:978)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getAvgColLenOf(StatsUtils.java:916)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getColStatisticsFromExpression(StatsUtils.java:1374)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getColStatisticsFromExprMap(StatsUtils.java:1197)
> at
> org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$GroupByStatsRule.process(StatsRulesProcFactory.java:1009)
> at
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
> at
> org.apache.hadoop.hive.ql.lib.LevelOrderWalker.walk(LevelOrderWalker.java:143)
> at
> org.apache.hadoop.hive.ql.lib.LevelOrderWalker.startWalking(LevelOrderWalker.java:122)
> at
> org.apache.hadoop.hive.ql.optimizer.stats.annotation.AnnotateWithStatistics.transform(AnnotateWithStatistics.java:78)
> at
> org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.runStatsAnnotation(SparkCompiler.java:240)
> {noformat}
> Issue should be there in master as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20927) The beeline has a dead cycle, resulting in high CPU.
[
https://issues.apache.org/jira/browse/HIVE-20927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yinghua_zh updated HIVE-20927:
--
Affects Version/s: 2.3.0
> The beeline has a dead cycle, resulting in high CPU.
>
>
> Key: HIVE-20927
> URL: https://issues.apache.org/jira/browse/HIVE-20927
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 2.3.0
>Reporter: yinghua_zh
>Priority: Major
>
> When beeline executes the command and places it for a period of time, it is
> found that the CPU of the beeline process is very high. It is the code that
> has a dead loop.The stack is as follows:
> "Thread-4" #34 daemon prio=5 os_prio=0 tid=0x7fbd6636d800 nid=0x162be
> sleeping[0x7fbd2e042000]
> java.lang.Thread.State: RUNNABLE
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.fillInStackTrace(Throwable.java:783)
> - locked <0x00076edb39d8> (a java.sql.SQLException)
> at java.lang.Throwable.(Throwable.java:265)
> at java.lang.Exception.(Exception.java:66)
> at java.sql.SQLException.(SQLException.java:119)
> at org.apache.hive.jdbc.HiveStatement.checkConnection(HiveStatement.java:377)
> at org.apache.hive.jdbc.HiveStatement.getQueryLog(HiveStatement.java:865)
> at org.apache.hive.jdbc.HiveStatement.getQueryLog(HiveStatement.java:849)
> at
> org.apache.hive.beeline.Commands$LogRunnable.updateQueryLog(Commands.java:1213)
> at org.apache.hive.beeline.Commands$LogRunnable.run(Commands.java:1221)
> at java.lang.Thread.run(Thread.java:745)
>
> The dead cycle code is as is as follows:
> @Override public void run() {
> while (hiveStatement.hasMoreLogs()) {
> try {
> updateQueryLog();
> Thread.sleep(queryProgressInterval);
> } catch (SQLException e) {
> commands.error(new SQLWarning(e));
> } catch (InterruptedException e) {
> commands.debug("Getting log thread is interrupted, since query is done!");
> commands.showRemainingLogsIfAny(hiveStatement);
> }
> }
> }
> }
>
>
> when the mehtod (checkConnection("getQueryLog");)occurs exception,the
> condition "hiveStatement.hasMoreLogs()" is still true,exist the dead lop,but
> the thread sleep code is not be run,so the CPU is high!
>
>
>
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20927) The beeline has a dead cycle, resulting in high CPU.
[
https://issues.apache.org/jira/browse/HIVE-20927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yinghua_zh updated HIVE-20927:
--
Component/s: Clients
> The beeline has a dead cycle, resulting in high CPU.
>
>
> Key: HIVE-20927
> URL: https://issues.apache.org/jira/browse/HIVE-20927
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Reporter: yinghua_zh
>Priority: Major
>
> When beeline executes the command and places it for a period of time, it is
> found that the CPU of the beeline process is very high. It is the code that
> has a dead loop.The stack is as follows:
> "Thread-4" #34 daemon prio=5 os_prio=0 tid=0x7fbd6636d800 nid=0x162be
> sleeping[0x7fbd2e042000]
> java.lang.Thread.State: RUNNABLE
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.fillInStackTrace(Throwable.java:783)
> - locked <0x00076edb39d8> (a java.sql.SQLException)
> at java.lang.Throwable.(Throwable.java:265)
> at java.lang.Exception.(Exception.java:66)
> at java.sql.SQLException.(SQLException.java:119)
> at org.apache.hive.jdbc.HiveStatement.checkConnection(HiveStatement.java:377)
> at org.apache.hive.jdbc.HiveStatement.getQueryLog(HiveStatement.java:865)
> at org.apache.hive.jdbc.HiveStatement.getQueryLog(HiveStatement.java:849)
> at
> org.apache.hive.beeline.Commands$LogRunnable.updateQueryLog(Commands.java:1213)
> at org.apache.hive.beeline.Commands$LogRunnable.run(Commands.java:1221)
> at java.lang.Thread.run(Thread.java:745)
>
> The dead cycle code is as is as follows:
> @Override public void run() {
> while (hiveStatement.hasMoreLogs()) {
> try {
> updateQueryLog();
> Thread.sleep(queryProgressInterval);
> } catch (SQLException e) {
> commands.error(new SQLWarning(e));
> } catch (InterruptedException e) {
> commands.debug("Getting log thread is interrupted, since query is done!");
> commands.showRemainingLogsIfAny(hiveStatement);
> }
> }
> }
> }
>
>
> when the mehtod (checkConnection("getQueryLog");)occurs exception,the
> condition "hiveStatement.hasMoreLogs()" is still true,exist the dead lop,but
> the thread sleep code is not be run,so the CPU is high!
>
>
>
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20920) Use SQL constraints to improve join reordering algorithm (II)
[
https://issues.apache.org/jira/browse/HIVE-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688993#comment-16688993
]
Hive QA commented on HIVE-20920:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948439/HIVE-20920.02.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14957/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14957/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14957/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-11-16 03:50:39.652
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-14957/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-11-16 03:50:39.656
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
1225ca9..156ed46 master -> origin/master
+ git reset --hard HEAD
HEAD is now at 1225ca9 HIVE-20842: Fix logic introduced in HIVE-20660 to
estimate statistics for group by (Vineet Garg, reviewed by Ashutosh Chauhan)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 156ed46 HIVE-20918: Flag to enable/disable pushdown of
computation from Calcite into JDBC connection (Jesus Camacho Rodriguez,
reviewed by Thejas M Nair)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-11-16 03:50:40.642
+ rm -rf ../yetus_PreCommit-HIVE-Build-14957
+ mkdir ../yetus_PreCommit-HIVE-Build-14957
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-14957
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-14957/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error:
a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java:
does not exist in index
error:
a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query11.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query14.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query17.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query18.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query24.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query25.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query29.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query35.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query4.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query46.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query47.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query54.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query57.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query58.q.out:
does not exist in index
error:
[jira] [Commented] (HIVE-20920) Use SQL constraints to improve join reordering algorithm (II)
[
https://issues.apache.org/jira/browse/HIVE-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688992#comment-16688992
]
Hive QA commented on HIVE-20920:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948430/HIVE-20920.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 15543 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=197)
[druidmini_masking.q,druidmini_joins.q,druid_timestamptz.q]
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query15]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query19]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query24]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query47]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query54]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query57]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query64]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query6]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query72]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query15]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query19]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query24]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query45]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query47]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query54]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query57]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query64]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query6]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query70]
(batchId=273)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query72]
(batchId=273)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14956/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14956/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14956/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 21 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948430 - PreCommit-HIVE-Build
> Use SQL constraints to improve join reordering algorithm (II)
> -
>
> Key: HIVE-20920
> URL: https://issues.apache.org/jira/browse/HIVE-20920
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20920.01.patch, HIVE-20920.02.patch,
> HIVE-20920.patch
>
>
> Follow-up of HIVE-20744.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20920) Use SQL constraints to improve join reordering algorithm (II)
[
https://issues.apache.org/jira/browse/HIVE-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688952#comment-16688952
]
Hive QA commented on HIVE-20920:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
41s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 4 new + 20 unchanged - 0 fixed
= 24 total (was 20) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
56s{color} | {color:red} ql generated 1 new + 2316 unchanged - 0 fixed = 2317
total (was 2316) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Potentially dangerous use of non-short-circuit logic in
org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdRowCount$SimpleTreeOnJoinKey.visit(RelNode,
int, RelNode) At HiveRelMdRowCount.java:logic in
org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdRowCount$SimpleTreeOnJoinKey.visit(RelNode,
int, RelNode) At HiveRelMdRowCount.java:[line 583] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14956/dev-support/hive-personality.sh
|
| git revision | master / 1225ca9 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14956/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14956/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14956/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Use SQL constraints to improve join reordering algorithm (II)
> -
>
> Key: HIVE-20920
> URL: https://issues.apache.org/jira/browse/HIVE-20920
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20920.01.patch, HIVE-20920.02.patch,
> HIVE-20920.patch
>
>
> Follow-up of HIVE-20744.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20920) Use SQL constraints to improve join reordering algorithm (II)
[ https://issues.apache.org/jira/browse/HIVE-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20920: --- Attachment: HIVE-20920.02.patch > Use SQL constraints to improve join reordering algorithm (II) > - > > Key: HIVE-20920 > URL: https://issues.apache.org/jira/browse/HIVE-20920 > Project: Hive > Issue Type: Improvement > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20920.01.patch, HIVE-20920.02.patch, > HIVE-20920.patch > > > Follow-up of HIVE-20744. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20918) Flag to enable/disable pushdown of computation from Calcite into JDBC connection
[ https://issues.apache.org/jira/browse/HIVE-20918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20918: --- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master, thanks [~thejas] > Flag to enable/disable pushdown of computation from Calcite into JDBC > connection > > > Key: HIVE-20918 > URL: https://issues.apache.org/jira/browse/HIVE-20918 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20918.01.patch, HIVE-20918.01.patch, > HIVE-20918.01.patch, HIVE-20918.01.patch, HIVE-20918.patch > > > Currently, there is no way to disable it. We will add a flag for that. By > default, pushdown of computation will be enabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[
https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688924#comment-16688924
]
Hive QA commented on HIVE-20919:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948409/HIVE-20919.05.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15538 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=196)
[druidmini_dynamic_partition.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_test_insert.q,druidkafkamini_delimited.q]
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=258)
org.apache.hive.jdbc.TestSSL.testSSLConnectionWithProperty (batchId=258)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14955/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14955/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14955/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948409 - PreCommit-HIVE-Build
> Break up UpdateDeleteSemanticAnalyzer
> -
>
> Key: HIVE-20919
> URL: https://issues.apache.org/jira/browse/HIVE-20919
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.2.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch,
> HIVE-20919.03.patch, HIVE-20919.04.patch, HIVE-20919.05.patch
>
>
> UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge
> queries by rewriting them to a different form. This is a clear violation of
> [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and
> therefore needs to be refactored. An abstract ancestor needs to take the
> common part, and each of the specific tasks should be handled by a separate
> class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20920) Use SQL constraints to improve join reordering algorithm (II)
[ https://issues.apache.org/jira/browse/HIVE-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20920: --- Attachment: HIVE-20920.01.patch > Use SQL constraints to improve join reordering algorithm (II) > - > > Key: HIVE-20920 > URL: https://issues.apache.org/jira/browse/HIVE-20920 > Project: Hive > Issue Type: Improvement > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20920.01.patch, HIVE-20920.patch > > > Follow-up of HIVE-20744. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[
https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688899#comment-16688899
]
Hive QA commented on HIVE-20919:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 6 new + 77 unchanged - 152
fixed = 83 total (was 229) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
50s{color} | {color:green} ql generated 0 new + 2315 unchanged - 1 fixed = 2315
total (was 2316) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14955/dev-support/hive-personality.sh
|
| git revision | master / 07529f7 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14955/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14955/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Break up UpdateDeleteSemanticAnalyzer
> -
>
> Key: HIVE-20919
> URL: https://issues.apache.org/jira/browse/HIVE-20919
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.2.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch,
> HIVE-20919.03.patch, HIVE-20919.04.patch, HIVE-20919.05.patch
>
>
> UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge
> queries by rewriting them to a different form. This is a clear violation of
> [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and
> therefore needs to be refactored. An abstract ancestor needs to take the
> common part, and each of the specific tasks should be handled by a separate
> class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-14204) Optimize loading dynamic partitions
[
https://issues.apache.org/jira/browse/HIVE-14204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1669#comment-1669
]
Yuming Wang commented on HIVE-14204:
Could we backport this patch to branch-1?
> Optimize loading dynamic partitions
>
>
> Key: HIVE-14204
> URL: https://issues.apache.org/jira/browse/HIVE-14204
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Labels: TODOC2.2
> Fix For: 2.2.0
>
> Attachments: HIVE-14204.1.patch, HIVE-14204.3.patch,
> HIVE-14204.4.patch, HIVE-14204.6.patch, HIVE-14204.7.patch, HIVE-14204.8.patch
>
>
> Lots of time is spent in sequential fashion to load dynamic partitioned
> dataset in driver side. E.g simple dynamic partitioned load as follows takes
> 300+ seconds
> {noformat}
> INSERT INTO web_sales_test partition(ws_sold_date_sk) select * from
> tpcds_bin_partitioned_orc_200.web_sales;
> Time taken to load dynamic partitions: 309.22 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20842) Fix logic introduced in HIVE-20660 to estimate statistics for group by
[ https://issues.apache.org/jira/browse/HIVE-20842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20842: --- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master > Fix logic introduced in HIVE-20660 to estimate statistics for group by > -- > > Key: HIVE-20842 > URL: https://issues.apache.org/jira/browse/HIVE-20842 > Project: Hive > Issue Type: Bug > Components: Statistics >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20842.1.patch, HIVE-20842.10.patch, > HIVE-20842.11.patch, HIVE-20842.12.patch, HIVE-20842.2.patch, > HIVE-20842.3.patch, HIVE-20842.4.patch, HIVE-20842.5.patch, > HIVE-20842.6.patch, HIVE-20842.7.patch, HIVE-20842.8.patch, HIVE-20842.9.patch > > > HIVE-20660 introduced better estimation for group by operator. But the logic > did not account for Partial and Full group by separately. > For partial group by parallelism (i.e. number of tasks) should be taken into > account. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-12897) Improve dynamic partition loading
[ https://issues.apache.org/jira/browse/HIVE-12897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688891#comment-16688891 ] Yuming Wang commented on HIVE-12897: Could we backport this patch to branch-1? > Improve dynamic partition loading > - > > Key: HIVE-12897 > URL: https://issues.apache.org/jira/browse/HIVE-12897 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Affects Versions: 0.11.0, 0.12.0, 0.13.0, 0.14.0, 1.0.0, 1.2.0, 1.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 2.1.0 > > Attachments: HIVE-12897.2.patch, HIVE-12897.3.patch, > HIVE-12897.4.patch, HIVE-12897.5.patch, HIVE-12897.patch > > > There are many redundant calls to metastore which is not needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20926) Semi join reduction hint fails when bloom filter entries are high or when there are no stats
[ https://issues.apache.org/jira/browse/HIVE-20926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20926: --- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master > Semi join reduction hint fails when bloom filter entries are high or when > there are no stats > > > Key: HIVE-20926 > URL: https://issues.apache.org/jira/browse/HIVE-20926 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20926.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20926) Semi join reduction hint fails when bloom filter entries are high or when there are no stats
[ https://issues.apache.org/jira/browse/HIVE-20926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1662#comment-1662 ] Ashutosh Chauhan commented on HIVE-20926: - +1 > Semi join reduction hint fails when bloom filter entries are high or when > there are no stats > > > Key: HIVE-20926 > URL: https://issues.apache.org/jira/browse/HIVE-20926 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20926.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20926) Semi join reduction hint fails when bloom filter entries are high or when there are no stats
[
https://issues.apache.org/jira/browse/HIVE-20926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688878#comment-16688878
]
Hive QA commented on HIVE-20926:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948403/HIVE-20926.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15543 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14954/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14954/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14954/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948403 - PreCommit-HIVE-Build
> Semi join reduction hint fails when bloom filter entries are high or when
> there are no stats
>
>
> Key: HIVE-20926
> URL: https://issues.apache.org/jira/browse/HIVE-20926
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20926.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20699) Query based compactor for full CRUD Acid tables
[
https://issues.apache.org/jira/browse/HIVE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-20699:
Attachment: HIVE-20699.1.patch
> Query based compactor for full CRUD Acid tables
> ---
>
> Key: HIVE-20699
> URL: https://issues.apache.org/jira/browse/HIVE-20699
> Project: Hive
> Issue Type: New Feature
> Components: Transactions
>Affects Versions: 3.1.0
>Reporter: Eugene Koifman
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-20699.1.patch, HIVE-20699.1.patch
>
>
> Currently the Acid compactor is implemented as generated MR job
> ({{CompactorMR.java}}).
> It could also be expressed as a Hive query that reads from a given partition
> and writes data back to the same partition. This will merge the deltas and
> 'apply' the delete events. The simplest would be to just use Insert
> Overwrite but that will change all ROW__IDs which we don't want.
> Need to implement this in a way that preserves ROW__IDs and creates a new
> {{base_x}} directory to handle Major compaction.
> Minor compaction will be investigated separately.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20926) Semi join reduction hint fails when bloom filter entries are high or when there are no stats
[
https://issues.apache.org/jira/browse/HIVE-20926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688849#comment-16688849
]
Hive QA commented on HIVE-20926:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
44s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14954/dev-support/hive-personality.sh
|
| git revision | master / 07529f7 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: storage-api ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14954/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Semi join reduction hint fails when bloom filter entries are high or when
> there are no stats
>
>
> Key: HIVE-20926
> URL: https://issues.apache.org/jira/browse/HIVE-20926
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20926.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17411) LLAP IO may incorrectly release a refcount in some rare cases
[ https://issues.apache.org/jira/browse/HIVE-17411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-17411: Description: java.io.IOException: java.lang.AssertionError: Unexpected refCount -1: 0x69a390d5(-1) In a large stream whose buffers are not reused, and that is separated into many CB (e.g. due to a small ORC compression buffer size), it may happen that some, but not all, buffers that are read together as a unit are evicted from cache. If CacheBuffer follows BufferChunk in the buffer list when a stream like this is read, the latter will be converted to ProcCacheChunk; it is possible for early refcount release logic from the former to release the refcount (for a dictionary stream, the initial refCount is always released early), and then backtrack to the latter to see if we can unlock more buffers. It would then to decref an uninitialized MemoryBuffer in ProcCacheChunk because ProcCacheChunk looks like a CacheChunk. PCC initial refcounts are released separately after the data is uncompressed. I'm assuming this would almost never happen with non-stripe-level streams because one would need a large RG to span 2+ CBs, no overlap with next/previous RGs in 2+ buffers for the early release to kick in, and an unfortunate eviction order. However it's possible with large-ish dictionaries. was: In a large stream whose buffers are not reused, and that is separated into many CB (e.g. due to a small ORC compression buffer size), it may happen that some, but not all, buffers that are read together as a unit are evicted from cache. If CacheBuffer follows BufferChunk in the buffer list when a stream like this is read, the latter will be converted to ProcCacheChunk; it is possible for early refcount release logic from the former to release the refcount (for a dictionary stream, the initial refCount is always released early), and then backtrack to the latter to see if we can unlock more buffers. It would then to decref an uninitialized MemoryBuffer in ProcCacheChunk because ProcCacheChunk looks like a CacheChunk. PCC initial refcounts are released separately after the data is uncompressed. I'm assuming this would almost never happen with non-stripe-level streams because one would need a large RG to span 2+ CBs, no overlap with next/previous RGs in 2+ buffers for the early release to kick in, and an unfortunate eviction order. However it's possible with large-ish dictionaries. > LLAP IO may incorrectly release a refcount in some rare cases > - > > Key: HIVE-17411 > URL: https://issues.apache.org/jira/browse/HIVE-17411 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 2.3.0, 3.0.0, 2.4.0 > > Attachments: HIVE-17411.patch > > > java.io.IOException: java.lang.AssertionError: Unexpected refCount -1: > 0x69a390d5(-1) > In a large stream whose buffers are not reused, and that is separated into > many CB (e.g. due to a small ORC compression buffer size), it may happen that > some, but not all, buffers that are read together as a unit are evicted from > cache. > If CacheBuffer follows BufferChunk in the buffer list when a stream like this > is read, the latter will be converted to ProcCacheChunk; it is possible for > early refcount release logic from the former to release the refcount (for a > dictionary stream, the initial refCount is always released early), and then > backtrack to the latter to see if we can unlock more buffers. It would then > to decref an uninitialized MemoryBuffer in ProcCacheChunk because > ProcCacheChunk looks like a CacheChunk. PCC initial refcounts are released > separately after the data is uncompressed. > I'm assuming this would almost never happen with non-stripe-level streams > because one would need a large RG to span 2+ CBs, no overlap with > next/previous RGs in 2+ buffers for the early release to kick in, and an > unfortunate eviction order. However it's possible with large-ish dictionaries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20916) Fix typo in JSONCreateDatabaseMessage and add test for alter database
[
https://issues.apache.org/jira/browse/HIVE-20916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688828#comment-16688828
]
Hive QA commented on HIVE-20916:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948398/HIVE-20916.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15544 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14953/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14953/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14953/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948398 - PreCommit-HIVE-Build
> Fix typo in JSONCreateDatabaseMessage and add test for alter database
> -
>
> Key: HIVE-20916
> URL: https://issues.apache.org/jira/browse/HIVE-20916
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-20916.01.patch, HIVE-20916.02.patch,
> HIVE-20916.03.patch
>
>
> {code}
> public JSONCreateDatabaseMessage(String server, String servicePrincipal,
> Database db,
> Long timestamp) {
> this.server = server;
> this.servicePrincipal = servicePrincipal;
> this.db = db.getName();
> this.timestamp = timestamp;
> try {
> this.dbJson = MessageBuilder.createDatabaseObjJson(db);
> } catch (TException ex) {
> throw new IllegalArgumentException("Could not serialize Function
> object", ex);
> }
> checkValid();
> }
> {code}
> The exception message should say Database instead of Function. Also, the
> {{TestDbNotificationListener#createDatabase}} should be modified to make sure
> that the deserialized database object from the dbJson field matches with the
> original database object
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Status: Open (was: Patch Available) > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch, > HIVE-20919.03.patch, HIVE-20919.04.patch, HIVE-20919.05.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Status: Patch Available (was: Open) > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch, > HIVE-20919.03.patch, HIVE-20919.04.patch, HIVE-20919.05.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Attachment: HIVE-20919.05.patch > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch, > HIVE-20919.03.patch, HIVE-20919.04.patch, HIVE-20919.05.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20916) Fix typo in JSONCreateDatabaseMessage and add test for alter database
[
https://issues.apache.org/jira/browse/HIVE-20916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688773#comment-16688773
]
Hive QA commented on HIVE-20916:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
3s{color} | {color:blue} standalone-metastore/metastore-server in master has
185 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14953/dev-support/hive-personality.sh
|
| git revision | master / 07529f7 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: itests/hcatalog-unit standalone-metastore/metastore-server U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14953/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix typo in JSONCreateDatabaseMessage and add test for alter database
> -
>
> Key: HIVE-20916
> URL: https://issues.apache.org/jira/browse/HIVE-20916
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-20916.01.patch, HIVE-20916.02.patch,
> HIVE-20916.03.patch
>
>
> {code}
> public JSONCreateDatabaseMessage(String server, String servicePrincipal,
> Database db,
> Long timestamp) {
> this.server = server;
> this.servicePrincipal = servicePrincipal;
> this.db = db.getName();
> this.timestamp = timestamp;
> try {
> this.dbJson = MessageBuilder.createDatabaseObjJson(db);
> } catch (TException ex) {
> throw new IllegalArgumentException("Could not serialize Function
> object", ex);
> }
> checkValid();
> }
> {code}
> The exception message should say Database instead of Function. Also, the
> {{TestDbNotificationListener#createDatabase}} should be modified to make sure
> that the deserialized database object from the dbJson field matches with the
> original
[jira] [Commented] (HIVE-20842) Fix logic introduced in HIVE-20660 to estimate statistics for group by
[
https://issues.apache.org/jira/browse/HIVE-20842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688749#comment-16688749
]
Hive QA commented on HIVE-20842:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948378/HIVE-20842.12.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15543 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14952/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14952/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14952/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948378 - PreCommit-HIVE-Build
> Fix logic introduced in HIVE-20660 to estimate statistics for group by
> --
>
> Key: HIVE-20842
> URL: https://issues.apache.org/jira/browse/HIVE-20842
> Project: Hive
> Issue Type: Bug
> Components: Statistics
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20842.1.patch, HIVE-20842.10.patch,
> HIVE-20842.11.patch, HIVE-20842.12.patch, HIVE-20842.2.patch,
> HIVE-20842.3.patch, HIVE-20842.4.patch, HIVE-20842.5.patch,
> HIVE-20842.6.patch, HIVE-20842.7.patch, HIVE-20842.8.patch, HIVE-20842.9.patch
>
>
> HIVE-20660 introduced better estimation for group by operator. But the logic
> did not account for Partial and Full group by separately.
> For partial group by parallelism (i.e. number of tasks) should be taken into
> account.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20926) Semi join reduction hint fails when bloom filter entries are high or when there are no stats
[ https://issues.apache.org/jira/browse/HIVE-20926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20926: --- Attachment: HIVE-20926.1.patch > Semi join reduction hint fails when bloom filter entries are high or when > there are no stats > > > Key: HIVE-20926 > URL: https://issues.apache.org/jira/browse/HIVE-20926 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20926.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20926) Semi join reduction hint fails when bloom filter entries are high or when there are no stats
[ https://issues.apache.org/jira/browse/HIVE-20926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20926: --- Status: Patch Available (was: Open) > Semi join reduction hint fails when bloom filter entries are high or when > there are no stats > > > Key: HIVE-20926 > URL: https://issues.apache.org/jira/browse/HIVE-20926 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20926.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20926) Semi join reduction hint fails when bloom filter entries are high or when there are no stats
[ https://issues.apache.org/jira/browse/HIVE-20926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-20926: -- > Semi join reduction hint fails when bloom filter entries are high or when > there are no stats > > > Key: HIVE-20926 > URL: https://issues.apache.org/jira/browse/HIVE-20926 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20916) Fix typo in JSONCreateDatabaseMessage and add test for alter database
[
https://issues.apache.org/jira/browse/HIVE-20916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-20916:
---
Attachment: HIVE-20916.03.patch
> Fix typo in JSONCreateDatabaseMessage and add test for alter database
> -
>
> Key: HIVE-20916
> URL: https://issues.apache.org/jira/browse/HIVE-20916
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-20916.01.patch, HIVE-20916.02.patch,
> HIVE-20916.03.patch
>
>
> {code}
> public JSONCreateDatabaseMessage(String server, String servicePrincipal,
> Database db,
> Long timestamp) {
> this.server = server;
> this.servicePrincipal = servicePrincipal;
> this.db = db.getName();
> this.timestamp = timestamp;
> try {
> this.dbJson = MessageBuilder.createDatabaseObjJson(db);
> } catch (TException ex) {
> throw new IllegalArgumentException("Could not serialize Function
> object", ex);
> }
> checkValid();
> }
> {code}
> The exception message should say Database instead of Function. Also, the
> {{TestDbNotificationListener#createDatabase}} should be modified to make sure
> that the deserialized database object from the dbJson field matches with the
> original database object
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20842) Fix logic introduced in HIVE-20660 to estimate statistics for group by
[
https://issues.apache.org/jira/browse/HIVE-20842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688709#comment-16688709
]
Hive QA commented on HIVE-20842:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
44s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} ql: The patch generated 0 new + 37 unchanged - 1
fixed = 37 total (was 38) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14952/dev-support/hive-personality.sh
|
| git revision | master / 07529f7 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14952/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix logic introduced in HIVE-20660 to estimate statistics for group by
> --
>
> Key: HIVE-20842
> URL: https://issues.apache.org/jira/browse/HIVE-20842
> Project: Hive
> Issue Type: Bug
> Components: Statistics
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20842.1.patch, HIVE-20842.10.patch,
> HIVE-20842.11.patch, HIVE-20842.12.patch, HIVE-20842.2.patch,
> HIVE-20842.3.patch, HIVE-20842.4.patch, HIVE-20842.5.patch,
> HIVE-20842.6.patch, HIVE-20842.7.patch, HIVE-20842.8.patch, HIVE-20842.9.patch
>
>
> HIVE-20660 introduced better estimation for group by operator. But the logic
> did not account for Partial and Full group by separately.
> For partial group by parallelism (i.e. number of tasks) should be taken into
> account.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20918) Flag to enable/disable pushdown of computation from Calcite into JDBC connection
[
https://issues.apache.org/jira/browse/HIVE-20918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688684#comment-16688684
]
Hive QA commented on HIVE-20918:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948358/HIVE-20918.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15543 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14951/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14951/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14951/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948358 - PreCommit-HIVE-Build
> Flag to enable/disable pushdown of computation from Calcite into JDBC
> connection
>
>
> Key: HIVE-20918
> URL: https://issues.apache.org/jira/browse/HIVE-20918
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20918.01.patch, HIVE-20918.01.patch,
> HIVE-20918.01.patch, HIVE-20918.01.patch, HIVE-20918.patch
>
>
> Currently, there is no way to disable it. We will add a flag for that. By
> default, pushdown of computation will be enabled.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20918) Flag to enable/disable pushdown of computation from Calcite into JDBC connection
[
https://issues.apache.org/jira/browse/HIVE-20918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688631#comment-16688631
]
Hive QA commented on HIVE-20918:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
46s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14951/dev-support/hive-personality.sh
|
| git revision | master / 07529f7 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14951/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Flag to enable/disable pushdown of computation from Calcite into JDBC
> connection
>
>
> Key: HIVE-20918
> URL: https://issues.apache.org/jira/browse/HIVE-20918
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20918.01.patch, HIVE-20918.01.patch,
> HIVE-20918.01.patch, HIVE-20918.01.patch, HIVE-20918.patch
>
>
> Currently, there is no way to disable it. We will add a flag for that. By
> default, pushdown of computation will be enabled.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20842) Fix logic introduced in HIVE-20660 to estimate statistics for group by
[ https://issues.apache.org/jira/browse/HIVE-20842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20842: --- Status: Patch Available (was: Open) > Fix logic introduced in HIVE-20660 to estimate statistics for group by > -- > > Key: HIVE-20842 > URL: https://issues.apache.org/jira/browse/HIVE-20842 > Project: Hive > Issue Type: Bug > Components: Statistics >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20842.1.patch, HIVE-20842.10.patch, > HIVE-20842.11.patch, HIVE-20842.12.patch, HIVE-20842.2.patch, > HIVE-20842.3.patch, HIVE-20842.4.patch, HIVE-20842.5.patch, > HIVE-20842.6.patch, HIVE-20842.7.patch, HIVE-20842.8.patch, HIVE-20842.9.patch > > > HIVE-20660 introduced better estimation for group by operator. But the logic > did not account for Partial and Full group by separately. > For partial group by parallelism (i.e. number of tasks) should be taken into > account. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20842) Fix logic introduced in HIVE-20660 to estimate statistics for group by
[ https://issues.apache.org/jira/browse/HIVE-20842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20842: --- Attachment: HIVE-20842.12.patch > Fix logic introduced in HIVE-20660 to estimate statistics for group by > -- > > Key: HIVE-20842 > URL: https://issues.apache.org/jira/browse/HIVE-20842 > Project: Hive > Issue Type: Bug > Components: Statistics >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20842.1.patch, HIVE-20842.10.patch, > HIVE-20842.11.patch, HIVE-20842.12.patch, HIVE-20842.2.patch, > HIVE-20842.3.patch, HIVE-20842.4.patch, HIVE-20842.5.patch, > HIVE-20842.6.patch, HIVE-20842.7.patch, HIVE-20842.8.patch, HIVE-20842.9.patch > > > HIVE-20660 introduced better estimation for group by operator. But the logic > did not account for Partial and Full group by separately. > For partial group by parallelism (i.e. number of tasks) should be taken into > account. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20842) Fix logic introduced in HIVE-20660 to estimate statistics for group by
[ https://issues.apache.org/jira/browse/HIVE-20842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20842: --- Status: Open (was: Patch Available) > Fix logic introduced in HIVE-20660 to estimate statistics for group by > -- > > Key: HIVE-20842 > URL: https://issues.apache.org/jira/browse/HIVE-20842 > Project: Hive > Issue Type: Bug > Components: Statistics >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20842.1.patch, HIVE-20842.10.patch, > HIVE-20842.11.patch, HIVE-20842.12.patch, HIVE-20842.2.patch, > HIVE-20842.3.patch, HIVE-20842.4.patch, HIVE-20842.5.patch, > HIVE-20842.6.patch, HIVE-20842.7.patch, HIVE-20842.8.patch, HIVE-20842.9.patch > > > HIVE-20660 introduced better estimation for group by operator. But the logic > did not account for Partial and Full group by separately. > For partial group by parallelism (i.e. number of tasks) should be taken into > account. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20786) Maven Build Failed with group id is too big
[
https://issues.apache.org/jira/browse/HIVE-20786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szehon Ho updated HIVE-20786:
-
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
Committed to master, thanks Vihang for the review
> Maven Build Failed with group id is too big
>
>
> Key: HIVE-20786
> URL: https://issues.apache.org/jira/browse/HIVE-20786
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
> Environment:
> OS: MacOS 10.13.6
> Java:
> {code}
> java version "1.8.0_192"
> Java(TM) SE Runtime Environment (build 1.8.0_192-b12)
> Java HotSpot(TM) 64-Bit Server VM (build 25.192-b12, mixed mode)
> {code}
> Maven:
> {code}
> Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe;
> 2018-06-18T02:33:14+08:00)
> Maven home: /usr/local/Cellar/maven/3.5.4/libexec
> Java version: 1.8.0_192, vendor: Oracle Corporation, runtime:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre
> Default locale: en_CN, platform encoding: UTF-8
> OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac"
> {code}
>
>
>Reporter: PENG Zhengshuai
>Assignee: Szehon Ho
>Priority: Major
> Labels: maven
> Fix For: 4.0.0
>
> Attachments: HIVE-20786.2.patch, HIVE-20786.patch,
> hive_build_error.log
>
>
> When executing
> {code}
> mvn clean install -DskipTests
> {code}
> Build Failed:
> {code}
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Hive Storage API 2.7.0-SNAPSHOT SUCCESS [ 5.299
> s]
> [INFO] Hive 4.0.0-SNAPSHOT SUCCESS [ 0.750
> s]
> [INFO] Hive Classifications ... SUCCESS [ 1.057
> s]
> [INFO] Hive Shims Common .. SUCCESS [ 3.882
> s]
> [INFO] Hive Shims 0.23 SUCCESS [ 5.020
> s]
> [INFO] Hive Shims Scheduler ... SUCCESS [ 2.587
> s]
> [INFO] Hive Shims . SUCCESS [ 2.038
> s]
> [INFO] Hive Common SUCCESS [ 6.921
> s]
> [INFO] Hive Service RPC ... SUCCESS [ 3.503
> s]
> [INFO] Hive Serde . SUCCESS [ 6.322
> s]
> [INFO] Hive Standalone Metastore .. FAILURE [ 0.557
> s]
> [INFO] Hive Standalone Metastore Common Code .. SKIPPED
> [INFO] Hive Metastore . SKIPPED
> [INFO] Hive Vector-Code-Gen Utilities . SKIPPED
> [INFO] Hive Llap Common ... SKIPPED
> [INFO] Hive Llap Client ... SKIPPED
> [INFO] Hive Llap Tez .. SKIPPED
> [INFO] Hive Spark Remote Client ... SKIPPED
> [INFO] Hive Metastore Server .. SKIPPED
> [INFO] Hive Query Language SKIPPED
> [INFO] Hive Llap Server ... SKIPPED
> [INFO] Hive Service ... SKIPPED
> [INFO] Hive Accumulo Handler .. SKIPPED
> [INFO] Hive JDBC .. SKIPPED
> [INFO] Hive Beeline ... SKIPPED
> [INFO] Hive CLI ... SKIPPED
> [INFO] Hive Contrib ... SKIPPED
> [INFO] Hive Druid Handler . SKIPPED
> [INFO] Hive HBase Handler . SKIPPED
> [INFO] Hive JDBC Handler .. SKIPPED
> [INFO] Hive HCatalog .. SKIPPED
> [INFO] Hive HCatalog Core . SKIPPED
> [INFO] Hive HCatalog Pig Adapter .. SKIPPED
> [INFO] Hive HCatalog Server Extensions SKIPPED
> [INFO] Hive HCatalog Webhcat Java Client .. SKIPPED
> [INFO] Hive HCatalog Webhcat .. SKIPPED
> [INFO] Hive HCatalog Streaming SKIPPED
> [INFO] Hive HPL/SQL ... SKIPPED
> [INFO] Hive Streaming . SKIPPED
> [INFO] Hive Llap External Client .. SKIPPED
> [INFO] Hive Shims Aggregator .. SKIPPED
> [INFO] Hive Kryo Registrator .. SKIPPED
> [INFO] Hive TestUtils . SKIPPED
> [INFO] Hive Kafka Storage Handler
[jira] [Commented] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[
https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688590#comment-16688590
]
Hive QA commented on HIVE-20919:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948345/HIVE-20919.04.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15538 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=195)
[druidmini_test_ts.q,druidmini_expressions.q,druid_timestamptz2.q,druidmini_test_alter.q,druidkafkamini_csv.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=86)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_limit]
(batchId=182)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14950/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14950/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14950/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948345 - PreCommit-HIVE-Build
> Break up UpdateDeleteSemanticAnalyzer
> -
>
> Key: HIVE-20919
> URL: https://issues.apache.org/jira/browse/HIVE-20919
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.2.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch,
> HIVE-20919.03.patch, HIVE-20919.04.patch
>
>
> UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge
> queries by rewriting them to a different form. This is a clear violation of
> [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and
> therefore needs to be refactored. An abstract ancestor needs to take the
> common part, and each of the specific tasks should be handled by a separate
> class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20923) masking_12 qtest expects wrong inputs
[
https://issues.apache.org/jira/browse/HIVE-20923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688553#comment-16688553
]
Jesus Camacho Rodriguez commented on HIVE-20923:
[~klcopp], this does not seem to be a bug, as behavior conforms with the
masking policies defined for the test:
https://github.com/apache/hive/blob/6340a81f487c407f63cf2a00e39b2f905328d74e/itests/util/src/main/java/org/apache/hadoop/hive/ql/security/authorization/plugin/sqlstd/SQLStdHiveAuthorizationValidatorForTest.java#L146
Explain plan should shed more light into what is going on underneath.
> masking_12 qtest expects wrong inputs
> -
>
> Key: HIVE-20923
> URL: https://issues.apache.org/jira/browse/HIVE-20923
> Project: Hive
> Issue Type: Bug
> Components: Query Planning, Test, Tests, Views
>Reporter: Karen Coppage
>Priority: Major
>
> It appears this qtest expects the wrong behavior, and should fail but doesn't.
> Below Queries 3 and 4 should not include default@src as Input
> Query 1
> {noformat}
> PREHOOK: query: create table `masking_test_subq_n1` as select cast(key as
> int) as key, value from src
> PREHOOK: type: CREATETABLE_AS_SELECT
> PREHOOK: Input: default@src
> PREHOOK: Output: database:default
> PREHOOK: Output: default@masking_test_subq_n1
> POSTHOOK: query: create table `masking_test_subq_n1` as select cast(key as
> int) as key, value from src
> POSTHOOK: type: CREATETABLE_AS_SELECT
> POSTHOOK: Input: default@src
> POSTHOOK: Output: database:default
> POSTHOOK: Output: default@masking_test_subq_n1
> POSTHOOK: Lineage: masking_test_subq_n1.key EXPRESSION
> [(src)src.FieldSchema(name:key, type:string, comment:default), ]
> POSTHOOK: Lineage: masking_test_subq_n1.value SIMPLE
> [(src)src.FieldSchema(name:value, type:string, comment:default), ]{noformat}
> Query 2
> {noformat}
> PREHOOK: query: create view `v1_n9` as select * from `masking_test_subq_n1`
> PREHOOK: type: CREATEVIEW
> PREHOOK: Input: default@masking_test_subq_n1
> PREHOOK: Output: database:default
> PREHOOK: Output: default@v1_n9
> POSTHOOK: query: create view `v1_n9` as select * from `masking_test_subq_n1`
> POSTHOOK: type: CREATEVIEW
> POSTHOOK: Input: default@masking_test_subq_n1
> POSTHOOK: Output: database:default
> POSTHOOK: Output: default@v1_n9
> POSTHOOK: Lineage: v1_n9.key SIMPLE
> [(masking_test_subq_n1)masking_test_subq_n1.FieldSchema(name:key, type:int,
> comment:null), ]
> POSTHOOK: Lineage: v1_n9.value SIMPLE
> [(masking_test_subq_n1)masking_test_subq_n1.FieldSchema(name:value,
> type:string, comment:null), ]
> {noformat}
> Query 3 (the explain output here describes a TableScan on table src)
> {noformat}
> PREHOOK: query: explain
> select * from `v1_n9`
> limit 20
> PREHOOK: type: QUERY
> PREHOOK: Input: default@masking_test_subq_n1
> PREHOOK: Input: default@src
> PREHOOK: Input: default@v1_n9
> A masked pattern was here
> POSTHOOK: query: explain
> select * from `v1_n9`
> limit 20
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@masking_test_subq_n1
> POSTHOOK: Input: default@src
> POSTHOOK: Input: default@v1_n9
> {noformat}
> Query 4
> {noformat}
> PREHOOK: query: select * from `v1_n9`
> limit 20
> PREHOOK: type: QUERY
> PREHOOK: Input: default@masking_test_subq_n1
> PREHOOK: Input: default@src
> PREHOOK: Input: default@v1_n9
> A masked pattern was here
> POSTHOOK: query: select * from `v1_n9`
> limit 20
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@masking_test_subq_n1
> POSTHOOK: Input: default@src
> POSTHOOK: Input: default@v1_n9
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[
https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688551#comment-16688551
]
Hive QA commented on HIVE-20919:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 6 new + 77 unchanged - 152
fixed = 83 total (was 229) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
53s{color} | {color:green} ql generated 0 new + 2315 unchanged - 1 fixed = 2315
total (was 2316) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14950/dev-support/hive-personality.sh
|
| git revision | master / 6fbdf37 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14950/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14950/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Break up UpdateDeleteSemanticAnalyzer
> -
>
> Key: HIVE-20919
> URL: https://issues.apache.org/jira/browse/HIVE-20919
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.2.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch,
> HIVE-20919.03.patch, HIVE-20919.04.patch
>
>
> UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge
> queries by rewriting them to a different form. This is a clear violation of
> [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and
> therefore needs to be refactored. An abstract ancestor needs to take the
> common part, and each of the specific tasks should be handled by a separate
> class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20360) QTest: ignore driver/qtest exclusions if -Dqfile param is set
[
https://issues.apache.org/jira/browse/HIVE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688515#comment-16688515
]
Hive QA commented on HIVE-20360:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948336/HIVE-20360.06.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15538 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=196)
[druidmini_dynamic_partition.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_test_insert.q,druidkafkamini_delimited.q]
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=197)
[druidmini_masking.q,druidmini_joins.q,druid_timestamptz.q]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_limit]
(batchId=182)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14949/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14949/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14949/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948336 - PreCommit-HIVE-Build
> QTest: ignore driver/qtest exclusions if -Dqfile param is set
> -
>
> Key: HIVE-20360
> URL: https://issues.apache.org/jira/browse/HIVE-20360
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Affects Versions: 3.1.0
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-20360.01.patch, HIVE-20360.02.patch,
> HIVE-20360.03.patch, HIVE-20360.04.patch, HIVE-20360.05.patch,
> HIVE-20360.06.patch
>
>
> Sometimes I need to run qtests with another driver for testing purposes. In
> this case I have to edit testconfiguration.properties which seems a bit
> hacky, even if it's temporary.
> In this case, no tests will run (however there's a log message):
> {code:java}
> mvn test -Pitests -pl itests/qtest -pl itests/util -Dtest=TestCliDriver
> -Dqfile=bucketizedhiveinputformat.q
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20360) QTest: ignore driver/qtest exclusions if -Dqfile param is set
[
https://issues.apache.org/jira/browse/HIVE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688451#comment-16688451
]
Hive QA commented on HIVE-20360:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} itests/util in master has 48 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} itests/util: The patch generated 2 new + 68 unchanged
- 5 fixed = 70 total (was 73) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14949/dev-support/hive-personality.sh
|
| git revision | master / 6fbdf37 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14949/yetus/diff-checkstyle-itests_util.txt
|
| modules | C: itests/util U: itests/util |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14949/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> QTest: ignore driver/qtest exclusions if -Dqfile param is set
> -
>
> Key: HIVE-20360
> URL: https://issues.apache.org/jira/browse/HIVE-20360
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Affects Versions: 3.1.0
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-20360.01.patch, HIVE-20360.02.patch,
> HIVE-20360.03.patch, HIVE-20360.04.patch, HIVE-20360.05.patch,
> HIVE-20360.06.patch
>
>
> Sometimes I need to run qtests with another driver for testing purposes. In
> this case I have to edit testconfiguration.properties which seems a bit
> hacky, even if it's temporary.
> In this case, no tests will run (however there's a log message):
> {code:java}
> mvn test -Pitests -pl itests/qtest -pl itests/util -Dtest=TestCliDriver
> -Dqfile=bucketizedhiveinputformat.q
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17300) WebUI query plan graphs
[ https://issues.apache.org/jira/browse/HIVE-17300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688439#comment-16688439 ] Wang Haihua commented on HIVE-17300: Great contribution! Should we rebase it to 1.x? > WebUI query plan graphs > --- > > Key: HIVE-17300 > URL: https://issues.apache.org/jira/browse/HIVE-17300 > Project: Hive > Issue Type: Sub-task > Components: Web UI >Affects Versions: 4.0.0 >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Labels: beginner, features, patch > Fix For: 4.0.0 > > Attachments: HIVE-17300.10.patch, HIVE-17300.10.patch, > HIVE-17300.10.patch, HIVE-17300.3.patch, HIVE-17300.4.patch, > HIVE-17300.5.patch, HIVE-17300.6.patch, HIVE-17300.7.patch, > HIVE-17300.7.patch, HIVE-17300.8.patch, HIVE-17300.8.patch, > HIVE-17300.8.patch, HIVE-17300.8.patch, HIVE-17300.9.patch, HIVE-17300.patch, > complete_success.png, full_mapred_stats.png, graph_with_mapred_stats.png, > last_stage_error.png, last_stage_running.png, non_mapred_task_selected.png > > > Hi all, > I’m working on a feature of the Hive WebUI Query Plan tab that would provide > the option to display the query plan as a nice graph (scroll down for > screenshots). If you click on one of the graph’s stages, the plan for that > stage appears as text below. > Stages are color-coded if they have a status (Success, Error, Running), and > the rest are grayed out. Coloring is based on status already available in the > WebUI, under the Stages tab. > There is an additional option to display stats for MapReduce tasks. This > includes the job’s ID, tracking URL (where the logs are found), and mapper > and reducer numbers/progress, among other info. > The library I’m using for the graph is called vis.js (http://visjs.org/). It > has an Apache license, and the only necessary file to be included from this > library is about 700 KB. > I tried to keep server-side changes minimal, and graph generation is taken > care of by the client. Plans with more than a given number of stages > (default: 25) won't be displayed in order to preserve resources. > I’d love to hear any and all input from the community about this feature: do > you think it’s useful, and is there anything important I’m missing? > Thanks, > Karen Coppage > Review request: https://reviews.apache.org/r/61663/ > Any input is welcome! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20918) Flag to enable/disable pushdown of computation from Calcite into JDBC connection
[ https://issues.apache.org/jira/browse/HIVE-20918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20918: --- Attachment: HIVE-20918.01.patch > Flag to enable/disable pushdown of computation from Calcite into JDBC > connection > > > Key: HIVE-20918 > URL: https://issues.apache.org/jira/browse/HIVE-20918 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20918.01.patch, HIVE-20918.01.patch, > HIVE-20918.01.patch, HIVE-20918.01.patch, HIVE-20918.patch > > > Currently, there is no way to disable it. We will add a flag for that. By > default, pushdown of computation will be enabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20914) MRScratchDir permission denied when "hive.server2.enable.doAs", "hive.exec.submitviachild" are set to "true" and impersonated/proxy user is used
[
https://issues.apache.org/jira/browse/HIVE-20914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688429#comment-16688429
]
Hive QA commented on HIVE-20914:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948324/HIVE-20914.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 15543 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=197)
[druidmini_masking.q,druidmini_joins.q,druid_timestamptz.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parallel_orderby]
(batchId=58)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_rp_limit]
(batchId=171)
org.apache.hadoop.hive.metastore.TestObjectStore.catalogs (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDatabaseOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testTableOps (batchId=232)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14948/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14948/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14948/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948324 - PreCommit-HIVE-Build
> MRScratchDir permission denied when "hive.server2.enable.doAs",
> "hive.exec.submitviachild" are set to "true" and impersonated/proxy user is
> used
>
>
> Key: HIVE-20914
> URL: https://issues.apache.org/jira/browse/HIVE-20914
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20914.1.patch, HIVE-20914.2.patch,
> HIVE-20914.3.patch
>
>
> The above issue could be reproduced in none Kerberos cluster using the below
> steps:
> 1. Set "hive.exec.submitviachild" value to "true".
> 2. Run a count query not using "hive" user.
> {code}beeline -u 'jdbc:hive2://localhost:1' -n hdfs{code}
> There is no issue when we try to execute the same query using the "hive" user.
> {code:java}
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.hadoop.security.AccessControlException: Permission denied:
> user=hive, access=EXECUTE, inode="/tmp/hive/hdfs":hdfs:supergroup:drwx--
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:279)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:260)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkTraverse(DefaultAuthorizationProvider.java:201)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkPermission(DefaultAuthorizationProvider.java:154)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:152)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:3877)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:3860)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkTraverse(FSDirectory.java:3847)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkTraverse(FSNamesystem.java:6822)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:4551)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:4529)
> at
>
[jira] [Commented] (HIVE-20914) MRScratchDir permission denied when "hive.server2.enable.doAs", "hive.exec.submitviachild" are set to "true" and impersonated/proxy user is used
[
https://issues.apache.org/jira/browse/HIVE-20914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688375#comment-16688375
]
Hive QA commented on HIVE-20914:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
36s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14948/dev-support/hive-personality.sh
|
| git revision | master / 6fbdf37 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14948/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> MRScratchDir permission denied when "hive.server2.enable.doAs",
> "hive.exec.submitviachild" are set to "true" and impersonated/proxy user is
> used
>
>
> Key: HIVE-20914
> URL: https://issues.apache.org/jira/browse/HIVE-20914
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20914.1.patch, HIVE-20914.2.patch,
> HIVE-20914.3.patch
>
>
> The above issue could be reproduced in none Kerberos cluster using the below
> steps:
> 1. Set "hive.exec.submitviachild" value to "true".
> 2. Run a count query not using "hive" user.
> {code}beeline -u 'jdbc:hive2://localhost:1' -n hdfs{code}
> There is no issue when we try to execute the same query using the "hive" user.
> {code:java}
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.hadoop.security.AccessControlException: Permission denied:
> user=hive, access=EXECUTE, inode="/tmp/hive/hdfs":hdfs:supergroup:drwx--
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:279)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:260)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkTraverse(DefaultAuthorizationProvider.java:201)
> at
>
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Status: Open (was: Patch Available) > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch, > HIVE-20919.03.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Attachment: HIVE-20919.04.patch > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch, > HIVE-20919.03.patch, HIVE-20919.04.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Status: Patch Available (was: Open) > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch, > HIVE-20919.03.patch, HIVE-20919.04.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[
https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688308#comment-16688308
]
Hive QA commented on HIVE-20919:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948321/HIVE-20919.03.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14947/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14947/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14947/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-11-15 16:29:51.233
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-14947/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-11-15 16:29:51.236
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 6fbdf37 HIVE-20730: Do delete event filtering even if
hive.acid.index is not there (Saurabh Seth via Eugene Koifman)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 6fbdf37 HIVE-20730: Do delete event filtering even if
hive.acid.index is not there (Saurabh Seth via Eugene Koifman)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-11-15 16:29:51.957
+ rm -rf ../yetus_PreCommit-HIVE-Build-14947
+ mkdir ../yetus_PreCommit-HIVE-Build-14947
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-14947
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-14947/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error:
a/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java:
does not exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/parse/UpdateDeleteSemanticAnalyzer.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/plan/AlterTableDesc.java: does
not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/plan/ExportWork.java: does not
exist in index
Going to apply patch with: git apply -p1
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc4508975307820368727.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc4508975307820368727.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
(process-resource-bundles) on project hive-pre-upgrade: Execution
process-resource-bundles of goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process failed.
ConcurrentModificationException -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
[jira] [Commented] (HIVE-20818) Views created with a WHERE subquery will regard views referenced in the subquery as direct input
[
https://issues.apache.org/jira/browse/HIVE-20818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688305#comment-16688305
]
Hive QA commented on HIVE-20818:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948309/HIVE-20818.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15545 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_12] (batchId=1)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_limit]
(batchId=182)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14946/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14946/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14946/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948309 - PreCommit-HIVE-Build
> Views created with a WHERE subquery will regard views referenced in the
> subquery as direct input
>
>
> Key: HIVE-20818
> URL: https://issues.apache.org/jira/browse/HIVE-20818
> Project: Hive
> Issue Type: Bug
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-20818.2.patch, HIVE-20818.3.patch,
> HIVE-20818.4.patch, HIVE-20818.5.patch, HIVE-20818.patch
>
>
> If Hive is configured with an authorization hook like Sentry, and a view is
> created with a WHERE clause referencing a different view' user has no access
> to, user cannot access the view as view' is considered direct input.
> For example:
> {code:java}
> create database db1;
> create database db2;
> create database db3;
>
> create table db1.table1 (cola string, colb string, colc string);
> insert into db1.table1 values ('a','b','c');
> insert into db1.table1 values ('x','y','z');
> CREATE VIEW db2.view1 AS SELECT cola, colb, colc FROM db1.table1 WHERE
> cola="x";
> CREATE VIEW db2.view2 AS SELECT table1.cola, table1.colb, table1.colc FROM
> db1.table1 WHERE table1.cola NOT IN (SELECT view1.cola FROM db2.view1);
> create view db3.view3 as select * from db2.view2
> {code}
> If test_user has read permission for only db3 (but not db1 or db2), their
> query
> {code:java}
> select * from db3.view3;{code}
> will fail with :
> {code:java}
> Error while compiling statement: FAILED: SemanticException No valid
> privileges User test_user does not have privileges for QUERY The required
> privileges: Server=server1->Db=db2->Table=view1->action=select; {code}
> WHERE IN and WHERE EXISTS cause the same issue.
> Cascading views created with no WHERE clauses (i.e. with simple SELECTs and
> FROM clauses) work fine.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20818) Views created with a WHERE subquery will regard views referenced in the subquery as direct input
[
https://issues.apache.org/jira/browse/HIVE-20818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688233#comment-16688233
]
Hive QA commented on HIVE-20818:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14946/dev-support/hive-personality.sh
|
| git revision | master / 6fbdf37 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14946/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Views created with a WHERE subquery will regard views referenced in the
> subquery as direct input
>
>
> Key: HIVE-20818
> URL: https://issues.apache.org/jira/browse/HIVE-20818
> Project: Hive
> Issue Type: Bug
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-20818.2.patch, HIVE-20818.3.patch,
> HIVE-20818.4.patch, HIVE-20818.5.patch, HIVE-20818.patch
>
>
> If Hive is configured with an authorization hook like Sentry, and a view is
> created with a WHERE clause referencing a different view' user has no access
> to, user cannot access the view as view' is considered direct input.
> For example:
> {code:java}
> create database db1;
> create database db2;
> create database db3;
>
> create table db1.table1 (cola string, colb string, colc string);
> insert into db1.table1 values ('a','b','c');
> insert into db1.table1 values ('x','y','z');
> CREATE VIEW db2.view1 AS SELECT cola, colb, colc FROM db1.table1 WHERE
> cola="x";
> CREATE VIEW db2.view2 AS SELECT table1.cola, table1.colb, table1.colc FROM
> db1.table1 WHERE table1.cola NOT IN (SELECT view1.cola FROM db2.view1);
> create view db3.view3 as select * from db2.view2
> {code}
> If test_user has read permission for only db3 (but not db1 or db2), their
> query
> {code:java}
> select * from db3.view3;{code}
> will fail with :
> {code:java}
> Error while compiling statement: FAILED: SemanticException No valid
> privileges User test_user does not have privileges for QUERY The required
> privileges:
[jira] [Commented] (HIVE-20914) MRScratchDir permission denied when "hive.server2.enable.doAs", "hive.exec.submitviachild" are set to "true" and impersonated/proxy user is used
[
https://issues.apache.org/jira/browse/HIVE-20914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688187#comment-16688187
]
Hive QA commented on HIVE-20914:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948308/HIVE-20914.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15543 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=197)
[druidmini_masking.q,druidmini_joins.q,druid_timestamptz.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14945/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14945/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14945/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948308 - PreCommit-HIVE-Build
> MRScratchDir permission denied when "hive.server2.enable.doAs",
> "hive.exec.submitviachild" are set to "true" and impersonated/proxy user is
> used
>
>
> Key: HIVE-20914
> URL: https://issues.apache.org/jira/browse/HIVE-20914
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20914.1.patch, HIVE-20914.2.patch,
> HIVE-20914.3.patch
>
>
> The above issue could be reproduced in none Kerberos cluster using the below
> steps:
> 1. Set "hive.exec.submitviachild" value to "true".
> 2. Run a count query not using "hive" user.
> {code}beeline -u 'jdbc:hive2://localhost:1' -n hdfs{code}
> There is no issue when we try to execute the same query using the "hive" user.
> {code:java}
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.hadoop.security.AccessControlException: Permission denied:
> user=hive, access=EXECUTE, inode="/tmp/hive/hdfs":hdfs:supergroup:drwx--
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:279)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:260)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkTraverse(DefaultAuthorizationProvider.java:201)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkPermission(DefaultAuthorizationProvider.java:154)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:152)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:3877)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:3860)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkTraverse(FSDirectory.java:3847)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkTraverse(FSNamesystem.java:6822)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:4551)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:4529)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4502)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:884)
> at
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.mkdirs(AuthorizationProviderProxyClientProtocol.java:328)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:641)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073) at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2281) at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2277) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
>
[jira] [Updated] (HIVE-20360) QTest: ignore driver/qtest exclusions if -Dqfile param is set
[
https://issues.apache.org/jira/browse/HIVE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-20360:
Attachment: HIVE-20360.06.patch
> QTest: ignore driver/qtest exclusions if -Dqfile param is set
> -
>
> Key: HIVE-20360
> URL: https://issues.apache.org/jira/browse/HIVE-20360
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Affects Versions: 3.1.0
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-20360.01.patch, HIVE-20360.02.patch,
> HIVE-20360.03.patch, HIVE-20360.04.patch, HIVE-20360.05.patch,
> HIVE-20360.06.patch
>
>
> Sometimes I need to run qtests with another driver for testing purposes. In
> this case I have to edit testconfiguration.properties which seems a bit
> hacky, even if it's temporary.
> In this case, no tests will run (however there's a log message):
> {code:java}
> mvn test -Pitests -pl itests/qtest -pl itests/util -Dtest=TestCliDriver
> -Dqfile=bucketizedhiveinputformat.q
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20914) MRScratchDir permission denied when "hive.server2.enable.doAs", "hive.exec.submitviachild" are set to "true" and impersonated/proxy user is used
[
https://issues.apache.org/jira/browse/HIVE-20914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688136#comment-16688136
]
Hive QA commented on HIVE-20914:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
36s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14945/dev-support/hive-personality.sh
|
| git revision | master / 6fbdf37 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14945/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> MRScratchDir permission denied when "hive.server2.enable.doAs",
> "hive.exec.submitviachild" are set to "true" and impersonated/proxy user is
> used
>
>
> Key: HIVE-20914
> URL: https://issues.apache.org/jira/browse/HIVE-20914
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20914.1.patch, HIVE-20914.2.patch,
> HIVE-20914.3.patch
>
>
> The above issue could be reproduced in none Kerberos cluster using the below
> steps:
> 1. Set "hive.exec.submitviachild" value to "true".
> 2. Run a count query not using "hive" user.
> {code}beeline -u 'jdbc:hive2://localhost:1' -n hdfs{code}
> There is no issue when we try to execute the same query using the "hive" user.
> {code:java}
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.hadoop.security.AccessControlException: Permission denied:
> user=hive, access=EXECUTE, inode="/tmp/hive/hdfs":hdfs:supergroup:drwx--
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:279)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:260)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkTraverse(DefaultAuthorizationProvider.java:201)
> at
>
[jira] [Updated] (HIVE-20914) MRScratchDir permission denied when "hive.server2.enable.doAs", "hive.exec.submitviachild" are set to "true" and impersonated/proxy user is used
[
https://issues.apache.org/jira/browse/HIVE-20914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-20914:
--
Attachment: HIVE-20914.3.patch
> MRScratchDir permission denied when "hive.server2.enable.doAs",
> "hive.exec.submitviachild" are set to "true" and impersonated/proxy user is
> used
>
>
> Key: HIVE-20914
> URL: https://issues.apache.org/jira/browse/HIVE-20914
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20914.1.patch, HIVE-20914.2.patch,
> HIVE-20914.3.patch
>
>
> The above issue could be reproduced in none Kerberos cluster using the below
> steps:
> 1. Set "hive.exec.submitviachild" value to "true".
> 2. Run a count query not using "hive" user.
> {code}beeline -u 'jdbc:hive2://localhost:1' -n hdfs{code}
> There is no issue when we try to execute the same query using the "hive" user.
> {code:java}
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.hadoop.security.AccessControlException: Permission denied:
> user=hive, access=EXECUTE, inode="/tmp/hive/hdfs":hdfs:supergroup:drwx--
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:279)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:260)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkTraverse(DefaultAuthorizationProvider.java:201)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkPermission(DefaultAuthorizationProvider.java:154)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:152)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:3877)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:3860)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkTraverse(FSDirectory.java:3847)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkTraverse(FSNamesystem.java:6822)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:4551)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:4529)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4502)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:884)
> at
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.mkdirs(AuthorizationProviderProxyClientProtocol.java:328)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:641)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073) at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2281) at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2277) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2275) at
> org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:285) at
> org.apache.hadoop.hive.ql.Context.getMRScratchDir(Context.java:328) at
> org.apache.hadoop.hive.ql.Context.getMRTmpPath(Context.java:444) at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:243) at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:771) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.util.RunJar.run(RunJar.java:221) at
> org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20924) Property 'hive.driver.parallel.compilation.global.limit' should be immutable at runtime
[
https://issues.apache.org/jira/browse/HIVE-20924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688100#comment-16688100
]
Hive QA commented on HIVE-20924:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948307/HIVE-20924.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 15543 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.addNoSuchTable[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.createGetDrop[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.createTableWithConstraintsPkInOtherCatalog[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.createTableWithConstraintsPk[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.doubleAddUniqueConstraint[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.getNoSuchCatalog[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.getNoSuchDb[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.getNoSuchTable[Remote]
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.inOtherCatalog[Remote]
(batchId=222)
org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow.testComplexQuery (batchId=259)
org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow.testKillQuery (batchId=259)
org.apache.hive.service.cli.session.TestSessionManagerMetrics.testAbandonedSessionMetrics
(batchId=236)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14944/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14944/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14944/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948307 - PreCommit-HIVE-Build
> Property 'hive.driver.parallel.compilation.global.limit' should be immutable
> at runtime
> ---
>
> Key: HIVE-20924
> URL: https://issues.apache.org/jira/browse/HIVE-20924
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20924.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20924) Property 'hive.driver.parallel.compilation.global.limit' should be immutable at runtime
[
https://issues.apache.org/jira/browse/HIVE-20924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688012#comment-16688012
]
Hive QA commented on HIVE-20924:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14944/dev-support/hive-personality.sh
|
| git revision | master / 6fbdf37 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14944/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Property 'hive.driver.parallel.compilation.global.limit' should be immutable
> at runtime
> ---
>
> Key: HIVE-20924
> URL: https://issues.apache.org/jira/browse/HIVE-20924
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20924.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Attachment: HIVE-20919.03.patch > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch, > HIVE-20919.03.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Status: Patch Available (was: Open) > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch, > HIVE-20919.03.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Status: Open (was: Patch Available) > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch, > HIVE-20919.03.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[
https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687979#comment-16687979
]
Hive QA commented on HIVE-20919:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948285/HIVE-20919.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15528 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=154)
[intersect_all.q,unionDistinct_1.q,table_nonprintable.q,orc_llap_counters1.q,mm_cttas.q,whroot_external1.q,global_limit.q,cte_2.q,rcfile_createas1.q,dynamic_partition_pruning_2.q,intersect_merge.q,results_cache_diff_fs.q,cttl.q,parallel_colstats.q,load_hdfs_file_with_space_in_the_name.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14943/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14943/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14943/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948285 - PreCommit-HIVE-Build
> Break up UpdateDeleteSemanticAnalyzer
> -
>
> Key: HIVE-20919
> URL: https://issues.apache.org/jira/browse/HIVE-20919
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.2.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch
>
>
> UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge
> queries by rewriting them to a different form. This is a clear violation of
> [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and
> therefore needs to be refactored. An abstract ancestor needs to take the
> common part, and each of the specific tasks should be handled by a separate
> class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[
https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687919#comment-16687919
]
Hive QA commented on HIVE-20919:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
42s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 6 new + 77 unchanged - 152
fixed = 83 total (was 229) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s{color} | {color:green} ql generated 0 new + 2315 unchanged - 1 fixed = 2315
total (was 2316) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14943/dev-support/hive-personality.sh
|
| git revision | master / 6fbdf37 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14943/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14943/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Break up UpdateDeleteSemanticAnalyzer
> -
>
> Key: HIVE-20919
> URL: https://issues.apache.org/jira/browse/HIVE-20919
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.2.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch
>
>
> UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge
> queries by rewriting them to a different form. This is a clear violation of
> [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and
> therefore needs to be refactored. An abstract ancestor needs to take the
> common part, and each of the specific tasks should be handled by a separate
> class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20923) masking_12 qtest expects wrong inputs
[
https://issues.apache.org/jira/browse/HIVE-20923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687877#comment-16687877
]
Yongzhi Chen commented on HIVE-20923:
-
[~klcopp], I think you are right, default@src should not be the input for query
select * from `v1_n9` .
The issue might be CTAS' dependencies (input) should only be checked when parse
the CTAS statement (itself), it should not be checked when the created table
used in other queries. The issue looks like masking_test_subq_n1 is checked as
part of CTAS in the query, but should not: it is only an ordinary table.
> masking_12 qtest expects wrong inputs
> -
>
> Key: HIVE-20923
> URL: https://issues.apache.org/jira/browse/HIVE-20923
> Project: Hive
> Issue Type: Bug
> Components: Query Planning, Test, Tests, Views
>Reporter: Karen Coppage
>Priority: Major
>
> It appears this qtest expects the wrong behavior, and should fail but doesn't.
> Below Queries 3 and 4 should not include default@src as Input
> Query 1
> {noformat}
> PREHOOK: query: create table `masking_test_subq_n1` as select cast(key as
> int) as key, value from src
> PREHOOK: type: CREATETABLE_AS_SELECT
> PREHOOK: Input: default@src
> PREHOOK: Output: database:default
> PREHOOK: Output: default@masking_test_subq_n1
> POSTHOOK: query: create table `masking_test_subq_n1` as select cast(key as
> int) as key, value from src
> POSTHOOK: type: CREATETABLE_AS_SELECT
> POSTHOOK: Input: default@src
> POSTHOOK: Output: database:default
> POSTHOOK: Output: default@masking_test_subq_n1
> POSTHOOK: Lineage: masking_test_subq_n1.key EXPRESSION
> [(src)src.FieldSchema(name:key, type:string, comment:default), ]
> POSTHOOK: Lineage: masking_test_subq_n1.value SIMPLE
> [(src)src.FieldSchema(name:value, type:string, comment:default), ]{noformat}
> Query 2
> {noformat}
> PREHOOK: query: create view `v1_n9` as select * from `masking_test_subq_n1`
> PREHOOK: type: CREATEVIEW
> PREHOOK: Input: default@masking_test_subq_n1
> PREHOOK: Output: database:default
> PREHOOK: Output: default@v1_n9
> POSTHOOK: query: create view `v1_n9` as select * from `masking_test_subq_n1`
> POSTHOOK: type: CREATEVIEW
> POSTHOOK: Input: default@masking_test_subq_n1
> POSTHOOK: Output: database:default
> POSTHOOK: Output: default@v1_n9
> POSTHOOK: Lineage: v1_n9.key SIMPLE
> [(masking_test_subq_n1)masking_test_subq_n1.FieldSchema(name:key, type:int,
> comment:null), ]
> POSTHOOK: Lineage: v1_n9.value SIMPLE
> [(masking_test_subq_n1)masking_test_subq_n1.FieldSchema(name:value,
> type:string, comment:null), ]
> {noformat}
> Query 3 (the explain output here describes a TableScan on table src)
> {noformat}
> PREHOOK: query: explain
> select * from `v1_n9`
> limit 20
> PREHOOK: type: QUERY
> PREHOOK: Input: default@masking_test_subq_n1
> PREHOOK: Input: default@src
> PREHOOK: Input: default@v1_n9
> A masked pattern was here
> POSTHOOK: query: explain
> select * from `v1_n9`
> limit 20
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@masking_test_subq_n1
> POSTHOOK: Input: default@src
> POSTHOOK: Input: default@v1_n9
> {noformat}
> Query 4
> {noformat}
> PREHOOK: query: select * from `v1_n9`
> limit 20
> PREHOOK: type: QUERY
> PREHOOK: Input: default@masking_test_subq_n1
> PREHOOK: Input: default@src
> PREHOOK: Input: default@v1_n9
> A masked pattern was here
> POSTHOOK: query: select * from `v1_n9`
> limit 20
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@masking_test_subq_n1
> POSTHOOK: Input: default@src
> POSTHOOK: Input: default@v1_n9
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20921) Oracle backed DbLockManager fail when drop/truncate acid table with large partitions
[
https://issues.apache.org/jira/browse/HIVE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687870#comment-16687870
]
Hive QA commented on HIVE-20921:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948272/HIVE-20921.branch-2.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14942/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14942/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14942/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-11-15 11:23:21.085
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-14942/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z branch-2.1 ]]
+ [[ -d apache-github-branch-2.1-source ]]
+ [[ ! -d apache-github-branch-2.1-source/.git ]]
+ [[ ! -d apache-github-branch-2.1-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-11-15 11:23:21.128
+ cd apache-github-branch-2.1-source
+ git fetch origin
>From https://github.com/apache/hive
5a62503..292a98f branch-2.1 -> origin/branch-2.1
* [new branch] HIVE-19429 -> origin/HIVE-19429
d4bbcfd..7fecb6f branch-1 -> origin/branch-1
18ddf46..9bcdb54 branch-1.2 -> origin/branch-1.2
564d529..0083145 branch-2 -> origin/branch-2
120476d..b148507 branch-2.2 -> origin/branch-2.2
6f4c35c..3b1d4fd branch-2.3 -> origin/branch-2.3
* [new branch] branch-3 -> origin/branch-3
* [new branch] branch-3.0 -> origin/branch-3.0
* [new branch] branch-3.1 -> origin/branch-3.1
81853c1..ba9cd79 hive-14535 -> origin/hive-14535
* [new branch] hive-18320-acid-repl -> origin/hive-18320-acid-repl
d012f86..6fbdf37 master -> origin/master
* [new branch] master-tez092 -> origin/master-tez092
* [new branch] master-txnstats -> origin/master-txnstats
* [new branch] standalone-metastore -> origin/standalone-metastore
* [new branch] storage-branch-2.5 -> origin/storage-branch-2.5
* [new branch] storage-branch-2.6 -> origin/storage-branch-2.6
* [new branch] storage-branch-2.7 -> origin/storage-branch-2.7
* [new tag] rel/storage-release-2.6.1 -> rel/storage-release-2.6.1
* [new tag] rel/storage-release-2.7.0 -> rel/storage-release-2.7.0
* [new tag] storage-release-2.7.0-rc1 -> storage-release-2.7.0-rc1
* [new tag] rel/release-2.3.1 -> rel/release-2.3.1
* [new tag] rel/release-2.3.2 -> rel/release-2.3.2
* [new tag] rel/release-3.0.0 -> rel/release-3.0.0
* [new tag] rel/release-3.1.0 -> rel/release-3.1.0
* [new tag] rel/release-3.1.1 -> rel/release-3.1.1
* [new tag] rel/standalone-metastore-release-3.0.0 ->
rel/standalone-metastore-release-3.0.0
* [new tag] rel/storage-release-2.5.0 -> rel/storage-release-2.5.0
* [new tag] rel/storage-release-2.6.0 -> rel/storage-release-2.6.0
+ git reset --hard HEAD
HEAD is now at 5a62503 HIVE-16683. Backport of ORC-125 to fix incorrect
handling of future
+ git clean -f -d
+ git checkout branch-2.1
Already on 'branch-2.1'
Your branch is behind 'origin/branch-2.1' by 4 commits, and can be
fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/branch-2.1
HEAD is now at 292a98f HIVE-16480: Empty vector batches of floats or doubles
gets EOFException (Owen O'Malley via Jesus Camacho Rodriguez)
+ git merge --ff-only origin/branch-2.1
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-11-15 11:23:52.251
+ rm -rf ../yetus_PreCommit-HIVE-Build-14942
+ mkdir ../yetus_PreCommit-HIVE-Build-14942
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-14942
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-14942/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
[jira] [Commented] (HIVE-20920) Use SQL constraints to improve join reordering algorithm (II)
[
https://issues.apache.org/jira/browse/HIVE-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687867#comment-16687867
]
Hive QA commented on HIVE-20920:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948257/HIVE-20920.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15538 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=86)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions
(batchId=259)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=259)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14941/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14941/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14941/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948257 - PreCommit-HIVE-Build
> Use SQL constraints to improve join reordering algorithm (II)
> -
>
> Key: HIVE-20920
> URL: https://issues.apache.org/jira/browse/HIVE-20920
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20920.patch
>
>
> Follow-up of HIVE-20744.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20818) Views created with a WHERE subquery will regard views referenced in the subquery as direct input
[
https://issues.apache.org/jira/browse/HIVE-20818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage updated HIVE-20818:
-
Attachment: HIVE-20818.5.patch
Status: Patch Available (was: Open)
> Views created with a WHERE subquery will regard views referenced in the
> subquery as direct input
>
>
> Key: HIVE-20818
> URL: https://issues.apache.org/jira/browse/HIVE-20818
> Project: Hive
> Issue Type: Bug
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-20818.2.patch, HIVE-20818.3.patch,
> HIVE-20818.4.patch, HIVE-20818.5.patch, HIVE-20818.patch
>
>
> If Hive is configured with an authorization hook like Sentry, and a view is
> created with a WHERE clause referencing a different view' user has no access
> to, user cannot access the view as view' is considered direct input.
> For example:
> {code:java}
> create database db1;
> create database db2;
> create database db3;
>
> create table db1.table1 (cola string, colb string, colc string);
> insert into db1.table1 values ('a','b','c');
> insert into db1.table1 values ('x','y','z');
> CREATE VIEW db2.view1 AS SELECT cola, colb, colc FROM db1.table1 WHERE
> cola="x";
> CREATE VIEW db2.view2 AS SELECT table1.cola, table1.colb, table1.colc FROM
> db1.table1 WHERE table1.cola NOT IN (SELECT view1.cola FROM db2.view1);
> create view db3.view3 as select * from db2.view2
> {code}
> If test_user has read permission for only db3 (but not db1 or db2), their
> query
> {code:java}
> select * from db3.view3;{code}
> will fail with :
> {code:java}
> Error while compiling statement: FAILED: SemanticException No valid
> privileges User test_user does not have privileges for QUERY The required
> privileges: Server=server1->Db=db2->Table=view1->action=select; {code}
> WHERE IN and WHERE EXISTS cause the same issue.
> Cascading views created with no WHERE clauses (i.e. with simple SELECTs and
> FROM clauses) work fine.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20914) MRScratchDir permission denied when "hive.server2.enable.doAs", "hive.exec.submitviachild" are set to "true" and impersonated/proxy user is used
[
https://issues.apache.org/jira/browse/HIVE-20914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-20914:
--
Attachment: HIVE-20914.2.patch
> MRScratchDir permission denied when "hive.server2.enable.doAs",
> "hive.exec.submitviachild" are set to "true" and impersonated/proxy user is
> used
>
>
> Key: HIVE-20914
> URL: https://issues.apache.org/jira/browse/HIVE-20914
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20914.1.patch, HIVE-20914.2.patch
>
>
> The above issue could be reproduced in none Kerberos cluster using the below
> steps:
> 1. Set "hive.exec.submitviachild" value to "true".
> 2. Run a count query not using "hive" user.
> {code}beeline -u 'jdbc:hive2://localhost:1' -n hdfs{code}
> There is no issue when we try to execute the same query using the "hive" user.
> {code:java}
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.hadoop.security.AccessControlException: Permission denied:
> user=hive, access=EXECUTE, inode="/tmp/hive/hdfs":hdfs:supergroup:drwx--
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:279)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:260)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkTraverse(DefaultAuthorizationProvider.java:201)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkPermission(DefaultAuthorizationProvider.java:154)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:152)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:3877)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:3860)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkTraverse(FSDirectory.java:3847)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkTraverse(FSNamesystem.java:6822)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:4551)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:4529)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4502)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:884)
> at
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.mkdirs(AuthorizationProviderProxyClientProtocol.java:328)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:641)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073) at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2281) at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2277) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2275) at
> org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:285) at
> org.apache.hadoop.hive.ql.Context.getMRScratchDir(Context.java:328) at
> org.apache.hadoop.hive.ql.Context.getMRTmpPath(Context.java:444) at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:243) at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:771) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.util.RunJar.run(RunJar.java:221) at
> org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20924) Property 'hive.driver.parallel.compilation.global.limit' should be immutable at runtime
[ https://issues.apache.org/jira/browse/HIVE-20924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-20924: -- Status: Patch Available (was: Open) > Property 'hive.driver.parallel.compilation.global.limit' should be immutable > at runtime > --- > > Key: HIVE-20924 > URL: https://issues.apache.org/jira/browse/HIVE-20924 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-20924.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20924) Property 'hive.driver.parallel.compilation.global.limit' should be immutable at runtime
[ https://issues.apache.org/jira/browse/HIVE-20924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-20924: -- Attachment: HIVE-20924.1.patch > Property 'hive.driver.parallel.compilation.global.limit' should be immutable > at runtime > --- > > Key: HIVE-20924 > URL: https://issues.apache.org/jira/browse/HIVE-20924 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Denys Kuzmenko >Priority: Major > Attachments: HIVE-20924.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20924) Property 'hive.driver.parallel.compilation.global.limit' should be immutable at runtime
[ https://issues.apache.org/jira/browse/HIVE-20924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko reassigned HIVE-20924: - Assignee: Denys Kuzmenko > Property 'hive.driver.parallel.compilation.global.limit' should be immutable > at runtime > --- > > Key: HIVE-20924 > URL: https://issues.apache.org/jira/browse/HIVE-20924 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-20924.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-14557) Nullpointer When both SkewJoin and Mapjoin Enabled
[
https://issues.apache.org/jira/browse/HIVE-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687821#comment-16687821
]
Yongzhi Chen commented on HIVE-14557:
-
PATCH 3 looks good. +1
> Nullpointer When both SkewJoin and Mapjoin Enabled
> ---
>
> Key: HIVE-14557
> URL: https://issues.apache.org/jira/browse/HIVE-14557
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 1.1.0, 2.1.0
>Reporter: Nemon Lou
>Assignee: Rui Li
>Priority: Major
> Attachments: HIVE-14557.2.patch, HIVE-14557.3.patch,
> HIVE-14557.3.patch, HIVE-14557.patch
>
>
> The following sql failed with return code 2 on mr.
> {noformat}
> create table a(id int,id1 int);
> create table b(id int,id1 int);
> create table c(id int,id1 int);
> set hive.optimize.skewjoin=true;
> select a.id,b.id,c.id1 from a,b,c where a.id=b.id and a.id1=c.id1;
> {noformat}
> Error log as follows:
> {noformat}
> 2016-08-17 21:13:42,081 INFO [main]
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper:
> Id =0
>
> Id =21
>
> Id =28
>
> Id =16
>
> <\Children>
> Id = 28 null<\Parent>
> <\FS>
> <\Children>
> Id = 21 nullId = 33
> Id =33
> null
> <\Children>
> <\Parent>
> <\HASHTABLEDUMMY><\Parent>
> <\MAPJOIN>
> <\Children>
> Id = 0 null<\Parent>
> <\TS>
> <\Children>
> <\Parent>
> <\MAP>
> 2016-08-17 21:13:42,084 INFO [main]
> org.apache.hadoop.hive.ql.exec.TableScanOperator: Initializing operator TS[21]
> 2016-08-17 21:13:42,084 INFO [main]
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper: Initializing dummy operator
> 2016-08-17 21:13:42,086 INFO [main]
> org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:0,
> RECORDS_IN:0,
> 2016-08-17 21:13:42,087 ERROR [main]
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper: Hit error while closing
> operators - failing tree
> 2016-08-17 21:13:42,088 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException: Hive Runtime Error
> while closing operators
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:207)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.closeOp(MapJoinOperator.java:474)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:682)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:696)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:696)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189)
> ... 8 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20818) Views created with a WHERE subquery will regard views referenced in the subquery as direct input
[
https://issues.apache.org/jira/browse/HIVE-20818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage updated HIVE-20818:
-
Status: Open (was: Patch Available)
> Views created with a WHERE subquery will regard views referenced in the
> subquery as direct input
>
>
> Key: HIVE-20818
> URL: https://issues.apache.org/jira/browse/HIVE-20818
> Project: Hive
> Issue Type: Bug
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-20818.2.patch, HIVE-20818.3.patch,
> HIVE-20818.4.patch, HIVE-20818.patch
>
>
> If Hive is configured with an authorization hook like Sentry, and a view is
> created with a WHERE clause referencing a different view' user has no access
> to, user cannot access the view as view' is considered direct input.
> For example:
> {code:java}
> create database db1;
> create database db2;
> create database db3;
>
> create table db1.table1 (cola string, colb string, colc string);
> insert into db1.table1 values ('a','b','c');
> insert into db1.table1 values ('x','y','z');
> CREATE VIEW db2.view1 AS SELECT cola, colb, colc FROM db1.table1 WHERE
> cola="x";
> CREATE VIEW db2.view2 AS SELECT table1.cola, table1.colb, table1.colc FROM
> db1.table1 WHERE table1.cola NOT IN (SELECT view1.cola FROM db2.view1);
> create view db3.view3 as select * from db2.view2
> {code}
> If test_user has read permission for only db3 (but not db1 or db2), their
> query
> {code:java}
> select * from db3.view3;{code}
> will fail with :
> {code:java}
> Error while compiling statement: FAILED: SemanticException No valid
> privileges User test_user does not have privileges for QUERY The required
> privileges: Server=server1->Db=db2->Table=view1->action=select; {code}
> WHERE IN and WHERE EXISTS cause the same issue.
> Cascading views created with no WHERE clauses (i.e. with simple SELECTs and
> FROM clauses) work fine.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20920) Use SQL constraints to improve join reordering algorithm (II)
[
https://issues.apache.org/jira/browse/HIVE-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687779#comment-16687779
]
Hive QA commented on HIVE-20920:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 3 new + 20 unchanged - 0 fixed
= 23 total (was 20) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14941/dev-support/hive-personality.sh
|
| git revision | master / 6fbdf37 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14941/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14941/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Use SQL constraints to improve join reordering algorithm (II)
> -
>
> Key: HIVE-20920
> URL: https://issues.apache.org/jira/browse/HIVE-20920
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20920.patch
>
>
> Follow-up of HIVE-20744.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20923) masking_12 qtest expects wrong inputs
[
https://issues.apache.org/jira/browse/HIVE-20923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687773#comment-16687773
]
Karen Coppage commented on HIVE-20923:
--
[~daijy] since you exposed this problem, and [~jcamachorodriguez] since you
created the test, and [~ychena]: Can you tell me if I am incorrect, and if not
then any insights into what the issue might be? Thanks.
> masking_12 qtest expects wrong inputs
> -
>
> Key: HIVE-20923
> URL: https://issues.apache.org/jira/browse/HIVE-20923
> Project: Hive
> Issue Type: Bug
> Components: Query Planning, Test, Tests, Views
>Reporter: Karen Coppage
>Priority: Major
>
> It appears this qtest expects the wrong behavior, and should fail but doesn't.
> Below Queries 3 and 4 should not include default@src as Input
> Query 1
> {noformat}
> PREHOOK: query: create table `masking_test_subq_n1` as select cast(key as
> int) as key, value from src
> PREHOOK: type: CREATETABLE_AS_SELECT
> PREHOOK: Input: default@src
> PREHOOK: Output: database:default
> PREHOOK: Output: default@masking_test_subq_n1
> POSTHOOK: query: create table `masking_test_subq_n1` as select cast(key as
> int) as key, value from src
> POSTHOOK: type: CREATETABLE_AS_SELECT
> POSTHOOK: Input: default@src
> POSTHOOK: Output: database:default
> POSTHOOK: Output: default@masking_test_subq_n1
> POSTHOOK: Lineage: masking_test_subq_n1.key EXPRESSION
> [(src)src.FieldSchema(name:key, type:string, comment:default), ]
> POSTHOOK: Lineage: masking_test_subq_n1.value SIMPLE
> [(src)src.FieldSchema(name:value, type:string, comment:default), ]{noformat}
> Query 2
> {noformat}
> PREHOOK: query: create view `v1_n9` as select * from `masking_test_subq_n1`
> PREHOOK: type: CREATEVIEW
> PREHOOK: Input: default@masking_test_subq_n1
> PREHOOK: Output: database:default
> PREHOOK: Output: default@v1_n9
> POSTHOOK: query: create view `v1_n9` as select * from `masking_test_subq_n1`
> POSTHOOK: type: CREATEVIEW
> POSTHOOK: Input: default@masking_test_subq_n1
> POSTHOOK: Output: database:default
> POSTHOOK: Output: default@v1_n9
> POSTHOOK: Lineage: v1_n9.key SIMPLE
> [(masking_test_subq_n1)masking_test_subq_n1.FieldSchema(name:key, type:int,
> comment:null), ]
> POSTHOOK: Lineage: v1_n9.value SIMPLE
> [(masking_test_subq_n1)masking_test_subq_n1.FieldSchema(name:value,
> type:string, comment:null), ]
> {noformat}
> Query 3 (the explain output here describes a TableScan on table src)
> {noformat}
> PREHOOK: query: explain
> select * from `v1_n9`
> limit 20
> PREHOOK: type: QUERY
> PREHOOK: Input: default@masking_test_subq_n1
> PREHOOK: Input: default@src
> PREHOOK: Input: default@v1_n9
> A masked pattern was here
> POSTHOOK: query: explain
> select * from `v1_n9`
> limit 20
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@masking_test_subq_n1
> POSTHOOK: Input: default@src
> POSTHOOK: Input: default@v1_n9
> {noformat}
> Query 4
> {noformat}
> PREHOOK: query: select * from `v1_n9`
> limit 20
> PREHOOK: type: QUERY
> PREHOOK: Input: default@masking_test_subq_n1
> PREHOOK: Input: default@src
> PREHOOK: Input: default@v1_n9
> A masked pattern was here
> POSTHOOK: query: select * from `v1_n9`
> limit 20
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@masking_test_subq_n1
> POSTHOOK: Input: default@src
> POSTHOOK: Input: default@v1_n9
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20911) External Table Replication for Hive
[
https://issues.apache.org/jira/browse/HIVE-20911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
anishek updated HIVE-20911:
---
Description:
External tables are not replicated currently as part of hive replication. As
part of this jira we want to enable that.
Approach:
* Target cluster will have a top level base directory config that will be used
to copy all data relevant to external tables. This will be provided via the
*with* clause in the *repl load* command. This base path will be prefixed to
the path of the same external table on source cluster.
* Since changes to directories on the external table can happen without hive
knowing it, hence we cant capture the relevant events when ever new data is
added or removed, we will have to copy the data from the source path to target
path for external tables every time we run incremental replication.
** this will require incremental *repl dump* to now create an additional file
*\_external\_tables\_info* with data in the following form
{code}
tableName,base64Encoded(tableDataLocation)
{code}
** *repl load* will read the *\_external\_tables\_info* to identify what
locations are to be copied from source to target and create corresponding tasks
for them.
* New External tables will be created with metadata only with no data copied as
part of regular tasks while incremental load/bootstrap load.
* Bootstrap dump will also create *\_external\_tables\_info* which will be
used to copy data from source to target as part of boostrap load.
* Since bootstrap load will most probably create a DAG to that can use
parallelism in the execution phase, the hdfs copy related tasks are created
only once the bootstrap phase is complete.
* Since incremental load results in a DAG with only sequential execution (
events applied in sequence ) to effectively use the parallelism capability in
execution mode we created tasks for hdfs copy along with the incremental
related DAG, This requires a few basic calculations to approximately meet the
configured value in "hive.repl.approx.max.load.tasks"
was:
External tables are not replicated currently as part of hive replication. As
part of this jira we want to enable that.
Approach:
* Target cluster will have a top level base directory config that will be used
to copy all data relevant to external tables. This will be provided via the
*with* clause in the *repl load* command. This base path will be prefixed to
the path of the same external table on source cluster.
* Since changes to directories on the external table can happen without hive
knowing it, hence we cant capture the relevant events when ever new data is
added or removed, we will have to copy the data from the source path to target
path for external tables every time we run incremental replication.
** this will require incremental *repl dump* to now create an additional file
*\_external\_tables\_info* with data in the following form
{code}
tableName,base64Encoded(tableDataLocation)
{code}
** *repl load* will read the *\_external\_tables\_info* to identify what
locations are to be copied from source to target and create corresponding tasks
for them.
* New External tables will be created with metadata only with no data copied as
part of regular tasks while incremental load/bootstrap load.
* Bootstrap dump will also create *\_external\_tables\_info* which will be
used to copy data from source to target as part of boostrap load.
> External Table Replication for Hive
> ---
>
> Key: HIVE-20911
> URL: https://issues.apache.org/jira/browse/HIVE-20911
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: anishek
>Assignee: anishek
>Priority: Critical
> Fix For: 4.0.0
>
>
> External tables are not replicated currently as part of hive replication. As
> part of this jira we want to enable that.
> Approach:
> * Target cluster will have a top level base directory config that will be
> used to copy all data relevant to external tables. This will be provided via
> the *with* clause in the *repl load* command. This base path will be prefixed
> to the path of the same external table on source cluster.
> * Since changes to directories on the external table can happen without hive
> knowing it, hence we cant capture the relevant events when ever new data is
> added or removed, we will have to copy the data from the source path to
> target path for external tables every time we run incremental replication.
> ** this will require incremental *repl dump* to now create an additional
> file *\_external\_tables\_info* with data in the following form
> {code}
> tableName,base64Encoded(tableDataLocation)
> {code}
> ** *repl load* will read the *\_external\_tables\_info* to identify what
> locations are to be
[jira] [Updated] (HIVE-20922) HiveServer2 user is switched to another occasionally
[ https://issues.apache.org/jira/browse/HIVE-20922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao Junliang updated HIVE-20922: Description: The HiveServer2（version 1.2.1）authentication configuration is: authentication type: kerberos kerberos principal: hive/h...@dc1.fh.com Other infomation: client principal: n...@dc1.fh.com yarn service principal:yarn/_h...@dc1.fh.com After the hive service was started, application could use jdbc to access hive service. But sometime later, clients got the exception. The user of the hive service was switched to another one (yarn). In my environment, Hive and Yarn were deployed in same machine, the user of yarn service is yarn/_h...@dc1.fh.com. Before the exception occured, the hive service was not restarted. == Main logs: 2018-11-11 23:59:35 [HiveServer2-Handler-Pool: Thread-115739] INFO org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(294) -Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V8 2018-11-11 23:59:35 [HiveServer2-Handler-Pool: Thread-115739] WARN org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(308) -Error opening session: org.apache.hive.service.cli.HiveSQLException: Failed to open new session: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: yarn/mas...@dc1.fh.com is not allowed to impersonate nbhd at org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:266) at org.apache.hive.service.cli.CLIService.openSessionWithImpersonation(CLIService.java:202) at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:402) at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:297) at org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1253) at org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1238) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:692) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: yarn/mas...@dc1.fh.com is not allowed to impersonate nbhd at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:83) at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) at com.sun.proxy.$Proxy19.open(Unknown Source) at org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:258) ... 12 more Caused by: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: yarn/mas...@dc1.fh.com is not allowed to impersonate nbhd at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522) at org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:137) at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) ... 20 more Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: yarn/mas...@dc1.fh.com is not allowed to impersonate nbhd at org.apache.hadoop.ipc.Client.call(Client.java:1468) at org.apache.hadoop.ipc.Client.call(Client.java:1399) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at com.sun.proxy.$Proxy14.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:752) at
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Status: Patch Available (was: Open) > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Status: Open (was: Patch Available) > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Attachment: HIVE-20919.02.patch > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20918) Flag to enable/disable pushdown of computation from Calcite into JDBC connection
[
https://issues.apache.org/jira/browse/HIVE-20918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687655#comment-16687655
]
Hive QA commented on HIVE-20918:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948252/HIVE-20918.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15543 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_limit]
(batchId=182)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14940/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14940/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14940/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948252 - PreCommit-HIVE-Build
> Flag to enable/disable pushdown of computation from Calcite into JDBC
> connection
>
>
> Key: HIVE-20918
> URL: https://issues.apache.org/jira/browse/HIVE-20918
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20918.01.patch, HIVE-20918.01.patch,
> HIVE-20918.01.patch, HIVE-20918.patch
>
>
> Currently, there is no way to disable it. We will add a flag for that. By
> default, pushdown of computation will be enabled.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16081) allow "0.23" shim creation for Hadoop 3
[ https://issues.apache.org/jira/browse/HIVE-16081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687636#comment-16687636 ] Yuming Wang commented on HIVE-16081: [~sershe] Cloud we backport this to branch-1? > allow "0.23" shim creation for Hadoop 3 > --- > > Key: HIVE-16081 > URL: https://issues.apache.org/jira/browse/HIVE-16081 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 2.3.0 > > Attachments: HIVE-16081.patch > > > We only have one shim, so for now, we are going to use that. Hopefully we > won't need shims and build profiles for Hadoop 3 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20918) Flag to enable/disable pushdown of computation from Calcite into JDBC connection
[
https://issues.apache.org/jira/browse/HIVE-20918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687629#comment-16687629
]
Hive QA commented on HIVE-20918:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
41s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14940/dev-support/hive-personality.sh
|
| git revision | master / 6fbdf37 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14940/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Flag to enable/disable pushdown of computation from Calcite into JDBC
> connection
>
>
> Key: HIVE-20918
> URL: https://issues.apache.org/jira/browse/HIVE-20918
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20918.01.patch, HIVE-20918.01.patch,
> HIVE-20918.01.patch, HIVE-20918.patch
>
>
> Currently, there is no way to disable it. We will add a flag for that. By
> default, pushdown of computation will be enabled.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
