date:20180411

[jira] [Commented] (HIVE-19104) When test MetaStore is started with retry the instances should be independent

2018-04-11 Thread Peter Vary (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434294#comment-16434294
 ] 

Peter Vary commented on HIVE-19104:
---

bq. Why not just delete the entry for javax.jdo.option.ConnectionURL from 
data/conf/hive-site.xml and force tests to explicitly set it (or change 
MetaStoreUtils.java to ignore the set value). 

The purpose of the MetaStoreTestUtils is to allow easy metastore instance 
creation for tests and the user should not have to bother with details, like 
making the jdbc url independent. On the other hand I did want to have the 
possibility to define the jdbc url if needed. This is the same approach I 
followed with the warehouse dir. If you have better ideas, I am open for 
anything, since I am not entirely comfortable with this solution.

bq. Why are the changes to TestHiveMetaStore necessary?

Otherwise when running the second test the @Before method will regenerate the 
conf changes (port settings, warehouse dir changes etc)

> When test MetaStore is started with retry the instances should be independent
> -
>
> Key: HIVE-19104
> URL: https://issues.apache.org/jira/browse/HIVE-19104
> Project: Hive
>  Issue Type: Improvement
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19104.2.patch, HIVE-19104.3.patch, HIVE-19104.patch
>
>
> When multiple MetaStore instances are started with 
> {{MetaStoreTestUtils.startMetaStoreWithRetry}} currently they use the same 
> JDBC url, and warehouse directory. This can cause problem in the tests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19130) NPE is thrown when REPL LOAD applied drop partition event.

2018-04-11 Thread Thejas M Nair (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434282#comment-16434282
 ] 

Thejas M Nair commented on HIVE-19130:
--

+1


> NPE is thrown when REPL LOAD applied drop partition event.
> --
>
> Key: HIVE-19130
> URL: https://issues.apache.org/jira/browse/HIVE-19130
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: DR, Replication, pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19130.01.patch
>
>
> During incremental replication, if we split the events batch as follows, then 
> the REPL LOAD on second batch throws NPE.
> Batch-1: CREATE_TABLE(t1) -> ADD_PARTITION(t1.p1) -> DROP_PARTITION (t1.p1)
> Batch-2: DROP_TABLE(t1) ->  CREATE_TABLE(t1) -> ADD_PARTITION(t1.p1) -> 
> DROP_PARTITION (t1.p1)
> {code}
> 2018-04-05 16:20:36,531 ERROR [HiveServer2-Background-Pool: Thread-107044]: 
> metadata.Hive (Hive.java:getTable(1219)) - Table catalog_sales_new not found: 
> new5_tpcds_real_bin_partitioned_orc_1000.catalog_sales_new table not found
> 2018-04-05 16:20:36,538 ERROR [HiveServer2-Background-Pool: Thread-107044]: 
> exec.DDLTask (DDLTask.java:failed(540)) - 
> org.apache.hadoop.hive.ql.metadata.HiveException
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:4016)
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:3983)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:341)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:162)
> at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1765)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1506)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1303)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1165)
> at 
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197)
> at 
> org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76)
> at 
> org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
> at 
> org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:266)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NullPointerException
> at 
> org.apache.hadoop.hive.ql.metadata.Hive.getPartitionsByExpr(Hive.java:2613)
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:4008)
> ... 23 more
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19174) Enhance HMS to support drop of partitions with partition names as input

2018-04-11 Thread Janaki Lahorani (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434276#comment-16434276
 ] 

Janaki Lahorani commented on HIVE-19174:


Based on the following comment from [~pvary] on HIVE-17824:
The metastore thrift interface provides the possibility to drop partitions 
based on names, so we have to add this possibility to the IMetaStoreClient 
interface too, and use this here. And probably at 
https://github.com/apache/hive/blob/master/hcatalog/webhcat/java-client/src/main/java/org/apache/hive/hcatalog/api/HCatClientHMSImpl.java#L514
 too.


> Enhance HMS to support drop of partitions with partition names as input
> ---
>
> Key: HIVE-19174
> URL: https://issues.apache.org/jira/browse/HIVE-19174
> Project: Hive
>  Issue Type: Improvement
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-17824) msck repair table should drop the missing partitions from metastore

2018-04-11 Thread Janaki Lahorani (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-17824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434277#comment-16434277
 ] 

Janaki Lahorani commented on HIVE-17824:


Thanks [~pvary].  I filed HIVE-19174 to track the same.

> msck repair table should drop the missing partitions from metastore
> ---
>
> Key: HIVE-17824
> URL: https://issues.apache.org/jira/browse/HIVE-17824
> Project: Hive
>  Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-17824.1.patch, HIVE-17824.2.patch, 
> HIVE-17824.3.patch
>
>
> {{msck repair table }} is often used in environments where the new 
> partitions are loaded as directories on HDFS or S3 and users want to create 
> the missing partitions in bulk. However, currently it only supports addition 
> of missing partitions. If there are any partitions which are present in 
> metastore but not on the FileSystem, it should also delete them so that it 
> truly repairs the table metadata.
> We should be careful not to break backwards compatibility so we should either 
> introduce a new config or keyword to add support to delete unnecessary 
> partitions from the metastore. This way users who want the old behavior can 
> easily turn it off. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19174) Enhance HMS to support drop of partitions with partition names as input

2018-04-11 Thread Janaki Lahorani (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Janaki Lahorani reassigned HIVE-19174:
--


> Enhance HMS to support drop of partitions with partition names as input
> ---
>
> Key: HIVE-19174
> URL: https://issues.apache.org/jira/browse/HIVE-19174
> Project: Hive
>  Issue Type: Improvement
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19104) When test MetaStore is started with retry the instances should be independent

2018-04-11 Thread Sahil Takiar (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434264#comment-16434264
 ] 

Sahil Takiar commented on HIVE-19104:
-

A few comments:

Why not just delete the entry for {{javax.jdo.option.ConnectionURL}} from 
{{data/conf/hive-site.xml}} and force tests to explicitly set it (or change 
{{MetaStoreUtils.java}} to ignore the set value). This seems a bit cleaner (and 
easier to follow) than replacing {{junit_metastore_db}} with 
{{junit_metastore_db_[port]}}. Ideally, the name of the database set in the 
connection URL in the {{hive-site.xml}} shouldn't matter, this approach adds an 
implicit dependency on the name.

Why are the changes to {{TestHiveMetaStore}} necessary?

> When test MetaStore is started with retry the instances should be independent
> -
>
> Key: HIVE-19104
> URL: https://issues.apache.org/jira/browse/HIVE-19104
> Project: Hive
>  Issue Type: Improvement
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19104.2.patch, HIVE-19104.3.patch, HIVE-19104.patch
>
>
> When multiple MetaStore instances are started with 
> {{MetaStoreTestUtils.startMetaStoreWithRetry}} currently they use the same 
> JDBC url, and warehouse directory. This can cause problem in the tests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18469) HS2UI: Introduce separate option to show query on web ui

2018-04-11 Thread Bharathkrishna Guruvayoor Murali (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434240#comment-16434240
 ] 

Bharathkrishna Guruvayoor Murali commented on HIVE-18469:
-

Hi [~kgyrtkirk] , Can I work on this JIRA?

> HS2UI: Introduce separate option to show query on web ui
> 
>
> Key: HIVE-18469
> URL: https://issues.apache.org/jira/browse/HIVE-18469
> Project: Hive
>  Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Priority: Major
>
> currently {{ConfVars.HIVE_LOG_EXPLAIN_OUTPUT}} enables 2 features:
> * log the query to the console (even thru beeline)
> * shows the query on the web ui
> I've enabled it...and ever since then my beeline is always flooded with an 
> {{explain extended}} output...which is very verbose; even for simple queries.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19170) Fix TestMiniDruidKafkaCliDriver

2018-04-11 Thread Zoltan Haindrich (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434228#comment-16434228
 ] 

Zoltan Haindrich commented on HIVE-19170:
-

[the config references 
it|https://github.com/apache/hive/blob/f8a671d8cfe8a26d1d12c51f93207ec92577c796/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CliConfigs.java#L196];
 but unfortunately the handler method threats non defined input as empty; and 
in that case it will include all files from the querydir


> Fix TestMiniDruidKafkaCliDriver
> ---
>
> Key: HIVE-19170
> URL: https://issues.apache.org/jira/browse/HIVE-19170
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-19170.patch
>
>
> added in HIVE-18976 
> the property key {{druid.kafka.query.files}} doesn't exists in 
> testconfiguration.properties.
> because of this TestMiniDruidKafkaCliDriver tries to run *all* qtests...which 
> time out...and produce 
> {code}
> TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed 
> out) (batchId=252)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19170) Fix TestMiniDruidKafkaCliDriver

2018-04-11 Thread Zoltan Haindrich (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434229#comment-16434229
 ] 

Zoltan Haindrich commented on HIVE-19170:
-

+1

> Fix TestMiniDruidKafkaCliDriver
> ---
>
> Key: HIVE-19170
> URL: https://issues.apache.org/jira/browse/HIVE-19170
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-19170.patch
>
>
> added in HIVE-18976 
> the property key {{druid.kafka.query.files}} doesn't exists in 
> testconfiguration.properties.
> because of this TestMiniDruidKafkaCliDriver tries to run *all* qtests...which 
> time out...and produce 
> {code}
> TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed 
> out) (batchId=252)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19170) Fix TestMiniDruidKafkaCliDriver

2018-04-11 Thread Ashutosh Chauhan (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434215#comment-16434215
 ] 

Ashutosh Chauhan commented on HIVE-19170:
-

I dont see \{{druid.kafka.query.files}} anywhere being used. How does driver 
pick up this config?

> Fix TestMiniDruidKafkaCliDriver
> ---
>
> Key: HIVE-19170
> URL: https://issues.apache.org/jira/browse/HIVE-19170
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-19170.patch
>
>
> added in HIVE-18976 
> the property key {{druid.kafka.query.files}} doesn't exists in 
> testconfiguration.properties.
> because of this TestMiniDruidKafkaCliDriver tries to run *all* qtests...which 
> time out...and produce 
> {code}
> TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed 
> out) (batchId=252)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19147) Fix PerfCliDrivers: Tpcds30T missed CAT_NAME change

2018-04-11 Thread Ashutosh Chauhan (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434199#comment-16434199
 ] 

Ashutosh Chauhan commented on HIVE-19147:
-

+1

> Fix PerfCliDrivers: Tpcds30T missed CAT_NAME change
> ---
>
> Key: HIVE-19147
> URL: https://issues.apache.org/jira/browse/HIVE-19147
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19147.01.patch
>
>
> it seems the baked metastore dump misses the CAT_NAME field added by some 
> recent metastore change



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19172) NPE due to null EnvironmentContext in DDLTask

2018-04-11 Thread Ashutosh Chauhan (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434192#comment-16434192
 ] 

Ashutosh Chauhan commented on HIVE-19172:
-

+1 pending tests

> NPE due to null EnvironmentContext in DDLTask
> -
>
> Key: HIVE-19172
> URL: https://issues.apache.org/jira/browse/HIVE-19172
> Project: Hive
>  Issue Type: Task
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-19172.patch
>
>
> Stack Trace -
> {code}
> 2018-04-11T02:52:51,386 ERROR [5f2e24bf-ac93-4977-84fe-aa2c5f674ea4 main] 
> exec.DDLTask: java.lang.NullPointerException
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.alterTable(DDLTask.java:3539)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:392)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1414)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19163) TestAcidOnTez testGetSplitsLocks failure

2018-04-11 Thread Eugene Koifman (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugene Koifman updated HIVE-19163:
--
Component/s: Transactions
 Test

> TestAcidOnTez testGetSplitsLocks failure
> 
>
> Key: HIVE-19163
> URL: https://issues.apache.org/jira/browse/HIVE-19163
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test, Transactions
>Reporter: Vineet Garg
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Fix For: 3.0.0
>
>
> Error Stack:
> {code}
> java.lang.AssertionError
>   at 
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getValidWriteIds(DbTxnManager.java:820)
>   at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.applyMaterializedViewRewriting(CalcitePlanner.java:2073)
>   at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1701)
>   at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1593)
>   at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:118)
>   at 
> org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:1052)
>   at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:154)
>   at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:111)
>   at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1398)
>   at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:1414)
>   at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:434)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12056)
>   at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:314)
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:287)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:645)
>   at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1655)
>   at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1602)
>   at 
> org.apache.hadoop.hive.ql.udf.generic.GenericUDTFGetSplits.createPlanFragment(GenericUDTFGetSplits.java:252)
>   at 
> org.apache.hadoop.hive.ql.udf.generic.GenericUDTFGetSplits.process(GenericUDTFGetSplits.java:201)
>   at 
> org.apache.hadoop.hive.ql.exec.UDTFOperator.process(UDTFOperator.java:116)
>   at 
> org.apache.hadoop.hive.ql.exec.Operator.baseForward(Operator.java:984)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:930)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:917)
>   at 
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.Operator.baseForward(Operator.java:984)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:930)
>   at 
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:125)
>   at 
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:492)
>   at 
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:484)
>   at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:145)
>   at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:2470)
>   at 
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.getResults(ReExecDriver.java:215)
>   at 
> org.apache.hadoop.hive.ql.TestAcidOnTez.runStatementOnDriver(TestAcidOnTez.java:879)
>   at 
> org.apache.hadoop.hive.ql.TestAcidOnTez.testGetSplitsLocks(TestAcidOnTez.java:737)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
>   at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>   at

[jira] [Commented] (HIVE-18988) Support bootstrap replication of ACID tables

2018-04-11 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434176#comment-16434176
 ] 

Hive QA commented on HIVE-18988:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  3m  
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  3m  
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
19s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
54s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  3m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  3m  
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
47s{color} | {color:red} ql: The patch generated 10 new + 510 unchanged - 2 
fixed = 520 total (was 512) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
32s{color} | {color:red} standalone-metastore: The patch generated 1 new + 1540 
unchanged - 7 fixed = 1541 total (was 1547) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch has 23 line(s) that end in whitespace. Use 
git apply --whitespace=fix <>. Refer 
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  3m 
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
15s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 45s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10140/dev-support/hive-personality.sh
 |
| git revision | master / f8a671d |
| Default Java | 1.8.0_111 |
| mvninstall | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10140/yetus/patch-mvninstall-hcatalog_server-extensions.txt
 |
| mvninstall | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10140/yetus/patch-mvninstall-ql.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10140/yetus/diff-checkstyle-ql.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10140/yetus/diff-checkstyle-standalone-metastore.txt
 |
| whitespace | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10140/yetus/whitespace-eol.txt
 |
| modules | C: hcatalog/server-extensions itests/hive-unit ql 
standalone-metastore U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10140/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Support bootstrap replication of ACID tables
> 
>
> Key: HIVE-18988
> URL: https://issues.apache.org/jira/browse/HIVE-18988
> Project: Hive
>  Issue Type: Sub-task
>  Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: ACID, DR,

[jira] [Commented] (HIVE-18831) Differentiate errors that are thrown by Spark tasks

2018-04-11 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434101#comment-16434101
 ] 

Hive QA commented on HIVE-18831:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918317/HIVE-18831.91.patch

{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 28 failed/errored test(s), 13282 tests 
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=253)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=253)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Commented] (HIVE-19171) Persist runtime statistics in metastore

2018-04-11 Thread Alan Gates (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434088#comment-16434088
 ] 

Alan Gates commented on HIVE-19171:
---

What kind of statistics do you have in mind?

> Persist runtime statistics in metastore
> ---
>
> Key: HIVE-19171
> URL: https://issues.apache.org/jira/browse/HIVE-19171
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures

2018-04-11 Thread Alan Gates (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434085#comment-16434085
 ] 

Alan Gates commented on HIVE-19164:
---

Sure.

> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test
>Reporter: Vineet Garg
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.0
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws 
> an exception {{No matching version found}}. 
> To fetch schema version this method executes {code:sql} SELECT FROM 
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever 
> reason execution returns empty result set resulting in the exception. Both 
> test failures are due to this exception. I tried debugging the query 
> execution but didn't really go nowhere with it. I suspect this could be due 
> to recent metastore changes. I tried reproducing this outside test but with 
> no success.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19164) TestMetastoreVersion failures

2018-04-11 Thread Alan Gates (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates reassigned HIVE-19164:
-

Assignee: Alan Gates

> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test
>Reporter: Vineet Garg
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.0
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws 
> an exception {{No matching version found}}. 
> To fetch schema version this method executes {code:sql} SELECT FROM 
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever 
> reason execution returns empty result set resulting in the exception. Both 
> test failures are due to this exception. I tried debugging the query 
> execution but didn't really go nowhere with it. I suspect this could be due 
> to recent metastore changes. I tried reproducing this outside test but with 
> no success.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-17477) PidFilePatternConverter not rolling log according to their pid

2018-04-11 Thread Bohdan Chupika (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bohdan Chupika updated HIVE-17477:
--
Attachment: HIVE-17477.1.patch

> PidFilePatternConverter not rolling log according to their pid 
> ---
>
> Key: HIVE-17477
> URL: https://issues.apache.org/jira/browse/HIVE-17477
> Project: Hive
>  Issue Type: Bug
>  Components: Logging
>Reporter: Vlad Gudikov
>Assignee: Bohdan Chupika
>Priority: Major
> Attachments: HIVE-17477.1.patch
>
>
> We can use pid  in th filePattern:
> appender.DRFA.filePattern = 
> ${sys:hive.log.dir}/${sys:hive.log.file}.%d{-MM-dd}.%pid
> But when its time roll the logs at the end of the RollingAppender just 
> renames hive.log by applying the pattern described above and keeps logs that 
> are not related to the process this log was named by. The issue is that all 
> processes are writing to the same log and we cannot separate them by pid.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-17477) PidFilePatternConverter not rolling log according to their pid

2018-04-11 Thread Bohdan Chupika (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bohdan Chupika updated HIVE-17477:
--
Status: Patch Available  (was: Open)

> PidFilePatternConverter not rolling log according to their pid 
> ---
>
> Key: HIVE-17477
> URL: https://issues.apache.org/jira/browse/HIVE-17477
> Project: Hive
>  Issue Type: Bug
>  Components: Logging
>Reporter: Vlad Gudikov
>Assignee: Bohdan Chupika
>Priority: Major
> Attachments: HIVE-17477.1.patch
>
>
> We can use pid  in th filePattern:
> appender.DRFA.filePattern = 
> ${sys:hive.log.dir}/${sys:hive.log.file}.%d{-MM-dd}.%pid
> But when its time roll the logs at the end of the RollingAppender just 
> renames hive.log by applying the pattern described above and keeps logs that 
> are not related to the process this log was named by. The issue is that all 
> processes are writing to the same log and we cannot separate them by pid.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-17477) PidFilePatternConverter not rolling log according to their pid

2018-04-11 Thread Bohdan Chupika (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bohdan Chupika reassigned HIVE-17477:
-

Assignee: Bohdan Chupika

> PidFilePatternConverter not rolling log according to their pid 
> ---
>
> Key: HIVE-17477
> URL: https://issues.apache.org/jira/browse/HIVE-17477
> Project: Hive
>  Issue Type: Bug
>  Components: Logging
>Reporter: Vlad Gudikov
>Assignee: Bohdan Chupika
>Priority: Major
>
> We can use pid  in th filePattern:
> appender.DRFA.filePattern = 
> ${sys:hive.log.dir}/${sys:hive.log.file}.%d{-MM-dd}.%pid
> But when its time roll the logs at the end of the RollingAppender just 
> renames hive.log by applying the pattern described above and keeps logs that 
> are not related to the process this log was named by. The issue is that all 
> processes are writing to the same log and we cannot separate them by pid.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19173) Add Storage Handler runtime information as part of DESCRIBE EXTENDED

2018-04-11 Thread Nishant Bangarwa (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishant Bangarwa reassigned HIVE-19173:
---


> Add Storage Handler runtime information as part of DESCRIBE EXTENDED
> 
>
> Key: HIVE-19173
> URL: https://issues.apache.org/jira/browse/HIVE-19173
> Project: Hive
>  Issue Type: Task
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
>
> Follow up for https://issues.apache.org/jira/browse/HIVE-18976 
> Kafka Indexing Service in Druid has a runtime state associated with it. 
> Druid publishes this runtime state as KafkaSupervisorReport which has latest 
> offsets as reported by Kafka, the consumer lag per partition, as well as the 
> aggregate lag of all partitions.
> This information is quite useful to know whether a kafka-indexing-service 
> backed table has latest info or not. 
> This task is to add a this information as part of the output of DESCRIBE 
> EXTENDED statement



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-11181) Update people change for new PMC members

2018-04-11 Thread Matt McCline (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-11181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434027#comment-16434027
 ] 

Matt McCline commented on HIVE-11181:
-

[~csun] How do I safely add myself as a Hive PMC member to 
[https://hive.apache.org/people.html]  ?  What is the github URL for clone, 
etc. Thanks

> Update people change for new PMC members
> 
>
> Key: HIVE-11181
> URL: https://issues.apache.org/jira/browse/HIVE-11181
> Project: Hive
>  Issue Type: Task
>  Components: Website
>Reporter: Chao Sun
>Assignee: Chao Sun
>Priority: Major
> Attachments: HIVE-11181.patch
>
>
> As suggested in the title.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18831) Differentiate errors that are thrown by Spark tasks

2018-04-11 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434016#comment-16434016
 ] 

Hive QA commented on HIVE-18831:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m  
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
40s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m  
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
44s{color} | {color:red} ql: The patch generated 1 new + 56 unchanged - 4 fixed 
= 57 total (was 60) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
11s{color} | {color:red} spark-client: The patch generated 6 new + 25 unchanged 
- 1 fixed = 31 total (was 26) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
38s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
16s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 36s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10139/dev-support/hive-personality.sh
 |
| git revision | master / f8a671d |
| Default Java | 1.8.0_111 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10139/yetus/diff-checkstyle-ql.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10139/yetus/diff-checkstyle-spark-client.txt
 |
| modules | C: itests itests/util ql spark-client U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10139/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Differentiate errors that are thrown by Spark tasks
> ---
>
> Key: HIVE-18831
> URL: https://issues.apache.org/jira/browse/HIVE-18831
> Project: Hive
>  Issue Type: Sub-task
>  Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18831.1.patch, HIVE-18831.2.patch, 
> HIVE-18831.3.patch, HIVE-18831.4.patch, HIVE-18831.6.patch, 
> HIVE-18831.7.patch, HIVE-18831.8.WIP.patch, HIVE-18831.9.patch, 
> HIVE-18831.90.patch, HIVE-18831.91.patch
>
>
> We propagate exceptions from Spark task failures to the client well, but we 
> don't differentiate between errors from HS2 / RSC vs. errors thrown by 
> individual tasks.
> Main motivation is that when the client sees a propagated Spark exception its 
> difficult to know what part of the excution threw the exception.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19009) Retain and use runtime statistics thru out a session

2018-04-11 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433966#comment-16433966
 ] 

Hive QA commented on HIVE-19009:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918535/HIVE-19009.03.patch

{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 41 failed/errored test(s), 13262 tests 
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=253)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=253)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=93)

[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Issue Comment Deleted] (HIVE-19130) NPE is thrown when REPL LOAD applied drop partition event.

2018-04-11 Thread mahesh kumar behera (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mahesh kumar behera updated HIVE-19130:
---
Comment: was deleted

(was: The path looks fine to me ..

On Wed, Apr 11, 2018 at 11:10 AM, Thejas M Nair (JIRA) 

)

> NPE is thrown when REPL LOAD applied drop partition event.
> --
>
> Key: HIVE-19130
> URL: https://issues.apache.org/jira/browse/HIVE-19130
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: DR, Replication, pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19130.01.patch
>
>
> During incremental replication, if we split the events batch as follows, then 
> the REPL LOAD on second batch throws NPE.
> Batch-1: CREATE_TABLE(t1) -> ADD_PARTITION(t1.p1) -> DROP_PARTITION (t1.p1)
> Batch-2: DROP_TABLE(t1) ->  CREATE_TABLE(t1) -> ADD_PARTITION(t1.p1) -> 
> DROP_PARTITION (t1.p1)
> {code}
> 2018-04-05 16:20:36,531 ERROR [HiveServer2-Background-Pool: Thread-107044]: 
> metadata.Hive (Hive.java:getTable(1219)) - Table catalog_sales_new not found: 
> new5_tpcds_real_bin_partitioned_orc_1000.catalog_sales_new table not found
> 2018-04-05 16:20:36,538 ERROR [HiveServer2-Background-Pool: Thread-107044]: 
> exec.DDLTask (DDLTask.java:failed(540)) - 
> org.apache.hadoop.hive.ql.metadata.HiveException
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:4016)
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:3983)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:341)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:162)
> at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1765)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1506)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1303)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1165)
> at 
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197)
> at 
> org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76)
> at 
> org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
> at 
> org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:266)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NullPointerException
> at 
> org.apache.hadoop.hive.ql.metadata.Hive.getPartitionsByExpr(Hive.java:2613)
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:4008)
> ... 23 more
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19170) Fix TestMiniDruidKafkaCliDriver

2018-04-11 Thread Nishant Bangarwa (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishant Bangarwa updated HIVE-19170:

Status: Patch Available  (was: Open)

[~kgyrtkirk] there is only 1 test for this driver at present. Attached a patch 
to add that to testconfiguration.

> Fix TestMiniDruidKafkaCliDriver
> ---
>
> Key: HIVE-19170
> URL: https://issues.apache.org/jira/browse/HIVE-19170
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-19170.patch
>
>
> added in HIVE-18976 
> the property key {{druid.kafka.query.files}} doesn't exists in 
> testconfiguration.properties.
> because of this TestMiniDruidKafkaCliDriver tries to run *all* qtests...which 
> time out...and produce 
> {code}
> TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed 
> out) (batchId=252)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19170) Fix TestMiniDruidKafkaCliDriver

2018-04-11 Thread Nishant Bangarwa (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishant Bangarwa reassigned HIVE-19170:
---

Assignee: Nishant Bangarwa

> Fix TestMiniDruidKafkaCliDriver
> ---
>
> Key: HIVE-19170
> URL: https://issues.apache.org/jira/browse/HIVE-19170
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-19170.patch
>
>
> added in HIVE-18976 
> the property key {{druid.kafka.query.files}} doesn't exists in 
> testconfiguration.properties.
> because of this TestMiniDruidKafkaCliDriver tries to run *all* qtests...which 
> time out...and produce 
> {code}
> TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed 
> out) (batchId=252)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19170) Fix TestMiniDruidKafkaCliDriver

2018-04-11 Thread Nishant Bangarwa (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishant Bangarwa updated HIVE-19170:

Attachment: HIVE-19170.patch

> Fix TestMiniDruidKafkaCliDriver
> ---
>
> Key: HIVE-19170
> URL: https://issues.apache.org/jira/browse/HIVE-19170
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-19170.patch
>
>
> added in HIVE-18976 
> the property key {{druid.kafka.query.files}} doesn't exists in 
> testconfiguration.properties.
> because of this TestMiniDruidKafkaCliDriver tries to run *all* qtests...which 
> time out...and produce 
> {code}
> TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed 
> out) (batchId=252)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19130) NPE is thrown when REPL LOAD applied drop partition event.

2018-04-11 Thread mahesh kumar behera (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433922#comment-16433922
 ] 

mahesh kumar behera commented on HIVE-19130:


[~thejas]

looks fine to me 

> NPE is thrown when REPL LOAD applied drop partition event.
> --
>
> Key: HIVE-19130
> URL: https://issues.apache.org/jira/browse/HIVE-19130
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: DR, Replication, pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19130.01.patch
>
>
> During incremental replication, if we split the events batch as follows, then 
> the REPL LOAD on second batch throws NPE.
> Batch-1: CREATE_TABLE(t1) -> ADD_PARTITION(t1.p1) -> DROP_PARTITION (t1.p1)
> Batch-2: DROP_TABLE(t1) ->  CREATE_TABLE(t1) -> ADD_PARTITION(t1.p1) -> 
> DROP_PARTITION (t1.p1)
> {code}
> 2018-04-05 16:20:36,531 ERROR [HiveServer2-Background-Pool: Thread-107044]: 
> metadata.Hive (Hive.java:getTable(1219)) - Table catalog_sales_new not found: 
> new5_tpcds_real_bin_partitioned_orc_1000.catalog_sales_new table not found
> 2018-04-05 16:20:36,538 ERROR [HiveServer2-Background-Pool: Thread-107044]: 
> exec.DDLTask (DDLTask.java:failed(540)) - 
> org.apache.hadoop.hive.ql.metadata.HiveException
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:4016)
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:3983)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:341)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:162)
> at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1765)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1506)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1303)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1165)
> at 
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197)
> at 
> org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76)
> at 
> org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
> at 
> org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:266)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NullPointerException
> at 
> org.apache.hadoop.hive.ql.metadata.Hive.getPartitionsByExpr(Hive.java:2613)
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:4008)
> ... 23 more
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19130) NPE is thrown when REPL LOAD applied drop partition event.

2018-04-11 Thread mahesh kumar behera (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433915#comment-16433915
 ] 

mahesh kumar behera commented on HIVE-19130:


The path looks fine to me ..

On Wed, Apr 11, 2018 at 11:10 AM, Thejas M Nair (JIRA) 



> NPE is thrown when REPL LOAD applied drop partition event.
> --
>
> Key: HIVE-19130
> URL: https://issues.apache.org/jira/browse/HIVE-19130
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: DR, Replication, pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19130.01.patch
>
>
> During incremental replication, if we split the events batch as follows, then 
> the REPL LOAD on second batch throws NPE.
> Batch-1: CREATE_TABLE(t1) -> ADD_PARTITION(t1.p1) -> DROP_PARTITION (t1.p1)
> Batch-2: DROP_TABLE(t1) ->  CREATE_TABLE(t1) -> ADD_PARTITION(t1.p1) -> 
> DROP_PARTITION (t1.p1)
> {code}
> 2018-04-05 16:20:36,531 ERROR [HiveServer2-Background-Pool: Thread-107044]: 
> metadata.Hive (Hive.java:getTable(1219)) - Table catalog_sales_new not found: 
> new5_tpcds_real_bin_partitioned_orc_1000.catalog_sales_new table not found
> 2018-04-05 16:20:36,538 ERROR [HiveServer2-Background-Pool: Thread-107044]: 
> exec.DDLTask (DDLTask.java:failed(540)) - 
> org.apache.hadoop.hive.ql.metadata.HiveException
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:4016)
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:3983)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:341)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:162)
> at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1765)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1506)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1303)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1165)
> at 
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197)
> at 
> org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76)
> at 
> org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
> at 
> org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:266)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NullPointerException
> at 
> org.apache.hadoop.hive.ql.metadata.Hive.getPartitionsByExpr(Hive.java:2613)
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:4008)
> ... 23 more
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19172) NPE due to null EnvironmentContext in DDLTask

2018-04-11 Thread Nishant Bangarwa (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433913#comment-16433913
 ] 

Nishant Bangarwa commented on HIVE-19172:
-

+cc [~ashutoshc] please review.

> NPE due to null EnvironmentContext in DDLTask
> -
>
> Key: HIVE-19172
> URL: https://issues.apache.org/jira/browse/HIVE-19172
> Project: Hive
>  Issue Type: Task
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-19172.patch
>
>
> Stack Trace -
> {code}
> 2018-04-11T02:52:51,386 ERROR [5f2e24bf-ac93-4977-84fe-aa2c5f674ea4 main] 
> exec.DDLTask: java.lang.NullPointerException
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.alterTable(DDLTask.java:3539)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:392)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1414)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19172) NPE due to null EnvironmentContext in DDLTask

2018-04-11 Thread Nishant Bangarwa (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishant Bangarwa updated HIVE-19172:

Attachment: HIVE-19172.patch

> NPE due to null EnvironmentContext in DDLTask
> -
>
> Key: HIVE-19172
> URL: https://issues.apache.org/jira/browse/HIVE-19172
> Project: Hive
>  Issue Type: Task
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-19172.patch
>
>
> Stack Trace -
> {code}
> 2018-04-11T02:52:51,386 ERROR [5f2e24bf-ac93-4977-84fe-aa2c5f674ea4 main] 
> exec.DDLTask: java.lang.NullPointerException
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.alterTable(DDLTask.java:3539)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:392)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1414)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19172) NPE due to null EnvironmentContext in DDLTask

2018-04-11 Thread Nishant Bangarwa (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishant Bangarwa updated HIVE-19172:

Status: Patch Available  (was: Open)

> NPE due to null EnvironmentContext in DDLTask
> -
>
> Key: HIVE-19172
> URL: https://issues.apache.org/jira/browse/HIVE-19172
> Project: Hive
>  Issue Type: Task
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-19172.patch
>
>
> Stack Trace -
> {code}
> 2018-04-11T02:52:51,386 ERROR [5f2e24bf-ac93-4977-84fe-aa2c5f674ea4 main] 
> exec.DDLTask: java.lang.NullPointerException
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.alterTable(DDLTask.java:3539)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:392)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1414)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19172) NPE due to null EnvironmentContext in DDLTask

2018-04-11 Thread Nishant Bangarwa (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433910#comment-16433910
 ] 

Nishant Bangarwa commented on HIVE-19172:
-

Note - This was reported by [~abstractdog] 

> NPE due to null EnvironmentContext in DDLTask
> -
>
> Key: HIVE-19172
> URL: https://issues.apache.org/jira/browse/HIVE-19172
> Project: Hive
>  Issue Type: Task
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
>
> Stack Trace -
> {code}
> 2018-04-11T02:52:51,386 ERROR [5f2e24bf-ac93-4977-84fe-aa2c5f674ea4 main] 
> exec.DDLTask: java.lang.NullPointerException
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.alterTable(DDLTask.java:3539)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:392)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1414)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19172) NPE due to null EnvironmentContext in DDLTask

2018-04-11 Thread Nishant Bangarwa (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishant Bangarwa reassigned HIVE-19172:
---


> NPE due to null EnvironmentContext in DDLTask
> -
>
> Key: HIVE-19172
> URL: https://issues.apache.org/jira/browse/HIVE-19172
> Project: Hive
>  Issue Type: Task
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
>
> Stack Trace -
> {code}
> 2018-04-11T02:52:51,386 ERROR [5f2e24bf-ac93-4977-84fe-aa2c5f674ea4 main] 
> exec.DDLTask: java.lang.NullPointerException
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.alterTable(DDLTask.java:3539)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:392)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1414)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19009) Retain and use runtime statistics thru out a session

2018-04-11 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433879#comment-16433879
 ] 

Hive QA commented on HIVE-19009:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
 1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
34s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
54s{color} | {color:red} ql: The patch generated 9 new + 1226 unchanged - 2 
fixed = 1235 total (was 1228) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
17s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 13s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10138/dev-support/hive-personality.sh
 |
| git revision | master / f8a671d |
| Default Java | 1.8.0_111 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10138/yetus/diff-checkstyle-ql.txt
 |
| modules | C: common itests ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10138/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Retain and use runtime statistics thru out a session
> 
>
> Key: HIVE-19009
> URL: https://issues.apache.org/jira/browse/HIVE-19009
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19009.01.patch, HIVE-19009.02.patch, 
> HIVE-19009.03.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19147) Fix PerfCliDrivers: Tpcds30T missed CAT_NAME change

2018-04-11 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433835#comment-16433835
 ] 

Hive QA commented on HIVE-19147:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918351/HIVE-19147.01.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 27 failed/errored test(s), 13266 tests 
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=253)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=253)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=93)

[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=95)

[jira] [Commented] (HIVE-19147) Fix PerfCliDrivers: Tpcds30T missed CAT_NAME change

2018-04-11 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433784#comment-16433784
 ] 

Hive QA commented on HIVE-19147:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  6m 
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  3m 
 4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  8m  
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  6m 
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  6m 
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  2m  
3s{color} | {color:red} root: The patch generated 16 new + 715 unchanged - 24 
fixed = 731 total (was 739) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
15s{color} | {color:red} itests/util: The patch generated 16 new + 73 unchanged 
- 23 fixed = 89 total (was 96) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
48s{color} | {color:green} ql: The patch generated 0 new + 642 unchanged - 1 
fixed = 642 total (was 643) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  8m 
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 54m 57s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  xml  javac  javadoc  findbugs  checkstyle  
compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10137/dev-support/hive-personality.sh
 |
| git revision | master / f8a671d |
| Default Java | 1.8.0_111 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10137/yetus/diff-checkstyle-root.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10137/yetus/diff-checkstyle-itests_util.txt
 |
| modules | C: . itests/util ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10137/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Fix PerfCliDrivers: Tpcds30T missed CAT_NAME change
> ---
>
> Key: HIVE-19147
> URL: https://issues.apache.org/jira/browse/HIVE-19147
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19147.01.patch
>
>
> it seems the baked metastore dump misses the CAT_NAME field added by some 
> recent metastore change



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19084) Test case in Hive Query Language fails with a java.lang.AssertionError.

2018-04-11 Thread Alisha Prabhu (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433781#comment-16433781
 ] 

Alisha Prabhu commented on HIVE-19084:
--

Hi [~steveyeom2017], thanks for the update. I am looking forward to a possible 
solution for the above issue.

> Test case in Hive Query Language fails with a java.lang.AssertionError.
> ---
>
> Key: HIVE-19084
> URL: https://issues.apache.org/jira/browse/HIVE-19084
> Project: Hive
>  Issue Type: Bug
>  Components: Test, Transactions
> Environment: uname -a
> Linux pts00607-vm3 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:46 UTC 
> 2018 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Assignee: Steve Yeom
>Priority: Major
> Attachments: HIVE-19084.1.patch
>
>
> The test case testInsertOverwriteForPartitionedMmTable in 
> TestTxnCommandsForMmTable.java and TestTxnCommandsForOrcMmTable.java fails 
> with a java.lang.AssertionError.
> Maven command used is mvn 
> -Dtest=TestTxnCommandsForMmTable#testInsertOverwriteForPartitionedMmTable test
> The test case fails as the listStatus function of the FileSystem does not 
> guarantee to return the List of files/directories status in a sorted order.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18831) Differentiate errors that are thrown by Spark tasks

2018-04-11 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433722#comment-16433722
 ] 

Hive QA commented on HIVE-18831:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918317/HIVE-18831.91.patch

{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 80 failed/errored test(s), 13679 tests 
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=253)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=253)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=96)

[jira] [Assigned] (HIVE-19171) Persist runtime statistics in metastore

2018-04-11 Thread Zoltan Haindrich (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich reassigned HIVE-19171:
---


> Persist runtime statistics in metastore
> ---
>
> Key: HIVE-19171
> URL: https://issues.apache.org/jira/browse/HIVE-19171
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19009) Retain and use runtime statistics thru out a session

2018-04-11 Thread Zoltan Haindrich (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich updated HIVE-19009:

Attachment: HIVE-19009.03.patch

> Retain and use runtime statistics thru out a session
> 
>
> Key: HIVE-19009
> URL: https://issues.apache.org/jira/browse/HIVE-19009
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19009.01.patch, HIVE-19009.02.patch, 
> HIVE-19009.03.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18831) Differentiate errors that are thrown by Spark tasks

2018-04-11 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433624#comment-16433624
 ] 

Hive QA commented on HIVE-18831:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m  
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m  
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
43s{color} | {color:red} ql: The patch generated 1 new + 56 unchanged - 4 fixed 
= 57 total (was 60) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
11s{color} | {color:red} spark-client: The patch generated 6 new + 25 unchanged 
- 1 fixed = 31 total (was 26) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
15s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 23s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10135/dev-support/hive-personality.sh
 |
| git revision | master / f8a671d |
| Default Java | 1.8.0_111 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10135/yetus/diff-checkstyle-ql.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10135/yetus/diff-checkstyle-spark-client.txt
 |
| modules | C: itests itests/util ql spark-client U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10135/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Differentiate errors that are thrown by Spark tasks
> ---
>
> Key: HIVE-18831
> URL: https://issues.apache.org/jira/browse/HIVE-18831
> Project: Hive
>  Issue Type: Sub-task
>  Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18831.1.patch, HIVE-18831.2.patch, 
> HIVE-18831.3.patch, HIVE-18831.4.patch, HIVE-18831.6.patch, 
> HIVE-18831.7.patch, HIVE-18831.8.WIP.patch, HIVE-18831.9.patch, 
> HIVE-18831.90.patch, HIVE-18831.91.patch
>
>
> We propagate exceptions from Spark task failures to the client well, but we 
> don't differentiate between errors from HS2 / RSC vs. errors thrown by 
> individual tasks.
> Main motivation is that when the client sees a propagated Spark exception its 
> difficult to know what part of the excution threw the exception.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19170) Fix TestMiniDruidKafkaCliDriver

2018-04-11 Thread Zoltan Haindrich (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433618#comment-16433618
 ] 

Zoltan Haindrich commented on HIVE-19170:
-

[~nishantbangarwa]: I'm not sure which tests supposed to run with this driver...

> Fix TestMiniDruidKafkaCliDriver
> ---
>
> Key: HIVE-19170
> URL: https://issues.apache.org/jira/browse/HIVE-19170
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Priority: Major
>
> added in HIVE-18976 
> the property key {{druid.kafka.query.files}} doesn't exists in 
> testconfiguration.properties.
> because of this TestMiniDruidKafkaCliDriver tries to run *all* qtests...which 
> time out...and produce 
> {code}
> TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed 
> out) (batchId=252)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-17824) msck repair table should drop the missing partitions from metastore

2018-04-11 Thread Peter Vary (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-17824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433595#comment-16433595
 ] 

Peter Vary commented on HIVE-17824:
---

This one looks good to me +1 (pending tests)
On one condition :), on a follow up jira we get rid of the expression based 
solution.
The metastore thrift interface provides the possibility to drop partitions 
based on names, so we have to add this possibility to the IMetaStoreClient 
interface too, and use this here. And probably at 
https://github.com/apache/hive/blob/master/hcatalog/webhcat/java-client/src/main/java/org/apache/hive/hcatalog/api/HCatClientHMSImpl.java#L514
 too.



> msck repair table should drop the missing partitions from metastore
> ---
>
> Key: HIVE-17824
> URL: https://issues.apache.org/jira/browse/HIVE-17824
> Project: Hive
>  Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-17824.1.patch, HIVE-17824.2.patch, 
> HIVE-17824.3.patch
>
>
> {{msck repair table }} is often used in environments where the new 
> partitions are loaded as directories on HDFS or S3 and users want to create 
> the missing partitions in bulk. However, currently it only supports addition 
> of missing partitions. If there are any partitions which are present in 
> metastore but not on the FileSystem, it should also delete them so that it 
> truly repairs the table metadata.
> We should be careful not to break backwards compatibility so we should either 
> introduce a new config or keyword to add support to delete unnecessary 
> partitions from the metastore. This way users who want the old behavior can 
> easily turn it off. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18831) Differentiate errors that are thrown by Spark tasks

2018-04-11 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433591#comment-16433591
 ] 

Hive QA commented on HIVE-18831:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918317/HIVE-18831.91.patch

{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 85 failed/errored test(s), 13679 tests 
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=253)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=253)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=96)

[jira] [Resolved] (HIVE-19077) Handle duplicate ptests requests standing in queue at the same time

2018-04-11 Thread Adam Szita (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Szita resolved HIVE-19077.
---
Resolution: Fixed

> Handle duplicate ptests requests standing in queue at the same time
> ---
>
> Key: HIVE-19077
> URL: https://issues.apache.org/jira/browse/HIVE-19077
> Project: Hive
>  Issue Type: Improvement
>  Components: Testing Infrastructure
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19077.0.patch, HIVE-19077.1.patch, 
> HIVE-19077.sslFix.patch
>
>
> I've been keeping on eye on our {{PreCommit-HIVE-Build}} job, and what I 
> noticed that sometimes huge queues can build up, that contain jira's more 
> than once. (Yesterday I've seen a queue of 40, having 31 distinct jiras..)
> Simple scenario is that I upload a patch, it gets queued for ptest (already 
> long queue), and 3 hours later I will update it, re-upload and re-queue. Now 
> the current ptest infra seems to be smart enough to always deal with the 
> latest patch, so what will happen is that the same patch will be tested 2 
> times (with ~3 hours) diff, most probably with same result.
> I propose we do some deduplication - if ptest starts running the request for 
> Jira X, then it can take a look on the current queue, and see if X is there 
> again. If so, it can skip for now, it will be picked up later anyway.
> In practice this means that if you reconsider your patch and update it, your 
> original place in the queue will be gone (like as a penalty for changing it), 
> but overall it saves resources for the whole community.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-18906) Lower Logging for "Using direct SQL"

2018-04-11 Thread Antal Sinkovits (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antal Sinkovits reassigned HIVE-18906:
--

Assignee: Antal Sinkovits

> Lower Logging for "Using direct SQL"
> 
>
> Key: HIVE-18906
> URL: https://issues.apache.org/jira/browse/HIVE-18906
> Project: Hive
>  Issue Type: Improvement
>Reporter: BELUGA BEHR
>Assignee: Antal Sinkovits
>Priority: Trivial
>  Labels: noob
>
> https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L181
> {code}
> 2018-02-26 14:32:50,620  INFO  
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-199]: 
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:33:09,566  INFO  
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-172]: 
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:33:43,170  INFO  
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-172]: 
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:33:46,301  INFO  
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-199]: 
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:34:16,966  INFO  
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-172]: 
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:34:18,072  INFO  
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-196]: 
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:34:25,064  INFO  
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-199]: 
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:34:51,781  INFO  
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-172]: 
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:35:04,685  INFO  
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-199]: 
> Using direct SQL, underlying DB is MYSQL
> 2018-02-26 14:35:26,689  INFO  
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql: [pool-4-thread-172]: 
> Using direct SQL, underlying DB is MYSQL
> {code}
> I wouldn't mind knowing this information when the service starts up and 
> detects that there is a Direct SQL compatible database, but I don't need to 
> know about this every time.
> Please lower to _debug_ level logging and parameterize with SLF4J.  Direct 
> SQL is a performance thing and not something I care about during the normal 
> course of operation (unless there is an error which would presumably be 
> covered by an _error_ logging message).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19077) Handle duplicate ptests requests standing in queue at the same time

2018-04-11 Thread Adam Szita (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Szita updated HIVE-19077:
--
Attachment: HIVE-19077.sslFix.patch

> Handle duplicate ptests requests standing in queue at the same time
> ---
>
> Key: HIVE-19077
> URL: https://issues.apache.org/jira/browse/HIVE-19077
> Project: Hive
>  Issue Type: Improvement
>  Components: Testing Infrastructure
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19077.0.patch, HIVE-19077.1.patch, 
> HIVE-19077.sslFix.patch
>
>
> I've been keeping on eye on our {{PreCommit-HIVE-Build}} job, and what I 
> noticed that sometimes huge queues can build up, that contain jira's more 
> than once. (Yesterday I've seen a queue of 40, having 31 distinct jiras..)
> Simple scenario is that I upload a patch, it gets queued for ptest (already 
> long queue), and 3 hours later I will update it, re-upload and re-queue. Now 
> the current ptest infra seems to be smart enough to always deal with the 
> latest patch, so what will happen is that the same patch will be tested 2 
> times (with ~3 hours) diff, most probably with same result.
> I propose we do some deduplication - if ptest starts running the request for 
> Jira X, then it can take a look on the current queue, and see if X is there 
> again. If so, it can skip for now, it will be picked up later anyway.
> In practice this means that if you reconsider your patch and update it, your 
> original place in the queue will be gone (like as a penalty for changing it), 
> but overall it saves resources for the whole community.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Reopened] (HIVE-19077) Handle duplicate ptests requests standing in queue at the same time

2018-04-11 Thread Adam Szita (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Szita reopened HIVE-19077:
---

> Handle duplicate ptests requests standing in queue at the same time
> ---
>
> Key: HIVE-19077
> URL: https://issues.apache.org/jira/browse/HIVE-19077
> Project: Hive
>  Issue Type: Improvement
>  Components: Testing Infrastructure
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19077.0.patch, HIVE-19077.1.patch, 
> HIVE-19077.sslFix.patch
>
>
> I've been keeping on eye on our {{PreCommit-HIVE-Build}} job, and what I 
> noticed that sometimes huge queues can build up, that contain jira's more 
> than once. (Yesterday I've seen a queue of 40, having 31 distinct jiras..)
> Simple scenario is that I upload a patch, it gets queued for ptest (already 
> long queue), and 3 hours later I will update it, re-upload and re-queue. Now 
> the current ptest infra seems to be smart enough to always deal with the 
> latest patch, so what will happen is that the same patch will be tested 2 
> times (with ~3 hours) diff, most probably with same result.
> I propose we do some deduplication - if ptest starts running the request for 
> Jira X, then it can take a look on the current queue, and see if X is there 
> again. If so, it can skip for now, it will be picked up later anyway.
> In practice this means that if you reconsider your patch and update it, your 
> original place in the queue will be gone (like as a penalty for changing it), 
> but overall it saves resources for the whole community.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19159) TestMTQueries.testMTQueries1 failure

2018-04-11 Thread Laszlo Bodor (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433568#comment-16433568
 ] 

Laszlo Bodor commented on HIVE-19159:
-

[~vgarg] : I'll look into it.

> TestMTQueries.testMTQueries1 failure
> 
>
> Key: HIVE-19159
> URL: https://issues.apache.org/jira/browse/HIVE-19159
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test
>Reporter: Vineet Garg
>Assignee: Laszlo Bodor
>Priority: Critical
> Fix For: 3.0.0
>
>
> I have confirmed that HIVE-18051 caused this failure



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19159) TestMTQueries.testMTQueries1 failure

2018-04-11 Thread Laszlo Bodor (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laszlo Bodor reassigned HIVE-19159:
---

Assignee: Laszlo Bodor

> TestMTQueries.testMTQueries1 failure
> 
>
> Key: HIVE-19159
> URL: https://issues.apache.org/jira/browse/HIVE-19159
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test
>Reporter: Vineet Garg
>Assignee: Laszlo Bodor
>Priority: Critical
> Fix For: 3.0.0
>
>
> I have confirmed that HIVE-18051 caused this failure



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HIVE-19148) derby/MV_CREATION_METADATA misses CAT_NAME column

2018-04-11 Thread Zoltan Haindrich (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich resolved HIVE-19148.
-
Resolution: Not A Problem

thank you [~alangates]! then I'll just close this :)

> derby/MV_CREATION_METADATA misses CAT_NAME column
> -
>
> Key: HIVE-19148
> URL: https://issues.apache.org/jira/browse/HIVE-19148
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19148.01.patch
>
>
> It seems like the upgrade patches for derby misses CAT_NAME for the table 
> MV_CREATION_METADATA



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18831) Differentiate errors that are thrown by Spark tasks

2018-04-11 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433535#comment-16433535
 ] 

Hive QA commented on HIVE-18831:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m  
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m  
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m  
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
45s{color} | {color:red} ql: The patch generated 1 new + 56 unchanged - 4 fixed 
= 57 total (was 60) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
11s{color} | {color:red} spark-client: The patch generated 6 new + 25 unchanged 
- 1 fixed = 31 total (was 26) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
15s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 28s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10134/dev-support/hive-personality.sh
 |
| git revision | master / f6dbcac |
| Default Java | 1.8.0_111 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10134/yetus/diff-checkstyle-ql.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10134/yetus/diff-checkstyle-spark-client.txt
 |
| modules | C: itests itests/util ql spark-client U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10134/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Differentiate errors that are thrown by Spark tasks
> ---
>
> Key: HIVE-18831
> URL: https://issues.apache.org/jira/browse/HIVE-18831
> Project: Hive
>  Issue Type: Sub-task
>  Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18831.1.patch, HIVE-18831.2.patch, 
> HIVE-18831.3.patch, HIVE-18831.4.patch, HIVE-18831.6.patch, 
> HIVE-18831.7.patch, HIVE-18831.8.WIP.patch, HIVE-18831.9.patch, 
> HIVE-18831.90.patch, HIVE-18831.91.patch
>
>
> We propagate exceptions from Spark task failures to the client well, but we 
> don't differentiate between errors from HS2 / RSC vs. errors thrown by 
> individual tasks.
> Main motivation is that when the client sees a propagated Spark exception its 
> difficult to know what part of the excution threw the exception.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19077) Handle duplicate ptests requests standing in queue at the same time

2018-04-11 Thread Peter Vary (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Vary updated HIVE-19077:
--
   Resolution: Fixed
Fix Version/s: 3.1.0
   Status: Resolved  (was: Patch Available)

Pushed to master.
Thanks for the patch [~szita]!

> Handle duplicate ptests requests standing in queue at the same time
> ---
>
> Key: HIVE-19077
> URL: https://issues.apache.org/jira/browse/HIVE-19077
> Project: Hive
>  Issue Type: Improvement
>  Components: Testing Infrastructure
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19077.0.patch, HIVE-19077.1.patch
>
>
> I've been keeping on eye on our {{PreCommit-HIVE-Build}} job, and what I 
> noticed that sometimes huge queues can build up, that contain jira's more 
> than once. (Yesterday I've seen a queue of 40, having 31 distinct jiras..)
> Simple scenario is that I upload a patch, it gets queued for ptest (already 
> long queue), and 3 hours later I will update it, re-upload and re-queue. Now 
> the current ptest infra seems to be smart enough to always deal with the 
> latest patch, so what will happen is that the same patch will be tested 2 
> times (with ~3 hours) diff, most probably with same result.
> I propose we do some deduplication - if ptest starts running the request for 
> Jira X, then it can take a look on the current queue, and see if X is there 
> again. If so, it can skip for now, it will be picked up later anyway.
> In practice this means that if you reconsider your patch and update it, your 
> original place in the queue will be gone (like as a penalty for changing it), 
> but overall it saves resources for the whole community.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19048) Initscript errors are ignored

2018-04-11 Thread Zoltan Haindrich (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433521#comment-16433521
 ] 

Zoltan Haindrich commented on HIVE-19048:
-

cool! I think people would expect this behaviour by default :)
+1 pending tests
fyi: [~pvary]

> Initscript errors are ignored
> -
>
> Key: HIVE-19048
> URL: https://issues.apache.org/jira/browse/HIVE-19048
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Reporter: Zoltan Haindrich
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
> Attachments: HIVE-19048.1.patch
>
>
> I've been running some queries for a while when I've noticed that my 
> initscript has an error; and beeline stops interpreting the initscript after 
> encountering the first error.
> {code}
> echo 'invalid;' > init.sql
> echo 'select 1;' > s1.sql
> beeline -u jdbc:hive2://localhost:1/ -n hive -i init.sql -f s1.sql 
> [...]
> Running init script init.sql
> 0: jdbc:hive2://localhost:1/> invalid;
> Error: Error while compiling statement: FAILED: ParseException line 1:0 
> cannot recognize input near 'invalid' '' '' (state=42000,code=4)
> 0: jdbc:hive2://localhost:1/> select 1;
> [...]
> $ echo $?
> 0
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18910) Migrate to Murmur hash for shuffle and bucketing

2018-04-11 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433515#comment-16433515
 ] 

Hive QA commented on HIVE-18910:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918482/HIVE-18910.28.patch

{color:green}SUCCESS:{color} +1 due to 17 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 394 failed/errored test(s), 13556 tests 
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=252)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed 
out) (batchId=246)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=252)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed 
out) (batchId=246)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=252)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=252)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=94)

[jira] [Updated] (HIVE-19167) Map data type doesn't keep the order of the key/values pairs as read (Part 2, The Sequel or SQL)

2018-04-11 Thread Matt McCline (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-19167:

Status: Patch Available  (was: Open)

> Map data type doesn't keep the order of the key/values pairs as read (Part 2, 
> The Sequel or SQL)   
> ---
>
> Key: HIVE-19167
> URL: https://issues.apache.org/jira/browse/HIVE-19167
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.1.0
>
> Attachments: HIVE-19167.01.patch
>
>
> HIVE-19116: "Vectorization: Vector Map data type doesn't keep the order of 
> the key/values pairs as read" didn't fix all the places where HashMap is used 
> instead of LinkedHashMap.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19167) Map data type doesn't keep the order of the key/values pairs as read (Part 2, The Sequel or SQL)

2018-04-11 Thread Matt McCline (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-19167:

Attachment: HIVE-19167.01.patch

> Map data type doesn't keep the order of the key/values pairs as read (Part 2, 
> The Sequel or SQL)   
> ---
>
> Key: HIVE-19167
> URL: https://issues.apache.org/jira/browse/HIVE-19167
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.1.0
>
> Attachments: HIVE-19167.01.patch
>
>
> HIVE-19116: "Vectorization: Vector Map data type doesn't keep the order of 
> the key/values pairs as read" didn't fix all the places where HashMap is used 
> instead of LinkedHashMap.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19169) llap: Timed out after 90 secs

2018-04-11 Thread Zoltan Haindrich (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433507#comment-16433507
 ] 

Zoltan Haindrich commented on HIVE-19169:
-

to provide better context: I'm working on reoptimizing the queries - so these 
queries are known to have a bad initial plan (usually map join is used - but 
its not gonna work; and will eventually get an oom) the problem is that it 
fails with timeout - in which case there won't be a retry...

> llap: Timed out after 90 secs
> -
>
> Key: HIVE-19169
> URL: https://issues.apache.org/jira/browse/HIVE-19169
> Project: Hive
>  Issue Type: Bug
>  Components: llap
>Reporter: Zoltan Haindrich
>Priority: Major
>
> a more or less recent hive/master with tpcds1000; while running only 1 query 
> at a time on the cluster; sometimes a "timeout" happens - this could be even 
> a misconfiguration problem...I'm not sure I've set it up correctly.
> what I see from: it seems like the attempt have entered the queue and 
> accepted - after almost 90 seconds there are messages that it will be 
> pre-empted - and the 90 sec timeout happens.
> hive.log; example: attempt_1522319554594_0065_19_05_000119_14 failed
> {code}
> 2018-04-10T13:20:25,178 ERROR [HiveServer2-Background-Pool: Thread-4194]: 
> SessionState (SessionState.java:printError(1214)) - Vertex failed, 
> vertexName=Reducer 3, vertexId=vertex_1522319554594_0065_19_05, 
> diagnostics=[Task failed, taskId=task_1522319554594_0065_19_05_000119, 
> diagnostics=[TaskAttempt 0 killed, TaskAttempt 1 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_1 Timed out after 90 
> secs], TaskAttempt 2 killed, TaskAttempt 3 killed, TaskAttempt 4 killed, 
> TaskAttempt 5 killed, TaskAttempt 6 killed, TaskAttempt 7 killed, TaskAttempt 
> 8 killed, TaskAttempt 9 killed, TaskAttempt 10 killed, TaskAttempt 11 killed, 
> TaskAttempt 12 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_12 Timed out after 90 
> secs], TaskAttempt 13 killed, TaskAttempt 14 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_14 Timed out after 90 
> secs], TaskAttempt 15 failed, 
> info=[org.apache.hadoop.ipc.RemoteException(java.lang.RuntimeException): 
> attempt_1522319554594_0065_19_08_18_6 was not registered and couldn't be 
> removed
> 2018-04-10T13:20:25,260 ERROR [HiveServer2-Background-Pool: Thread-4194]: 
> ql.Driver (SessionState.java:printError(1214)) - FAILED: Execution Error, 
> return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, 
> vertexName=Reducer 3, vertexId=vertex_1522319554594_0065_19_05, 
> diagnostics=[Task failed, taskId=task_1522319554594_0065_19_05_000119, 
> diagnostics=[TaskAttempt 0 killed, TaskAttempt 1 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_1 Timed out after 90 
> secs], TaskAttempt 2 killed, TaskAttempt 3 killed, TaskAttempt 4 killed, 
> TaskAttempt 5 killed, TaskAttempt 6 killed, TaskAttempt 7 killed, TaskAttempt 
> 8 killed, TaskAttempt 9 killed, TaskAttempt 10 killed, TaskAttempt 11 killed, 
> TaskAttempt 12 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_12 Timed out after 90 
> secs], TaskAttempt 13 killed, TaskAttempt 14 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_14 Timed out after 90 
> secs], TaskAttempt 15 failed, 
> info=[org.apache.hadoop.ipc.RemoteException(java.lang.RuntimeException): 
> attempt_1522319554594_0065_19_08_18_6 was not registered and couldn't be 
> removed
> org.apache.hive.service.cli.HiveSQLException: Error while processing 
> statement: FAILED: Execution Error, return code 2 from 
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Reducer 
> 3, vertexId=vertex_1522319554594_0065_19_05, diagnostics=[Task failed, 
> taskId=task_1522319554594_0065_19_05_000119, diagnostics=[TaskAttempt 0 
> killed, TaskAttempt 1 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_1 Timed out after 90 
> secs], TaskAttempt 2 killed, TaskAttempt 3 killed, TaskAttempt 4 killed, 
> TaskAttempt 5 killed, TaskAttempt 6 killed, TaskAttempt 7 killed, TaskAttempt 
> 8 killed, TaskAttempt 9 killed, TaskAttempt 10 killed, TaskAttempt 11 killed, 
> TaskAttempt 12 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_12 Timed out after 90 
> secs], TaskAttempt 13 killed, TaskAttempt 14 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_14 Timed out after 90 
> secs], TaskAttempt 15 failed, 
> info=[org.apache.hadoop.ipc.RemoteException(java.lang.RuntimeException): 
> attempt_1522319554594_0065_19_08_18_6 was not registered and couldn't be 
> removed
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Vertex failed, 
> vertexName=Reducer 3, vertexId=vertex_1522319554594_0065_19_05, 
> diagnostics=[Task

[jira] [Updated] (HIVE-19169) llap: Timed out after 90 secs

2018-04-11 Thread Zoltan Haindrich (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich updated HIVE-19169:

Description: 
a more or less recent hive/master with tpcds1000; while running only 1 query at 
a time on the cluster; sometimes a "timeout" happens - this could be even a 
misconfiguration problem...I'm not sure I've set it up correctly.

what I see from: it seems like the attempt have entered the queue and accepted 
- after almost 90 seconds there are messages that it will be pre-empted - and 
the 90 sec timeout happens.

hive.log; example: attempt_1522319554594_0065_19_05_000119_14 failed
{code}
2018-04-10T13:20:25,178 ERROR [HiveServer2-Background-Pool: Thread-4194]: 
SessionState (SessionState.java:printError(1214)) - Vertex failed, 
vertexName=Reducer 3, vertexId=vertex_1522319554594_0065_19_05, 
diagnostics=[Task failed, taskId=task_1522319554594_0065_19_05_000119, 
diagnostics=[TaskAttempt 0 killed, TaskAttempt 1 failed, 
info=[AttemptID:attempt_1522319554594_0065_19_05_000119_1 Timed out after 90 
secs], TaskAttempt 2 killed, TaskAttempt 3 killed, TaskAttempt 4 killed, 
TaskAttempt 5 killed, TaskAttempt 6 killed, TaskAttempt 7 killed, TaskAttempt 8 
killed, TaskAttempt 9 killed, TaskAttempt 10 killed, TaskAttempt 11 killed, 
TaskAttempt 12 failed, 
info=[AttemptID:attempt_1522319554594_0065_19_05_000119_12 Timed out after 90 
secs], TaskAttempt 13 killed, TaskAttempt 14 failed, 
info=[AttemptID:attempt_1522319554594_0065_19_05_000119_14 Timed out after 90 
secs], TaskAttempt 15 failed, 
info=[org.apache.hadoop.ipc.RemoteException(java.lang.RuntimeException): 
attempt_1522319554594_0065_19_08_18_6 was not registered and couldn't be 
removed
2018-04-10T13:20:25,260 ERROR [HiveServer2-Background-Pool: Thread-4194]: 
ql.Driver (SessionState.java:printError(1214)) - FAILED: Execution Error, 
return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, 
vertexName=Reducer 3, vertexId=vertex_1522319554594_0065_19_05, 
diagnostics=[Task failed, taskId=task_1522319554594_0065_19_05_000119, 
diagnostics=[TaskAttempt 0 killed, TaskAttempt 1 failed, 
info=[AttemptID:attempt_1522319554594_0065_19_05_000119_1 Timed out after 90 
secs], TaskAttempt 2 killed, TaskAttempt 3 killed, TaskAttempt 4 killed, 
TaskAttempt 5 killed, TaskAttempt 6 killed, TaskAttempt 7 killed, TaskAttempt 8 
killed, TaskAttempt 9 killed, TaskAttempt 10 killed, TaskAttempt 11 killed, 
TaskAttempt 12 failed, 
info=[AttemptID:attempt_1522319554594_0065_19_05_000119_12 Timed out after 90 
secs], TaskAttempt 13 killed, TaskAttempt 14 failed, 
info=[AttemptID:attempt_1522319554594_0065_19_05_000119_14 Timed out after 90 
secs], TaskAttempt 15 failed, 
info=[org.apache.hadoop.ipc.RemoteException(java.lang.RuntimeException): 
attempt_1522319554594_0065_19_08_18_6 was not registered and couldn't be 
removed
org.apache.hive.service.cli.HiveSQLException: Error while processing statement: 
FAILED: Execution Error, return code 2 from 
org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Reducer 
3, vertexId=vertex_1522319554594_0065_19_05, diagnostics=[Task failed, 
taskId=task_1522319554594_0065_19_05_000119, diagnostics=[TaskAttempt 0 killed, 
TaskAttempt 1 failed, info=[AttemptID:attempt_1522319554594_0065_19_05_000119_1 
Timed out after 90 secs], TaskAttempt 2 killed, TaskAttempt 3 killed, 
TaskAttempt 4 killed, TaskAttempt 5 killed, TaskAttempt 6 killed, TaskAttempt 7 
killed, TaskAttempt 8 killed, TaskAttempt 9 killed, TaskAttempt 10 killed, 
TaskAttempt 11 killed, TaskAttempt 12 failed, 
info=[AttemptID:attempt_1522319554594_0065_19_05_000119_12 Timed out after 90 
secs], TaskAttempt 13 killed, TaskAttempt 14 failed, 
info=[AttemptID:attempt_1522319554594_0065_19_05_000119_14 Timed out after 90 
secs], TaskAttempt 15 failed, 
info=[org.apache.hadoop.ipc.RemoteException(java.lang.RuntimeException): 
attempt_1522319554594_0065_19_08_18_6 was not registered and couldn't be 
removed
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Vertex failed, 
vertexName=Reducer 3, vertexId=vertex_1522319554594_0065_19_05, 
diagnostics=[Task failed, taskId=task_1522319554594_0065_19_05_000119, 
diagnostics=[TaskAttempt 0 killed, TaskAttempt 1 failed, 
info=[AttemptID:attempt_1522319554594_0065_19_05_000119_1 Timed out after 90 
secs], TaskAttempt 2 killed, TaskAttempt 3 killed, TaskAttempt 4 killed, 
TaskAttempt 5 killed, TaskAttempt 6 killed, TaskAttempt 7 killed, TaskAttempt 8 
killed, TaskAttempt 9 killed, TaskAttempt 10 killed, TaskAttempt 11 killed, 
TaskAttempt 12 failed, 
info=[AttemptID:attempt_1522319554594_0065_19_05_000119_12 Timed out after 90 
secs], TaskAttempt 13 killed, TaskAttempt 14 failed, 
info=[AttemptID:attempt_1522319554594_0065_19_05_000119_14 Timed out after 90 
secs], TaskAttempt 15 failed, 
info=[org.apache.hadoop.ipc.RemoteException(java.lang.RuntimeException):

[jira] [Commented] (HIVE-19169) llap: Timed out after 90 secs

2018-04-11 Thread Zoltan Haindrich (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433495#comment-16433495
 ] 

Zoltan Haindrich commented on HIVE-19169:
-

this could be known issue ; but I wasn't able to locate a relevant jira - I'm 
using an at least 2-3 weeks old version of master

> llap: Timed out after 90 secs
> -
>
> Key: HIVE-19169
> URL: https://issues.apache.org/jira/browse/HIVE-19169
> Project: Hive
>  Issue Type: Bug
>  Components: llap
>Reporter: Zoltan Haindrich
>Priority: Major
>
> a more or less recent hive/master with tpcds1000; while running only 1 query 
> at a time on the cluster; sometimes a "timeout" happens - this could be even 
> a misconfiguration problem...I'm not sure I've set it up correctly.
> what I see from: it seems like the attempt have entered the queue and 
> accepted - after almost 90 seconds there are messages that it will be 
> pre-empted - and the 90 sec timeout happens.
> hive.log; example: attempt_1522319554594_0065_19_05_000119_14 failed
> {code}
> 2018-04-10T13:20:25,178 ERROR [HiveServer2-Background-Pool: Thread-4194]: 
> SessionState (SessionState.java:printError(1214)) - Vertex failed, 
> vertexName=Reducer 3, vertexId=vertex_1522319554594_0065_19_05, 
> diagnostics=[Task failed, taskId=task_1522319554594_0065_19_05_000119, 
> diagnostics=[TaskAttempt 0 killed, TaskAttempt 1 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_1 Timed out after 90 
> secs], TaskAttempt 2 killed, TaskAttempt 3 killed, TaskAttempt 4 killed, 
> TaskAttempt 5 killed, TaskAttempt 6 killed, TaskAttempt 7 killed, TaskAttempt 
> 8 killed, TaskAttempt 9 killed, TaskAttempt 10 killed, TaskAttempt 11 killed, 
> TaskAttempt 12 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_12 Timed out after 90 
> secs], TaskAttempt 13 killed, TaskAttempt 14 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_14 Timed out after 90 
> secs], TaskAttempt 15 failed, 
> info=[org.apache.hadoop.ipc.RemoteException(java.lang.RuntimeException): 
> attempt_1522319554594_0065_19_08_18_6 was not registered and couldn't be 
> removed
> 2018-04-10T13:20:25,260 ERROR [HiveServer2-Background-Pool: Thread-4194]: 
> ql.Driver (SessionState.java:printError(1214)) - FAILED: Execution Error, 
> return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, 
> vertexName=Reducer 3, vertexId=vertex_1522319554594_0065_19_05, 
> diagnostics=[Task failed, taskId=task_1522319554594_0065_19_05_000119, 
> diagnostics=[TaskAttempt 0 killed, TaskAttempt 1 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_1 Timed out after 90 
> secs], TaskAttempt 2 killed, TaskAttempt 3 killed, TaskAttempt 4 killed, 
> TaskAttempt 5 killed, TaskAttempt 6 killed, TaskAttempt 7 killed, TaskAttempt 
> 8 killed, TaskAttempt 9 killed, TaskAttempt 10 killed, TaskAttempt 11 killed, 
> TaskAttempt 12 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_12 Timed out after 90 
> secs], TaskAttempt 13 killed, TaskAttempt 14 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_14 Timed out after 90 
> secs], TaskAttempt 15 failed, 
> info=[org.apache.hadoop.ipc.RemoteException(java.lang.RuntimeException): 
> attempt_1522319554594_0065_19_08_18_6 was not registered and couldn't be 
> removed
> org.apache.hive.service.cli.HiveSQLException: Error while processing 
> statement: FAILED: Execution Error, return code 2 from 
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Reducer 
> 3, vertexId=vertex_1522319554594_0065_19_05, diagnostics=[Task failed, 
> taskId=task_1522319554594_0065_19_05_000119, diagnostics=[TaskAttempt 0 
> killed, TaskAttempt 1 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_1 Timed out after 90 
> secs], TaskAttempt 2 killed, TaskAttempt 3 killed, TaskAttempt 4 killed, 
> TaskAttempt 5 killed, TaskAttempt 6 killed, TaskAttempt 7 killed, TaskAttempt 
> 8 killed, TaskAttempt 9 killed, TaskAttempt 10 killed, TaskAttempt 11 killed, 
> TaskAttempt 12 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_12 Timed out after 90 
> secs], TaskAttempt 13 killed, TaskAttempt 14 failed, 
> info=[AttemptID:attempt_1522319554594_0065_19_05_000119_14 Timed out after 90 
> secs], TaskAttempt 15 failed, 
> info=[org.apache.hadoop.ipc.RemoteException(java.lang.RuntimeException): 
> attempt_1522319554594_0065_19_08_18_6 was not registered and couldn't be 
> removed
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Vertex failed, 
> vertexName=Reducer 3, vertexId=vertex_1522319554594_0065_19_05, 
> diagnostics=[Task failed, taskId=task_1522319554594_0065_19_05_000119, 
> diagnostics=[TaskAttempt 0 killed, TaskAttempt 1 failed, 
>

[jira] [Commented] (HIVE-18910) Migrate to Murmur hash for shuffle and bucketing

2018-04-11 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433472#comment-16433472
 ] 

Hive QA commented on HIVE-18910:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  4m 
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  3m 
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  4m 
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
20s{color} | {color:red} streaming in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
57s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  4m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  4m 
53s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
14s{color} | {color:red} storage-api: The patch generated 3 new + 97 unchanged 
- 3 fixed = 100 total (was 100) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
21s{color} | {color:red} serde: The patch generated 150 new + 214 unchanged - 3 
fixed = 364 total (was 217) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
17s{color} | {color:red} hcatalog/streaming: The patch generated 1 new + 257 
unchanged - 0 fixed = 258 total (was 257) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
15s{color} | {color:red} hcatalog/webhcat/java-client: The patch generated 1 
new + 147 unchanged - 0 fixed = 148 total (was 147) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
21s{color} | {color:red} itests/hive-unit: The patch generated 5 new + 264 
unchanged - 0 fixed = 269 total (was 264) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m  
4s{color} | {color:red} ql: The patch generated 26 new + 1665 unchanged - 3 
fixed = 1691 total (was 1668) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
1s{color} | {color:red} The patch 248 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  4m 
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  0m 
15s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 42m 25s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10133/dev-support/hive-personality.sh
 |
| git revision | master / f6dbcac |
| Default Java | 1.8.0_111 |
| mvninstall | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10133/yetus/patch-mvninstall-hcatalog_streaming.txt
 |
| mvninstall | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10133/yetus/patch-mvninstall-ql.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10133/yetus/diff-checkstyle-storage-api.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10133/yetus/diff-checkstyle-serde.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-10133/yetus/diff-checkstyle-hcatalog_streaming.txt
 |
| checkstyle |

[jira] [Assigned] (HIVE-19168) Ranger changes for llap commands

2018-04-11 Thread Prasanth Jayachandran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran reassigned HIVE-19168:



> Ranger changes for llap commands
> 
>
> Key: HIVE-19168
> URL: https://issues.apache.org/jira/browse/HIVE-19168
> Project: Hive
>  Issue Type: Bug
>  Components: Authorization
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
>
> New llap commands "llap cluster -info" and "llap cache -purge" require some 
> changes so that Ranger can log the commands for auditing. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-18320) Support ACID Tables Replication

2018-04-11 Thread anishek (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

anishek updated HIVE-18320:
---
Attachment: (was: Hive Replication- ACID Tables.pdf)

> Support ACID Tables Replication
> ---
>
> Key: HIVE-18320
> URL: https://issues.apache.org/jira/browse/HIVE-18320
> Project: Hive
>  Issue Type: New Feature
>  Components: HiveServer2, Metastore, repl, Transactions
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: ACID, DR, replication
> Attachments: Hive Replication_ ACID Tables.pdf
>
>
> Currently, Full ACID and MM (Micro-Managed) tables are not supported for 
> replication.
> Need to support it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-18320) Support ACID Tables Replication

2018-04-11 Thread Sankar Hariappan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sankar Hariappan updated HIVE-18320:

Attachment: Hive Replication_ ACID Tables.pdf

> Support ACID Tables Replication
> ---
>
> Key: HIVE-18320
> URL: https://issues.apache.org/jira/browse/HIVE-18320
> Project: Hive
>  Issue Type: New Feature
>  Components: HiveServer2, Metastore, repl, Transactions
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
>  Labels: ACID, DR, replication
> Attachments: Hive Replication- ACID Tables.pdf, Hive Replication_ 
> ACID Tables.pdf
>
>
> Currently, Full ACID and MM (Micro-Managed) tables are not supported for 
> replication.
> Need to support it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19093) some parts of the Driver runs from the "Background-Pool" in HS2

2018-04-11 Thread Bharathkrishna Guruvayoor Murali (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bharathkrishna Guruvayoor Murali reassigned HIVE-19093:
---

Assignee: Bharathkrishna Guruvayoor Murali

> some parts of the Driver runs from the "Background-Pool" in HS2
> ---
>
> Key: HIVE-19093
> URL: https://issues.apache.org/jira/browse/HIVE-19093
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
>
> I was looking into perflog results; and the fact that Driver.run open / close 
> happens on a different thread caught my eye - this might cause real problems 
> since {{Session.get()}} will return an entirely different session in the 
> aftermath...most notably there are some lock related calls like: releaseLocks
> {code}
> 2018-04-03T08:36:53,488 DEBUG [2c81c6c1-aa6f-4609-8250-5b1a5360a8ba 
> HiveServer2-Handler-Pool: Thread-16242]: log.PerfLogger 
> (PerfLogger.java:PerfLogBegin(132)) -  from=org.apache.hadoop.hive.ql.Driver>
> 2018-04-03T08:37:21,791 DEBUG [HiveServer2-Background-Pool: Thread-16247]: 
> log.PerfLogger (PerfLogger.java:PerfLogEnd(172)) -  method=Driver.run start=1522744613488 end=1522744641791 duration=28303 
> from=org.apache.hadoop.hive.ql.Driver>
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19167) Map data type doesn't keep the order of the key/values pairs as read (Part 2, The Sequel or SQL)

2018-04-11 Thread Matt McCline (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline reassigned HIVE-19167:
---


> Map data type doesn't keep the order of the key/values pairs as read (Part 2, 
> The Sequel or SQL)   
> ---
>
> Key: HIVE-19167
> URL: https://issues.apache.org/jira/browse/HIVE-19167
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.1.0
>
>
> HIVE-19116: "Vectorization: Vector Map data type doesn't keep the order of 
> the key/values pairs as read" didn't fix all the places where HashMap is used 
> instead of LinkedHashMap.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

< 1 2

101 - 169 of 169 matches

Mail list logo