[jira] [Created] (HIVE-12951) Reduce Spark executor prewarm timeout to 5s

2016-01-27 Thread Xuefu Zhang (JIRA) 
Xuefu Zhang created HIVE-12951:
--

 Summary: Reduce Spark executor prewarm timeout to 5s
 Key: HIVE-12951
 URL: https://issues.apache.org/jira/browse/HIVE-12951
 Project: Hive
  Issue Type: Bug
  Components: Spark
Affects Versions: 1.2.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang


Currently it's set to 30s, which tends to be longer than needed. Reduce it to 
5s, only considering jvm startup time. (Eventually, we may want to make this 
configurable.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-12949) Fix description of hive.server2.logging.operation.level

2016-01-27 Thread Mohit Sabharwal (JIRA) 
Mohit Sabharwal created HIVE-12949:
--

 Summary: Fix description of hive.server2.logging.operation.level
 Key: HIVE-12949
 URL: https://issues.apache.org/jira/browse/HIVE-12949
 Project: Hive
  Issue Type: Bug
  Components: Configuration
Reporter: Mohit Sabharwal
Assignee: Mohit Sabharwal


Description of {{hive.server2.logging.operation.level}} states that this 
property is available to set at session level. This is not correct. The 
property is only settable at HS2 level currently.

A log4j appender is created just once currently - based on logging level 
specified by this config, at OperationManager init time in 
initOperationLogCapture(). And there is only one instance of OperationManager.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-12947) SMB join in tez has ClassCastException when container reuse is on

2016-01-27 Thread Vikram Dixit K (JIRA) 
Vikram Dixit K created HIVE-12947:
-

 Summary: SMB join in tez has ClassCastException when container 
reuse is on
 Key: HIVE-12947
 URL: https://issues.apache.org/jira/browse/HIVE-12947
 Project: Hive
  Issue Type: Bug
  Components: Tez
Affects Versions: 2.0.0
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K


{code}
java.lang.RuntimeException: Map operator initialization failed
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:247)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:147)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)
at 
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassCastException: 
org.apache.hadoop.hive.ql.exec.FileSinkOperator cannot be cast to 
org.apache.hadoop.hive.ql.exec.DummyStoreOperator
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.getJoinParentOp(MapRecordProcessor.java:300)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.getJoinParentOp(MapRecordProcessor.java:302)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.getJoinParentOp(MapRecordProcessor.java:302)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.getJoinParentOp(MapRecordProcessor.java:302)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.getJoinParentOp(MapRecordProcessor.java:302)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:189)
... 15 more
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-12950) support NullRowInputFormat in LLAP for nullscan cases

2016-01-27 Thread Sergey Shelukhin (JIRA) 
Sergey Shelukhin created HIVE-12950:
---

 Summary: support NullRowInputFormat in LLAP for nullscan cases
 Key: HIVE-12950
 URL: https://issues.apache.org/jira/browse/HIVE-12950
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-12953) Update description of hive.ppd.remove.duplicatefilters in HiveConf.java

2016-01-27 Thread Shannon Ladymon (JIRA) 
Shannon Ladymon created HIVE-12953:
--

 Summary: Update description of hive.ppd.remove.duplicatefilters in 
HiveConf.java
 Key: HIVE-12953
 URL: https://issues.apache.org/jira/browse/HIVE-12953
 Project: Hive
  Issue Type: Improvement
Affects Versions: 1.2.1
Reporter: Shannon Ladymon
Assignee: Shannon Ladymon
Priority: Minor


Per [discussion in HIVE-1638 | 
https://issues.apache.org/jira/browse/HIVE-1538?focusedCommentId=14568180=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14568180],
 we should update the description of the the *hive.ppd.remove.duplicatefilters* 
parameter in HiveConf.java.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 41807: further improve user level explain

2016-01-27 Thread John Pullokkaran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41807/#review116704
---


Ship it!




Ship It!

- John Pullokkaran


On Jan. 21, 2016, 7:46 a.m., pengcheng xiong wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41807/
> ---
> 
> (Updated Jan. 21, 2016, 7:46 a.m.)
> 
> 
> Review request for hive and John Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-12709
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Attr.java 
> 5d355d2 
>   common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Op.java 
> d0c1037 
>   common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Stage.java 
> 455d59f 
>   
> common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/TezJsonParser.java
>  b193fef 
>   
> common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/TezJsonParserUtils.java
>  PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Vertex.java 
> be01b8b 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java f48db6a 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/AbstractOperatorDesc.java 
> bc67e5a 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/BaseWork.java ad34b98 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 3f38f74 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableLikeDesc.java f051712 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FileSinkDesc.java 40a8477 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/FilterDesc.java d04cb78 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/GroupByDesc.java 0f2855e 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/HashTableSinkDesc.java 7c8eee2 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/JoinDesc.java eb83fd6 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/LateralViewJoinDesc.java 12f01e5 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/LoadTableDesc.java 427aac1 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java 4b93e7c 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 5bea6fb 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 2f69b7f 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceWork.java 1c31962 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/SelectDesc.java e7bbab4 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/Statistics.java 4e52bac 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/TableDesc.java 2fdb0a1 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/TableScanDesc.java 098aa89 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/PTFQueryInputDef.java 
> ae55741 
>   
> ql/src/java/org/apache/hadoop/hive/ql/plan/ptf/PartitionedTableFunctionDef.java
>  46f106c 
>   ql/src/test/queries/clientpositive/explainuser_4.q PRE-CREATION 
>   ql/src/test/results/clientpositive/perf/query12.q.out 648d64d 
>   ql/src/test/results/clientpositive/perf/query13.q.out 6113d18 
>   ql/src/test/results/clientpositive/perf/query15.q.out c237828 
>   ql/src/test/results/clientpositive/perf/query17.q.out 0e42d05 
>   ql/src/test/results/clientpositive/perf/query18.q.out 83f21dc 
>   ql/src/test/results/clientpositive/perf/query19.q.out d565839 
>   ql/src/test/results/clientpositive/perf/query20.q.out 40c9a17 
>   ql/src/test/results/clientpositive/perf/query21.q.out 2634d3d 
>   ql/src/test/results/clientpositive/perf/query22.q.out 72935ed 
>   ql/src/test/results/clientpositive/perf/query25.q.out dabc1c5 
>   ql/src/test/results/clientpositive/perf/query26.q.out 84cc238 
>   ql/src/test/results/clientpositive/perf/query27.q.out ac38683 
>   ql/src/test/results/clientpositive/perf/query28.q.out cac9ac3 
>   ql/src/test/results/clientpositive/perf/query29.q.out 8991f35 
>   ql/src/test/results/clientpositive/perf/query3.q.out 871ad74 
>   ql/src/test/results/clientpositive/perf/query31.q.out 909d64c 
>   ql/src/test/results/clientpositive/perf/query32.q.out da2ac8f 
>   ql/src/test/results/clientpositive/perf/query34.q.out 1cb7c28 
>   ql/src/test/results/clientpositive/perf/query39.q.out 9f3e650 
>   ql/src/test/results/clientpositive/perf/query40.q.out b2d6262 
>   ql/src/test/results/clientpositive/perf/query42.q.out 9ede45d 
>   ql/src/test/results/clientpositive/perf/query43.q.out 7fa4f53 
>   ql/src/test/results/clientpositive/perf/query45.q.out f55f20c 
>   ql/src/test/results/clientpositive/perf/query46.q.out 0ded912 
>   ql/src/test/results/clientpositive/perf/query48.q.out 40b76f0 
>   ql/src/test/results/clientpositive/perf/query50.q.out 8b41d05 
>   ql/src/test/results/clientpositive/perf/query51.q.out f57af2c 
>   ql/src/test/results/clientpositive/perf/query52.q.out 7e7224b 
>   ql/src/test/results/clientpositive/perf/query54.q.out 56c70b7

Re: Review Request 42487: Use bit vector to track NDV

2016-01-27 Thread John Pullokkaran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42487/#review116706
---


Ship it!




Ship It!

- John Pullokkaran


On Jan. 26, 2016, 8:10 a.m., pengcheng xiong wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42487/
> ---
> 
> (Updated Jan. 26, 2016, 8:10 a.m.)
> 
> 
> Review request for hive, Alan Gates and John Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-12763
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/HiveStatsUtils.java 9193f80 
>   
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/hbase/TestHBaseSchemaTool.java
>  9fbbf90 
>   itests/src/test/resources/testconfiguration.properties 8318c3a 
>   metastore/if/hive_metastore.thrift 81837e6 
>   metastore/pom.xml a8e84a1 
>   
> metastore/src/gen/protobuf/gen-java/org/apache/hadoop/hive/metastore/hbase/HbaseMetastoreProto.java
>  39a7278 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h ce1d7da 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 0203b06 
>   
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/BinaryColumnStatsData.java
>  84e393c 
>   
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/BooleanColumnStatsData.java
>  6aa4668 
>   
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DateColumnStatsData.java
>  2ebb811 
>   
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DecimalColumnStatsData.java
>  720176a 
>   
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DoubleColumnStatsData.java
>  5d48b5d 
>   
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/LongColumnStatsData.java
>  2f41c5a 
>   
> metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StringColumnStatsData.java
>  bd8a922 
>   metastore/src/gen/thrift/gen-php/metastore/Types.php 380e6d0 
>   metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 409c247 
>   metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb a473611 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/NumDistinctValueEstimator.java
>  PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseUtils.java 
> f4df2e2 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/StatsCache.java 
> 5ec60be 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/BinaryColumnStatsAggregator.java
>  bbd2c7b 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/BooleanColumnStatsAggregator.java
>  9047f68 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/ColumnStatsAggregator.java
>  217b654 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/ColumnStatsAggregatorFactory.java
>  a8dbc1f 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/DecimalColumnStatsAggregator.java
>  ec25b31 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/DoubleColumnStatsAggregator.java
>  71af0ac 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/LongColumnStatsAggregator.java
>  15b8cf7 
>   
> metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/StringColumnStatsAggregator.java
>  fe1a04c 
>   
> metastore/src/protobuf/org/apache/hadoop/hive/metastore/hbase/hbase_metastore_proto.proto
>  0d0ef89 
>   
> metastore/src/test/org/apache/hadoop/hive/metastore/hbase/TestHBaseStoreBitVector.java
>  PRE-CREATION 
>   ql/pom.xml 358cd2a 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsTask.java 7914471 
>   
> ql/src/java/org/apache/hadoop/hive/ql/parse/ColumnStatsSemanticAnalyzer.java 
> 1f30cbd 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java b4cf58f 
>   
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFComputeStats.java
>  0e96f89 
>   ql/src/test/queries/clientpositive/tez_aggr_part_stats.q PRE-CREATION 
>   ql/src/test/results/clientpositive/char_udf1.q.java1.7.out bfed116 
>   ql/src/test/results/clientpositive/columnstats_partlvl.q.out b7c9075 
>   ql/src/test/results/clientpositive/columnstats_partlvl_dp.q.out 9685202 
>   ql/src/test/results/clientpositive/columnstats_tbllvl.q.out 0aadae3 
>   ql/src/test/results/clientpositive/compute_stats_date.q.out b57a862 
>   ql/src/test/results/clientpositive/compute_stats_decimal.q.out 35abb37 
>   ql/src/test/results/clientpositive/compute_stats_double.q.out f6b4052 
>   ql/src/test/results/clientpositive/compute_stats_empty_table.q.out f76c760 
>   ql/src/test/results/clientpositive/compute_stats_long.q.out 2c6171d 
>

Re: [CONF] Apache Hive > Presentations

2016-01-27 Thread Lefty Leverenz 
+1

Good idea, Sergey.

-- Lefty


On Wed, Jan 27, 2016 at 11:47 AM, Szehon Ho  wrote:

> Yea makes sense to me (esp as "Older Presentations" are already at the
> bottom).  I thought it had been in reverse order at one point.
>
> On Wed, Jan 27, 2016 at 11:41 AM, Sergey Shelukhin  >
> wrote:
>
> > Btw, would it make sense to keep the page in reverse chronological order,
> > to have the recent stuff on top?
> >
> > From: "Xuefu Zhang (Confluence)" 
> > Date: Tuesday, January 26, 2016 at 20:11
> > To: Sergey Shelukhin 
> > Subject: [CONF] Apache Hive > Presentations
> >
> >
> > <
> https://cwiki.apache.org/confluence/display/~xuefu?src=mail=1453867871770=com.atlassian.confluence.plugins.confluence-file-notifications%3Afile-content-update-notification=8aa980874e36a1eb014e36a2ced410d2>
> Xuefu
> > Zhang *added* a file to a page
> >
> > [image: attachment icon]
> > <
> https://cwiki.apache.org/confluence/display/Hive/Presentations?preview=%2F27362054%2F61337443%2FHive+User+Group+Meeting+Jan+2016.pdf=mail=1453867871770=com.atlassian.confluence.plugins.confluence-file-notifications%3Afile-content-update-notification=8aa980874e36a1eb014e36a2ced410d2=view>
> Hive
> > User Group Meeting Jan 2016.pdf
> > <
> https://cwiki.apache.org/confluence/display/Hive/Presentations?preview=%2F27362054%2F61337443%2FHive+User+Group+Meeting+Jan+2016.pdf=mail=1453867871770=com.atlassian.confluence.plugins.confluence-file-notifications%3Afile-content-update-notification=8aa980874e36a1eb014e36a2ced410d2=view
> >
> > [image: Presentations Icon]
> > <
> https://cwiki.apache.org/confluence/display/Hive/Presentations?src=mail=1453867871770=com.atlassian.confluence.plugins.confluence-file-notifications%3Afile-content-update-notification=8aa980874e36a1eb014e36a2ced410d2
> >
> > Presentations
> > <
> https://cwiki.apache.org/confluence/display/Hive/Presentations?src=mail=1453867871770=com.atlassian.confluence.plugins.confluence-file-notifications%3Afile-content-update-notification=8aa980874e36a1eb014e36a2ced410d2
> >
> >
> > Stop watching page
> > <
> https://cwiki.apache.org/confluence/users/removepagenotification.action?pageId=27362054=mail=1453867871770=com.atlassian.confluence.plugins.confluence-file-notifications%3Afile-content-update-notification=8aa980874e36a1eb014e36a2ced410d2=stop-watching
> >
> > •
> > Manage notifications
> > <
> https://cwiki.apache.org/confluence/users/editmyemailsettings.action?src=mail=1453867871770=com.atlassian.confluence.plugins.confluence-file-notifications%3Afile-content-update-notification=8aa980874e36a1eb014e36a2ced410d2=manage
> >
> > [image: Confluence logo big]
> > This message was sent by Atlassian Confluence 5.8.4
> >
>

[GitHub] hive pull request: HIVE-12943: Fixing USE DEFAULT by adding the 'd...

2016-01-27 Thread kasjain 
GitHub user kasjain opened a pull request:

https://github.com/apache/hive/pull/65

HIVE-12943: Fixing USE DEFAULT by adding the 'default' keyword to grammer

Added the "default" keyword to the grammer files HiveParser.g, HiveLexer.g 
and Identifier.g

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kasjain/hive branch-1.2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/65.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #65


commit ae0e374637b39f4847c0413d4fdbe69a2d7f1fb9
Author: Kashish Jain 
Date:   2016-01-28T05:31:09Z

HIVE-12943: Fixing USE DEFAULT by adding the 'default' keyword to grammer




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

HIVE-12943- Use default not working

2016-01-27 Thread Kashish Jain 
Hi,


I want to discuss about this bug and the possible resolutions for this. I have 
done a fix in my local and the issue seems to be resolved. The pull request for 
the same is https://github.com/apache/hive/pull/65

 Can someone review this please?


Thanks

Kashish Jain

[jira] [Created] (HIVE-12948) Update branch-2.0 before cutting the RC

2016-01-27 Thread Sergey Shelukhin (JIRA) 
Sergey Shelukhin created HIVE-12948:
---

 Summary: Update branch-2.0 before cutting the RC
 Key: HIVE-12948
 URL: https://issues.apache.org/jira/browse/HIVE-12948
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Attachments: HIVE-12948.patch

One blocker remaining, hopefully the RC will come soon. I will update the 
change list from here: 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12332641=Text=12310843=Create
 right before the RC. For now, update README and the tracking branch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-12942) Remove Yarn WebApps from LLAP daemon instance

2016-01-27 Thread Gopal V (JIRA) 
Gopal V created HIVE-12942:
--

 Summary: Remove Yarn WebApps from LLAP daemon instance
 Key: HIVE-12942
 URL: https://issues.apache.org/jira/browse/HIVE-12942
 Project: Hive
  Issue Type: Improvement
  Components: llap
Affects Versions: 2.1.0
Reporter: Gopal V
Assignee: Gopal V


Yarn WebApps obscures web app contexts and makes it inconvenient.

Switch out that impl and instead use org.apache.hive.http.* ones from 
hive-common, which can be modified more easily.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 41928: HIVE-12777: Add capability to restore session

2016-01-27 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41928/
---

(Updated Jan. 27, 2016, 8:55 p.m.)


Review request for hive.


Bugs: HIVE-12777
https://issues.apache.org/jira/browse/HIVE-12777


Repository: hive-git


Description
---

Extensions using Hive session handles should be able to restore the hive 
session from the handle. 

Apache Lens depends on a fork of hive and that fork has such a capability. 

Relevant commit: 
https://github.com/InMobi/hive/commit/931fe9116161a18952c082c14223ad6745fefe00#diff-0acb35f7cab7492f522b0c40ce3ce1be


Diffs (updated)
-

  common/pom.xml 8141f75df6b649c0d9f66fd7c6efa2989e42938a 
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
2c25cae570b33dd03e33be3df45a964cc0a7f071 
  common/src/java/org/apache/hadoop/hive/conf/Validator.java 
04a305dbf15376c116ae660cdb8a42bc948ae6e3 
  data/files/windowing_distinct.txt PRE-CREATION 
  errata.txt 6ac1d41aa1476553a77d0c5a58f4a4c7c3d891f7 
  
hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatAuthUtil.java
 6dce9c4b1d1218ce1f8b4ebffee0eaacb31d25cd 
  
hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/DynamicPartitionFileRecordWriterContainer.java
 320ace4352c1212bab093d38635c4d70005bd473 
  
hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FosterStorageHandler.java
 bc56d77c00bd5ac908c913948ad93f89c60d57ab 
  
hcatalog/core/src/test/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/TestHCatAuthUtil.java
 830dcb8119c255127f64a128a1d7b66f041bfb3e 
  
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatMultiOutputFormat.java
 8148faa8741952b0f92c33eaaa5ca908dfa47365 
  
hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/TestStreaming.java
 ff2598f9a8e4d3e83292e5b85bf137aaea53a24d 
  
hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/mutate/StreamingAssert.java
 339e9ef8a5517ee17e84ab63ea168a01c415db73 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/hbase/TestHBaseSchemaTool.java
 24bebb821ebff6707620f363cac9ee5ce47eff50 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/hbase/TestHBaseStoreIntegration.java
 c62190421cb979790d8c1df5c952292c39fd2ab3 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestMetastoreAuthorizationProvider.java
 4529ce396e209d7a8e7648b2213bcab12cb28e50 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCompactor.java
 da367ca363ab11cb371236beab71cc2d6c43525b 
  itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java 
09905ea1fe98ea5aaf12b902081a41b53ed9ad4e 
  itests/src/test/resources/testconfiguration.properties 
a2ccfe0e86512021c70e79e04242c16325435e0f 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 
fa16020361e05059e4d81dbb0c52e3ade1c9cdc6 
  
llap-client/src/java/org/apache/hadoop/hive/llap/registry/ServiceInstanceSet.java
 388b5f3ab8154943ae8a6a5d387b6e3766480585 
  
llap-client/src/java/org/apache/hadoop/hive/llap/registry/impl/LlapFixedRegistryImpl.java
 ef9de320c5c112bef6c5293e8caea980ade38eb1 
  
llap-client/src/java/org/apache/hadoop/hive/llap/registry/impl/LlapRegistryService.java
 740f3734d179d206b6f73407e0bea702f7464113 
  
llap-client/src/java/org/apache/hadoop/hive/llap/registry/impl/LlapYarnRegistryImpl.java
 fc2ebf2347845325da2042dda05cfe6c1d2f983c 
  
llap-server/src/gen/protobuf/gen-java/org/apache/hadoop/hive/llap/daemon/rpc/LlapDaemonProtocolProtos.java
 d2180e55e8008b22d74efe6d155a16271e357f72 
  llap-server/src/java/org/apache/hadoop/hive/llap/cache/BuddyAllocator.java 
ab4df5dbd9eac092b8691bcdfdf4aacbfbf80ce8 
  
llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelCacheMemoryManager.java
 992da8e78599174becf53063d310821369fca2a9 
  
llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelLrfuCachePolicy.java
 84910d74269f61e010a0ce1061d2d3591d93e745 
  llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapServiceDriver.java 
d6e1a6e68630f73ee5cf3acd6dc2acb5b01202a2 
  llap-server/src/java/org/apache/hadoop/hive/llap/daemon/HistoryLogger.java 
3c9ad2427c2d9f5ca3c6dcea892bc29b7e8ac891 
  
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/KilledTaskHandler.java 
7cb433bfd9c10ec678fbd022fb634aa9ee0643ea 
  
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/QueryFailedHandler.java 
4e62a68f763c3c43324b2034885d260b8b214c9c 
  llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/AMReporter.java 
f6711d80acf847f1babd6ef522b332a21a7ada61 
  
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/ContainerRunnerImpl.java
 0d85671381e00510959cb5e4ba5fd9f2220fbdf6 
  
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/FunctionLocalizer.java
 PRE-CREATION 
  llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/LlapDaemon.java

Review Request 42758: Improve dynamic partition loading

2016-01-27 Thread Ashutosh Chauhan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42758/
---

Review request for hive and Prasanth_J.


Bugs: HIVE-12907
https://issues.apache.org/jira/browse/HIVE-12907


Repository: hive-git


Description
---

Reduces number of calls on metastore


Diffs
-

  common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java 029d415 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/TestDbNotificationListener.java
 56f5c8b 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java efb50b2 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java c8895c2 

Diff: https://reviews.apache.org/r/42758/diff/


Testing
---

Existing tests.


Thanks,

Ashutosh Chauhan

Review Request 42858: HIVE-12944: Support SUM(DISTINCT) for partitioning query.

2016-01-27 Thread Aihua Xu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42858/
---

Review request for hive and Szehon Ho.


Repository: hive-git


Description
---

HIVE-12944: Support SUM(DISTINCT) for partitioning query.


Diffs
-

  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/functions/HiveSqlSumAggFunction.java
 056eaeb0d707342185763dc5d5af900523e5512d 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/SqlFunctionConverter.java
 75c38fa0d30ec4a3fcf240b7f5d6e3c739a3aa17 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCount.java 
f526c437c21713b02f991ce3a8ee52a0dbd885a2 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 
09680086ee25b3f512061cb60946b55db934345c 
  ql/src/test/queries/clientpositive/windowing_distinct.q 
94f4044b8a491e9e13ebe7fd879fd49715860e94 
  ql/src/test/results/clientpositive/windowing_distinct.q.out 
50f8ff81934c6a9338e74e9e6d4116d5f2b3ba4b 

Diff: https://reviews.apache.org/r/42858/diff/


Testing
---


Thanks,

Aihua Xu

[jira] [Created] (HIVE-12944) Support SUM(DISTINCT) for partitioning query.

2016-01-27 Thread Aihua Xu (JIRA) 
Aihua Xu created HIVE-12944:
---

 Summary: Support SUM(DISTINCT) for partitioning query.
 Key: HIVE-12944
 URL: https://issues.apache.org/jira/browse/HIVE-12944
 Project: Hive
  Issue Type: Sub-task
  Components: PTF-Windowing
Affects Versions: 2.1.0
Reporter: Aihua Xu
Assignee: Aihua Xu
 Fix For: 2.1.0


We need to support avg(distinct), count(distinct), sum(distinct) for the parent 
jira HIVE-9534. Separate the work for count(distinct) in this subtask.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 42671: Improve dynamic partition loading

2016-01-27 Thread j . prasanth . j 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42671/#review116592
---


Ship it!




Ship It!

- Prasanth_J


On Jan. 27, 2016, 7 a.m., Ashutosh Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42671/
> ---
> 
> (Updated Jan. 27, 2016, 7 a.m.)
> 
> 
> Review request for hive, Gopal V and Prasanth_J.
> 
> 
> Bugs: HIVE-12897
> https://issues.apache.org/jira/browse/HIVE-12897
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Reduces number of calls to Metastore.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 97fe7bc 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 
> eee7f1b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
> e044c73 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java 3289cfc 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java efb50b2 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/index/RewriteParseContextGenerator.java
>  48105de 
>   
> ql/src/java/org/apache/hadoop/hive/ql/parse/ExplainSQRewriteSemanticAnalyzer.java
>  2c2339a 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/ExplainSemanticAnalyzer.java 
> c1e9ec1 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/DynamicPartitionCtx.java 95d5635 
> 
> Diff: https://reviews.apache.org/r/42671/diff/
> 
> 
> Testing
> ---
> 
> Regression test suite.
> 
> 
> Thanks,
> 
> Ashutosh Chauhan
> 
>

Re: Review Request 41928: HIVE-12777: Add capability to restore session

2016-01-27 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41928/
---

(Updated Jan. 28, 2016, 12:48 p.m.)


Review request for hive.


Bugs: HIVE-12777
https://issues.apache.org/jira/browse/HIVE-12777


Repository: hive-git


Description
---

Extensions using Hive session handles should be able to restore the hive 
session from the handle. 

Apache Lens depends on a fork of hive and that fork has such a capability. 

Relevant commit: 
https://github.com/InMobi/hive/commit/931fe9116161a18952c082c14223ad6745fefe00#diff-0acb35f7cab7492f522b0c40ce3ce1be


Diffs (updated)
-

  service/src/java/org/apache/hive/service/cli/CLIService.java 
0d5ae1e491bdcf98a12f5eef7b35f23c7b9dfc00 
  service/src/java/org/apache/hive/service/cli/SessionHandle.java 
4e4f0211de09408168c8da669626fb49f3db364c 
  service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 
3c5700bd114f6b792df10df23e57dad9b9e689f6 
  
service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java
 c56d97afc969ea57810e9b3222b20e086e847d67 
  service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
de18f1d93502704e7a8fbb8d31d93bd770c4d232 
  service/src/test/org/apache/hive/service/cli/CLIServiceRestoreTest.java 
PRE-CREATION 

Diff: https://reviews.apache.org/r/41928/diff/


Testing
---


Thanks,

Rajat Khandelwal

Re: Review Request 42758: Improve dynamic partition loading

2016-01-27 Thread j . prasanth . j 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42758/#review116597
---


Ship it!




Ship It!

- Prasanth_J


On Jan. 27, 2016, 4:10 p.m., Ashutosh Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42758/
> ---
> 
> (Updated Jan. 27, 2016, 4:10 p.m.)
> 
> 
> Review request for hive and Prasanth_J.
> 
> 
> Bugs: HIVE-12907
> https://issues.apache.org/jira/browse/HIVE-12907
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Reduces number of calls on metastore
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java 029d415 
>   
> itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/TestDbNotificationListener.java
>  56f5c8b 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java efb50b2 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java c8895c2 
> 
> Diff: https://reviews.apache.org/r/42758/diff/
> 
> 
> Testing
> ---
> 
> Existing tests.
> 
> 
> Thanks,
> 
> Ashutosh Chauhan
> 
>

Re: Review Request 42858: HIVE-12944: Support SUM(DISTINCT) for partitioning query.

2016-01-27 Thread Szehon Ho 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42858/#review116607
---


Fix it, then Ship it!




Good job, same logic as count(distinct).  Dont have many useful comments, just 
very small code-style nits, can fix on commit.


ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/SqlFunctionConverter.java
 (line 233)


Space after the cast for code-style consistency.



ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java (line 95)


Space after cast



ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java (line 129)


TYPE should be T or Type for codestyle.



ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java (line 219)


Same



ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java (line 336)


Same



ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java (line 450)


Same


- Szehon Ho


On Jan. 27, 2016, 4:31 p.m., Aihua Xu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42858/
> ---
> 
> (Updated Jan. 27, 2016, 4:31 p.m.)
> 
> 
> Review request for hive and Szehon Ho.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-12944: Support SUM(DISTINCT) for partitioning query.
> 
> 
> Diffs
> -
> 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/functions/HiveSqlSumAggFunction.java
>  056eaeb0d707342185763dc5d5af900523e5512d 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/SqlFunctionConverter.java
>  75c38fa0d30ec4a3fcf240b7f5d6e3c739a3aa17 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCount.java 
> f526c437c21713b02f991ce3a8ee52a0dbd885a2 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 
> 09680086ee25b3f512061cb60946b55db934345c 
>   ql/src/test/queries/clientpositive/windowing_distinct.q 
> 94f4044b8a491e9e13ebe7fd879fd49715860e94 
>   ql/src/test/results/clientpositive/windowing_distinct.q.out 
> 50f8ff81934c6a9338e74e9e6d4116d5f2b3ba4b 
> 
> Diff: https://reviews.apache.org/r/42858/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Aihua Xu
> 
>

[jira] [Created] (HIVE-12945) Bucket pruning: bucketing for -ve hashcodes have historical issues

2016-01-27 Thread Gopal V (JIRA) 
Gopal V created HIVE-12945:
--

 Summary: Bucket pruning: bucketing for -ve hashcodes have 
historical issues
 Key: HIVE-12945
 URL: https://issues.apache.org/jira/browse/HIVE-12945
 Project: Hive
  Issue Type: Bug
  Components: Tez
Affects Versions: 2.0.0
Reporter: Gopal V
Assignee: Gopal V


The different ETL pathways differed in reducer choice slightly for -ve 
hashcodes.

{code}
(hashCode & Integer.MAX_VALUE) % numberOfBuckets;
!=
Math.abs(hashCode) % numberOfBuckets
{code}

Add a backwards compat option, which can be used to protect against old data 
left over from 0.13.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 42777: HIVE-12550 : Cache and display last N completed queries in HS2 WebUI

2016-01-27 Thread Szehon Ho 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42777/
---

(Updated Jan. 27, 2016, 7:38 p.m.)


Review request for hive.


Changes
---

Address review comments.


Bugs: HIVE-12550
https://issues.apache.org/jira/browse/HIVE-12550


Repository: hive-git


Description
---

Use a Guava 'EvictingQueue' to cache finished SQLOperation info.  Bump up Guava 
version to get this.  The information is saved when the SQLOperation is closed.

Also fix a small discrepancy where Elapsed Time (existing field) was comparing 
against lastAccessTime , which gets updated whenever state changes and some 
other places.  Chaged to compare against operation's start time.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 97fe7bc 
  pom.xml 2066518 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 75187cf 
  service/src/java/org/apache/hive/service/cli/operation/Operation.java 113eddf 
  service/src/java/org/apache/hive/service/cli/operation/OperationManager.java 
92135cd 
  service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java 
c8a69b9 
  service/src/java/org/apache/hive/service/cli/operation/SQLOperationInfo.java 
PRE-CREATION 
  service/src/resources/hive-webapps/hiveserver2/hiveserver2.jsp a91b008 

Diff: https://reviews.apache.org/r/42777/diff/


Testing
---

Manual testing, with screenshot attached.


Thanks,

Szehon Ho

Re: [CONF] Apache Hive > Presentations

2016-01-27 Thread Sergey Shelukhin 
Btw, would it make sense to keep the page in reverse chronological order, to 
have the recent stuff on top?

From: "Xuefu Zhang (Confluence)" 
>
Date: Tuesday, January 26, 2016 at 20:11
To: Sergey Shelukhin >
Subject: [CONF] Apache Hive > Presentations

[cid:avatar_f95f0e9a808957f862efa8d84812b58b]
 Xuefu Zhang added a file to a page


[attachment 
icon]
Hive User Group Meeting Jan 
2016.pdf

[Presentations 
Icon]
  
Presentations





Stop watching 
page
 •
Manage 
notifications

[Confluence logo big]

This message was sent by Atlassian Confluence 5.8.4

Re: Review Request 42777: HIVE-12550 : Cache and display last N completed queries in HS2 WebUI

2016-01-27 Thread Aihua Xu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42777/#review116612
---


Ship it!




Ship It!

- Aihua Xu


On Jan. 27, 2016, 7:38 p.m., Szehon Ho wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42777/
> ---
> 
> (Updated Jan. 27, 2016, 7:38 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-12550
> https://issues.apache.org/jira/browse/HIVE-12550
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Use a Guava 'EvictingQueue' to cache finished SQLOperation info.  Bump up 
> Guava version to get this.  The information is saved when the SQLOperation is 
> closed.
> 
> Also fix a small discrepancy where Elapsed Time (existing field) was 
> comparing against lastAccessTime , which gets updated whenever state changes 
> and some other places.  Chaged to compare against operation's start time.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 97fe7bc 
>   pom.xml 2066518 
>   ql/src/java/org/apache/hadoop/hive/ql/Driver.java 75187cf 
>   service/src/java/org/apache/hive/service/cli/operation/Operation.java 
> 113eddf 
>   
> service/src/java/org/apache/hive/service/cli/operation/OperationManager.java 
> 92135cd 
>   service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java 
> c8a69b9 
>   
> service/src/java/org/apache/hive/service/cli/operation/SQLOperationInfo.java 
> PRE-CREATION 
>   service/src/resources/hive-webapps/hiveserver2/hiveserver2.jsp a91b008 
> 
> Diff: https://reviews.apache.org/r/42777/diff/
> 
> 
> Testing
> ---
> 
> Manual testing, with screenshot attached.
> 
> 
> Thanks,
> 
> Szehon Ho
> 
>

[jira] [Created] (HIVE-12946) alter table should also add default scheme and authority for the location similar to create table

2016-01-27 Thread Aihua Xu (JIRA) 
Aihua Xu created HIVE-12946:
---

 Summary: alter table should also add default scheme and authority 
for the location similar to create table 
 Key: HIVE-12946
 URL: https://issues.apache.org/jira/browse/HIVE-12946
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Affects Versions: 2.1.0
Reporter: Aihua Xu
Assignee: Aihua Xu


When you call create table command  (create table ... location '/abc') and 
check the database, you will notice that the table location has the scheme. 
While alter table set location '/abc' would throw an error "{0}  is not 
absolute or has no scheme information.  Please specify a complete absolute uri 
with scheme information. /abc.

Seems it should be consistent.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 42777: HIVE-12550 : Cache and display last N completed queries in HS2 WebUI

2016-01-27 Thread Mohit Sabharwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42777/#review116611
---


Ship it!




Ship It!

- Mohit Sabharwal


On Jan. 27, 2016, 7:38 p.m., Szehon Ho wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42777/
> ---
> 
> (Updated Jan. 27, 2016, 7:38 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-12550
> https://issues.apache.org/jira/browse/HIVE-12550
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Use a Guava 'EvictingQueue' to cache finished SQLOperation info.  Bump up 
> Guava version to get this.  The information is saved when the SQLOperation is 
> closed.
> 
> Also fix a small discrepancy where Elapsed Time (existing field) was 
> comparing against lastAccessTime , which gets updated whenever state changes 
> and some other places.  Chaged to compare against operation's start time.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 97fe7bc 
>   pom.xml 2066518 
>   ql/src/java/org/apache/hadoop/hive/ql/Driver.java 75187cf 
>   service/src/java/org/apache/hive/service/cli/operation/Operation.java 
> 113eddf 
>   
> service/src/java/org/apache/hive/service/cli/operation/OperationManager.java 
> 92135cd 
>   service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java 
> c8a69b9 
>   
> service/src/java/org/apache/hive/service/cli/operation/SQLOperationInfo.java 
> PRE-CREATION 
>   service/src/resources/hive-webapps/hiveserver2/hiveserver2.jsp a91b008 
> 
> Diff: https://reviews.apache.org/r/42777/diff/
> 
> 
> Testing
> ---
> 
> Manual testing, with screenshot attached.
> 
> 
> Thanks,
> 
> Szehon Ho
> 
>

Re: [CONF] Apache Hive > Presentations

2016-01-27 Thread Szehon Ho 
Yea makes sense to me (esp as "Older Presentations" are already at the
bottom).  I thought it had been in reverse order at one point.

On Wed, Jan 27, 2016 at 11:41 AM, Sergey Shelukhin 
wrote:

> Btw, would it make sense to keep the page in reverse chronological order,
> to have the recent stuff on top?
>
> From: "Xuefu Zhang (Confluence)" 
> Date: Tuesday, January 26, 2016 at 20:11
> To: Sergey Shelukhin 
> Subject: [CONF] Apache Hive > Presentations
>
>
> 
>  Xuefu
> Zhang *added* a file to a page
>
> [image: attachment icon]
> 
>  Hive
> User Group Meeting Jan 2016.pdf
> 
> [image: Presentations Icon]
> 
> Presentations
> 
>
> Stop watching page
> 
> •
> Manage notifications
> 
> [image: Confluence logo big]
> This message was sent by Atlassian Confluence 5.8.4
>

Re: Review Request 42858: HIVE-12944: Support SUM(DISTINCT) for partitioning query.

2016-01-27 Thread Aihua Xu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42858/
---

(Updated Jan. 27, 2016, 7:52 p.m.)


Review request for hive and Szehon Ho.


Repository: hive-git


Description
---

HIVE-12944: Support SUM(DISTINCT) for partitioning query.


Diffs (updated)
-

  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/functions/HiveSqlSumAggFunction.java
 056eaeb 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/SqlFunctionConverter.java
 75c38fa 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCount.java 
f526c43 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 0968008 
  ql/src/test/queries/clientpositive/windowing_distinct.q 94f4044 
  ql/src/test/results/clientpositive/windowing_distinct.q.out 50f8ff8 

Diff: https://reviews.apache.org/r/42858/diff/


Testing
---


Thanks,

Aihua Xu

[jira] [Created] (HIVE-12952) Show query sub-pages on webui

2016-01-27 Thread Szehon Ho (JIRA) 
Szehon Ho created HIVE-12952:


 Summary: Show query sub-pages on webui
 Key: HIVE-12952
 URL: https://issues.apache.org/jira/browse/HIVE-12952
 Project: Hive
  Issue Type: Sub-task
  Components: Diagnosability
Reporter: Szehon Ho
Assignee: Szehon Ho


Today there the queries showing in Running and Completed lists have some basic 
information like query string, elapsed time, state, user, etc.

It may be helpful to have even more information like:
* Job URL's, job status
* Explain plan (configurable)
* Error message (if failure)
* Dynamic metrics, incl:
 ** Number of Tables/partitions fetched
 ** Time taken in each method via perf-logger.

This JIRA aims to tackle some of those.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 41928: HIVE-12777: Add capability to restore session

2016-01-27 Thread Amareshwari Sriramadasu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41928/#review116580
---




service/src/java/org/apache/hive/service/cli/CLIService.java (line 207)


Shall we remove the return from the method, which is not adding any?



service/src/java/org/apache/hive/service/cli/session/SessionManager.java (line 
311)


Why is the ipAddress variable changed?



service/src/test/org/apache/hive/service/cli/CLIServiceRestoreTest.java (line 
36)


Do a stop() after the test?


- Amareshwari Sriramadasu


On Jan. 21, 2016, 1:39 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41928/
> ---
> 
> (Updated Jan. 21, 2016, 1:39 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-12777
> https://issues.apache.org/jira/browse/HIVE-12777
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Extensions using Hive session handles should be able to restore the hive 
> session from the handle. 
> 
> Apache Lens depends on a fork of hive and that fork has such a capability. 
> 
> Relevant commit: 
> https://github.com/InMobi/hive/commit/931fe9116161a18952c082c14223ad6745fefe00#diff-0acb35f7cab7492f522b0c40ce3ce1be
> 
> 
> Diffs
> -
> 
>   service/src/java/org/apache/hive/service/cli/CLIService.java 
> 0d5ae1e491bdcf98a12f5eef7b35f23c7b9dfc00 
>   service/src/java/org/apache/hive/service/cli/SessionHandle.java 
> 4e4f0211de09408168c8da669626fb49f3db364c 
>   service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 
> 3c5700bd114f6b792df10df23e57dad9b9e689f6 
>   
> service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java
>  c56d97afc969ea57810e9b3222b20e086e847d67 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
> de18f1d93502704e7a8fbb8d31d93bd770c4d232 
>   service/src/test/org/apache/hive/service/cli/CLIServiceRestoreTest.java 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/41928/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

[jira] [Created] (HIVE-12943) Use default doesnot work in Hive 1.2.1

2016-01-27 Thread Kashish Jain (JIRA) 
Kashish Jain created HIVE-12943:
---

 Summary: Use default doesnot work in Hive 1.2.1
 Key: HIVE-12943
 URL: https://issues.apache.org/jira/browse/HIVE-12943
 Project: Hive
  Issue Type: Bug
  Components: Database/Schema
Affects Versions: 1.2.1
Reporter: Kashish Jain
 Fix For: 1.3.0, 1.2.2, 1.2.1


"USE Default" does not work with the latest hive 1.2.1
The message is 
"
   Cannot recognize input near 'default' '' '' in switch database 
statement; line 1 pos 4
   NoViableAltException(81@[])
   at 
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier(HiveParser_IdentifiersParser.java:11577)
   at 
org.apache.hadoop.hive.ql.parse.HiveParser.identifier(HiveParser.java:46055)
"

According to my analysis this may be due to the removal of "KW_DEFAULT" keyword 
from the grammar of Hive parser according to the bug 
https://issues.apache.org/jira/browse/HIVE-6617

Adding the KW_DEFAULT keyword back to the HiveParser.g, HiveLexer.g and 
Identifier.g and It worked. Wish to push it to apache-hive



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 41928: HIVE-12777: Add capability to restore session

2016-01-27 Thread Rajat Khandelwal 


> On Jan. 27, 2016, 5:50 p.m., Amareshwari Sriramadasu wrote:
> > service/src/java/org/apache/hive/service/cli/CLIService.java, line 207
> > 
> >
> > Shall we remove the return from the method, which is not adding any?

yeah we can remove. right now it's getting used in test cases to compare return 
value with the value passed.


> On Jan. 27, 2016, 5:50 p.m., Amareshwari Sriramadasu wrote:
> > service/src/java/org/apache/hive/service/cli/session/SessionManager.java, 
> > line 311
> > 
> >
> > Why is the ipAddress variable changed?

Seems a mistake. Added back.


- Rajat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41928/#review116580
---


On Jan. 21, 2016, 7:09 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41928/
> ---
> 
> (Updated Jan. 21, 2016, 7:09 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-12777
> https://issues.apache.org/jira/browse/HIVE-12777
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Extensions using Hive session handles should be able to restore the hive 
> session from the handle. 
> 
> Apache Lens depends on a fork of hive and that fork has such a capability. 
> 
> Relevant commit: 
> https://github.com/InMobi/hive/commit/931fe9116161a18952c082c14223ad6745fefe00#diff-0acb35f7cab7492f522b0c40ce3ce1be
> 
> 
> Diffs
> -
> 
>   service/src/java/org/apache/hive/service/cli/CLIService.java 
> 0d5ae1e491bdcf98a12f5eef7b35f23c7b9dfc00 
>   service/src/java/org/apache/hive/service/cli/SessionHandle.java 
> 4e4f0211de09408168c8da669626fb49f3db364c 
>   service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 
> 3c5700bd114f6b792df10df23e57dad9b9e689f6 
>   
> service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java
>  c56d97afc969ea57810e9b3222b20e086e847d67 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
> de18f1d93502704e7a8fbb8d31d93bd770c4d232 
>   service/src/test/org/apache/hive/service/cli/CLIServiceRestoreTest.java 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/41928/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>