[jira] [Created] (HIVE-5123) group by on a same key producing wrong result
cyril liao created HIVE-5123: Summary: group by on a same key producing wrong result Key: HIVE-5123 URL: https://issues.apache.org/jira/browse/HIVE-5123 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.11.0 Reporter: cyril liao grouping by on a same key twice will run a single mapreduce-job,producing wrong result. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-5124) group by without map aggregation lead to mapreduce exception
cyril liao created HIVE-5124: Summary: group by without map aggregation lead to mapreduce exception Key: HIVE-5124 URL: https://issues.apache.org/jira/browse/HIVE-5124 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.11.0 Reporter: cyril liao On my environment, the same query but diffent by seting hive.map.aggr with true or flase,produce different result. With hive.map.aggr=false,tasktracker report the following exception: java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:485) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:420) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083) at org.apache.hadoop.mapred.Child.main(Child.java:249) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ... 9 more Caused by: java.lang.RuntimeException: Reduce operator initialization failed at org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:160) ... 14 more Caused by: java.lang.RuntimeException: cannot find field value from [0:_col0, 1:_col1, 2:_col2, 3:_col3] at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:366) at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:143) at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:82) at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:299) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407) at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:438) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:153) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Implement directory/table level access control in HDFS
Hi folks,
HDFS has a POSIX-like permission model, using R,W,X and owner, group, other
for access control. It is good most of the time, except for:
1. Data need to be shared among users
group can be used for access control, and the users has to be in the same
GROUP as the data. the GROUP here stand for the sharing relationship
between users and data. If many sharing relationships exists, there are
many groups. It is hard to manage.
2. Hive
Hive use a table based access control model, user can have SELECT, UPDATE,
CREATE, DROP privileges on certain table, which means R/W permission in
HDFS. However, Hive’s table based authorization doesn’t match HDFS’s
POSIX-like model. For hive user accessing HDFS, Group permissions can be
deployed, which introduces many groups, or big groups contains many sharing
relationship.
Inspired by RDBMS’s way of manage data, a directory level access control
based on authorized user impersonate can be implemented as a extension to
POSIX-like permission model.
it consist of:
1. ACLFileSystem
2. authorization manager: hold access control information and a shared
secret with namenode
3. authenticator(embedded in namenode)
Take hive as a example, owner of the data is user DW. The procedure is:
1. user submit a hive query or a hcatalog job to access DW’s data, we can
get the read table/partition and write table/partition, and the
corresponding hdfs path. Then a RPC call to authorization manager is
invoked, send
{user, tablename, table_path, w/r}
2. authorization manager do a authorization check to find whether it is
allowed. If allowed, reply a encrypted tablepath:
{realuser, encrypted(tablepath+w/r)}
realuser here stand for the owner of the requested data
3. ACLFilesystem extends FileSystem and when a open(path) call is invoked ,
it replace the path to encrypted(tablepath+w/r) and invoke the namenode RPC
call, such as
open(realuser, encrypted(tablepath+w/r), null)
If the user is requesting a partition path, the rpc call can be invoked as
open(realuser, encrypted(tablepath+w/r), path_suffix)
4. Namenode pick up the RPC call, decrypt the encrypted(hdfspath+w/r) with
the shared secret to verify whether it is fake. If it is true, check w/r
operation, join the tablepath and path_suffix, and invoke the call as
hdfspath owner, for example user DW.
delegation token or something else can be used as the shared secret, and
authorization manager can be integrated into hive metastore.
In general, I propose a HDFS user impersonate mechanism and a authorization
mechanism based on HDFS user impersonation.
If the community is interested, I will file a jira for HDFS user
impersonation and a jira for authorization manager soon.
Thoughts?
Thanks a lot
Erik.fang
[jira] [Commented] (HIVE-5022) Decimal Arithmetic generates NULL value
[
https://issues.apache.org/jira/browse/HIVE-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13744769#comment-13744769
]
Hive QA commented on HIVE-5022:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12598711/HIVE-5022.3.patch.txt
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 2885 tests executed
*Failed tests:*
{noformat}
org.apache.hcatalog.listener.TestNotificationListener.testAMQListener
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/483/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/483/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
Decimal Arithmetic generates NULL value
---
Key: HIVE-5022
URL: https://issues.apache.org/jira/browse/HIVE-5022
Project: Hive
Issue Type: Bug
Components: Types
Affects Versions: 0.11.0
Environment: Hortonworks 1.3 running Hive 0.11.0.1.3.0.0-107
Reporter: Kevin Soo Hoo
Assignee: Teddy Choi
Attachments: HIVE-5022.1.patch.txt, HIVE-5022.2.patch.txt,
HIVE-5022.3.patch.txt
When a decimal division is the first operation, the quotient cannot be
multiplied in a subsequent calculation. Instead, a NULL is returned.
The following yield NULL results:
select (cast (4.53 as decimal) / cast(25.86 as decimal)) * cast(0.087 as
decimal) from tablename limit 1;
select cast (4.53 as decimal) / cast(25.86 as decimal) * cast(0.087 as
decimal) from tablename limit 1;
If we move the multiplication operation to be first, then it will
successfully calculate the result.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4617) ExecuteStatementAsync call to run a query in non-blocking mode
[ https://issues.apache.org/jira/browse/HIVE-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-4617: -- Attachment: HIVE-4617.D12417.1.patch vaibhavgumashta requested code review of HIVE-4617 [jira] ExecuteStatementAsync call to run a query in non-blocking mode. Reviewers: JIRA (Based on the code in HIVE-4569 published by Jaideep Dhok) Provide a way to run a queries asynchronously. Current executeStatement call blocks until the query run is complete. TEST PLAN Unit Tests for async execute REVISION DETAIL https://reviews.facebook.net/D12417 AFFECTED FILES common/src/java/org/apache/hadoop/hive/conf/HiveConf.java metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote service/if/TCLIService.thrift service/src/gen/thrift/gen-cpp/TCLIService.cpp service/src/gen/thrift/gen-cpp/TCLIService.h service/src/gen/thrift/gen-cpp/TCLIService_server.skeleton.cpp service/src/gen/thrift/gen-cpp/TCLIService_types.cpp service/src/gen/thrift/gen-cpp/TCLIService_types.h service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCLIService.java service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TExecuteStatementAsyncReq.java service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TExecuteStatementAsyncResp.java service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetTablesReq.java service/src/gen/thrift/gen-php/TCLIService.php service/src/gen/thrift/gen-py/TCLIService/TCLIService-remote service/src/gen/thrift/gen-py/TCLIService/TCLIService.py service/src/gen/thrift/gen-py/TCLIService/ttypes.py service/src/gen/thrift/gen-py/hive_service/ThriftHive-remote service/src/gen/thrift/gen-rb/t_c_l_i_service.rb service/src/gen/thrift/gen-rb/t_c_l_i_service_types.rb service/src/java/org/apache/hive/service/cli/CLIService.java service/src/java/org/apache/hive/service/cli/CLIServiceClient.java service/src/java/org/apache/hive/service/cli/EmbeddedCLIServiceClient.java service/src/java/org/apache/hive/service/cli/ICLIService.java service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java service/src/java/org/apache/hive/service/cli/session/HiveSession.java service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java service/src/java/org/apache/hive/service/cli/session/SessionManager.java service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIServiceClient.java service/src/test/org/apache/hive/service/cli/CLIServiceTest.java MANAGE HERALD RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/29655/ To: JIRA, vaibhavgumashta ExecuteStatementAsync call to run a query in non-blocking mode -- Key: HIVE-4617 URL: https://issues.apache.org/jira/browse/HIVE-4617 Project: Hive Issue Type: Improvement Components: HiveServer2 Affects Versions: 0.11.0 Reporter: Jaideep Dhok Assignee: Jaideep Dhok Attachments: HIVE-4617.D12417.1.patch Provide a way to run a queries asynchronously. Current executeStatement call blocks until the query run is complete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4617) ExecuteStatementAsync call to run a query in non-blocking mode
[ https://issues.apache.org/jira/browse/HIVE-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13744771#comment-13744771 ] Vaibhav Gumashta commented on HIVE-4617: These are changes from HIVE-4569 specific to the async execution API. Adding here for a more focussed discussion. ExecuteStatementAsync call to run a query in non-blocking mode -- Key: HIVE-4617 URL: https://issues.apache.org/jira/browse/HIVE-4617 Project: Hive Issue Type: Improvement Components: HiveServer2 Affects Versions: 0.11.0 Reporter: Jaideep Dhok Assignee: Jaideep Dhok Attachments: HIVE-4617.D12417.1.patch Provide a way to run a queries asynchronously. Current executeStatement call blocks until the query run is complete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4645) Stat information like numFiles and totalSize is not correct when sub-directory is exists
[
https://issues.apache.org/jira/browse/HIVE-4645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13744802#comment-13744802
]
Hive QA commented on HIVE-4645:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12598893/HIVE-4645.D11037.4.patch
{color:green}SUCCESS:{color} +1 2885 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/484/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/484/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
Stat information like numFiles and totalSize is not correct when
sub-directory is exists
Key: HIVE-4645
URL: https://issues.apache.org/jira/browse/HIVE-4645
Project: Hive
Issue Type: Test
Components: Statistics
Reporter: Navis
Assignee: Navis
Priority: Trivial
Fix For: 0.12.0
Attachments: HIVE-4645.D11037.1.patch, HIVE-4645.D11037.2.patch,
HIVE-4645.D11037.3.patch, HIVE-4645.D11037.4.patch
The test infer_bucket_sort_list_bucket.q returns 4096 as totalSize but it's
size of parent directory, not sum of file size.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5019) Use StringBuffer instead of += (issue 1)
[ https://issues.apache.org/jira/browse/HIVE-5019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13744803#comment-13744803 ] Benjamin Jakobus commented on HIVE-5019: Done. Use StringBuffer instead of += (issue 1) Key: HIVE-5019 URL: https://issues.apache.org/jira/browse/HIVE-5019 Project: Hive Issue Type: Sub-task Reporter: Benjamin Jakobus Assignee: Benjamin Jakobus Fix For: 0.12.0 Attachments: HIVE-5019.2.patch.txt, HIVE-5019.3.patch.txt Issue 1 - use of StringBuilder over += inside loops. java/org/apache/hadoop/hive/ql/optimizer/physical/GenMRSkewJoinProcessor.java java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java java/org/apache/hadoop/hive/ql/parse/PTFTranslator.java java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java java/org/apache/hadoop/hive/ql/plan/ConditionalResolverMergeFiles.java java/org/apache/hadoop/hive/ql/plan/PlanUtils.java java/org/apache/hadoop/hive/ql/security/authorization/BitSetCheckedAuthorizationProvider.java java/org/apache/hadoop/hive/ql/stats/jdbc/JDBCStatsUtils.java java/org/apache/hadoop/hive/ql/udf/UDFLike.java java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSentences.java java/org/apache/hadoop/hive/ql/udf/generic/NumDistinctValueEstimator.java java/org/apache/hadoop/hive/ql/udf/ptf/NPath.java -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13744814#comment-13744814 ] Teddy Choi commented on HIVE-1555: -- [~ashutoshc], I want to take it. But I'm not sure whether I can do it. I'll take a look. JDBC Storage Handler Key: HIVE-1555 URL: https://issues.apache.org/jira/browse/HIVE-1555 Project: Hive Issue Type: New Feature Components: JDBC Reporter: Bob Robertson Assignee: Andrew Wilson Attachments: JDBCStorageHandler Design Doc.pdf Original Estimate: 24h Remaining Estimate: 24h With the Cassandra and HBase Storage Handlers I thought it would make sense to include a generic JDBC RDBMS Storage Handler so that you could import a standard DB table into Hive. Many people must want to perform HiveQL joins, etc against tables in other systems etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4779) Enhance coverage of package org.apache.hadoop.hive.ql.udf
[ https://issues.apache.org/jira/browse/HIVE-4779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13744815#comment-13744815 ] Ivan A. Veselovsky commented on HIVE-4779: -- I see you rebased the patch, thanks a lot. Enhance coverage of package org.apache.hadoop.hive.ql.udf - Key: HIVE-4779 URL: https://issues.apache.org/jira/browse/HIVE-4779 Project: Hive Issue Type: Test Affects Versions: 0.12.0 Reporter: Ivan A. Veselovsky Assignee: Ivan A. Veselovsky Attachments: HIVE-4779.patch, HIVE-4779-trunk--N1.patch Enhance coverage of package org.apache.hadoop.hive.ql.udf up to 80%. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-664) optimize UDF split
[ https://issues.apache.org/jira/browse/HIVE-664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi reassigned HIVE-664: --- Assignee: Teddy Choi optimize UDF split -- Key: HIVE-664 URL: https://issues.apache.org/jira/browse/HIVE-664 Project: Hive Issue Type: Bug Components: UDF Reporter: Namit Jain Assignee: Teddy Choi Labels: optimization Min Zhou added a comment - 21/Jul/09 07:34 AM It's very useful for us . some comments: 1. Can you implement it directly with Text ? Avoiding string decoding and encoding would be faster. Of course that trick may lead to another problem, as String.split uses a regular expression for splitting. 2. getDisplayString() always return a string in lowercase. [ Show » ] Min Zhou added a comment - 21/Jul/09 07:34 AM It's very useful for us . some comments: 1. Can you implement it directly with Text ? Avoiding string decoding and encoding would be faster. Of course that trick may lead to another problem, as String.split uses a regular expression for splitting. 2. getDisplayString() always return a string in lowercase. [ Permlink | « Hide ] Namit Jain added a comment - 21/Jul/09 09:22 AM Committed. Thanks Emil [ Show » ] Namit Jain added a comment - 21/Jul/09 09:22 AM Committed. Thanks Emil [ Permlink | « Hide ] Emil Ibrishimov added a comment - 21/Jul/09 10:48 AM There are some easy (compromise) ways to optimize split: 1. Check if the regex argument actually contains some regex specific characters and if it doesn't, do a straightforward split without converting to strings. 2. Assume some default value for the second argument (for example - split(str) to be equivalent to split(str, ' ') and optimize for this value 3. Have two separate split functions - one that does regex and one that splits around plain text. I think that 1 is a good choice and can be done rather quickly. [ Show » ] Emil Ibrishimov added a comment - 21/Jul/09 10:48 AM There are some easy (compromise) ways to optimize split: 1. Check if the regex argument actually contains some regex specific characters and if it doesn't, do a straightforward split without converting to strings. 2. Assume some default value for the second argument (for example - split(str) to be equivalent to split(str, ' ') and optimize for this value 3. Have two separate split functions - one that does regex and one that splits around plain text. I think that 1 is a good choice and can be done rather quickly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[ANNOUNCE] New Hive Committer - Thejas Nair
The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas!
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congrats Thejas! On Aug 20, 2013 3:32 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas! -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-5121) Remove obsolete code on SemanticAnalyzer#genJoinTree
[
https://issues.apache.org/jira/browse/HIVE-5121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13744845#comment-13744845
]
Hive QA commented on HIVE-5121:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12598901/HIVE-5121.D12405.1.patch
{color:green}SUCCESS:{color} +1 2885 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/485/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/485/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
Remove obsolete code on SemanticAnalyzer#genJoinTree
Key: HIVE-5121
URL: https://issues.apache.org/jira/browse/HIVE-5121
Project: Hive
Issue Type: Improvement
Affects Versions: 0.11.0
Environment: ubuntu 12.04
Reporter: Azrael Park
Assignee: Azrael Park
Priority: Trivial
Attachments: HIVE-5121.D12405.1.patch
Remove obsolete code on SemanticAnalyzer#genJoinTree.
{noformat}
children[1] = alias;
joinTree.setBaseSrc(children);
- aliasToOpInfo.get(alias);
joinTree.setId(qb.getId());
joinTree.getAliasToOpInfo().put(
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-5124) group by without map aggregation lead to mapreduce exception
[ https://issues.apache.org/jira/browse/HIVE-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bing Li reassigned HIVE-5124: - Assignee: Bing Li group by without map aggregation lead to mapreduce exception Key: HIVE-5124 URL: https://issues.apache.org/jira/browse/HIVE-5124 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.11.0 Reporter: cyril liao Assignee: Bing Li On my environment, the same query but diffent by seting hive.map.aggr with true or flase,produce different result. With hive.map.aggr=false,tasktracker report the following exception: java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:485) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:420) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083) at org.apache.hadoop.mapred.Child.main(Child.java:249) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ... 9 more Caused by: java.lang.RuntimeException: Reduce operator initialization failed at org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:160) ... 14 more Caused by: java.lang.RuntimeException: cannot find field value from [0:_col0, 1:_col1, 2:_col2, 3:_col3] at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:366) at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:143) at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:82) at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:299) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407) at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:438) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:153) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congrats Thejas! 在 2013年8月20日星期二，Carl Steinbach 写道： The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas!
[jira] [Updated] (HIVE-4943) An explode function that includes the item's position in the array
[ https://issues.apache.org/jira/browse/HIVE-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niko Stahl updated HIVE-4943: - Attachment: HIVE-4943.2.patch An explode function that includes the item's position in the array -- Key: HIVE-4943 URL: https://issues.apache.org/jira/browse/HIVE-4943 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.11.0 Reporter: Niko Stahl Labels: patch Fix For: 0.11.0 Attachments: HIVE-4943.1.patch, HIVE-4943.2.patch Original Estimate: 8h Remaining Estimate: 8h A function that explodes an array and includes an output column with the position of each item in the original array. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4943) An explode function that includes the item's position in the array
[ https://issues.apache.org/jira/browse/HIVE-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13744888#comment-13744888 ] Niko Stahl commented on HIVE-4943: -- Yes, it's quite specific but very useful when the ordering of the original array matters. I've uploaded a new patch that should pass those failing tests. Thanks for your feedback. An explode function that includes the item's position in the array -- Key: HIVE-4943 URL: https://issues.apache.org/jira/browse/HIVE-4943 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.11.0 Reporter: Niko Stahl Labels: patch Fix For: 0.11.0 Attachments: HIVE-4943.1.patch, HIVE-4943.2.patch Original Estimate: 8h Remaining Estimate: 8h A function that explodes an array and includes an output column with the position of each item in the original array. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4214) OVER accepts general expression instead of just function
[
https://issues.apache.org/jira/browse/HIVE-4214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13744920#comment-13744920
]
Hive QA commented on HIVE-4214:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12598902/HIVE-4214.3.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 2887 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/486/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/486/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
OVER accepts general expression instead of just function
Key: HIVE-4214
URL: https://issues.apache.org/jira/browse/HIVE-4214
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Affects Versions: 0.11.0
Reporter: Alan Gates
Assignee: Ashutosh Chauhan
Attachments: HIVE-4214.1.patch, HIVE-4214.3.patch, HIVE-4214.patch
The query:
select s, i, avg(d) / 10.0 over (partition by s order by i) from over100k;
runs (and produces meaningless output).
Over should not allow the arithmetic expression. Only a UDAF or PTF function
should be valid there. The correct way to write this query should be
select s, i, avg(d) over (partition by s order by i) / 10. 0 from over100k;
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: No java compiler available exception for HWI
Hi, Eric et al Did you resolve this failure? I'm using Hive-0.11.0, and get the same error when access to HWI via browser. I already set the following properties in hive-site.xml - hive.hwi.listen.host - hive.hwi.listen.port - hive.hwi.war.file And copied two jasper jars into hive/lib: - jasper-compiler-5.5.23.jar - jasper-runtime-5.5.23.jar Thanks, - Bing 2013/8/20 Bing Li sarah.lib...@gmail.com Hi, Eric et al Did you resolve this failure? I'm using Hive-0.11.0, and get the same error when access to HWI via browser. I already set the following properties in hive-site.xml - hive.hwi.listen.host - hive.hwi.listen.port - hive.hwi.war.file And copied two jasper jars into hive/lib: - jasper-compiler-5.5.23.jar - jasper-runtime-5.5.23.jar Thanks, - Bing 2013/3/30 Eric Chu e...@rocketfuel.com Hi, I'm running Hive 0.10 and I want to support HWI (besides CLI and HUE). When I started HWI I didn't get any error. However, when I went to Hive Server Address:/hwi on my browser I saw the error below complaining about No Java compiler available. My JAVA_HOME is set to /usr/lib/jvm/java-1.6.0-sun-1.6.0.16. Besides https://cwiki.apache.org/Hive/hivewebinterface.html, there's not much documentation on HWI. I'm wondering if anyone else has seen this or has any idea about what's wrong? Thanks. Eric Problem accessing /hwi/. Reason: No Java compiler available Caused by: java.lang.IllegalStateException: No Java compiler available at org.apache.jasper.JspCompilationContext.createCompiler(JspCompilationContext.java:225) at org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:560) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:299) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:315) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:265) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:327) at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:126) at org.mortbay.jetty.servlet.DefaultServlet.doGet(DefaultServlet.java:503) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.handler.RequestLogHandler.handle(RequestLogHandler.java:49) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congratulations! 2013/8/20 Clark Yang (杨卓荦) yangzhuo...@gmail.com: Congrats Thejas! 在 2013年8月20日星期二，Carl Steinbach 写道： The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas!
[jira] [Updated] (HIVE-5111) ExprNodeColumnDesc doesn't distinguish partition and virtual columns, causing partition pruner to receive the latter
[ https://issues.apache.org/jira/browse/HIVE-5111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-5111: Resolution: Fixed Fix Version/s: 0.12.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Sergey! ExprNodeColumnDesc doesn't distinguish partition and virtual columns, causing partition pruner to receive the latter Key: HIVE-5111 URL: https://issues.apache.org/jira/browse/HIVE-5111 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.12.0 Attachments: HIVE-5111.D12393.1.patch See HIVE-3926 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5106) HCatFieldSchema overrides equals() but not hashCode()
[ https://issues.apache.org/jira/browse/HIVE-5106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-5106: Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Eugene! HCatFieldSchema overrides equals() but not hashCode() - Key: HIVE-5106 URL: https://issues.apache.org/jira/browse/HIVE-5106 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Fix For: 0.12.0 Attachments: HIVE-5106.patch It's likely that objects of this type are not hashed today but would lead to very nasty bugs if they ever will be. Looks like it was introduced in HCATALOG-438. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4978) [WebHCat] Close the PrintWriter after writing data
[ https://issues.apache.org/jira/browse/HIVE-4978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-4978: Status: Open (was: Patch Available) Should close the writer only if it's writing on file. [WebHCat] Close the PrintWriter after writing data -- Key: HIVE-4978 URL: https://issues.apache.org/jira/browse/HIVE-4978 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.11.0 Reporter: Venki Korukanti Priority: Minor Fix For: 0.12.0 Attachments: HIVE-4978-1.patch We are not closing the PrintWriter after writing data into it. I haven't seen any problems so far, but it is good to close the PrintWriter so that resources are released properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congratulations, Thejas! On Tue, Aug 20, 2013 at 5:43 AM, Navis류승우 navis@nexr.com wrote: Congratulations! 2013/8/20 Clark Yang (杨卓荦) yangzhuo...@gmail.com: Congrats Thejas! 在 2013年8月20日星期二，Carl Steinbach 写道： The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas!
[jira] [Commented] (HIVE-5122) Add partition for multiple partition ignores locations for non-first partitions
[
https://issues.apache.org/jira/browse/HIVE-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13744982#comment-13744982
]
Hive QA commented on HIVE-5122:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12598904/HIVE-5122.D12411.1.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 2885 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view_partitioned
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/487/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/487/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
Add partition for multiple partition ignores locations for non-first
partitions
---
Key: HIVE-5122
URL: https://issues.apache.org/jira/browse/HIVE-5122
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-5122.D12411.1.patch
http://www.mail-archive.com/user@hive.apache.org/msg09151.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congrats Thejas. Where's the Party tonight !! :P On Tue, Aug 20, 2013 at 5:43 PM, Navis류승우 navis@nexr.com wrote: Congratulations! 2013/8/20 Clark Yang (杨卓荦) yangzhuo...@gmail.com: Congrats Thejas! 在 2013年8月20日星期二，Carl Steinbach 写道： The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas! -- *Muhammad Hamza Asad*
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13744998#comment-13744998 ] Ashutosh Chauhan commented on HIVE-1555: [~teddy.choi] I think you can take it up. I am not seeing any activity from previous contributors of this jira. JDBC Storage Handler Key: HIVE-1555 URL: https://issues.apache.org/jira/browse/HIVE-1555 Project: Hive Issue Type: New Feature Components: JDBC Reporter: Bob Robertson Assignee: Andrew Wilson Attachments: JDBCStorageHandler Design Doc.pdf Original Estimate: 24h Remaining Estimate: 24h With the Cassandra and HBase Storage Handlers I thought it would make sense to include a generic JDBC RDBMS Storage Handler so that you could import a standard DB table into Hive. Many people must want to perform HiveQL joins, etc against tables in other systems etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [Discuss] project chop up
Just an update. This is going very well: NFO] Nothing to compile - all classes are up to date [INFO] [INFO] Reactor Summary: [INFO] [INFO] Apache Hive ... SUCCESS [0.002s] [INFO] hive-shims-x .. SUCCESS [1.210s] [INFO] hive-shims-20 . SUCCESS [0.125s] [INFO] hive-common ... SUCCESS [0.082s] [INFO] hive-serde SUCCESS [2.521s] [INFO] hive-metastore SUCCESS [10.818s] [INFO] hive-exec . SUCCESS [4.521s] [INFO] hive-avro . SUCCESS [1.582s] [INFO] hive-zookeeper SUCCESS [0.519s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 21.613s [INFO] Finished at: Tue Aug 20 10:23:34 EDT 2013 [INFO] Final Memory: 39M/408M Though I did some short cuts and disabled some tests. We can build hive very fast, including incremental builds. Also we are using maven plugins to compile antlr, thrift, protobuf, datanucleas and building those every time. On Fri, Aug 16, 2013 at 11:16 PM, Xuefu Zhang xzh...@cloudera.com wrote: Thanks, Edward. I'm big +1 to mavenize Hive. Hive has long reached a point where it's hard to manage its build using ant. I'd like to help on this too. Thanks, Xuefu On Fri, Aug 16, 2013 at 7:31 PM, Edward Capriolo edlinuxg...@gmail.com wrote: For those interested in pitching in. https://github.com/edwardcapriolo/hive On Fri, Aug 16, 2013 at 11:58 AM, Edward Capriolo edlinuxg...@gmail.com wrote: Summary from hive-irc channel. Minor edits for spell check/grammar. The last 10 lines are a summary of the key points. [10:59:17] ecapriolo noland: et all. Do you want to talk about hive in maven? [11:01:06] smonchi [~ ro...@host34-189-dynamic.23-79-r.retail.telecomitalia.it] has quit IRC: Quit: ... 'cause there is no patch for human stupidity ... [11:10:04] noland ecapriolo: yeah that sounds good to me! [11:10:22] noland I saw you created the jira but haven't had time to look [11:10:32] ecapriolo So I found a few things [11:10:49] ecapriolo In common there is one or two testats that actually fork a process :) [11:10:56] ecapriolo and use build.test.resources [11:11:12] ecapriolo Some serde, uses some methods from ql in testing [11:11:27] ecapriolo and shims really needs a separate hadoop test shim [11:11:32] ecapriolo But that is all simple stuff [11:11:47] ecapriolo The biggest problem is I do not know how to solve shims with maven [11:11:50] ecapriolo do you have any ideas [11:11:52] ecapriolo ? [11:13:00] noland That one is going to be a challenge. It might be that in that section we have to drop down to ant [11:14:44] noland Is it a requirement that we build both the .20 and .23 shims for a package as we do today? [11:16:46] ecapriolo I was thinking we can do it like a JDBC driver [11:16:59] ecapriolo Se separate out the interface of shims [11:17:22] ecapriolo And then at runtime we drop in a driver implementing [11:17:34] Wertax [~wer...@wolfkamp.xs4all.nl] has quit IRC: Remote host closed the connection [11:17:36] ecapriolo That or we could use maven's profile system [11:18:09] ecapriolo It seems that everything else can actually link against hadoop-0.20.2 as a provided dependency [11:18:37] noland Yeah either would work. The driver method would probably require use to use ant build both the drivers? [11:18:44] noland I am a fan of mvn profiles [11:19:05] ecapriolo I was thinking we kinda separate the shim out into its own project,, not a module [11:19:10] ecapriolo to achive that jdbc thing [11:19:27] ecapriolo But I do not have a solution yet, I was looking to farm that out to someone smart...like you :) [11:19:33] noland :) [11:19:47] ecapriolo All I know is that we need a test shim because HadoopShim requires hadoop-test jars [11:20:10] ecapriolo then the Mini stuff is only used in qtest anyway [11:20:48] ecapriolo Is this something you want to help with? I was thinking of spinning up a github [11:20:50] noland I think that the separate projects would work and perhaps nicely. [11:21:01] noland Yeah I'd be interested in helping! [11:21:17] noland But I am going on vacation starting next week for about 10 days [11:21:27] ecapriolo Ah cool where are you going? [11:21:37] noland Netherlands [11:21:42] noland Biking around and such [11:23:52] noland The one thing I was thinking about with regards to a branch is keeping history. We'll want to
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congratulations Thejas! On Tue, Aug 20, 2013 at 9:06 AM, Xuefu Zhang xzh...@cloudera.com wrote: Congratulations, Thejas! On Tue, Aug 20, 2013 at 5:43 AM, Navis류승우 navis@nexr.com wrote: Congratulations! 2013/8/20 Clark Yang (杨卓荦) yangzhuo...@gmail.com: Congrats Thejas! 在 2013年8月20日星期二，Carl Steinbach 写道： The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas! -- Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congratulations Thejas! Jarcec On Tue, Aug 20, 2013 at 03:31:48AM -0700, Carl Steinbach wrote: The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas! signature.asc Description: Digital signature
[jira] [Updated] (HIVE-5120) document what hive.server2.thrift.sasl.qop values mean in hive-default.xml.template
[ https://issues.apache.org/jira/browse/HIVE-5120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-5120: --- Resolution: Fixed Fix Version/s: 0.12.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Thejas! document what hive.server2.thrift.sasl.qop values mean in hive-default.xml.template --- Key: HIVE-5120 URL: https://issues.apache.org/jira/browse/HIVE-5120 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.12.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Fix For: 0.12.0 Attachments: HIVE-5120.1.patch, HIVE-5120.2.patch Current description of configuration does not say what the values for the hive.server2.thrift.sasl.qop property mean, and also does not say that it works only with kerberos auth turned on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5121) Remove obsolete code on SemanticAnalyzer#genJoinTree
[
https://issues.apache.org/jira/browse/HIVE-5121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-5121:
---
Resolution: Fixed
Fix Version/s: 0.12.0
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks, Azrael!
Remove obsolete code on SemanticAnalyzer#genJoinTree
Key: HIVE-5121
URL: https://issues.apache.org/jira/browse/HIVE-5121
Project: Hive
Issue Type: Improvement
Affects Versions: 0.11.0
Environment: ubuntu 12.04
Reporter: Azrael Park
Assignee: Azrael Park
Priority: Trivial
Fix For: 0.12.0
Attachments: HIVE-5121.D12405.1.patch
Remove obsolete code on SemanticAnalyzer#genJoinTree.
{noformat}
children[1] = alias;
joinTree.setBaseSrc(children);
- aliasToOpInfo.get(alias);
joinTree.setId(qb.getId());
joinTree.getAliasToOpInfo().put(
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4299) exported metadata by HIVE-3068 cannot be imported because of wrong file name
[
https://issues.apache.org/jira/browse/HIVE-4299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-4299:
---
Resolution: Fixed
Fix Version/s: 0.12.0
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks, Sho for patch. Thanks, Ed for fixing test failures!
exported metadata by HIVE-3068 cannot be imported because of wrong file name
Key: HIVE-4299
URL: https://issues.apache.org/jira/browse/HIVE-4299
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.11.0
Reporter: Sho Shimauchi
Assignee: Edward Capriolo
Fix For: 0.12.0
Attachments: HIVE-4299.1.patch.txt, HIVE-4299.4.patch.txt,
HIVE-4299.5.patch.txt, HIVE-4299.patch
h2. Symptom
When DROP TABLE a table, metadata of the table is generated to be able to
import the dropped table again.
However, the exported metadata name is 'table name.metadata'.
Since ImportSemanticAnalyzer allows only '_metadata' as metadata filename,
user have to rename the metadata file to import the table.
h2. How to reproduce
Set the following setting to hive-site.xml:
{code}
property
namehive.metastore.pre.event.listeners/name
valueorg.apache.hadoop.hive.ql.parse.MetaDataExportListener/value
/property
{code}
Then run the following queries:
{code}
CREATE TABLE test_table (id INT, name STRING);
DROP TABLE test_table;
IMPORT TABLE test_table_imported FROM '/path/to/metadata/file';
FAILED: SemanticException [Error 10027]: Invalid path
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congrats Thejas! On Tue, Aug 20, 2013 at 7:32 AM, Jarek Jarcec Cecho jar...@apache.orgwrote: Congratulations Thejas! Jarcec On Tue, Aug 20, 2013 at 03:31:48AM -0700, Carl Steinbach wrote: The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas! -- *Note that I'm no longer using my Yahoo! email address. Please email me at billgra...@gmail.com going forward.*
[jira] [Commented] (HIVE-5077) Provide an option to run local task in process
[ https://issues.apache.org/jira/browse/HIVE-5077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745035#comment-13745035 ] Ashutosh Chauhan commented on HIVE-5077: Since no technical justification is provided, I will consider this veto as void. Provide an option to run local task in process -- Key: HIVE-5077 URL: https://issues.apache.org/jira/browse/HIVE-5077 Project: Hive Issue Type: Bug Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-5077.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4779) Enhance coverage of package org.apache.hadoop.hive.ql.udf
[
https://issues.apache.org/jira/browse/HIVE-4779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745072#comment-13745072
]
Hive QA commented on HIVE-4779:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12598909/HIVE-4779.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 2893 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket5
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/488/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/488/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
Enhance coverage of package org.apache.hadoop.hive.ql.udf
-
Key: HIVE-4779
URL: https://issues.apache.org/jira/browse/HIVE-4779
Project: Hive
Issue Type: Test
Affects Versions: 0.12.0
Reporter: Ivan A. Veselovsky
Assignee: Ivan A. Veselovsky
Attachments: HIVE-4779.patch, HIVE-4779-trunk--N1.patch
Enhance coverage of package org.apache.hadoop.hive.ql.udf up to 80%.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5107) Change hive's build to maven
[ https://issues.apache.org/jira/browse/HIVE-5107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745083#comment-13745083 ] Owen O'Malley commented on HIVE-5107: - Suggestions: * separate the ql and exec jars. We should have a jar that includes just the Hive code and not the dependencies. * remove the ant dir, we don't need it after moving to maven. * i'd suggest making the top level the parent pom and then making a separate aggregation directory for packaging * if you want to look at the attempt i started back in april, i pushed it to https://github.com/omalley/hive on the maven branch. Change hive's build to maven Key: HIVE-5107 URL: https://issues.apache.org/jira/browse/HIVE-5107 Project: Hive Issue Type: Task Reporter: Edward Capriolo Assignee: Edward Capriolo I can not cope with hive's build infrastructure any more. I have started working on porting the project to maven. When I have some solid progess i will github the entire thing for review. Then we can talk about switching the project somehow. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5077) Provide an option to run local task in process
[ https://issues.apache.org/jira/browse/HIVE-5077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745082#comment-13745082 ] Edward Capriolo commented on HIVE-5077: --- What is the technical justification for adding the feature? That is all I want to know, but no one wants to give me an answer. Provide an option to run local task in process -- Key: HIVE-5077 URL: https://issues.apache.org/jira/browse/HIVE-5077 Project: Hive Issue Type: Bug Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-5077.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5077) Provide an option to run local task in process
[
https://issues.apache.org/jira/browse/HIVE-5077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745086#comment-13745086
]
Edward Capriolo commented on HIVE-5077:
---
{quote}
To prevent vetos from being used capriciously, they must be accompanied by a
technical justification showing why the change is bad (opens a security
exposure, negatively affects performance, etc. ). A veto without a
justification is invalid and has no weight.
{quote}
Are you trying to argue that we should allow people to add features with no use
case an no description, just because I can not prove it's bad?
Provide an option to run local task in process
--
Key: HIVE-5077
URL: https://issues.apache.org/jira/browse/HIVE-5077
Project: Hive
Issue Type: Bug
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Attachments: HIVE-5077.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HIVE-5077) Provide an option to run local task in process
[
https://issues.apache.org/jira/browse/HIVE-5077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745086#comment-13745086
]
Edward Capriolo edited comment on HIVE-5077 at 8/20/13 3:57 PM:
{quote}
To prevent vetos from being used capriciously, they must be accompanied by a
technical justification showing why the change is bad (opens a security
exposure, negatively affects performance, etc. ). A veto without a
justification is invalid and has no weight.
{quote}
Are you trying to argue that we should allow people to add features with no use
case an no description, just because I can not prove it's bad? I can't argue
the technical justification for my -1 until you give me technical justification
why we need this feature.
was (Author: appodictic):
{quote}
To prevent vetos from being used capriciously, they must be accompanied by a
technical justification showing why the change is bad (opens a security
exposure, negatively affects performance, etc. ). A veto without a
justification is invalid and has no weight.
{quote}
Are you trying to argue that we should allow people to add features with no use
case an no description, just because I can not prove it's bad?
Provide an option to run local task in process
--
Key: HIVE-5077
URL: https://issues.apache.org/jira/browse/HIVE-5077
Project: Hive
Issue Type: Bug
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Attachments: HIVE-5077.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congrats Thejas! On Aug 20, 2013, at 8:00 AM, Bill Graham billgra...@gmail.commailto:billgra...@gmail.com wrote: Congrats Thejas! On Tue, Aug 20, 2013 at 7:32 AM, Jarek Jarcec Cecho jar...@apache.orgmailto:jar...@apache.org wrote: Congratulations Thejas! Jarcec On Tue, Aug 20, 2013 at 03:31:48AM -0700, Carl Steinbach wrote: The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas! -- Note that I'm no longer using my Yahoo! email address. Please email me at billgra...@gmail.commailto:billgra...@gmail.com going forward.
[jira] [Commented] (HIVE-5077) Provide an option to run local task in process
[ https://issues.apache.org/jira/browse/HIVE-5077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745092#comment-13745092 ] Edward Capriolo commented on HIVE-5077: --- technical justification showing why the change is bad Technical justification: 1) Issues had no description and no obvious use case. 2) Other issues https://issues.apache.org/jira/browse/HIVE-5054 seem to suggest running in-process is a bad thing http://wiki.apache.org/hadoop/CodeReviewChecklist (in bold) 1) unit tests exist for bug fixes and new features, or a rationale is given in Jira for why there is no test I will gladly remove my -1 when it is explained to me what this feature does. Provide an option to run local task in process -- Key: HIVE-5077 URL: https://issues.apache.org/jira/browse/HIVE-5077 Project: Hive Issue Type: Bug Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-5077.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5077) Provide an option to run local task in process
[
https://issues.apache.org/jira/browse/HIVE-5077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745094#comment-13745094
]
Edward Capriolo commented on HIVE-5077:
---
https://cwiki.apache.org/Hive/howtocontribute.html
Making Changes
Before you start, send a message to the Hive developer mailing list, or file a
bug report in JIRA.
{bold}Describe your proposed changes and check that they fit in with what
others are doing and have planned for the project. Be patient, it may take
folks a while to understand your requirements.{bold}
Modify the source code and add some features using your favorite IDE.
Provide an option to run local task in process
--
Key: HIVE-5077
URL: https://issues.apache.org/jira/browse/HIVE-5077
Project: Hive
Issue Type: Bug
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Attachments: HIVE-5077.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congrats Thejas! On Tue, Aug 20, 2013 at 3:31 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas! -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Updated] (HIVE-4645) Stat information like numFiles and totalSize is not correct when sub-directory is exists
[ https://issues.apache.org/jira/browse/HIVE-4645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-4645: --- Resolution: Fixed Status: Resolved (was: Patch Available) I committed this to trunk! Thank you very much for the contribution!! Stat information like numFiles and totalSize is not correct when sub-directory is exists Key: HIVE-4645 URL: https://issues.apache.org/jira/browse/HIVE-4645 Project: Hive Issue Type: Test Components: Statistics Reporter: Navis Assignee: Navis Priority: Trivial Fix For: 0.12.0 Attachments: HIVE-4645.D11037.1.patch, HIVE-4645.D11037.2.patch, HIVE-4645.D11037.3.patch, HIVE-4645.D11037.4.patch The test infer_bucket_sort_list_bucket.q returns 4096 as totalSize but it's size of parent directory, not sum of file size. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4645) Stat information like numFiles and totalSize is not correct when sub-directory is exists
[ https://issues.apache.org/jira/browse/HIVE-4645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745115#comment-13745115 ] Phabricator commented on HIVE-4645: --- brock has accepted the revision HIVE-4645 [jira] Stat information like numFiles and totalSize is not correct when sub-directory is exists. Thanks Navis!! REVISION DETAIL https://reviews.facebook.net/D11037 BRANCH HIVE-4645 ARCANIST PROJECT hive To: JIRA, brock, navis Cc: brock Stat information like numFiles and totalSize is not correct when sub-directory is exists Key: HIVE-4645 URL: https://issues.apache.org/jira/browse/HIVE-4645 Project: Hive Issue Type: Test Components: Statistics Reporter: Navis Assignee: Navis Priority: Trivial Fix For: 0.12.0 Attachments: HIVE-4645.D11037.1.patch, HIVE-4645.D11037.2.patch, HIVE-4645.D11037.3.patch, HIVE-4645.D11037.4.patch The test infer_bucket_sort_list_bucket.q returns 4096 as totalSize but it's size of parent directory, not sum of file size. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4925) Modify Hive build to enable compiling and running Hive with JDK7
[ https://issues.apache.org/jira/browse/HIVE-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745118#comment-13745118 ] Xuefu Zhang commented on HIVE-4925: --- [~ashutoshc] [~brocknoland] Could we have a close on this? Modify Hive build to enable compiling and running Hive with JDK7 Key: HIVE-4925 URL: https://issues.apache.org/jira/browse/HIVE-4925 Project: Hive Issue Type: Sub-task Components: Build Infrastructure Affects Versions: 0.11.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4925) Modify Hive build to enable compiling and running Hive with JDK7
[ https://issues.apache.org/jira/browse/HIVE-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745122#comment-13745122 ] Brock Noland commented on HIVE-4925: Sure, let's remove it for now and add it later if required. Modify Hive build to enable compiling and running Hive with JDK7 Key: HIVE-4925 URL: https://issues.apache.org/jira/browse/HIVE-4925 Project: Hive Issue Type: Sub-task Components: Build Infrastructure Affects Versions: 0.11.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congrats Tejas! On Tue, Aug 20, 2013 at 9:32 AM, Eugene Koifman ekoif...@hortonworks.comwrote: Congrats Thejas! On Tue, Aug 20, 2013 at 3:31 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas! CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-4925) Modify Hive build to enable compiling and running Hive with JDK7
[ https://issues.apache.org/jira/browse/HIVE-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745132#comment-13745132 ] Ashutosh Chauhan commented on HIVE-4925: yeah.. lets reintroduce this later if we find a need for it. Modify Hive build to enable compiling and running Hive with JDK7 Key: HIVE-4925 URL: https://issues.apache.org/jira/browse/HIVE-4925 Project: Hive Issue Type: Sub-task Components: Build Infrastructure Affects Versions: 0.11.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congratulation! On Tue, Aug 20, 2013 at 4:56 PM, Shreepadma Venugopalan shreepa...@cloudera.com wrote: Congrats Tejas! On Tue, Aug 20, 2013 at 9:32 AM, Eugene Koifman ekoif...@hortonworks.com wrote: Congrats Thejas! On Tue, Aug 20, 2013 at 3:31 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas! CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congrats Thejas!! On Aug 20, 2013, at 10:01 AM, Daniel Dai da...@hortonworks.com wrote: Congratulation! On Tue, Aug 20, 2013 at 4:56 PM, Shreepadma Venugopalan shreepa...@cloudera.com wrote: Congrats Tejas! On Tue, Aug 20, 2013 at 9:32 AM, Eugene Koifman ekoif...@hortonworks.com wrote: Congrats Thejas! On Tue, Aug 20, 2013 at 3:31 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas! CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congrats Thejas! On Tue, Aug 20, 2013 at 9:56 AM, Shreepadma Venugopalan shreepa...@cloudera.com wrote: Congrats Tejas! On Tue, Aug 20, 2013 at 9:32 AM, Eugene Koifman ekoif...@hortonworks.com wrote: Congrats Thejas! On Tue, Aug 20, 2013 at 3:31 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas! CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-4331) Integrated StorageHandler for Hive and HCat using the HiveStorageHandler
[ https://issues.apache.org/jira/browse/HIVE-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745136#comment-13745136 ] Francis Liu commented on HIVE-4331: --- Ashutosh, I don't think what you're suggesting will work. Since HiveOFImpl is merely a container for initializing real OFs that are contained in defined FileSinkOps. Looking at HIVE-2764, HiveOFImpl was introduced to guarantee checkOutputSpecs was called on each real OF so that OFs such as HBase has a chance of adding a delegation token. It seems the HiveOFImpl name is misleading since it doesn't extend HiveOF and is not used in the same way. Let me know if I missed anything. The requirement that a StorageHandler's OF has to extend HiveOF pervasive all over the hive code. ie A lot of classes have HiveOutputFormat (tableDesc, FileSinkOP, etc) as a member variable instead of OF. So it's not possible to just change this expectation at the end point of the code path. With storage handlers I don't see why we still need HiveOF. We should migrate all supported OFs to use storage handlers and remove HiveOF. HivePassThru is an step in that direction. StorageHandlers will no longer have to extend HiveOF and HiveOF does not have to leak into HCat code to get interoperability working. Integrated StorageHandler for Hive and HCat using the HiveStorageHandler Key: HIVE-4331 URL: https://issues.apache.org/jira/browse/HIVE-4331 Project: Hive Issue Type: Task Components: HCatalog Affects Versions: 0.11.0, 0.12.0 Reporter: Ashutosh Chauhan Assignee: Viraj Bhat Attachments: HIVE4331_07-17.patch, StorageHandlerDesign_HIVE4331.pdf 1) Deprecate the HCatHBaseStorageHandler and RevisionManager from HCatalog. These will now continue to function but internally they will use the DefaultStorageHandler from Hive. They will be removed in future release of Hive. 2) Design a HivePassThroughFormat so that any new StorageHandler in Hive will bypass the HiveOutputFormat. We will use this class in Hive's HBaseStorageHandler instead of the HiveHBaseTableOutputFormat. 3) Write new unit tests in the HCat's storagehandler so that systems such as Pig and Map Reduce can use the Hive's HBaseStorageHandler instead of the HCatHBaseStorageHandler. 4) Make sure all the old and new unit tests pass without backward compatibility (except known issues as described in the Design Document). 5) Replace all instances of the HCat source code, which point to HCatStorageHandler to use theHiveStorageHandler including the FosterStorageHandler. I have attached the design document for the same and will attach a patch to this Jira. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congrats !! On Tue, Aug 20, 2013 at 10:03 AM, Vaibhav Gumashta vgumas...@hortonworks.com wrote: Congrats Thejas! On Tue, Aug 20, 2013 at 9:56 AM, Shreepadma Venugopalan shreepa...@cloudera.com wrote: Congrats Tejas! On Tue, Aug 20, 2013 at 9:32 AM, Eugene Koifman ekoif...@hortonworks.com wrote: Congrats Thejas! On Tue, Aug 20, 2013 at 3:31 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas! CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Updated] (HIVE-1511) Hive plan serialization is slow
[ https://issues.apache.org/jira/browse/HIVE-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-1511: --- Attachment: HIVE-1511.6.patch v6 of the patch. Hive plan serialization is slow --- Key: HIVE-1511 URL: https://issues.apache.org/jira/browse/HIVE-1511 Project: Hive Issue Type: Improvement Affects Versions: 0.7.0 Reporter: Ning Zhang Assignee: Mohammad Kamrul Islam Attachments: HIVE-1511.4.patch, HIVE-1511.5.patch, HIVE-1511.6.patch, HIVE-1511.patch, HIVE-1511-wip2.patch, HIVE-1511-wip3.patch, HIVE-1511-wip4.patch, HIVE-1511-wip.patch As reported by Edward Capriolo: For reference I did this as a test case SELECT * FROM src where key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR ...(100 more of these) No OOM but I gave up after the test case did not go anywhere for about 2 minutes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4925) Modify Hive build to enable compiling and running Hive with JDK7
[ https://issues.apache.org/jira/browse/HIVE-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-4925: -- Fix Version/s: 0.12.0 Status: Patch Available (was: Open) Modify Hive build to enable compiling and running Hive with JDK7 Key: HIVE-4925 URL: https://issues.apache.org/jira/browse/HIVE-4925 Project: Hive Issue Type: Sub-task Components: Build Infrastructure Affects Versions: 0.11.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang Fix For: 0.12.0 Attachments: HIVE-4925.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4925) Modify Hive build to enable compiling and running Hive with JDK7
[ https://issues.apache.org/jira/browse/HIVE-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-4925: -- Attachment: HIVE-4925.patch Attached patch that removes a few usage of javac.version in the build. Modify Hive build to enable compiling and running Hive with JDK7 Key: HIVE-4925 URL: https://issues.apache.org/jira/browse/HIVE-4925 Project: Hive Issue Type: Sub-task Components: Build Infrastructure Affects Versions: 0.11.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang Attachments: HIVE-4925.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5107) Change hive's build to maven
[ https://issues.apache.org/jira/browse/HIVE-5107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745143#comment-13745143 ] Edward Capriolo commented on HIVE-5107: --- remove the ant dir, we don't need it after moving to maven. Right, I was actually thinking we can have the project in a state where either could build it for a while but it might be nice to cut bait with the old system. separate the ql and exec jars. We should have a jar that includes just the Hive code and not the dependencies. Makes sense. I have just been focused on removing input formats and zk. I will look at what you did here: https://github.com/omalley/hive. I am punting on many issues on my first pass, but there is lots of things I do not know exactly how they will be solved yet, Change hive's build to maven Key: HIVE-5107 URL: https://issues.apache.org/jira/browse/HIVE-5107 Project: Hive Issue Type: Task Reporter: Edward Capriolo Assignee: Edward Capriolo I can not cope with hive's build infrastructure any more. I have started working on porting the project to maven. When I have some solid progess i will github the entire thing for review. Then we can talk about switching the project somehow. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5107) Change hive's build to maven
[ https://issues.apache.org/jira/browse/HIVE-5107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745148#comment-13745148 ] Edward Capriolo commented on HIVE-5107: --- I like what you did with the shims, that was one I just punted on. Change hive's build to maven Key: HIVE-5107 URL: https://issues.apache.org/jira/browse/HIVE-5107 Project: Hive Issue Type: Task Reporter: Edward Capriolo Assignee: Edward Capriolo I can not cope with hive's build infrastructure any more. I have started working on porting the project to maven. When I have some solid progess i will github the entire thing for review. Then we can talk about switching the project somehow. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congrats Thejas! On Tue, Aug 20, 2013 at 10:08 AM, Hari Subramaniyan hsubramani...@hortonworks.com wrote: Congrats !! On Tue, Aug 20, 2013 at 10:03 AM, Vaibhav Gumashta vgumas...@hortonworks.com wrote: Congrats Thejas! On Tue, Aug 20, 2013 at 9:56 AM, Shreepadma Venugopalan shreepa...@cloudera.com wrote: Congrats Tejas! On Tue, Aug 20, 2013 at 9:32 AM, Eugene Koifman ekoif...@hortonworks.com wrote: Congrats Thejas! On Tue, Aug 20, 2013 at 3:31 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas! CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Created] (HIVE-5125) Better error messages from Windowing queries
Harish Butani created HIVE-5125: --- Summary: Better error messages from Windowing queries Key: HIVE-5125 URL: https://issues.apache.org/jira/browse/HIVE-5125 Project: Hive Issue Type: Bug Components: PTF-Windowing Reporter: Harish Butani Assignee: Harish Butani There are 2 issues. - Ranking invocations must notify user of their relation to the Order clause. - When all fn invocations have the same Partition and Order spec, there is no need to talk about componentization in the error message. Here is more details about these issues. Pasting message posted on Hive User Group: Regarding the issue posted with rank and dense_rank. The example posted was: CREATE TABLE test (a INT); EXPLAIN SELECT DENSE_RANK() OVER (PARTITION BY a), a FROM test; Some comments on this: 1. The underlying issue is the bug that ranking functions had to be lowercase. This was fixed in Jira 4954 by Edward. The ranking functions are rank, dense_rank, percent_rank and cume_dist. Some details about this: we check for ranking functions and this check was looking for lowercase names. More below on why we do this check. The rest of hive: parser, functionRegistry etc is not case sensitive. So from a language perspective these functions are not case sensitive. 2. The ranking functions don't accept any arguments. They operate on the expressions specified in the order spec of the window clause. Internally the ranking functions are implemented as UDAFs, so what we do is pass the order expressions as arguments to the ranking functions. So from a language perspective invoke ranking functions without any arguments. 3. Now the e.g. above is saying partition by 'a'; but no order is specified. Since no order is specified for the ranking functions the order expression assumed is the partition expressions, in this it is 'a'. So the output of this e.g. would be all rows would have a dense_rank = 1. 4. The explanation for this case: DENSE_RANK() OVER (PARTITION BY a) you got: FAILED: SemanticException Failed to breakup Windowing invocations into Groups. At least 1 group must only depend on input columns. Also check for circular dependencies. Underlying error: org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException: One or more arguments are expected. The problem is DENSE_RANK didn't get flagged as a ranking function because of the lowercase bug. This had the effect that we add the order expressions as arguments. It got compiled into a dense_rank UDAF invocation. Which is than expecting at least 1 arg. 5. The explanation for this case: dense_rank(a) OVER (PARTITION BY a) this is the right behavior. But I can see the confusion. The error message is not helpful. We should say something to the effect that ranking functions work of order expressions. Adding a Jira for this. 6. What is is 'Failed to breakup Windowing invocations into Groups. At least 1 group must only depend on input columns. Also check for circular dependencies.' We allow for multiple window invocations in a Query. These may be on different partition and order specifications. Internally we go through a process of breaking up Window invocations into Groups and do translation on each Group. Again in this case, we should provide better error messages. This message is confusing. When there is only 1 Group, we should just output the underlying message. Adding a Jira for this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4617) ExecuteStatementAsync call to run a query in non-blocking mode
[ https://issues.apache.org/jira/browse/HIVE-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-4617: --- Status: Patch Available (was: Reopened) ExecuteStatementAsync call to run a query in non-blocking mode -- Key: HIVE-4617 URL: https://issues.apache.org/jira/browse/HIVE-4617 Project: Hive Issue Type: Improvement Components: HiveServer2 Affects Versions: 0.11.0 Reporter: Jaideep Dhok Assignee: Vaibhav Gumashta Attachments: HIVE-4617.D12417.1.patch Provide a way to run a queries asynchronously. Current executeStatement call blocks until the query run is complete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: DataNucleus patches for Hive
Xuefu, Indeed that's the problem and ant very-clean also fixed the problem. Thanks, Eric On Tue, Aug 6, 2013 at 5:10 PM, Xuefu Zhang xzh...@cloudera.com wrote: You probably have mix-matched datanucleus libraries in your lib/ folder. You should only three 3.2.x files if you do ls datanu*. To avoid that, you need to do ant very-clean before building hive. --Xuefu On Tue, Aug 6, 2013 at 4:39 PM, Eric Chu e...@rocketfuel.com wrote: Hi, I'm a bit confused about what DataNucleus patches we should get for *Hive 0.11 with JDK 6*. It'd be great if people working on that could shed some light on the subject. Thanks in advance! After installing Hive 0.11 and applying the patch for HIVE-4619 (or else MR queries will result in error), I will sometimes (but not always) see DataNucleus error log messages as below: 13/08/06 12:32:33 ERROR DataNucleus.Plugin: Bundle org.eclipse.jdt.core requires org.eclipse.core.resources but it cannot be resolved. 13/08/06 12:32:33 ERROR DataNucleus.Plugin: Bundle org.eclipse.jdt.core requires org.eclipse.core.runtime but it cannot be resolved. 13/08/06 12:32:33 ERROR DataNucleus.Plugin: Bundle org.eclipse.jdt.core requires org.eclipse.text but it cannot be resolved. HIVE-2015 was opened for this issue, but the last comment there said it was fixed through HIVE-3632, whose purpose is to upgrade DataNucleus to support JDK 7. HIVE-3632 in turn was blocked by HIVE-4900 and broke HIVE-4942, both of which have patches available in 12. So I ended up applying HIVE-4900, HIVE-3632, and HIVE-4942, in that order. However, afterward, any query (MR, select *, show tables) would result in error. In the end, after I applied the patch for HIVE-4900, I haven't seen those error messages and queries are still running fine. Are the other patches not compatible with JDK 6? *Error messages for SHOW TABLES after applying HIVE-4900, HIVE-3632, and HIVE-4942:* FAILED: Error in metadata: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient 13/08/06 12:31:01 ERROR exec.Task: FAILED: Error in metadata: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1140) at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1125) at org.apache.hadoop.hive.ql.exec.DDLTask.showTables(DDLTask.java:2265) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:337) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:144) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1361) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1146) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:948) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:261) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:218) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:415) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:758) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:208) Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1139) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.init(RetryingMetaStoreClient.java:51) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:61) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2284) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2295) at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1136) ... 18 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1137) ... 23 more
[jira] [Assigned] (HIVE-5112) Upgrade protobuf to 2.5 from 2.4
[ https://issues.apache.org/jira/browse/HIVE-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley reassigned HIVE-5112: --- Assignee: Owen O'Malley Upgrade protobuf to 2.5 from 2.4 Key: HIVE-5112 URL: https://issues.apache.org/jira/browse/HIVE-5112 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Owen O'Malley Hadoop and Hbase have both upgraded protobuf. We should as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congratulations Thejas!! On Tue, Aug 20, 2013 at 10:20 AM, Vikram Dixit vik...@hortonworks.comwrote: Congrats Thejas! On Tue, Aug 20, 2013 at 10:08 AM, Hari Subramaniyan hsubramani...@hortonworks.com wrote: Congrats !! On Tue, Aug 20, 2013 at 10:03 AM, Vaibhav Gumashta vgumas...@hortonworks.com wrote: Congrats Thejas! On Tue, Aug 20, 2013 at 9:56 AM, Shreepadma Venugopalan shreepa...@cloudera.com wrote: Congrats Tejas! On Tue, Aug 20, 2013 at 9:32 AM, Eugene Koifman ekoif...@hortonworks.com wrote: Congrats Thejas! On Tue, Aug 20, 2013 at 3:31 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas! CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-4978) [WebHCat] Close the PrintWriter after writing data
[ https://issues.apache.org/jira/browse/HIVE-4978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745178#comment-13745178 ] Venki Korukanti commented on HIVE-4978: --- Modified the patch to close only if the out stream is not a standard stream. [WebHCat] Close the PrintWriter after writing data -- Key: HIVE-4978 URL: https://issues.apache.org/jira/browse/HIVE-4978 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.11.0 Reporter: Venki Korukanti Priority: Minor Fix For: 0.12.0 Attachments: HIVE-4978-1.patch, HIVE-4978-2.patch We are not closing the PrintWriter after writing data into it. I haven't seen any problems so far, but it is good to close the PrintWriter so that resources are released properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4978) [WebHCat] Close the PrintWriter after writing data
[ https://issues.apache.org/jira/browse/HIVE-4978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venki Korukanti updated HIVE-4978: -- Attachment: HIVE-4978-2.patch [WebHCat] Close the PrintWriter after writing data -- Key: HIVE-4978 URL: https://issues.apache.org/jira/browse/HIVE-4978 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.11.0 Reporter: Venki Korukanti Priority: Minor Fix For: 0.12.0 Attachments: HIVE-4978-1.patch, HIVE-4978-2.patch We are not closing the PrintWriter after writing data into it. I haven't seen any problems so far, but it is good to close the PrintWriter so that resources are released properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4601) WebHCat, Templeton need to support proxy users
[ https://issues.apache.org/jira/browse/HIVE-4601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-4601: - Status: Patch Available (was: Open) WebHCat, Templeton need to support proxy users -- Key: HIVE-4601 URL: https://issues.apache.org/jira/browse/HIVE-4601 Project: Hive Issue Type: Improvement Components: HCatalog Affects Versions: 0.11.0 Reporter: Dilli Arumugam Assignee: Eugene Koifman Labels: proxy, templeton Fix For: 0.12.0 Attachments: HIVE-4601.2.patch, HIVE-4601.3.patch, HIVE-4601.4.patch, HIVE-4601.patch We have a use case where a Gateway would provide unified and controlled access to secure hadoop cluster. The Gateway itself would authenticate to secure WebHDFS, Oozie and Templeton with SPNego. The Gateway would authenticate the end user with http basic and would assert the end user identity as douser argument in the calls to downstream WebHDFS, Oozie and Templeton. This works fine with WebHDFS and Oozie. But, does not work for Templeton as Templeton does not support proxy users. Hence, request to add this improvement to Templeton. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4601) WebHCat, Templeton need to support proxy users
[ https://issues.apache.org/jira/browse/HIVE-4601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-4601: - Attachment: HIVE-4601.4.patch HIVE-4601.4.patch made long Kerberos name to short name translation independent of Hadoop version WebHCat, Templeton need to support proxy users -- Key: HIVE-4601 URL: https://issues.apache.org/jira/browse/HIVE-4601 Project: Hive Issue Type: Improvement Components: HCatalog Affects Versions: 0.11.0 Reporter: Dilli Arumugam Assignee: Eugene Koifman Labels: proxy, templeton Fix For: 0.12.0 Attachments: HIVE-4601.2.patch, HIVE-4601.3.patch, HIVE-4601.4.patch, HIVE-4601.patch We have a use case where a Gateway would provide unified and controlled access to secure hadoop cluster. The Gateway itself would authenticate to secure WebHDFS, Oozie and Templeton with SPNego. The Gateway would authenticate the end user with http basic and would assert the end user identity as douser argument in the calls to downstream WebHDFS, Oozie and Templeton. This works fine with WebHDFS and Oozie. But, does not work for Templeton as Templeton does not support proxy users. Hence, request to add this improvement to Templeton. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1511) Hive plan serialization is slow
[ https://issues.apache.org/jira/browse/HIVE-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745185#comment-13745185 ] Brock Noland commented on HIVE-1511: Making MapredWork.opParseCtxMap transient seems to fix that. Not sure if that will cause us problems later. Hive plan serialization is slow --- Key: HIVE-1511 URL: https://issues.apache.org/jira/browse/HIVE-1511 Project: Hive Issue Type: Improvement Affects Versions: 0.7.0 Reporter: Ning Zhang Assignee: Mohammad Kamrul Islam Attachments: HIVE-1511.4.patch, HIVE-1511.5.patch, HIVE-1511.6.patch, HIVE-1511.patch, HIVE-1511-wip2.patch, HIVE-1511-wip3.patch, HIVE-1511-wip4.patch, HIVE-1511-wip.patch As reported by Edward Capriolo: For reference I did this as a test case SELECT * FROM src where key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR ...(100 more of these) No OOM but I gave up after the test case did not go anywhere for about 2 minutes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congratulations Thejas! chris On 8/20/13 3:31 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas!
Re: [ANNOUNCE] New Hive Committer - Thejas Nair
Congrats Thejas! thanks Prasad On Tue, Aug 20, 2013 at 3:31 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache Hive project. Please join me in congratulating Thejas!
[jira] [Commented] (HIVE-4788) RCFile and bzip2 compression not working
[ https://issues.apache.org/jira/browse/HIVE-4788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745222#comment-13745222 ] Tzur Turkenitz commented on HIVE-4788: -- This bug is persistent on hive 0.10.0 shipped with HDP RCFile and bzip2 compression not working Key: HIVE-4788 URL: https://issues.apache.org/jira/browse/HIVE-4788 Project: Hive Issue Type: Bug Components: Compression Affects Versions: 0.10.0 Environment: CDH4.2 Reporter: Johndee Burks Priority: Minor The issue is that Bzip2 compressed rcfile data is encountering an error when being queried even the most simple query select *. The issue is easily reproducible using the following. Create a table and load the sample data below. DDL: create table source_data (a string, b string) row format delimited fields terminated by ','; Sample data: apple,sauce Test: Do the following and you should receive the error listed below for the rcfile table with bz2 compression. create table rc_nobz2 (a string, b string) stored as rcfile; insert into table rc_nobz2 select * from source_txt; SET io.seqfile.compression.type=BLOCK; SET hive.exec.compress.output=true; SET mapred.compress.map.output=true; SET mapred.output.compress=true; SET mapred.output.compression.codec=org.apache.hadoop.io.compress.BZip2Codec; create table rc_bz2 (a string, b string) stored as rcfile; insert into table rc_bz2 select * from source_txt; hive select * from rc_bz2; Failed with exception java.io.IOException:java.io.IOException: Stream is not BZip2 formatted: expected 'h' as first byte but got '�' hive select * from rc_nobz2; apple sauce -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4388) HBase tests fail against Hadoop 2
[ https://issues.apache.org/jira/browse/HIVE-4388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745229#comment-13745229 ] Brock Noland commented on HIVE-4388: Works for me! HBase tests fail against Hadoop 2 - Key: HIVE-4388 URL: https://issues.apache.org/jira/browse/HIVE-4388 Project: Hive Issue Type: Bug Components: HBase Handler Reporter: Gunther Hagleitner Assignee: Brock Noland Attachments: HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388-wip.txt Currently we're building by default against 0.92. When you run against hadoop 2 (-Dhadoop.mr.rev=23) builds fail because of: HBASE-5963. HIVE-3861 upgrades the version of hbase used. This will get you past the problem in HBASE-5963 (which was fixed in 0.94.1) but fails with: HBASE-6396. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1511) Hive plan serialization is slow
[ https://issues.apache.org/jira/browse/HIVE-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-1511: --- Attachment: HIVE-1511.7.patch v7 changes opParseCtxMap to transient. Hive plan serialization is slow --- Key: HIVE-1511 URL: https://issues.apache.org/jira/browse/HIVE-1511 Project: Hive Issue Type: Improvement Affects Versions: 0.7.0 Reporter: Ning Zhang Assignee: Mohammad Kamrul Islam Attachments: HIVE-1511.4.patch, HIVE-1511.5.patch, HIVE-1511.6.patch, HIVE-1511.7.patch, HIVE-1511.patch, HIVE-1511-wip2.patch, HIVE-1511-wip3.patch, HIVE-1511-wip4.patch, HIVE-1511-wip.patch As reported by Edward Capriolo: For reference I did this as a test case SELECT * FROM src where key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR ...(100 more of these) No OOM but I gave up after the test case did not go anywhere for about 2 minutes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5112) Upgrade protobuf to 2.5 from 2.4
[ https://issues.apache.org/jira/browse/HIVE-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-5112: Status: Patch Available (was: Open) Upgrade protobuf to 2.5 from 2.4 Key: HIVE-5112 URL: https://issues.apache.org/jira/browse/HIVE-5112 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Owen O'Malley Hadoop and Hbase have both upgraded protobuf. We should as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5112) Upgrade protobuf to 2.5 from 2.4
[ https://issues.apache.org/jira/browse/HIVE-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-5112: -- Attachment: HIVE-5112.D12429.1.patch omalley requested code review of HIVE-5112 [jira] Upgrade protobuf to 2.5 from 2.4. Reviewers: JIRA HIVE-5112. Upgrade protobuf to 2.5.0. Hadoop and Hbase have both upgraded protobuf. We should as well. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D12429 AFFECTED FILES ivy/libraries.properties ql/src/gen/protobuf/gen-java/org/apache/hadoop/hive/ql/io/orc/OrcProto.java MANAGE HERALD RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/29667/ To: JIRA, omalley Upgrade protobuf to 2.5 from 2.4 Key: HIVE-5112 URL: https://issues.apache.org/jira/browse/HIVE-5112 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Owen O'Malley Attachments: HIVE-5112.D12429.1.patch Hadoop and Hbase have both upgraded protobuf. We should as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4214) OVER accepts general expression instead of just function
[ https://issues.apache.org/jira/browse/HIVE-4214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745246#comment-13745246 ] Harish Butani commented on HIVE-4214: - +1 OVER accepts general expression instead of just function Key: HIVE-4214 URL: https://issues.apache.org/jira/browse/HIVE-4214 Project: Hive Issue Type: Bug Components: PTF-Windowing Affects Versions: 0.11.0 Reporter: Alan Gates Assignee: Ashutosh Chauhan Attachments: HIVE-4214.1.patch, HIVE-4214.3.patch, HIVE-4214.patch The query: select s, i, avg(d) / 10.0 over (partition by s order by i) from over100k; runs (and produces meaningless output). Over should not allow the arithmetic expression. Only a UDAF or PTF function should be valid there. The correct way to write this query should be select s, i, avg(d) over (partition by s order by i) / 10. 0 from over100k; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5112) Upgrade protobuf to 2.5 from 2.4
[ https://issues.apache.org/jira/browse/HIVE-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745247#comment-13745247 ] Phabricator commented on HIVE-5112: --- hagleitn has commented on the revision HIVE-5112 [jira] Upgrade protobuf to 2.5 from 2.4. LGTM: +1 REVISION DETAIL https://reviews.facebook.net/D12429 To: JIRA, omalley Cc: hagleitn Upgrade protobuf to 2.5 from 2.4 Key: HIVE-5112 URL: https://issues.apache.org/jira/browse/HIVE-5112 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Owen O'Malley Attachments: HIVE-5112.D12429.1.patch Hadoop and Hbase have both upgraded protobuf. We should as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5117) orc_dictionary_threshold is not deterministic
[ https://issues.apache.org/jira/browse/HIVE-5117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-5117: --- Resolution: Fixed Fix Version/s: 0.12.0 Status: Resolved (was: Patch Available) Thank you for the patch Navis and thank you for the review Ashutosh! I committed this. orc_dictionary_threshold is not deterministic - Key: HIVE-5117 URL: https://issues.apache.org/jira/browse/HIVE-5117 Project: Hive Issue Type: Test Components: Tests Reporter: Navis Assignee: Navis Priority: Trivial Fix For: 0.12.0 Attachments: HIVE-5117.D12363.1.patch orc_dictionary_threshold.q makes different result on hadoop2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5117) orc_dictionary_threshold is not deterministic
[ https://issues.apache.org/jira/browse/HIVE-5117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745258#comment-13745258 ] Brock Noland commented on HIVE-5117: Once we get the hadoop2 tests passing I think we have two precommit jobs. Once for hadoop1 and once for hadoop2. orc_dictionary_threshold is not deterministic - Key: HIVE-5117 URL: https://issues.apache.org/jira/browse/HIVE-5117 Project: Hive Issue Type: Test Components: Tests Reporter: Navis Assignee: Navis Priority: Trivial Fix For: 0.12.0 Attachments: HIVE-5117.D12363.1.patch orc_dictionary_threshold.q makes different result on hadoop2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5112) Upgrade protobuf to 2.5 from 2.4
[ https://issues.apache.org/jira/browse/HIVE-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745259#comment-13745259 ] Gunther Hagleitner commented on HIVE-5112: -- Code looks good - some questions though: - Is this going to work with 20S, where hadoop still relies on protobuf 2.4? - Is the binary orc format going to be ok with the upgrade (orc files generated with 2.4, reading with 2.5)? Upgrade protobuf to 2.5 from 2.4 Key: HIVE-5112 URL: https://issues.apache.org/jira/browse/HIVE-5112 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Owen O'Malley Attachments: HIVE-5112.D12429.1.patch Hadoop and Hbase have both upgraded protobuf. We should as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5112) Upgrade protobuf to 2.5 from 2.4
[ https://issues.apache.org/jira/browse/HIVE-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745261#comment-13745261 ] Edward Capriolo commented on HIVE-5112: --- 2.4 and 2.5 generated byte-code for protobuf is not the same compatible. Upgrade protobuf to 2.5 from 2.4 Key: HIVE-5112 URL: https://issues.apache.org/jira/browse/HIVE-5112 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Owen O'Malley Attachments: HIVE-5112.D12429.1.patch Hadoop and Hbase have both upgraded protobuf. We should as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HIVE-5112) Upgrade protobuf to 2.5 from 2.4
[ https://issues.apache.org/jira/browse/HIVE-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745261#comment-13745261 ] Edward Capriolo edited comment on HIVE-5112 at 8/20/13 6:42 PM: 2.4 and 2.5 generated source for protobuf is not compatible. If you generate 2.5 protobuf 2.4.1.jar can not compile with it. was (Author: appodictic): 2.4 and 2.5 generated source for protobuf is not the same compatible. If you generate 2.5 protobuf 2.4.1.jar can not compile with it. Upgrade protobuf to 2.5 from 2.4 Key: HIVE-5112 URL: https://issues.apache.org/jira/browse/HIVE-5112 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Owen O'Malley Attachments: HIVE-5112.D12429.1.patch Hadoop and Hbase have both upgraded protobuf. We should as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HIVE-5112) Upgrade protobuf to 2.5 from 2.4
[ https://issues.apache.org/jira/browse/HIVE-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745261#comment-13745261 ] Edward Capriolo edited comment on HIVE-5112 at 8/20/13 6:41 PM: 2.4 and 2.5 generated source for protobuf is not the same compatible. If you generate 2.5 protobuf 2.4.1.jar can not compile with it. was (Author: appodictic): 2.4 and 2.5 generated byte-code for protobuf is not the same compatible. Upgrade protobuf to 2.5 from 2.4 Key: HIVE-5112 URL: https://issues.apache.org/jira/browse/HIVE-5112 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Owen O'Malley Attachments: HIVE-5112.D12429.1.patch Hadoop and Hbase have both upgraded protobuf. We should as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5112) Upgrade protobuf to 2.5 from 2.4
[ https://issues.apache.org/jira/browse/HIVE-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745282#comment-13745282 ] Edward Capriolo commented on HIVE-5112: --- In the maven branch I am going to push to remove generated source code. It helps avoid issues like this. Upgrade protobuf to 2.5 from 2.4 Key: HIVE-5112 URL: https://issues.apache.org/jira/browse/HIVE-5112 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Owen O'Malley Attachments: HIVE-5112.D12429.1.patch Hadoop and Hbase have both upgraded protobuf. We should as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4844) Add char/varchar data types
[ https://issues.apache.org/jira/browse/HIVE-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745284#comment-13745284 ] Xuefu Zhang commented on HIVE-4844: --- [~jdere] It seems that the patch here and what you posted on fb rb are out of sync. Would you mind updating your patch here? Also, fb rb seems showing diff incorrectly, which is a little bothersome. Thanks. Add char/varchar data types --- Key: HIVE-4844 URL: https://issues.apache.org/jira/browse/HIVE-4844 Project: Hive Issue Type: New Feature Components: Types Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-4844.1.patch.hack Add new char/varchar data types which have support for more SQL-compliant behavior, such as SQL string comparison semantics, max length, etc. NO PRECOMMIT TESTS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4645) Stat information like numFiles and totalSize is not correct when sub-directory is exists
[ https://issues.apache.org/jira/browse/HIVE-4645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745300#comment-13745300 ] Hudson commented on HIVE-4645: -- FAILURE: Integrated in Hive-trunk-hadoop2-ptest #65 (See [https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/65/]) HIVE-4645: Stat information like numFiles and totalSize is not correct when sub-directory is exists (Navis via Brock Noland) (brock: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1515865) * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/StatsTask.java * /hive/trunk/ql/src/test/queries/clientpositive/list_bucket_dml_7.q * /hive/trunk/ql/src/test/queries/clientpositive/list_bucket_dml_8.q * /hive/trunk/ql/src/test/results/clientpositive/infer_bucket_sort_list_bucket.q.out * /hive/trunk/ql/src/test/results/clientpositive/list_bucket_dml_7.q.out * /hive/trunk/ql/src/test/results/clientpositive/list_bucket_dml_8.q.out * /hive/trunk/ql/src/test/results/clientpositive/stats_noscan_2.q.out Stat information like numFiles and totalSize is not correct when sub-directory is exists Key: HIVE-4645 URL: https://issues.apache.org/jira/browse/HIVE-4645 Project: Hive Issue Type: Test Components: Statistics Reporter: Navis Assignee: Navis Priority: Trivial Fix For: 0.12.0 Attachments: HIVE-4645.D11037.1.patch, HIVE-4645.D11037.2.patch, HIVE-4645.D11037.3.patch, HIVE-4645.D11037.4.patch The test infer_bucket_sort_list_bucket.q returns 4096 as totalSize but it's size of parent directory, not sum of file size. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3978) HIVE_AUX_JARS_PATH should have : instead of , as separator since it gets appended to HADOOP_CLASSPATH
[ https://issues.apache.org/jira/browse/HIVE-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745299#comment-13745299 ] Hudson commented on HIVE-3978: -- FAILURE: Integrated in Hive-trunk-hadoop2-ptest #65 (See [https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/65/]) HIVE-3978 : HIVE_AUX_JARS_PATH should have : instead of , as separator since it gets appended to HADOOP_CLASSPATH (Arup Malakar via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1515697) * /hive/trunk/bin/hive HIVE_AUX_JARS_PATH should have : instead of , as separator since it gets appended to HADOOP_CLASSPATH - Key: HIVE-3978 URL: https://issues.apache.org/jira/browse/HIVE-3978 Project: Hive Issue Type: Bug Environment: hive-0.10 hcatalog-0.5 hadoop 0.23 hbase 0.94 Reporter: Arup Malakar Assignee: Arup Malakar Fix For: 0.12.0 Attachments: HIVE-3978_branch_0.10_0.patch, HIVE-3978_trunk_0.patch The following code gets executed only in case of cygwin. HIVE_AUX_JARS_PATH=`echo $HIVE_AUX_JARS_PATH | sed 's/,/:/g'` But since HIVE_AUX_JARS_PATH gets added to HADOOP_CLASSPATH, the comma should get replaced by : for all cases. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5075) bug in ExprProcFactory.genPruner
[ https://issues.apache.org/jira/browse/HIVE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745298#comment-13745298 ] Hudson commented on HIVE-5075: -- FAILURE: Integrated in Hive-trunk-hadoop2-ptest #65 (See [https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/65/]) HIVE-5075 : bug in ExprProcFactory.genPruner (Sergey Shelukhin via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1515694) * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/ExprProcCtx.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/ExprProcFactory.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpProcFactory.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/OpWalkerCtx.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java bug in ExprProcFactory.genPruner Key: HIVE-5075 URL: https://issues.apache.org/jira/browse/HIVE-5075 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.12.0 Attachments: HIVE-5075.D12261.1.patch, HIVE-5075.D12261.2.patch The implications are not clear to me yet, I am just familiarizing myself with code around these parts, but ExprProcFactory.genPruner intends to return a value (indicating whether there are non-partition columns) via a primitive boolean parameter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4299) exported metadata by HIVE-3068 cannot be imported because of wrong file name
[
https://issues.apache.org/jira/browse/HIVE-4299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745296#comment-13745296
]
Hudson commented on HIVE-4299:
--
FAILURE: Integrated in Hive-trunk-hadoop2-ptest #65 (See
[https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/65/])
HIVE-4299 : exported metadata by HIVE3068 cannot be imported because of wrong
file name (Sho Shimauchi Edward Capriolo via Ashutosh Chauhan) (hashutosh:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1515839)
*
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java
*
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/MetaDataExportListener.java
exported metadata by HIVE-3068 cannot be imported because of wrong file name
Key: HIVE-4299
URL: https://issues.apache.org/jira/browse/HIVE-4299
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.11.0
Reporter: Sho Shimauchi
Assignee: Edward Capriolo
Fix For: 0.12.0
Attachments: HIVE-4299.1.patch.txt, HIVE-4299.4.patch.txt,
HIVE-4299.5.patch.txt, HIVE-4299.patch
h2. Symptom
When DROP TABLE a table, metadata of the table is generated to be able to
import the dropped table again.
However, the exported metadata name is 'table name.metadata'.
Since ImportSemanticAnalyzer allows only '_metadata' as metadata filename,
user have to rename the metadata file to import the table.
h2. How to reproduce
Set the following setting to hive-site.xml:
{code}
property
namehive.metastore.pre.event.listeners/name
valueorg.apache.hadoop.hive.ql.parse.MetaDataExportListener/value
/property
{code}
Then run the following queries:
{code}
CREATE TABLE test_table (id INT, name STRING);
DROP TABLE test_table;
IMPORT TABLE test_table_imported FROM '/path/to/metadata/file';
FAILED: SemanticException [Error 10027]: Invalid path
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5106) HCatFieldSchema overrides equals() but not hashCode()
[ https://issues.apache.org/jira/browse/HIVE-5106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745297#comment-13745297 ] Hudson commented on HIVE-5106: -- FAILURE: Integrated in Hive-trunk-hadoop2-ptest #65 (See [https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/65/]) HIVE-5106 : HCatFieldSchema overrides equals() but not hashCode() (Eugene Koifman via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1515815) * /hive/trunk/hcatalog/core/src/main/java/org/apache/hcatalog/data/schema/HCatFieldSchema.java * /hive/trunk/hcatalog/core/src/test/java/org/apache/hcatalog/data/schema/TestHCatSchema.java HCatFieldSchema overrides equals() but not hashCode() - Key: HIVE-5106 URL: https://issues.apache.org/jira/browse/HIVE-5106 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 0.12.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Fix For: 0.12.0 Attachments: HIVE-5106.patch It's likely that objects of this type are not hashed today but would lead to very nasty bugs if they ever will be. Looks like it was introduced in HCATALOG-438. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5111) ExprNodeColumnDesc doesn't distinguish partition and virtual columns, causing partition pruner to receive the latter
[ https://issues.apache.org/jira/browse/HIVE-5111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745301#comment-13745301 ] Hudson commented on HIVE-5111: -- FAILURE: Integrated in Hive-trunk-hadoop2-ptest #65 (See [https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/65/]) HIVE-5111 : ExprNodeColumnDesc doesn't distinguish partition and virtual columns, causing partition pruner to receive the latter (Sergey Shelukhin via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1515811) * /hive/trunk/common/src/java/org/apache/hadoop/hive/common/ObjectPair.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartExprEvalUtils.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java ExprNodeColumnDesc doesn't distinguish partition and virtual columns, causing partition pruner to receive the latter Key: HIVE-5111 URL: https://issues.apache.org/jira/browse/HIVE-5111 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.12.0 Attachments: HIVE-5111.D12393.1.patch See HIVE-3926 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5120) document what hive.server2.thrift.sasl.qop values mean in hive-default.xml.template
[ https://issues.apache.org/jira/browse/HIVE-5120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745295#comment-13745295 ] Hudson commented on HIVE-5120: -- FAILURE: Integrated in Hive-trunk-hadoop2-ptest #65 (See [https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/65/]) HIVE-5120 : document what hive.server2.thrift.sasl.qop values mean in hive-default.xml.template (Thejas Nair via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1515837) * /hive/trunk/conf/hive-default.xml.template document what hive.server2.thrift.sasl.qop values mean in hive-default.xml.template --- Key: HIVE-5120 URL: https://issues.apache.org/jira/browse/HIVE-5120 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.12.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Fix For: 0.12.0 Attachments: HIVE-5120.1.patch, HIVE-5120.2.patch Current description of configuration does not say what the values for the hive.server2.thrift.sasl.qop property mean, and also does not say that it works only with kerberos auth turned on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5121) Remove obsolete code on SemanticAnalyzer#genJoinTree
[
https://issues.apache.org/jira/browse/HIVE-5121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745302#comment-13745302
]
Hudson commented on HIVE-5121:
--
FAILURE: Integrated in Hive-trunk-hadoop2-ptest #65 (See
[https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/65/])
HIVE-5121 : Remove obsolete code on SemanticAnalyzer#genJoinTree (Azrael Park
via Ashutosh Chauhan) (hashutosh:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1515838)
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
Remove obsolete code on SemanticAnalyzer#genJoinTree
Key: HIVE-5121
URL: https://issues.apache.org/jira/browse/HIVE-5121
Project: Hive
Issue Type: Improvement
Affects Versions: 0.11.0
Environment: ubuntu 12.04
Reporter: Azrael Park
Assignee: Azrael Park
Priority: Trivial
Fix For: 0.12.0
Attachments: HIVE-5121.D12405.1.patch
Remove obsolete code on SemanticAnalyzer#genJoinTree.
{noformat}
children[1] = alias;
joinTree.setBaseSrc(children);
- aliasToOpInfo.get(alias);
joinTree.setId(qb.getId());
joinTree.getAliasToOpInfo().put(
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4779) Enhance coverage of package org.apache.hadoop.hive.ql.udf
[ https://issues.apache.org/jira/browse/HIVE-4779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745315#comment-13745315 ] Ashutosh Chauhan commented on HIVE-4779: Ran bucket5.q, didnt fail for me. Patch checked in trunk. Thanks, Ivan for the cleanup and tests.! Enhance coverage of package org.apache.hadoop.hive.ql.udf - Key: HIVE-4779 URL: https://issues.apache.org/jira/browse/HIVE-4779 Project: Hive Issue Type: Test Affects Versions: 0.12.0 Reporter: Ivan A. Veselovsky Assignee: Ivan A. Veselovsky Attachments: HIVE-4779.patch, HIVE-4779-trunk--N1.patch Enhance coverage of package org.apache.hadoop.hive.ql.udf up to 80%. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4779) Enhance coverage of package org.apache.hadoop.hive.ql.udf
[ https://issues.apache.org/jira/browse/HIVE-4779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-4779: --- Resolution: Fixed Fix Version/s: 0.12.0 Status: Resolved (was: Patch Available) Enhance coverage of package org.apache.hadoop.hive.ql.udf - Key: HIVE-4779 URL: https://issues.apache.org/jira/browse/HIVE-4779 Project: Hive Issue Type: Test Affects Versions: 0.12.0 Reporter: Ivan A. Veselovsky Assignee: Ivan A. Veselovsky Fix For: 0.12.0 Attachments: HIVE-4779.patch, HIVE-4779-trunk--N1.patch Enhance coverage of package org.apache.hadoop.hive.ql.udf up to 80%. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4844) Add char/varchar data types
[ https://issues.apache.org/jira/browse/HIVE-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745319#comment-13745319 ] Jason Dere commented on HIVE-4844: -- I'd expect those two patches to be very different as there were a number of changes made between the first patch here and the one on phabricator. What is the issue you see with the diff on phabricator? It was too large to upload using the arc command so I had to use the --less-context option. Add char/varchar data types --- Key: HIVE-4844 URL: https://issues.apache.org/jira/browse/HIVE-4844 Project: Hive Issue Type: New Feature Components: Types Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-4844.1.patch.hack Add new char/varchar data types which have support for more SQL-compliant behavior, such as SQL string comparison semantics, max length, etc. NO PRECOMMIT TESTS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
