Re: Asking for permissions to edit wiki

[Lefty Leverenz]

You now have wiki edit permissions.  Welcome to the Hive wiki team!

-- Lefty

On Thu, Jul 14, 2016 at 3:25 PM, Tao Li  wrote:

> Thanks Lefty. My username is: tli
>
> From: Lefty Leverenz 
> Date: Thursday, July 14, 2016 at 12:10 AM
> To: "dev@hive.apache.org" 
> Cc: Thejas Nair , Tao LI 
> Subject: Re: Asking for permissions to edit wiki
>
> Tao Li, you just need to get a Confluence account and tell us your
> username, as described here:  About This Wiki -- How to get permission to
> edit
> .
>
>
> -- Lefty
>
>
> On Wed, Jul 13, 2016 at 5:03 PM, Tao Li  wrote:
>
>> Hi,
>>
>> I am from Hortonworks and just started working on Hive. I would like to
>> be granted permissions to edit the wiki page so that I can make some
>> contributions there. Please help. Thanks.
>>
>
>

Re: Review Request 49881: HIVE-14204: Optimize loading loaddynamic partitions

[Rajesh Balamohan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49881/
---

(Updated July 15, 2016, 2:26 a.m.)


Review request for hive and Ashutosh Chauhan.


Changes
---

Reduced the number of calls to metastore.


Bugs: HIVE-14204
https://issues.apache.org/jira/browse/HIVE-14204


Repository: hive-git


Description
---

Lots of time is spent in sequential fashion to load dynamic partitioned dataset 
in driver side. 

E.g simple dynamic partitioned load as follows takes 300+ seconds

INSERT INTO web_sales_test partition(ws_sold_date_sk) select * from 
tpcds_bin_partitioned_orc_200.web_sales;

Time taken to load dynamic partitions: 309.22 seconds


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 82abd52 

Diff: https://reviews.apache.org/r/49881/diff/


Testing
---


Thanks,

Rajesh Balamohan

[jira] [Created] (HIVE-14245) NoClassDefFoundError when starting LLAP daemon

[Rajesh Balamohan (JIRA)]

Rajesh Balamohan created HIVE-14245:
---

 Summary: NoClassDefFoundError when starting LLAP daemon
 Key: HIVE-14245
 URL: https://issues.apache.org/jira/browse/HIVE-14245
 Project: Hive
  Issue Type: Bug
Reporter: Rajesh Balamohan
Priority: Minor


Env: hive master branch

{noformat}
2016-07-14T20:40:00,646  WARN [main[]] conf.Configuration: hive-site.xml:an 
attempt to override final parameter: 
hive.server2.tez.sessions.per.default.queue;  Ignoring.
2016-07-14T20:40:00,652  WARN [main[]] impl.LlapDaemon: Failed to start LLAP 
Daemon with exception
java.lang.NoClassDefFoundError: 
org/apache/hadoop/registry/client/binding/RegistryUtils$ServiceRecordMarshal
at 
org.apache.hadoop.hive.llap.registry.impl.LlapZookeeperRegistryImpl.(LlapZookeeperRegistryImpl.java:134)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.hive.llap.registry.impl.LlapRegistryService.serviceInit(LlapRegistryService.java:84)
 ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) 
~[hadoop-common-2.7.1.jar:?]
at 
org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.serviceStart(LlapDaemon.java:369)
 ~[hive-llap-server-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) 
~[hadoop-common-2.7.1.jar:?]
at 
org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.main(LlapDaemon.java:460) 
[hive-llap-server-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
Caused by: java.lang.ClassNotFoundException: 
org.apache.hadoop.registry.client.binding.RegistryUtils$ServiceRecordMarshal
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) 
~[?:1.8.0_65]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_65]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) 
~[?:1.8.0_65]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_65]
... 6 more
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: One Mapper runs in ACID tables

[Gopal Vijayaraghavan]

>Want to confirm why we  have implemented it in this way and if any
>correction in understanding is required please update

This is not implemented intentionally - the issue is that delta files show
up as 0 sized splits.

You should be getting exactly as many splits as you have buckets x
partitions in Hive-2.0.

This fix can't be backported to hive-1.2.1 without a corresponding Tez
upgrade to 0.7.x.




Cheers,
Gopal

One Mapper runs in ACID tables

[Ashish Singh]

Hi Team,

I have created ACID tables in Hive , while executing queries found a very
unusual behavior
Only one mapper runs if we have only delta files and in order to optimize
it we have to run major compactions to merge to base files

Want to confirm why we  have implemented it in this way and if any
correction in understanding is required please update






Thanks,
Ashish  Singh

Re: Review Request 49919: HIVE-14135 : beeline output not formatted correctly for large column widths

[Mohit Sabharwal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49919/#review142322
---


Ship it!




Ship It!

- Mohit Sabharwal


On July 14, 2016, 11:12 p.m., Vihang Karajgaonkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49919/
> ---
> 
> (Updated July 14, 2016, 11:12 p.m.)
> 
> 
> Review request for hive, Mohit Sabharwal, Sergio Pena, Sahil Takiar, and 
> Thejas Nair.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-14135 : beeline output not formatted correctly for large column widths
> 
> 
> Diffs
> -
> 
>   beeline/pom.xml a720d0835314221ec3bd9e8d354d148498ff794c 
>   beeline/src/java/org/apache/hive/beeline/BeeLineOpts.java 
> 5aaa38527734d46de037352ff51e54e0ae1cede0 
>   beeline/src/java/org/apache/hive/beeline/BufferedRows.java 
> 962c5319bb7e6e448979e1cef80a086cadd2ecc6 
>   beeline/src/test/org/apache/hive/beeline/TestBufferedRows.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/49919/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vihang Karajgaonkar
> 
>

[GitHub] hive pull request #86: HIVE-14242. Backport of ORC-53.

[omalley]

GitHub user omalley opened a pull request:

https://github.com/apache/hive/pull/86

HIVE-14242. Backport of ORC-53.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/omalley/hive hive-14242

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/86.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #86


commit c1170e9b8d7eeb6034551e978e9f17bf0f621ca2
Author: Owen O'Malley 
Date:   2016-07-14T23:53:41Z

HIVE-14242. Backport of ORC-53.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (HIVE-14244) bucketmap right outer join query throws ArrayIndexOutOfBoundsException

[Jagruti Varia (JIRA)]

Jagruti Varia created HIVE-14244:


 Summary: bucketmap right outer join query throws 
ArrayIndexOutOfBoundsException
 Key: HIVE-14244
 URL: https://issues.apache.org/jira/browse/HIVE-14244
 Project: Hive
  Issue Type: Bug
Affects Versions: 2.1.0, 1.3.0
Reporter: Jagruti Varia
Assignee: Zhiyuan Yang


bucketmap right outer join on partitioned bucketed table throws this error:
{noformat}
Vertex failed, vertexName=Map 1, vertexId=vertex_1466710232033_0539_6_00, 
diagnostics=[Task failed, taskId=task_1466710232033_0539_6_00_00, 
diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( 
failure ) : 
attempt_1466710232033_0539_6_00_00_0:java.lang.RuntimeException: 
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: 
Hive Runtime Error while processing row 
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
at 
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at 
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at 
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at 
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at 
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: 
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
processing row 
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:95)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:70)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:393)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:185)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error 
while processing row 
at 
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:850)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
... 17 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
java.lang.ArrayIndexOutOfBoundsException: -1
at 
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.process(ReduceSinkOperator.java:416)
at 
org.apache.hadoop.hive.ql.exec.vector.VectorReduceSinkOperator.process(VectorReduceSinkOperator.java:104)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:879)
at 
org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130)
at 
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:762)
... 18 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
at 
org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter.write(UnorderedPartitionedKVWriter.java:314)
at 
org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter.write(UnorderedPartitionedKVWriter.java:257)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor$TezKVOutputCollector.collect(TezProcessor.java:253)
at 
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.collect(ReduceSinkOperator.java:552)
at 
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.process(ReduceSinkOperator.java:398)
... 22 more
], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) : 
attempt_1466710232033_0539_6_00_00_1:java.lang.RuntimeException: 
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: 
Hive Runtime Error while processing row 
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
at 
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at 

Review Request 50054: CTAS with UNION ALL puts the wrong stats + count(*) = 0 in Tez

[pengcheng xiong]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50054/
---

Review request for hive and Ashutosh Chauhan.


Repository: hive-git


Description
---

HIVE-14236


Diffs
-

  itests/src/test/resources/testconfiguration.properties 73fcb03 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java 812a943 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java c626add 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java f555741 
  ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java 32bf24d 
  ql/src/java/org/apache/hadoop/hive/ql/plan/FileSinkDesc.java 07ed4fd 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsCollectionContext.java 
ae6f2ac 
  ql/src/java/org/apache/hadoop/hive/ql/stats/fs/FSStatsPublisher.java 3a49b30 
  ql/src/test/queries/clientpositive/union_stats.q PRE-CREATION 
  ql/src/test/results/clientpositive/tez/union_stats.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/union_stats.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/50054/diff/


Testing
---


Thanks,

pengcheng xiong

Re: Review Request 49919: HIVE-14135 : beeline output not formatted correctly for large column widths

[Vihang Karajgaonkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49919/
---

(Updated July 14, 2016, 11:12 p.m.)


Review request for hive, Mohit Sabharwal, Sergio Pena, Sahil Takiar, and Thejas 
Nair.


Changes
---

Added Mohit's suggestions.


Repository: hive-git


Description
---

HIVE-14135 : beeline output not formatted correctly for large column widths


Diffs (updated)
-

  beeline/pom.xml a720d0835314221ec3bd9e8d354d148498ff794c 
  beeline/src/java/org/apache/hive/beeline/BeeLineOpts.java 
5aaa38527734d46de037352ff51e54e0ae1cede0 
  beeline/src/java/org/apache/hive/beeline/BufferedRows.java 
962c5319bb7e6e448979e1cef80a086cadd2ecc6 
  beeline/src/test/org/apache/hive/beeline/TestBufferedRows.java PRE-CREATION 

Diff: https://reviews.apache.org/r/49919/diff/


Testing
---


Thanks,

Vihang Karajgaonkar

Review Request 50053: HIVE-14239: Change OperationManager to serve each session and make it threadsafe

[Aihua Xu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50053/
---

Review request for hive.


Repository: hive-git


Description
---

HIVE-14239: Change OperationManager to serve each session and make it threadsafe


Diffs
-

  
itests/hive-unit/src/test/java/org/apache/hive/service/cli/session/TestHiveSessionImpl.java
 d58a9133bfd6e1b56cc5c5206ed35aa057c7e69e 
  
itests/hive-unit/src/test/java/org/apache/hive/service/cli/session/TestQueryDisplay.java
 cc18ce79568941caeb20ea7e5708adbb13078547 
  service-rpc/if/TCLIService.thrift 5a9a785945b1396f4b4f124372abff5f15cfe785 
  service-rpc/src/gen/thrift/gen-cpp/TCLIService_types.h 
d23b3cd548e30786f0485425efedeb134910eda5 
  service-rpc/src/gen/thrift/gen-cpp/TCLIService_types.cpp 
0f53cb24c90fd621b21d48a9a7d895ff181e92f5 
  
service-rpc/src/gen/thrift/gen-javabean/org/apache/hive/service/rpc/thrift/TOperationHandle.java
 9eaf2be3ed5ea4b7a58a1626a1ad8b79961d184d 
  service-rpc/src/gen/thrift/gen-php/Types.php 
a6a257f08bcdf93a1a7f8446efac35a916050b45 
  service-rpc/src/gen/thrift/gen-py/TCLIService/ttypes.py 
fcd330f30dd6d310c036920a2ee1c0f74f03a782 
  service-rpc/src/gen/thrift/gen-rb/t_c_l_i_service_types.rb 
71148a0a84bc7a47b192db4b951e815fbd7bb5c9 
  service/src/java/org/apache/hive/service/cli/CLIService.java 
ed52b4a0a1995f2038669e187b0f3706e31d6340 
  service/src/java/org/apache/hive/service/cli/OperationHandle.java 
267a6f84872b6ba4e7fbe71aaabe2465322c996a 
  service/src/java/org/apache/hive/service/cli/operation/LogDivertAppender.java 
eaf1acbcfeb687eeebf1b3f7eed2099241fc46a2 
  service/src/java/org/apache/hive/service/cli/operation/Operation.java 
021c1fe598f45e41fce4bc136e953b83eeaa686f 
  service/src/java/org/apache/hive/service/cli/operation/OperationManager.java 
c26a6112db70ed216df25a952e1ab93c1a5c1de0 
  service/src/java/org/apache/hive/service/cli/session/HiveSessionBase.java 
9436a25a003ca8e1f0a6495485db569ecf3f0647 
  service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 
73416359ee2c8ab2d83af79ab21ec64e37cee1be 
  service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
4f5a8525beca0ff11071d0215a3b17b00bdc06f9 
  service/src/java/org/apache/hive/service/servlet/QueryProfileServlet.java 
8fa447a386c5a7a000b04453618f502e1681364e 
  service/src/resources/hive-webapps/hiveserver2/hiveserver2.jsp 
3c187b66702e0067cf11d99cdd9a7e358f24a50a 
  
service/src/test/org/apache/hive/service/cli/session/TestSessionGlobalInitFile.java
 55a325db7b154949dea4575d9ced013467d40f1f 

Diff: https://reviews.apache.org/r/50053/diff/


Testing
---


Thanks,

Aihua Xu

[jira] [Created] (HIVE-14243) Cleanup reflection in llap log routing

[Prasanth Jayachandran (JIRA)]

Prasanth Jayachandran created HIVE-14243:


 Summary: Cleanup reflection in llap log routing
 Key: HIVE-14243
 URL: https://issues.apache.org/jira/browse/HIVE-14243
 Project: Hive
  Issue Type: Bug
  Components: Logging
Affects Versions: 2.2.0
Reporter: Prasanth Jayachandran
Assignee: Prasanth Jayachandran


Follow up to fix reflection in HIVE-9756



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 49919: HIVE-14135 : beeline output not formatted correctly for large column widths

[Mohit Sabharwal]

> On July 14, 2016, 9:57 p.m., Vihang Karajgaonkar wrote:
> > beeline/src/java/org/apache/hive/beeline/BufferedRows.java, line 76
> > 
> >
> > Thanks for reviewing. I thought of adding it in the above loop like you 
> > mentioned, but then went against that for readability reasons since 
> > max.length should not be too big (it is numbers of columns in a row). I 
> > will add it now in the loop.
> > 
> > btw, If we have to add it in the loop it should be 
> > Math.min(Math.max(max[j], row.sizes[j] + 1), maxColumnWidth); Did you 
> > mean the same?

Ah, right, that should me min not max.


- Mohit


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49919/#review142292
---


On July 12, 2016, 5:51 p.m., Vihang Karajgaonkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49919/
> ---
> 
> (Updated July 12, 2016, 5:51 p.m.)
> 
> 
> Review request for hive, Mohit Sabharwal, Sergio Pena, Sahil Takiar, and 
> Thejas Nair.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-14135 : beeline output not formatted correctly for large column widths
> 
> 
> Diffs
> -
> 
>   beeline/pom.xml a720d0835314221ec3bd9e8d354d148498ff794c 
>   beeline/src/java/org/apache/hive/beeline/BeeLineOpts.java 
> 5aaa38527734d46de037352ff51e54e0ae1cede0 
>   beeline/src/java/org/apache/hive/beeline/BufferedRows.java 
> 962c5319bb7e6e448979e1cef80a086cadd2ecc6 
>   beeline/src/test/org/apache/hive/beeline/TestBufferedRows.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/49919/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vihang Karajgaonkar
> 
>

Re: Review Request 49919: HIVE-14135 : beeline output not formatted correctly for large column widths

[Vihang Karajgaonkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49919/#review142292
---




beeline/src/java/org/apache/hive/beeline/BufferedRows.java (line 76)


Thanks for reviewing. I thought of adding it in the above loop like you 
mentioned, but then went against that for readability reasons since max.length 
should not be too big (it is numbers of columns in a row). I will add it now in 
the loop.

btw, If we have to add it in the loop it should be 
Math.min(Math.max(max[j], row.sizes[j] + 1), maxColumnWidth); Did you mean 
the same?


- Vihang Karajgaonkar


On July 12, 2016, 5:51 p.m., Vihang Karajgaonkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49919/
> ---
> 
> (Updated July 12, 2016, 5:51 p.m.)
> 
> 
> Review request for hive, Mohit Sabharwal, Sergio Pena, Sahil Takiar, and 
> Thejas Nair.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-14135 : beeline output not formatted correctly for large column widths
> 
> 
> Diffs
> -
> 
>   beeline/pom.xml a720d0835314221ec3bd9e8d354d148498ff794c 
>   beeline/src/java/org/apache/hive/beeline/BeeLineOpts.java 
> 5aaa38527734d46de037352ff51e54e0ae1cede0 
>   beeline/src/java/org/apache/hive/beeline/BufferedRows.java 
> 962c5319bb7e6e448979e1cef80a086cadd2ecc6 
>   beeline/src/test/org/apache/hive/beeline/TestBufferedRows.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/49919/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vihang Karajgaonkar
> 
>

Re: Review Request 49919: HIVE-14135 : beeline output not formatted correctly for large column widths

[Mohit Sabharwal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49919/#review142287
---




beeline/src/java/org/apache/hive/beeline/BufferedRows.java (line 76)


wondering if this logic can be folded into the prior loop itself ?
 max((max[j], row.sizes[j] + 1)),maxColumnWidth) ?



beeline/src/test/org/apache/hive/beeline/TestBufferedRows.java (line 1)


Missing Apache header



beeline/src/test/org/apache/hive/beeline/TestBufferedRows.java (line 16)


nit: extra line


- Mohit Sabharwal


On July 12, 2016, 5:51 p.m., Vihang Karajgaonkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49919/
> ---
> 
> (Updated July 12, 2016, 5:51 p.m.)
> 
> 
> Review request for hive, Mohit Sabharwal, Sergio Pena, Sahil Takiar, and 
> Thejas Nair.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-14135 : beeline output not formatted correctly for large column widths
> 
> 
> Diffs
> -
> 
>   beeline/pom.xml a720d0835314221ec3bd9e8d354d148498ff794c 
>   beeline/src/java/org/apache/hive/beeline/BeeLineOpts.java 
> 5aaa38527734d46de037352ff51e54e0ae1cede0 
>   beeline/src/java/org/apache/hive/beeline/BufferedRows.java 
> 962c5319bb7e6e448979e1cef80a086cadd2ecc6 
>   beeline/src/test/org/apache/hive/beeline/TestBufferedRows.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/49919/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vihang Karajgaonkar
> 
>

Review Request 50049: HIVE-13966: DbNotificationListener: can loose DDL operation notifications

[Rahul Sharma]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50049/
---

Review request for hive.


Repository: hive-git


Description
---

HIVE-13966: DbNotificationListener: can loose DDL operation notifications


Diffs
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
bae1825e0699cc358349fda7731241adf0dcf2f6 
  
hcatalog/server-extensions/src/main/java/org/apache/hive/hcatalog/listener/DbNotificationListener.java
 172f58d435ba06b4c3df0344a3f1f6567a5e970c 
  
itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/TestTransactionalDbNotificationListener.java
 PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/AlterHandler.java 
dedd4497adfcc9d57090a943f6bb4f35ea87fa61 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 
7b8459556f54ad8d6e38526796c2ca0c48525cfb 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
c6c1e11429e7a4b4389d3d9ec12e02dadb986049 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
5adfa02b4dcdac5c77ef90f07a45d5dc5b7d9d2d 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 
bbd47b8442d70a81c0c60dc9a4adda943e18034a 
  
metastore/src/java/org/apache/hadoop/hive/metastore/TransactionalMetaStoreEventListener.java
 PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
c65c7a42eda4dfeb136dd5223fb9f527a6a87465 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 1ea72a0d4e7f5d7fa1df4cf44166bdf7565b1e59 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 3e6acc74a58db330de63b3fefde17752025135a8 

Diff: https://reviews.apache.org/r/50049/diff/


Testing
---


Thanks,

Rahul Sharma

[jira] [Created] (HIVE-14242) Backport ORC-53 to Hive

[Owen O'Malley (JIRA)]

Owen O'Malley created HIVE-14242:


 Summary: Backport ORC-53 to Hive
 Key: HIVE-14242
 URL: https://issues.apache.org/jira/browse/HIVE-14242
 Project: Hive
  Issue Type: Bug
  Components: ORC
Reporter: Owen O'Malley
Assignee: Owen O'Malley


ORC-53 was mostly about the mapreduce shims for ORC, but it fixed a problem in 
TypeDescription that should be backported to Hive.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14241) Acid clashes with ConfVars.HIVEFETCHTASKCONVERSION <> "none"

[Eugene Koifman (JIRA)]

Eugene Koifman created HIVE-14241:
-

 Summary: Acid clashes with ConfVars.HIVEFETCHTASKCONVERSION <> 
"none"
 Key: HIVE-14241
 URL: https://issues.apache.org/jira/browse/HIVE-14241
 Project: Hive
  Issue Type: Bug
  Components: Transactions
Affects Versions: 2.0.0, 1.3.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
Priority: Critical
 Fix For: 1.3.0, 2.2.0, 2.1.1


Some queries are optimized so as not to create an MR job. This somehow causes 
the Configuration object in FetchOperator to be passed to the operator before 
Driver.recordValidTxns() is called. So then to this op it looks like there are 
no valid txns and it returns nothing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Asking for permissions to edit wiki

[Tao Li]

Thanks Lefty. My username is: tli

From: Lefty Leverenz >
Date: Thursday, July 14, 2016 at 12:10 AM
To: "dev@hive.apache.org" 
>
Cc: Thejas Nair >, Tao LI 
>
Subject: Re: Asking for permissions to edit wiki

Tao Li, you just need to get a Confluence account and tell us your username, as 
described here:  About This Wiki -- How to get permission to 
edit.

-- Lefty


On Wed, Jul 13, 2016 at 5:03 PM, Tao Li 
> wrote:
Hi,

I am from Hortonworks and just started working on Hive. I would like to be 
granted permissions to edit the wiki page so that I can make some contributions 
there. Please help. Thanks.

[jira] [Created] (HIVE-14240) HoS itests shouldn't depend on a Spark distribution

[Sahil Takiar (JIRA)]

Sahil Takiar created HIVE-14240:
---

 Summary: HoS itests shouldn't depend on a Spark distribution
 Key: HIVE-14240
 URL: https://issues.apache.org/jira/browse/HIVE-14240
 Project: Hive
  Issue Type: Improvement
  Components: Spark
Affects Versions: 2.0.1, 2.1.0, 2.0.0
Reporter: Sahil Takiar
Assignee: Sahil Takiar


The HoS integration tests download a full Spark Distribution (a tar-ball) from 
CloudFront. It uses this distribution to run Spark locally. It runs a few tests 
with Spark in embedded mode, and some tests against a local Spark on YARN 
cluster. The {{itests/pom.xml}} actually contains scripts to download the 
tar-ball from a pre-defined location.

This is problematic because the Spark Distribution shades all its dependencies, 
including Hadoop dependencies. This can cause problems when upgrading the 
Hadoop version for Hive (ref: HIVE-13930).

Removing it will also avoid having to download the tar-ball during every build, 
and simplify the build process for the itests module.

The Hive itests should instead directly depend on Spark artifacts published in 
Maven Central. It will require some effort to get this working. The current 
Hive Spark Client uses a launch script in the Spark installation to run Spark 
jobs. The script basically does some setup work and invokes 
org.apache.spark.deploy.SparkSubmit. It is possible to invoke this class 
directly, which avoids the need to have a full Spark distribution available 
locally (in fact this option already exists, but isn't tested).

There may be other issues around classpath conflicts between Hive and Spark. 
For example, Hive and Spark require different versions of Kyro. One solution to 
this would be to take Spark artifacts and shade Kyro inside them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 49655: HIVE-12646: beeline and HIVE CLI do not parse ; in quote properly

[Sergio Pena]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49655/#review142247
---



The patch looks good. Could you add more tests to validate different cases?

For instance:
select ';'
select '";"'
select "';'"
select "\';\'"
select "\";\""
select '\';\''

- Sergio Pena


On July 12, 2016, 4:04 a.m., Sahil Takiar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49655/
> ---
> 
> (Updated July 12, 2016, 4:04 a.m.)
> 
> 
> Review request for hive, Sergio Pena and Yongzhi Chen.
> 
> 
> Bugs: HIVE-12646
> https://issues.apache.org/jira/browse/HIVE-12646
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-12646: beeline and HIVE CLI do not parse ; in quote properly
> 
> Approach:
> 
>   * Modified the `Commands.execute(...)` command to iterate throught the 
> given input line character by character
>   * It looks for single and double quotes in order to track when the iterator 
> is inside a quotation block
>   * If the iterator is inside a quotation block and it finds a semicolon, it 
> ignores it, otherwise it treats it as it normally would
>   * Moved the logic for parsing the line into a helper method called 
> `getCmList(...)` which is responsible for returning a `List` of commands that 
> need to be run
> 
> 
> Diffs
> -
> 
>   beeline/src/java/org/apache/hive/beeline/Commands.java 3a204c0 
>   
> itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java
>  ecfeddb 
> 
> Diff: https://reviews.apache.org/r/49655/diff/
> 
> 
> Testing
> ---
> 
> Add a unit tests which checks that Beeline can successfully run queries that 
> contain semi-colons inside quotation blocks. Confirmed existing unit tests 
> pass.
> 
> 
> Thanks,
> 
> Sahil Takiar
> 
>

[jira] [Created] (HIVE-14239) Change OperationManager to serve each session and make it threadsafe

[Aihua Xu (JIRA)]

Aihua Xu created HIVE-14239:
---

 Summary: Change OperationManager to serve each session and make it 
threadsafe
 Key: HIVE-14239
 URL: https://issues.apache.org/jira/browse/HIVE-14239
 Project: Hive
  Issue Type: Sub-task
  Components: HiveServer2
Affects Versions: 2.0.1
Reporter: Aihua Xu
Assignee: Aihua Xu


Currently OperationManager is designed to hold all the operations from all the 
sessions. It will cause the functions to unnecessarily sync between the 
operations across the sessions. And also there are some functions are not 
threadsafe which could be the cause for invalid OperationHandle.

Plan to change OperationManager to just serve one session and make it a helper 
class for HiveSession rather than a service which doesn't seem to  be needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14238) Ownership shouldn't be checked if external table location doesn't exist

[Rui Li (JIRA)]

Rui Li created HIVE-14238:
-

 Summary: Ownership shouldn't be checked if external table location 
doesn't exist
 Key: HIVE-14238
 URL: https://issues.apache.org/jira/browse/HIVE-14238
 Project: Hive
  Issue Type: Bug
Reporter: Rui Li
Assignee: Rui Li


When creating external table with SQL authorization, we require RWX permission 
+ ownership of the table location. If the location doesn't exist, we check on 
parent dir (recursively), which means we require the user owns everything under 
parent dir. I think this is not necessary - we don't have to check ownership of 
parent dir, or we just check non-recursively.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [jira] [Commented] (HIVE-14135) beeline output not formatted correctly for large column widths

[Lefty Leverenz]

That looks like a cut-&-paste error.  Here's what the wiki says for
--maxWidth:  "The maximum width to display before truncating data, in
characters, when *outputformat* is table. Default is to query the terminal
for current width, then fall back to 80."

-- Lefty

On Wed, Jul 13, 2016 at 8:15 PM, Vihang Karajgaonkar (JIRA)  wrote:

>
> [
> https://issues.apache.org/jira/browse/HIVE-14135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15376046#comment-15376046
> ]
>
> Vihang Karajgaonkar commented on HIVE-14135:
> 
>
> Thanks [~spena] for taking a look. Can you take a look at the latest
> review (version 2 in the review board)? The setup() method is removed.
>
> I think DEFAULT_MAX_WIDTH is the max line (row) width and
> DEFAULT_MAX_COLUMN_WIDTH is the max width for column. According to
> documentation
> https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients
>
> --maxWidth=MAXWIDTH The maximum column width, in characters, when
> outputformat is table. Default is 15.
> Usage: beeline --maxColumnWidth=25
>
> --maxColumnWidth=MAXCOLWIDTHThe maximum column width, in characters,
> when outputformat is table. Default is 15.
> Usage: beeline --maxColumnWidth=25
>
>
>
> > beeline output not formatted correctly for large column widths
> > --
> >
> > Key: HIVE-14135
> > URL: https://issues.apache.org/jira/browse/HIVE-14135
> > Project: Hive
> >  Issue Type: Bug
> >  Components: Beeline
> >Affects Versions: 2.2.0
> >Reporter: Vihang Karajgaonkar
> >Assignee: Vihang Karajgaonkar
> > Attachments: HIVE-14135.1.patch, HIVE-14135.2.patch,
> longKeyValues.txt, output_after.txt, output_before.txt
> >
> >
> > If the column width is too large then beeline uses the maximum column
> width when normalizing all the column widths. In order to reproduce the
> issue, run set -v;
> > Once the configuration variables is classpath which can be extremely
> large width (41k characters in my environment).
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>

Re: Asking for permissions to edit wiki

[Lefty Leverenz]

Tao Li, you just need to get a Confluence account and tell us your
username, as described here:  About This Wiki -- How to get permission to
edit

.

-- Lefty


On Wed, Jul 13, 2016 at 5:03 PM, Tao Li  wrote:

> Hi,
>
> I am from Hortonworks and just started working on Hive. I would like to be
> granted permissions to edit the wiki page so that I can make some
> contributions there. Please help. Thanks.
>

[jira] [Created] (HIVE-14237) Move stats collection logic out of transaction for create table

[Ashutosh Chauhan (JIRA)]

Ashutosh Chauhan created HIVE-14237:
---

 Summary: Move stats collection logic out of transaction for create 
table
 Key: HIVE-14237
 URL: https://issues.apache.org/jira/browse/HIVE-14237
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Affects Versions: 2.1.0, 2.0.0, 1.1.0, 1.2.0, 1.0.0
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan


Stats collection logic interacts with FS which may take long time. There is no 
need to do that inside DB transaction.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 50018: HIVE-13989: Extended ACLs are not handled according to specification

[Chris Drome]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50018/
---

Review request for hive.


Repository: hive-git


Description
---

HIVE-13989: Extended ACLs are not handled according to specification


Diffs
-

  
hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java
 9db3dc1b5d1d1354aaeb4850c6e7ec0b61682fe1 
  
itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/ql/security/TestExtendedAcls.java
 b7983797fed107aeb5e0bc53bc452cfaed95fdf9 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/FolderPermissionBase.java
 2ae9cc0cecf15ab03e3bad9ff298bad74ee6bbc0 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 
611266f88ac62ef3cbe8810c7dda2b70d582b94d 
  shims/common/src/main/java/org/apache/hadoop/hive/io/HdfsUtils.java 
70a6857464a38d9a425511b78b54d4231f131f1f 

Diff: https://reviews.apache.org/r/50018/diff/


Testing
---


Thanks,

Chris Drome