[jira] [Commented] (HIVE-4187) QL build-grammar target fails after HIVE-4148

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604118#comment-13604118
 ] 

Gunther Hagleitner commented on HIVE-4187:
--

I saw your comments on HIVE-4148 as well. Yes, I see your point, but I do think 
that dependencies specified in fewer places leads to something that's easier to 
maintain. 

My thinking was:

- I believe the modules aren't/can't be built in isolation anyways, so why 
duplicate all the deps 
- Nothing seems to enforce that each module has a complete set of deps, so it's 
best effort at best
- I see duplicated "hacks" and "broken pom file" stuff in the ivy scripts that 
I'd rather have in one place. 

The build speed is secondary, but I also don't understand how your system is 
faster with more dependencies to resolve than with fewer.

I am not married to HIVE-4148 either, although if the decision is to enforce 
that each module specifies all it deps directly, I'd like to go over the patch 
again and see what can and can't be removed rather than just reverting.

Having said that, I was simply jumping on this one first, because a broken 
build seems more urgent and I want to fix that right away. However, in order to 
do that I would like to know how your build is broken. Ivy should pick up the 
right version, the build machine doesn't have the problem and I can't reproduce 
it. Do you have any pointers?

> QL build-grammar target fails after HIVE-4148
> -
>
> Key: HIVE-4187
> URL: https://issues.apache.org/jira/browse/HIVE-4187
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Gunther Hagleitner
>Priority: Critical
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4196) Support for Streaming Partitions in Hive

[Roshan Naik (JIRA)]

Roshan Naik created HIVE-4196:
-

 Summary: Support for Streaming Partitions in Hive
 Key: HIVE-4196
 URL: https://issues.apache.org/jira/browse/HIVE-4196
 Project: Hive
  Issue Type: New Feature
  Components: Database/Schema, HCatalog
Affects Versions: 0.10.1
Reporter: Roshan Naik


Motivation: Allow Hive users to immediately query data streaming in through 
clients such as Flume.


Currently Hive partitions must be created after all the data for the partition 
is available. Thereafter, data in the partitions is considered immutable. 

This proposal introduces the notion of a streaming partition into which new 
files an be committed periodically and made available for queries before the 
partition is closed and converted into a standard partition.

The admin enables streaming partition on a table using DDL. He provides the 
following pieces of information:
- Name of the partition in the table on which streaming is enabled
- Frequency at which the streaming partition should be closed and converted 
into a standard partition.

Tables with streaming partition enabled will be partitioned by one and only one 
column. It is assumed that this column will contain a timestamp.

Closing the current streaming partition converts it into a standard partition. 
Based on the specified frequency, the current streaming partition  is closed 
and a new one created for future writes. This is referred to as 'rolling the 
partition'.


A streaming partition's life cycle is as follows:

 - A new streaming partition is instantiated for writes

 - Streaming clients request (via webhcat) for a HDFS file name into which they 
can write a chunk of records for a specific table.

 - Streaming clients write a chunk (via webhdfs) to that file and commit it(via 
webhcat). Committing merely indicates that the chunk has been written 
completely and ready for serving queries.  

 - When the partition is rolled, all committed chunks are swept into single 
directory and a standard partition pointing to that directory is created. The 
streaming partition is closed and new streaming partition is created. Rolling 
the partition is atomic. Streaming clients are agnostic of partition rolling.  

 - Hive queries will be able to query the partition that is currently open for 
streaming. only committed chunks will be visible. read consistency will be 
ensured so that repeated reads of the same partition will be idempotent for the 
lifespan of the query.



Partition rolling requires an active agent/thread running to check when it is 
time to roll and trigger the roll. This could be either be achieved by using an 
external agent such as Oozie (preferably) or an internal agent.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4187) QL build-grammar target fails after HIVE-4148

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604098#comment-13604098
 ] 

Carl Steinbach commented on HIVE-4187:
--

As far as I can tell the changes that were made in HIVE-4148
don't make the build any faster. I also think that relying on
Ivy's transitive dependency resolution mechanism to resolve
direct dependencies is a really bad idea, and actually makes
the build harder to maintain. Please let me know if you have
evidence to the contrary. Otherwise I would appreciate it if
you would resolve this problem by reverting HIVE-4148.

> QL build-grammar target fails after HIVE-4148
> -
>
> Key: HIVE-4187
> URL: https://issues.apache.org/jira/browse/HIVE-4187
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Gunther Hagleitner
>Priority: Critical
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4187) QL build-grammar target fails after HIVE-4148

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604096#comment-13604096
 ] 

Carl Steinbach commented on HIVE-4187:
--

Yes, I use 'ant clean package'. 

> QL build-grammar target fails after HIVE-4148
> -
>
> Key: HIVE-4187
> URL: https://issues.apache.org/jira/browse/HIVE-4187
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Gunther Hagleitner
>Priority: Critical
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4139) MiniDFS shim does not work for hadoop 2

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604090#comment-13604090
 ] 

Gunther Hagleitner commented on HIVE-4139:
--

- Was there a reason to add this flag -XX:+CMSClassUnloadingEnabled

When running the unit tests against the hadoop 2 minidfs/minimr stuff, you run 
out of PermGen space. This setting (as well as the one to increase the PermGen 
space for junit fixes the issue.

- Shim for DFS: There's a change in MiniDFS that changes the return type to a 
sub type of one of the public methods (getFilesystem). API compatible, but not 
binary compatible. By creating the shim, each version will be compiled against 
the right version of hadoop.

- Shim for MiniMRCluster: There was some really ugly code in QTestUtil, that 
used exceptions to distinguish between hadoop 1/2 and initialized the 
MiniMRcluster accordingly. By creating shims for these you can just init that 
stuff differently for each hadoop version (the apis are the same, but the 
behavior and configs have changed)

- Yes, these dependencies are needed. The build system won't pick them up 
otherwise.

> MiniDFS shim does not work for hadoop 2
> ---
>
> Key: HIVE-4139
> URL: https://issues.apache.org/jira/browse/HIVE-4139
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-4139.1.patch, HIVE-4139.2.patch, HIVE-4139.3.patch
>
>
> There's an incompatibility between hadoop 1 & 2 wrt to the MiniDfsCluster 
> class. That causes the hadoop 2 line Minimr tests to fail with a 
> "MethodNotFound" exception.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4172) JDBC2 does not support VOID type

[Prasad Mujumdar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad Mujumdar updated HIVE-4172:
--

Component/s: HiveServer2

> JDBC2 does not support VOID type
> 
>
> Key: HIVE-4172
> URL: https://issues.apache.org/jira/browse/HIVE-4172
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2, JDBC
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
>  Labels: HiveServer2
>
> In beeline, "select key, null from src" fails with exception,
> {noformat}
> org.apache.hive.service.cli.HiveSQLException: Error running query: 
> java.lang.NullPointerException
>   at 
> org.apache.hive.service.cli.operation.SQLOperation.run(SQLOperation.java:112)
>   at 
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveSessionImpl.java:166)
>   at 
> org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:148)
>   at 
> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:183)
>   at 
> org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1133)
>   at 
> org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1118)
>   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>   at 
> org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:39)
>   at 
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>   at java.lang.Thread.run(Thread.java:662)
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4187) QL build-grammar target fails after HIVE-4148

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604085#comment-13604085
 ] 

Gunther Hagleitner commented on HIVE-4187:
--

Downgraded my ant version to 1.8.1. Cleared all caches and built. Still don't 
see the problem.

> QL build-grammar target fails after HIVE-4148
> -
>
> Key: HIVE-4187
> URL: https://issues.apache.org/jira/browse/HIVE-4187
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Gunther Hagleitner
>Priority: Critical
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4194) JDBC2: HiveDriver should not throw RuntimeException when passed an invalid URL

[Prasad Mujumdar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad Mujumdar updated HIVE-4194:
--

Component/s: HiveServer2

> JDBC2: HiveDriver should not throw RuntimeException when passed an invalid URL
> --
>
> Key: HIVE-4194
> URL: https://issues.apache.org/jira/browse/HIVE-4194
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, JDBC
>Affects Versions: 0.11.0
>Reporter: Richard Ding
>Assignee: Richard Ding
>
> As per JDBC 3.0 Spec (section 9.2)
> "If the Driver implementation understands the URL, it will return a 
> Connection object; otherwise it returns null"
> Currently HiveConnection constructor will throw IllegalArgumentException if 
> url string doesn't start with "jdbc:hive2". This exception should be caught 
> by HiveDriver.connect and return null.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4188) TestJdbcDriver2.testDescribeTable failing consistently

[Prasad Mujumdar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad Mujumdar updated HIVE-4188:
--

Attachment: HIVE-4188-1.patch

> TestJdbcDriver2.testDescribeTable failing consistently
> --
>
> Key: HIVE-4188
> URL: https://issues.apache.org/jira/browse/HIVE-4188
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Tests
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Prasad Mujumdar
> Attachments: HIVE-4188-1.patch
>
>
> Running in Linux on a clean checkout after running ant very-clean package, 
> the test TestJdbcDriver2.testDescribeTable fails consistently with 
> Column name 'under_col' not found expected: but was:<# col_name >
> junit.framework.ComparisonFailure: Column name 'under_col' not found 
> expected: but was:<# col_name >
> at junit.framework.Assert.assertEquals(Assert.java:81)
> at 
> org.apache.hive.jdbc.TestJdbcDriver2.testDescribeTable(TestJdbcDriver2.java:815)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:154)
> at junit.framework.TestCase.runBare(TestCase.java:127)
> at junit.framework.TestResult$1.protect(TestResult.java:106)
> at junit.framework.TestResult.runProtected(TestResult.java:124)
> at junit.framework.TestResult.run(TestResult.java:109)
> at junit.framework.TestCase.run(TestCase.java:118)
> at junit.framework.TestSuite.runTest(TestSuite.java:208)
> at junit.framework.TestSuite.run(TestSuite.java:203)
> at 
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4188) TestJdbcDriver2.testDescribeTable failing consistently

[Prasad Mujumdar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad Mujumdar updated HIVE-4188:
--

Status: Patch Available  (was: Open)

Patch attached

> TestJdbcDriver2.testDescribeTable failing consistently
> --
>
> Key: HIVE-4188
> URL: https://issues.apache.org/jira/browse/HIVE-4188
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Tests
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Prasad Mujumdar
> Attachments: HIVE-4188-1.patch
>
>
> Running in Linux on a clean checkout after running ant very-clean package, 
> the test TestJdbcDriver2.testDescribeTable fails consistently with 
> Column name 'under_col' not found expected: but was:<# col_name >
> junit.framework.ComparisonFailure: Column name 'under_col' not found 
> expected: but was:<# col_name >
> at junit.framework.Assert.assertEquals(Assert.java:81)
> at 
> org.apache.hive.jdbc.TestJdbcDriver2.testDescribeTable(TestJdbcDriver2.java:815)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:154)
> at junit.framework.TestCase.runBare(TestCase.java:127)
> at junit.framework.TestResult$1.protect(TestResult.java:106)
> at junit.framework.TestResult.runProtected(TestResult.java:124)
> at junit.framework.TestResult.run(TestResult.java:109)
> at junit.framework.TestCase.run(TestCase.java:118)
> at junit.framework.TestSuite.runTest(TestSuite.java:208)
> at junit.framework.TestSuite.run(TestSuite.java:203)
> at 
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4187) QL build-grammar target fails after HIVE-4148

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604070#comment-13604070
 ] 

Gunther Hagleitner commented on HIVE-4187:
--

Looking at this right now. Not sure how to reproduce. Can you share how you 
build? Is it just "ant clean package"?

> QL build-grammar target fails after HIVE-4148
> -
>
> Key: HIVE-4187
> URL: https://issues.apache.org/jira/browse/HIVE-4187
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Gunther Hagleitner
>Priority: Critical
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4067) Followup to HIVE-701: reduce ambiguity in grammar

[Samuel Yuan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604068#comment-13604068
 ] 

Samuel Yuan commented on HIVE-4067:
---

The primary motivation was that it was tricky to add new keywords, which are 
almost always reserved words by default in Hive, because doing so could easily 
break existing queries. The changes for HIVE-701 make it easy to add 
non-reserved keywords in the future. HIVE-701 also removes the reserved status 
of most keywords, to prevent recently introduced keywords from breaking queries.

I can undo the changes for the Hive keywords which are reserved in SQL 2003, 
but would there be any reason to do so besides to adhere to the standard, given 
that the grammar can support leaving them non-reserved?

> Followup to HIVE-701: reduce ambiguity in grammar
> -
>
> Key: HIVE-4067
> URL: https://issues.apache.org/jira/browse/HIVE-4067
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.11.0
>Reporter: Samuel Yuan
>Assignee: Samuel Yuan
>Priority: Minor
> Attachments: HIVE-4067.D8883.1.patch
>
>
> After HIVE-701 the grammar has become much more ambiguous, and the 
> compilation generates a large number of warnings. Making FROM, DISTINCT, 
> PRESERVE, COLUMN, ALL, AND, OR, and NOT reserved keywords again reduces the 
> number of warnings to 134, up from the original 81 warnings but down from the 
> 565 after HIVE-701. Most of the remaining ambiguity is trivial, an example 
> being "KW_ELEM_TYPE | KW_KEY_TYPE | KW_VALUE_TYPE | identifier", and they are 
> all correctly handled by ANTLR.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4195) Avro SerDe causes incorrect behavior in unrelated tables

[Skye Wanderman-Milne (JIRA)]

Skye Wanderman-Milne created HIVE-4195:
--

 Summary: Avro SerDe causes incorrect behavior in unrelated tables
 Key: HIVE-4195
 URL: https://issues.apache.org/jira/browse/HIVE-4195
 Project: Hive
  Issue Type: Bug
Reporter: Skye Wanderman-Milne


When I run a file that first creates an Avro table using the Avro SerDe, then 
immediately creates an LZO text table and inserts data into the LZO table, the 
resulting LZO table contain Avro data files. When I remove the Avro CREATE 
TABLE statement, the LZO table contains .lzo files as expected.

{noformat}
DROP TABLE IF EXISTS avro_table;
CREATE EXTERNAL TABLE avro_table
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS
INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
TBLPROPERTIES ('avro.schema.literal' = '{
"namespace": "testing.hive.avro.serde",
"name": "test_record",
"type": "record",
"fields": [
{"name":"int1", "type":"long"},
{"name":"string1", "type":"string"}
]
}');

DROP TABLE IF EXISTS lzo_table;
CREATE EXTERNAL TABLE lzo_table (
id int,
bool_col boolean,
tinyint_col tinyint,
smallint_col smallint,
int_col int,
bigint_col bigint,
float_col float,
double_col double,
date_string_col string,
string_col string,
timestamp_col timestamp)
STORED AS 
INPUTFORMAT 'com.hadoop.mapred.DeprecatedLzoTextInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
;

SET hive.exec.compress.output=true;
SET mapred.output.compression.type=BLOCK;
SET mapred.output.compression.codec=com.hadoop.compression.lzo.LzopCodec;
SET hive.exec.dynamic.partition.mode=nonstrict;
SET hive.exec.dynamic.partition=true;
SET mapred.max.split.size=25600;
SET hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat;
insert overwrite table lzo_table SELECT id, bool_col, tinyint_col, 
smallint_col, int_col, bigint_col, float_col, double_col, date_string_col, 
string_col, timestamp_col FROM src_table;
{noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4194) JDBC2: HiveDriver should not throw RuntimeException when passed an invalid URL

[Richard Ding (JIRA)]

Richard Ding created HIVE-4194:
--

 Summary: JDBC2: HiveDriver should not throw RuntimeException when 
passed an invalid URL
 Key: HIVE-4194
 URL: https://issues.apache.org/jira/browse/HIVE-4194
 Project: Hive
  Issue Type: Bug
  Components: JDBC
Affects Versions: 0.11.0
Reporter: Richard Ding
Assignee: Richard Ding


As per JDBC 3.0 Spec (section 9.2)
"If the Driver implementation understands the URL, it will return a 
Connection object; otherwise it returns null"

Currently HiveConnection constructor will throw IllegalArgumentException if url 
string doesn't start with "jdbc:hive2". This exception should be caught by 
HiveDriver.connect and return null.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Merging HCatalog into Hive

[Carl Steinbach]

Hi Alan,

I committed HIVE-4145, created an HCatalog component on JIRA, and
updated the asf-authorization-template to give the HCatalog committers
karma on the hcatalog subdirectory. At this point I think everything should
be ready to go. Let me know if you run into any problems.

Thanks.

Carl

On Wed, Mar 13, 2013 at 11:56 AM, Alan Gates  wrote:

> Proposed changes look good to me.  And you don't need an infra ticket to
> grant karma.  Since you're Hive VP you can do it.  See
> http://www.apache.org/dev/pmc.html#SVNaccess
>
> Alan.
>
> On Mar 10, 2013, at 9:29 PM, Carl Steinbach wrote:
>
> > Hi Alan,
> >
> > I submitted a patch that creates the hcatalog directory and makes some
> other necessary
> > changes here:
> >
> > https://issues.apache.org/jira/browse/HIVE-4145
> >
> > Once this is committed I will contact ASFINFRA and ask them to grant the
> HCatalog
> > committers karma.
> >
> > Thanks.
> >
> > Carl
> >
> > On Sat, Mar 9, 2013 at 12:54 PM, Alan Gates 
> wrote:
> > Alright, I've gotten some feedback from Brock around the JIRA stuff and
> Carl in a live conversation expressed his desire to move hcat into the Hive
> namespace sooner rather than later.  So the proposal is that we'd move the
> code to org.apache.hive.hcatalog, though we would create shell classes and
> interfaces in org.apache.hcatalog for all public classes and interfaces so
> that it will be backward compatible.  I'm fine with doing this now.
> >
> > So, let's get started.  Carl, could you create an hcatalog directory
> under trunk/hive and grant the listed hcat committers karma on it?  Then
> I'll get started on moving the actual code.
> >
> > Alan.
> >
> > On Feb 24, 2013, at 12:22 PM, Brock Noland wrote:
> >
> > > Looks good from my perspective and I glad to see this moving forward.
> > >
> > > Regarding #4 (JIRA)
> > >
> > > "I don't know if there's a way to upload existing JIRAs into Hive's
> JIRA,
> > > but I think it would be better to leave them where they are."
> > >
> > > JIRA has a bulk move feature, but I am curious as why we would leave
> them
> > > under the old project? There might be good reason to orphan them, but
> my
> > > first thought is that it would be nice to have them under the HIVE
> project
> > > simply for search purposes.
> > >
> > > Brock
> > >
> > >
> > >
> > >
> > > On Fri, Feb 22, 2013 at 7:12 PM, Alan Gates 
> wrote:
> > >
> > >> Alright, our vote has passed, it's time to get on with merging
> HCatalog
> > >> into Hive.  Here's the things I can think of we need to deal with.
>  Please
> > >> add additional issues I've missed:
> > >>
> > >> 1) Moving the code
> > >> 2) Dealing with domain names in the code
> > >> 3) The mailing lists
> > >> 4) The JIRA
> > >> 5) The website
> > >> 6) Committer rights
> > >> 7) Make a proposal for how HCat is released going forward
> > >> 8) Publish an FAQ
> > >>
> > >> Proposals for how we handle these:
> > >> Below I propose an approach for how to handle each of these.  Feedback
> > >> welcome.
> > >>
> > >> 1) Moving the code
> > >> I propose that HCat move into a subdirectory of Hive.  This fits
> nicely
> > >> into Hive's structure since it already has metastore, ql, etc.  We'd
> just
> > >> add 'hcatalog' as a new directory.  This directory would contain
> hcatalog
> > >> as it is today.  It does not follow Hive's standard build model so
> we'd
> > >> need to do some work to make it so that building Hive also builds
> HCat, but
> > >> this should be minimal.
> > >>
> > >> 2) Dealing with domain names
> > >> HCat code currently is under org.apache.hcatalog.  Do we want to
> change
> > >> it?  In time we probably should change it to match the rest of Hive
> > >> (org.apache.hadoop.hive.hcatalog).  We need to do this in a backward
> > >> compatible way.  I propose we leave it as is for now and if we decide
> to in
> > >> the future we can move the actual code to
> org.apache.hadoop.hive.hcatalog
> > >> and create shell classes under org.apache.hcatalog.
> > >>
> > >> 3) The mailing lists
> > >> Given that our goal is to merge the projects and not create a
> subproject
> > >> we should merge the mailing lists rather than keep hcat specific
> lists.  We
> > >> can ask infra to remove hcatalog-*@incubator.apache.org and forward
> any
> > >> new mail to the appropriate Hive lists.  We need to find out if they
> can
> > >> auto-subscribe people from the hcat lists to the hive lists.  Given
> that
> > >> traffic on the Hive lists is an order of magnitude higher we should
> warn
> > >> people before we auto-subscribe them and allow them a chance to get
> off.
> > >>
> > >> 4) JIRA
> > >> We can create an hcatalog component in Hive's JIRA.  All new HCat
> issues
> > >> could be filed there.  I don't know if there's a way to upload
> existing
> > >> JIRAs into Hive's JIRA, but I think it would be better to leave them
> where
> > >> they are.  We should see if infra can turn off the ability to create
> new
> > >> JIRAs in hcatalog.
> > >>
> > >> 5) Website
> > >> We will ne

[jira] [Created] (HIVE-4193) OVER clauses with BETWEEN in the window definition produce wrong results

[Alan Gates (JIRA)]

Alan Gates created HIVE-4193:


 Summary: OVER clauses with BETWEEN in the window definition 
produce wrong results
 Key: HIVE-4193
 URL: https://issues.apache.org/jira/browse/HIVE-4193
 Project: Hive
  Issue Type: Bug
  Components: PTF-Windowing
Affects Versions: 0.11.0
Reporter: Alan Gates


Window queries that define a windowing clause that has a termination row often 
(though not all) return incorrect results.  For example, from our test queries 
all of the following return incorrect results:

{code}
select s, sum(f) over (partition by t order by b 
   rows between current row and unbounded following) 
from over100k;

select s, avg(f) over (partition by b order by d 
   rows between 5 preceding and current row) 
from over100k;

select s, avg(f) over (partition by bin order by s 
   rows between current row and 5 following) 
from over100k;

select s, avg(d) over (partition by i order by f desc 
   rows between 5 preceding and 5 following) 
from over100k;
{code}


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4145) Create hcatalog stub directory and add it to the build

[Carl Steinbach (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-4145:
-

Component/s: HCatalog

> Create hcatalog stub directory and add it to the build
> --
>
> Key: HIVE-4145
> URL: https://issues.apache.org/jira/browse/HIVE-4145
> Project: Hive
>  Issue Type: Task
>  Components: Build Infrastructure, HCatalog
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.11.0
>
> Attachments: HIVE-4145.1.patch.txt, HIVE-4145.2.patch.txt, 
> HIVE-4145.3.patch.txt
>
>
> Alan has requested that we create a directory for hcatalog and give the 
> HCatalog submodule committers karma on it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4145) Create hcatalog stub directory and add it to the build

[Carl Steinbach (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-4145:
-

   Resolution: Fixed
Fix Version/s: 0.11.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed to trunk.

> Create hcatalog stub directory and add it to the build
> --
>
> Key: HIVE-4145
> URL: https://issues.apache.org/jira/browse/HIVE-4145
> Project: Hive
>  Issue Type: Task
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.11.0
>
> Attachments: HIVE-4145.1.patch.txt, HIVE-4145.2.patch.txt, 
> HIVE-4145.3.patch.txt
>
>
> Alan has requested that we create a directory for hcatalog and give the 
> HCatalog submodule committers karma on it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4192) Use of LEAD in an OVER clauses causes the query to fail

[Alan Gates (JIRA)]

Alan Gates created HIVE-4192:


 Summary: Use of LEAD in an OVER clauses causes the query to fail
 Key: HIVE-4192
 URL: https://issues.apache.org/jira/browse/HIVE-4192
 Project: Hive
  Issue Type: Bug
  Components: PTF-Windowing
Affects Versions: 0.11.0
Reporter: Alan Gates


Running a query like:

{code}
select i, lead(s) over (partition by bin order by d desc 
rows between current row and 1 following) 
from over100k;
{code}

gives an error:

{code}
FAILED: SemanticException Function lead((TOK_TABLE_OR_COL s)) 
org.apache.hadoop.hive.ql.parse.WindowingSpec$WindowSpec@13e15f7 as _wcol0 
doesn't support windowing
{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4184) Document HiveServer2 setup under the admin documentation on hive wiki

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604020#comment-13604020
 ] 

Carl Steinbach commented on HIVE-4184:
--

One more thing: the hiveserver2 launch script doesn't have a .sh suffix.

> Document HiveServer2 setup under the admin documentation on hive wiki 
> --
>
> Key: HIVE-4184
> URL: https://issues.apache.org/jira/browse/HIVE-4184
> Project: Hive
>  Issue Type: Bug
>  Components: Documentation, HiveServer2
>Reporter: Prasad Mujumdar
>Assignee: Prasad Mujumdar
>
> The setup/configuration instructions for HiveServer are available on  
> https://cwiki.apache.org/confluence/display/Hive/AdminManual+SettingUpHiveServer
> We should include similar details for HiveServer2 configuration

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (HIVE-4184) Document HiveServer2 setup under the admin documentation on hive wiki

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604008#comment-13604008
 ] 

Carl Steinbach edited comment on HIVE-4184 at 3/16/13 12:03 AM:


[~prasadm] please go ahead and add this to the wiki. What you have looks good 
to me. One thing I'd like to add is that HiveServer2 was intentionally designed 
to be independent of the RPC layer. Currently we only support Thrift, but in 
the future we should also be able to support Avro or Protobufs without having 
to modify the upper layers of the system.

  was (Author: cwsteinbach):
@prasadm please go ahead and add this to the wiki. What you have looks good 
to me. One thing I'd like to add is that HiveServer2 was intentionally designed 
to be independent of the RPC layer. Currently we only support Thrift, but in 
the future we should also be able to support Avro or Protobufs without having 
to modify the upper layers of the system.
  
> Document HiveServer2 setup under the admin documentation on hive wiki 
> --
>
> Key: HIVE-4184
> URL: https://issues.apache.org/jira/browse/HIVE-4184
> Project: Hive
>  Issue Type: Bug
>  Components: Documentation, HiveServer2
>Reporter: Prasad Mujumdar
>Assignee: Prasad Mujumdar
>
> The setup/configuration instructions for HiveServer are available on  
> https://cwiki.apache.org/confluence/display/Hive/AdminManual+SettingUpHiveServer
> We should include similar details for HiveServer2 configuration

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4184) Document HiveServer2 setup under the admin documentation on hive wiki

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604008#comment-13604008
 ] 

Carl Steinbach commented on HIVE-4184:
--

@prasadm please go ahead and add this to the wiki. What you have looks good to 
me. One thing I'd like to add is that HiveServer2 was intentionally designed to 
be independent of the RPC layer. Currently we only support Thrift, but in the 
future we should also be able to support Avro or Protobufs without having to 
modify the upper layers of the system.

> Document HiveServer2 setup under the admin documentation on hive wiki 
> --
>
> Key: HIVE-4184
> URL: https://issues.apache.org/jira/browse/HIVE-4184
> Project: Hive
>  Issue Type: Bug
>  Components: Documentation, HiveServer2
>Reporter: Prasad Mujumdar
>Assignee: Prasad Mujumdar
>
> The setup/configuration instructions for HiveServer are available on  
> https://cwiki.apache.org/confluence/display/Hive/AdminManual+SettingUpHiveServer
> We should include similar details for HiveServer2 configuration

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4188) TestJdbcDriver2.testDescribeTable failing consistently

[Kevin Wilfong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604006#comment-13604006
 ] 

Kevin Wilfong commented on HIVE-4188:
-

Could you attach the patch to the JIRA and mark it Patch Available if it's 
ready for review.

> TestJdbcDriver2.testDescribeTable failing consistently
> --
>
> Key: HIVE-4188
> URL: https://issues.apache.org/jira/browse/HIVE-4188
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Tests
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Prasad Mujumdar
>
> Running in Linux on a clean checkout after running ant very-clean package, 
> the test TestJdbcDriver2.testDescribeTable fails consistently with 
> Column name 'under_col' not found expected: but was:<# col_name >
> junit.framework.ComparisonFailure: Column name 'under_col' not found 
> expected: but was:<# col_name >
> at junit.framework.Assert.assertEquals(Assert.java:81)
> at 
> org.apache.hive.jdbc.TestJdbcDriver2.testDescribeTable(TestJdbcDriver2.java:815)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:154)
> at junit.framework.TestCase.runBare(TestCase.java:127)
> at junit.framework.TestResult$1.protect(TestResult.java:106)
> at junit.framework.TestResult.runProtected(TestResult.java:124)
> at junit.framework.TestResult.run(TestResult.java:109)
> at junit.framework.TestCase.run(TestCase.java:118)
> at junit.framework.TestSuite.runTest(TestSuite.java:208)
> at junit.framework.TestSuite.run(TestSuite.java:203)
> at 
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4179) NonBlockingOpDeDup does not merge SEL operators correctly

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604007#comment-13604007
 ] 

Gunther Hagleitner commented on HIVE-4179:
--

https://reviews.facebook.net/D9471

> NonBlockingOpDeDup does not merge SEL operators correctly
> -
>
> Key: HIVE-4179
> URL: https://issues.apache.org/jira/browse/HIVE-4179
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-4179.1.patch, HIVE-4179.2.patch
>
>
> The input columns list for SEL operations isn't merged properly in the 
> optimization. The best way to see this is running union_remove_22.q with 
> -Dhadoop.mr.rev=23. The plan shows lost UDFs and a broken lineage for one 
> column.
> Note: union_remove tests do not run on hadoop 1 or 0.20.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4188) TestJdbcDriver2.testDescribeTable failing consistently

[Prasad Mujumdar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604005#comment-13604005
 ] 

Prasad Mujumdar commented on HIVE-4188:
---

Review request at https://reviews.facebook.net/D9477

> TestJdbcDriver2.testDescribeTable failing consistently
> --
>
> Key: HIVE-4188
> URL: https://issues.apache.org/jira/browse/HIVE-4188
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Tests
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Prasad Mujumdar
>
> Running in Linux on a clean checkout after running ant very-clean package, 
> the test TestJdbcDriver2.testDescribeTable fails consistently with 
> Column name 'under_col' not found expected: but was:<# col_name >
> junit.framework.ComparisonFailure: Column name 'under_col' not found 
> expected: but was:<# col_name >
> at junit.framework.Assert.assertEquals(Assert.java:81)
> at 
> org.apache.hive.jdbc.TestJdbcDriver2.testDescribeTable(TestJdbcDriver2.java:815)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:154)
> at junit.framework.TestCase.runBare(TestCase.java:127)
> at junit.framework.TestResult$1.protect(TestResult.java:106)
> at junit.framework.TestResult.runProtected(TestResult.java:124)
> at junit.framework.TestResult.run(TestResult.java:109)
> at junit.framework.TestCase.run(TestCase.java:118)
> at junit.framework.TestSuite.runTest(TestSuite.java:208)
> at junit.framework.TestSuite.run(TestSuite.java:203)
> at 
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4188) TestJdbcDriver2.testDescribeTable failing consistently

[Prasad Mujumdar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad Mujumdar updated HIVE-4188:
--

Component/s: HiveServer2

> TestJdbcDriver2.testDescribeTable failing consistently
> --
>
> Key: HIVE-4188
> URL: https://issues.apache.org/jira/browse/HIVE-4188
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Tests
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Prasad Mujumdar
>
> Running in Linux on a clean checkout after running ant very-clean package, 
> the test TestJdbcDriver2.testDescribeTable fails consistently with 
> Column name 'under_col' not found expected: but was:<# col_name >
> junit.framework.ComparisonFailure: Column name 'under_col' not found 
> expected: but was:<# col_name >
> at junit.framework.Assert.assertEquals(Assert.java:81)
> at 
> org.apache.hive.jdbc.TestJdbcDriver2.testDescribeTable(TestJdbcDriver2.java:815)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:154)
> at junit.framework.TestCase.runBare(TestCase.java:127)
> at junit.framework.TestResult$1.protect(TestResult.java:106)
> at junit.framework.TestResult.runProtected(TestResult.java:124)
> at junit.framework.TestResult.run(TestResult.java:109)
> at junit.framework.TestCase.run(TestCase.java:118)
> at junit.framework.TestSuite.runTest(TestSuite.java:208)
> at junit.framework.TestSuite.run(TestSuite.java:203)
> at 
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3861) Upgrade hbase dependency to 0.94.2

[Sushanth Sowmyan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603993#comment-13603993
 ] 

Sushanth Sowmyan commented on HIVE-3861:


I think this patch needs to be updated, it's not applying cleanly for me on 
trunk currently.

However, that said, one more fix is required for this patch - it brings in 
yammer which brings in a second version of slf4j, which causes two slf4j jars 
present in the hive/lib directory, which causes other commands like pig 
--useHCatalog to fail as an integration point.

It needs the yammer dependency in hbase-handler/ivy.xml additionally patched as 
follows:

{noformat}
-
+
+  
+
{noformat}


> Upgrade hbase dependency to 0.94.2
> --
>
> Key: HIVE-3861
> URL: https://issues.apache.org/jira/browse/HIVE-3861
> Project: Hive
>  Issue Type: Bug
>  Components: HBase Handler
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-3861.patch
>
>
> Hive tests fail to run against hbase v0.94.2. Proposing to upgrade the 
> dependency and change the test setup to properly work with the newer version.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4179) NonBlockingOpDeDup does not merge SEL operators correctly

[Gunther Hagleitner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-4179:
-

Status: Patch Available  (was: Open)

> NonBlockingOpDeDup does not merge SEL operators correctly
> -
>
> Key: HIVE-4179
> URL: https://issues.apache.org/jira/browse/HIVE-4179
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-4179.1.patch, HIVE-4179.2.patch
>
>
> The input columns list for SEL operations isn't merged properly in the 
> optimization. The best way to see this is running union_remove_22.q with 
> -Dhadoop.mr.rev=23. The plan shows lost UDFs and a broken lineage for one 
> column.
> Note: union_remove tests do not run on hadoop 1 or 0.20.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4179) NonBlockingOpDeDup does not merge SEL operators correctly

[Gunther Hagleitner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-4179:
-

Attachment: HIVE-4179.2.patch

> NonBlockingOpDeDup does not merge SEL operators correctly
> -
>
> Key: HIVE-4179
> URL: https://issues.apache.org/jira/browse/HIVE-4179
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-4179.1.patch, HIVE-4179.2.patch
>
>
> The input columns list for SEL operations isn't merged properly in the 
> optimization. The best way to see this is running union_remove_22.q with 
> -Dhadoop.mr.rev=23. The plan shows lost UDFs and a broken lineage for one 
> column.
> Note: union_remove tests do not run on hadoop 1 or 0.20.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4179) NonBlockingOpDeDup does not merge SEL operators correctly

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603970#comment-13603970
 ] 

Gunther Hagleitner commented on HIVE-4179:
--

Query:

{noformat}
insert overwrite table outputTbl1
SELECT a.key, concat(a.values, a.values), concat(a.values, a.values)
FROM (
  SELECT key, count(1) as values from inputTbl1 group by key
  UNION ALL
  SELECT key, count(1) as values from inputTbl1 group by key
) a;
{noformat}

Before:

{noformat}
  outputColumnNames: _col0, _col1
  Select Operator
expressions:
  expr: _col0
  type: string
  expr: UDFToLong(_col1)
  type: bigint
  expr: UDFToLong(_col2)
  type: bigint
outputColumnNames: _col0, _col1, _col2
{noformat}

After:

{noformat}
  outputColumnNames: _col0, _col1
  Select Operator
expressions:
  expr: _col0
  type: string
  expr: UDFToLong(concat(_col1, _col1))
  type: bigint
  expr: UDFToLong(concat(_col1, _col1))
  type: bigint
outputColumnNames: _col0, _col1, _col2
{noformat}

> NonBlockingOpDeDup does not merge SEL operators correctly
> -
>
> Key: HIVE-4179
> URL: https://issues.apache.org/jira/browse/HIVE-4179
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-4179.1.patch, HIVE-4179.2.patch
>
>
> The input columns list for SEL operations isn't merged properly in the 
> optimization. The best way to see this is running union_remove_22.q with 
> -Dhadoop.mr.rev=23. The plan shows lost UDFs and a broken lineage for one 
> column.
> Note: union_remove tests do not run on hadoop 1 or 0.20.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4138) ORC's union object inspector returns a type name that isn't parseable by TypeInfoUtils

[Kevin Wilfong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-4138:


Status: Open  (was: Patch Available)

I'm getting a failure from TestOrcStruct

> ORC's union object inspector returns a type name that isn't parseable by 
> TypeInfoUtils
> --
>
> Key: HIVE-4138
> URL: https://issues.apache.org/jira/browse/HIVE-4138
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Attachments: h-4138.patch, HIVE-4138.D9219.1.patch
>
>
> Currently the typename returned by ORC's union object inspector isn't 
> parseable by TypeInfoUtils. The format needs to be union.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4145) Create hcatalog stub directory and add it to the build

[Travis Crawford (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603941#comment-13603941
 ] 

Travis Crawford commented on HIVE-4145:
---

Hey [~cwsteinbach] - this looks good to create the initial directory. There 
will likely be some integration-related build changes but this will let us get 
started. +1

> Create hcatalog stub directory and add it to the build
> --
>
> Key: HIVE-4145
> URL: https://issues.apache.org/jira/browse/HIVE-4145
> Project: Hive
>  Issue Type: Task
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: HIVE-4145.1.patch.txt, HIVE-4145.2.patch.txt, 
> HIVE-4145.3.patch.txt
>
>
> Alan has requested that we create a directory for hcatalog and give the 
> HCatalog submodule committers karma on it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4191) describe table output always prints as if formatted keyword is specified

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-4191:


Attachment: HIVE-4191.1.patch

HIVE-4191.1.patch - a patch that changes format of default describe table 
command as hive 0.9

> describe table output always prints as if formatted keyword is specified
> 
>
> Key: HIVE-4191
> URL: https://issues.apache.org/jira/browse/HIVE-4191
> Project: Hive
>  Issue Type: Bug
>  Components: CLI, HiveServer2
>Affects Versions: 0.10.0
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Attachments: HIVE-4191.1.patch
>
>
> With the change in HIVE-3140, describe table output prints like the format 
> expected from "describe *formatted* table". ie, the headers are included and 
> there is padding with space for the fields. 
> This is a non backward compatible change, we should discuss if this change in 
> the formatting of output should remain. 
> This has impact on hiveserver2, it has been relying on the old format, and 
> with this change it prints additional headers and fields with space padding.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4191) describe table output always prints as if formatted keyword is specified

[Thejas M Nair (JIRA)]

Thejas M Nair created HIVE-4191:
---

 Summary: describe table output always prints as if formatted 
keyword is specified
 Key: HIVE-4191
 URL: https://issues.apache.org/jira/browse/HIVE-4191
 Project: Hive
  Issue Type: Bug
  Components: CLI, HiveServer2
Affects Versions: 0.10.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair


With the change in HIVE-3140, describe table output prints like the format 
expected from "describe *formatted* table". ie, the headers are included and 
there is padding with space for the fields. 
This is a non backward compatible change, we should discuss if this change in 
the formatting of output should remain. 
This has impact on hiveserver2, it has been relying on the old format, and with 
this change it prints additional headers and fields with space padding.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4145) Create hcatalog stub directory and add it to the build

[Kevin Wilfong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603905#comment-13603905
 ] 

Kevin Wilfong commented on HIVE-4145:
-

+1 thanks for fixing the other Eclipse issues

> Create hcatalog stub directory and add it to the build
> --
>
> Key: HIVE-4145
> URL: https://issues.apache.org/jira/browse/HIVE-4145
> Project: Hive
>  Issue Type: Task
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: HIVE-4145.1.patch.txt, HIVE-4145.2.patch.txt, 
> HIVE-4145.3.patch.txt
>
>
> Alan has requested that we create a directory for hcatalog and give the 
> HCatalog submodule committers karma on it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4190) OVER clauses with ORDER BY not getting windowing set properly

[Alan Gates (JIRA)]

Alan Gates created HIVE-4190:


 Summary: OVER clauses with ORDER BY not getting windowing set 
properly
 Key: HIVE-4190
 URL: https://issues.apache.org/jira/browse/HIVE-4190
 Project: Hive
  Issue Type: Bug
  Components: PTF-Windowing
Affects Versions: 0.11.0
Reporter: Alan Gates


Given a query like:

select s, avg(f) over (partition by si order by d) from over100k;

Hive is not setting the window frame properly.  The order by creates an 
implicit window frame of 'unbounded preceding' but Hive is treating the above 
query as if it has no window.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4145) Create hcatalog stub directory and add it to the build

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603890#comment-13603890
 ] 

Carl Steinbach commented on HIVE-4145:
--

@kevinwilfong: I updated the diff. Please take a look. Thanks.

> Create hcatalog stub directory and add it to the build
> --
>
> Key: HIVE-4145
> URL: https://issues.apache.org/jira/browse/HIVE-4145
> Project: Hive
>  Issue Type: Task
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: HIVE-4145.1.patch.txt, HIVE-4145.2.patch.txt, 
> HIVE-4145.3.patch.txt
>
>
> Alan has requested that we create a directory for hcatalog and give the 
> HCatalog submodule committers karma on it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4145) Create hcatalog stub directory and add it to the build

[Carl Steinbach (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-4145:
-

Attachment: HIVE-4145.3.patch.txt

> Create hcatalog stub directory and add it to the build
> --
>
> Key: HIVE-4145
> URL: https://issues.apache.org/jira/browse/HIVE-4145
> Project: Hive
>  Issue Type: Task
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: HIVE-4145.1.patch.txt, HIVE-4145.2.patch.txt, 
> HIVE-4145.3.patch.txt
>
>
> Alan has requested that we create a directory for hcatalog and give the 
> HCatalog submodule committers karma on it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: HIVE-4145. Create hcatalog stub directory and add it to the build

[Carl Steinbach]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9848/
---

(Updated March 15, 2013, 9:47 p.m.)


Review request for hive and Ashutosh Chauhan.


Description
---

This patch creates an hcatalog stub directory. Alan requested this. Once the 
patch is committed I will contact ASFINFRA and request that they grant karma on 
the directory to the hcatalog submodule committers.


This addresses bug HIVE-4145.
https://issues.apache.org/jira/browse/HIVE-4145


Diffs (updated)
-

  .gitignore e5383d4 
  build-common.xml cab3942 
  build.properties b96fe60 
  build.xml 9e656d6 
  common/src/gen/org/apache/hive/common/package-info.java dfba75d 
  eclipse-templates/.classpath f5c580b 
  hcatalog/build.xml PRE-CREATION 
  hcatalog/ivy.xml PRE-CREATION 
  hcatalog/src/java/org/apache/hive/hcatalog/package-info.java PRE-CREATION 
  hcatalog/src/test/.gitignore PRE-CREATION 

Diff: https://reviews.apache.org/r/9848/diff/


Testing
---


Thanks,

Carl Steinbach

[jira] [Updated] (HIVE-3958) support partial scan for analyze command - RCFile

[Gang Tim Liu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gang Tim Liu updated HIVE-3958:
---

Attachment: HIVE-3958.patch.1

> support partial scan for analyze command - RCFile
> -
>
> Key: HIVE-3958
> URL: https://issues.apache.org/jira/browse/HIVE-3958
> Project: Hive
>  Issue Type: Improvement
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Attachments: HIVE-3958.patch.1
>
>
> analyze commands allows us to collect statistics on existing 
> tables/partitions. It works great but might be slow since it scans all files.
> There are 2 ways to speed it up:
> 1. collect stats without file scan. It may not collect all stats but good and 
> fast enough for use case. HIVE-3917 addresses it
> 2. collect stats via partial file scan. It doesn't scan all content of files 
> but part of it to get file metadata. some examples are 
> https://cwiki.apache.org/Hive/rcfilecat.html for RCFile, ORC ( HIVE-3874 ) 
> and HFile of Hbase
> This jira is targeted to address the #2. More specifically RCFile format.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3958) support partial scan for analyze command - RCFile

[Gang Tim Liu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gang Tim Liu updated HIVE-3958:
---

Status: Patch Available  (was: In Progress)

> support partial scan for analyze command - RCFile
> -
>
> Key: HIVE-3958
> URL: https://issues.apache.org/jira/browse/HIVE-3958
> Project: Hive
>  Issue Type: Improvement
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Attachments: HIVE-3958.patch.1
>
>
> analyze commands allows us to collect statistics on existing 
> tables/partitions. It works great but might be slow since it scans all files.
> There are 2 ways to speed it up:
> 1. collect stats without file scan. It may not collect all stats but good and 
> fast enough for use case. HIVE-3917 addresses it
> 2. collect stats via partial file scan. It doesn't scan all content of files 
> but part of it to get file metadata. some examples are 
> https://cwiki.apache.org/Hive/rcfilecat.html for RCFile, ORC ( HIVE-3874 ) 
> and HFile of Hbase
> This jira is targeted to address the #2. More specifically RCFile format.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: HIVE-4145. Create hcatalog stub directory and add it to the build

[Carl Steinbach]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9848/
---

(Updated March 15, 2013, 9:34 p.m.)


Review request for hive and Ashutosh Chauhan.


Changes
---

Updated the diff to fix the eclipse template file.


Description
---

This patch creates an hcatalog stub directory. Alan requested this. Once the 
patch is committed I will contact ASFINFRA and request that they grant karma on 
the directory to the hcatalog submodule committers.


This addresses bug HIVE-4145.
https://issues.apache.org/jira/browse/HIVE-4145


Diffs (updated)
-

  build-common.xml cab3942 
  build.properties b96fe60 
  build.xml 9e656d6 
  eclipse-templates/.classpath f5c580b 
  hcatalog/build.xml PRE-CREATION 
  hcatalog/ivy.xml PRE-CREATION 
  hcatalog/src/java/org/apache/hive/hcatalog/package-info.java PRE-CREATION 
  hcatalog/src/test/.gitignore PRE-CREATION 

Diff: https://reviews.apache.org/r/9848/diff/


Testing
---


Thanks,

Carl Steinbach

[jira] [Updated] (HIVE-4145) Create hcatalog stub directory and add it to the build

[Carl Steinbach (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-4145:
-

Attachment: HIVE-4145.2.patch.txt

> Create hcatalog stub directory and add it to the build
> --
>
> Key: HIVE-4145
> URL: https://issues.apache.org/jira/browse/HIVE-4145
> Project: Hive
>  Issue Type: Task
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: HIVE-4145.1.patch.txt, HIVE-4145.2.patch.txt
>
>
> Alan has requested that we create a directory for hcatalog and give the 
> HCatalog submodule committers karma on it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4162) disable TestBeeLineDriver

[Kevin Wilfong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-4162:


Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed, thanks Thejas.

> disable TestBeeLineDriver
> -
>
> Key: HIVE-4162
> URL: https://issues.apache.org/jira/browse/HIVE-4162
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Fix For: 0.11.0
>
> Attachments: HIVE-4162.1.patch
>
>
> See HIVE-4161. We should disable the TestBeeLineDriver test cases. In its 
> current state, it was not supposed to be enabled by default.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4070) Like operator in Hive is case sensitive while in MySQL (and most likely other DBs) it's case insensitive

[Gwen Shapira (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603853#comment-13603853
 ] 

Gwen Shapira commented on HIVE-4070:


Oracle's "LIKE" (as well as any other char/varchar comparison) is case 
sensitive.
No matter how HiveQL behaves it can't be consistent with every SQL 
implementation out there. 

> Like operator in Hive is case sensitive while in MySQL (and most likely other 
> DBs) it's case insensitive
> 
>
> Key: HIVE-4070
> URL: https://issues.apache.org/jira/browse/HIVE-4070
> Project: Hive
>  Issue Type: Bug
>  Components: UDF
>Affects Versions: 0.10.0
>Reporter: Mark Grover
>Assignee: Mark Grover
> Fix For: 0.11.0
>
>
> Hive's like operator seems to be case sensitive.
> See 
> https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLike.java#L164
> However, MySQL's like operator is case insensitive. I don't have other DB's 
> (like PostgreSQL) installed and handy but I am guessing their LIKE is case 
> insensitive as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4070) Like operator in Hive is case sensitive while in MySQL (and most likely other DBs) it's case insensitive

[Gwen Shapira (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gwen Shapira updated HIVE-4070:
---

Priority: Trivial  (was: Major)

> Like operator in Hive is case sensitive while in MySQL (and most likely other 
> DBs) it's case insensitive
> 
>
> Key: HIVE-4070
> URL: https://issues.apache.org/jira/browse/HIVE-4070
> Project: Hive
>  Issue Type: Bug
>  Components: UDF
>Affects Versions: 0.10.0
>Reporter: Mark Grover
>Assignee: Mark Grover
>Priority: Trivial
> Fix For: 0.11.0
>
>
> Hive's like operator seems to be case sensitive.
> See 
> https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLike.java#L164
> However, MySQL's like operator is case insensitive. I don't have other DB's 
> (like PostgreSQL) installed and handy but I am guessing their LIKE is case 
> insensitive as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4184) Document HiveServer2 setup under the admin documentation on hive wiki

[Prasad Mujumdar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603831#comment-13603831
 ] 

Prasad Mujumdar commented on HIVE-4184:
---

[~cwsteinbach] I can add these initial versions to wiki and update it based on 
the feedback.

> Document HiveServer2 setup under the admin documentation on hive wiki 
> --
>
> Key: HIVE-4184
> URL: https://issues.apache.org/jira/browse/HIVE-4184
> Project: Hive
>  Issue Type: Bug
>  Components: Documentation, HiveServer2
>Reporter: Prasad Mujumdar
>Assignee: Prasad Mujumdar
>
> The setup/configuration instructions for HiveServer are available on  
> https://cwiki.apache.org/confluence/display/Hive/AdminManual+SettingUpHiveServer
> We should include similar details for HiveServer2 configuration

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4189) ORC fails with String column that ends in lots of nulls

[Kevin Wilfong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-4189:


Status: Patch Available  (was: Open)

> ORC fails with String column that ends in lots of nulls
> ---
>
> Key: HIVE-4189
> URL: https://issues.apache.org/jira/browse/HIVE-4189
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-4189.1.patch.txt
>
>
> When ORC attempts to write out a string column that ends in enough nulls to 
> span an index stride, StringTreeWriter's writeStripe method will get an 
> exception from TreeWriter's writeStripe method
> Column has wrong number of index entries found: x expected: y
> This is caused by rowIndexValueCount having multiple entries equal to the 
> number of non-null rows in the column, combined with the fact that 
> StringTreeWriter has special logic for constructing its index.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4189) ORC fails with String column that ends in lots of nulls

[Kevin Wilfong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603819#comment-13603819
 ] 

Kevin Wilfong commented on HIVE-4189:
-

https://reviews.facebook.net/D9465

> ORC fails with String column that ends in lots of nulls
> ---
>
> Key: HIVE-4189
> URL: https://issues.apache.org/jira/browse/HIVE-4189
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-4189.1.patch.txt
>
>
> When ORC attempts to write out a string column that ends in enough nulls to 
> span an index stride, StringTreeWriter's writeStripe method will get an 
> exception from TreeWriter's writeStripe method
> Column has wrong number of index entries found: x expected: y
> This is caused by rowIndexValueCount having multiple entries equal to the 
> number of non-null rows in the column, combined with the fact that 
> StringTreeWriter has special logic for constructing its index.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4189) ORC fails with String column that ends in lots of nulls

[Kevin Wilfong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-4189:


Attachment: HIVE-4189.1.patch.txt

> ORC fails with String column that ends in lots of nulls
> ---
>
> Key: HIVE-4189
> URL: https://issues.apache.org/jira/browse/HIVE-4189
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-4189.1.patch.txt
>
>
> When ORC attempts to write out a string column that ends in enough nulls to 
> span an index stride, StringTreeWriter's writeStripe method will get an 
> exception from TreeWriter's writeStripe method
> Column has wrong number of index entries found: x expected: y
> This is caused by rowIndexValueCount having multiple entries equal to the 
> number of non-null rows in the column, combined with the fact that 
> StringTreeWriter has special logic for constructing its index.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4186) NPE in ReduceSinkDeDuplication

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603816#comment-13603816
 ] 

Ashutosh Chauhan commented on HIVE-4186:


[~rhbutani] Can you add testcase in a .q file and add it in your patch ?

> NPE in ReduceSinkDeDuplication
> --
>
> Key: HIVE-4186
> URL: https://issues.apache.org/jira/browse/HIVE-4186
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Harish Butani
> Attachments: HIVE-4186.1.patch.txt
>
>
> When you have a sequence of RedueSinks on constants you get this error:
> {noformat}
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hive.ql.optimizer.ReduceSinkDeDuplication$ReduceSinkDeduplicateProcFactory$ReducerReducerProc.getPartitionAndKeyColumnMapping(ReduceSinkDeDuplication.java:416)
> {noformat}
> The e.g. to generate this si:
> {noformat}
> select p_name from (select p_name from part distribute by 1 sort by 1) p 
> distribute by 1 sort by 1
> {noformat}
> Sorry for the contrived e.g., but this actually happens when we stack 
> windowing clauses (see PTF-Windowing branch)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4189) ORC fails with String column that ends in lots of nulls

[Kevin Wilfong (JIRA)]

Kevin Wilfong created HIVE-4189:
---

 Summary: ORC fails with String column that ends in lots of nulls
 Key: HIVE-4189
 URL: https://issues.apache.org/jira/browse/HIVE-4189
 Project: Hive
  Issue Type: Bug
  Components: Serializers/Deserializers
Affects Versions: 0.11.0
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong


When ORC attempts to write out a string column that ends in enough nulls to 
span an index stride, StringTreeWriter's writeStripe method will get an 
exception from TreeWriter's writeStripe method

Column has wrong number of index entries found: x expected: y

This is caused by rowIndexValueCount having multiple entries equal to the 
number of non-null rows in the column, combined with the fact that 
StringTreeWriter has special logic for constructing its index.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4188) TestJdbcDriver2.testDescribeTable failing consistently

[Kevin Wilfong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603797#comment-13603797
 ] 

Kevin Wilfong commented on HIVE-4188:
-

Thanks Prasad

> TestJdbcDriver2.testDescribeTable failing consistently
> --
>
> Key: HIVE-4188
> URL: https://issues.apache.org/jira/browse/HIVE-4188
> Project: Hive
>  Issue Type: Bug
>  Components: Tests
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Prasad Mujumdar
>
> Running in Linux on a clean checkout after running ant very-clean package, 
> the test TestJdbcDriver2.testDescribeTable fails consistently with 
> Column name 'under_col' not found expected: but was:<# col_name >
> junit.framework.ComparisonFailure: Column name 'under_col' not found 
> expected: but was:<# col_name >
> at junit.framework.Assert.assertEquals(Assert.java:81)
> at 
> org.apache.hive.jdbc.TestJdbcDriver2.testDescribeTable(TestJdbcDriver2.java:815)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:154)
> at junit.framework.TestCase.runBare(TestCase.java:127)
> at junit.framework.TestResult$1.protect(TestResult.java:106)
> at junit.framework.TestResult.runProtected(TestResult.java:124)
> at junit.framework.TestResult.run(TestResult.java:109)
> at junit.framework.TestCase.run(TestCase.java:118)
> at junit.framework.TestSuite.runTest(TestSuite.java:208)
> at junit.framework.TestSuite.run(TestSuite.java:203)
> at 
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4188) TestJdbcDriver2.testDescribeTable failing consistently

[Prasad Mujumdar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603788#comment-13603788
 ] 

Prasad Mujumdar commented on HIVE-4188:
---

[~kevinwilfong] I will take a look.

> TestJdbcDriver2.testDescribeTable failing consistently
> --
>
> Key: HIVE-4188
> URL: https://issues.apache.org/jira/browse/HIVE-4188
> Project: Hive
>  Issue Type: Bug
>  Components: Tests
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Prasad Mujumdar
>
> Running in Linux on a clean checkout after running ant very-clean package, 
> the test TestJdbcDriver2.testDescribeTable fails consistently with 
> Column name 'under_col' not found expected: but was:<# col_name >
> junit.framework.ComparisonFailure: Column name 'under_col' not found 
> expected: but was:<# col_name >
> at junit.framework.Assert.assertEquals(Assert.java:81)
> at 
> org.apache.hive.jdbc.TestJdbcDriver2.testDescribeTable(TestJdbcDriver2.java:815)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:154)
> at junit.framework.TestCase.runBare(TestCase.java:127)
> at junit.framework.TestResult$1.protect(TestResult.java:106)
> at junit.framework.TestResult.runProtected(TestResult.java:124)
> at junit.framework.TestResult.run(TestResult.java:109)
> at junit.framework.TestCase.run(TestCase.java:118)
> at junit.framework.TestSuite.runTest(TestSuite.java:208)
> at junit.framework.TestSuite.run(TestSuite.java:203)
> at 
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-4188) TestJdbcDriver2.testDescribeTable failing consistently

[Prasad Mujumdar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad Mujumdar reassigned HIVE-4188:
-

Assignee: Prasad Mujumdar

> TestJdbcDriver2.testDescribeTable failing consistently
> --
>
> Key: HIVE-4188
> URL: https://issues.apache.org/jira/browse/HIVE-4188
> Project: Hive
>  Issue Type: Bug
>  Components: Tests
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Prasad Mujumdar
>
> Running in Linux on a clean checkout after running ant very-clean package, 
> the test TestJdbcDriver2.testDescribeTable fails consistently with 
> Column name 'under_col' not found expected: but was:<# col_name >
> junit.framework.ComparisonFailure: Column name 'under_col' not found 
> expected: but was:<# col_name >
> at junit.framework.Assert.assertEquals(Assert.java:81)
> at 
> org.apache.hive.jdbc.TestJdbcDriver2.testDescribeTable(TestJdbcDriver2.java:815)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:154)
> at junit.framework.TestCase.runBare(TestCase.java:127)
> at junit.framework.TestResult$1.protect(TestResult.java:106)
> at junit.framework.TestResult.runProtected(TestResult.java:124)
> at junit.framework.TestResult.run(TestResult.java:109)
> at junit.framework.TestCase.run(TestCase.java:118)
> at junit.framework.TestSuite.runTest(TestSuite.java:208)
> at junit.framework.TestSuite.run(TestSuite.java:203)
> at 
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
> at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4188) TestJdbcDriver2.testDescribeTable failing consistently

[Kevin Wilfong (JIRA)]

Kevin Wilfong created HIVE-4188:
---

 Summary: TestJdbcDriver2.testDescribeTable failing consistently
 Key: HIVE-4188
 URL: https://issues.apache.org/jira/browse/HIVE-4188
 Project: Hive
  Issue Type: Bug
  Components: Tests
Affects Versions: 0.11.0
Reporter: Kevin Wilfong


Running in Linux on a clean checkout after running ant very-clean package, the 
test TestJdbcDriver2.testDescribeTable fails consistently with 

Column name 'under_col' not found expected: but was:<# col_name >

junit.framework.ComparisonFailure: Column name 'under_col' not found 
expected: but was:<# col_name >
at junit.framework.Assert.assertEquals(Assert.java:81)
at 
org.apache.hive.jdbc.TestJdbcDriver2.testDescribeTable(TestJdbcDriver2.java:815)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:154)
at junit.framework.TestCase.runBare(TestCase.java:127)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: subscribe to your lists

[Alan Gates]

Send email to user-subscr...@hive.apache.org and dev-subscr...@hive.apache.org.

Alan.

On Mar 7, 2013, at 7:17 AM, Lin Picouleau wrote:

> Hi,
> 
> I would like to subscribe to your lists to get involved with Hive project.
> 
> Thank you!
> 
> Lin Picouleau

[jira] [Commented] (HIVE-4148) Cleanup aisle "ivy"

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603776#comment-13603776
 ] 

Carl Steinbach commented on HIVE-4148:
--

This patch is causing the build to fail on my machine.
I created HIVE-4187 to track this problem and assigned
it to Gunther.

I am able to fix the failure by adding antlr and antlr-runtime
back to ql/ivy.xml.

I think we should consider reverting this patch for the following
reasons:

* It makes maintenance harder since it converts explicit dependencies into 
transitive ones. For example, hive-exec has a direct compile-time dependency on 
the antlr parser generator, but it now relies on a transitive dependency via 
hive-metastore in order to satisfy this. This is also brittle since it means 
that hive-exec will break if the antlr dependency is removed from 
metastore/ivy.xml.
* I don't see any performance improvement with this change in place. I tried 
doing a fresh build several times with and without HIVE-4148, and the version 
without HIVE-4148 often finishes a couple seconds faster. This is pretty much 
what you would expect since Ivy should be using its local cache to resolve most 
of these dependencies.


> Cleanup aisle "ivy"
> ---
>
> Key: HIVE-4148
> URL: https://issues.apache.org/jira/browse/HIVE-4148
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Fix For: 0.11.0
>
> Attachments: HIVE-4148.patch
>
>
> Lot's of duplicated dependencies in the modules' ivy configs. Makes compiling 
> slow and maintenance hard. This patch cleans up these dependencies.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4145) Create hcatalog stub directory and add it to the build

[Kevin Wilfong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603754#comment-13603754
 ] 

Kevin Wilfong commented on HIVE-4145:
-

Could you add an entry to eclipse-templates/.classpath as well for hcatalog?

> Create hcatalog stub directory and add it to the build
> --
>
> Key: HIVE-4145
> URL: https://issues.apache.org/jira/browse/HIVE-4145
> Project: Hive
>  Issue Type: Task
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: HIVE-4145.1.patch.txt
>
>
> Alan has requested that we create a directory for hcatalog and give the 
> HCatalog submodule committers karma on it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4185) Document HiveServer2 JDBC and Beeline CLI in the user documentation

[Prasad Mujumdar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603741#comment-13603741
 ] 

Prasad Mujumdar commented on HIVE-4185:
---

JDBC -
Hive includes a new JDBC client driver for HiveServer2. It supports both 
embedded and remote access to HiveServer2.
The JDBC connection URL format has prefix is jdbc:hive2:// and the Driver class 
is org.apache.hive.jdbc.HiveDriver. Note that this is different from the old 
hiveserver. For remote server, the URL format is 
jdbc:hive2://:/. For embedded server, the URL format is 
jdbc:hive2:// (no host or port)
When connecting to HiveServer2 with kerberos authentication, the URL format is 
jdbc:hive2://:/;principal=. 
The client needs to have a valid Kerberos ticket in the ticket cache before 
connecting. In case of LDAP or customer pass through authentication, the client 
needs to pass the valid user name and password to JDBC connection API.
Beeline Shell -
Hive includes a new command shell Beeline that works with HiveServer2. Its a 
JDBC client that is based on SQLLine CLI (http://sqlline.sourceforge.net/). 
There’s an detailed documentation of the SQLLine options available at 
http://sqlline.sourceforge.net/#manual which is applicable to Beeline.

> Document HiveServer2 JDBC and Beeline CLI in the user documentation 
> 
>
> Key: HIVE-4185
> URL: https://issues.apache.org/jira/browse/HIVE-4185
> Project: Hive
>  Issue Type: Bug
>  Components: Documentation, HiveServer2
>Reporter: Prasad Mujumdar
>Assignee: Prasad Mujumdar
>
> The user documentation 
> https://cwiki.apache.org/confluence/display/Hive/HiveClient includes 
> information about client connection to thrift server.
> We need to include HiveServer2 client details (JDBC, Beeline)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4184) Document HiveServer2 setup under the admin documentation on hive wiki

[Prasad Mujumdar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603739#comment-13603739
 ] 

Prasad Mujumdar commented on HIVE-4184:
---

JDBC -
   Hive includes a new JDBC client driver for HiveServer2. It supports both 
embedded and remote access to HiveServer2.
The JDBC connection URL format has prefix is jdbc:hive2:// and the Driver class 
is  org.apache.hive.jdbc.HiveDriver. Note that this is different from the old 
hiveserver. For remote server, the URL format is 
jdbc:hive2://:/. For embedded server, the URL format is 
jdbc:hive2:// (no host or port)
When connecting to HiveServer2 with kerberos authentication, the URL format is 
jdbc:hive2://:/;principal=. 
The client needs to have a valid Kerberos ticket in the ticket cache before 
connecting. In case of LDAP or customer pass through authentication, the client 
needs to pass the valid user name and password to JDBC connection API.

Beeline Shell -
  Hive includes a new command shell Beeline that works with HiveServer2. Its a 
JDBC client that is based on SQLLine CLI (http://sqlline.sourceforge.net/). 
There’s an detailed documentation of the SQLLine options available at 
http://sqlline.sourceforge.net/#manual which is applicable to Beeline.

> Document HiveServer2 setup under the admin documentation on hive wiki 
> --
>
> Key: HIVE-4184
> URL: https://issues.apache.org/jira/browse/HIVE-4184
> Project: Hive
>  Issue Type: Bug
>  Components: Documentation, HiveServer2
>Reporter: Prasad Mujumdar
>Assignee: Prasad Mujumdar
>
> The setup/configuration instructions for HiveServer are available on  
> https://cwiki.apache.org/confluence/display/Hive/AdminManual+SettingUpHiveServer
> We should include similar details for HiveServer2 configuration

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4041) Support multiple partitionings in a single Query

[Phabricator (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603738#comment-13603738
 ] 

Phabricator commented on HIVE-4041:
---

hbutani has commented on the revision "HIVE-4041 [jira] Support multiple 
partitionings in a single Query".

INLINE COMMENTS
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java:212
 Yes the translation code could do with your review.
  We were not paying much attention to optimization at the time we wrote it.
  So the TableFuncDef holds onto ShapeDetails(input, output..)
  The Shape class has Serde props that we use to reconstruct the OIs during 
runtime.
  This happens in PTFTranslator. Read the translate for WindowingSpec method 
(line 138) in PTFTranslator.
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkDeDuplication.java:415
 Just added a Jira 4186 for this.
  ql/src/java/org/apache/hadoop/hive/ql/parse/PTFTranslator.java:482 yes I 
agree. I just don't want to make these changes in this Jira. Want to only add 
multi partition support here.
  ql/src/java/org/apache/hadoop/hive/ql/parse/WindowingComponentizer.java:38 I 
will add more comments. So there is:
  - 1 PTFOp
  - It can contain one or more PTF invocations.
  - When the PTF is WindowTableFunc, it can contain 1 or more UDAFs; they have 
the same partitioning.
  - During translation we create a WindowingSpec for each destination with 
Windowing(over clauses).
  - Here we then componentize the single WindowingSpec into multiple 
WindowingSpecs based on the partitioning.
  ql/src/java/org/apache/hadoop/hive/ql/parse/WindowingComponentizer.java:85 
Yes the having is to be removed. Haven't gotten around to it. Again didn't want 
to make this change in this Jira. It is on my todo.
  ql/src/test/queries/clientpositive/windowing_multipartitioning.q:21 I added 
all the multipartition tests from the spreadsheet except the ones that have no 
order. Those I will add once we resolve how to handle no order.

REVISION DETAIL
  https://reviews.facebook.net/D9381

To: JIRA, ashutoshc, hbutani


> Support multiple partitionings in a single Query
> 
>
> Key: HIVE-4041
> URL: https://issues.apache.org/jira/browse/HIVE-4041
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Harish Butani
>Assignee: Harish Butani
> Attachments: HIVE-4041.D9381.1.patch, WindowingComponentization.pdf
>
>
> Currently we disallow queries if the partition specifications of all Wdw fns 
> are not the same. We can relax this by generating multiple PTFOps based on 
> the unique partitionings in a Query. For partitionings that only differ in 
> sort, we can introduce a sort step in between PTFOps, which can happen in the 
> same Reduce task.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4187) QL build-grammar target fails after HIVE-4148

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603735#comment-13603735
 ] 

Carl Steinbach commented on HIVE-4187:
--

On my machine the build consistently fails during execution of the ql 
build-grammar target
with the following error:

{code}
build-grammar:
 [echo] Project: ql
 [echo] Building Grammar 
/Users/carl/Work/repos/hive-test/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
  
 [java] ANTLR Parser Generator  Version 3.0.1 (August 13, 2007)  1989-2007
 [java] 
/Users/carl/Work/repos/hive-test/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g
 [java] 
/Users/carl/Work/repos/hive-test/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g
 [java] error(100): 
/Users/carl/Work/repos/hive-test/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g:27:8:
 syntax error: antlr: 
/Users/carl/Work/repos/hive-test/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g:27:8:
 unexpected token: SelectClauseParser
 [java] error(100): 
/Users/carl/Work/repos/hive-test/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g:27:44:
 syntax error: antlr: 
/Users/carl/Work/repos/hive-test/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g:27:44:
 unexpected token: ,
{code}

This error was introduced by HIVE-4148. It appears that the changes in
HIVE-4148 cause Ant to put antlr-3.0.1.jar on the classpath instead of
antlr-3.4.jar

I'm using Ant 1.8.1


> QL build-grammar target fails after HIVE-4148
> -
>
> Key: HIVE-4187
> URL: https://issues.apache.org/jira/browse/HIVE-4187
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Gunther Hagleitner
>Priority: Critical
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4184) Document HiveServer2 setup under the admin documentation on hive wiki

[Prasad Mujumdar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603732#comment-13603732
 ] 

Prasad Mujumdar commented on HIVE-4184:
---

  HiveServer2 (HS2) is a thrift base server that enables remote clients to 
execute queries against hive and retrieve the results. Its an improved version 
of HiveServer that supports multi-client concurrency and authentication. It 
uses a new thrift interface with concurrency support. It is  designed to 
provide better support for open API clients like JDBC and ODBC. The thrift IDL 
is available at 
https://github.com/apache/hive/blob/trunk/service/if/TCLIService.thrift

How to configure -
  Configuration properties in hive-site.xml
hive.server2.thrift.min.worker.threads - Number of minimum worker threads, 
default 5.
hive.server2.thrift.max.worker.threads - Number of minimum worker threads, 
default 100
hive.server2.thrift.port - Tcp port to listen on , default 1
hive.server2.thrift.bind.host - Tcp interface to bind to

  Env 
HIVE_SERVER2_THRIFT_BIND_HOST - optional tcp host interface to bind to. 
Overrides the config file setting
HIVE_SERVER2_THRIFT_PORT - optional tcp port# to listen on, default 1. 
Overrides the config file setting

How to start
  hiveserver2.sh
  OR
  hive --service hiveserver2

Authentication -
  HiveServer2 support Anonymous (no auth), Kerberos, pass through LDAP and 
pluggable custom authentication.
  Configuration -
 hive.server2.authentication - Authentication mode, default NONE. Options 
are NONE, KERBEROS, LDAP and CUSTOM
 hive.server2.authentication.kerberos.principal - Kerberos principal for 
server
 hive.server2.authentication.kerberos.keytab - Keytab for server principal
 hive.server2.authentication.ldap.url - LDAP url
 hive.server2.authentication.ldap.baseDN - LDAP base DN
 hive.server2.custom.authentication.class - Custom authentication class 
that implements org.apache.hive.service.auth.PasswdAuthenticationProvider 
interface

Impersonation -
   By default HiveServer2 performs the query processing as the user that 
started the server process. It can be enabled to impersonate the connected 
user. 
   Configuration -
  hive.server2.enable.impersonation - Impersonate the connected user, 
default false
   OR
  hive.server2.enable.doAs - Impersonate the connected user, default false
  fs.hdfs.impl.disable.cache - Disable filesystem cache, default false


> Document HiveServer2 setup under the admin documentation on hive wiki 
> --
>
> Key: HIVE-4184
> URL: https://issues.apache.org/jira/browse/HIVE-4184
> Project: Hive
>  Issue Type: Bug
>  Components: Documentation, HiveServer2
>Reporter: Prasad Mujumdar
>Assignee: Prasad Mujumdar
>
> The setup/configuration instructions for HiveServer are available on  
> https://cwiki.apache.org/confluence/display/Hive/AdminManual+SettingUpHiveServer
> We should include similar details for HiveServer2 configuration

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4187) QL build-grammar target fails after HIVE-4148

[Carl Steinbach (JIRA)]

Carl Steinbach created HIVE-4187:


 Summary: QL build-grammar target fails after HIVE-4148
 Key: HIVE-4187
 URL: https://issues.apache.org/jira/browse/HIVE-4187
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Reporter: Carl Steinbach
Assignee: Gunther Hagleitner
Priority: Critical




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4186) NPE in ReduceSinkDeDuplication

[Harish Butani (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603728#comment-13603728
 ] 

Harish Butani commented on HIVE-4186:
-

patch is attached.

> NPE in ReduceSinkDeDuplication
> --
>
> Key: HIVE-4186
> URL: https://issues.apache.org/jira/browse/HIVE-4186
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Harish Butani
> Attachments: HIVE-4186.1.patch.txt
>
>
> When you have a sequence of RedueSinks on constants you get this error:
> {noformat}
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hive.ql.optimizer.ReduceSinkDeDuplication$ReduceSinkDeduplicateProcFactory$ReducerReducerProc.getPartitionAndKeyColumnMapping(ReduceSinkDeDuplication.java:416)
> {noformat}
> The e.g. to generate this si:
> {noformat}
> select p_name from (select p_name from part distribute by 1 sort by 1) p 
> distribute by 1 sort by 1
> {noformat}
> Sorry for the contrived e.g., but this actually happens when we stack 
> windowing clauses (see PTF-Windowing branch)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4186) NPE in ReduceSinkDeDuplication

[Harish Butani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Butani updated HIVE-4186:


Attachment: HIVE-4186.1.patch.txt

> NPE in ReduceSinkDeDuplication
> --
>
> Key: HIVE-4186
> URL: https://issues.apache.org/jira/browse/HIVE-4186
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Harish Butani
> Attachments: HIVE-4186.1.patch.txt
>
>
> When you have a sequence of RedueSinks on constants you get this error:
> {noformat}
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hive.ql.optimizer.ReduceSinkDeDuplication$ReduceSinkDeduplicateProcFactory$ReducerReducerProc.getPartitionAndKeyColumnMapping(ReduceSinkDeDuplication.java:416)
> {noformat}
> The e.g. to generate this si:
> {noformat}
> select p_name from (select p_name from part distribute by 1 sort by 1) p 
> distribute by 1 sort by 1
> {noformat}
> Sorry for the contrived e.g., but this actually happens when we stack 
> windowing clauses (see PTF-Windowing branch)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4186) NPE in ReduceSinkDeDuplication

[Harish Butani (JIRA)]

Harish Butani created HIVE-4186:
---

 Summary: NPE in ReduceSinkDeDuplication
 Key: HIVE-4186
 URL: https://issues.apache.org/jira/browse/HIVE-4186
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Harish Butani


When you have a sequence of RedueSinks on constants you get this error:
{noformat}
java.lang.NullPointerException
at 
org.apache.hadoop.hive.ql.optimizer.ReduceSinkDeDuplication$ReduceSinkDeduplicateProcFactory$ReducerReducerProc.getPartitionAndKeyColumnMapping(ReduceSinkDeDuplication.java:416)
{noformat}

The e.g. to generate this si:
{noformat}
select p_name from (select p_name from part distribute by 1 sort by 1) p 
distribute by 1 sort by 1
{noformat}

Sorry for the contrived e.g., but this actually happens when we stack windowing 
clauses (see PTF-Windowing branch)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4185) Document HiveServer2 JDBC and Beeline CLI in the user documentation

[Prasad Mujumdar (JIRA)]

Prasad Mujumdar created HIVE-4185:
-

 Summary: Document HiveServer2 JDBC and Beeline CLI in the user 
documentation 
 Key: HIVE-4185
 URL: https://issues.apache.org/jira/browse/HIVE-4185
 Project: Hive
  Issue Type: Bug
  Components: Documentation, HiveServer2
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar


The user documentation 
https://cwiki.apache.org/confluence/display/Hive/HiveClient includes 
information about client connection to thrift server.
We need to include HiveServer2 client details (JDBC, Beeline)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4184) Document HiveServer2 setup under the admin documentation on hive wiki

[Prasad Mujumdar (JIRA)]

Prasad Mujumdar created HIVE-4184:
-

 Summary: Document HiveServer2 setup under the admin documentation 
on hive wiki 
 Key: HIVE-4184
 URL: https://issues.apache.org/jira/browse/HIVE-4184
 Project: Hive
  Issue Type: Bug
  Components: Documentation, HiveServer2
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar


The setup/configuration instructions for HiveServer are available on  
https://cwiki.apache.org/confluence/display/Hive/AdminManual+SettingUpHiveServer
We should include similar details for HiveServer2 configuration

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4147) Slow Hive JDBC in concurrency mode to create/drop table

[Carl Steinbach (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-4147:
-

Component/s: HiveServer2

> Slow Hive JDBC in concurrency mode to create/drop table
> ---
>
> Key: HIVE-4147
> URL: https://issues.apache.org/jira/browse/HIVE-4147
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2, JDBC
>Affects Versions: 0.10.0
>Reporter: Alexander Alten-Lorenz
>Assignee: Brock Noland
>
> It's very slow using hive jdbc in concurrency mode to create/drop table, 
> which is 20 times slower than using HiveMetatstoreClient.
> test steps: 
> 1. create 100 different hive table one by one by using hive jdbc: "create 
> table .." 
> 2. drop table one by one by using hive jdbc: "drop table .." and timing 
> 3. create 100 different hive table one by one by using hive jdbc: "create 
> table .." 
> 4. drop tables one by one by using new 
> HiveMetatstoreClient().dropTable("default", "table_name") and timing
> results 
> step 2 is 20 times slower than step 4. 
> basically hive jdbc is 20 times slower than HiveMetatstoreClient not only 
> create/table, but also the same kind of calls.
> Dropping tables via this low level API could cause issues if there are any 
> clients concurrently querying the table. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4183) Implement Style changes to InspectorFactories

[Brock Noland (JIRA)]

Brock Noland created HIVE-4183:
--

 Summary: Implement Style changes to InspectorFactories
 Key: HIVE-4183
 URL: https://issues.apache.org/jira/browse/HIVE-4183
 Project: Hive
  Issue Type: Bug
Reporter: Brock Noland
Assignee: Brock Noland
Priority: Trivial


In HIVE-4141 we updated the InspectorFactories to use concurrent data 
structures. In the review Owen had requested some style updates. That is the 
original code typed the variables as HashMap and I just modified it to be 
ConcurrentHashMap. I updated the review item but forgot to attach the patch to 
JIRA so the patch which was committed did not have these style updates.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4145) Create hcatalog stub directory and add it to the build

[Gang Tim Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603604#comment-13603604
 ] 

Gang Tim Liu commented on HIVE-4145:


+1

> Create hcatalog stub directory and add it to the build
> --
>
> Key: HIVE-4145
> URL: https://issues.apache.org/jira/browse/HIVE-4145
> Project: Hive
>  Issue Type: Task
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: HIVE-4145.1.patch.txt
>
>
> Alan has requested that we create a directory for hcatalog and give the 
> HCatalog submodule committers karma on it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-4147) Slow Hive JDBC in concurrency mode to create/drop table

[Brock Noland (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland reassigned HIVE-4147:
--

Assignee: Brock Noland

> Slow Hive JDBC in concurrency mode to create/drop table
> ---
>
> Key: HIVE-4147
> URL: https://issues.apache.org/jira/browse/HIVE-4147
> Project: Hive
>  Issue Type: Improvement
>  Components: JDBC
>Affects Versions: 0.10.0
>Reporter: Alexander Alten-Lorenz
>Assignee: Brock Noland
>
> It's very slow using hive jdbc in concurrency mode to create/drop table, 
> which is 20 times slower than using HiveMetatstoreClient.
> test steps: 
> 1. create 100 different hive table one by one by using hive jdbc: "create 
> table .." 
> 2. drop table one by one by using hive jdbc: "drop table .." and timing 
> 3. create 100 different hive table one by one by using hive jdbc: "create 
> table .." 
> 4. drop tables one by one by using new 
> HiveMetatstoreClient().dropTable("default", "table_name") and timing
> results 
> step 2 is 20 times slower than step 4. 
> basically hive jdbc is 20 times slower than HiveMetatstoreClient not only 
> create/table, but also the same kind of calls.
> Dropping tables via this low level API could cause issues if there are any 
> clients concurrently querying the table. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4041) Support multiple partitionings in a single Query

[Phabricator (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603591#comment-13603591
 ] 

Phabricator commented on HIVE-4041:
---

ashutoshc has commented on the revision "HIVE-4041 [jira] Support multiple 
partitionings in a single Query".

  Some more questions.

INLINE COMMENTS
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkDeDuplication.java:415
 I see. I thought with following query I can simulate the same problem even on 
trunk.
  select 1 from over10k group by 1;

  But this didn't result in NPE and query ran successfully. Is this query good 
approximation to simulate this path ? My motivation is somehow to simulate this 
code path without over clause and thus expose bug on trunk and fix it there, so 
we don't need to do this in branch.
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java:212
 Hmm. I think we hold on to the schema for PTFOp way too early in semantic 
phase. Apart from changes required here, this holding on to the schema is not 
playing well with other compile time optimization which hive does after 
semantic analysis. Other operators don't do this. I think we need to spend a 
bit of time on this. Can you point to me where we hold on to schema in 
SemanticAnalyzer and why is it necessary?
  ql/src/java/org/apache/hadoop/hive/ql/parse/PTFTranslator.java:482 I am fine 
with doing it in follow-up. But if possible we should get rid of this. This 
probably result in runtime perf impact since I think this will force hadoop 
secondary sort so that values for a given key come out sorted. Further, adding 
extra constraints will lessen the opportunity to do compile time optimizations 
like filter push down (see my comments on HIVE-4180).
  ql/src/java/org/apache/hadoop/hive/ql/parse/WindowingComponentizer.java:38 It 
will be good to define group more concretely. If I am getting this right, this 
is group of over functions which has same partitioning. Is that correct ?
  So, a group may have multiple functions associated with it (but all on same 
partitioning). So, group -> one PTFOp on which there will be multiple functions 
working? Or a group implies multiple PTFOp chained in same reducer one after 
other each working on their own function.
  ql/src/java/org/apache/hadoop/hive/ql/parse/WindowingComponentizer.java:85 
Which filter this is? Is this having clause ? But I thought we already removed 
support for that. If not, I think we should. Or this regular where clause. If 
later, we should not consume other operators of query in PTOperator.
  ql/src/test/queries/clientpositive/windowing_multipartitioning.q:21 It will 
be good to add more tests from the google document which I shared with you. It 
has multipartitioning tests towards the end.

REVISION DETAIL
  https://reviews.facebook.net/D9381

To: JIRA, ashutoshc, hbutani


> Support multiple partitionings in a single Query
> 
>
> Key: HIVE-4041
> URL: https://issues.apache.org/jira/browse/HIVE-4041
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Harish Butani
>Assignee: Harish Butani
> Attachments: HIVE-4041.D9381.1.patch, WindowingComponentization.pdf
>
>
> Currently we disallow queries if the partition specifications of all Wdw fns 
> are not the same. We can relax this by generating multiple PTFOps based on 
> the unique partitionings in a Query. For partitionings that only differ in 
> sort, we can introduce a sort step in between PTFOps, which can happen in the 
> same Reduce task.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3963) Allow Hive to connect to RDBMS

[Maxime LANCIAUX (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603506#comment-13603506
 ] 

Maxime LANCIAUX commented on HIVE-3963:
---

I will work on improving the implementation soon so I am looking for any advice 
(especially how to remove the need to the DUAL table) ! Thanks.

> Allow Hive to connect to RDBMS
> --
>
> Key: HIVE-3963
> URL: https://issues.apache.org/jira/browse/HIVE-3963
> Project: Hive
>  Issue Type: New Feature
>  Components: Import/Export, JDBC, SQL, StorageHandler
>Affects Versions: 0.10.0, 0.9.1, 0.11.0
>Reporter: Maxime LANCIAUX
> Fix For: 0.10.1
>
> Attachments: patchfile
>
>
> I am thinking about something like :
> SELECT jdbcload('driver','url','user','password','sql') FROM dual;
> There is already a JIRA https://issues.apache.org/jira/browse/HIVE-1555 for 
> JDBCStorageHandler

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4182) doAS does not work with HiveServer2 in non-kerberos mode with local job

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-4182:


Attachment: HIVE-4182.1.patch

HIVE-4182.1.patch - the correct (initial) patch

> doAS does not work with HiveServer2 in non-kerberos mode with local job
> ---
>
> Key: HIVE-4182
> URL: https://issues.apache.org/jira/browse/HIVE-4182
> Project: Hive
>  Issue Type: Bug
>Reporter: Thejas M Nair
>  Labels: HiveServer2
> Attachments: HIVE-4182.1.patch
>
>
> When HiveServer2 is configured without kerberos security enabled, and the 
> query gets launched as a local map-reduce job, the job runs as the user hive 
> server is running as , instead of the user who submitted the query.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4182) doAS does not work with HiveServer2 in non-kerberos mode with local job

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-4182:


Attachment: (was: HIVE-4168.1.patch)

> doAS does not work with HiveServer2 in non-kerberos mode with local job
> ---
>
> Key: HIVE-4182
> URL: https://issues.apache.org/jira/browse/HIVE-4182
> Project: Hive
>  Issue Type: Bug
>Reporter: Thejas M Nair
>  Labels: HiveServer2
>
> When HiveServer2 is configured without kerberos security enabled, and the 
> query gets launched as a local map-reduce job, the job runs as the user hive 
> server is running as , instead of the user who submitted the query.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-1662) Add file pruning into Hive.

[Rajat Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603487#comment-13603487
 ] 

Rajat Jain commented on HIVE-1662:
--

We are following this patch because we wanted to implement something similar. 
Is it possible if we can do the pruning much before this step - in 
GenMapRedUtils.setTaskPlan. This will take care of the reducer estimation as 
well. Are there any issues with this?

> Add file pruning into Hive.
> ---
>
> Key: HIVE-1662
> URL: https://issues.apache.org/jira/browse/HIVE-1662
> Project: Hive
>  Issue Type: New Feature
>Reporter: He Yongqiang
>Assignee: Navis
> Attachments: HIVE-1662.D8391.1.patch, HIVE-1662.D8391.2.patch, 
> HIVE-1662.D8391.3.patch, HIVE-1662.D8391.4.patch
>
>
> now hive support filename virtual column. 
> if a file name filter presents in a query, hive should be able to only add 
> files which passed the filter to input paths.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4182) doAS does not work with HiveServer2 in non-kerberos mode with local job

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-4182:


Attachment: HIVE-4168.1.patch

HIVE-4168.1.patch - initial patch

> doAS does not work with HiveServer2 in non-kerberos mode with local job
> ---
>
> Key: HIVE-4182
> URL: https://issues.apache.org/jira/browse/HIVE-4182
> Project: Hive
>  Issue Type: Bug
>Reporter: Thejas M Nair
>  Labels: HiveServer2
> Attachments: HIVE-4168.1.patch
>
>
> When HiveServer2 is configured without kerberos security enabled, and the 
> query gets launched as a local map-reduce job, the job runs as the user hive 
> server is running as , instead of the user who submitted the query.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4182) doAS does not work with HiveServer2 in non-kerberos mode with local job

[Thejas M Nair (JIRA)]

Thejas M Nair created HIVE-4182:
---

 Summary: doAS does not work with HiveServer2 in non-kerberos mode 
with local job
 Key: HIVE-4182
 URL: https://issues.apache.org/jira/browse/HIVE-4182
 Project: Hive
  Issue Type: Bug
Reporter: Thejas M Nair


When HiveServer2 is configured without kerberos security enabled, and the query 
gets launched as a local map-reduce job, the job runs as the user hive server 
is running as , instead of the user who submitted the query.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-4141) InspectorFactories contains static HashMaps which can cause infinite loop

[Brock Noland (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland reassigned HIVE-4141:
--

Assignee: Brock Noland

> InspectorFactories contains static HashMaps which can cause infinite loop
> -
>
> Key: HIVE-4141
> URL: https://issues.apache.org/jira/browse/HIVE-4141
> Project: Hive
>  Issue Type: Sub-task
>  Components: Server Infrastructure
>Reporter: Brock Noland
>Assignee: Brock Noland
>Priority: Blocker
> Fix For: 0.11.0
>
> Attachments: HIVE-4141-1.patch, HIVE-4141-2.patch
>
>
> When many clients hit hs2, hs2 can get stuck in an infinite loop due to 
> concurrent modification of the static maps here:
> https://github.com/apache/hive/blob/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/LazyObjectInspectorFactory.java
> and in other ObjectFactories. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4141) InspectorFactories contains static HashMaps which can cause infinite loop

[Brock Noland (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1360#comment-1360
 ] 

Brock Noland commented on HIVE-4141:


Sorry looks like the first patch was committed. The differences were only style 
improvements over the original code so I don't think this is an issue. I'll 
file a follow up JIRA to implement the style improvements requested by Owen in 
the review.

> InspectorFactories contains static HashMaps which can cause infinite loop
> -
>
> Key: HIVE-4141
> URL: https://issues.apache.org/jira/browse/HIVE-4141
> Project: Hive
>  Issue Type: Sub-task
>  Components: Server Infrastructure
>Reporter: Brock Noland
>Priority: Blocker
> Fix For: 0.11.0
>
> Attachments: HIVE-4141-1.patch, HIVE-4141-2.patch
>
>
> When many clients hit hs2, hs2 can get stuck in an infinite loop due to 
> concurrent modification of the static maps here:
> https://github.com/apache/hive/blob/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/LazyObjectInspectorFactory.java
> and in other ObjectFactories. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4141) InspectorFactories contains static HashMaps which can cause infinite loop

[Brock Noland (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland updated HIVE-4141:
---

Attachment: HIVE-4141-2.patch

Uploaded this to RB a few days back but forgot to upload here.

> InspectorFactories contains static HashMaps which can cause infinite loop
> -
>
> Key: HIVE-4141
> URL: https://issues.apache.org/jira/browse/HIVE-4141
> Project: Hive
>  Issue Type: Sub-task
>  Components: Server Infrastructure
>Reporter: Brock Noland
>Priority: Blocker
> Fix For: 0.11.0
>
> Attachments: HIVE-4141-1.patch, HIVE-4141-2.patch
>
>
> When many clients hit hs2, hs2 can get stuck in an infinite loop due to 
> concurrent modification of the static maps here:
> https://github.com/apache/hive/blob/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/LazyObjectInspectorFactory.java
> and in other ObjectFactories. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira