[jira] [Updated] (HIVE-3073) Hive List Bucketing - DML support

2012-11-30 Thread Gang Tim Liu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gang Tim Liu updated HIVE-3073:
---

Attachment: HIVE-3073.patch.22

> Hive List Bucketing - DML support 
> --
>
> Key: HIVE-3073
> URL: https://issues.apache.org/jira/browse/HIVE-3073
> Project: Hive
>  Issue Type: New Feature
>  Components: SQL
>Affects Versions: 0.10.0
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Attachments: HIVE-3073.patch.12, HIVE-3073.patch.13, 
> HIVE-3073.patch.15, HIVE-3073.patch.18, HIVE-3073.patch.19, 
> HIVE-3073.patch.21, HIVE-3073.patch.22
>
>
> If a hive table column has skewed keys, query performance on non-skewed key 
> is always impacted. Hive List Bucketing feature will address it:
> https://cwiki.apache.org/Hive/listbucketing.html
> This jira issue will track DML change for the feature:
> 1. single skewed column
> 2. manual load data

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3724) Metastore tests use hardcoded ports

2012-11-30 Thread Ashutosh Chauhan (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-3724:
---

   Resolution: Fixed
Fix Version/s: 0.11
   Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks, Kevin!

> Metastore tests use hardcoded ports
> ---
>
> Key: HIVE-3724
> URL: https://issues.apache.org/jira/browse/HIVE-3724
> Project: Hive
>  Issue Type: Bug
>  Components: Tests
>Affects Versions: 0.10.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
>Priority: Minor
> Fix For: 0.11
>
> Attachments: HIVE-3724.1.patch.txt, HIVE-3724.2.patch.txt
>
>
> Several of the metastore tests use hardcoded ports for remote metastore 
> Thrift servers.  This is causing transient failures in Jenkins, e.g. 
> https://builds.apache.org/job/Hive-trunk-h0.21/1804/
> A few tests already dynamically determine free ports, and this logic can be 
> shared.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HIVE-3612) RetryingHMSHandler should retry on MetaExceptions if the cause was a JDOException

2012-11-30 Thread Ashutosh Chauhan (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan resolved HIVE-3612.


Resolution: Invalid

> RetryingHMSHandler should retry on MetaExceptions if the cause was a 
> JDOException
> -
>
> Key: HIVE-3612
> URL: https://issues.apache.org/jira/browse/HIVE-3612
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Affects Versions: 0.10.0
>Reporter: Kevin Wilfong
>Assignee: Jean Xu
>
> The RetryingHMSHandler would be much more effective if it retried on 
> MetaExceptions wrapped around JDOExceptions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3720) Expand and standardize authorization in Hive

2012-11-30 Thread Shreepadma Venugopalan (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507831#comment-13507831
 ] 

Shreepadma Venugopalan commented on HIVE-3720:
--

@Namit: The authorization model in this proposal mirrors that of MySQL as 
closely as possible. The proposal also documents wherever there is a deviation 
from MySQL's authorization model. Since Hive's data model is based on that of 
MySQL, it would make a lot of sense to base the authorization model on MySQL's 
as well. The proposed functionality is not necessarily a superset of the 
existing authorization functionality but subsumes some of the existing 
functionality. While the existing implementation supports authorization on some 
HiveQL operations, it doesn't secure all of the operations, provide a way to 
bootstrap the system etc. This proposal expands authorization to all HiveQL 
operations and direct metadata operations that can be performed by invoking the 
metastore Thrift API. 

As discussed earlier, since the proposed model standardizes the authorization 
model to mirror that of MySQL, it deviates from the existing model where ever 
the existing implementation deviates from the authorization model of MySQL or 
other RDBMSs. The proposed model is also more fine grained and supports 
hierarchical privileges much like an RDBMS. For instance, the proposed model 
supports CREATE, ALTER, DROP privileges on objects whereas the current model 
supports an ALTER_METADATA privilege that includes the privileges needed to 
perform CREATE, ALTER, DROP etc. Note that one of the goals is to propose an 
authorization model such that finer grained privileges can be added in as 
necessary later. 

Since the existing implementation is not complete, it unclear at this point 
what part of the functionality has been completely implemented. Perhaps we can 
mark the existing functionality in the wiki once we start implementing the 
proposed model. Thanks.

> Expand and standardize authorization in Hive
> 
>
> Key: HIVE-3720
> URL: https://issues.apache.org/jira/browse/HIVE-3720
> Project: Hive
>  Issue Type: Improvement
>  Components: Authorization
>Affects Versions: 0.9.0
>Reporter: Shreepadma Venugopalan
>Assignee: Shreepadma Venugopalan
> Attachments: Hive_Authorization_Functionality.pdf
>
>
> The existing implementation of authorization in Hive is not complete. 
> Additionally the existing implementation has security holes. This JIRA is an 
> umbrella JIRA  for a) extending authorization to all SQL operations and 
> direct metadata operations, and b) standardizing the authorization model and 
> its semantics to mirror that of MySQL as closely as possible.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2380) Add Binary Datatype in Hive

2012-11-30 Thread Chuck Connell (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507810#comment-13507810
 ] 

Chuck Connell commented on HIVE-2380:
-

Let me state my earlier comment another way... If the BINARY column type 
"works" in narrow sense, but it is impossible to get binary data from a file 
into Hive, then this feature is essentially useless. So we need some 
explanation for how to use the existing DDL features to support BINARY columns, 
or a new file format that allows it.

> Add Binary Datatype in Hive
> ---
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
>  Issue Type: New Feature
>  Components: Serializers/Deserializers
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch, 
> hive-2380_4.patch, hive-2380.patch
>
>
> Add bytearray as a primitive data type.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1827 - Fixed

2012-11-30 Thread Apache Jenkins Server 
Changes for Build #1784
[kevinwilfong] HIVE-3675. NaN does not work correctly for round(n). (njain via 
kevinwilfong)

[cws] HIVE-3651. bucketmapjoin?.q tests fail with hadoop 0.23 (Prasad Mujumdar 
via cws)


Changes for Build #1785
[namit] HIVE-3613 Implement grouping_id function
(Ian Gorbachev via namit)

[namit] HIVE-3692 Update parallel test documentation
(Ivan Gorbachev via namit)

[namit] HIVE-3649 Hive List Bucketing - enhance DDL to specify list bucketing 
table
(Gang Tim Liu via namit)


Changes for Build #1786
[namit] HIVE-3696 Revert HIVE-3483 which causes performance regression
(Gang Tim Liu via namit)


Changes for Build #1787
[kevinwilfong] HIVE-3621. Make prompt in Hive CLI configurable. (Jingwei Lu via 
kevinwilfong)

[kevinwilfong] HIVE-3695. TestParse breaks due to HIVE-3675. (njain via 
kevinwilfong)


Changes for Build #1788
[kevinwilfong] HIVE-3557. Access to external URLs in hivetest.py. (Ivan 
Gorbachev via kevinwilfong)


Changes for Build #1789
[hashutosh] HIVE-3662 : TestHiveServer: testScratchDirShouldClearWhileStartup 
is failing on Windows (Kanna Karanam via Ashutosh Chauhan)

[hashutosh] HIVE-3659 : TestHiveHistory::testQueryloglocParentDirNotExist Test 
fails on Windows because of some resource leaks in ZK (Kanna Karanam via 
Ashutosh Chauhan)

[hashutosh] HIVE-3663 Unable to display the MR Job file path on Windows in case 
of MR job failures.  (Kanna Karanam via Ashutosh Chauhan)


Changes for Build #1790

Changes for Build #1791

Changes for Build #1792

Changes for Build #1793
[hashutosh] HIVE-3704 : name of some metastore scripts are not per convention 
(Ashutosh Chauhan)


Changes for Build #1794
[hashutosh] HIVE-3243 : ignore white space between entries of hive/hbase table 
mapping (Shengsheng Huang via Ashutosh Chauhan)

[hashutosh] HIVE-3215 : JobDebugger should use RunningJob.getTrackingURL 
(Bhushan Mandhani via Ashutosh Chauhan)


Changes for Build #1795
[cws] HIVE-3437. 0.23 compatibility: fix unit tests when building against 0.23 
(Chris Drome via cws)

[hashutosh] HIVE-3626 : RetryingHMSHandler should wrap JDOException inside 
MetaException (Bhushan Mandhani via Ashutosh Chauhan)

[hashutosh] HIVE-3560 : Hive always prints a warning message when using remote 
metastore (Travis Crawford via Ashutosh Chauhan)


Changes for Build #1796

Changes for Build #1797
[hashutosh] HIVE-3664 : Avoid to create a symlink for hive-contrib.jar file in 
dist\lib folder. (Kanna Karanam via Ashutosh Chauhan)


Changes for Build #1798
[namit] HIVE-3706 getBoolVar in FileSinkOperator can be optimized
(Kevin Wilfong via namit)

[namit] HIVE-3707 Round map/reduce progress down when it is in the range [99.5, 
100)
(Kevin Wilfong via namit)

[namit] HIVE-3471 Implement grouping sets in hive
(Ivan Gorbachev via namit)


Changes for Build #1799
[hashutosh] HIVE-3291 : fix fs resolvers (Ashish Singh via Ashutosh Chauhan)

[hashutosh] HIVE-3680 : Include Table information in Hive's AddPartitionEvent. 
(Mithun Radhakrishnan via Ashutosh Chauhan)


Changes for Build #1800
[hashutosh] HIVE-3520 : ivysettings.xml does not let you override 
.m2/repository (Raja Aluri via Ashutosh Chauhan)

[hashutosh] HIVE-3435 : Get pdk pluginTest passed when triggered from both 
builtin tests and pdk tests on hadoop23  (Zhenxiao Luo via Ashutosh Chauhan)


Changes for Build #1801

Changes for Build #1802

Changes for Build #1803

Changes for Build #1804

Changes for Build #1805

Changes for Build #1806
[hashutosh] Preparing for 0.11 development

[kevinwilfong] HIVE-3647. map-side groupby wrongly due to HIVE-3432. (njain via 
kevinwilfong)


Changes for Build #1807
[kevinwilfong] HIVE-3679. Unescape partition names returned by show partitions. 
(Wonho Kim via kevinwilfong)


Changes for Build #1808
[kevinwilfong] Remove extra files accidentally committed with HIVE-3679. 
(kevinwilfong)


Changes for Build #1809

Changes for Build #1810

Changes for Build #1811
[namit] HIVE-3735 PTest doesn't work due to hive snapshot version upgrade to 11
(Gang Tim Liu via namit)

[namit] HIVE-3722 Create index fails on CLI using remote metastore
(Kevin Wilfong via namit)


Changes for Build #1812
[namit] HIVE-3703 Hive Query Explain Plan JSON not being created properly
(Mayank Garg via namit)

[namit] HIVE-3721 ALTER TABLE ADD PARTS should check for valid partition spec 
and throw a SemanticException
if part spec is not valid (Pamela Vagata via namit)


Changes for Build #1813

Changes for Build #1814
[hashutosh] HIVE-3698 : enhanceModel.notRequired is incorrectly determined 
(Chris Drome via Ashutosh Chauhan)


Changes for Build #1815

Changes for Build #1816

Changes for Build #1817
[hashutosh] HIVE-3736 : hive unit test case build failure. (Ashish Singh via 
Ashutosh Chauhan)


Changes for Build #1818

Changes for Build #1819

Changes for Build #1820

Changes for Build #1821
[hashutosh] HIVE-3749 : New test cases added by HIVE-3676 in insert1.q is not 
deterministic (Navis via Ashutosh Chauhan)

[hashutosh] HIVE-3648 : Hive

[jira] [Updated] (HIVE-3073) Hive List Bucketing - DML support

2012-11-30 Thread Gang Tim Liu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gang Tim Liu updated HIVE-3073:
---

Status: Patch Available  (was: In Progress)

patch is available.

> Hive List Bucketing - DML support 
> --
>
> Key: HIVE-3073
> URL: https://issues.apache.org/jira/browse/HIVE-3073
> Project: Hive
>  Issue Type: New Feature
>  Components: SQL
>Affects Versions: 0.10.0
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Attachments: HIVE-3073.patch.12, HIVE-3073.patch.13, 
> HIVE-3073.patch.15, HIVE-3073.patch.18, HIVE-3073.patch.19, HIVE-3073.patch.21
>
>
> If a hive table column has skewed keys, query performance on non-skewed key 
> is always impacted. Hive List Bucketing feature will address it:
> https://cwiki.apache.org/Hive/listbucketing.html
> This jira issue will track DML change for the feature:
> 1. single skewed column
> 2. manual load data

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3073) Hive List Bucketing - DML support

2012-11-30 Thread Gang Tim Liu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gang Tim Liu updated HIVE-3073:
---

Attachment: HIVE-3073.patch.21

> Hive List Bucketing - DML support 
> --
>
> Key: HIVE-3073
> URL: https://issues.apache.org/jira/browse/HIVE-3073
> Project: Hive
>  Issue Type: New Feature
>  Components: SQL
>Affects Versions: 0.10.0
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Attachments: HIVE-3073.patch.12, HIVE-3073.patch.13, 
> HIVE-3073.patch.15, HIVE-3073.patch.18, HIVE-3073.patch.19, HIVE-3073.patch.21
>
>
> If a hive table column has skewed keys, query performance on non-skewed key 
> is always impacted. Hive List Bucketing feature will address it:
> https://cwiki.apache.org/Hive/listbucketing.html
> This jira issue will track DML change for the feature:
> 1. single skewed column
> 2. manual load data

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Work started] (HIVE-3073) Hive List Bucketing - DML support

2012-11-30 Thread Gang Tim Liu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HIVE-3073 started by Gang Tim Liu.

> Hive List Bucketing - DML support 
> --
>
> Key: HIVE-3073
> URL: https://issues.apache.org/jira/browse/HIVE-3073
> Project: Hive
>  Issue Type: New Feature
>  Components: SQL
>Affects Versions: 0.10.0
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Attachments: HIVE-3073.patch.12, HIVE-3073.patch.13, 
> HIVE-3073.patch.15, HIVE-3073.patch.18, HIVE-3073.patch.19, HIVE-3073.patch.21
>
>
> If a hive table column has skewed keys, query performance on non-skewed key 
> is always impacted. Hive List Bucketing feature will address it:
> https://cwiki.apache.org/Hive/listbucketing.html
> This jira issue will track DML change for the feature:
> 1. single skewed column
> 2. manual load data

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3400) Add Retries to Hive MetaStore Connections

2012-11-30 Thread Bhushan Mandhani (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-3400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507768#comment-13507768
 ] 

Bhushan Mandhani commented on HIVE-3400:


Ashutosh, we no longer need HIVE-3612. Jean is about to abandon that diff. I 
think we should keep these RetryingRawStore changes here since 
RetryingHMSHandler already catches JDOExceptions. But I can take it out if you 
prefer that.

> Add Retries to Hive MetaStore Connections
> -
>
> Key: HIVE-3400
> URL: https://issues.apache.org/jira/browse/HIVE-3400
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Bhushan Mandhani
>Assignee: Bhushan Mandhani
>Priority: Minor
>  Labels: metastore
> Attachments: HIVE-3400.1.patch.txt, HIVE-3400.2.patch.txt
>
>
> Currently, when using Thrift to access the MetaStore, if the Thrift host 
> dies, there is no mechanism to reconnect to some other host even if the 
> MetaStore URIs variable in the Conf contains multiple hosts. Hive should 
> retry and reconnect rather than throwing a communication link error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3612) RetryingHMSHandler should retry on MetaExceptions if the cause was a JDOException

2012-11-30 Thread Jean Xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean Xu updated HIVE-3612:
--

Status: Open  (was: Patch Available)

This jira is not needed any more . 

> RetryingHMSHandler should retry on MetaExceptions if the cause was a 
> JDOException
> -
>
> Key: HIVE-3612
> URL: https://issues.apache.org/jira/browse/HIVE-3612
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Affects Versions: 0.10.0
>Reporter: Kevin Wilfong
>Assignee: Jean Xu
>
> The RetryingHMSHandler would be much more effective if it retried on 
> MetaExceptions wrapped around JDOExceptions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2380) Add Binary Datatype in Hive

2012-11-30 Thread Chuck Connell (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507761#comment-13507761
 ] 

Chuck Connell commented on HIVE-2380:
-

I am trying to use this feature (BINARY columns) and I believe I have the 
perfect use-case for it, but I am missing something. 

Here is the background... I have some files that each contain just one logical 
field, which is a binary object. (The files are Google Protobuf format.) I want 
to put these binary files into a larger file, where each protobuf is a logical 
record. Then I want to define a Hive table that stores each protobuf as one 
row, with the entire protobuf object in one BINARY column. Then I will use a 
custom UDF to select/query the binary object. 

This is about as simple as can be for putting binary data into Hive. But all of 
the test cases for this jira seem to draw the binary columns from another 
existing table and CAST them. I want to load the files from disk.

What file format should I use to package the binary rows? What should the Hive 
table definition be? I cannot use TEXTFILE, since the binary may contain 
newlines. Many of my attempts have choked on the newlines.

Thanks very much,
Chuck Connell
Nuance
Burlington, MA



> Add Binary Datatype in Hive
> ---
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
>  Issue Type: New Feature
>  Components: Serializers/Deserializers
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch, 
> hive-2380_4.patch, hive-2380.patch
>
>
> Add bytearray as a primitive data type.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3747) Provide hive operation name for hookContext

2012-11-30 Thread Shreepadma Venugopalan (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shreepadma Venugopalan updated HIVE-3747:
-

Attachment: HIVE-3747.1.patch.txt

> Provide hive operation name for hookContext
> ---
>
> Key: HIVE-3747
> URL: https://issues.apache.org/jira/browse/HIVE-3747
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Sudhanshu Arora
>Assignee: Shreepadma Venugopalan
> Attachments: HIVE-3747.1.patch.txt
>
>
> The hookContext exposed through ExecuteWithHookContext, does not provide the 
> name of the Hive operation. 
> The following public API should be added in HookContext.
> public String getOperationName() {
> return SessionState.get().getHiveOperation().name();
> }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3747) Provide hive operation name for hookContext

2012-11-30 Thread Shreepadma Venugopalan (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shreepadma Venugopalan updated HIVE-3747:
-

Status: Patch Available  (was: Open)

> Provide hive operation name for hookContext
> ---
>
> Key: HIVE-3747
> URL: https://issues.apache.org/jira/browse/HIVE-3747
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Sudhanshu Arora
>Assignee: Shreepadma Venugopalan
> Attachments: HIVE-3747.1.patch.txt
>
>
> The hookContext exposed through ExecuteWithHookContext, does not provide the 
> name of the Hive operation. 
> The following public API should be added in HookContext.
> public String getOperationName() {
> return SessionState.get().getHiveOperation().name();
> }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-3747) Provide hive operation name for hookContext

2012-11-30 Thread Shreepadma Venugopalan (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shreepadma Venugopalan reassigned HIVE-3747:


Assignee: Shreepadma Venugopalan

> Provide hive operation name for hookContext
> ---
>
> Key: HIVE-3747
> URL: https://issues.apache.org/jira/browse/HIVE-3747
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Sudhanshu Arora
>Assignee: Shreepadma Venugopalan
>
> The hookContext exposed through ExecuteWithHookContext, does not provide the 
> name of the Hive operation. 
> The following public API should be added in HookContext.
> public String getOperationName() {
> return SessionState.get().getHiveOperation().name();
> }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3400) Add Retries to Hive MetaStore Connections

2012-11-30 Thread Ashutosh Chauhan (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-3400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507722#comment-13507722
 ] 

Ashutosh Chauhan commented on HIVE-3400:


Bhushan, You have some changes in RetryingRawStore.java which looks like are 
unrelated to this patch. It seems like HIVE-3612 is tracking similar 
enhancements. Will you mind taking those changes out and put on in HIVE-3612?

> Add Retries to Hive MetaStore Connections
> -
>
> Key: HIVE-3400
> URL: https://issues.apache.org/jira/browse/HIVE-3400
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Bhushan Mandhani
>Assignee: Bhushan Mandhani
>Priority: Minor
>  Labels: metastore
> Attachments: HIVE-3400.1.patch.txt, HIVE-3400.2.patch.txt
>
>
> Currently, when using Thrift to access the MetaStore, if the Thrift host 
> dies, there is no mechanism to reconnect to some other host even if the 
> MetaStore URIs variable in the Conf contains multiple hosts. Hive should 
> retry and reconnect rather than throwing a communication link error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification

2012-11-30 Thread Phabricator (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-3384:
--

Attachment: D6873.5.patch

mbautin updated the revision "[jira] [HIVE-3384] HIVE JDBC module won't compile 
under JDK1.7 as new methods added in JDBC specification".
Reviewers: cdrome, weidongbian, JIRA, ashutoshc

  Rebasing on trunk changes.

REVISION DETAIL
  https://reviews.facebook.net/D6873

AFFECTED FILES
  build-common.xml
  build.properties
  build.xml
  jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveBaseResultSet.java
  jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveCallableStatement.java
  jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveConnection.java
  jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveDataSource.java
  jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveDatabaseMetaData.java
  jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveDriver.java
  jdbc/src/java/org/apache/hadoop/hive/jdbc/HivePreparedStatement.java
  jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveQueryResultSet.java
  jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveStatement.java
  metastore/build.xml
  pdk/scripts/build-plugin.xml

To: cdrome, weidongbian, JIRA, ashutoshc, mbautin


> HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC 
> specification
> --
>
> Key: HIVE-3384
> URL: https://issues.apache.org/jira/browse/HIVE-3384
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Reporter: Weidong Bian
>Priority: Minor
> Attachments: D6873-0.9.1.patch, D6873.1.patch, D6873.2.patch, 
> D6873.3.patch, D6873.4.patch, D6873.5.patch, HIVE-3384.2.patch, 
> HIVE-3384.patch, HIVE-JDK7-JDBC.patch
>
>
> jdbc module couldn't be compiled with jdk7 as it adds some abstract method in 
> the JDBC specification 
> some error info:
>  error: HiveCallableStatement is not abstract and does not override abstract
> method getObject(String,Class) in CallableStatement
> .
> .
> .

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3400) Add Retries to Hive MetaStore Connections

2012-11-30 Thread Bhushan Mandhani (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-3400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507626#comment-13507626
 ] 

Bhushan Mandhani commented on HIVE-3400:


Ashutosh, I've uploaded and submitted the latest patch. Thanks.

> Add Retries to Hive MetaStore Connections
> -
>
> Key: HIVE-3400
> URL: https://issues.apache.org/jira/browse/HIVE-3400
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Bhushan Mandhani
>Assignee: Bhushan Mandhani
>Priority: Minor
>  Labels: metastore
> Attachments: HIVE-3400.1.patch.txt, HIVE-3400.2.patch.txt
>
>
> Currently, when using Thrift to access the MetaStore, if the Thrift host 
> dies, there is no mechanism to reconnect to some other host even if the 
> MetaStore URIs variable in the Conf contains multiple hosts. Hive should 
> retry and reconnect rather than throwing a communication link error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Bad performance of Hive meta store for tables with large number of partitions

2012-11-30 Thread Benyi Wang 
Does JDO support the features in EclipseLink in this article?

http://java-persistence-performance.blogspot.com/2010/08/batch-fetching-optimizing-object-graph.html


On Fri, Nov 30, 2012 at 11:36 AM, Benyi Wang  wrote:

> We have some tables with 15K ~ 20K partitions. If I run a query scanning a
> lot of partitions, Hive could use more than 10 minutes to commit the mapred
> job.
>
> The problem is caused by ObjectStore.getPartitionsByNames when Hive
> semantic analyzer tries to prune partitions. This method sends a lot of
> queries to our MySQL database to retrieve ALL information about partitions.
> Because MPartition and MStroageDescriptor are converted into Partition and
> StorageDescriptor, every field will be accessed during conversion, in other
> words, even the fields has nothing to do with partition pruning, such as
> BucketCols. In our case, 10 queries for each partition will be sent to the
> database and each query may take 40ms.
>
> This is known ORM 1+N problem. But it is really bad user experience.
>
> Actually we assembly Partition objects manually, it would only need about
> 10 queries for a group of partitions (default size is 300). In our
> environment, it only needs 40 seconds for 30K partitions: 30K / 300 * 10 *
> 40.
>
> I tried to this way:
>
> 1. Fetch MPartition with fetch group and fetch_size_greedy, so one query
> can get MPartition's primary fields and MStorageDescriptor cached.
> 2. Get all descriptors into a list "msds", run another query to get
> MStorageDescriptor with filter like "msds.contains(this)", all cached
> descriptors will be refreshed in one query instead of n queries.
>
> This works well for 1-1 relations, but not on 1-N relation like
> MPartition.values. I didn't find a way to populate those fields in just one
> query.
>
> Because JDO mapping doesn't work well in the conversion (MPartition ->
> Partition), I'm wondering if it is worth doing like this:
>
> 1. Query each table in SQL directly PARTITIONS, SDS, etcs.
> 2. Assembly Partition objects
>
> This is a hack and the code will be really bad. But I didn't find JDO
> support "FETCH JOIN" or "Batch fetch".
>
> Any thoughts?
>

[jira] [Updated] (HIVE-3400) Add Retries to Hive MetaStore Connections

2012-11-30 Thread Bhushan Mandhani (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhushan Mandhani updated HIVE-3400:
---

Labels: metastore  (was: )
Status: Patch Available  (was: Open)

> Add Retries to Hive MetaStore Connections
> -
>
> Key: HIVE-3400
> URL: https://issues.apache.org/jira/browse/HIVE-3400
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Bhushan Mandhani
>Assignee: Bhushan Mandhani
>Priority: Minor
>  Labels: metastore
> Attachments: HIVE-3400.1.patch.txt, HIVE-3400.2.patch.txt
>
>
> Currently, when using Thrift to access the MetaStore, if the Thrift host 
> dies, there is no mechanism to reconnect to some other host even if the 
> MetaStore URIs variable in the Conf contains multiple hosts. Hive should 
> retry and reconnect rather than throwing a communication link error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3400) Add Retries to Hive MetaStore Connections

2012-11-30 Thread Bhushan Mandhani (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhushan Mandhani updated HIVE-3400:
---

Attachment: HIVE-3400.2.patch.txt

> Add Retries to Hive MetaStore Connections
> -
>
> Key: HIVE-3400
> URL: https://issues.apache.org/jira/browse/HIVE-3400
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Bhushan Mandhani
>Assignee: Bhushan Mandhani
>Priority: Minor
>  Labels: metastore
> Attachments: HIVE-3400.1.patch.txt, HIVE-3400.2.patch.txt
>
>
> Currently, when using Thrift to access the MetaStore, if the Thrift host 
> dies, there is no mechanism to reconnect to some other host even if the 
> MetaStore URIs variable in the Conf contains multiple hosts. Hive should 
> retry and reconnect rather than throwing a communication link error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Exception with datanucleus while running hive tests in eclipse

2012-11-30 Thread Mahsa Mofidpoor 
I ran into the same problem on the same MAC OS version.
This seems to be a JVM command line issue. it exceeds its limits and it's
platform independent. I know IntelliJ Idea handles this case.

On Wed, May 16, 2012 at 5:40 PM, kulkarni.swar...@gmail.com <
kulkarni.swar...@gmail.com> wrote:

> I installed datanucleus eclipse plugin as I realized that it is needed to
> run some of the hive tests in eclipse. While trying to run the enhancer
> tool, I keep getting this exception:
>
> "Exception occurred executing command line. Cannot run program
> "/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java"
> ...Argument list is too long"
>
> Has anyone else encountered this too? My machine is running OS X 10.7.
>
> Thanks,
>
> Swarnim
>

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #215

2012-11-30 Thread Apache Jenkins Server 
See 


--
[...truncated 5844 lines...]
[ivy:resolve]  (42kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] asm#asm;3.2!asm.jar (140ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/com/sun/jersey/jersey-core/1.8/jersey-core-1.8.jar
 ...
[ivy:resolve] . (447kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/com/sun/jersey/jersey-json/1.8/jersey-json-1.8.jar
 ...
[ivy:resolve]  (144kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/com/sun/jersey/jersey-server/1.8/jersey-server-1.8.jar
 ...
[ivy:resolve] 

 (678kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
com.sun.jersey#jersey-server;1.8!jersey-server.jar(bundle) (106ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/tomcat/jasper-compiler/5.5.23/jasper-compiler-5.5.23.jar
 ...
[ivy:resolve]  (398kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/tomcat/jasper-runtime/5.5.23/jasper-runtime-5.5.23.jar
 ...
[ivy:resolve] . (75kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] tomcat#jasper-runtime;5.5.23!jasper-runtime.jar 
(69ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar ...
[ivy:resolve]  (98kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] javax.servlet.jsp#jsp-api;2.1!jsp-api.jar (33ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/commons-logging/commons-logging-api/1.1/commons-logging-api-1.1.jar
 ...
[ivy:resolve] ... (43kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/log4j/log4j/1.2.15/log4j-1.2.15.jar ...
[ivy:resolve] .. (382kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] log4j#log4j;1.2.15!log4j.jar (83ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/net/java/dev/jets3t/jets3t/0.6.1/jets3t-0.6.1.jar 
...
[ivy:resolve]  (314kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/commons-lang/commons-lang/2.5/commons-lang-2.5.jar
 ...
[ivy:resolve] . (272kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] commons-lang#commons-lang;2.5!commons-lang.jar 
(46ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar ...
[ivy:resolve] .. (24kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] org.slf4j#slf4j-api;1.6.1!slf4j-api.jar (30ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar
 ...
[ivy:resolve] .. (9kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] org.slf4j#slf4j-log4j12;1.6.1!slf4j-log4j12.jar 
(68ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/aspectj/aspectjrt/1.6.5/aspectjrt-1.6.5.jar 
...
[ivy:resolve] . (113kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] org.aspectj#aspectjrt;1.6.5!aspectjrt.jar (173ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/apache/avro/avro/1.5.3/avro-1.5.3.jar ...
[ivy:resolve] . (257kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] org.apache.avro#avro;1.5.3!avro.jar (98ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/apache/avro/avro-ipc/1.5.3/avro-ipc-1.5.3.jar 
...
[ivy:resolve] . (164kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] org.apache.avro#avro-ipc;1.5.3!avro-ipc.jar (70ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/com/google/protobuf/protobuf-java/2.4.0a/protobuf-java-2.4.0a.jar
 ...
[ivy:resolve] . (439kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
com.google.protobuf#protobuf-java;2.4.0a!protobuf-java.jar (183ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-auth/0.23.3/hadoop-auth-0.23.3.jar
 ...
[ivy:resolve] ... (41kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
org.apache.hadoop#hadoop-auth;0.23.3!hadoop-auth.jar (211ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/com/googlecode/json-simple/json-simple/1.1/json-simple-1.1.jar
 ...
[ivy:resolve] ... (15kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
com.googlecode.json-simple#json-simple;1.1!json-simple.jar (82ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/jdiff/jdiff/1.0.9/jdiff-1.0.9.jar ...
[ivy:resolve] ... (126kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar
 ...
[ivy:resolve] ... (32kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar
 ...
[ivy

Re: Username/password authentication support in HiveServer

2012-11-30 Thread Mark Grover 
Hi Yu Gao,
The submission of HiveServer2 is being tracked by
https://issues.apache.org/jira/browse/HIVE-2935
Keep your eyes out for when the status of that JIRA changes:-)

Mark

On Thu, Nov 29, 2012 at 11:18 PM, 郜瑜  wrote:

> Hi all,
>
> I noticed that HiveServer2 will have username/password authentication. When
> will this HiveServer2 be available?
>
> As we know, the existing HiveServer does not have any authentication
> mechanism, so there's no security control on client connections and
> operations. It would be good to introduce authentication infrastructure to
> it as well, so I opened jira
> HIVE-3719
> and put the design there to achieve this.
>
> Regards,
> Yu Gao
>

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21 #215

2012-11-30 Thread Apache Jenkins Server 
See 

--
[...truncated 5810 lines...]
[ivy:resolve]  (33kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
org.apache.hadoop#hadoop-mapreduce-client-jobclient;0.23.3!hadoop-mapreduce-client-jobclient.jar
 (124ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-jobclient/0.23.3/hadoop-mapreduce-client-jobclient-0.23.3-tests.jar
 ...
[ivy:resolve] 
.
 (1264kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-annotations/0.23.3/hadoop-annotations-0.23.3.jar
 ...
[ivy:resolve] ... (16kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
org.apache.hadoop#hadoop-annotations;0.23.3!hadoop-annotations.jar (248ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar
 ...
[ivy:resolve] ... (297kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
commons-httpclient#commons-httpclient;3.1!commons-httpclient.jar (41ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/commons-io/commons-io/2.1/commons-io-2.1.jar ...
[ivy:resolve] .. (159kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] commons-io#commons-io;2.1!commons-io.jar (83ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/javax/servlet/servlet-api/2.5/servlet-api-2.5.jar 
...
[ivy:resolve] .. (102kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] javax.servlet#servlet-api;2.5!servlet-api.jar 
(34ms)
[ivy:resolve] downloading http://repo1.maven.org/maven2/asm/asm/3.2/asm-3.2.jar 
...
[ivy:resolve]  (42kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] asm#asm;3.2!asm.jar (31ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/com/sun/jersey/jersey-core/1.8/jersey-core-1.8.jar
 ...
[ivy:resolve] ... (447kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/com/sun/jersey/jersey-json/1.8/jersey-json-1.8.jar
 ...
[ivy:resolve] .. (144kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
com.sun.jersey#jersey-json;1.8!jersey-json.jar(bundle) (175ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/com/sun/jersey/jersey-server/1.8/jersey-server-1.8.jar
 ...
[ivy:resolve] 
 (678kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/tomcat/jasper-compiler/5.5.23/jasper-compiler-5.5.23.jar
 ...
[ivy:resolve] . (398kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] tomcat#jasper-compiler;5.5.23!jasper-compiler.jar 
(97ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/tomcat/jasper-runtime/5.5.23/jasper-runtime-5.5.23.jar
 ...
[ivy:resolve] .. (75kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar ...
[ivy:resolve] . (98kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/commons-logging/commons-logging-api/1.1/commons-logging-api-1.1.jar
 ...
[ivy:resolve]  (43kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] 
commons-logging#commons-logging-api;1.1!commons-logging-api.jar (86ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/log4j/log4j/1.2.15/log4j-1.2.15.jar ...
[ivy:resolve]  (382kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/net/java/dev/jets3t/jets3t/0.6.1/jets3t-0.6.1.jar 
...
[ivy:resolve] . (314kB)
[ivy:resolve] .. (0kB)
[ivy:resolve]   [SUCCESSFUL ] net.java.dev.jets3t#jets3t;0.6.1!jets3t.jar 
(199ms)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/commons-lang/commons-lang/2.5/commons-lang-2.5.jar
 ...
[ivy:resolve] ... (272kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar ...
[ivy:resolve] .. (24kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar
 ...
[ivy:resolve] .. (9kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/aspectj/aspectjrt/1.6.5/aspectjrt-1.6.5.jar 
...
[ivy:resolve] .. (113kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/apache/avro/avro/1.5.3/avro-1.5.3.jar ...
[ivy:resolve] . (257kB)
[ivy:resolve] .. (0kB)
[ivy:resolve] downloading 
http://repo1.maven.org/maven2/org/apache/avro/avro-ipc/1.5.3/avro-ipc-1.5.3.jar 
...
[ivy:resolve] .. (164kB)
[i

[jira] [Commented] (HIVE-2250) "DESCRIBE EXTENDED table_name" shows inconsistent compression information.

2012-11-30 Thread Harsh J (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507271#comment-13507271
 ] 

Harsh J commented on HIVE-2250:
---

If we don't really make use of the IS_COMPRESSED attribute of a table, should 
we just get rid of it (or at least not print it in the {{describe 
extended/formatted}} output, which causes great confusion as it is always 
certainly {{No}})?

> "DESCRIBE EXTENDED table_name" shows inconsistent compression information.
> --
>
> Key: HIVE-2250
> URL: https://issues.apache.org/jira/browse/HIVE-2250
> Project: Hive
>  Issue Type: Bug
>  Components: CLI, Diagnosability
>Affects Versions: 0.7.0
> Environment: RHEL, Full Cloudera stack
>Reporter: Travis Powell
>Assignee: subramanian raghunathan
>Priority: Critical
> Attachments: HIVE-2250.patch
>
>
> Commands executed in this order:
> user@node # hive
> hive> SET hive.exec.compress.output=true; 
> hive> SET io.seqfile.compression.type=BLOCK;
> hive> CREATE TABLE table_name ( [...] ) ROW FORMAT DELIMITED FIELDS 
> TERMINATED BY '\t' STORED AS SEQUENCEFILE;
> hive> CREATE TABLE staging_table ( [...] ) ROW FORMAT DELIMITED FIELDS 
> TERMINATED BY '\t';
> hive> LOAD DATA LOCAL INPATH 'file:///root/input/' OVERWRITE INTO TABLE 
> staging_table;
> hive> INSERT OVERWRITE TABLE table_name SELECT * FROM staging_table;
> (Map reduce job to change to sequence file...)
> hive> DESCRIBE EXTENDED table_name;
> Detailed Table Information  Table(tableName:table_name, 
> dbName:benchmarking, owner:root, createTime:1309480053, lastAccessTime:0, 
> retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:session_key, 
> type:string, comment:null), FieldSchema(name:remote_address, type:string, 
> comment:null), FieldSchema(name:canister_lssn, type:string, comment:null), 
> FieldSchema(name:canister_session_id, type:bigint, comment:null), 
> FieldSchema(name:tltsid, type:string, comment:null), FieldSchema(name:tltuid, 
> type:string, comment:null), FieldSchema(name:tltvid, type:string, 
> comment:null), FieldSchema(name:canister_server, type:string, comment:null), 
> FieldSchema(name:session_timestamp, type:string, comment:null), 
> FieldSchema(name:session_duration, type:string, comment:null), 
> FieldSchema(name:hit_count, type:bigint, comment:null), 
> FieldSchema(name:http_user_agent, type:string, comment:null), 
> FieldSchema(name:extractid, type:bigint, comment:null), 
> FieldSchema(name:site_link, type:string, comment:null), FieldSchema(name:dt, 
> type:string, comment:null), FieldSchema(name:hour, type:int, comment:null)], 
> location:hdfs://hadoop2/user/hive/warehouse/benchmarking.db/table_name, 
> inputFormat:org.apache.hadoop.mapred.SequenceFileInputFormat, 
> outputFormat:org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, 
> compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, 
> serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, 
> parameters:{serialization.format=   , field.delim=
> *** SEE ABOVE: Compression is set to FALSE, even though contents of table is 
> compressed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira