Simple Hive Query

2012-03-05 Thread hadoop hive
Hi Folks ,

i m facing a problem like i have a table a(k1 int ,k2 int ,k3 int   );

and data present is like

*9,*1,*3
3,4,5
12,32,54
*9,12,32
12,*9,21

i need row which doesn't have *9 in any fields.

i wrote a query

1: select k1,k2,k3 from a where (k1<>'\*9' or k2<>'\*9' or k3<>'\*9');

2: select k1,k2,k3 from a where (k1<>'*9' or k2<>'*9' or k3<>'*9');

3: select k1,k2,k3 from a where (k1<>"*9" or k2<>"*9" or k3<>"*9");

any of these case is not working.

i need the result like

*9,*1,*3
3,4,5
12,32,54
*9,12,32
12,*9,21


Thanks
Vikas Srivastava


Hive-trunk-h0.21 - Build # 1290 - Fixed

2012-03-05 Thread Apache Jenkins Server
Changes for Build #1280

Changes for Build #1281

Changes for Build #1282

Changes for Build #1283

Changes for Build #1284
[namit] HIVE-2825 Concatenating a partition does not inherit location from table
(Kevin Wilfong via namit)


Changes for Build #1285

Changes for Build #1286

Changes for Build #1288

Changes for Build #1289

Changes for Build #1290
[namit] HIVE-2833 Fix test failures caused by HIVE-2716
(Kevin Wilfong via namit)




All tests passed

The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1290)

Status: Fixed

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1290/ to 
view the results.

[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class

2012-03-05 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222306#comment-13222306
 ] 

Hudson commented on HIVE-2716:
--

Integrated in Hive-trunk-h0.21 #1290 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1290/])
HIVE-2833 Fix test failures caused by HIVE-2716
(Kevin Wilfong via namit) (Revision 1296946)

 Result = SUCCESS
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296946
Files : 
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RetryingRawStore.java
* 
/hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyJdoConnectionUrlHook.java
* 
/hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
* 
/hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreConnectionUrlHook.java


> Move retry logic in HiveMetaStore to a separe class
> ---
>
> Key: HIVE-2716
> URL: https://issues.apache.org/jira/browse/HIVE-2716
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Affects Versions: 0.9.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.9.0
>
> Attachments: HIVE-2716.D1227.1.patch, HIVE-2716.D1227.2.patch, 
> HIVE-2716.D1227.3.patch, HIVE-2716.D1227.4.patch, HIVE-2716.D2055.1.patch, 
> HIVE-2716.patch
>
>
> In HIVE-1219, method retrying for raw store operation are introduced to 
> handle jdo operations more robustly. However, the abstraction for the 
> RawStore operations can be moved to a separate class implementing RawStore, 
> which should clean up the code base for HiveMetaStore. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2833) Fix test failures caused by HIVE-2716

2012-03-05 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222307#comment-13222307
 ] 

Hudson commented on HIVE-2833:
--

Integrated in Hive-trunk-h0.21 #1290 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1290/])
HIVE-2833 Fix test failures caused by HIVE-2716
(Kevin Wilfong via namit) (Revision 1296946)

 Result = SUCCESS
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296946
Files : 
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RetryingRawStore.java
* 
/hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyJdoConnectionUrlHook.java
* 
/hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
* 
/hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreConnectionUrlHook.java


> Fix test failures caused by HIVE-2716
> -
>
> Key: HIVE-2833
> URL: https://issues.apache.org/jira/browse/HIVE-2833
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Reporter: Carl Steinbach
>Assignee: Kevin Wilfong
> Attachments: HIVE-2716.D2055.1.patch, HIVE-2833.D2055.2.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Re: Simple Hive Query

2012-03-05 Thread Wouter de Bie
Hi,

Your table specifies 3 ints, however, some of your rows contain strings. If you 
specify you table as strings and then later parse the fields to an int (by 
doing some replacements and casting it to an int), it should work. 

// Wouter


On Monday, March 5, 2012 at 12:55 PM, hadoop hive wrote:

> Hi Folks ,
> 
> i m facing a problem like i have a table a(k1 int ,k2 int ,k3 int   );
> 
> and data present is like 
> 
> *9,*1,*3
> 3,4,5
> 12,32,54
> *9,12,32
> 12,*9,21
> 
> i need row which doesn't have *9 in any fields.
> 
> i wrote a query
> 
> 1: select k1,k2,k3 from a where (k1<>'\*9' or k2<>'\*9' or k3<>'\*9'); 
> 
> 2: select k1,k2,k3 from a where (k1<>'*9' or k2<>'*9' or k3<>'*9');
> 
> 3: select k1,k2,k3 from a where (k1<>"*9" or k2<>"*9" or k3<>"*9"); 
> 
> any of these case is not working.
> 
> i need the result like
> 
> *9,*1,*3
> 3,4,5
> 12,32,54
> *9,12,32
> 12,*9,21
> 
> 
> Thanks 
> Vikas Srivastava
> 
> 




Hive 0.8.1 build errors using IBM JAVA

2012-03-05 Thread Amir Sanjar
I am new to HIVE, so forgive me if this is a stupid question :)
The plan is to build HIVE using IBM JAVA 6, however during the final steps
of the build we get below errors, please help:
Note:   we use command "ant clean package" .. also hbase;0.89.0-SNAPSHOT
.jar and hbase;0.89.0-SNAPSHOT-test.jar are in .../hive-0.8.1/lib

 

 

 

 

 
 [echo] Project: hbase-handler  
 
 [ivy:resolve] :: loading settings :: file = 
/home//development/hive-0.8.1/src/ivy/ivysettings.xml   

 [ivy:resolve]  
 
 [ivy:resolve]  WARNINGS
 
 [ivy:resolve] module not found: org.apache.hbase#hbase;0.89.0-SNAPSHOT 
 
 [ivy:resolve]  apache-snapshot: tried  
 
 [ivy:resolve]  
 
 
https://repository.apache.org/content/repositories/snapshots/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.pom
   
 [ivy:resolve] -- artifact org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar:
 
 [ivy:resolve]  
 
 
https://repository.apache.org/content/repositories/snapshots/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
   
 [ivy:resolve] -- artifact 
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar(test-jar): 
  
 [ivy:resolve]  
 
 
https://repository.apache.org/content/repositories/snapshots/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT-tests.jar
 
 [ivy:resolve]  maven2: tried   
 
 [ivy:resolve] 
http://repo1.maven.org/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.pom

 [ivy:resolve] -- artifact org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar:
 
 [ivy:resolve] 
http://repo1.maven.org/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar

 [ivy:resolve] -- artifact 
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar(test-jar): 
  
 [ivy:resolve] 
http://repo1.maven.org/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar

 [ivy:resolve]  datanucleus-repo: tried 
 
 [ivy:resolve] -- artifact org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar:
 
 [ivy:resolve] 
http://www.datanucleus.org/downloads/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
  
 [ivy:resolve] -- artifact 
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar(test-jar): 
  
 [ivy:resolve] 
http://www.datanucleus.org/downloads/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
  
 [ivy:resolve]  hadoop-source: tried
 
 [ivy:resolve] -- artifact org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar:
 
 [ivy:resolve] 
http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hbase-0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
 
 [ivy:resolve] -- artifact 
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar(test-jar): 
  
 [ivy:resolve] 
http://mirror.facebook.net/facebook/hive-deps/hadoop/c

Re: Hive 0.8.1 build errors using IBM JAVA

2012-03-05 Thread Edward Capriolo
I am sure you are going to love hearing this since you are from IBM,
but there have been a few outstanding issues with Hive and the IBM
jvm.

I just searched to find one as an example.
https://issues.apache.org/jira/browse/HIVE-1686

I am not sure how many users in the wild are running Hive with that
JVM. This does not look like a JVM issue but could be. Do other JVMs
produce the same issue for you?


On Mon, Mar 5, 2012 at 10:55 AM, Amir Sanjar  wrote:
> I am new to HIVE, so forgive me if this is a stupid question :)
> The plan is to build HIVE using IBM JAVA 6, however during the final steps
> of the build we get below errors, please help:
> Note:   we use command "ant clean package" .. also hbase;0.89.0-SNAPSHOT
> .jar and hbase;0.89.0-SNAPSHOT-test.jar are in .../hive-0.8.1/lib
>
>
>
>
>
>  [echo] Project: hbase-handler
>  [ivy:resolve] :: loading settings :: file = 
> /home//development/hive-0.8.1/src/ivy/ivysettings.xml
>  [ivy:resolve]
>  [ivy:resolve]  WARNINGS
>  [ivy:resolve] module not found: org.apache.hbase#hbase;0.89.0-SNAPSHOT
>  [ivy:resolve]  apache-snapshot: tried
>  [ivy:resolve]
>  https://repository.apache.org/content/repositories/snapshots/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.pom
>  [ivy:resolve] -- artifact org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar:
>  [ivy:resolve]
>  https://repository.apache.org/content/repositories/snapshots/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
>  [ivy:resolve] -- artifact 
> org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar(test-jar):
>  [ivy:resolve]
>  https://repository.apache.org/content/repositories/snapshots/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT-tests.jar
>  [ivy:resolve]  maven2: tried
>  [ivy:resolve] 
> http://repo1.maven.org/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.pom
>  [ivy:resolve] -- artifact org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar:
>  [ivy:resolve] 
> http://repo1.maven.org/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
>  [ivy:resolve] -- artifact 
> org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar(test-jar):
>  [ivy:resolve] 
> http://repo1.maven.org/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
>  [ivy:resolve]  datanucleus-repo: tried
>  [ivy:resolve] -- artifact org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar:
>  [ivy:resolve] 
> http://www.datanucleus.org/downloads/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
>  [ivy:resolve] -- artifact 
> org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar(test-jar):
>  [ivy:resolve] 
> http://www.datanucleus.org/downloads/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
>  [ivy:resolve]  hadoop-source: tried
>  [ivy:resolve] -- artifact org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar:
>  [ivy:resolve] 
> http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hbase-0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
>  [ivy:resolve] -- artifact 
> org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar(test-jar):
>  [ivy:resolve] 
> http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hbase-0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
>  [ivy:resolve]  hadoop-source2: tried
>  [ivy:resolve] -- artifact org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar:
>  [ivy:resolve] 
> http://archive.cloudera.com/hive-deps/hadoop/core/hbase-0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
>  [ivy:resolve] -- artifact 
> org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar(test-jar):
>  [ivy:resolve] 
> http://archive.cloudera.com/hive-deps/hadoop/core/hbase-0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
>  [ivy:resolve] ::
>  [ivy:resolve] :: UNRESOLVED DEPENDENCIES ::
>  [ivy:resolve] ::
>  [ivy:resolve] :: org.apache.hbase#hbase;0.89.0-SNAPSHOT: not found
>  [ivy:resolve] ::
>  [ivy:resolve]
>
>
>
>
>
> Best Regards
> Amir Sanjar
>
> Linux System Management Architect and Lead
> IBM Senior Software Engineer
> Phone# 512-286-8393
> Fax#      512-838-8858
>
>


Re: Hive 0.8.1 build errors using IBM JAVA

2012-03-05 Thread Amir Sanjar
I undertsand but IBM JVM  is the only JVM that works on POWER..It looks
like it compiles fine but
it has something to do with packaging.. For some reasons, hbaseXXX jar
files don't get copied to hive-0.8.1/src/build/ivy/lib/default..






From:   Edward Capriolo 
To: dev@hive.apache.org
Date:   03/05/2012 10:21 AM
Subject:Re: Hive 0.8.1 build errors using IBM JAVA



I am sure you are going to love hearing this since you are from IBM,
but there have been a few outstanding issues with Hive and the IBM
jvm.

I just searched to find one as an example.
https://issues.apache.org/jira/browse/HIVE-1686

I am not sure how many users in the wild are running Hive with that
JVM. This does not look like a JVM issue but could be. Do other JVMs
produce the same issue for you?


On Mon, Mar 5, 2012 at 10:55 AM, Amir Sanjar  wrote:
> I am new to HIVE, so forgive me if this is a stupid question :)
> The plan is to build HIVE using IBM JAVA 6, however during the final
steps
> of the build we get below errors, please help:
> Note:   we use command "ant clean package" .. also hbase;0.89.0-SNAPSHOT
> .jar and hbase;0.89.0-SNAPSHOT-test.jar are in .../hive-0.8.1/lib
>
>
>
>
>
>  [echo] Project: hbase-handler
>  [ivy:resolve] :: loading settings :: file
= /home//development/hive-0.8.1/src/ivy/ivysettings.xml
>  [ivy:resolve]
>  [ivy:resolve]  WARNINGS
>  [ivy:resolve] module not found: org.apache.hbase#hbase;0.89.0-SNAPSHOT
>  [ivy:resolve]  apache-snapshot: tried
>  [ivy:resolve]
>
https://repository.apache.org/content/repositories/snapshots/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.pom

>  [ivy:resolve] -- artifact
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar:
>  [ivy:resolve]
>
https://repository.apache.org/content/repositories/snapshots/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar

>  [ivy:resolve] -- artifact
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar(test-jar):
>  [ivy:resolve]
>
https://repository.apache.org/content/repositories/snapshots/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT-tests.jar

>  [ivy:resolve]  maven2: tried
>  [ivy:resolve]
http://repo1.maven.org/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.pom

>  [ivy:resolve] -- artifact
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar:
>  [ivy:resolve]
http://repo1.maven.org/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar

>  [ivy:resolve] -- artifact
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar(test-jar):
>  [ivy:resolve]
http://repo1.maven.org/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar

>  [ivy:resolve]  datanucleus-repo: tried
>  [ivy:resolve] -- artifact
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar:
>  [ivy:resolve]
http://www.datanucleus.org/downloads/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar

>  [ivy:resolve] -- artifact
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar(test-jar):
>  [ivy:resolve]
http://www.datanucleus.org/downloads/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar

>  [ivy:resolve]  hadoop-source: tried
>  [ivy:resolve] -- artifact
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar:
>  [ivy:resolve]
http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hbase-0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar

>  [ivy:resolve] -- artifact
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar(test-jar):
>  [ivy:resolve]
http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hbase-0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar

>  [ivy:resolve]  hadoop-source2: tried
>  [ivy:resolve] -- artifact
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar:
>  [ivy:resolve]
http://archive.cloudera.com/hive-deps/hadoop/core/hbase-0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar

>  [ivy:resolve] -- artifact
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar(test-jar):
>  [ivy:resolve]
http://archive.cloudera.com/hive-deps/hadoop/core/hbase-0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar

>  [ivy:resolve] ::
>  [ivy:resolve] :: UNRESOLVED DEPENDENCIES ::
>  [ivy:resolve] ::
>  [ivy:resolve] :: org.apache.hbase#hbase;0.89.0-SNAPSHOT: not found
>  [ivy:resolve] ::
>  [ivy:resolve]
>
>
>
>
>
> Best Regards
> Amir Sanjar
>
> Linux System Management Architect and Lead
> IBM Senior Software Engineer
> Phone# 512-286-8393
> Fax#      512-838-8858
>
>





[jira] [Commented] (HIVE-2836) CREATE TABLE LIKE does not copy table type

2012-03-05 Thread Kevin Wilfong (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222482#comment-13222482
 ] 

Kevin Wilfong commented on HIVE-2836:
-

@Aniket

Thanks for pointing that out.

> CREATE TABLE LIKE does not copy table type
> --
>
> Key: HIVE-2836
> URL: https://issues.apache.org/jira/browse/HIVE-2836
> Project: Hive
>  Issue Type: Bug
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
>
> CREATE TABLE t LIKE t2
> If t2 is external t will still be managed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HIVE-2836) CREATE TABLE LIKE does not copy table type

2012-03-05 Thread Kevin Wilfong (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong resolved HIVE-2836.
-

Resolution: Not A Problem

> CREATE TABLE LIKE does not copy table type
> --
>
> Key: HIVE-2836
> URL: https://issues.apache.org/jira/browse/HIVE-2836
> Project: Hive
>  Issue Type: Bug
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
>
> CREATE TABLE t LIKE t2
> If t2 is external t will still be managed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2810) Implement NULL-safe equality operator <=>

2012-03-05 Thread Phabricator (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222500#comment-13222500
 ] 

Phabricator commented on HIVE-2810:
---

cwsteinbach has accepted the revision "HIVE-2810 [jira] Implement NULL-safe 
equality operator <=>".

  +1. Will commit if tests pass.

REVISION DETAIL
  https://reviews.facebook.net/D1791

BRANCH
  DPAL-843


> Implement NULL-safe equality operator <=>
> -
>
> Key: HIVE-2810
> URL: https://issues.apache.org/jira/browse/HIVE-2810
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor, UDF
>Reporter: Carl Steinbach
>Assignee: Navis
> Fix For: 0.9.0
>
> Attachments: HIVE-2810.D1791.1.patch, HIVE-2810.D1791.2.patch, 
> HIVE-2810.D1791.3.patch, HIVE-2810.D1791.4.patch
>
>
> Ref: 
> http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_equal-to

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2771) Add support for filter pushdown for key ranges in hbase for keys of type string

2012-03-05 Thread Phabricator (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222511#comment-13222511
 ] 

Phabricator commented on HIVE-2771:
---

cwsteinbach has accepted the revision "HIVE-2771 [jira] Add support for filter 
pushdown for key ranges in hbase for keys of type string".

  +1. Will commit if tests pass.

REVISION DETAIL
  https://reviews.facebook.net/D1551

BRANCH
  svn


> Add support for filter pushdown for key ranges in hbase for keys of type 
> string
> ---
>
> Key: HIVE-2771
> URL: https://issues.apache.org/jira/browse/HIVE-2771
> Project: Hive
>  Issue Type: Sub-task
>  Components: HBase Handler
>Affects Versions: 0.7.0, 0.7.1, 0.8.0, 0.8.1
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-2771.D1551.1.patch, HIVE-2771.D1551.2.patch
>
>
> This is a subtask of HIVE-1643

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Hive-0.8.1-SNAPSHOT-h0.21 - Build # 213 - Fixed

2012-03-05 Thread Apache Jenkins Server
Changes for Build #212

Changes for Build #213



All tests passed

The Apache Jenkins build system has built Hive-0.8.1-SNAPSHOT-h0.21 (build #213)

Status: Fixed

Check console output at 
https://builds.apache.org/job/Hive-0.8.1-SNAPSHOT-h0.21/213/ to view the 
results.

[jira] [Updated] (HIVE-2834) Diff masking it too aggressive in index_bitmap*.q and index_compact*.q tests

2012-03-05 Thread Phabricator (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-2834:
--

Attachment: HIVE-2834.D2109.1.patch

cwsteinbach requested code review of "HIVE-2834 [jira] Diff masking it too 
aggressive in index_bitmap*.q and index_compact*.q tests".
Reviewers: JIRA

  HIVE-2834. Diff masking it too aggressive in index_bitmap*.q and 
index_compact*.q tests

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D2109

AFFECTED FILES
  ql/src/test/queries/clientpositive/index_bitmap.q
  ql/src/test/queries/clientpositive/index_bitmap1.q
  ql/src/test/queries/clientpositive/index_bitmap2.q
  ql/src/test/queries/clientpositive/index_bitmap3.q
  ql/src/test/queries/clientpositive/index_bitmap_auto.q
  ql/src/test/queries/clientpositive/index_bitmap_rc.q
  ql/src/test/queries/clientpositive/index_compact.q
  ql/src/test/queries/clientpositive/index_compact_1.q
  ql/src/test/queries/clientpositive/index_compact_2.q
  ql/src/test/queries/clientpositive/index_compact_3.q
  ql/src/test/queries/clientpositive/virtual_column.q
  ql/src/test/results/clientpositive/index_bitmap.q.out
  ql/src/test/results/clientpositive/index_bitmap1.q.out
  ql/src/test/results/clientpositive/index_bitmap2.q.out
  ql/src/test/results/clientpositive/index_bitmap3.q.out
  ql/src/test/results/clientpositive/index_bitmap_auto.q.out
  ql/src/test/results/clientpositive/index_bitmap_rc.q.out
  ql/src/test/results/clientpositive/index_compact.q.out
  ql/src/test/results/clientpositive/index_compact_1.q.out
  ql/src/test/results/clientpositive/index_compact_2.q.out
  ql/src/test/results/clientpositive/index_compact_3.q.out
  ql/src/test/results/clientpositive/virtual_column.q.out

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/4557/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


> Diff masking it too aggressive in index_bitmap*.q and index_compact*.q tests
> 
>
> Key: HIVE-2834
> URL: https://issues.apache.org/jira/browse/HIVE-2834
> Project: Hive
>  Issue Type: Bug
>  Components: Tests
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: HIVE-2834.D2061.1.patch, HIVE-2834.D2109.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2838) cleanup readentity/writeentity

2012-03-05 Thread Ashutosh Chauhan (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222760#comment-13222760
 ] 

Ashutosh Chauhan commented on HIVE-2838:


@Namit,
Can you briefly explain why there should be one common entity instead of 
read/write entity?

> cleanup readentity/writeentity
> --
>
> Key: HIVE-2838
> URL: https://issues.apache.org/jira/browse/HIVE-2838
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
>
> Ideally, there should be one common entity instead of readentity/writeentity.
> Unfortunately, that would be a backward incompatible change since users os 
> hive might have written
> there own hooks, where they are using readentity/writeentity.
> We should atleast create a common class, and then we can deprecate read/write 
> entity later, for a new release.
> For now, I propose to make a backward compatible change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs

2012-03-05 Thread Andrew Bayer (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Bayer updated HIVE-2646:
---

Status: Patch Available  (was: Open)

> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> 
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.8.0
>Reporter: Andrew Bayer
>Priority: Critical
> Attachments: HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is 
> problematic - depending on the tarball and extracting the jars from there, 
> rather than depending on the jars directly. It'd be great if this was fixed 
> to actually have the jar dependencies defined directly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs

2012-03-05 Thread Andrew Bayer (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Bayer updated HIVE-2646:
---

Attachment: HIVE-2646.diff.txt

This should apply cleanly to trunk, and I think it also encompasses the fixes 
for HIVE-2745.

> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> 
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.8.0
>Reporter: Andrew Bayer
>Priority: Critical
> Attachments: HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is 
> problematic - depending on the tarball and extracting the jars from there, 
> rather than depending on the jars directly. It'd be great if this was fixed 
> to actually have the jar dependencies defined directly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Assigned] (HIVE-2711) Make the header of RCFile unique

2012-03-05 Thread Owen O'Malley (Assigned) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Owen O'Malley reassigned HIVE-2711:
---

Assignee: Owen O'Malley

> Make the header of RCFile unique
> 
>
> Key: HIVE-2711
> URL: https://issues.apache.org/jira/browse/HIVE-2711
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
>
> The RCFile implementation was copied from Hadoop's SequenceFile and copied 
> the 'magic' string in the header. This means that you can't use the header to 
> distinguish between RCFiles and SequenceFiles.
> I'd propose that we create a new header for RCFiles (RCF?) to replace the 
> current SEQ. To maintain compatibility, we'll need to continue to accept the 
> current 'SEQ\06' and just make new files contain the new header.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2711) Make the header of RCFile unique

2012-03-05 Thread Phabricator (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-2711:
--

Attachment: HIVE-2711.D2115.1.patch

omalley requested code review of "HIVE-2711 [jira] Make the header of RCFile 
unique".
Reviewers: JIRA

  HIVE-2711

  Make the header of RCFile unique wrt SequenceFile

  The RCFile implementation was copied from Hadoop's SequenceFile and copied 
the 'magic' string in the header. This means that you can't use the header to 
distinguish between RCFiles and SequenceFiles.

  I'd propose that we create a new header for RCFiles (RCF?) to replace the 
current SEQ. To maintain compatibility, we'll need to continue to accept the 
current 'SEQ\06' and just make new files contain the new header.

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D2115

AFFECTED FILES
  ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java
  ql/src/test/data/rc-file-v0.rc
  ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/4587/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


> Make the header of RCFile unique
> 
>
> Key: HIVE-2711
> URL: https://issues.apache.org/jira/browse/HIVE-2711
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Attachments: HIVE-2711.D2115.1.patch
>
>
> The RCFile implementation was copied from Hadoop's SequenceFile and copied 
> the 'magic' string in the header. This means that you can't use the header to 
> distinguish between RCFiles and SequenceFiles.
> I'd propose that we create a new header for RCFiles (RCF?) to replace the 
> current SEQ. To maintain compatibility, we'll need to continue to accept the 
> current 'SEQ\06' and just make new files contain the new header.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2428) ADD ARCHIVE has no effect when query runs in local mode

2012-03-05 Thread Phabricator (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-2428:
--

Attachment: HIVE-2428.D2121.1.patch

kevinwilfong requested code review of "HIVE-2428 [jira] ADD ARCHIVE has no 
effect when query runs in local mode".
Reviewers: JIRA

  https://issues.apache.org/jira/browse/HIVE-2428

  Added the -archives flag to the command run when a map reduce task is run 
locally. It runs essentially exactly as it does for the -files flag.  This 
fixes the issue, I have added a test to verify that.

  When you ADD ARCHIVE a file and the query runs in local mode, the file is not 
available at all.

  For example running the commands

  echo '#!/bin/bash\nls -l 1>&2;read' > foo.sh

  hive -e 'ADD FILE foo.sh; ADD ARCHIVE /tmp/mapreduce_test.zip; SELECT 
TRANSFORM USING "sh foo.sh" FROM src'

  when the query is allowed to run in local mode, only information for foo.sh 
is returned.

  However, if the query is not allowed to run in local mode, information for 
both foo.sh and mapreduce_test.zip is returned.

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D2121

AFFECTED FILES
  data/files/add_archive_local.zip
  data/scripts/add_archive_local_script
  ql/src/test/results/clientpositive/add_archive_local.q.out
  ql/src/test/queries/clientpositive/add_archive_local.q
  ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/4599/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


> ADD ARCHIVE has no effect when query runs in local mode
> ---
>
> Key: HIVE-2428
> URL: https://issues.apache.org/jira/browse/HIVE-2428
> Project: Hive
>  Issue Type: Bug
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-2428.1.patch.txt, HIVE-2428.D2121.1.patch
>
>
> When you ADD ARCHIVE a file and the query runs in local mode, the file is not 
> available at all.
> For example running the commands
> echo '#!/bin/bash\nls -l 1>&2;read' > foo.sh
> hive -e 'ADD FILE foo.sh; ADD ARCHIVE /tmp/mapreduce_test.zip; SELECT 
> TRANSFORM(*) USING "sh foo.sh" FROM src'
> when the query is allowed to run in local mode, only information for foo.sh 
> is returned.
> However, if the query is not allowed to run in local mode, information for 
> both foo.sh and mapreduce_test.zip is returned.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2428) ADD ARCHIVE has no effect when query runs in local mode

2012-03-05 Thread Kevin Wilfong (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2428:


Attachment: add_archive_local.zip

> ADD ARCHIVE has no effect when query runs in local mode
> ---
>
> Key: HIVE-2428
> URL: https://issues.apache.org/jira/browse/HIVE-2428
> Project: Hive
>  Issue Type: Bug
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-2428.1.patch.txt, HIVE-2428.D2121.1.patch, 
> add_archive_local.zip
>
>
> When you ADD ARCHIVE a file and the query runs in local mode, the file is not 
> available at all.
> For example running the commands
> echo '#!/bin/bash\nls -l 1>&2;read' > foo.sh
> hive -e 'ADD FILE foo.sh; ADD ARCHIVE /tmp/mapreduce_test.zip; SELECT 
> TRANSFORM(*) USING "sh foo.sh" FROM src'
> when the query is allowed to run in local mode, only information for foo.sh 
> is returned.
> However, if the query is not allowed to run in local mode, information for 
> both foo.sh and mapreduce_test.zip is returned.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2428) ADD ARCHIVE has no effect when query runs in local mode

2012-03-05 Thread Kevin Wilfong (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2428:


Status: Patch Available  (was: Open)

> ADD ARCHIVE has no effect when query runs in local mode
> ---
>
> Key: HIVE-2428
> URL: https://issues.apache.org/jira/browse/HIVE-2428
> Project: Hive
>  Issue Type: Bug
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-2428.1.patch.txt, HIVE-2428.D2121.1.patch, 
> add_archive_local.zip
>
>
> When you ADD ARCHIVE a file and the query runs in local mode, the file is not 
> available at all.
> For example running the commands
> echo '#!/bin/bash\nls -l 1>&2;read' > foo.sh
> hive -e 'ADD FILE foo.sh; ADD ARCHIVE /tmp/mapreduce_test.zip; SELECT 
> TRANSFORM(*) USING "sh foo.sh" FROM src'
> when the query is allowed to run in local mode, only information for foo.sh 
> is returned.
> However, if the query is not allowed to run in local mode, information for 
> both foo.sh and mapreduce_test.zip is returned.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2428) ADD ARCHIVE has no effect when query runs in local mode

2012-03-05 Thread Phabricator (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222852#comment-13222852
 ] 

Phabricator commented on HIVE-2428:
---

kevinwilfong has commented on the revision "HIVE-2428 [jira] ADD ARCHIVE has no 
effect when query runs in local mode".

  Attached the zip file used by the test to the JIRA.

  Before this fix was committed, add_archive_local.zip would not appear in the 
output when the test q file is run.

REVISION DETAIL
  https://reviews.facebook.net/D2121


> ADD ARCHIVE has no effect when query runs in local mode
> ---
>
> Key: HIVE-2428
> URL: https://issues.apache.org/jira/browse/HIVE-2428
> Project: Hive
>  Issue Type: Bug
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-2428.1.patch.txt, HIVE-2428.D2121.1.patch, 
> add_archive_local.zip
>
>
> When you ADD ARCHIVE a file and the query runs in local mode, the file is not 
> available at all.
> For example running the commands
> echo '#!/bin/bash\nls -l 1>&2;read' > foo.sh
> hive -e 'ADD FILE foo.sh; ADD ARCHIVE /tmp/mapreduce_test.zip; SELECT 
> TRANSFORM(*) USING "sh foo.sh" FROM src'
> when the query is allowed to run in local mode, only information for foo.sh 
> is returned.
> However, if the query is not allowed to run in local mode, information for 
> both foo.sh and mapreduce_test.zip is returned.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2764) Obtain delegation tokens for MR jobs in secure hbase setup

2012-03-05 Thread Enis Soztutar (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222862#comment-13222862
 ] 

Enis Soztutar commented on HIVE-2764:
-

We had some discussion with Ashutosh about how to best approach this. As a part 
of this issue, we have to obtain delegation tokens from the storage handler 
(hbase in this case), and then pass this to the submitted job. However, 
HiveStorageHandler.configureTableJobProperties() does not expose the JobConf 
object, or an interface to configure the job's credentials. There seems a 
couple of options we can go for:
1. At HiveStorageHandler.configureTableJobProperties(), create and save the job 
credentials somewhere, and at the time of job submit, merge all the credentials 
before submitting the job. 
2. Change the HiveStorageHandler.configureTableJobProperties() to get the 
actual Job to be submitted, so that Storage handlers have a real shot at 
configuring the job. 
3. Add a method like HiveStorageHandler.configureJobCredentials(), and call it 
from HIF.getSplits() and HiveHBaseTableOutputFormat.checkOutputSpecs(). 

Number 1 and 3 seems hackish, and we will want to give the storagehandlers 
ability to configure the actual job. So we are leaning towards that. But this 
will also be merged with HIVE-2773. I am still trying to make this work, so any 
comments / suggestions are more than welcome. 

> Obtain delegation tokens for MR jobs in secure hbase setup  
> 
>
> Key: HIVE-2764
> URL: https://issues.apache.org/jira/browse/HIVE-2764
> Project: Hive
>  Issue Type: Improvement
>  Components: HBase Handler, Security
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>
> As discussed in HCATALOG-244, in a secure hbase setup with 0.92, we need to 
> obtain delegation tokens for hbase and save it in jobconf, so that tasks can 
> access region servers. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2764) Obtain delegation tokens for MR jobs in secure hbase setup

2012-03-05 Thread Enis Soztutar (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated HIVE-2764:


Attachment: HIVE-2764_v0.patch

Attaching a patch for demonstration of yet another approach of adding the 
Credentials to TableDesc. This patch is still WIP.

I have tested this with fully secure hadoop  +hbase. Compiles with 
-Dhadoop.version=1.0.0

> Obtain delegation tokens for MR jobs in secure hbase setup  
> 
>
> Key: HIVE-2764
> URL: https://issues.apache.org/jira/browse/HIVE-2764
> Project: Hive
>  Issue Type: Improvement
>  Components: HBase Handler, Security
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: HIVE-2764_v0.patch
>
>
> As discussed in HCATALOG-244, in a secure hbase setup with 0.92, we need to 
> obtain delegation tokens for hbase and save it in jobconf, so that tasks can 
> access region servers. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HIVE-2840) INPUT__FILE__NAME virtual column returns unqualified paths on Hadoop 0.23

2012-03-05 Thread Carl Steinbach (Created) (JIRA)
INPUT__FILE__NAME virtual column returns unqualified paths on Hadoop 0.23
-

 Key: HIVE-2840
 URL: https://issues.apache.org/jira/browse/HIVE-2840
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Carl Steinbach
Assignee: Carl Steinbach




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2840) INPUT__FILE__NAME virtual column returns unqualified paths on Hadoop 0.23

2012-03-05 Thread Carl Steinbach (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222983#comment-13222983
 ] 

Carl Steinbach commented on HIVE-2840:
--

On 0.20.1 the values generated by the INPUT__FILE__NAME virtual column are 
fully qualified:

{noformat}
hive> SELECT INPUT__FILE__NAME FROM src LIMIT 1;
file:/user/hive/warehouse/src/kv1.txt
{noformat}

However, the same query on 0.23.1 produces unqualified results:

{noformat}
hive> SELECT INPUT__FILE__NAME FROM src LIMIT 1;
/user/hive/warehouse/src/kv1.txt
{noformat}


> INPUT__FILE__NAME virtual column returns unqualified paths on Hadoop 0.23
> -
>
> Key: HIVE-2840
> URL: https://issues.apache.org/jira/browse/HIVE-2840
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2840) INPUT__FILE__NAME virtual column returns unqualified paths on Hadoop 0.23

2012-03-05 Thread Phabricator (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-2840:
--

Attachment: HIVE-2840.D2127.1.patch

cwsteinbach requested code review of "HIVE-2840 [jira] INPUT__FILE__NAME 
virtual column returns unqualified paths on Hadoop 0.23".
Reviewers: JIRA, ashutoshc

  HIVE-2840. INPUT__FILE__NAME virtual column returns unqualified paths on 
Hadoop 0.23

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D2127

AFFECTED FILES
  ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/4605/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


> INPUT__FILE__NAME virtual column returns unqualified paths on Hadoop 0.23
> -
>
> Key: HIVE-2840
> URL: https://issues.apache.org/jira/browse/HIVE-2840
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: HIVE-2840.D2127.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2764) Obtain delegation tokens for MR jobs in secure hbase setup

2012-03-05 Thread Francis Liu (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222994#comment-13222994
 ] 

Francis Liu commented on HIVE-2764:
---

Having making getCredentials() exposed as a TableDesc member sounds ok to me as 
it really is a special property of JobConf. 

Though it breaks the expectation of tableDesc being readonly and jobProperties 
being the only one mutable for configureTableProperties(). I wonder if we can 
remove jobProperties altogether and use tableDesc.jobProperties to set 
properties. As tableDesc.jobProperties is always a new HashMap when 
configureTableProperties() is called?

> Obtain delegation tokens for MR jobs in secure hbase setup  
> 
>
> Key: HIVE-2764
> URL: https://issues.apache.org/jira/browse/HIVE-2764
> Project: Hive
>  Issue Type: Improvement
>  Components: HBase Handler, Security
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: HIVE-2764_v0.patch
>
>
> As discussed in HCATALOG-244, in a secure hbase setup with 0.92, we need to 
> obtain delegation tokens for hbase and save it in jobconf, so that tasks can 
> access region servers. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira