[jira] [Created] (HIVE-2997) Store sort order of table in the metastore

2012-05-02 Thread Jeff Hammerbacher (JIRA)
Jeff Hammerbacher created HIVE-2997:
---

 Summary: Store sort order of table in the metastore
 Key: HIVE-2997
 URL: https://issues.apache.org/jira/browse/HIVE-2997
 Project: Hive
  Issue Type: New Feature
  Components: Metastore
Reporter: Jeff Hammerbacher


If a table or view is sorted on a specific column, it would be useful to record 
this fact in the metastore.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Re: Hive projects for Google Summer of code 2012 ?

2012-05-02 Thread Carl Steinbach
Ed,

Did you see this?

http://google-melange.appspot.com/gsoc/project/google/gsoc2012/bharathv/29002

On Wed, Mar 7, 2012 at 12:15 PM, Edward Capriolo edlinuxg...@gmail.comwrote:

 For the record we are 0-2 in gsoc.

 On Wed, Mar 7, 2012 at 1:42 AM, Namit Jain nj...@fb.com wrote:
  Done
 
 
  On 3/6/12 10:32 PM, bharath vissapragada
  bharathvissapragada1...@gmail.com wrote:
 
 Hi Namit,
 
 Is it possible to add Hive-1362 [1] to the list? I am interested in
 working
 on that and I've contacted Ashutosh regarding that.
 
 Thanks and Regards,
 
 
 [1]  https://issues.apache.org/jira/browse/HIVE-1362
 
 On Wed, Mar 7, 2012 at 6:46 AM, Namit Jain nj...@fb.com wrote:
 
 
 
 https://issues.apache.org/jira/secure/IssueNavigator.jspa?requestId=12319
 27
  0
 
 
 
  0 was in the next line by mistake
  Look for HIVE jiras.
 
 
 
  On 3/6/12 7:37 PM, Alexis De La Cruz Toledo alexis...@gmail.com
 wrote:
 
  Sorry Mr. Namit Jain I can't see the jiras.
  
  Thanks
  
  El 6 de marzo de 2012 19:41, Namit Jain nj...@fb.com escribió:
  
   I filed a couple of gsoc 2012 jiras.
  
   Please submit your proposal if you are interested.
  
  
  
 
 
 https://issues.apache.org/jira/secure/IssueNavigator.jspa?requestId=12319
  27
   0
  
  
  
  
   Thanks,
   -namit
  
  
   On 2/7/12 6:32 PM, Alexis De La Cruz Toledo alexis...@gmail.com
  wrote:
  
   Hi Namit Jain, I'm Alexis, I'm a master student, I'm studying in
   Cinvestav,
   df Mexico.
   I'm interesting in collaborating with Hive. I'd like to do my
 thesis
  work
   in something about Hive.
   I'd like to participate in Google Summer of Code 2012 too.
   
   The issues that you propose I find interesting, particularly in the
  next
   issues:
   * The topic 2. Indexed Joins.
   * [PO] Optimize Joins using Bloom Filters of this page
   https://cwiki.apache.org/confluence/display/Hive/Roadmap.
   
   Can you tell me something more about it?
   What is problem to be solved?
   What benefits we hope to gain?
   
   This because I want to raise my thesis problem.
   By another hand, can you be my mentor in Google Summer of Code, if
 I
  work
   with this topics?
   
   Thanks.
   
   El 5 de febrero de 2012 19:58, Namit Jain nj...@fb.com escribió:
   
Hi Alexis/Bharath,
   
Great to see your interests. If you looking for ideas, some
 things
  that
will be very useful are:
   
1. Removing the map-join hint completely.
  Moving all processing to runtime.
  Currently, bucketed map joins and sort merge joins are
 completely
driven off hints.
  It would be very helpful to the community, and also clean up
 the
  code
   a
lot.
   
2. Indexed Joins.
  Something that would be really useful -
  If the basic infrastructure is ready, it can eventually be used
 to
   join
tables
  stored outside also (say Hbase).
   
3. Metastore understanding hierarchy.
  For eg: if a table is partitioned by ds and hr,
  A valid partition on ds does not exist. This is a very common
  usecase
on many
  applications, and the current work-around is to have signal
 tables
  for
ds
  un-necessarily complicating the process.
   
   
If you are interested, I would be happy to provide more details.
   
   
Thanks,
-namit
   
   
On 2/4/12 11:57 AM, Ashutosh Chauhan hashut...@apache.org
 wrote:
   
Hi Alexis,

Great to see your interest. Feel free to come up with concrete
  proposal
and
submit to GSoC. Its certainly heartening to see folks interested
 in
   making
contributions to the Hive Project.

Ashutosh
On Sat, Feb 4, 2012 at 10:48, Alexis De La Cruz Toledo
alexis...@gmail.comwrote:

 Hi Ashutosh, I'm interesting in hive,
 I'd like to improve the compilation process,
 I have been that the plan query tree generated
 by Hive can be optimized, and I'd like
 to participate in Google Summer of code 2012.
 What do you say?

 Regards.


 El 4 de febrero de 2012 12:29, Ashutosh Chauhan 
   hashut...@apache.org
 escribió:

  Hey Bharath,
 
  Great to see your enthusiasm for Hive! I would be happy to
  mentor
   you
for
  the project.  For the start, you can take a look at
  https://cwiki.apache.org/confluence/display/Hive/Roadmapfor
 a
   list
of
  open
  projects in Hive. The document is bit dated, so some of
 those
   projects
 may
  not be relevant. But, its a good source to start with to see
 if
   any of
  these projects excite you.
 
  Hope it helps,
  Ashutosh
 
  On Sat, Feb 4, 2012 at 08:47, bharath vissapragada 
  bharathvissapragada1...@gmail.com wrote:
 
   Hey list, devs,
  
   Google summer of code, 2012 's notification [1] has been
  released
and
   mentoring organizations can submit their proposals to
 Google
  for
  opensource
   projects.
  
   Any of the devs interested in mentoring students 

[jira] [Commented] (HIVE-2996) minor bug in hooks (NPE)

2012-05-02 Thread Kevin Wilfong (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266420#comment-13266420
 ] 

Kevin Wilfong commented on HIVE-2996:
-

@Namit: The tests pass, but see Carl's comment.

@Carl: This patch, and the one it fixes, HIVE-2986, are just the first step in 
an effort by us to open source our environment, specifically the hooks we use, 
to initially allow us to be better able to work with other open source 
contributors to debug issues when a patch breaks them (as has happened in the 
past).  Neither patch provides (nor attempts to provide) any sort of test 
framework.

Creating a test framework is a goal, and some parts may overlap in the patch 
you mention in HIVE-2985 which should be investigated.  But again, the point of 
these two patches is not to create a test framework, and do not overlap with 
the patch you mention in HIVE-2985.

 minor bug in hooks (NPE)
 

 Key: HIVE-2996
 URL: https://issues.apache.org/jira/browse/HIVE-2996
 Project: Hive
  Issue Type: Bug
Reporter: Namit Jain
 Attachments: HIVE-2996.D2991.1.patch


 NPE in AbstractSmcConfigHook.java

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Execution Performance of my JobFlow (JobFlow consists of Hive and JDBC) on Amazon EMR

2012-05-02 Thread Bhavesh Shah
Hello all,

I have setup my job on Amazon Elastic Map Reduce some how after reading the
various links and some blogs. Now I am running my java code which has task
in Hive and JDBC.

I have test my program on local machine (single cluster). It is executing
well with no errors. But to increase the execution speed. Now I start
running that program on Amazon EMR but my execution is too slow compare to
single cluster machine. I am running my program with 2 Medium sized
instance.

Why is it so? Pls let me know about this.

Thanks.


-- 
Regards,
Bhavesh Shah


Re: Hive projects for Google Summer of code 2012 ?

2012-05-02 Thread Edward Capriolo
I meant 0-2 over the past few years. I was wrong in any case. My bad.

On Wed, May 2, 2012 at 2:49 AM, Carl Steinbach c...@cloudera.com wrote:
 Ed,

 Did you see this?

 http://google-melange.appspot.com/gsoc/project/google/gsoc2012/bharathv/29002

 On Wed, Mar 7, 2012 at 12:15 PM, Edward Capriolo edlinuxg...@gmail.comwrote:

 For the record we are 0-2 in gsoc.

 On Wed, Mar 7, 2012 at 1:42 AM, Namit Jain nj...@fb.com wrote:
  Done
 
 
  On 3/6/12 10:32 PM, bharath vissapragada
  bharathvissapragada1...@gmail.com wrote:
 
 Hi Namit,
 
 Is it possible to add Hive-1362 [1] to the list? I am interested in
 working
 on that and I've contacted Ashutosh regarding that.
 
 Thanks and Regards,
 
 
 [1]  https://issues.apache.org/jira/browse/HIVE-1362
 
 On Wed, Mar 7, 2012 at 6:46 AM, Namit Jain nj...@fb.com wrote:
 
 
 
 https://issues.apache.org/jira/secure/IssueNavigator.jspa?requestId=12319
 27
  0
 
 
 
  0 was in the next line by mistake
  Look for HIVE jiras.
 
 
 
  On 3/6/12 7:37 PM, Alexis De La Cruz Toledo alexis...@gmail.com
 wrote:
 
  Sorry Mr. Namit Jain I can't see the jiras.
  
  Thanks
  
  El 6 de marzo de 2012 19:41, Namit Jain nj...@fb.com escribió:
  
   I filed a couple of gsoc 2012 jiras.
  
   Please submit your proposal if you are interested.
  
  
  
 
 
 https://issues.apache.org/jira/secure/IssueNavigator.jspa?requestId=12319
  27
   0
  
  
  
  
   Thanks,
   -namit
  
  
   On 2/7/12 6:32 PM, Alexis De La Cruz Toledo alexis...@gmail.com
  wrote:
  
   Hi Namit Jain, I'm Alexis, I'm a master student, I'm studying in
   Cinvestav,
   df Mexico.
   I'm interesting in collaborating with Hive. I'd like to do my
 thesis
  work
   in something about Hive.
   I'd like to participate in Google Summer of Code 2012 too.
   
   The issues that you propose I find interesting, particularly in the
  next
   issues:
   * The topic 2. Indexed Joins.
   * [PO] Optimize Joins using Bloom Filters of this page
   https://cwiki.apache.org/confluence/display/Hive/Roadmap.
   
   Can you tell me something more about it?
   What is problem to be solved?
   What benefits we hope to gain?
   
   This because I want to raise my thesis problem.
   By another hand, can you be my mentor in Google Summer of Code, if
 I
  work
   with this topics?
   
   Thanks.
   
   El 5 de febrero de 2012 19:58, Namit Jain nj...@fb.com escribió:
   
Hi Alexis/Bharath,
   
Great to see your interests. If you looking for ideas, some
 things
  that
will be very useful are:
   
1. Removing the map-join hint completely.
  Moving all processing to runtime.
  Currently, bucketed map joins and sort merge joins are
 completely
driven off hints.
  It would be very helpful to the community, and also clean up
 the
  code
   a
lot.
   
2. Indexed Joins.
  Something that would be really useful -
  If the basic infrastructure is ready, it can eventually be used
 to
   join
tables
  stored outside also (say Hbase).
   
3. Metastore understanding hierarchy.
  For eg: if a table is partitioned by ds and hr,
  A valid partition on ds does not exist. This is a very common
  usecase
on many
  applications, and the current work-around is to have signal
 tables
  for
ds
  un-necessarily complicating the process.
   
   
If you are interested, I would be happy to provide more details.
   
   
Thanks,
-namit
   
   
On 2/4/12 11:57 AM, Ashutosh Chauhan hashut...@apache.org
 wrote:
   
Hi Alexis,

Great to see your interest. Feel free to come up with concrete
  proposal
and
submit to GSoC. Its certainly heartening to see folks interested
 in
   making
contributions to the Hive Project.

Ashutosh
On Sat, Feb 4, 2012 at 10:48, Alexis De La Cruz Toledo
alexis...@gmail.comwrote:

 Hi Ashutosh, I'm interesting in hive,
 I'd like to improve the compilation process,
 I have been that the plan query tree generated
 by Hive can be optimized, and I'd like
 to participate in Google Summer of code 2012.
 What do you say?

 Regards.


 El 4 de febrero de 2012 12:29, Ashutosh Chauhan 
   hashut...@apache.org
 escribió:

  Hey Bharath,
 
  Great to see your enthusiasm for Hive! I would be happy to
  mentor
   you
for
  the project.  For the start, you can take a look at
  https://cwiki.apache.org/confluence/display/Hive/Roadmapfor
 a
   list
of
  open
  projects in Hive. The document is bit dated, so some of
 those
   projects
 may
  not be relevant. But, its a good source to start with to see
 if
   any of
  these projects excite you.
 
  Hope it helps,
  Ashutosh
 
  On Sat, Feb 4, 2012 at 08:47, bharath vissapragada 
  bharathvissapragada1...@gmail.com wrote:
 
   Hey list, devs,
  
   Google summer of code, 2012 's notification [1] has been
  released
and
   mentoring 

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #2

2012-05-02 Thread Apache Jenkins Server
See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/2/

--
[...truncated 9030 lines...]
|  default |   8   |   0   |   0   |   0   ||   8   |   0   |
|  master  |   0   |   0   |   0   |   0   ||   0   |   0   |
|  compile |   0   |   0   |   0   |   0   ||   0   |   0   |
|  runtime |   0   |   0   |   0   |   0   ||   0   |   0   |
-
[ivy:retrieve] :: retrieving :: org.apache.hive#hive-shims
[ivy:retrieve]  confs: [default, master, compile, runtime]
[ivy:retrieve]  0 artifacts copied, 8 already retrieved (0kB/3ms)

install-hadoopcore-internal:

build_shims:
 [echo] Project: shims
 [echo] Compiling 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/shims/src/common/java;/home/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.20/java
 against hadoop 0.20.1 
(https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/2/artifact/hive/build/hadoopcore/hadoop-0.20.1)
 [echo] Building shims 0.20S

ivy-retrieve-hadoop-source:
 [echo] Project: shims
 [echo] hadoop.version.ant-internal: 1.0.0
[ivy:retrieve] :: loading settings :: file = 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml
[ivy:retrieve] :: resolving dependencies :: 
org.apache.hive#hive-shims;0.9.1-SNAPSHOT
[ivy:retrieve]  confs: [default, master, compile, runtime]
[ivy:retrieve]  found org.apache.hadoop#hadoop-core;1.0.0 in maven2
[ivy:retrieve]  found commons-cli#commons-cli;1.2 in maven2
[ivy:retrieve]  found xmlenc#xmlenc;0.52 in maven2
[ivy:retrieve]  found commons-httpclient#commons-httpclient;3.0.1 in maven2
[ivy:retrieve]  found junit#junit;3.8.1 in maven2
[ivy:retrieve]  found commons-logging#commons-logging;1.0.3 in maven2
[ivy:retrieve]  found commons-codec#commons-codec;1.4 in maven2
[ivy:retrieve]  found org.apache.commons#commons-math;2.1 in maven2
[ivy:retrieve]  found commons-configuration#commons-configuration;1.6 in maven2
[ivy:retrieve]  found commons-collections#commons-collections;3.2.1 in maven2
[ivy:retrieve]  found commons-lang#commons-lang;2.4 in maven2
[ivy:retrieve]  found commons-logging#commons-logging;1.1.1 in maven2
[ivy:retrieve]  found commons-digester#commons-digester;1.8 in maven2
[ivy:retrieve]  found commons-beanutils#commons-beanutils;1.7.0 in maven2
[ivy:retrieve]  found commons-beanutils#commons-beanutils-core;1.8.0 in maven2
[ivy:retrieve]  found commons-net#commons-net;1.4.1 in maven2
[ivy:retrieve]  found oro#oro;2.0.8 in maven2
[ivy:retrieve]  found org.mortbay.jetty#jetty;6.1.26 in maven2
[ivy:retrieve]  found org.mortbay.jetty#jetty-util;6.1.26 in maven2
[ivy:retrieve]  found org.mortbay.jetty#servlet-api;2.5-20081211 in maven2
[ivy:retrieve]  found tomcat#jasper-runtime;5.5.12 in maven2
[ivy:retrieve]  found tomcat#jasper-compiler;5.5.12 in maven2
[ivy:retrieve]  found org.mortbay.jetty#jsp-api-2.1;6.1.14 in maven2
[ivy:retrieve]  found org.mortbay.jetty#servlet-api-2.5;6.1.14 in maven2
[ivy:retrieve]  found org.mortbay.jetty#jsp-2.1;6.1.14 in maven2
[ivy:retrieve]  found org.eclipse.jdt#core;3.1.1 in maven2
[ivy:retrieve]  found ant#ant;1.6.5 in maven2
[ivy:retrieve]  found commons-el#commons-el;1.0 in maven2
[ivy:retrieve]  found net.java.dev.jets3t#jets3t;0.7.1 in maven2
[ivy:retrieve]  found net.sf.kosmosfs#kfs;0.3 in maven2
[ivy:retrieve]  found hsqldb#hsqldb;1.8.0.10 in maven2
[ivy:retrieve]  found org.apache.zookeeper#zookeeper;3.4.3 in maven2
[ivy:retrieve]  found org.apache.thrift#libthrift;0.7.0 in maven2
[ivy:retrieve]  found commons-logging#commons-logging;1.0.4 in maven2
[ivy:retrieve]  found commons-logging#commons-logging-api;1.0.4 in maven2
[ivy:retrieve]  found com.google.guava#guava;r09 in maven2
[ivy:retrieve]  found org.codehaus.jackson#jackson-core-asl;1.8.8 in maven2
[ivy:retrieve]  found org.codehaus.jackson#jackson-mapper-asl;1.8.8 in maven2
[ivy:retrieve] :: resolution report :: resolve 235ms :: artifacts dl 14ms
[ivy:retrieve]  :: evicted modules:
[ivy:retrieve]  commons-logging#commons-logging;1.0.3 by 
[[commons-logging#commons-logging;1.0.4, 
commons-logging#commons-logging;1.1.1]] in [default]
[ivy:retrieve]  commons-codec#commons-codec;1.2 by 
[commons-codec#commons-codec;1.4] in [default]
[ivy:retrieve]  commons-logging#commons-logging;1.1 by 
[[commons-logging#commons-logging;1.0.4, 
commons-logging#commons-logging;1.1.1]] in [default]
[ivy:retrieve]  commons-codec#commons-codec;1.3 by 
[commons-codec#commons-codec;1.4] in [default]
[ivy:retrieve]  commons-httpclient#commons-httpclient;3.1 by 
[commons-httpclient#commons-httpclient;3.0.1] in [default]
-
|  |modules||   artifacts   |
|  

[jira] [Resolved] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs

2012-05-02 Thread Ashutosh Chauhan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan resolved HIVE-2646.


Resolution: Fixed

Merged in 0.9 branch.

 Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
 

 Key: HIVE-2646
 URL: https://issues.apache.org/jira/browse/HIVE-2646
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Affects Versions: 0.8.0
Reporter: Andrew Bayer
Assignee: Andrew Bayer
Priority: Critical
 Fix For: 0.10.0, 0.9.1

 Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, 
 HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, 
 HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, 
 HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, 
 HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, 
 HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, 
 HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, 
 HIVE-2646.diff.txt


 The current Hive Ivy dependency logic for its Hadoop dependencies is 
 problematic - depending on the tarball and extracting the jars from there, 
 rather than depending on the jars directly. It'd be great if this was fixed 
 to actually have the jar dependencies defined directly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2976) Fix maven-build Ant target

2012-05-02 Thread Ashutosh Chauhan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-2976:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Merged in branch 0.9

 Fix maven-build Ant target
 --

 Key: HIVE-2976
 URL: https://issues.apache.org/jira/browse/HIVE-2976
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Reporter: Carl Steinbach
Assignee: Thomas Weise
 Fix For: 0.10.0, 0.9.1

 Attachments: HIVE-2976-maven.diff.txt, HIVE-2976.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2757) hive can't find hadoop executor scripts without HADOOP_HOME set

2012-05-02 Thread Roman Shaposhnik (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Shaposhnik updated HIVE-2757:
---

Attachment: HIVE-2757.patch.txt

Sorry for the delay -- it seems I don't have much luck with running Hive unit 
tests on my machine :-(

On the positive side -- I tested this patch against a real cluster and it works 
as expected.

 hive can't find hadoop executor scripts without HADOOP_HOME set
 ---

 Key: HIVE-2757
 URL: https://issues.apache.org/jira/browse/HIVE-2757
 Project: Hive
  Issue Type: Bug
  Components: CLI
Affects Versions: 0.8.0
Reporter: Roman Shaposhnik
 Attachments: HIVE-2757.patch.txt, HIVE-2757.patch.txt, hive-2757.diff


 The trouble is that in Hadoop 0.23 HADOOP_HOME has been deprecated. I think 
 it would be really nice if bin/hive can be modified to capture the which 
 hadoop
 and pass that as a property into the JVM.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2757) hive can't find hadoop executor scripts without HADOOP_HOME set

2012-05-02 Thread Roman Shaposhnik (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Shaposhnik updated HIVE-2757:
---

Status: Patch Available  (was: Open)

 hive can't find hadoop executor scripts without HADOOP_HOME set
 ---

 Key: HIVE-2757
 URL: https://issues.apache.org/jira/browse/HIVE-2757
 Project: Hive
  Issue Type: Bug
  Components: CLI
Affects Versions: 0.8.0
Reporter: Roman Shaposhnik
 Attachments: HIVE-2757.patch.txt, HIVE-2757.patch.txt, hive-2757.diff


 The trouble is that in Hadoop 0.23 HADOOP_HOME has been deprecated. I think 
 it would be really nice if bin/hive can be modified to capture the which 
 hadoop
 and pass that as a property into the JVM.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2757) hive can't find hadoop executor scripts without HADOOP_HOME set

2012-05-02 Thread Roman Shaposhnik (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266715#comment-13266715
 ] 

Roman Shaposhnik commented on HIVE-2757:


@Ashutosh,

I think at this point I'd go for the minimally invasive patch (which is 
attached). Potential improvements would include merging the stuff that Buddhika 
posted (and better yet, following up with a patch that migrates us to 
ProcessBuilder). However, I believe all of those things need to be handled in 
separate JIRAs.

@Edward, 

I agree with your comments on standardizing the environment. It is fine for 
Hive to keep depending on env. variables, but it also needs to have reasonable 
defaults.



 hive can't find hadoop executor scripts without HADOOP_HOME set
 ---

 Key: HIVE-2757
 URL: https://issues.apache.org/jira/browse/HIVE-2757
 Project: Hive
  Issue Type: Bug
  Components: CLI
Affects Versions: 0.8.0
Reporter: Roman Shaposhnik
 Attachments: HIVE-2757.patch.txt, HIVE-2757.patch.txt, hive-2757.diff


 The trouble is that in Hadoop 0.23 HADOOP_HOME has been deprecated. I think 
 it would be really nice if bin/hive can be modified to capture the which 
 hadoop
 and pass that as a property into the JVM.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2976) Fix maven-build Ant target

2012-05-02 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266730#comment-13266730
 ] 

Hudson commented on HIVE-2976:
--

Integrated in Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #3 (See 
[https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/3/])
HIVE-2976 Fix maven-build Ant target  Merged in from trunk. (Andrew Bayer 
and Thomas Weise) (Revision 1333095)

 Result = FAILURE
hashutosh : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1333095
Files : 
* /hive/branches/branch-0.9
* /hive/branches/branch-0.9/build.xml
* /hive/branches/branch-0.9/builtins/build.xml


 Fix maven-build Ant target
 --

 Key: HIVE-2976
 URL: https://issues.apache.org/jira/browse/HIVE-2976
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Reporter: Carl Steinbach
Assignee: Thomas Weise
 Fix For: 0.10.0, 0.9.1

 Attachments: HIVE-2976-maven.diff.txt, HIVE-2976.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs

2012-05-02 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266729#comment-13266729
 ] 

Hudson commented on HIVE-2646:
--

Integrated in Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #3 (See 
[https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/3/])
HIVE-2646 Hive Ivy dependencies on Hadoop should depend on jars directly, 
not tarballs. Merged in from trunk. (Andrew Bayer and Thomas Weise) (Revision 
1333094)

 Result = FAILURE
hashutosh : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1333094
Files : 
* /hive/branches/branch-0.9
* /hive/branches/branch-0.9/build-common.xml
* /hive/branches/branch-0.9/build.properties
* /hive/branches/branch-0.9/build.xml
* /hive/branches/branch-0.9/builtins/build.xml
* /hive/branches/branch-0.9/builtins/ivy.xml
* /hive/branches/branch-0.9/cli/ivy.xml
* /hive/branches/branch-0.9/common/ivy.xml
* /hive/branches/branch-0.9/contrib/build.xml
* /hive/branches/branch-0.9/contrib/ivy.xml
* /hive/branches/branch-0.9/hbase-handler/build.xml
* /hive/branches/branch-0.9/hbase-handler/ivy.xml
* /hive/branches/branch-0.9/hwi/build.xml
* /hive/branches/branch-0.9/hwi/ivy.xml
* /hive/branches/branch-0.9/ivy/common-configurations.xml
* /hive/branches/branch-0.9/ivy/ivysettings.xml
* /hive/branches/branch-0.9/ivy/libraries.properties
* /hive/branches/branch-0.9/jdbc/build.xml
* /hive/branches/branch-0.9/jdbc/ivy.xml
* /hive/branches/branch-0.9/metastore/ivy.xml
* /hive/branches/branch-0.9/pdk/build.xml
* /hive/branches/branch-0.9/pdk/ivy.xml
* /hive/branches/branch-0.9/pdk/scripts/build-plugin.xml
* /hive/branches/branch-0.9/ql/build.xml
* /hive/branches/branch-0.9/ql/ivy.xml
* /hive/branches/branch-0.9/serde/ivy.xml
* /hive/branches/branch-0.9/service/build.xml
* /hive/branches/branch-0.9/service/ivy.xml
* /hive/branches/branch-0.9/shims/build.xml
* /hive/branches/branch-0.9/shims/ivy.xml
* /hive/branches/branch-0.9/testutils/hadoop


 Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
 

 Key: HIVE-2646
 URL: https://issues.apache.org/jira/browse/HIVE-2646
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Affects Versions: 0.8.0
Reporter: Andrew Bayer
Assignee: Andrew Bayer
Priority: Critical
 Fix For: 0.10.0, 0.9.1

 Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, 
 HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, 
 HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, 
 HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, 
 HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, 
 HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, 
 HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, 
 HIVE-2646.diff.txt


 The current Hive Ivy dependency logic for its Hadoop dependencies is 
 problematic - depending on the tarball and extracting the jars from there, 
 rather than depending on the jars directly. It'd be great if this was fixed 
 to actually have the jar dependencies defined directly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #3

2012-05-02 Thread Apache Jenkins Server
See 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/3/changes

Changes:

[hashutosh] HIVE-2976 Fix maven-build Ant target  Merged in from trunk. (Andrew 
Bayer and Thomas Weise)

[hashutosh] HIVE-2646 Hive Ivy dependencies on Hadoop should depend on jars 
directly, not tarballs. Merged in from trunk. (Andrew Bayer and Thomas Weise)

--
[...truncated 10110 lines...]

ivy-resolve-test:
 [echo] Project: odbc

ivy-retrieve-test:
 [echo] Project: odbc

compile-test:
 [echo] Project: odbc

create-dirs:
 [echo] Project: serde
 [copy] Warning: 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/serde/src/test/resources
 does not exist.

init:
 [echo] Project: serde

ivy-init-settings:
 [echo] Project: serde

ivy-resolve:
 [echo] Project: serde
[ivy:resolve] :: loading settings :: file = 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml
[ivy:report] Processing 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/3/artifact/hive/build/ivy/resolution-cache/org.apache.hive-hive-serde-default.xml
 to 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/3/artifact/hive/build/ivy/report/org.apache.hive-hive-serde-default.html

ivy-retrieve:
 [echo] Project: serde

dynamic-serde:

compile:
 [echo] Project: serde

ivy-resolve-test:
 [echo] Project: serde

ivy-retrieve-test:
 [echo] Project: serde

compile-test:
 [echo] Project: serde
[javac] Compiling 26 source files to 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/3/artifact/hive/build/serde/test/classes
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.

create-dirs:
 [echo] Project: service
 [copy] Warning: 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/service/src/test/resources
 does not exist.

init:
 [echo] Project: service

ivy-init-settings:
 [echo] Project: service

ivy-resolve:
 [echo] Project: service
[ivy:resolve] :: loading settings :: file = 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml
[ivy:report] Processing 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/3/artifact/hive/build/ivy/resolution-cache/org.apache.hive-hive-service-default.xml
 to 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/3/artifact/hive/build/ivy/report/org.apache.hive-hive-service-default.html

ivy-retrieve:
 [echo] Project: service

compile:
 [echo] Project: service

ivy-resolve-test:
 [echo] Project: service

ivy-retrieve-test:
 [echo] Project: service

compile-test:
 [echo] Project: service
[javac] Compiling 2 source files to 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/3/artifact/hive/build/service/test/classes

test:
 [echo] Project: hive

test-shims:
 [echo] Project: hive

test-conditions:
 [echo] Project: shims

gen-test:
 [echo] Project: shims

create-dirs:
 [echo] Project: shims
 [copy] Warning: 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/shims/src/test/resources
 does not exist.

init:
 [echo] Project: shims

ivy-init-settings:
 [echo] Project: shims

ivy-resolve:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml
[ivy:report] Processing 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/3/artifact/hive/build/ivy/resolution-cache/org.apache.hive-hive-shims-default.xml
 to 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/3/artifact/hive/build/ivy/report/org.apache.hive-hive-shims-default.html

ivy-retrieve:
 [echo] Project: shims

compile:
 [echo] Project: shims
 [echo] Building shims 0.20

build_shims:
 [echo] Project: shims
 [echo] Compiling 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/shims/src/common/java;/home/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.20/java
 against hadoop 0.20.2 
(https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/3/artifact/hive/build/hadoopcore/hadoop-0.20.2)

ivy-init-settings:
 [echo] Project: shims

ivy-resolve-hadoop-shim:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml

ivy-retrieve-hadoop-shim:
 [echo] Project: shims
 [echo] Building shims 0.20S

build_shims:
 [echo] 

[jira] [Commented] (HIVE-2996) minor bug in hooks (NPE)

2012-05-02 Thread Carl Steinbach (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266798#comment-13266798
 ] 

Carl Steinbach commented on HIVE-2996:
--

@Namit: My main concern is that many of the hooks which were committed in 
HIVE-2986 reference non-public APIs. For example, I see references to 
SessionState, QueryPlan, DDLTask, ExplainTask, HiveOperation, etc. It looks 
like one consequence of adding these hooks is that developers will now have to 
treat them as de facto public APIs. Does that sound accurate? If so, what's the 
proposed policy for handling future changes to these internal APIs? What will 
be the process for modifying the hooks which were added in HIVE-2986, and where 
will the tests that reference these hooks live? If a change breaks one of these 
tests, who is responsible for fixing it?

 minor bug in hooks (NPE)
 

 Key: HIVE-2996
 URL: https://issues.apache.org/jira/browse/HIVE-2996
 Project: Hive
  Issue Type: Bug
Reporter: Namit Jain
 Attachments: HIVE-2996.D2991.1.patch


 NPE in AbstractSmcConfigHook.java

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2757) hive can't find hadoop executor scripts without HADOOP_HOME set

2012-05-02 Thread Carl Steinbach (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266866#comment-13266866
 ] 

Carl Steinbach commented on HIVE-2757:
--

@Roman: Can you submit a phabricator review request? Thanks.
https://cwiki.apache.org/Hive/phabricatorcodereview.html

 hive can't find hadoop executor scripts without HADOOP_HOME set
 ---

 Key: HIVE-2757
 URL: https://issues.apache.org/jira/browse/HIVE-2757
 Project: Hive
  Issue Type: Bug
  Components: CLI
Affects Versions: 0.8.0
Reporter: Roman Shaposhnik
 Attachments: HIVE-2757.patch.txt, HIVE-2757.patch.txt, hive-2757.diff


 The trouble is that in Hadoop 0.23 HADOOP_HOME has been deprecated. I think 
 it would be really nice if bin/hive can be modified to capture the which 
 hadoop
 and pass that as a property into the JVM.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2996) minor bug in hooks (NPE)

2012-05-02 Thread Namit Jain (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266902#comment-13266902
 ] 

Namit Jain commented on HIVE-2996:
--

@Carl, this is a pretty big change, and I did not want to wait for all the 
cleanups to happen before I could get it in.
Having said that, I will be providing 2 sets of tests for these soon: 
1. Unit tests
2. End to End tests which will use these hooks.

Once these tests are in, it would be very easy to modify the hooks. It is just 
I did not want to wait for the whole patch to go in a single patch - 
this is not touching any hive code, and should not break anything.

 minor bug in hooks (NPE)
 

 Key: HIVE-2996
 URL: https://issues.apache.org/jira/browse/HIVE-2996
 Project: Hive
  Issue Type: Bug
Reporter: Namit Jain
 Attachments: HIVE-2996.D2991.1.patch


 NPE in AbstractSmcConfigHook.java

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2975) Filter parsing does not recognize '!=' as operator and silently ignores invalid tokens

2012-05-02 Thread Phabricator (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266905#comment-13266905
 ] 

Phabricator commented on HIVE-2975:
---

ashutoshc has commented on the revision HIVE-2975 [jira] Filter parsing does 
not recognize '!=' as operator and silently ignores invalid tokens.

  Other then those two comments, patch looks good.

INLINE COMMENTS
  
metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java:178
 Can you also include Operator.NOTEQUALS2 in this expression so that it works 
for listTableNamesByFilter() api as well.
  
metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:1839 
Can you also add a test for table filtering within a database. 
testTableFilter() method will be best suited for it.

REVISION DETAIL
  https://reviews.facebook.net/D2931


 Filter parsing does not recognize '!=' as operator and silently ignores 
 invalid tokens
 --

 Key: HIVE-2975
 URL: https://issues.apache.org/jira/browse/HIVE-2975
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Reporter: Thomas Weise
Assignee: Thomas Weise
 Fix For: 0.9.1

 Attachments: HIVE-2975.D2931.1.patch, HIVE-2975.D2931.2.patch


 Should support operator as alternative to '' and should not produce false 
 positives on lexer errors.
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Hive-trunk-h0.21 - Build # 1408 - Failure

2012-05-02 Thread Apache Jenkins Server
Changes for Build #1408



1 tests failed.
REGRESSION:  
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_scriptfile1

Error Message:
Unexpected exception See build/ql/tmp/hive.log, or try ant test ... 
-Dtest.silent=false to get more logs.

Stack Trace:
junit.framework.AssertionFailedError: Unexpected exception
See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get 
more logs.
at junit.framework.Assert.fail(Assert.java:50)
at 
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_scriptfile1(TestMinimrCliDriver.java:573)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)




The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1408)

Status: Failure

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1408/ to 
view the results.

[jira] [Commented] (HIVE-2996) minor bug in hooks (NPE)

2012-05-02 Thread Carl Steinbach (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266922#comment-13266922
 ] 

Carl Steinbach commented on HIVE-2996:
--


@Namit: I'm not concerned that these hooks are going to break
anything. Actually, I have the opposite concern, namely that the
new hooks and tests are going to make it much harder for Hive
developers to improve internal, non-public APIs by locking down
internal, non-public implementation details. We already have more
than a thousand tests designed to catch changes to the public
APIs (HQL, the public listener and hook interfaces, the Thrift
APIs, etc), and HIVE-2670 is set to add better support for
running these tests in the sort of e2e scenarios that members of
the community encounter in production environments.

Since this is such a big change I propose that this work be done
on a separate branch, either here or on github, and then when
it's finished we can give the committers a chance to evaluate it
as a whole. How does that sound?

 minor bug in hooks (NPE)
 

 Key: HIVE-2996
 URL: https://issues.apache.org/jira/browse/HIVE-2996
 Project: Hive
  Issue Type: Bug
Reporter: Namit Jain
 Attachments: HIVE-2996.D2991.1.patch


 NPE in AbstractSmcConfigHook.java

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2996) minor bug in hooks (NPE)

2012-05-02 Thread Namit Jain (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266930#comment-13266930
 ] 

Namit Jain commented on HIVE-2996:
--

@Carl, this will be a iterative process, and the changes are only going in 
contrib.
The internal, non-public APIs are locked for facebook anyway, and we run into 
these
problems at deployment time. It would be very difficult to keep it in sync. in 
a separate 
branch.

Right now, there are no tests for these, so any change by any contributor 
cannot break anything 
in these hooks. As I will be adding tests for these hooks, I will be converting 
these hooks to 
not use any private data-structures/APIs. That is the reason, I did not blindly 
any existing tests.
In the last few pushes, we have run into random issues, which could have been 
addressed before had
this test been in place.

As I said before, the purpose of HIVE-2670 is similar, but having the facebook 
deployment in the open
makes us feel much more comfortable in running the recent branch of hive. I 
really dont want Facebook 
to have its own branch for hive, which makes it very difficult to sync. up 
later. The problem is that
if I keep developing in a separate branch, i will diverge very quickly and be 
never able to converge.

 minor bug in hooks (NPE)
 

 Key: HIVE-2996
 URL: https://issues.apache.org/jira/browse/HIVE-2996
 Project: Hive
  Issue Type: Bug
Reporter: Namit Jain
 Attachments: HIVE-2996.D2991.1.patch


 NPE in AbstractSmcConfigHook.java

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HIVE-2998) Making Hive run on Windows Server and Windows Azure environment

2012-05-02 Thread Lengning Liu (JIRA)
Lengning Liu created HIVE-2998:
--

 Summary: Making Hive run on Windows Server and Windows Azure 
environment
 Key: HIVE-2998
 URL: https://issues.apache.org/jira/browse/HIVE-2998
 Project: Hive
  Issue Type: Improvement
Affects Versions: 0.8.1, 0.7.1
 Environment: Windows Server 2008 R2 and Windows Azure
Reporter: Lengning Liu


This is the master JIRA for improvements on Hive that target on making Hive 
running natively on Windows Server and Windows Azure environment. Microsoft has 
done an initial work to make Hive (release 0.7.1 and 0.8.1) to run on Windows 
environment natively, and wants to contribute this work back to the community. 
The patches to sub-tasks of this JIRA are based on this work. The end to end 
HiveQL queries that Microsoft used for testing pass. We are working on 
investigating failed unit test cases at this moment and hopefully reaching the 
required bar in the next few months. We are posting the patches now so that we 
can get early feedback from the community and collaborate with other 
contributors who are interested in this work in the future. Thanks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-895) Add SerDe for Avro serialized data

2012-05-02 Thread Jakob Homan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jakob Homan updated HIVE-895:
-

Attachment: HIVE-895-draft.patch

Here's a first draft of the port to ASF.  It corresponds to the 
mergeHive8ToMaster branch on github, which has all the latest fixes and is 
compatible with Hive 8.  Need to re-format to Hive style and run full unit 
tests.  

One thing of concern is that the avroserde relies on the ql package, which 
required a change to the build script to build serde afterwards.  Is there a 
defined dependency for Hive's modules, and if so does this break that?  If so, 
the other option would be to move this to the contrib package, but to me 
contrib is a dirty word and I'd like to avoid that.  

Also, this bundles the avro serde into the serde jar.  It'd be nice for those 
not using Avro to not require it, but Avro is already a build-time dependency 
so it's not a new problem.  Eventually it'd be nice to have a separate jar with 
just the serde in it to make the code more modular.

I'll finish the port in the next couple of days, but take a glance and comment 
if you'd like.

 Add SerDe for Avro serialized data
 --

 Key: HIVE-895
 URL: https://issues.apache.org/jira/browse/HIVE-895
 Project: Hive
  Issue Type: New Feature
  Components: Serializers/Deserializers
Reporter: Jeff Hammerbacher
Assignee: Jakob Homan
 Attachments: HIVE-895-draft.patch


 As Avro continues to mature, having a SerDe to allow HiveQL queries over Avro 
 data seems like a solid win.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2990) Remove hadoop-source Ivy resolvers and Ant targets

2012-05-02 Thread Ashutosh Chauhan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-2990:
---

   Resolution: Fixed
Fix Version/s: 0.10.0
   Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks, Carl!

 Remove hadoop-source Ivy resolvers and Ant targets
 --

 Key: HIVE-2990
 URL: https://issues.apache.org/jira/browse/HIVE-2990
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Affects Versions: 0.10.0
Reporter: Carl Steinbach
Assignee: Carl Steinbach
 Fix For: 0.10.0

 Attachments: HIVE-2990.D2973.1.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2998) Making Hive run on Windows Server and Windows Azure environment

2012-05-02 Thread Lengning Liu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lengning Liu updated HIVE-2998:
---

Description: This is the master JIRA for improvements on Hive that target 
on making Hive running natively on Windows Server and Windows Azure 
environment. Microsoft has done an initial work to make Hive (release 0.7.1 and 
0.8.1) to run on Windows environment natively, and wants to contribute this 
work back to the community. The patches to sub-tasks of this JIRA are based on 
this work. The end-to-end HiveQL query tests pass. We are working on 
investigating failed unit test cases at this moment and hopefully reaching the 
required bar in the next few months. We are posting the patches now so that we 
can get early feedback from the community and collaborate with other 
contributors who are interested in this work in the future. Thanks.  (was: This 
is the master JIRA for improvements on Hive that target on making Hive running 
natively on Windows Server and Windows Azure environment. Microsoft has done an 
initial work to make Hive (release 0.7.1 and 0.8.1) to run on Windows 
environment natively, and wants to contribute this work back to the community. 
The patches to sub-tasks of this JIRA are based on this work. The end to end 
HiveQL queries that Microsoft used for testing pass. We are working on 
investigating failed unit test cases at this moment and hopefully reaching the 
required bar in the next few months. We are posting the patches now so that we 
can get early feedback from the community and collaborate with other 
contributors who are interested in this work in the future. Thanks.)

 Making Hive run on Windows Server and Windows Azure environment
 ---

 Key: HIVE-2998
 URL: https://issues.apache.org/jira/browse/HIVE-2998
 Project: Hive
  Issue Type: Improvement
Affects Versions: 0.7.1, 0.8.1
 Environment: Windows Server 2008 R2 and Windows Azure
Reporter: Lengning Liu

 This is the master JIRA for improvements on Hive that target on making Hive 
 running natively on Windows Server and Windows Azure environment. Microsoft 
 has done an initial work to make Hive (release 0.7.1 and 0.8.1) to run on 
 Windows environment natively, and wants to contribute this work back to the 
 community. The patches to sub-tasks of this JIRA are based on this work. The 
 end-to-end HiveQL query tests pass. We are working on investigating failed 
 unit test cases at this moment and hopefully reaching the required bar in the 
 next few months. We are posting the patches now so that we can get early 
 feedback from the community and collaborate with other contributors who are 
 interested in this work in the future. Thanks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Assigned] (HIVE-2529) metastore 0.8 upgrade script for PostgreSQL

2012-05-02 Thread Zhenxiao Luo (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo reassigned HIVE-2529:
--

Assignee: Zhenxiao Luo  (was: Carl Steinbach)

 metastore 0.8 upgrade script for PostgreSQL 
 

 Key: HIVE-2529
 URL: https://issues.apache.org/jira/browse/HIVE-2529
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Affects Versions: 0.8.0
Reporter: John Sichi
Assignee: Zhenxiao Luo
Priority: Blocker

 I think you mentioned that this was in the works.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HIVE-2999) Offline build is not working

2012-05-02 Thread Navis (JIRA)
Navis created HIVE-2999:
---

 Summary: Offline build is not working
 Key: HIVE-2999
 URL: https://issues.apache.org/jira/browse/HIVE-2999
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Affects Versions: 0.10.0
Reporter: Navis


It's fine without -Doffline=true option. But with offline option (ant 
-Doffline=true clean package), it's failing with error message like this.

{noformat}
ivy-retrieve:
 [echo] Project: common
[ivy:retrieve] :: loading settings :: file = 
/home/navis/apache/oss-hive/ivy/ivysettings.xml
[ivy:retrieve] 
[ivy:retrieve] :: problems summary ::
[ivy:retrieve]  WARNINGS
[ivy:retrieve]  module not found: org.apache.hadoop#hadoop-common;0.20.2
[ivy:retrieve]   local: tried
[ivy:retrieve]
/home/navis/.ivy2/local/org.apache.hadoop/hadoop-common/0.20.2/ivys/ivy.xml
[ivy:retrieve]-- artifact 
org.apache.hadoop#hadoop-common;0.20.2!hadoop-common.jar:
[ivy:retrieve]
/home/navis/.ivy2/local/org.apache.hadoop/hadoop-common/0.20.2/jars/hadoop-common.jar
[ivy:retrieve]   apache-snapshot: tried
[ivy:retrieve]
https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.20.2/hadoop-common-0.20.2.pom
[ivy:retrieve]-- artifact 
org.apache.hadoop#hadoop-common;0.20.2!hadoop-common.jar:
[ivy:retrieve]
https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.20.2/hadoop-common-0.20.2.jar
[ivy:retrieve]   maven2: tried
[ivy:retrieve]
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.20.2/hadoop-common-0.20.2.pom
[ivy:retrieve]-- artifact 
org.apache.hadoop#hadoop-common;0.20.2!hadoop-common.jar:
[ivy:retrieve]
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.20.2/hadoop-common-0.20.2.jar
[ivy:retrieve]   datanucleus-repo: tried
[ivy:retrieve]-- artifact 
org.apache.hadoop#hadoop-common;0.20.2!hadoop-common.jar:
[ivy:retrieve]
http://www.datanucleus.org/downloads/maven2/org/apache/hadoop/hadoop-common/0.20.2/hadoop-common-0.20.2.jar
[ivy:retrieve]   hadoop-source: tried
[ivy:retrieve]-- artifact 
org.apache.hadoop#hadoop-common;0.20.2!hadoop-common.jar:
[ivy:retrieve]
http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-common-0.20.2/hadoop-common-0.20.2.jar
[ivy:retrieve]   hadoop-source2: tried
[ivy:retrieve]-- artifact 
org.apache.hadoop#hadoop-common;0.20.2!hadoop-common.jar:
[ivy:retrieve]
http://archive.cloudera.com/hive-deps/hadoop/core/hadoop-common-0.20.2/hadoop-common-0.20.2.jar
[ivy:retrieve]  module not found: org.apache.hadoop#hadoop-auth;0.20.2
[ivy:retrieve]   local: tried
[ivy:retrieve]
/home/navis/.ivy2/local/org.apache.hadoop/hadoop-auth/0.20.2/ivys/ivy.xml
[ivy:retrieve]-- artifact 
org.apache.hadoop#hadoop-auth;0.20.2!hadoop-auth.jar:
[ivy:retrieve]
/home/navis/.ivy2/local/org.apache.hadoop/hadoop-auth/0.20.2/jars/hadoop-auth.jar
[ivy:retrieve]   apache-snapshot: tried
[ivy:retrieve]
https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-auth/0.20.2/hadoop-auth-0.20.2.pom
[ivy:retrieve]-- artifact 
org.apache.hadoop#hadoop-auth;0.20.2!hadoop-auth.jar:
[ivy:retrieve]
https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-auth/0.20.2/hadoop-auth-0.20.2.jar
[ivy:retrieve]   maven2: tried
[ivy:retrieve]
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-auth/0.20.2/hadoop-auth-0.20.2.pom
[ivy:retrieve]-- artifact 
org.apache.hadoop#hadoop-auth;0.20.2!hadoop-auth.jar:
[ivy:retrieve]
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-auth/0.20.2/hadoop-auth-0.20.2.jar
[ivy:retrieve]   datanucleus-repo: tried
[ivy:retrieve]-- artifact 
org.apache.hadoop#hadoop-auth;0.20.2!hadoop-auth.jar:
[ivy:retrieve]
http://www.datanucleus.org/downloads/maven2/org/apache/hadoop/hadoop-auth/0.20.2/hadoop-auth-0.20.2.jar
[ivy:retrieve]   hadoop-source: tried
[ivy:retrieve]-- artifact 
org.apache.hadoop#hadoop-auth;0.20.2!hadoop-auth.jar:
[ivy:retrieve]
http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-auth-0.20.2/hadoop-auth-0.20.2.jar
[ivy:retrieve]   hadoop-source2: tried
[ivy:retrieve]-- artifact 
org.apache.hadoop#hadoop-auth;0.20.2!hadoop-auth.jar:
[ivy:retrieve]
http://archive.cloudera.com/hive-deps/hadoop/core/hadoop-auth-0.20.2/hadoop-auth-0.20.2.jar
[ivy:retrieve]  module not found: 
org.apache.hadoop#hadoop-mapreduce-client-core;0.20.2
[ivy:retrieve]   local: tried
[ivy:retrieve]
/home/navis/.ivy2/local/org.apache.hadoop/hadoop-mapreduce-client-core/0.20.2/ivys/ivy.xml
[ivy:retrieve]-- artifact 
org.apache.hadoop#hadoop-mapreduce-client-core;0.20.2!hadoop-mapreduce-client-core.jar:
[ivy:retrieve]

[jira] [Updated] (HIVE-2999) Offline build is not working

2012-05-02 Thread Navis (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-2999:


Attachment: HIVE-2999.1.patch.txt

 Offline build is not working
 

 Key: HIVE-2999
 URL: https://issues.apache.org/jira/browse/HIVE-2999
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Affects Versions: 0.10.0
Reporter: Navis
 Attachments: HIVE-2999.1.patch.txt


 It's fine without -Doffline=true option. But with offline option (ant 
 -Doffline=true clean package), it's failing with error message like this.
 {noformat}
 ivy-retrieve:
  [echo] Project: common
 [ivy:retrieve] :: loading settings :: file = 
 /home/navis/apache/oss-hive/ivy/ivysettings.xml
 [ivy:retrieve] 
 [ivy:retrieve] :: problems summary ::
 [ivy:retrieve]  WARNINGS
 [ivy:retrieve]module not found: 
 org.apache.hadoop#hadoop-common;0.20.2
 [ivy:retrieve] local: tried
 [ivy:retrieve]  
 /home/navis/.ivy2/local/org.apache.hadoop/hadoop-common/0.20.2/ivys/ivy.xml
 [ivy:retrieve]  -- artifact 
 org.apache.hadoop#hadoop-common;0.20.2!hadoop-common.jar:
 [ivy:retrieve]  
 /home/navis/.ivy2/local/org.apache.hadoop/hadoop-common/0.20.2/jars/hadoop-common.jar
 [ivy:retrieve] apache-snapshot: tried
 [ivy:retrieve]  
 https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.20.2/hadoop-common-0.20.2.pom
 [ivy:retrieve]  -- artifact 
 org.apache.hadoop#hadoop-common;0.20.2!hadoop-common.jar:
 [ivy:retrieve]  
 https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.20.2/hadoop-common-0.20.2.jar
 [ivy:retrieve] maven2: tried
 [ivy:retrieve]  
 http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.20.2/hadoop-common-0.20.2.pom
 [ivy:retrieve]  -- artifact 
 org.apache.hadoop#hadoop-common;0.20.2!hadoop-common.jar:
 [ivy:retrieve]  
 http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.20.2/hadoop-common-0.20.2.jar
 [ivy:retrieve] datanucleus-repo: tried
 [ivy:retrieve]  -- artifact 
 org.apache.hadoop#hadoop-common;0.20.2!hadoop-common.jar:
 [ivy:retrieve]  
 http://www.datanucleus.org/downloads/maven2/org/apache/hadoop/hadoop-common/0.20.2/hadoop-common-0.20.2.jar
 [ivy:retrieve] hadoop-source: tried
 [ivy:retrieve]  -- artifact 
 org.apache.hadoop#hadoop-common;0.20.2!hadoop-common.jar:
 [ivy:retrieve]  
 http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-common-0.20.2/hadoop-common-0.20.2.jar
 [ivy:retrieve] hadoop-source2: tried
 [ivy:retrieve]  -- artifact 
 org.apache.hadoop#hadoop-common;0.20.2!hadoop-common.jar:
 [ivy:retrieve]  
 http://archive.cloudera.com/hive-deps/hadoop/core/hadoop-common-0.20.2/hadoop-common-0.20.2.jar
 [ivy:retrieve]module not found: 
 org.apache.hadoop#hadoop-auth;0.20.2
 [ivy:retrieve] local: tried
 [ivy:retrieve]  
 /home/navis/.ivy2/local/org.apache.hadoop/hadoop-auth/0.20.2/ivys/ivy.xml
 [ivy:retrieve]  -- artifact 
 org.apache.hadoop#hadoop-auth;0.20.2!hadoop-auth.jar:
 [ivy:retrieve]  
 /home/navis/.ivy2/local/org.apache.hadoop/hadoop-auth/0.20.2/jars/hadoop-auth.jar
 [ivy:retrieve] apache-snapshot: tried
 [ivy:retrieve]  
 https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-auth/0.20.2/hadoop-auth-0.20.2.pom
 [ivy:retrieve]  -- artifact 
 org.apache.hadoop#hadoop-auth;0.20.2!hadoop-auth.jar:
 [ivy:retrieve]  
 https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-auth/0.20.2/hadoop-auth-0.20.2.jar
 [ivy:retrieve] maven2: tried
 [ivy:retrieve]  
 http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-auth/0.20.2/hadoop-auth-0.20.2.pom
 [ivy:retrieve]  -- artifact 
 org.apache.hadoop#hadoop-auth;0.20.2!hadoop-auth.jar:
 [ivy:retrieve]  
 http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-auth/0.20.2/hadoop-auth-0.20.2.jar
 [ivy:retrieve] datanucleus-repo: tried
 [ivy:retrieve]  -- artifact 
 org.apache.hadoop#hadoop-auth;0.20.2!hadoop-auth.jar:
 [ivy:retrieve]  
 http://www.datanucleus.org/downloads/maven2/org/apache/hadoop/hadoop-auth/0.20.2/hadoop-auth-0.20.2.jar
 [ivy:retrieve] hadoop-source: tried
 [ivy:retrieve]  -- artifact 
 org.apache.hadoop#hadoop-auth;0.20.2!hadoop-auth.jar:
 [ivy:retrieve]  
 http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-auth-0.20.2/hadoop-auth-0.20.2.jar
 [ivy:retrieve] hadoop-source2: tried
 [ivy:retrieve]  -- artifact 
 

[jira] [Commented] (HIVE-2996) minor bug in hooks (NPE)

2012-05-02 Thread Kevin Wilfong (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13267057#comment-13267057
 ] 

Kevin Wilfong commented on HIVE-2996:
-

@Carl
Can I commit this patch (our internal environment is severely broken without 
this fix)?

We can move the discussion to HIVE-2985

 minor bug in hooks (NPE)
 

 Key: HIVE-2996
 URL: https://issues.apache.org/jira/browse/HIVE-2996
 Project: Hive
  Issue Type: Bug
Reporter: Namit Jain
 Attachments: HIVE-2996.D2991.1.patch


 NPE in AbstractSmcConfigHook.java

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HIVE-3000) Potential infinite loop / log spew in ZookeeperHiveLockManager

2012-05-02 Thread Paul Yang (JIRA)
Paul Yang created HIVE-3000:
---

 Summary: Potential infinite loop / log spew in 
ZookeeperHiveLockManager
 Key: HIVE-3000
 URL: https://issues.apache.org/jira/browse/HIVE-3000
 Project: Hive
  Issue Type: Bug
  Components: Locking
Affects Versions: 0.9.0
Reporter: Paul Yang


See ZookeeperHiveLockManger.lock()

If Zookeeper is in a bad state, it's possible to get an exception (e.g. 
org.apache.zookeeper.KeeperException$SessionExpiredException) when we call 
lockPrimitive(). There is a bug in the exception handler where the loop does 
not exit because the break in the switch statement gets out the switch, not the 
do..while loop. Because tryNum was not incremented due to the exception, 
lockPrimitive() will be called in an infinite loop, as fast as possible. Since 
the exception is printed for each call, Hive will produce significant log spew.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2998) Making Hive run on Windows Server and Windows Azure environment

2012-05-02 Thread Edward Capriolo (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13267069#comment-13267069
 ] 

Edward Capriolo commented on HIVE-2998:
---

It is more about keeping it working then making it work. Hive has ran on 
cygwin+windows before. Because the Jenkins environment does not run on windows 
systems it can break at any time and no one will know. 

 Making Hive run on Windows Server and Windows Azure environment
 ---

 Key: HIVE-2998
 URL: https://issues.apache.org/jira/browse/HIVE-2998
 Project: Hive
  Issue Type: Improvement
Affects Versions: 0.7.1, 0.8.1
 Environment: Windows Server 2008 R2 and Windows Azure
Reporter: Lengning Liu

 This is the master JIRA for improvements on Hive that target on making Hive 
 running natively on Windows Server and Windows Azure environment. Microsoft 
 has done an initial work to make Hive (release 0.7.1 and 0.8.1) to run on 
 Windows environment natively, and wants to contribute this work back to the 
 community. The patches to sub-tasks of this JIRA are based on this work. The 
 end-to-end HiveQL query tests pass. We are working on investigating failed 
 unit test cases at this moment and hopefully reaching the required bar in the 
 next few months. We are posting the patches now so that we can get early 
 feedback from the community and collaborate with other contributors who are 
 interested in this work in the future. Thanks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2987) SELECTing nulls returns nothing

2012-05-02 Thread Navis (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13267073#comment-13267073
 ] 

Navis commented on HIVE-2987:
-

It seemed not possible to discern empty row from not-existing row in hbase.

 SELECTing nulls returns nothing
 ---

 Key: HIVE-2987
 URL: https://issues.apache.org/jira/browse/HIVE-2987
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.9.0
 Environment: Tested using 0.9.0rc1, hbase 0.92.1, hadoop 0.20.2-cdh3u2
Reporter: Oliver Meyn
Priority: Critical

 Given an hbase table defined as 'test' with a single column family 'a', 
 rowkey of type string, and two rows as follows:
 key:1,a:lat=60.0,a:long=50.0,a:precision=10
 key:2,a:lat=54
 And an hive table created overtop of it as follows:
 CREATE EXTERNAL TABLE hbase_test (
   id STRING,
   latitude STRING,
   longitude STRING,
   precision STRING
 )
 STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
 WITH SERDEPROPERTIES (hbase.columns.mapping = 
 :key#s,a:lat#s,a:long#s,a:precision#s)
 TBLPROPERTIES(
   hbase.table.name = test,
   hbase.table.default.storage.type = binary
 );
 The query SELECT id, precision FROM hbase_test WHERE id = '2' returns no 
 result.  Expected behaviour is to return:
 '2',NULL
 If the query is changed to include a non-null result, eg SELECT id, latitude, 
 precision FROM hbase_test WHERE id = '2' the result is as expected:
 '2','54',NULL

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2917) Add support for various charsets in LazySimpleSerDe

2012-05-02 Thread Kai Zhang (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kai Zhang updated HIVE-2917:


Status: Patch Available  (was: Open)

 Add support for various charsets in LazySimpleSerDe
 ---

 Key: HIVE-2917
 URL: https://issues.apache.org/jira/browse/HIVE-2917
 Project: Hive
  Issue Type: New Feature
  Components: CLI, Serializers/Deserializers
Affects Versions: 0.9.0
Reporter: Kai Zhang
 Attachments: HIVE-2917.1.patch.txt, HIVE-2917.2.patch.txt, 
 HIVE-2917.3.patch.txt, HIVE-2917.D2619.1.patch


 Currently hive can only serialize/deserialize data encoded in utf-8.
 It would be useful to specify the data's charset when creating the table.
 The idea is to add a new keyword CHARSET to set charset at table level.
 For example:
 CREATE TABLE tbl1 (col1 STRING) ROW FORMAT CHARET GBK DELIMITED FIELDS 
 TERMINATED BY '\t';
 Another place to use CHARSET is in TRANSFORM clause.
 For example:
 SELECT TRANSFORM(col1, col2) ROW FORMAT CHARSET 'gbk'
 USING 'some_script'
 AS (col3, col4) ROW FORMAT CHARSET 'utf-8';

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs

2012-05-02 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13267201#comment-13267201
 ] 

Hudson commented on HIVE-2646:
--

Integrated in Hive-0.9.1-SNAPSHOT-h0.21 #3 (See 
[https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/3/])
HIVE-2646 Hive Ivy dependencies on Hadoop should depend on jars directly, 
not tarballs. Merged in from trunk. (Andrew Bayer and Thomas Weise) (Revision 
1333094)

 Result = ABORTED
hashutosh : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1333094
Files : 
* /hive/branches/branch-0.9
* /hive/branches/branch-0.9/build-common.xml
* /hive/branches/branch-0.9/build.properties
* /hive/branches/branch-0.9/build.xml
* /hive/branches/branch-0.9/builtins/build.xml
* /hive/branches/branch-0.9/builtins/ivy.xml
* /hive/branches/branch-0.9/cli/ivy.xml
* /hive/branches/branch-0.9/common/ivy.xml
* /hive/branches/branch-0.9/contrib/build.xml
* /hive/branches/branch-0.9/contrib/ivy.xml
* /hive/branches/branch-0.9/hbase-handler/build.xml
* /hive/branches/branch-0.9/hbase-handler/ivy.xml
* /hive/branches/branch-0.9/hwi/build.xml
* /hive/branches/branch-0.9/hwi/ivy.xml
* /hive/branches/branch-0.9/ivy/common-configurations.xml
* /hive/branches/branch-0.9/ivy/ivysettings.xml
* /hive/branches/branch-0.9/ivy/libraries.properties
* /hive/branches/branch-0.9/jdbc/build.xml
* /hive/branches/branch-0.9/jdbc/ivy.xml
* /hive/branches/branch-0.9/metastore/ivy.xml
* /hive/branches/branch-0.9/pdk/build.xml
* /hive/branches/branch-0.9/pdk/ivy.xml
* /hive/branches/branch-0.9/pdk/scripts/build-plugin.xml
* /hive/branches/branch-0.9/ql/build.xml
* /hive/branches/branch-0.9/ql/ivy.xml
* /hive/branches/branch-0.9/serde/ivy.xml
* /hive/branches/branch-0.9/service/build.xml
* /hive/branches/branch-0.9/service/ivy.xml
* /hive/branches/branch-0.9/shims/build.xml
* /hive/branches/branch-0.9/shims/ivy.xml
* /hive/branches/branch-0.9/testutils/hadoop


 Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
 

 Key: HIVE-2646
 URL: https://issues.apache.org/jira/browse/HIVE-2646
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Affects Versions: 0.8.0
Reporter: Andrew Bayer
Assignee: Andrew Bayer
Priority: Critical
 Fix For: 0.10.0, 0.9.1

 Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, 
 HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, 
 HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, 
 HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, 
 HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, 
 HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, 
 HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, 
 HIVE-2646.diff.txt


 The current Hive Ivy dependency logic for its Hadoop dependencies is 
 problematic - depending on the tarball and extracting the jars from there, 
 rather than depending on the jars directly. It'd be great if this was fixed 
 to actually have the jar dependencies defined directly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2976) Fix maven-build Ant target

2012-05-02 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13267202#comment-13267202
 ] 

Hudson commented on HIVE-2976:
--

Integrated in Hive-0.9.1-SNAPSHOT-h0.21 #3 (See 
[https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/3/])
HIVE-2976 Fix maven-build Ant target  Merged in from trunk. (Andrew Bayer 
and Thomas Weise) (Revision 1333095)

 Result = ABORTED
hashutosh : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1333095
Files : 
* /hive/branches/branch-0.9
* /hive/branches/branch-0.9/build.xml
* /hive/branches/branch-0.9/builtins/build.xml


 Fix maven-build Ant target
 --

 Key: HIVE-2976
 URL: https://issues.apache.org/jira/browse/HIVE-2976
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Reporter: Carl Steinbach
Assignee: Thomas Weise
 Fix For: 0.10.0, 0.9.1

 Attachments: HIVE-2976-maven.diff.txt, HIVE-2976.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira