Hive-trunk-h0.21 - Build # 1365 - Failure

2012-04-10 Thread Apache Jenkins Server 
Changes for Build #1365
[hashutosh] Upping the version number from 0.9 to 0.10 on trunk.




1 tests failed.
REGRESSION:  
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1

Error Message:
Unexpected exception See build/ql/tmp/hive.log, or try ant test ... 
-Dtest.silent=false to get more logs.

Stack Trace:
junit.framework.AssertionFailedError: Unexpected exception
See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get 
more logs.
at junit.framework.Assert.fail(Assert.java:47)
at 
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1(TestNegativeCliDriver.java:10418)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:154)
at junit.framework.TestCase.runBare(TestCase.java:127)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)




The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1365)

Status: Failure

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1365/ to 
view the results.

[jira] [Commented] (HIVE-2826) ALTER VIEW ADD PARTITION needs IF NOT EXISTS.

2012-04-10 Thread Adam Kramer (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250498#comment-13250498
 ] 

Adam Kramer commented on HIVE-2826:
---

Ugh, sorry. The weird difference in syntax is what made me think this had not 
been implemented.

 ALTER VIEW ADD PARTITION needs IF NOT EXISTS.
 -

 Key: HIVE-2826
 URL: https://issues.apache.org/jira/browse/HIVE-2826
 Project: Hive
  Issue Type: Improvement
Reporter: Adam Kramer

 Support for ALTER TABLE x ADD IF NOT EXISTS PARTITION was added in HIVE-1106. 
 The same semantics should be supported for ALTER VIEW.
 WAS:
 When trying to add a partition, if the partition already exists, we shouldn't 
 necessarily cause the job to fail...just like we have CREATE TABLE IF NOT 
 EXISTS and DROP TABLE IF EXISTS, we should have ALTER TABLE foo ADD PARTITION 
 IF NOT EXISTS (ds=whatever).
 Ditto for ALTER VIEW.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2826) ALTER VIEW ADD PARTITION needs IF NOT EXISTS.

2012-04-10 Thread Adam Kramer (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250499#comment-13250499
 ] 

Adam Kramer commented on HIVE-2826:
---

Also, it is implemented for views, too.

 ALTER VIEW ADD PARTITION needs IF NOT EXISTS.
 -

 Key: HIVE-2826
 URL: https://issues.apache.org/jira/browse/HIVE-2826
 Project: Hive
  Issue Type: Improvement
Reporter: Adam Kramer

 Support for ALTER TABLE x ADD IF NOT EXISTS PARTITION was added in HIVE-1106. 
 The same semantics should be supported for ALTER VIEW.
 WAS:
 When trying to add a partition, if the partition already exists, we shouldn't 
 necessarily cause the job to fail...just like we have CREATE TABLE IF NOT 
 EXISTS and DROP TABLE IF EXISTS, we should have ALTER TABLE foo ADD PARTITION 
 IF NOT EXISTS (ds=whatever).
 Ditto for ALTER VIEW.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2826) ALTER VIEW ADD (IF NOT EXISTS) PARTITION syntax is bad.

2012-04-10 Thread Adam Kramer (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Kramer updated HIVE-2826:
--

Description: 
The syntax, ALTER TABLE x ADD IF NOT EXISTS PARTITION, puts the IF NOT EXISTS 
before the subject of the statement. Not only is this weird grammar, but it is 
also inconsistent with other uses of IF NOT EXISTS, for example, CREATE 
TABLE IF NOT EXISTS and DROP TABLE IF EXISTS.

It would be a minor improvement to align the syntaxes.


  was:
Support for ALTER TABLE x ADD IF NOT EXISTS PARTITION was added in HIVE-1106. 
The same semantics should be supported for ALTER VIEW.

WAS:
When trying to add a partition, if the partition already exists, we shouldn't 
necessarily cause the job to fail...just like we have CREATE TABLE IF NOT 
EXISTS and DROP TABLE IF EXISTS, we should have ALTER TABLE foo ADD PARTITION 
IF NOT EXISTS (ds=whatever).

Ditto for ALTER VIEW.

Summary: ALTER VIEW ADD (IF NOT EXISTS) PARTITION syntax is bad.  (was: 
ALTER VIEW ADD PARTITION needs IF NOT EXISTS.)

 ALTER VIEW ADD (IF NOT EXISTS) PARTITION syntax is bad.
 ---

 Key: HIVE-2826
 URL: https://issues.apache.org/jira/browse/HIVE-2826
 Project: Hive
  Issue Type: Improvement
Reporter: Adam Kramer

 The syntax, ALTER TABLE x ADD IF NOT EXISTS PARTITION, puts the IF NOT 
 EXISTS before the subject of the statement. Not only is this weird grammar, 
 but it is also inconsistent with other uses of IF NOT EXISTS, for example, 
 CREATE TABLE IF NOT EXISTS and DROP TABLE IF EXISTS.
 It would be a minor improvement to align the syntaxes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2826) ALTER VIEW ADD (IF NOT EXISTS) PARTITION syntax is bad.

2012-04-10 Thread Adam Kramer (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Kramer updated HIVE-2826:
--

Priority: Minor  (was: Major)

 ALTER VIEW ADD (IF NOT EXISTS) PARTITION syntax is bad.
 ---

 Key: HIVE-2826
 URL: https://issues.apache.org/jira/browse/HIVE-2826
 Project: Hive
  Issue Type: Improvement
Reporter: Adam Kramer
Priority: Minor

 The syntax, ALTER TABLE x ADD IF NOT EXISTS PARTITION, puts the IF NOT 
 EXISTS before the subject of the statement. Not only is this weird grammar, 
 but it is also inconsistent with other uses of IF NOT EXISTS, for example, 
 CREATE TABLE IF NOT EXISTS and DROP TABLE IF EXISTS.
 It would be a minor improvement to align the syntaxes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2940) RCFileMergeMapper Prints To Standard Output Even In Silent Mode

2012-04-10 Thread Thiruvel Thirumoolan (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-2940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thiruvel Thirumoolan updated HIVE-2940:
---

Attachment: HIVE-2940.patch

 RCFileMergeMapper Prints To Standard Output Even In Silent Mode
 ---

 Key: HIVE-2940
 URL: https://issues.apache.org/jira/browse/HIVE-2940
 Project: Hive
  Issue Type: Task
Affects Versions: 0.8.1, 0.9.0
Reporter: Thiruvel Thirumoolan
Assignee: Thiruvel Thirumoolan
Priority: Trivial
 Fix For: 0.10

 Attachments: HIVE-2940.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2940) RCFileMergeMapper Prints To Standard Output Even In Silent Mode

2012-04-10 Thread Thiruvel Thirumoolan (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-2940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thiruvel Thirumoolan updated HIVE-2940:
---

Description: A very minor issue. One of our users was running partition 
concatenate in local mode as part of their application and this was a little 
noisy.

 RCFileMergeMapper Prints To Standard Output Even In Silent Mode
 ---

 Key: HIVE-2940
 URL: https://issues.apache.org/jira/browse/HIVE-2940
 Project: Hive
  Issue Type: Task
Affects Versions: 0.8.1, 0.9.0
Reporter: Thiruvel Thirumoolan
Assignee: Thiruvel Thirumoolan
Priority: Trivial
 Fix For: 0.10

 Attachments: HIVE-2940.patch


 A very minor issue. One of our users was running partition concatenate in 
 local mode as part of their application and this was a little noisy.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2940) RCFileMergeMapper Prints To Standard Output Even In Silent Mode

2012-04-10 Thread Thiruvel Thirumoolan (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250536#comment-13250536
 ] 

Thiruvel Thirumoolan commented on HIVE-2940:


Patch uploaded to https://reviews.facebook.net/D2715

 RCFileMergeMapper Prints To Standard Output Even In Silent Mode
 ---

 Key: HIVE-2940
 URL: https://issues.apache.org/jira/browse/HIVE-2940
 Project: Hive
  Issue Type: Task
Affects Versions: 0.8.1, 0.9.0
Reporter: Thiruvel Thirumoolan
Assignee: Thiruvel Thirumoolan
Priority: Trivial
 Fix For: 0.10

 Attachments: HIVE-2940.patch


 A very minor issue. One of our users was running partition concatenate in 
 local mode as part of their application and this was a little noisy.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2940) RCFileMergeMapper Prints To Standard Output Even In Silent Mode

2012-04-10 Thread Thiruvel Thirumoolan (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-2940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thiruvel Thirumoolan updated HIVE-2940:
---

Status: Patch Available  (was: Open)

 RCFileMergeMapper Prints To Standard Output Even In Silent Mode
 ---

 Key: HIVE-2940
 URL: https://issues.apache.org/jira/browse/HIVE-2940
 Project: Hive
  Issue Type: Task
Affects Versions: 0.8.1, 0.9.0
Reporter: Thiruvel Thirumoolan
Assignee: Thiruvel Thirumoolan
Priority: Trivial
 Fix For: 0.10

 Attachments: HIVE-2940.patch


 A very minor issue. One of our users was running partition concatenate in 
 local mode as part of their application and this was a little noisy.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2910) Improve the HWI interface

2012-04-10 Thread Hugo Trippaers (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hugo Trippaers updated HIVE-2910:
-

Attachment: glyphicons-halflings-white.png
glyphicons-halflings.png
hive-2910.4.patch.txt

This patch included the required changes to the NOTICE file for inclusion of 
both Bootstrap and Glyphicons. The images are included separately and should be 
placed in hwi/src/web/img

 Improve the HWI interface
 -

 Key: HIVE-2910
 URL: https://issues.apache.org/jira/browse/HIVE-2910
 Project: Hive
  Issue Type: Improvement
  Components: Web UI
Reporter: Hugo Trippaers
Assignee: Hugo Trippaers
Priority: Minor
  Labels: newbie, patch
 Attachments: glyphicons-halflings-white.png, 
 glyphicons-halflings.png, hive-2910.3.patch.log, hive-2910.3.patch.txt, 
 hive-2910.4.patch.txt, hive-hwi-2.patch, hive-hwi.patch, screenie001.PNG, 
 screenie002.PNG


 I've made some improvements to the HWI interface with the Twitter bootstrap 
 system. I'm looking for feedback on the new design.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2910) Improve the HWI interface

2012-04-10 Thread Hugo Trippaers (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250574#comment-13250574
 ] 

Hugo Trippaers commented on HIVE-2910:
--

Owen,  the bootstrap.min.css is not modified by me. It's generated by the 
webform at the bootstrap site on github.  I've updated the notice file and 
added a new patch with these changes.

 Improve the HWI interface
 -

 Key: HIVE-2910
 URL: https://issues.apache.org/jira/browse/HIVE-2910
 Project: Hive
  Issue Type: Improvement
  Components: Web UI
Reporter: Hugo Trippaers
Assignee: Hugo Trippaers
Priority: Minor
  Labels: newbie, patch
 Attachments: glyphicons-halflings-white.png, 
 glyphicons-halflings.png, hive-2910.3.patch.log, hive-2910.3.patch.txt, 
 hive-2910.4.patch.txt, hive-hwi-2.patch, hive-hwi.patch, screenie001.PNG, 
 screenie002.PNG


 I've made some improvements to the HWI interface with the Twitter bootstrap 
 system. I'm looking for feedback on the new design.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2767) Optionally use framed transport with metastore

2012-04-10 Thread Travis Crawford (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250707#comment-13250707
 ] 

Travis Crawford commented on HIVE-2767:
---

@ashutosh - Yeah this test does seem problematic. It works fine in IntelliJ but 
I can't get it to pass with the command you gave, even when doing a clean trunk 
build. Looking at Jenkins the test works fine, so perhaps its something to do 
with my machine (osx 10.7.3).

I'm looking into why the test doesn't work on trunk, then will see if this 
change affects it.

 Optionally use framed transport with metastore
 --

 Key: HIVE-2767
 URL: https://issues.apache.org/jira/browse/HIVE-2767
 Project: Hive
  Issue Type: New Feature
  Components: Metastore
Reporter: Travis Crawford
Assignee: Travis Crawford
 Attachments: HIVE-2767.D2661.1.patch, HIVE-2767.D2661.2.patch, 
 HIVE-2767.D2661.3.patch, HIVE-2767.patch.txt, HIVE-2767_a.patch.txt


 Users may want/need to use thrift's framed transport when communicating with 
 the Hive MetaStore. This patch adds a new property 
 {{hive.metastore.thrift.framed.transport.enabled}} that enables the framed 
 transport (defaults to off, aka no change from before the patch). This 
 property must be set for both clients and the HMS server.
 It wasn't immediately clear how to use the framed transport with SASL, so as 
 written an exception is thrown if you try starting the server with both 
 options. If SASL and the framed transport will indeed work together I can 
 update the patch (although I don't have a secured environment to test in).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2767) Optionally use framed transport with metastore

2012-04-10 Thread Ashutosh Chauhan (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250711#comment-13250711
 ] 

Ashutosh Chauhan commented on HIVE-2767:


@Travis,
It has nothing to do with your patch. Problem exists on trunk. See, HIVE-2937 
And ya, its a race condition, so it doesn't show up always.

 Optionally use framed transport with metastore
 --

 Key: HIVE-2767
 URL: https://issues.apache.org/jira/browse/HIVE-2767
 Project: Hive
  Issue Type: New Feature
  Components: Metastore
Reporter: Travis Crawford
Assignee: Travis Crawford
 Attachments: HIVE-2767.D2661.1.patch, HIVE-2767.D2661.2.patch, 
 HIVE-2767.D2661.3.patch, HIVE-2767.patch.txt, HIVE-2767_a.patch.txt


 Users may want/need to use thrift's framed transport when communicating with 
 the Hive MetaStore. This patch adds a new property 
 {{hive.metastore.thrift.framed.transport.enabled}} that enables the framed 
 transport (defaults to off, aka no change from before the patch). This 
 property must be set for both clients and the HMS server.
 It wasn't immediately clear how to use the framed transport with SASL, so as 
 written an exception is thrown if you try starting the server with both 
 options. If SASL and the framed transport will indeed work together I can 
 update the patch (although I don't have a secured environment to test in).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2767) Optionally use framed transport with metastore

2012-04-10 Thread Travis Crawford (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250718#comment-13250718
 ] 

Travis Crawford commented on HIVE-2767:
---

Cool - thanks for the pointer! I'll watch that issue and afterwards rebase if 
necessary and update.

 Optionally use framed transport with metastore
 --

 Key: HIVE-2767
 URL: https://issues.apache.org/jira/browse/HIVE-2767
 Project: Hive
  Issue Type: New Feature
  Components: Metastore
Reporter: Travis Crawford
Assignee: Travis Crawford
 Attachments: HIVE-2767.D2661.1.patch, HIVE-2767.D2661.2.patch, 
 HIVE-2767.D2661.3.patch, HIVE-2767.patch.txt, HIVE-2767_a.patch.txt


 Users may want/need to use thrift's framed transport when communicating with 
 the Hive MetaStore. This patch adds a new property 
 {{hive.metastore.thrift.framed.transport.enabled}} that enables the framed 
 transport (defaults to off, aka no change from before the patch). This 
 property must be set for both clients and the HMS server.
 It wasn't immediately clear how to use the framed transport with SASL, so as 
 written an exception is thrown if you try starting the server with both 
 options. If SASL and the framed transport will indeed work together I can 
 update the patch (although I don't have a secured environment to test in).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2928) Support for Oracle-backed Hive-Metastore (longvarchar to clob in package.jdo)

2012-04-10 Thread Mithun Radhakrishnan (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250786#comment-13250786
 ] 

Mithun Radhakrishnan commented on HIVE-2928:


@Carl: Thank you for the pointer.

Would your suggestion be that changing the datatype in Hive's pacakge.jdo won't 
be acceptable because there might be deployments with MySQL with JDO 
column-validations turned on?

 Support for Oracle-backed Hive-Metastore (longvarchar to clob in 
 package.jdo)
 -

 Key: HIVE-2928
 URL: https://issues.apache.org/jira/browse/HIVE-2928
 Project: Hive
  Issue Type: New Feature
  Components: Metastore
Affects Versions: 0.9.0
Reporter: Mithun Radhakrishnan
 Attachments: HIVE-2928.patch


 I'm trying to get the Hive-Metastore to work when backed by an Oracle 
 backend. There's a change to hive's package.jdo that I'd like advice/comments 
 on.
 One sticking point on working with Oracle has been the TBLS table (MTable) 
 and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). 
 Oracle doesn't support more than one LONGVARCHAR property per table (for 
 reason of legacy), and prefers that one use CLOBs instead. If one switches to 
 CLOB properties, with no modification to hive's package.jdo, one sees the 
 following exception:
 quote
 Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
 (datastore), but type expected was LONGVARCHAR (metadata). Please check that
 the type in the datastore and the type specified in the MetaData are
 consistent.
 org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
 Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
 (datastore), but type expected was LONGVARCHAR (metadata). Please check that
 the type in the datastore and the type specified in the MetaData are
 consistent.
 at
 org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
 at
 org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
 /quote
 But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of 
 LONGVARCHARs, things look promising:
 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet 
 to scale-test.)
 2. These modified hive-libraries work as is with pre-existing mysql 
 metastores. Migrating data isn't a worry.
 3. The unit-tests seem to run through. 
 Would there be opposition to changing the package.jdo's LONGVARCHAR 
 references to CLOB, if this works with mysql and with Oracle? 
 Mithun
 P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm 
 testing, for the related issue of creating the required tables in Oracle.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2907) Hive error when dropping a table with large number of partitions

2012-04-10 Thread Kevin Wilfong (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250791#comment-13250791
 ] 

Kevin Wilfong commented on HIVE-2907:
-

+1 tests passed, will commit.

 Hive error when dropping a table with large number of partitions
 

 Key: HIVE-2907
 URL: https://issues.apache.org/jira/browse/HIVE-2907
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.9.0
 Environment: General. Hive Metastore bug.
Reporter: Mousom Dhar Gupta
Assignee: Mousom Dhar Gupta
Priority: Minor
 Fix For: 0.9.0

 Attachments: HIVE-2907.1.patch.txt, HIVE-2907.2.patch.txt, 
 HIVE-2907.3.patch.txt, HIVE-2907.D2505.1.patch, HIVE-2907.D2505.2.patch, 
 HIVE-2907.D2505.3.patch, HIVE-2907.D2505.4.patch, HIVE-2907.D2505.5.patch, 
 HIVE-2907.D2505.6.patch, HIVE-2907.D2505.7.patch

   Original Estimate: 10h
  Remaining Estimate: 10h

 Running into an Out Of Memory error when trying to drop a table with 128K 
 partitions.
 The methods dropTable in 
 metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
 and dropTable in ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
 encounter out of memory errors 
 when dropping tables with lots of partitions because they try to load the 
 metadata for every partition into memory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HIVE-2907) Hive error when dropping a table with large number of partitions

2012-04-10 Thread Kevin Wilfong (Resolved) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong resolved HIVE-2907.
-

Resolution: Fixed

Committed.  Thanks Mousom.

 Hive error when dropping a table with large number of partitions
 

 Key: HIVE-2907
 URL: https://issues.apache.org/jira/browse/HIVE-2907
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.9.0
 Environment: General. Hive Metastore bug.
Reporter: Mousom Dhar Gupta
Assignee: Mousom Dhar Gupta
Priority: Minor
 Fix For: 0.9.0

 Attachments: HIVE-2907.1.patch.txt, HIVE-2907.2.patch.txt, 
 HIVE-2907.3.patch.txt, HIVE-2907.D2505.1.patch, HIVE-2907.D2505.2.patch, 
 HIVE-2907.D2505.3.patch, HIVE-2907.D2505.4.patch, HIVE-2907.D2505.5.patch, 
 HIVE-2907.D2505.6.patch, HIVE-2907.D2505.7.patch

   Original Estimate: 10h
  Remaining Estimate: 10h

 Running into an Out Of Memory error when trying to drop a table with 128K 
 partitions.
 The methods dropTable in 
 metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
 and dropTable in ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
 encounter out of memory errors 
 when dropping tables with lots of partitions because they try to load the 
 metadata for every partition into memory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2530) Implement SHOW TBLPROPERTIES

2012-04-10 Thread Phabricator (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250840#comment-13250840
 ] 

Phabricator commented on HIVE-2530:
---

kevinwilfong has accepted the revision HIVE-2530 [jira] Implement SHOW 
TBLPROPERTIES.

  +1 will commit if tests pass.

REVISION DETAIL
  https://reviews.facebook.net/D2589

BRANCH
  svn


 Implement SHOW TBLPROPERTIES
 

 Key: HIVE-2530
 URL: https://issues.apache.org/jira/browse/HIVE-2530
 Project: Hive
  Issue Type: New Feature
Reporter: Adam Kramer
Assignee: Lei Zhao
Priority: Minor
 Attachments: HIVE-2530.D2589.1.patch, HIVE-2530.D2589.2.patch, 
 HIVE-2530.D2589.3.patch


 Since table properties can be defined arbitrarily, they should be easy for a 
 user to query from the command-line.
 SHOW TBLPROPERTIES tblname;
 ...would show all of them, one per row, key \t value
 SHOW TBLPROPERTIES tblname (FOOBAR);
 ...would just show the value for the FOOBAR tblproperty.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2530) Implement SHOW TBLPROPERTIES

2012-04-10 Thread Kevin Wilfong (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250843#comment-13250843
 ] 

Kevin Wilfong commented on HIVE-2530:
-

Ashutosh: I agree that the information can already be viewed via a describe 
formatted command, however, for tables with a large number of properties, or 
even a large number of columns or large column comments, it becomes difficult 
for users to quickly find the property they're looking for.  This will help 
provide a clean view and ease readability.

 Implement SHOW TBLPROPERTIES
 

 Key: HIVE-2530
 URL: https://issues.apache.org/jira/browse/HIVE-2530
 Project: Hive
  Issue Type: New Feature
Reporter: Adam Kramer
Assignee: Lei Zhao
Priority: Minor
 Attachments: HIVE-2530.D2589.1.patch, HIVE-2530.D2589.2.patch, 
 HIVE-2530.D2589.3.patch


 Since table properties can be defined arbitrarily, they should be easy for a 
 user to query from the command-line.
 SHOW TBLPROPERTIES tblname;
 ...would show all of them, one per row, key \t value
 SHOW TBLPROPERTIES tblname (FOOBAR);
 ...would just show the value for the FOOBAR tblproperty.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2883) Metastore client doesnt close connection properly

2012-04-10 Thread Travis Crawford (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250953#comment-13250953
 ] 

Travis Crawford commented on HIVE-2883:
---

I patched this in and ran a pig query through HCatalog, and the Unable to 
shutdown local metastore client error went away, and the query produced a 
correct result. LGTM.

 Metastore client doesnt close connection properly
 -

 Key: HIVE-2883
 URL: https://issues.apache.org/jira/browse/HIVE-2883
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.9.0
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
 Fix For: 0.9.0

 Attachments: HIVE-2883.D2613.1.patch


 While closing connection, it always fail with following trace. Seemingly, it 
 doesnt have any harmful effects.
 {code}
 12/03/20 10:55:02 ERROR hive.metastore: Unable to shutdown local metastore 
 client
 org.apache.thrift.transport.TTransportException: Cannot write to null 
 outputStream
   at 
 org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:142)
   at 
 org.apache.thrift.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.java:163)
   at 
 org.apache.thrift.protocol.TBinaryProtocol.writeMessageBegin(TBinaryProtocol.java:91)
   at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
   at 
 com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:421)
   at 
 com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:415)
   at 
 org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:310)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2928) Support for Oracle-backed Hive-Metastore (longvarchar to clob in package.jdo)

2012-04-10 Thread Carl Steinbach (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250981#comment-13250981
 ] 

Carl Steinbach commented on HIVE-2928:
--

We currently have JDO schema validation disabled by default since it affects 
performance. However, if modifying Hive's package.jdo to work with Oracle means 
that we're forfeiting this feature, I'd at least like to know that up front.

Also, there was an exchange a couple years ago on the hive-user list where John 
made an interesting suggestion:

http://mail-archives.apache.org/mod_mbox/hadoop-hive-user/201006.mbox/%3cbd1fe08f-5edf-4d90-a741-8b703cd06...@facebook.com%3E

bq. Another option is to precreate your schema in Oracle and then tell JDO not 
to try to create/update
it automatically.

Would you mind trying this out to see if it works? If it does then I think that 
might be the optimal solution for now.


 Support for Oracle-backed Hive-Metastore (longvarchar to clob in 
 package.jdo)
 -

 Key: HIVE-2928
 URL: https://issues.apache.org/jira/browse/HIVE-2928
 Project: Hive
  Issue Type: New Feature
  Components: Metastore
Affects Versions: 0.9.0
Reporter: Mithun Radhakrishnan
 Attachments: HIVE-2928.patch


 I'm trying to get the Hive-Metastore to work when backed by an Oracle 
 backend. There's a change to hive's package.jdo that I'd like advice/comments 
 on.
 One sticking point on working with Oracle has been the TBLS table (MTable) 
 and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). 
 Oracle doesn't support more than one LONGVARCHAR property per table (for 
 reason of legacy), and prefers that one use CLOBs instead. If one switches to 
 CLOB properties, with no modification to hive's package.jdo, one sees the 
 following exception:
 quote
 Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
 (datastore), but type expected was LONGVARCHAR (metadata). Please check that
 the type in the datastore and the type specified in the MetaData are
 consistent.
 org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
 Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
 (datastore), but type expected was LONGVARCHAR (metadata). Please check that
 the type in the datastore and the type specified in the MetaData are
 consistent.
 at
 org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
 at
 org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
 /quote
 But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of 
 LONGVARCHARs, things look promising:
 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet 
 to scale-test.)
 2. These modified hive-libraries work as is with pre-existing mysql 
 metastores. Migrating data isn't a worry.
 3. The unit-tests seem to run through. 
 Would there be opposition to changing the package.jdo's LONGVARCHAR 
 references to CLOB, if this works with mysql and with Oracle? 
 Mithun
 P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm 
 testing, for the related issue of creating the required tables in Oracle.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-0.8.1-SNAPSHOT-h0.21 - Build # 249 - Still Failing

2012-04-10 Thread Apache Jenkins Server 
Changes for Build #247

Changes for Build #248

Changes for Build #249



3 tests failed.
REGRESSION:  
org.apache.hadoop.hive.cli.TestHBaseMinimrCliDriver.testCliDriver_hbase_bulk

Error Message:
Unexpected exception

Stack Trace:
junit.framework.AssertionFailedError: Unexpected exception
at junit.framework.Assert.fail(Assert.java:50)
at 
org.apache.hadoop.hive.cli.TestHBaseMinimrCliDriver.testCliDriver_hbase_bulk(TestHBaseMinimrCliDriver.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.extensions.TestSetup.run(TestSetup.java:27)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)


REGRESSION:  
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_add_part_exist

Error Message:
Unexpected exception in setup

Stack Trace:
junit.framework.AssertionFailedError: Unexpected exception in setup
at junit.framework.Assert.fail(Assert.java:50)
at org.apache.hadoop.hive.cli.TestCliDriver.setUp(TestCliDriver.java:60)
at junit.framework.TestCase.runBare(TestCase.java:132)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)


REGRESSION:  org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter1

Error Message:
Unexpected exception in setup

Stack Trace:
junit.framework.AssertionFailedError: Unexpected exception in setup
at junit.framework.Assert.fail(Assert.java:50)
at org.apache.hadoop.hive.cli.TestCliDriver.setUp(TestCliDriver.java:60)
at junit.framework.TestCase.runBare(TestCase.java:132)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)




The Apache Jenkins build system has built Hive-0.8.1-SNAPSHOT-h0.21 (build #249)

Status: Still Failing

Check console output at 
https://builds.apache.org/job/Hive-0.8.1-SNAPSHOT-h0.21/249/ to view the 
results.

[jira] [Created] (HIVE-2942) substr on string containing UTF-8 characters produces StringIndexOutOfBoundsException

2012-04-10 Thread Kevin Wilfong (Created) (JIRA) 
substr on string containing UTF-8 characters produces 
StringIndexOutOfBoundsException
-

 Key: HIVE-2942
 URL: https://issues.apache.org/jira/browse/HIVE-2942
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong


After HIVE-2792, the substr function produces a StringIndexOutOfBoundsException 
when called on a string containing UTF-8 characters without the length argument 
being present.

E.g.
select substr(str, 1) from table1;

now fails with that exception if str contains a UTF-8 character for any row in 
the table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1366 - Still Failing

2012-04-10 Thread Apache Jenkins Server 
Changes for Build #1365
[hashutosh] Upping the version number from 0.9 to 0.10 on trunk.


Changes for Build #1366



1 tests failed.
FAILED:  
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1

Error Message:
Unexpected exception See build/ql/tmp/hive.log, or try ant test ... 
-Dtest.silent=false to get more logs.

Stack Trace:
junit.framework.AssertionFailedError: Unexpected exception
See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get 
more logs.
at junit.framework.Assert.fail(Assert.java:47)
at 
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1(TestNegativeCliDriver.java:10418)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:154)
at junit.framework.TestCase.runBare(TestCase.java:127)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)




The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1366)

Status: Still Failing

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1366/ to 
view the results.

[jira] [Updated] (HIVE-2942) substr on string containing UTF-8 characters produces StringIndexOutOfBoundsException

2012-04-10 Thread Phabricator (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-2942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-2942:
--

Attachment: HIVE-2942.D2727.1.patch

kevinwilfong requested code review of HIVE-2942 [jira] substr on string 
containing UTF-8 characters produces StringIndexOutOfBoundsException.
Reviewers: JIRA

  https://issues.apache.org/jira/browse/HIVE-2942

  Fixed UDFSubstr so that for strings, the substr now succeeds if there is a 
UTF-8 character by using the string length instead of the Text length.

  Also, updated QTestUtil so that we can now write tests which include UTF-8 
characters.

  After HIVE-2792, the substr function produces a 
StringIndexOutOfBoundsException when called on a string containing UTF-8 
characters without the length argument being present.

  E.g.
  select substr(str, 1) from table1;

  now fails with that exception if str contains a UTF-8 character for any row 
in the table.

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D2727

AFFECTED FILES
  ql/src/test/results/clientpositive/udf_substr.q.out
  ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java
  ql/src/test/queries/clientpositive/udf_substr.q
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSubstr.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/6219/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


 substr on string containing UTF-8 characters produces 
 StringIndexOutOfBoundsException
 -

 Key: HIVE-2942
 URL: https://issues.apache.org/jira/browse/HIVE-2942
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2942.D2727.1.patch


 After HIVE-2792, the substr function produces a 
 StringIndexOutOfBoundsException when called on a string containing UTF-8 
 characters without the length argument being present.
 E.g.
 select substr(str, 1) from table1;
 now fails with that exception if str contains a UTF-8 character for any row 
 in the table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2084) Upgrade datanucleus from 2.0.3 to 3.0.1

2012-04-10 Thread Sushanth Sowmyan (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251051#comment-13251051
 ] 

Sushanth Sowmyan commented on HIVE-2084:


@Ashutosh : As per above comment of mine, ignore the one directly on jira - 
that was generated from hcatalog with a bunch of other unnecessary directory 
structures associated with it ( hive/external/ ). Please check the one on 
phabricator.

 Upgrade datanucleus from 2.0.3 to 3.0.1
 ---

 Key: HIVE-2084
 URL: https://issues.apache.org/jira/browse/HIVE-2084
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Reporter: Ning Zhang
Assignee: Sushanth Sowmyan
  Labels: datanucleus
 Attachments: HIVE-2084.1.patch.txt, HIVE-2084.2.patch.txt, 
 HIVE-2084.D2397.1.patch, HIVE-2084.patch


 It seems the datanucleus 2.2.3 does a better join in caching. The time it 
 takes to get the same set of partition objects takes about 1/4 of the time it 
 took for the first time. While with 2.0.3, it took almost the same amount of 
 time in the second execution. We should retest the test case mentioned in 
 HIVE-1853, HIVE-1862.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-2943) Escaping quote in partition value encodes both backslash and quote in the partition name

2012-04-10 Thread Alan Gates (Created) (JIRA) 
Escaping quote in partition value encodes both backslash and quote in the 
partition name


 Key: HIVE-2943
 URL: https://issues.apache.org/jira/browse/HIVE-2943
 Project: Hive
  Issue Type: Bug
  Components: SQL
Affects Versions: 0.9.0
Reporter: Alan Gates


{code}
create table foo (a string, c int) partitioned by (b string) stored as textfile;
alter table foo add partition (b = 'abc\'def') location '/bla';
show partitions foo;
OK
b=abc%5C%27def
{code}

The \ has been encoded in as %5C in addition to the single quote (%27).  The 
parser should remove the backslash in this context, since it is here to escape 
the single quote.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2943) Escaping quote in partition value encodes both backslash and quote in the partition name

2012-04-10 Thread Alan Gates (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-2943:
-

Priority: Minor  (was: Major)

 Escaping quote in partition value encodes both backslash and quote in the 
 partition name
 

 Key: HIVE-2943
 URL: https://issues.apache.org/jira/browse/HIVE-2943
 Project: Hive
  Issue Type: Bug
  Components: SQL
Affects Versions: 0.9.0
Reporter: Alan Gates
Priority: Minor

 {code}
 create table foo (a string, c int) partitioned by (b string) stored as 
 textfile;
 alter table foo add partition (b = 'abc\'def') location '/bla';
 show partitions foo;
 OK
 b=abc%5C%27def
 {code}
 The \ has been encoded in as %5C in addition to the single quote (%27).  The 
 parser should remove the backslash in this context, since it is here to 
 escape the single quote.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs

2012-04-10 Thread Phabricator (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251140#comment-13251140
 ] 

Phabricator commented on HIVE-2646:
---

cwsteinbach has requested changes to the revision HIVE-2646 [jira] Hive Ivy 
dependencies on Hadoop should depend on jars directly, not tarballs.

  Executing the following ant command results in a failure:

  % ant clean package test -Dtestcase=TestNegativeCliDriver -Dqfile=autolocal1.q

  WARNINGS
  [ivy:resolve] module not found: 
org.apache.hive#hive-builtins;0.10.0-SNAPSHOT

INLINE COMMENTS
  ivy/common-configurations.xml:28 Is it possible to make these names 
consistent, e.g. change hadoop0.20.shim to hadoop20.shim?

REVISION DETAIL
  https://reviews.facebook.net/D2133

BRANCH
  HIVE-2646-dev-branch


 Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
 

 Key: HIVE-2646
 URL: https://issues.apache.org/jira/browse/HIVE-2646
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Affects Versions: 0.8.0
Reporter: Andrew Bayer
Assignee: Andrew Bayer
Priority: Critical
 Fix For: 0.9.0

 Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, 
 HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, 
 HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, 
 HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, 
 HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt


 The current Hive Ivy dependency logic for its Hadoop dependencies is 
 problematic - depending on the tarball and extracting the jars from there, 
 rather than depending on the jars directly. It'd be great if this was fixed 
 to actually have the jar dependencies defined directly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs

2012-04-10 Thread Phabricator (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251161#comment-13251161
 ] 

Phabricator commented on HIVE-2646:
---

abayer has commented on the revision HIVE-2646 [jira] Hive Ivy dependencies on 
Hadoop should depend on jars directly, not tarballs.

  'k, think I found the problem with hive-builtins - since its jar is built 
outside of the build-common.xml jar target, it wasn't getting published to the 
local repo. re: the shims configuration names - they've got to be consistent 
with the other strings in shims/build.xml, so unless those can change too (and 
I'm not sure if they can), they need to stay as they are.

REVISION DETAIL
  https://reviews.facebook.net/D2133

BRANCH
  HIVE-2646-dev-branch


 Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
 

 Key: HIVE-2646
 URL: https://issues.apache.org/jira/browse/HIVE-2646
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Affects Versions: 0.8.0
Reporter: Andrew Bayer
Assignee: Andrew Bayer
Priority: Critical
 Fix For: 0.9.0

 Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, 
 HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, 
 HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, 
 HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, 
 HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt


 The current Hive Ivy dependency logic for its Hadoop dependencies is 
 problematic - depending on the tarball and extracting the jars from there, 
 rather than depending on the jars directly. It'd be great if this was fixed 
 to actually have the jar dependencies defined directly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs

2012-04-10 Thread Phabricator (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-2646:
--

Attachment: HIVE-2646.D2133.14.patch

abayer updated the revision HIVE-2646 [jira] Hive Ivy dependencies on Hadoop 
should depend on jars directly, not tarballs.
Reviewers: JIRA, cwsteinbach

  Added ivy:publish for builtins, rebased on trunk.

REVISION DETAIL
  https://reviews.facebook.net/D2133

AFFECTED FILES
  build-common.xml
  build.properties
  build.xml
  builtins/build.xml
  builtins/ivy.xml
  cli/ivy.xml
  common/ivy.xml
  contrib/build.xml
  contrib/ivy.xml
  hbase-handler/build.xml
  hbase-handler/ivy.xml
  hwi/build.xml
  hwi/ivy.xml
  ivy/common-configurations.xml
  ivy/ivysettings.xml
  ivy/libraries.properties
  jdbc/build.xml
  jdbc/ivy.xml
  metastore/ivy.xml
  pdk/ivy.xml
  pdk/scripts/build-plugin.xml
  ql/build.xml
  ql/ivy.xml
  serde/ivy.xml
  service/build.xml
  service/ivy.xml
  shims/build.xml
  shims/ivy.xml
  testutils/hadoop


 Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
 

 Key: HIVE-2646
 URL: https://issues.apache.org/jira/browse/HIVE-2646
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Affects Versions: 0.8.0
Reporter: Andrew Bayer
Assignee: Andrew Bayer
Priority: Critical
 Fix For: 0.9.0

 Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, 
 HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, 
 HIVE-2646.D2133.14.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, 
 HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, 
 HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, 
 HIVE-2646.diff.txt


 The current Hive Ivy dependency logic for its Hadoop dependencies is 
 problematic - depending on the tarball and extracting the jars from there, 
 rather than depending on the jars directly. It'd be great if this was fixed 
 to actually have the jar dependencies defined directly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Problems with Arc/Phabricator

2012-04-10 Thread Carl Steinbach 
Arc won't let me apply the patch for HIVE-2646. I've also noticed some
other problems (e.g. missing TEST PLAN in checkin comments), and was
wondering how we can get these problems resolved? Since most of this work
was done at Facebook, who is the designated point of contact for
Arc/Phabricator issues going forward?


Here's the error I get when applying HIVE-2646:

% arc patch D2133 --trace
Loading phutil library 'arc_jira_lib' from
'/Users/carl/Work/repos/hive-test/.arc_jira_lib'...
 [0] exec $ (cd '/Users/carl/Work/repos/hive-test'; git rev-parse
--show-cdup)
 [0] exec 13,599 us
 [1] conduit differential.getdiff()
 [1] conduit 608,237 us
 [2] exec $ (cd '/Users/carl/Work/repos/hive-test/'; git apply --index
--reject)
 [2] exec 13,178 us

Fatal error: Uncaught exception 'CommandException' with message 'Command
'(cd '/Users/carl/Work/repos/hive-test/'; git apply --index --reject)'
failed with error #128:
stdout:

stderr:
stdin:10: trailing whitespace.

stdin:111: trailing whitespace.
fileset dir=${build.ivy.lib.dir}/default includes=*.jar
fatal: corrupt patch at line 609

' in /Users/carl/.local/pkg/libphutil/src/future/exec/ExecFuture.php:346
Stack trace:
#0
/Users/carl/.local/pkg/arcanist/src/workflow/patch/ArcanistPatchWorkflow.php(412):
ExecFuture-resolvex()
#1 /Users/carl/.local/pkg/arcanist/scripts/arcanist.php(266):
ArcanistPatchWorkflow-run()
#2 {main}
  thrown in /Users/carl/.local/pkg/libphutil/src/future/exec/ExecFuture.php
on line 346


Thanks.

Carl

Re: Problems with Arc/Phabricator

2012-04-10 Thread Kevin Wilfong 
There's a list of ways to report issues/get help on the side bar of this
site http://phabricator.org/

It suggests #phabricator irc://chat.freenode.net/phabricator as the best
place to get help with issues.

On 4/10/12 5:42 PM, Carl Steinbach c...@cloudera.com wrote:

Arc won't let me apply the patch for HIVE-2646. I've also noticed some
other problems (e.g. missing TEST PLAN in checkin comments), and was
wondering how we can get these problems resolved? Since most of this work
was done at Facebook, who is the designated point of contact for
Arc/Phabricator issues going forward?


Here's the error I get when applying HIVE-2646:

% arc patch D2133 --trace
Loading phutil library 'arc_jira_lib' from
'/Users/carl/Work/repos/hive-test/.arc_jira_lib'...
 [0] exec $ (cd '/Users/carl/Work/repos/hive-test'; git rev-parse
--show-cdup)
 [0] exec 13,599 us
 [1] conduit differential.getdiff()
 [1] conduit 608,237 us
 [2] exec $ (cd '/Users/carl/Work/repos/hive-test/'; git apply
--index
--reject)
 [2] exec 13,178 us

Fatal error: Uncaught exception 'CommandException' with message 'Command
'(cd '/Users/carl/Work/repos/hive-test/'; git apply --index --reject)'
failed with error #128:
stdout:

stderr:
stdin:10: trailing whitespace.

stdin:111: trailing whitespace.
fileset dir=${build.ivy.lib.dir}/default includes=*.jar
fatal: corrupt patch at line 609

' in /Users/carl/.local/pkg/libphutil/src/future/exec/ExecFuture.php:346
Stack trace:
#0
/Users/carl/.local/pkg/arcanist/src/workflow/patch/ArcanistPatchWorkflow.p
hp(412):
ExecFuture-resolvex()
#1 /Users/carl/.local/pkg/arcanist/scripts/arcanist.php(266):
ArcanistPatchWorkflow-run()
#2 {main}
  thrown in 
/Users/carl/.local/pkg/libphutil/src/future/exec/ExecFuture.php
on line 346


Thanks.

Carl

Re: Problems with Arc/Phabricator

2012-04-10 Thread Carl Steinbach 
Ok, I'll submit an issue. But then what? Who is going to fix it? Perhaps in
the meantime we should just switch back to using reviewboard while we wait
for Arc and Phabricator to mature as open source projects.


On Tue, Apr 10, 2012 at 5:58 PM, Kevin Wilfong kevinwilf...@fb.com wrote:

 There's a list of ways to report issues/get help on the side bar of this
 site http://phabricator.org/

 It suggests #phabricator irc://chat.freenode.net/phabricator as the best
 place to get help with issues.

 On 4/10/12 5:42 PM, Carl Steinbach c...@cloudera.com wrote:

 Arc won't let me apply the patch for HIVE-2646. I've also noticed some
 other problems (e.g. missing TEST PLAN in checkin comments), and was
 wondering how we can get these problems resolved? Since most of this work
 was done at Facebook, who is the designated point of contact for
 Arc/Phabricator issues going forward?
 
 
 Here's the error I get when applying HIVE-2646:
 
 % arc patch D2133 --trace
 Loading phutil library 'arc_jira_lib' from
 '/Users/carl/Work/repos/hive-test/.arc_jira_lib'...
  [0] exec $ (cd '/Users/carl/Work/repos/hive-test'; git rev-parse
 --show-cdup)
  [0] exec 13,599 us
  [1] conduit differential.getdiff()
  [1] conduit 608,237 us
  [2] exec $ (cd '/Users/carl/Work/repos/hive-test/'; git apply
 --index
 --reject)
  [2] exec 13,178 us
 
 Fatal error: Uncaught exception 'CommandException' with message 'Command
 '(cd '/Users/carl/Work/repos/hive-test/'; git apply --index --reject)'
 failed with error #128:
 stdout:
 
 stderr:
 stdin:10: trailing whitespace.
 
 stdin:111: trailing whitespace.
 fileset dir=${build.ivy.lib.dir}/default includes=*.jar
 fatal: corrupt patch at line 609
 
 ' in /Users/carl/.local/pkg/libphutil/src/future/exec/ExecFuture.php:346
 Stack trace:
 #0
 /Users/carl/.local/pkg/arcanist/src/workflow/patch/ArcanistPatchWorkflow.p
 hp(412):
 ExecFuture-resolvex()
 #1 /Users/carl/.local/pkg/arcanist/scripts/arcanist.php(266):
 ArcanistPatchWorkflow-run()
 #2 {main}
   thrown in
 /Users/carl/.local/pkg/libphutil/src/future/exec/ExecFuture.php
 on line 346
 
 
 Thanks.
 
 Carl

Re: Problems with Arc/Phabricator

2012-04-10 Thread John Sichi 
I'll take a look.  We asked ASF infra to make the review board
instance read only to make sure the transition went through
completely, so it can't be used for new patches.

JVS

On Tue, Apr 10, 2012 at 6:27 PM, Kevin Wilfong kevinwilf...@fb.com wrote:
 +John Sichi as he did a lot of the work on getting Hive to use Phabricator.

 Is there anything preventing people from using Review Board for patches now,
 except maybe convention?  As for applying patches, in either case, you can
 download the patch from the JIRA site.

 To be clear, I am not trying to aggressively force Phabricator on Hive,
 these are just suggestions.

 From: Carl Steinbach c...@cloudera.com
 Date: Tue, 10 Apr 2012 18:10:44 -0700
 To: Microsoft Office User kevinwilf...@fb.com
 Cc: dev@hive.apache.org dev@hive.apache.org
 Subject: Re: Problems with Arc/Phabricator

 Ok, I'll submit an issue. But then what? Who is going to fix it? Perhaps in
 the meantime we should just switch back to using reviewboard while we wait
 for Arc and Phabricator to mature as open source projects.


 On Tue, Apr 10, 2012 at 5:58 PM, Kevin Wilfong kevinwilf...@fb.com wrote:

 There's a list of ways to report issues/get help on the side bar of this
 site http://phabricator.org/

 It suggests #phabricator irc://chat.freenode.net/phabricator as the best
 place to get help with issues.

 On 4/10/12 5:42 PM, Carl Steinbach c...@cloudera.com wrote:

 Arc won't let me apply the patch for HIVE-2646. I've also noticed some
 other problems (e.g. missing TEST PLAN in checkin comments), and was
 wondering how we can get these problems resolved? Since most of this work
 was done at Facebook, who is the designated point of contact for
 Arc/Phabricator issues going forward?
 
 
 Here's the error I get when applying HIVE-2646:
 
 % arc patch D2133 --trace
 Loading phutil library 'arc_jira_lib' from
 '/Users/carl/Work/repos/hive-test/.arc_jira_lib'...
  [0] exec $ (cd '/Users/carl/Work/repos/hive-test'; git rev-parse
 --show-cdup)
  [0] exec 13,599 us
  [1] conduit differential.getdiff()
  [1] conduit 608,237 us
  [2] exec $ (cd '/Users/carl/Work/repos/hive-test/'; git apply
 --index
 --reject)
  [2] exec 13,178 us
 
 Fatal error: Uncaught exception 'CommandException' with message 'Command
 '(cd '/Users/carl/Work/repos/hive-test/'; git apply --index --reject)'
 failed with error #128:
 stdout:
 
 stderr:
 stdin:10: trailing whitespace.
 
 stdin:111: trailing whitespace.
     fileset dir=${build.ivy.lib.dir}/default includes=*.jar
 fatal: corrupt patch at line 609
 
 ' in /Users/carl/.local/pkg/libphutil/src/future/exec/ExecFuture.php:346
 Stack trace:
 #0

  /Users/carl/.local/pkg/arcanist/src/workflow/patch/ArcanistPatchWorkflow.p
 hp(412):
 ExecFuture-resolvex()
 #1 /Users/carl/.local/pkg/arcanist/scripts/arcanist.php(266):
 ArcanistPatchWorkflow-run()
 #2 {main}
   thrown in
 /Users/carl/.local/pkg/libphutil/src/future/exec/ExecFuture.php
 on line 346
 
 
 Thanks.
 
 Carl

[jira] [Commented] (HIVE-2907) Hive error when dropping a table with large number of partitions

2012-04-10 Thread Hudson (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251235#comment-13251235
 ] 

Hudson commented on HIVE-2907:
--

Integrated in Hive-trunk-h0.21 #1367 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1367/])
HIVE-2907. Hive error when dropping a table with large number of 
partitions. (mousom via kevinwilfong) (Revision 1311850)

 Result = FAILURE
kevinwilfong : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1311850
Files : 
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/conf/hive-default.xml.template
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
* 
/hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
* /hive/trunk/ql/src/test/queries/clientnegative/drop_table_failure3.q
* /hive/trunk/ql/src/test/queries/clientpositive/drop_table2.q
* /hive/trunk/ql/src/test/results/clientnegative/drop_table_failure3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/drop_table2.q.out


 Hive error when dropping a table with large number of partitions
 

 Key: HIVE-2907
 URL: https://issues.apache.org/jira/browse/HIVE-2907
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.9.0
 Environment: General. Hive Metastore bug.
Reporter: Mousom Dhar Gupta
Assignee: Mousom Dhar Gupta
Priority: Minor
 Fix For: 0.9.0

 Attachments: HIVE-2907.1.patch.txt, HIVE-2907.2.patch.txt, 
 HIVE-2907.3.patch.txt, HIVE-2907.D2505.1.patch, HIVE-2907.D2505.2.patch, 
 HIVE-2907.D2505.3.patch, HIVE-2907.D2505.4.patch, HIVE-2907.D2505.5.patch, 
 HIVE-2907.D2505.6.patch, HIVE-2907.D2505.7.patch

   Original Estimate: 10h
  Remaining Estimate: 10h

 Running into an Out Of Memory error when trying to drop a table with 128K 
 partitions.
 The methods dropTable in 
 metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 
 and dropTable in ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
 encounter out of memory errors 
 when dropping tables with lots of partitions because they try to load the 
 metadata for every partition into memory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1367 - Still Failing

2012-04-10 Thread Apache Jenkins Server 
Changes for Build #1365
[hashutosh] Upping the version number from 0.9 to 0.10 on trunk.


Changes for Build #1366

Changes for Build #1367
[kevinwilfong] HIVE-2907. Hive error when dropping a table with large number of 
partitions. (mousom via kevinwilfong)




1 tests failed.
REGRESSION:  org.apache.hadoop.hive.hwi.TestHWIServer.testServerInit

Error Message:
Address already in use

Stack Trace:
java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383)
at java.net.ServerSocket.bind(ServerSocket.java:328)
at java.net.ServerSocket.init(ServerSocket.java:194)
at 
org.mortbay.jetty.bio.SocketConnector.newServerSocket(SocketConnector.java:80)
at org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)
at 
org.mortbay.jetty.AbstractConnector.doStart(AbstractConnector.java:283)
at 
org.mortbay.jetty.bio.SocketConnector.doStart(SocketConnector.java:147)
at 
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.mortbay.jetty.Server.doStart(Server.java:233)
at 
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at 
org.apache.hadoop.hive.hwi.TestHWIServer.testServerInit(TestHWIServer.java:71)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:154)
at junit.framework.TestCase.runBare(TestCase.java:127)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)




The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1367)

Status: Still Failing

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1367/ to 
view the results.

[jira] [Created] (HIVE-2944) dummy issue for testing arc

2012-04-10 Thread John Sichi (Created) (JIRA) 
dummy issue for testing arc
---

 Key: HIVE-2944
 URL: https://issues.apache.org/jira/browse/HIVE-2944
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Reporter: John Sichi
Assignee: John Sichi




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2944) dummy issue for testing arc

2012-04-10 Thread Phabricator (Updated) (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-2944:
--

Attachment: HIVE-2944.D2733.1.patch

jsichi requested code review of HIVE-2944 [jira] dummy issue for testing arc.
Reviewers: JIRA

  https://issues.apache.org/jira/browse/HIVE-2944

  Update README



TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D2733

AFFECTED FILES
  README.txt

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/6243/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


 dummy issue for testing arc
 ---

 Key: HIVE-2944
 URL: https://issues.apache.org/jira/browse/HIVE-2944
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Reporter: John Sichi
Assignee: John Sichi
 Attachments: HIVE-2944.D2733.1.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Problems with Arc/Phabricator

2012-04-10 Thread John Sichi 
For HIVE-2646:  the diagnostics aren't great, but I think it's just
choking on a patch that needs to be rebased, since as Kevin says, if I
download the latest .patch from JIRA and apply that with patch -p0, I
get the correct failure message.  So just ask the author to rebase
again?

Regarding the test plans:  Carl, could you be more specific about what
is going wrong so I can attempt to reproduce the problem?

JVS

On Tue, Apr 10, 2012 at 6:27 PM, Kevin Wilfong kevinwilf...@fb.com wrote:
 +John Sichi as he did a lot of the work on getting Hive to use Phabricator.

 Is there anything preventing people from using Review Board for patches now,
 except maybe convention?  As for applying patches, in either case, you can
 download the patch from the JIRA site.

 To be clear, I am not trying to aggressively force Phabricator on Hive,
 these are just suggestions.

 From: Carl Steinbach c...@cloudera.com
 Date: Tue, 10 Apr 2012 18:10:44 -0700
 To: Microsoft Office User kevinwilf...@fb.com
 Cc: dev@hive.apache.org dev@hive.apache.org
 Subject: Re: Problems with Arc/Phabricator

 Ok, I'll submit an issue. But then what? Who is going to fix it? Perhaps in
 the meantime we should just switch back to using reviewboard while we wait
 for Arc and Phabricator to mature as open source projects.


 On Tue, Apr 10, 2012 at 5:58 PM, Kevin Wilfong kevinwilf...@fb.com wrote:

 There's a list of ways to report issues/get help on the side bar of this
 site http://phabricator.org/

 It suggests #phabricator irc://chat.freenode.net/phabricator as the best
 place to get help with issues.

 On 4/10/12 5:42 PM, Carl Steinbach c...@cloudera.com wrote:

 Arc won't let me apply the patch for HIVE-2646. I've also noticed some
 other problems (e.g. missing TEST PLAN in checkin comments), and was
 wondering how we can get these problems resolved? Since most of this work
 was done at Facebook, who is the designated point of contact for
 Arc/Phabricator issues going forward?
 
 
 Here's the error I get when applying HIVE-2646:
 
 % arc patch D2133 --trace
 Loading phutil library 'arc_jira_lib' from
 '/Users/carl/Work/repos/hive-test/.arc_jira_lib'...
  [0] exec $ (cd '/Users/carl/Work/repos/hive-test'; git rev-parse
 --show-cdup)
  [0] exec 13,599 us
  [1] conduit differential.getdiff()
  [1] conduit 608,237 us
  [2] exec $ (cd '/Users/carl/Work/repos/hive-test/'; git apply
 --index
 --reject)
  [2] exec 13,178 us
 
 Fatal error: Uncaught exception 'CommandException' with message 'Command
 '(cd '/Users/carl/Work/repos/hive-test/'; git apply --index --reject)'
 failed with error #128:
 stdout:
 
 stderr:
 stdin:10: trailing whitespace.
 
 stdin:111: trailing whitespace.
     fileset dir=${build.ivy.lib.dir}/default includes=*.jar
 fatal: corrupt patch at line 609
 
 ' in /Users/carl/.local/pkg/libphutil/src/future/exec/ExecFuture.php:346
 Stack trace:
 #0

  /Users/carl/.local/pkg/arcanist/src/workflow/patch/ArcanistPatchWorkflow.p
 hp(412):
 ExecFuture-resolvex()
 #1 /Users/carl/.local/pkg/arcanist/scripts/arcanist.php(266):
 ArcanistPatchWorkflow-run()
 #2 {main}
   thrown in
 /Users/carl/.local/pkg/libphutil/src/future/exec/ExecFuture.php
 on line 346
 
 
 Thanks.
 
 Carl

[jira] [Commented] (HIVE-2614) Thrift api to know if partition exists or not

2012-04-10 Thread Amareshwari Sriramadasu (Commented) (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251329#comment-13251329
 ] 

Amareshwari Sriramadasu commented on HIVE-2614:
---

bq. @Amareshwari: Can you give a concrete example of a use-case for this new 
call? In general I think providing calls like this is a bad idea since it 
encourages people to accidentally add time-of-check-to-time-of-use bugs, e.g. 
HIVE-2938.
Main motivation here is to improve performance of exists check, which would 
just return boolean instead of the full partition. The use-case we were looking 
mainly was filtering i.e. processing the partitions that do not exists yet and 
add them.


 Thrift api to know if partition exists or not
 -

 Key: HIVE-2614
 URL: https://issues.apache.org/jira/browse/HIVE-2614
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Reporter: Amareshwari Sriramadasu
Assignee: Amareshwari Sriramadasu
 Fix For: 0.9.0


 Currently if we want to know if a partition exists or not, we should call 
 get_partition() which constructs the partition and returns. This would 
 require joining multiple tables at the metastore end and it might not scale 
 for many such operations.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-2945) Hive Client is hanging since MRApp Master doesnt exist after the task is finished

2012-04-10 Thread Kristam Subba Swathi (Created) (JIRA) 
Hive Client is hanging since MRApp Master doesnt exist after the task is 
finished 
--

 Key: HIVE-2945
 URL: https://issues.apache.org/jira/browse/HIVE-2945
 Project: Hive
  Issue Type: Bug
  Components: Clients
Reporter: Kristam Subba Swathi


Submit a hive query which uses mapred to give the result(Say join query)
If we submit a job to resourceManager process. it will create a MRAppMaster 
process to excute this task .
MRAppMaster will create YarnChild process to excute Map task or reduce task. 
after task is finished,resourceManager will kill MRAppMaster and YarnChild 
process.
When the hive client is about to get the progress from MRAppMaster ,the task is 
already finished ,so MRAppMaster process don't exist. so the client will try to 
connect it for ever.
that's why hive client hangs.



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira