Re: Book 'Programming Hive' from O'Reilly now available!

[Gabi D]

Congratulations Edward!
first order (for Outbrain) already on the way :)
Gabi

On Sun, Sep 30, 2012 at 1:51 AM, Edward Capriolo wrote:

> Hello all,
>
> I wanted to let you know that "Programming Hive" from O'Reilly is now
> available!
>
> http://shop.oreilly.com/product/0636920023555.do
>
> I could not have said enough in the dedication about how absolutely
> awesome Hive and Hadoop are! It is a great joy to work with hive and
> to get to write the book.
>
> One of the things I love about the book is that it had three authors,
> five case studies, and two entire sections written by others. This is
> much like the code base itself, which has a number of major
> contributions by a number of people and organizations.
>
> There is a long run down of people to thank for both the great work in
> hive as well as those who supported my involvement. I hope the book
> does all your great work justice!
>
> Thank you,
> Edward
>

[jira] [Commented] (HIVE-3467) BucketMapJoinOptimizer should optimize joins on partition columns

[Zhenxiao Luo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466422#comment-13466422
 ] 

Zhenxiao Luo commented on HIVE-3467:


Currently, BucketMapJoinOptimizer does not keep Partition information in its 
aliasToPartitionBucketNumberMapping and aliasToPartitionBucketFileNamesMapping, 
without information of Partition Columns, could not do the partition aware 
optimization. How about adding Partition info into the map:


-  LinkedHashMap> aliasToPartitionBucketNumberMapping 
=
-  new LinkedHashMap>();
-  LinkedHashMap>> 
aliasToPartitionBucketFileNamesMapping =
-  new LinkedHashMap>>();
+
+  // (alias to )
+  // AND (alias to )
+  // one pair for each partition
+  // partition key/values info is needed in optimization
+  LinkedHashMap>>
+aliasToPartitionBucketNumberMapping =
+new LinkedHashMap>>();
+  LinkedHashMap>>>
+aliasToPartitionBucketFileNamesMapping =
+new LinkedHashMap>>>();


> BucketMapJoinOptimizer should optimize joins on partition columns
> -
>
> Key: HIVE-3467
> URL: https://issues.apache.org/jira/browse/HIVE-3467
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.10.0
>Reporter: Kevin Wilfong
>
> Consider the query:
> SELECT * FROM t1 JOIN t2 on t1.part = t2.part and t1.key = t2.key;
> Where t1 and t2 are partitioned by part and bucketed by key.
> Suppose part take values 1 and 2 and t1 and t2 are bucketed into 2 buckets.
> The bucket map join optimizer will put the first bucket of part=1 and part=2 
> partitions of t2 into the same mapper as that of part=1 partition of t1.  It 
> will do the same for the part=2 partition of t1.
> It could take advantage of the partition values and send the first bucket of 
> only the part=1 partitions of t1 and t2 into one mapper and the first bucket 
> of only the part=2 partitions into another.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-3467) BucketMapJoinOptimizer should optimize joins on partition columns

[Zhenxiao Luo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo reassigned HIVE-3467:
--

Assignee: Zhenxiao Luo

> BucketMapJoinOptimizer should optimize joins on partition columns
> -
>
> Key: HIVE-3467
> URL: https://issues.apache.org/jira/browse/HIVE-3467
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.10.0
>Reporter: Kevin Wilfong
>Assignee: Zhenxiao Luo
>
> Consider the query:
> SELECT * FROM t1 JOIN t2 on t1.part = t2.part and t1.key = t2.key;
> Where t1 and t2 are partitioned by part and bucketed by key.
> Suppose part take values 1 and 2 and t1 and t2 are bucketed into 2 buckets.
> The bucket map join optimizer will put the first bucket of part=1 and part=2 
> partitions of t2 into the same mapper as that of part=1 partition of t1.  It 
> will do the same for the part=2 partition of t1.
> It could take advantage of the partition values and send the first bucket of 
> only the part=1 partitions of t1 and t2 into one mapper and the first bucket 
> of only the part=2 partitions into another.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-3426) union with same source should be optimized

[Zhenxiao Luo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo reassigned HIVE-3426:
--

Assignee: Zhenxiao Luo

> union with same source should be optimized
> --
>
> Key: HIVE-3426
> URL: https://issues.apache.org/jira/browse/HIVE-3426
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Zhenxiao Luo
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3426) union with same source should be optimized

[Zhenxiao Luo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466419#comment-13466419
 ] 

Zhenxiao Luo commented on HIVE-3426:


TS -- FIL -- SEL
\
 \
  UNION
 /
/
TS -- TIL -- SEL

Needs to be updated into:

TS --FIL -- SEL

And, the FIL condition needs to be updated,

eg.

select key from srcpart where key = 484
union all
select key from srcpart where key = 409

should be updated into:

select key from srcpart where key = 484 OR key = 409



> union with same source should be optimized
> --
>
> Key: HIVE-3426
> URL: https://issues.apache.org/jira/browse/HIVE-3426
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-1977) DESCRIBE TABLE syntax doesn't support specifying a database qualified table name

[Zhenxiao Luo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo updated HIVE-1977:
---

Status: Patch Available  (was: Open)

> DESCRIBE TABLE syntax doesn't support specifying a database qualified table 
> name
> 
>
> Key: HIVE-1977
> URL: https://issues.apache.org/jira/browse/HIVE-1977
> Project: Hive
>  Issue Type: Bug
>  Components: Database/Schema, Query Processor, SQL
>Reporter: Carl Steinbach
>Assignee: Zhenxiao Luo
> Attachments: HIVE-1977.1.patch.txt
>
>
> The syntax for DESCRIBE is broken. It should be:
> {code}
> DESCRIBE [EXTENDED] [database DOT]table [column]
> {code}
> but is actually
> {code}
> DESCRIBE [EXTENDED] table[DOT col_name]
> {code}
> Ref: http://dev.mysql.com/doc/refman/5.0/en/describe.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-1977) DESCRIBE TABLE syntax doesn't support specifying a database qualified table name

[Zhenxiao Luo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo updated HIVE-1977:
---

Attachment: HIVE-1977.1.patch.txt

> DESCRIBE TABLE syntax doesn't support specifying a database qualified table 
> name
> 
>
> Key: HIVE-1977
> URL: https://issues.apache.org/jira/browse/HIVE-1977
> Project: Hive
>  Issue Type: Bug
>  Components: Database/Schema, Query Processor, SQL
>Reporter: Carl Steinbach
>Assignee: Zhenxiao Luo
> Attachments: HIVE-1977.1.patch.txt
>
>
> The syntax for DESCRIBE is broken. It should be:
> {code}
> DESCRIBE [EXTENDED] [database DOT]table [column]
> {code}
> but is actually
> {code}
> DESCRIBE [EXTENDED] table[DOT col_name]
> {code}
> Ref: http://dev.mysql.com/doc/refman/5.0/en/describe.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-1977) DESCRIBE TABLE syntax doesn't support specifying a database qualified table name

[Zhenxiao Luo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466417#comment-13466417
 ] 

Zhenxiao Luo commented on HIVE-1977:


Review Request submitted at:
https://reviews.facebook.net/D5763

> DESCRIBE TABLE syntax doesn't support specifying a database qualified table 
> name
> 
>
> Key: HIVE-1977
> URL: https://issues.apache.org/jira/browse/HIVE-1977
> Project: Hive
>  Issue Type: Bug
>  Components: Database/Schema, Query Processor, SQL
>Reporter: Carl Steinbach
>Assignee: Zhenxiao Luo
> Attachments: HIVE-1977.1.patch.txt
>
>
> The syntax for DESCRIBE is broken. It should be:
> {code}
> DESCRIBE [EXTENDED] [database DOT]table [column]
> {code}
> but is actually
> {code}
> DESCRIBE [EXTENDED] table[DOT col_name]
> {code}
> Ref: http://dev.mysql.com/doc/refman/5.0/en/describe.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-1367) cluster by multiple columns does not work if parenthesis is present

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466416#comment-13466416
 ] 

Namit Jain commented on HIVE-1367:
--

running tests

> cluster by multiple columns does not work if parenthesis is present
> ---
>
> Key: HIVE-1367
> URL: https://issues.apache.org/jira/browse/HIVE-1367
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Zhenxiao Luo
> Attachments: HIVE-1367.1.patch.txt
>
>
> The following query:
> select ...  from src cluster by (key, value)
> throws a compile error:
> whereas the query
> select ...  from src cluster by key, value
> works fine

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-1367) cluster by multiple columns does not work if parenthesis is present

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466415#comment-13466415
 ] 

Namit Jain commented on HIVE-1367:
--

+1

> cluster by multiple columns does not work if parenthesis is present
> ---
>
> Key: HIVE-1367
> URL: https://issues.apache.org/jira/browse/HIVE-1367
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Zhenxiao Luo
> Attachments: HIVE-1367.1.patch.txt
>
>
> The following query:
> select ...  from src cluster by (key, value)
> throws a compile error:
> whereas the query
> select ...  from src cluster by key, value
> works fine

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-1367) cluster by multiple columns does not work if parenthesis is present

[Zhenxiao Luo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo updated HIVE-1367:
---

Attachment: HIVE-1367.1.patch.txt

> cluster by multiple columns does not work if parenthesis is present
> ---
>
> Key: HIVE-1367
> URL: https://issues.apache.org/jira/browse/HIVE-1367
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Zhenxiao Luo
> Attachments: HIVE-1367.1.patch.txt
>
>
> The following query:
> select ...  from src cluster by (key, value)
> throws a compile error:
> whereas the query
> select ...  from src cluster by key, value
> works fine

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-1367) cluster by multiple columns does not work if parenthesis is present

[Zhenxiao Luo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo updated HIVE-1367:
---

Status: Patch Available  (was: Open)

> cluster by multiple columns does not work if parenthesis is present
> ---
>
> Key: HIVE-1367
> URL: https://issues.apache.org/jira/browse/HIVE-1367
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Zhenxiao Luo
> Attachments: HIVE-1367.1.patch.txt
>
>
> The following query:
> select ...  from src cluster by (key, value)
> throws a compile error:
> whereas the query
> select ...  from src cluster by key, value
> works fine

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-1367) cluster by multiple columns does not work if parenthesis is present

[Zhenxiao Luo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466414#comment-13466414
 ] 

Zhenxiao Luo commented on HIVE-1367:


Review Request submitted at:
https://reviews.facebook.net/D5757

> cluster by multiple columns does not work if parenthesis is present
> ---
>
> Key: HIVE-1367
> URL: https://issues.apache.org/jira/browse/HIVE-1367
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
> Attachments: HIVE-1367.1.patch.txt
>
>
> The following query:
> select ...  from src cluster by (key, value)
> throws a compile error:
> whereas the query
> select ...  from src cluster by key, value
> works fine

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-1367) cluster by multiple columns does not work if parenthesis is present

[Zhenxiao Luo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo reassigned HIVE-1367:
--

Assignee: Zhenxiao Luo

> cluster by multiple columns does not work if parenthesis is present
> ---
>
> Key: HIVE-1367
> URL: https://issues.apache.org/jira/browse/HIVE-1367
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Zhenxiao Luo
> Attachments: HIVE-1367.1.patch.txt
>
>
> The following query:
> select ...  from src cluster by (key, value)
> throws a compile error:
> whereas the query
> select ...  from src cluster by key, value
> works fine

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2874) Renaming external partition changes location

[Zhenxiao Luo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo updated HIVE-2874:
---

Status: Patch Available  (was: Open)

> Renaming external partition changes location
> 
>
> Key: HIVE-2874
> URL: https://issues.apache.org/jira/browse/HIVE-2874
> Project: Hive
>  Issue Type: Bug
>Reporter: Kevin Wilfong
>Assignee: Zhenxiao Luo
> Attachments: HIVE-2874.1.patch.txt
>
>
> Renaming an external partition will change the location of that partition to 
> the default location of a managed partition with the same name.
> E.g. If ex_table is external and has partition part=1 with location 
> /.../managed_table/part=1
> Calling ALTER TABLE ex_table PARTITION (part = '1') RENAME TO PARTITION (part 
> = '2');
> Will change the location of the partition to /.../ex_table/part=2

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2874) Renaming external partition changes location

[Zhenxiao Luo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466413#comment-13466413
 ] 

Zhenxiao Luo commented on HIVE-2874:


Review Request submitted at:
https://reviews.facebook.net/D5751

> Renaming external partition changes location
> 
>
> Key: HIVE-2874
> URL: https://issues.apache.org/jira/browse/HIVE-2874
> Project: Hive
>  Issue Type: Bug
>Reporter: Kevin Wilfong
>Assignee: Zhenxiao Luo
> Attachments: HIVE-2874.1.patch.txt
>
>
> Renaming an external partition will change the location of that partition to 
> the default location of a managed partition with the same name.
> E.g. If ex_table is external and has partition part=1 with location 
> /.../managed_table/part=1
> Calling ALTER TABLE ex_table PARTITION (part = '1') RENAME TO PARTITION (part 
> = '2');
> Will change the location of the partition to /.../ex_table/part=2

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2874) Renaming external partition changes location

[Zhenxiao Luo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo updated HIVE-2874:
---

Attachment: HIVE-2874.1.patch.txt

> Renaming external partition changes location
> 
>
> Key: HIVE-2874
> URL: https://issues.apache.org/jira/browse/HIVE-2874
> Project: Hive
>  Issue Type: Bug
>Reporter: Kevin Wilfong
>Assignee: Zhenxiao Luo
> Attachments: HIVE-2874.1.patch.txt
>
>
> Renaming an external partition will change the location of that partition to 
> the default location of a managed partition with the same name.
> E.g. If ex_table is external and has partition part=1 with location 
> /.../managed_table/part=1
> Calling ALTER TABLE ex_table PARTITION (part = '1') RENAME TO PARTITION (part 
> = '2');
> Will change the location of the partition to /.../ex_table/part=2

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3493) aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two different ways

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3493:
-

Status: Open  (was: Patch Available)

> aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two 
> different ways
> --
>
> Key: HIVE-3493
> URL: https://issues.apache.org/jira/browse/HIVE-3493
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.10.0
>Reporter: Yin Huai
>Assignee: Yin Huai
>Priority: Minor
> Attachments: HIVE-3493.1.patch.txt, HIVE-3493.2.patch.txt
>
>
> aggName in 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(String,
>  ArrayList, ASTNode, boolean, boolean) is generated by two 
> different ways. One is String aggName = value.getChild(0).getText(); and 
> another is String aggName = unescapeIdentifier(value.getChild(0).getText());. 
> When a aggregation function is involved in a view, we may get a error.
> You can try the query below (from create_view.q) to replay the error.
> {code:sql}
> set hive.map.aggr=false;
> CREATE TEMPORARY FUNCTION test_max AS
> 'org.apache.hadoop.hive.ql.udf.UDAFTestMax';
> CREATE VIEW view9(m) AS
> SELECT test_max(length(value))
> FROM src;
> DESCRIBE EXTENDED view9;
> DESCRIBE FORMATTED view9;
> SELECT * FROM view9;
> {code}
> Here is the log
> {code}
> 2012-09-20 07:26:15,176 DEBUG exec.FunctionRegistry 
> (FunctionRegistry.java:getGenericUDAFResolver(849)) - Looking up GenericUDAF: 
> `test_max`
> 2012-09-20 07:26:15,181 ERROR ql.Driver (SessionState.java:printError(400)) - 
> FAILED: SemanticException Line 1:33 Function argument type mismatch 
> '`test_max`' in definition of VIEW view9 [
> SELECT `_c0` AS `m` FROM (SELECT `test_max`(length(`src`.`value`))
> FROM `default`.`src`) `view9`
> ] used as view9 at Line 2:14: Looking for UDAF Evaluator"`test_max`" with 
> parameters 
> [org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector@5afa2b2b]
> org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:33 Function 
> argument type mismatch '`test_max`' in definition of VIEW view9 [
> SELECT `_c0` AS `m` FROM (SELECT `test_max`(length(`src`.`value`))
> FROM `default`.`src`) `view9`
> ] used as view9 at Line 2:14: Looking for UDAF Evaluator"`test_max`" with 
> parameters 
> [org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector@5afa2b2b]
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:2394)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanGroupByOperator(SemanticAnalyzer.java:2561)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlan1MR(SemanticAnalyzer.java:3341)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:6140)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6903)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6843)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6864)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6843)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6864)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7484)
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:245)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:431)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:903)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>   at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:713)
>   at 
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view(TestCliDriver.java:125)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at junit.framework.TestCase.runTest(TestCase.java:168)
>   at junit.framework.TestCase.runBare(TestCase.java:134)
>   at junit.framework.TestRe

[jira] [Commented] (HIVE-3493) aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two different ways

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466412#comment-13466412
 ] 

Namit Jain commented on HIVE-3493:
--

Can you do the same change on line 3281 also ?
Otherwise, it looks good.

> aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two 
> different ways
> --
>
> Key: HIVE-3493
> URL: https://issues.apache.org/jira/browse/HIVE-3493
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.10.0
>Reporter: Yin Huai
>Assignee: Yin Huai
>Priority: Minor
> Attachments: HIVE-3493.1.patch.txt, HIVE-3493.2.patch.txt
>
>
> aggName in 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(String,
>  ArrayList, ASTNode, boolean, boolean) is generated by two 
> different ways. One is String aggName = value.getChild(0).getText(); and 
> another is String aggName = unescapeIdentifier(value.getChild(0).getText());. 
> When a aggregation function is involved in a view, we may get a error.
> You can try the query below (from create_view.q) to replay the error.
> {code:sql}
> set hive.map.aggr=false;
> CREATE TEMPORARY FUNCTION test_max AS
> 'org.apache.hadoop.hive.ql.udf.UDAFTestMax';
> CREATE VIEW view9(m) AS
> SELECT test_max(length(value))
> FROM src;
> DESCRIBE EXTENDED view9;
> DESCRIBE FORMATTED view9;
> SELECT * FROM view9;
> {code}
> Here is the log
> {code}
> 2012-09-20 07:26:15,176 DEBUG exec.FunctionRegistry 
> (FunctionRegistry.java:getGenericUDAFResolver(849)) - Looking up GenericUDAF: 
> `test_max`
> 2012-09-20 07:26:15,181 ERROR ql.Driver (SessionState.java:printError(400)) - 
> FAILED: SemanticException Line 1:33 Function argument type mismatch 
> '`test_max`' in definition of VIEW view9 [
> SELECT `_c0` AS `m` FROM (SELECT `test_max`(length(`src`.`value`))
> FROM `default`.`src`) `view9`
> ] used as view9 at Line 2:14: Looking for UDAF Evaluator"`test_max`" with 
> parameters 
> [org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector@5afa2b2b]
> org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:33 Function 
> argument type mismatch '`test_max`' in definition of VIEW view9 [
> SELECT `_c0` AS `m` FROM (SELECT `test_max`(length(`src`.`value`))
> FROM `default`.`src`) `view9`
> ] used as view9 at Line 2:14: Looking for UDAF Evaluator"`test_max`" with 
> parameters 
> [org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector@5afa2b2b]
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:2394)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanGroupByOperator(SemanticAnalyzer.java:2561)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlan1MR(SemanticAnalyzer.java:3341)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:6140)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6903)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6843)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6864)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6843)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6864)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7484)
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:245)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:431)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:903)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>   at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:713)
>   at 
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view(TestCliDriver.java:125)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at junit.framework.TestCase.runTest(TestCase.java:

[jira] [Commented] (HIVE-2874) Renaming external partition changes location

[Zhenxiao Luo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466410#comment-13466410
 ] 

Zhenxiao Luo commented on HIVE-2874:


if it is an EXTERNAL_TABLE, the table's location is in StorageDescriptor, the 
new patch shares the same parent path as the old path.

If it is a MANAGED_TABLE, the table's location could be get from Warehouse.

> Renaming external partition changes location
> 
>
> Key: HIVE-2874
> URL: https://issues.apache.org/jira/browse/HIVE-2874
> Project: Hive
>  Issue Type: Bug
>Reporter: Kevin Wilfong
>
> Renaming an external partition will change the location of that partition to 
> the default location of a managed partition with the same name.
> E.g. If ex_table is external and has partition part=1 with location 
> /.../managed_table/part=1
> Calling ALTER TABLE ex_table PARTITION (part = '1') RENAME TO PARTITION (part 
> = '2');
> Will change the location of the partition to /.../ex_table/part=2

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-2874) Renaming external partition changes location

[Zhenxiao Luo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo reassigned HIVE-2874:
--

Assignee: Zhenxiao Luo

> Renaming external partition changes location
> 
>
> Key: HIVE-2874
> URL: https://issues.apache.org/jira/browse/HIVE-2874
> Project: Hive
>  Issue Type: Bug
>Reporter: Kevin Wilfong
>Assignee: Zhenxiao Luo
>
> Renaming an external partition will change the location of that partition to 
> the default location of a managed partition with the same name.
> E.g. If ex_table is external and has partition part=1 with location 
> /.../managed_table/part=1
> Calling ALTER TABLE ex_table PARTITION (part = '1') RENAME TO PARTITION (part 
> = '2');
> Will change the location of the partition to /.../ex_table/part=2

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3515) metadata_export_drop.q causes failure of other tests

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-3515:
---

Fix Version/s: 0.10.0

> metadata_export_drop.q causes failure of other tests
> 
>
> Key: HIVE-3515
> URL: https://issues.apache.org/jira/browse/HIVE-3515
> Project: Hive
>  Issue Type: Bug
>  Components: Tests
>Reporter: Ivan Gorbachev
>Assignee: Ivan Gorbachev
> Fix For: 0.10.0
>
> Attachments: jira-3515.1.patch
>
>
> metadata_export_drop.q causes failure of other tests on cleanup stage.
> {quote}
> Exception: java.lang.IllegalArgumentException: java.net.URISyntaxException: 
> Relative path in absolute URI: 
> file:../build/ql/test/data/exports/HIVE-3427/src.2012-09-28-11-38-17
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative 
> path in absolute URI: 
> file:../build/ql/test/data/exports/HIVE-3427/src.2012-09-28-11-38-17
>   at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:845)
>   at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:821)
>   at org.apache.hadoop.hive.ql.QTestUtil.cleanUp(QTestUtil.java:445)
>   at org.apache.hadoop.hive.ql.QTestUtil.shutdown(QTestUtil.java:300)
>   at 
> org.apache.hadoop.hive.cli.TestCliDriver.tearDown(TestCliDriver.java:87)
>   at junit.framework.TestCase.runBare(TestCase.java:140)
>   at junit.framework.TestResult$1.protect(TestResult.java:110)
>   at junit.framework.TestResult.runProtected(TestResult.java:128)
>   at junit.framework.TestResult.run(TestResult.java:113)
>   at junit.framework.TestCase.run(TestCase.java:124)
>   at junit.framework.TestSuite.runTest(TestSuite.java:232)
>   at junit.framework.TestSuite.run(TestSuite.java:227)
>   at 
> org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
>   at 
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: 
> Relative path in absolute URI: 
> file:../build/ql/test/data/exports/HIVE-3427/src.2012-09-28-11-38-17
>   at org.apache.hadoop.fs.Path.initialize(Path.java:140)
>   at org.apache.hadoop.fs.Path.(Path.java:132)
>   at 
> org.apache.hadoop.fs.ProxyFileSystem.swizzleParamPath(ProxyFileSystem.java:56)
>   at org.apache.hadoop.fs.ProxyFileSystem.mkdirs(ProxyFileSystem.java:214)
>   at 
> org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
>   at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1120)
>   at 
> org.apache.hadoop.hive.ql.parse.MetaDataExportListener.export_meta_data(MetaDataExportListener.java:81)
>   at 
> org.apache.hadoop.hive.ql.parse.MetaDataExportListener.onEvent(MetaDataExportListener.java:106)
>   at 
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_core(HiveMetaStore.java:1024)
>   at 
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table(HiveMetaStore.java:1185)
>   at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropTable(HiveMetaStoreClient.java:566)
>   at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:839)
>   ... 17 more
> Caused by: java.net.URISyntaxException: Relative path in absolute URI: 
> file:../build/ql/test/data/exports/HIVE-3427/src.2012-09-28-11-38-17
>   at java.net.URI.checkPath(URI.java:1787)
>   at java.net.URI.(URI.java:735)
>   at org.apache.hadoop.fs.Path.initialize(Path.java:137)
>   ... 28 more
> {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1709 - Still Failing

[Apache Jenkins Server]

Changes for Build #1708

Changes for Build #1709
[namit] HIVE-3515 metadata_export_drop.q causes failure of other tests
(Ivan Gorbachev via namit)




1 tests failed.
REGRESSION:  
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_aggregator_error_1

Error Message:
Forked Java VM exited abnormally. Please note the time in the report does not 
reflect the time until the VM exit.

Stack Trace:
junit.framework.AssertionFailedError: Forked Java VM exited abnormally. Please 
note the time in the report does not reflect the time until the VM exit.
at 
net.sf.antcontrib.logic.ForTask.doSequentialIteration(ForTask.java:259)
at net.sf.antcontrib.logic.ForTask.doToken(ForTask.java:268)
at net.sf.antcontrib.logic.ForTask.doTheTasks(ForTask.java:324)
at net.sf.antcontrib.logic.ForTask.execute(ForTask.java:244)




The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1709)

Status: Still Failing

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1709/ to 
view the results.

[jira] [Commented] (HIVE-3515) metadata_export_drop.q causes failure of other tests

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466400#comment-13466400
 ] 

Hudson commented on HIVE-3515:
--

Integrated in Hive-trunk-h0.21 #1709 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1709/])
HIVE-3515 metadata_export_drop.q causes failure of other tests
(Ivan Gorbachev via namit) (Revision 1391848)

 Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1391848
Files : 
* /hive/trunk/ql/src/test/queries/clientpositive/metadata_export_drop.q


> metadata_export_drop.q causes failure of other tests
> 
>
> Key: HIVE-3515
> URL: https://issues.apache.org/jira/browse/HIVE-3515
> Project: Hive
>  Issue Type: Bug
>  Components: Tests
>Reporter: Ivan Gorbachev
>Assignee: Ivan Gorbachev
> Attachments: jira-3515.1.patch
>
>
> metadata_export_drop.q causes failure of other tests on cleanup stage.
> {quote}
> Exception: java.lang.IllegalArgumentException: java.net.URISyntaxException: 
> Relative path in absolute URI: 
> file:../build/ql/test/data/exports/HIVE-3427/src.2012-09-28-11-38-17
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative 
> path in absolute URI: 
> file:../build/ql/test/data/exports/HIVE-3427/src.2012-09-28-11-38-17
>   at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:845)
>   at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:821)
>   at org.apache.hadoop.hive.ql.QTestUtil.cleanUp(QTestUtil.java:445)
>   at org.apache.hadoop.hive.ql.QTestUtil.shutdown(QTestUtil.java:300)
>   at 
> org.apache.hadoop.hive.cli.TestCliDriver.tearDown(TestCliDriver.java:87)
>   at junit.framework.TestCase.runBare(TestCase.java:140)
>   at junit.framework.TestResult$1.protect(TestResult.java:110)
>   at junit.framework.TestResult.runProtected(TestResult.java:128)
>   at junit.framework.TestResult.run(TestResult.java:113)
>   at junit.framework.TestCase.run(TestCase.java:124)
>   at junit.framework.TestSuite.runTest(TestSuite.java:232)
>   at junit.framework.TestSuite.run(TestSuite.java:227)
>   at 
> org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
>   at 
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: 
> Relative path in absolute URI: 
> file:../build/ql/test/data/exports/HIVE-3427/src.2012-09-28-11-38-17
>   at org.apache.hadoop.fs.Path.initialize(Path.java:140)
>   at org.apache.hadoop.fs.Path.(Path.java:132)
>   at 
> org.apache.hadoop.fs.ProxyFileSystem.swizzleParamPath(ProxyFileSystem.java:56)
>   at org.apache.hadoop.fs.ProxyFileSystem.mkdirs(ProxyFileSystem.java:214)
>   at 
> org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
>   at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1120)
>   at 
> org.apache.hadoop.hive.ql.parse.MetaDataExportListener.export_meta_data(MetaDataExportListener.java:81)
>   at 
> org.apache.hadoop.hive.ql.parse.MetaDataExportListener.onEvent(MetaDataExportListener.java:106)
>   at 
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_core(HiveMetaStore.java:1024)
>   at 
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table(HiveMetaStore.java:1185)
>   at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropTable(HiveMetaStoreClient.java:566)
>   at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:839)
>   ... 17 more
> Caused by: java.net.URISyntaxException: Relative path in absolute URI: 
> file:../build/ql/test/data/exports/HIVE-3427/src.2012-09-28-11-38-17
>   at java.net.URI.checkPath(URI.java:1787)
>   at java.net.URI.(URI.java:735)
>   at org.apache.hadoop.fs.Path.initialize(Path.java:137)
>   ... 28 more
> {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3493) aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two different ways

[Yin Huai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yin Huai updated HIVE-3493:
---

Status: Patch Available  (was: Open)

> aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two 
> different ways
> --
>
> Key: HIVE-3493
> URL: https://issues.apache.org/jira/browse/HIVE-3493
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.10.0
>Reporter: Yin Huai
>Assignee: Yin Huai
>Priority: Minor
> Attachments: HIVE-3493.1.patch.txt, HIVE-3493.2.patch.txt
>
>
> aggName in 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(String,
>  ArrayList, ASTNode, boolean, boolean) is generated by two 
> different ways. One is String aggName = value.getChild(0).getText(); and 
> another is String aggName = unescapeIdentifier(value.getChild(0).getText());. 
> When a aggregation function is involved in a view, we may get a error.
> You can try the query below (from create_view.q) to replay the error.
> {code:sql}
> set hive.map.aggr=false;
> CREATE TEMPORARY FUNCTION test_max AS
> 'org.apache.hadoop.hive.ql.udf.UDAFTestMax';
> CREATE VIEW view9(m) AS
> SELECT test_max(length(value))
> FROM src;
> DESCRIBE EXTENDED view9;
> DESCRIBE FORMATTED view9;
> SELECT * FROM view9;
> {code}
> Here is the log
> {code}
> 2012-09-20 07:26:15,176 DEBUG exec.FunctionRegistry 
> (FunctionRegistry.java:getGenericUDAFResolver(849)) - Looking up GenericUDAF: 
> `test_max`
> 2012-09-20 07:26:15,181 ERROR ql.Driver (SessionState.java:printError(400)) - 
> FAILED: SemanticException Line 1:33 Function argument type mismatch 
> '`test_max`' in definition of VIEW view9 [
> SELECT `_c0` AS `m` FROM (SELECT `test_max`(length(`src`.`value`))
> FROM `default`.`src`) `view9`
> ] used as view9 at Line 2:14: Looking for UDAF Evaluator"`test_max`" with 
> parameters 
> [org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector@5afa2b2b]
> org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:33 Function 
> argument type mismatch '`test_max`' in definition of VIEW view9 [
> SELECT `_c0` AS `m` FROM (SELECT `test_max`(length(`src`.`value`))
> FROM `default`.`src`) `view9`
> ] used as view9 at Line 2:14: Looking for UDAF Evaluator"`test_max`" with 
> parameters 
> [org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector@5afa2b2b]
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:2394)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanGroupByOperator(SemanticAnalyzer.java:2561)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlan1MR(SemanticAnalyzer.java:3341)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:6140)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6903)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6843)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6864)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6843)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6864)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7484)
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:245)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:431)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:903)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>   at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:713)
>   at 
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view(TestCliDriver.java:125)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at junit.framework.TestCase.runTest(TestCase.java:168)
>   at junit.framework.TestCase.runBare(TestCase.java:134)
>   at junit.framework.TestResult

[jira] [Updated] (HIVE-3493) aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two different ways

[Yin Huai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yin Huai updated HIVE-3493:
---

Attachment: HIVE-3493.2.patch.txt

sure. In the updated patch, in create_view.q, view9 is created twice with 
disabled and enabled map-side aggregation, respectively. Reviewboard has been 
also updated.

> aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two 
> different ways
> --
>
> Key: HIVE-3493
> URL: https://issues.apache.org/jira/browse/HIVE-3493
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.10.0
>Reporter: Yin Huai
>Assignee: Yin Huai
>Priority: Minor
> Attachments: HIVE-3493.1.patch.txt, HIVE-3493.2.patch.txt
>
>
> aggName in 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(String,
>  ArrayList, ASTNode, boolean, boolean) is generated by two 
> different ways. One is String aggName = value.getChild(0).getText(); and 
> another is String aggName = unescapeIdentifier(value.getChild(0).getText());. 
> When a aggregation function is involved in a view, we may get a error.
> You can try the query below (from create_view.q) to replay the error.
> {code:sql}
> set hive.map.aggr=false;
> CREATE TEMPORARY FUNCTION test_max AS
> 'org.apache.hadoop.hive.ql.udf.UDAFTestMax';
> CREATE VIEW view9(m) AS
> SELECT test_max(length(value))
> FROM src;
> DESCRIBE EXTENDED view9;
> DESCRIBE FORMATTED view9;
> SELECT * FROM view9;
> {code}
> Here is the log
> {code}
> 2012-09-20 07:26:15,176 DEBUG exec.FunctionRegistry 
> (FunctionRegistry.java:getGenericUDAFResolver(849)) - Looking up GenericUDAF: 
> `test_max`
> 2012-09-20 07:26:15,181 ERROR ql.Driver (SessionState.java:printError(400)) - 
> FAILED: SemanticException Line 1:33 Function argument type mismatch 
> '`test_max`' in definition of VIEW view9 [
> SELECT `_c0` AS `m` FROM (SELECT `test_max`(length(`src`.`value`))
> FROM `default`.`src`) `view9`
> ] used as view9 at Line 2:14: Looking for UDAF Evaluator"`test_max`" with 
> parameters 
> [org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector@5afa2b2b]
> org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:33 Function 
> argument type mismatch '`test_max`' in definition of VIEW view9 [
> SELECT `_c0` AS `m` FROM (SELECT `test_max`(length(`src`.`value`))
> FROM `default`.`src`) `view9`
> ] used as view9 at Line 2:14: Looking for UDAF Evaluator"`test_max`" with 
> parameters 
> [org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector@5afa2b2b]
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:2394)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanGroupByOperator(SemanticAnalyzer.java:2561)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlan1MR(SemanticAnalyzer.java:3341)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:6140)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6903)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6843)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6864)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6843)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6864)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7484)
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:245)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:431)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:903)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>   at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:713)
>   at 
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view(TestCliDriver.java:125)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Me

Re: Review Request: HIVE-3493 aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two different ways

[Yin Huai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7222/
---

(Updated Sept. 30, 2012, 2:21 a.m.)


Review request for hive.


Changes
---

add a test case


Description
---

Please see jira for detailed description


This addresses bug HIVE-3493.
https://issues.apache.org/jira/browse/HIVE-3493


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 3f2 
  ql/src/test/queries/clientpositive/create_view.q 0af3442 
  ql/src/test/results/clientpositive/create_view.q.out 3b78f1e 

Diff: https://reviews.apache.org/r/7222/diff/


Testing
---

All tests pass.


Thanks,

Yin Huai

Book 'Programming Hive' from O'Reilly now available!

[Edward Capriolo]

Hello all,

I wanted to let you know that "Programming Hive" from O'Reilly is now available!

http://shop.oreilly.com/product/0636920023555.do

I could not have said enough in the dedication about how absolutely
awesome Hive and Hadoop are! It is a great joy to work with hive and
to get to write the book.

One of the things I love about the book is that it had three authors,
five case studies, and two entire sections written by others. This is
much like the code base itself, which has a number of major
contributions by a number of people and organizations.

There is a long run down of people to thank for both the great work in
hive as well as those who supported my involvement. I hope the book
does all your great work justice!

Thank you,
Edward

Hive-trunk-h0.21 - Build # 1708 - Failure

[Apache Jenkins Server]

Changes for Build #1708



2 tests failed.
REGRESSION:  
org.apache.hadoop.hive.metastore.TestMetaStoreEventListener.testListener

Error Message:
java.net.SocketException: Broken pipe

Stack Trace:
org.apache.thrift.transport.TTransportException: java.net.SocketException: 
Broken pipe
at 
org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:147)
at 
org.apache.thrift.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.java:163)
at 
org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:186)
at 
org.apache.thrift.protocol.TBinaryProtocol.writeMessageBegin(TBinaryProtocol.java:92)
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_get_database(ThriftHiveMetastore.java:372)
at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_database(ThriftHiveMetastore.java:364)
at 
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClient.java:705)
at 
org.apache.hadoop.hive.metastore.TestMetaStoreEventListener.testListener(TestMetaStoreEventListener.java:190)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:232)
at junit.framework.TestSuite.run(TestSuite.java:227)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
Caused by: java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at 
org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:145)
... 23 more


REGRESSION:  
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1

Error Message:
Unexpected exception See build/ql/tmp/hive.log, or try "ant test ... 
-Dtest.silent=false" to get more logs.

Stack Trace:
junit.framework.AssertionFailedError: Unexpected exception
See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get 
more logs.
at junit.framework.Assert.fail(Assert.java:47)
at 
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1(TestNegativeCliDriver.java:11512)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:232)
at junit.framework.TestSuite.run(TestSuite.java:227)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)




The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1708)

Status: Failure

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1708/ to 
view the results.

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21 #153

[Apache Jenkins Server]

See 

--
[...truncated 36564 lines...]
[junit] POSTHOOK: query: select count(1) as cnt from testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: 
file:/tmp/jenkins/hive_2012-09-29_13-53-38_326_5503619170406277930/-mr-1
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] Hive history 
file=
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] OK
[junit] Copying file: 

[junit] PREHOOK: query: create table testhivedrivertable (num int)
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: create table testhivedrivertable (num int)
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: load data local inpath 
'
 into table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] Copying data from 

[junit] Loading data to table default.testhivedrivertable
[junit] POSTHOOK: query: load data local inpath 
'
 into table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: select * from testhivedrivertable limit 10
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: 
file:/tmp/jenkins/hive_2012-09-29_13-53-42_607_8961015802259237604/-mr-1
[junit] POSTHOOK: query: select * from testhivedrivertable limit 10
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: 
file:/tmp/jenkins/hive_2012-09-29_13-53-42_607_8961015802259237604/-mr-1
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] Hive history 
file=
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] OK
[junit] PREHOOK: query: create table testhivedrivertable (num int)
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: create table testhivedrivertable (num int)
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] Hive history 
file=
[junit] Hive history 
file=
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: drop table testhivedrivertable
   

[jira] [Created] (HIVE-3516) Fast incremental statistics computation on column in Hive tables

[Shreepadma Venugopalan (JIRA)]

Shreepadma Venugopalan created HIVE-3516:


 Summary: Fast incremental statistics computation on column in Hive 
tables
 Key: HIVE-3516
 URL: https://issues.apache.org/jira/browse/HIVE-3516
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Reporter: Shreepadma Venugopalan
Assignee: Shreepadma Venugopalan


Statistics computed on Hive columns in partition can be rolled up to avoid 
scanning the table again to compute column statistics at the table(global) 
level. While its straightforward to roll up some statistics such as max, min, 
avgcollen, maxcollen etc, rolling up other statistics such as ndv requires 
maintaining intermediate state. This ticket covers the task of a) maintaining 
the necessary intermediate state needed to roll up partition level statistics 
b) detecting that the partition level statistics can be rolled up and actually 
computing table level statistics from partition level statistics.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3516) Fast incremental statistics computation on column in Hive tables

[Shreepadma Venugopalan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466298#comment-13466298
 ] 

Shreepadma Venugopalan commented on HIVE-3516:
--

HIVE-1362 adds support for computing statistics on columns in tables and 
partitions.

> Fast incremental statistics computation on column in Hive tables
> 
>
> Key: HIVE-3516
> URL: https://issues.apache.org/jira/browse/HIVE-3516
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Reporter: Shreepadma Venugopalan
>Assignee: Shreepadma Venugopalan
>
> Statistics computed on Hive columns in partition can be rolled up to avoid 
> scanning the table again to compute column statistics at the table(global) 
> level. While its straightforward to roll up some statistics such as max, min, 
> avgcollen, maxcollen etc, rolling up other statistics such as ndv requires 
> maintaining intermediate state. This ticket covers the task of a) maintaining 
> the necessary intermediate state needed to roll up partition level statistics 
> b) detecting that the partition level statistics can be rolled up and 
> actually computing table level statistics from partition level statistics.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3433) Implement CUBE and ROLLUP operators in Hive

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3433:
-

Issue Type: New Feature  (was: Bug)

> Implement CUBE and ROLLUP operators in Hive
> ---
>
> Key: HIVE-3433
> URL: https://issues.apache.org/jira/browse/HIVE-3433
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Sambavi Muthukrishnan
>Assignee: Namit Jain
> Attachments: hive.3433.1.patch, hive.3433.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3433) Implement CUBE and ROLLUP operators in Hive

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3433:
-

Affects Version/s: (was: 0.9.0)

> Implement CUBE and ROLLUP operators in Hive
> ---
>
> Key: HIVE-3433
> URL: https://issues.apache.org/jira/browse/HIVE-3433
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Sambavi Muthukrishnan
>Assignee: Namit Jain
> Attachments: hive.3433.1.patch, hive.3433.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3433) Implement CUBE and ROLLUP operators in Hive

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3433:
-

Status: Patch Available  (was: Open)

> Implement CUBE and ROLLUP operators in Hive
> ---
>
> Key: HIVE-3433
> URL: https://issues.apache.org/jira/browse/HIVE-3433
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.9.0
>Reporter: Sambavi Muthukrishnan
>Assignee: Namit Jain
> Attachments: hive.3433.1.patch, hive.3433.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3433) Implement CUBE and ROLLUP operators in Hive

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466286#comment-13466286
 ] 

Namit Jain commented on HIVE-3433:
--

All the tests passed.

The current implementation can be optimized.
In this patch, at the time of aggregation, all the values corresponding to the 
grouping sets are passed.
This will increase the data across the map-reduce boundary.
It is still better than the current work-around for the cube and rollup in 
hive, which is to perform
multiple group bys for the same base table.

> Implement CUBE and ROLLUP operators in Hive
> ---
>
> Key: HIVE-3433
> URL: https://issues.apache.org/jira/browse/HIVE-3433
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.9.0
>Reporter: Sambavi Muthukrishnan
>Assignee: Namit Jain
> Attachments: hive.3433.1.patch, hive.3433.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3433) Implement CUBE and ROLLUP operators in Hive

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3433:
-

Attachment: hive.3433.1.patch

> Implement CUBE and ROLLUP operators in Hive
> ---
>
> Key: HIVE-3433
> URL: https://issues.apache.org/jira/browse/HIVE-3433
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.9.0
>Reporter: Sambavi Muthukrishnan
>Assignee: Namit Jain
> Attachments: hive.3433.1.patch, hive.3433.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3493) aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two different ways

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3493:
-

Status: Open  (was: Patch Available)

The code changes look good.
Can you add the test as mentioned in the jira ?

> aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two 
> different ways
> --
>
> Key: HIVE-3493
> URL: https://issues.apache.org/jira/browse/HIVE-3493
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.10.0
>Reporter: Yin Huai
>Assignee: Yin Huai
>Priority: Minor
> Attachments: HIVE-3493.1.patch.txt
>
>
> aggName in 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(String,
>  ArrayList, ASTNode, boolean, boolean) is generated by two 
> different ways. One is String aggName = value.getChild(0).getText(); and 
> another is String aggName = unescapeIdentifier(value.getChild(0).getText());. 
> When a aggregation function is involved in a view, we may get a error.
> You can try the query below (from create_view.q) to replay the error.
> {code:sql}
> set hive.map.aggr=false;
> CREATE TEMPORARY FUNCTION test_max AS
> 'org.apache.hadoop.hive.ql.udf.UDAFTestMax';
> CREATE VIEW view9(m) AS
> SELECT test_max(length(value))
> FROM src;
> DESCRIBE EXTENDED view9;
> DESCRIBE FORMATTED view9;
> SELECT * FROM view9;
> {code}
> Here is the log
> {code}
> 2012-09-20 07:26:15,176 DEBUG exec.FunctionRegistry 
> (FunctionRegistry.java:getGenericUDAFResolver(849)) - Looking up GenericUDAF: 
> `test_max`
> 2012-09-20 07:26:15,181 ERROR ql.Driver (SessionState.java:printError(400)) - 
> FAILED: SemanticException Line 1:33 Function argument type mismatch 
> '`test_max`' in definition of VIEW view9 [
> SELECT `_c0` AS `m` FROM (SELECT `test_max`(length(`src`.`value`))
> FROM `default`.`src`) `view9`
> ] used as view9 at Line 2:14: Looking for UDAF Evaluator"`test_max`" with 
> parameters 
> [org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector@5afa2b2b]
> org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:33 Function 
> argument type mismatch '`test_max`' in definition of VIEW view9 [
> SELECT `_c0` AS `m` FROM (SELECT `test_max`(length(`src`.`value`))
> FROM `default`.`src`) `view9`
> ] used as view9 at Line 2:14: Looking for UDAF Evaluator"`test_max`" with 
> parameters 
> [org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector@5afa2b2b]
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:2394)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanGroupByOperator(SemanticAnalyzer.java:2561)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlan1MR(SemanticAnalyzer.java:3341)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:6140)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6903)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6843)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6864)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6843)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6864)
>   at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7484)
>   at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:245)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:431)
>   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:903)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
>   at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:713)
>   at 
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view(TestCliDriver.java:125)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at junit.framework.TestCase.runTest(TestCase.java:168)
>   at junit.framework.Tes

[jira] [Updated] (HIVE-3501) Track table and keys used in joins and group bys for logging

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3501:
-

Affects Version/s: (was: 0.9.0)
   0.10.0

> Track table and keys used in joins and group bys for logging
> 
>
> Key: HIVE-3501
> URL: https://issues.apache.org/jira/browse/HIVE-3501
> Project: Hive
>  Issue Type: Task
>  Components: Query Processor
>Affects Versions: 0.10.0
>Reporter: Sambavi Muthukrishnan
>Assignee: Sambavi Muthukrishnan
>Priority: Minor
> Attachments: table_access_keys.1.patch, table_access_keys.2.patch
>
>   Original Estimate: 96h
>  Remaining Estimate: 96h
>
> For all operators that could benefit from bucketing, it will be useful to 
> keep track of and log the table names and key column names in order for the 
> operator to be converted to the bucketed version. This task is to track this 
> information for joins and group bys when the keys can be directly mapped back 
> to table scans and columns on that table. This information will be tracked on 
> the QueryPlan object so it is available to any pre/post execution hooks for 
> logging.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3501) Track table and keys used in joins and group bys for logging

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3501:
-

Status: Open  (was: Patch Available)

comments on phabricator

> Track table and keys used in joins and group bys for logging
> 
>
> Key: HIVE-3501
> URL: https://issues.apache.org/jira/browse/HIVE-3501
> Project: Hive
>  Issue Type: Task
>  Components: Query Processor
>Affects Versions: 0.9.0
>Reporter: Sambavi Muthukrishnan
>Assignee: Sambavi Muthukrishnan
>Priority: Minor
> Attachments: table_access_keys.1.patch, table_access_keys.2.patch
>
>   Original Estimate: 96h
>  Remaining Estimate: 96h
>
> For all operators that could benefit from bucketing, it will be useful to 
> keep track of and log the table names and key column names in order for the 
> operator to be converted to the bucketed version. This task is to track this 
> information for joins and group bys when the keys can be directly mapped back 
> to table scans and columns on that table. This information will be tracked on 
> the QueryPlan object so it is available to any pre/post execution hooks for 
> logging.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3515) metadata_export_drop.q causes failure of other tests

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3515:
-

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed. Thanks Ivan

> metadata_export_drop.q causes failure of other tests
> 
>
> Key: HIVE-3515
> URL: https://issues.apache.org/jira/browse/HIVE-3515
> Project: Hive
>  Issue Type: Bug
>  Components: Tests
>Reporter: Ivan Gorbachev
>Assignee: Ivan Gorbachev
> Attachments: jira-3515.1.patch
>
>
> metadata_export_drop.q causes failure of other tests on cleanup stage.
> {quote}
> Exception: java.lang.IllegalArgumentException: java.net.URISyntaxException: 
> Relative path in absolute URI: 
> file:../build/ql/test/data/exports/HIVE-3427/src.2012-09-28-11-38-17
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative 
> path in absolute URI: 
> file:../build/ql/test/data/exports/HIVE-3427/src.2012-09-28-11-38-17
>   at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:845)
>   at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:821)
>   at org.apache.hadoop.hive.ql.QTestUtil.cleanUp(QTestUtil.java:445)
>   at org.apache.hadoop.hive.ql.QTestUtil.shutdown(QTestUtil.java:300)
>   at 
> org.apache.hadoop.hive.cli.TestCliDriver.tearDown(TestCliDriver.java:87)
>   at junit.framework.TestCase.runBare(TestCase.java:140)
>   at junit.framework.TestResult$1.protect(TestResult.java:110)
>   at junit.framework.TestResult.runProtected(TestResult.java:128)
>   at junit.framework.TestResult.run(TestResult.java:113)
>   at junit.framework.TestCase.run(TestCase.java:124)
>   at junit.framework.TestSuite.runTest(TestSuite.java:232)
>   at junit.framework.TestSuite.run(TestSuite.java:227)
>   at 
> org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
>   at 
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: 
> Relative path in absolute URI: 
> file:../build/ql/test/data/exports/HIVE-3427/src.2012-09-28-11-38-17
>   at org.apache.hadoop.fs.Path.initialize(Path.java:140)
>   at org.apache.hadoop.fs.Path.(Path.java:132)
>   at 
> org.apache.hadoop.fs.ProxyFileSystem.swizzleParamPath(ProxyFileSystem.java:56)
>   at org.apache.hadoop.fs.ProxyFileSystem.mkdirs(ProxyFileSystem.java:214)
>   at 
> org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
>   at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1120)
>   at 
> org.apache.hadoop.hive.ql.parse.MetaDataExportListener.export_meta_data(MetaDataExportListener.java:81)
>   at 
> org.apache.hadoop.hive.ql.parse.MetaDataExportListener.onEvent(MetaDataExportListener.java:106)
>   at 
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_core(HiveMetaStore.java:1024)
>   at 
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table(HiveMetaStore.java:1185)
>   at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropTable(HiveMetaStoreClient.java:566)
>   at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:839)
>   ... 17 more
> Caused by: java.net.URISyntaxException: Relative path in absolute URI: 
> file:../build/ql/test/data/exports/HIVE-3427/src.2012-09-28-11-38-17
>   at java.net.URI.checkPath(URI.java:1787)
>   at java.net.URI.(URI.java:735)
>   at org.apache.hadoop.fs.Path.initialize(Path.java:137)
>   ... 28 more
> {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3515) metadata_export_drop.q causes failure of other tests

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466272#comment-13466272
 ] 

Namit Jain commented on HIVE-3515:
--

+1

The change individually looks good.
However, it exposes a bigger issue that the tests have side-affects.
Ideally, QTestUtil should make sure that the tests do not have any 
side-affects, and initialize the
parameters everytime. Can you file a new jira to track that, and give this 
usecase.

> metadata_export_drop.q causes failure of other tests
> 
>
> Key: HIVE-3515
> URL: https://issues.apache.org/jira/browse/HIVE-3515
> Project: Hive
>  Issue Type: Bug
>  Components: Tests
>Reporter: Ivan Gorbachev
>Assignee: Ivan Gorbachev
> Attachments: jira-3515.1.patch
>
>
> metadata_export_drop.q causes failure of other tests on cleanup stage.
> {quote}
> Exception: java.lang.IllegalArgumentException: java.net.URISyntaxException: 
> Relative path in absolute URI: 
> file:../build/ql/test/data/exports/HIVE-3427/src.2012-09-28-11-38-17
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative 
> path in absolute URI: 
> file:../build/ql/test/data/exports/HIVE-3427/src.2012-09-28-11-38-17
>   at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:845)
>   at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:821)
>   at org.apache.hadoop.hive.ql.QTestUtil.cleanUp(QTestUtil.java:445)
>   at org.apache.hadoop.hive.ql.QTestUtil.shutdown(QTestUtil.java:300)
>   at 
> org.apache.hadoop.hive.cli.TestCliDriver.tearDown(TestCliDriver.java:87)
>   at junit.framework.TestCase.runBare(TestCase.java:140)
>   at junit.framework.TestResult$1.protect(TestResult.java:110)
>   at junit.framework.TestResult.runProtected(TestResult.java:128)
>   at junit.framework.TestResult.run(TestResult.java:113)
>   at junit.framework.TestCase.run(TestCase.java:124)
>   at junit.framework.TestSuite.runTest(TestSuite.java:232)
>   at junit.framework.TestSuite.run(TestSuite.java:227)
>   at 
> org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
>   at 
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
>   at 
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: 
> Relative path in absolute URI: 
> file:../build/ql/test/data/exports/HIVE-3427/src.2012-09-28-11-38-17
>   at org.apache.hadoop.fs.Path.initialize(Path.java:140)
>   at org.apache.hadoop.fs.Path.(Path.java:132)
>   at 
> org.apache.hadoop.fs.ProxyFileSystem.swizzleParamPath(ProxyFileSystem.java:56)
>   at org.apache.hadoop.fs.ProxyFileSystem.mkdirs(ProxyFileSystem.java:214)
>   at 
> org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
>   at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1120)
>   at 
> org.apache.hadoop.hive.ql.parse.MetaDataExportListener.export_meta_data(MetaDataExportListener.java:81)
>   at 
> org.apache.hadoop.hive.ql.parse.MetaDataExportListener.onEvent(MetaDataExportListener.java:106)
>   at 
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_core(HiveMetaStore.java:1024)
>   at 
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table(HiveMetaStore.java:1185)
>   at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropTable(HiveMetaStoreClient.java:566)
>   at org.apache.hadoop.hive.ql.metadata.Hive.dropTable(Hive.java:839)
>   ... 17 more
> Caused by: java.net.URISyntaxException: Relative path in absolute URI: 
> file:../build/ql/test/data/exports/HIVE-3427/src.2012-09-28-11-38-17
>   at java.net.URI.checkPath(URI.java:1787)
>   at java.net.URI.(URI.java:735)
>   at org.apache.hadoop.fs.Path.initialize(Path.java:137)
>   ... 28 more
> {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3276) optimize union sub-queries

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3276:
-

Attachment: hive.3276.9.patch

> optimize union sub-queries
> --
>
> Key: HIVE-3276
> URL: https://issues.apache.org/jira/browse/HIVE-3276
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: HIVE-3276.1.patch, hive.3276.2.patch, hive.3276.3.patch, 
> hive.3276.4.patch, hive.3276.5.patch, hive.3276.6.patch, hive.3276.7.patch, 
> hive.3276.8.patch, hive.3276.9.patch
>
>
> It might be a good idea to optimize simple union queries containing 
> map-reduce jobs in at least one of the sub-qeuries.
> For eg:
> a query like:
> insert overwrite table T1 partition P1
> select * from 
> (
>   subq1
> union all
>   subq2
> ) u;
> today creates 3 map-reduce jobs, one for subq1, another for subq2 and 
> the final one for the union. 
> It might be a good idea to optimize this. Instead of creating the union 
> task, it might be simpler to create a move task (or something like a move
> task), where the outputs of the two sub-queries will be moved to the final 
> directory. This can easily extend to more than 2 sub-queries in the union.
> This is very useful if there is a select * followed by filesink after the
> union. This can be independently useful, and also be used to optimize the
> skewed joins https://cwiki.apache.org/Hive/skewed-join-optimization.html.
> If there is a select, filter between the union and the filesink, the select
> and the filter can be moved before the union, and the follow-up job can
> still be removed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #153

[Apache Jenkins Server]

See 


--
[...truncated 10215 lines...]
 [echo] Project: odbc
 [copy] Warning: 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/odbc/src/conf
 does not exist.

ivy-resolve-test:
 [echo] Project: odbc

ivy-retrieve-test:
 [echo] Project: odbc

compile-test:
 [echo] Project: odbc

create-dirs:
 [echo] Project: serde
 [copy] Warning: 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/serde/src/test/resources
 does not exist.

init:
 [echo] Project: serde

ivy-init-settings:
 [echo] Project: serde

ivy-resolve:
 [echo] Project: serde
[ivy:resolve] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml
[ivy:report] Processing 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/resolution-cache/org.apache.hive-hive-serde-default.xml
 to 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/report/org.apache.hive-hive-serde-default.html

ivy-retrieve:
 [echo] Project: serde

dynamic-serde:

compile:
 [echo] Project: serde

ivy-resolve-test:
 [echo] Project: serde

ivy-retrieve-test:
 [echo] Project: serde

compile-test:
 [echo] Project: serde
[javac] Compiling 26 source files to 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/serde/test/classes
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.

create-dirs:
 [echo] Project: service
 [copy] Warning: 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/service/src/test/resources
 does not exist.

init:
 [echo] Project: service

ivy-init-settings:
 [echo] Project: service

ivy-resolve:
 [echo] Project: service
[ivy:resolve] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml
[ivy:report] Processing 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/resolution-cache/org.apache.hive-hive-service-default.xml
 to 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/report/org.apache.hive-hive-service-default.html

ivy-retrieve:
 [echo] Project: service

compile:
 [echo] Project: service

ivy-resolve-test:
 [echo] Project: service

ivy-retrieve-test:
 [echo] Project: service

compile-test:
 [echo] Project: service
[javac] Compiling 2 source files to 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/service/test/classes

test:
 [echo] Project: hive

test-shims:
 [echo] Project: hive

test-conditions:
 [echo] Project: shims

gen-test:
 [echo] Project: shims

create-dirs:
 [echo] Project: shims
 [copy] Warning: 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/test/resources
 does not exist.

init:
 [echo] Project: shims

ivy-init-settings:
 [echo] Project: shims

ivy-resolve:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml
[ivy:report] Processing 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/resolution-cache/org.apache.hive-hive-shims-default.xml
 to 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/report/org.apache.hive-hive-shims-default.html

ivy-retrieve:
 [echo] Project: shims

compile:
 [echo] Project: shims
 [echo] Building shims 0.20

build_shims:
 [echo] Project: shims
 [echo] Compiling 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.20/java
 against hadoop 0.20.2 
(/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/hadoopcore/hadoop-0.20.2)

ivy-init-settings:
 [echo] Project: shims

ivy-resolve-hadoop-shim:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml

ivy-retrieve-hadoop-shim:
 [echo] Project: shims
 [echo] Building shims 0.20S

build_shims:
 [echo] Project: shims
 [echo] Compiling 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common/ja

[jira] [Commented] (HIVE-3499) add tests to use bucketing metadata for partitions

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466176#comment-13466176
 ] 

Namit Jain commented on HIVE-3499:
--

addressed comments

> add tests to use bucketing metadata for partitions
> --
>
> Key: HIVE-3499
> URL: https://issues.apache.org/jira/browse/HIVE-3499
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3499.1.patch
>
>
> What happens if the bucketing metadata is different for different partitions ?
> I dont mean, different number of buckets, but what happens if the partitions 
> are 
> bucketed on different keys.
> Do we get wrong results ?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira