date:20120210

[jira] [Updated] (HIVE-2612) support hive table/partitions exists in more than one region

2012-02-10 Thread Namit Jain (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-2612:
-

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed. Thanks Kevin and Yongqiang

> support hive table/partitions exists in more than one region
> 
>
> Key: HIVE-2612
> URL: https://issues.apache.org/jira/browse/HIVE-2612
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore
>Reporter: He Yongqiang
>Assignee: Kevin Wilfong
> Attachments: HIVE-2612.1.patch, HIVE-2612.2.patch.txt, 
> HIVE-2612.3.patch.txt, HIVE-2612.4.patch.txt, HIVE-2612.6.patch.txt, 
> HIVE-2612.7.patch.txt, HIVE-2612.8.patch.txt, HIVE-2612.D1569.1.patch, 
> HIVE-2612.D1569.2.patch, HIVE-2612.D1569.3.patch, HIVE-2612.D1569.4.patch, 
> HIVE-2612.D1569.5.patch, HIVE-2612.D1569.6.patch, HIVE-2612.D1569.7.patch, 
> HIVE-2612.D1707.1.patch, hive.2612.5.patch
>
>
> 1) add region object into hive metastore
> 2) each partition/table has a primary region and a list of living regions, 
> and also data location in each region

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-2800) NPE in "create index" without comment clause in external metastore

2012-02-10 Thread Daniel Dai (Created) (JIRA)

NPE in "create index" without comment clause in external metastore
--

 Key: HIVE-2800
 URL: https://issues.apache.org/jira/browse/HIVE-2800
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.8.1, 0.9.0
Reporter: Daniel Dai
Priority: Minor


This happens only when using external metastore (with --hiveconf 
hive.metastore.uris=thrift://localhost:8088 --hiveconf 
hive.metastore.local=false). Also if I gave a comment in the statement, this 
exception go away.

Here is the statement:
create index test111 on table hcat_test(name) as 'compact' with deferred 
rebuild;

Here is the stack:
2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - 
FAILED: Error in metadata: java.lang.NullPointerException
org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
at org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
at 
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
at 
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.NullPointerException
at 
org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
at 
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
... 17 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (HIVE-2785) support use region

2012-02-10 Thread Namit Jain (Issue Comment Edited) (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205700#comment-13205700
 ] 

Namit Jain edited comment on HIVE-2785 at 2/11/12 12:16 AM:


The semantics are as follows:

use physicalRegion;

Use the default region based on the primary region of the table being queried.
Choose the jobtracker, dfs from the region provided.

use physicalRegion ;

Use the region specified.

The query should only succeed if all the partitions of the table are present in 
the primary region


  was (Author: namit):
The semantics are as follows:

use region;

Use the default region based on the primary region of the table being queried.
Choose the jobtracker, dfs from the region provided.

use region ;

Use the region specified.

The query should only succeed if all the partitions of the table are present in 
the primary region

  
> support use region
> --
>
> Key: HIVE-2785
> URL: https://issues.apache.org/jira/browse/HIVE-2785
> Project: Hive
>  Issue Type: New Feature
>Reporter: Namit Jain
>Assignee: Namit Jain
>
> use physicalRegion;
> use physicalRegion ;
> should be supported

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2785) support use region

2012-02-10 Thread Namit Jain (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-2785:
-

Description: 
use physicalRegion;

use physicalRegion ;

should be supported

  was:
use region;

use region ;

should be supported


> support use region
> --
>
> Key: HIVE-2785
> URL: https://issues.apache.org/jira/browse/HIVE-2785
> Project: Hive
>  Issue Type: New Feature
>Reporter: Namit Jain
>Assignee: Namit Jain
>
> use physicalRegion;
> use physicalRegion ;
> should be supported

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2612) support hive table/partitions exists in more than one region

2012-02-10 Thread Kevin Wilfong (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2612:


Attachment: HIVE-2612.8.patch.txt

> support hive table/partitions exists in more than one region
> 
>
> Key: HIVE-2612
> URL: https://issues.apache.org/jira/browse/HIVE-2612
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore
>Reporter: He Yongqiang
>Assignee: Kevin Wilfong
> Attachments: HIVE-2612.1.patch, HIVE-2612.2.patch.txt, 
> HIVE-2612.3.patch.txt, HIVE-2612.4.patch.txt, HIVE-2612.6.patch.txt, 
> HIVE-2612.7.patch.txt, HIVE-2612.8.patch.txt, HIVE-2612.D1569.1.patch, 
> HIVE-2612.D1569.2.patch, HIVE-2612.D1569.3.patch, HIVE-2612.D1569.4.patch, 
> HIVE-2612.D1569.5.patch, HIVE-2612.D1569.6.patch, HIVE-2612.D1569.7.patch, 
> HIVE-2612.D1707.1.patch, hive.2612.5.patch
>
>
> 1) add region object into hive metastore
> 2) each partition/table has a primary region and a list of living regions, 
> and also data location in each region

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-2799) change the following thrift apis to add a region

2012-02-10 Thread Namit Jain (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain reassigned HIVE-2799:


Assignee: Kevin Wilfong  (was: Namit Jain)

> change the following thrift apis to add a region
> 
>
> Key: HIVE-2799
> URL: https://issues.apache.org/jira/browse/HIVE-2799
> Project: Hive
>  Issue Type: New Feature
>Reporter: Namit Jain
>Assignee: Kevin Wilfong
>
>  list get_tables(1: string db_name, 2: string pattern) throws (1: 
> MetaException o1)
>   list get_all_tables(1: string db_name) throws (1: MetaException o1)
>   Table get_table(1:string dbname, 2:string tbl_name)
>throws (1:MetaException o1, 2:NoSuchObjectException o2)
>   list get_table_objects_by_name(1:string dbname, 2:list 
> tbl_names)
>throws (1:MetaException o1, 2:InvalidOperationException o2, 
> 3:UnknownDBException o3)
>   list get_table_names_by_filter(1:string dbname, 2:string filter, 
> 3:i16 max_tables=-1)
>throws (1:MetaException o1, 
> 2:InvalidOperationException o2, 3:UnknownDBException o3)
>   Partition add_partition(1:Partition new_part)
>throws(1:InvalidObjectException o1, 
> 2:AlreadyExistsException o2, 3:MetaException o3)
>   i32 add_partitions(1:list new_parts)
>throws(1:InvalidObjectException o1, 
> 2:AlreadyExistsException o2, 3:MetaException o3)
>   Partition append_partition(1:string db_name, 2:string tbl_name, 
> 3:list part_vals)
>throws (1:InvalidObjectException o1, 
> 2:AlreadyExistsException o2, 3:MetaException o3)
>   Partition append_partition_by_name(1:string db_name, 2:string tbl_name, 
> 3:string part_name)
>throws (1:InvalidObjectException o1, 
> 2:AlreadyExistsException o2, 3:MetaException o3)
>   bool drop_partition(1:string db_name, 2:string tbl_name, 3:list 
> part_vals, 4:bool deleteData)
>throws(1:NoSuchObjectException o1, 2:MetaException o2)
>   bool drop_partition_by_name(1:string db_name, 2:string tbl_name, 3:string 
> part_name, 4:bool deleteData)
>throws(1:NoSuchObjectException o1, 2:MetaException o2)
>   Partition get_partition(1:string db_name, 2:string tbl_name, 3:list 
> part_vals)
>throws(1:MetaException o1, 2:NoSuchObjectException o2)
>   Partition get_partition_with_auth(1:string db_name, 2:string tbl_name, 
> 3:list part_vals,
>   4: string user_name, 5: list group_names) 
> throws(1:MetaException o1, 2:NoSuchObjectException o2)
>   Partition get_partition_by_name(1:string db_name 2:string tbl_name, 
> 3:string part_name)
>throws(1:MetaException o1, 2:NoSuchObjectException o2)
>   list get_partitions(1:string db_name, 2:string tbl_name, 3:i16 
> max_parts=-1)
>throws(1:NoSuchObjectException o1, 2:MetaException o2)
>   list get_partitions_with_auth(1:string db_name, 2:string 
> tbl_name, 3:i16 max_parts=-1,
>  4: string user_name, 5: list group_names) 
> throws(1:NoSuchObjectException o1, 2:MetaException o2)
>   list get_partition_names(1:string db_name, 2:string tbl_name, 3:i16 
> max_parts=-1)
>throws(1:MetaException o2)
>   list get_partitions_ps(1:string db_name 2:string tbl_name
> 3:list part_vals, 4:i16 max_parts=-1)
>throws(1:MetaException o1, 2:NoSuchObjectException o2)
>   list get_partitions_ps_with_auth(1:string db_name, 2:string 
> tbl_name, 3:list part_vals, 4:i16 max_parts=-1,
>  5: string user_name, 6: list group_names) 
> throws(1:NoSuchObjectException o1, 2:MetaException o2)
>   list get_partition_names_ps(1:string db_name,
> 2:string tbl_name, 3:list part_vals, 4:i16 max_parts=-1)
>throws(1:MetaException o1, 2:NoSuchObjectException o2)
>   list get_partitions_by_filter(1:string db_name 2:string tbl_name
> 3:string filter, 4:i16 max_parts=-1)
>throws(1:MetaException o1, 2:NoSuchObjectException o2)
>   list get_partitions_by_names(1:string db_name 2:string tbl_name 
> 3:list names)
>throws(1:MetaException o1, 2:NoSuchObjectException o2)
>   bool drop_index_by_name(1:string db_name, 2:string tbl_name, 3:string 
> index_name, 4:bool deleteData)
>throws(1:NoSuchObjectException o1, 2:MetaException o2)
>   Index get_index_by_name(1:string db_name 2:string tbl_name, 3:string 
> index_name)
>throws(1:MetaException o1, 2:NoSuchObjectException o2)
>   list get_indexes(1:string db_name, 2:string tbl_name, 3:i16 
> max_indexes=-1)
>throws(1:NoSuchObjectException o1, 2:MetaException o2)
>   list get_index_names(1:string db_name, 2:string tbl_name, 3:i16 
> max_indexes=-1)
>

[jira] [Updated] (HIVE-2612) support hive table/partitions exists in more than one region

2012-02-10 Thread Phabricator (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-2612:
--

Attachment: HIVE-2612.D1707.1.patch

kevinwilfong requested code review of "HIVE-2612 [jira] support hive 
table/partitions coexists in more than one region".
Reviewers: JIRA, njain

  https://issues.apache.org/jira/browse/HIVE-2612

  1) add region object into hive metastore
  2) each partition/table has a primary region and a list of living regions, 
and also data location in each region

  This is just an initial patch, it sets up the metastore so that we can begin 
to support multiple regions, it simply adds a default primary region to each 
table/partition and adds an empty list of secondary regions.

TEST PLAN
  Ran most of the unit tests.

REVISION DETAIL
  https://reviews.facebook.net/D1707

AFFECTED FILES
  conf/hive-default.xml.template
  metastore/scripts/upgrade/derby/010-HIVE-2612.derby.sql
  metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql
  metastore/scripts/upgrade/derby/hive-schema-0.10.0.derby.sql
  metastore/scripts/upgrade/mysql/010-HIVE-2612.mysql.sql
  metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql
  metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql
  metastore/scripts/upgrade/postgres/010-HIVE-2612.postgres.sql
  metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
  metastore/src/model/package.jdo
  
metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java
  
metastore/src/model/org/apache/hadoop/hive/metastore/model/MRegionStorageDescriptor.java
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
  metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
  metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py
  metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
  metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
  metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
  metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/RegionStorageDescriptor.java
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java
  metastore/src/gen/thrift/gen-php/hive_metastore_constants.php
  metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php
  metastore/src/gen/thrift/gen-php/hive_metastore_types.php
  metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php
  metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php
  metastore/if/hive_metastore.thrift
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
  contrib/src/test/results/clientnegative/serde_regex.q.out
  contrib/src/test/results/clientpositive/serde_regex.q.out
  contrib/src/test/results/clientpositive/fileformat_base64.q.out
  ql/src/test/results/clientpositive/inputddl3.q.out
  ql/src/test/results/clientpositive/rcfile_createas1.q.out
  ql/src/test/results/clientpositive/input15.q.out
  ql/src/test/results/clientpositive/union25.q.out
  ql/src/test/results/clientpositive/smb_mapjoin9.q.out
  ql/src/test/results/clientpositive/inputddl1.q.out
  ql/src/test/results/clientpositive/symlink_text_input_format.q.out
  ql/src/test/results/clientpositive/create_union_table.q.out
  ql/src/test/results/clientpositive/ctas.q.out
  ql/src/test/results/clientpositive/merge3.q.out
  ql/src/test/results/clientpositive/fileformat_text.q.out
  ql/src/test/results/clientpositive/inputddl2.q.out
  ql/src/test/results/clientpositive/fileformat_sequencefile.q.out
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java
  ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java
  ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java
  ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/3639/

Tip: u

[jira] [Updated] (HIVE-2612) support hive table/partitions exists in more than one region

2012-02-10 Thread Kevin Wilfong (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2612:


Status: Patch Available  (was: Open)

> support hive table/partitions exists in more than one region
> 
>
> Key: HIVE-2612
> URL: https://issues.apache.org/jira/browse/HIVE-2612
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore
>Reporter: He Yongqiang
>Assignee: Kevin Wilfong
> Attachments: HIVE-2612.1.patch, HIVE-2612.2.patch.txt, 
> HIVE-2612.3.patch.txt, HIVE-2612.4.patch.txt, HIVE-2612.6.patch.txt, 
> HIVE-2612.7.patch.txt, HIVE-2612.D1569.1.patch, HIVE-2612.D1569.2.patch, 
> HIVE-2612.D1569.3.patch, HIVE-2612.D1569.4.patch, HIVE-2612.D1569.5.patch, 
> HIVE-2612.D1569.6.patch, HIVE-2612.D1569.7.patch, HIVE-2612.D1707.1.patch, 
> hive.2612.5.patch
>
>
> 1) add region object into hive metastore
> 2) each partition/table has a primary region and a list of living regions, 
> and also data location in each region

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-2612) support hive table/partitions exists in more than one region

2012-02-10 Thread Kevin Wilfong (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong reassigned HIVE-2612:
---

Assignee: Kevin Wilfong  (was: Namit Jain)

> support hive table/partitions exists in more than one region
> 
>
> Key: HIVE-2612
> URL: https://issues.apache.org/jira/browse/HIVE-2612
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore
>Reporter: He Yongqiang
>Assignee: Kevin Wilfong
> Attachments: HIVE-2612.1.patch, HIVE-2612.2.patch.txt, 
> HIVE-2612.3.patch.txt, HIVE-2612.4.patch.txt, HIVE-2612.6.patch.txt, 
> HIVE-2612.7.patch.txt, HIVE-2612.D1569.1.patch, HIVE-2612.D1569.2.patch, 
> HIVE-2612.D1569.3.patch, HIVE-2612.D1569.4.patch, HIVE-2612.D1569.5.patch, 
> HIVE-2612.D1569.6.patch, HIVE-2612.D1569.7.patch, hive.2612.5.patch
>
>
> 1) add region object into hive metastore
> 2) each partition/table has a primary region and a list of living regions, 
> and also data location in each region

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2302) Allow grant privileges on granting privileges.

2012-02-10 Thread Carl Steinbach (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-2302:
-

Component/s: Authorization

> Allow grant privileges on granting privileges.
> --
>
> Key: HIVE-2302
> URL: https://issues.apache.org/jira/browse/HIVE-2302
> Project: Hive
>  Issue Type: Improvement
>  Components: Authorization, Security
>Reporter: Guy Doulberg
>Assignee: Jakob Homan
>
> Today any user can grant him and any other users privileges on schemas and 
> tables.
> This way the administrator can not be sure that the rules he had apply are 
> fulfilled.
>   

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2612) support hive table/partitions exists in more than one region

2012-02-10 Thread Kevin Wilfong (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205793#comment-13205793
 ] 

Kevin Wilfong commented on HIVE-2612:
-

Updated the patch with some small changes:
cleaned up a few remaining mentions of cluster instead of region
always check/set the primary region name for a table in the metastore

> support hive table/partitions exists in more than one region
> 
>
> Key: HIVE-2612
> URL: https://issues.apache.org/jira/browse/HIVE-2612
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore
>Reporter: He Yongqiang
>Assignee: Namit Jain
> Attachments: HIVE-2612.1.patch, HIVE-2612.2.patch.txt, 
> HIVE-2612.3.patch.txt, HIVE-2612.4.patch.txt, HIVE-2612.6.patch.txt, 
> HIVE-2612.7.patch.txt, HIVE-2612.D1569.1.patch, HIVE-2612.D1569.2.patch, 
> HIVE-2612.D1569.3.patch, HIVE-2612.D1569.4.patch, HIVE-2612.D1569.5.patch, 
> HIVE-2612.D1569.6.patch, HIVE-2612.D1569.7.patch, hive.2612.5.patch
>
>
> 1) add region object into hive metastore
> 2) each partition/table has a primary region and a list of living regions, 
> and also data location in each region

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2612) support hive table/partitions exists in more than one region

2012-02-10 Thread Kevin Wilfong (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2612:


Attachment: HIVE-2612.7.patch.txt

> support hive table/partitions exists in more than one region
> 
>
> Key: HIVE-2612
> URL: https://issues.apache.org/jira/browse/HIVE-2612
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore
>Reporter: He Yongqiang
>Assignee: Namit Jain
> Attachments: HIVE-2612.1.patch, HIVE-2612.2.patch.txt, 
> HIVE-2612.3.patch.txt, HIVE-2612.4.patch.txt, HIVE-2612.6.patch.txt, 
> HIVE-2612.7.patch.txt, HIVE-2612.D1569.1.patch, HIVE-2612.D1569.2.patch, 
> HIVE-2612.D1569.3.patch, HIVE-2612.D1569.4.patch, HIVE-2612.D1569.5.patch, 
> HIVE-2612.D1569.6.patch, HIVE-2612.D1569.7.patch, hive.2612.5.patch
>
>
> 1) add region object into hive metastore
> 2) each partition/table has a primary region and a list of living regions, 
> and also data location in each region

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2612) support hive table/partitions exists in more than one region

2012-02-10 Thread Kevin Wilfong (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2612:


Attachment: HIVE-2612.6.patch.txt

> support hive table/partitions exists in more than one region
> 
>
> Key: HIVE-2612
> URL: https://issues.apache.org/jira/browse/HIVE-2612
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore
>Reporter: He Yongqiang
>Assignee: Namit Jain
> Attachments: HIVE-2612.1.patch, HIVE-2612.2.patch.txt, 
> HIVE-2612.3.patch.txt, HIVE-2612.4.patch.txt, HIVE-2612.6.patch.txt, 
> HIVE-2612.D1569.1.patch, HIVE-2612.D1569.2.patch, HIVE-2612.D1569.3.patch, 
> HIVE-2612.D1569.4.patch, HIVE-2612.D1569.5.patch, HIVE-2612.D1569.6.patch, 
> HIVE-2612.D1569.7.patch, hive.2612.5.patch
>
>
> 1) add region object into hive metastore
> 2) each partition/table has a primary region and a list of living regions, 
> and also data location in each region

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2612) support hive table/partitions exists in more than one region

2012-02-10 Thread Kevin Wilfong (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205762#comment-13205762
 ] 

Kevin Wilfong commented on HIVE-2612:
-

Attached a patch containing the updated metastore upgrade scripts.

> support hive table/partitions exists in more than one region
> 
>
> Key: HIVE-2612
> URL: https://issues.apache.org/jira/browse/HIVE-2612
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore
>Reporter: He Yongqiang
>Assignee: Namit Jain
> Attachments: HIVE-2612.1.patch, HIVE-2612.2.patch.txt, 
> HIVE-2612.3.patch.txt, HIVE-2612.4.patch.txt, HIVE-2612.6.patch.txt, 
> HIVE-2612.D1569.1.patch, HIVE-2612.D1569.2.patch, HIVE-2612.D1569.3.patch, 
> HIVE-2612.D1569.4.patch, HIVE-2612.D1569.5.patch, HIVE-2612.D1569.6.patch, 
> HIVE-2612.D1569.7.patch, hive.2612.5.patch
>
>
> 1) add region object into hive metastore
> 2) each partition/table has a primary region and a list of living regions, 
> and also data location in each region

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2206) add a new optimizer for query correlation discovery and optimization

2012-02-10 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205742#comment-13205742
 ] 

jirapos...@reviews.apache.org commented on HIVE-2206:
-


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2001/
---

(Updated 2012-02-10 20:49:01.177796)


Review request for hive.


Changes
---

updated patch on revision 1237253. Will generate the patch based on the latest 
trunk latter. 


Summary
---

This optimizer exploits intra-query correlations and merges multiple correlated 
MapReduce jobs into one jobs.


This addresses bug HIVE-2206.
https://issues.apache.org/jira/browse/HIVE-2206


Diffs (updated)
-

  trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/BaseReduceSinkOperator.java 
PRE-CREATION 
  
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationCompositeOperator.java
 PRE-CREATION 
  
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationLocalSimulativeReduceSinkOperator.java
 PRE-CREATION 
  
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationReducerDispatchOperator.java
 PRE-CREATION 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecReducer.java 1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java 1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 
1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 
1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java 
1237326 
  
trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizer.java 
PRE-CREATION 
  
trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizerUtils.java
 PRE-CREATION 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/BaseReduceSinkDesc.java 
PRE-CREATION 
  
trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CorrelationCompositeDesc.java 
PRE-CREATION 
  
trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CorrelationLocalSimulativeReduceSinkDesc.java
 PRE-CREATION 
  
trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CorrelationReducerDispatchDesc.java
 PRE-CREATION 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java 1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableScanDesc.java 1237326 
  trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/TestExecDriver.java 1237326 
  trunk/ql/src/test/results/compiler/plan/groupby1.q.xml 1237326 
  trunk/ql/src/test/results/compiler/plan/groupby2.q.xml 1237326 
  trunk/ql/src/test/results/compiler/plan/groupby3.q.xml 1237326 
  trunk/ql/src/test/results/compiler/plan/groupby5.q.xml 1237326 

Diff: https://reviews.apache.org/r/2001/diff


Testing
---


Thanks,

Yin



> add a new optimizer for query correlation discovery and optimization
> 
>
> Key: HIVE-2206
> URL: https://issues.apache.org/jira/browse/HIVE-2206
> Project: Hive
>  Issue Type: New Feature
>Reporter: He Yongqiang
>Assignee: Yin Huai
> Attachments: HIVE-2206.1.patch.txt, HIVE-2206.2.patch.txt, 
> HIVE-2206.3.patch.txt, HIVE-2206.4.patch.txt, HIVE-2206.5-1.patch.txt, 
> HIVE-2206.5.patch.txt, HIVE-2206.6.patch.txt, HIVE-2206.7.patch.txt, 
> HIVE-2206.8-r1237253.patch.txt, HIVE-2206.8.r1224646.patch.txt, 
> YSmartPatchForHive.patch, testQueries.2.q
>
>
> reference:
> http://www.cse.ohio-state.edu/hpcs/WWW/HTML/publications/papers/TR-11-7.pdf

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-02-10 Thread Yin Huai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2001/
---

(Updated 2012-02-10 20:49:01.177796)


Review request for hive.


Changes
---

updated patch on revision 1237253. Will generate the patch based on the latest 
trunk latter. 


Summary
---

This optimizer exploits intra-query correlations and merges multiple correlated 
MapReduce jobs into one jobs.


This addresses bug HIVE-2206.
https://issues.apache.org/jira/browse/HIVE-2206


Diffs (updated)
-

  trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/BaseReduceSinkOperator.java 
PRE-CREATION 
  
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationCompositeOperator.java
 PRE-CREATION 
  
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationLocalSimulativeReduceSinkOperator.java
 PRE-CREATION 
  
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationReducerDispatchOperator.java
 PRE-CREATION 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecReducer.java 1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java 1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 
1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 
1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java 
1237326 
  
trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizer.java 
PRE-CREATION 
  
trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizerUtils.java
 PRE-CREATION 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/BaseReduceSinkDesc.java 
PRE-CREATION 
  
trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CorrelationCompositeDesc.java 
PRE-CREATION 
  
trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CorrelationLocalSimulativeReduceSinkDesc.java
 PRE-CREATION 
  
trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CorrelationReducerDispatchDesc.java
 PRE-CREATION 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java 1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java 1237326 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableScanDesc.java 1237326 
  trunk/ql/src/test/org/apache/hadoop/hive/ql/exec/TestExecDriver.java 1237326 
  trunk/ql/src/test/results/compiler/plan/groupby1.q.xml 1237326 
  trunk/ql/src/test/results/compiler/plan/groupby2.q.xml 1237326 
  trunk/ql/src/test/results/compiler/plan/groupby3.q.xml 1237326 
  trunk/ql/src/test/results/compiler/plan/groupby5.q.xml 1237326 

Diff: https://reviews.apache.org/r/2001/diff


Testing
---


Thanks,

Yin

[jira] [Commented] (HIVE-2604) Add UberCompressor Serde/Codec to contrib which allows per-column compression strategies

2012-02-10 Thread He Yongqiang (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205731#comment-13205731
 ] 

He Yongqiang commented on HIVE-2604:


+1, will commit after tests pass

> Add UberCompressor Serde/Codec to contrib which allows per-column compression 
> strategies
> 
>
> Key: HIVE-2604
> URL: https://issues.apache.org/jira/browse/HIVE-2604
> Project: Hive
>  Issue Type: Sub-task
>  Components: Contrib
>Reporter: Krishna Kumar
>Assignee: Krishna Kumar
> Attachments: HIVE-2604.D1011.1.patch, HIVE-2604.D1011.2.patch, 
> HIVE-2604.v0.patch, HIVE-2604.v1.patch, HIVE-2604.v2.patch
>
>
> The strategies supported are
> 1. using a specified codec on the column
> 2. using a specific codec on the column which is serialized via a specific 
> serde
> 3. using a specific "TypeSpecificCompressor" instance

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2623) Add Integer type compressors

2012-02-10 Thread He Yongqiang (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205722#comment-13205722
 ] 

He Yongqiang commented on HIVE-2623:


Krishna Kumar, this is awesome number. Do you have some CPU information about 
this? How much more CPU it requires for compression and decompression?

> Add Integer type compressors
> 
>
> Key: HIVE-2623
> URL: https://issues.apache.org/jira/browse/HIVE-2623
> Project: Hive
>  Issue Type: Sub-task
>  Components: Contrib
>Reporter: Krishna Kumar
>Assignee: Krishna Kumar
>Priority: Minor
> Attachments: HIVE-2623.v0.patch, HIVE-2623.v1.patch, 
> HIVE-2623.v2.patch, data.tar.gz
>
>
> Type-specific compressors for integers.
> Starting with elias gamma which prefers small values as per a power-law like 
> distribution. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-2799) change the following thrift apis to add a region

2012-02-10 Thread Namit Jain (Created) (JIRA)

change the following thrift apis to add a region


 Key: HIVE-2799
 URL: https://issues.apache.org/jira/browse/HIVE-2799
 Project: Hive
  Issue Type: New Feature
Reporter: Namit Jain
Assignee: Namit Jain


 list get_tables(1: string db_name, 2: string pattern) throws (1: 
MetaException o1)
  list get_all_tables(1: string db_name) throws (1: MetaException o1)
  Table get_table(1:string dbname, 2:string tbl_name)
   throws (1:MetaException o1, 2:NoSuchObjectException o2)
  list get_table_objects_by_name(1:string dbname, 2:list 
tbl_names)
   throws (1:MetaException o1, 2:InvalidOperationException o2, 
3:UnknownDBException o3)
  list get_table_names_by_filter(1:string dbname, 2:string filter, 
3:i16 max_tables=-1)
   throws (1:MetaException o1, 2:InvalidOperationException 
o2, 3:UnknownDBException o3)
  Partition add_partition(1:Partition new_part)
   throws(1:InvalidObjectException o1, 
2:AlreadyExistsException o2, 3:MetaException o3)
  i32 add_partitions(1:list new_parts)
   throws(1:InvalidObjectException o1, 
2:AlreadyExistsException o2, 3:MetaException o3)
  Partition append_partition(1:string db_name, 2:string tbl_name, 
3:list part_vals)
   throws (1:InvalidObjectException o1, 
2:AlreadyExistsException o2, 3:MetaException o3)
  Partition append_partition_by_name(1:string db_name, 2:string tbl_name, 
3:string part_name)
   throws (1:InvalidObjectException o1, 
2:AlreadyExistsException o2, 3:MetaException o3)
  bool drop_partition(1:string db_name, 2:string tbl_name, 3:list 
part_vals, 4:bool deleteData)
   throws(1:NoSuchObjectException o1, 2:MetaException o2)
  bool drop_partition_by_name(1:string db_name, 2:string tbl_name, 3:string 
part_name, 4:bool deleteData)
   throws(1:NoSuchObjectException o1, 2:MetaException o2)
  Partition get_partition(1:string db_name, 2:string tbl_name, 3:list 
part_vals)
   throws(1:MetaException o1, 2:NoSuchObjectException o2)

  Partition get_partition_with_auth(1:string db_name, 2:string tbl_name, 
3:list part_vals,
  4: string user_name, 5: list group_names) throws(1:MetaException 
o1, 2:NoSuchObjectException o2)

  Partition get_partition_by_name(1:string db_name 2:string tbl_name, 3:string 
part_name)
   throws(1:MetaException o1, 2:NoSuchObjectException o2)
  list get_partitions(1:string db_name, 2:string tbl_name, 3:i16 
max_parts=-1)
   throws(1:NoSuchObjectException o1, 2:MetaException o2)
  list get_partitions_with_auth(1:string db_name, 2:string tbl_name, 
3:i16 max_parts=-1,
 4: string user_name, 5: list group_names) 
throws(1:NoSuchObjectException o1, 2:MetaException o2)

  list get_partition_names(1:string db_name, 2:string tbl_name, 3:i16 
max_parts=-1)
   throws(1:MetaException o2)
  list get_partitions_ps(1:string db_name 2:string tbl_name
3:list part_vals, 4:i16 max_parts=-1)
   throws(1:MetaException o1, 2:NoSuchObjectException o2)
  list get_partitions_ps_with_auth(1:string db_name, 2:string 
tbl_name, 3:list part_vals, 4:i16 max_parts=-1,
 5: string user_name, 6: list group_names) 
throws(1:NoSuchObjectException o1, 2:MetaException o2)

  list get_partition_names_ps(1:string db_name,
2:string tbl_name, 3:list part_vals, 4:i16 max_parts=-1)
   throws(1:MetaException o1, 2:NoSuchObjectException o2)
  list get_partitions_by_filter(1:string db_name 2:string tbl_name
3:string filter, 4:i16 max_parts=-1)
   throws(1:MetaException o1, 2:NoSuchObjectException o2)
  list get_partitions_by_names(1:string db_name 2:string tbl_name 
3:list names)
   throws(1:MetaException o1, 2:NoSuchObjectException o2)
  bool drop_index_by_name(1:string db_name, 2:string tbl_name, 3:string 
index_name, 4:bool deleteData)
   throws(1:NoSuchObjectException o1, 2:MetaException o2)
  Index get_index_by_name(1:string db_name 2:string tbl_name, 3:string 
index_name)
   throws(1:MetaException o1, 2:NoSuchObjectException o2)

  list get_indexes(1:string db_name, 2:string tbl_name, 3:i16 
max_indexes=-1)
   throws(1:NoSuchObjectException o1, 2:MetaException o2)
  list get_index_names(1:string db_name, 2:string tbl_name, 3:i16 
max_indexes=-1)
   throws(1:MetaException o2)


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1253 - Still Failing

2012-02-10 Thread Apache Jenkins Server

Changes for Build #1209

Changes for Build #1210
[namit] HIVE-2674 get_partitions_ps throws TApplicationException if table 
doesn't
exist (Kevin Wilfong via namit)


Changes for Build #1211
[cws] HIVE-2203. Extend concat_ws() UDF to support arrays of strings (Zhenxiao 
Luo via cws)

[cws] HIVE-2279. Implement sort(array) UDF (Zhenxiao Luo via cws)


Changes for Build #1212
[hashutosh] HIVE-2589 : Newly created partition should inherit properties from 
table (Ashutosh Chauhan)


Changes for Build #1213

Changes for Build #1214

Changes for Build #1215

Changes for Build #1216
[cws] HIVE-2724. Remove unused lib/log4j-1.2.15.jar (Andrew Bayer via cws)


Changes for Build #1217

Changes for Build #1218

Changes for Build #1219
[hashutosh] HIVE-2665 : Support for metastore service specific HADOOP_OPTS 
environment setting (thw via hashutosh)


Changes for Build #1220
[namit] HIVE-2727 add a testcase for partitioned view on union and base tables 
have
index (He Yongqiang via namit)


Changes for Build #1221
[hashutosh] HIVE-2746 : Metastore client doesn't log properly in case of 
connection failure to server (hashutosh)

[cws] HIVE-2698 [jira] Enable Hadoop-1.0.0 in Hive
(Enis Söztutar via Carl Steinbach)

Summary:
third version of the patch

Hadoop-1.0.0 is recently released, which is AFAIK, API compatible to the 0.20S
release.

Test Plan: EMPTY

Reviewers: JIRA, cwsteinbach

Reviewed By: cwsteinbach

CC: cwsteinbach, enis

Differential Revision: https://reviews.facebook.net/D1389


Changes for Build #1222
[namit] HIVE-2750 Hive multi group by single reducer optimization causes 
invalid column
reference error (Kevin Wilfong via namit)


Changes for Build #1223

Changes for Build #1224
[cws] HIVE-2734 [jira] Fix some nondeterministic test output
(Zhenxiao Luo via Carl Steinbach)

Summary:
HIVE-2734: Fix some nondeterministic test output

Many Hive query tests lack an ORDER BY clause, and consequently the ordering
of the rows in the result set is nondeterministic:

groupby1_limit
input11_limit
input1_limit
input_lazyserde
join18_multi_distinct
join_1to1
join_casesensitive
join_filters
join_nulls
merge3
rcfile_columnar
rcfile_lazydecompress
rcfile_union
sample10
udf_sentences
union24
columnarserde_create_shortcut
combine1
global_limit

Test Plan: EMPTY

Reviewers: JIRA, cwsteinbach

Reviewed By: cwsteinbach

CC: zhenxiao, cwsteinbach

Differential Revision: https://reviews.facebook.net/D1449

[namit] HIVE-2754 NPE in union with lateral view
(Yongqiang He via namit)


Changes for Build #1225

Changes for Build #1226

Changes for Build #1227
[namit] HIVE-2755 union follwowed by union_subq does not work if the subquery 
union
has reducers (He Yongqiang via namit)


Changes for Build #1228

Changes for Build #1229
[hashutosh] HIVE-2735: PlanUtils.configureTableJobPropertiesForStorageHandler() 
is not called for partitioned table (sushanth via ashutosh)


Changes for Build #1230
[cws] HIVE-2760 [jira] TestCliDriver should log elapsed time

Summary: HIVE-2760. TestCliDriver should log elapsed time

Test Plan: EMPTY

Reviewers: JIRA, ashutoshc

Reviewed By: ashutoshc

CC: ashutoshc, cwsteinbach

Differential Revision: https://reviews.facebook.net/D1503

[cws] HIVE-2662 [jira] Add Ant configuration property for dumping classpath of 
tests

Summary: HIVE-2662. Add Ant configuration property for dumping classpath of
tests

Test Plan: EMPTY

Reviewers: JIRA, jsichi, ashutoshc

Reviewed By: ashutoshc

CC: ashutoshc

Differential Revision: https://reviews.facebook.net/D903


Changes for Build #1231
[hashutosh] HIVE_2645: Hive Web Server startup messages logs incorrect path it 
is searching for WAR (Edward Capriolo via Ashutosh Chauhan)


Changes for Build #1232

Changes for Build #1233
[sdong] HIVE-2249 When creating constant expression for numbers, try to infer 
type from another comparison operand, instead of trying to use integer first, 
and then long and double (Zhiqiu Kong via Siying Dong)


Changes for Build #1234

Changes for Build #1235
[heyongqiang] HIVE-2765 hbase handler uses ZooKeeperConnectionException which 
is not compatible with HBase versions other than 0.89 (Pei Yue via He Yongqiang)


Changes for Build #1236

Changes for Build #1237

Changes for Build #1238
[heyongqiang] HIVE-2772 [jira] make union31.q deterministic
(Namit Jain via Yongqiang He)

Summary:
https://issues.apache.org/jira/browse/HIVE-2772

HIVE-2772



Test Plan: EMPTY

Reviewers: JIRA, ashutoshc

Reviewed By: ashutoshc

CC: ashutoshc

Differential Revision: https://reviews.facebook.net/D1557

[kevinwilfong] HIVE-2758 Metastore is caching too aggressively (Kevin Wilfong 
reviewed by Carl Steinbach)


Changes for Build #1239

Changes for Build #1240
[namit] HIVE-2762 Alter Table Partition Concatenate Fails On Certain Characters
(Kevin Wilfong via namit)


Changes for Build #1241
[namit] HIVE-2756 Views should be added to the inputs of queries.
(Yongqiang He via namit)


Changes for Build #1242

Changes for Build #1243

Changes for Build #1244

C

[jira] [Commented] (HIVE-2786) Throw an error if the user tries to insert a table into a region other than the primary region

2012-02-10 Thread Namit Jain (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205706#comment-13205706
 ] 

Namit Jain commented on HIVE-2786:
--

The semantics are as follows:

If the table/partition is present in the primary region, and the insert is 
happening in the secondary region, verify that the table/partition is the same 
(the file size should be exactly
the same). Otherwise, the operation fails.

If the table/partition is present in the secondary region, and the insert is 
happening in the primary region, verify that the table/partition is the same 
(the file size should be exactly
the same). Otherwise, the secondary region needs to be deleted.

> Throw an error if the user tries to insert a table into a region other than 
> the primary region
> --
>
> Key: HIVE-2786
> URL: https://issues.apache.org/jira/browse/HIVE-2786
> Project: Hive
>  Issue Type: New Feature
>Reporter: Namit Jain
>Assignee: Namit Jain
>
> By default, the user can only insert into the primary region.
> Add an option to insert into the secondary region also.
> The config variable is 'hive.insert.secondary.regions' - default for that 
> variable is false

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-2798) add an option to change the primary region for a table

2012-02-10 Thread Namit Jain (Created) (JIRA)

add an option to change the primary region for a table
--

 Key: HIVE-2798
 URL: https://issues.apache.org/jira/browse/HIVE-2798
 Project: Hive
  Issue Type: New Feature
Reporter: Namit Jain
Assignee: Namit Jain


This should error out if any of the partitions are not present in the primary 
region

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-2786) Throw an error if the user tries to insert a table into a region other than the primary region

2012-02-10 Thread Namit Jain (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain reassigned HIVE-2786:


Assignee: Namit Jain

> Throw an error if the user tries to insert a table into a region other than 
> the primary region
> --
>
> Key: HIVE-2786
> URL: https://issues.apache.org/jira/browse/HIVE-2786
> Project: Hive
>  Issue Type: New Feature
>Reporter: Namit Jain
>Assignee: Namit Jain
>
> By default, the user can only insert into the primary region.
> Add an option to insert into the secondary region also.
> The config variable is 'hive.insert.secondary.regions' - default for that 
> variable is false

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2786) Throw an error if the user tries to insert a table into a region other than the primary region

2012-02-10 Thread Namit Jain (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-2786:
-

Description: 
By default, the user can only insert into the primary region.

Add an option to insert into the secondary region also.
The config variable is 'hive.insert.secondary.regions' - default for that 
variable is false

  was:
By default, the user can only insert into the primary cluster.

Add an option to insert into the secondary clusters also

Summary: Throw an error if the user tries to insert a table into a 
region other than the primary region  (was: Throw an error if the user tries to 
insert a table into a cluster other than the primary cluster)

> Throw an error if the user tries to insert a table into a region other than 
> the primary region
> --
>
> Key: HIVE-2786
> URL: https://issues.apache.org/jira/browse/HIVE-2786
> Project: Hive
>  Issue Type: New Feature
>Reporter: Namit Jain
>
> By default, the user can only insert into the primary region.
> Add an option to insert into the secondary region also.
> The config variable is 'hive.insert.secondary.regions' - default for that 
> variable is false

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-2785) support use region

2012-02-10 Thread Namit Jain (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain reassigned HIVE-2785:


Assignee: Namit Jain

> support use region
> --
>
> Key: HIVE-2785
> URL: https://issues.apache.org/jira/browse/HIVE-2785
> Project: Hive
>  Issue Type: New Feature
>Reporter: Namit Jain
>Assignee: Namit Jain
>
> use region;
> use region ;
> should be supported

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2785) support use cluster

2012-02-10 Thread Namit Jain (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205700#comment-13205700
 ] 

Namit Jain commented on HIVE-2785:
--

The semantics are as follows:

use region;

Use the default region based on the primary region of the table being queried.
Choose the jobtracker, dfs from the region provided.

use region ;

Use the region specified.

The query should only succeed if all the partitions of the table are present in 
the primary region


> support use cluster
> ---
>
> Key: HIVE-2785
> URL: https://issues.apache.org/jira/browse/HIVE-2785
> Project: Hive
>  Issue Type: New Feature
>Reporter: Namit Jain
>
> use cluster;
> use cluster ;
> should be supported

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2785) support use region

2012-02-10 Thread Namit Jain (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-2785:
-

Description: 
use region;

use region ;

should be supported

  was:
use cluster;

use cluster ;

should be supported

Summary: support use region  (was: support use cluster)

> support use region
> --
>
> Key: HIVE-2785
> URL: https://issues.apache.org/jira/browse/HIVE-2785
> Project: Hive
>  Issue Type: New Feature
>Reporter: Namit Jain
>
> use region;
> use region ;
> should be supported

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2797) Make the IP address of a Thrift client available to HMSHandler.

2012-02-10 Thread Kevin Wilfong (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2797:


Assignee: Kevin Wilfong
  Status: Patch Available  (was: Open)

> Make the IP address of a Thrift client available to HMSHandler.
> ---
>
> Key: HIVE-2797
> URL: https://issues.apache.org/jira/browse/HIVE-2797
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-2797.D1701.1.patch
>
>
> Currently, in unsecured mode, metastore Thrift calls are, from the 
> HMSHandler's point of view, anonymous.  If we expose the IP address of the 
> Thrift client to the HMSHandler from the Processor, this will help to give 
> some context, in particular for audit logging, of where the call is coming 
> from.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2797) Make the IP address of a Thrift client available to HMSHandler.

2012-02-10 Thread Kevin Wilfong (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2797:


Component/s: Metastore

> Make the IP address of a Thrift client available to HMSHandler.
> ---
>
> Key: HIVE-2797
> URL: https://issues.apache.org/jira/browse/HIVE-2797
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Kevin Wilfong
> Attachments: HIVE-2797.D1701.1.patch
>
>
> Currently, in unsecured mode, metastore Thrift calls are, from the 
> HMSHandler's point of view, anonymous.  If we expose the IP address of the 
> Thrift client to the HMSHandler from the Processor, this will help to give 
> some context, in particular for audit logging, of where the call is coming 
> from.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2612) support hive table/partitions exists in more than one region

2012-02-10 Thread Namit Jain (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205688#comment-13205688
 ] 

Namit Jain commented on HIVE-2612:
--

Changed 'cluster' to 'region' and ran the testclidriver tests.
Overwrote a bunch of test files.

Kevin, can you change the upgrade files, since the schema has changed.

> support hive table/partitions exists in more than one region
> 
>
> Key: HIVE-2612
> URL: https://issues.apache.org/jira/browse/HIVE-2612
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore
>Reporter: He Yongqiang
>Assignee: Namit Jain
> Attachments: HIVE-2612.1.patch, HIVE-2612.2.patch.txt, 
> HIVE-2612.3.patch.txt, HIVE-2612.4.patch.txt, HIVE-2612.D1569.1.patch, 
> HIVE-2612.D1569.2.patch, HIVE-2612.D1569.3.patch, HIVE-2612.D1569.4.patch, 
> HIVE-2612.D1569.5.patch, HIVE-2612.D1569.6.patch, HIVE-2612.D1569.7.patch, 
> hive.2612.5.patch
>
>
> 1) add region object into hive metastore
> 2) each partition/table has a primary region and a list of living regions, 
> and also data location in each region

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2612) support hive table/partitions exists in more than one region

2012-02-10 Thread Namit Jain (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-2612:
-

Attachment: hive.2612.5.patch

> support hive table/partitions exists in more than one region
> 
>
> Key: HIVE-2612
> URL: https://issues.apache.org/jira/browse/HIVE-2612
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore
>Reporter: He Yongqiang
>Assignee: Namit Jain
> Attachments: HIVE-2612.1.patch, HIVE-2612.2.patch.txt, 
> HIVE-2612.3.patch.txt, HIVE-2612.4.patch.txt, HIVE-2612.D1569.1.patch, 
> HIVE-2612.D1569.2.patch, HIVE-2612.D1569.3.patch, HIVE-2612.D1569.4.patch, 
> HIVE-2612.D1569.5.patch, HIVE-2612.D1569.6.patch, HIVE-2612.D1569.7.patch, 
> hive.2612.5.patch
>
>
> 1) add region object into hive metastore
> 2) each partition/table has a primary region and a list of living regions, 
> and also data location in each region

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2797) Make the IP address of a Thrift client available to HMSHandler.

2012-02-10 Thread Phabricator (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-2797:
--

Attachment: HIVE-2797.D1701.1.patch

kevinwilfong requested code review of "HIVE-2797 [jira] Make the IP address of 
a Thrift client available to HMSHandler.".
Reviewers: JIRA

  https://issues.apache.org/jira/browse/HIVE-2797

  Created a new subclass of ThriftHiveMetastore.Processor which passes the IP 
address to the HMSHandler.  Unsecure connections will use this processor.  The 
IP address is stored in a thread local variable, so that multiple connections 
will each maintain there own IP address.

  Currently, in unsecured mode, metastore Thrift calls are, from the 
HMSHandler's point of view, anonymous.  If we expose the IP address of the 
Thrift client to the HMSHandler from the Processor, this will help to give some 
context, in particular for audit logging, of where the call is coming from.

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D1701

AFFECTED FILES
  metastore/src/java/org/apache/hadoop/hive/metastore/TUGIBasedProcessor.java
  
metastore/src/java/org/apache/hadoop/hive/metastore/TSetIpAddressProcessor.java
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/3633/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


> Make the IP address of a Thrift client available to HMSHandler.
> ---
>
> Key: HIVE-2797
> URL: https://issues.apache.org/jira/browse/HIVE-2797
> Project: Hive
>  Issue Type: Improvement
>Reporter: Kevin Wilfong
> Attachments: HIVE-2797.D1701.1.patch
>
>
> Currently, in unsecured mode, metastore Thrift calls are, from the 
> HMSHandler's point of view, anonymous.  If we expose the IP address of the 
> Thrift client to the HMSHandler from the Processor, this will help to give 
> some context, in particular for audit logging, of where the call is coming 
> from.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-2797) Make the IP address of a Thrift client available to HMSHandler.

2012-02-10 Thread Kevin Wilfong (Created) (JIRA)

Make the IP address of a Thrift client available to HMSHandler.
---

 Key: HIVE-2797
 URL: https://issues.apache.org/jira/browse/HIVE-2797
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong


Currently, in unsecured mode, metastore Thrift calls are, from the HMSHandler's 
point of view, anonymous.  If we expose the IP address of the Thrift client to 
the HMSHandler from the Processor, this will help to give some context, in 
particular for audit logging, of where the call is coming from.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2206) add a new optimizer for query correlation discovery and optimization

2012-02-10 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205654#comment-13205654
 ] 

jirapos...@reviews.apache.org commented on HIVE-2206:
-



bq.  On 2012-02-10 17:38:09, Kevin Wilfong wrote:
bq.  > I've started reviewing this, here's my comments so far.  I'll continue 
to look over it.

I will update this patch soon. 


bq.  On 2012-02-10 17:38:09, Kevin Wilfong wrote:
bq.  > trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 453
bq.  > 
bq.  >
bq.  > Does this have to default to false, does anything break if it's true?
bq.  > 
bq.  > Similarly, have you tried running the tests with this set to true?

I have not tried running the tests with this set to true. I will do it when I 
find a revision which can pass all unit tests (btw, any suggestion on which 
revision should I use?). In my opinion, since this optimizer is kind of 
complicated and it is still being developed, it will be safer to default it to 
false and let users to decide when to use it than default it to true.


bq.  On 2012-02-10 17:38:09, Kevin Wilfong wrote:
bq.  > 
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationCompositeOperator.java,
 line 101
bq.  > 
bq.  >
bq.  > It's not clear to me why we need both setRowNumber and processOp.

Since a CorrelationCompositeOperator may have multiple parents, I used a buffer 
to store the output of parents of the CorrelationCompositeOperator (shown 
processOp method). The TableScanOperator will trigger the setRowNumber method 
and then CorrelationCompositeOperator will decide the operationPathTags of this 
row based on the contents in the buffer and then forward the row in its buffer 
to its child. So, setRowNumber in here is used to evaluate the 
operationPathTags of the row in the buffer before the 
CorrelationCompositeOperator gets the new row. 


bq.  On 2012-02-10 17:38:09, Kevin Wilfong wrote:
bq.  > 
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationCompositeOperator.java,
 lines 150-177
bq.  > 
bq.  >
bq.  > Putting this code in a helper method would be better than having it 
both here and in setRowNumber.

I will do it. 


bq.  On 2012-02-10 17:38:09, Kevin Wilfong wrote:
bq.  > 
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationLocalSimulativeReduceSinkOperator.java,
 line 274
bq.  > 
bq.  >
bq.  > Does this commented out code need to be kept?

This commented out code is not needed. I will delete it. 


bq.  On 2012-02-10 17:38:09, Kevin Wilfong wrote:
bq.  > trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java, line 1337
bq.  > 
bq.  >
bq.  > I couldn't find a CorrelationFakeReduceSinkOperator class.

CorrelationLocalSimulativeReduceSinkOperator was named as 
CorrelationFakeReduceSinkOperator. I will use the right name in the comment. 


bq.  On 2012-02-10 17:38:09, Kevin Wilfong wrote:
bq.  > 
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java, line 
273
bq.  > 
bq.  >
bq.  > Tabs are bad, could you change them to spaces, at least in the new 
code your introducing.

I will change the format of my code. Thanks for letting me know.


bq.  On 2012-02-10 17:38:09, Kevin Wilfong wrote:
bq.  > 
trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizer.java,
 line 239
bq.  > 
bq.  >
bq.  > I take it from this line it's a requirement that in order for this 
correlation optimization to be attempted every reduce sink has to be followed 
only by children with a single child.
bq.  > 
bq.  > Could this be relaxed?  Could the optimization simply not be applied 
if there is an operator between two ReduceSinks that has more than one child?
bq.  > 
bq.  > Also, if there is a ReduceSink which is not followed by another 
ReduceSink, but is followed by an operator with more than one child, this 
prevents the optimization from being used, even though it shouldn't have an 
effect.
bq.  > 
bq.  > Also, regarding checking if the size <=1, if the size <1 the next 
line will throw an exception.

Only "assert op.getChildOperators().size() > 0;" is needed at here. Thank you 
for letting me know. 


bq.  On 2012-02-10 17:38:09, Kevin Wilfong wrote:
bq.  > 
trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizer.java,
 line 335
bq.  > 
bq.  >
bq.

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-02-10 Thread Yin Huai

> On 2012-02-10 17:38:09, Kevin Wilfong wrote:
> > I've started reviewing this, here's my comments so far.  I'll continue to 
> > look over it.

I will update this patch soon. 

> On 2012-02-10 17:38:09, Kevin Wilfong wrote:
> > trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 453
> > 
> >
> > Does this have to default to false, does anything break if it's true?
> > 
> > Similarly, have you tried running the tests with this set to true?

I have not tried running the tests with this set to true. I will do it when I 
find a revision which can pass all unit tests (btw, any suggestion on which 
revision should I use?). In my opinion, since this optimizer is kind of 
complicated and it is still being developed, it will be safer to default it to 
false and let users to decide when to use it than default it to true.

> On 2012-02-10 17:38:09, Kevin Wilfong wrote:
> > trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationCompositeOperator.java,
> >  line 101
> > 
> >
> > It's not clear to me why we need both setRowNumber and processOp.

Since a CorrelationCompositeOperator may have multiple parents, I used a buffer 
to store the output of parents of the CorrelationCompositeOperator (shown 
processOp method). The TableScanOperator will trigger the setRowNumber method 
and then CorrelationCompositeOperator will decide the operationPathTags of this 
row based on the contents in the buffer and then forward the row in its buffer 
to its child. So, setRowNumber in here is used to evaluate the 
operationPathTags of the row in the buffer before the 
CorrelationCompositeOperator gets the new row. 

> On 2012-02-10 17:38:09, Kevin Wilfong wrote:
> > trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationCompositeOperator.java,
> >  lines 150-177
> > 
> >
> > Putting this code in a helper method would be better than having it 
> > both here and in setRowNumber.

I will do it. 

> On 2012-02-10 17:38:09, Kevin Wilfong wrote:
> > trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationLocalSimulativeReduceSinkOperator.java,
> >  line 274
> > 
> >
> > Does this commented out code need to be kept?

This commented out code is not needed. I will delete it. 

> On 2012-02-10 17:38:09, Kevin Wilfong wrote:
> > trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java, line 1337
> > 
> >
> > I couldn't find a CorrelationFakeReduceSinkOperator class.

CorrelationLocalSimulativeReduceSinkOperator was named as 
CorrelationFakeReduceSinkOperator. I will use the right name in the comment. 

> On 2012-02-10 17:38:09, Kevin Wilfong wrote:
> > trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java, 
> > line 273
> > 
> >
> > Tabs are bad, could you change them to spaces, at least in the new code 
> > your introducing.

I will change the format of my code. Thanks for letting me know.

> On 2012-02-10 17:38:09, Kevin Wilfong wrote:
> > trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizer.java,
> >  line 239
> > 
> >
> > I take it from this line it's a requirement that in order for this 
> > correlation optimization to be attempted every reduce sink has to be 
> > followed only by children with a single child.
> > 
> > Could this be relaxed?  Could the optimization simply not be applied if 
> > there is an operator between two ReduceSinks that has more than one child?
> > 
> > Also, if there is a ReduceSink which is not followed by another 
> > ReduceSink, but is followed by an operator with more than one child, this 
> > prevents the optimization from being used, even though it shouldn't have an 
> > effect.
> > 
> > Also, regarding checking if the size <=1, if the size <1 the next line 
> > will throw an exception.

Only "assert op.getChildOperators().size() > 0;" is needed at here. Thank you 
for letting me know. 

> On 2012-02-10 17:38:09, Kevin Wilfong wrote:
> > trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizer.java,
> >  line 335
> > 
> >
> > findNextChildReduceSinkOperator can return null, do you need to check 
> > for this?

findNextChildReduceSinkOperator will not return null since its input will not 
be the last ReduceSinkOperator before the FileSinkOperator. For example, 
suppose that we have a plan tree like (some operators)->RS1->(some 
operators)->RS2->(some operators)->FS. The input of 
findNex

[jira] [Commented] (HIVE-2206) add a new optimizer for query correlation discovery and optimization

2012-02-10 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205570#comment-13205570
 ] 

jirapos...@reviews.apache.org commented on HIVE-2206:
-


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2001/#review4912
---


I've started reviewing this, here's my comments so far.  I'll continue to look 
over it.


trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java


Does this have to default to false, does anything break if it's true?

Similarly, have you tried running the tests with this set to true?



trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationCompositeOperator.java


It's not clear to me why we need both setRowNumber and processOp.



trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationCompositeOperator.java


Putting this code in a helper method would be better than having it both 
here and in setRowNumber.



trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationLocalSimulativeReduceSinkOperator.java


Does this commented out code need to be kept?



trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java


I couldn't find a CorrelationFakeReduceSinkOperator class.



trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java


Tabs are bad, could you change them to spaces, at least in the new code 
your introducing.



trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizer.java


I take it from this line it's a requirement that in order for this 
correlation optimization to be attempted every reduce sink has to be followed 
only by children with a single child.

Could this be relaxed?  Could the optimization simply not be applied if 
there is an operator between two ReduceSinks that has more than one child?

Also, if there is a ReduceSink which is not followed by another ReduceSink, 
but is followed by an operator with more than one child, this prevents the 
optimization from being used, even though it shouldn't have an effect.

Also, regarding checking if the size <=1, if the size <1 the next line will 
throw an exception.



trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizer.java


findNextChildReduceSinkOperator can return null, do you need to check for 
this?


- Kevin


On 2012-01-29 17:56:48, Yin Huai wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2001/
bq.  ---
bq.  
bq.  (Updated 2012-01-29 17:56:48)
bq.  
bq.  
bq.  Review request for hive.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This optimizer exploits intra-query correlations and merges multiple 
correlated MapReduce jobs into one jobs.
bq.  
bq.  
bq.  This addresses bug HIVE-2206.
bq.  https://issues.apache.org/jira/browse/HIVE-2206
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1237326 
bq.
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/BaseReduceSinkOperator.java 
PRE-CREATION 
bq.
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationCompositeOperator.java
 PRE-CREATION 
bq.
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationLocalSimulativeReduceSinkOperator.java
 PRE-CREATION 
bq.
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationReducerDispatchOperator.java
 PRE-CREATION 
bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecReducer.java 
1237326 
bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java 1237326 
bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 
1237326 
bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 
1237326 
bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 
1237326 
bq.trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java 
1237326 
bq.
trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizer.java 
PRE-CREATION 
bq.
trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizerUtils.java
 PRE-CREATION 
bq.
trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
1237326 
bq.trunk/ql/src

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

2012-02-10 Thread Kevin Wilfong

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2001/#review4912
---

I've started reviewing this, here's my comments so far.  I'll continue to look 
over it.

trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java

Does this have to default to false, does anything break if it's true?

Similarly, have you tried running the tests with this set to true?

trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationCompositeOperator.java

It's not clear to me why we need both setRowNumber and processOp.

trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationCompositeOperator.java

Putting this code in a helper method would be better than having it both 
here and in setRowNumber.

trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationLocalSimulativeReduceSinkOperator.java

Does this commented out code need to be kept?

trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java

I couldn't find a CorrelationFakeReduceSinkOperator class.

trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java

Tabs are bad, could you change them to spaces, at least in the new code 
your introducing.

trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizer.java

I take it from this line it's a requirement that in order for this 
correlation optimization to be attempted every reduce sink has to be followed 
only by children with a single child.

Could this be relaxed?  Could the optimization simply not be applied if 
there is an operator between two ReduceSinks that has more than one child?

Also, if there is a ReduceSink which is not followed by another ReduceSink, 
but is followed by an operator with more than one child, this prevents the 
optimization from being used, even though it shouldn't have an effect.

Also, regarding checking if the size <=1, if the size <1 the next line will 
throw an exception.

trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizer.java

findNextChildReduceSinkOperator can return null, do you need to check for 
this?

- Kevin

On 2012-01-29 17:56:48, Yin Huai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2001/
> ---
> 
> (Updated 2012-01-29 17:56:48)
> 
> 
> Review request for hive.
> 
> 
> Summary
> ---
> 
> This optimizer exploits intra-query correlations and merges multiple 
> correlated MapReduce jobs into one jobs.
> 
> 
> This addresses bug HIVE-2206.
> https://issues.apache.org/jira/browse/HIVE-2206
> 
> 
> Diffs
> -
> 
>   trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1237326 
>   
> trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/BaseReduceSinkOperator.java 
> PRE-CREATION 
>   
> trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationCompositeOperator.java
>  PRE-CREATION 
>   
> trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationLocalSimulativeReduceSinkOperator.java
>  PRE-CREATION 
>   
> trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationReducerDispatchOperator.java
>  PRE-CREATION 
>   trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecReducer.java 1237326 
>   trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java 1237326 
>   trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 
> 1237326 
>   trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 
> 1237326 
>   trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 
> 1237326 
>   trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java 
> 1237326 
>   
> trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizer.java
>  PRE-CREATION 
>   
> trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizerUtils.java
>  PRE-CREATION 
>   trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
> 1237326 
>   trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 
> 1237326 
>   trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 1237326 
>   trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> 1237326 
>   trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/BaseReduceSinkDesc.java 
> PRE-CREATION 
>   
> trunk/ql/src/java/org/apache/h

[jira] [Commented] (HIVE-2539) Enable passing username/password via JDBC

2012-02-10 Thread chunqing xie (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205466#comment-13205466
 ] 

chunqing xie commented on HIVE-2539:


DriverManager.getConnection() with username/password or without 
username/password are both tested.

> Enable passing username/password via JDBC
> -
>
> Key: HIVE-2539
> URL: https://issues.apache.org/jira/browse/HIVE-2539
> Project: Hive
>  Issue Type: Improvement
>  Components: JDBC
>Affects Versions: 0.7.1
>Reporter: Sriram Krishnan
>  Labels: patch
> Attachments: HIVE-2539.PATCH
>
>
> Changing the username and/or the password seems to have no effect (also 
> confirmed here: https://cwiki.apache.org/Hive/hivejdbcinterface.html). 
>Connection con = 
> DriverManager.getConnection("jdbc:hive://localhost:1/default", "", "");
> Would be beneficial to pass the username/password via JDBC - and also for the 
> server to honor the username password being passed (may be dependent of that 
> being fixed first).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2539) Enable passing username/password via JDBC

2012-02-10 Thread chunqing xie (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chunqing xie updated HIVE-2539:
---

Labels: patch  (was: )
Status: Patch Available  (was: Open)

The username and password is processed in HiveConnection.java

> Enable passing username/password via JDBC
> -
>
> Key: HIVE-2539
> URL: https://issues.apache.org/jira/browse/HIVE-2539
> Project: Hive
>  Issue Type: Improvement
>  Components: JDBC
>Affects Versions: 0.7.1
>Reporter: Sriram Krishnan
>  Labels: patch
> Attachments: HIVE-2539.PATCH
>
>
> Changing the username and/or the password seems to have no effect (also 
> confirmed here: https://cwiki.apache.org/Hive/hivejdbcinterface.html). 
>Connection con = 
> DriverManager.getConnection("jdbc:hive://localhost:1/default", "", "");
> Would be beneficial to pass the username/password via JDBC - and also for the 
> server to honor the username password being passed (may be dependent of that 
> being fixed first).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2539) Enable passing username/password via JDBC

2012-02-10 Thread chunqing xie (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-2539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chunqing xie updated HIVE-2539:
---

Attachment: HIVE-2539.PATCH

> Enable passing username/password via JDBC
> -
>
> Key: HIVE-2539
> URL: https://issues.apache.org/jira/browse/HIVE-2539
> Project: Hive
>  Issue Type: Improvement
>  Components: JDBC
>Affects Versions: 0.7.1
>Reporter: Sriram Krishnan
>  Labels: patch
> Attachments: HIVE-2539.PATCH
>
>
> Changing the username and/or the password seems to have no effect (also 
> confirmed here: https://cwiki.apache.org/Hive/hivejdbcinterface.html). 
>Connection con = 
> DriverManager.getConnection("jdbc:hive://localhost:1/default", "", "");
> Would be beneficial to pass the username/password via JDBC - and also for the 
> server to honor the username password being passed (may be dependent of that 
> being fixed first).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1252 - Still Failing

2012-02-10 Thread Apache Jenkins Server

Changes for Build #1209

Changes for Build #1210
[namit] HIVE-2674 get_partitions_ps throws TApplicationException if table 
doesn't
exist (Kevin Wilfong via namit)


Changes for Build #1211
[cws] HIVE-2203. Extend concat_ws() UDF to support arrays of strings (Zhenxiao 
Luo via cws)

[cws] HIVE-2279. Implement sort(array) UDF (Zhenxiao Luo via cws)


Changes for Build #1212
[hashutosh] HIVE-2589 : Newly created partition should inherit properties from 
table (Ashutosh Chauhan)


Changes for Build #1213

Changes for Build #1214

Changes for Build #1215

Changes for Build #1216
[cws] HIVE-2724. Remove unused lib/log4j-1.2.15.jar (Andrew Bayer via cws)


Changes for Build #1217

Changes for Build #1218

Changes for Build #1219
[hashutosh] HIVE-2665 : Support for metastore service specific HADOOP_OPTS 
environment setting (thw via hashutosh)


Changes for Build #1220
[namit] HIVE-2727 add a testcase for partitioned view on union and base tables 
have
index (He Yongqiang via namit)


Changes for Build #1221
[hashutosh] HIVE-2746 : Metastore client doesn't log properly in case of 
connection failure to server (hashutosh)

[cws] HIVE-2698 [jira] Enable Hadoop-1.0.0 in Hive
(Enis Söztutar via Carl Steinbach)

Summary:
third version of the patch

Hadoop-1.0.0 is recently released, which is AFAIK, API compatible to the 0.20S
release.

Test Plan: EMPTY

Reviewers: JIRA, cwsteinbach

Reviewed By: cwsteinbach

CC: cwsteinbach, enis

Differential Revision: https://reviews.facebook.net/D1389


Changes for Build #1222
[namit] HIVE-2750 Hive multi group by single reducer optimization causes 
invalid column
reference error (Kevin Wilfong via namit)


Changes for Build #1223

Changes for Build #1224
[cws] HIVE-2734 [jira] Fix some nondeterministic test output
(Zhenxiao Luo via Carl Steinbach)

Summary:
HIVE-2734: Fix some nondeterministic test output

Many Hive query tests lack an ORDER BY clause, and consequently the ordering
of the rows in the result set is nondeterministic:

groupby1_limit
input11_limit
input1_limit
input_lazyserde
join18_multi_distinct
join_1to1
join_casesensitive
join_filters
join_nulls
merge3
rcfile_columnar
rcfile_lazydecompress
rcfile_union
sample10
udf_sentences
union24
columnarserde_create_shortcut
combine1
global_limit

Test Plan: EMPTY

Reviewers: JIRA, cwsteinbach

Reviewed By: cwsteinbach

CC: zhenxiao, cwsteinbach

Differential Revision: https://reviews.facebook.net/D1449

[namit] HIVE-2754 NPE in union with lateral view
(Yongqiang He via namit)


Changes for Build #1225

Changes for Build #1226

Changes for Build #1227
[namit] HIVE-2755 union follwowed by union_subq does not work if the subquery 
union
has reducers (He Yongqiang via namit)


Changes for Build #1228

Changes for Build #1229
[hashutosh] HIVE-2735: PlanUtils.configureTableJobPropertiesForStorageHandler() 
is not called for partitioned table (sushanth via ashutosh)


Changes for Build #1230
[cws] HIVE-2760 [jira] TestCliDriver should log elapsed time

Summary: HIVE-2760. TestCliDriver should log elapsed time

Test Plan: EMPTY

Reviewers: JIRA, ashutoshc

Reviewed By: ashutoshc

CC: ashutoshc, cwsteinbach

Differential Revision: https://reviews.facebook.net/D1503

[cws] HIVE-2662 [jira] Add Ant configuration property for dumping classpath of 
tests

Summary: HIVE-2662. Add Ant configuration property for dumping classpath of
tests

Test Plan: EMPTY

Reviewers: JIRA, jsichi, ashutoshc

Reviewed By: ashutoshc

CC: ashutoshc

Differential Revision: https://reviews.facebook.net/D903


Changes for Build #1231
[hashutosh] HIVE_2645: Hive Web Server startup messages logs incorrect path it 
is searching for WAR (Edward Capriolo via Ashutosh Chauhan)


Changes for Build #1232

Changes for Build #1233
[sdong] HIVE-2249 When creating constant expression for numbers, try to infer 
type from another comparison operand, instead of trying to use integer first, 
and then long and double (Zhiqiu Kong via Siying Dong)


Changes for Build #1234

Changes for Build #1235
[heyongqiang] HIVE-2765 hbase handler uses ZooKeeperConnectionException which 
is not compatible with HBase versions other than 0.89 (Pei Yue via He Yongqiang)


Changes for Build #1236

Changes for Build #1237

Changes for Build #1238
[heyongqiang] HIVE-2772 [jira] make union31.q deterministic
(Namit Jain via Yongqiang He)

Summary:
https://issues.apache.org/jira/browse/HIVE-2772

HIVE-2772



Test Plan: EMPTY

Reviewers: JIRA, ashutoshc

Reviewed By: ashutoshc

CC: ashutoshc

Differential Revision: https://reviews.facebook.net/D1557

[kevinwilfong] HIVE-2758 Metastore is caching too aggressively (Kevin Wilfong 
reviewed by Carl Steinbach)


Changes for Build #1239

Changes for Build #1240
[namit] HIVE-2762 Alter Table Partition Concatenate Fails On Certain Characters
(Kevin Wilfong via namit)


Changes for Build #1241
[namit] HIVE-2756 Views should be added to the inputs of queries.
(Yongqiang He via namit)


Changes for Build #1242

Changes for Build #1243

Changes for Build #1244

C

[jira] [Commented] (HIVE-2795) View partitions do not have a storage descriptor

2012-02-10 Thread Hudson (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205354#comment-13205354
 ] 

Hudson commented on HIVE-2795:
--

Integrated in Hive-trunk-h0.21 #1252 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1252/])
HIVE-2795 View partitions do not have a storage descriptor
(Kevin Wilfong via namit)

namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1242682
Files : 
* /hive/trunk/metastore/scripts/upgrade/001-HIVE-2795.update_view_partitions.py
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
* 
/hive/trunk/ql/src/test/queries/clientpositive/describe_formatted_view_partitioned.q
* 
/hive/trunk/ql/src/test/results/clientpositive/describe_formatted_view_partitioned.q.out


> View partitions do not have a storage descriptor
> 
>
> Key: HIVE-2795
> URL: https://issues.apache.org/jira/browse/HIVE-2795
> Project: Hive
>  Issue Type: Improvement
>Reporter: Namit Jain
>Assignee: Kevin Wilfong
> Attachments: HIVE-2795.D1683.1.patch, HIVE-2795.D1683.2.patch, 
> HIVE-2795.D1683.3.patch
>
>
> Besides being an inconsistency, it causes errors.
> Calling describe formatted on a view partition throws an exception
> java.lang.NullPointerException
>   at org.apache.hadoop.hive.ql.metadata.Partition.getCols(Partition.java:505) 
>  
>   at org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2570)
> because it does not have a column descriptor, which is part of the storage 
> descriptor.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

42 matches

Mail list logo