[jira] [Commented] (HIVE-2799) change the following thrift apis to add a region
[ https://issues.apache.org/jira/browse/HIVE-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209485#comment-13209485 ] Ashutosh Chauhan commented on HIVE-2799: As a developer of Hive, I am definitely in favor of keeping code simpler and interfaces cleaner. As an open source user, I am fine with wrapping the new calls in existing apis. change the following thrift apis to add a region Key: HIVE-2799 URL: https://issues.apache.org/jira/browse/HIVE-2799 Project: Hive Issue Type: New Feature Components: Metastore, Thrift API Reporter: Namit Jain Assignee: Kevin Wilfong liststring get_tables(1: string db_name, 2: string pattern) throws (1: MetaException o1) liststring get_all_tables(1: string db_name) throws (1: MetaException o1) Table get_table(1:string dbname, 2:string tbl_name) throws (1:MetaException o1, 2:NoSuchObjectException o2) listTable get_table_objects_by_name(1:string dbname, 2:liststring tbl_names) throws (1:MetaException o1, 2:InvalidOperationException o2, 3:UnknownDBException o3) liststring get_table_names_by_filter(1:string dbname, 2:string filter, 3:i16 max_tables=-1) throws (1:MetaException o1, 2:InvalidOperationException o2, 3:UnknownDBException o3) Partition add_partition(1:Partition new_part) throws(1:InvalidObjectException o1, 2:AlreadyExistsException o2, 3:MetaException o3) i32 add_partitions(1:listPartition new_parts) throws(1:InvalidObjectException o1, 2:AlreadyExistsException o2, 3:MetaException o3) Partition append_partition(1:string db_name, 2:string tbl_name, 3:liststring part_vals) throws (1:InvalidObjectException o1, 2:AlreadyExistsException o2, 3:MetaException o3) Partition append_partition_by_name(1:string db_name, 2:string tbl_name, 3:string part_name) throws (1:InvalidObjectException o1, 2:AlreadyExistsException o2, 3:MetaException o3) bool drop_partition(1:string db_name, 2:string tbl_name, 3:liststring part_vals, 4:bool deleteData) throws(1:NoSuchObjectException o1, 2:MetaException o2) bool drop_partition_by_name(1:string db_name, 2:string tbl_name, 3:string part_name, 4:bool deleteData) throws(1:NoSuchObjectException o1, 2:MetaException o2) Partition get_partition(1:string db_name, 2:string tbl_name, 3:liststring part_vals) throws(1:MetaException o1, 2:NoSuchObjectException o2) Partition get_partition_with_auth(1:string db_name, 2:string tbl_name, 3:liststring part_vals, 4: string user_name, 5: liststring group_names) throws(1:MetaException o1, 2:NoSuchObjectException o2) Partition get_partition_by_name(1:string db_name 2:string tbl_name, 3:string part_name) throws(1:MetaException o1, 2:NoSuchObjectException o2) listPartition get_partitions(1:string db_name, 2:string tbl_name, 3:i16 max_parts=-1) throws(1:NoSuchObjectException o1, 2:MetaException o2) listPartition get_partitions_with_auth(1:string db_name, 2:string tbl_name, 3:i16 max_parts=-1, 4: string user_name, 5: liststring group_names) throws(1:NoSuchObjectException o1, 2:MetaException o2) liststring get_partition_names(1:string db_name, 2:string tbl_name, 3:i16 max_parts=-1) throws(1:MetaException o2) listPartition get_partitions_ps(1:string db_name 2:string tbl_name 3:liststring part_vals, 4:i16 max_parts=-1) throws(1:MetaException o1, 2:NoSuchObjectException o2) listPartition get_partitions_ps_with_auth(1:string db_name, 2:string tbl_name, 3:liststring part_vals, 4:i16 max_parts=-1, 5: string user_name, 6: liststring group_names) throws(1:NoSuchObjectException o1, 2:MetaException o2) liststring get_partition_names_ps(1:string db_name, 2:string tbl_name, 3:liststring part_vals, 4:i16 max_parts=-1) throws(1:MetaException o1, 2:NoSuchObjectException o2) listPartition get_partitions_by_filter(1:string db_name 2:string tbl_name 3:string filter, 4:i16 max_parts=-1) throws(1:MetaException o1, 2:NoSuchObjectException o2) listPartition get_partitions_by_names(1:string db_name 2:string tbl_name 3:liststring names) throws(1:MetaException o1, 2:NoSuchObjectException o2) bool drop_index_by_name(1:string db_name, 2:string tbl_name, 3:string index_name, 4:bool deleteData) throws(1:NoSuchObjectException o1, 2:MetaException o2) Index get_index_by_name(1:string db_name 2:string tbl_name, 3:string index_name)
Push Hive 0.8.1 jars to maven central?
Carl, Can you please push the Hive 0.8.1 jars to Maven central? Thanks, Owen
Re: Push Hive 0.8.1 jars to maven central?
Hi Owen, Sure. Can you point me to a doc that explains how to do that? Thanks. Carl On Thu, Feb 16, 2012 at 8:56 AM, Owen O'Malley omal...@apache.org wrote: Carl, Can you please push the Hive 0.8.1 jars to Maven central? Thanks, Owen
[jira] [Commented] (HIVE-1054) CHANGE COLUMN does not support changing partition column types.
[ https://issues.apache.org/jira/browse/HIVE-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209524#comment-13209524 ] Kevin Wilfong commented on HIVE-1054: - @Ashutosh What if we only allow changing the type of a partition column? I agree adding/dropping/renaming partition columns has a host of difficulties to manage, but I thought changing a partition column's type shouldn't involve a change in hdfs, simply a metadata change. CHANGE COLUMN does not support changing partition column types. - Key: HIVE-1054 URL: https://issues.apache.org/jira/browse/HIVE-1054 Project: Hive Issue Type: Bug Reporter: He Yongqiang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Push Hive 0.8.1 jars to maven central?
On Thu, Feb 16, 2012 at 9:19 AM, Carl Steinbach c...@apache.org wrote: Hi Owen, Sure. Can you point me to a doc that explains how to do that? http://www.apache.org/dev/publishing-maven-artifacts.html Thanks, Roman.
[jira] [Commented] (HIVE-1054) CHANGE COLUMN does not support changing partition column types.
[ https://issues.apache.org/jira/browse/HIVE-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209634#comment-13209634 ] Ashutosh Chauhan commented on HIVE-1054: One problem there could be that all partition values are stored as string, so type info is completely lost in mysql. As a result partition filters cannot be pushed in mysql for any types other then string. This is also documented in listPartitionsByFilter() api of metastoreclient. Though, I don't find that this is enforced anywhere (that might itself be a bug). As a result, changing a type of partition column from string to int or vice-versa might result in change in behavior. CHANGE COLUMN does not support changing partition column types. - Key: HIVE-1054 URL: https://issues.apache.org/jira/browse/HIVE-1054 Project: Hive Issue Type: Bug Reporter: He Yongqiang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2807) Add compression tests
Add compression tests - Key: HIVE-2807 URL: https://issues.apache.org/jira/browse/HIVE-2807 Project: Hive Issue Type: Task Components: Compression, Tests Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2808) Add inputs/outputs for create table if not exists
Add inputs/outputs for create table if not exists - Key: HIVE-2808 URL: https://issues.apache.org/jira/browse/HIVE-2808 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Kevin Wilfong create table if not exists T(..) should have the following semantics: 1. If the table does not exist, T should be a output 2. If T already exists, T should be a input and output. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2809) StorageHandler authorization providers
StorageHandler authorization providers -- Key: HIVE-2809 URL: https://issues.apache.org/jira/browse/HIVE-2809 Project: Hive Issue Type: New Feature Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar In this issue, we would like to discuss the possibility of supplementing the Hive authorization model with authorization at the storage level. As discussed in HIVE-1943, Hive should also check for operation permissions in hdfs and hbase, since otherwise, data and metadata can be in an inconsistent state or be orphaned. Going a step further, some of the setups might not need the full featured auth model by Hive, but want to rely on managing the permissions at the data layer. In this model, the metadata operations are checked first from hdfs/hbase and it is allowed only if they are allowed at the data layer. The semantics are documented at https://cwiki.apache.org/confluence/display/HCATALOG/Hcat+Security+Design. So, the goals of this issue are: - Port storage handler specific authorization providers, and the StorageDelegationAuthorizationProvider from HCATALOG-245 and HCATALOG-260 to Hive. - Keep current Hive's default authorization provider, and enable user to use this and/or the storage one. auth providers are already configurable. - Move the manual checks that had to be performed about authorization in Hcat to Hive, specifically: -- CREATE DATABASE/TABLE, ADD PARTITION statements does not call HiveAuthorizationProvider.authorize() with the candidate objects, which means that we cannot do checks against defined LOCATION. -- HiveOperation does not define sufficient Privileges for most of the operations, especially database operations. -- For some of the operations, Hive SemanticAnalyzer does not add the changed object as a WriteEntity or ReadEntity. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Hive build is back to green
Great news! Thanks for fixing this Ashutosh! comparisons. Fix was to do export LANG=en_US.UTF-8 in environment of build machine. Is this something that we can set in Hive's build.xml file? Thanks. Carl
[jira] [Updated] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2716: -- Attachment: HIVE-2716.D1227.2.patch enis updated the revision HIVE-2716 [jira] Move retry logic in HiveMetaStore to a separe class. Reviewers: JIRA Rebased the patch. REVISION DETAIL https://reviews.facebook.net/D1227 AFFECTED FILES metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java metastore/src/java/org/apache/hadoop/hive/metastore/RetryingRawStore.java metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerTask.java metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HIVE-2716.D1227.1.patch, HIVE-2716.D1227.2.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209958#comment-13209958 ] Phabricator commented on HIVE-2716: --- cwsteinbach has commented on the revision HIVE-2716 [jira] Move retry logic in HiveMetaStore to a separe class. What's the motivation for this patch? Is this change required by something on the HCat side? REVISION DETAIL https://reviews.facebook.net/D1227 Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HIVE-2716.D1227.1.patch, HIVE-2716.D1227.2.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209965#comment-13209965 ] Phabricator commented on HIVE-2716: --- ashutoshc has commented on the revision HIVE-2716 [jira] Move retry logic in HiveMetaStore to a separe class. Enis will know more, but AFAIK HCatalog won't get anything out of this change. I like the idea of the patch for refactoring HiveMetaStore.java which is unwieldy in its current form, thus making Hive codebase easier to read and follow. Do you have any specific concerns? REVISION DETAIL https://reviews.facebook.net/D1227 Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HIVE-2716.D1227.1.patch, HIVE-2716.D1227.2.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209975#comment-13209975 ] Phabricator commented on HIVE-2716: --- enis has commented on the revision HIVE-2716 [jira] Move retry logic in HiveMetaStore to a separe class. What's the motivation for this patch? Is this change required by something on the HCat side? I created the issue and the patch while working on the HMS. The patch refactors the retry logic out of HMS. The Command wrapper inside HMS makes the code very hard to follow, and requires some ugly code for handling checked exceptions, and is error prone is someone forgets wrapping a new method in RawStore. REVISION DETAIL https://reviews.facebook.net/D1227 Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HIVE-2716.D1227.1.patch, HIVE-2716.D1227.2.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2811) Export LANG=en_US.UTF-8 to environment while running tests
Export LANG=en_US.UTF-8 to environment while running tests -- Key: HIVE-2811 URL: https://issues.apache.org/jira/browse/HIVE-2811 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.9.0 Reporter: Ashutosh Chauhan Fix For: 0.9.0 Most OS has this value by default, but in some cases (on apache build machines) it has a different value making inputddl5.q test to fail. This variable should be exported while running the tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Hive build is back to green
Good idea! Created HIVE-2811 for this. Thanks, Ashutosh On Thu, Feb 16, 2012 at 14:41, Carl Steinbach c...@cloudera.com wrote: Great news! Thanks for fixing this Ashutosh! comparisons. Fix was to do export LANG=en_US.UTF-8 in environment of build machine. Is this something that we can set in Hive's build.xml file? Thanks. Carl
[jira] [Commented] (HIVE-2716) Move retry logic in HiveMetaStore to a separe class
[ https://issues.apache.org/jira/browse/HIVE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209984#comment-13209984 ] Phabricator commented on HIVE-2716: --- cwsteinbach has requested changes to the revision HIVE-2716 [jira] Move retry logic in HiveMetaStore to a separe class. INLINE COMMENTS metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:136 Is this change required, or are you just providing coverage for the new dropType(String, boolean) method? Were exceptions somehow getting masked earlier? metastore/src/java/org/apache/hadoop/hive/metastore/RetryingRawStore.java:38 It's not clear to me why HCatalog needs access to this class. Can you please provide more details? REVISION DETAIL https://reviews.facebook.net/D1227 BRANCH ms-retry-logic Move retry logic in HiveMetaStore to a separe class --- Key: HIVE-2716 URL: https://issues.apache.org/jira/browse/HIVE-2716 Project: Hive Issue Type: Sub-task Components: Metastore Affects Versions: 0.9.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HIVE-2716.D1227.1.patch, HIVE-2716.D1227.2.patch In HIVE-1219, method retrying for raw store operation are introduced to handle jdo operations more robustly. However, the abstraction for the RawStore operations can be moved to a separate class implementing RawStore, which should clean up the code base for HiveMetaStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
