[jira] [Updated] (HIVE-2933) analyze command throw NPE when table doesn't exists
[ https://issues.apache.org/jira/browse/HIVE-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ransom.hezhiqiang updated HIVE-2933: Attachment: HIVE-2933-0.8.1-1.patch analyze command throw NPE when table doesn't exists --- Key: HIVE-2933 URL: https://issues.apache.org/jira/browse/HIVE-2933 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.8.1 Reporter: alex gemini Priority: Minor Attachments: HIVE-2933-0.8.1-1.patch analyze command throw NPE when table doesn't exists -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2933) analyze command throw NPE when table doesn't exists
[ https://issues.apache.org/jira/browse/HIVE-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ransom.hezhiqiang updated HIVE-2933: Status: Patch Available (was: Open) analyze command throw NPE when table doesn't exists --- Key: HIVE-2933 URL: https://issues.apache.org/jira/browse/HIVE-2933 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.8.1 Reporter: alex gemini Priority: Minor Attachments: HIVE-2933-0.8.1-1.patch analyze command throw NPE when table doesn't exists -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2886) distinct with order by fails with Java SQL exception.
[ https://issues.apache.org/jira/browse/HIVE-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13249786#comment-13249786 ] ransom.hezhiqiang commented on HIVE-2886: - distinct can only effect on one table distinct with order by fails with Java SQL exception. - Key: HIVE-2886 URL: https://issues.apache.org/jira/browse/HIVE-2886 Project: Hive Issue Type: Bug Components: SQL Affects Versions: 0.7.1 Reporter: Mauro Cazzari The following select: select distinct TXT_1.`a`, TXT_1.`b` from `MYTAB` TXT_1 order by TXT_1.`a` asc fails with a Java SQL exception. Note that if the distinct or the table alias is removed from the SQL, the statement executes fine. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2910) Improve the HWI interface
[ https://issues.apache.org/jira/browse/HIVE-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13249819#comment-13249819 ] Edward Capriolo commented on HIVE-2910: --- CDH much be running an older version. That page has been removed from HWI entirely because that information is no longer available. Improve the HWI interface - Key: HIVE-2910 URL: https://issues.apache.org/jira/browse/HIVE-2910 Project: Hive Issue Type: Improvement Components: Web UI Reporter: Hugo Trippaers Assignee: Hugo Trippaers Priority: Minor Labels: newbie, patch Attachments: hive-2910.3.patch.log, hive-2910.3.patch.txt, hive-hwi-2.patch, hive-hwi.patch, screenie001.PNG, screenie002.PNG I've made some improvements to the HWI interface with the Twitter bootstrap system. I'm looking for feedback on the new design. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Professional Hiring: Architect and Developer in Hadoop Area ( Beijing, China )
国际著名大型IT企业（排名前3位）开发中心招聘Hadoop技术专家（北京）-非猎头 职位描述： Hadoop系统和平台开发（架构师，资深开发人员） 职位要求： 1.有设计开发大型分布式系统的经验（工作年限3年以上，架构师5年以上），hadoop大型实际应用经验优先 2.良好的编程和调试经验(java or c++/c)，扎实的计算机理论基础，快速的学习能力 3. 沟通和合作能力强，熟练使用英语（包括口语） *我们将提供有竞争力的待遇，欢迎加入我们* 有意请发简历到邮箱： sarah.lib...@gmail.com
[jira] [Commented] (HIVE-2928) Support for Oracle-backed Hive-Metastore (longvarchar to clob in package.jdo)
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13249946#comment-13249946 ] Mithun Radhakrishnan commented on HIVE-2928: @Ashutosh: The data-type change is in the package.jdo. I've verified that hive-jars built with CLOB work against a pre-existing MySQL metastore-server (whose datatypes haven't been changed). I didn't have to migrate. Support for Oracle-backed Hive-Metastore (longvarchar to clob in package.jdo) - Key: HIVE-2928 URL: https://issues.apache.org/jira/browse/HIVE-2928 Project: Hive Issue Type: New Feature Components: Metastore Affects Versions: 0.9.0 Reporter: Mithun Radhakrishnan Attachments: HIVE-2928.patch I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on. One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception: quote Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB (datastore), but type expected was LONGVARCHAR (metadata). Please check that the type in the datastore and the type specified in the MetaData are consistent. org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException: Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB (datastore), but type expected was LONGVARCHAR (metadata). Please check that the type in the datastore and the type specified in the MetaData are consistent. at org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521) at org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2 /quote But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising: 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.) 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry. 3. The unit-tests seem to run through. Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle? Mithun P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2910) Improve the HWI interface
[ https://issues.apache.org/jira/browse/HIVE-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13249972#comment-13249972 ] Hugo Trippaers commented on HIVE-2910: -- Ashutosh Chauhan, i don't know much about the license stuff. The images distributed as part of twitter bootstrap (http://twitter.github.com/bootstrap/) which i used to build the css. The images itself are from the site http://glyphicons.com/ and are release under the Creative Commons V3 (http://creativecommons.org/licenses/by/3.0/deed.en) Is this ok for inclusion? Improve the HWI interface - Key: HIVE-2910 URL: https://issues.apache.org/jira/browse/HIVE-2910 Project: Hive Issue Type: Improvement Components: Web UI Reporter: Hugo Trippaers Assignee: Hugo Trippaers Priority: Minor Labels: newbie, patch Attachments: hive-2910.3.patch.log, hive-2910.3.patch.txt, hive-hwi-2.patch, hive-hwi.patch, screenie001.PNG, screenie002.PNG I've made some improvements to the HWI interface with the Twitter bootstrap system. I'm looking for feedback on the new design. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1721) use bloom filters to improve the performance of joins
[ https://issues.apache.org/jira/browse/HIVE-1721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13249977#comment-13249977 ] Alan Gates commented on HIVE-1721: -- It is possible to do bloom filter generation in parallel, see Pig's BuildBloom UDF for an example. It does require a serial reduce phase, but it is quite small since it involves ORing the bitmaps from all of the Bloom filters built in each map phase. use bloom filters to improve the performance of joins - Key: HIVE-1721 URL: https://issues.apache.org/jira/browse/HIVE-1721 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Namit Jain Labels: gsoc, gsoc2012, optimization In case of map-joins, it is likely that the big table will not find many matching rows from the small table. Currently, we perform a hash-map lookup for every row in the big table, which can be pretty expensive. It might be useful to try out a bloom-filter containing all the elements in the small table. Each element from the big table is first searched in the bloom filter, and only in case of a positive match, the small table hash table is explored. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2504) Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory
[ https://issues.apache.org/jira/browse/HIVE-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated HIVE-2504: - Attachment: HIVE-2504-1.patch Added hive.warehouse.subdir.inherit.perms which is false by default. The default behaviour of hive-0.8 stays. i.e directories will be created with the permissions of dfs.umask or dfs.umaskmode. If hive.warehouse.subdir.inherit.perms is set to true, then table directories will inherit the permission of the default warehouse or the custom database location. This comes in handy when you have databases created with different permissions or if the warehouse directory has permissions like 775. Ashutosh, Your argument of warehouse having 700 permissions will not hold. If warehouse has 700 and even if the table directories have 755 or 775 they will not be accessible by any one other than the owner because if you don't have access to the parent directory you cannot access sub-directories in dfs (Same as Linux). So the warehouse has to be at least 755 or 750 to start with. So with the initial patch, the sub-directories would have been created with 775. But with my patch, they would have been created the same as warehouse directory(755 or 750) which would still allow read access to group. But anyways to avoid any confusion and provide backward compatibility added the new config. Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory - Key: HIVE-2504 URL: https://issues.apache.org/jira/browse/HIVE-2504 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Rohini Palaniswamy Fix For: 0.9.0 Attachments: HIVE-2504-1.patch, HIVE-2504.patch, HIVE-2504.patch When the Hive Metastore creates a subdirectory in the Hive warehouse for a new table it does so with the default HDFS permissions. Since the default dfs.umask value is 022, this means that the new subdirectory will not inherit the group write permissions of the hive warehouse directory. We should make the umask used by Warehouse.mkdirs() configurable, and set it to use a default value of 002. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2764) Obtain delegation tokens for MR jobs in secure hbase setup
[ https://issues.apache.org/jira/browse/HIVE-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2764: -- Attachment: HIVE-2764.D2205.5.patch enis updated the revision HIVE-2764 [jira] Obtain delegation tokens for MR jobs in secure hbase setup. Reviewers: JIRA, ashutoshc Fixed one more issue with the tests. All tests pass now. REVISION DETAIL https://reviews.facebook.net/D2205 AFFECTED FILES hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableOutputFormat.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHFileOutputFormat.java ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java ql/src/java/org/apache/hadoop/hive/ql/io/HiveNullValueSequenceFileOutputFormat.java ql/src/java/org/apache/hadoop/hive/ql/io/HiveOutputFormat.java ql/src/java/org/apache/hadoop/hive/ql/io/HiveOutputFormatImpl.java ql/src/java/org/apache/hadoop/hive/ql/io/HiveSequenceFileOutputFormat.java ql/src/java/org/apache/hadoop/hive/ql/io/RCFileOutputFormat.java ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java shims/src/common-secure/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java Obtain delegation tokens for MR jobs in secure hbase setup Key: HIVE-2764 URL: https://issues.apache.org/jira/browse/HIVE-2764 Project: Hive Issue Type: Improvement Components: HBase Handler, Security Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HIVE-2764.D2205.1.patch, HIVE-2764.D2205.2.patch, HIVE-2764.D2205.3.patch, HIVE-2764.D2205.4.patch, HIVE-2764.D2205.5.patch, HIVE-2764_v0.patch As discussed in HCATALOG-244, in a secure hbase setup with 0.92, we need to obtain delegation tokens for hbase and save it in jobconf, so that tasks can access region servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2935) Implement HiveServer2
Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250072#comment-13250072 ] Carl Steinbach commented on HIVE-2935: -- This is an umbrella JIRA for the HiveServer2 work. A proposal for the HiveServer2 Thrift API is available here: https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Thrift+API Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2504) Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory
[ https://issues.apache.org/jira/browse/HIVE-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2504: - Resolution: Fixed Assignee: Chinna Rao Lalam (was: Rohini Palaniswamy) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Closing this ticket again since Chinna's patch for HIVE-2504 was committed back in January. Please do not reopen tickets that have been marked fixed/resolved (i.e. tickets for which a patch has already been committed). Open a new ticket if you think there's a problem with the original patch. Thanks. Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory - Key: HIVE-2504 URL: https://issues.apache.org/jira/browse/HIVE-2504 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Chinna Rao Lalam Fix For: 0.9.0 Attachments: HIVE-2504-1.patch, HIVE-2504.patch, HIVE-2504.patch When the Hive Metastore creates a subdirectory in the Hive warehouse for a new table it does so with the default HDFS permissions. Since the default dfs.umask value is 022, this means that the new subdirectory will not inherit the group write permissions of the hive warehouse directory. We should make the umask used by Warehouse.mkdirs() configurable, and set it to use a default value of 002. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2936) Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory
Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory - Key: HIVE-2936 URL: https://issues.apache.org/jira/browse/HIVE-2936 Project: Hive Issue Type: Bug Components: Metastore Reporter: Rohini Palaniswamy Assignee: Chinna Rao Lalam Fix For: 0.9.0 Attachments: HIVE-2504-1.patch, HIVE-2504.patch, HIVE-2504.patch When the Hive Metastore creates a subdirectory in the Hive warehouse for a new table it does so with the default HDFS permissions. Since the default dfs.umask value is 022, this means that the new subdirectory will not inherit the group write permissions of the hive warehouse directory. We should make the umask used by Warehouse.mkdirs() configurable, and set it to use a default value of 002. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2936) Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory
[ https://issues.apache.org/jira/browse/HIVE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy reassigned HIVE-2936: Assignee: Rohini Palaniswamy (was: Chinna Rao Lalam) Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory - Key: HIVE-2936 URL: https://issues.apache.org/jira/browse/HIVE-2936 Project: Hive Issue Type: Bug Components: Metastore Reporter: Rohini Palaniswamy Assignee: Rohini Palaniswamy Fix For: 0.9.0 Attachments: HIVE-2504-1.patch, HIVE-2504.patch, HIVE-2504.patch When the Hive Metastore creates a subdirectory in the Hive warehouse for a new table it does so with the default HDFS permissions. Since the default dfs.umask value is 022, this means that the new subdirectory will not inherit the group write permissions of the hive warehouse directory. We should make the umask used by Warehouse.mkdirs() configurable, and set it to use a default value of 002. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2936) Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory
[ https://issues.apache.org/jira/browse/HIVE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated HIVE-2936: - Description: When the Hive Metastore creates a subdirectory in the Hive warehouse for a new table it does so with the default HDFS permissions derived from dfs.umask or dfs.umaskmode. There should be a option to inherit the permissions of the parent directory (default warehouse or custom database directory) so that the table directories have the same permissions as the database directories. was: When the Hive Metastore creates a subdirectory in the Hive warehouse for a new table it does so with the default HDFS permissions. Since the default dfs.umask value is 022, this means that the new subdirectory will not inherit the group write permissions of the hive warehouse directory. We should make the umask used by Warehouse.mkdirs() configurable, and set it to use a default value of 002. Issue Type: New Feature (was: Bug) Hadoop Flags: (was: Reviewed) Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory - Key: HIVE-2936 URL: https://issues.apache.org/jira/browse/HIVE-2936 Project: Hive Issue Type: New Feature Components: Metastore Reporter: Rohini Palaniswamy Assignee: Rohini Palaniswamy Fix For: 0.9.0 Attachments: HIVE-2504-1.patch, HIVE-2504.patch, HIVE-2504.patch When the Hive Metastore creates a subdirectory in the Hive warehouse for a new table it does so with the default HDFS permissions derived from dfs.umask or dfs.umaskmode. There should be a option to inherit the permissions of the parent directory (default warehouse or custom database directory) so that the table directories have the same permissions as the database directories. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2928) Support for Oracle-backed Hive-Metastore (longvarchar to clob in package.jdo)
[ https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250145#comment-13250145 ] Mithun Radhakrishnan commented on HIVE-2928: (Just re-verified that this is the case.) Support for Oracle-backed Hive-Metastore (longvarchar to clob in package.jdo) - Key: HIVE-2928 URL: https://issues.apache.org/jira/browse/HIVE-2928 Project: Hive Issue Type: New Feature Components: Metastore Affects Versions: 0.9.0 Reporter: Mithun Radhakrishnan Attachments: HIVE-2928.patch I'm trying to get the Hive-Metastore to work when backed by an Oracle backend. There's a change to hive's package.jdo that I'd like advice/comments on. One sticking point on working with Oracle has been the TBLS table (MTable) and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT). Oracle doesn't support more than one LONGVARCHAR property per table (for reason of legacy), and prefers that one use CLOBs instead. If one switches to CLOB properties, with no modification to hive's package.jdo, one sees the following exception: quote Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB (datastore), but type expected was LONGVARCHAR (metadata). Please check that the type in the datastore and the type specified in the MetaData are consistent. org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException: Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB (datastore), but type expected was LONGVARCHAR (metadata). Please check that the type in the datastore and the type specified in the MetaData are consistent. at org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521) at org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2 /quote But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of LONGVARCHARs, things look promising: 1. The exception no longer occurs. Things seem to work with Oracle. (I've yet to scale-test.) 2. These modified hive-libraries work as is with pre-existing mysql metastores. Migrating data isn't a worry. 3. The unit-tests seem to run through. Would there be opposition to changing the package.jdo's LONGVARCHAR references to CLOB, if this works with mysql and with Oracle? Mithun P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm testing, for the related issue of creating the required tables in Oracle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1805) Ability to create dynamic partitions atomically
[ https://issues.apache.org/jira/browse/HIVE-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1805: - Component/s: Metastore Ability to create dynamic partitions atomically --- Key: HIVE-1805 URL: https://issues.apache.org/jira/browse/HIVE-1805 Project: Hive Issue Type: New Feature Components: Metastore Reporter: Namit Jain -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2937) TestHiveServerSessions hangs when executed directly
[
https://issues.apache.org/jira/browse/HIVE-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2937:
Description:
{code}
ant test -Doffline=true -Dtestcase=TestHiveServerSessions
{code}
Hangs infinitely.
I couldn't imagine exact cause of the problem, but found that by adding 'new
HiveServer.HiveServerHandler();' in setup(), test resulted to success.
was:
code}
ant test -Doffline=true -Dtestcase=TestHiveServerSessions
{code}
Hangs infinitely.
I couldn't imagine exact cause of the problem, but found that by adding 'new
HiveServer.HiveServerHandler();' in setup(), test resulted to success.
TestHiveServerSessions hangs when executed directly
---
Key: HIVE-2937
URL: https://issues.apache.org/jira/browse/HIVE-2937
Project: Hive
Issue Type: Test
Reporter: Navis
Priority: Trivial
{code}
ant test -Doffline=true -Dtestcase=TestHiveServerSessions
{code}
Hangs infinitely.
I couldn't imagine exact cause of the problem, but found that by adding 'new
HiveServer.HiveServerHandler();' in setup(), test resulted to success.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2938) ALTER TABLE ADD IF NOT EXISTS PARTITION is not atomic
ALTER TABLE ADD IF NOT EXISTS PARTITION is not atomic - Key: HIVE-2938 URL: https://issues.apache.org/jira/browse/HIVE-2938 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.8.1, 0.8.0, 0.7.0, 0.6.0, 0.5.0 Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2084) Upgrade datanucleus from 2.0.3 to 3.0.1
[ https://issues.apache.org/jira/browse/HIVE-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250161#comment-13250161 ] Ashutosh Chauhan commented on HIVE-2084: @Sushanth, You have two patches up, one via phabricator another directly on jira and they have quite bit of differences among them. Which one you are proposing for review? Upgrade datanucleus from 2.0.3 to 3.0.1 --- Key: HIVE-2084 URL: https://issues.apache.org/jira/browse/HIVE-2084 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Ning Zhang Assignee: Sushanth Sowmyan Labels: datanucleus Attachments: HIVE-2084.1.patch.txt, HIVE-2084.2.patch.txt, HIVE-2084.D2397.1.patch, HIVE-2084.patch It seems the datanucleus 2.2.3 does a better join in caching. The time it takes to get the same set of partition objects takes about 1/4 of the time it took for the first time. While with 2.0.3, it took almost the same amount of time in the second execution. We should retest the test case mentioned in HIVE-1853, HIVE-1862. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2938) ALTER TABLE ADD IF NOT EXISTS PARTITION is not atomic
[ https://issues.apache.org/jira/browse/HIVE-2938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250162#comment-13250162 ] Carl Steinbach commented on HIVE-2938: -- Support for ALTER TABLE ADD IF NOT EXISTS PARTITION was added in HIVE-1106. However, this patch introduced a time-of-check-to-time-of-use bug since the check to see if the partition already exists is done in a transaction separate from the one in which the partition is actually created. We need to provide support for IF NOT EXISTS at the level of the Metastore Thrift API in order to support this correctly. ALTER TABLE ADD IF NOT EXISTS PARTITION is not atomic - Key: HIVE-2938 URL: https://issues.apache.org/jira/browse/HIVE-2938 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.5.0, 0.6.0, 0.7.0, 0.8.0, 0.8.1 Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2736) Hive UDFs cannot emit binary constants
[ https://issues.apache.org/jira/browse/HIVE-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Tromans updated HIVE-2736: - Attachment: HIVE-2736.1.patch.txt Hive UDFs cannot emit binary constants -- Key: HIVE-2736 URL: https://issues.apache.org/jira/browse/HIVE-2736 Project: Hive Issue Type: Bug Components: Query Processor, Serializers/Deserializers, UDF Affects Versions: 0.9.0 Reporter: Philip Tromans Assignee: Philip Tromans Priority: Minor Labels: newbie Fix For: 0.9.0 Attachments: HIVE-2736.1.patch.txt Original Estimate: 4h Remaining Estimate: 4h I recently wrote a UDF which emits BINARY values (as implemented in [HIVE-2380|https://issues.apache.org/jira/browse/HIVE-2380]). When testing this, I encountered the following exception (because I was evaluating f(g(constant string))) and g() was emitting a BytesWritable type. FAILED: Hive Internal Error: java.lang.RuntimeException(Internal error: Cannot find ConstantObjectInspector for BINARY) java.lang.RuntimeException: Internal error: Cannot find ConstantObjectInspector for BINARY at org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory.getPrimitiveWritableConstantObjectInspector(PrimitiveObjectInspectorFactory.java:196) at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getConstantObjectInspector(ObjectInspectorUtils.java:899) at org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:128) at org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:214) at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:684) at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:805) at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:89) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:88) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:125) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:102) at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:161) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:7708) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:2301) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:2103) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:6126) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:6097) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6723) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7484) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:243) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:430) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:889) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) It looks like a pretty simple fix - add a case for BINARY in PrimitiveObjectInspectorFactory.getPrimitiveWritableConstantObjectInspector() and implement a WritableConstantByteArrayObjectInspector class (almost identical to the others). I'm happy to do this, although this is my first foray into the world of contributing to FOSS so I might end up asking a few stupid questions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators:
Hive-0.8.1-SNAPSHOT-h0.21 - Build # 248 - Failure
Changes for Build #247 Changes for Build #248 1 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:50) at org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1(TestNegativeCliDriver.java:9440) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) The Apache Jenkins build system has built Hive-0.8.1-SNAPSHOT-h0.21 (build #248) Status: Failure Check console output at https://builds.apache.org/job/Hive-0.8.1-SNAPSHOT-h0.21/248/ to view the results.
[jira] [Commented] (HIVE-1977) DESCRIBE TABLE syntax doesn't support specifying a database qualified table name
[
https://issues.apache.org/jira/browse/HIVE-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250180#comment-13250180
]
Zhenxiao Luo commented on HIVE-1977:
currently the following is working:
1. $describe table;
2. $describe table.column;
while, these are not working:
3. $describe column;
4. $describe database.table;
5. $describe database.table.column;
always errors about the starting element is not a valid Table.
Hive should support #3 and #5, since these are standard syntax, by updating
Hive.g and related parser/lexer.
For #4, a new configuration property may be needed to distinguish #2 with #4.
Or, shall we always try matching both #2 and #4 when doing parsing? And error
out if and only if neither matches? In this way, no new configuration property
needed.
This is also related to HIVE-2228.
DESCRIBE TABLE syntax doesn't support specifying a database qualified table
name
Key: HIVE-1977
URL: https://issues.apache.org/jira/browse/HIVE-1977
Project: Hive
Issue Type: Bug
Components: Database/Schema, Query Processor, SQL
Reporter: Carl Steinbach
Assignee: Zhenxiao Luo
The syntax for DESCRIBE is broken. It should be:
{code}
DESCRIBE [EXTENDED] [database DOT]table [column]
{code}
but is actually
{code}
DESCRIBE [EXTENDED] table[DOT col_name]
{code}
Ref: http://dev.mysql.com/doc/refman/5.0/en/describe.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1977) DESCRIBE TABLE syntax doesn't support specifying a database qualified table name
[
https://issues.apache.org/jira/browse/HIVE-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250193#comment-13250193
]
Carl Steinbach commented on HIVE-1977:
--
bq. 3. $describe column;
This doesn't make sense since tables in the same schema are allowed to have
columns with the same name. The correct syntax for describing a column is:
DESCRIBE table column
or
DESCRIBE db.table column
DESCRIBE TABLE syntax doesn't support specifying a database qualified table
name
Key: HIVE-1977
URL: https://issues.apache.org/jira/browse/HIVE-1977
Project: Hive
Issue Type: Bug
Components: Database/Schema, Query Processor, SQL
Reporter: Carl Steinbach
Assignee: Zhenxiao Luo
The syntax for DESCRIBE is broken. It should be:
{code}
DESCRIBE [EXTENDED] [database DOT]table [column]
{code}
but is actually
{code}
DESCRIBE [EXTENDED] table[DOT col_name]
{code}
Ref: http://dev.mysql.com/doc/refman/5.0/en/describe.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2914) HiveConnection constructor ignores passed-in properties object
[
https://issues.apache.org/jira/browse/HIVE-2914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-2914:
--
Attachment: HIVE-2914.D2691.1.patch
navis requested code review of HIVE-2914 [jira] HiveConnection constructor
ignores passed-in properties object.
Reviewers: JIRA
DPAL-1078 HiveConnection constructor ignores passed-in properties object
In local mode HiveConf should initialize itself with passed in properties and
in remote mode, connection should execute series of set command for all the
properties.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D2691
AFFECTED FILES
cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java
service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/6177/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
HiveConnection constructor ignores passed-in properties object
--
Key: HIVE-2914
URL: https://issues.apache.org/jira/browse/HIVE-2914
Project: Hive
Issue Type: Bug
Components: JDBC
Reporter: Ashutosh Chauhan
Attachments: HIVE-2914.D2691.1.patch
In local mode HiveConf should initialize itself with passed in properties and
in remote mode, connection should execute series of {{set}} command for all
the properties.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1977) DESCRIBE TABLE syntax doesn't support specifying a database qualified table name
[
https://issues.apache.org/jira/browse/HIVE-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250203#comment-13250203
]
Zhenxiao Luo commented on HIVE-1977:
@Carl: Get it. Should follow the correct syntax.
In summary, should support the following:
1. $describe table;
2. $describe database.table;
3. $describe table column;
4. $describe database.table column;
5. $describe table.column;
6. $describe database.table.column;
Currently,
#1 is working
#2 is error out for database not a valid table name
#3 and #4 are having exceptions for invalid syntax
#5 is working
How about #6, are we going to support it?
currently, #6 also error out for database not a valid table name
And, to distinguish #2 and #5, a new configuration property is needed, or we
try both, and error out only if neither matches?
DESCRIBE TABLE syntax doesn't support specifying a database qualified table
name
Key: HIVE-1977
URL: https://issues.apache.org/jira/browse/HIVE-1977
Project: Hive
Issue Type: Bug
Components: Database/Schema, Query Processor, SQL
Reporter: Carl Steinbach
Assignee: Zhenxiao Luo
The syntax for DESCRIBE is broken. It should be:
{code}
DESCRIBE [EXTENDED] [database DOT]table [column]
{code}
but is actually
{code}
DESCRIBE [EXTENDED] table[DOT col_name]
{code}
Ref: http://dev.mysql.com/doc/refman/5.0/en/describe.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2937) TestHiveServerSessions hangs when executed directly
[
https://issues.apache.org/jira/browse/HIVE-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-2937:
--
Attachment: HIVE-2937.D2697.1.patch
navis requested code review of HIVE-2937 [jira] TestHiveServerSessions hangs
when executed directly.
Reviewers: JIRA
DPAL-1079 TestHiveServerSessions hangs when executed directly
ant test -Doffline=true -Dtestcase=TestHiveServerSessions
Hangs infinitely.
I couldn't imagine exact cause of the problem, but found that by adding 'new
HiveServer.HiveServerHandler();' in setup(), test resulted to success.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D2697
AFFECTED FILES
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/6183/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
TestHiveServerSessions hangs when executed directly
---
Key: HIVE-2937
URL: https://issues.apache.org/jira/browse/HIVE-2937
Project: Hive
Issue Type: Test
Reporter: Navis
Priority: Trivial
Attachments: HIVE-2937.D2697.1.patch
{code}
ant test -Doffline=true -Dtestcase=TestHiveServerSessions
{code}
Hangs infinitely.
I couldn't imagine exact cause of the problem, but found that by adding 'new
HiveServer.HiveServerHandler();' in setup(), test resulted to success.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2937) TestHiveServerSessions hangs when executed directly
[
https://issues.apache.org/jira/browse/HIVE-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250238#comment-13250238
]
Navis commented on HIVE-2937:
-
@Ashutosh Chauhan
It was related to initialization of metastore. Just added static sync in
HMSHandler#createDefaultDB and seemed enough to solve this.
TestHiveServerSessions hangs when executed directly
---
Key: HIVE-2937
URL: https://issues.apache.org/jira/browse/HIVE-2937
Project: Hive
Issue Type: Test
Reporter: Navis
Priority: Trivial
Attachments: HIVE-2937.D2697.1.patch
{code}
ant test -Doffline=true -Dtestcase=TestHiveServerSessions
{code}
Hangs infinitely.
I couldn't imagine exact cause of the problem, but found that by adding 'new
HiveServer.HiveServerHandler();' in setup(), test resulted to success.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2937) TestHiveServerSessions hangs when executed directly
[
https://issues.apache.org/jira/browse/HIVE-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis reassigned HIVE-2937:
---
Assignee: Navis
TestHiveServerSessions hangs when executed directly
---
Key: HIVE-2937
URL: https://issues.apache.org/jira/browse/HIVE-2937
Project: Hive
Issue Type: Test
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-2937.D2697.1.patch
{code}
ant test -Doffline=true -Dtestcase=TestHiveServerSessions
{code}
Hangs infinitely.
I couldn't imagine exact cause of the problem, but found that by adding 'new
HiveServer.HiveServerHandler();' in setup(), test resulted to success.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2937) TestHiveServerSessions hangs when executed directly
[
https://issues.apache.org/jira/browse/HIVE-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2937:
Status: Patch Available (was: Open)
TestHiveServerSessions hangs when executed directly
---
Key: HIVE-2937
URL: https://issues.apache.org/jira/browse/HIVE-2937
Project: Hive
Issue Type: Test
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-2937.D2697.1.patch
{code}
ant test -Doffline=true -Dtestcase=TestHiveServerSessions
{code}
Hangs infinitely.
I couldn't imagine exact cause of the problem, but found that by adding 'new
HiveServer.HiveServerHandler();' in setup(), test resulted to success.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2937) TestHiveServerSessions hangs when executed directly
[
https://issues.apache.org/jira/browse/HIVE-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250243#comment-13250243
]
Ashutosh Chauhan commented on HIVE-2937:
@Navis
Can you add Namit in reviewer list?
TestHiveServerSessions hangs when executed directly
---
Key: HIVE-2937
URL: https://issues.apache.org/jira/browse/HIVE-2937
Project: Hive
Issue Type: Test
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-2937.D2697.1.patch
{code}
ant test -Doffline=true -Dtestcase=TestHiveServerSessions
{code}
Hangs infinitely.
I couldn't imagine exact cause of the problem, but found that by adding 'new
HiveServer.HiveServerHandler();' in setup(), test resulted to success.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2914) HiveConnection constructor ignores passed-in properties object
[
https://issues.apache.org/jira/browse/HIVE-2914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis reassigned HIVE-2914:
---
Assignee: Navis
HiveConnection constructor ignores passed-in properties object
--
Key: HIVE-2914
URL: https://issues.apache.org/jira/browse/HIVE-2914
Project: Hive
Issue Type: Bug
Components: JDBC
Reporter: Ashutosh Chauhan
Assignee: Navis
Attachments: HIVE-2914.D2691.1.patch
In local mode HiveConf should initialize itself with passed in properties and
in remote mode, connection should execute series of {{set}} command for all
the properties.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2918) Hive Dynamic Partition Insert - move task not considering 'hive.exec.max.dynamic.partitions' from CLI
[ https://issues.apache.org/jira/browse/HIVE-2918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2918: -- Attachment: HIVE-2918.D2703.1.patch cwsteinbach requested code review of HIVE-2918 [jira] Hive Dynamic Partition Insert - move task not considering 'hive.exec.max.dynamic.partitions' from CLI. Reviewers: JIRA HIVE-2918. Hive Dynamic Partition Insert - move task not considering 'hive.exec.max.dynamic.partitions' from CLI Dynamic Partition insert showing an error with the number of partitions created even after the default value of 'hive.exec.max.dynamic.partitions' is bumped high to 2000. Error Message: Failed with exception Number of dynamic partitions created is 1413, which is more than 1000. To solve this try to set hive.exec.max.dynamic.partitions to at least 1413. These are the following properties set on hive CLI hive set hive.exec.dynamic.partition=true; hive set hive.exec.dynamic.partition.mode=nonstrict; hive set hive.exec.max.dynamic.partitions=2000; hive set hive.exec.max.dynamic.partitions.pernode=2000; This is the query with console error log hive INSERT OVERWRITE TABLE partn_dyn Partition (pobox) SELECT country,state,pobox FROM non_partn_dyn; Total MapReduce jobs = 2 Launching Job 1 out of 2 Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_201204021529_0002, Tracking URL = http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201204021529_0002 Kill Command = /usr/lib/hadoop/bin/hadoop job -Dmapred.job.tracker=0.0.0.0:8021 -kill job_201204021529_0002 2012-04-02 16:05:28,619 Stage-1 map = 0%, reduce = 0% 2012-04-02 16:05:39,701 Stage-1 map = 100%, reduce = 0% 2012-04-02 16:05:50,800 Stage-1 map = 100%, reduce = 100% Ended Job = job_201204021529_0002 Ended Job = 248865587, job is filtered out (removed at runtime). Moving data to: hdfs://0.0.0.0/tmp/hive-cloudera/hive_2012-04-02_16-05-24_919_5976014408587784412/-ext-1 Loading data to table default.partn_dyn partition (pobox=null) Failed with exception Number of dynamic partitions created is 1413, which is more than 1000. To solve this try to set hive.exec.max.dynamic.partitions to at least 1413. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask I checked the job.xml of the first map only job, there the value hive.exec.max.dynamic.partitions=2000 is reflected but the move task is taking the default value from hive-site.xml . If I change the value in hive-site.xml then the job completes successfully. Bottom line,the property 'hive.exec.max.dynamic.partitions'set on CLI is not being considered by move task TEST PLAN NONE REVISION DETAIL https://reviews.facebook.net/D2703 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java ql/src/test/queries/clientnegative/dyn_part_max.q ql/src/test/queries/clientnegative/dyn_part_max_per_node.q ql/src/test/results/clientnegative/dyn_part_max.q.out ql/src/test/results/clientnegative/dyn_part_max_per_node.q.out MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/6189/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. Hive Dynamic Partition Insert - move task not considering 'hive.exec.max.dynamic.partitions' from CLI - Key: HIVE-2918 URL: https://issues.apache.org/jira/browse/HIVE-2918 Project: Hive Issue Type: Bug Affects Versions: 0.7.1, 0.8.0, 0.8.1 Environment: Cent OS 64 bit Reporter: Bejoy KS Assignee: Carl Steinbach Attachments: HIVE-2918.D2703.1.patch Dynamic Partition insert showing an error with the number of partitions created even after the default value of 'hive.exec.max.dynamic.partitions' is bumped high to 2000. Error Message: Failed with exception Number of dynamic partitions created is 1413, which is more than 1000. To solve this try to set hive.exec.max.dynamic.partitions to at least 1413. These are the following properties set on hive CLI hive set hive.exec.dynamic.partition=true; hive set hive.exec.dynamic.partition.mode=nonstrict; hive set hive.exec.max.dynamic.partitions=2000; hive set hive.exec.max.dynamic.partitions.pernode=2000; This is the query with console error log hive INSERT OVERWRITE TABLE partn_dyn Partition (pobox) SELECT country,state,pobox FROM non_partn_dyn; Total MapReduce jobs = 2 Launching Job 1 out of 2 Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_201204021529_0002, Tracking URL = http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201204021529_0002
Re: Hive 0.9 release
As per the plan I am going to create a branch now. Please hold off any commits till I send an email for all-clear. Thanks, Ashutosh On Fri, Apr 6, 2012 at 15:00, Owen O'Malley omal...@apache.org wrote: I think we also need to get the RAT report cleaned up. Apache projects aren't supposed to release while they have files without the Apache header. I've filed HIVE-2930 to fix all of the issues. While working on it, I found that one of the files was added by HIVE-2246. HIVE-2246 was contributed by Sohan Jain, who hasn't filed an ICLA, and doesn't have the jira box checked for contribution. Does someone know him and can ask him to state on the jira that he intended to contribute it? Failing that, I believe he was working at Facebook at the time, so someone else who is still there can upload the patch to the jira? All of this brings up a challenge in that Phabricator and the Apache review tool upload patches to jira without providing a way to check the contribute to Apache box. Without the checkbox we should only commit patches from people who have filed ICLAs. Is there a way to add an option the arc command that will check the box? Even having it *always* check the box is better than having it not check the box. (Although it should warn users that it is doing so.) Thoughts? -- Owen
[jira] [Updated] (HIVE-2858) Cache remote map reduce job stack traces for additional logging
[ https://issues.apache.org/jira/browse/HIVE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2858: --- Fix Version/s: 0.9.0 Cache remote map reduce job stack traces for additional logging --- Key: HIVE-2858 URL: https://issues.apache.org/jira/browse/HIVE-2858 Project: Hive Issue Type: Improvement Components: Logging Reporter: Kevin Wilfong Assignee: Kevin Wilfong Fix For: 0.9.0 Attachments: HIVE-2858.D2223.1.patch, HIVE-2858.D2223.2.patch Currently we are parsing the task logs for failed jobs for information to display to the user in the CLI. In addition, we could parse those logs for stack traces and store e them in the SessionState. This way, when we log failed queries, these will give us a decent idea of why those queries failed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2929) race condition in DAG execute tasks for hive
[ https://issues.apache.org/jira/browse/HIVE-2929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2929: --- Fix Version/s: 0.10 race condition in DAG execute tasks for hive Key: HIVE-2929 URL: https://issues.apache.org/jira/browse/HIVE-2929 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Fix For: 0.10 Attachments: hive.2929.1.patch select ... ( SubQuery involving MapReduce union all SubQuery involving MapReduce ); or select ... (SubQuery involving MapReduce) join (SubQuery involving MapReduce) ; If both the subQueries finish at nearly the same time, there is a race condition in which the results of the subQuery finishing last will be completely missed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Hive 0.9 release
All-clear. Trunk is now open for commits. Since HIVE-2929 have resulted in intermittent test failures. (See, HIVE-2937), I branched right before HIVE-2929. Additionally, I merged in HIVE-2764 in 0.9 I also added version 0.10 on jira, so any commits on trunk now must have 0.10 as fix version. Thanks, Ashutosh On Mon, Apr 9, 2012 at 17:01, Ashutosh Chauhan hashut...@apache.org wrote: As per the plan I am going to create a branch now. Please hold off any commits till I send an email for all-clear. Thanks, Ashutosh On Fri, Apr 6, 2012 at 15:00, Owen O'Malley omal...@apache.org wrote: I think we also need to get the RAT report cleaned up. Apache projects aren't supposed to release while they have files without the Apache header. I've filed HIVE-2930 to fix all of the issues. While working on it, I found that one of the files was added by HIVE-2246. HIVE-2246 was contributed by Sohan Jain, who hasn't filed an ICLA, and doesn't have the jira box checked for contribution. Does someone know him and can ask him to state on the jira that he intended to contribute it? Failing that, I believe he was working at Facebook at the time, so someone else who is still there can upload the patch to the jira? All of this brings up a challenge in that Phabricator and the Apache review tool upload patches to jira without providing a way to check the contribute to Apache box. Without the checkbox we should only commit patches from people who have filed ICLAs. Is there a way to add an option the arc command that will check the box? Even having it *always* check the box is better than having it not check the box. (Although it should warn users that it is doing so.) Thoughts? -- Owen
[jira] [Commented] (HIVE-2928) Support for Oracle-backed Hive-Metastore (longvarchar to clob in package.jdo)
[
https://issues.apache.org/jira/browse/HIVE-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250361#comment-13250361
]
Carl Steinbach commented on HIVE-2928:
--
@Mithun: What happens on MySQL if you turn column validation on (e.g.
datanucleus.validateColumns=true at startup)? There's a note on the Datanucleus
site that indicates this won't work:
http://www.datanucleus.org/products/datanucleus/rdbms/support.html
{quote}
You can specify BLOB, CLOB JDBC types when using MySQL with DataNucleus but
you must turn validation of columns OFF. This is because these types are not
supported by the MySQL JDBC driver and it returns them as
LONGVARBINARY/LONGVARCHAR when querying the column type.
{quote}
Support for Oracle-backed Hive-Metastore (longvarchar to clob in
package.jdo)
-
Key: HIVE-2928
URL: https://issues.apache.org/jira/browse/HIVE-2928
Project: Hive
Issue Type: New Feature
Components: Metastore
Affects Versions: 0.9.0
Reporter: Mithun Radhakrishnan
Attachments: HIVE-2928.patch
I'm trying to get the Hive-Metastore to work when backed by an Oracle
backend. There's a change to hive's package.jdo that I'd like advice/comments
on.
One sticking point on working with Oracle has been the TBLS table (MTable)
and its 2 LONGVARCHAR properties (VIEW_ORIGINAL_TEXT and VIEW_EXPANDED_TEXT).
Oracle doesn't support more than one LONGVARCHAR property per table (for
reason of legacy), and prefers that one use CLOBs instead. If one switches to
CLOB properties, with no modification to hive's package.jdo, one sees the
following exception:
quote
Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
(datastore), but type expected was LONGVARCHAR (metadata). Please check that
the type in the datastore and the type specified in the MetaData are
consistent.
org.datanucleus.store.rdbms.exceptions.IncompatibleDataTypeException:
Incompatible data type for column TBLS.VIEW_EXPANDED_TEXT : was CLOB
(datastore), but type expected was LONGVARCHAR (metadata). Please check that
the type in the datastore and the type specified in the MetaData are
consistent.
at
org.datanucleus.store.rdbms.table.ColumnImpl.validate(ColumnImpl.java:521)
at
org.datanucleus.store.rdbms.table.TableImpl.validateColumns(TableImpl.java:2
/quote
But if one rebuilds Hive with the package.jdo changed to use CLOBs instead of
LONGVARCHARs, things look promising:
1. The exception no longer occurs. Things seem to work with Oracle. (I've yet
to scale-test.)
2. These modified hive-libraries work as is with pre-existing mysql
metastores. Migrating data isn't a worry.
3. The unit-tests seem to run through.
Would there be opposition to changing the package.jdo's LONGVARCHAR
references to CLOB, if this works with mysql and with Oracle?
Mithun
P.S. I also have a working hive-schema-0.9.0-oracle.sql script that I'm
testing, for the related issue of creating the required tables in Oracle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2764) Obtain delegation tokens for MR jobs in secure hbase setup
[ https://issues.apache.org/jira/browse/HIVE-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250410#comment-13250410 ] Hudson commented on HIVE-2764: -- Integrated in Hive-trunk-h0.21 #1364 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1364/]) HIVE-2764: Obtain delegation tokens for MR jobs in secure hbase setup (Enis Soztutar via Ashutosh Chauhan) (Revision 1311418) Result = SUCCESS hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1311418 Files : * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableOutputFormat.java * /hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHFileOutputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveNullValueSequenceFileOutputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveOutputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveOutputFormatImpl.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveSequenceFileOutputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFileOutputFormat.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java * /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java * /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java * /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java Obtain delegation tokens for MR jobs in secure hbase setup Key: HIVE-2764 URL: https://issues.apache.org/jira/browse/HIVE-2764 Project: Hive Issue Type: Improvement Components: HBase Handler, Security Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.9.0 Attachments: HIVE-2764.D2205.1.patch, HIVE-2764.D2205.2.patch, HIVE-2764.D2205.3.patch, HIVE-2764.D2205.4.patch, HIVE-2764.D2205.5.patch, HIVE-2764_v0.patch As discussed in HCATALOG-244, in a secure hbase setup with 0.92, we need to obtain delegation tokens for hbase and save it in jobconf, so that tasks can access region servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2939) LazyArray.getList changes array it previously returned
[ https://issues.apache.org/jira/browse/HIVE-2939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Kabiljo updated HIVE-2939: --- Priority: Minor (was: Major) LazyArray.getList changes array it previously returned --- Key: HIVE-2939 URL: https://issues.apache.org/jira/browse/HIVE-2939 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.8.1 Reporter: Igor Kabiljo Priority: Minor Simple query like: SELECT a, e FROM ikabiljo_test_string_array LATERAL VIEW EXPLODE(a) x1 AS e (table contains one column - ARRAYSTRING - and has one row - [b,c,a] ) fails with ConcurrentModificationException, since LazyArray.getList changes the cached array it returns. LazyArray.getList can easily: - return cached array if present, without clearing and refiling. Hive is already not going to work properly if you change input parameters in an UDF. If that doesn't sound good - it can return Collections.unmodifiableList - or just not cache anything Same is true for LazyMap.getMap -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-2939) LazyArray.getList changes array it previously returned
LazyArray.getList changes array it previously returned --- Key: HIVE-2939 URL: https://issues.apache.org/jira/browse/HIVE-2939 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.8.1 Reporter: Igor Kabiljo Simple query like: SELECT a, e FROM ikabiljo_test_string_array LATERAL VIEW EXPLODE(a) x1 AS e (table contains one column - ARRAYSTRING - and has one row - [b,c,a] ) fails with ConcurrentModificationException, since LazyArray.getList changes the cached array it returns. LazyArray.getList can easily: - return cached array if present, without clearing and refiling. Hive is already not going to work properly if you change input parameters in an UDF. If that doesn't sound good - it can return Collections.unmodifiableList - or just not cache anything Same is true for LazyMap.getMap -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
