Re: [ANNOUNCE] New Hive Committer - Yin Huai
Amazing news, congratz Yin! Well deserved! On Sep 4, 2013, at 6:49 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Yin Huai a committer on the Apache Hive project. Please join me in congratulating Yin! Thanks. Carl -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF
Re: VOTE: moving hive from forest to Apache CMS
+1 (non-binding) On Jul 21, 2013, at 8:25 PM, Mark Grover grover.markgro...@gmail.com wrote: +1 (non-binding) On Sun, Jul 21, 2013 at 11:08 AM, Jarek Jarcec Cecho jar...@apache.org wrote: +1 (non-binding) Jarcec On Sun, Jul 21, 2013 at 01:53:39PM -0400, Edward Capriolo wrote: http://hive.apache.org is generated by forest, a rather cumbersome and confusing way to run a website. Forest is difficult to maintain and publish updates with. As a nail in the coffin forest does not even work well with recent versions of java. This vote is to move the site to: Apache CMShttps://www.apache.org/dev/cms.html and away from forest. Brock Noland has offered to move the site, and I am offering to help him and look it over. Vote +1 if you support the move to Apache CMS. (This is the one case where cutting down a forest is a very good idea :) Edward -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF
Re: [ANNOUNCE] New Hive Committer - Gunther Hagleitner
Congratulations, Gunther. Well deserved! On Jul 21, 2013, at 10:00 AM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Gunther Hagleitner a committer on the Apache Hive project. Congratulations Gunther! Carl -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF
Re: [DISCUSS] moving website to Apache CMS
+1 (non - binding) :) Agree with Ed. Thanks to bringing this to attention, Brock. On Jul 18, 2013, at 7:37 AM, Edward Capriolo edlinuxg...@gmail.com wrote: I like the idea. markdown is easy to learn. The correct system uses forest which is a big PITA.
Re: [ANNOUNCE] New Hive Committer - Brock Noland
Congratulations, Brock! On Jul 15, 2013, at 10:29 PM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has passed a vote to make Brock Noland a new committer on the project. Brock, please submit your ICLA to the Apache Software Foundation as described here: http://www.apache.org/licenses/#clas Thanks. Carl
Re: [VOTE] Apache Hive 0.11.0 Release Candidate 2
+1 (non-binding) - downloaded and verified checksum - build works - test works - Alex On May 17, 2013, at 9:30 PM, Ashish Thusoo athu...@qubole.com wrote: +1 Downloaded tar-ball, ran some tests and a few test queries. Ashish Ashish Thusoo http://www.linkedin.com/pub/ashish-thusoo/0/5a8/50 CEO and Co-founder, Qubole http://www.qubole.com - a cloud based service that makes big data easy for analysts and data engineers On Wed, May 15, 2013 at 8:13 PM, Jitendra Pandey jiten...@hortonworks.comwrote: +1 (non binding) Downloaded tar-ball, verified checksums, installed one node cluster and ran a few queries. On Wed, May 15, 2013 at 7:53 PM, Edward Capriolo edlinuxg...@gmail.com wrote: +1 On Wed, May 15, 2013 at 10:44 PM, Ashutosh Chauhan hashut...@apache.org wrote: +1 Built from sources, ran few unit tests and some simple queries against 1-node cluster. Thanks, Ashutosh On Wed, May 15, 2013 at 5:30 PM, Navis류승우 navis@nexr.com wrote: +1 - built from source, passed all tests (without assertion fail or conversion to backup task) - working good with queries from running-site - some complaints on missing HIVE-4172 (void type for JDBC2) Thanks 2013/5/12 Owen O'Malley omal...@apache.org: Based on feedback from everyone, I have respun release candidate, RC2. Please take a look. We've fixed 7 problems with the previous RC: * Release notes were incorrect * HIVE-4018 - MapJoin failing with Distributed Cache error * HIVE-4421 - Improve memory usage by ORC dictionaries * HIVE-4500 - Ensure that HiveServer 2 closes log files. * HIVE-4494 - ORC map columns get class cast exception in some contexts * HIVE-4498 - Fix TestBeeLineWithArgs failure * HIVE-4505 - Hive can't load transforms with remote scripts * HIVE-4527 - Fix the eclipse template Source tag for RC2 is at: https://svn.apache.org/repos/asf/hive/tags/release-0.11.0rc2 Source tar ball and convenience binary artifacts can be found at: http://people.apache.org/~omalley/hive-0.11.0rc2/ This release has many goodies including HiveServer2, integrated hcatalog, windowing and analytical functions, decimal data type, better query planning, performance enhancements and various bug fixes. In total, we resolved more than 350 issues. Full list of fixed issues can be found at: http://s.apache.org/8Fr Voting will conclude in 72 hours. Hive PMC Members: Please test and vote. Thanks, Owen -- http://hortonworks.com/download/ -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF
[jira] [Commented] (HIVE-3983) Select on table with hbase storage handler fails with an SASL error
[
https://issues.apache.org/jira/browse/HIVE-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13651833#comment-13651833
]
Alexander Alten-Lorenz commented on HIVE-3983:
--
Add the hbase-config into hive-site.xml:
property
namehive.aux.jars.path/name
valuefile:///etc/hbase/conf//value
/property
Select on table with hbase storage handler fails with an SASL error
---
Key: HIVE-3983
URL: https://issues.apache.org/jira/browse/HIVE-3983
Project: Hive
Issue Type: Bug
Environment: hive-0.10
hbase-0.94.5.5
hadoop-0.23.3.1
hcatalog-0.5
Reporter: Arup Malakar
The table is created using the following query:
{code}
CREATE TABLE hbase_table_1(key int, value string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (hbase.columns.mapping = :key,cf1:val)
TBLPROPERTIES (hbase.table.name = xyz);
{code}
Doing a select on the table launches a map-reduce job. But the job fails with
the following error:
{code}
2013-02-02 01:31:07,500 FATAL [IPC Server handler 3 on 40118]
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
attempt_1348093718159_1501_m_00_0 - exited : java.io.IOException:
java.lang.RuntimeException: SASL authentication failed. The most likely cause
is missing or invalid credentials. Consider 'kinit'.
at
org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
at
org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:243)
at
org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:522)
at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.init(MapTask.java:160)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:381)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:157)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:152)
Caused by: java.lang.RuntimeException: SASL authentication failed. The most
likely cause is missing or invalid credentials. Consider 'kinit'.
at
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$1.run(SecureClient.java:242)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
at org.apache.hadoop.hbase.security.User.call(User.java:590)
at org.apache.hadoop.hbase.security.User.access$700(User.java:51)
at
org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:444)
at
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.handleSaslConnectionFailure(SecureClient.java:203)
at
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:291)
at
org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1124)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:974)
at
org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:104)
at $Proxy12.getProtocolVersion(Unknown Source)
at
org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.java:146)
at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1335)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1291)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1278)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:987
Re: [VOTE] Apache Hive 0.11.0 Release Candidate 0
Yes, a bit large. Looks like that the hive-0.11.0.tar.gz have a build and test env included. On Apr 30, 2013, at 9:34 AM, Carl Steinbach cwsteinb...@gmail.com wrote: I think the source tarball must be corrupted. It's 664MB in size, which is roughly 630MB larger than the 0.10.0 release tarball. I haven't been able to take a look at it yet because the apache archive site keeps throttling my connection midway through the download. On Mon, Apr 29, 2013 at 10:42 PM, Ashutosh Chauhan hashut...@apache.orgwrote: Hey all, I am excited to announce availability of Apache Hive 0.11.0 Release Candidate 0 at: http://people.apache.org/~hashutosh/hive-0.11.0-rc0/ Maven artifacts are available here: https://repository.apache.org/content/repositories/orgapachehive-154/ This release has many goodies including HiveServer2, windowing and analytical functions, decimal data type, better query planning, performance enhancements and various bug fixes. In total, we resolved more than 350 issues. Full list of fixed issues can be found at: http://s.apache.org/8Fr Voting will conclude in 72 hours. Hive PMC Members: Please test and vote. Thanks, Ashutosh (On behalf of Hive contributors who made 0.11 a possibility) -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF
[jira] [Created] (HIVE-4147) Slow Hive JDBC in concurrency mode to create/drop table
Alexander Alten-Lorenz created HIVE-4147: Summary: Slow Hive JDBC in concurrency mode to create/drop table Key: HIVE-4147 URL: https://issues.apache.org/jira/browse/HIVE-4147 Project: Hive Issue Type: Improvement Components: JDBC Affects Versions: 0.10.0 Reporter: Alexander Alten-Lorenz It's very slow using hive jdbc in concurrency mode to create/drop table, which is 20 times slower than using HiveMetatstoreClient. test steps: 1. create 100 different hive table one by one by using hive jdbc: create table .. 2. drop table one by one by using hive jdbc: drop table .. and timing 3. create 100 different hive table one by one by using hive jdbc: create table .. 4. drop tables one by one by using new HiveMetatstoreClient().dropTable(default, table_name) and timing results step 2 is 20 times slower than step 4. basically hive jdbc is 20 times slower than HiveMetatstoreClient not only create/table, but also the same kind of calls. Dropping tables via this low level API could cause issues if there are any clients concurrently querying the table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4147) Slow Hive JDBC in concurrency mode to create/drop table
[ https://issues.apache.org/jira/browse/HIVE-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13598597#comment-13598597 ] Alexander Alten-Lorenz commented on HIVE-4147: -- HiveMetatstoreClient is using hive.metastore.uris=thrift://IP:9083 to connect to metastore service which is started as standonle mode using port 9083. We've used HiveServer2, yes. Slow Hive JDBC in concurrency mode to create/drop table --- Key: HIVE-4147 URL: https://issues.apache.org/jira/browse/HIVE-4147 Project: Hive Issue Type: Improvement Components: JDBC Affects Versions: 0.10.0 Reporter: Alexander Alten-Lorenz It's very slow using hive jdbc in concurrency mode to create/drop table, which is 20 times slower than using HiveMetatstoreClient. test steps: 1. create 100 different hive table one by one by using hive jdbc: create table .. 2. drop table one by one by using hive jdbc: drop table .. and timing 3. create 100 different hive table one by one by using hive jdbc: create table .. 4. drop tables one by one by using new HiveMetatstoreClient().dropTable(default, table_name) and timing results step 2 is 20 times slower than step 4. basically hive jdbc is 20 times slower than HiveMetatstoreClient not only create/table, but also the same kind of calls. Dropping tables via this low level API could cause issues if there are any clients concurrently querying the table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] Kevin Wilfong elected to Hive PMC
Way to go, Kevin! Conratz! - Alex On Mar 4, 2013, at 8:54 PM, Carl Steinbach c...@apache.org wrote: On behalf of the Apache Hive PMC I am pleased to welcome Kevin Wilfong as a member of the Apache Hive PMC. Please join me in congratulating Kevin on his new role! Thanks. Carl -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF
Re: Apply to join the Hive-Email-list
Hey, this is a open list, just follow http://hive.apache.org/mailing_lists.html and pickup the lists you want to join. Best, Alex On Feb 21, 2013, at 10:08 AM, shuozhe shuozhe...@163.com wrote: Hi, This is a Hive developer ,so I want to join the Hive-Email-list to know more about the Hive. thanks for your permission. Thank you. Shuozhe Jia (operation department developer) Kanbox.com -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF
Re: [VOTE] Graduate HCatalog from the incubator and become part of Hive
+1, non-binding - Alex On Feb 5, 2013, at 10:06 AM, Sushanth Sowmyan khorg...@gmail.com wrote: And my axe! Erm... I mean, my +1. On Mon, Feb 4, 2013 at 10:18 PM, Alan Gates ga...@hortonworks.com wrote: FYI. Alan. Begin forwarded message: From: Alan Gates ga...@hortonworks.com Date: February 4, 2013 10:18:09 PM PST To: hcatalog-...@incubator.apache.org Subject: [VOTE] Graduate HCatalog from the incubator and become part of Hive The Hive PMC has voted to accept HCatalog as a submodule of Hive. You can see the vote thread at http://mail-archives.apache.org/mod_mbox/hive-dev/201301.mbox/%3cCACf6RrzktBYD0suZxn3Pfv8XkR=vgwszrzyb_2qvesuj2vh...@mail.gmail.com%3e . We now need to vote to graduate from the incubator and become a submodule of Hive. This entails the following: 1) the establishment of an HCatalog submodule in the Apache Hive Project; 2) the adoption of the Apache HCatalog codebase into the Hive HCatalog submodule; and 3) adding all currently active HCatalog committers as submodule committers on the Hive HCatalog submodule. Definitions for all these can be found in the (now adopted) Hive bylaws at https://cwiki.apache.org/confluence/display/Hive/Proposed+Changes+to+Hive+Bylaws+for+Submodule+Committer. This vote will stay open for at least 72 hours (thus 23:00 PST on 2/7/13). PPMC members votes are binding in this vote, though input from all is welcome. If this vote passes the next step will be to submit the graduation motion to the Incubator PMC. Here's my +1. Alan. -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF
Re: [VOTE] Amend Hive Bylaws + Add HCatalog Submodule
+1 and +1 too (unbinding) - Alex On Jan 30, 2013, at 2:23 AM, Gunther Hagleitner ghagleit...@hortonworks.com wrote: +1 and +1 Thanks, Gunther. On Tue, Jan 29, 2013 at 5:18 PM, Edward Capriolo edlinuxg...@gmail.comwrote: Measure 1: +1 Measure 2: +1 On Mon, Jan 28, 2013 at 2:47 PM, Carl Steinbach c...@apache.org wrote: I am calling a vote on the following two measures. Measure 1: Amend Hive Bylaws to Define Submodules and Submodule Committers If this measure passes the Apache Hive Project Bylaws will be amended with the following changes: https://cwiki.apache.org/confluence/display/Hive/Proposed+Changes+to+Hive+Bylaws+for+Submodule+Committers The motivation for these changes is discussed in the following email thread which appeared on the hive-dev and hcatalog-dev mailing lists: http://markmail.org/thread/u5nap7ghvyo7euqa Measure 2: Create HCatalog Submodule and Adopt HCatalog Codebase This measure provides for 1) the establishment of an HCatalog submodule in the Apache Hive Project, 2) the adoption of the Apache HCatalog codebase into the Hive HCatalog submodule, and 3) adding all currently active HCatalog committers as submodule committers on the Hive HCatalog submodule. Passage of this measure depends on the passage of Measure 1. Voting: Both measures require +1 votes from 2/3 of active Hive PMC members in order to pass. All participants in the Hive project are encouraged to vote on these measures, but only votes from active Hive PMC members are binding. The voting period commences immediately and shall last a minimum of six days. Voting is carried out by replying to this email thread. You must indicate which measure you are voting on in order for your vote to be counted. More details about the voting process can be found in the Apache Hive Project Bylaws: https://cwiki.apache.org/confluence/display/Hive/Bylaws -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF
[jira] [Commented] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13552512#comment-13552512 ] Alexander Alten-Lorenz commented on HIVE-3635: -- Sorry, missed this. I fixed the indentation only and replaced the attached one with the fixed one. No other code changes where done. allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Attachments: HIVE-3635.patch interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] Apache Hive 0.10.0 Released
+1, thank you! On Jan 12, 2013, at 12:57 AM, Carl Steinbach c...@apache.org wrote: Great work everyone! On Fri, Jan 11, 2013 at 12:37 AM, Lefty Leverenz le...@hortonworks.comwrote: Congratulations! – Lefty On Thu, Jan 10, 2013 at 11:59 PM, Ashutosh Chauhan hashut...@apache.org wrote: The Apache Hive team is proud to announce the the release of Apache Hive version 0.10.0. The Apache Hive (TM) data warehouse software facilitates querying and managing large datasets residing in distributed storage. Built on top of Apache Hadoop (TM), it provides: * Tools to enable easy data extract/transform/load (ETL) * A mechanism to impose structure on a variety of data formats * Access to files stored either directly in Apache HDFS (TM) or in other data storage systems such as Apache HBase (TM) * Query execution via MapReduce For Hive release details and downloads, please visit:http://hive.apache.org/releases.html Hive 0.10.0 Release Notes are available here: https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12320745styleName=TextprojectId=12310843 More than 350 jira issues are fixed in this release. We would like to thank many contributors who made this release possible. Regards, The Apache Hive Team -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF
[jira] [Assigned] (HIVE-3463) Add CASCADING to MySQL's InnoDB schema
[ https://issues.apache.org/jira/browse/HIVE-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz reassigned HIVE-3463: Assignee: (was: Alexander Alten-Lorenz) Add CASCADING to MySQL's InnoDB schema -- Key: HIVE-3463 URL: https://issues.apache.org/jira/browse/HIVE-3463 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Cascading could help to cleanup the tables when a FK is deleted. http://dev.mysql.com/doc/refman/5.5/en/innodb-foreign-key-constraints.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3463) Add CASCADING to MySQL's InnoDB schema
[ https://issues.apache.org/jira/browse/HIVE-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13547764#comment-13547764 ] Alexander Alten-Lorenz commented on HIVE-3463: -- Since the switch into a new DN version needs large work I going ahead and mark as Unassigned. Add CASCADING to MySQL's InnoDB schema -- Key: HIVE-3463 URL: https://issues.apache.org/jira/browse/HIVE-3463 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Cascading could help to cleanup the tables when a FK is deleted. http://dev.mysql.com/doc/refman/5.5/en/innodb-foreign-key-constraints.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Apache Hive 0.10.0 Release Candidate 0
+1 (non binding) - ran build, tests - sample queries ran too Thanks for driving this, Alex On Jan 9, 2013, at 6:19 AM, Namit Jain nj...@fb.com wrote: +1 Build from src. Ran some sanity tests both from bin and compiled src - they looked good On 12/27/12 3:08 AM, Ashutosh Chauhan ashutosh.chau...@gmail.com wrote: +1 Built from sources. Ran unit tests. All looked good. Thanks, Ashutosh On Fri, Dec 21, 2012 at 2:25 PM, Alan Gates ga...@hortonworks.com wrote: +1 (non-binding) Checked the check sums and key signatures. Installed it and ran a few queries. All looked good. As a note Hive should be offering a src only release and a convenience binary rather than two binaries, one with the source and one without. See the thread on general@incubator discussing this: http://mail-archives.apache.org/mod_mbox/incubator-general/201203.mbox/%3 CCAOFYJNY%3DEjVHrWVvAedR3OKwCv-BkTaCbEu0ufp7OZR_gpCTiA%40mail.gmail.com%3 E I think this can be solved later and need not block this release. Alan. On Dec 18, 2012, at 10:23 PM, Ashutosh Chauhan wrote: Apache Hive 0.10.0 Release Candidate 0 is available here: http://people.apache.org/~hashutosh/hive-0.10.0-rc0/ Maven artifacts are available here: https://repository.apache.org/content/repositories/orgapachehive-049/org/ apache/hive/ Release notes are available at: https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12320745s tyleName=TextprojectId=12310843Create=Createatl_token=A5KQ-2QAV-T4JA-F DED%7C70f39c6dd3cf337eaa0e3a0359687cf608903879%7Clin Voting will conclude in 72 hours. Hive PMC Members: Please test and vote. Thanks, Ashutosh -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF
[jira] [Updated] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3635: - Attachment: (was: HIVE-3635.patch) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3635.patch interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3635: - Status: Patch Available (was: Open) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3635.patch interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3635: - Attachment: HIVE-3635.patch allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3635.patch interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500878#comment-13500878 ] Alexander Alten-Lorenz commented on HIVE-3635: -- Replaced available patch here with the newer one. allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3635.patch interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7759/ --- (Updated Nov. 20, 2012, 7:11 a.m.) Review request for hive. Changes --- indentation fixed Description --- interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way This addresses bug HIVE-3635. https://issues.apache.org/jira/browse/HIVE-3635 Diffs (updated) - serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyBoolean.java c741c3a Diff: https://reviews.apache.org/r/7759/diff/ Testing --- Thanks, Alexander Alten-Lorenz
[jira] [Created] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
Alexander Alten-Lorenz created HIVE-3635: Summary: allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3635: - Status: Patch Available (was: Open) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3635: - Attachment: HIVE-3635.patch allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3635.patch interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7759/ --- Review request for hive. Description --- interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way This addresses bug HIVE-3635. https://issues.apache.org/jira/browse/HIVE-3635 Diffs - serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyBoolean.java c741c3a Diff: https://reviews.apache.org/r/7759/diff/ Testing --- Thanks, Alexander Alten-Lorenz
[jira] [Commented] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485874#comment-13485874 ] Alexander Alten-Lorenz commented on HIVE-3635: -- https://reviews.apache.org/r/7759/ allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3635.patch interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3636) Catch the NPe when using ^D to exit from CLI
[ https://issues.apache.org/jira/browse/HIVE-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3636: - Fix Version/s: 0.10.0 Affects Version/s: 0.9.0 Status: Patch Available (was: Open) Catch the NPe when using ^D to exit from CLI Key: HIVE-3636 URL: https://issues.apache.org/jira/browse/HIVE-3636 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3636) Catch the NPe when using ^D to exit from CLI
Alexander Alten-Lorenz created HIVE-3636: Summary: Catch the NPe when using ^D to exit from CLI Key: HIVE-3636 URL: https://issues.apache.org/jira/browse/HIVE-3636 Project: Hive Issue Type: Improvement Components: CLI Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3636) Catch the NPe when using ^D to exit from CLI
[ https://issues.apache.org/jira/browse/HIVE-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3636: - Attachment: HIVE-3636.patch Catch the NPe when using ^D to exit from CLI Key: HIVE-3636 URL: https://issues.apache.org/jira/browse/HIVE-3636 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3636.patch The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: Catch the NPe when using ^D to exit from CLI
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7760/ --- Review request for hive. Description --- The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace This addresses bug HIVE-3636. https://issues.apache.org/jira/browse/HIVE-3636 Diffs - cli/src/java/org/apache/hadoop/hive/cli/CliSessionState.java dfb30e2 Diff: https://reviews.apache.org/r/7760/diff/ Testing --- Thanks, Alexander Alten-Lorenz
[jira] [Commented] (HIVE-3636) Catch the NPe when using ^D to exit from CLI
[ https://issues.apache.org/jira/browse/HIVE-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485877#comment-13485877 ] Alexander Alten-Lorenz commented on HIVE-3636: -- https://reviews.apache.org/r/7760/ Catch the NPe when using ^D to exit from CLI Key: HIVE-3636 URL: https://issues.apache.org/jira/browse/HIVE-3636 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3636.patch The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3636) Catch the NPe when using ^D to exit from CLI
[ https://issues.apache.org/jira/browse/HIVE-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3636: - Resolution: Fixed Status: Resolved (was: Patch Available) Catch the NPe when using ^D to exit from CLI Key: HIVE-3636 URL: https://issues.apache.org/jira/browse/HIVE-3636 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3636.patch The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. Originally created by Frank Fejes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3635: - Description: interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. (was: interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. Originally created by Frank Fejes. ) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3635.patch interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486668#comment-13486668 ] Alexander Alten-Lorenz commented on HIVE-3635: -- +1 - will do allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3635.patch interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3636) Catch the NPe when using ^D to exit from CLI
[ https://issues.apache.org/jira/browse/HIVE-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3636: - Description: The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. (was: The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. Originally created by Frank Fejes.) Catch the NPe when using ^D to exit from CLI Key: HIVE-3636 URL: https://issues.apache.org/jira/browse/HIVE-3636 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3636.patch The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3636) Catch the NPe when using ^D to exit from CLI
[ https://issues.apache.org/jira/browse/HIVE-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486669#comment-13486669 ] Alexander Alten-Lorenz commented on HIVE-3636: -- Thanks, set to Fixed Catch the NPe when using ^D to exit from CLI Key: HIVE-3636 URL: https://issues.apache.org/jira/browse/HIVE-3636 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Fix For: 0.10.0 Attachments: HIVE-3636.patch The exit patch is just a quick hack to catch the NPE in order to allow ^D to exit hive without a stacktrace. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3463) Add CASCADING to MySQL's InnoDB schema
[ https://issues.apache.org/jira/browse/HIVE-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13463554#comment-13463554 ] Alexander Alten-Lorenz commented on HIVE-3463: -- Yes, we have to patch the upgrade scripts as well the jdo package (since the db will created if not present). But we've to know which relations we need to implement cascading. Any thoughts? Add CASCADING to MySQL's InnoDB schema -- Key: HIVE-3463 URL: https://issues.apache.org/jira/browse/HIVE-3463 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Cascading could help to cleanup the tables when a FK is deleted. http://dev.mysql.com/doc/refman/5.5/en/innodb-foreign-key-constraints.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3463) Add CASCADING to MySQL's InnoDB schema
[
https://issues.apache.org/jira/browse/HIVE-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13461612#comment-13461612
]
Alexander Alten-Lorenz commented on HIVE-3463:
--
We've to decide on which table we need cascading to create the appr. schema.
Example for SERDE (not really tested, only a design pattern)
{code}
CREATE TABLE SERDES (
SERDE_ID VARCHAR not null,
NAME VARCHAR not null,
SLIB VARCHAR,
SERDE_PARAMS STRING,
PARAM_KEY VARCHAR not null,
PARAM_VALUE VARCHAR not null,
PRIMARY KEY (PARAM_KEY,PARAM_VALUE),
KEY pkey (PARAM_KEY),
FOREIGN KEY (PARAM_VALUE) REFERENCES SERDE_ID (id)
ON DELETE CASCADE
ON UPDATE CASCADE,
FOREIGN KEY (NAME) REFERENCES SERDE_ID (id)
ON DELETE CASCADE
ON UPDATE CASCADE,
);
{code}
We need to know:
- which tables and rows have to be a relationship
- what make sense to describe as a CASCADE
- are the schema we deliver good enough to implement bulk query fetching?
Thanks,
Alex
Add CASCADING to MySQL's InnoDB schema
--
Key: HIVE-3463
URL: https://issues.apache.org/jira/browse/HIVE-3463
Project: Hive
Issue Type: Improvement
Components: Metastore
Affects Versions: 0.9.0
Reporter: Alexander Alten-Lorenz
Assignee: Alexander Alten-Lorenz
Cascading could help to cleanup the tables when a FK is deleted.
http://dev.mysql.com/doc/refman/5.5/en/innodb-foreign-key-constraints.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3463) Add CASCADING to MySQL's InnoDB schema
[ https://issues.apache.org/jira/browse/HIVE-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13461738#comment-13461738 ] Alexander Alten-Lorenz commented on HIVE-3463: -- I think so, since the schema is describing the design, and DN maps only. I do some tests atm. Add CASCADING to MySQL's InnoDB schema -- Key: HIVE-3463 URL: https://issues.apache.org/jira/browse/HIVE-3463 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Alexander Alten-Lorenz Cascading could help to cleanup the tables when a FK is deleted. http://dev.mysql.com/doc/refman/5.5/en/innodb-foreign-key-constraints.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3463) Add CASCADING to MySQL's InnoDB schema
[ https://issues.apache.org/jira/browse/HIVE-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459424#comment-13459424 ] Alexander Alten-Lorenz commented on HIVE-3463: -- More insight: 1. The drop table command deletes entries in different tables as UNIT transactions, so if there is a communication break or any other uncertain condition, database could rollback that individual transaction leaving database in inconsistent state. 2. The drop table is not serialized action, which is causing Replication break 3. As cascade is not defined in the FK definition during table creation, the stale could present in certain conditions. 4. The database with id 511 is showing the tables of default database who’s db_id is 1 5. The select on any portioned table with a limit operator scans through all the partitions using UNIT transactions, which just query single entry per transaction, means to say, if a table has got 10 partitions, it does 10 queries to database before showing the result. Instead, it should have done one bulk query fetching all the partition details at once. Example: In our database we are having 3500 partitioned tables, with total of 95 partitions. Every day we get hit on 90% of tables. And 60% of the users say, the queries takes its own sweet time to return, even though we kept the metastore level performance(MySQL) at 2miliSec to 20 miliSec, the query time at hive is always varies between few minutes to few hours. Even simple USE statement takes up to 5 min in some cases. (new thing I am bringing it up, as it took some time for me to confirm this) 6. Dead locks happening on hive tables, during normal operations, and quite a number of rollbacks happening at database side. What I noticed is the transaction serialization issue, which is issue a delete before an update, see below for the detail on the dead lock. With upgrade to mysql 5.5, we are not seeing it not so frequently now. One nit, Cascading for InnoDB is a great feature, since Hive doesn't support MySQL's binlog replication (Transaction level 'READ-COMMITTED' in InnoDB is not safe for binlog mode 'STATEMENT'). I was digging a bit deeper, that could be avoided by setting SET GLOBAL binlog_format = 'ROW';: inside of MySQL. But this needs to be confirmed first. I raise the FR up. Add CASCADING to MySQL's InnoDB schema -- Key: HIVE-3463 URL: https://issues.apache.org/jira/browse/HIVE-3463 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Priority: Minor Cascading could help to cleanup the tables when a FK is deleted. http://dev.mysql.com/doc/refman/5.5/en/innodb-foreign-key-constraints.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3463) Add CASCADING to MySQL's InnoDB schema
[ https://issues.apache.org/jira/browse/HIVE-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3463: - Priority: Major (was: Minor) Add CASCADING to MySQL's InnoDB schema -- Key: HIVE-3463 URL: https://issues.apache.org/jira/browse/HIVE-3463 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Cascading could help to cleanup the tables when a FK is deleted. http://dev.mysql.com/doc/refman/5.5/en/innodb-foreign-key-constraints.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3463) Add CASCADING to MySQL's InnoDB schema
Alexander Alten-Lorenz created HIVE-3463: Summary: Add CASCADING to MySQL's InnoDB schema Key: HIVE-3463 URL: https://issues.apache.org/jira/browse/HIVE-3463 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Priority: Minor Cascading could help to cleanup the tables when a FK is deleted. http://dev.mysql.com/doc/refman/5.5/en/innodb-foreign-key-constraints.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2058) MySQL Upgrade scripts missing new defaults for two table's columns
[ https://issues.apache.org/jira/browse/HIVE-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz reassigned HIVE-2058: Assignee: Alexander Alten-Lorenz MySQL Upgrade scripts missing new defaults for two table's columns -- Key: HIVE-2058 URL: https://issues.apache.org/jira/browse/HIVE-2058 Project: Hive Issue Type: Bug Components: Metastore Reporter: Stephen Tunney Assignee: Alexander Alten-Lorenz Priority: Blocker Upgraded from 0.5.0 to 0.7.0, and the upgrade scripts to 0.6.0 and 0.7.0 did not have two defaults that are necessary for being able to create a hive table. The columns missing default values are: COLUMNS.INTEGER_IDX SDS.IS_COMPRESSED I set them both to zero(0) (false for IS_COMPRESSED, obviously) The absence of these two default prevents the ability to create a table in Hive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3424) Error by upgrading a Hive 0.7.0 database to 0.8.0 (008-HIVE-2246.mysql.sql)
Alexander Alten-Lorenz created HIVE-3424: Summary: Error by upgrading a Hive 0.7.0 database to 0.8.0 (008-HIVE-2246.mysql.sql) Key: HIVE-3424 URL: https://issues.apache.org/jira/browse/HIVE-3424 Project: Hive Issue Type: Bug Components: Database/Schema Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Priority: Blocker Fix For: 0.9.1 ++ | Inserted table columns into COLUMNS_V2 | ++ | Inserted table columns into COLUMNS_V2 | ++ +--+ | Completed migrating idxs | +--+ | Completed migrating idxs | +--+ ERROR 1025 (HY000) at line 250 in file: '008-HIVE-2246.mysql.sql': Error on rename of './metastore/COLUMNS_OLD' to './metastore/#sql2-14e6-6eb' (errno: 152) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3424) Error by upgrading a Hive 0.7.0 database to 0.8.0 (008-HIVE-2246.mysql.sql)
[ https://issues.apache.org/jira/browse/HIVE-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13447140#comment-13447140 ] Alexander Alten-Lorenz commented on HIVE-3424: -- please assign to me, thanks Error by upgrading a Hive 0.7.0 database to 0.8.0 (008-HIVE-2246.mysql.sql) Key: HIVE-3424 URL: https://issues.apache.org/jira/browse/HIVE-3424 Project: Hive Issue Type: Bug Components: Database/Schema Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Priority: Blocker Fix For: 0.9.1 ++ | Inserted table columns into COLUMNS_V2 | ++ | Inserted table columns into COLUMNS_V2 | ++ +--+ | Completed migrating idxs | +--+ | Completed migrating idxs | +--+ ERROR 1025 (HY000) at line 250 in file: '008-HIVE-2246.mysql.sql': Error on rename of './metastore/COLUMNS_OLD' to './metastore/#sql2-14e6-6eb' (errno: 152) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3424) Error by upgrading a Hive 0.7.0 database to 0.8.0 (008-HIVE-2246.mysql.sql)
[ https://issues.apache.org/jira/browse/HIVE-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3424: - Attachment: HIVE-3424.patch Error by upgrading a Hive 0.7.0 database to 0.8.0 (008-HIVE-2246.mysql.sql) Key: HIVE-3424 URL: https://issues.apache.org/jira/browse/HIVE-3424 Project: Hive Issue Type: Bug Components: Database/Schema Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Priority: Blocker Fix For: 0.9.1 Attachments: HIVE-3424.patch ++ | Inserted table columns into COLUMNS_V2 | ++ | Inserted table columns into COLUMNS_V2 | ++ +--+ | Completed migrating idxs | +--+ | Completed migrating idxs | +--+ ERROR 1025 (HY000) at line 250 in file: '008-HIVE-2246.mysql.sql': Error on rename of './metastore/COLUMNS_OLD' to './metastore/#sql2-14e6-6eb' (errno: 152) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3424) Error by upgrading a Hive 0.7.0 database to 0.8.0 (008-HIVE-2246.mysql.sql)
[ https://issues.apache.org/jira/browse/HIVE-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3424: - Release Note: wrapped SET FOREIGN_KEY_CHECKS = 0|1 around the table Status: Patch Available (was: Open) Error by upgrading a Hive 0.7.0 database to 0.8.0 (008-HIVE-2246.mysql.sql) Key: HIVE-3424 URL: https://issues.apache.org/jira/browse/HIVE-3424 Project: Hive Issue Type: Bug Components: Database/Schema Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Priority: Blocker Fix For: 0.9.1 Attachments: HIVE-3424.patch ++ | Inserted table columns into COLUMNS_V2 | ++ | Inserted table columns into COLUMNS_V2 | ++ +--+ | Completed migrating idxs | +--+ | Completed migrating idxs | +--+ ERROR 1025 (HY000) at line 250 in file: '008-HIVE-2246.mysql.sql': Error on rename of './metastore/COLUMNS_OLD' to './metastore/#sql2-14e6-6eb' (errno: 152) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: Disable FK checks on COLUMNS in 008-HIVE-2246.mysql.sql
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/6894/ --- Review request for hive. Description --- By upgrading a MySQL metastore sometimes the schema get's into a foreign key issue: ++ | Inserted table columns into COLUMNS_V2 | ++ | Inserted table columns into COLUMNS_V2 | ++ +--+ | Completed migrating idxs | +--+ | Completed migrating idxs | +--+ ERROR 1025 (HY000) at line 250 in file: '008-HIVE-2246.mysql.sql': Error on rename of './metastore/COLUMNS_OLD' to './metastore/#sql2-14e6-6eb' (errno: 152) These patch disables FK checks before the table will be copied, and enable them after. This addresses bug HIVE-3424. https://issues.apache.org/jira/browse/HIVE-3424 Diffs - metastore/scripts/upgrade/mysql/008-HIVE-2246.mysql.sql 3ac8d2e Diff: https://reviews.apache.org/r/6894/diff/ Testing --- Thanks, Alexander Alten-Lorenz
[jira] [Updated] (HIVE-3424) Error by upgrading a Hive 0.7.0 database to 0.8.0 (008-HIVE-2246.mysql.sql)
[
https://issues.apache.org/jira/browse/HIVE-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Alten-Lorenz updated HIVE-3424:
-
Description:
{code}
++
| Inserted table columns into COLUMNS_V2 |
++
| Inserted table columns into COLUMNS_V2 |
++
+--+
| Completed migrating idxs |
+--+
| Completed migrating idxs |
+--+
ERROR 1025 (HY000) at line 250 in file: '008-HIVE-2246.mysql.sql': Error on
rename of './metastore/COLUMNS_OLD' to './metastore/#sql2-14e6-6eb' (errno: 152)
{code}
was:
++
| Inserted table columns into COLUMNS_V2 |
++
| Inserted table columns into COLUMNS_V2 |
++
+--+
| Completed migrating idxs |
+--+
| Completed migrating idxs |
+--+
ERROR 1025 (HY000) at line 250 in file: '008-HIVE-2246.mysql.sql': Error on
rename of './metastore/COLUMNS_OLD' to './metastore/#sql2-14e6-6eb' (errno: 152)
Error by upgrading a Hive 0.7.0 database to 0.8.0 (008-HIVE-2246.mysql.sql)
Key: HIVE-3424
URL: https://issues.apache.org/jira/browse/HIVE-3424
Project: Hive
Issue Type: Bug
Components: Database/Schema
Affects Versions: 0.9.0
Reporter: Alexander Alten-Lorenz
Assignee: Alexander Alten-Lorenz
Priority: Blocker
Fix For: 0.9.1
Attachments: HIVE-3424.patch
{code}
++
| Inserted table columns into COLUMNS_V2 |
++
| Inserted table columns into COLUMNS_V2 |
++
+--+
| Completed migrating idxs |
+--+
| Completed migrating idxs |
+--+
ERROR 1025 (HY000) at line 250 in file: '008-HIVE-2246.mysql.sql': Error on
rename of './metastore/COLUMNS_OLD' to './metastore/#sql2-14e6-6eb' (errno:
152)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3387) meta data file size exceeds limit
Alexander Alten-Lorenz created HIVE-3387: Summary: meta data file size exceeds limit Key: HIVE-3387 URL: https://issues.apache.org/jira/browse/HIVE-3387 Project: Hive Issue Type: Bug Affects Versions: 0.7.1 Reporter: Alexander Alten-Lorenz Fix For: 0.9.1 The cause is certainly that we use an array list instead of a set structure in the split locations API. Looks like a bug in Hive's CombineFileInputFormat. Reproduce: Set mapreduce.jobtracker.split.metainfo.maxsize=1 when submitting the Hive query. Run a big hive query that write data into a partitioned table. Due to the large number of splits, you encounter an exception on the job submitted to Hadoop and the exception said: meta data size exceeds 1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
