RE: release 0.6
Carl, Now that all the blocking jiras for 0.6 have been committed, can we release 0.6, say end of the week ? We can give some notice to people if they want to file a blocker in the next 2-3 days. Thanks, -namit From: Namit Jain [nj...@facebook.com] Sent: Friday, October 01, 2010 9:44 AM To: Carl Steinbach Cc: hive-dev@hadoop.apache.org Subject: RE: release 0.6 I am not sure what kind of downtime would it involve for us (facebook). We will have to make a copy of the production metastore, and then perform the changes. If that takes a long time, we will have to come up with some quicker upgrade solutions - We will try to do that today, and get back to you. Thanks, -namit From: Carl Steinbach [mailto:c...@cloudera.com] Sent: Thursday, September 30, 2010 11:23 PM To: Namit Jain Cc: hive-dev@hadoop.apache.org Subject: Re: release 0.6 Hi Namit, It used to be much higher in the beginning but quite a few users reported problems on some mysql dbs. 767 seemed to work most dbs. before committing this can someone test this on some different dbs (with and without UTF encoding)? Copying my response to Prasad from HIVE-1364: It's possible that people who ran into problems before were using a version of MySQL older than 5.0.3. These versions supported a 255 byte max length for VARCHARs. It's also possible that older versions of the package.jdo mapping contained more indexes, in which case the 767 byte limit holds. Also, UTF encoding should not make a difference since these are byte lengths, not character lengths. Another point is that HIVE-675 added two 4000 byte VARCHARs to the mapping, and this patch is present in both trunk and the 0.6.0 branch. I haven't heard that anyone is experiencing problems because of this. Do we really need it for 0.6, or should we test it properly/take our time and then commit it if needed. Yes, I think we really need these changes. Several people have already commented on the list about hitting the 767 byte limit while using the HBase storage handler. What kind of testing regimen do think is necessary for this change? Thanks. Carl
Re: release 0.6
Hi Namit, Sounds like a good plan to me. Carl On Wed, Oct 6, 2010 at 2:04 PM, Namit Jain nj...@facebook.com wrote: Carl, Now that all the blocking jiras for 0.6 have been committed, can we release 0.6, say end of the week ? We can give some notice to people if they want to file a blocker in the next 2-3 days. Thanks, -namit From: Namit Jain [nj...@facebook.com] Sent: Friday, October 01, 2010 9:44 AM To: Carl Steinbach Cc: hive-dev@hadoop.apache.org Subject: RE: release 0.6 I am not sure what kind of downtime would it involve for us (facebook). We will have to make a copy of the production metastore, and then perform the changes. If that takes a long time, we will have to come up with some quicker upgrade solutions - We will try to do that today, and get back to you. Thanks, -namit From: Carl Steinbach [mailto:c...@cloudera.com] Sent: Thursday, September 30, 2010 11:23 PM To: Namit Jain Cc: hive-dev@hadoop.apache.org Subject: Re: release 0.6 Hi Namit, It used to be much higher in the beginning but quite a few users reported problems on some mysql dbs. 767 seemed to work most dbs. before committing this can someone test this on some different dbs (with and without UTF encoding)? Copying my response to Prasad from HIVE-1364: It's possible that people who ran into problems before were using a version of MySQL older than 5.0.3. These versions supported a 255 byte max length for VARCHARs. It's also possible that older versions of the package.jdo mapping contained more indexes, in which case the 767 byte limit holds. Also, UTF encoding should not make a difference since these are byte lengths, not character lengths. Another point is that HIVE-675 added two 4000 byte VARCHARs to the mapping, and this patch is present in both trunk and the 0.6.0 branch. I haven't heard that anyone is experiencing problems because of this. Do we really need it for 0.6, or should we test it properly/take our time and then commit it if needed. Yes, I think we really need these changes. Several people have already commented on the list about hitting the 767 byte limit while using the HBase storage handler. What kind of testing regimen do think is necessary for this change? Thanks. Carl
RE: release 0.6
I am not sure what kind of downtime would it involve for us (facebook). We will have to make a copy of the production metastore, and then perform the changes. If that takes a long time, we will have to come up with some quicker upgrade solutions - We will try to do that today, and get back to you. Thanks, -namit From: Carl Steinbach [mailto:c...@cloudera.com] Sent: Thursday, September 30, 2010 11:23 PM To: Namit Jain Cc: hive-dev@hadoop.apache.org Subject: Re: release 0.6 Hi Namit, It used to be much higher in the beginning but quite a few users reported problems on some mysql dbs. 767 seemed to work most dbs. before committing this can someone test this on some different dbs (with and without UTF encoding)? Copying my response to Prasad from HIVE-1364: It's possible that people who ran into problems before were using a version of MySQL older than 5.0.3. These versions supported a 255 byte max length for VARCHARs. It's also possible that older versions of the package.jdo mapping contained more indexes, in which case the 767 byte limit holds. Also, UTF encoding should not make a difference since these are byte lengths, not character lengths. Another point is that HIVE-675 added two 4000 byte VARCHARs to the mapping, and this patch is present in both trunk and the 0.6.0 branch. I haven't heard that anyone is experiencing problems because of this. Do we really need it for 0.6, or should we test it properly/take our time and then commit it if needed. Yes, I think we really need these changes. Several people have already commented on the list about hitting the 767 byte limit while using the HBase storage handler. What kind of testing regimen do think is necessary for this change? Thanks. Carl
Re: release 0.6
Hi Namit, HIVE-1364 is required since various people have found the current size limitations to be too small, especially when using the HBase storage handler and views. Please review and commit this patch. HIVE-1427 is also a requirement due to metastore schema changes that were made in the following tickets: HIVE-675, HIVE-972 and HIVE-1068. This ticket is also blocked on the changes proposed in HIVE-1364. I will go ahead and post upgrade scripts that assume the changes in HIVE-1364 are committed. In the meantime please look at HIVE-1364 and let me know whether or not this patch needs to be modified. Thanks. Carl On Thu, Sep 30, 2010 at 4:04 PM, Namit Jain nj...@facebook.com wrote: Hi Carl, Are the following needed for the 0.6 https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=truemode=hidesorter/order=DESCsorter/field=priorityresolution=-1pid=12310843fixfor=12314524? Can we set a target date for the release of the branch ? Thanks, -namit
RE: release 0.6
HIVE-1364: The comment from Prasad is a little concerning: It used to be much higher in the beginning but quite a few users reported problems on some mysql dbs. 767 seemed to work most dbs. before committing this can someone test this on some different dbs (with and without UTF encoding)? Do we really need it for 0.6, or should we test it properly/take our time and then commit it if needed. HIVE-1427 is needed nevertheless. Thanks, -namit From: Carl Steinbach [c...@cloudera.com] Sent: Thursday, September 30, 2010 6:40 PM To: hive-dev@hadoop.apache.org; Namit Jain Subject: Re: release 0.6 Hi Namit, HIVE-1364 is required since various people have found the current size limitations to be too small, especially when using the HBase storage handler and views. Please review and commit this patch. HIVE-1427 is also a requirement due to metastore schema changes that were made in the following tickets: HIVE-675, HIVE-972 and HIVE-1068. This ticket is also blocked on the changes proposed in HIVE-1364. I will go ahead and post upgrade scripts that assume the changes in HIVE-1364 are committed. In the meantime please look at HIVE-1364 and let me know whether or not this patch needs to be modified. Thanks. Carl On Thu, Sep 30, 2010 at 4:04 PM, Namit Jain nj...@facebook.commailto:nj...@facebook.com wrote: Hi Carl, Are the following needed for the 0.6 https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=truemode=hidesorter/order=DESCsorter/field=priorityresolution=-1pid=12310843fixfor=12314524 ? Can we set a target date for the release of the branch ? Thanks, -namit
