Heads up: merge HDFS-3042 branch next week?

2012-05-12 Thread Todd Lipcon 
Hi all,

I'd like to merge the HDFS-3042 (auto-failover) branch back to trunk
next week. There are a few improvements still up on JIRA against the
branch, but it's mostly done and we have been QAing it for over a
month, including some testing at customer sites.

Over the next day or two, I'm going to make sure all the test cases
pass and there are no new findbugs/warnings introduced by the branch.
Meanwhile, I wanted to give the heads up that I plan to call a vote
really soon.

Thanks
-Todd
-- 
Todd Lipcon
Software Engineer, Cloudera

RE: Heads up: merge HDFS-3042 branch next week?

2012-05-12 Thread Uma Maheswara Rao G 
That's great news Todd. Even we also have started testing HDFS-3042. It works 
well.

+1 for your merge plan.

Thanks,
Uma

From: Todd Lipcon [t...@cloudera.com]
Sent: Sunday, May 13, 2012 1:26 AM
To: general@hadoop.apache.org
Subject: Heads up: merge HDFS-3042 branch next week?

Hi all,

I'd like to merge the HDFS-3042 (auto-failover) branch back to trunk
next week. There are a few improvements still up on JIRA against the
branch, but it's mostly done and we have been QAing it for over a
month, including some testing at customer sites.

Over the next day or two, I'm going to make sure all the test cases
pass and there are no new findbugs/warnings introduced by the branch.
Meanwhile, I wanted to give the heads up that I plan to call a vote
really soon.

Thanks
-Todd
--
Todd Lipcon
Software Engineer, Cloudera

Re: Heads up: merge HDFS-3042 branch next week?

2012-05-12 Thread Eli Collins 
+1   Sounds great Todd.

On Sat, May 12, 2012 at 12:56 PM, Todd Lipcon  wrote:
> Hi all,
>
> I'd like to merge the HDFS-3042 (auto-failover) branch back to trunk
> next week. There are a few improvements still up on JIRA against the
> branch, but it's mostly done and we have been QAing it for over a
> month, including some testing at customer sites.
>
> Over the next day or two, I'm going to make sure all the test cases
> pass and there are no new findbugs/warnings introduced by the branch.
> Meanwhile, I wanted to give the heads up that I plan to call a vote
> really soon.
>
> Thanks
> -Todd
> --
> Todd Lipcon
> Software Engineer, Cloudera

Re: [VOTE] Release hadoop-2.0.0-alpha

2012-05-12 Thread Todd Lipcon 
Looking at the release tag vs the current state of branch-2, I have
two concerns from the point of view of HDFS:

1) We reverted HDFS-3157 in branch-2 because it sends deletions for
corrupt replicas without properly going through the "corrupt block"
path. We saw this cause data loss in TestPipelinesFailover. So, I'm
nervous about putting it in a release, even labeled as alpha.

2) HADOOP-8285 and HADOOP-8366 changed the wire format for the RPC
envelope in branch-2, but didn't make it into this rc. So, that would
mean that future alphas would not be protocol-compatible with this
alpha. Per a discussion a few weeks ago, I think we all were in
agreement that, if possible, we'd like all 2.x to be compatible for
client-server communication, at least (even if we don't support
cross-version for the intra-cluster protocols)

Do other folks think it's worth rolling an rc1? I would propose either:
a) Revert HDFS-3157 and commit HADOOP-8285 and HADOOP-8366 on
branch-2.0.0-alpha, so these are the only changes since rc0. Roll a
new rc1 from here.
or:
b) Discard the current branch-2.0.0-alpha and re-branch from the
current state of branch-2.

-Todd

On Fri, May 11, 2012 at 7:19 PM, Eli Collins  wrote:
> +1  I installed the build on a 6 node cluster and kicked the tires,
> didn't find any blocking issues.
>
> Btw in the future better to build from the svn repo so the revision is
> an svn rev from the release branch. Eg 1336254 instead of 40e90d3c7
> which is from the git mirror, this way we're consistent across
> releases.
>
> hadoop-2.0.0-alpha $ ./bin/hadoop version
> Hadoop 2.0.0-alpha
> Subversion 
> git://devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common
> -r 40e90d3c7e5d71aedcdc2d9cc55d078e78944c55
> Compiled by hortonmu on Wed May  9 16:19:55 UTC 2012
> From source with checksum 3d9a13a31ef3a9ab4b5cba1f982ab888
>
>
> On Wed, May 9, 2012 at 9:58 AM, Arun C Murthy  wrote:
>> I've created a release candidate for hadoop-2.0.0-alpha that I would like to 
>> release.
>>
>> It is available at: 
>> http://people.apache.org/~acmurthy/hadoop-2.0.0-alpha-rc0/
>>
>> The maven artifacts are available via repository.apache.org.
>>
>> Please try the release and vote; the vote will run for the usual 7 days.
>>
>> This is a big milestone for the Apache Hadoop community - congratulations 
>> and thanks for all the contributions!
>>
>> thanks,
>> Arun
>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>



-- 
Todd Lipcon
Software Engineer, Cloudera