Re: Next releases

2013-11-20 Thread Arun C Murthy
Jason,

 I'm glad to see we are converging. I'll update the Roadmap wiki with details 
about major/minor/patch releases.

 Here is a straight-forward approach for now: I'll just roll contents of 
branch-2.2 as a 2.3-rc0 candidate right-away. This way we don't have to get 
embroiled in details of individual patches (there are too many). Next up, I'll 
roll 2.4 in December.

 Thoughts?

thanks,
Arun

On Nov 13, 2013, at 1:55 PM, Jason Lowe  wrote:

> I think a lot of confusion comes from the fact that the 2.x line is starting 
> to mature.  Before this there wasn't such a big contention of what went into 
> patch vs. minor releases and often the lines were blurred between the two.  
> However now we have significant customers and products starting to use 2.x as 
> a base, which means we need to start treating it like we treat 1.x.  That 
> means getting serious about what we should put into a patch release vs. what 
> we postpone to a minor release.
> 
> Here's my $0.02 on recent proposals:
> 
> +1 to releasing more often in general.  A lot of the rush to put changes into 
> a patch release is because it can be a very long time between any kind of 
> release.  If minor releases are more frequent then I hope there would be less 
> of a need to rush something or hold up a release.
> 
> +1 to limiting checkins of patch releases to Blockers/Criticals.  If 
> necessary committers check into trunk/branch-2 only and defer to the patch 
> release manager for the patch release merge.  Then there should be fewer 
> surprises for everyone what ended up in a patch release and less likely the 
> patch release becomes destabilized from the sheer amount of code churn.  
> Maybe this won't be necessary if everyone understands that the patch release 
> isn't the only way to get a change out in timely manner.
> 
> As for 2.2.1, again I think it's expectations for what that release means.  
> If it's really just a patch release then there shouldn't be features in it 
> and tons of code churn, but I think many were treating it as the next vehicle 
> to deliver changes in general.  If we think 2.2.1 is just as good or better 
> than 2.2.0 then let's wrap it up and move to a more disciplined approach for 
> subsequent patch releases and more frequent minor releases.
> 
> Jason
> 
> On 11/13/2013 12:10 PM, Arun C Murthy wrote:
>> On Nov 12, 2013, at 1:54 PM, Todd Lipcon  wrote:
>> 
>>> On Mon, Nov 11, 2013 at 2:57 PM, Colin McCabe wrote:
>>> 
 To be honest, I'm not aware of anything in 2.2.1 that shouldn't be
 there.  However, I have only been following the HDFS and common side
 of things so I may not have the full picture.  Arun, can you give a
 specific example of something you'd like to "blow away"?
>> There are bunch of issues in YARN/MapReduce which clearly aren't *critical*, 
>> similarly in HDFS a cursory glance showed up some 
>> *enhancements*/*improvements* in CHANGES.txt which aren't necessary for a 
>> patch release, plus things like:
>> 
>>  HADOOP-9623 
>> Update jets3t dependency to 0.9.0
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>>  
>> Having said that, the HDFS devs know their code the best.
>> 
>>> I agree with Colin. If we've been backporting things into a patch release
>>> (third version component) which don't belong, we should explicitly call out
>>> those patches, so we can learn from our mistakes and have a discussion
>>> about what belongs.
>> Good point.
>> 
>> Here is a straw man proposal:
>> 
>> 
>> A patch (third version) release should only include *blocker* bugs which are 
>> critical from an operational, security or data-integrity issues.
>> 
>> This way, we can ensure that a minor series release (2.2.x or 2.3.x or 
>> 2.4.x) is always release-able, and more importantly, deploy-able at any 
>> point in time.
>> 
>> 
>> 
>> Sandy did bring up a related point about timing of releases and the urge for 
>> everyone to cram features/fixes into a dot release.
>> 
>> So, we could remedy that situation by doing a release every 4-6 weeks (2.3, 
>> 2.4 etc.) and keep the patch releases limited to blocker bugs.
>> 
>> Thoughts?
>> 
>> thanks,
>> Arun
>> 
>> 
>> 
>> 
>> 
> 

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/



-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.


[jira] [Created] (HADOOP-10120) Additional sliding window metrics

2013-11-20 Thread Andrew Wang (JIRA)
Andrew Wang created HADOOP-10120:


 Summary: Additional sliding window metrics
 Key: HADOOP-10120
 URL: https://issues.apache.org/jira/browse/HADOOP-10120
 Project: Hadoop Common
  Issue Type: New Feature
  Components: metrics
Affects Versions: 2.2.0
Reporter: Andrew Wang
Assignee: Andrew Wang


For HDFS-5350 we'd like to report the last few fsimage transfer times as a 
health metric. This would mean (for example) a sliding window of the last 10 
transfer times, when it was last updated, the total count. It'd be nice to have 
a metrics class that did this.

It'd also be interesting to have some kind of time-based sliding window for 
statistics like counts and averages. This would let us answer questions like 
"how many RPCs happened in the last 10s? minute? 5 minutes? 10 minutes?". 
Commutative metrics like counts and averages are easy to aggregate in this 
fashion.



--
This message was sent by Atlassian JIRA
(v6.1#6144)