Re: Will there be a 2.2 patch releases?
On Sat, Jan 4, 2014 at 9:48 AM, Raymie Stata wrote:
> I took a look at items in 2.3 and 2.4, as well as CDH5 and HDP2 (also
> looked at a few of the patches to assess their risk levels), and came
> up with the following strawman propose of bug-patches to be included
> in a 2.2.1 release:
>
> HADOOP-10029 [major] - Specifying har file to MR job fails in secure
> cluster
>
> HDFS-5089 [major] - When a LayoutVersion supports SNAPSHOT, it must
> support FSIMAGE_NAME_OPTIMIZATION
> HDFS-5403 [major] - WebHdfs client cannot communicate with older
> WebHdfs servers post HDFS-5306
> HDFS-5433 [critical] - When reloading fsimage during checkpointing, we
> should clear existing snapshottable directories
>
> MAPREDUCE-5028 [critical] - Maps fail when io.sort.mb is set to high value
>
Has only been fixed in branch-1. Still to be fixed in branch-2; IMO, it is
critical, but not enough to include it in 2.2.1
>
> YARN-1295 [major] - In UnixLocalWrapperScriptBuilder, using bash -c
> can cause Text file busy errors
> YARN-1374 [blocker] - Resource Manager fails to start due to
> ConcurrentModificationException
>
I don't think this is a bug in 2.2. It was introduced by changes in
branch-2, not in 2.2
> YARN-1176 [critical] - RM web services ClusterMetricsInfo total nodes
> doesn't include unhealthy nodes
>
> There are lots of outstanding bug fixes, so this list is definitely a
> bit arbitrary, but it seemed like a good list to me. Any thoughts?
>
>
> On Fri, Jan 3, 2014 at 5:26 PM, Sandy Ryza
> wrote:
> > Re-reading the thread, it seems what I said about 2.2.1 never happening
> was
> > incorrect. My impression is still that nobody has plans to drive a 2.2.1
> > release on any particular timeline.
> >
> > The changes that are now in 2.3 have been moved out of the branch-2.2.1.
> I
> > suppose the idea is that changes slated for 2.2.1 should be committed
> both
> > to branch-2.2 and branch-2.2.1.
> >
> > -Sandy
> >
> >
> > On Fri, Jan 3, 2014 at 4:57 PM, Raymie Stata
> wrote:
> >
> >> Yes, that thread is part of what's confusing me. Arun's initial 11/8
> >> message suggests that there would be room for blocker fixes leading to
> >> a 2.2.1 patch release ("...and then be very careful about including
> >> only *blocker* fixes in branch-2.2"). And nothing else in that thread
> >> suggests that there wouldn't be a patch release. And yet, Sandy seems
> >> to think that "2.2.1 isn't happening at all" (YARN-1295), a view
> >> that's consistent with the currently confused state of the repo
> >> (branch-2.2.1 exists but not released, branch-2.2 version is
> >> 2.2.2-SNAPSHOT).
> >>
> >> Seems to me that we should be planning for a 2.2.1 patch release at
> >> some point...
> >>
> >> Raymie
> >>
> >>
> >>
> >> On Fri, Jan 3, 2014 at 1:17 AM, Steve Loughran
> >> wrote:
> >> > the last discussion on this was in november -I presume that's still
> the
> >> plan
> >> >
> >> >
> >>
> http://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201311.mbox/%3CA31E1430-33BE-437C-A61E-050F9A67C109%40hortonworks.com%3E
> >> >
> >> >
> >> > On 3 January 2014 04:10, Raymie Stata wrote:
> >> >
> >> >> Nudge, any thoughts?
> >> >>
> >> >> On Sun, Dec 29, 2013 at 1:25 AM, Raymie Stata
> >> >> wrote:
> >> >> > In discussing YARN-1295 it's become clear that I'm confused about
> the
> >> >> > outcome of the "Next releases" thread. I had assumed there would
> be
> >> >> > patch releases to 2.2, and indeed one would be coming out early Q1.
> >> >> > Is this correct?
> >> >> >
> >> >> > If so, then things seem a little messed-up right now in 2.2-land.
> >> >> > There already is a branch-2.2.1, but there hasn't been a release.
> And
> >> >> > branch-2.2 has Maven version 2.2.2-SNAPSHOT. Due to the "2.3
> rename"
> >> >> > a few weeks ago, it might be that the first patch release for 2.2
> >> >> > needs to be 2.2.2. But if so, notice these lists of fixes for
> 2.2.1:
> >> >> >
> >> >> >
> https://issues.apache.org/jira/browse/YARN/fixforversion/12325667
> >> >> >
> https://issues.apache.org/jira/browse/HDFS/fixforversion/12325666
> >> >> >
> >> >> > Do these need to have their fix-versions updated?
> >> >> >
> >> >> > Raymie
> >> >> >
> >> >> >
> >> >> > P.S. While we're on the subject of point releases, let me check my
> >> >> assumptions.
> >> >> >
> >> >> > I assumed that, for release x.y.z, fixes deemed to be "critical bug
> >> >> > fixes" would be put into branch-x.y as a matter of course. The
> Maven
> >> >> > release-number in branch-x.y would be x.y.(z+1)-SNAPSHOT, and JIRAs
> >> >> > (to be) committed to branch-x.y would have x.y.(z+1) as one of
> their
> >> >> > fix-versions.
> >> >> >
> >> >> > When enough fixes have accumulated to warrant a release, or when a
> fix
> >> >> > comes up that is critical enough to warrant an immediate release,
> then
> >> >> > branch-x-y is branched to branch-x.y.(z+1), and a release is made.
> >> >> >
> >> >> > (As Hadoop itself moves from x.y to x.(y+1) and then x.(y+2), the
> >> >> > threshol
Re: Will there be a 2.2 patch releases?
I took a look at items in 2.3 and 2.4, as well as CDH5 and HDP2 (also
looked at a few of the patches to assess their risk levels), and came
up with the following strawman propose of bug-patches to be included
in a 2.2.1 release:
HADOOP-10029 [major] - Specifying har file to MR job fails in secure cluster
HDFS-5089 [major] - When a LayoutVersion supports SNAPSHOT, it must
support FSIMAGE_NAME_OPTIMIZATION
HDFS-5403 [major] - WebHdfs client cannot communicate with older
WebHdfs servers post HDFS-5306
HDFS-5433 [critical] - When reloading fsimage during checkpointing, we
should clear existing snapshottable directories
MAPREDUCE-5028 [critical] - Maps fail when io.sort.mb is set to high value
YARN-1295 [major] - In UnixLocalWrapperScriptBuilder, using bash -c
can cause Text file busy errors
YARN-1374 [blocker] - Resource Manager fails to start due to
ConcurrentModificationException
YARN-1176 [critical] - RM web services ClusterMetricsInfo total nodes
doesn't include unhealthy nodes
There are lots of outstanding bug fixes, so this list is definitely a
bit arbitrary, but it seemed like a good list to me. Any thoughts?
On Fri, Jan 3, 2014 at 5:26 PM, Sandy Ryza wrote:
> Re-reading the thread, it seems what I said about 2.2.1 never happening was
> incorrect. My impression is still that nobody has plans to drive a 2.2.1
> release on any particular timeline.
>
> The changes that are now in 2.3 have been moved out of the branch-2.2.1. I
> suppose the idea is that changes slated for 2.2.1 should be committed both
> to branch-2.2 and branch-2.2.1.
>
> -Sandy
>
>
> On Fri, Jan 3, 2014 at 4:57 PM, Raymie Stata wrote:
>
>> Yes, that thread is part of what's confusing me. Arun's initial 11/8
>> message suggests that there would be room for blocker fixes leading to
>> a 2.2.1 patch release ("...and then be very careful about including
>> only *blocker* fixes in branch-2.2"). And nothing else in that thread
>> suggests that there wouldn't be a patch release. And yet, Sandy seems
>> to think that "2.2.1 isn't happening at all" (YARN-1295), a view
>> that's consistent with the currently confused state of the repo
>> (branch-2.2.1 exists but not released, branch-2.2 version is
>> 2.2.2-SNAPSHOT).
>>
>> Seems to me that we should be planning for a 2.2.1 patch release at
>> some point...
>>
>> Raymie
>>
>>
>>
>> On Fri, Jan 3, 2014 at 1:17 AM, Steve Loughran
>> wrote:
>> > the last discussion on this was in november -I presume that's still the
>> plan
>> >
>> >
>> http://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201311.mbox/%3CA31E1430-33BE-437C-A61E-050F9A67C109%40hortonworks.com%3E
>> >
>> >
>> > On 3 January 2014 04:10, Raymie Stata wrote:
>> >
>> >> Nudge, any thoughts?
>> >>
>> >> On Sun, Dec 29, 2013 at 1:25 AM, Raymie Stata
>> >> wrote:
>> >> > In discussing YARN-1295 it's become clear that I'm confused about the
>> >> > outcome of the "Next releases" thread. I had assumed there would be
>> >> > patch releases to 2.2, and indeed one would be coming out early Q1.
>> >> > Is this correct?
>> >> >
>> >> > If so, then things seem a little messed-up right now in 2.2-land.
>> >> > There already is a branch-2.2.1, but there hasn't been a release. And
>> >> > branch-2.2 has Maven version 2.2.2-SNAPSHOT. Due to the "2.3 rename"
>> >> > a few weeks ago, it might be that the first patch release for 2.2
>> >> > needs to be 2.2.2. But if so, notice these lists of fixes for 2.2.1:
>> >> >
>> >> > https://issues.apache.org/jira/browse/YARN/fixforversion/12325667
>> >> > https://issues.apache.org/jira/browse/HDFS/fixforversion/12325666
>> >> >
>> >> > Do these need to have their fix-versions updated?
>> >> >
>> >> > Raymie
>> >> >
>> >> >
>> >> > P.S. While we're on the subject of point releases, let me check my
>> >> assumptions.
>> >> >
>> >> > I assumed that, for release x.y.z, fixes deemed to be "critical bug
>> >> > fixes" would be put into branch-x.y as a matter of course. The Maven
>> >> > release-number in branch-x.y would be x.y.(z+1)-SNAPSHOT, and JIRAs
>> >> > (to be) committed to branch-x.y would have x.y.(z+1) as one of their
>> >> > fix-versions.
>> >> >
>> >> > When enough fixes have accumulated to warrant a release, or when a fix
>> >> > comes up that is critical enough to warrant an immediate release, then
>> >> > branch-x-y is branched to branch-x.y.(z+1), and a release is made.
>> >> >
>> >> > (As Hadoop itself moves from x.y to x.(y+1) and then x.(y+2), the
>> >> > threshold for what is considered to be a "critical bug" would
>> >> > naturally start to rise, as the effort of back-porting goes up.)
>> >> >
>> >> > Do I have it right?
>> >>
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or entity
>> to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the reader
>> > of this message is not the intended recipient, you are hereb
Re: Will there be a 2.2 patch releases?
Re-reading the thread, it seems what I said about 2.2.1 never happening was
incorrect. My impression is still that nobody has plans to drive a 2.2.1
release on any particular timeline.
The changes that are now in 2.3 have been moved out of the branch-2.2.1. I
suppose the idea is that changes slated for 2.2.1 should be committed both
to branch-2.2 and branch-2.2.1.
-Sandy
On Fri, Jan 3, 2014 at 4:57 PM, Raymie Stata wrote:
> Yes, that thread is part of what's confusing me. Arun's initial 11/8
> message suggests that there would be room for blocker fixes leading to
> a 2.2.1 patch release ("...and then be very careful about including
> only *blocker* fixes in branch-2.2"). And nothing else in that thread
> suggests that there wouldn't be a patch release. And yet, Sandy seems
> to think that "2.2.1 isn't happening at all" (YARN-1295), a view
> that's consistent with the currently confused state of the repo
> (branch-2.2.1 exists but not released, branch-2.2 version is
> 2.2.2-SNAPSHOT).
>
> Seems to me that we should be planning for a 2.2.1 patch release at
> some point...
>
> Raymie
>
>
>
> On Fri, Jan 3, 2014 at 1:17 AM, Steve Loughran
> wrote:
> > the last discussion on this was in november -I presume that's still the
> plan
> >
> >
> http://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201311.mbox/%3CA31E1430-33BE-437C-A61E-050F9A67C109%40hortonworks.com%3E
> >
> >
> > On 3 January 2014 04:10, Raymie Stata wrote:
> >
> >> Nudge, any thoughts?
> >>
> >> On Sun, Dec 29, 2013 at 1:25 AM, Raymie Stata
> >> wrote:
> >> > In discussing YARN-1295 it's become clear that I'm confused about the
> >> > outcome of the "Next releases" thread. I had assumed there would be
> >> > patch releases to 2.2, and indeed one would be coming out early Q1.
> >> > Is this correct?
> >> >
> >> > If so, then things seem a little messed-up right now in 2.2-land.
> >> > There already is a branch-2.2.1, but there hasn't been a release. And
> >> > branch-2.2 has Maven version 2.2.2-SNAPSHOT. Due to the "2.3 rename"
> >> > a few weeks ago, it might be that the first patch release for 2.2
> >> > needs to be 2.2.2. But if so, notice these lists of fixes for 2.2.1:
> >> >
> >> > https://issues.apache.org/jira/browse/YARN/fixforversion/12325667
> >> > https://issues.apache.org/jira/browse/HDFS/fixforversion/12325666
> >> >
> >> > Do these need to have their fix-versions updated?
> >> >
> >> > Raymie
> >> >
> >> >
> >> > P.S. While we're on the subject of point releases, let me check my
> >> assumptions.
> >> >
> >> > I assumed that, for release x.y.z, fixes deemed to be "critical bug
> >> > fixes" would be put into branch-x.y as a matter of course. The Maven
> >> > release-number in branch-x.y would be x.y.(z+1)-SNAPSHOT, and JIRAs
> >> > (to be) committed to branch-x.y would have x.y.(z+1) as one of their
> >> > fix-versions.
> >> >
> >> > When enough fixes have accumulated to warrant a release, or when a fix
> >> > comes up that is critical enough to warrant an immediate release, then
> >> > branch-x-y is branched to branch-x.y.(z+1), and a release is made.
> >> >
> >> > (As Hadoop itself moves from x.y to x.(y+1) and then x.(y+2), the
> >> > threshold for what is considered to be a "critical bug" would
> >> > naturally start to rise, as the effort of back-porting goes up.)
> >> >
> >> > Do I have it right?
> >>
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
>
Re: Will there be a 2.2 patch releases?
Yes, that thread is part of what's confusing me. Arun's initial 11/8
message suggests that there would be room for blocker fixes leading to
a 2.2.1 patch release ("...and then be very careful about including
only *blocker* fixes in branch-2.2"). And nothing else in that thread
suggests that there wouldn't be a patch release. And yet, Sandy seems
to think that "2.2.1 isn't happening at all" (YARN-1295), a view
that's consistent with the currently confused state of the repo
(branch-2.2.1 exists but not released, branch-2.2 version is
2.2.2-SNAPSHOT).
Seems to me that we should be planning for a 2.2.1 patch release at
some point...
Raymie
On Fri, Jan 3, 2014 at 1:17 AM, Steve Loughran wrote:
> the last discussion on this was in november -I presume that's still the plan
>
> http://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201311.mbox/%3CA31E1430-33BE-437C-A61E-050F9A67C109%40hortonworks.com%3E
>
>
> On 3 January 2014 04:10, Raymie Stata wrote:
>
>> Nudge, any thoughts?
>>
>> On Sun, Dec 29, 2013 at 1:25 AM, Raymie Stata
>> wrote:
>> > In discussing YARN-1295 it's become clear that I'm confused about the
>> > outcome of the "Next releases" thread. I had assumed there would be
>> > patch releases to 2.2, and indeed one would be coming out early Q1.
>> > Is this correct?
>> >
>> > If so, then things seem a little messed-up right now in 2.2-land.
>> > There already is a branch-2.2.1, but there hasn't been a release. And
>> > branch-2.2 has Maven version 2.2.2-SNAPSHOT. Due to the "2.3 rename"
>> > a few weeks ago, it might be that the first patch release for 2.2
>> > needs to be 2.2.2. But if so, notice these lists of fixes for 2.2.1:
>> >
>> > https://issues.apache.org/jira/browse/YARN/fixforversion/12325667
>> > https://issues.apache.org/jira/browse/HDFS/fixforversion/12325666
>> >
>> > Do these need to have their fix-versions updated?
>> >
>> > Raymie
>> >
>> >
>> > P.S. While we're on the subject of point releases, let me check my
>> assumptions.
>> >
>> > I assumed that, for release x.y.z, fixes deemed to be "critical bug
>> > fixes" would be put into branch-x.y as a matter of course. The Maven
>> > release-number in branch-x.y would be x.y.(z+1)-SNAPSHOT, and JIRAs
>> > (to be) committed to branch-x.y would have x.y.(z+1) as one of their
>> > fix-versions.
>> >
>> > When enough fixes have accumulated to warrant a release, or when a fix
>> > comes up that is critical enough to warrant an immediate release, then
>> > branch-x-y is branched to branch-x.y.(z+1), and a release is made.
>> >
>> > (As Hadoop itself moves from x.y to x.(y+1) and then x.(y+2), the
>> > threshold for what is considered to be a "critical bug" would
>> > naturally start to rise, as the effort of back-porting goes up.)
>> >
>> > Do I have it right?
>>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Will there be a 2.2 patch releases?
the last discussion on this was in november -I presume that's still the plan http://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201311.mbox/%3CA31E1430-33BE-437C-A61E-050F9A67C109%40hortonworks.com%3E On 3 January 2014 04:10, Raymie Stata wrote: > Nudge, any thoughts? > > On Sun, Dec 29, 2013 at 1:25 AM, Raymie Stata > wrote: > > In discussing YARN-1295 it's become clear that I'm confused about the > > outcome of the "Next releases" thread. I had assumed there would be > > patch releases to 2.2, and indeed one would be coming out early Q1. > > Is this correct? > > > > If so, then things seem a little messed-up right now in 2.2-land. > > There already is a branch-2.2.1, but there hasn't been a release. And > > branch-2.2 has Maven version 2.2.2-SNAPSHOT. Due to the "2.3 rename" > > a few weeks ago, it might be that the first patch release for 2.2 > > needs to be 2.2.2. But if so, notice these lists of fixes for 2.2.1: > > > > https://issues.apache.org/jira/browse/YARN/fixforversion/12325667 > > https://issues.apache.org/jira/browse/HDFS/fixforversion/12325666 > > > > Do these need to have their fix-versions updated? > > > > Raymie > > > > > > P.S. While we're on the subject of point releases, let me check my > assumptions. > > > > I assumed that, for release x.y.z, fixes deemed to be "critical bug > > fixes" would be put into branch-x.y as a matter of course. The Maven > > release-number in branch-x.y would be x.y.(z+1)-SNAPSHOT, and JIRAs > > (to be) committed to branch-x.y would have x.y.(z+1) as one of their > > fix-versions. > > > > When enough fixes have accumulated to warrant a release, or when a fix > > comes up that is critical enough to warrant an immediate release, then > > branch-x-y is branched to branch-x.y.(z+1), and a release is made. > > > > (As Hadoop itself moves from x.y to x.(y+1) and then x.(y+2), the > > threshold for what is considered to be a "critical bug" would > > naturally start to rise, as the effort of back-porting goes up.) > > > > Do I have it right? > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Will there be a 2.2 patch releases?
Nudge, any thoughts? On Sun, Dec 29, 2013 at 1:25 AM, Raymie Stata wrote: > In discussing YARN-1295 it's become clear that I'm confused about the > outcome of the "Next releases" thread. I had assumed there would be > patch releases to 2.2, and indeed one would be coming out early Q1. > Is this correct? > > If so, then things seem a little messed-up right now in 2.2-land. > There already is a branch-2.2.1, but there hasn't been a release. And > branch-2.2 has Maven version 2.2.2-SNAPSHOT. Due to the "2.3 rename" > a few weeks ago, it might be that the first patch release for 2.2 > needs to be 2.2.2. But if so, notice these lists of fixes for 2.2.1: > > https://issues.apache.org/jira/browse/YARN/fixforversion/12325667 > https://issues.apache.org/jira/browse/HDFS/fixforversion/12325666 > > Do these need to have their fix-versions updated? > > Raymie > > > P.S. While we're on the subject of point releases, let me check my > assumptions. > > I assumed that, for release x.y.z, fixes deemed to be "critical bug > fixes" would be put into branch-x.y as a matter of course. The Maven > release-number in branch-x.y would be x.y.(z+1)-SNAPSHOT, and JIRAs > (to be) committed to branch-x.y would have x.y.(z+1) as one of their > fix-versions. > > When enough fixes have accumulated to warrant a release, or when a fix > comes up that is critical enough to warrant an immediate release, then > branch-x-y is branched to branch-x.y.(z+1), and a release is made. > > (As Hadoop itself moves from x.y to x.(y+1) and then x.(y+2), the > threshold for what is considered to be a "critical bug" would > naturally start to rise, as the effort of back-porting goes up.) > > Do I have it right?
Will there be a 2.2 patch releases?
In discussing YARN-1295 it's become clear that I'm confused about the outcome of the "Next releases" thread. I had assumed there would be patch releases to 2.2, and indeed one would be coming out early Q1. Is this correct? If so, then things seem a little messed-up right now in 2.2-land. There already is a branch-2.2.1, but there hasn't been a release. And branch-2.2 has Maven version 2.2.2-SNAPSHOT. Due to the "2.3 rename" a few weeks ago, it might be that the first patch release for 2.2 needs to be 2.2.2. But if so, notice these lists of fixes for 2.2.1: https://issues.apache.org/jira/browse/YARN/fixforversion/12325667 https://issues.apache.org/jira/browse/HDFS/fixforversion/12325666 Do these need to have their fix-versions updated? Raymie P.S. While we're on the subject of point releases, let me check my assumptions. I assumed that, for release x.y.z, fixes deemed to be "critical bug fixes" would be put into branch-x.y as a matter of course. The Maven release-number in branch-x.y would be x.y.(z+1)-SNAPSHOT, and JIRAs (to be) committed to branch-x.y would have x.y.(z+1) as one of their fix-versions. When enough fixes have accumulated to warrant a release, or when a fix comes up that is critical enough to warrant an immediate release, then branch-x-y is branched to branch-x.y.(z+1), and a release is made. (As Hadoop itself moves from x.y to x.(y+1) and then x.(y+2), the threshold for what is considered to be a "critical bug" would naturally start to rise, as the effort of back-porting goes up.) Do I have it right?
