Created a page for 2.7.4 release.
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.7.4
If you want to edit this wiki, please ping me.
Regards,
Akira
On 2017/05/23 4:42, Brahma Reddy Battula wrote:
Hi Konstantin Shvachko
how about creating a wiki page for 2.7.4 release status like 2.8 and trunk in
following link.??
https://cwiki.apache.org/confluence/display/HADOOP
________________________________
From: Konstantin Shvachko <shv.had...@gmail.com>
Sent: Saturday, May 13, 2017 3:58 AM
To: Akira Ajisaka
Cc: Hadoop Common; Hdfs-dev; mapreduce-...@hadoop.apache.org;
yarn-...@hadoop.apache.org
Subject: Re: About 2.7.4 Release
Latest update on the links and filters. Here is the correct link for the
filter:
https://issues.apache.org/jira/secure/IssueNavigator.jspa?requestId=12340814
Also updated: https://s.apache.org/Dzg4
Had to do some Jira debugging. Sorry for confusion.
Thanks,
--Konstantin
On Wed, May 10, 2017 at 2:30 PM, Konstantin Shvachko <shv.had...@gmail.com>
wrote:
Hey Akira,
I didn't have private filters. Most probably Jira caches something.
Your filter is in the right direction, but for some reason it lists only
22 issues, while mine has 29.
It misses e.g. YARN-5543 <https://issues.apache.org/jira/browse/YARN-5543>
.
Anyways, I created a Jira filter now "Hadoop 2.7.4 release blockers",
shared it with "everybody", and updated my link to point to that filter. So
you can use any of the three methods below to get the correct list:
1. Go to https://s.apache.org/Dzg4
2. Go to the filter via
https://issues.apache.org/jira/issues?filter=12340814
or by finding "Hadoop 2.7.4 release blockers" filter in the jira
3. On Advanced issues search page paste this:
project in (HDFS, HADOOP, YARN, MAPREDUCE) AND labels = release-blocker
AND "Target Version/s" = 2.7.4
Hope this solves the confusion for which issues are included.
Please LMK if it doesn't, as it is important.
Thanks,
--Konstantin
On Tue, May 9, 2017 at 9:58 AM, Akira Ajisaka <aajis...@apache.org> wrote:
Hi Konstantin,
Thank you for volunteering as release manager!
Actually the original link works fine: https://s.apache.org/Dzg4
I couldn't see the link. Maybe is it private filter?
Here is a link I generated: https://s.apache.org/ehKy
This filter includes resolved issue and excludes fixversion == 2.7.4
Thanks and Regards,
Akira
On 2017/05/08 19:20, Konstantin Shvachko wrote:
Hi Brahma Reddy Battula,
Actually the original link works fine: https://s.apache.org/Dzg4
Your link excludes closed and resolved issues, which needs backporting,
and
which we cannot reopen, as discussed in this thread earlier.
Looked through the issues you proposed:
HDFS-9311 <https://issues.apache.org/jira/browse/HDFS-9311>
Seems like a new feature. It helps failover to standby node when primary
is
under heavy load, but it introduces new APIs, addresses, config
parameters.
And needs at least one follow up jira.
Looks like a backward compatible change, though.
Did you have a chance to run it in production?
+1 on
HDFS-10987 <https://issues.apache.org/jira/browse/HDFS-10987>
[HDFS-10987] Make Decommission less expensive when lot of
...<https://issues.apache.org/jira/browse/HDFS-10987>
issues.apache.org
When user want to decommission a node which having 50M blocks ,it could hold
the namesystem lock for long time.We've seen it is taking 36 sec. As we knew
during this ...
HDFS-9902 <https://issues.apache.org/jira/browse/HDFS-9902>
[HDFS-9902] Support different values of dfs.datanode.du
...<https://issues.apache.org/jira/browse/HDFS-9902>
issues.apache.org
Now Hadoop support different storage type for DISK, SSD, ARCHIVE and RAM_DISK,
but they share one configuration dfs.datanode.du.reserved. The DISK size may be
several ...
HDFS-8312 <https://issues.apache.org/jira/browse/HDFS-8312>
Trash does not descent into child directories to check for
...<https://issues.apache.org/jira/browse/HDFS-8312>
issues.apache.org
HDFS trash does not descent into child directory to check if user has
permission to delete files. For example: Run the following command to
initialize directory ...
HADOOP-14100 <https://issues.apache.org/jira/browse/HADOOP-14100>
Upgrade Jsch jar to latest version to fix vulnerability in
...<https://issues.apache.org/jira/browse/HADOOP-14100>
issues.apache.org
Recently there was on vulnerability reported on jsch library. Its fixed in
latest 0.1.54 version before CVE was made public.
https://cve.mitre.org/cgi-bin/cvename.cgi ...
Added them to 2.7.4 release. You should see them via the above link now.
Would be good if you could attach backport patches for some of them?
Appreciate your help,
--Konstantin
On Mon, May 8, 2017 at 8:39 AM, Brahma Reddy Battula <
brahmareddy.batt...@huawei.com> wrote:
Looks following link is not correct..
https://s.apache.org/Dzg4
It should be like following..?
https://s.apache.org/wi3U
Apart from Konstantin mentioned,Following also good to go..? let me know
your thoughts on this.
For Large Cluster:
=============
https://issues.apache.org/jira/browse/HDFS-9311===Life Line Protocol
https://issues.apache.org/jira/browse/HDFS-10987===Deecommission
Expensive when lot's of blocks are present
https://issues.apache.org/jira/browse/HDFS-9902===
"dfs.datanode.du.reserved" per Storage Type
For Security:
=========
https://issues.apache.org/jira/browse/HDFS-8312===Trash does not
descent
into child directories to check for permission
https://issues.apache.org/jira/browse/HADOOP-14100===Upgrade Jsch jar
to
latest version to fix vulnerability in old versions
Regards
Brahma Reddy Battula
-----Original Message-----
From: Erik Krogen [mailto:ekro...@linkedin.com.INVALID]
Sent: 06 May 2017 02:40
To: Konstantin Shvachko
Cc: Zhe Zhang; Hadoop Common; Hdfs-dev; mapreduce-...@hadoop.apache.org
;
yarn-...@hadoop.apache.org
Subject: Re: About 2.7.4 Release
List LGTM Konstantin!
Let's say that we will only create a new tracking JIRA for patches which
do not backport cleanly, to avoid having too many lying around.
Otherwise
we can directly attach to old ticket. If a clean backport does happen to
break a test the nightly build will help us catch it.
Erik
On Thu, May 4, 2017 at 7:21 PM, Konstantin Shvachko <
shv.had...@gmail.com>
wrote:
Great Zhe. Let's monitor the build.
I marked all jiras I knew of for inclusion into 2.7.4 as I described
before.
Target Version/s: 2.7.4
Label: release-blocker
Here is the link to the list: https://s.apache.org/Dzg4 Please let me
know if I missed anything.
And feel free to pick up any. Most of backports are pretty
straightforward, but not all.
We can create tracking jiras for backporting if you need to run
Jenkins on the patch (and since Allen does not allow reopening them).
But I think the final patch should be attached to the original jira.
Otherwise history will be hard to follow.
Thanks,
--Konstantin
On Wed, May 3, 2017 at 4:53 PM, Zhe Zhang <z...@apache.org> wrote:
Thanks for volunteering as RM Konstantin! The plan LGTM.
I've created a nightly Jenkins job for branch-2.7 (unit tests):
https://builds.apache.org/job/Hadoop-branch2.7-nightly/
On Wed, May 3, 2017 at 12:42 AM Konstantin Shvachko <
shv.had...@gmail.com>
wrote:
Hey guys,
I and a few of my colleagues would like to help here and move 2.7.4
release forward. A few points in this regard.
1. Reading through this thread since March 1 I see that Vinod
hinted on managing the release. Vinod, if you still want the job /
have bandwidth will be happy to work with you.
Otherwise I am glad to volunteer as the release manager.
2. In addition to current blockers and criticals, I would like to
propose
a
few issues to be included in the release, see the list below. Those
are mostly bug fixes and optimizations, which we already have in
our
internal
branch and run in production. Plus one minor feature "node
labeling", which we found very handy, when you have heterogeneous
environments and mixed workloads, like MR and Spark.
3. For marking issues for the release I propose to
- set the target version to 2.7.4, and
- add a new label "release-blocker"
That way we will know issues targeted for the release without
reopening them for backports.
4. I see quite a few people are interested in the release. With all
the help I think we can target to release by the end of May.
Other things include fixing CHANGES.txt and fixing Jenkins build
for
2.7.4
branch.
Thanks,
--Konstantin
========== List of issue for 2.7.4 ===========
------ Backports
HADOOP-12975 <https://issues.apache.org/jira/browse/HADOOP-12975>.
Add
du
jitters
HDFS-9710 <https://issues.apache.org/jira/browse/HDFS-9710>. IBR
batching
HDFS-10715 <https://issues.apache.org/jira/browse/HDFS-10715>. NPE
when applying AvailableSpaceBlockPlacementPolicy
HDFS-2538 <https://issues.apache.org/jira/browse/HDFS-2538>. fsck
removal
of dot printing
HDFS-8131 <https://issues.apache.org/jira/browse/HDFS-8131>.
space-balanced
policy for balancer
HDFS-8549 <https://issues.apache.org/jira/browse/HDFS-8549>. abort
balancer if upgrade in progress
HDFS-9412 <https://issues.apache.org/jira/browse/HDFS-9412>. skip
small blocks in getBlocks
YARN-1471 <https://issues.apache.org/jira/browse/YARN-1471>. SLS
simulator
YARN-4302 <https://issues.apache.org/jira/browse/YARN-4302>. SLS
YARN-4367 <https://issues.apache.org/jira/browse/YARN-4367>. SLS
YARN-4612 <https://issues.apache.org/jira/browse/YARN-4612>. SLS
----- Node labeling
MAPREDUCE-6304
<https://issues.apache.org/jira/browse/MAPREDUCE-6304>
YARN-2943 <https://issues.apache.org/jira/browse/YARN-2943>
YARN-4109 <https://issues.apache.org/jira/browse/YARN-4109>
YARN-4140 <https://issues.apache.org/jira/browse/YARN-4140>
YARN-4250 <https://issues.apache.org/jira/browse/YARN-4250>
YARN-4925 <https://issues.apache.org/jira/browse/YARN-4925>
--
Zhe Zhang
Apache Hadoop Committer
http://zhe-thoughts.github.io/about/ | @oldcap
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
