Really looking forward to getting this in.
Couple of questions:
* Can you maybe comment a bit on the type of scale testing done ?
Specifically, the number of resources tested with and any point where it is
discovered that performance might take a hit. Also, given that we do not
have AM's that curr
Xiao Chen created HDFS-12359:
Summary: Re-encryption should operate with minimum KMS ACL
requirements.
Key: HDFS-12359
URL: https://issues.apache.org/jira/browse/HDFS-12359
Project: Hadoop HDFS
Hi all,
I've written up a status report for the current state of Hadoop 3 on the
wiki. I've also pasted it below for your convenience.
Cheers,
Andrew
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates
2017-08-25
Another month flew by without an update. This is a
Jonathan, thanks for the heads up. I don't have much familiarity with YARN,
but gave the PBs and pom changes a look, and left a few small comments on
the umbrella JIRA.
This seems like a smaller change than some of the other branch merges we're
discussing, but I'm again reticent about adding scope
Here's a summary of some 1-on-1 conversations I had with contributors of
the different features I'm tracking.
Storage Policy Satisfier (Uma Gangumalla)
* Target version: 3.1.0, maybe 3.0.0 GA
* We're resolving some design discussions on JIRA. Plan is to do some MVP
work on the API to get this into
Hanisha Koneru created HDFS-12358:
-
Summary: Catch IOException when transferring edit log to Journal
current dir through JN sync
Key: HDFS-12358
URL: https://issues.apache.org/jira/browse/HDFS-12358
P
Hi Subru,
Thanks for your vote and your response!
Regarding your question about merging to branch2, I think, after the merge
to trunk is done, it will be good to merge to branch-2 as soon as we can.
A lot of testing with trunk has been done presently and it will be good to
repeat the same with th
[
https://issues.apache.org/jira/browse/HDFS-10825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy resolved HDFS-10825.
---
Resolution: Duplicate
Fix Version/s: 3.0.0-beta1
2.9.0
HDFS-
> From a release management perspective, it's *extremely* reasonable to block
> the inclusion of new features a month from the planned release date. A
> typical software development lifecycle includes weeks of feature freeze and
> weeks of code freeze. It is no knock on any developer or any feat
Thanks Subru for voting.
> What are the timelines you are looking for getting this into branch-2?
We haven't yet decided on it and were thinking of discussing this in detail
within the team after merge to trunk.
The timelines would depend on whether we release whatever we merge to trunk
in 2.9 o
[
https://issues.apache.org/jira/browse/HDFS-11220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy resolved HDFS-11220.
---
Resolution: Workaround
Fix Version/s: 3.0.0-beta1
The core issue described in t
[
https://issues.apache.org/jira/browse/HDFS-11218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy resolved HDFS-11218.
---
Resolution: Workaround
The core issue described in the jira is not a problem any more.
[
https://issues.apache.org/jira/browse/HDFS-11720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy resolved HDFS-11720.
---
Resolution: Duplicate
Fix Version/s: 3.0.0-beta1
HDFS-12217 solved the issue de
Yongjun Zhang created HDFS-12357:
Summary: Let NameNode to bypass external attribute provider for
special user
Key: HDFS-12357
URL: https://issues.apache.org/jira/browse/HDFS-12357
Project: Hadoop HDF
+1 (binding).
I have been following the effort and had few design discussions around the
team especially about how it integrates with Federation. Overall I feel
it's a welcome improvement to YARN.
What are the timelines you are looking for getting this into branch-2?
Thanks,
Subru
On Fri, Aug 2
Allen Wittenauer wrote:
> Doesn't this place an undue burden on the contributor with the first
> incompatible patch to prove worthiness? What happens if it is decided that
> it's not good enough?
It is a burden for that first, "this can't go anywhere else but 4.x"
change, but arguably that sho
Plan looks good to me.
+1
Regards,
Uma
On 8/25/17, 10:36 AM, "Andrew Wang" wrote:
>Hi folks,
>
>With 3.0.0-beta1 fast approaching, I wanted to go over the proposed
>branching strategy.
>
>In the early 2.x days, moving trunk immediately to 3.0.0 was a mistake.
>branch-2 and trunk were virtually
Hi Andrew,
Thanks for updating the proposal, +1.
- Wangda
On Fri, Aug 25, 2017 at 12:16 PM, Allen Wittenauer wrote:
>
> > On Aug 25, 2017, at 10:36 AM, Andrew Wang
> wrote:
>
> > Until we need to make incompatible changes, there's no need for
> > a Hadoop 4.0 version.
>
> Some questions:
>
>
Hanisha Koneru created HDFS-12356:
-
Summary: Unit test for JN sync during Rolling Upgrade
Key: HDFS-12356
URL: https://issues.apache.org/jira/browse/HDFS-12356
Project: Hadoop HDFS
Issue Type
Hi Andrew,
Thanks for starting the discussion - we have a feature YARN-5734 for API
based scheduler configuration that I feel is pretty close to merge (also "a
few weeks"). It's almost completely code and API additions and we were
careful to design it so that it's compatible (feature is also turne
> On Aug 25, 2017, at 10:36 AM, Andrew Wang wrote:
> Until we need to make incompatible changes, there's no need for
> a Hadoop 4.0 version.
Some questions:
Doesn't this place an undue burden on the contributor with the first
incompatible patch to prove worthiness? What happens if it
+1 from me too for the branching proposal.
On Fri, Aug 25, 2017 at 12:56 PM, Eric Payne <
eric.payne1...@yahoo.com.invalid> wrote:
> +1 for this branching proposal.-Eric
>
>
> From: Andrew Wang
> To: "common-...@hadoop.apache.org" ; "
> mapreduce-...@hadoop.apache.org" ; "
> hdfs-dev@hado
Resource profile is similar to TSv2, the feature is:
- Alpha feature, we will not freeze new added APIs. And all added APIs are
explicitly marked to @Unstable.
- Allow rolling upgrade from branch-2.
- Touched existing code, but we have, and will continue tests to make sure
changes are safe.
Discus
+1 for this branching proposal.-Eric
From: Andrew Wang
To: "common-...@hadoop.apache.org" ;
"mapreduce-...@hadoop.apache.org" ;
"hdfs-dev@hadoop.apache.org" ;
"yarn-...@hadoop.apache.org"
Sent: Friday, August 25, 2017 12:36 PM
Subject: [DISCUSS] Branches and versions for Hadoop 3
On 25 August 2017 at 22:39, Andrew Wang wrote:
> Hi Rohith,
>
> Given that we're advertising TSv2 as an alpha feature, I think we're
> allowed to break compatibility. Let's make sure this is clear in the
> release notes and documentation.
>
> That said, with TSv2 phase 2, is the API going to be
Hi folks,
With 3.0.0-beta1 fast approaching, I wanted to go over the proposed
branching strategy.
In the early 2.x days, moving trunk immediately to 3.0.0 was a mistake.
branch-2 and trunk were virtually identical, and increased backport
complexity. Until we need to make incompatible changes, the
Hi Jason,
I agree with this proposal. I'll start another email thread spelling this
out, and gather additional feedback.
Best,
Andrew
On Fri, Aug 25, 2017 at 6:27 AM, Jason Lowe wrote:
> Andrew Wang wrote:
>
>
>> This means I'll cut branch-3 and
>> branch-3.0, and move trunk to 4.0.0 before th
Hi Rohith,
Given that we're advertising TSv2 as an alpha feature, I think we're
allowed to break compatibility. Let's make sure this is clear in the
release notes and documentation.
That said, with TSv2 phase 2, is the API going to be frozen? The umbrella
JIRA refers to "TSv2 alpha2" which indica
+1 (binding)
I've built the current branch, and checked out a few basic areas including
documentation. Also perused the most recent changes that went in.
Thanks much for the great team work! I look forward to seeing it in action.
Regards,
Sangjin
On Fri, Aug 25, 2017 at 9:27 AM, Haibo Chen wro
For more details, see
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/503/
[Aug 24, 2017 7:26:37 AM] (jzhuge) HDFS-12318. Fix IOException condition for
openInfo in DFSInputStream.
[Aug 24, 2017 11:04:38 AM] (bibinchundatt) YARN-7074. Fix NM state store update
comment. Contributed
Rushabh S Shah created HDFS-12355:
-
Summary: Webhdfs needs to support encryption zones.
Key: HDFS-12355
URL: https://issues.apache.org/jira/browse/HDFS-12355
Project: Hadoop HDFS
Issue Type:
Andrew Wang wrote:
> This means I'll cut branch-3 and
> branch-3.0, and move trunk to 4.0.0 before these VOTEs end. This will open
> up development for Hadoop 3.1.0 and 4.0.0.
I can see a need for branch-3.0, but please do not create branch-3. Doing
so will relegate trunk back to the "patch pu
Weiwei Yang created HDFS-12354:
--
Summary: Improve the throttle algorithm in Datanode
BlockDeletingService
Key: HDFS-12354
URL: https://issues.apache.org/jira/browse/HDFS-12354
Project: Hadoop HDFS
33 matches
Mail list logo