[jira] [Resolved] (HDFS-6932) Balancer and Mover tools should ignore replicas on RAM_DISK

2014-09-24 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal resolved HDFS-6932. - Resolution: Fixed Fix Version/s: HDFS-6581 Hadoop Flags: Reviewed +1 Committed to the

[jira] [Resolved] (HDFS-7143) Fix findbugs warnings in HDFS-6581 branch

2014-09-24 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal resolved HDFS-7143. - Resolution: Fixed Fix Version/s: HDFS-6581 Hadoop Flags: Reviewed Thanks a lot for fixi

[jira] [Resolved] (HDFS-7089) Fix findbugs warnings in the HDFS-6581 branch

2014-09-24 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal resolved HDFS-7089. - Resolution: Duplicate Fix Version/s: HDFS-6581 > Fix findbugs warnings in the HDFS-6581 bran

Re: Thinking ahead to hadoop-2.6

2014-09-24 Thread Vinod Kumar Vavilapalli
We can branch off in a week or two so that work on branch-2 itself can go ahead with other features that can't fit in 2.6. Independent of that, we can then decide on the timeline of the release candidates once branch-2.6 is close to being done w.r.t the planned features. Branching it off can let u

Re: Thinking ahead to hadoop-2.6

2014-09-24 Thread Vinod Kumar Vavilapalli
Thank for restarting this, Arun! The following are the efforts that I am involved in directly or indirectly and shepherding from YARN's point of view: rolling-upgrades (YARN-666), work-preserving RM restart (YARN-556), minimal long-running services support (YARN-896), including service-registry (Y

[jira] [Created] (HDFS-7143) Fix findbugs warnings in HDFS-6581 branch

2014-09-24 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-7143: - Summary: Fix findbugs warnings in HDFS-6581 branch Key: HDFS-7143 URL: https://issues.apache.org/jira/browse/HDFS-7143 Project: Hadoop HDFS Issue T

Re: [VOTE] Merge HDFS-6581 to trunk - Writing to replicas in memory.

2014-09-24 Thread Colin McCabe
On Wed, Sep 24, 2014 at 4:03 PM, Arpit Agarwal wrote: >> I would appreciate in the future if benchmarks were posted >> a day or two before a merge vote of a performance improvement was >> suggested. >> I feel that it would have been better to float the idea of a >> merge on the JIRA before actuall

Re: Thinking ahead to hadoop-2.6

2014-09-24 Thread Jing Zhao
Just to give a quick update about the current status of HDFS-6584 (archival storage). So far after HDFS-7081 got committed this morning, the main functionalities have already been finished. As a summary two new DistributedFileSystem APIs are added: getStoragePolicies and setStoragePolicy. We have a

Re: [VOTE] Merge HDFS-6581 to trunk - Writing to replicas in memory.

2014-09-24 Thread Arpit Agarwal
> I would appreciate in the future if benchmarks were posted > a day or two before a merge vote of a performance improvement was > suggested. > I feel that it would have been better to float the idea of a > merge on the JIRA before actually calling it, Colin, I noted my intention to start the merg

Re: [VOTE] Merge HDFS-6581 to trunk - Writing to replicas in memory.

2014-09-24 Thread Suresh Srinivas
On Wed, Sep 24, 2014 at 2:19 PM, Colin McCabe wrote: > On Wed, Sep 24, 2014 at 11:12 AM, Suresh Srinivas > wrote: > > Features are done in a separate branch for two reasons: 1) During a > feature > > development the branch may be not functional 2) The high level approach > and > > design is not

Re: Thinking ahead to hadoop-2.6

2014-09-24 Thread Jitendra Pandey
I also believe its worth a week's wait to include HDFS-6584 and HDFS-6581 in 2.6. On Wed, Sep 24, 2014 at 3:28 PM, Suresh Srinivas wrote: > Given some of the features are in final stages of stabilization, > Arun, we should hold off creating 2.6 branch or building an RC by a week? > All the featu

Re: Thinking ahead to hadoop-2.6

2014-09-24 Thread Suresh Srinivas
Given some of the features are in final stages of stabilization, Arun, we should hold off creating 2.6 branch or building an RC by a week? All the features in flux are important ones and worth delaying the release by a week. On Wed, Sep 24, 2014 at 11:36 AM, Andrew Wang wrote: > Hey Nicholas, >

[jira] [Resolved] (HDFS-7141) Integrate HDFS-6581 memory usage with HDFS-4949

2014-09-24 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal resolved HDFS-7141. - Resolution: Duplicate > Integrate HDFS-6581 memory usage with HDFS-4949 > -

Re: [VOTE] Merge HDFS-6581 to trunk - Writing to replicas in memory.

2014-09-24 Thread Colin McCabe
On Wed, Sep 24, 2014 at 11:12 AM, Suresh Srinivas wrote: > Features are done in a separate branch for two reasons: 1) During a feature > development the branch may be not functional 2) The high level approach and > design is not very clear and development can continue while that is being > sorted

[jira] [Created] (HDFS-7142) Implement a 2Q eviction strategy for HDFS-6581

2014-09-24 Thread Colin Patrick McCabe (JIRA)
Colin Patrick McCabe created HDFS-7142: -- Summary: Implement a 2Q eviction strategy for HDFS-6581 Key: HDFS-7142 URL: https://issues.apache.org/jira/browse/HDFS-7142 Project: Hadoop HDFS

[jira] [Created] (HDFS-7141) Integrate HDFS-6581 memory usage with HDFS-4949

2014-09-24 Thread Colin Patrick McCabe (JIRA)
Colin Patrick McCabe created HDFS-7141: -- Summary: Integrate HDFS-6581 memory usage with HDFS-4949 Key: HDFS-7141 URL: https://issues.apache.org/jira/browse/HDFS-7141 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-5722) Implement compression in the HTTP server of SNN / SBN instead of FSImage

2014-09-24 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai resolved HDFS-5722. -- Resolution: Invalid As the current PB-based fsimage format compresses each section, this jira is no long

Re: Thinking ahead to hadoop-2.6

2014-09-24 Thread Andrew Wang
Hey Nicholas, My concern about Archival Storage isn't related to the code quality or the size of the feature. I think that you and Jing did good work. My concern is that once we ship, we're locked into that set of archival storage APIs, and these APIs are not yet finalized. Simply being able to tu

Re: [VOTE] Merge HDFS-6581 to trunk - Writing to replicas in memory.

2014-09-24 Thread Jitendra Pandey
I would consider following as blockers to merge to trunk: a) Any bugs that makes the feature unusable or cause instability or cause any regression. b) Any bad design choices that add technical debt. c) Any missing functionality, that makes the feature useless or difficult to use for its intended u

[jira] [Created] (HDFS-7140) Add a tool to list all the existing block storage policies

2014-09-24 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-7140: --- Summary: Add a tool to list all the existing block storage policies Key: HDFS-7140 URL: https://issues.apache.org/jira/browse/HDFS-7140 Project: Hadoop HDFS Issue Type

Re: [VOTE] Merge HDFS-6581 to trunk - Writing to replicas in memory.

2014-09-24 Thread Suresh Srinivas
I am +1 on the merge. On Wed, Sep 24, 2014 at 11:12 AM, Suresh Srinivas wrote: > Features are done in a separate branch for two reasons: 1) During a > feature development the branch may be not functional 2) The high level > approach and design is not very clear and development can continue while

Re: [VOTE] Merge HDFS-6581 to trunk - Writing to replicas in memory.

2014-09-24 Thread Suresh Srinivas
Features are done in a separate branch for two reasons: 1) During a feature development the branch may be not functional 2) The high level approach and design is not very clear and development can continue while that is being sorted out. In case of this feature, clearly (2) is not an issue. We have

[jira] [Created] (HDFS-7139) Unit test for creating encryption zone on root path

2014-09-24 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-7139: --- Summary: Unit test for creating encryption zone on root path Key: HDFS-7139 URL: https://issues.apache.org/jira/browse/HDFS-7139 Project: Hadoop HDFS Issue Type: Impro

Re: [VOTE] Merge HDFS-6581 to trunk - Writing to replicas in memory.

2014-09-24 Thread Arpit Agarwal
Colin, the feature should be evaluated on its technical merits and not on the speed of development. The high level approach was decided during the community hangout on Apr 30 - five months ago - which you attended. Nothing we have developed is contrary to what was decided that day. We have discuss

[jira] [Created] (HDFS-7138) Fix hftp to work with encryption

2014-09-24 Thread Charles Lamb (JIRA)
Charles Lamb created HDFS-7138: -- Summary: Fix hftp to work with encryption Key: HDFS-7138 URL: https://issues.apache.org/jira/browse/HDFS-7138 Project: Hadoop HDFS Issue Type: Sub-task