[jira] [Resolved] (HDFS-1056) Multi-node RPC deadlocks during block recovery

[ https://issues.apache.org/jira/browse/HDFS-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1056. Resolution: Unresolved Target Version/s: (was: ) closing as stale. > Multi-node

[jira] [Resolved] (HDFS-7605) hadoop distcp hftp://192.168.80.31:50070/user/wp hdfs://192.168.210.10:8020/

[ https://issues.apache.org/jira/browse/HDFS-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-7605. Resolution: Cannot Reproduce > hadoop distcp hftp://192.168.80.31:50070/user/wp hdfs://192.16

[jira] [Resolved] (HDFS-1951) Null pointer exception comes when Namenode recovery happens and there is no response from client to NN more than the hardlimit for NN recovery and the current block is mo

[ https://issues.apache.org/jira/browse/HDFS-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1951. Resolution: Won't Fix > Null pointer exception comes when Namenode recovery happens and there

[jira] [Resolved] (HDFS-2075) Add "Number of Reporting Nodes" to namenode web UI

[ https://issues.apache.org/jira/browse/HDFS-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-2075. Resolution: Unresolved closing as stale > Add "Number of Reporting Nodes" to namenode web UI

[jira] [Resolved] (HDFS-2723) tomcat tar in the apache archive is corrupted

[ https://issues.apache.org/jira/browse/HDFS-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-2723. Resolution: Cannot Reproduce > tomcat tar in the apache archive is corrupted > --

[jira] [Resolved] (HDFS-2774) Use TestDFSIO to test HDFS, and Failed with the exception: All datanodes are bad. Aborting...

[ https://issues.apache.org/jira/browse/HDFS-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-2774. Resolution: Won't Fix > Use TestDFSIO to test HDFS, and Failed with the exception: All datano

[jira] [Resolved] (HDFS-1961) New architectural documentation created

[ https://issues.apache.org/jira/browse/HDFS-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1961. Resolution: Won't Fix Closing as stale. > New architectural documentation created >

[jira] [Resolved] (HDFS-2300) TestFileAppend4 and TestMultiThreadedSync fail on 20.append and 20-security.

[ https://issues.apache.org/jira/browse/HDFS-2300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-2300. Resolution: Fixed Target Version/s: 0.20.205.0, 0.20-append (was: 0.20-append, 0.20

[jira] [Resolved] (HDFS-1262) Failed pipeline creation during append leaves lease hanging on NN

[ https://issues.apache.org/jira/browse/HDFS-1262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1262. Resolution: Won't Fix > Failed pipeline creation during append leaves lease hanging on NN > -

[jira] [Resolved] (HDFS-1982) Null pointer exception is thrown when NN restarts with a block lesser in size than the block that is present in DN1 but the generation stamp is greater in the NN

[ https://issues.apache.org/jira/browse/HDFS-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1982. Resolution: Won't Fix > Null pointer exception is thrown when NN restarts with a block lesser

[jira] [Resolved] (HDFS-1488) hadoop will terminate Web service process when a hadoop mapreduce task is finished.

[ https://issues.apache.org/jira/browse/HDFS-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1488. Resolution: Not a Problem > hadoop will terminate Web service process when a hadoop mapreduce

[jira] [Resolved] (HDFS-1247) Improvements to HDFS-1204 test

[ https://issues.apache.org/jira/browse/HDFS-1247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1247. Resolution: Won't Fix stale > Improvements to HDFS-1204 test > -

Re: Looking to a Hadoop 3 release

Hi Raymie, Konst proposed just releasing off of trunk rather than cutting a branch-2, and there was general agreement there. So, consider #3 abandoned. 1&2 can be achieved at the same time, we just need to avoid using JDK8 language features in trunk so things can be backported. Best, Andrew On M

[jira] [Resolved] (HDFS-1578) First step towards data transter protocol compatibility: support DatanodeProtocol#getDataTransferProtocolVersion

[ https://issues.apache.org/jira/browse/HDFS-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1578. Resolution: Later probably fixed with protobuf. closing as stale. > First step towards data

[jira] [Resolved] (HDFS-2213) DataNode gets stuck while shutting down minicluster

[ https://issues.apache.org/jira/browse/HDFS-2213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-2213. Resolution: Fixed closing as stale > DataNode gets stuck while shutting down minicluster > -

[jira] [Resolved] (HDFS-1553) Hftp file read should retry a different datanode if the chosen best datanode fails to connect to NameNode

[ https://issues.apache.org/jira/browse/HDFS-1553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1553. Resolution: Won't Fix hftp is dead > Hftp file read should retry a different datanode if the

[jira] [Resolved] (HDFS-1566) Test that covers full partition

[ https://issues.apache.org/jira/browse/HDFS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1566. Resolution: Later closing as later as recommended. > Test that covers full partition > ---

[jira] [Resolved] (HDFS-2248) Port recoverLease API from append 0.20 to trunk

[ https://issues.apache.org/jira/browse/HDFS-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-2248. Resolution: Later probably stale. closing. > Port recoverLease API from append 0.20 to trunk

[jira] [Resolved] (HDFS-2192) Update HDFS dependency of Java for deb package

[ https://issues.apache.org/jira/browse/HDFS-2192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-2192. Resolution: Won't Fix we've stop building packages. closing as won't fix > Update HDFS depen

[jira] [Resolved] (HDFS-2336) hftp only supports remote hdfs servers

[ https://issues.apache.org/jira/browse/HDFS-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-2336. Resolution: Won't Fix hftp is dead. closing as won't fix. > hftp only supports remote hdfs s

[jira] [Resolved] (HDFS-2182) Exceptions in DataXceiver#run can result in a zombie datanode

[ https://issues.apache.org/jira/browse/HDFS-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-2182. Resolution: Later closing this as stale > Exceptions in DataXceiver#run can result in a zomb

Re: Hadoop 3.x: what about shipping trunk as a 2.x release in 2015?

Between this and the other thread, I’m seeing: * companies that were forced to make internal forks because their patches were ignored are now considered the deciders for whether we move forward * 5 years since the last branch off of trunk is considered ‘soon’ * M

Re: Hadoop - Major releases

Hi Mayank, > 1. We would be moving to Hadoop -3 (Not this year though) however I don't > see we can do another JDK upgrade so soon. So the point I am trying to make > is we should be supporting jdk 7 as well for Hadoop-3. > > We'll still be releasing 2.x releases for a while, with similar feature

[jira] [Created] (HDFS-7912) Erasure Coding: track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks

Jing Zhao created HDFS-7912: --- Summary: Erasure Coding: track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks Key: HDFS-7912 URL: https://issues.apache.org/jira/browse/HDFS-7912

[jira] [Resolved] (HDFS-7853) Erasure coding: extend LocatedBlocks to support reading from striped files

[ https://issues.apache.org/jira/browse/HDFS-7853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-7853. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed I've committed this. > Erasu

[jira] [Created] (HDFS-7911) Buffer Overflow when running HBase on HDFS Encryption Zone

Xiaoyu Yao created HDFS-7911: Summary: Buffer Overflow when running HBase on HDFS Encryption Zone Key: HDFS-7911 URL: https://issues.apache.org/jira/browse/HDFS-7911 Project: Hadoop HDFS Issue Ty

[jira] [Created] (HDFS-7910) Modify NNThroughputBenchmark to be able to provide some metrics

Charles Lamb created HDFS-7910: -- Summary: Modify NNThroughputBenchmark to be able to provide some metrics Key: HDFS-7910 URL: https://issues.apache.org/jira/browse/HDFS-7910 Project: Hadoop HDFS

Re: Hadoop 3.x: what about shipping trunk as a 2.x release in 2015?

On 09/03/2015 15:56, "Andrew Wang" wrote: >I find this proposal very surprising. We've intentionally deferred >incompatible changes to trunk, because they are incompatible and do not >belong in a minor release. Now we are supposed to blur our eyes and >release >these changes anyway? I don't see

Re: Hadoop - Major releases

Hi Andrew, I wish things are as simple as you are pointing out. At least they are not for us so far. Couple of things 1. We would be moving to Hadoop -3 (Not this year though) however I don't see we can do another JDK upgrade so soon. So the point I am trying to make is we should be supporting j

[jira] [Created] (HDFS-7909) Improve TestMemoryManager and TestProbingHashTable

Colin Patrick McCabe created HDFS-7909: -- Summary: Improve TestMemoryManager and TestProbingHashTable Key: HDFS-7909 URL: https://issues.apache.org/jira/browse/HDFS-7909 Project: Hadoop HDFS

Re: Hadoop 3.x: what about shipping trunk as a 2.x release in 2015?

I find this proposal very surprising. We've intentionally deferred incompatible changes to trunk, because they are incompatible and do not belong in a minor release. Now we are supposed to blur our eyes and release these changes anyway? I don't see this ending well. One higher-level goal we should

Re: Hadoop 3.x: what about shipping trunk as a 2.x release in 2015?

Colin, Do you have a list of incompatible changes other than the shell-script rewrite? If we do have others we'd have to fix them anyway for the current plan on hadoop-3.x right? So, I don't see the difference? Arun From: Colin P. McCabe Sent: Monday,

Re: Hadoop - Major releases

Hi Mayank, Note that Hadoop 3 does not mean the end of updates for Hadoop 2.x, which will keep supporting JDK7 for a while yet. Someone on the original thread also proposed keeping Hadoop 3 JDK7-source compatible to make backports to 2.x easier. I support this. Note also that the jump from Hadoop

[jira] [Resolved] (HDFS-3742) libhdfs symbol visibility issues

[ https://issues.apache.org/jira/browse/HDFS-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe resolved HDFS-3742. Resolution: Duplicate Fix Version/s: 2.7.0 implemented in HDFS-7879 > libhdfs

[jira] [Reopened] (HDFS-2605) CHANGES.txt has two "Release 0.21.1" sections

[ https://issues.apache.org/jira/browse/HDFS-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer reopened HDFS-2605: reopening--I checked the wrong changes file. :) > CHANGES.txt has two "Release 0.21.1" sections

[jira] [Resolved] (HDFS-2605) CHANGES.txt has two "Release 0.21.1" sections

[ https://issues.apache.org/jira/browse/HDFS-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-2605. Resolution: Fixed stale > CHANGES.txt has two "Release 0.21.1" sections > --

[jira] [Resolved] (HDFS-2224) The libhdfs test is broken

[ https://issues.apache.org/jira/browse/HDFS-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-2224. Resolution: Fixed stale > The libhdfs test is broken > -- > >

Re: Hadoop 3.x: what about shipping trunk as a 2.x release in 2015?

Java 7 will be end-of-lifed in April 2015. I think it would be unwise to plan a new Hadoop release against a version of Java that is almost obsolete and (soon) no longer receiving security updates. I think people will be willing to roll out a new version of Java for Hadoop 3.x. Similarly, the wh

Re: Hadoop 3.x: what about shipping trunk as a 2.x release in 2015?

Steve, From: Steve Loughran Sent: Monday, March 09, 2015 2:15 PM To: mapreduce-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org; common-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Hadoop 3.x: what about shipping trunk as a 2.x release in 2

Re: Hadoop - Major releases

Hi Guys, >From my prospective @ ebay we are not going to upgrade to JDK 8 any time soon we just upgraded to 7 and not want to move further at least this year so I will request you guys not to drop the support for JDK 7 as that would be very crucial for us to move forward. We also just completed o

Hadoop 3.x: what about shipping trunk as a 2.x release in 2015?

If 3.x is going to be Java 8 & not backwards compatible, I don't expect anyone wanting to use this in production until some time deep into 2016. Issue: JDK 8 vs 7 It will require Hadoop clusters to move up to Java 8. While there's dev pull for this, there's ops pull against this: people are st

[jira] [Created] (HDFS-7908) Use larger value for fs.s3a.connection.timeout and change the unit to seconds.

Lei (Eddy) Xu created HDFS-7908: --- Summary: Use larger value for fs.s3a.connection.timeout and change the unit to seconds. Key: HDFS-7908 URL: https://issues.apache.org/jira/browse/HDFS-7908 Project: Had

[jira] [Resolved] (HDFS-7905) Refactor the checking the minimum replication logic in BlockManager

[ https://issues.apache.org/jira/browse/HDFS-7905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-7905. - Resolution: Won't Fix Looks like instead of doing this refactoring in trunk, it is easier to keep the chan

[jira] [Created] (HDFS-7907) Update DecommissionManager to support striped blocks

Jing Zhao created HDFS-7907: --- Summary: Update DecommissionManager to support striped blocks Key: HDFS-7907 URL: https://issues.apache.org/jira/browse/HDFS-7907 Project: Hadoop HDFS Issue Type: Sub-

Re: Looking to a Hadoop 3 release

> On Mar 5, 2015, at 3:21 PM, Siddharth Seth wrote: > > 2) Simplification of configs - potentially separating client side configs > and those used by daemons. This is another source of perpetual confusion > for users. + 1 on this. sanjay

[jira] [Created] (HDFS-7906) BlockReport of a RUR can have a FINALIZED original replica from Truncate

Plamen Jeliazkov created HDFS-7906: -- Summary: BlockReport of a RUR can have a FINALIZED original replica from Truncate Key: HDFS-7906 URL: https://issues.apache.org/jira/browse/HDFS-7906 Project: Had

Re: Looking to a Hadoop 3 release

On Mar 6, 2015, at 5:20 PM, Chris Douglas wrote: > On Fri, Mar 6, 2015 at 4:32 PM, Vinod Kumar Vavilapalli > wrote: >> I'd encourage everyone to post their wish list on the Roadmap wiki that >> *warrants* making incompatible changes forcing us to go 3.x. > > This is a useful exercise, but not

[jira] [Created] (HDFS-7905) Refactor the checking the minimum replication logic in BlockManager

Jing Zhao created HDFS-7905: --- Summary: Refactor the checking the minimum replication logic in BlockManager Key: HDFS-7905 URL: https://issues.apache.org/jira/browse/HDFS-7905 Project: Hadoop HDFS

Hadoop-Hdfs-trunk-Java8 - Build # 118 - Still Failing

See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/118/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 12318 lines...] [INFO] Deleting /home/jenkins/jenki

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #118

See Changes: [stevel] HADOOP-11670. Regression: s3a auth setup broken. (Adam Budde via stevel) [junping_du] YARN-3296. Mark ResourceCalculatorProcessTree class as Public for configurable resource monitoring. Contributed by Hit

Build failed in Jenkins: Hadoop-Hdfs-trunk #2059

See Changes: [stevel] HADOOP-11670. Regression: s3a auth setup broken. (Adam Budde via stevel) [junping_du] YARN-3296. Mark ResourceCalculatorProcessTree class as Public for configurable resource monitoring. Contributed by Hitesh S

Hadoop - Major releases

Over the last few days, we have had lots of discussions that have intertwined several major themes: # When/why do we make major Hadoop releases? # When/how do we move to major JDK versions? # To a lesser extent, we have debated another theme: what do we do about trunk? For now, let's park