Re: [DISCUSS] Remove append?

On 3/22/12 5:41 PM, "Eli Collins" wrote: >On Thu, Mar 22, 2012 at 4:27 PM, wrote: >> Eli, >> >> I think by "current definition of visible length", you mean that once a >> client opens a file and gets block list, it will always be able to read >>up >> to the length at open. >> > >I was thinkin

Re: [DISCUSS] Remove append?

On 3/22/12 10:25 AM, "Eli Collins" wrote: >On Thu, Mar 22, 2012 at 1:26 AM, Konstantin Shvachko > wrote: >> Eli, >> >> I went over the entire discussion on the topic, and did not get it. Is >> there a problem with append? We know it does not work in hadoop-1, >> only flush() does. Is there anyt

Re: [DISCUSS] Remove append?

On 3/20/12 5:37 PM, "Eli Collins" wrote: >Append introduces non-trivial design and code complexity, which is not >worth the cost if we don't have real users. Removing append means we >have the property that HDFS blocks, when finalized, are immutable. >This significantly simplifies the design an

[jira] [Created] (HDFS-3140) Support multiple network interfaces

Support multiple network interfaces --- Key: HDFS-3140 URL: https://issues.apache.org/jira/browse/HDFS-3140 Project: Hadoop HDFS Issue Type: New Feature Reporter: Eli Collins Assignee:

[jira] [Created] (HDFS-3139) Minor Datanode logging improvement

Minor Datanode logging improvement -- Key: HDFS-3139 URL: https://issues.apache.org/jira/browse/HDFS-3139 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Reporter: Eli

[jira] [Created] (HDFS-3138) Move DatanodeInfo#ipcPort and hostName to DatanodeID

Move DatanodeInfo#ipcPort and hostName to DatanodeID Key: HDFS-3138 URL: https://issues.apache.org/jira/browse/HDFS-3138 Project: Hadoop HDFS Issue Type: Improvement Reporter:

[jira] [Created] (HDFS-3137) Bump LAST_UPGRADABLE_LAYOUT_VERSION

Bump LAST_UPGRADABLE_LAYOUT_VERSION --- Key: HDFS-3137 URL: https://issues.apache.org/jira/browse/HDFS-3137 Project: Hadoop HDFS Issue Type: Improvement Reporter: Eli Collins Assignee:

[jira] [Created] (HDFS-3136) Multiple SLF4J binding warning

Multiple SLF4J binding warning -- Key: HDFS-3136 URL: https://issues.apache.org/jira/browse/HDFS-3136 Project: Hadoop HDFS Issue Type: Bug Components: build Affects Versions: 0.23.0 Repor

[jira] [Created] (HDFS-3135) Build a war file for HttpFS instead of packaging the server (tomcat) along with the application.

Build a war file for HttpFS instead of packaging the server (tomcat) along with the application. Key: HDFS-3135 URL: https://issues.apache.org/jira/browse/HDFS-3135

[jira] [Created] (HDFS-3134) harden edit log loader against malformed or malicious input

harden edit log loader against malformed or malicious input --- Key: HDFS-3134 URL: https://issues.apache.org/jira/browse/HDFS-3134 Project: Hadoop HDFS Issue Type: Bug Repo

[jira] [Created] (HDFS-3133) Add support for DFS upgrade with HA enabled

Add support for DFS upgrade with HA enabled --- Key: HDFS-3133 URL: https://issues.apache.org/jira/browse/HDFS-3133 Project: Hadoop HDFS Issue Type: Improvement Components: ha, name-node

Re: svn commit: r1304067 - in /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project: ./ bin/ conf/ hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/ hadoop-mapreduce-exam

Got it. Thanks Nicholas! Although I still find it intriguing that it is always a constant set of paths that "track" svn:mergeinfo stuff. Perhaps a property needs to be unset on them. My SVN-fu isn't very strong to know what exactly is the issue though. On Fri, Mar 23, 2012 at 11:31 PM, Tsz Wo Sze

Re: svn commit: r1304067 - in /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project: ./ bin/ conf/ hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/ hadoop-mapreduce-exam

Hi Harsh, $svn --version svn, version 1.6.17 (r1128011)    compiled Nov  8 2011, 18:14:46 I think the version is irrelevant.  You have to merge under the sub-project directory, i.e. "svn merge -c /path/to/trunk/hadoop-hdfs-project/hadoop-hdfs .", provided that all the changes are that sub-di

[jira] [Created] (HDFS-3132) Findbugs warning on HDFS trunk

Findbugs warning on HDFS trunk -- Key: HDFS-3132 URL: https://issues.apache.org/jira/browse/HDFS-3132 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.24.0, 0.23.3

Re: [DISCUSS] Remove append?

I think Yarn/MR might be able to benefit from the ability to append to logs in hdfs. It might reduce some of the after-the-fact copying of logs into hdfs. Daryn On Mar 22, 2012, at 8:18 PM, Dhruba Borthakur wrote: > I think "append" would be useful. But not precisely sure which applications >

Build failed in Jenkins: Hadoop-Hdfs-trunk #993

See Changes: [todd] HDFS-3071. haadmin failover command does not provide enough detail when target NN is not ready to be active. Contributed by Todd Lipcon. [eli] HADOOP-8159. NetworkTopology: getLeaf should check for invalid topolo

Hadoop-Hdfs-trunk - Build # 993 - Still Failing

See https://builds.apache.org/job/Hadoop-Hdfs-trunk/993/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 12611 lines...] [INFO]

Re: libhdfs process fork problem

Hi Tareq, This is because libhdfs will keep a bit of state data (especially if the master was connected to HDFS). Three suggestions: 1) [Likely to work] Fork the children first, then do any HDFS actions in the master. Alternately, don't have the master do any HDFS actions; have it fork a chil