Re: Willing to work on Issue - HDFS-4901 - Where to start?

Vivek, I have added you as HDFS contributor and assigned the jira to you. Regards, Suresh On Mon, Jul 8, 2013 at 6:30 PM, Vivek Ganesan wrote: > Thanks!! > > I have sent my ICLA to the mail address secret...@apache.org , as > mentioned > in the text of ICLA > > Could you please assign the JIRA

[jira] [Created] (HDFS-4968) Provide configuration option for FileSystem symlink resolution

Andrew Wang created HDFS-4968: - Summary: Provide configuration option for FileSystem symlink resolution Key: HDFS-4968 URL: https://issues.apache.org/jira/browse/HDFS-4968 Project: Hadoop HDFS I

[jira] [Created] (HDFS-4967) Generate block ID sequentially cannot work with QJM HA

Fengdong Yu created HDFS-4967: - Summary: Generate block ID sequentially cannot work with QJM HA Key: HDFS-4967 URL: https://issues.apache.org/jira/browse/HDFS-4967 Project: Hadoop HDFS Issue Type

Re: data loss after cluster wide power loss

Thanks. Suresh and Kihwal are right-- renames are journalled, but not necessarily durable (stored to disk). I was getting mixed up with HDFS semantics, in which we actually do make the journal durable before returning success to the client. It might be a good idea for HDFS to fsync the file desc

Re: Willing to work on Issue - HDFS-4901 - Where to start?

Thanks!! I have sent my ICLA to the mail address secret...@apache.org , as mentioned in the text of ICLA Could you please assign the JIRA to me? Thank you. Regards, Vivek On Mon, Jul 8, 2013 at 8:05 PM, Suresh Srinivas wrote: > Vivek, > > For information on how to contribute see - > http://w

Re: Can I move block data directly?

I know nothing. It seems that circumventing normal operations could be very bad. There was an example of something similar at hadoop summit. Some very experienced contributors decided they should edit meta data. they broke their cluster. Just say no ! ;) On Jul 8, 2013 9:01 PM, "Azuryy Yu"

RE: [VOTE] Release Apache Hadoop 0.23.9

+1, downloaded the release, verified signs, ran examples and succeeded. Thanks Devaraj k -Original Message- From: Thomas Graves [mailto:tgra...@yahoo-inc.com] Sent: 01 July 2013 22:50 To: common-...@hadoop.apache.org Cc: hdfs-dev@hadoop.apache.org; mapreduce-...@hadoop.apache.org; yarn

[jira] [Created] (HDFS-4966) implement advisory caching for RawLocalFilesystem

Colin Patrick McCabe created HDFS-4966: -- Summary: implement advisory caching for RawLocalFilesystem Key: HDFS-4966 URL: https://issues.apache.org/jira/browse/HDFS-4966 Project: Hadoop HDFS

Re: Can I move block data directly?

Hi Harsh, I also do agree with you that this is crude. and balancer is the right way. I just want to slove the problem very quickly. and only a few nodes involved. Thanks. On Tue, Jul 9, 2013 at 8:50 AM, Harsh J wrote: > Eitan, > > The block to host mapping isn't persisted in the metadata.

Re: Can I move block data directly?

Eitan, The block to host mapping isn't persisted in the metadata. This is also the reason why the steps include a restart, which will re-trigger a block report (and avoid gotchas) that will update the NN of the new listing at each DN. Thats what makes this method "crude" at the same time - you're

[jira] [Created] (HDFS-4965) Make datanodes support multiple QOP for RPC and streaming protocols

Benoy Antony created HDFS-4965: -- Summary: Make datanodes support multiple QOP for RPC and streaming protocols Key: HDFS-4965 URL: https://issues.apache.org/jira/browse/HDFS-4965 Project: Hadoop HDFS

[jira] [Created] (HDFS-4964) Enable multiple QOP for NameNode

Benoy Antony created HDFS-4964: -- Summary: Enable multiple QOP for NameNode Key: HDFS-4964 URL: https://issues.apache.org/jira/browse/HDFS-4964 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-4963) Improve multihoming support in namenode

Arpit Agarwal created HDFS-4963: --- Summary: Improve multihoming support in namenode Key: HDFS-4963 URL: https://issues.apache.org/jira/browse/HDFS-4963 Project: Hadoop HDFS Issue Type: Bug

Re: [VOTE] Release Apache Hadoop 0.23.9

+1 downloaded the release. verified sigs and sums. verified docs and verified native compiles on ubuntu and runs examples as expected. On Mon, Jul 8, 2013 at 3:48 PM, Kihwal Lee wrote: > +1 Downloaded it and ran several sample tests in a pseudo-distributed > cluster. > > > Kihwal > > On 7/1/13

Re: [VOTE] Release Apache Hadoop 0.23.9

+1 Downloaded it and ran several sample tests in a pseudo-distributed cluster. Kihwal On 7/1/13 12:20 PM, "Thomas Graves" wrote: >I've created a release candidate (RC0) for hadoop-0.23.9 that I would like >to release. > >The RC is available at: >http://people.apache.org/~tgraves/hadoop-0.23.9-

Re: Willing to work on Issue - HDFS-4901 - Where to start?

In the past, when posting a patch, one could grant the license to ASF. That is choice is no longer available post jira upgrade. Since then, the only option I see is signing ICLA, to grant license for the contributed work. However, after doing a bit of research, I see that ICLA is not required. See

[jira] [Resolved] (HDFS-3092) Enable journal protocol based editlog streaming for standby namenode

[ https://issues.apache.org/jira/browse/HDFS-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas resolved HDFS-3092. --- Resolution: Won't Fix > Enable journal protocol based editlog streaming for standby nameno

Re: Willing to work on Issue - HDFS-4901 - Where to start?

Hi Suresh, I was not aware of registering an iCLA with ASF. Can you please elaborate a little bit, it says fill up the form , sign it and send it to secret...@apache.org. My doubt is once it is sent after signing , what will be my status as a contributor because I have contributed quite a few Had

[jira] [Resolved] (HDFS-3182) Add class to manage JournalList

[ https://issues.apache.org/jira/browse/HDFS-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-3182. --- Resolution: Won't Fix This was obsoleted by other work in HDFS-3077 > Add class to ma

[jira] [Created] (HDFS-4962) Use enum for nfs constants

Tsz Wo (Nicholas), SZE created HDFS-4962: Summary: Use enum for nfs constants Key: HDFS-4962 URL: https://issues.apache.org/jira/browse/HDFS-4962 Project: Hadoop HDFS Issue Type: Sub-

Re: Willing to work on Issue - HDFS-4901 - Where to start?

Vivek, For information on how to contribute see - http://wiki.apache.org/hadoop/HowToContribute Please register an iCLA with ASF - see http://www.apache.org/licenses/icla.txt Once this is done, I will add you as a contributor and assign the jira to you. Regards, Suresh On Mon, Jul 8, 2013 at 3

Re: Can I move block data directly?

Hi Eitan, yup, The namenode has a mapping of blocks to datanodes, which keeping in memory. and you are also right, DN also keeps block structure in memory. but you've noticed, I restart the DN after I moved block data manually. then during the data node's start, Block Scanner can scan the block

Re: Can I move block data directly?

Hi Azurry, I'd also like to be able to manually move blocks. One piece that is missing in your current approach is updating any block mappings that the cluster relies on. The namenode has a mapping of blocks to datanodes, and the datanode has, as the comments say, a "block -> stream of bytes" mapp

Re: [VOTE] Release Apache Hadoop 0.23.9

Friendly reminder this vote ends today please try it out if you have time. Thanks, Tom Graves On 7/1/13 12:20 PM, "Thomas Graves" wrote: >I've created a release candidate (RC0) for hadoop-0.23.9 that I would like >to release. > >The RC is available at: >http://people.apache.org/~tgraves/hadoop-

Hadoop-Hdfs-trunk - Build # 1454 - Still Failing

See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1454/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 10783 lines...] [INFO] Building Apache Hadoop HDFS Projec

Build failed in Jenkins: Hadoop-Hdfs-trunk #1454

See Changes: [szetszwo] HDFS-4645. Move from randomly generated block ID to sequentially generated block ID. Contributed by Arpit Agarwal -- [...truncated 10590 lines...] Tests run: 50, Fail

Jenkins build became unstable: Hadoop-Hdfs-0.23-Build #662

See

Hadoop-Hdfs-0.23-Build - Build # 662 - Unstable

See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/662/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 11843 lines...] [INFO] [INFO] --- maven-source-plugi

Re: Can I move block data directly?

If the imbalance is across data nodes then you need to run the balancer. Sent from my iPad On Jul 8, 2013, at 1:15 AM, Azuryy Yu wrote: > Hi Dear all, > > There are some unbalanced data nodes in my cluster, some nodes reached more > than 95% disk usage. > > so Can I move some block data from

Willing to work on Issue - HDFS-4901 - Where to start?

Hi, I am willing to work on issue HDFS-4901 . I am new to hadoop development community. I would like to contribute. Can anyone give me URLs/docs which deal with hadoop (in specific hdfs) contribution process? Thank you. Regards, Vivek Ganesan (v

Facing issues with hadoop compression

Hello, Is there any way of decompressing LZ4 files? I am using the following: Cloudera version: cdh4.2 Linux version: Linux 2.6.18-348.6.1.el5 (red hat) Hadoop version: 2.0.0 Mahout Version: 0.7 Thank you. Regards, Sugato