RE: How do I copy files from my linux file system to HDFS using a java prog?

This program can read from local file system as well as HDFS. Try Thanks Suresh -Original Message- From: Ajey Shah [mailto:[EMAIL PROTECTED] Sent: Wednesday, May 07, 2008 4:04 AM To: core-user@hadoop.apache.org Subject: RE: How do I copy files from my linux file system to HDFS using a

Collecting output not to file

Hey, >From the examples that I have seen thus far, all of the results from the >reduce function are being written to a file. Instead of writing results to a >file, I want to store them and inspect them after the job is completed. (I >think that I need to implement my own OutputCollector, but I

Re: Monthly Hadoop user group meetings

Thirded. I'm doing my machine learning experiment on a hadoop cluster and eagering to acquire more info on it. :-) 2008/5/7, Leon Mergen <[EMAIL PROTECTED]>: > > On Tue, May 6, 2008 at 6:59 PM, Cole Flournoy <[EMAIL PROTECTED]> > wrote: > > > Is there anyway we could set up some off site web cam c

Re: Hadoop Resiliency

Hi Arv, 1) look for info on "secondary NameNode" on the Hadoop wiki and ML archives. 2) I don't think a NN is supposed to restart a killed DN. I haven't tried it, haven't seen it, but haven't read that anywhere either. 3) I think bin/start-dfs.sh is what you are after, no? Or at least the one

Re: How do I copy files from my linux file system to HDFS using a java prog?

I think that file names of the form "file://directory-path" should work to give you local file access using this program. On 5/6/08 3:34 PM, "Ajey Shah" <[EMAIL PROTECTED]> wrote: > > Thanks Suresh. But even this program reads and writes from the HDFS. What i > need to do is read from my norma

RE: How do I copy files from my linux file system to HDFS using a java prog?

Thanks Suresh. But even this program reads and writes from the HDFS. What i need to do is read from my normal local linux harddrive and write to the HDFS. I'm sorry if I misunderstood your program. Thanks for replying. :) Babu, Suresh wrote: > > > Try this program. Modify the HDFS configura

Re: HDFS: fault tolerance to block losses with namenode failure

Starting in 0.17 release, an application can invoke DFSOutputStream.fsync() to persist block locations for a file even before the file is closed. thanks, dhruba On Tue, May 6, 2008 at 8:11 AM, Cagdas Gerede <[EMAIL PROTECTED]> wrote: > If you are writing 10 blocks for a file and let's say in 10t

Re: Monthly Hadoop user group meetings

On Tue, May 6, 2008 at 6:59 PM, Cole Flournoy <[EMAIL PROTECTED]> wrote: > Is there anyway we could set up some off site web cam conferencing > abilities? I would love to attend, but I am on the east coast. Seconded. I'm from Europe, and am pretty sure that I will watch any video about a Hadoop

Hadoop Resiliency

Hi folks, I'm new to hadoop and just had a few questions regards resiliency i) Does hadoop support redundant NameNodes? I didn't see any mention of it. ii) In a distributed setup, when you kill a DataNode, should the NameNode restart it automatically? I see the NameNode detects (eventually) th

Re: Monthly Hadoop user group meetings

-Will the meeting be recorded as electronic type? I wanna take part in , but I 'm not in US. :D -- deSign thE fuTure http://www.freeis.cn/

copyFromLocal strangeness

Hi All, I recently encountered the following exception when attempting to write to our hadoop grid This is from the name-node log: 2008-05-05 16:11:01,966 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 54310, call create(/2008-05-05/ONE-2008-05-05-14.gz, DFSClient_-164311132, false,

Re: Monthly Hadoop user group meetings

Is there anyway we could set up some off site web cam conferencing abilities? I would love to attend, but I am on the east coast. Thanks, Cole On Tue, May 6, 2008 at 12:53 PM, Ajay Anand <[EMAIL PROTECTED]> wrote: > One of the things we had discussed at the Hadoop summit was to set up > monthly

Monthly Hadoop user group meetings

One of the things we had discussed at the Hadoop summit was to set up monthly user group meetings to discuss topics of interest to the hadoop community. We have scheduled the first of these meetings for May 21st from 6 to 7:30 pm at the Yahoo! Mission College campus. You can register for this at ht

Re: Groovy Scripting for Hadoop

Have you seen my grool system which allows simple MR programs to be written simply? yes, I did take a look. your success with it was part of the reason I went with Groovy first, instead of Jython or Jruby. In addition, I have been working on a layer over Zookeeper to handle collection of

Re: How to perform FILE IO with Hadoop DFS

Please find my response inline, Thanks, -Vikas On Tue, May 6, 2008 at 3:43 PM, Steve Loughran <[EMAIL PROTECTED]> wrote: > vikas wrote: > > > Thank you very much for the right link. It really helped. As many others > > even I'm waiting for > > "Append to files in HDFS" > > > > Is there any thing

Re: How to perform FILE IO with Hadoop DFS

vikas wrote: Thank you very much for the right link. It really helped. As many others even I'm waiting for "Append to files in HDFS" Is there any thing which I can do to raise its priority. Does HADOOP Developer community is tracking any request counter for a particular feature to raise ones pr