Re: How-to use DFSClient's BlockReader from Java

2012-01-10 Thread Todd Lipcon
On Tue, Jan 10, 2012 at 4:32 AM, David Pavlis david.pav...@javlin.eu wrote: Hi Todd, Understand, I will re-think the MR approach. Nonetheless I like the idea with getting the block id and accessing the locally stored file directly You really don't want to do this - what happens when the

HDFS Problems After Programmatic Access

2012-01-10 Thread Apurv Verma
When I run this command, I get an error message. This problem started first when I first accessed HDFS programatically. I am on a single node cluster in ubuntu. hadoop dfs -copyFromLocal /home/apurv/bigdataLocal/macbeth.txt /home/apurv/bigdataRemote/ 12/01/11 08:07:49 WARN hdfs.DFSClient:

RE: HDFS Problems After Programmatic Access

2012-01-10 Thread Uma Maheswara Rao G
Looks namenode is not giving the nodes. Can you please check Datanode is running properly? From: Apurv Verma [dapu...@gmail.com] Sent: Wednesday, January 11, 2012 8:15 AM To: hdfs-user@hadoop.apache.org Subject: HDFS Problems After Programmatic Access When I

Re: HDFS Problems After Programmatic Access

2012-01-10 Thread Harsh J
Apurv, Take a look at the FAQ: http://wiki.apache.org/hadoop/FAQ#What_does_.22file_could_only_be_replicated_to_0_nodes.2C_instead_of_1.22_mean.3F On 11-Jan-2012, at 8:15 AM, Apurv Verma wrote: When I run this command, I get an error message. This problem started first when I first accessed

Fwd: HDFS Federation Exception

2012-01-10 Thread Praveen Sripati
Hi, I am trying to setup a HDFS federation and getting the below error. Also, pasted the core-site.xml and hdfs-site.xml at the bottom of the mail. Did I miss something in the configuration files? 2012-01-11 12:12:15,759 ERROR namenode.NameNode (NameNode.java:main(803)) - Exception in namenode

Custom web app

2012-01-10 Thread Michael Lok
Hi folks, I was wondering if it's possible to develop a custom front end which allows monitoring/management of jobs/tasks and hdfs. From what i can see from the built in webapp, all JSPs retrieve the relevant NameNode/TaskTracker objects from the application instance; which is set during

Re : Custom web app

2012-01-10 Thread Franck Besnard
Hello Yes this is quite trivial and for example ibm did that in their big insights flavor of hadoop The hadoop alrzady has quite a lot of instrumentation regarding monitoring and you can extend the embedded jetty to add more features. Ideally this webapp should nit be bundled within the nodes

Re: Re : Custom web app

2012-01-10 Thread Michael Lok
Hi Franck, Thanks for the info. However, I'm looking at decoupling this app from the embedded Jetty in Hadoop. Ideally, it should be deployed in a separate server as the NameNode. Is this possible? On Wed, Jan 11, 2012 at 3:23 PM, Franck Besnard besn0...@yahoo.com wrote: Hello Yes this