Hadoop dfs can't allocate memory with enough hard disk space when data gets huge

2009-10-18 Thread Kunsheng Chen
I and running a hadoop program to perform MapReduce work on files inside a folder. My program is basically doing Map and Reduce work, each line of any file is a pair of string, and the result is a string associate with occurence inside all files. The program works fine until the number of file

Seattle / NW Hadoop, Lucene, Apache "Cloud Stack" Meetup, Wed Oct 28 6:45pm

2009-10-18 Thread Bradford Stephens
Greetings, (You're receiving this e-mail because you're on a DL or I think you'd be interested) It's time for another Hadoop/Lucene/Apache "Cloud" stack meetup! This month it'll be on Wednesday, the 28th, at 6:45 pm. A *huge* thanks for everyone who showed up last month, and to Facebook for send

proper way to configure classes required by mapper job

2009-10-18 Thread yz5od2
Hi, What is the preferred method to distribute the classes (in various Jars) to my Hadoop instances, that are required by my Mapper class? thanks!

Re: Project ideas !

2009-10-18 Thread tim robertson
> Has any publications been done in this area? (The spatial processing on > hadoop) > G Sudha Sadasivam I saw Chris who does Cascading tweet about someone building RTree indexes recently in hadoop and wanted to follow up with him about who was doing that. For my part I very hastily wrote a blog

HTTP secure access

2009-10-18 Thread Bogdan M. Maryniuk
Hi! Well, I have a kinda simple question, but I can not spot a proper doc for it: how you, guys, restricting access to the web interfaces? :-) It is somewhere in jetty or there is no feature like this? I am OK with a simple basic authentication, but I don't really like when others are staring at l

Re: Datanode Throwing NoRouteToHostException

2009-10-18 Thread Last-chance Architect
Do you by chance have any firewall configuration blocking those ports (though I'd imagine the error would be different)? Out of curiousity, does the command 'hostname' return the correct name on both boxes? Lajos lazikid wrote: I need some help with setting up a Hadoop cluster. The datanod

Re: Datanode Throwing NoRouteToHostException

2009-10-18 Thread Huy Phan
1. Can you try to netstat on your namenode machine to make sure that port 54310 is opened ? 2. If it's opened already, check if iptables 's blocking your connection or not. Best, Huy Phan lazikid wrote: I need some help with setting up a Hadoop cluster. The datanode on the slave is not comin

Datanode Throwing NoRouteToHostException

2009-10-18 Thread lazikid
I need some help with setting up a Hadoop cluster. The datanode on the slave is not coming up throwing java.net.NoRouteToHostException: No route to host. Please see the details below. I have a centos master and a fedora slave. Both have Java 6 and Hadoop 0.20.1. I instaleed hadoop under /opt in b