Hi Igor, I am not sure if Hadoop is designed for realtime requests. I have a feeling that you are trying to use Hadoop in a way that it isnot designed for. From my experience, Hadoop cluster will be much slower than "local" hadoop mode when processing smaller dataset, because there is always extra overhead of task and job management in cluster mode.
Praveen ________________________________________ From: ext Igor Bubkin [igb...@gmail.com] Sent: Tuesday, February 01, 2011 3:19 AM To: common-iss...@hadoop.apache.org Cc: common-user@hadoop.apache.org Subject: How to speed up of Map/Reduce job? Hello everybody I have a problem. I installed Hadoop on 2-nodes cluster and run Wordcount example. It takes about 20 sec for processing of 1,5MB text file. We want to use Map/Reduce in real time (interactive: by user's requests). User can't wait for his request 20 sec. This is too long. Is it possible to reduce time of Map/Reduce job? Or may be I misunderstand something? BR, Igor Babkin, Mifors.com
