Hi, For "near" real time performance you may try Hbase. I had read about Streamy doing this, and their hadoop-world-nyc ppt is available on their blog: http://devblog.streamy.com/2009/07/24/streamy-hadoop-summit-hbase-goes-realtime/
Amogh On 11/25/09 1:31 AM, "onur ascigil" <onurasci...@hotmail.com> wrote: Thanks for your reply! I am running 0.20.1 in pseudo-distributed mode in Ubuntu. I want to run interactive jobs with Hadoop and trying to see if Hadoop is suitable for that purpose or not. I wonder if there is anybody using Hadoop for interactive jobs where given a query, output is returned within an acceptable amount of time. Is Hadoop meant to be used only for batch processing? > From: rekha...@yahoo-inc.com > To: common-user@hadoop.apache.org > Date: Mon, 23 Nov 2009 22:30:43 -0800 > Subject: Re: Hadoop Performance > > Hi, > > Not sure about your hadoop version, and havent done much on single m/c setup > myself. However there is a IPC improvement bug filed @ > https://issues.apache.org/jira/browse/HADOOP-2864.Thanks! > > On 11/24/09 11:22 AM, "onur ascigil" <onurasci...@hotmail.com> wrote: > > > > I am running Hadoop on a single machine and have some questions about its > performance. > I have a simple java program that runs breadth first search on a graph > with 5 nodes. It involves several map-reduce iterations. > > I observed that, Hadoop takes too long to produce > results on such a simple job. So I attached a java profiler to my mapreduce > job > (runJar) to see what is going on. The java profiler reported several IPC > connections to ports 54310 and 54311. Each of these IPCs to Jobtracker and > HDFS takes around 10 seconds! > > First of all why are these IPCs take this long? > And I am wondering if there is anyway to improve > the performance of these IPC calls. Does Hadoop > have such a large fixed-cost ? > > I would really appreciate any comments or suggestions. > Thanks in advance, > Onur > > _________________________________________________________________ > Windows Live Hotmail: Your friends can get your Facebook updates, right from > Hotmail®. > http://www.microsoft.com/middleeast/windows/windowslive/see-it-in-action/social-network-basics.aspx?ocid=PID23461::T:WLMTAGL:ON:WL:en-xm:SI_SB_4:092009 > _________________________________________________________________ Windows Live Hotmail: Your friends can get your Facebook updates, right from Hotmail®. http://www.microsoft.com/middleeast/windows/windowslive/see-it-in-action/social-network-basics.aspx?ocid=PID23461::T:WLMTAGL:ON:WL:en-xm:SI_SB_4:092009
