If you go to the jobtracker's web UI, it provides plenty of details about each job. Even with all the default settings of a typical hadoop/hive installation, 4 minutes for 2200 rows is extremely slow. It feels like there is some kind of problem but it is hard to guess what that could be. Digging through the web UI can tell you how much time is spent on map vs reduce. It can also provide some insight into the I/O operations.
Good luck! -Vijay On Mon, Mar 7, 2011 at 9:24 PM, abhishek pathak <forever_yours_a...@yahoo.co.in> wrote: > I suspected as such.My system is a Core2Duo,1.86 Ghz.I understand that > map-reduce is not instantaneous, just wanted to confirm that 2200 rows in 4 > minutes is indeeed not normal behaviour.Could you point me at some places > where i can get some info on how to tune this up? > Regards, > Abhishek > ________________________________ > From: Ajo Fod <ajo....@gmail.com> > To: user@hive.apache.org > Sent: Mon, 7 March, 2011 9:21:51 PM > Subject: Re: Hive too slow? > > In my experience, hive is not instantaneous like other DBs, but 4 minutes to > count 2200 rows seems unreasonable. > > For comparison my query of 169k rows one one computer with 4 cores running > 1Ghz (approx) took 20 seconds. > > Cheers, > Ajo. > > On Mon, Mar 7, 2011 at 1:19 AM, abhishek pathak > <forever_yours_a...@yahoo.co.in> wrote: >> >> Hi, >> I am a hive newbie.I just finished setting up hive on a cluster of two >> servers for my organisation.As a test drill, we operated some simple >> queries.It took the standard map-reduce algorithm around 4 minutes just to >> execute this query: >> count(1) from tablename; >> The answer returned was around 2200.Clearly, this is not a big number by >> hadoop standards.My question is whether this is a standard performance or is >> there some configuration that is not optimised?Will scaling up of data to >> say,50 times, produce any drastic slowness?I tried reading the documentation >> but was not clear on these issues, and i would like to have an idea before >> this setup starts working in a production environment. >> Thanks in advance, >> Regards, >> Abhishek Pathak >> >> > > >
