I reverted back to the old build and the same query is working fine now. How do I find out the SVN revision of the old build?
Saurabh. On Wed, Dec 30, 2009 at 12:24 PM, Saurabh Nanda <[email protected]>wrote: > The rate at which "Map input bytes" and "Map input records" is growing is > extremely slow. Is something wrong with the HDFS configuration? But, it was > working perfectly fine with the previous Hive version. > > Saurabh. > > > On Wed, Dec 30, 2009 at 12:20 PM, Saurabh Nanda <[email protected]>wrote: > >> Attached are the task logs of one of the tasks. >> >> Saurabh. >> >> >> On Wed, Dec 30, 2009 at 12:16 PM, Zheng Shao <[email protected]> wrote: >> >>> This should be compiled into a single map-only job. >>> Can you take a look at the progress and the task logs of the job? >>> >>> We are not aware of any changes that might cause this problem. >>> >>> Zheng >>> >>> On Tue, Dec 29, 2009 at 10:35 PM, Saurabh Nanda <[email protected]> >>> wrote: >>> > Picking up data from the 'raw' table, filtering the unwanted lines and >>> > inserting into 'raw_compressed' table which is stored as sequencefile: >>> > >>> > insert overwrite table raw_compressed partition(dt='2009-04-01') select >>> line >>> > from raw where dt='2009-04-01' and lower(line) rlike '.*get >>> > .*/confirmation.*http.*' and not lower(line) rlike >>> > >>> '(/images.*?|/styles.*?|/javascripts.*?|/adserver.*?|.*?favicon.*?|/includes/thwarte-logo.html.*)'; >>> > >>> > Saurabh. >>> > >>> > On Wed, Dec 30, 2009 at 11:59 AM, Zheng Shao <[email protected]> wrote: >>> >> >>> >> What is the import query? Do you mean "load data"? >>> >> Can you give an example? >>> >> >>> >> Zheng >>> >> >>> >> On Tue, Dec 29, 2009 at 10:22 PM, Saurabh Nanda < >>> [email protected]> >>> >> wrote: >>> >> > Also has something changed drastically in Hive over the last 2-3 >>> months? >>> >> > A >>> >> > simply import query seems to be taking forever now! >>> >> > >>> >> > Saurabh. >>> >> > >>> >> > On Wed, Dec 30, 2009 at 11:48 AM, Saurabh Nanda < >>> [email protected]> >>> >> > wrote: >>> >> >> >>> >> >> I'm taking a look at the HDFS directories through the web interface >>> and >>> >> >> I >>> >> >> can see only 5 files there, not 6. I tried creating the partition >>> using >>> >> >> the >>> >> >> ADD PARTITION command. After that all 6 files get imported >>> >> >> successfully. >>> >> >> >>> >> >> Saurabh. >>> >> >> >>> >> >> On Wed, Dec 30, 2009 at 11:39 AM, Zheng Shao <[email protected]> >>> wrote: >>> >> >>> >>> >> >>> Can you list the HDFS directories? Are the files in the >>> corresponding >>> >> >>> directories yet? >>> >> >>> >>> >> >>> >>> >> >>> Zheng >>> >> >>> >>> >> >>> On Tue, Dec 29, 2009 at 9:57 PM, Saurabh Nanda >>> >> >>> <[email protected]> >>> >> >>> wrote: >>> >> >>> > Hi, >>> >> >>> > >>> >> >>> > I'm revisiting Hive after a long hiatus, so I may not be aware >>> of >>> >> >>> > any >>> >> >>> > new >>> >> >>> > developments. I had written a script some time back to import >>> >> >>> > webserver >>> >> >>> > logs >>> >> >>> > for a day into a new partition. The same script now running on >>> the >>> >> >>> > latest >>> >> >>> > version of Hive (r894548 compiled off trunk) seems to be >>> >> >>> > misbehaving. >>> >> >>> > >>> >> >>> > I'm importing about 6 files into each partition. However, after >>> the >>> >> >>> > script >>> >> >>> > ends, only 5 files show up in each partition. Do I need to >>> >> >>> > explicitly >>> >> >>> > issue >>> >> >>> > the ADD PARTITION command before loading data? Isn't the >>> partition >>> >> >>> > implicitly created? >>> >> >>> > >>> >> >>> > Saurabh. >>> >> >>> > -- >>> >> >>> > http://nandz.blogspot.com >>> >> >>> > http://foodieforlife.blogspot.com >>> >> >>> > >>> >> >>> >>> >> >>> >>> >> >>> >>> >> >>> -- >>> >> >>> Yours, >>> >> >>> Zheng >>> >> >> >>> >> >> >>> >> >> >>> >> >> -- >>> >> >> http://nandz.blogspot.com >>> >> >> http://foodieforlife.blogspot.com >>> >> > >>> >> > >>> >> > >>> >> > -- >>> >> > http://nandz.blogspot.com >>> >> > http://foodieforlife.blogspot.com >>> >> > >>> >> >>> >> >>> >> >>> >> -- >>> >> Yours, >>> >> Zheng >>> > >>> > >>> > >>> > -- >>> > http://nandz.blogspot.com >>> > http://foodieforlife.blogspot.com >>> > >>> >>> >>> >>> -- >>> Yours, >>> Zheng >>> >> >> >> >> -- >> http://nandz.blogspot.com >> http://foodieforlife.blogspot.com >> > > > > -- > http://nandz.blogspot.com > http://foodieforlife.blogspot.com > -- http://nandz.blogspot.com http://foodieforlife.blogspot.com
