[ANN] Hadoop Summit Call for Papers Now Open

Just a quick note to let everyone know that the Call for Papers for this year's Hadoop Summit is currently open. Please visit www.hadoopsummit.org for more information. The submission deadline for abstracts is February 22nd. This year's Hadoop Summit will be even bigger and better than past year

Re: What's the best practice of loading logs into hdfs while using hive to do log analytic?

Yes. You can use partitioned tables in hive to append in a new table without moving the data. For flume you can define small sinks, but you're right, the file in hdfs is closed and written when flume send the closing. Please note, the gzip codec has no marker inside so you have to wait till flu

Re: What's the best practice of loading logs into hdfs while using hive to do log analytic?

hi Bejoy and Alex, thank you for your advice. Actually I have look at Scribe first, and I have heard of Flume. I look at flume's user guide just now, and flume seems promising, as Bejoy said , the flume collector can dump data into hdfs when the collector buffer reaches a particular size of afte

Get race condition in hive with script_pipe.q ( the same as HIVE-1491)

Hi,All I got the same error message described in [HIVE-1491] ( https://issues.apache.org/jira/browse/HIVE-1491) when ran script_pipe.q on Open JDK. Will hive plan to resolve this race condition? Details: = [junit] Begin query: script_pipe.q [junit] Ended

Re: Error when Creating an UDF

Btw, welcome on the list, JC! :) -- Alexander Lorenz http://mapredit.blogspot.com On Feb 7, 2012, at 10:57 AM, Jean-Charles Thomas wrote: > Hi, > > thanks for the replies, I get it now to work, the jar file was not correctly > done. > Now it works like a charm. > > Regards, > > Jean-Char

Re: What's the best practice of loading logs into hdfs while using hive to do log analytic?

Hi, a first start with flume: http://mapredit.blogspot.com/2011/10/centralized-logfile-management-across.html Facebook's scribe could also be work for you. - Alex -- Alexander Lorenz http://mapredit.blogspot.com On Feb 7, 2012, at 11:03 AM, Xiaobin She wrote: > Hi all, > > Sorry if it is not

What's the best practice of loading logs into hdfs while using hive to do log analytic?

Hi all, Sorry if it is not appropriate to send one thread into two maillist. ** I'm tring to use hadoop and hive to do some log analytic jobs. Our system generate lots of logs every day, for example, it produce about 370GB logs(including lots of log files) yesterday, and every day the logs increa

AW: Error when Creating an UDF

Hi, thanks for the replies, I get it now to work, the jar file was not correctly done. Now it works like a charm. Regards, Jean-Charles Von: bejoy...@yahoo.com [mailto:bejoy...@yahoo.com] Gesendet: Montag, 6. Februar 2012 19:48 An: user@hive.apache.org Betreff: Re: Error when Creating an UDF