Thank you Bejoy. Cheers! Manoj.
On Mon, Sep 10, 2012 at 1:36 PM, Bejoy Ks <bejoy.had...@gmail.com> wrote: > Hi Manoj > > From my limited knowledge on file appends in hdfs , i have seen more > recommendations to use sync() in the latest releases than using append(). > Let us wait for some commiter to authoritatively comment on 'the production > readiness of append()' . :) > > Regards > Bejoy KS > > > On Mon, Sep 10, 2012 at 11:03 AM, Manoj Babu <manoj...@gmail.com> wrote: > >> Thank you Bejoy. >> >> Does file append is production stable? >> >> >> Cheers! >> Manoj. >> >> >> >> On Sun, Sep 9, 2012 at 10:19 PM, Bejoy KS <bejoy.had...@gmail.com> wrote: >> >>> ** >>> Hi Manoj >>> >>> You can load daily logs into a individual directories in hdfs and >>> process them daily. Keep those results in hdfs or hbase or dbs etc. Every >>> day do the processing, get the results and aggregate the same with the >>> previously aggregated results till date. >>> >>> Regards >>> Bejoy KS >>> >>> Sent from handheld, please excuse typos. >>> ------------------------------ >>> *From: * Manoj Babu <manoj...@gmail.com> >>> *Date: *Sun, 9 Sep 2012 21:28:54 +0530 >>> *To: *<mapreduce-user@hadoop.apache.org> >>> *ReplyTo: * mapreduce-user@hadoop.apache.org >>> *Subject: *Reg: parsing all files & file append >>> >>> Hi All, >>> >>> I have two questions, providing info on it will be helpful. >>> >>> 1, I am using hadoop to analyze and to find top n search term metric's >>> from logs. >>> If any new log file is added to HDFS then again we are running the job >>> to find the metrics. >>> Daily we will be getting log files and we are parsing the whole file and >>> getting the metric's. >>> All the log file's are parsed daily to get the latest metric's is there >>> any way is there any way to avoid this? >>> >>> 2, Does file append is production stable? >>> >>> Cheers! >>> Manoj. >>> >>> >> >
