I have tried 1.6, replaying log has been faster, but not enough. We have G bytes of logs, replaying these logs still takes us hours even days. This is frustrating, and has been the biggest concern for us to use it in a larger scale.
2015-06-01 15:32 GMT+08:00 Hari Shreedharan <[email protected]>: > 1.6 has been released. We were waiting for maven central to sync up. Now > that it is on central, I will post the update on the site tomorrow. > > > On Sunday, May 31, 2015, Shady Xu <[email protected]> wrote: > >> I noticed that Flume 1.6 has been released on Github but not the official >> website. I have compiled some of the modules from source myself (for other >> reasons), but I'm not sure compiling the whole project is a good idea. >> >> We have tons of data, every time we change the configurations, replaying >> log takes us way too many hours... >> >> 2015-04-17 12:38 GMT+08:00 Hari Shreedharan <[email protected]>: >> >>> Changes that went into Flume 1.6 should improve replay time. Flume 1.6 >>> will be out in a few days. >>> >>> >>> Thanks, >>> Hari >>> >>> On Thu, Apr 16, 2015 at 7:55 PM, Shady Xu <[email protected]> wrote: >>> >>>> Every time I restart Flume NG, it will try to replay the log and the >>>> process usually takes hours. During this time, Flume does not take any data >>>> from the source. >>>> >>>> So how can I make the replay faster? >>>> >>> >>> >> > > -- > > Thanks, > Hari > >
