Hi all, I thought I'd give an update from the ElasticSearchSink perspective. I know that we're very keen to see a 1.4 release and will test RCs, etc. when they come out when priorities allow.
1. The patch to build with elasticsearch 0.90.1 is in FLUME-2049 and the review is here: https://reviews.apache.org/r/11581/ I think we really need that one in Flume because otherwise anybody who wants to use Flume with anything other than elasticsearch 0.19 (really old) will have to do a custom compile. 2. I mentioned issues with Flume logging via itself. I got those to a point where I thought I understood the problem and have raised work items either in Flume or with Log4j2 to document. These do not hold up Flume 1.4.0 3. Ideally, we'd do FLUME-2092 to document that the index name needs to match a Kibana setting. 4. It would probably be nice to get the patch from FLUME-2089 in (unexpected character sets cause a YAMLException). However, I think we need a unit test for this. Cheers, Edward On Wed, May 22, 2013 at 9:28 AM, Edward Sargisson <[email protected]> wrote: > Hi All, > +1/+1 for 1.4 and Mike. > > > I'm very keen to have a 1.4 for the environments I manage. There's a lot > of stuff I'm keen on in there. > > On my pre-1.4 list: > 1. compile with elasticsearch 0.90 > 2. figure out file channel state issue which is stopping Flume logging via > itself. > > 1. Currently we compile with es 0.19. If somebody wants to run es 0.20 > they have to recompile (es made an interface change that is source > compatible but requires a recompile). es 0.90 has been out for 2-ish weeks > so safe enough to change the compile to. I think I'll raise an empty Jira > to record this. > > 2. I haven't reported this because I haven't isolated it well enough. I'm > having issues with the 1.3.1 file channel which I'd like to resolve. > > Cheers, > Edward > > "Hi folks, > We have had over 100 commits since 1.3.1, and a bunch of new features and > improvements including a Thrift source, much improved ElasticSearch sink, > support for a new plugins directory and layout, compression support in the > avro sink/source, improved checkpointing in the file channel and more, plus > a lot of bug fixes. > > It seems to me that it's time to start thinking about cutting a 1.4 > release. I would be happy to volunteer to RM the release. Worth noting that > I will be unavailable for the next two weeks... but after that I'd be happy > to pick this up and run with it. That's also a decent amount of time for > people to get moving on patches and reviews for their favorite features, > bug fixes, etc. > > If this all sounds OK, I'd like to suggest targeting the last week of June > as a release date. If we can release in time for Hadoop Summit then that > would be pretty nice. Otherwise, if something comes up and we can't get the > release out that week, let's shoot for the first week of July at the > latest. > > Please let me know your thoughts. > > Regards, > Mike > > +1 for Flume 1.4 > +1 for Mike being RM. > > > Cheers, > Hari" >
