We have been using elasticsearch-hadoop to write to ElasticSearch from Storm. It looks good so far. Any thoughts on adopting ElasticSearch Hadoop as part of Storm external rather than trying to write and maintain our own bolt in Storm Project.
Any thoughts? On Tue, Mar 22, 2016 at 2:19 PM, Aaron.Dossett <aaron.doss...@target.com> wrote: > No, we hadn’t looked at that, but once we are streaming into elastic at > scale (not there yet) I would be interested to compare. > > From: Tech Id <tech.login....@gmail.com> > Reply-To: "user@storm.apache.org" <user@storm.apache.org> > Date: Monday, March 21, 2016 at 1:21 PM > To: "user@storm.apache.org" <user@storm.apache.org> > Subject: Re: external/storm-elasticsearch - upgrade requested > > Thanks Aaron, > > Did you have a chance to compare the elasticsearch-hadoop ( > https://www.elastic.co/guide/en/elasticsearch/hadoop/current/storm.html) > with the storm-elasticsearch ( > https://github.com/apache/storm/tree/master/external/storm-elasticsearch) > ? > > Former uses REST client while the latter uses TransportClient. > > It would be interesting to know some performance numbers between them. > > Thanks ! > > On Mon, Mar 21, 2016 at 5:40 AM, Aaron.Dossett <aaron.doss...@target.com> > wrote: > >> Hi, >> >> The best way to make feature requests like this is via the Apache Jira. ( >> https://issues.apache.org/jira/browse/STORM/) >> >> As it happens, my team (at Target) has adapted the elastic search bolt >> for 2.X and we are using it in production. We opened a JIRA ( >> https://issues.apache.org/jira/browse/STORM-1475) to track that and >> contribute our change back to the project. We’re cleaning a couple of >> things up right now, but I’m hopeful we will contribute it back soon. >> >> Your second request could be a nice feature enhancement, can you open a >> JIRA for it? In my experience, these external bolts are a really nice way >> to get involved with the project as well. You could try adding a couple of >> features and contributing those. >> >> Thanks! >> >> From: Tech Id <tech.login....@gmail.com> >> Reply-To: "user@storm.apache.org" <user@storm.apache.org> >> Date: Friday, March 18, 2016 at 9:45 PM >> To: "user@storm.apache.org" <user@storm.apache.org> >> Subject: external/storm-elasticsearch - upgrade requested >> >> Hi, >> >> I see that the version of elastic-search used >> in external/storm-elasticsearch/pom.xml is quite old (1.6.0) while the >> latest elastic-search is 2.2.0 >> >> 2.x version is not compatible with 1.x version of elastic-search and so I >> request you to upgrade. >> >> Also, current storm-elasticsearch uses TransportClient of elastic-search >> but does not expose many of the useful options in the TransportClient (like >> routing). >> >> Request you to expose them to the users too. >> >> >> Thanks >> Tid >> > >
