Is it possible to share the method and class name? It would be somewhere in client package I guess. I shall do the rest, else would need to re-invent the wheel :)
On Wed, Apr 1, 2015 at 3:30 AM, Otis Gospodnetic <[email protected]> wrote: > Hi, > > Unfortunately don't have a patch, though I think this is just 1 method that > needs to be added. Don't think ELB provides the info needed, so one needs > to just have timeout. > > Otis > -- > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > Solr & Elasticsearch Support * http://sematext.com/ > > > On Tue, Mar 31, 2015 at 4:25 PM, Hari Shreedharan <[email protected] >> wrote: > >> I am not very familiar with ELB, but if you do have a patch that will make >> Flume work better - I’ll be glad to review it. I think the HDFS, HBase, >> Morphline, Solr, Thrift, and Avro sinks will need it (similar logic, but >> different impl). >> >> >> >> >> Thanks, Hari >> >> On Tue, Mar 31, 2015 at 9:27 AM, Ashish <[email protected]> wrote: >> >> > Does ELB provide any kind of status code when it rejects the request >> > or it just doesn't send any response? >> > Perhaps adding a JIRA would be great. Doesn't ELB times out connection >> > which are Idle for more than 60 sec? >> > thanks >> > ashish >> > On Tue, Mar 31, 2015 at 7:50 PM, Otis Gospodnetic >> > <[email protected]> wrote: >> >> Hi, >> >> >> >> I don't have a very good bug report, but I want to share one problem >> with >> >> Flume we observed the other day. Specifically, with Elasticsearch Sink >> and >> >> AWS ELB, but others may have this problem, too. >> >> >> >> In short, ELB has this notion of a Spillover where it can reject >> requests >> >> when it can't handle the load. Unfortunately, it doesn't actually >> signal >> >> this by closing the connection, so Flume's connection remains open, I >> >> believe in some java.net....... readSocket(....) method. >> >> >> >> To work around this we had to add a socket timeout in our custom >> version of >> >> Flume's Elasticsearch Sink. >> >> >> >> I think it's worth fixing this in Flume and checking if other Sinks need >> >> this sort of timeout. >> >> >> >> HTH >> >> >> >> Otis >> >> -- >> >> Monitoring * Alerting * Anomaly Detection * Centralized Log Management >> >> Solr & Elasticsearch Support * http://sematext.com/ >> > -- >> > thanks >> > ashish >> > Blog: http://www.ashishpaliwal.com/blog >> > My Photo Galleries: http://www.pbase.com/ashishpaliwal >> -- thanks ashish Blog: http://www.ashishpaliwal.com/blog My Photo Galleries: http://www.pbase.com/ashishpaliwal
