Re: [heka] heka and elasticsearch bulk upload

2016-04-29 Thread Ramin Ali Dousti
ng-as-within-30s-while-bulk-indexing-documents/42305/3 > > > 2016-04-28 14:54 GMT+02:00 Ramin Ali Dousti : > >> Hi, >> >> The ES version is "2.2.0". >> >> This is the HTTP response. Look for the status 503 in the payload: >> >> T 127.0.0

Re: [heka] heka and elasticsearch bulk upload

2016-04-28 Thread Ramin Ali Dousti
- "_id": "AVRTguzBSCHabxnyv4qq", - "_version": 1, - "_shards": { - "total": 3, - "successful": 3, - "failed": 0 }, - "status"

[heka] heka and elasticsearch bulk upload

2016-04-26 Thread Ramin Ali Dousti
Hi, I have an ES output that bulk uploads to a cluster. The HTTP status code is 200 OK but the reply payload says that it failed the upload for a few of the items. But heka doesn't seem to care about the failed items. I looked at the code and it says: https://github.com/mozilla-services/heka/blob

Re: [heka] heka wedged

2016-03-30 Thread Ramin Ali Dousti
o a file and monitor that the file is growing, or send > them to a downstream listener that will notice if the heartbeats stop > coming. > > -r > > > On 03/30/2016 11:13 AM, Ramin Ali Dousti wrote: > >> Hi Timur, >> >> 1) wedged: >> In my

Re: [heka] heka wedged

2016-03-30 Thread Ramin Ali Dousti
bug_encoder" > message_matcher="TRUE" > > This makes heka dump all messages to stdout. If you store that in logs > hope you rotate them based on size reached. > > > Best regards, > Timur > > On Wed, Mar 30, 2016 at 2:22 AM, Ramin Ali Dousti > wrote: >

[heka] heka wedged

2016-03-29 Thread Ramin Ali Dousti
Hi, I have a nagging question. I read in the documentations and I also experienced it myself that hekad might become "wedged" and there would be no activity within the system while the process seems up from the outside. My question is what are the conditions that this might happen? The reason I as

Re: [heka] heka monitoring

2016-02-04 Thread Ramin Ali Dousti
a-3, though, > which leads us to turtles all the way down. Or maybe you trust that if > heka-3 is set up to do absolutely *nothing* other than listen for heka-2's > heartbeats, then process level monitoring is good enough, since the chance > of it getting wedged when doing absol

Re: [heka] heka monitoring

2016-02-01 Thread Ramin Ali Dousti
te: > On 2016-01-14 14:59, Ramin Ali Dousti wrote: > >> Hi, >> >> What is the best way of monitoring the proper working of a heka >> instance? Let me give a concrete example: I have heka-1 that has log >> files as input and TCP outputs to another heka instance. T

[heka] heka monitoring

2016-01-14 Thread Ramin Ali Dousti
Hi, What is the best way of monitoring the proper working of a heka instance? Let me give a concrete example: I have heka-1 that has log files as input and TCP outputs to another heka instance. The second one would receive the stream from the first one and publishes to elastic search as well as in

[heka] Geo IP

2016-01-06 Thread Ramin Ali Dousti
Hi, I have an array of heka servers that would parse my application logs and use TcpOutput with ProtobufEncoder to send the produced Heka Messages to a centralized heka daemon. The message would have a field that represents an IP address. On this centralized heka daemon I'd like to do a goe IP loo