Sorry, I don't quite understand the question. There are no "steps" in
Heka. Heka's pipeline is the following:
- Raw data comes in via an Input plugin
- A Splitter plugin slices the raw data into discrete records
- A Decoder plugin maps the raw data to a Heka message structure;
MultiDecoder can be used to apply more than one Decoder to the same
input data
- Messages are handed to the router
- Router checks the message_matcher for all Filter and Output plugins,
delivers the message to any that are a match
- Optionally, Filter plugins might generate new messages that will be
dropped onto the router
- Output plugin invokes an Encoder to turn the message back into a raw
data stream
- Output plugin sends data out
If you explain what you're trying to accomplish at a high level, I could
probably point you in the right direction, but the current language that
you're using doesn't make much sense to me.
-r
On 10/06/2015 02:06 AM, Christian Kniep wrote:
Hey y'all,
on that topic ‘kinda replacing Logstash’.
I am not mistaken it’s possible to chain multiple steps together as one
would do in logstash, correct?
I use the [ProcStatDecoder] and pass it in a couple of steps towards the
[LogOutput] by stating the previous step in the ‘message_type’.
As far as I understood it (or hope it works) it will parse all the
config and afterwards create the correct workflow based on the matcher
and name of the steps.
Cheers
Christian
--
Christian Kniep |Release Engineer
www.gaikai.com <http://www.gaikai.com/>
On 06 Oct 2015, at 10:32, Robert Gardam
<[email protected]
<mailto:[email protected]>> wrote:
I managed to get it to work. I think the only way to make this work is
to use the multi-decoder as I still need all other syslog messages to be
processed in heka and passed onto es.
On 05/10/15 22:58, Rob Miller wrote:
On 10/5/15 9:59 AM, Robert Gardam wrote:
Hi All,
I'm just trying out moving from logstash and I'm wondering how does one
push one message from one decoder into another decoder after it has been
decoded the first time.
Essentially i'm decoding json rsyslog messages from a tcp input and then
I want to pass only one field into the nginx access log decoder.
Most of the work for the nginx access decoder is done in a module, so
I'd recommend just copying the nginx decoder code and adding your JSON
parsing right in the same decoder.
I can multi decode, but this only seems to decode json first but not
pass the json message into the second decoder.
MultiDecoder should work, albeit less efficiently. You have to set
`cascade_strategy = "all"` (the default is "first-wins") for your use
case, though; did you set that?
-r
_______________________________________________
Heka mailing list
[email protected] <mailto:[email protected]>
https://mail.mozilla.org/listinfo/heka
--
Robert Gardam
DevOps Engineer
Native Instruments GmbH
+49-30-611035-1775
www.native-instruments.com <http://www.native-instruments.com>
_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
