well, logger wraps it in a full syslog-formatted message, so the raw input has a lot more than just the text string.
i gisted an example: https://gist.github.com/nathwill/c57c99c737882c1ad993 On Wed, Aug 19, 2015 at 2:32 PM, Christian Kniep <[email protected]> wrote: > Bonus question, but I don’t want to stress it to much… > > What is different if I send the string via ‘logger’? > #### > logger -n 127.0.0.1 -P 5514 -T Test6 > #### > This only shows the closing TCP connection but now decoding/splitting… > #### > 2015/08/19 23:32:20 Decoder 'TcpInput-newline_decoder-127.0.0.1': stopped > #### > > Good n8 from Berlin > Christian > > On 19 Aug 2015, at 23:13, Nathan Williams <[email protected]> > wrote: > > i also noticed that that echo isn't actually inserting newlines, but > literal '\n' instead. fwiw, it works a bit better using printf instead. > > On Wed, Aug 19, 2015 at 2:10 PM, Christian Kniep <[email protected]> > wrote: > >> Cool, >> >> thanks guys… that seems to solve my issue! >> >> #### >> # cat /etc/heka/hekad.toml >> [hekad] >> maxprocs = 2 >> >> [TcpInput] >> address = ":5514" >> splitter = "newline_splitter" >> decoder = "better_decoder" >> >> [newline_splitter] >> type = "TokenSplitter" >> delimiter = '\n' >> >> [better_decoder] >> type = "ScribbleDecoder" >> [better_decoder.message_fields] >> Type = "MyType" >> >> [gross_decoder] >> type = "PayloadRegexDecoder" >> match_regex = "^(?P<Message>.+)" >> >> [PayloadEncoder] >> append_newlines = true >> >> [LogOutput] >> message_matcher = "TRUE" >> encoder = “PayloadEncoder" >> ##### >> >> Cheers >> Christian >> >> > On 19 Aug 2015, at 23:06, Rob Miller <[email protected]> wrote: >> > >> > What you're seeing is normal. When your client shuts down, the TCP >> connection closes, so Heka spins down the goroutines for handling that >> connection. Each connection gets its own decoder instance, so you're seeing >> the ProtobufDecoder for that connection exiting. >> > >> > Since TcpInput defaults to using ProtobufDecoder, you'll need to >> specify something to override the default. Ideally you'd do some actual >> parsing here, but if you don't need to do any parsing the most minimal >> performance impact would probably be to use a ScribbleDecoder to scribble >> some value onto each message. >> > >> > -r >> > >> > >> > On 08/19/2015 01:44 PM, Christian Kniep wrote: >> >> Hey y’all, >> >> >> >> after opening an issue on github I am still not sure how to create the >> >> TCP input to allow me sending arbitrary strings. >> >> https://github.com/mozilla-services/heka/issues/1683 >> >> >> >> I am creating a docker image out of it: >> >> https://github.com/qnib/docker-heka >> >> >> >> My heka.toml looks as follows: >> >> ############## >> >> [hekad] >> >> maxprocs = 2 >> >> >> >> [TcpInput] >> >> address = ":5514" >> >> splitter = "newline_splitter" >> >> >> >> [newline_splitter] >> >> type = "RegexSplitter" >> >> delimiter = '\n' >> >> delimiter_eol = true >> >> >> >> [split_on_space] >> >> type = "TokenSplitter" >> >> delimiter = " " >> >> >> >> [PayloadEncoder] >> >> append_newlines = true >> >> >> >> [LogOutput] >> >> message_matcher = "TRUE" >> >> encoder = “PayloadEncoder" >> >> ############## >> >> >> >> If I send a string the TCPInput stops: >> >> ############## >> >> [root@heka tmp]# /opt/heka/bin/hekad -config=/etc/heka/hekad.toml >> >> 2015/08/19 22:38:14 Pre-loading: [LogOutput] >> >> 2015/08/19 22:38:14 Pre-loading: [TcpInput] >> >> 2015/08/19 22:38:14 Pre-loading: [newline_splitter] >> >> 2015/08/19 22:38:14 Pre-loading: [split_on_space] >> >> 2015/08/19 22:38:14 Pre-loading: [PayloadEncoder] >> >> 2015/08/19 22:38:14 Pre-loading: [HekaFramingSplitter] >> >> 2015/08/19 22:38:14 Loading: [HekaFramingSplitter] >> >> 2015/08/19 22:38:14 Pre-loading: [NullSplitter] >> >> 2015/08/19 22:38:14 Loading: [NullSplitter] >> >> 2015/08/19 22:38:14 Pre-loading: [ProtobufDecoder] >> >> 2015/08/19 22:38:14 Loading: [ProtobufDecoder] >> >> 2015/08/19 22:38:14 Pre-loading: [ProtobufEncoder] >> >> 2015/08/19 22:38:14 Loading: [ProtobufEncoder] >> >> 2015/08/19 22:38:14 Pre-loading: [TokenSplitter] >> >> 2015/08/19 22:38:14 Loading: [TokenSplitter] >> >> 2015/08/19 22:38:14 Loading: [PayloadEncoder] >> >> 2015/08/19 22:38:14 Loading: [newline_splitter] >> >> 2015/08/19 22:38:14 Loading: [split_on_space] >> >> 2015/08/19 22:38:14 Loading: [TcpInput] >> >> 2015/08/19 22:38:14 Loading: [LogOutput] >> >> 2015/08/19 22:38:14 Starting hekad... >> >> 2015/08/19 22:38:14 Output started: LogOutput >> >> 2015/08/19 22:38:14 MessageRouter started. >> >> 2015/08/19 22:38:14 Input started: TcpInput >> >> 2015/08/19 22:38:18 Decoder 'TcpInput-ProtobufDecoder-::1': stopped >> >> 2015/08/19 22:38:18 >> >> ############## >> >> While sending something like this: >> >> # echo "huhu\n"|nc -w1 localhost 5514 >> >> >> >> Cheers >> >> Christian >> >> >> >> >> >> _______________________________________________ >> >> Heka mailing list >> >> [email protected] >> >> https://mail.mozilla.org/listinfo/heka >> >> >> >> >> _______________________________________________ >> Heka mailing list >> [email protected] >> https://mail.mozilla.org/listinfo/heka >> >> > _______________________________________________ > Heka mailing list > [email protected] > https://mail.mozilla.org/listinfo/heka > > >
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
