On 04/14/2015 02:51 PM, Cristian Falcas wrote:
Hi Rob,
I'm reading on host A logs from /dev/log.
Using LogstreamerInput, I'm guessing? If so, then they hostname *should* be
getting set by the LogstreamerInput, which means that if the hostname is
disappearing then it's probably inadvertently happening in the decoder.
No hostname is set here by
heka and there is none in the messages (a few of them have localhost as
the hostname, but the vast majority have nothing). Those logs are
shipped to host B. I was expecting that at least the TCPOutput plugin
will set the hostname for the messages, but this doesn’t seem to be the
case.
Not sure why you'd expect an output to set the hostname for outgoing messages.
Hostname is usually related to where a message originates, not the most recent
box a message came from.
I want to set the hostname after reading them from /dev/log, but I don't
know how to do that either. I'm using a sandbox decoder for this, if
that matters.
To be clear, you're using a SandboxDecoder with your LogstreamerInput on host
A? Is that a custom decoder, or one that Heka ships with? If it's custom, can
you paste the source code somewhere so I can take a peek?
-r
Thank you,
Cristian Falcas
On Wed, Apr 15, 2015 at 12:21 AM, Rob Miller <[email protected]
<mailto:[email protected]>> wrote:
On 04/09/2015 10:20 PM, Cristian Falcas wrote:
Thanks. So the hostname is not supposed to be set automatically, no?
You haven't given enough detail for me to know. A LogstreamerInput
that is reading a text log file does default to setting the local
hostname for each message, although you can override the hostname
that you want to use in the config. This may be (and often is)
overwritten by a decoder, however. A LogstreamerInput that is
loading a file full of protobuf encoded Heka messages will not set
the hostname, because it's presumed that the messages will already
have the hostname set. The former case is much more common than the
latter case, so generally a LogstreamerInput *will* set a hostname
by default.
A TcpInput behaves similarly. If it's receiving a stream of raw text
data, the text will end up as the message payload, and the hostname
will be set to the remote address of the TCP connection. If it's
receiving a stream of protobuf encoded messages, the messages
themselves will already have the hostname set. For TcpInputs, the
protobuf encoded case is the more common one, so generally TcpInputs
will not set a default hostname.
From your question, it's not clear to me which hostname (A or B)
you're expecting to see, nor where.
-r
On Thu, Apr 9, 2015 at 8:15 PM, Rob Miller <[email protected]
<mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>> wrote:
Are you doing any parsing at all of the data that you're
reading,
currently? Regardless, you can probably just add a
ScribbleDecoder
to set the Hostname field on all of the records coming in
through
that particular LogstreamerInput. If a SandboxDecoder is
already in
play, you can tweak that to set the Hostname field in
addition to
whatever other work it's already doing.
-r
On 04/09/2015 07:48 AM, Cristian Falcas wrote:
Hello,
I'm trying to send messages from host A to host B and I was
expecting
that the hostname will be set automatically.
I'm reading on host A from /dev/log and forward them to
host B
with TcpOutput.
Is this normal and I need to use a
"PayloadRegexDecoder" or a
sendbox
plugin in order to set this?
Thank you,
Cristian Falcas
___________________________________________________
Heka mailing list
[email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>
https://mail.mozilla.org/____listinfo/heka
<https://mail.mozilla.org/__listinfo/heka>
<https://mail.mozilla.org/__listinfo/heka
<https://mail.mozilla.org/listinfo/heka>>
_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
