I should also specify that when testing I have an explicit type declaration
in the section.
Eli
On Mon, Feb 8, 2016 at 1:39 PM Eli Flesher <efles...@lyft.com> wrote:
> I didn’t realize that the identifier at the top of the toml stanza was
> anything other than the type declaration. Doing some testing it looks like
> it’s a name but that it can’t include either ‘.’ or ‘/‘ or it will be
> interpreted as the plugin type.
>
> Is there any documentation or guidance for naming projects that will have
> a large number of these type of namespaces?
>
>
> Eli
>
> On Mon, Feb 8, 2016 at 1:11 PM Rob Miller <rmil...@mozilla.com> wrote:
>
>> With no differentiator the Logger value will be the name you give the
>> LogstreamerInput plugin, which has to be unique. The only way a single
>> LogstreamerInput can generate multiple streams is by specifying a
>> differentiator, in which case the Logger value will get the
>> differentiator's name for each stream.
>>
>> The only possible issue I can see is that if you specify the same
>> differentiator name structure in two different LogstreamerInput
>> configurations, Heka won't notice and you'll get name collision. So don't
>> do that, and you should be fine.
>>
>> -r
>>
>>
>> On 02/08/2016 01:04 PM, Eli Flesher wrote:
>> > Yeah, exactly right, I just need a unique identifier to the stream.
>> >
>> > Right now, I’m using the Logger in a way similar to this:
>> >
>> > record := LogAgentRecord{
>> > ...
>> > Path: pack.Message.Logger,
>> > Events: payload,
>> > Datetime: ts,
>> > ...
>> > }
>> >
>> > However, what get’s encoded when that Struct is marshaled to json is:
>> >
>> > "path":”LogstreamerInput"
>> >
>> > However, I don’t have a differentiator specified nor do I have more than
>> > one LogStreamerInput configured while testing. Let me try both of these
>> > and I’ll update the thread.
>> >
>> > Any other suggestions would be appreciated as well.
>> >
>> > Thanks,
>> >
>> >
>> > Eli
>> >
>> > On Mon, Feb 8, 2016 at 12:25 PM Rob Miller <rmil...@mozilla.com
>> > <mailto:rmil...@mozilla.com>> wrote:
>> >
>> > Hrm. File names are tricky, b/c the LogstreamerInput usually thinks
>> in
>> > terms of log *streams*, not files, so the name of the file that a
>> > particular record came from isn't readily available.
>> >
>> > I suspect that you don't actually need the file, though, but that
>> some
>> > unique identifier for your stream is actually enough. If that's the
>> > case, you should check out the `Logger` value on your generated
>> > messages. That will contain your differentiator value, if you have a
>> > differentiator specified. If not, it should at least be unique for
>> each
>> > LogstreamerInput in use.
>> >
>> > If that's not good enough, you might have to do something yourself
>> in
>> > the decoding layer. Hopefully Logger will suffice, though.
>> >
>> > -r
>> >
>> >
>> > On 02/08/2016 11:33 AM, Eli Flesher wrote:
>> > > Hello Everyone,
>> > >
>> > > We’re exploring using Heka internally as a daemon to push up
>> > > periodically written files and submit changes to an API for
>> > consumption.
>> > >
>> > > As part of this project, I’ve written an Encoder that formats
>> > messages
>> > > from several LogStreamerInputs into our message format before
>> > going to
>> > > an Output plugin that submits the messages.
>> > >
>> > > I’ve run into one problem. I need to know the file name being
>> > monitored
>> > > in order to determine parts of the message body. Looking at the
>> > > LogStreamerInput code, I can’t see a way that this is exposed,
>> > but I’m
>> > > also very new to Go.
>> > >
>> > > Can anyone guide me on how I might accomplish this?
>> > >
>> > >
>> > > Eli
>> > > --
>> > > —
>> > > *Elijah Flesher* | *Lyft* <http://lyft.me/> | /Software
>> Engineer/
>> > > 206.661.4697 | @eliflesher
>> > >
>> > >
>> > > _______________________________________________
>> > > Heka mailing list
>> > > Heka@mozilla.org <mailto:Heka@mozilla.org>
>> > > https://mail.mozilla.org/listinfo/heka
>> > >
>> >
>> > --
>> > —
>> > *Elijah Flesher* | *Lyft* <http://lyft.me/> | /Software Engineer/
>> > 206.661.4697 | @eliflesher
>> >
>> >
>> > _______________________________________________
>> > Heka mailing list
>> > Heka@mozilla.org
>> > https://mail.mozilla.org/listinfo/heka
>> >
>>
>> --
> —
> *Elijah Flesher* | *Lyft* <http://lyft.me/> | *Software Engineer*
> 206.661.4697 | @eliflesher
>
--
—
*Elijah Flesher* | *Lyft* <http://lyft.me/> | *Software Engineer*
206.661.4697 | @eliflesher
_______________________________________________
Heka mailing list
Heka@mozilla.org
https://mail.mozilla.org/listinfo/heka
