Re: [graylog2] Sidecar: When would you need more than one tag in a configuration?

[Werner vd Merwe]

Great point Marius!

Your explanation makes sense as well, I am beginning to better understand the 
intent. Great product and awesome idea, this final touches is the exciting last 
steps leading up to version 1.0!

I see this similarly to puppet and roles manifests. Having a tag per computer 
role and having the role made up of profile tags rather than, as you mentioned, 
the confusion of a mega bucket with a large amount of small puzzle pieces. 

Looking forward to spend some mind time on this when I am back in the country 
next month!


> On 19/07/2016, at 11:45, Marius Sturm  wrote:
> 
> Hi,
> thanks for the feedback, that's very helpful for us. Otherwise we never know 
> if the concepts are understood by the users or not.
> The current implementation is merging all configurations that are fetched 
> throug the provided tags. So let's say that your sidecar starts with three 
> tags, internally it's
> fetching all three configurations and merge them together to a single new 
> one. So you end up with a configuration that includes all 
> outputs/inputs/snippets from all three configurations. That said, the only 
> limitation is in the web interface. Currently you can't use an output from 
> one configuration with an input from another configuration.
> But that could be changed relatively easily.
> 
> Our goal is to find a solution that is intuitive and scales at the same time. 
> When you think of not having 1-5 configurations but maybe a few dozens. 
> Putting all inputs/output/snippets into one big bucket would be pretty 
> confusing imo. Losing the overview means you never know what will actually be 
> configured on the target system.
> 
> So I am very open to better ideas but they need to solve the problem for 
> bigger and smaller setups. Maybe there is some holy grail we missed till now.
> 
> Cheers,
> Marius
> 
> 
> 
>> On 18 July 2016 at 19:02, 123Dev  wrote:
>> Agreed, the tag is confusing to us too. 
>> On Graylog, if I have 3 configurations.
>> Config1 - tag1
>> Config2 - tag2
>> Config3 - tag3
>> On the collector side, I was wrongly expecting that if I set tag1 and tag2, 
>> the client would get both configurations.
>> But that didn't work
>> 
>> Because each configuration to be configured needs its own output, and the 
>> generated nxlog.conf did not get the two outputs or the auto-generated two 
>> routes.
>> 
>> I think Inputs, Outputs, Snippets and even routes (which cannot be defined 
>> now) need to be decoupled from a configuration and be allowed to be defined 
>> independently each in its own collections (similar to how the rules are done 
>> in pipelines) for convenience (you don't want to repeat for each config) and 
>> functionality (see below the nxlog internal example)
>> Then a configuration can pull one or many of the above components from the 
>> collections, and then applied to a collector.
>> 
>> If we don't decouple, then if we want to have nxlog internal logs logged to 
>> Graylog, we would need to define something like this in Snippets for each 
>> config (repeated and modified to match the output id)
>> 
>> Module  im_internal
>> 
>> 
>> 
>>   Path internal => 977ad164136aa0330cf2b422
>> 
>> 
>> 
>> and if the client has multiple tags, would it get multiple copies? single 
>> copy? which output? which route?
>> 
>> Sorry if we're understanding this thing totally wrong.
>> 
>> Thanks
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Graylog Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to graylog2+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/graylog2/0eb49603-18b4-4e1e-98f7-02ccfd35ef08%40googlegroups.com.
>> 
>> For more options, visit https://groups.google.com/d/optout.
> 
> 
> 
> -- 
> Developer
> 
> Tel.: +49 (0)40 609 452 077
> Fax.: +49 (0)40 609 452 078
> 
> TORCH GmbH - A Graylog Company
> Poolstraße 21
> 20335 Hamburg
> Germany
> 
> https://www.graylog.com
> 
> Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175
> Geschäftsführer: Lennart Koopmann (CEO)
> -- 
> You received this message because you are subscribed to a topic in the Google 
> Groups "Graylog Users" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/graylog2/KL6kn_4hm6k/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to 
> graylog2+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/graylog2/CAMqbBbJK8TLr6JuwQUx-gj8UjDuwFrZ84O5UqRtAxb%2B5t--0NA%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view

Re: [graylog2] Sidecar: When would you need more than one tag in a configuration?

[Marius Sturm]

Hi,
thanks for the feedback, that's very helpful for us. Otherwise we never
know if the concepts are understood by the users or not.
The current implementation is merging all configurations that are fetched
throug the provided tags. So let's say that your sidecar starts with three
tags, internally it's
fetching all three configurations and merge them together to a single new
one. So you end up with a configuration that includes all
outputs/inputs/snippets from all three configurations. That said, the only
limitation is in the web interface. Currently you can't use an output from
one configuration with an input from another configuration.
But that could be changed relatively easily.

Our goal is to find a solution that is intuitive and scales at the same
time. When you think of not having 1-5 configurations but maybe a few
dozens. Putting all inputs/output/snippets into one big bucket would be
pretty confusing imo. Losing the overview means you never know what will
actually be configured on the target system.

So I am very open to better ideas but they need to solve the problem for
bigger and smaller setups. Maybe there is some holy grail we missed till
now.

Cheers,
Marius



On 18 July 2016 at 19:02, 123Dev  wrote:

> Agreed, the tag is confusing to us too.
> On Graylog, if I have 3 configurations.
>
>- Config1 - tag1
>- Config2 - tag2
>- Config3 - tag3
>
> On the collector side, I was wrongly expecting that if I set tag1 and
> tag2, the client would get both configurations.
> But that didn't work
>
> Because each configuration to be configured needs its own output, and the
> generated nxlog.conf did not get the two outputs or the auto-generated two
> routes.
>
> I think Inputs, Outputs, Snippets and even routes (which cannot be defined
> now) need to be decoupled from a configuration and be allowed to be defined
> independently each in its own collections (similar to how the rules are
> done in pipelines) for convenience (you don't want to repeat for each
> config) and functionality (see below the nxlog internal example)
> Then a configuration can pull one or many of the above components from the
> collections, and then applied to a collector.
>
> If we don't decouple, then if we want to have nxlog internal logs logged
> to Graylog, we would need to define something like this in Snippets for
> each config (repeated and modified to match the output id)
> 
> Module  im_internal
> 
>
> 
>   Path internal => 977ad164136aa0330cf2b422
> 
>
>
> and if the client has multiple tags, would it get multiple copies? single
> copy? which output? which route?
>
> Sorry if we're understanding this thing totally wrong.
>
> Thanks
>
>
>
>
>
>
>
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Graylog Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to graylog2+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/graylog2/0eb49603-18b4-4e1e-98f7-02ccfd35ef08%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Developer

Tel.: +49 (0)40 609 452 077
Fax.: +49 (0)40 609 452 078

TORCH GmbH - A Graylog Company
Poolstraße 21
20335 Hamburg
Germany

https://www.graylog.com 

Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175
Geschäftsführer: Lennart Koopmann (CEO)

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CAMqbBbJK8TLr6JuwQUx-gj8UjDuwFrZ84O5UqRtAxb%2B5t--0NA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Sidecar: When would you need more than one tag in a configuration?

[123Dev]

Agreed, the tag is confusing to us too. 
On Graylog, if I have 3 configurations.

   - Config1 - tag1 
   - Config2 - tag2
   - Config3 - tag3

On the collector side, I was wrongly expecting that if I set tag1 and tag2, 
the client would get both configurations.
But that didn't work

Because each configuration to be configured needs its own output, and the 
generated nxlog.conf did not get the two outputs or the auto-generated two 
routes.

I think Inputs, Outputs, Snippets and even routes (which cannot be defined 
now) need to be decoupled from a configuration and be allowed to be defined 
independently each in its own collections (similar to how the rules are 
done in pipelines) for convenience (you don't want to repeat for each 
config) and functionality (see below the nxlog internal example)
Then a configuration can pull one or many of the above components from the 
collections, and then applied to a collector.

If we don't decouple, then if we want to have nxlog internal logs logged to 
Graylog, we would need to define something like this in Snippets for each 
config (repeated and modified to match the output id)

Module  im_internal



  Path internal => 977ad164136aa0330cf2b422



and if the client has multiple tags, would it get multiple copies? single 
copy? which output? which route?

Sorry if we're understanding this thing totally wrong.

Thanks












-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/0eb49603-18b4-4e1e-98f7-02ccfd35ef08%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Sidecar: When would you need more than one tag in a configuration?

[Marius Sturm]

Hi,
this depends on your tagging schema, let's say you have three classes of
machines 'database', 'application' and 'web_server' and you want to apply a
general configuration to all of them. E.g. a file input for
/var/log/messages, then you can create a configuration for that and use all
three tags.
But yes, to not get confused too much you should use at least on one side
just a single tag either the server side or the collector side.

Cheers,
Marius


On 16 July 2016 at 05:20, Werner van der Merwe 
wrote:

> Hi, this confuses me a bit.
>
> I understand a host can have multiple tags to combine multiple
> configurations, for example an apache server can have tags linux and apache.
>
> As I understand it, for this, two configurations will be created, one with
> a tag called linux and one with a tag called apache.
>
> So form that point I can understand it, but in what circumstances will a
> configuration have more than one tag?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Graylog Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to graylog2+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/graylog2/086e7094-c096-4ed7-8baf-dd96f1e1730c%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Developer

Tel.: +49 (0)40 609 452 077
Fax.: +49 (0)40 609 452 078

TORCH GmbH - A Graylog Company
Poolstraße 21
20335 Hamburg
Germany

https://www.graylog.com 

Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175
Geschäftsführer: Lennart Koopmann (CEO)

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CAMqbBbKuptJMH1J0%3DkcAG%3DCAwn0X-4YHx6U33%3DH1o4dTWnrkJg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Sidecar: When would you need more than one tag in a configuration?

[Werner van der Merwe]

Hi, this confuses me a bit.

I understand a host can have multiple tags to combine multiple 
configurations, for example an apache server can have tags linux and apache.

As I understand it, for this, two configurations will be created, one with 
a tag called linux and one with a tag called apache.

So form that point I can understand it, but in what circumstances will a 
configuration have more than one tag?

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/086e7094-c096-4ed7-8baf-dd96f1e1730c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.