Re: [Bro-Dev] Broker's remote logging (BIT-1784)

2017-02-07 Thread Robin Sommer
On Tue, Feb 07, 2017 at 02:46 +, you wrote: > It was actually always confusing to me that a remote log entry versus > a local log entry would be processed differently regarding the log_* > events. I know, it's a bit confusing. Some of that is historic and part of trying to maintain

Re: [Bro-Dev] Broker's remote logging (BIT-1784)

2017-02-06 Thread Siwek, Jon
> On Feb 6, 2017, at 10:43 AM, Robin Sommer wrote: > > I propose that for now we make Broker work like the current model, and > then we go from there. If we need more, we can add that later. The > less semantic differences we have between old and new communication, > the easier

Re: [Bro-Dev] Broker's remote logging (BIT-1784)

2017-02-06 Thread Robin Sommer
On Sun, Feb 05, 2017 at 22:04 +, you wrote: > I understand the argument that the old semantics (manager not running > log events/filters) may be more performant, though, I’d consider > whether the internal comm. framework or the base/user scripts should > be the one to decide. I agree that

Re: [Bro-Dev] Broker's remote logging (BIT-1784)

2017-02-05 Thread Siwek, Jon
> On Jan 31, 2017, at 5:41 PM, Azoff, Justin S wrote: > >> I'm wondering if there's a reason that in the Broker case things >> *have* to be this way. Is there something that prevents the Broker >> manager from doing the same as the RemoteSerializer? > > Jon would know

Re: [Bro-Dev] Broker's remote logging (BIT-1784)

2017-02-01 Thread Seth Hall
> On Jan 31, 2017, at 5:41 PM, Robin Sommer wrote: > > For RemoteSerializer, log events and log > filters run only on the originating nodes; those guys make all > decisions about what's getting logged exactly and they then send that > on to the manager, which just writes out the

Re: [Bro-Dev] Broker's remote logging (BIT-1784)

2017-01-31 Thread Robin Sommer
On Tue, Jan 31, 2017 at 18:44 -0800, you wrote: > Purely from an API perspective, could we just move the call from one > Write() function to the other? I think the answer is yes, but I've looked at bit more at the code and I think I see where the challenge is: that 2nd Write() method (the one

Re: [Bro-Dev] Broker's remote logging (BIT-1784)

2017-01-31 Thread Matthias Vallentin
> I'm wondering if there's a reason that in the Broker case things > *have* to be this way. Is there something that prevents the Broker > manager from doing the same as the RemoteSerializer? Some background: when Broker sends to a log topic, the message has the structure of a pair (id, (x, y, z,

Re: [Bro-Dev] Broker's remote logging (BIT-1784)

2017-01-31 Thread Alan Commike
On Tue, Jan 31, 2017 at 3:48 PM Azoff, Justin S wrote: > > 2. Having the logger node be as much of a dumb byte mover as possible is > best for performance reasons. Having the log events and log filters run on > the workers lets that functionality scale out across the nodes.

Re: [Bro-Dev] Broker's remote logging (BIT-1784)

2017-01-31 Thread Azoff, Justin S
> On Jan 31, 2017, at 5:41 PM, Robin Sommer wrote: > ... > The fact that RemoteSerializer and broker::Manager are calling > different Write() functions seems to be a broader issue: we get > different semantics that way. For RemoteSerializer, log events and log > filters run only

[Bro-Dev] Broker's remote logging (BIT-1784)

2017-01-31 Thread Robin Sommer
Taking from ticket to the mailing list as I'm looking for some input. https://bro-tracker.atlassian.net/browse/BIT-1784 says: > The change from the older communication code is that > RemoteSerializer::ProcessLogWrite used to do > > success = log_mgr->Write(id_val, writer_val, path,