On Wed, Aug 8, 2018 at 2:50 PM Robin Sommer <ro...@corelight.com> wrote:
> > * enable message forwarding by default (meaning re-implement the one > > or two subscription patterns that might create a cycle) > > Haven't quite made up my mind on this one. In principlel yes, but > right now a host needs to be subscribed to a topic to forward it if I > remember than right. That may limit how we use topics, not sure (e.g., > if a worker wanted to talk to other workers, with "real" > forwarding/routing they'd just publish to the worker topic and that > message would get routed there, but not be processed at the > intermediary hops as well. With our current forwarding, the hops would > need to subscribe to the worker topic as well and hence the event got > raised there, too.) Yeah, that's how I also understand the current mechanisms would work. Maybe can split it into two separate questions: (1) enable the "explicit/manual" forwarding by default? (2) re-implement any existing subscription cycles? Answer to (2) may pragmatically be "yes" because they'd be known to cause problems if ever (1) did become enabled (and also could be problematic for a more sophisticated/automatic/implicit routing system should that become available in the future... at least I think it's a problem, but then again maybe connection-cycles would also still be a problem at that point, not quite sure). Answer to (1) may be "no" because we don't have a use for it at the moment -- having the forwarding-nodes also raise events is not ideal, but if we solved that would it be useful? Maybe an idea would be extend the subscribe() API in Bro: function Broker::subscribe(topic_prefix: string, forward_only: bool &default=F); I recall that we have access to both the message/event as well as the topic string on the receiver side, so could be possible to detect whether or not to raise the event depending on whether the topic only has a matching subscription prefix that is marked as forward_only. With that you could do something like: # On Manager Broker::subscribe(worker_to_worker_topic, T); # On Worker Broker::subscribe(worker_to_worker_topic); Broker::publish(worker_to_worker_topic, my_event); There, my_event would be distributed from one worker to all workers via the manager, but not sure that's as usable/dynamic as the current "relay" mechanism because you also get a load-balancing scheme to go along with it. Here, you'd only ever want to pick a single manager or proxy to do the forwarding (subscribing like this on all proxies causes all proxies to forward to all workers resulting in undesired event duplication.) So I guess that's still to say I'm not sure what the use of the current forwarding mechanism would be if it were enabled. Also maybe begs the question for later regarding the "real" routing mechanism: I suppose that would need to be smart enough to do automatic load-balancing in the case of there being more than one route to a subscriber. - Jon _______________________________________________ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev