> > It implements a fake known hosts and scan detection policy. > > the main things to figure out is: > > * How to work out the proper node_count at runtime. I think on a real bro > cluster the Cluster namespace has the data I need for this, including which > nodes are reachable. > > * How to handle one node becoming unreachable or a new node showing up. > Ideally bro would use a form of consistent ring hashing. > > If this were worked out, and implemented for logging as well, you could run a > bro cluster with 2 'manager' nodes and have a fully functioning cluster even > if one of them died. > > As is, I can probably use this on our test cluster to run 4 data nodes and > distribute scan detection to 4 cpu cores.
I got this to work on a real cluster running the broker integration branch! https://github.com/JustinAzoff/broker_distributed_events/blob/master/distributed_broker.bro Took me a while to figure out that a change somewhere made Broker::publish_topic required to send messages to other nodes, and that to raise notices on a datanode bro needs: redef datanode2manager_events += {"Notice::cluster_notice"}; The main screwy thing is that I couldn't figure out a consistent way to enumerate the data nodes[1], so I just hardcoded the index in the name: [datanode-0] type=datanode host=bro-test [datanode-1] type=datanode host=bro-test etc. Once I had the base functions in place I changed my scan.bro from #this is a worker2manager event event Scan::scan_attempt(scanner, attempt); to local args = Broker::event_args(scan_attempt, scanner, attempt); Cluster::send_event_hashed(scanner, args); And.. it just worked. I have 8 data nodes running that are each handling 1/8th of the scan detection aggregation. -- - Justin Azoff [1] They are stored in a set, and the 'first' data node on different workers was not always the same... would need to convert the set of data nodes to a list and sort it. _______________________________________________ bro-dev mailing list [email protected] http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
