> On Oct 6, 2017, at 5:58 PM, Robin Sommer <[email protected]> wrote: > > In the most simple version of this, the cluster framework would just > hard-code a subscription to "bro/cluster/". And then scripts like the > Intel framework would just publish all their events to "bro/cluster/" > directly through Broker. > > To allow for distinguishing by node type we can define separate topic > hierarchies: "bro/cluster/{manager,worker,logger}/". Each node > subscribes to the hierarchy corresponding to its type, and each script > publishes according to where it wants to send events to (again > directly using the Broker API).
Yeah, that could be a better way to approach it, thanks. I’ll try to go back and rework the design around that topic hierarchy/naming convention (that was the part I was most unsure about). > >> Broker Framework API >> -------------------- > > I'm wondering if these store operations should become part of the > Cluster framework instead. If we added them to the Broker framework, > we'd have two separate store APIs there: one low-level version mapping > directly to the C++ Broker API, and one higher-level that configures > things like location of the DB files. That could be confusing. Yeah could be. I’ll try moving more stuff into Cluster and see if it still makes sense to me. > I like this. One additional idea: while I see that it's generally the > user who wants to configure which backend to use, the script author > may know already if it's data that should be persistent across > execution; I'm guessing that's usually implied by the script's > semantics. We could give InitStore() an additional boolean > "persistent" to indicate that. Ack. >> # User needs to be able to choose data store backends and which cluster node >> the >> # the master store lives on. They can either do this manually, or BroControl >> # will autogenerate the following in cluster-layout.bro: > > I don't really like the idea of autogenerating this, as it's pretty > complex information. Usually, the Broker::default_* values should be > fine, right? For the few cases where one wants to tweak that on a > per-store bassis, using a manual redef on the table sounds fine to me. It’s just a matter of where you expect most users to feel comfortable making customizations: in Bro scripts or in a broctl config file. I think it’s fine to first assume it won’t be needed often and so only provide the customization via Bro scripts directly. If we learn later that it’s a pain point for users, it’s easy add the "simpler" config file interface via broctl to help autogenerate it. > Hmm, actually, what would you think about using functions instead of > tables? We could model this similar to how the logging framework does > filters: there's a default filter installed, but you can retrieve and > update it. Here there'd be a default StoreInfo, which one can update. I think I went with the ‘redef’ interface first because it’s impossible for a user to screw up order of operations there, where with functions you can (technically) have some &priority mishaps on bro_init() since the InitStore() function is also going to be running in bro_init(). Maybe the key point is that these customizations only make sense to happen once before init time? i.e. a function would imply calling it anytime at runtime could yield a useful result, but at the moment, we’re not allowing changing a store’s backend or master node dynamically at runtime, just once before bro_init(). So if you think that’s something to anticipate in the future, I’d agree that just using functions from the start would be better. >> redef Broker::default_store_dir = "/home/jon/stores"; > > Can the default_store_dir be set to some standard location through > BroControl? Would be neat if this all just worked in the standard case > without any custom configuration at all. Yeah, should be possible, I think I had just given a random example in the above. - Jon _______________________________________________ bro-dev mailing list [email protected] http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
