> On Feb 10, 2017, at 11:49 AM, Matthias Vallentin <vallen...@icir.org> wrote: > > Concretely: can you describe (without Bro script code) what "client-side > load-balancing and failover" means? Who is the client and what state > needs to be resilient to failure? I don't think we have a working > definition of "data node" either. My hunch is that they are involved in > MapReduce computation and perhaps represent the reducers, but I'm not > sure. > > Matthias
Yes.. exactly like reducers. In this case, the clients are the workers and the servers are the manager/logger/datanode I want to send events containing data up to data nodes so they can be aggregated, but I don't want the data node to be a single point of failure or bottleneck. scan detection doesn't require coordination. The data just needs to be partitioned by source address. This also applies for: * Known hosts (partition on host) * Known services (partition on host or host+service) * Known certs (partition on cert hash) * Intel (partition on seen value) * Notices (partition on identifier) * DHCP (partition on mac address) as far as state, the data nodes COULD replicate their state to the other data nodes, but that's a whole separate issue. Initially the goal would just to be able to fail over from one data node to the next in the case of an outage. State on that data note would be lost if it wasn't replicated, but new work would be able to be performed instead of the system grinding to a halt. -- - Justin Azoff _______________________________________________ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
