On Tue, Feb 13, 2018 at 11:41 -0600, Jon wrote: > If the goals is to prevent loss of data, then don't we need more than > just buffering, like message acknowledgements from the peer?
Yeah, I wouldn't see it as bullet-proof reliability, rather as a best effort "let's no needlesly drop stuff on the floor" kind of thing. I'm thinking less here of the cluster setting (where things can get complex and we'd usually restart everything anyways), and more of external agents streaming stuff into Bro, like with the osquery plugin. If one needs to restart the receiving-side Bro, it would be nice to not just drop any activity reported in the meantime. With that perspective, it would really just need just a bit of buffering of messages that cannot be sent out right now. And if in the end they still don't make it, that's not the end of the world. > And if you still planned on message routing/auto-forwarding being more > widely used, I think you would want to buffer the message while the > longest subscribed *path* has a down node? I was thinking to do the buffering at the routing/hop-level. The messsage would get as far as it can at first. If a peer is down that a node would have normally forwarded to, it'd buffer for a bit until that comes back (but I realize this makes it even more fuzzy which peers to wait for: in a flexible topology peers could come and go all the time; see below). That said, I'm now wondering if such buffering functionality should really be located inside CAF, as that's in charge of low-level message propagation. > Yeah, I'm also unclear if there's anyway you can tell if the peer is > supposed to be permanent vs. transient in come cases. We could make that an explicit endpoint option: "for this peer, on disconnect buffer stuff it would normally receive until it comes back (subject to some limits)". We may need a better way to identify the same peer though, just IP probably wouldn't work well. Maybe through some ID/name sent during the handshake? One would need to configure such a name for peers when turning on the buffering. > Last observation is that I think any of these types of changes would > be to the internal messaging pattern/protocol and so maybe reasonable > to change/improve in subsequent releases in a way that's transparent > to users. Yeah, nothing to get in immediately, still needs some thinking. I'm getting the sense though that we'll need it for some applications, osquery being the main one on my mind. Robin -- Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin _______________________________________________ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev