[ https://bro-tracker.atlassian.net/browse/BIT-1388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias Vallentin reassigned BIT-1388: --------------------------------------- Assignee: Matthias Vallentin > Broker's integration in Bro's main/run loop > ------------------------------------------- > > Key: BIT-1388 > URL: https://bro-tracker.atlassian.net/browse/BIT-1388 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro, Broker > Reporter: Jon Siwek > Assignee: Matthias Vallentin > Fix For: 2.5 > > > * There's a cost to Broker queues being idle. Whenever Broker gets a chance > to process messages, it looks for updates to all > connections/message-queues/data-stores. That involves sending synchronous > messages between actors, and for empty queues, it just gets back an empty > deque object it needs to destroy. > * Broker queues integrate into Bro's run loop by exposing a file descriptor > that's ready when the queue is non-empty. Users have the capability of > adding arbitrary numbers of queues at run-time (e.g. they can freely add > subscriptions to any amount of logs, events, etc.). Relying on select() may > become a bottleneck if someone has hundreds of Broker queues, or could > possibly break on some systems if FD_SETSIZE is limited to 1024. > Ideas on how to fix: > * Improve Bro's main run loop and dedicate an IOSource to each Broker queue > (instead of sharing a single IOSource like they do now). There might be > several things that could be tweaked in the main run loop, but at a minimum, > epoll()/kqueue() could alternatively replace select(). Could also think > about using something like libev > (http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod) to abstract what > particular polling backend is used. Might even be able to use libev's timers > to fix how Bro's timers are currently coupled w/ there being an active > IOSource consistently driving time forward. > * Move the draining of Broker queues completely off to their own threads. > This maybe is adding a bit too much complexity (Broker/CAF uses threads for > queues, then Bro would use more threads just to talk to those other > threads...). Since CAF becomes a requirement, it may be simpler to start > replacing/allowing some areas of Bro's threading to be done w/ CAF actors. > And then if Broker exposed an optional API to talk directly w/ CAF actors, > the integration w/ Bro may actually become more straightforward. > And those ideas don't have to be mutually exclusive. -- This message was sent by Atlassian JIRA (v7.2.0-OD-05-030#72002) _______________________________________________ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev