Re: [Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

2020-07-18 Thread Bob Murphy
> On Jul 17, 2020, at 3:01 AM, Robin Sommer wrote: > > On Wed, Jul 15, 2020 at 14:57 -0700, Bob Murphy wrote: > >> use a single, common logging API, but let it send its output to >> different output mechanisms that support different use cases. > > I get that in general. It's just that afaik

Re: [Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

2020-07-17 Thread Robin Sommer
On Wed, Jul 15, 2020 at 14:57 -0700, Bob Murphy wrote: > use a single, common logging API, but let it send its output to > different output mechanisms that support different use cases. I get that in general. It's just that afaik this is the first time this need comes up. Adding a

Re: [Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

2020-07-15 Thread Bob Murphy
> > On Jul 15, 2020, at 12:52 AM, Robin Sommer wrote: > Reading through this thread, I'm wondering if we should focus on > improving identification of log lines in terms of where they come from > and when they were generated, while keeping to go through the existing > mechanism of sending

Re: [Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

2020-07-15 Thread Robin Sommer
Reading through this thread, I'm wondering if we should focus on improving identification of log lines in terms of where they come from and when they were generated, while keeping to go through the existing mechanism of sending messages back to main process for output (so that we don't need the

Re: [Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

2020-07-14 Thread Bob Murphy
> On Jul 14, 2020, at 1:14 PM, Jon Siwek wrote: > > On Tue, Jul 14, 2020 at 11:56 AM Bob Murphy wrote: > >> The code you show should give correct ordering on when Foo() and Bar() >> finish. > > Wondering what's meant by "correct ordering" here. Bar() can finish > before Foo() and yet

Re: [Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

2020-07-14 Thread Jon Siwek
On Tue, Jul 14, 2020 at 11:56 AM Bob Murphy wrote: > The code you show should give correct ordering on when Foo() and Bar() finish. Wondering what's meant by "correct ordering" here. Bar() can finish before Foo() and yet debug.log can report "I did Foo" before "I did Bar" for whatever

Re: [Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

2020-07-14 Thread Bob Murphy
> On Jul 14, 2020, at 11:35 AM, Jon Siwek wrote: > > On Tue, Jul 14, 2020 at 8:05 AM Bob Murphy wrote: > >> My current implementation does just use a mutex to control access to the >> output file, and reports the time of generation. > > I was also trying to break down a couple distinct

Re: [Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

2020-07-14 Thread Jon Siwek
On Tue, Jul 14, 2020 at 8:05 AM Bob Murphy wrote: > My current implementation does just use a mutex to control access to the > output file, and reports the time of generation. I was also trying to break down a couple distinct requirements and wondered if that actually covers the 2nd: (1) Fix

Re: [Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

2020-07-14 Thread Bob Murphy
> On Jul 13, 2020, at 1:42 PM, Jon Siwek wrote: > > On Fri, Jul 10, 2020 at 11:00 AM Bob Murphy wrote: > >> The development I was doing sometimes required me to examine the debug >> messages from different threads in the chronological order they were >> generated. But if I understand it

Re: [Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

2020-07-13 Thread Jon Siwek
On Fri, Jul 10, 2020 at 11:00 AM Bob Murphy wrote: > The development I was doing sometimes required me to examine the debug > messages from different threads in the chronological order they were > generated. But if I understand it correctly, the threading framework’s > logging doesn’t

Re: [Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

2020-07-10 Thread Bob Murphy
Hi Johanna, I wasn’t aware of that call, but it also wouldn’t have done what I needed. If I understand the code correctly, each MsgThread has a FIFO queue that it pushes messages onto. Later on, the main thread occasionally runs a loop where it handles all the queued messages from the first

Re: [Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

2020-07-09 Thread Johanna Amann
On 9 Jul 2020, at 16:57, Bob Murphy wrote: > Right now, if you try to use Zeek's debug logging facilities in > DebugLogger.h concurrently from multiple threads, the contents of > debug.log can get mixed up and look like like "word salad". Is there a reason why you didn’t just use the Debug

[Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

2020-07-09 Thread Bob Murphy
Right now, if you try to use Zeek's debug logging facilities in DebugLogger.h concurrently from multiple threads, the contents of debug.log can get mixed up and look like like "word salad". I've been working on log writers for Zeek. Those operate in different threads, and using Zeek's current