On Thu, Jul 16, 2020 at 17:15 -0700, Bob Murphy wrote:
> Here’s how it would work:
It would be helpful to see a draft API for the full batch writing
functionality to see how the pieces would work together. Could you
mock that up?
That said, couple of thoughts:
> 2. The failure_type value
>> For batching, I was thinking of having a way to send back a std::vector of
>> structs that would be something like this:
>>
>> struct failure_info {
>>uint32_t index_in_batch;
>>uint16_t failure_type;
>>uint16_t recovery_suggestion;
>> };
>
> This is almost starting to sound a
On 15 Jul 2020, at 20:45, Bob Murphy wrote:
>> On Jul 15, 2020, at 1:09 AM, Robin Sommer
>> wrote:
>>
>> Not quite sure what this would look like. Right now we just shut down
>> the thread on error, right? Can you elaborate how "report those
>> failures to other Zeek components" and "make
> On Jul 15, 2020, at 1:09 AM, Robin Sommer wrote:
>
> On Thu, Jul 09, 2020 at 18:19 -0700, Bob Murphy wrote:
>
>> Proposed Solution: Add a new optional API for writing a batch all at once,
>> while
>> still supporting older log writers that don't need to write batches.
>
> That sounds good
On Thu, Jul 09, 2020 at 18:19 -0700, Bob Murphy wrote:
> Proposed Solution: Add a new optional API for writing a batch all at once,
> while
> still supporting older log writers that don't need to write batches.
That sounds good to me, a PR with the proposed API would be great.
> a. For
Summary
This proposal is aimed at solving two intertwined problems in Zeek's log-
writing system:
Problem: Batch writing code duplication
- Some log writers need to send multiple log records at a time in "batches".
These include writers that send data to elasticsearch, splunk hec, kinesis,
