[
https://issues.apache.org/jira/browse/NIFI-1221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Witt updated NIFI-1221:
------------------------------
Attachment: 0001-NIFI-1221-fixing-test-failures-contrib-failed-line-m.patch
> ListenSyslog should support batching
> ------------------------------------
>
> Key: NIFI-1221
> URL: https://issues.apache.org/jira/browse/NIFI-1221
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Mark Payne
> Assignee: Mark Payne
> Fix For: 0.4.0
>
> Attachments:
> 0001-NIFI-1221-Support-batching-of-Syslog-messages.patch,
> 0001-NIFI-1221-fixing-test-failures-contrib-failed-line-m.patch
>
>
> Currently, performance of ListenSyslog is pretty reasonable. If I configure
> logging to WARN level, and I use a Yield Duration of 0 ms, 3 threads, 25 ms
> run duration, then I can push about 23,000 messages per second to a single
> NiFi node without any loss, with message parsing enabled.
> However, I think we can do a lot better than that. Since these Syslog events
> are just log messages, they lend themselves very well to concatenation. We
> should have a Max Batch Size property as well as a Message Delimiter property.
> If using batch, though, it's important that we do not allow the Parse
> Messages property to be true, since it doesn't really make sense to add
> attributes if we have multiple messages.
> However, since we cannot parse the messages if bundled together, we should
> have a separate ParseSyslog processor that does parse them. This way, we can
> route specific events to a ParseSyslog processor. For instance, using
> RouteText to pull out events of interest.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
