[
https://issues.apache.org/jira/browse/NIFI-1221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15030360#comment-15030360
]
Joseph Witt commented on NIFI-1221:
-----------------------------------
with patch build tests fail
Failed tests:
TestListenSyslog.testTCPMultipleConnection:206
expected:<[TCP:/]//127.0.0.1:0> but was:<[tcp:]//127.0.0.1:0>
TestListenSyslog.testTCPSingleConnection:154 expected:<[TCP:/]//127.0.0.1:0>
but was:<[tcp:]//127.0.0.1:0>
TestListenSyslog.testUDP:103 expected:<[UDP:/]//127.0.0.1:0> but
was:<[udp:]//127.0.0.1:0>
Adjusting tests to only verify that the transit URI is set and that it starts
with the proper protocol. Will also have that patch fix a contrib-check issue
and move the initialization of the demarcator bytes to onScheduled.
> ListenSyslog should support batching
> ------------------------------------
>
> Key: NIFI-1221
> URL: https://issues.apache.org/jira/browse/NIFI-1221
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Mark Payne
> Assignee: Mark Payne
> Fix For: 0.4.0
>
> Attachments: 0001-NIFI-1221-Support-batching-of-Syslog-messages.patch
>
>
> Currently, performance of ListenSyslog is pretty reasonable. If I configure
> logging to WARN level, and I use a Yield Duration of 0 ms, 3 threads, 25 ms
> run duration, then I can push about 23,000 messages per second to a single
> NiFi node without any loss, with message parsing enabled.
> However, I think we can do a lot better than that. Since these Syslog events
> are just log messages, they lend themselves very well to concatenation. We
> should have a Max Batch Size property as well as a Message Delimiter property.
> If using batch, though, it's important that we do not allow the Parse
> Messages property to be true, since it doesn't really make sense to add
> attributes if we have multiple messages.
> However, since we cannot parse the messages if bundled together, we should
> have a separate ParseSyslog processor that does parse them. This way, we can
> route specific events to a ParseSyslog processor. For instance, using
> RouteText to pull out events of interest.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
