[
https://issues.apache.org/jira/browse/FLUME-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siva Kotipalli updated FLUME-2779:
----------------------------------
Attachment: multilineSpool.patch
1. MultilineDeSerilizer.java
-- deserializer.logstartregexkey -- regxKey which is expected to log line
start
-- deserializer.maxLogLineLength -- this is the max log length that is
considered for transfer to sink, the remaining portion is trucated.
2. The details and options are provided in TestMultiLineDeserializer.java
> MultiLine Deserializer for Spooling DIrectory Source
> ----------------------------------------------------
>
> Key: FLUME-2779
> URL: https://issues.apache.org/jira/browse/FLUME-2779
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: 1.7
> Reporter: Siva Kotipalli
> Priority: Minor
> Attachments: multilineSpool.patch
>
>
> 1. Spool Directory source defaults to LINE, this has issue when multiline log
> events like stack traces and have request/responses.
> 2. Proposal to have log line start regex Key to aggregate all the log lines
> till the next regex key is found.
> 3. As there is aggregation needed based on startKey, maxLineLength is dropped
> on deserializer and included maxLogLineLength as alternative, this basically
> truncates the event based on the length and remaining part is never
> transferred to sink.
> 4. logstartregexkey is introduced on Deserializer to include the start regex
> key.
> 5. Patch available with unit tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
