[
https://issues.apache.org/jira/browse/FLUME-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662892#comment-14662892
]
Juhani Connolly commented on FLUME-2498:
----------------------------------------
Pardon my being forward, but are these actually features we actually need for
the patch to be released?
I think we should be fixing any actual bugs(the /r /n issue you mentioned), and
documentation(e.g. documenting it is not appropriate for binary files) and then
committing. After that others are free to further improve on the source by
adding deserializer support rather than further delaying inclusion. Committing
without deserializer support does not strike me as harmful to users, just a
missing feature that would be nice to have and would be an appropriate
follow-up patch(same with most of the other suggestions)
As you mentioned, it is pretty mature in implementation. It's been in
production use for about an year now, on a very large number of servers. Trying
to throw in more features in this patch(rather than a separate one) is just
going to mean additional debugging and delays. Inclusion has no impact on other
components so it is not harmful to them, and the main considerations should be
"is it needed"(I would say yes) and "does it work as documented"(possibly
needing a newline handling fix and documentation on what it does/does not
handle). Committing it opens it up to modification by more people to contribute
the features they would like to see added.
> Implement Taildir Source
> ------------------------
>
> Key: FLUME-2498
> URL: https://issues.apache.org/jira/browse/FLUME-2498
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Reporter: Satoshi Iijima
> Fix For: v1.7.0
>
> Attachments: FLUME-2498-2.patch, FLUME-2498-3.patch, FLUME-2498.patch
>
>
> This is the proposal of implementing a new tailing source.
> This source watches the specified files, and tails them in nearly real-time
> once appends are detected to these files.
> * This source is reliable and will not miss data even when the tailing files
> rotate.
> * It periodically writes the last read position of each file in a position
> file using the JSON format.
> * If Flume is stopped or down for some reason, it can restart tailing from
> the position written on the existing position file.
> * It can add event headers to each tailing file group.
> A attached patch includes a config documentation of this.
> This source requires Unix-style file system and Java 1.7 or later.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
