[ https://issues.apache.org/jira/browse/FLUME-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386457#comment-15386457 ]
Mike Percy commented on FLUME-2653: ----------------------------------- Hmm, one more thing. I wonder what happens if you specify this in the configuration file: {code} my_agent.sinks.my_hdfs_sink.hdfs.inUseSuffix = {code} Won't that achieve a similar effect? Maybe this patch is not needed, and we already have this feature (but people just don't know how to use it)? I haven't investigated this in depth but that question also needs to be answered. Regarding the comment by [~yobibytes] about the Cloudera Manager validation, someone from Cloudera should investigate whether that is a real Flume bug, a Flume validation bug, or a Cloudera Manager validation bug. If I recall correctly, Cloudera Manager relies on information from Flume itself for most of its configuration validation. If it's a Flume-related bug, we can fix it here, otherwise that needs to be handled elsewhere (not as part of the Apache Flume project) > Allow inUseSuffix to be null/empty > ---------------------------------- > > Key: FLUME-2653 > URL: https://issues.apache.org/jira/browse/FLUME-2653 > Project: Flume > Issue Type: Improvement > Components: Sinks+Sources > Affects Versions: v1.5.1 > Reporter: Andrew Jones > Assignee: bimal tandel > Labels: docs-missing, hdfssink > Fix For: v1.7.0 > > Attachments: FLUME-2653.patch > > > At the moment, it doesn't seem possible to set the null/empty. We've tried > {{''}} which just adds the quotes to the end, and setting to nothing, which > just uses the default {{.tmp}}. > We want the _in use_ file to have the same name as the _closed_ file, so we > can read from files that are in use without the file moving from underneath > us. In our use case, we know that an in use file is still readable and > parseable, because it is just text with a JSON document per line. > It looks like [the HDFS sink > code|https://github.com/apache/flume/blob/542b1695033d330eb00ae81713fdc838b88332b6/flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java#L618] > can handle this change already, but at the moment there is no way to set the > {{bucketPath}} and {{targetPath}} to be the same. -- This message was sent by Atlassian JIRA (v6.3.4#6332)