[
https://issues.apache.org/jira/browse/FLUME-2725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denes Arvay updated FLUME-2725:
-------------------------------
Attachment: FLUME-2725-3.patch
> HDFS Sink does not use configured timezone for rounding
> -------------------------------------------------------
>
> Key: FLUME-2725
> URL: https://issues.apache.org/jira/browse/FLUME-2725
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Reporter: Eric Czech
> Assignee: Denes Arvay
> Priority: Minor
> Attachments: FLUME-2725-2.patch, FLUME-2725-3.patch, FLUME-2725.patch
>
>
> When a BucketPath used by an HDFS sink is configured to run with some
> roundUnit and roundValue > 1 (e.g. 6 hours), the "roundDown" function used by
> BucketPath does not actually round the date correctly.
> That function calls TimestampRoundDownUtil which creates a Calendar instance
> using the *local* timezone to truncate a unix timestamp rather than the
> TimeZone that the sink was configured to convert dates to paths with (and
> that timezone is already available in the BucketPath class but it just isn't
> passed to TimestampRoundDownUtil).
> The net effect of this is that if a flume jvm is running on a system with an
> EST clock while trying to write, say, 6 hour directories in UTC time, the
> directories are written with the hours 04, 10, 16, 22 rather than 00, 06, 12,
> 18 like you would expect.
> I found a workaround for this by passing
> "-Duser.timezone=<hdfs_sink_timezone>" as a system property, but I wanted to
> create a ticket for this since it seems like it would be very minimal effort
> to carry that configured timezone down into the rounding utility as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
