[ https://issues.apache.org/jira/browse/FLINK-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Knauf reopened FLINK-2672: ------------------------------------- Re-opening in accordance with https://issues.apache.org/jira/browse/FLINK-23206. > Add partitioned output format to HDFS RollingSink > ------------------------------------------------- > > Key: FLINK-2672 > URL: https://issues.apache.org/jira/browse/FLINK-2672 > Project: Flink > Issue Type: Improvement > Components: Connectors / Common, Connectors / FileSystem > Affects Versions: 0.10.0 > Reporter: Mohamed Amine ABDESSEMED > Priority: Minor > Labels: auto-closed, features > > An interesting use case of the HDFS Sink is to dispatch data into multiple > directories depending of attributes present in the source data. > For example, for some data with a timestamp and a status fields, we want to > write it into different directories using a pattern like : > /somepath/%{timestamp}/%{status} > The expected results are somethings like: > /somepath/some_timestamp/wellformed > /somepath/some_timestamp/malformed > /somepath/some_timestamp/incomplete > ... > etc > To support this functionality the bucketing and checkpointing logics need to > be changed. > Note: For now, this can be done using the current version of the Rolling HDFS > Sink (https://github.com/apache/flink/pull/1084) with the help of splitting > data streams and having multiple HDFS sinks -- This message was sent by Atlassian Jira (v8.3.4#803005)