[
https://issues.apache.org/jira/browse/FLINK-13609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16921068#comment-16921068
]
Joao Boto commented on FLINK-13609:
-----------------------------------
[~kkl0u] I understand what you say about timestamp, but could be more easy to
not see mistakes where there are none
I'm seeing some jumps on the same bucket
bucket 1 -> {{part-0-1, part-0-2, part-0-4}}
bucket 2 -> {{part-0-3, part-0-5}}
Now I understand why this happens, but the first time I was thinking that part3
on bucket1 are missing for some strange error.
> StreamingFileSink - reset part counter on bucket change
> -------------------------------------------------------
>
> Key: FLINK-13609
> URL: https://issues.apache.org/jira/browse/FLINK-13609
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
> Reporter: Joao Boto
> Priority: Major
>
> When writing to files using StreamingFileSink on bucket change we expect that
> partcounter will reset its counter to 0
> as a example
> * using DateTimeBucketAssigner using ({color:#6a8759}yyyy/MM/dd/HH{color})
> * and ten files hour (for simplicity)
> this will create the:
> * bucket 2019/08/07/00 with files partfile-0-0 to partfile-0-9
> * bucket 2019/08/07/01 with files partfile-0-10 to partfile-0-19
> * bucket 2019/08/07/02 with files partfile-0-20 to partfile-0-29
> and we expect this:
> * bucket 2019/08/07/00 with files partfile-0-0 to partfile-0-9
> * bucket 2019/08/07/01 with files partfile-0-0 to partfile-0-9
> * bucket 2019/08/07/02 with files partfile-0-0 to partfile-0-9
>
> [~kkl0u] i don't know if it's the expected behavior (or this can be
> configured)
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
