[ https://issues.apache.org/jira/browse/FLINK-18479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150693#comment-17150693 ]
Jun Zhang commented on FLINK-18479: ----------------------------------- Hi,[~lzljs3620320] ,In this problem, the result of my test is that there is no problem with the watermark, but there is a problem with the toMills method, which cannot be controlled by the user. The steps of my test are: 1. The system tzone is UTC+08:00, 2. the tzone of the original data is UTC+08:00 3. The tzone of the watermark is UTC+08:00 4. {code:java} LocalDateTime partTime = extractor.extract( partitionKeys, extractPartitionValues(new Path(partition)));{code} This code gets a local time, but the toMills method treats this partTime as UTC time, The result of the toMills method is 16 hours more than UTC time and 8 hours more than watermark, which makes it impossible to trigger the submission. > can not commit partition when set partition time > ------------------------------------------------ > > Key: FLINK-18479 > URL: https://issues.apache.org/jira/browse/FLINK-18479 > Project: Flink > Issue Type: Bug > Components: FileSystems > Affects Versions: 1.11.0 > Reporter: Jun Zhang > Priority: Major > Labels: pull-request-available > Fix For: 1.11.1 > > > when we write streaming data to filesystem, and select the 'partition time' , > we can not commit the partition when write finished. > > {code:java} > LocalDateTime partTime = extractor.extract( > partitionKeys, extractPartitionValues(new Path(partition))); > if (watermark > toMills(partTime) + commitDelay) { > needCommit.add(partition); > iter.remove(); > } > {code} > when we set a not UTC zone, and submit the partition, the method 'toMills' > will get the UTC mills ,for example ,in UTC/GMT+08:00 ,the watermark will > less than the toMills , so we can not commit the partition forever. > if we use a local time , not utc , it will be ok in UTC zone and other zone. > -- This message was sent by Atlassian Jira (v8.3.4#803005)