[
https://issues.apache.org/jira/browse/FLINK-38735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naijie Liu updated FLINK-38735:
-------------------------------
Description:
h1. How to reproduce :
# use file system streaming sink, set `sink.partition-commit.trigger` to
`partition-time`
# stop job, update the config to `process-time`
# restore job from latest state
# when job started, got exception :
```
java.util.NoSuchElementException: null at
java.util.ArrayList$Itr.next(ArrayList.java:864) ~[?:1.8.0_261] at
org.apache.flink.connector.file.table.stream.ProcTimeCommitTrigger.<init>(ProcTimeCommitTrigger.java:66)
~[flink-connector-files-1.20.1.jar:1.20.1] at
org.apache.flink.connector.file.table.stream.PartitionCommitTrigger.create(PartitionCommitTrigger.java:72)
~[flink-connector-files-1.20.1.jar:1.20.1] at
org.apache.flink.connector.file.table.stream.PartitionCommitter.initializeState(PartitionCommitter.java:113)
~[flink-connector-files-1.20.1.jar:1.20.1] at
org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.initializeOperatorState(StreamOperatorStateHandler.java:147)
~[flink-dist-1.20.1.jar:1.20.1
```
h1. Issue analytics
the code is
[flink/flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/stream/ProcTimeCommitTrigger.java
at release-2.1.1 · apache/flink ·
GitHub|https://github.com/apache/flink/blob/release-2.1.1/flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/stream/ProcTimeCommitTrigger.java#L66]
When I restore job without `pending-partitions-with-time` state, it still get
element from state list, but the list is empty, so got `NoSuchElementException`
h1. Improve
Do we need fix this issue, only get/put when state has element
Or this is expected behavior (restoring from an empty need throw exception), I
think we can update it to a more meaningful alert message.
_*I want fix this issue, could you assign this task to me, thx~*_
was:
## How to reproduce :
# use file system streaming sink, set `sink.partition-commit.trigger` to
`partition-time`
# stop job, update the config to `process-time`
# restore job from latest state
# when job started, got exception :
```
java.util.NoSuchElementException: null at
java.util.ArrayList$Itr.next(ArrayList.java:864) ~[?:1.8.0_261] at
org.apache.flink.connector.file.table.stream.ProcTimeCommitTrigger.<init>(ProcTimeCommitTrigger.java:66)
~[flink-connector-files-1.20.1.jar:1.20.1] at
org.apache.flink.connector.file.table.stream.PartitionCommitTrigger.create(PartitionCommitTrigger.java:72)
~[flink-connector-files-1.20.1.jar:1.20.1] at
org.apache.flink.connector.file.table.stream.PartitionCommitter.initializeState(PartitionCommitter.java:113)
~[flink-connector-files-1.20.1.jar:1.20.1] at
org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.initializeOperatorState(StreamOperatorStateHandler.java:147)
~[flink-dist-1.20.1.jar:1.20.1
```
# Issue analytics
the code is
[flink/flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/stream/ProcTimeCommitTrigger.java
at release-2.1.1 · apache/flink ·
GitHub|https://github.com/apache/flink/blob/release-2.1.1/flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/stream/ProcTimeCommitTrigger.java#L66]
When I restore job without `pending-partitions-with-time` state, it still get
element from state list, but the list is empty, so got `NoSuchElementException`
# Improve
Do we need fix this issue, only get/put when state has element
Or this is expected behavior (restoring from an empty need throw exception), I
think we can update it to a more meaningful alert message.
I want fix this issue, could you assign this task to me, thx~
> ProcTimeCommitTrigger restore issue without state
> -------------------------------------------------
>
> Key: FLINK-38735
> URL: https://issues.apache.org/jira/browse/FLINK-38735
> Project: Flink
> Issue Type: Bug
> Components: Connectors / FileSystem
> Reporter: Naijie Liu
> Priority: Major
>
> h1. How to reproduce :
> # use file system streaming sink, set `sink.partition-commit.trigger` to
> `partition-time`
> # stop job, update the config to `process-time`
> # restore job from latest state
> # when job started, got exception :
> ```
> java.util.NoSuchElementException: null at
> java.util.ArrayList$Itr.next(ArrayList.java:864) ~[?:1.8.0_261] at
> org.apache.flink.connector.file.table.stream.ProcTimeCommitTrigger.<init>(ProcTimeCommitTrigger.java:66)
> ~[flink-connector-files-1.20.1.jar:1.20.1] at
> org.apache.flink.connector.file.table.stream.PartitionCommitTrigger.create(PartitionCommitTrigger.java:72)
> ~[flink-connector-files-1.20.1.jar:1.20.1] at
> org.apache.flink.connector.file.table.stream.PartitionCommitter.initializeState(PartitionCommitter.java:113)
> ~[flink-connector-files-1.20.1.jar:1.20.1] at
> org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.initializeOperatorState(StreamOperatorStateHandler.java:147)
> ~[flink-dist-1.20.1.jar:1.20.1
> ```
>
> h1. Issue analytics
> the code is
> [flink/flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/stream/ProcTimeCommitTrigger.java
> at release-2.1.1 · apache/flink ·
> GitHub|https://github.com/apache/flink/blob/release-2.1.1/flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/stream/ProcTimeCommitTrigger.java#L66]
>
> When I restore job without `pending-partitions-with-time` state, it still get
> element from state list, but the list is empty, so got
> `NoSuchElementException`
>
> h1. Improve
> Do we need fix this issue, only get/put when state has element
> Or this is expected behavior (restoring from an empty need throw exception),
> I think we can update it to a more meaningful alert message.
>
>
> _*I want fix this issue, could you assign this task to me, thx~*_
>
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
