[
https://issues.apache.org/jira/browse/FLINK-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15350387#comment-15350387
]
Geoffrey Mon commented on FLINK-4098:
-------------------------------------
Sorry, I must have made a typo in the previous comment. With some further
investigation, I found that the call to
{{NonReusingMutableToRegularIteratorWrapper.hasNext()}} (not getNext) in
{{PythonStreamer.streamBufferWithoutGroups}}
(at
[L257|https://github.com/apache/flink/blob/93c61c097a889c55f412ff31524c749851ca872f/flink-libraries/flink-python/src/main/java/org/apache/flink/python/api/streaming/data/PythonStreamer.java#L257])
hangs indefinitely because
{{NonReusingMutableToRegularIteratorWrapper.hasNext()}} tries to get the
[{{.next()}} object from its
{{source}}|https://github.com/apache/flink/blob/b8299bf92d8e3dbe140dd89602699394019b783d/flink-runtime/src/main/java/org/apache/flink/runtime/util/NonReusingMutableToRegularIteratorWrapper.java#L54],
which is a {{CountingMutableObjectIterator}}. The
{{CountingMutableObjectIterator}} in turn calls a {{ReaderIterator}} which
calls {{MutableRecordReader.getNextRecord()}}. On the first iteration of the
while loop in that function, the Reader wants {{.getNextBufferOrEvent()}} from
its {{SingleInputGate}} which endlessly polls waiting for data [at
L421|https://github.com/apache/flink/blob/c78b3c49e0e82874cbfa71e88bf28b99ed395610/flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/SingleInputGate.java#L421].
Thanks for your advice.
> Iteration support in Python API
> -------------------------------
>
> Key: FLINK-4098
> URL: https://issues.apache.org/jira/browse/FLINK-4098
> Project: Flink
> Issue Type: New Feature
> Components: Python API
> Affects Versions: 1.0.2
> Reporter: Geoffrey Mon
> Priority: Minor
>
> Bulk and delta iterations are not supported in the Python API. Currently
> working on this at https://github.com/GEOFBOT/flink
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
