[jira] [Commented] (FLINK-18767) Streaming job stuck when disabling operator chaining
[
https://issues.apache.org/jira/browse/FLINK-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170037#comment-17170037
]
Nico Kruber commented on FLINK-18767:
-
Just to document it here: setting a buffer timeout of -1 means "no output
flusher" and if the buffer isn't filled (like in my case), it won't be flushed.
I meant to use an interval of 0 which flushes always. We also confirmed that
this one is working, so nothing wrong here, except my code.
> Streaming job stuck when disabling operator chaining
>
>
> Key: FLINK-18767
> URL: https://issues.apache.org/jira/browse/FLINK-18767
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Network
>Affects Versions: 1.9.3, 1.10.1, 1.11.1
>Reporter: Nico Kruber
>Assignee: Zhijiang
>Priority: Critical
>
> The following code is stuck sending data from the source to the map operator.
> Two settings seem to have an influence here: {{env.setBufferTimeout(-1);}}
> and {{env.disableOperatorChaining();}} - if I remove either of these, the job
> works as expected.
> (I pre-populated my Kafka topic with one element to reproduce easily)
> {code}
> StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment();
> // comment either these two and the job works
> env.setBufferTimeout(-1);
> env.disableOperatorChaining();
> Properties properties = new Properties();
> properties.setProperty("bootstrap.servers", "localhost:9092");
> properties.setProperty("group.id", "test");
> FlinkKafkaConsumer consumer = new FlinkKafkaConsumer<>("topic",
> new SimpleStringSchema(), properties);
> consumer.setStartFromEarliest();
> DataStreamSource input = env.addSource(consumer);
> input
> .map((x) -> x)
> .print();
> env.execute();
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-18767) Streaming job stuck when disabling operator chaining
[
https://issues.apache.org/jira/browse/FLINK-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169904#comment-17169904
]
Zhijiang commented on FLINK-18767:
--
After confirming with [~NicoK] offline, it had some misleading before and not a
real problem, so close it.
> Streaming job stuck when disabling operator chaining
>
>
> Key: FLINK-18767
> URL: https://issues.apache.org/jira/browse/FLINK-18767
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Network
>Affects Versions: 1.9.3, 1.10.1, 1.11.1
>Reporter: Nico Kruber
>Assignee: Zhijiang
>Priority: Critical
>
> The following code is stuck sending data from the source to the map operator.
> Two settings seem to have an influence here: {{env.setBufferTimeout(-1);}}
> and {{env.disableOperatorChaining();}} - if I remove either of these, the job
> works as expected.
> (I pre-populated my Kafka topic with one element to reproduce easily)
> {code}
> StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment();
> // comment either these two and the job works
> env.setBufferTimeout(-1);
> env.disableOperatorChaining();
> Properties properties = new Properties();
> properties.setProperty("bootstrap.servers", "localhost:9092");
> properties.setProperty("group.id", "test");
> FlinkKafkaConsumer consumer = new FlinkKafkaConsumer<>("topic",
> new SimpleStringSchema(), properties);
> consumer.setStartFromEarliest();
> DataStreamSource input = env.addSource(consumer);
> input
> .map((x) -> x)
> .print();
> env.execute();
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-18767) Streaming job stuck when disabling operator chaining
[
https://issues.apache.org/jira/browse/FLINK-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169769#comment-17169769
]
Nico Kruber commented on FLINK-18767:
-
[~zjwang] I actually tried 1.8.2 and it showed the same behaviour (but didn't
try any earlier version). Why did you remove that tag?
> Streaming job stuck when disabling operator chaining
>
>
> Key: FLINK-18767
> URL: https://issues.apache.org/jira/browse/FLINK-18767
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Network
>Affects Versions: 1.9.3, 1.10.1, 1.11.1
>Reporter: Nico Kruber
>Assignee: Zhijiang
>Priority: Critical
>
> The following code is stuck sending data from the source to the map operator.
> Two settings seem to have an influence here: {{env.setBufferTimeout(-1);}}
> and {{env.disableOperatorChaining();}} - if I remove either of these, the job
> works as expected.
> (I pre-populated my Kafka topic with one element to reproduce easily)
> {code}
> StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment();
> // comment either these two and the job works
> env.setBufferTimeout(-1);
> env.disableOperatorChaining();
> Properties properties = new Properties();
> properties.setProperty("bootstrap.servers", "localhost:9092");
> properties.setProperty("group.id", "test");
> FlinkKafkaConsumer consumer = new FlinkKafkaConsumer<>("topic",
> new SimpleStringSchema(), properties);
> consumer.setStartFromEarliest();
> DataStreamSource input = env.addSource(consumer);
> input
> .map((x) -> x)
> .print();
> env.execute();
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
