[jira] [Comment Edited] (FLINK-29183) flink job ended before window time up while using bounded stream source

2022-10-20 Thread Yun Gao (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-29183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17620855#comment-17620855
 ] 

Yun Gao edited comment on FLINK-29183 at 10/20/22 7:39 AM:
---

Hi [~landlord] , this is indeed a known issue that currently Flink will ignores 
all the register times at the end of bounded stream. Recently we have some 
discussion on this issue under 
https://issues.apache.org/jira/browse/FLINK-18647.   

We are planning to allow users to specify how to deal with the timers at the 
end of streams. I'll open a FLIP and discussion for that soon. 

If you feel ok I'll first close this issue and let's unify the discussion in 
FLINK-18647 and the mail list. 


was (Author: gaoyunhaii):
Hi [~landlord] , this is indeed a known issue that currently Flink will ignores 
all the register times at the end of bounded stream. Recently we have some 
discussion on this issue under 
https://issues.apache.org/jira/browse/FLINK-18647.   

We are planning to allow users to specify how to deal with the timers at the 
end of streams. I'll open a FLIP and discussion for that soon. 

> flink job ended before window time up while using bounded stream source
> ---
>
> Key: FLINK-29183
> URL: https://issues.apache.org/jira/browse/FLINK-29183
> Project: Flink
>  Issue Type: Bug
>  Components: API / DataStream
>Affects Versions: 1.15.2
>Reporter: Lu
>Priority: Major
> Attachments: image-2022-09-02-15-33-08-459.png, 
> image-2022-09-02-15-35-18-590.png, image-2022-09-06-18-09-40-132.png, 
> image-2022-09-06-18-09-43-745.png, image-2022-09-06-18-10-11-848.png, 
> image-2022-09-06-18-11-02-915.png
>
>
> !image-2022-09-02-15-35-18-590.png|width=574,height=306!
>  
> when i use a long time window while using bounded stream in STREAMING 
> mode，the job shutdown before the window has been triggered and then i have 
> nothing to print.
> if i use BATCH mode with the same code,  it will work. But the batch shuffle 
> is so expensive so i still prefer STREAMING. 
> Is there any method could fix this problem? Or should i have mistake in my 
> code?
> Looking forward to your reply.  Thanks in advance.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Comment Edited] (FLINK-29183) flink job ended before window time up while using bounded stream source

2022-09-04 Thread Weijie Guo (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-29183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17600229#comment-17600229
 ] 

Weijie Guo edited comment on FLINK-29183 at 9/5/22 6:55 AM:


[~landlord] I think this is the semantic different between streaming and batch, 
I'm not sure if there are configuration options that can control this behavior, 
but if you just want to use pipelined shuffle in batch mode, you can configure 
execution.batch-shuffle-mode: ALL_ EXCHANGES_ PIPELINED.


was (Author: weijie guo):
[~landlord] I think this is the semantic different between streaming and batch, 
I'm not sure if there are configuration options that can control this behavior, 
but if you just want to use pipelined shuffle in batch mode, you can configure 
execution batch-shuffle-mode: ALL_ EXCHANGES_ PIPELINED.

> flink job ended before window time up while using bounded stream source
> ---
>
> Key: FLINK-29183
> URL: https://issues.apache.org/jira/browse/FLINK-29183
> Project: Flink
>  Issue Type: Bug
>  Components: API / DataStream
>Affects Versions: 1.15.2
>Reporter: Lu
>Priority: Major
> Attachments: image-2022-09-02-15-33-08-459.png, 
> image-2022-09-02-15-35-18-590.png
>
>
> !image-2022-09-02-15-35-18-590.png|width=574,height=306!
>  
> when i use a long time window while using bounded stream in STREAMING 
> mode，the job shutdown before the window has been triggered and then i have 
> nothing to print.
> if i use BATCH mode with the same code,  it will work. But the batch shuffle 
> is so expensive so i still prefer STREAMING. 
> Is there any method could fix this problem? Or should i have mistake in my 
> code?
> Looking forward to your reply.  Thanks in advance.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)