[
https://issues.apache.org/jira/browse/FLINK-31212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lyn Zhang updated FLINK-31212:
------------------------------
Description:
I have a case in [^test.sql] that records in table_1 left join fail will be
discard by group window.
I check the interval join operator implements. If one record in left table join
right table fail, the record will not be emitted realtime but emitted waiting
for half join bound time. In the test.sql, table_1 left join table_2 in 5
minute bound, and the output will delay 2.5 minute this will cause window
discard the records.
h2. testing
h4. input:
!image-2023-02-24-17-58-44-461.png!
{"n":"n1","ts":"2023-02-24 14:00:00"} \{"n":"n2","ts":"2023-02-24 14:00:00"}
\{"n":"n1","ts":"2023-02-24 14:06:01"}
!image-2023-02-24-17-58-57-238.png!
{"n":"n1","ts":"2023-02-24 14:00:00","v":111} \{"n":"n1","ts":"2023-02-24
14:06:01","v":111}
h4. output:
expect:
!image-2023-02-24-18-00-52-891.png!
real:
!image-2023-02-24-17-59-25-179.png!
I remove this logic in [https://github.com/apache/flink/pull/22014] Please
help to review this PR.
was:
I have a case in [^test.sql] that records in table_1 left join fail will be
discard by group window.
I check the interval join operator implements. If one record in left table join
right table fail, the record will not be emitted realtime but emitted waiting
for half join bound time. In the test.sql, table_1 left join table_2 in 5
minute bound, and the output will delay 2.5 minute this will cause window
discard the records.
h2. testing
h4. input:
!image-2023-02-24-17-58-44-461.png!
{"n":"n1","ts":"2023-02-24 14:00:00"}
{"n":"n2","ts":"2023-02-24 14:00:00"} \{"n":"n1","ts":"2023-02-24 14:06:01"}
!image-2023-02-24-17-58-57-238.png!
{"n":"n1","ts":"2023-02-24 14:00:00","v":111}
{"n":"n1","ts":"2023-02-24 14:06:01","v":111}
h4. output:
expect:
!image-2023-02-24-18-00-52-891.png!
real:
!image-2023-02-24-17-59-25-179.png!
I remove this logic in [https://github.com/apache/flink/pull/22014] Please
help to review this PR.
> Data lost if window group after interval left join
> --------------------------------------------------
>
> Key: FLINK-31212
> URL: https://issues.apache.org/jira/browse/FLINK-31212
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
> Affects Versions: 1.8.4
> Reporter: Lyn Zhang
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2023-02-24-17-58-44-461.png,
> image-2023-02-24-17-58-57-238.png, image-2023-02-24-17-59-25-179.png,
> image-2023-02-24-18-00-52-891.png, test.sql
>
>
>
> I have a case in [^test.sql] that records in table_1 left join fail will be
> discard by group window.
> I check the interval join operator implements. If one record in left table
> join right table fail, the record will not be emitted realtime but emitted
> waiting for half join bound time. In the test.sql, table_1 left join table_2
> in 5 minute bound, and the output will delay 2.5 minute this will cause
> window discard the records.
> h2. testing
> h4. input:
> !image-2023-02-24-17-58-44-461.png!
> {"n":"n1","ts":"2023-02-24 14:00:00"} \{"n":"n2","ts":"2023-02-24 14:00:00"}
> \{"n":"n1","ts":"2023-02-24 14:06:01"}
> !image-2023-02-24-17-58-57-238.png!
> {"n":"n1","ts":"2023-02-24 14:00:00","v":111} \{"n":"n1","ts":"2023-02-24
> 14:06:01","v":111}
> h4. output:
> expect:
> !image-2023-02-24-18-00-52-891.png!
> real:
> !image-2023-02-24-17-59-25-179.png!
> I remove this logic in [https://github.com/apache/flink/pull/22014] Please
> help to review this PR.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
