Flink 1.17.2 planned?
Hi team, are there any infos about a bugfix release 1.17.2 available? E.g. will there be another bugfix release of 1.17 / approximate timing? We are hit by https://issues.apache.org/jira/browse/FLINK-32296 which leads to wrong SQL responses in some circumstances. Kind regards, Christian This e-mail is from Mapp Digital Group and its international legal entities and may contain information that is confidential. If you are not the intended recipient, do not read, copy or distribute the e-mail or any attachments. Instead, please notify the sender and delete the e-mail and any attachments.
Re: Flink 1.17.2 planned?
Hi, Christian We released 1.17.1 [1] in May, and the main focus of the community is currently on the 1.18 release, so 1.17.2 should be planned for after the 1.18 release! [1] https://flink.apache.org/2023/05/25/apache-flink-1.17.1-release-announcement/ Best, Ron Christian Lorenz via user 于2023年8月21日周一 17:33写道: > Hi team, > > > > are there any infos about a bugfix release 1.17.2 available? E.g. will > there be another bugfix release of 1.17 / approximate timing? > > We are hit by https://issues.apache.org/jira/browse/FLINK-32296 which > leads to wrong SQL responses in some circumstances. > > > > Kind regards, > > Christian > > This e-mail is from Mapp Digital Group and its international legal > entities and may contain information that is confidential. > If you are not the intended recipient, do not read, copy or distribute the > e-mail or any attachments. Instead, please notify the sender and delete the > e-mail and any attachments. >
Request-Response flow for real-time analytics
Hi, We are currently evaluating Flink for our analytics engine. We would appreciate any help with our experiment in using flink for real-time request-response use-case. To demonstrate the current use-case: our application produces events of the following form: {id, customerId, amount, timestamp} We calculate some continuous aggregates triggered by each event produced and use them to decide on the action. Examples of Aggregates: sum of amount total, amount group by customerId, amount per day(group-by customer), per month etc. One approach we considered is to correlate the aggregates with the `Id`, So for the following input events: {1, "CUST1", 100, $TS1} {2, "CUST2", 5, $TS2} {3, "CUST1", 15, $TS3} We would generate the following(ignoring timestamp for now) into kafka: {1, "TOTAL", 100} , {1, "GROUPBYCUSTTOTAL", "CUST1", 100} {2, "TOTAL", 105} , {2, "GROUPBYCUSTTOTAL", "CUST2", 5} {3, "TOTAL", 120} , {3, "GROUPBYCUSTTOTAL", "CUST1", 115} And our application would read from kafka and process them. So the flow looks like: Application -- kafka---> flink --> kafka <--- Application We want to keep our current request - response model i.e. we need all continuous aggregates out for every ingested event into flink, before we can further process the said event. Unfortunately we don't see a way to do this in flink-SQL: As the aggregates would not have the requestId for us to correlate with e.g. for the following simple continuous query: SELECT sum(amount) from EVENTS We have tried doing this with flink-Datastream API: KeyedProcessFunction with MapState per window, and collecting in processElement and using Kafka sink. A sample code for the windowing would look like the following: public void processElement(Transaction transaction, KeyedProcessFunction.Context context, Collector collector) throws Exception { () collector.collect(new Aggregate(transaction.getId(), context.getCurrentKey(), agg0, evTime)); } If we were to use FlinkSQL instead, how would we accomplish this functionality? If there are any alternative approaches to accomplish this while maintaining our invariant: every event must produce all aggregates that consume the corresponding event, we would love to hear from the community. Regards, Jiten -- *The information contained in this transmission (including any attachments) is confidential and may be privileged. It is intended only for the use of the individual or entity named above. If you are not the intended recipient; dissemination, distribution, or copy of this communication is strictly prohibited. If you have received this communication in error, please erase all copies of this message and its attachments and notify me immediately.*
Re: Flink 1.17.2 planned?
Hi Christian, Thanks for reaching out. Liked Ron pointed out that the community is focusing on the 1.18 release. If you are facing urgent issues, would you like to volunteer as the release manager of 1.17.2 and drive the release? Theoretically, everyone could be the release manager of a bugs fix release. Best regards, Jing On Tue, Aug 22, 2023 at 3:41 AM liu ron wrote: > Hi, Christian > > We released 1.17.1 [1] in May, and the main focus of the community is > currently on the 1.18 release, so 1.17.2 should be planned for after the > 1.18 release! > > [1] > https://flink.apache.org/2023/05/25/apache-flink-1.17.1-release-announcement/ > > > Best, > Ron > > Christian Lorenz via user 于2023年8月21日周一 17:33写道: > >> Hi team, >> >> >> >> are there any infos about a bugfix release 1.17.2 available? E.g. will >> there be another bugfix release of 1.17 / approximate timing? >> >> We are hit by https://issues.apache.org/jira/browse/FLINK-32296 which >> leads to wrong SQL responses in some circumstances. >> >> >> >> Kind regards, >> >> Christian >> >> This e-mail is from Mapp Digital Group and its international legal >> entities and may contain information that is confidential. >> If you are not the intended recipient, do not read, copy or distribute >> the e-mail or any attachments. Instead, please notify the sender and delete >> the e-mail and any attachments. >> >