Hi, Could you clarify whether your question is about - connecting Table and DataStream APIs? [1] - matching the records using the Table API? - or matching in the DataStream API? [3] ...
You should probably look at the temporal joins to match records using Table API [2]; and connect/join/ or broadcast for the DataStream API [3]. The choice in the latter case depends on the matching logic. [1] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/data_stream_api/#converting-between-datastream-and-table [2] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/joins/#event-time-temporal-join [3] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/broadcast_state/ Regards, Roman On Tue, Feb 8, 2022 at 8:26 PM Shahdat Hossain <[email protected]> wrote: > > I am trying to match streaming data against my database table using Flink > Table API. I am able to match the data that exists in my streaming data and > the specific column of the database table. Now I need to activate some rules > that users have entered in the system (another database table) and I want to > evaluate the rules. What is the best approach? I have spent a lot of time > trying to figure it out within the Table API but I simply can't seem to > connect the dots. Is there another API that I should use to achieve this? > > Example of what I am trying to do: Multiple data from students coming in via > Kafka source. Student A, B, C, D and E data comes in with their grade of 70, > 80, 90, 95 and 98. I have a rule table that has let's say only 3 rows for > rule 1, 2 and 3. rule 1 says "if student C gets more than 85 send a thank you > note." Rule 2 says "if student D gets more than 92 send a thank you note." > Rule 3 says "if student E gets more than 96 send out a thank you note." Now I > can use Table API to do a join between the streaming data and the table to > only get C, D and E data. As soon as I get the match for C, I need to look > into the streaming data to get the student's score, compare that with the > rule, and need to execute the rule since in this case student C did get more > than 85. > > Your help is much appreciated. > > Regards, > Shahdat Hossain >
