Re: Flink reads data from JDBC table only on startup

2020-12-30 Thread Danny Chan
For your case, you should use a temporal table join syntax, and set up a refresh TTL for the RHS join cache. Taras Moisiuk 于2020年12月28日周一 下午7:21写道: > Hi Danny, > > I use regular join and it looks like: > > SELECT > ... > FROM *dynamic_kafka_table* k > JOIN *jdbc_table* j ON k.id = j.k_id > > > S

Re: Flink reads data from JDBC table only on startup

2020-12-28 Thread Taras Moisiuk
Hi Danny, I use regular join and it looks like: SELECT ... FROM *dynamic_kafka_table* k JOIN *jdbc_table* j ON k.id = j.k_id Should I set some additional conditions for this join? -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Flink reads data from JDBC table only on startup

2020-12-27 Thread Danny Chan
Hi Taras ~ There is a look up cache for temporal join but it default is false, see [1]. That means, by default FLINK SQL would lookup the external databases on each record from the JOIN LHS. Did you use the temporal table join syntax or normal stream-stream join syntax ? The temporal table join u

Flink reads data from JDBC table only on startup

2020-12-26 Thread Taras Moisiuk
Hi everyone! I'm using Flink 1.12.0 with SQL API. I'm developing a streaming job with join and insertion into postgreSQL. There is two tables in join: 1. Dynamic table based on kafka topic 2. Small lookup JDBC table >From what I can see Flink job reads data from JDBC table only on startup and mar