Re: Need help how to use Table API to join two Kafka streams

2023-02-08 Thread Martijn Visser
Moving the Dev mailing list to BCC and adding the User ML in this thread On Wed, Feb 8, 2023 at 8:08 AM Amir Hossein Sharifzadeh < amirsharifza...@gmail.com> wrote: > Thanks. If you look at the code, I am defining/creating the table as: > > create_kafka_source_ddl = """ > CREATE TABLE pay

Re: Need help how to use Table API to join two Kafka streams

2023-02-07 Thread Amir Hossein Sharifzadeh
Thanks. If you look at the code, I am defining/creating the table as: create_kafka_source_ddl = """ CREATE TABLE payment_msg( createTime VARCHAR, orderId BIGINT, payAmount DOUBLE, payPlatform INT, provinceId INT ) WITH (

Re: Need help how to use Table API to join two Kafka streams

2023-02-07 Thread Leonard Xu
> The result here is the representation of the statement execution result, NOT > the query output. > You can define a print connector table[1] as your sink table, and insert into > the query output to the sink table to achieve your goal. Correction: you can also iterate the output result from

Re: Need help how to use Table API to join two Kafka streams

2023-02-07 Thread Leonard Xu
> 1) *First*: In *payment_msg_proccessing.py > * > code, > I want to run a simple query on Kafka stream (payment_msg table) without > insertion data into the sink table (es_sink here) and do s

Re: Need help how to use Table API to join two Kafka streams

2023-02-07 Thread Amir Hossein Sharifzadeh
erializeValue(SerializedValue.java:67) > at > org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder.create(OperatorCoordinatorHolder.java:488) > at > org.apache.flink.runtime.executiongraph.ExecutionJobVertex.createOperatorCoordinatorHolder(ExecutionJobVertex.java:286) &

Re: Need help how to use Table API to join two Kafka streams

2023-02-07 Thread Amir Hossein Sharifzadeh
ot cause. Maybe > you can post the full error message . > > Best regards, > Yuxia > > -- > *发件人: *"Amir Hossein Sharifzadeh" > *收件人: *"yuxia" > *抄送: *"dev" > *发送时间: *星期二, 2023年 2 月 07日 上午 10:39:25 > *主题: *Re: Need help how to us

Re: Need help how to use Table API to join two Kafka streams

2023-02-06 Thread yuxia
don't need to cosume data in my cosumer class seperately and then insert them into those tables. The data will be consumed from what we implemented here. Best regards, Yuxia 发件人: "Amir Hossein Sharifzadeh" < [ mailto:amirsharifza...@gmail.com | amirsharifza...@gmail.com ]

Re: Need help how to use Table API to join two Kafka streams

2023-02-06 Thread Amir Hossein Sharifzadeh
tImpl.java:884) ... 13 more Best, Amir On Sun, Feb 5, 2023 at 8:20 PM yuxia wrote: > Hi, thanks for reaching me out. > For your question, you don't need to cosume data in my cosumer class > seperately and then insert them into those tables. The data will > be c

Re: Need help how to use Table API to join two Kafka streams

2023-02-05 Thread yuxia
t; 收件人: luoyu...@alumni.sjtu.edu.cn 发送时间: 星期日, 2023年 2 月 05日 上午 6:07:02 主题: Re: Need help how to use Table API to join two Kafka streams Dear Yuxia, [ mailto:dev@flink.apache.org | dev@flink.apache.org ] Thank you again for your help. I am implementing code in Python. But I am still have some con

Re: Need help how to use Table API to join two Kafka streams

2023-02-03 Thread Amir Hossein Sharifzadeh
re for more detail[1]. > [1] > https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/tableapi/#joins > > Best regards, > Yuxia > > - 原始邮件 - > 发件人: "Amir Hossein Sharifzadeh" > 收件人: "dev" > 发送时间: 星期五, 2023年 2 月 03日 上午 4:45:

Re: Need help how to use Table API to join two Kafka streams

2023-02-03 Thread yuxia
uot;ssn" === $"ssn1"); Also, you can refer here for more detail[1]. [1] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/tableapi/#joins Best regards, Yuxia - 原始邮件 - 发件人: "Amir Hossein Sharifzadeh" 收件人: "dev" 发送时间: 星期五, 2023年 2 月 03日 上午 4:45:08

Need help how to use Table API to join two Kafka streams

2023-02-02 Thread Amir Hossein Sharifzadeh
Hello, I have a Kafka producer and a Kafka consumer that produces and consumes multiple data respectively. You can think of two data sets here. Both datasets have a similar structure but carry different data. I want to implement a Table API to join two Kafka streams while I consume them. For exam