Re: DataStream.keyBy() with keys determined at run time

2022-07-10 Thread yidan zhao
You can use string, and serialize all keys to a string. Hemanga Borah 于2022年7月11日周一 09:49写道: > > Here is the documentation of the Tuple class: > https://nightlies.apache.org/flink/flink-docs-master/api/java/org/apache/flink/api/java/tuple/Tuple.html > > If you need a concrete class, you can go f

Re: DataStream.keyBy() with keys determined at run time

2022-07-10 Thread Hemanga Borah
Here is the documentation of the Tuple class: https://nightlies.apache.org/flink/flink-docs-master/api/java/org/apache/flink/api/java/tuple/Tuple.html If you need a concrete class, you can go from Tuple0 to Tuple25. On Sun, Jul 10, 2022 at 5:43 PM Thomas Wang wrote: > I didn't copy the exact er

Re: How can I convert a DataSet into a Table?

2022-07-10 Thread yuxia
I'm afraid we have no way to do such conversion in Flink 1.15. But for you case, which is to read from csv file in table api. You can try as follows: tableEnv.createTemporaryTable(" csvInput ", TableDescriptor.forConnector("filesystem") .schema(schema) .option("path", "/path/to/file") .for

Re: Does Table API connector, csv, has some option to ignore some columns

2022-07-10 Thread Shengkai Fang
Hi. In Flink SQL, you can select the column that you wants in the query. For example, you can use ``` SELECT col_a, col_b FROM some_table; ``` Best, Shengkai 于2022年7月9日周六 01:48写道: > Does Table API connector, CSV, has some option to ignore some columns in > source file? > For instance read on

Re: DataStream.keyBy() with keys determined at run time

2022-07-10 Thread Thomas Wang
I didn't copy the exact error message, but basically the idea of the error message is that I cannot use the abstract class Tuple and instead, I should use Tuple1, Tuple2 and etc. Thomas On Sun, Jul 10, 2022 at 12:47 PM Hemanga Borah wrote: > What error do you see? > > On Sun, Jul 10, 2022 at 6:

Re: DataStream.keyBy() with keys determined at run time

2022-07-10 Thread Hemanga Borah
What error do you see? On Sun, Jul 10, 2022 at 6:30 AM Thomas Wang wrote: > Hi, > > I have a use case where I need to call DataStream.keyBy() with keys loaded > from a configuration. The number of keys and their data types are variables > and is determined by the configuration. Once the configur

Re: [ANNOUNCE] Apache Flink 1.15.1 released

2022-07-10 Thread podunk
I run Flink in Windows and in version 1.15.1 Task Managers fails to start. Works without problems in 1.14.5     Sent: Friday, July 08, 2022 at 12:18 AM From: "David Anderson" To: "dev" , "user" , "user-zh" , annou...@apache.org Subject: [ANNOUNCE] Apache Flink 1.15.1 released   The Apa

DataStream.keyBy() with keys determined at run time

2022-07-10 Thread Thomas Wang
Hi, I have a use case where I need to call DataStream.keyBy() with keys loaded from a configuration. The number of keys and their data types are variables and is determined by the configuration. Once the configuration is loaded, they won't change. I'm trying to use the following key selector, but

Issues with watermark alignment in Flink 1.15

2022-07-10 Thread Jun Qin
Hi All I am trying watermark alignment in Flink 1.15 with: watermarkStrategy = WatermarkStrategy.<~>forBoundedOutOfOrderness( Duration.ofMillis(outOfOrderness)) .withWatermarkAlignment("wm-group", Duration.ofSeconds(10), Duration.ofSeconds(1)) .withTimestampAssign