Are metadata columns required to get declared in the table's schema?

2023-03-25 Thread Jie Han
Hi community, I want to query a metadata column from my table t. Do I need to declare it in the table schema explicitly? In spark, metadata columns are hidden columns, which means we don’t need to declare it in the table ddl, we only explicitly reference it in our query. For instance, select *

Re: Table API function and expression vs SQL

2023-03-25 Thread Mate Czagany
Hi, Please also keep in mind that restoring existing Table API jobs from savepoints when upgrading to a newer minor version of Flink, e.g. 1.16 -> 1.17 is not supported as the topology might change between these versions due to optimizer changes. See here for more information: https://nightlies.a

Re: Table API function and expression vs SQL

2023-03-25 Thread Hang Ruan
Hi, I think the SQL job is better. Flink SQL jobs can be easily shared with others for debugging. And it is more suitable for flow batch integration. For a small part of jobs which can not be expressed through SQL, we will choose a job by DataStream API. Best, Hang ravi_suryavanshi.yahoo.com via

Re: Flink 1.17 upgrade issue when using azure storage account for checkpoints/savepoints

2023-03-25 Thread 陳昌倬
On Sat, Mar 25, 2023 at 02:01:24PM +0530, Jessy Ping wrote: > Root cause: Caused by: java.util.concurrent.CompletionException: > java.lang.RuntimeException: java.lang.ClassNotFoundException: Class > org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback not found We have similar error

Flink 1.17 upgrade issue when using azure storage account for checkpoints/savepoints

2023-03-25 Thread Jessy Ping
Hi Team. The application failed to start after upgrading from Flink 1.16.1 to 1.17.0 in both kubernetes and docker. I didn't make any changes in flink configurations related to savepoints and checkpoints. Root cause: Caused by: java.util.concurrent.CompletionException: java.lang.RuntimeExcepti

Re: [ANNOUNCE] Apache Flink 1.17.0 released

2023-03-25 Thread Hang Ruan
Thanks for the great work ! Congrats all! Best, Hang Panagiotis Garefalakis 于2023年3月25日周六 03:22写道: > Congrats all! Well done! > > Cheers, > Panagiotis > > On Fri, Mar 24, 2023 at 2:46 AM Qingsheng Ren wrote: > > > I'd like to say thank you to all contributors of Flink 1.17. Your support > > an