Flink app reading parquet files from s3 and sink to kafka topics.

2024-01-09 Thread Lijuan Hou
Hi team, can I get some help and suggestions on how to resolve the following issue? Thank you in advance! I am implementing an app to read parquet files from s3, and sink to kafka topics. For s3 schema, I am using schema avsc files with header outside of topic schema content. I have made sure that

Could not register schema - Forbidden error when upgrading flink from 1.16.1 to 1.17.1

2023-10-23 Thread Lijuan Hou
Hi team, We encountered the schema forbidden issue during deployment with the changes of flink version upgrade (1.16.1 -> 1.17.1), hope to get some help here :pray:. In the changes of upgrade flink version from 1.16.1 to 1.17.1, we also switched from our customized schema-registry dependency to

Flink version upgrade issues (schema register forbidden error + incorrect kafka offset issue)

2023-10-20 Thread Lijuan Hou
Hi team, we encountered the schema forbidden issue during deployment with the changes of flink version upgrade (1.16.1 -> 1.17.1), and also encountered incorrect reset kafka offset issue during restoring our flink job from this change. Hope to get some help here. *1 - schema forbidden issue* In