Hi team,
We encountered the schema forbidden issue during deployment with the
changes of flink version upgrade (1.16.1 -> 1.17.1), hope to get some help
here :pray:.
In the changes of upgrade flink version from 1.16.1 to 1.17.1, we also
switched from our customized schema-registry dependency to
org.apache.flink:flink-avro-confluent-registry:1.17.1 . Previously we never
encountered this issue using our customized schema-registry, but after the
changes, the error in the job was like this:
97112e6_4e578e017158b6c826cf089a3d845171_10_5) switched from RUNNING
to FAILED on sessionizer-job-taskmanager-1-1 @ 192.168.71.13
(dataPort=40879).
java.io.IOException: Could not register schema in registry
at
org.apache.flink.formats.avro.registry.confluent.ConfluentSchemaRegistryCoder.writeSchema(ConfluentSchemaRegistryCoder.java:90)
~[?:?]
at
org.apache.flink.formats.avro.RegistryAvroSerializationSchema.serialize(RegistryAvroSerializationSchema.java:85)
~[?:?]
...
Caused by:
io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException:
Forbidden; error code: 403
at
io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:301)
~[?:?]
at
io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:371)
~[?:?]
...
at
org.apache.flink.formats.avro.registry.confluent.ConfluentSchemaRegistryCoder.writeSchema(ConfluentSchemaRegistryCoder.java:85)
~[?:?]
And the job keeps switching from RUNNING to CANCELING to CANCELED.
We checked the schema of the kafka topic we are producing to, it's
owned by other teams, not us. And the job just kept auto registering
this target schema, which cannot be disabled by setting
auto.register.schemas to false , that's why it said Forbidden.
We cannot change the owner or add another owner on that schema. With
this limitation, is there any way to avoid this issue while switching
to 1.17.1 with the
org.apache.flink:flink-avro-confluent-registry:1.17.1?
Thanks!
Lijuan
