Hi Chris,
Thank you for highlighting the scenario. I tested it using the JSON below, and
you're right we will need to include escape sequences in the schema.content for
the configuration. Although this makes the schema look a bit complex, we can
always provide it in as a file if needed. I have updated the KIP with the
details.
{
"name": "api-file-source-json",
"config": {
"value.converter.schemas.enable": true,
"value.converter.schema.content": "{\"type\": \"struct\", \"fields\": [{
\"field\": \"id\", \"type\": \"string\", \"optional\": false },{ \"field\":
\"name\", \"type\": \"string\", \"optional\": false }]}"
}
}
Thanks,
Priyanka.
From: Chris Egerton <fearthecel...@gmail.com>
Date: Wednesday, 10 July 2024 at 9:27 PM
To: dev@kafka.apache.org <dev@kafka.apache.org>
Subject: [EXTERNAL] Re: [VOTE] KIP-1054: Support external schemas in
JSONConverter
Hi Priyanka,
The issue is that connector configurations can also be submitted via the
REST API as JSON objects, and if a user tries to embed a schema directly in
that configuration as a string, it'll be fairly ugly. This does not apply
if using a properties file (in standalone mode) or the config provider
mechanism, which is why I think your testing may have missed this scenario.
If you still think that's not the case, can you share the exact connector
configurations you tested with, and instructions for building the JSON
converter with your proposed changes (can be as simple as "check out this
git branch from my repo").
Cheers,
Chris
On Wed, Jul 10, 2024, 11:36 Priyanka K U <priyanka....@ibm.com.invalid>
wrote:
> Hi Chris,
>
> We did the testing using the JAR that supports external schema
> functionality. This included specifying the external schema file in the
> Kafka properties file,
> as well as embedding an inline schema directly within the connector
> properties file .
>
> Thanks,
> Priyanka
>
> From: Chris Egerton <fearthecel...@gmail.com>
> Date: Friday, 5 July 2024 at 5:55 PM
> To: dev@kafka.apache.org <dev@kafka.apache.org>
> Subject: [EXTERNAL] Re: [VOTE] KIP-1054: Support external schemas in
> JSONConverter
> Hi Priyanka,
>
> How exactly did you test this feature? Was it in standalone mode (with a
> Java properties file), or with a JSON config submitted via the REST API?
>
> Cheers,
>
> Chris
>
>
