[
https://issues.apache.org/jira/browse/FLINK-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16549063#comment-16549063
]
Fabian Hueske commented on FLINK-9813:
--------------------------------------
Ah, OK I see. That makes total sense.
I agree that Avro schemas are widely used. It would be nice to generate CSV
schemas from that. However, we would need to check to what extend they are
compatible, i.e., which types are supported by CSV but not Avro and vice versa,
e.g., nested Avro types won't be supported.
We are also currently in the process of reworking how TableSources are defined,
separating connectors (FileSystem, Kafka, etc), Formats (Csv, Avro, ORC,
Parquet), and table schemas. There is already the
[Csv|https://github.com/apache/flink/blob/master/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/descriptors/Csv.scala]
format descriptor which could be extended by a {{Csv.avroSchema(String)}}
method to parse an Avro schema.
What do you think [~twalthr] [~flacombe]?
> Build xTableSource from Avro schemas
> ------------------------------------
>
> Key: FLINK-9813
> URL: https://issues.apache.org/jira/browse/FLINK-9813
> Project: Flink
> Issue Type: Wish
> Components: Table API & SQL
> Affects Versions: 1.5.0
> Reporter: François Lacombe
> Priority: Trivial
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> As Avro provide efficient data schemas formalism, it may be great to be able
> to build Flink Tables Sources with such files.
> More info about Avro schemas
> :[https://avro.apache.org/docs/1.8.1/spec.html#schemas]
> For instance, with CsvTableSource :
> Parser schemaParser = new Schema.Parser();
> Schema tableSchema = schemaParser.parse("avro.json");
> Builder bld = CsvTableSource.builder().schema(tableSchema);
>
> This would give me a fully available CsvTableSource with columns defined in
> avro.json
> It may be possible to do so for every TableSources since avro format is
> really common and versatile.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
