[ https://issues.apache.org/jira/browse/SPARK-24741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16536907#comment-16536907 ]
Marek Novotny commented on SPARK-24741: --------------------------------------- It would be nice if the build-in data source was interoperable with Structured Streaming and schema registries. (Maybe like [ABRiS|https://github.com/AbsaOSS/ABRiS] does?) cc [~felipesmmelo] > Have a built-in AVRO data source implementation > ----------------------------------------------- > > Key: SPARK-24741 > URL: https://issues.apache.org/jira/browse/SPARK-24741 > Project: Spark > Issue Type: New Feature > Components: SQL, Structured Streaming > Affects Versions: 2.4.0 > Reporter: Xiao Li > Priority: Major > > Apache Avro (https://avro.apache.org) is a data serialization format. It is > widely used in the Spark and Hadoop ecosystem, especially for Kafka-based > data pipelines. Using Spark-Avro package > (https://github.com/databricks/spark-avro), Spark SQL can read and write the > avro data. Making spark-Avro built-in can provide a better experience for > first-time users of Spark SQL and structured streaming. We expect the > built-in Avro data source can further improve the adoption of structured > streaming. We should consider inlining > https://github.com/databricks/spark-avro. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org