[ https://issues.apache.org/jira/browse/SPARK-24741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16543372#comment-16543372 ]
Xiao Li commented on SPARK-24741: --------------------------------- Please follow the umbrella JIRA: https://issues.apache.org/jira/browse/SPARK-24768. There are multiple tasks that are related to your scenario. > Have a built-in AVRO data source implementation > ----------------------------------------------- > > Key: SPARK-24741 > URL: https://issues.apache.org/jira/browse/SPARK-24741 > Project: Spark > Issue Type: New Feature > Components: SQL, Structured Streaming > Affects Versions: 2.4.0 > Reporter: Xiao Li > Assignee: Gengliang Wang > Priority: Major > > Apache Avro (https://avro.apache.org) is a data serialization format. It is > widely used in the Spark and Hadoop ecosystem, especially for Kafka-based > data pipelines. Using Spark-Avro package > (https://github.com/databricks/spark-avro), Spark SQL can read and write the > avro data. Making spark-Avro built-in can provide a better experience for > first-time users of Spark SQL and structured streaming. We expect the > built-in Avro data source can further improve the adoption of structured > streaming. We should consider inlining > https://github.com/databricks/spark-avro. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org