[
https://issues.apache.org/jira/browse/SPARK-19414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847571#comment-15847571
]
Shixiong Zhu commented on SPARK-19414:
--------------------------------------
[~samelamin] I would suggest that you take a look at the Kafka source:
https://github.com/apache/spark/tree/master/external/kafka-0-10-sql
The magic part is this file:
https://github.com/apache/spark/blob/master/external/kafka-0-10-sql/src/main/resources/META-INF/services/org.apache.spark.sql.sources.DataSourceRegister
> Inferring schema in a structured streaming source
> -------------------------------------------------
>
> Key: SPARK-19414
> URL: https://issues.apache.org/jira/browse/SPARK-19414
> Project: Spark
> Issue Type: Question
> Components: Structured Streaming
> Reporter: sam elamin
>
> Hi All
> I am writing a connector to BigQuery that uses structured streaming, my
> question is about schemas
> I would like to be able to infer the schema from BQ rather than pass it in,
> is there any way to overwrite the source schema in anything that extends
> org.apache.spark.sql.execution.streaming.Source
> Overriding the schema method doesnt seem to work
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
