[ https://issues.apache.org/jira/browse/SPARK-19414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847574#comment-15847574 ]
Shixiong Zhu commented on SPARK-19414: -------------------------------------- Oh, I see, You cannot infer schema when getting the DataFrame. The schema must be provided before running the streaming query as SQL needs it to verify the query. > Inferring schema in a structured streaming source > ------------------------------------------------- > > Key: SPARK-19414 > URL: https://issues.apache.org/jira/browse/SPARK-19414 > Project: Spark > Issue Type: Question > Components: Structured Streaming > Reporter: sam elamin > > Hi All > I am writing a connector to BigQuery that uses structured streaming, my > question is about schemas > I would like to be able to infer the schema from BQ rather than pass it in, > is there any way to overwrite the source schema in anything that extends > org.apache.spark.sql.execution.streaming.Source > Overriding the schema method doesnt seem to work -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org