[ https://issues.apache.org/jira/browse/SPARK-20432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Burak Yavuz closed SPARK-20432. ------------------------------- Resolution: Duplicate > Unioning two identical Streaming DataFrames fails during attribute resolution > ----------------------------------------------------------------------------- > > Key: SPARK-20432 > URL: https://issues.apache.org/jira/browse/SPARK-20432 > Project: Spark > Issue Type: Bug > Components: Structured Streaming > Affects Versions: 2.1.0 > Reporter: Burak Yavuz > > To reproduce, try unioning two identical Kafka Streams: > {code} > df = spark.readStream.format("kafka")... \ > .select(from_json(col("value").cast("string"), > simpleSchema).alias("parsed_value")) > df.union(df).writeStream... > {code} > Exception is confusing: > {code} > org.apache.spark.sql.AnalysisException: resolved attribute(s) value#526 > missing from > value#511,topic#512,partition#513,offset#514L,timestampType#516,key#510,timestamp#515 > in operator !Project [jsontostructs(...) AS parsed_value#357]; > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org