Anton Okolnychyi created SPARK-54157:
----------------------------------------
Summary: Fix refresh of DSv2 tables between Dataset executions
Key: SPARK-54157
URL: https://issues.apache.org/jira/browse/SPARK-54157
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.1.0
Reporter: Anton Okolnychyi
We must fix refresh of DSv2 tables between Dataset executions.
val df = spark.sql("SELECT * FROM t") // resolves to v1
df.show() // outputs v1
spark.sql("INSERT INTO t ... VALUES (...)") // move table to v2
df.show() // should output v2 and not v1 as today
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
