Thanks Russell. This <https://gite.lirmm.fr/yagoubi/spark/commit/6463e0b9e8067cce70602c5c9006a2546856a9d6#fecff1a3ad108a52192ba9cd6dd7b11a3d18871b_0_141> shows that the "refreshTable" and "invalidateTable" could be used to reload the metadata but they do not work in our case. I have tried to invoke the "schema()" with the updated schema from the "buildScan()" as well.
It will be helpful to have this feature enabled for DataSourceV1 as the schema evolves, i will check if this is an change that can be made. You mentioned that it works in DataSourceV2. Is there an implementation sample for persistent tables DataSourceV2 that works with spark 2.4.4? Thanks again. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
