wombatu-kun commented on code in PR #18975:
URL: https://github.com/apache/hudi/pull/18975#discussion_r4573555063


##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala:
##########
@@ -1338,6 +1333,45 @@ class HoodieSparkSqlWriterInternal {
 }
 
 object HoodieSparkSqlWriterInternal {
+  private val log = 
LoggerFactory.getLogger(classOf[HoodieSparkSqlWriterInternal])
+
+  /**
+   * Best-effort invalidation of the Spark catalog relation cache for the 
just-synced table(s), so
+   * subsequent reads in the same Spark session reflect the new write.
+   *
+   * The table name is always qualified with the sync database 
([[HIVE_DATABASE]] /
+   * `hoodie.datasource.hive_sync.database`) so a same-named table in another 
database (in
+   * particular the session's current/`default` database) is never resolved 
and refreshed by
+   * mistake.
+   *
+   * Any failure is logged and swallowed: by this point the data has already 
been committed and
+   * meta-synced successfully, so a cache-invalidation problem (a transient 
catalog error, or a
+   * same-named table backed by storage the writer cannot access) must never 
fail the write.
+   */
+  def refreshSparkCatalogTableCache(spark: SparkSession, syncDb: String, 
tableNames: Seq[String]): Unit = {

Review Comment:
   The AccessDenied in #18139 is thrown by 
`HoodieFileIndex.getConfigProperties` looking up `<current 
db>.<hoodie.table.name>` while the correctly qualified sync-db table is being 
resolved, and on master that lookup already runs only for reads that declare a 
bucket index and is best-effort. Is this meant as extra hardening on top of 
that, and if so could the description stop attributing #18139 to the refresh 
resolving the `default` table?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to