danny0405 commented on issue #5553: URL: https://github.com/apache/hudi/issues/5553#issuecomment-1134270047
Have fired a fix for flink here: https://github.com/apache/hudi/pull/5660 https://issues.apache.org/jira/browse/HUDI-3782 and https://issues.apache.org/jira/browse/HUDI-4138 In general, we should never make any side effect in the read code path of hoodie table config. And hoodie table metadata writer. I'm not sure how to fix this in Spark side, have two ways to fix on my mind: 1. make table config concurrency safe (not suggested because it is too heavy for a config) 2. make sure the metadata cleaning only happens once for the whole Job lifetime (still risky because there may be multiple jobs, but with very small probability). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
