[GitHub] [hudi] vinothchandar commented on pull request #3093: [HUDI-2031] JVM occasionally crashes during compaction when spark speculative execution is enabled
vinothchandar commented on pull request #3093: URL: https://github.com/apache/hudi/pull/3093#issuecomment-864728997 Generally speaking, I am bit skeptical on overall use of speculative execution. Most of the policies are pretty naive. Anyways, @garyli1019 Would love to get a JIRA with some steps to repro if possible. The writer is already resistant to stage retries, so it should no cause such dirty/uncommitted files lying around. Do you have the state of marker files when this happened? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] vinothchandar commented on pull request #3093: [HUDI-2031] JVM occasionally crashes during compaction when spark speculative execution is enabled
vinothchandar commented on pull request #3093: URL: https://github.com/apache/hudi/pull/3093#issuecomment-864355908 Change looks reasonable. let me think bit more around the scenarios before merging this. Anyway we can write a unit test around this scenario? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
