hudi-bot opened a new issue, #17219: URL: https://github.com/apache/hudi/issues/17219
For Spark structured streaming, after the end of first micro-batch, the write client is closed and hence triggers closure of timeline service. But subsequent micro-batches do succeed though. Currently, we explicitly override the marker type to be DIRECT in Spark structured streaming. We need to revisit the behavior of timeline service before we can enable timeline-server-based markers for Spark structured streaming. ## JIRA info - Link: https://issues.apache.org/jira/browse/HUDI-2865 - Type: Sub-task - Parent: https://issues.apache.org/jira/browse/HUDI-2271 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
