Gabor Somogyi created SPARK-29876: ------------------------------------- Summary: Delete/archive file source data in separate thread Key: SPARK-29876 URL: https://issues.apache.org/jira/browse/SPARK-29876 Project: Spark Issue Type: Improvement Components: Structured Streaming Affects Versions: 3.0.0 Reporter: Gabor Somogyi
SPARK-20568 added the possibility to clean up completed files in streaming query. Deleting/archiving uses the main thread which can slow down processing. It would be good to do this on separate thread(s). -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org