Siying Dong created SPARK-45504: ----------------------------------- Summary: RocksDB State Store Should Lower RocksDB Background Thread CPU Priority Key: SPARK-45504 URL: https://issues.apache.org/jira/browse/SPARK-45504 Project: Spark Issue Type: Task Components: Structured Streaming Affects Versions: 3.5.1 Reporter: Siying Dong
We can move RocksDB flush and compaction to lower CPU priority. They usually are background tasks and don't need to compete with task execution. For the case where a task may wait for some RocksDB background task to finish, such as checkpointing, or waiting async checkpointing to finish, the task slot is waiting so we are likely to have enough CPU for low pri CPU anyway. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org