[GitHub] [spark] holdenk commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-09 Thread GitBox
holdenk commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-776142652 Merged to the current dev branch This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] holdenk commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-09 Thread GitBox
holdenk commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-776140962 Actually, I think I'll just merge as is. If we decide to change the documentation we can do that in a follow up.

[GitHub] [spark] holdenk commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-09 Thread GitBox
holdenk commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-776129436 K8s failure is unrelated. If there are no more comments I'll merge this soon. This is an automated message

[GitHub] [spark] holdenk commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
holdenk commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774316948 I think the Kafka failures are unrelated. This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] holdenk commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
holdenk commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774275909 > In general, this mitigation cannot avoid the disk full situation fundamentally. Can we choose the migration target in a better way instead of this rejecting because this is

[GitHub] [spark] holdenk commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
holdenk commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774215330 cc @dongjoon-hyun This is an automated message from the Apache Git Service. To respond to the message, please