[jira] [Updated] (SPARK-42252) Deprecate spark.shuffle.unsafe.file.output.buffer and add a new config
[ https://issues.apache.org/jira/browse/SPARK-42252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-42252: --- Labels: pull-request-available (was: ) > Deprecate spark.shuffle.unsafe.file.output.buffer and add a new config > -- > > Key: SPARK-42252 > URL: https://issues.apache.org/jira/browse/SPARK-42252 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 3.0.0, 3.1.0, 3.2.0, 3.3.0, 3.4.0 >Reporter: Wei Guo >Priority: Minor > Labels: pull-request-available > > After Jira SPARK-28209 and PR > [25007|[https://github.com/apache/spark/pull/25007]], the new shuffle writer > api is proposed. All shuffle writers(BypassMergeSortShuffleWriter, > SortShuffleWriter, UnsafeShuffleWriter) are based on > LocalDiskShuffleMapOutputWriter to write local disk shuffle files. The config > spark.shuffle.unsafe.file.output.buffer used in > LocalDiskShuffleMapOutputWriter was only used in UnsafeShuffleWriter before. > > It's better to rename it and make it more suitable. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-42252) Deprecate spark.shuffle.unsafe.file.output.buffer and add a new config
[ https://issues.apache.org/jira/browse/SPARK-42252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42252: - Fix Version/s: (was: 3.5.0) > Deprecate spark.shuffle.unsafe.file.output.buffer and add a new config > -- > > Key: SPARK-42252 > URL: https://issues.apache.org/jira/browse/SPARK-42252 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 3.0.0, 3.1.0, 3.2.0, 3.3.0, 3.4.0 >Reporter: Wei Guo >Priority: Minor > > After Jira SPARK-28209 and PR > [25007|[https://github.com/apache/spark/pull/25007]], the new shuffle writer > api is proposed. All shuffle writers(BypassMergeSortShuffleWriter, > SortShuffleWriter, UnsafeShuffleWriter) are based on > LocalDiskShuffleMapOutputWriter to write local disk shuffle files. The config > spark.shuffle.unsafe.file.output.buffer used in > LocalDiskShuffleMapOutputWriter was only used in UnsafeShuffleWriter before. > > It's better to rename it and make it more suitable. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-42252) Deprecate spark.shuffle.unsafe.file.output.buffer and add a new config
[ https://issues.apache.org/jira/browse/SPARK-42252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42252: - Target Version/s: (was: 3.5.0) > Deprecate spark.shuffle.unsafe.file.output.buffer and add a new config > -- > > Key: SPARK-42252 > URL: https://issues.apache.org/jira/browse/SPARK-42252 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 3.0.0, 3.1.0, 3.2.0, 3.3.0, 3.4.0 >Reporter: Wei Guo >Priority: Minor > > After Jira SPARK-28209 and PR > [25007|[https://github.com/apache/spark/pull/25007]], the new shuffle writer > api is proposed. All shuffle writers(BypassMergeSortShuffleWriter, > SortShuffleWriter, UnsafeShuffleWriter) are based on > LocalDiskShuffleMapOutputWriter to write local disk shuffle files. The config > spark.shuffle.unsafe.file.output.buffer used in > LocalDiskShuffleMapOutputWriter was only used in UnsafeShuffleWriter before. > > It's better to rename it and make it more suitable. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-42252) Deprecate spark.shuffle.unsafe.file.output.buffer and add a new config
[ https://issues.apache.org/jira/browse/SPARK-42252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Guo updated SPARK-42252: Target Version/s: 3.5.0 (was: 3.4.0) > Deprecate spark.shuffle.unsafe.file.output.buffer and add a new config > -- > > Key: SPARK-42252 > URL: https://issues.apache.org/jira/browse/SPARK-42252 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 3.0.0 >Reporter: Wei Guo >Priority: Minor > Fix For: 3.4.0 > > > After Jira SPARK-28209 and PR > [25007|[https://github.com/apache/spark/pull/25007]], the new shuffle writer > api is proposed. All shuffle writers(BypassMergeSortShuffleWriter, > SortShuffleWriter, UnsafeShuffleWriter) are based on > LocalDiskShuffleMapOutputWriter to write local disk shuffle files. The config > spark.shuffle.unsafe.file.output.buffer used in > LocalDiskShuffleMapOutputWriter was only used in UnsafeShuffleWriter before. > > It's better to rename it and make it more suitable. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-42252) Deprecate spark.shuffle.unsafe.file.output.buffer and add a new config
[ https://issues.apache.org/jira/browse/SPARK-42252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Guo updated SPARK-42252: Affects Version/s: 3.3.0 3.2.0 3.1.0 3.4.0 > Deprecate spark.shuffle.unsafe.file.output.buffer and add a new config > -- > > Key: SPARK-42252 > URL: https://issues.apache.org/jira/browse/SPARK-42252 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 3.0.0, 3.1.0, 3.2.0, 3.3.0, 3.4.0 >Reporter: Wei Guo >Priority: Minor > Fix For: 3.5.0 > > > After Jira SPARK-28209 and PR > [25007|[https://github.com/apache/spark/pull/25007]], the new shuffle writer > api is proposed. All shuffle writers(BypassMergeSortShuffleWriter, > SortShuffleWriter, UnsafeShuffleWriter) are based on > LocalDiskShuffleMapOutputWriter to write local disk shuffle files. The config > spark.shuffle.unsafe.file.output.buffer used in > LocalDiskShuffleMapOutputWriter was only used in UnsafeShuffleWriter before. > > It's better to rename it and make it more suitable. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-42252) Deprecate spark.shuffle.unsafe.file.output.buffer and add a new config
[ https://issues.apache.org/jira/browse/SPARK-42252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Guo updated SPARK-42252: Fix Version/s: 3.5.0 (was: 3.4.0) > Deprecate spark.shuffle.unsafe.file.output.buffer and add a new config > -- > > Key: SPARK-42252 > URL: https://issues.apache.org/jira/browse/SPARK-42252 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 3.0.0 >Reporter: Wei Guo >Priority: Minor > Fix For: 3.5.0 > > > After Jira SPARK-28209 and PR > [25007|[https://github.com/apache/spark/pull/25007]], the new shuffle writer > api is proposed. All shuffle writers(BypassMergeSortShuffleWriter, > SortShuffleWriter, UnsafeShuffleWriter) are based on > LocalDiskShuffleMapOutputWriter to write local disk shuffle files. The config > spark.shuffle.unsafe.file.output.buffer used in > LocalDiskShuffleMapOutputWriter was only used in UnsafeShuffleWriter before. > > It's better to rename it and make it more suitable. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
