zhaomin1423 commented on PR #43371:
URL: https://github.com/apache/spark/pull/43371#issuecomment-1767747660
Thank you for your review and providing valuable suggestions. @mridulm
@LuciferYang @beliefer
--
This is an automated message from the Apache Git Service.
To respond to the
mridulm commented on PR #43371:
URL: https://github.com/apache/spark/pull/43371#issuecomment-1767743867
Merged to master.
Thanks for working on this @zhaomin1423 !
Thanks for the reviews @LuciferYang, @beliefer :-)
--
This is an automated message from the Apache Git Service.
To r
mridulm closed pull request #43371: [SPARK-45534][CORE] Use
java.lang.ref.Cleaner instead of finalize for RemoteBlockPushResolver
URL: https://github.com/apache/spark/pull/43371
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
zhaomin1423 commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1361416337
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -94,8 +95,8 @@
*/
public class RemoteBlockPushResolv
beliefer commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1361412044
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -94,8 +95,8 @@
*/
public class RemoteBlockPushResolver
zhaomin1423 commented on PR #43371:
URL: https://github.com/apache/spark/pull/43371#issuecomment-1764897751
> A few comments, mostly looks good.
All has been modified. Thank you very much for your comments.
--
This is an automated message from the Apache Git Service.
To respond to t
zhaomin1423 commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360968020
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1933,6 +1927,54 @@ public RoaringBitmap getMapTracker(
zhaomin1423 commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360966656
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1864,35 +1871,27 @@ private void finalizePartition() t
zhaomin1423 commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360966112
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1864,35 +1871,27 @@ private void finalizePartition() t
zhaomin1423 commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360965339
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1864,35 +1871,27 @@ private void finalizePartition() t
zhaomin1423 commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360964693
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1710,6 +1712,8 @@ public String toString() {
/**
mridulm commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360959374
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1864,35 +1871,27 @@ private void finalizePartition() throw
zhaomin1423 commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360953142
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1864,35 +1871,27 @@ private void finalizePartition() t
mridulm commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360941537
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1864,35 +1871,27 @@ private void finalizePartition() throw
mridulm commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360947933
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1933,6 +1927,54 @@ public RoaringBitmap getMapTracker() {
mridulm commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360946397
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1864,35 +1871,27 @@ private void finalizePartition() throw
mridulm commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360945069
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1864,35 +1871,27 @@ private void finalizePartition() throw
mridulm commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360941537
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1864,35 +1871,27 @@ private void finalizePartition() throw
mridulm commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360937228
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1710,6 +1712,8 @@ public String toString() {
/** Meta
zhaomin1423 commented on PR #43371:
URL: https://github.com/apache/spark/pull/43371#issuecomment-1764695641
There are some unrelated fail, I push again without making any changes after
rebase master.
--
This is an automated message from the Apache Git Service.
To respond to the message, p
beliefer commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360573952
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1933,6 +1927,49 @@ public RoaringBitmap getMapTracker() {
LuciferYang commented on PR #43371:
URL: https://github.com/apache/spark/pull/43371#issuecomment-1764359088
The change is fine to me. Could you take another look? @mridulm @beliefer
Thanks ~
--
This is an automated message from the Apache Git Service.
To respond to the message, please lo
zhaomin1423 commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360568448
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1887,33 +1881,45 @@ static void closeAllFilesAndDelete
LuciferYang commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360562956
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1864,34 +1871,26 @@ private void finalizePartition() t
beliefer commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360563141
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -827,9 +824,7 @@ public MergeStatuses
finalizeShuffleMerg
zhaomin1423 commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360558718
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1864,34 +1871,26 @@ private void finalizePartition() t
LuciferYang commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360526040
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1864,34 +1871,26 @@ private void finalizePartition() t
LuciferYang commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360524390
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1887,33 +1881,45 @@ static void closeAllFilesAndDelete
LuciferYang commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360523564
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -827,9 +824,7 @@ public MergeStatuses
finalizeShuffleM
beliefer commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360518425
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1887,33 +1881,45 @@ static void closeAllFilesAndDeleteIfN
zhaomin1423 commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360503923
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1933,6 +1924,49 @@ public RoaringBitmap getMapTracker(
zhaomin1423 commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360503697
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1864,34 +1871,23 @@ private void finalizePartition() t
LuciferYang commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360481397
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1933,6 +1924,49 @@ public RoaringBitmap getMapTracker(
zhaomin1423 commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360436665
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -537,7 +540,9 @@ void closeAndDeleteOutdatedPartitions(
zhaomin1423 commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360435276
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1734,6 +1735,8 @@ public static class AppShufflePartit
LuciferYang commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360177836
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -537,7 +540,9 @@ void closeAndDeleteOutdatedPartitions(
LuciferYang commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360165594
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -537,7 +540,9 @@ void closeAndDeleteOutdatedPartitions(
LuciferYang commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360165920
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1734,6 +1735,8 @@ public static class AppShufflePartit
LuciferYang commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360165594
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -537,7 +540,9 @@ void closeAndDeleteOutdatedPartitions(
beliefer commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360150033
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -481,7 +482,9 @@ void closeAndDeletePartitionsIfNeeded(
beliefer commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360148902
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1933,6 +1946,29 @@ public RoaringBitmap getMapTracker() {
mridulm commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360130919
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1864,7 +1875,14 @@ private void finalizePartition() throws
mridulm commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360129540
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -481,7 +482,9 @@ void closeAndDeletePartitionsIfNeeded(
beliefer commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360085452
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1933,6 +1946,29 @@ public RoaringBitmap getMapTracker() {
zhaomin1423 commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360016315
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1933,6 +1946,29 @@ public RoaringBitmap getMapTracker(
beliefer commented on code in PR #43371:
URL: https://github.com/apache/spark/pull/43371#discussion_r1360011335
##
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##
@@ -1933,6 +1946,29 @@ public RoaringBitmap getMapTracker() {
46 matches
Mail list logo
