spark git commit: [SPARK-17850][CORE] Add a flag to ignore corrupt files

Repository: spark Updated Branches: refs/heads/master eb69335cd -> 47776e7c0 [SPARK-17850][CORE] Add a flag to ignore corrupt files ## What changes were proposed in this pull request? Add a flag to ignore corrupt files. For Spark core, the configuration is `spark.files.ignoreCorruptFiles`. F

spark git commit: [SPARK-17850][CORE] Add a flag to ignore corrupt files (branch 1.6)

Repository: spark Updated Branches: refs/heads/branch-1.6 d3890deb7 -> 585c5657f [SPARK-17850][CORE] Add a flag to ignore corrupt files (branch 1.6) ## What changes were proposed in this pull request? This is the patch for 1.6. It only adds Spark conf `spark.files.ignoreCorruptFiles` because

spark git commit: [SPARK-17637][SCHEDULER] Packed scheduling for Spark tasks across executors

Repository: spark Updated Branches: refs/heads/master 36d81c2c6 -> ed1463341 [SPARK-17637][SCHEDULER] Packed scheduling for Spark tasks across executors ## What changes were proposed in this pull request? Restructure the code and implement two new task assigner. PackedAssigner: try to allocat

spark git commit: [SPARK-14914][CORE] Fix Resource not closed after using, mostly for unit tests

Repository: spark Updated Branches: refs/heads/master 0d95662e7 -> 8f0ea011a [SPARK-14914][CORE] Fix Resource not closed after using, mostly for unit tests ## What changes were proposed in this pull request? Close `FileStreams`, `ZipFiles` etc to release the resources after using. Not closin

spark git commit: [SPARK-16759][CORE] Add a configuration property to pass caller contexts of upstream applications into Spark

Repository: spark Updated Branches: refs/heads/master 46b2550bc -> 3af894511 [SPARK-16759][CORE] Add a configuration property to pass caller contexts of upstream applications into Spark ## What changes were proposed in this pull request? Many applications take Spark as a computing engine and

spark git commit: [SPARK-19244][CORE] Sort MemoryConsumers according to their memory usage when spilling

Repository: spark Updated Branches: refs/heads/master 52d4f6194 -> 2f523fa0c [SPARK-19244][CORE] Sort MemoryConsumers according to their memory usage when spilling ## What changes were proposed in this pull request? In `TaskMemoryManager `, when we acquire memory by calling `acquireExecutio

spark git commit: [SPARK-22866][K8S] Fix path issue in Kubernetes dockerfile

Repository: spark Updated Branches: refs/heads/master a36b78b0e -> 22e1849bc [SPARK-22866][K8S] Fix path issue in Kubernetes dockerfile ## What changes were proposed in this pull request? The path was recently changed in https://github.com/apache/spark/pull/19946, but the dockerfile was not

spark git commit: [SPARK-22465][CORE] Add a safety-check to RDD defaultPartitioner

Repository: spark Updated Branches: refs/heads/master 1219d7a43 -> 0bf1a74a7 [SPARK-22465][CORE] Add a safety-check to RDD defaultPartitioner ## What changes were proposed in this pull request? In choosing a Partitioner to use for a cogroup-like operation between a number of RDDs, the default

spark git commit: [SPARK-22465][FOLLOWUP] Update the number of partitions of default partitioner when defaultParallelism is set

Repository: spark Updated Branches: refs/heads/master b2ce17b4c -> 96cb60bc3 [SPARK-22465][FOLLOWUP] Update the number of partitions of default partitioner when defaultParallelism is set ## What changes were proposed in this pull request? #20002 purposed a way to safe check the default parti

spark git commit: [SPARK-22465][FOLLOWUP] Update the number of partitions of default partitioner when defaultParallelism is set

Repository: spark Updated Branches: refs/heads/branch-2.3 7241556d8 -> 832d69817 [SPARK-22465][FOLLOWUP] Update the number of partitions of default partitioner when defaultParallelism is set ## What changes were proposed in this pull request? #20002 purposed a way to safe check the default p

spark git commit: [SPARK-24948][SHS] Delegate check access permissions to the file system

Repository: spark Updated Branches: refs/heads/master 278984d5a -> 3c96937c7 [SPARK-24948][SHS] Delegate check access permissions to the file system ## What changes were proposed in this pull request? In `SparkHadoopUtil. checkAccessPermission`, we consider only basic permissions in order t

spark git commit: [SPARK-21549][CORE] Respect OutputFormats with no output directory provided

Repository: spark Updated Branches: refs/heads/master debcbec74 -> 2030f1951 [SPARK-21549][CORE] Respect OutputFormats with no output directory provided ## What changes were proposed in this pull request? Fix for https://issues.apache.org/jira/browse/SPARK-21549 JIRA issue. Since version 2.2

spark git commit: [SPARK-21549][CORE] Respect OutputFormats with no output directory provided

Repository: spark Updated Branches: refs/heads/branch-2.2 8a4e7dd89 -> 0d3f1667e [SPARK-21549][CORE] Respect OutputFormats with no output directory provided ## What changes were proposed in this pull request? Fix for https://issues.apache.org/jira/browse/SPARK-21549 JIRA issue. Since version

spark git commit: [SPARK-21549][CORE] Respect OutputFormats with no/invalid output directory provided

Repository: spark Updated Branches: refs/heads/master e8547ffb4 -> 13c155958 [SPARK-21549][CORE] Respect OutputFormats with no/invalid output directory provided ## What changes were proposed in this pull request? PR #19294 added support for null's - but spark 2.1 handled other error cases w

spark git commit: [SPARK-21549][CORE] Respect OutputFormats with no/invalid output directory provided

Repository: spark Updated Branches: refs/heads/branch-2.2 acbad83ec -> 6b6761e8f [SPARK-21549][CORE] Respect OutputFormats with no/invalid output directory provided ## What changes were proposed in this pull request? PR #19294 added support for null's - but spark 2.1 handled other error case

git commit: SPARK-2277: make TaskScheduler track hosts on rack

Repository: spark Updated Branches: refs/heads/master efc452a16 -> 33e64ecac SPARK-2277: make TaskScheduler track hosts on rack Hi mateiz, I've created [SPARK-2277](https://issues.apache.org/jira/browse/SPARK-2277) to make TaskScheduler track hosts on each rack. Please help to review, thanks

git commit: SPARK-1376. In the yarn-cluster submitter, rename "args" option to "arg"

Repository: spark Updated Branches: refs/heads/master 33b3c2a8c -> 564f1c137 SPARK-1376. In the yarn-cluster submitter, rename "args" option to "arg" Author: Sandy Ryza Closes #279 from sryza/sandy-spark-1376 and squashes the following commits: d8aebfa [Sandy Ryza] SPARK-1376. In the yarn-c

git commit: SPARK-1417: Spark on Yarn - spark UI link from resourcemanager is broken

Repository: spark Updated Branches: refs/heads/master 44f654eec -> 446bb3417 SPARK-1417: Spark on Yarn - spark UI link from resourcemanager is broken Author: Thomas Graves Closes #344 from tgravescs/SPARK-1417 and squashes the following commits: c450b5f [Thomas Graves] fix test e1c1d7e [Tho

[spark] branch master updated: [SPARK-32920][CORE][SHUFFLE][FOLLOW-UP] Fix to run push-based shuffle tests in DAGSchedulerSuite in ad-hoc manner

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new fbf53de [SPARK-32920][CORE][SHUFFLE][FOLLOW

[spark] branch branch-3.2 updated: [SPARK-32920][CORE][SHUFFLE][FOLLOW-UP] Fix to run push-based shuffle tests in DAGSchedulerSuite in ad-hoc manner

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 5fa7855 [SPARK-32920][CORE][SHUFFLE

[spark] branch master updated: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 4783fb7 [SPARK-35276][CORE] Calculate

[spark] branch branch-3.2 updated: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new d5022c3 [SPARK-35276][CORE

[spark] branch master updated: [SPARK-32922][SHUFFLE][CORE][FOLLOWUP] Fixes few issues when the executor tries to fetch push-merged blocks

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 6d2cbad [SPARK-32922][SHUFFLE][CORE

[spark] branch branch-3.2 updated: [SPARK-32922][SHUFFLE][CORE][FOLLOWUP] Fixes few issues when the executor tries to fetch push-merged blocks

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 595e825 [SPARK-32922][SHUFFLE][CORE

[spark] branch master updated: [SPARK-35546][SHUFFLE] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state in a better way

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new c77acf0 [SPARK-35546][SHUFFLE] Enable push

[spark] branch branch-3.2 updated: [SPARK-35546][SHUFFLE] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state in a better way

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 1907f0a [SPARK-35546][SHUFFLE

[spark] branch master updated: [SPARK-32920][FOLLOW-UP] Fix shuffleMergeFinalized directly calling rdd.getNumPartitions as RDD is not serialized to executor

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new ba1a7ce [SPARK-32920][FOLLOW-UP] Fix

[spark] branch branch-3.2 updated: [SPARK-32920][FOLLOW-UP] Fix shuffleMergeFinalized directly calling rdd.getNumPartitions as RDD is not serialized to executor

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 39d6e87 [SPARK-32920][FOLLOW-UP] Fix

[spark] branch master updated: [SPARK-36266][SHUFFLE] Rename classes in shuffle RPC used for block push operations

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new c4aa54e [SPARK-36266][SHUFFLE] Rename

[spark] branch branch-3.2 updated: [SPARK-36266][SHUFFLE] Rename classes in shuffle RPC used for block push operations

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new bbec381 [SPARK-36266][SHUFFLE

[spark] branch master updated: [SPARK-32919][FOLLOW-UP] Filter out driver in the merger locations and fix the return type of RemoveShufflePushMergerLocations

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 2a18f82 [SPARK-32919][FOLLOW-UP] Filter out

[spark] branch branch-3.2 updated: [SPARK-32919][FOLLOW-UP] Filter out driver in the merger locations and fix the return type of RemoveShufflePushMergerLocations

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new cd3ab00 [SPARK-32919][FOLLOW-UP

[spark] branch master updated: [SPARK-36206][CORE] Support shuffle data corruption diagnosis via shuffle checksum

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new a98d919 [SPARK-36206][CORE] Support shuffle

[spark] branch branch-3.2 updated: [SPARK-36206][CORE] Support shuffle data corruption diagnosis via shuffle checksum

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new df43300 [SPARK-36206][CORE] Support

[spark] branch master updated: [SPARK-32923][FOLLOW-UP] Clean up older shuffleMergeId shuffle files when finalize request for higher shuffleMergeId is received

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new d816949 [SPARK-32923][FOLLOW-UP] Clean up

[spark] branch branch-3.2 updated: [SPARK-32923][FOLLOW-UP] Clean up older shuffleMergeId shuffle files when finalize request for higher shuffleMergeId is received

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 0a98e51 [SPARK-32923][FOLLOW-UP

[spark] branch master updated: [SPARK-36173][CORE] Support getting CPU number in TaskContext

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new f6e6d11 [SPARK-36173][CORE] Support getting

[spark] branch master updated: [SPARK-36423][SHUFFLE] Randomize order of blocks in a push request to improve block merge ratio for push-based shuffle

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 6e72951 [SPARK-36423][SHUFFLE] Randomize

[spark] branch branch-3.2 updated: [SPARK-36423][SHUFFLE] Randomize order of blocks in a push request to improve block merge ratio for push-based shuffle

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 552a332 [SPARK-36423][SHUFFLE

[spark] branch master updated: [SPARK-36378][SHUFFLE] Switch to using RPCResponse to communicate common block push failures to the client

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 3f09093 [SPARK-36378][SHUFFLE] Switch to

[spark] branch branch-3.2 updated: [SPARK-36378][SHUFFLE] Switch to using RPCResponse to communicate common block push failures to the client

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new c6b683e [SPARK-36378][SHUFFLE

[spark] branch master updated: [SPARK-36483][CORE][TESTS] Fix intermittent test failures at Netty 4.1.52+

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new b8e2186 [SPARK-36483][CORE][TESTS] Fix

[spark] branch branch-3.2 updated: [SPARK-36483][CORE][TESTS] Fix intermittent test failures at Netty 4.1.52+

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 32d67d1 [SPARK-36483][CORE][TESTS

[spark] branch master updated: [SPARK-36374][SHUFFLE][DOC] Push-based shuffle high level user documentation

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 2270ecf [SPARK-36374][SHUFFLE][DOC] Push

[spark] branch branch-3.2 updated: [SPARK-36374][SHUFFLE][DOC] Push-based shuffle high level user documentation

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 233af3d [SPARK-36374][SHUFFLE][DOC

[spark] branch master updated: [SPARK-35548][CORE][SHUFFLE] Handling new attempt has started error message in BlockPushErrorHandler in client

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 05cd5f9 [SPARK-35548][CORE][SHUFFLE

[spark] branch branch-3.2 updated: [SPARK-35548][CORE][SHUFFLE] Handling new attempt has started error message in BlockPushErrorHandler in client

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 2fb62e0 [SPARK-35548][CORE][SHUFFLE

[spark] branch master updated: [SPARK-36374][FOLLOW-UP] Change config key spark.shuffle.server.mergedShuffleFileManagerImpl to spark.shuffle.push.server.mergedShuffleFileManagerImpl

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 7b2842e [SPARK-36374][FOLLOW-UP] Change

[spark] branch branch-3.2 updated: [SPARK-36374][FOLLOW-UP] Change config key spark.shuffle.server.mergedShuffleFileManagerImpl to spark.shuffle.push.server.mergedShuffleFileManagerImpl

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 0f2e318 [SPARK-36374][FOLLOW-UP

[spark] branch master updated: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 999473b [SPARK-36705][SHUFFLE] Disable push

[spark] branch branch-3.2 updated: [SPARK-36705][SHUFFLE] Disable push based shuffle when IO encryption is enabled or serializer is not relocatable

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 79e148e [SPARK-36705][SHUFFLE

[spark] branch master updated: [SPARK-36705][FOLLOW-UP] Fix unnecessary logWarning when PUSH_BASED_SHUFFLE_ENABLED is set to false

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 2d7dc7c [SPARK-36705][FOLLOW-UP] Fix

[spark] branch branch-3.2 updated: [SPARK-36705][FOLLOW-UP] Fix unnecessary logWarning when PUSH_BASED_SHUFFLE_ENABLED is set to false

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 4a486f4 [SPARK-36705][FOLLOW-UP] Fix

[spark] branch master updated: [SPARK-36615][CORE] Register shutdown hook earlier when start SC

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new b7d99e3 [SPARK-36615][CORE] Register

[spark] branch master updated: [SPARK-36705][FOLLOW-UP] Support the case when user's classes need to register for Kryo serialization

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new e5b01cd [SPARK-36705][FOLLOW-UP] Support the

[spark] branch branch-3.2 updated: [SPARK-36705][FOLLOW-UP] Support the case when user's classes need to register for Kryo serialization

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new c542297 [SPARK-36705][FOLLOW-UP

[spark] branch master updated: [SPARK-36798][CORE] Wait for listeners to finish before flushing metrics

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new d758ae3 [SPARK-36798][CORE] Wait for

[spark] branch master updated: [SPARK-36834][SHUFFLE] Add support for namespacing log lines emitted by external shuffle service

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 4072a22 [SPARK-36834][SHUFFLE] Add support

[spark] branch master updated: [SPARK-37023][CORE] Avoid fetching merge status when shuffleMergeEnabled is false for a shuffleDependency during retry

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new f1532a2 [SPARK-37023][CORE] Avoid fetching

[spark] branch branch-3.2 updated: [SPARK-37023][CORE] Avoid fetching merge status when shuffleMergeEnabled is false for a shuffleDependency during retry

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 9f96524 [SPARK-37023][CORE] Avoid

[spark] branch master updated: [SPARK-32663][CORE] Avoid individual closing of pooled TransportClients (which must be closed through the pool)

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 79b4dea [SPARK-32663][CORE] Avoid individual

[spark] branch branch-3.0 updated: [SPARK-32663][CORE] Avoid individual closing of pooled TransportClients (which must be closed through the pool)

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.0 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.0 by this push: new f73e6ca [SPARK-32663][CORE] Avoid

[spark] branch branch-3.0 updated: [SPARK-32738][CORE][3.0] Should reduce the number of active threads if fatal error happens in `Inbox.process`

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.0 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.0 by this push: new 17a5195 [SPARK-32738][CORE][3.0

[spark] branch branch-2.4 updated: [SPARK-32738][CORE][2.4] Should reduce the number of active threads if fatal error happens in `Inbox.process`

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.4 by this push: new b8e6fa7 [SPARK-32738][CORE][2.4

[spark] branch master updated: [SPARK-32915][CORE] Network-layer and shuffle RPC layer changes to support push shuffle blocks

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 82eea13 [SPARK-32915][CORE] Network-layer

[spark] branch master updated: [SPARK-33088][CORE] Enhance ExecutorPlugin API to include callbacks on task start and end events

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 8f4fc22 [SPARK-33088][CORE] Enhance

[spark] branch master updated: [SPARK-33185][YARN] Set up yarn.Client to print direct links to driver stdout/stderr

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 324275a [SPARK-33185][YARN] Set up

[spark] branch master updated: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 8113c88 [SPARK-32916][SHUFFLE

[spark] branch master updated: [SPARK-31069][CORE] Avoid repeat compute `chunksBeingTransferred` cause hight cpu cost in external shuffle service when `maxChunksBeingTransferred` use default value

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new f8b95dd [SPARK-31069][CORE] Avoid repeat

[spark] branch master updated: [SPARK-31069][CORE] Avoid repeat compute `chunksBeingTransferred` cause hight cpu cost in external shuffle service when `maxChunksBeingTransferred` use default value

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new dd32f45 [SPARK-31069][CORE] Avoid repeat

[spark] branch master updated: [SPARK-32919][SHUFFLE][TEST-MAVEN][TEST-HADOOP2.7] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging partitions

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 8218b48 [SPARK-32919][SHUFFLE][TEST-MAVEN

[spark] branch master updated: [SPARK-32918][SHUFFLE] RPC implementation to support control plane coordination for push-based shuffle

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 1bd897c [SPARK-32918][SHUFFLE] RPC

[spark] branch master updated: [SPARK-33185][YARN][FOLLOW-ON] Leverage RM's RPC API instead of REST to fetch driver log links in yarn.Client

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new f3c2583 [SPARK-33185][YARN][FOLLOW-ON

[spark] branch master updated: [SPARK-33669] Wrong error message from YARN application state monitor when sc.stop in yarn client mode

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 48f93af [SPARK-33669] Wrong error message

[spark] branch branch-3.1 updated: [SPARK-33669] Wrong error message from YARN application state monitor when sc.stop in yarn client mode

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.1 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.1 by this push: new 57ed4df [SPARK-33669] Wrong error

[spark] branch master updated: [SPARK-32916][SHUFFLE][TEST-MAVEN][TEST-HADOOP2.7] Ensure the number of chunks in meta file and index file are equal

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 0677c39 [SPARK-32916][SHUFFLE][TEST-MAVEN

[spark] branch branch-3.1 updated: [SPARK-32916][SHUFFLE][TEST-MAVEN][TEST-HADOOP2.7] Ensure the number of chunks in meta file and index file are equal

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.1 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.1 by this push: new b174ac7 [SPARK-32916][SHUFFLE][TEST

[spark-website] branch asf-site updated: Update committer page

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/spark-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 96a50b5 Update committer page

[spark] branch master updated: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 2d6eb00 [SPARK-32596][CORE] Clear Ivy

[spark] branch master updated: [SPARK-32119][CORE] ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes with --jars

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 1a4c8f7 [SPARK-32119][CORE] ExecutorPlugin

[spark] branch master updated: [SPARK-37461][YARN][FOLLOWUP] Refactor YARN Client code to avoid add unnecessary parameter of `appId`

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 0b42cd4 [SPARK-37461][YARN][FOLLOWUP

[spark] branch master updated: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new f6128a6 [SPARK-33701][SHUFFLE] Adaptive

[spark] branch master updated: [SPARK-37710][CORE] Add detailed log message for java.io.IOException occurring on Kryo flow

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 5d7031c [SPARK-37710][CORE] Add detailed log

[spark] branch master updated: [SPARK-37847][CORE][SHUFFLE] PushBlockStreamCallback#isStale should check null to avoid NPE

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 7463564 [SPARK-37847][CORE][SHUFFLE

[spark] branch master updated: [SPARK-37712][YARN] Spark request yarn cluster metrics slow cause delay

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new df7447b [SPARK-37712][YARN] Spark request

[spark] branch master updated: [SPARK-37675][SPARK-37793] Prevent overwriting of push shuffle merged files once the shuffle is finalized

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 9afb407 [SPARK-37675][SPARK-37793] Prevent

[spark] branch branch-3.2 updated: [SPARK-37675][SPARK-37793] Prevent overwriting of push shuffle merged files once the shuffle is finalized

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new 837c6b4 [SPARK-37675][SPARK-37793

[spark] branch master updated: [SPARK-34826][SHUFFLE] Adaptively fetch shuffle mergers for push based shuffle

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new c69f08f8 [SPARK-34826][SHUFFLE] Adaptively

[spark] branch master updated: [SPARK-37688][CORE] ExecutorMonitor should ignore SparkListenerBlockUpdated event if executor was not active

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 969d672 [SPARK-37688][CORE] ExecutorMonitor

[spark] branch master updated: [SPARK-38344][SHUFFLE] Avoid to submit task when there are no requests to push up in push-based shuffle

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 96bcb04 [SPARK-38344][SHUFFLE] Avoid to

[spark] branch master updated: [SPARK-38309][CORE] Fix SHS `shuffleTotalReads` and `shuffleTotalBlocks` percentile metrics

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 0ad7677 [SPARK-38309][CORE] Fix SHS

[spark] branch branch-3.2 updated: [SPARK-38309][CORE] Fix SHS `shuffleTotalReads` and `shuffleTotalBlocks` percentile metrics

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.2 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.2 by this push: new e067b12 [SPARK-38309][CORE] Fix SHS

[spark] branch branch-3.1 updated: [SPARK-38309][CORE] Fix SHS `shuffleTotalReads` and `shuffleTotalBlocks` percentile metrics

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.1 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.1 by this push: new 0277139 [SPARK-38309][CORE] Fix SHS

[spark] branch master updated: [SPARK-38428][SHUFFLE] Check the FetchShuffleBlocks message only once to improve iteration in external shuffle service

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 8fabd5e [SPARK-38428][SHUFFLE] Check the

[spark] branch master updated: [SPARK-38555][NETWORK][SHUFFLE] Avoid contention and get or create clientPools quickly in the TransportClientFactory

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 6d3e8eb [SPARK-38555][NETWORK][SHUFFLE

[spark] branch master updated (a826469d0be -> 2a1f9767213)

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git from a826469d0be [SPARK-38851][CORE][TESTS] Refactor `HistoryServerSuite` to add UTs for RocksDB add 2a1f9767213

[spark] branch master updated: [SPARK-32917][SHUFFLE][CORE] Adds support for executors to push shuffle blocks after successful map task completion

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new d00f069 [SPARK-32917][SHUFFLE][CORE] Adds

[spark] branch master updated: [SPARK-34069][CORE] Kill barrier tasks should respect SPARK_JOB_INTERRUPT_ON_CANCEL

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 65222b7 [SPARK-34069][CORE] Kill barrier

[spark] branch branch-3.1 updated: [SPARK-34069][CORE] Kill barrier tasks should respect SPARK_JOB_INTERRUPT_ON_CANCEL

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch branch-3.1 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-3.1 by this push: new 983dbb9 [SPARK-34069][CORE] Kill

[spark] branch master updated: [SPARK-24818][CORE] Support delay scheduling for barrier execution

This is an automated email from the ASF dual-hosted git repository. mridulm80 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 4dc16f2 [SPARK-24818][CORE] Support delay

  1   2   3   >