This is an automated email from the ASF dual-hosted git repository.
zuston pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/uniffle.git
from 2a32171b9 [#2569] feat(spark): Add statistic of shuffle read times
(#2598)
add 1e48bc673 [#2601] feat(spark): Introduce overlapping decompression for
shuffle read (#2602)
No new revisions were added by this update.
Summary of changes:
.../hadoop/mapreduce/task/reduce/RssFetcher.java | 2 +-
.../org/apache/spark/shuffle/RssSparkConfig.java | 12 +++
.../shuffle/reader/RssShuffleDataIterator.java | 47 +++++++---
.../spark/shuffle/reader/RssShuffleReader.java | 57 ++++++++----
.../library/common/shuffle/impl/RssTezFetcher.java | 2 +-
.../orderedgrouped/RssTezShuffleDataFetcher.java | 2 +-
.../uniffle/client/api/ShuffleReadClient.java | 4 +-
.../client/factory/ShuffleClientFactory.java | 34 +++++++
.../uniffle/client/impl/DecompressionWorker.java | 103 +++++++++++++++++++++
.../uniffle/client/impl/ShuffleReadClientImpl.java | 42 +++++++--
.../client/response/CompressedShuffleBlock.java | 5 +-
...fleBlock.java => DecompressedShuffleBlock.java} | 23 +++--
.../uniffle/client/response/ShuffleBlock.java | 10 +-
.../java/org/apache/uniffle/client/TestUtils.java | 4 +-
.../client/impl/DecompressionWorkerTest.java | 89 ++++++++++++++++++
.../client/impl/ShuffleReadClientImplTest.java | 4 +-
.../test/HybridStorageFaultToleranceBase.java | 4 +-
.../java/org/apache/uniffle/test/QuorumTest.java | 4 +-
.../uniffle/test/ShuffleServerWithHadoopTest.java | 4 +-
.../ShuffleServerWithKerberizedHadoopTest.java | 4 +-
20 files changed, 384 insertions(+), 72 deletions(-)
create mode 100644
client/src/main/java/org/apache/uniffle/client/impl/DecompressionWorker.java
copy
client/src/main/java/org/apache/uniffle/client/response/{CompressedShuffleBlock.java
=> DecompressedShuffleBlock.java} (65%)
copy
server/src/main/java/org/apache/uniffle/server/merge/MergeEventHandler.java =>
client/src/main/java/org/apache/uniffle/client/response/ShuffleBlock.java (83%)
create mode 100644
client/src/test/java/org/apache/uniffle/client/impl/DecompressionWorkerTest.java
