This is an automated email from the ASF dual-hosted git repository.
zhangzc pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
from 08a71fe663 [GLUTEN-9594][CH] Reorder join/aggregate keys, ensure keys
of all clauses are in the same order (#9595)
add 32f03d63a9 [GLUTEN-9606][CH]Support CH MergeTree + Delta
DeletionVector reading (#9609)
No new revisions were added by this update.
Summary of changes:
backends-clickhouse/pom.xml | 1 +
.../v2/clickhouse/metadata/AddFileTags.scala | 1 +
.../v2/clickhouse/metadata/AddFileTags.scala | 19 +-
.../gluten/sql/shims/delta32/Delta32Shims.scala | 41 ++++-
.../spark/sql/delta/PreprocessTableWithDVs.scala | 203 +++++++++++++++++++++
.../v2/clickhouse/metadata/AddFileTags.scala | 19 +-
.../source/DeltaMergeTreeFileFormat.scala | 40 +++-
.../GlutenDeltaMergeTreeDeletionVectorSuite.scala | 84 ++++++++-
.../org/apache/gluten/sql/shims/DeltaShims.scala | 20 +-
.../execution/GlutenMergeTreePartition.scala | 22 ++-
.../utils/MergeTreePartsPartitionsUtil.scala | 33 ++--
.../datasources/mergetree/MetaSerializer.scala | 18 +-
...lutenClickhouseMergetreeSoftAffinitySuite.scala | 3 +-
.../apache/spark/affinity/MixedAffinitySuite.scala | 2 +-
.../FillingDeltaInternalRowDeletedStep.cpp | 129 +++++++++++++
...Step.h => FillingDeltaInternalRowDeletedStep.h} | 22 ++-
.../Parser/RelParsers/MergeTreeRelParser.cpp | 57 +++++-
.../Parser/RelParsers/MergeTreeRelParser.h | 18 +-
.../Storages/MergeTree/SparkMergeTreeMeta.cpp | 4 +
.../Storages/MergeTree/SparkMergeTreeMeta.h | 2 +
.../Storages/SubstraitSource/Delta/DeltaMeta.h | 92 ++++++++++
.../SubstraitSource/Delta/DeltaParquetMeta.h | 44 -----
.../Storages/SubstraitSource/Delta/DeltaReader.cpp | 32 +---
.../Storages/SubstraitSource/Delta/DeltaReader.h | 20 +-
.../Storages/SubstraitSource/FileReader.cpp | 8 +-
.../Storages/SubstraitSource/ParquetFormatFile.cpp | 4 +-
26 files changed, 787 insertions(+), 151 deletions(-)
copy backends-clickhouse/{src =>
src-delta-20}/main/scala/org/apache/spark/sql/execution/datasources/v2/clickhouse/metadata/AddFileTags.scala
(99%)
copy backends-clickhouse/{src =>
src-delta-23}/main/scala/org/apache/spark/sql/execution/datasources/v2/clickhouse/metadata/AddFileTags.scala
(95%)
create mode 100644
backends-clickhouse/src-delta-32/main/scala/org/apache/spark/sql/delta/PreprocessTableWithDVs.scala
rename backends-clickhouse/{src =>
src-delta-32}/main/scala/org/apache/spark/sql/execution/datasources/v2/clickhouse/metadata/AddFileTags.scala
(95%)
create mode 100644
cpp-ch/local-engine/Operator/FillingDeltaInternalRowDeletedStep.cpp
copy cpp-ch/local-engine/Operator/{ReplicateRowsStep.h =>
FillingDeltaInternalRowDeletedStep.h} (53%)
create mode 100644
cpp-ch/local-engine/Storages/SubstraitSource/Delta/DeltaMeta.h
delete mode 100644
cpp-ch/local-engine/Storages/SubstraitSource/Delta/DeltaParquetMeta.h
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
