This is an automated email from the ASF dual-hosted git repository.
zhouyuan pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gluten.git
The following commit(s) were added to refs/heads/main by this push:
new 75a0cc9bbb [GLUTEN-6887][VL] Daily Update Velox Version (2026_11_06)
(#13559)
75a0cc9bbb is described below
commit 75a0cc9bbb004ce44bfa96ef74f04eb12bd3c7a2
Author: Gluten Performance Bot
<[email protected]>
AuthorDate: Fri Nov 6 14:33:20 2026 +0000
[GLUTEN-6887][VL] Daily Update Velox Version (2026_11_06) (#13559)
* [GLUTEN-6887][VL] Daily Update Velox Version (dft-2026_11_06)
Upstream Velox's New Commits:
afb370168 by Pedro Pedreira, perf(expression): Compute each expression's
hash once
75efde385 by Zhenyuan Zhao, feat(hive): Keep fileReadOps a split sets
(#20118)
7b059b1f0 by Raymond Lin, fix(fuzzer): Stop input-file query shapes at the
deadline (#19877)
420431858 by XiaoDu, refactor(nimble): Share the slice range check across
encodings (#20016)
87432f41a by mingyuzh, fix(vector): Initialise constants over lazy rows
(#19922)
f566e6403 by Hongze Zhang, feat: Support toIntermediate in
AggregateCompanionAdapter::MergeFunction
01ced0f18 by Jimmy Lu, feat(exec): Run fixed point iterations as
multi-stage plans (#20128)
31a5424b9 by Letian Jiang, perf(functions): Avoid date parser allocations
with min/max tracking
fbd6b4f49 by GALI PREM SAGAR, feat(cudf): Cast DATE and TIMESTAMP to
VARCHAR on the GPU
fc78d79de by Henry Dikeman, fix(nimble): Pass the batch reader's
FileIoContext to TabletReader (#20078)
13ac55fe9 by Henry Dikeman, fix(nimble): Pass the selective reader's
FileIoContext to TabletReader (#20077)
9b33ecc88 by Henry Dikeman, feat(nimble): Carry a FileIoContext through
TabletReader reads (#20076)
b932d0e34 by XiaoDu, feat(nimble): Slice PartitionedEliasFano natively
(#20015)
84d760ddc by Luis PeƱaranda, docs(sparksql): Describe extreme decimal scale
behavior (#20123)
73eb477a0 by Xiaoxuan, fix(nimble): Select ALP exponents by encoded size
2277b9ae9 by Rui Mo, refactor: Share evaluation of SwitchExpr and CaseExpr
871b4d1f2 by hu hengrui, perf(exec): Avoid per-cell size tracking for
fixed-width columns
cff4a1639 by Luis PeƱaranda, fix(sparksql): Correct shared decimal rounding
and overflow
924e90735 by macduan, refactor(paimon): Move Paimon connector out of Hive
03bea9b3c by Xiaoxuan, feat(nimble): Cut stripes at a raw size inside
batches (#20113)
896dbfd20 by XiaoDu, refactor(nimble): Slice with a value delta in
EncodingFactory (#20052)
ce4a98510 by Apurva Kumar, fix(dwio): Keep rows and nulls under an
AlwaysTrue filter (#20003)
33f899652 by Krishnakanth, feat(exec): Relax IndexLookupJoin output
requirements (#20106)
390e9d93f by Xiaoxuan, perf(nimble): Build the projector stream slicer
lazily
20859fb25 by Maria Basmanova, test: Finish the AggregationTester migration
and remove the old methods (#20100)
94a4d4152 by Maria Basmanova, feat: Make local vector index format pluggable
3b1e4101f by Xiaoxuan, perf(nimble): Decode Huffman values sequentially
9e55bbed4 by kevinwilfong, perf: Stream filtered inner merge-join groups
(#20104)
4bedd787b by Brody Smith, feat(hive): Forward fileReadOps from
FileDataSource to data and Iceberg delete file readers (#20011) (#20011)
9c3dd335b by Maria Basmanova, test: Migrate Spark aggregate function tests
to AggregationTester
baa845064 by Apurva Kumar, test(iceberg): Compare deletion vectors in split
benchmark (#20070)
8e869d3e2 by Pedro Pedreira, test(expression): Benchmark compiling large
expressions
78c6ae7d0 by Maria Basmanova, test: Migrate Presto aggregate function tests
to AggregationTester
3ade85c62 by tanjialiang, fix(nimble): Count metadata cache hits in
rawBytesRead (#19987)
46774b883 by Ke, test(nimble): Fuzz stripe groups and fully-null chunks
6ca3736eb by Xiaoxuan, perf(nimble): Reuse stream readers across stripes
3c9dc4a2d by shanhao-203, perf(nimble): Validate Hybrid FlatMap schema keys
in one pass (#20025)
e1b50d383 by Krishna Pai, refactor: Use std::identity instead of
folly::identity (#20088)
47d70f0fe by Maria Basmanova, test: Add AggregationTester for aggregation
function tests
d2d3b0a9f by shanhao-203, perf(nimble): Build Hybrid FlatMap reader keys
without strings (#20026)
b0328218a by macduan, refactor(paimon): Move shared file connector code to
common
3ce1e60ed by adheer-araokar, fix(spill): Let the page writer split spill
vectors of 2GB or more (#19843)
4ba4dc509 by adheer-araokar, fix(serializer): Reject Presto pages of 2GB or
more (#19842)
f0f6a167d by Zongze Li, perf(operator): Avoid repeated peer heap work in
TopN ranking
07791605e by Richard Barnes, refactor(parquet): Remove unreachable returns
in arrow test utilities
40b53f118 by Xiaoxuan, perf(nimble): Read ALP encoded values in one pass
25add5e38 by Krishna Pai, misc: Reconcile Type.h, BaseVector.h and Nimble
views with fbcode (#20067)
---
.github/workflows/util/iceberg-upstream/flaky-tests.csv | 1 +
cpp/velox/compute/VeloxBackend.cc | 4 ++--
cpp/velox/compute/delta/DeltaDataSource.h | 8 +++++++-
cpp/velox/compute/delta/DeltaSplitReader.h | 4 +++-
cpp/velox/memory/GlutenBufferedInputBuilder.h | 4 ++--
cpp/velox/tests/iceberg/IcebergWriteTest.cc | 5 ++---
ep/build-velox/src/get-velox.sh | 4 ++--
7 files changed, 19 insertions(+), 11 deletions(-)
diff --git a/.github/workflows/util/iceberg-upstream/flaky-tests.csv
b/.github/workflows/util/iceberg-upstream/flaky-tests.csv
index 7f5ec34e6d..1dceb3bd3f 100644
--- a/.github/workflows/util/iceberg-upstream/flaky-tests.csv
+++ b/.github/workflows/util/iceberg-upstream/flaky-tests.csv
@@ -18,3 +18,4 @@
class,method
# Example
#
org.apache.iceberg.spark.extensions.TestAlterTablePartitionFields,testDropPartitionAndSourceColumnLong
+org.apache.iceberg.spark.extensions.TestViews,createViewWithCTEReferencingTempView
diff --git a/cpp/velox/compute/VeloxBackend.cc
b/cpp/velox/compute/VeloxBackend.cc
index 52f48f38ef..449dfb7382 100644
--- a/cpp/velox/compute/VeloxBackend.cc
+++ b/cpp/velox/compute/VeloxBackend.cc
@@ -60,7 +60,7 @@
#include "utils/VeloxWriterUtils.h"
#include "velox/common/caching/SsdCache.h"
#include "velox/common/file/FileSystems.h"
-#include "velox/connectors/hive/BufferedInputBuilder.h"
+#include "velox/connectors/common/BufferedInputBuilder.h"
#include "velox/connectors/hive/HiveConnector.h"
#include "velox/connectors/hive/HiveDataSource.h"
#include "velox/connectors/hive/iceberg/IcebergConnector.h"
@@ -281,7 +281,7 @@ void VeloxBackend::init(
velox::orc::registerOrcReaderFactory();
velox::exec::ExprToSubfieldFilterParser::registerParser(std::make_unique<SparkExprToSubfieldFilterParser>(
backendConf_->get<bool>(kScanBloomFilterPushdownEnabled,
kScanBloomFilterPushdownEnabledDefault)));
-
velox::connector::hive::BufferedInputBuilder::registerBuilder(std::make_shared<GlutenBufferedInputBuilder>());
+
velox::connector::BufferedInputBuilder::registerBuilder(std::make_shared<GlutenBufferedInputBuilder>());
// Register Velox functions
registerAllFunctions();
diff --git a/cpp/velox/compute/delta/DeltaDataSource.h
b/cpp/velox/compute/delta/DeltaDataSource.h
index 64d796b812..a870bc9d61 100644
--- a/cpp/velox/compute/delta/DeltaDataSource.h
+++ b/cpp/velox/compute/delta/DeltaDataSource.h
@@ -20,7 +20,9 @@
#include "velox/connectors/hive/HiveDataSource.h"
#ifndef GLUTEN_VELOX_DELTA_USE_FILE_SPLIT_READER
-#if __has_include("velox/connectors/hive/FileSplitReader.h")
+#if __has_include("velox/connectors/common/FileSplitReader.h")
+#define GLUTEN_VELOX_DELTA_USE_FILE_SPLIT_READER 1
+#elif __has_include("velox/connectors/hive/FileSplitReader.h")
#define GLUTEN_VELOX_DELTA_USE_FILE_SPLIT_READER 1
#else
#define GLUTEN_VELOX_DELTA_USE_FILE_SPLIT_READER 0
@@ -28,7 +30,11 @@
#endif
#if GLUTEN_VELOX_DELTA_USE_FILE_SPLIT_READER
+#if __has_include("velox/connectors/common/FileSplitReader.h")
+#include "velox/connectors/common/FileSplitReader.h"
+#else
#include "velox/connectors/hive/FileSplitReader.h"
+#endif
#else
namespace facebook::velox::connector::hive {
class SplitReader;
diff --git a/cpp/velox/compute/delta/DeltaSplitReader.h
b/cpp/velox/compute/delta/DeltaSplitReader.h
index 88fad25828..2bc26519c9 100644
--- a/cpp/velox/compute/delta/DeltaSplitReader.h
+++ b/cpp/velox/compute/delta/DeltaSplitReader.h
@@ -21,7 +21,9 @@
#include "compute/delta/DeltaSplit.h"
#ifndef GLUTEN_VELOX_DELTA_USE_FILE_SPLIT_READER
-#if __has_include("velox/connectors/hive/FileSplitReader.h")
+#if __has_include("velox/connectors/common/FileSplitReader.h")
+#define GLUTEN_VELOX_DELTA_USE_FILE_SPLIT_READER 1
+#elif __has_include("velox/connectors/hive/FileSplitReader.h")
#define GLUTEN_VELOX_DELTA_USE_FILE_SPLIT_READER 1
#else
#define GLUTEN_VELOX_DELTA_USE_FILE_SPLIT_READER 0
diff --git a/cpp/velox/memory/GlutenBufferedInputBuilder.h
b/cpp/velox/memory/GlutenBufferedInputBuilder.h
index 25b05f9870..860f2521c7 100644
--- a/cpp/velox/memory/GlutenBufferedInputBuilder.h
+++ b/cpp/velox/memory/GlutenBufferedInputBuilder.h
@@ -18,13 +18,13 @@
#pragma once
#include "GlutenDirectBufferedInput.h"
-#include "velox/connectors/hive/BufferedInputBuilder.h"
+#include "velox/connectors/common/BufferedInputBuilder.h"
#include "velox/connectors/hive/FileHandle.h"
#include "velox/dwio/common/CachedBufferedInput.h"
namespace gluten {
-class GlutenBufferedInputBuilder : public
facebook::velox::connector::hive::BufferedInputBuilder {
+class GlutenBufferedInputBuilder : public
facebook::velox::connector::BufferedInputBuilder {
public:
std::unique_ptr<facebook::velox::dwio::common::BufferedInput> create(
const facebook::velox::FileHandle& fileHandle,
diff --git a/cpp/velox/tests/iceberg/IcebergWriteTest.cc
b/cpp/velox/tests/iceberg/IcebergWriteTest.cc
index b3a97fc9d3..a78739d7e9 100644
--- a/cpp/velox/tests/iceberg/IcebergWriteTest.cc
+++ b/cpp/velox/tests/iceberg/IcebergWriteTest.cc
@@ -19,7 +19,7 @@
#include "memory/VeloxColumnarBatch.h"
#include "utils/ConfigExtractor.h"
#include "utils/VeloxWriterUtils.h"
-#include "velox/connectors/hive/FileConnectorUtil.h"
+#include "velox/connectors/common/FileConnectorUtil.h"
#include "velox/connectors/hive/HiveConfig.h"
#include "velox/exec/tests/utils/TempDirectoryPath.h"
#include "velox/vector/tests/utils/VectorTestBase.h"
@@ -58,8 +58,7 @@ TEST_F(VeloxIcebergWriteTest, parquetWriterOptions) {
{"spark.gluten.sql.columnar.backend.velox.parquet_writer_compression_level",
std::to_string(level)},
{"spark.gluten.sql.columnar.backend.velox.parquet_writer_datapage_version",
version}});
auto session = createHiveConnectorSessionConfig(sparkConf);
- auto scopedConfigs =
- connector::hive::makeFormatScopedConfigs(hiveConfig, *session,
dwio::common::FileFormat::PARQUET);
+ auto scopedConfigs = connector::makeFormatScopedConfigs(hiveConfig,
*session, dwio::common::FileFormat::PARQUET);
auto options = std::static_pointer_cast<parquet::ParquetWriterOptions>(
factory.createFormatOptions(scopedConfigs.connectorConfig,
scopedConfigs.sessionProperties));
ASSERT_NE(options->codecOptions, nullptr);
diff --git a/ep/build-velox/src/get-velox.sh b/ep/build-velox/src/get-velox.sh
index 2245a84906..502d6643e8 100755
--- a/ep/build-velox/src/get-velox.sh
+++ b/ep/build-velox/src/get-velox.sh
@@ -18,8 +18,8 @@ set -exu
CURRENT_DIR=$(cd "$(dirname "$BASH_SOURCE")"; pwd)
VELOX_REPO=https://github.com/IBM/velox.git
-VELOX_BRANCH=dft-2026_11_05
-VELOX_ENHANCED_BRANCH=ibm-2026_11_05
+VELOX_BRANCH=dft-2026_11_06
+VELOX_ENHANCED_BRANCH=ibm-2026_11_06
VELOX_HOME=""
RUN_SETUP_SCRIPT=ON
ENABLE_ENHANCED_FEATURES=OFF
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]