This is an automated email from the ASF dual-hosted git repository.
jackylee-ch pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gluten.git
The following commit(s) were added to refs/heads/main by this push:
new 0b8d0f649a [MINOR][CH] Correct misleading Spark 3.2 comment in
ReadBufferBuilder (#12583)
0b8d0f649a is described below
commit 0b8d0f649adc8636834c768dd6f9cf704658c817
Author: YangJie <[email protected]>
AuthorDate: Wed Aug 26 13:30:01 2026 +0800
[MINOR][CH] Correct misleading Spark 3.2 comment in ReadBufferBuilder
(#12583)
---
cpp-ch/local-engine/Storages/SubstraitSource/ReadBufferBuilder.cpp | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/cpp-ch/local-engine/Storages/SubstraitSource/ReadBufferBuilder.cpp
b/cpp-ch/local-engine/Storages/SubstraitSource/ReadBufferBuilder.cpp
index 6d99965709..3ea544dc01 100644
--- a/cpp-ch/local-engine/Storages/SubstraitSource/ReadBufferBuilder.cpp
+++ b/cpp-ch/local-engine/Storages/SubstraitSource/ReadBufferBuilder.cpp
@@ -326,8 +326,9 @@ public:
{
if (!file_size.has_value())
{
- // only for spark3.2 file partition not contained file size
- // so first compute file size first
+ // Fallback for callers whose Substrait LocalFilesNode was
built without a
+ // `fileSizes` input (e.g. some Iceberg producers), so no
per-file `fileSize`
+ // property is available. Compute the size by opening the HDFS
file first.
auto tmp_read_buffer =
std::make_unique<DB::ReadBufferFromHDFS>(
hdfs_uri,
hdfs_file_path,
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
