github-actions[bot] commented on code in PR #66227:
URL: https://github.com/apache/doris/pull/66227#discussion_r4073608596
##########
thirdparty/build-thirdparty.sh:
##########
@@ -2372,6 +2704,40 @@ cleanup_package_source() {
fi
}
+# All-Rust rebuild: when the selected package list rebuilds BOTH Rust
+# packages (explicitly or via the default full list / --continue from an
+# earlier package), drop the installed Rust archives and the shared rustc
+# toolchain stamp before the loop, so a toolchain switch can actually start
+# -- otherwise the first Rust package would be rejected against the stamp
+# recorded for the old pair. A partial rebuild (one Rust package alone)
+# keeps the check strict on purpose: mixing a new-toolchain archive with the
+# installed old-toolchain one is exactly what must not land in
+# TP_INSTALL_DIR. If the paired rebuild fails midway, TP_INSTALL_DIR is left
+# without the removed Rust archives (a BE link then fails on the missing
+# lib rather than linking mismatched std copies), which is recoverable by
+# rerunning the same paired rebuild.
+rust_rebuild_lance=0
+rust_rebuild_paimon=0
+for package in "${packages[@]}"; do
+ case "${package}" in
+ lance_c) rust_rebuild_lance=1 ;;
+ paimon_rust) rust_rebuild_paimon=1 ;;
+ esac
+done
+if [[ "${rust_rebuild_lance}" -eq 1 && "${rust_rebuild_paimon}" -eq 1 ]]; then
Review Comment:
[P2] Base Rust cleanup on the resumed suffix
With --continue, packages is still the full default list, so this pre-loop
scan always sees both Rust packages and removes both archives. The later
PACKAGE_FOUND gate can skip them: --continue paimon_rust deletes liblance_c.a
but rebuilds only Paimon, while a resume point after Paimon deletes both and
rebuilds neither. A successful resume therefore leaves the install prefix
unlinkable. Compute this decision from the effective suffix after
start_package, or force the resume back to lance_c when a paired rebuild is
required.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/source/PaimonScanNode.java:
##########
@@ -123,11 +129,21 @@ public class PaimonScanNode extends FileQueryScanNode {
"doris.scan.manifest.parallelism-cap";
private static final String DORIS_SERIALIZED_SYSTEM_SOURCE =
"doris.serialized-system-source";
private static final String DORIS_SYSTEM_TABLE_TYPE =
"doris.system-table-type";
+ // Same key as the rust CoreOptions DELETION_VECTORS_MERGE_ON_READ_OPTION:
paimon
+ // 1.4 exposes no Java accessor for it, so the raw TableSchema option is
read.
+ private static final String DELETION_VECTORS_MERGE_ON_READ =
"deletion-vectors.merge-on-read";
private static final List<String> BACKEND_PAIMON_OPTIONS = Arrays.asList(
DORIS_ENABLE_JNI_IO_MANAGER,
DORIS_JNI_IO_MANAGER_TMP_DIR,
DORIS_JNI_IO_MANAGER_IMPL_CLASS,
DORIS_ENABLE_FILE_READER_ASYNC);
+ // The table-location URI schemes whose property translation into the
+ // paimon-rust FileIO key families is implemented (BE bridge) and open
+ // tested: s3 / s3a -> the s3.* family, oss -> the fs.oss.* family, plus
+ // the credential-free hdfs and local-filesystem parsers (hadoop conf and
+ // local paths pass through untouched). See isRustVerifiedLocationScheme.
+ private static final Set<String> RUST_VERIFIED_LOCATION_SCHEMES =
+ new HashSet<>(Arrays.asList("s3", "s3a", "oss", "hdfs", "file"));
Review Comment:
[P1] Keep authenticated HDFS scans on JNI
This allowlist treats every hdfs location as credential-free, but Doris
supports Kerberized HDFS catalogs and sends hadoop.security.authentication,
principal/keytab, username, and dfs/fs configuration. The pinned paimon-rust
HDFS parser consumes only hdfs.name-node and hdfs.enable-append, so those
catalogs still satisfy canUseRust and then open without their configured
identity; the same query that works through JNI fails authentication. Please
gate HDFS to a proven simple-auth shape, or implement the Doris HDFS
authentication contract in the native backend before selecting it.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/source/PaimonScanNode.java:
##########
@@ -333,6 +349,52 @@ private void setScanLevelPaimonOptions() {
}
}
+ /**
+ * Whether the table location's URI scheme is served by a paimon-rust
FileIO parser whose
+ * property translation the FE/BE bridge implements (the s3.* / fs.oss.*
key families for
+ * s3 / s3a / oss) or that needs no credentials at all (hdfs hadoop conf
and local
+ * filesystem paths). Every other scheme the pinned crate dispatches to
its own parser
+ * (cosn / obs / gs / abfs and friends) must fall back to the JNI reader
because Doris
+ * delivers those credentials only as AWS_* aliases that those parsers do
not read.
+ * A null location cannot be verified (and cannot ship paimon_table
either), so it is
+ * not rust-eligible.
+ */
+ @VisibleForTesting
+ static boolean isRustVerifiedLocationScheme(String location) {
+ if (location == null) {
+ return false;
+ }
+ int sep = location.indexOf("://");
+ if (sep <= 0) {
+ // No URI scheme: a plain local path reads through the crate's
local-filesystem
+ // parser, which needs no credentials.
+ return true;
+ }
+ return RUST_VERIFIED_LOCATION_SCHEMES.contains(
Review Comment:
[P1] Do not route uppercase URI schemes to case-sensitive Rust paths
The classifier deliberately accepts S3:// and OSS:// (the new test asserts
this) but the unchanged location is sent to Rust. The pinned backend lowercases
only storage dispatch; its object-store path extraction strips a lowercase
prefix from the original string, and the HDFS/file helpers likewise require
lowercase hdfs:// or file:/. These valid URI case variants therefore select
PAIMON_RUST and then fail to open instead of using JNI. Normalize the
transported scheme or require the original path to match the Rust parser before
setting canUseRust.
##########
regression-test/suites/external_table_p0/paimon/test_paimon_rust_reader_v2.groovy:
##########
@@ -0,0 +1,109 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+// Paimon rust reader under FileScannerV2: enable_paimon_rust_reader=true must
work with
+// the default enable_file_scanner_v2=true (before the v2 port, a rust split
was rejected by
+// FileScannerV2::is_supported and the reader was only reachable with v2
disabled).
+suite("test_paimon_rust_reader_v2", "p0,external") {
+ String enabled = context.config.otherConfigs.get("enablePaimonTest")
+ if (enabled == null || !enabled.equalsIgnoreCase("true")) {
+ logger.info("disabled paimon test")
+ return
+ }
+
+ String catalogName = "test_paimon_rust_reader_v2"
+ String hdfsPort = context.config.otherConfigs.get("hive2HdfsPort")
+ String externalEnvIp = context.config.otherConfigs.get("externalEnvIp")
+
+ try {
+ sql """drop catalog if exists ${catalogName}"""
+ sql """create catalog if not exists ${catalogName} properties (
+ "type" = "paimon",
+ "paimon.catalog.type" = "filesystem",
+ "warehouse" =
"hdfs://${externalEnvIp}:${hdfsPort}/user/doris/paimon1"
+ );"""
+ // Capture the settings this suite overrides so finally can restore
them.
+ def originalForceJni = sql("select @@force_jni_scanner")[0][0]
+ def originalV2 = sql("select @@enable_file_scanner_v2")[0][0]
+
+ sql """switch ${catalogName}"""
+ sql """use db1"""
+ // Force the logical (JNI / rust) reader path: the append parquet
tables
+ // below (all_table, all_table_with_parquet, append_table, ...) convert
+ // to raw native splits, which getSplits() would otherwise prefer —
+ // both differential legs would silently use the native reader and
+ // never reach the JNI / rust converters.
+ sql """set force_jni_scanner=true"""
+ // FileScannerV2 is the default; make the intent of this suite
explicit.
+ sql """set enable_file_scanner_v2=true"""
+
+ def testQueries = [
+ """select c1 from complex_all order by c1""",
+ // Filter on a data column exercises the v2 predicate pushdown
into the
+ // paimon-rust filter (PaimonRustPredicateConverter
global-index mode).
+ """select c1 from complex_all where c1 >= 2 order by c1""",
+ """select * from all_table order by c1""",
+ """select * from all_table_with_parquet where c13 like '13%'
order by c1""",
+ """select * from complex_tab order by c1""",
+ """select c3['a_test'], c3['b_test'], c3['bbb'], c3['ccc']
from complex_tab order by c3['a_test'], c3['b_test']""",
+ """select array_max(c2) c from complex_tab order by c""",
+ """select c20[0] c from complex_all order by c""",
+ // ORDER BY keeps the JNI/rust differential deterministic: the
two
+ // readers may return the same rows in different valid
file/batch
+ // order, and the suite compares the result lists as strings.
+ """select * from deletion_vector_orc order by id""",
+ """select * from deletion_vector_parquet order by id""",
+ // count(*) exercises the table-level metadata count path of
the v2 leaf reader.
+ """select count(*) from append_table""",
+ """select count(*) from merge_on_read_table""",
+ """select count(*) from deletion_vector_parquet""",
+ // count(*) with a filter must fall back to the real rust scan.
+ """select count(*) from all_table where c1 >= 2""",
+ // Full scans of the count-suite tables exercise merge-on-read
and
+ // deletion-vector batches through the v2 leaf reader.
+ """select * from append_table order by product_id""",
+ """select * from merge_on_read_table order by product_id""",
+ """select * from deletion_vector_parquet order by id"""
+ ]
+
+ // Default path is JNI when enable_paimon_rust_reader=false.
+ sql """set enable_paimon_rust_reader=false"""
+ def jniResults = testQueries.collect { query -> sql(query) }
+
+ sql """set enable_paimon_rust_reader=true"""
+ def rustResults = testQueries.collect { query -> sql(query) }
+
+ assertTrue(rustResults[0].size() > 0)
+ for (int i = 0; i < testQueries.size(); i++) {
+ assertEquals(jniResults[i].toString(), rustResults[i].toString())
+ }
+
+ // The v1 path must keep working when v2 is explicitly disabled: FE
only
+ // encodes PAIMON_RUST requests when enable_file_scanner_v2 is on (the
V1
+ // FileScanner rejects them), so with v2 disabled it selects JNI.
+ sql """set enable_file_scanner_v2=false"""
+ def v1RustResults = testQueries.collect { query -> sql(query) }
+ for (int i = 0; i < testQueries.size(); i++) {
+ assertEquals(jniResults[i].toString(), v1RustResults[i].toString())
+ }
+ } finally {
+ sql """set enable_paimon_rust_reader=false"""
+ sql """set force_jni_scanner=${originalForceJni}"""
Review Comment:
[P2] Declare the saved settings outside the try block
originalForceJni and originalV2 are local variables declared inside try, so
they are out of scope in this finally block. The suite will throw while
interpolating the cleanup SQL instead of restoring
enable_file_scanner_v2/force_jni_scanner (and can mask the original failure).
Declare both variables before try and assign them inside, as the sibling suites
do.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/source/PaimonScanNode.java:
##########
@@ -411,10 +473,227 @@ private void setPaimonParams(TFileRangeDesc rangeDesc,
PaimonSplit paimonSplit)
String fileFormat = getFileFormat(paimonSplit.getPathString());
if (split != null) {
+ // use jni reader / paimon-cpp reader / paimon-rust reader
rangeDesc.setFormatType(TFileFormatType.FORMAT_JNI);
- // A logical DataSplit may span multiple files, so keep it intact
for the JNI reader.
- fileDesc.setReaderType(TPaimonReaderType.PAIMON_JNI);
- fileDesc.setPaimonSplit(PaimonUtil.encodeObjectToString(split));
+ // paimon-cpp and paimon-rust both consume Paimon native binary
serialization,
+ // which only supports DataSplit. Any other split type falls back
to JNI.
+ boolean nativeSplit = split instanceof DataSplit;
+ // Fallback-read splits stay on JNI: FallbackDataSplit extends
+ // DataSplit, so the instanceof above passes, but its serializer
+ // appends an isFallback byte after the ordinary split that the
+ // pinned rust decoder rejects outright ("trailing bytes after
+ // DataSplit" — it requires full-buffer consumption), and even a
+ // permissive decode would still lack the second table identity
+ // needed to honor the fallback-side discriminator. Both sides of a
+ // FallbackReadFileStoreTable wrap their splits, so the table
+ // wrapper is gated as a whole (any split from it routes to JNI)
+ // until the rust ABI represents both sides; the FallbackSplit
+ // interface also catches a wrapper split regardless of how the
+ // table was resolved here.
+ boolean fallbackRead = split instanceof
FallbackReadFileStoreTable.FallbackSplit
+ || processedTable instanceof FallbackReadFileStoreTable;
+ // Serialize the same effective table that planning and the JNI
reader use.
+ // Relation options such as t@options('read.batch-size'='1') are
applied by
+ // getProcessedTable() (doInitialize caches it in processedTable),
and the
+ // rust reader derives its read batch size from the schema options
— the raw
+ // cached table would silently drop the override. Copies,
delegates and
+ // fallback wrappers of getProcessedTable() are still
FileStoreTable, so the
+ // instanceof gate keeps its semantics.
+ Table paimonTable = processedTable;
+ FileStoreTable paimonFileStoreTable =
+ paimonTable instanceof FileStoreTable ? (FileStoreTable)
paimonTable : null;
+ // query-auth.enabled tables stay on JNI: when catalog
authorization
+ // succeeds with no row filter or column mask, Paimon still leaves
an
+ // ordinary DataSplit (restricted results use QueryAuthSplit and
are
+ // already handled by the nativeSplit gate above), so this table
shape
+ // passes the compound gate — but the shipped schema keeps
+ // query-auth.enabled=true and the pinned rust ReadBuilder rejects
+ // every such table (its CoreOptions::ensure_read_authorized fails
+ // closed because the client cannot enforce the row filter / column
+ // masking), turning a valid authorized scan into a BE-open
failure.
+ // Until the authorization result can be transported and enforced
by
+ // the rust ABI, these tables route to JNI.
+ boolean queryAuthTable = false;
+ // Partial-update / aggregation tables with deletion vectors only
pass
+ // the rust reader in the fully materialized shape: the pinned rust
+ // read_pk rejects merge-engine=partial-update/aggregation with
+ // deletion-vectors.merge-on-read=true outright, and otherwise
requires
+ // every split to be compacted and known free of retract rows
+ // (DataSplit::is_fully_materialized_pk_dv). Their ordinary
DataSplits
+ // sail through the compound gate above, so without this check a
valid
+ // Java/JNI scan reaches BE and the rust open fails. Deduplicate
stays
+ // rust-eligible: its read_pk routes uncompacted splits to the KV
+ // reader, which applies the attached per-file DVs.
merge-on-read=true
+ // is a table option, so the whole table routes to JNI;
+ // non-materialized splits are gated per split below.
+ boolean puAggDeletionVectors = false;
+ boolean dvMergeOnRead = false;
+ if (paimonFileStoreTable != null) {
+ CoreOptions resolvedCoreOptions =
paimonFileStoreTable.coreOptions();
+ // Null-safe: a table handle whose CoreOptions is not resolved
+ // (e.g. some wrapper shapes) stays rust-eligible rather than
+ // failing the scan here — the rust open itself rejects such a
+ // table if the option is really set.
+ if (resolvedCoreOptions != null) {
+ queryAuthTable = resolvedCoreOptions.queryAuthEnabled();
+ CoreOptions.MergeEngine mergeEngine =
resolvedCoreOptions.mergeEngine();
+ if (resolvedCoreOptions.deletionVectorsEnabled()
+ && (mergeEngine ==
CoreOptions.MergeEngine.PARTIAL_UPDATE
+ || mergeEngine ==
CoreOptions.MergeEngine.AGGREGATE)) {
+ puAggDeletionVectors = true;
+ // The merge-engine and deletion-vectors.enabled checks
+ // above resolve through the Java CoreOptions
accessors,
+ // which the table builds from this same schema options
+ // map — the one the BE rust reader deserializes from
+ // the shipped schema JSON — so they cannot diverge
from
+ // what BE sees. merge-on-read has no Java accessor in
+ // paimon 1.4, so it is read raw from the map, with the
+ // rust parsing semantics (any case-insensitive "true"
+ // is on, default false).
+ TableSchema dvSchema = paimonFileStoreTable.schema();
+ Map<String, String> dvOptions = dvSchema == null ?
null : dvSchema.options();
+ String mergeOnRead = dvOptions == null
+ ? null :
dvOptions.get(DELETION_VECTORS_MERGE_ON_READ);
+ dvMergeOnRead = "true".equalsIgnoreCase(mergeOnRead);
+ }
+ }
+ }
+ // paimon-rust additionally requires (a) FileScannerV2: the V1
FileScanner
+ // explicitly rejects PAIMON_RUST, so with enable_file_scanner_v2
disabled
+ // the split falls back to JNI instead of encoding a rust request
that the
+ // selected scanner cannot consume, and (b) a FileStoreTable: BE
opens the
+ // table via paimon_table_from_schema_json, which needs the
resolved
+ // TableSchema that only FileStoreTable exposes via schema(). If
the table
+ // is not a FileStoreTable (e.g. a sys table backed by DataSplit),
we cannot
+ // ship a schema JSON, so fall back to CPP / JNI rather than
sending an
+ // incomplete PAIMON_RUST request that BE would reject.
+ //
+ // The paimon-rust S3 bridge maps static credentials, anonymous
+ // access (AWS_CREDENTIALS_PROVIDER_TYPE=ANONYMOUS -> s3.anonymous)
+ // and assume-role (AWS_ROLE_ARN / AWS_EXTERNAL_ID ->
+ // s3.assumed.role.*), but the remaining credential-provider modes
+ // are ambient JVM provider chains (ENV, SYSTEM_PROPERTIES,
+ // WEB_IDENTITY, CONTAINER, INSTANCE_PROFILE) with no paimon-rust
+ // equivalent — rust would silently sign with whatever the ambient
+ // chain resolves to. Gate those modes away from the rust reader
+ // here so the configured provider is honored via the JNI path.
+ boolean providerModeTranslatable = true;
+ String providerType = backendStorageProperties == null
+ ? null :
backendStorageProperties.get("AWS_CREDENTIALS_PROVIDER_TYPE");
+ if (providerType != null) {
+ String mode = providerType.trim().toUpperCase(Locale.ROOT);
+ providerModeTranslatable = mode.equals("DEFAULT")
+ || mode.equals("ANONYMOUS");
+ // The rust OSS FileIO parser (oss:// warehouses) has no
+ // skip-signature switch, so an anonymous OSS catalog cannot be
+ // served by the rust reader either — fall back to JNI.
+ if (mode.equals("ANONYMOUS")) {
+ String location = source.getTableLocation();
+ if (location != null && location.startsWith("oss://")) {
+ providerModeTranslatable = false;
+ }
+ }
+ }
+ // Incremental scans (binlog / changelog / delta / diff) must stay
+ // on the JNI path: this wire format carries only an ordinary
+ // DataSplit and the rust reader invokes TableRead::to_arrow, but
+ // paimon 1.4 marks incremental splits as streaming (which the
+ // pinned rust deserializer rejects), diff requires a separate
+ // IncrementalPlan instead of an ordinary plan, and ordinary
+ // primary-key reads can merge versions rather than return the
+ // changes — until the C ABI transports the mode and plan, the
+ // rust reader cannot express any of these.
+ TableScanParams incrementalParams = getScanParams();
+ boolean isIncremental = incrementalParams != null &&
incrementalParams.incrementalRead();
+ // ORC TIMESTAMP_WITH_LOCAL_TIME_ZONE schemas stay on JNI: the
pinned
+ // paimon-rust ORC decoder materializes LTZ instants shifted by the
+ // writer timezone (an upstream crate limitation), so a logical ORC
+ // DataSplit that selects rust (e.g. with force_jni_scanner=true or
+ // when raw conversion is unavailable) returns a different instant
+ // than JNI — applying the session timezone in BE cannot repair an
+ // epoch already shifted during decode. fileFormat is resolved per
+ // split above; a bucket-directory split without a file suffix
falls
+ // back to the table-level 'file.format' option, which matches the
+ // files Paimon writes for that table. Parquet LTZ is unaffected.
+ boolean orcLtzSchema = paimonFileStoreTable != null
+ && "orc".equals(fileFormat.toLowerCase(Locale.ROOT))
+ &&
paimonFileStoreTable.schema().fields().stream().anyMatch(field ->
+ field.type().getTypeRoot() ==
DataTypeRoot.TIMESTAMP_WITH_LOCAL_TIME_ZONE);
+ // Scheme capability gate: the pinned paimon-rust storage
+ // dispatcher (io/storage.rs) selects the FileIO parser from the
+ // table location's URI scheme, and libpaimon_c.a compiles in
+ // separate COS, OBS, GCS and Azdls parsers besides the OSS and S3
+ // ones. Doris normalizes every object store's credentials into
+ // the AWS_* / use_path_style aliases (see the *Properties storage
+ // classes), which the BE rust bridge translates only into the
+ // fs.oss.* and s3.* key families — a cosn:// / obs:// / gs:// /
+ // abfs:// warehouse would reach its scheme's parser without the
+ // key family it reads (fs.cosn.userinfo.*, fs.obs.*, gcs.*,
+ // azure.*) and fail the open instead of using JNI. Only the
+ // schemes whose property translation is implemented and
+ // open-tested (s3 / s3a / oss, via RUST_VERIFIED_LOCATION_SCHEMES)
+ // plus the credential-free hdfs and local-filesystem parsers stay
+ // rust-eligible; every other scheme falls back to JNI. A null
+ // location also routes to JNI: the rust path needs the
+ // paimon_table that only a real location can provide (BE rejects
+ // a split without it).
+ boolean schemeCapabilityVerified =
isRustVerifiedLocationScheme(source.getTableLocation());
+ // With merge-on-read=true the whole table already routes to JNI
(dvMergeOnRead);
+ // for the remaining partial-update/aggregation DV tables, a split
that is
+ // not fully materialized (uncompacted level-0 data, or
retractions not
+ // known to be applied — even a split with no deletion file
attached yet)
+ // fails the rust is_fully_materialized_pk_dv guard, so it falls
back per
+ // split instead of turning into a BE-open failure. nativeSplit and
+ // !fallbackRead only guard the cast — those splits already route
to JNI.
+ boolean splitDvNotMaterialized = puAggDeletionVectors &&
!dvMergeOnRead
+ && nativeSplit && !fallbackRead
+ && !isFullyMaterializedPkDvSplit((DataSplit) split);
+ boolean canUseRust = sessionVariable.isEnablePaimonRustReader()
+ && sessionVariable.enableFileScannerV2 && nativeSplit &&
!fallbackRead
+ && !isIncremental && providerModeTranslatable &&
!queryAuthTable
+ && !dvMergeOnRead && !splitDvNotMaterialized
+ && !orcLtzSchema && schemeCapabilityVerified &&
paimonFileStoreTable != null;
Review Comment:
[P1] Fall back for projected Variant columns
A projected Paimon VARIANT can reach this gate when enable_variant_v2 is on,
but the Rust leaf feeds its Arrow struct into
DataTypeVariantV2SerDe::read_column_from_arrow, which unconditionally returns
NOT_IMPLEMENTED_ERROR. Nested Variant reaches the same decoder through the
container SerDes. Thus valid Variant scans that work through JNI are selected
for Rust and fail during materialization. Exclude any projected type that
recursively contains Variant until this leaf has a Variant Arrow decoder.
##########
be/src/format_v2/table/paimon_rust_table_reader.cpp:
##########
@@ -0,0 +1,848 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "format_v2/table/paimon_rust_table_reader.h"
+
+#include <algorithm>
+#include <utility>
+
+#include "arrow/c/abi.h"
+#include "arrow/c/bridge.h"
+#include "arrow/record_batch.h"
+#include "arrow/result.h"
+#include "common/logging.h"
+#include "core/block/block.h"
+#include "core/block/column_with_type_and_name.h"
+#include "core/column/column_const.h"
+#include "exprs/vexpr_context.h"
+#include "exprs/vliteral.h"
+#include "format_v2/column_mapper.h"
+#include "format_v2/table/paimon_rust_predicate_converter.h"
+#include "runtime/descriptors.h"
+#include "runtime/file_scan_profile.h"
+#include "runtime/runtime_state.h"
+#include "util/string_util.h"
+#include "util/timezone_utils.h"
+#include "util/url_coding.h"
+
+extern "C" {
+#include "paimon_rust/paimon.h"
+}
+
+namespace doris::format::paimon {
+
+namespace {
+constexpr const char* VALUE_KIND_FIELD = "_VALUE_KIND";
+
+// ---------------------------------------------------------------------------
+// RAII wrappers over the paimon-rust C handles. Each handle is an opaque
+// pointer owned by Rust and released by a matching paimon_*_free function.
+// ---------------------------------------------------------------------------
+#define PAIMON_OWNED(type, freefn) \
+ struct type##_deleter { \
+ void operator()(paimon_##type* p) const { \
+ if (p) { \
+ freefn(p); \
+ } \
+ } \
+ }; \
+ using type##_ptr = std::unique_ptr<paimon_##type, type##_deleter>
+
+PAIMON_OWNED(table, paimon_table_free);
+PAIMON_OWNED(read_builder, paimon_read_builder_free);
+PAIMON_OWNED(plan, paimon_plan_free);
+PAIMON_OWNED(table_read, paimon_table_read_free);
+PAIMON_OWNED(record_batch_reader, paimon_record_batch_reader_free);
+PAIMON_OWNED(error, paimon_error_free);
+
+#undef PAIMON_OWNED
+
+// One Arrow batch (schema + array containers). Owning it requires a two-step
+// teardown that the unique_ptr deleters above can't express: first invoke the
+// Arrow C Data Interface `release` callback on each struct (hands buffers back
+// to the producer), then free the container structs via
paimon_arrow_batch_free.
+class ArrowBatch {
+public:
+ explicit ArrowBatch(paimon_arrow_batch batch) : batch_(batch) {}
+ ~ArrowBatch() {
+ auto* schema = static_cast<ArrowSchema*>(batch_.schema);
+ auto* array = static_cast<ArrowArray*>(batch_.array);
+ if (array && array->release) {
+ array->release(array);
+ }
+ if (schema && schema->release) {
+ schema->release(schema);
+ }
+ paimon_arrow_batch_free(batch_);
+ }
+
+ ArrowBatch(const ArrowBatch&) = delete;
+ ArrowBatch& operator=(const ArrowBatch&) = delete;
+
+ ArrowSchema* schema() const { return
static_cast<ArrowSchema*>(batch_.schema); }
+ ArrowArray* array() const { return static_cast<ArrowArray*>(batch_.array);
}
+
+private:
+ paimon_arrow_batch batch_;
+};
+
+// Render a paimon_error into a string. Takes ownership of `err` via RAII so it
+// is freed on every return path. Safe to call with nullptr.
+std::string consume_error(paimon_error* err) {
+ error_ptr owned(err);
+ if (!owned) {
+ return "unknown error";
+ }
+ std::string msg;
+ if (owned->message.data != nullptr && owned->message.len > 0) {
+ msg.assign(reinterpret_cast<const char*>(owned->message.data),
owned->message.len);
+ }
+ return "code=" + std::to_string(owned->code) + ", msg=" + msg;
+}
+
+// Render storage option KEYS for diagnostics. Values are never rendered:
+// credential keys arrive under many spellings and cases (AWS_SECRET_KEY,
+// AWS_TOKEN, fs.oss.accessKeySecret, s3.secret-key, ...), and a key-name
+// blocklist that misses one alias leaks the value into the INFO log, so
+// only the key names are printed at all.
+std::string format_options(const std::map<std::string, std::string>& options) {
+ std::string out;
+ for (const auto& kv : options) {
+ if (!out.empty()) {
+ out += ", ";
+ }
+ out += kv.first;
+ }
+ return out;
+}
+
+} // namespace
+
+// Paimon-rust handles. Order of members matters: destruction runs in reverse
+// declaration order, and the read_builder depends on the table while the arrow
+// reader depends on the whole pipeline above it. So the table MUST be declared
+// first (destroyed last) and the record batch reader last.
+struct PaimonRustTableReader::PaimonHandles {
+ table_ptr table;
+ read_builder_ptr read_builder;
+ plan_ptr plan;
+ table_read_ptr table_read;
+ record_batch_reader_ptr reader;
+};
+
+PaimonRustTableReader::PaimonRustTableReader() = default;
+
+PaimonRustTableReader::~PaimonRustTableReader() = default;
+
+Status PaimonRustTableReader::init(format::TableReadOptions&& options) {
+ RETURN_IF_ERROR(format::TableReader::init(std::move(options)));
+ {
+ // Base and derived scopes must not overlap on the same counter:
RuntimeProfile timers
+ // add deltas, so nested use would double-count instead of extending
lifecycle coverage.
+ SCOPED_TIMER(_profile.total_timer);
+ SCOPED_TIMER(_profile.init_timer);
+ // Materialize TIMESTAMP_LTZ in the session timezone — the same
+ // convention as the JNI reader (PaimonJniScanner reads time_zone from
+ // its scan params) and lance_reader. Timezone-naive (paimon TIMESTAMP)
+ // arrow values are decoded in UTC by the DateTimeV2 serde regardless
+ // of _ctz, so NTZ wall-clock semantics are preserved.
+ DORIS_CHECK(_runtime_state != nullptr);
+ _ctz = _runtime_state->timezone_obj();
+ if (_scanner_profile != nullptr) {
+ file_scan_profile::ensure_hierarchy(_scanner_profile);
+ _rust_total_time = ADD_CHILD_TIMER(_scanner_profile,
"PaimonRustReader",
+
file_scan_profile::TABLE_READER);
+ _rust_open_split_time =
+ ADD_CHILD_TIMER(_scanner_profile, "OpenSplitTime",
"PaimonRustReader");
+ _rust_read_batch_time =
+ ADD_CHILD_TIMER(_scanner_profile, "ReadBatchTime",
"PaimonRustReader");
+ _rust_arrow_to_block_time =
+ ADD_CHILD_TIMER(_scanner_profile, "ArrowToBlockTime",
"PaimonRustReader");
+ }
+ // Projected column name -> fixed output position, registered with
both the exact and
+ // the lower-case spelling so mixed-case Rust schema output still
resolves (v1
+ // semantics: exact match first, lower-case fallback on lookup).
+ _output_name_to_idx.reserve(_projected_columns.size() * 2);
+ for (size_t idx = 0; idx < _projected_columns.size(); ++idx) {
+ _output_name_to_idx.emplace(_projected_columns[idx].name, idx);
+
_output_name_to_idx.emplace(to_lower(_projected_columns[idx].name), idx);
+ }
+ }
+ return Status::OK();
+}
+
+Status PaimonRustTableReader::prepare_split(const format::SplitReadOptions&
options) {
+ // EOF belongs to the previous split. Keep it set after closing that split
so repeated reads
+ // are idempotent, and clear it only when a new split is explicitly
prepared.
+ _close_split_reader();
+ _split_eof = false;
+ _current_range = options.current_range;
+ RETURN_IF_ERROR(format::TableReader::prepare_split(options));
+ if (current_split_pruned()) {
+ return Status::OK();
+ }
+ if (_is_table_level_count_active()) {
+ // No rust pipeline is opened; get_block emits the synthetic count
rows.
+ return Status::OK();
+ }
+ RETURN_IF_ERROR(_validate_rust_split(options.current_range));
+ {
+ SCOPED_TIMER(_profile.total_timer);
+ SCOPED_TIMER(_profile.prepare_split_timer);
+ SCOPED_TIMER(_rust_open_split_time);
+ RETURN_IF_ERROR(_open_split_reader(options.current_range));
+ }
+ return Status::OK();
+}
+
+Status PaimonRustTableReader::get_block(Block* block, bool* eos) {
+ SCOPED_TIMER(_profile.total_timer);
+ SCOPED_TIMER(_profile.exec_timer);
+ SCOPED_TIMER(_rust_total_time);
+ DORIS_CHECK(block != nullptr);
+ DORIS_CHECK(eos != nullptr);
+ DORIS_CHECK(block->columns() == _projected_columns.size());
+ block->clear_column_data(_projected_columns.size());
+ *eos = false;
+
+ if (_is_table_level_count_active()) {
+ return _read_table_level_count(block, eos);
+ }
+
+ // num_splits == 0 yields an empty (but valid) stream: report EOF.
+ if (_split_eof) {
+ *eos = true;
+ return Status::OK();
+ }
+ if (!_handles || !_handles->reader) {
+ return Status::InternalError("paimon-rust reader is not initialized");
+ }
+
+ while (true) {
+ // Mirror the base TableReader cancellation contract so a cancelled
query does not
+ // drain the whole split.
+ if (_io_ctx != nullptr && _io_ctx->should_stop) {
+ _split_eof = true;
+ _close_split_reader();
+ *eos = true;
+ return Status::OK();
+ }
+
+ paimon_result_next_batch next;
+ {
+ SCOPED_TIMER(_rust_read_batch_time);
+ next = paimon_record_batch_reader_next(_handles->reader.get());
+ }
+ if (next.error != nullptr) {
+ return Status::InternalError("paimon-rust read batch failed: {}",
+ consume_error(next.error));
+ }
+ // End of stream: both pointers are null.
+ if (next.batch.array == nullptr && next.batch.schema == nullptr) {
+ _split_eof = true;
+ _close_split_reader();
+ *eos = true;
+ return Status::OK();
+ }
+
+ // RAII: the batch's Arrow release callbacks + container free run when
+ // `batch` leaves this scope, including on any early return.
+ ArrowBatch batch(next.batch);
+
+ auto* c_array = batch.array();
+ auto* c_schema = batch.schema();
+ arrow::Result<std::shared_ptr<arrow::RecordBatch>> import_result =
+ arrow::ImportRecordBatch(c_array, c_schema);
+ if (!import_result.ok()) {
+ return Status::InternalError("failed to import paimon-rust arrow
batch: {}",
+ import_result.status().message());
+ }
+
+ auto record_batch = std::move(import_result).ValueUnsafe();
+ const auto rows = static_cast<size_t>(record_batch->num_rows());
+ if (rows == 0) {
+ // Skip empty batches and keep draining the stream.
+ continue;
+ }
+ RETURN_IF_ERROR(_fill_block_from_record_batch(record_batch, block,
rows));
+ _record_scan_rows(rows);
+ *eos = false;
+ return Status::OK();
+ }
+}
+
+Status PaimonRustTableReader::abort_split() {
+ {
+ SCOPED_TIMER(_profile.total_timer);
+ SCOPED_TIMER(_profile.close_timer);
+ _close_split_reader();
+ _split_eof = false;
+ }
+ return format::TableReader::abort_split();
+}
+
+#ifdef BE_TEST
+std::string PaimonRustTableReader::TEST_format_options(
+ const std::map<std::string, std::string>& options) {
+ return format_options(options);
+}
+
+std::map<std::string, std::string> PaimonRustTableReader::TEST_build_options(
+ TFileScanRangeParams* scan_params, const TFileRangeDesc& range) {
+ TFileScanRangeParams* previous_params = _scan_params;
+ TFileRangeDesc previous_range = _current_range;
+ _scan_params = scan_params;
+ _current_range = range;
+ std::map<std::string, std::string> options = _build_options();
+ _scan_params = previous_params;
+ _current_range = std::move(previous_range);
+ return options;
+}
+#endif
+
+Status PaimonRustTableReader::close() {
+ {
+ SCOPED_TIMER(_profile.total_timer);
+ SCOPED_TIMER(_profile.close_timer);
+ _close_split_reader();
+ _close_table();
+ }
+ return format::TableReader::close();
+}
+
+Status PaimonRustTableReader::_validate_rust_split(const TFileRangeDesc&
range) const {
+ if (!range.__isset.table_format_params ||
!range.table_format_params.__isset.paimon_params) {
+ return Status::InternalError(
+ "missing paimon_params for paimon rust reader, possibly caused
by FE/BE protocol "
+ "mismatch");
+ }
+ const auto& params = range.table_format_params.paimon_params;
+ if (!params.__isset.paimon_split || params.paimon_split.empty()) {
+ return Status::InternalError(
+ "missing paimon_split for paimon rust reader, possibly caused
by FE/BE protocol "
+ "mismatch");
+ }
+ if (params.__isset.reader_type && params.reader_type !=
TPaimonReaderType::PAIMON_RUST) {
+ return Status::InternalError(
+ "invalid reader_type for paimon rust reader, possibly caused
by FE/BE protocol "
+ "mismatch");
+ }
+ if (!_resolve_table_path(range).has_value()) {
+ return Status::InternalError(
+ "paimon-rust missing paimon_table; cannot resolve paimon table
location");
+ }
+ if (!_resolve_db_name(range).has_value()) {
+ return Status::InternalError(
+ "paimon-rust missing db_name; cannot open paimon table via
schema json");
+ }
+ if (!_resolve_table_name(range).has_value()) {
+ return Status::InternalError(
+ "paimon-rust missing table_name; cannot open paimon table via
schema json");
+ }
+ if (!_resolve_table_schema_json(range).has_value()) {
+ return Status::InternalError(
+ "paimon-rust missing paimon_table_schema_json; cannot open
paimon table via "
+ "schema json");
+ }
+ return Status::OK();
+}
+
+Status PaimonRustTableReader::_open_split_reader(const TFileRangeDesc& range) {
+ // 1. Decode the FE-planned split first so we fail fast (and without any
+ // filesystem IO) when it is missing or malformed.
+ std::string split_bytes;
+ RETURN_IF_ERROR(_decode_split_bytes(&split_bytes));
+
+ // 2. Resolve identifier + table_path + FE-supplied TableSchema JSON.
+ auto table_path = _resolve_table_path(range).value();
+ auto db_name = _resolve_db_name(range).value();
+ auto table_name = _resolve_table_name(range).value();
+ auto schema_json = _resolve_table_schema_json(range).value();
+ auto branch_opt = _resolve_branch(range);
+
+ // 3. Assemble storage options: FE-supplied paimon options + hadoop_conf +
+ // OSS/S3 → AWS_* translations. These feed FileIO only (per
+ // paimon_table_from_schema_json contract); they are NOT merged into the
+ // supplied table schema.
+ auto options = _build_options();
+
+ auto opened_table_key =
+ std::make_tuple(table_path, schema_json, db_name, table_name,
branch_opt, options);
+ if (!_handles || !_handles->table || _opened_table_key !=
opened_table_key) {
+ // A paimon scan reads one table, so the handle is opened at most once
per
+ // distinct identity (e.g. re-created after a close); splits of the
same
+ // table reuse it and only rebuild the read pipeline below.
+ _close_table();
+ _handles = std::make_unique<PaimonHandles>();
+
+ std::vector<paimon_option> c_options;
+ c_options.reserve(options.size());
+ for (const auto& kv : options) {
+ c_options.push_back(paimon_option {kv.first.c_str(),
kv.second.c_str()});
+ }
+
+ LOG(INFO) << "paimon-rust opening table via schema json: db=" <<
db_name
+ << " table=" << table_name << " path=" << table_path
+ << " branch=" << (branch_opt.has_value() ?
branch_opt.value() : "main")
+ << " storage_options=[" << format_options(options) << "]";
+
+ // Build the table directly from the FE-supplied schema JSON. The Rust
+ // side rejects null / empty branch, so we default to paimon's
canonical
+ // "main" sentinel when FE did not set paimon_branch (i.e. the table is
+ // on the main branch — matches upstream
Identifier.DEFAULT_MAIN_BRANCH).
+ const std::string& branch_str = branch_opt.has_value() ?
branch_opt.value() : "main";
+ paimon_result_get_table tbl_res = paimon_table_from_schema_json(
+ table_path.c_str(), schema_json.c_str(), db_name.c_str(),
table_name.c_str(),
+ branch_str.c_str(), c_options.empty() ? nullptr :
c_options.data(),
+ c_options.size());
+ if (tbl_res.error != nullptr) {
+ return Status::InternalError(
+ "paimon-rust table_from_schema_json failed: db={} table={}
err={}", db_name,
+ table_name, consume_error(tbl_res.error));
+ }
+ _handles->table.reset(tbl_res.table);
+ _opened_table_key = std::move(opened_table_key);
+ }
+
+ // 4. Build the read pipeline: read_builder -> case-insensitive ->
projection.
+ paimon_result_read_builder rb_res =
paimon_table_new_read_builder(_handles->table.get());
+ if (rb_res.error != nullptr) {
+ return Status::InternalError("paimon-rust new read builder failed: {}",
+ consume_error(rb_res.error));
+ }
+ _handles->read_builder.reset(rb_res.read_builder);
+
+ // Fold column casing on the Rust side so FE-normalized lowercase names
+ // resolve against tables with mixed-case column definitions.
+ if (paimon_error* case_err =
+
paimon_read_builder_with_case_sensitive(_handles->read_builder.get(), false)) {
+ return Status::InternalError("paimon-rust set case_sensitive failed:
{}",
+ consume_error(case_err));
+ }
+
+ // Partition keys are excluded: they are materialized from split metadata
+ // (see _fill_non_arrow_columns), and paimon-rust does not emit them.
+ auto read_columns = _build_read_columns();
+ std::vector<const char*> projection;
+ projection.reserve(read_columns.size() + 1);
+ for (const auto& col : read_columns) {
+ projection.push_back(col.c_str());
+ }
+ projection.push_back(nullptr);
+ if (paimon_error* proj_err =
paimon_read_builder_with_projection(_handles->read_builder.get(),
+
projection.data())) {
+ return Status::InternalError("paimon-rust set projection failed: {}",
+ consume_error(proj_err));
+ }
+
+ // Convert the scanner conjuncts into a paimon-rust filter and apply it.
+ RETURN_IF_ERROR(_apply_predicate());
+
+ // 5. Deserialize the FE-planned split into a one-split plan, so this
+ // scanner reads exactly the split it was assigned rather than replanning
+ // the whole table. The wire form is identical to what paimon-cpp consumes
+ // (`paimon::table::DataSplit::serialize`).
+ paimon_result_plan plan_res = paimon_plan_from_split_bytes(
+ reinterpret_cast<const uint8_t*>(split_bytes.data()),
split_bytes.size());
+ if (plan_res.error != nullptr) {
+ return Status::InternalError("paimon-rust build plan failed: {}",
+ consume_error(plan_res.error));
+ }
+ _handles->plan.reset(plan_res.plan);
+
+ size_t num_splits = paimon_plan_num_splits(_handles->plan.get());
+ if (num_splits == 0) {
+ _split_eof = true;
+ return Status::OK();
+ }
+
+ // 6. Open the arrow stream over the plan.
+ paimon_result_new_read read_res =
paimon_read_builder_new_read(_handles->read_builder.get());
+ if (read_res.error != nullptr) {
+ return Status::InternalError("paimon-rust new read failed: {}",
+ consume_error(read_res.error));
+ }
+ _handles->table_read.reset(read_res.read);
+
+ paimon_result_record_batch_reader rdr_res = paimon_table_read_to_arrow(
+ _handles->table_read.get(), _handles->plan.get(), /*offset=*/0,
/*length=*/num_splits);
+ if (rdr_res.error != nullptr) {
+ return Status::InternalError("paimon-rust open arrow reader failed:
{}",
+ consume_error(rdr_res.error));
+ }
+ _handles->reader.reset(rdr_res.reader);
+ return Status::OK();
+}
+
+void PaimonRustTableReader::_close_split_reader() {
+ if (!_handles) {
+ return;
+ }
+ // Reverse of the declaration order in PaimonHandles.
+ _handles->reader.reset();
+ _handles->table_read.reset();
+ _handles->plan.reset();
+ _handles->read_builder.reset();
+}
+
+void PaimonRustTableReader::_close_table() {
+ if (!_handles) {
+ return;
+ }
+ _close_split_reader();
+ _handles->table.reset();
+ _opened_table_key.reset();
+}
+
+Status PaimonRustTableReader::_apply_predicate() {
+ if (_conjuncts.empty() || !_handles || !_handles->table ||
!_handles->read_builder) {
+ return Status::OK();
+ }
+ LOG(INFO) << "paimon-rust predicate pushdown: " << _conjuncts.size() << "
conjunct(s) input";
+ // The conjunct VSlotRefs carry table global indices (positions), so the v2
+ // converter mode resolves fields by the projected column names; partition
+ // keys are excluded because the rust reader does not read them.
+ std::vector<std::string> names;
+ std::vector<DataTypePtr> types;
+ names.reserve(_projected_columns.size());
+ types.reserve(_projected_columns.size());
+ for (const auto& col : _projected_columns) {
+ if (col.is_partition_key) {
+ continue;
+ }
+ names.push_back(col.name);
+ types.push_back(col.type);
+ }
+ PaimonRustPredicateConverter converter(names, types,
_handles->table.get());
+ paimon_predicate* predicate = converter.build(_conjuncts);
+ if (predicate == nullptr) {
+ LOG(INFO) << "paimon-rust predicate pushdown: nothing convertible, no
filter applied";
+ return Status::OK();
+ }
+ // paimon_read_builder_with_filter consumes the predicate (ownership moves
to
+ // the builder) on every path, so we must not free it here.
+ if (paimon_error* err =
+ paimon_read_builder_with_filter(_handles->read_builder.get(),
predicate)) {
+ return Status::InternalError("paimon-rust apply filter failed: {}",
consume_error(err));
+ }
+ LOG(INFO) << "paimon-rust predicate pushdown: applied";
+ return Status::OK();
+}
+
+Status PaimonRustTableReader::_fill_block_from_record_batch(
+ const std::shared_ptr<arrow::RecordBatch>& batch, Block* block, size_t
rows) {
+ SCOPED_TIMER(_rust_arrow_to_block_time);
+ DORIS_CHECK(batch != nullptr);
+ DORIS_CHECK(block != nullptr);
+ std::unordered_set<size_t> materialized_indices;
+ materialized_indices.reserve(_projected_columns.size());
+ {
+ auto columns_guard = block->mutate_columns_scoped();
+ auto& columns = columns_guard.mutable_columns();
+ for (int c = 0; c < batch->num_columns(); ++c) {
+ const auto& field = batch->schema()->field(c);
+ if (field->name() == VALUE_KIND_FIELD) {
+ continue;
+ }
+ // Projected column names are FE-normalized to lowercase.
+ // paimon-rust's case_sensitive=false setting also case-folds
column
+ // names in the schema output, so exact match works — but tolerate
+ // mixed-case Rust output by folding here as well.
+ auto it = _output_name_to_idx.find(field->name());
+ if (it == _output_name_to_idx.end()) {
+ it = _output_name_to_idx.find(to_lower(field->name()));
+ }
+ if (it == _output_name_to_idx.end()) {
+ // Skip columns that are not in the block (e.g. columns
dropped by
+ // slot pruning).
+ continue;
+ }
+ const auto output_idx = it->second;
+ if (!materialized_indices.emplace(output_idx).second) {
+ return Status::InternalError("paimon-rust returned duplicate
column '{}'",
+ field->name());
+ }
+ try {
+
RETURN_IF_ERROR(columns_guard.get_datatype_by_position(output_idx)
+ ->get_serde()
+
->read_column_from_arrow(*columns[output_idx],
+
batch->column(c).get(), 0, rows,
+ _ctz));
+ } catch (Exception& e) {
+ return Status::InternalError("Failed to convert from arrow to
block: {}", e.what());
+ }
+ }
+ }
+ // Partition columns and other projected columns absent from the arrow
batch
+ // are back-filled from split metadata / defaults.
+ return _fill_non_arrow_columns(block, rows, materialized_indices);
Review Comment:
[P1] Apply bounded-string truncation on the direct Rust path
This direct Arrow path returns after filling absent columns and never runs
TableReader::finalize_chunk, whose final step enforces
truncate_char_or_varchar_columns. The pinned Rust reader maps both historical
and current CHAR/VARCHAR types to lengthless Arrow Utf8, so after narrowing
VARCHAR(10) to VARCHAR(3), an old value such as abcdefghij is still returned
unchanged here while the normal path returns abc when truncation is enabled.
Apply the shared bounded-string truncation semantics before returning this
block (or otherwise preserve and enforce the file/table bounds).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
