924060929 commented on code in PR #66913:
URL: https://github.com/apache/doris/pull/66913#discussion_r3909888786
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/source/IcebergScanNode.java:
##########
@@ -307,12 +309,16 @@ protected void doInitialize() throws UserException {
// These tables are always readable regardless of format
version
formatVersion = MIN_DELETE_FILE_SUPPORT_VERSION;
}
- preExecutionAuthenticator =
source.getCatalog().getExecutionAuthenticator();
- storagePropertiesMap =
VendedCredentialsFactory.getStoragePropertiesMapWithVendedCredentials(
-
source.getCatalog().getCatalogProperty().getMetastoreProperties(),
-
source.getCatalog().getCatalogProperty().getStoragePropertiesMap(),
- icebergTable
- );
+ if (runtimeContext == null) {
+ preExecutionAuthenticator =
source.getCatalog().getExecutionAuthenticator();
+ storagePropertiesMap =
VendedCredentialsFactory.getStoragePropertiesMapWithVendedCredentials(
+
source.getCatalog().getCatalogProperty().getMetastoreProperties(),
+
source.getCatalog().getCatalogProperty().getStoragePropertiesMap(),
icebergTable);
+ } else {
+ preExecutionAuthenticator = runtimeContext.getAuthenticator();
Review Comment:
Fixed in 7ccd2dd646c. IcebergScanNode now captures both schema-mapping
options from the pinned IcebergSnapshotCacheValue before partition/schema
initialization, so schema conversion, delete handling, partition metadata, and
FE-to-BE thrift flags all use the frozen scan generation. Added a focused test
covering the frozen options after refresh.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergExternalMetaCache.java:
##########
@@ -133,10 +147,34 @@ IcebergSnapshotEntryKey.class,
IcebergSnapshotCacheValue.class, defaultEntryCach
public Table getIcebergTable(ExternalTable dorisTable) {
NameMapping nameMapping = dorisTable.getOrBuildNameMapping();
- return
tableEntry.get(nameMapping.getCtlId()).get(nameMapping).getIcebergTable();
+ IcebergTableCacheValue.Lease lease = statementLease(nameMapping);
+ if (lease != null) {
+ return lease.getIcebergTable();
+ }
+ throw new IllegalStateException("Iceberg table access outside a
statement must use a scoped borrower");
Review Comment:
Fixed in 7ccd2dd646c. Native and HMS Iceberg background schema callers now
project snapshot, schema, and native row-lineage metadata inside one
statement/operation lease. Pinned snapshots reuse their retained generation;
no-context getFullSchema/getColumn/getSchemaCacheValue no longer reacquire a
live table generation. Added focused native, HMS, and scoped-cache tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
