JingsongLi commented on code in PR #10178: URL: https://github.com/apache/paimon/pull/10178#discussion_r4245350271
########## paimon-filesystems/paimon-oss-impl/src/main/java/org/apache/paimon/oss/RESTTokenCredentialsProvider.java: ########## @@ -0,0 +1,112 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.paimon.oss; + +import org.apache.paimon.options.Options; +import org.apache.paimon.rest.RESTToken; +import org.apache.paimon.rest.RESTTokenRefresher; + +import com.aliyun.oss.common.auth.Credentials; +import com.aliyun.oss.common.auth.CredentialsProvider; +import com.aliyun.oss.common.auth.DefaultCredentials; +import org.apache.hadoop.conf.Configuration; + +import java.net.URI; +import java.util.Locale; +import java.util.Map; + +import static org.apache.paimon.utils.Preconditions.checkArgument; + +/** + * An OSS {@link CredentialsProvider} that refreshes the data token of a REST catalog table by + * itself and is asked on every request, so streams that are already open also pick up refreshed + * tokens. + */ +public class RESTTokenCredentialsProvider implements CredentialsProvider { + + /** Prefix under which {@link OSSFileIO} passes the catalog options to this provider. */ + static final String CATALOG_OPTIONS_PREFIX = "fs.oss.rest-token.catalog."; + + private static final String ACCESS_KEY_ID = "fs.oss.accessKeyId"; + private static final String ACCESS_KEY_SECRET = "fs.oss.accessKeySecret"; + private static final String SECURITY_TOKEN = "fs.oss.securityToken"; + + private final RESTTokenRefresher refresher; + + // The last token and the credentials built from it, swapped together. + private volatile Resolved last; + + public RESTTokenCredentialsProvider(URI uri, Configuration conf) { + Options options = new Options(conf.getPropsWithPrefix(CATALOG_OPTIONS_PREFIX)); + checkArgument( + RESTTokenRefresher.isConfigured(options), + "No REST catalog table configured under '%s'.", + CATALOG_OPTIONS_PREFIX); + this.refresher = RESTTokenRefresher.fromOptions(options); + } + + @Override + public void setCredentials(Credentials credentials) { + throw new UnsupportedOperationException( + "Credentials come from the REST catalog and cannot be set."); + } + + @Override + public Credentials getCredentials() { + RESTToken token = refresher.token(); Review Comment: [P2] Check presigned validity against the credentials actually used to sign This provider can now refresh independently while `RESTTokenFileIO.createBlobPresignedUrl` is materializing a BLOB, but that wrapper's before/after validity checks still refer to its original `FileIOWithToken.token`. With cached token A having 61 minutes left, a 10-minute request passes; materialization crosses A's refresh window, this provider obtains shorter-lived B, and OSSBlobPresigner's SDK call signs with B while the wrapper accepts the URL using A's remaining lifetime. I exercised the production wrapper, refresher, OSSBlobPresigner and inherited OSS SDK signing against real local REST HTTP, with storage HEAD/multipart-copy replies stubbed and coherent logical clock advancement. The returned URL used B and its actual Expires timestamp was 179,490 ms later than B's actual expiresAtMillis. A fresh wrapper starting with B correctly rejected the same validity; a static-A signing control remained within A's lifetime. Please carry the signing token's expiry/minimum-validity requir ement through the presign handoff and validate that actual bundle after materialization, rather than only the wrapper token snapshot. Include rotation to a shorter-lived token during materialization in the regression coverage. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
