sundapeng commented on code in PR #10350:
URL: https://github.com/apache/paimon/pull/10350#discussion_r4367886601
##########
paimon-filesystems/paimon-jindo/src/main/java/org/apache/paimon/jindo/JindoFileIO.java:
##########
@@ -181,29 +225,40 @@ public Options hadoopOptions(Path path, String opType) {
shouldCache = writeCacheEnabled && shouldCache(path);
} else if (opType.equalsIgnoreCase("meta")) {
shouldCache = metaCacheEnabled && shouldCache(path);
+ } else if (opType.equalsIgnoreCase("exists")) {
+ shouldCache = existsCacheEnabled && shouldCache(path);
}
- if (shouldCache) {
- return hadoopOptionsWithCache;
- } else {
+ if (!shouldCache) {
return hadoopOptions;
+ } else if (cacheRouting != null) {
+ return cacheTargetOptions.get(cacheRouting.targetOf(path).name);
Review Comment:
Thanks, 5af45bcc21 takes the second option. With io-cache routing,
`hadoopOptions(path, opType)` now returns the origin's OSS options for every
path and operation, so the Lance and Vortex converters only build clients for
the OSS endpoint and never see an HTTP, IP or path-style target. The
per-request selection moves to the package-private `routedHadoopOptions`, which
the routing tests use; JindoCache mode keeps its per-path options.
`JindoIoCacheRoutingTest#testLanceAndVortexGetTheOssOptions` configures an
`http://` path-style cluster target and an accelerator target, and checks that
`hadoopOptions` returns the OSS options for data, manifest and LATEST paths
under read, write and meta, while `routedHadoopOptions` still selects the
cluster for data reads. Routing native format IO through a target can follow
once the converters honor the selected scheme and addressing.
##########
paimon-filesystems/paimon-jindo/src/main/java/org/apache/paimon/jindo/JindoFileIO.java:
##########
@@ -181,29 +225,40 @@ public Options hadoopOptions(Path path, String opType) {
shouldCache = writeCacheEnabled && shouldCache(path);
} else if (opType.equalsIgnoreCase("meta")) {
shouldCache = metaCacheEnabled && shouldCache(path);
+ } else if (opType.equalsIgnoreCase("exists")) {
+ shouldCache = existsCacheEnabled && shouldCache(path);
}
- if (shouldCache) {
- return hadoopOptionsWithCache;
- } else {
+ if (!shouldCache) {
return hadoopOptions;
+ } else if (cacheRouting != null) {
+ return cacheTargetOptions.get(cacheRouting.targetOf(path).name);
Review Comment:
Thanks, 5af45bcc21 takes the second option. With io-cache routing,
`hadoopOptions(path, opType)` now returns the origin's OSS options for every
path and operation, so the Lance and Vortex converters only build clients for
the OSS endpoint and never see an HTTP, IP or path-style target. The
per-request selection moves to the package-private `routedHadoopOptions`, which
the routing tests use; JindoCache mode keeps its per-path options.
`JindoIoCacheRoutingTest#testLanceAndVortexGetTheOssOptions` configures an
`http://` path-style cluster target and an accelerator target, and checks that
`hadoopOptions` returns the OSS options for data, manifest and LATEST paths
under read, write and meta; the existing per-target test still gets the cluster
for data reads through `routedHadoopOptions`. Routing native format IO through
a target can follow once the converters honor the selected scheme and
addressing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
