github-actions[bot] commented on code in PR #69527:
URL: https://github.com/apache/doris/pull/69527#discussion_r4366424083
##########
fe/fe-core/src/main/java/org/apache/doris/tablefunction/LocalTableValuedFunction.java:
##########
@@ -135,6 +143,38 @@ private void getFileListFromBackend() throws
AnalysisException {
}
}
+ /** Apply LOCAL admission to sinks that select the built-in local JNI
writer as well. */
+ public static void checkSinkEnabled(String tvfName, Map<String, String>
properties) {
+ String writerClass = properties.getOrDefault("writer_class", "");
+ // JNI consumes a C string, and ScannerLoader accepts slash-separated
class names.
+ int nul = writerClass.indexOf('\0');
+ if (nul >= 0) {
+ writerClass = writerClass.substring(0, nul);
+ }
+ if ("local".equalsIgnoreCase(tvfName)
+ ||
"org.apache.doris.writer.LocalFileJniWriter".equals(writerClass.replace('/',
'.'))) {
Review Comment:
[P1] Gate native HDFS sinks that resolve to local storage
With `enable_local_tvf=false`, a global LOAD user can still run `INSERT INTO
hdfs('file_path'='/tmp/local_gate_', 'format'='csv', 'fs.defaultFS'='file:///')
SELECT 1` without any `writer_class`. This predicate admits it, and
`TVFTableSink` calls `StorageProperties.createPrimary` without URI validation,
preserving the local default filesystem. The native `FILE_HDFS`/`FileFactory`
route then reaches [the pinned libhdfs provider
selection](https://github.com/apache/doris-thirdparty/blob/hadoop-3.4.2.2-for-doris/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/hdfs.c#L690-L909),
which selects Hadoop's local filesystem and creates BE-local files. With
`delete_existing_files=true` and a prefix such as `/tmp/victim/prefix_`, the
same configuration also makes `InsertIntoTVFCommand` call
`DFSFileSystem.deleteDirectory`, recursively deleting `/tmp/victim/` on the FE
before BE execution. Gate or reject effective local HDFS providers before
either side effect, and add dis
abled INSERT/EXPLAIN and final-binding coverage. This is distinct from the
fixed LocalFileJniWriter thread: it uses the native HDFS path with no JNI
writer class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
