morningman commented on code in PR #44267:
URL: https://github.com/apache/doris/pull/44267#discussion_r1865305204


##########
fe/be-java-extensions/hudi-scanner/src/main/scala/org/apache/doris/hudi/BaseSplitReader.scala:
##########
@@ -497,8 +499,11 @@ abstract class BaseSplitReader(val split: HoodieSplit) {
                                 options: Map[String, String],
                                 hadoopConf: Configuration): PartitionedFile => 
Iterator[InternalRow] = {
     partitionedFile => {
-      val reader = new HoodieAvroHFileReader(
-        hadoopConf, partitionedFile.filePath.toPath, new 
CacheConfig(hadoopConf))
+      var hadoopStorageConfiguration = new 
HadoopStorageConfiguration(hadoopConf);
+      var sotragePath = new StoragePath(partitionedFile.toPath.toUri.getPath);

Review Comment:
   ```suggestion
         var storagePath = new 
StoragePath(partitionedFile.toPath.toUri.getPath);
   ```



##########
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/source/PaimonScanNode.java:
##########
@@ -142,6 +141,19 @@ public static <T> String encodeObjectToString(T t) {
     @Override
     protected void setScanParams(TFileRangeDesc rangeDesc, Split split) {
         if (split instanceof PaimonSplit) {
+            PaimonSplit paimonSplit = (PaimonSplit) split;
+            if (rangeDesc.getFormatType() == TFileFormatType.FORMAT_JNI
+                    && paimonSplit.getSplit() == null) {
+                // fall back to JNI reader

Review Comment:
   ```suggestion
                   // fall back to native reader
   ```



##########
be/src/vec/exec/scan/vfile_scanner.cpp:
##########
@@ -754,28 +750,10 @@ Status VFileScanner::_get_next_reader() {
 
         // create reader for specific format
         Status init_status;
-        TFileFormatType::type format_type = _params->format_type;
+        TFileFormatType::type format_type = range.format_type;
         // JNI reader can only push down column value range
         bool push_down_predicates =
                 !_is_load && _params->format_type != 
TFileFormatType::FORMAT_JNI;
-        if (format_type == TFileFormatType::FORMAT_JNI && 
range.__isset.table_format_params) {

Review Comment:
   This need to be reserved.
   Because when upgrading, the BE will be upgrade, and the old FE will send old 
format request to new BE.
   You can reserve this logic but mark it as deprecated in 3.1



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Reply via email to