voonhous commented on code in PR #19478:
URL: https://github.com/apache/hudi/pull/19478#discussion_r3709744044
##########
hudi-trino/src/main/java/io/trino/plugin/hudi/split/HudiSplitFactory.java:
##########
@@ -127,7 +127,10 @@ private static List<HudiSplit> createSplitsForBaseFile(
}
ImmutableList.Builder<HudiSplit> splits = ImmutableList.builder();
- long targetSplitSizeInBytes = Math.max(targetSplitSize.toBytes(),
baseFile.getPathInfo().getBlockSize());
+ // Slicing is governed solely by the target split size; the block size
reported by
+ // storage is not meaningful on object stores and must not influence
split sizing.
+ long targetSplitSizeInBytes = targetSplitSize.toBytes();
Review Comment:
Moved it into the `HudiSplitFactory` constructor next to the
`requireNonNull`s, and made `createHudiSplits` **private** so the constructor
is the only entry point that can be handed a bad target.
`testCreateHudiSplitsWithZeroTargetSplitSize` now asserts on construction
instead of on a file slice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
