[
https://issues.apache.org/jira/browse/HADOOP-19291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17887919#comment-17887919
]
ASF GitHub Bot commented on HADOOP-19291:
-----------------------------------------
steveloughran commented on PR #7079:
URL: https://github.com/apache/hadoop/pull/7079#issuecomment-2402228009
+1 to cutting the validation in raw local; the way the nio reads are
executed it is safe.
I should note that it only surfaces with file:// ; ChecksumFS + RawLocal,
with a large enough file that multiple large ranges to be supplied close
to/adjac
We missed this because we were only doing small files in the unit tests and
testing against HDFS and S3 in the full stack tests. "our tests were too good".
> `CombinedFileRange.merge` should not convert disjoint ranges into overlapped
> ones
> ---------------------------------------------------------------------------------
>
> Key: HADOOP-19291
> URL: https://issues.apache.org/jira/browse/HADOOP-19291
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
> Affects Versions: 3.3.9, 3.5.0, 3.4.1
> Reporter: Dongjoon Hyun
> Priority: Major
> Labels: pull-request-available
> Attachments: Screenshot 2024-09-28 at 22.02.01.png
>
>
> Currently, Hadoop has a bug to convert disjoint ranges into overlapped ones
> and eventually fails by itself.
> {code:java}
> + public void testMergeSortedRanges() {
> + List<FileRange> input = asList(
> + createFileRange(13816220, 24, null),
> + createFileRange(13816244, 7423960, null)
> + );
> + assertIsNotOrderedDisjoint(input, 100, 800);
> + final List<CombinedFileRange> outputList = mergeSortedRanges(
> + sortRangeList(input), 100, 1001, 2500);
> +
> + assertRangeListSize(outputList, 1);
> + assertFileRange(outputList.get(0), 13816200, 7424100);
> + }
> {code}
> !Screenshot 2024-09-28 at 22.02.01.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
