[jira] [Updated] (HDFS-11379) DFSInputStream may infinite loop requesting block locations
[ https://issues.apache.org/jira/browse/HDFS-11379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-11379: - Resolution: Fixed Fix Version/s: 2.7.4 Status: Resolved (was: Patch Available) > DFSInputStream may infinite loop requesting block locations > --- > > Key: HDFS-11379 > URL: https://issues.apache.org/jira/browse/HDFS-11379 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs-client >Affects Versions: 2.7.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha3 > > Attachments: HDFS-11379.branch-2.7.patch, HDFS-11379.branch-2.patch, > HDFS-11379.trunk.patch > > > DFSInputStream creation caches file size and initial range of locations. If > the file is truncated (or replaced) and the client attempts to read outside > the initial range, the client goes into a tight infinite looping requesting > locations for the nonexistent range. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11379) DFSInputStream may infinite loop requesting block locations
[ https://issues.apache.org/jira/browse/HDFS-11379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-11379: - Status: Patch Available (was: Reopened) > DFSInputStream may infinite loop requesting block locations > --- > > Key: HDFS-11379 > URL: https://issues.apache.org/jira/browse/HDFS-11379 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs-client >Affects Versions: 2.7.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Fix For: 2.8.0, 3.0.0-alpha3 > > Attachments: HDFS-11379.branch-2.7.patch, HDFS-11379.branch-2.patch, > HDFS-11379.trunk.patch > > > DFSInputStream creation caches file size and initial range of locations. If > the file is truncated (or replaced) and the client attempts to read outside > the initial range, the client goes into a tight infinite looping requesting > locations for the nonexistent range. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11379) DFSInputStream may infinite loop requesting block locations
[
https://issues.apache.org/jira/browse/HDFS-11379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Chen updated HDFS-11379:
-
Attachment: HDFS-11379.branch-2.7.patch
2.7 patch has some trivial conflicts: {{fetchBlockAt}} signature, imports in
the test class, and the renamed
{{DFSConfigKeys.DFS_CLIENT_READ_PREFETCH_SIZE_KEY}}
> DFSInputStream may infinite loop requesting block locations
> ---
>
> Key: HDFS-11379
> URL: https://issues.apache.org/jira/browse/HDFS-11379
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Affects Versions: 2.7.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Fix For: 2.8.0, 3.0.0-alpha3
>
> Attachments: HDFS-11379.branch-2.7.patch, HDFS-11379.branch-2.patch,
> HDFS-11379.trunk.patch
>
>
> DFSInputStream creation caches file size and initial range of locations. If
> the file is truncated (or replaced) and the client attempts to read outside
> the initial range, the client goes into a tight infinite looping requesting
> locations for the nonexistent range.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11379) DFSInputStream may infinite loop requesting block locations
[ https://issues.apache.org/jira/browse/HDFS-11379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11379: -- Fix Version/s: (was: 2.8.1) 2.8.0 > DFSInputStream may infinite loop requesting block locations > --- > > Key: HDFS-11379 > URL: https://issues.apache.org/jira/browse/HDFS-11379 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs-client >Affects Versions: 2.7.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Fix For: 2.8.0, 3.0.0-alpha3 > > Attachments: HDFS-11379.branch-2.patch, HDFS-11379.trunk.patch > > > DFSInputStream creation caches file size and initial range of locations. If > the file is truncated (or replaced) and the client attempts to read outside > the initial range, the client goes into a tight infinite looping requesting > locations for the nonexistent range. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11379) DFSInputStream may infinite loop requesting block locations
[ https://issues.apache.org/jira/browse/HDFS-11379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11379: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.1 3.0.0-alpha3 Status: Resolved (was: Patch Available) I've committed this to trunk, branch-2 and branch-2.8. > DFSInputStream may infinite loop requesting block locations > --- > > Key: HDFS-11379 > URL: https://issues.apache.org/jira/browse/HDFS-11379 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs-client >Affects Versions: 2.7.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Fix For: 3.0.0-alpha3, 2.8.1 > > Attachments: HDFS-11379.branch-2.patch, HDFS-11379.trunk.patch > > > DFSInputStream creation caches file size and initial range of locations. If > the file is truncated (or replaced) and the client attempts to read outside > the initial range, the client goes into a tight infinite looping requesting > locations for the nonexistent range. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11379) DFSInputStream may infinite loop requesting block locations
[ https://issues.apache.org/jira/browse/HDFS-11379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HDFS-11379: --- Target Version/s: 2.9.0, 3.0.0-alpha3, 2.8.1 (was: 2.8.1) > DFSInputStream may infinite loop requesting block locations > --- > > Key: HDFS-11379 > URL: https://issues.apache.org/jira/browse/HDFS-11379 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs-client >Affects Versions: 2.7.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HDFS-11379.branch-2.patch, HDFS-11379.trunk.patch > > > DFSInputStream creation caches file size and initial range of locations. If > the file is truncated (or replaced) and the client attempts to read outside > the initial range, the client goes into a tight infinite looping requesting > locations for the nonexistent range. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11379) DFSInputStream may infinite loop requesting block locations
[ https://issues.apache.org/jira/browse/HDFS-11379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-11379: --- Attachment: HDFS-11379.branch-2.patch HDFS-11379.trunk.patch Main issue is getFinalizedBlockRange takes an offset and length. It will loop forever until it receives blocks that satisfy length, even if no locations were returned. Simplest change is to abort when no locations are returned in the expected range. Changed two duplicated code regions, sans whether to cache, to invoke a common method for fetching locations. > DFSInputStream may infinite loop requesting block locations > --- > > Key: HDFS-11379 > URL: https://issues.apache.org/jira/browse/HDFS-11379 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs-client >Affects Versions: 2.7.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HDFS-11379.branch-2.patch, HDFS-11379.trunk.patch > > > DFSInputStream creation caches file size and initial range of locations. If > the file is truncated (or replaced) and the client attempts to read outside > the initial range, the client goes into a tight infinite looping requesting > locations for the nonexistent range. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11379) DFSInputStream may infinite loop requesting block locations
[ https://issues.apache.org/jira/browse/HDFS-11379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-11379: --- Status: Patch Available (was: Open) > DFSInputStream may infinite loop requesting block locations > --- > > Key: HDFS-11379 > URL: https://issues.apache.org/jira/browse/HDFS-11379 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs-client >Affects Versions: 2.7.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Critical > Attachments: HDFS-11379.branch-2.patch, HDFS-11379.trunk.patch > > > DFSInputStream creation caches file size and initial range of locations. If > the file is truncated (or replaced) and the client attempts to read outside > the initial range, the client goes into a tight infinite looping requesting > locations for the nonexistent range. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
