[jira] [Updated] (HDFS-15119) Allow expiration of cached locations in DFSInputStream

2020-01-27 Thread Ahmed Hussein (Jira)


 [ 
https://issues.apache.org/jira/browse/HDFS-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ahmed Hussein updated HDFS-15119:
-
Attachment: HDFS-15119-branch-2.10.003.patch

> Allow expiration of cached locations in DFSInputStream
> --
>
> Key: HDFS-15119
> URL: https://issues.apache.org/jira/browse/HDFS-15119
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: dfsclient
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Minor
> Fix For: 3.3.0, 3.1.4, 3.2.2
>
> Attachments: HDFS-15119-branch-2.10.003.patch, HDFS-15119.001.patch, 
> HDFS-15119.002.patch, HDFS-15119.003.patch
>
>
> Staleness and other transient conditions can affect reads for a long time 
> since the block locations may not be re-fetched. It makes sense to make 
> cached locations to expire.
> For example, we may not take advantage of local-reads since the nodes are 
> blacklisted and have not been updated.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDFS-15119) Allow expiration of cached locations in DFSInputStream

2020-01-24 Thread Kihwal Lee (Jira)


 [ 
https://issues.apache.org/jira/browse/HDFS-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kihwal Lee updated HDFS-15119:
--
Fix Version/s: 3.2.2
   3.1.4
   3.3.0
 Hadoop Flags: Reviewed
   Resolution: Fixed
   Status: Resolved  (was: Patch Available)

> Allow expiration of cached locations in DFSInputStream
> --
>
> Key: HDFS-15119
> URL: https://issues.apache.org/jira/browse/HDFS-15119
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: dfsclient
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Minor
> Fix For: 3.3.0, 3.1.4, 3.2.2
>
> Attachments: HDFS-15119.001.patch, HDFS-15119.002.patch, 
> HDFS-15119.003.patch
>
>
> Staleness and other transient conditions can affect reads for a long time 
> since the block locations may not be re-fetched. It makes sense to make 
> cached locations to expire.
> For example, we may not take advantage of local-reads since the nodes are 
> blacklisted and have not been updated.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDFS-15119) Allow expiration of cached locations in DFSInputStream

2020-01-15 Thread Ahmed Hussein (Jira)


 [ 
https://issues.apache.org/jira/browse/HDFS-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ahmed Hussein updated HDFS-15119:
-
Attachment: HDFS-15119.003.patch
Status: Patch Available  (was: Open)

> Allow expiration of cached locations in DFSInputStream
> --
>
> Key: HDFS-15119
> URL: https://issues.apache.org/jira/browse/HDFS-15119
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: dfsclient
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Minor
> Attachments: HDFS-15119.001.patch, HDFS-15119.002.patch, 
> HDFS-15119.003.patch
>
>
> Staleness and other transient conditions can affect reads for a long time 
> since the block locations may not be re-fetched. It makes sense to make 
> cached locations to expire.
> For example, we may not take advantage of local-reads since the nodes are 
> blacklisted and have not been updated.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDFS-15119) Allow expiration of cached locations in DFSInputStream

2020-01-14 Thread Ahmed Hussein (Jira)


 [ 
https://issues.apache.org/jira/browse/HDFS-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ahmed Hussein updated HDFS-15119:
-
Attachment: HDFS-15119.002.patch

> Allow expiration of cached locations in DFSInputStream
> --
>
> Key: HDFS-15119
> URL: https://issues.apache.org/jira/browse/HDFS-15119
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: dfsclient
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Minor
> Attachments: HDFS-15119.001.patch, HDFS-15119.002.patch
>
>
> Staleness and other transient conditions can affect reads for a long time 
> since the block locations may not be re-fetched. It makes sense to make 
> cached locations to expire.
> For example, we may not take advantage of local-reads since the nodes are 
> blacklisted and have not been updated.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDFS-15119) Allow expiration of cached locations in DFSInputStream

2020-01-14 Thread Ahmed Hussein (Jira)


 [ 
https://issues.apache.org/jira/browse/HDFS-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ahmed Hussein updated HDFS-15119:
-
Status: In Progress  (was: Patch Available)

> Allow expiration of cached locations in DFSInputStream
> --
>
> Key: HDFS-15119
> URL: https://issues.apache.org/jira/browse/HDFS-15119
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: dfsclient
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Minor
> Attachments: HDFS-15119.001.patch
>
>
> Staleness and other transient conditions can affect reads for a long time 
> since the block locations may not be re-fetched. It makes sense to make 
> cached locations to expire.
> For example, we may not take advantage of local-reads since the nodes are 
> blacklisted and have not been updated.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Updated] (HDFS-15119) Allow expiration of cached locations in DFSInputStream

2020-01-13 Thread Ahmed Hussein (Jira)


 [ 
https://issues.apache.org/jira/browse/HDFS-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ahmed Hussein updated HDFS-15119:
-
Attachment: HDFS-15119.001.patch
Status: Patch Available  (was: Open)

> Allow expiration of cached locations in DFSInputStream
> --
>
> Key: HDFS-15119
> URL: https://issues.apache.org/jira/browse/HDFS-15119
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: dfsclient
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Minor
> Attachments: HDFS-15119.001.patch
>
>
> Staleness and other transient conditions can affect reads for a long time 
> since the block locations may not be re-fetched. It makes sense to make 
> cached locations to expire.
> For example, we may not take advantage of local-reads since the nodes are 
> blacklisted and have not been updated.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org