[jira] [Updated] (HDFS-7531) Improve the concurrent access on FsVolumeList

2015-08-30 Thread Vinod Kumar Vavilapalli (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kumar Vavilapalli updated HDFS-7531:
--
Target Version/s: 2.7.0, 3.0.0  (was: 3.0.0, 2.7.0)
   Fix Version/s: 2.6.1

[~sjlee0] backported this to 2.6.1, with minor changes to accomodate older 
JDKs. I just pushed the commit to 2.6.1 after running compilation and 
TestFsDatasetImpl which changed in the patch.

> Improve the concurrent access on FsVolumeList
> -
>
> Key: HDFS-7531
> URL: https://issues.apache.org/jira/browse/HDFS-7531
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.6.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
>  Labels: 2.6.1-candidate
> Fix For: 2.7.0, 2.6.1
>
> Attachments: HDFS-7531.000.patch, HDFS-7531.001.patch, 
> HDFS-7531.002.patch
>
>
> {{FsVolumeList}} uses {{synchronized}} to protect the update on 
> {{FsVolumeList#volumes}}, while various operations (e.g., {{checkDirs()}}, 
> {{getAvailable()}}) iterate {{volumes}} without protection.
> This JIRA proposes to use {{AtomicReference}} to encapture {{volumes}} to 
> provide better concurrent access.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HDFS-7531) Improve the concurrent access on FsVolumeList

2015-07-17 Thread Vinod Kumar Vavilapalli (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kumar Vavilapalli updated HDFS-7531:
--
Labels: 2.6.1-candidate  (was: )

> Improve the concurrent access on FsVolumeList
> -
>
> Key: HDFS-7531
> URL: https://issues.apache.org/jira/browse/HDFS-7531
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.6.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
>  Labels: 2.6.1-candidate
> Fix For: 2.7.0
>
> Attachments: HDFS-7531.000.patch, HDFS-7531.001.patch, 
> HDFS-7531.002.patch
>
>
> {{FsVolumeList}} uses {{synchronized}} to protect the update on 
> {{FsVolumeList#volumes}}, while various operations (e.g., {{checkDirs()}}, 
> {{getAvailable()}}) iterate {{volumes}} without protection.
> This JIRA proposes to use {{AtomicReference}} to encapture {{volumes}} to 
> provide better concurrent access.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HDFS-7531) Improve the concurrent access on FsVolumeList

2014-12-17 Thread Colin Patrick McCabe (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colin Patrick McCabe updated HDFS-7531:
---
   Resolution: Fixed
Fix Version/s: 2.7.0
   Status: Resolved  (was: Patch Available)

+1.  Thanks, Eddy

> Improve the concurrent access on FsVolumeList
> -
>
> Key: HDFS-7531
> URL: https://issues.apache.org/jira/browse/HDFS-7531
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.6.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Fix For: 2.7.0
>
> Attachments: HDFS-7531.000.patch, HDFS-7531.001.patch, 
> HDFS-7531.002.patch
>
>
> {{FsVolumeList}} uses {{synchronized}} to protect the update on 
> {{FsVolumeList#volumes}}, while various operations (e.g., {{checkDirs()}}, 
> {{getAvailable()}}) iterate {{volumes}} without protection.
> This JIRA proposes to use {{AtomicReference}} to encapture {{volumes}} to 
> provide better concurrent access.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HDFS-7531) Improve the concurrent access on FsVolumeList

2014-12-17 Thread Lei (Eddy) Xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lei (Eddy) Xu updated HDFS-7531:

Attachment: HDFS-7531.002.patch

Thanks for the suggestions [~cmccabe].

I updated the patch based on your comments.

> Improve the concurrent access on FsVolumeList
> -
>
> Key: HDFS-7531
> URL: https://issues.apache.org/jira/browse/HDFS-7531
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.6.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-7531.000.patch, HDFS-7531.001.patch, 
> HDFS-7531.002.patch
>
>
> {{FsVolumeList}} uses {{synchronized}} to protect the update on 
> {{FsVolumeList#volumes}}, while various operations (e.g., {{checkDirs()}}, 
> {{getAvailable()}}) iterate {{volumes}} without protection.
> This JIRA proposes to use {{AtomicReference}} to encapture {{volumes}} to 
> provide better concurrent access.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HDFS-7531) Improve the concurrent access on FsVolumeList

2014-12-16 Thread Lei (Eddy) Xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lei (Eddy) Xu updated HDFS-7531:

Attachment: HDFS-7531.001.patch

Thanks for the reviews, [~cmccabe] and [~wheat9] !

I updated the patch to use {{compareAndSet}} instead of {{synchronized}} for 
{{addVolume}} and {{removeVolume}}.

Also changed to use JDK 7 features. 

> Improve the concurrent access on FsVolumeList
> -
>
> Key: HDFS-7531
> URL: https://issues.apache.org/jira/browse/HDFS-7531
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.6.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-7531.000.patch, HDFS-7531.001.patch
>
>
> {{FsVolumeList}} uses {{synchronized}} to protect the update on 
> {{FsVolumeList#volumes}}, while various operations (e.g., {{checkDirs()}}, 
> {{getAvailable()}}) iterate {{volumes}} without protection.
> This JIRA proposes to use {{AtomicReference}} to encapture {{volumes}} to 
> provide better concurrent access.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HDFS-7531) Improve the concurrent access on FsVolumeList

2014-12-15 Thread Lei (Eddy) Xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lei (Eddy) Xu updated HDFS-7531:

Summary: Improve the concurrent access on FsVolumeList  (was: Improve the 
concurrent access of FsVolumeList)

> Improve the concurrent access on FsVolumeList
> -
>
> Key: HDFS-7531
> URL: https://issues.apache.org/jira/browse/HDFS-7531
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.6.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-7531.000.patch
>
>
> {{FsVolumeList}} uses {{synchronized}} to protect the update on 
> {{FsVolumeList#volumes}}, while various operations (e.g., {{checkDirs()}}, 
> {{getAvailable()}}) iterate {{volumes}} without protection.
> This JIRA proposes to use {{AtomicReference}} to encapture {{volumes}} to 
> provide better concurrent access.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)