[jira] [Commented] (HDFS-9666) Enable hdfs-client to read even remote SSD/RAM prior to local disk replica to improve random read

2017-06-07 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040413#comment-16040413 ] Yu Li commented on HDFS-9666: - Thanks for chiming in with performance data [~whisper_deng]. Maybe we should

[jira] [Commented] (HDFS-10690) Optimize insertion/removal of replica in ShortCircuitCache.java

2016-08-09 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413040#comment-15413040 ] Yu Li commented on HDFS-10690: -- {quote} Run GET queries with 64 YCSB processes for 30 minutes, record the QPS

[jira] [Commented] (HDFS-9666) Enable hdfs-client to read even remote SSD/RAM prior to local disk replica to improve random read

2016-01-20 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110188#comment-15110188 ] Yu Li commented on HDFS-9666: - bq. However it looked like the benefits of reading from remote RAM were

[jira] [Commented] (HDFS-6441) Add ability to exclude/include few datanodes while balancing

2014-07-21 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068447#comment-14068447 ] Yu Li commented on HDFS-6441: - Hi [~szetszwo] [~aagarwal] and [~benoyantony], Sorry for the

[jira] [Commented] (HDFS-6441) Add ability to Ignore few datanodes while balancing

2014-05-21 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14005524#comment-14005524 ] Yu Li commented on HDFS-6441: - This is kind of duplicated with HDFS-6010 Add ability to

[jira] [Commented] (HDFS-6010) Make balancer able to balance data among specified servers

2014-03-29 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13951867#comment-13951867 ] Yu Li commented on HDFS-6010: - Thanks for the review and comments Tsz. {quote} I think

[jira] [Commented] (HDFS-6010) Make balancer able to balance data among specified servers

2014-03-28 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13950587#comment-13950587 ] Yu Li commented on HDFS-6010: - Ok, thanks in advance [~szetszwo] Make balancer able to

[jira] [Commented] (HDFS-6010) Make balancer able to balance data among specified servers

2014-03-26 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13947857#comment-13947857 ] Yu Li commented on HDFS-6010: - Hi [~szetszwo], Since hadoop QA test has passed, could you

[jira] [Commented] (HDFS-6010) Make balancer able to balance data among specified servers

2014-03-24 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13945142#comment-13945142 ] Yu Li commented on HDFS-6010: - The UT failure is caused by a bug of TestBalancer, here is

[jira] [Updated] (HDFS-6010) Make balancer able to balance data among specified servers

2014-03-24 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HDFS-6010: Attachment: HDFS-6010-trunk_V2.patch Attach the new patch with fix of the UT failure as mentioned above, and

[jira] [Updated] (HDFS-6010) Make balancer able to balance data among specified servers

2014-03-24 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HDFS-6010: Status: Patch Available (was: Open) Make balancer able to balance data among specified servers

[jira] [Updated] (HDFS-6010) Make balancer able to balance data among specified servers

2014-03-24 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HDFS-6010: Status: Open (was: Patch Available) Make balancer able to balance data among specified servers

[jira] [Commented] (HDFS-6010) Make balancer able to balance data among specified servers

2014-03-18 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13939277#comment-13939277 ] Yu Li commented on HDFS-6010: - Hi [~szetszwo], [~sanjay.radia] and [~devaraj], Is it ok for me

[jira] [Updated] (HDFS-6010) Make balancer able to balance data among specified servers

2014-03-18 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HDFS-6010: Labels: balancer (was: ) Status: Patch Available (was: In Progress) Submitting patch for hadoop QA to test.

[jira] [Commented] (HDFS-6009) Tools based on favored node feature for isolation

2014-03-14 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13934814#comment-13934814 ] Yu Li commented on HDFS-6009: - {quote} In particular, what caused the failure in your case? Is

[jira] [Commented] (HDFS-6009) Tools based on favored node feature for isolation

2014-03-12 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931955#comment-13931955 ] Yu Li commented on HDFS-6009: - Hi [~thanhdo], Yes, the data are replicated, so there won't be

[jira] [Commented] (HDFS-6010) Make balancer able to balance data among specified servers

2014-03-12 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931974#comment-13931974 ] Yu Li commented on HDFS-6010: - Hi [~devaraj], it seems we are waiting for your comment here.

[jira] [Commented] (HDFS-6010) Make balancer able to balance data among specified servers

2014-03-12 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932891#comment-13932891 ] Yu Li commented on HDFS-6010: - {quote} You know how things work when there are deadlines to

[jira] [Commented] (HDFS-6010) Make balancer able to balance data among specified servers

2014-03-09 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925426#comment-13925426 ] Yu Li commented on HDFS-6010: - Hi [~szetszwo], How do you think about the use case? Does it

[jira] [Commented] (HDFS-6010) Make balancer able to balance data among specified servers

2014-03-06 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13922183#comment-13922183 ] Yu Li commented on HDFS-6010: - Thanks [~devaraj] for the reply and CC Nicholas! Hi

[jira] [Commented] (HDFS-6010) Make balancer able to balance data among specified servers

2014-03-03 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918074#comment-13918074 ] Yu Li commented on HDFS-6010: - Hi [~devaraj], Any comments? Or is it ok for me to submit the

[jira] [Updated] (HDFS-6010) Make balancer able to balance data among specified servers

2014-02-25 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HDFS-6010: Attachment: HDFS-6010-trunk.patch Attach the first patch against trunk, below is the test-patch result on my local

[jira] [Updated] (HDFS-6012) Tool for checking whether all blocks under a path are placed on specified nodes

2014-02-25 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HDFS-6012: Issue Type: Task (was: Improvement) Tool for checking whether all blocks under a path are placed on specified

[jira] [Updated] (HDFS-6009) Tools based on favored node feature for isolation

2014-02-25 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HDFS-6009: Issue Type: Task (was: New Feature) Tools based on favored node feature for isolation

[jira] [Commented] (HDFS-6010) Make balancer able to balance data among specified servers

2014-02-25 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13912420#comment-13912420 ] Yu Li commented on HDFS-6010: - Hi [~devaraj], Sorry for bothering but I noticed you

[jira] [Created] (HDFS-6009) Tools based on favored node feature for isolation

2014-02-24 Thread Yu Li (JIRA)
Yu Li created HDFS-6009: --- Summary: Tools based on favored node feature for isolation Key: HDFS-6009 URL: https://issues.apache.org/jira/browse/HDFS-6009 Project: Hadoop HDFS Issue Type: New Feature

[jira] [Created] (HDFS-6010) Make balancer able to balance data among specified servers

2014-02-24 Thread Yu Li (JIRA)
Yu Li created HDFS-6010: --- Summary: Make balancer able to balance data among specified servers Key: HDFS-6010 URL: https://issues.apache.org/jira/browse/HDFS-6010 Project: Hadoop HDFS Issue Type:

[jira] [Assigned] (HDFS-6010) Make balancer able to balance data among specified servers

2014-02-24 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li reassigned HDFS-6010: --- Assignee: Yu Li Make balancer able to balance data among specified servers

[jira] [Created] (HDFS-6011) Make it able to specify balancer bandwidth for specified nodes

2014-02-24 Thread Yu Li (JIRA)
Yu Li created HDFS-6011: --- Summary: Make it able to specify balancer bandwidth for specified nodes Key: HDFS-6011 URL: https://issues.apache.org/jira/browse/HDFS-6011 Project: Hadoop HDFS Issue Type:

[jira] [Created] (HDFS-6012) Tool for checking whether all blocks under a path are placed on specified nodes

2014-02-24 Thread Yu Li (JIRA)
Yu Li created HDFS-6012: --- Summary: Tool for checking whether all blocks under a path are placed on specified nodes Key: HDFS-6012 URL: https://issues.apache.org/jira/browse/HDFS-6012 Project: Hadoop HDFS

[jira] [Work started] (HDFS-6010) Make balancer able to balance data among specified servers

2014-02-24 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-6010 started by Yu Li. Make balancer able to balance data among specified servers --

[jira] [Commented] (HDFS-2994) If lease soft limit is recovered successfully the append can fail

2014-01-07 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13864016#comment-13864016 ] Yu Li commented on HDFS-2994: - Happen to find this JIRA already integrated into 2.1.1-beta

[jira] [Created] (HDFS-5706) Should reset Balancer.Parameters.DEFALUT.policy in TestBalancer

2013-12-29 Thread Yu Li (JIRA)
Yu Li created HDFS-5706: --- Summary: Should reset Balancer.Parameters.DEFALUT.policy in TestBalancer Key: HDFS-5706 URL: https://issues.apache.org/jira/browse/HDFS-5706 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-5706) Should reset Balancer.Parameters.DEFALUT.policy in TestBalancer

2013-12-29 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HDFS-5706: Description: Now in TestBalancer.java, more than one test case will invoke the private method runBalancer, in

[jira] [Updated] (HDFS-5706) Should reset Balancer.Parameters.DEFALUT.policy in TestBalancer

2013-12-29 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HDFS-5706: Description: Now in TestBalancer.java, more than one test case will invoke the private method runBalancer, in

[jira] [Updated] (HDFS-5706) Should reset Balancer.Parameters.DEFALUT.policy in TestBalancer

2013-12-29 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HDFS-5706: Description: Now in TestBalancer.java, more than one test case will invoke the private method runBalancer, in

[jira] [Updated] (HDFS-5706) Should reset Balancer.Parameters.DEFALUT.policy in TestBalancer

2013-12-29 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HDFS-5706: Affects Version/s: 2.2.0 Should reset Balancer.Parameters.DEFALUT.policy in TestBalancer

[jira] [Assigned] (HDFS-5706) Should reset Balancer.Parameters.DEFALUT.policy in TestBalancer

2013-12-29 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li reassigned HDFS-5706: --- Assignee: Yu Li Should reset Balancer.Parameters.DEFALUT.policy in TestBalancer

[jira] [Resolved] (HDFS-5706) Should reset Balancer.Parameters.DEFALUT.policy in TestBalancer

2013-12-29 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li resolved HDFS-5706. - Resolution: Duplicate After more careful investigation, the issue is already fixed along with the patch of

[jira] [Updated] (HDFS-5022) Add explicit error message in log when datanode went out of service because of free disk space hit dfs.datanode.du.reserved

2013-07-24 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HDFS-5022: Summary: Add explicit error message in log when datanode went out of service because of free disk space hit

[jira] [Updated] (HDFS-5022) Add explicit error message in log when datanode went out of service because of low disk space

2013-07-24 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HDFS-5022: Description: Currently, if dfs.datanode.du.reserved is set and a datanode run out of configured disk space, it

[jira] [Commented] (HDFS-5022) Add explicit error message in log when datanode went out of service because of free disk space hit dfs.datanode.du.reserved

2013-07-24 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718229#comment-13718229 ] Yu Li commented on HDFS-5022: - Hi Jim, It's my fault w/o mentioning the condition. According

[jira] [Created] (HDFS-5022) Add explicit error message in log when datanode went out of service because of low disk space

2013-07-23 Thread Yu Li (JIRA)
Yu Li created HDFS-5022: --- Summary: Add explicit error message in log when datanode went out of service because of low disk space Key: HDFS-5022 URL: https://issues.apache.org/jira/browse/HDFS-5022 Project:

[jira] [Commented] (HDFS-4720) Misleading warning message in WebhdfsFileSystem when trying to check whether path exist using webhdfs url

2013-04-22 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13637833#comment-13637833 ] Yu Li commented on HDFS-4720: - Hello [~jerryhe], The message and stack I observed(as shown

[jira] [Created] (HDFS-4720) Misleading warning message in WebhdfsFileSystem when trying to check whether path exist using webhdfs url

2013-04-21 Thread Yu Li (JIRA)
Yu Li created HDFS-4720: --- Summary: Misleading warning message in WebhdfsFileSystem when trying to check whether path exist using webhdfs url Key: HDFS-4720 URL: https://issues.apache.org/jira/browse/HDFS-4720

[jira] [Commented] (HDFS-4720) Misleading warning message in WebhdfsFileSystem when trying to check whether path exist using webhdfs url

2013-04-21 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13637525#comment-13637525 ] Yu Li commented on HDFS-4720: - Checking the source in trunk, the RemoteException didn't include

[jira] [Updated] (HDFS-4720) Misleading warning message in WebhdfsFileSystem when trying to check whether path exist using webhdfs url

2013-04-21 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HDFS-4720: Attachment: HDFS-4720-trunk.patch Misleading warning message in WebhdfsFileSystem when trying to check whether

[jira] [Commented] (HDFS-4720) Misleading warning message in WebhdfsFileSystem when trying to check whether path exist using webhdfs url

2013-04-21 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13637526#comment-13637526 ] Yu Li commented on HDFS-4720: - Here is the result of test-patch in sun jdk 1.6u21:

[jira] [Updated] (HDFS-4262) Backport HTTPFS to Branch 1

2012-12-05 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HDFS-4262: Attachment: HDFS-4262-github.patch 03-resolve-proxyuser-related-issue.patch Backport HTTPFS to

[jira] [Commented] (HDFS-4262) Backport HTTPFS to Branch 1

2012-12-05 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13510680#comment-13510680 ] Yu Li commented on HDFS-4262: - Thanks Alejandro. After applying the two patches you supplied,

[jira] [Work started] (HDFS-4262) Backport HTTPFS to Branch 1

2012-12-05 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-4262 started by Yu Li. Backport HTTPFS to Branch 1 --- Key: HDFS-4262 URL:

[jira] [Commented] (HDFS-4262) Backport HTTPFS to Branch 1

2012-12-05 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13510709#comment-13510709 ] Yu Li commented on HDFS-4262: - Next step I will try to put the httpfs source codes into

[jira] [Commented] (HDFS-4262) Backport HTTPFS to Branch 1

2012-12-04 Thread Yu Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13509804#comment-13509804 ] Yu Li commented on HDFS-4262: - Yes, Alejandro, please upload your delta patch, I believe it's a