[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-04-10 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13965839#comment-13965839 ] Chris Nauroth commented on HDFS-5776: - FYI, I've discovered that the {{DFSClient}} can

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-25 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13911633#comment-13911633 ] Hudson commented on HDFS-5776: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1709 (See [htt

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-25 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13911577#comment-13911577 ] Hudson commented on HDFS-5776: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1684 (See [https://

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-25 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13911477#comment-13911477 ] Hudson commented on HDFS-5776: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #492 (See [https://b

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-24 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13910935#comment-13910935 ] Hudson commented on HDFS-5776: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5216 (See [https:

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13907863#comment-13907863 ] Liang Xie commented on HDFS-5776: - bq. little discernible overall difference in spite of my

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-20 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13907451#comment-13907451 ] stack commented on HDFS-5776: - Thanks for the numbers [~xieliang007] I ran some loadings yeste

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-19 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13906611#comment-13906611 ] Liang Xie commented on HDFS-5776: - Huge thanks, [~saint@gmail.com] ! Here is the YCSB

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-19 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13906242#comment-13906242 ] Hadoop QA commented on HDFS-5776: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-14 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13902288#comment-13902288 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-14 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13901843#comment-13901843 ] Jing Zhao commented on HDFS-5776: - Thanks for updating the patch [~stack]. The latest patch

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-13 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900133#comment-13900133 ] Hadoop QA commented on HDFS-5776: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-12 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13899961#comment-13899961 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-09 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13896288#comment-13896288 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-07 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895461#comment-13895461 ] Liang Xie commented on HDFS-5776: - We need to have a NPE check here since the chosenNode be

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-07 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895441#comment-13895441 ] Liang Xie commented on HDFS-5776: - Thank you, Michael! > Support 'hedged' reads in DFSClie

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-07 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895440#comment-13895440 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-07 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895423#comment-13895423 ] Liang Xie commented on HDFS-5776: - I am from holiday now, will try to catch up the above co

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13891912#comment-13891912 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-30 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13887329#comment-13887329 ] Liang Xie commented on HDFS-5776: - [~saint@gmail.com], yeh, i need your help, thanks! :

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-30 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13887039#comment-13887039 ] stack commented on HDFS-5776: - [~jingzhao] Thanks for taking the time to look and the great fee

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-29 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13886172#comment-13886172 ] Jing Zhao commented on HDFS-5776: - Thanks for the feedback [~stack]. My first question is

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-29 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13886142#comment-13886142 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-29 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13886107#comment-13886107 ] stack commented on HDFS-5776: - [~jingzhao] Thanks for the new input. Please help me better und

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-29 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13886072#comment-13886072 ] Jing Zhao commented on HDFS-5776: - Thanks for updating the patch, [~xieliang007] and [~stac

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-29 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13885813#comment-13885813 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-29 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13885512#comment-13885512 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13885092#comment-13885092 ] stack commented on HDFS-5776: - bq. what do you think ? That looks good to me [~xieliang007]

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13885091#comment-13885091 ] Liang Xie commented on HDFS-5776: - bq. I think a better way is to add this check in chooseD

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13885083#comment-13885083 ] Liang Xie commented on HDFS-5776: - bq. we do not check threadpool in enableHedgedReads. Thi

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13884710#comment-13884710 ] Arpit Agarwal commented on HDFS-5776: - I've stated my concerns but if there is broad co

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13884694#comment-13884694 ] stack commented on HDFS-5776: - Thanks lads. We are almost there. [~xieliang007] It is better

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13884573#comment-13884573 ] Suresh Srinivas commented on HDFS-5776: --- bq. We do not check other configuration sett

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13884554#comment-13884554 ] Colin Patrick McCabe commented on HDFS-5776: [~arpitagarwal] : if I understand

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13884361#comment-13884361 ] Suresh Srinivas commented on HDFS-5776: --- bq. Could we create another JIRA to track th

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13884351#comment-13884351 ] Arpit Agarwal commented on HDFS-5776: - [~stack] I am basically +1 on the v9 patch at th

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13883944#comment-13883944 ] Liang Xie commented on HDFS-5776: - Could we create another JIRA to track those disagreement

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13883940#comment-13883940 ] Jing Zhao commented on HDFS-5776: - Another thing for enoughNodesForHedgedRead. The current

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13883928#comment-13883928 ] Jing Zhao commented on HDFS-5776: - bq. it's more flexible if we provide instance level dis

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-27 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13883823#comment-13883823 ] Liang Xie commented on HDFS-5776: - bq. Isn't the call to actualGetFromOneDataNode wrapped i

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13883746#comment-13883746 ] Hadoop QA commented on HDFS-5776: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-27 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13883717#comment-13883717 ] stack commented on HDFS-5776: - [~arpitagarwal] Would v10 be palatable? You say OK to v9 above

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-27 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13883685#comment-13883685 ] Arpit Agarwal commented on HDFS-5776: - {quote} Yes, that would be perfect sometimes, bu

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13883666#comment-13883666 ] Jing Zhao commented on HDFS-5776: - Thanks for the work [~xieliang007]! I will review your l

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13883079#comment-13883079 ] Colin Patrick McCabe commented on HDFS-5776: bq. Fine, let me introduce a hard

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882655#comment-13882655 ] Hadoop QA commented on HDFS-5776: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-26 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882606#comment-13882606 ] Liang Xie commented on HDFS-5776: - [~saint@gmail.com] bq. If we are too often running

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-25 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882037#comment-13882037 ] stack commented on HDFS-5776: - bq. I might be misunderstanding, but it seems like this should b

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-25 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881827#comment-13881827 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-24 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881661#comment-13881661 ] Colin Patrick McCabe commented on HDFS-5776: I might be misunderstanding, but i

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-24 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881341#comment-13881341 ] Arpit Agarwal commented on HDFS-5776: - I reviewed the v8 patch. The implementation of {

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-24 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881305#comment-13881305 ] Arpit Agarwal commented on HDFS-5776: - {quote} I don't see why we wouldn't expose this

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-24 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881281#comment-13881281 ] stack commented on HDFS-5776: - So, to enable, we set DFS_DFSCLIENT_HEDGED_READ_THREADPOOL_SIZE

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-23 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13880783#comment-13880783 ] Liang Xie commented on HDFS-5776: - v8 add the enoughNodesForHedgedRead() function to sanity

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-23 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13880762#comment-13880762 ] Liang Xie commented on HDFS-5776: - bq. Can this scenario be possible? In hedgedFetchBlockBy

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-23 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13880757#comment-13880757 ] stack commented on HDFS-5776: - [~xieliang007] what you think of the new comments above by the l

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-23 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13880746#comment-13880746 ] stack commented on HDFS-5776: - bq. An alternative way to do this is to have an "Allow-Hedged-Re

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-23 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13880284#comment-13880284 ] Colin Patrick McCabe commented on HDFS-5776: By the way, my previous comment wa

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-23 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13880280#comment-13880280 ] Colin Patrick McCabe commented on HDFS-5776: bq. One note on 1.. Is static ever

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-23 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13880279#comment-13880279 ] Jing Zhao commented on HDFS-5776: - # In DFSClient, I agree with Arpit that we should remove

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-23 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13879680#comment-13879680 ] Liang Xie commented on HDFS-5776: - [~arpitagarwal], thanks for your nice review! bq. The c

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-22 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13879476#comment-13879476 ] stack commented on HDFS-5776: - [~arpitagarwal] Great review (from a bystander). One note on 1.

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-22 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13879326#comment-13879326 ] Arpit Agarwal commented on HDFS-5776: - Hi Liang, thanks for this contribution to HDFS!

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-22 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13879324#comment-13879324 ] Colin Patrick McCabe commented on HDFS-5776: I like the idea of calling this "h

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-22 Thread Enis Soztutar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878997#comment-13878997 ] Enis Soztutar commented on HDFS-5776: - bq. About naming, personally i'd like to keep "h

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-22 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878485#comment-13878485 ] Hadoop QA commented on HDFS-5776: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-22 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878459#comment-13878459 ] Liang Xie commented on HDFS-5776: - Added a bit codes into YCSB to get 99.9thPercentile late

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-21 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878382#comment-13878382 ] Liang Xie commented on HDFS-5776: - About naming, personally i'd like to keep "hedged" uncha

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-21 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878184#comment-13878184 ] Liang Xie commented on HDFS-5776: - [~enis], thanks for your nice comments. bq. sleeping is

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-21 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878028#comment-13878028 ] stack commented on HDFS-5776: - (Trying to help out w/ naming...) [~enis] You don't like 'hedge

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-21 Thread Enis Soztutar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877996#comment-13877996 ] Enis Soztutar commented on HDFS-5776: - Nice work Liang. You have beat us to implement t

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877344#comment-13877344 ] Hadoop QA commented on HDFS-5776: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877268#comment-13877268 ] Liang Xie commented on HDFS-5776: - bq. I suppose moving this into finally would be messier

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877263#comment-13877263 ] Liang Xie commented on HDFS-5776: - bq. When we get to the end of the pipeline here; i.e. al

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877255#comment-13877255 ] Liang Xie commented on HDFS-5776: - bq. For sure this will always trigger though the number

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877231#comment-13877231 ] Liang Xie commented on HDFS-5776: - {quote} getHedgedReadsThreadPool Should be kept internal

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877177#comment-13877177 ] Liang Xie commented on HDFS-5776: - bq. Shut down this executor in the finally? Don't want i

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877173#comment-13877173 ] Liang Xie commented on HDFS-5776: - {quote} Nit: Should the '60' below here: + Thread.sleep(

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877161#comment-13877161 ] Liang Xie commented on HDFS-5776: - bq. Nit: This seems extraneous in the new Callable: + re

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877154#comment-13877154 ] Liang Xie commented on HDFS-5776: - bq. Let's keep TestPread as a test of just pread, and ha

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877146#comment-13877146 ] Liang Xie commented on HDFS-5776: - bq. I guess we probably don't need to special-case local

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877145#comment-13877145 ] Liang Xie commented on HDFS-5776: - [~saint@gmail.com], [~cmccabe], hedged reads doesn'

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13876842#comment-13876842 ] Colin Patrick McCabe commented on HDFS-5776: bq. Having hedged read on could am

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13876802#comment-13876802 ] stack commented on HDFS-5776: - bq. I think that when the data being read is local, you will not

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13876777#comment-13876777 ] Colin Patrick McCabe commented on HDFS-5776: Let's keep TestPread as a test of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13876733#comment-13876733 ] stack commented on HDFS-5776: - Nice numbers [~xieliang007] When you get a chance, there were a

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13876388#comment-13876388 ] Hadoop QA commented on HDFS-5776: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-17 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13875453#comment-13875453 ] stack commented on HDFS-5776: - This looks like copy paste issue: + // assert that there w

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-17 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13875161#comment-13875161 ] stack commented on HDFS-5776: - [~xieliang007] Nit: The 'return null' necessary in below given

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-17 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874818#comment-13874818 ] Liang Xie commented on HDFS-5776: - all are green now, nice:) > Support 'hedged' reads in D

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874620#comment-13874620 ] Hadoop QA commented on HDFS-5776: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-16 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873239#comment-13873239 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-16 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873136#comment-13873136 ] Liang Xie commented on HDFS-5776: - "mvn clean test -Dtest=TestPread" passed locally:) > Su

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-15 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13872402#comment-13872402 ] stack commented on HDFS-5776: - I had a go at the subject for you [~xieliang007] I am a native