[jira] [Created] (HDFS-10648) Expose Balancer metrics through Metrics2
Mark Wagner created HDFS-10648: -- Summary: Expose Balancer metrics through Metrics2 Key: HDFS-10648 URL: https://issues.apache.org/jira/browse/HDFS-10648 Project: Hadoop HDFS Issue Type: New Feature Reporter: Mark Wagner The Balancer currently prints progress information to the console. For deployments that run the balancer frequently, it would be helpful to collect those metrics for publishing to the available sinks. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-4777) File creation with overwrite flag set to true results in logSync holding namesystem lock
[ https://issues.apache.org/jira/browse/HDFS-4777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated HDFS-4777: -- Resolution: Duplicate Status: Resolved (was: Patch Available) This was resolved by HDFS-6886 and duplicates HDFS-6871. File creation with overwrite flag set to true results in logSync holding namesystem lock Key: HDFS-4777 URL: https://issues.apache.org/jira/browse/HDFS-4777 Project: Hadoop HDFS Issue Type: Bug Components: namenode Affects Versions: 0.23.0, 2.0.0-alpha Reporter: Suresh Srinivas Assignee: Suresh Srinivas Labels: BB2015-05-TBR Attachments: HDFS-4777.patch FSNamesystem#startFileInternal calls delete. Delete method releases the write lock, making parts of startFileInternal code unintentionally executed without write lock being held. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-4549) WebHDFS hits a Jetty performance issue
[ https://issues.apache.org/jira/browse/HDFS-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated HDFS-4549: -- Attachment: HDFS-4549.2.patch I've backported HDFS-3577, HDFS-3318, and HDFS-3788 as best as possible. However, I'm seeing performance problems above 2GB. It looks like when the MessageBodyWriter reports a length greater than 2GB, Jersey is reverting to chunked transfer, causing the previously seen performance problems with Jetty. I tested this on a 2.0.3 deployment and got the same results (chunked transfer above a certain size). [~szetszwo], can you expand on the motivation for using MessageBodyWriter and OpenEntity instead of manually setting the header values? Would it be problematic to make all branches use the method of my first patch? It seems that Jersey is taking the MessageBodyWriter information only as a suggestion. WebHDFS hits a Jetty performance issue -- Key: HDFS-4549 URL: https://issues.apache.org/jira/browse/HDFS-4549 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 1.1.2 Reporter: Mark Wagner Assignee: Mark Wagner Attachments: example4549.txt, HDFS-4549.1.patch, HDFS-4549.2.patch WebHDFS on branch-1 is hitting a Jetty issue for me when it does chunked transfers. This is the same Jetty issue as MAPREDUCE-4399. I have not observed this on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4549) WebHDFS hits a Jetty performance issue
[ https://issues.apache.org/jira/browse/HDFS-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated HDFS-4549: -- Attachment: example4549.txt WebHDFS hits a Jetty performance issue -- Key: HDFS-4549 URL: https://issues.apache.org/jira/browse/HDFS-4549 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 1.1.2 Reporter: Mark Wagner Assignee: Mark Wagner Attachments: example4549.txt, HDFS-4549.1.patch, HDFS-4549.2.patch WebHDFS on branch-1 is hitting a Jetty issue for me when it does chunked transfers. This is the same Jetty issue as MAPREDUCE-4399. I have not observed this on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4549) WebHDFS hits a Jetty performance issue
[
https://issues.apache.org/jira/browse/HDFS-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13635687#comment-13635687
]
Mark Wagner commented on HDFS-4549:
---
bq. I think we should leave Jersey to decide what to use, chunk encoding or not.
I agree that that would usually be best, however this is causing significant
performance problems, so I think it needs to be handled somehow. I just took a
look at the memory usage when pulling a 1GB file (which does use
content-length) and didn't see any memory usage increase. I'll look into this
further though.
{quote}
Is the Jetty version use by Hadoop 1 that has issues with chunk encoding?
Also, if we don't see the problem in trunk/branch-2, why change it there? not
broken, don't fix it.
{quote}
The issue has been observed in 6.1.26, which is used by branch-1, branch-2, and
trunk. This problem was first seen (chunked encoding being used, leading to
performance loss) in branch-1 (1.0.4 to be precise) for files of any size. I
didn't see it on branch-2/trunk because I didn't test larger files. When I was
backporting though, I noticed that it was still slow above 2GB, and it turns
out the same is true for branch-2/trunk (I've attached logs for a demonstration
of this issue using 2.0.3-alpha).
WebHDFS hits a Jetty performance issue
--
Key: HDFS-4549
URL: https://issues.apache.org/jira/browse/HDFS-4549
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 1.1.2
Reporter: Mark Wagner
Assignee: Mark Wagner
Attachments: example4549.txt, HDFS-4549.1.patch, HDFS-4549.2.patch
WebHDFS on branch-1 is hitting a Jetty issue for me when it does chunked
transfers. This is the same Jetty issue as MAPREDUCE-4399. I have not
observed this on trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4549) WebHDFS hits a Jetty performance issue
[ https://issues.apache.org/jira/browse/HDFS-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13635695#comment-13635695 ] Mark Wagner commented on HDFS-4549: --- [~szetszwo], that makes sense. Yes, I see performance issues above 2GB on 2.0.3. WebHDFS hits a Jetty performance issue -- Key: HDFS-4549 URL: https://issues.apache.org/jira/browse/HDFS-4549 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 1.1.2 Reporter: Mark Wagner Assignee: Mark Wagner Attachments: example4549.txt, HDFS-4549.1.patch, HDFS-4549.2.patch WebHDFS on branch-1 is hitting a Jetty issue for me when it does chunked transfers. This is the same Jetty issue as MAPREDUCE-4399. I have not observed this on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4715) Backport HDFS-3577 and other related WebHDFS issue to branch-1
[ https://issues.apache.org/jira/browse/HDFS-4715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated HDFS-4715: -- Attachment: HDFS-4751.1.patch Addressed your points, and got things as close to trunk as possible. Backport HDFS-3577 and other related WebHDFS issue to branch-1 -- Key: HDFS-4715 URL: https://issues.apache.org/jira/browse/HDFS-4715 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Reporter: Tsz Wo (Nicholas), SZE Assignee: Mark Wagner Attachments: HDFS-4751.1.patch The related JIRAs are HDFS-3577, HDFS-3318, and HDFS-3788. Backporting them can fix some WebHDFS performance issues in branch-1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4715) Backport HDFS-3577 and other related WebHDFS issue to branch-1
[ https://issues.apache.org/jira/browse/HDFS-4715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated HDFS-4715: -- Status: Patch Available (was: Open) Backport HDFS-3577 and other related WebHDFS issue to branch-1 -- Key: HDFS-4715 URL: https://issues.apache.org/jira/browse/HDFS-4715 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Reporter: Tsz Wo (Nicholas), SZE Assignee: Mark Wagner Attachments: HDFS-4751.1.patch The related JIRAs are HDFS-3577, HDFS-3318, and HDFS-3788. Backporting them can fix some WebHDFS performance issues in branch-1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4549) WebHDFS hits a Jetty performance issue
[ https://issues.apache.org/jira/browse/HDFS-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13633410#comment-13633410 ] Mark Wagner commented on HDFS-4549: --- That seems like a reasonable idea. I'll look into that. WebHDFS hits a Jetty performance issue -- Key: HDFS-4549 URL: https://issues.apache.org/jira/browse/HDFS-4549 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 1.1.2 Reporter: Mark Wagner Assignee: Mark Wagner Attachments: HDFS-4549.1.patch WebHDFS on branch-1 is hitting a Jetty issue for me when it does chunked transfers. This is the same Jetty issue as MAPREDUCE-4399. I have not observed this on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4549) WebHDFS hits a Jetty performance issue
[ https://issues.apache.org/jira/browse/HDFS-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated HDFS-4549: -- Status: Open (was: Patch Available) WebHDFS hits a Jetty performance issue -- Key: HDFS-4549 URL: https://issues.apache.org/jira/browse/HDFS-4549 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 1.1.2 Reporter: Mark Wagner Assignee: Mark Wagner Attachments: HDFS-4549.1.patch WebHDFS on branch-1 is hitting a Jetty issue for me when it does chunked transfers. This is the same Jetty issue as MAPREDUCE-4399. I have not observed this on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4549) WebHDFS hits a Jetty performance issue
[ https://issues.apache.org/jira/browse/HDFS-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605939#comment-13605939 ] Mark Wagner commented on HDFS-4549: --- Could I get somebody to look at this? WebHDFS hits a Jetty performance issue -- Key: HDFS-4549 URL: https://issues.apache.org/jira/browse/HDFS-4549 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 1.1.2 Reporter: Mark Wagner Assignee: Mark Wagner Attachments: HDFS-4549.1.patch WebHDFS on branch-1 is hitting a Jetty issue for me when it does chunked transfers. This is the same Jetty issue as MAPREDUCE-4399. I have not observed this on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4551) Change WebHDFS buffersize behavior to improve default performance
[
https://issues.apache.org/jira/browse/HDFS-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13595334#comment-13595334
]
Mark Wagner commented on HDFS-4551:
---
Hi Nicholas,
I have observed significant performance increase (7-8x) at copying a 1GB file
when the server has set io.file.buffer.size at 64kB (and the buffer size isn't
specified in the request). Of course you can manually set the buffer size to
4096 bytes, but then that affects the buffer size used to open the file also.
I think we may have gotten crossed up about what is changing. This patch only
changes the buffer size to copy from the FileInputStream onto network. My
understanding is that both WebHDFS and hftp on trunk eventually end up at:
{code:title=IOUtils.java|borderStyle=solid}
126 public static void copyBytes(InputStream in, OutputStream out, long count,
127 boolean close) throws IOException {
128byte buf[] = new byte[4096];
129long bytesRemaining = count;
{code}
which is what this patch is trying to match. Is that your understanding also?
There's an argument to be made that this should be configurable, but I figured
it best to copy what trunk does.
Change WebHDFS buffersize behavior to improve default performance
-
Key: HDFS-4551
URL: https://issues.apache.org/jira/browse/HDFS-4551
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 1.1.2
Reporter: Mark Wagner
Assignee: Mark Wagner
Attachments: HDFS-4551.1.patch
Currently on 1.X branch, the buffer size used to copy bytes to network
defaults to io.file.buffer.size. This causes performance problems if that
buffersize is large.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4551) Change WebHDFS buffersize behavior to improve default performance
[ https://issues.apache.org/jira/browse/HDFS-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13595481#comment-13595481 ] Mark Wagner commented on HDFS-4551: --- I'm not sure what's best, but io.file.buffer.size=64KB is the setting that I first noticed this on. Although it defaults to 4KB, everything I've seen recommended values from 16KB to 128KB, so I don't think 64KB is an unusual choice. Change WebHDFS buffersize behavior to improve default performance - Key: HDFS-4551 URL: https://issues.apache.org/jira/browse/HDFS-4551 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 1.1.2 Reporter: Mark Wagner Assignee: Mark Wagner Attachments: HDFS-4551.1.patch Currently on 1.X branch, the buffer size used to copy bytes to network defaults to io.file.buffer.size. This causes performance problems if that buffersize is large. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4551) Change WebHDFS buffersize behavior to improve default performance
[ https://issues.apache.org/jira/browse/HDFS-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13595554#comment-13595554 ] Mark Wagner commented on HDFS-4551: --- I haven't done any real testing for different values of io.file.buffer.size, but it seemed that 4KB and 64KB had similar performance (at least when pulling over WebHDFS). They may be different under load, though. I can look at this more, but I think determining a good value for that parameter is outside the scope of this JIRA. Would you agree? Change WebHDFS buffersize behavior to improve default performance - Key: HDFS-4551 URL: https://issues.apache.org/jira/browse/HDFS-4551 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 1.1.2 Reporter: Mark Wagner Assignee: Mark Wagner Attachments: HDFS-4551.1.patch Currently on 1.X branch, the buffer size used to copy bytes to network defaults to io.file.buffer.size. This causes performance problems if that buffersize is large. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-4549) WebHDFS hits a Jetty performance issue
Mark Wagner created HDFS-4549: - Summary: WebHDFS hits a Jetty performance issue Key: HDFS-4549 URL: https://issues.apache.org/jira/browse/HDFS-4549 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 1.1.2 Reporter: Mark Wagner WebHDFS on branch-1 is hitting a Jetty issue for me when it does chunked transfers. This is the same Jetty issue as MAPREDUCE-4399. I have not observed this on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4549) WebHDFS hits a Jetty performance issue
[ https://issues.apache.org/jira/browse/HDFS-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated HDFS-4549: -- Attachment: HDFS-4549.1.patch I've attached my patch. The 'content-length' field has been added to the HTTP response in order to workaround the Jetty issue. This patch includes a backported HDFS-3318. WebHDFS hits a Jetty performance issue -- Key: HDFS-4549 URL: https://issues.apache.org/jira/browse/HDFS-4549 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 1.1.2 Reporter: Mark Wagner Attachments: HDFS-4549.1.patch WebHDFS on branch-1 is hitting a Jetty issue for me when it does chunked transfers. This is the same Jetty issue as MAPREDUCE-4399. I have not observed this on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-4551) Change WebHDFS buffersize behavior to improve default performance
Mark Wagner created HDFS-4551: - Summary: Change WebHDFS buffersize behavior to improve default performance Key: HDFS-4551 URL: https://issues.apache.org/jira/browse/HDFS-4551 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 1.1.2 Reporter: Mark Wagner Currently on 1.X branch, the buffer size used to copy bytes to network defaults to io.file.buffer.size. This causes performance problems if that buffersize is large. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-4551) Change WebHDFS buffersize behavior to improve default performance
[ https://issues.apache.org/jira/browse/HDFS-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner reassigned HDFS-4551: - Assignee: Mark Wagner Change WebHDFS buffersize behavior to improve default performance - Key: HDFS-4551 URL: https://issues.apache.org/jira/browse/HDFS-4551 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 1.1.2 Reporter: Mark Wagner Assignee: Mark Wagner Currently on 1.X branch, the buffer size used to copy bytes to network defaults to io.file.buffer.size. This causes performance problems if that buffersize is large. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4551) Change WebHDFS buffersize behavior to improve default performance
[ https://issues.apache.org/jira/browse/HDFS-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated HDFS-4551: -- Attachment: HDFS-4551.1.patch I've attached a patch which hardcodes the copy buffer size to 4096. This matches the behavior of hftp and WebHDFS on trunk. Change WebHDFS buffersize behavior to improve default performance - Key: HDFS-4551 URL: https://issues.apache.org/jira/browse/HDFS-4551 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 1.1.2 Reporter: Mark Wagner Assignee: Mark Wagner Attachments: HDFS-4551.1.patch Currently on 1.X branch, the buffer size used to copy bytes to network defaults to io.file.buffer.size. This causes performance problems if that buffersize is large. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4551) Change WebHDFS buffersize behavior to improve default performance
[ https://issues.apache.org/jira/browse/HDFS-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated HDFS-4551: -- Status: Patch Available (was: Open) Change WebHDFS buffersize behavior to improve default performance - Key: HDFS-4551 URL: https://issues.apache.org/jira/browse/HDFS-4551 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 1.1.2 Reporter: Mark Wagner Assignee: Mark Wagner Attachments: HDFS-4551.1.patch Currently on 1.X branch, the buffer size used to copy bytes to network defaults to io.file.buffer.size. This causes performance problems if that buffersize is large. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4549) WebHDFS hits a Jetty performance issue
[ https://issues.apache.org/jira/browse/HDFS-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated HDFS-4549: -- Status: Patch Available (was: Open) WebHDFS hits a Jetty performance issue -- Key: HDFS-4549 URL: https://issues.apache.org/jira/browse/HDFS-4549 Project: Hadoop HDFS Issue Type: Bug Components: webhdfs Affects Versions: 1.1.2 Reporter: Mark Wagner Assignee: Mark Wagner Attachments: HDFS-4549.1.patch WebHDFS on branch-1 is hitting a Jetty issue for me when it does chunked transfers. This is the same Jetty issue as MAPREDUCE-4399. I have not observed this on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
