[jira] [Commented] (HDFS-9080) update htrace version to 4.0
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903721#comment-14903721 ] Hadoop QA commented on HDFS-9080: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 25m 30s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 11 new or modified test files. | | {color:red}-1{color} | javac | 0m 25s | The patch appears to cause the build to fail. | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12761759/HDFS-9080.010.patch | | Optional Tests | javadoc javac unit findbugs checkstyle site | | git revision | trunk / cc2b473 | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12612/console | This message was automatically generated. > update htrace version to 4.0 > > > Key: HDFS-9080 > URL: https://issues.apache.org/jira/browse/HDFS-9080 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe > Attachments: HDFS-9080.001.patch, HDFS-9080.002.patch, > HDFS-9080.003.patch, HDFS-9080.004.patch, HDFS-9080.005.patch, > HDFS-9080.006.patch, HDFS-9080.007.patch, HDFS-9080.009.patch, > HDFS-9080.010.patch, tracing-fsshell-put.png > > > Update the HTrace library version Hadoop uses to htrace 4.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9080) update htrace version to 4.0
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903662#comment-14903662 ] Colin Patrick McCabe commented on HDFS-9080: Thanks for the reviews. bq. Suggest you say what \[the DFSClient lifecycle issues\] are on commit: I added a note about HADOOP-6356. bq. In the .md file, you don't close the scope, just the tracer. Do you want to add a line closing scope? So, this is actually an example of the Java 7 "try-with-resources" idiom. https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html I'm going to rework this though, because I'm afraid some people may misinterpret this code and think that they don't need to close trace scopes. bq. hdfs-default.xml: This should be fs.client.htrace.sampler.classes. Fixed, thanks bq. {{fs.client.htrace.sampler.classes}}... also should be moved to core-default.xml? FS_CLIENT_HTRACE_PREFIX is defined in CommonConfigurationKeys.java of hadoop-common Yes, let's move it to {{core-default.xml}}. bq. dfs.htrace.spanreceiver.classes is sometimes found in Tracing.md. It should be fixed as hadoop.htrace.span.receiver.classes. I added a reference to {{core-default.xml}} so that we don't have to change 2 files next time. > update htrace version to 4.0 > > > Key: HDFS-9080 > URL: https://issues.apache.org/jira/browse/HDFS-9080 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe > Attachments: HDFS-9080.001.patch, HDFS-9080.002.patch, > HDFS-9080.003.patch, HDFS-9080.004.patch, HDFS-9080.005.patch, > HDFS-9080.006.patch, HDFS-9080.007.patch, HDFS-9080.009.patch, > tracing-fsshell-put.png > > > Update the HTrace library version Hadoop uses to htrace 4.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9080) update htrace version to 4.0
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14902653#comment-14902653 ] Masatake Iwasaki commented on HDFS-9080: * hdfs-default.xml: This should be {{fs.client.htrace.sampler.classes}}. This also should be moved to core-default.xml? FS_CLIENT_HTRACE_PREFIX is defined in CommonConfigurationKeys.java of hadoop-common. {code} - dfs.htrace.spanreceiver.classes + dfs.client.htrace.sampler.classes {code} * TestHdfsConfigFields.java: We don't need this if the default configuration above is moved to core-default.xml. {code} -// Ignore SpanReceiveHost properties -xmlPropsToSkipCompare.add("dfs.htrace.spanreceiver.classes"); -xmlPropsToSkipCompare.add("dfs.client.htrace.spanreceiver.classes"); +// Ignore HTrace properties +xmlPropsToSkipCompare.add("fs.client.htrace"); +xmlPropsToSkipCompare.add("hadoop.htrace"); {code} * {{dfs.htrace.spanreceiver.classes}} is sometimes found in Tracing.md. It should be fixed as {{hadoop.htrace.span.receiver.classes}}. > update htrace version to 4.0 > > > Key: HDFS-9080 > URL: https://issues.apache.org/jira/browse/HDFS-9080 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe > Attachments: HDFS-9080.001.patch, HDFS-9080.002.patch, > HDFS-9080.003.patch, HDFS-9080.004.patch, HDFS-9080.005.patch, > HDFS-9080.006.patch, HDFS-9080.007.patch, HDFS-9080.009.patch, > tracing-fsshell-put.png > > > Update the HTrace library version Hadoop uses to htrace 4.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9080) update htrace version to 4.0
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14902637#comment-14902637 ] Masatake Iwasaki commented on HDFS-9080: Thanks for working on this, [~cmccabe]. I tested the 009 patch in pseudo distributed cluster and it worked fine. I attached the trace of {{hdfs dfs -Dfs.shell.htrace.span.receiver.classes=org.apache.htrace.impl.HTracedRESTReceiver -Dfs.shell.htrace.sampler.classes=AlwaysSampler -put /tmp/test.txt /tmp/}}. > update htrace version to 4.0 > > > Key: HDFS-9080 > URL: https://issues.apache.org/jira/browse/HDFS-9080 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe > Attachments: HDFS-9080.001.patch, HDFS-9080.002.patch, > HDFS-9080.003.patch, HDFS-9080.004.patch, HDFS-9080.005.patch, > HDFS-9080.006.patch, HDFS-9080.007.patch, HDFS-9080.009.patch, > tracing-fsshell-put.png > > > Update the HTrace library version Hadoop uses to htrace 4.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9080) update htrace version to 4.0
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14900088#comment-14900088 ] stack commented on HDFS-9080: - +1 You might add a picture of trace UI showing some traced HDFS ops that can only happen if up on htrace 4.0 to help with justification for this patch but all looks good to me. Suggest you say what these are on commit: {code} 9* It would be better if this weren't global, but there are some issues with 30 * the lifecycle of FileSystem and AbstractFileSystem objects that make that 31 * impossible. {code} In the .md file, you don't close the scope, just the tracer. Do you want to add a line closing scope? > update htrace version to 4.0 > > > Key: HDFS-9080 > URL: https://issues.apache.org/jira/browse/HDFS-9080 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe > Attachments: HDFS-9080.001.patch, HDFS-9080.002.patch, > HDFS-9080.003.patch, HDFS-9080.004.patch, HDFS-9080.005.patch, > HDFS-9080.006.patch, HDFS-9080.007.patch, HDFS-9080.009.patch > > > Update the HTrace library version Hadoop uses to htrace 4.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9080) update htrace version to 4.0
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14877431#comment-14877431 ] Hadoop QA commented on HDFS-9080: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 23m 32s | Findbugs (version 3.0.0) appears to be broken on trunk. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 11 new or modified test files. | | {color:red}-1{color} | javac | 0m 25s | The patch appears to cause the build to fail. | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12761318/HDFS-9080.009.patch | | Optional Tests | javadoc javac unit findbugs checkstyle site | | git revision | trunk / 3a9c707 | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12562/console | This message was automatically generated. > update htrace version to 4.0 > > > Key: HDFS-9080 > URL: https://issues.apache.org/jira/browse/HDFS-9080 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe > Attachments: HDFS-9080.001.patch, HDFS-9080.002.patch, > HDFS-9080.003.patch, HDFS-9080.004.patch, HDFS-9080.005.patch, > HDFS-9080.006.patch, HDFS-9080.007.patch, HDFS-9080.009.patch > > > Update the HTrace library version Hadoop uses to htrace 4.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9080) update htrace version to 4.0
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14876323#comment-14876323 ] Hadoop QA commented on HDFS-9080: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 22m 51s | Findbugs (version 3.0.0) appears to be broken on trunk. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 11 new or modified test files. | | {color:red}-1{color} | javac | 2m 10s | The patch appears to cause the build to fail. | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12761181/HDFS-9080.007.patch | | Optional Tests | javadoc javac unit findbugs checkstyle site | | git revision | trunk / f97dc03 | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12547/console | This message was automatically generated. > update htrace version to 4.0 > > > Key: HDFS-9080 > URL: https://issues.apache.org/jira/browse/HDFS-9080 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe > Attachments: HDFS-9080.001.patch, HDFS-9080.002.patch, > HDFS-9080.003.patch, HDFS-9080.004.patch, HDFS-9080.005.patch, > HDFS-9080.006.patch, HDFS-9080.007.patch > > > Update the HTrace library version Hadoop uses to htrace 4.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9080) update htrace version to 4.0
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14876247#comment-14876247 ] Colin Patrick McCabe commented on HDFS-9080: * fix a case where some tests were using a FileSystem without initializing the Tracer object * remove unused imports * give FsTracer a private constructor to make checkstyle happy * break a line with > 80 characters * Add workaround for HTRACE-253 * Add "final" to FileSystem#getTracer * FsShell#init: fix typo in htrace shell configuration name * TestFsShell: use Tracer constants for config key names. > update htrace version to 4.0 > > > Key: HDFS-9080 > URL: https://issues.apache.org/jira/browse/HDFS-9080 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe > Attachments: HDFS-9080.001.patch, HDFS-9080.002.patch, > HDFS-9080.003.patch, HDFS-9080.004.patch, HDFS-9080.005.patch, > HDFS-9080.006.patch, HDFS-9080.007.patch > > > Update the HTrace library version Hadoop uses to htrace 4.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9080) update htrace version to 4.0
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14805028#comment-14805028 ] Hadoop QA commented on HDFS-9080: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 24m 46s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 10 new or modified test files. | | {color:green}+1{color} | javac | 8m 18s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 22s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 24s | The applied patch does not increase the total number of release audit warnings. | | {color:green}+1{color} | site | 3m 6s | Site still builds. | | {color:red}-1{color} | checkstyle | 2m 43s | The applied patch generated 10 new checkstyle issues (total was 718, now 716). | | {color:red}-1{color} | checkstyle | 3m 44s | The applied patch generated 13 new checkstyle issues (total was 159, now 162). | | {color:red}-1{color} | whitespace | 1m 6s | The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 38s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 35s | The patch built with eclipse:eclipse. | | {color:red}-1{color} | findbugs | 6m 35s | The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. | | {color:red}-1{color} | common tests | 22m 5s | Tests failed in hadoop-common. | | {color:red}-1{color} | hdfs tests | 162m 43s | Tests failed in hadoop-hdfs. | | {color:green}+1{color} | hdfs tests | 0m 28s | Tests passed in hadoop-hdfs-client. | | | | 245m 57s | | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-common | | Failed unit tests | hadoop.fs.TestFsShellCopy | | | hadoop.fs.TestFSMainOperationsLocalFileSystem | | | hadoop.fs.shell.TestPathData | | | hadoop.fs.TestPath | | | hadoop.fs.shell.TestCopyPreserveFlag | | | hadoop.fs.TestTrash | | | hadoop.fs.TestFilterFileSystem | | | hadoop.fs.TestFsShell | | | hadoop.fs.TestHarFileSystem | | | hadoop.fs.shell.TestLs | | | hadoop.fs.shell.TestMove | | | hadoop.fs.TestFsShellReturnCode | | | hadoop.fs.viewfs.TestViewFsTrash | | | hadoop.hdfs.TestBlockReaderFactory | | | hadoop.hdfs.TestRemoteBlockReader | | | hadoop.hdfs.server.namenode.TestFsck | | | hadoop.tools.TestHdfsConfigFields | | | hadoop.hdfs.TestDFSOutputStream | | | hadoop.hdfs.TestClientBlockVerification | | | hadoop.cli.TestHDFSCLI | | | hadoop.hdfs.TestDFSShell | | | hadoop.hdfs.TestBlockReaderLocal | | | hadoop.fs.TestGlobPaths | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12757193/HDFS-9080.006.patch | | Optional Tests | javadoc javac unit findbugs checkstyle site | | git revision | trunk / 6b97fa6 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/12529/artifact/patchprocess/diffcheckstylehadoop-common.txt https://builds.apache.org/job/PreCommit-HDFS-Build/12529/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt | | whitespace | https://builds.apache.org/job/PreCommit-HDFS-Build/12529/artifact/patchprocess/whitespace.txt | | Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12529/artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html | | hadoop-common test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12529/artifact/patchprocess/testrun_hadoop-common.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12529/artifact/patchprocess/testrun_hadoop-hdfs.txt | | hadoop-hdfs-client test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12529/artifact/patchprocess/testrun_hadoop-hdfs-client.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12529/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12529/console | This message was automatically generated. > update htrace version to 4.0 > > > Key: HDFS-9080 > URL: https://issues.apache.org/jira/browse/HDFS-9080 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe > Attachments: HDFS-9080.001.patch, HDFS-9080.002.patch, > HDFS-9080.003.patch, HDFS-9080.004.patch, HDFS-9080.005.patch, > HDFS-9080.006.patch > >
[jira] [Commented] (HDFS-9080) update htrace version to 4.0
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14804927#comment-14804927 ] Hadoop QA commented on HDFS-9080: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 24m 42s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 10 new or modified test files. | | {color:green}+1{color} | javac | 8m 1s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 12s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 24s | The applied patch does not increase the total number of release audit warnings. | | {color:green}+1{color} | site | 3m 7s | Site still builds. | | {color:red}-1{color} | checkstyle | 2m 37s | The applied patch generated 10 new checkstyle issues (total was 718, now 716). | | {color:red}-1{color} | checkstyle | 3m 39s | The applied patch generated 13 new checkstyle issues (total was 159, now 162). | | {color:red}-1{color} | whitespace | 1m 8s | The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 36s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 36s | The patch built with eclipse:eclipse. | | {color:red}-1{color} | findbugs | 6m 23s | The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. | | {color:red}-1{color} | common tests | 21m 53s | Tests failed in hadoop-common. | | {color:red}-1{color} | hdfs tests | 0m 23s | Tests failed in hadoop-hdfs. | | {color:green}+1{color} | hdfs tests | 0m 27s | Tests passed in hadoop-hdfs-client. | | | | 82m 38s | | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-common | | Failed unit tests | hadoop.fs.TestHarFileSystem | | | hadoop.fs.TestFsShellReturnCode | | | hadoop.fs.shell.TestPathData | | | hadoop.fs.shell.TestMove | | | hadoop.fs.TestTrash | | | hadoop.fs.TestFsShellCopy | | | hadoop.fs.TestPath | | | hadoop.fs.TestFSMainOperationsLocalFileSystem | | | hadoop.fs.shell.TestCopyPreserveFlag | | | hadoop.fs.TestFilterFileSystem | | | hadoop.fs.TestFsShell | | | hadoop.fs.viewfs.TestViewFsTrash | | | hadoop.fs.shell.TestLs | | Failed build | hadoop-hdfs | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12757193/HDFS-9080.006.patch | | Optional Tests | javadoc javac unit findbugs checkstyle site | | git revision | trunk / 6b97fa6 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/12530/artifact/patchprocess/diffcheckstylehadoop-common.txt https://builds.apache.org/job/PreCommit-HDFS-Build/12530/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt | | whitespace | https://builds.apache.org/job/PreCommit-HDFS-Build/12530/artifact/patchprocess/whitespace.txt | | Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12530/artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html | | hadoop-common test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12530/artifact/patchprocess/testrun_hadoop-common.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12530/artifact/patchprocess/testrun_hadoop-hdfs.txt | | hadoop-hdfs-client test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12530/artifact/patchprocess/testrun_hadoop-hdfs-client.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12530/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12530/console | This message was automatically generated. > update htrace version to 4.0 > > > Key: HDFS-9080 > URL: https://issues.apache.org/jira/browse/HDFS-9080 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe > Attachments: HDFS-9080.001.patch, HDFS-9080.002.patch, > HDFS-9080.003.patch, HDFS-9080.004.patch, HDFS-9080.005.patch, > HDFS-9080.006.patch > > > Update the HTrace library version Hadoop uses to htrace 4.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9080) update htrace version to 4.0
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14804720#comment-14804720 ] Colin Patrick McCabe commented on HDFS-9080: Thanks for the review. bq. You want to add hostname+port to this name("FSClient").? ... at least as an annotation? It's automatically added by HTrace, depending on the configuration of {{tracer.id}}. bq. You change the names of the pb params but use old ids. Thats a no-no. Add the new params with new ids or, why not just note that you are stuffing high and low into trace and span from 4.0.0 out... and RPC stays the same. Hmm, those names are never sent over the wire, just the positions and types. But I guess I can leave them as-is. bq. Should TracerConfigurationManager do Sampler too? Or that can be another patch It definitely should, but that would be a new feature :) Right now you can only change {{SpanReceivers}} via RPC, not {{Samplers}}. That's definitely an oversight we should fix in another patch. bq. Below import \[not\] needed in your doc. Fixed bq. Unused 32 import org.apache.htrace.core.Tracer; in ClientContext? Fixed bq. I can see why you have Tracer as static on FileSystem. What will happen in case where two FileSystems in same JVM? Not important I suppose (there'll be no tracing in any setup – unit test – that does such a thing). Thats fine. It'll be OK. The Tracer methods are thread-safe. bq. I suppose this is correct... assuming that the parent at position zero is first parent Can it be null? (It throws exception in this case? A null parent down here would be an error?) No, it can't be null. > update htrace version to 4.0 > > > Key: HDFS-9080 > URL: https://issues.apache.org/jira/browse/HDFS-9080 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe > Attachments: HDFS-9080.001.patch, HDFS-9080.002.patch, > HDFS-9080.003.patch, HDFS-9080.004.patch > > > Update the HTrace library version Hadoop uses to htrace 4.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9080) update htrace version to 4.0
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14804708#comment-14804708 ] Hadoop QA commented on HDFS-9080: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 24m 46s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 10 new or modified test files. | | {color:green}+1{color} | javac | 8m 2s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 17s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 24s | The applied patch does not increase the total number of release audit warnings. | | {color:green}+1{color} | site | 3m 5s | Site still builds. | | {color:red}-1{color} | checkstyle | 2m 41s | The applied patch generated 10 new checkstyle issues (total was 718, now 716). | | {color:red}-1{color} | checkstyle | 3m 46s | The applied patch generated 13 new checkstyle issues (total was 160, now 163). | | {color:red}-1{color} | whitespace | 1m 8s | The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 37s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 36s | The patch built with eclipse:eclipse. | | {color:red}-1{color} | findbugs | 6m 28s | The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. | | {color:red}-1{color} | common tests | 22m 48s | Tests failed in hadoop-common. | | {color:red}-1{color} | hdfs tests | 37m 27s | Tests failed in hadoop-hdfs. | | {color:green}+1{color} | hdfs tests | 0m 28s | Tests passed in hadoop-hdfs-client. | | | | 120m 58s | | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-common | | Failed unit tests | hadoop.fs.shell.TestLs | | | hadoop.fs.shell.TestMove | | | hadoop.fs.TestTrash | | | hadoop.fs.viewfs.TestViewFsTrash | | | hadoop.fs.shell.TestCopyPreserveFlag | | | hadoop.fs.TestPath | | | hadoop.fs.shell.TestPathData | | | hadoop.fs.TestFSMainOperationsLocalFileSystem | | | hadoop.fs.TestHarFileSystem | | | hadoop.fs.TestFsShell | | | hadoop.fs.TestFsShellReturnCode | | | hadoop.fs.TestFilterFileSystem | | | hadoop.fs.TestFsShellCopy | | | hadoop.hdfs.server.namenode.TestHDFSConcat | | | hadoop.hdfs.server.blockmanagement.TestSequentialBlockId | | | hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics | | | hadoop.hdfs.server.namenode.ha.TestLossyRetryInvocationHandler | | | hadoop.hdfs.server.namenode.TestMetadataVersionOutput | | | hadoop.hdfs.server.namenode.TestFSNamesystemMBean | | | hadoop.hdfs.TestDFSInputStream | | | hadoop.hdfs.TestDFSStorageStateRecovery | | | hadoop.hdfs.TestSeekBug | | | hadoop.hdfs.TestFileAppendRestart | | | hadoop.hdfs.server.namenode.TestNameNodeXAttr | | | hadoop.hdfs.server.datanode.TestDatanodeProtocolRetryPolicy | | | hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica | | | hadoop.TestRefreshCallQueue | | | hadoop.hdfs.TestHdfsAdmin | | | hadoop.hdfs.TestGetBlocks | | | hadoop.hdfs.tools.TestDFSZKFailoverController | | | hadoop.hdfs.server.mover.TestMover | | | hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling | | | hadoop.hdfs.TestDFSUtil | | | hadoop.hdfs.qjournal.client.TestQJMWithFaults | | | hadoop.hdfs.TestHDFSServerPorts | | | hadoop.hdfs.server.datanode.TestHSync | | | hadoop.hdfs.server.namenode.TestStartup | | | hadoop.hdfs.web.TestWebHdfsTokens | | | hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer | | | hadoop.hdfs.TestDisableConnCache | | | hadoop.hdfs.server.datanode.TestDeleteBlockPool | | | hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock | | | hadoop.hdfs.TestRollingUpgradeRollback | | | hadoop.hdfs.TestDatanodeDeath | | | hadoop.hdfs.server.namenode.ha.TestHAMetrics | | | hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery | | | hadoop.hdfs.server.namenode.TestFileLimit | | | hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM | | | hadoop.hdfs.server.namenode.snapshot.TestSnapshotListing | | | hadoop.hdfs.TestParallelShortCircuitLegacyRead | | | hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes | | | hadoop.hdfs.server.datanode.TestBlockRecovery | | | hadoop.hdfs.server.datanode.TestDataNodeMetrics | | | hadoop.hdfs.tools.TestDelegationTokenFetcher | | | hadoop.hdfs.TestFileAppend4 | | | hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks | | | hadoop.hdfs.server.namenode.TestStorageRestore | | | hadoop.hdfs.server.namenode.TestLargeDirectoryDelete | | | hadoop.hdfs.server.blockmanagement.Tes
[jira] [Commented] (HDFS-9080) update htrace version to 4.0
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14804679#comment-14804679 ] stack commented on HDFS-9080: - You want to add hostname+port to thisname("FSClient").? ... at least as an annotation? This is NOT catastrophic: 76LOG.info("Failed to add SpanReceiver " + info.getClassName() + so INFO seems fine but what happens to the thrown exception? Is it going to be panic stations in the tiers above? You change the names of the pb params but use old ids. Thats a no-no. Add the new params with new ids or, why not just note that you are stuffing high and low into trace and span from 4.0.0 out... and RPC stays the same. 64optional int64 parentIdHigh = 1; 65optional int64 parentIdLow = 2; Above happens in two places. Should TracerConfigurationManager do Sampler too? Or that can be another patch. Below import needed in your doc. import org.apache.htrace.core.Sampler; Spelling static final String SEHLL_HTRACE_PREFIX = "dfs.shell.htrace."; Unused 32 import org.apache.htrace.core.Tracer; in ClientContext? I like the old name over this one TraceScope scope = newPathTraceScope("getBlockSize", f); : i.e. getPathT Same for newSrcDstTraceScope I can see why you have Tracer as static on FileSystem. What will happen in case where two FileSystems in same JVM? Not important I suppose (there'll be no tracing in any setup -- unit test -- that does such a thing). Thats fine. I suppose this is correct... assuming that the parent at position zero is first parent Can it be null? (It throws exception in this case? A null parent down here would be an error?) 560 scope = FileSystem.tracer.newScope("dataStreamer", parents[0]); 561 scope.getSpan().setParents(parents); > update htrace version to 4.0 > > > Key: HDFS-9080 > URL: https://issues.apache.org/jira/browse/HDFS-9080 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe > Attachments: HDFS-9080.001.patch, HDFS-9080.002.patch, > HDFS-9080.003.patch, HDFS-9080.004.patch > > > Update the HTrace library version Hadoop uses to htrace 4.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9080) update htrace version to 4.0
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14791255#comment-14791255 ] Hadoop QA commented on HDFS-9080: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 22m 34s | Findbugs (version 3.0.0) appears to be broken on trunk. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 10 new or modified test files. | | {color:green}+1{color} | javac | 8m 0s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 8s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 24s | The applied patch does not increase the total number of release audit warnings. | | {color:green}+1{color} | site | 3m 7s | Site still builds. | | {color:red}-1{color} | checkstyle | 2m 7s | The applied patch generated 4 new checkstyle issues (total was 681, now 674). | | {color:red}-1{color} | whitespace | 0m 52s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 38s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 6m 14s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:red}-1{color} | common tests | 23m 40s | Tests failed in hadoop-common. | | {color:red}-1{color} | hdfs tests | 175m 6s | Tests failed in hadoop-hdfs. | | {color:green}+1{color} | hdfs tests | 0m 31s | Tests passed in hadoop-hdfs-client. | | | | 255m 33s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.fs.viewfs.TestFcCreateMkdirLocalFs | | | hadoop.fs.viewfs.TestViewFsLocalFs | | | hadoop.fs.viewfs.TestFcPermissionsLocalFs | | | hadoop.fs.TestLocalFsFCStatistics | | | hadoop.fs.TestS3_LocalFileContextURI | | | hadoop.fs.TestLocal_S3FileContextURI | | | hadoop.fs.TestSymlinkLocalFSFileContext | | | hadoop.ipc.TestDecayRpcScheduler | | | hadoop.fs.TestFcLocalFsPermission | | | hadoop.fs.TestLocalFSFileContextMainOperations | | | hadoop.fs.TestLocalFSFileContextCreateMkdir | | | hadoop.fs.TestFileContextDeleteOnExit | | | hadoop.fs.viewfs.TestViewFsWithAuthorityLocalFs | | | hadoop.fs.viewfs.TestChRootedFs | | | hadoop.fs.TestFsShell | | | hadoop.fs.viewfs.TestFcMainOperationsLocalFs | | | hadoop.fs.TestFcLocalFsUtil | | | hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes | | | hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys | | | hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional | | | hadoop.hdfs.server.namenode.TestStorageRestore | | | hadoop.hdfs.qjournal.TestNNWithQJM | | | hadoop.hdfs.server.namenode.TestSecureNameNode | | | hadoop.hdfs.server.namenode.TestGenericJournalConf | | | hadoop.hdfs.server.namenode.TestEditLog | | | hadoop.hdfs.TestRollingUpgradeDowngrade | | | hadoop.hdfs.tools.TestGetGroups | | | hadoop.hdfs.server.namenode.TestNameNodeRecovery | | | hadoop.hdfs.server.namenode.TestProtectedDirectories | | | hadoop.hdfs.TestHdfsAdmin | | | hadoop.hdfs.server.namenode.TestEditLogJournalFailures | | | hadoop.hdfs.TestDFSMkdirs | | | hadoop.hdfs.TestDatanodeConfig | | | hadoop.hdfs.server.namenode.TestBackupNode | | | hadoop.hdfs.server.namenode.TestCheckpoint | | | hadoop.tools.TestHdfsConfigFields | | | hadoop.hdfs.server.namenode.TestStartup | | | hadoop.hdfs.web.TestWebHDFSOAuth2 | | | hadoop.hdfs.server.namenode.TestNameNodeMXBean | | | hadoop.hdfs.TestRollingUpgradeRollback | | | hadoop.hdfs.TestDFSUpgradeFromImage | | | hadoop.hdfs.security.TestDelegationToken | | | hadoop.hdfs.server.namenode.TestSecondaryWebUi | | | hadoop.hdfs.server.datanode.TestBlockScanner | | | hadoop.hdfs.server.namenode.TestTransferFsImage | | | hadoop.hdfs.server.namenode.TestCreateEditsLog | | Timed out tests | org.apache.hadoop.hdfs.TestSeekBug | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12756319/HDFS-9080.003.patch | | Optional Tests | javadoc javac unit findbugs checkstyle site | | git revision | trunk / bf2f2b4 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/12488/artifact/patchprocess/diffcheckstylehadoop-common.txt | | whitespace | https://builds.apache.org/job/PreCommit-HDFS-Build/12488/artifact/patchprocess/whitespace.txt | | hadoop-common test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12488/artifact/patchprocess/testrun_hadoop-common.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12488/artifact/patchprocess/testrun_hadoop-hdfs.txt | | ha
[jira] [Commented] (HDFS-9080) update htrace version to 4.0
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746130#comment-14746130 ] Hadoop QA commented on HDFS-9080: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | patch | 0m 0s | The patch command could not apply the patch during dryrun. | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12756063/HDFS-9080.002.patch | | Optional Tests | javadoc javac unit findbugs checkstyle site | | git revision | trunk / 8c1cdb1 | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12457/console | This message was automatically generated. > update htrace version to 4.0 > > > Key: HDFS-9080 > URL: https://issues.apache.org/jira/browse/HDFS-9080 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Colin Patrick McCabe >Assignee: Colin Patrick McCabe > Attachments: HDFS-9080.001.patch, HDFS-9080.002.patch > > > Update the HTrace library version Hadoop uses to htrace 4.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9080) update htrace version to 4.0
[ https://issues.apache.org/jira/browse/HDFS-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14744879#comment-14744879 ] Hadoop QA commented on HDFS-9080: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 28m 6s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 10 new or modified test files. | | {color:green}+1{color} | javac | 10m 3s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 13m 20s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 27s | The applied patch does not increase the total number of release audit warnings. | | {color:green}+1{color} | site | 3m 55s | Site still builds. | | {color:red}-1{color} | checkstyle | 3m 4s | The applied patch generated 10 new checkstyle issues (total was 681, now 680). | | {color:red}-1{color} | whitespace | 0m 57s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 2m 8s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 44s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 7m 47s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:red}-1{color} | common tests | 23m 23s | Tests failed in hadoop-common. | | {color:red}-1{color} | hdfs tests | 32m 11s | Tests failed in hadoop-hdfs. | | {color:red}-1{color} | hdfs tests | 0m 22s | Tests failed in hadoop-hdfs-client. | | | | 127m 13s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.fs.viewfs.TestFcCreateMkdirLocalFs | | | hadoop.fs.viewfs.TestViewFsLocalFs | | | hadoop.fs.viewfs.TestFcPermissionsLocalFs | | | hadoop.fs.TestLocalFsFCStatistics | | | hadoop.fs.TestS3_LocalFileContextURI | | | hadoop.fs.TestLocal_S3FileContextURI | | | hadoop.fs.TestSymlinkLocalFSFileContext | | | hadoop.fs.TestFcLocalFsPermission | | | hadoop.fs.TestLocalFSFileContextMainOperations | | | hadoop.fs.TestLocalFSFileContextCreateMkdir | | | hadoop.fs.TestFileContextDeleteOnExit | | | hadoop.fs.viewfs.TestViewFsWithAuthorityLocalFs | | | hadoop.fs.viewfs.TestChRootedFs | | | hadoop.fs.TestFsShell | | | hadoop.fs.viewfs.TestFcMainOperationsLocalFs | | | hadoop.fs.TestFcLocalFsUtil | | | hadoop.hdfs.TestFileStatus | | | hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes | | | hadoop.hdfs.server.namenode.TestINodeFile | | | hadoop.fs.contract.hdfs.TestHDFSContractOpen | | | hadoop.hdfs.server.datanode.TestFsDatasetCache | | | hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart | | | hadoop.hdfs.TestFileCreationDelete | | | hadoop.hdfs.server.namenode.ha.TestHASafeMode | | | hadoop.hdfs.TestEncryptionZonesWithHA | | | hadoop.fs.contract.hdfs.TestHDFSContractMkdir | | | hadoop.hdfs.TestDFSShell | | | hadoop.hdfs.server.namenode.TestNameNodeXAttr | | | hadoop.hdfs.qjournal.TestMiniJournalCluster | | | hadoop.hdfs.server.namenode.ha.TestBootstrapStandby | | | hadoop.hdfs.shortcircuit.TestShortCircuitCache | | | hadoop.hdfs.TestDecommission | | | hadoop.hdfs.server.namenode.TestFSEditLogLoader | | | hadoop.hdfs.server.namenode.ha.TestDelegationTokensWithHA | | | hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes | | | hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages | | | hadoop.hdfs.server.datanode.TestCachingStrategy | | | hadoop.hdfs.server.datanode.TestDirectoryScanner | | | hadoop.cli.TestXAttrCLI | | | hadoop.hdfs.server.namenode.TestDeleteRace | | | hadoop.hdfs.server.namenode.TestParallelImageWrite | | | hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys | | | hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional | | | hadoop.hdfs.server.namenode.TestSaveNamespace | | | hadoop.hdfs.TestReplaceDatanodeOnFailure | | | hadoop.hdfs.TestDFSRename | | | hadoop.hdfs.server.namenode.TestFsck | | | hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA | | | hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement | | | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration | | | hadoop.hdfs.server.datanode.TestDeleteBlockPool | | | hadoop.hdfs.TestRemoteBlockReader2 | | | hadoop.hdfs.server.namenode.TestStorageRestore | | | hadoop.hdfs.server.namenode.TestFileLimit | | | hadoop.hdfs.server.blockmanagement.TestNodeCount | | | hadoop.hdfs.TestEncryptionZones | | | hadoop.fs.contract.hdfs.TestHDFSContractSetTimes | | | hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots | | | hadoop.hdfs.qjournal.TestNNWithQJM | |