[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16594241#comment-16594241 ] Colin Garcia commented on HBASE-20306: -- Added your suggestions! > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch, HBASE-20306.001.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20306: - Attachment: HBASE-20306.001.patch > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch, HBASE-20306.001.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16594244#comment-16594244 ] Colin Garcia commented on HBASE-20306: -- {code:java} 2018-08-27 14:29:33,296 INFO [MultiThreadedAction-ProgressReporter-1535405323258] util.MultiThreadedAction: [W:20] Keys=87549, cols=985.4 K, time=00:00:50 Overall: [keys/s= 1749, latency=11.34 ms] Current: [keys/s=1732, latency=11.47 ms], wroteUpTo=-1 2018-08-27 14:29:38,299 INFO [MultiThreadedAction-ProgressReporter-1535405323258] util.MultiThreadedAction: [W:20] Keys=96588, cols=1.1 M, time=00:00:55 Overall: [keys/s= 1754, latency=11.31 ms] Current: [keys/s=1807, latency=10.98 ms], wroteUpTo=-1 2018-08-27 14:29:43,303 INFO [MultiThreadedAction-ProgressReporter-1535405323258] util.MultiThreadedAction: RUN SUMMARY: W Keys=96588, cols=1.1 M, time=00:00:55 Overall: [keys/s= 1754, latency=11.31 ms] 2018-08-27 14:29:43,324 INFO [ReadOnlyZKClient-localhost:2181@0x0b6b1987-EventThread] zookeeper.ClientCnxn: EventThread shut down for session: 0x1657d2d21d30023 {code} > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch, HBASE-20306.001.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20779) Server version is not right when enable TABLES_ON_MASTER
[ https://issues.apache.org/jira/browse/HBASE-20779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20779: - Description: When enble TABLES_ON_MASTER, master will be a region server to carry regions. So it will report to itself, too. And we get server version from rpc call. But master report to itself will skip rpc call. Then ServerManager will record a wrong version 0. (was: When eable TABLES_ON_MASTER, master will be a region server to carry regions. So it will report to itself, too. And we get server version from rpc call. But master report to itself will skip rpc call. Then ServerManager will record a wrong version 0.) > Server version is not right when enable TABLES_ON_MASTER > > > Key: HBASE-20779 > URL: https://issues.apache.org/jira/browse/HBASE-20779 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Priority: Major > > When enble TABLES_ON_MASTER, master will be a region server to carry regions. > So it will report to itself, too. And we get server version from rpc call. > But master report to itself will skip rpc call. Then ServerManager will > record a wrong version 0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20779) Server version is not right when enable TABLES_ON_MASTER
[ https://issues.apache.org/jira/browse/HBASE-20779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20779: - Description: When enable TABLES_ON_MASTER, master will be a region server to carry regions. So it will report to itself, too. And we get server version from rpc call. But master report to itself will skip rpc call. Then ServerManager will record a wrong version 0. (was: When enble TABLES_ON_MASTER, master will be a region server to carry regions. So it will report to itself, too. And we get server version from rpc call. But master report to itself will skip rpc call. Then ServerManager will record a wrong version 0.) > Server version is not right when enable TABLES_ON_MASTER > > > Key: HBASE-20779 > URL: https://issues.apache.org/jira/browse/HBASE-20779 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Priority: Major > > When enable TABLES_ON_MASTER, master will be a region server to carry > regions. So it will report to itself, too. And we get server version from rpc > call. But master report to itself will skip rpc call. Then ServerManager will > record a wrong version 0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20307) LoadTestTool prints too much zookeeper logging
[ https://issues.apache.org/jira/browse/HBASE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597962#comment-16597962 ] Colin Garcia commented on HBASE-20307: -- Bump. Just wondering if you guys had any thoughts about the proposed changes? > LoadTestTool prints too much zookeeper logging > -- > > Key: HBASE-20307 > URL: https://issues.apache.org/jira/browse/HBASE-20307 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20307.000.patch > > > When running ltt there is a ton of ZK related cruft that I probably don't > care about. Hide it behind -verbose flag or point people at log4j > configuration but don't print it by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20307) LoadTestTool prints too much zookeeper logging
[ https://issues.apache.org/jira/browse/HBASE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20307: - Attachment: HBASE-20307.001.patch > LoadTestTool prints too much zookeeper logging > -- > > Key: HBASE-20307 > URL: https://issues.apache.org/jira/browse/HBASE-20307 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20307.000.patch, HBASE-20307.001.patch > > > When running ltt there is a ton of ZK related cruft that I probably don't > care about. Hide it behind -verbose flag or point people at log4j > configuration but don't print it by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612569#comment-16612569 ] Colin Garcia commented on HBASE-20306: -- [~mdrob] I was wondering if you've had a chance to look this over? > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch, HBASE-20306.001.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20306: - Attachment: HBASE-20306.002.patch > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch, HBASE-20306.001.patch, > HBASE-20306.002.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16614096#comment-16614096 ] Colin Garcia commented on HBASE-20306: -- Thanks for the comments Andrew. I've updated using the getHistogramReport functionality. The only concern I have is that the values won't exactly match the "Overall Status" due to some casting to long from doubles. Here is a snippet of the output. Let me know what you think :) hoping this is a step in the right direction {code:java} 2018-09-13 14:30:52,732 INFO [MultiThreadedAction-ProgressReporter-1536874207705] util.MultiThreadedAction: [W:20] Keys=82495, cols=927.8 K, time=00:00:45 Overall: [keys/s= 1832, latency=10.82 ms] Current: [keys/s=2058, latency=9.64 ms], wroteUpTo=-1 2018-09-13 14:30:57,733 INFO [MultiThreadedAction-ProgressReporter-1536874207705] util.MultiThreadedAction: [W:20] Keys=92516, cols=1.0 M, time=00:00:50 Overall: [keys/s= 1849, latency=10.72 ms] Current: [keys/s=2004, latency=9.89 ms], wroteUpTo=-1 2018-09-13 14:31:02,738 INFO [MultiThreadedAction-ProgressReporter-1536874207705] util.MultiThreadedAction: RUN SUMMARY KEYS PER SECOND: mean=1849.90, min=1108.00, max=2065.00, stdDev=291.71, 50th=1945.50, 75th=2017.50, 95th=2065.00, 99th=2065.00, 99.9th=2065.00, 99.99th=2065.00, 99.999th=2065.00 LATENCY: mean=10.60, min=9.00, max=17.00, stdDev=2.41, 50th=10.00, 75th=10.50, 95th=17.00, 99th=17.00, 99.9th=17.00, 99.99th=17.00, 99.999th=17.00 {code} > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch, HBASE-20306.001.patch, > HBASE-20306.002.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16636241#comment-16636241 ] Colin Garcia commented on HBASE-20306: -- [~apurtell] Thanks! Strangely, I ran with the same input and both threads finished (although it took some time). Could there be a race condition here? I don't see how one could run indefinitely as at some point the worker threads will finish, setting isProgressReporter to false. Can you paste what your output was? Fixing the other pieces you suggested now. > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306-branch-1.000.patch, HBASE-20306.000.patch, > HBASE-20306.001.patch, HBASE-20306.002.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20306: - Attachment: HBASE-20306.003.patch > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306-branch-1.000.patch, HBASE-20306.000.patch, > HBASE-20306.001.patch, HBASE-20306.002.patch, HBASE-20306.003.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16636264#comment-16636264 ] Colin Garcia commented on HBASE-20306: -- I see. But I don't think the old if/else is right then either. If the reader progress was printing the same status, that would have to mean that there were still worker threads running (as they were formerly the same check), so the old if/else wouldn't have terminated either. This problem might be elsewhere, unrelated to the change but having something to do with how these worker threads are exiting. > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306-branch-1.000.patch, HBASE-20306.000.patch, > HBASE-20306.001.patch, HBASE-20306.002.patch, HBASE-20306.003.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16636264#comment-16636264 ] Colin Garcia edited comment on HBASE-20306 at 10/3/18 12:16 AM: I see. But I don't think the old if/else is right then either. If the reader progress was printing the same status, that would have to mean that there were still worker threads running (as they were formerly the same check), so the old if/else wouldn't have terminated either. This problem might be elsewhere, unrelated to the change but having something to do with how these worker threads are exiting. I'm rerunning to see if I can mimic the behavior you describe. was (Author: colingarcia): I see. But I don't think the old if/else is right then either. If the reader progress was printing the same status, that would have to mean that there were still worker threads running (as they were formerly the same check), so the old if/else wouldn't have terminated either. This problem might be elsewhere, unrelated to the change but having something to do with how these worker threads are exiting. > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306-branch-1.000.patch, HBASE-20306.000.patch, > HBASE-20306.001.patch, HBASE-20306.002.patch, HBASE-20306.003.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16636274#comment-16636274 ] Colin Garcia commented on HBASE-20306: -- That's fine, I agree with this assessment. I'm just spitballing to determine what the cause could be. Will keep investigating. > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306-branch-1.000.patch, HBASE-20306.000.patch, > HBASE-20306.001.patch, HBASE-20306.002.patch, HBASE-20306.003.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16636299#comment-16636299 ] Colin Garcia commented on HBASE-20306: -- My output from the latest run, for reference: {code:java} ~/hbase/bin(master*) » ./hbase ltt -read 100:10 -write 10:100:10 -num_keys 100 {code} {code:java} 2018-10-02 17:42:15,369 INFO [MultiThreadedAction-ProgressReporter-1538525614665] util.MultiThreadedAction: [W:11] Keys=993612, cols=10.9 M, time=00:28:40 Overall: [keys/s= 577, latency=17.27 ms] Current: [keys/s=566, latency=17.62 ms], wroteUpTo=993597, wroteQSize=4 2018-10-02 17:42:15,423 INFO [MultiThreadedAction-ProgressReporter-1538525614699] util.MultiThreadedAction: [R:10] Keys=9144453, cols=155.6 M, time=00:28:40 Overall: [keys/s= 5314, latency=0.90 ms] Current: [keys/s=5187, latency=0.95 ms], verified=9144453 2018-10-02 17:42:20,369 INFO [MultiThreadedAction-ProgressReporter-1538525614665] util.MultiThreadedAction: [W:11] Keys=996682, cols=10.9 M, time=00:28:45 Overall: [keys/s= 577, latency=17.26 ms] Current: [keys/s=614, latency=16.24 ms], wroteUpTo=996669, wroteQSize=2 2018-10-02 17:42:20,425 INFO [MultiThreadedAction-ProgressReporter-1538525614699] util.MultiThreadedAction: [R:10] Keys=9171711, cols=156.1 M, time=00:28:45 Overall: [keys/s= 5314, latency=0.90 ms] Current: [keys/s=5451, latency=0.84 ms], verified=9171711 2018-10-02 17:42:25,369 INFO [MultiThreadedAction-ProgressReporter-1538525614665] util.MultiThreadedAction: [W:11] Keys=999798, cols=11.0 M, time=00:28:50 Overall: [keys/s= 577, latency=17.26 ms] Current: [keys/s=623, latency=16.01 ms], wroteUpTo=999780, wroteQSize=7 2018-10-02 17:42:25,429 INFO [MultiThreadedAction-ProgressReporter-1538525614699] util.MultiThreadedAction: [R:10] Keys=9199696, cols=156.5 M, time=00:28:50 Overall: [keys/s= 5315, latency=0.90 ms] Current: [keys/s=5597, latency=0.81 ms], verified=9199699 2018-10-02 17:42:30,406 INFO [MultiThreadedAction-ProgressReporter-1538525614665] util.MultiThreadedAction: RUN SUMMARY KEYS PER SECOND [W]: mean=577.53, min=322.00, max=729.00, stdDev=65.36, 50th=586.00, 75th=622.00, 95th=663.00, 99th=711.30, 99.9th=729.00, 99.99th=729.00, 99.999th=729.00 LATENCY [W]: mean=17.05, min=13.00, max=30.00, stdDev=2.42, 50th=16.00, 75th=18.00, 95th=22.00, 99th=27.00, 99.9th=30.00, 99.99th=30.00, 99.999th=30.00 2018-10-02 17:42:30,434 INFO [MultiThreadedAction-ProgressReporter-1538525614699] util.MultiThreadedAction: RUN SUMMARY KEYS PER SECOND [R]: mean=5317.31, min=2953.00, max=6900.00, stdDev=639.58, 50th=5366.00, 75th=5727.00, 95th=6248.65, 99th=6598.89, 99.9th=6900.00, 99.99th=6900.00, 99.999th=6900.00 LATENCY [R]: mean=0.28, min=0.00, max=2.00, stdDev=0.46, 50th=0.00, 75th=1.00, 95th=1.00, 99th=1.00, 99.9th=2.00, 99.99th=2.00, 99.999th=2.00 2018-10-02 17:42:30,783 INFO [main] zookeeper.ReadOnlyZKClient: Close zookeeper connection 0x31aa3ca5 to localhost:2181 Failed to write keys: 0 2018-10-02 17:42:30,794 INFO [main] zookeeper.ReadOnlyZKClient: Close zookeeper connection 0x6b44435b to localhost:2181{code} > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306-branch-1.000.patch, HBASE-20306.000.patch, > HBASE-20306.001.patch, HBASE-20306.002.patch, HBASE-20306.003.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642281#comment-16642281 ] Colin Garcia commented on HBASE-20306: -- Hey Andrew. I was wondering what test env you were using to run the ltt command? I've been running it locally, and haven't been able to replicate. Thanks in advance > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306-branch-1.000.patch, HBASE-20306.000.patch, > HBASE-20306.001.patch, HBASE-20306.002.patch, HBASE-20306.003.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662655#comment-16662655 ] Colin Garcia commented on HBASE-20306: -- Andrew, is it possible for me to use your test cluster? I've ran locally several times and have been unable to replicate the behavior you've described. > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306-branch-1.000.patch, HBASE-20306.000.patch, > HBASE-20306.001.patch, HBASE-20306.002.patch, HBASE-20306.003.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20307) LoadTestTool prints too much zookeeper logging
[ https://issues.apache.org/jira/browse/HBASE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586394#comment-16586394 ] Colin Garcia commented on HBASE-20307: -- Cc: [~liuml07] > LoadTestTool prints too much zookeeper logging > -- > > Key: HBASE-20307 > URL: https://issues.apache.org/jira/browse/HBASE-20307 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Priority: Major > Labels: beginner > > When running ltt there is a ton of ZK related cruft that I probably don't > care about. Hide it behind -verbose flag or point people at log4j > configuration but don't print it by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20307) LoadTestTool prints too much zookeeper logging
[ https://issues.apache.org/jira/browse/HBASE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586392#comment-16586392 ] Colin Garcia commented on HBASE-20307: -- Hi [~mdrob], new to HBase and was looking to take this on. I've ran the following command: {code:java} ~/hbase/bin(master) » ./hbase ltt -tn TestTable -read 10:1000 -num_keys 1000 {code} And have posted the output below. I was wondering if we're looking to hide the logs of the form: [ReadOnlyZKClient-localhost*]? Will ReadOnlyZKClient be called for all the various inputs? Output for reference: {code:java} ~/hbase/bin(master) » ./hbase ltt -tn TestTable -read 10:1000 -num_keys 1000 Key range: [0..999] Multi-gets (value of 1 means no multigets): 1 Percent of keys to verify: 10 Reader threads: 1000 2018-08-17 14:51:14,391 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2018-08-17 14:51:15,600 INFO [main] zookeeper.ReadOnlyZKClient: Connect 0x7193666c to localhost:2181 with session timeout=9ms, retries 30, retry interval 1000ms, keepAlive=6ms 2018-08-17 14:51:15,610 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT 2018-08-17 14:51:15,610 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client environment:host.name=colingarc-ltm8.internal.salesforce.com 2018-08-17 14:51:15,610 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client environment:java.version=1.8.0_181 2018-08-17 14:51:15,610 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 2018-08-17 14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client environment:java.home=/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre 2018-08-17 14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: cia/.m2/repository/javax/validation/validation-api/1.1.0.Final/validation-api-1.1.0.Final.jar:/Users/colin.garcia/hbase/hbase-examples/target/hbase-examples-3.0.0-SNAPSHOT.jar:/Users/colin.garcia/hbase/hbase-endpoint/target/hbase-endpoint-3.0.0-SNAPSHOT.jar:/Users/colin.garcia/.m2/repository/org/apache/curator/curator-framework/4.0.0/curator-framework-4.0.0.jar:/Users/colin.garcia/.m2/repository/org/apache/curator/curator-client/4.0.0/curator-client-4.0.0.jar:/Users/colin.garcia/.m2/repository/org/apache/curator/curator-recipes/4.0.0/curator-recipes-4.0.0.jar:/Users/colin.garcia/hbase/hbase-zookeeper/target/hbase-zookeeper-3.0.0-SNAPSHOT.jar:/Users/colin.garcia/hbase/hbase-rsgroup/target/hbase-rsgroup-3.0.0-SNAPSHOT.jar:/Users/colin.garcia/hbase/hbase-annotations/target/hbase-annotations-3.0.0-SNAPSHOT-tests.jar:/Users/colin.garcia/.m2/repository/org/apache/yetus/audience-annotations/0.5.0/audience-annotations-0.5.0.jar:/Users/colin.garcia/.m2/repository/junit/junit/4.12/junit-4.12.jar 2018-08-17 14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client environment:java.library.path=/Users/colin.garcia/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:. 2018-08-17 14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/var/folders/ct/l2m2v7h95nb6n_cvp0jf73p1l_c_hb/T/ 2018-08-17 14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client environment:java.compiler= 2018-08-17 14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client environment:os.name=Mac OS X 2018-08-17 14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client environment:os.arch=x86_64 2018-08-17 14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client environment:os.version=10.13.6 2018-08-17 14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client environment:user.name=colin.garcia 2018-08-17 14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client environment:user.home=/Users/colin.garcia 2018-08-17 14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client environment:user.dir=/Users/colin.garcia/hbase/bin 2018-08-17 14:51:15,612 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Initiating client connection, connectString
[jira] [Commented] (HBASE-20307) LoadTestTool prints too much zookeeper logging
[ https://issues.apache.org/jira/browse/HBASE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586656#comment-16586656 ] Colin Garcia commented on HBASE-20307: -- [~mdrob] [~stack], thanks for the responses! Makes sense. Discussing this with others, we have a workaround proposal and want to see what you think. I'll post a diff with the changes and a copy of the logs below. We set the ZK logging level to WARN in the LoadTestTool class, which means right now it only works for LoadTestTool (currently experimenting to see if it can be added to ReadOnlyZKClient so that we don't have to add this code to each tool). Here is an updated look at the logs: {code:java} ~/hbase/bin(master*) » ./hbase ltt -tn TestTable -read 10:1000 -num_keys 1000 Key range: [0..999] Multi-gets (value of 1 means no multigets): 1 Percent of keys to verify: 10 Reader threads: 1000 2018-08-20 15:59:08,307 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2018-08-20 15:59:08,526 INFO [main] zookeeper.ReadOnlyZKClient: Connect 0x03835c46 to localhost:2181 with session timeout=9ms, retries 30, retry interval 1000ms, keepAlive=6ms 2018-08-20 15:59:08,574 INFO [ReadOnlyZKClient-localhost:2181@0x03835c46-SendThread(localhost:2181)] zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2018-08-20 15:59:08,606 INFO [ReadOnlyZKClient-localhost:2181@0x03835c46-SendThread(localhost:2181)] zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:2181, initiating session 2018-08-20 15:59:08,614 INFO [ReadOnlyZKClient-localhost:2181@0x03835c46-SendThread(localhost:2181)] zookeeper.ClientCnxn: Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x165599020930005, negotiated timeout = 4 2018-08-20 15:59:13,376 INFO [main] client.RpcRetryingCallerImpl: Call exception, tries=6, retries=16, started=4634 ms ago, cancelled=false, msg=org.apache.hadoop.hbase.PleaseHoldException: Master is initializing 2018-08-20 15:59:17,440 INFO [main] hbase.HBaseCommonTestingUtility: Number of live regionservers: 1, pre-splitting table into 5 regions (regions per server: 5) 2018-08-20 15:59:17,579 WARN [main] hbase.HBaseCommonTestingUtility: Table TestTable already exists, continuing 2018-08-20 15:59:17,580 INFO [main] client.ConnectionImplementation: Closing master protocol: MasterService 2018-08-20 15:59:17,580 INFO [main] zookeeper.ReadOnlyZKClient: Close zookeeper connection 0x03835c46 to localhost:2181 2018-08-20 15:59:17,583 INFO [ReadOnlyZKClient-localhost:2181@0x03835c46-EventThread] zookeeper.ClientCnxn: EventThread shut down for session: 0x165599020930005 2018-08-20 15:59:17,597 INFO [main] zookeeper.ReadOnlyZKClient: Connect 0x68fa0ba8 to localhost:2181 with session timeout=9ms, retries 30, retry interval 1000ms, keepAlive=6ms 2018-08-20 15:59:17,598 INFO [ReadOnlyZKClient-localhost:2181@0x68fa0ba8-SendThread(localhost:2181)] zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2018-08-20 15:59:17,599 INFO [ReadOnlyZKClient-localhost:2181@0x68fa0ba8-SendThread(localhost:2181)] zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:2181, initiating session 2018-08-20 15:59:17,600 INFO [ReadOnlyZKClient-localhost:2181@0x68fa0ba8-SendThread(localhost:2181)] zookeeper.ClientCnxn: Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x165599020930006, negotiated timeout = 4 2018-08-20 15:59:17,621 INFO [main] util.LoadTestTool: Disabling table TestTable 2018-08-20 15:59:17,622 INFO [main] client.HBaseAdmin: Started disable of TestTable 2018-08-20 15:59:18,068 INFO [main] client.HBaseAdmin: Operation: DISABLE, Table Name: default:TestTable, procId: 54 completed 2018-08-20 15:59:18,219 INFO [main] util.LoadTestTool: Enabling table TestTable 2018-08-20 15:59:18,220 INFO [main] client.HBaseAdmin: Started enable of TestTable 2018-08-20 15:59:18,963 INFO [main] client.HBaseAdmin: Operation: ENABLE, Table Name: default:TestTable, procId: 61 completed 2018-08-20 15:59:18,963 INFO [main] client.ConnectionImplementation: Closing master protocol: MasterService 2018-08-20 15:59:18,963 INFO [main] zookeeper.ReadOnlyZKClient: Close zookeeper connection 0x68fa0ba8 to localhost:2181 2018-08-20 15:59:18,966 INFO [ReadOnlyZKClient-localhost:2181@0x68fa0ba8-EventThread] zookeeper.ClientCnxn: EventThread shut down for session: 0x165599020930006 2018-08-20 15:59:18,967 INFO [main] zookeeper.ReadOnlyZKClient: Connect 0x66b7550d to localhost:2181 with session timeout=9ms, retries 30, retry interval 1000ms,
[jira] [Commented] (HBASE-20307) LoadTestTool prints too much zookeeper logging
[ https://issues.apache.org/jira/browse/HBASE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586670#comment-16586670 ] Colin Garcia commented on HBASE-20307: -- Can I be added as committer to post the patch? Here is the diff (it's a small change): {code:java} ~/hbase/bin(master*) » cat HBASE-20307.000.patch colin.garcia@colingarc-ltm8 diff --git a/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java b/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java index 41788588ff..0c0023b593 100644 --- a/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java +++ b/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java @@ -36,7 +36,10 @@ import org.apache.hadoop.hbase.HConstants; import org.apache.hadoop.hbase.TableName; import org.apache.hadoop.hbase.client.ColumnFamilyDescriptor; import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder; +import org.apache.log4j.Level; +import org.apache.log4j.LogManager; import org.apache.yetus.audience.InterfaceAudience; +import org.apache.zookeeper.ZooKeeper; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.apache.hadoop.hbase.client.Admin; @@ -565,6 +568,7 @@ public class LoadTestTool extends AbstractHBaseTool { @Override protected int doWork() throws IOException { + LogManager.getLogger(ZooKeeper.class.getName()).setLevel(Level.WARN); if (numTables > 1) { return parallelLoadTables(); } else { {code} Note that we can add another line if we'd like to remove the logs about ClientCnxn, but that approach doesn't seem optimal. Is there a root ZK Logger we can set to WARN to remove any logs of Zookeeper.*? > LoadTestTool prints too much zookeeper logging > -- > > Key: HBASE-20307 > URL: https://issues.apache.org/jira/browse/HBASE-20307 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Priority: Major > Labels: beginner > > When running ltt there is a ton of ZK related cruft that I probably don't > care about. Hide it behind -verbose flag or point people at log4j > configuration but don't print it by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20307) LoadTestTool prints too much zookeeper logging
[ https://issues.apache.org/jira/browse/HBASE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20307: - Attachment: HBASE-20307.000.patch > LoadTestTool prints too much zookeeper logging > -- > > Key: HBASE-20307 > URL: https://issues.apache.org/jira/browse/HBASE-20307 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20307.000.patch > > > When running ltt there is a ton of ZK related cruft that I probably don't > care about. Hide it behind -verbose flag or point people at log4j > configuration but don't print it by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587823#comment-16587823 ] Colin Garcia commented on HBASE-20306: -- [~jojochuang] I was wondering what progress you've made? Looking to take a look into this as well :) > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Wei-Chiu Chuang >Priority: Major > Labels: beginner > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590796#comment-16590796 ] Colin Garcia commented on HBASE-20306: -- [~mdrob] I've been looking at this. Based on the output below, do you just want a final line that's something like the output below? Should I remove the Overall at each step and just put in overall at the end? Thanks in advance :) {code:java} 2018-08-23 13:52:52,435 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=45976, cols=972.4 K, time=00:00:30 Overall: [keys/s= 1531, latency=644.90 ms] Current: [keys/s=2017, latency=492.38 ms], verified=4704 2018-08-23 13:52:57,439 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=56588, cols=1.2 M, time=00:00:35 Overall: [keys/s= 1615, latency=610.22 ms] Current: [keys/s=2122, latency=459.96 ms], verified=5755 2018-08-23 13:53:02,440 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=63442, cols=1.3 M, time=00:00:40 Overall: [keys/s= 1584, latency=620.65 ms] Current: [keys/s=1370, latency=706.78 ms], verified=6454 2018-08-23 13:53:07,442 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=72112, cols=1.5 M, time=00:00:45 Overall: [keys/s= 1601, latency=618.87 ms] Current: [keys/s=1734, latency=605.84 ms], verified=7293 2018-08-23 13:53:12,443 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=82197, cols=1.7 M, time=00:00:50 Overall: [keys/s= 1642, latency=598.89 ms] Current: [keys/s=2017, latency=455.99 ms], verified=8272 2018-08-23 13:53:17,444 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=94334, cols=1.9 M, time=00:00:55 Overall: [keys/s= 1714, latency=580.54 ms] Current: [keys/s=2427, latency=456.27 ms], verified=9511 2018-08-23 13:53:22,445 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: RUN SUMMARY: Keys=94334, cols=2.1 M, time=00:00:55 Overall: [keys/s= 1714, latency=580.54 ms] {code} > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Wei-Chiu Chuang >Priority: Major > Labels: beginner > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590796#comment-16590796 ] Colin Garcia edited comment on HBASE-20306 at 8/23/18 8:56 PM: --- [~mdrob] I've been looking at this. Based on the output below, do you just want a final line that's something like the output below? Should I remove the Overall at each step and just put in overall at the end? Thanks in advance :) {code:java} 2018-08-23 13:52:52,435 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=45976, cols=972.4 K, time=00:00:30 Overall: [keys/s= 1531, latency=644.90 ms] Current: [keys/s=2017, latency=492.38 ms], verified=4704 2018-08-23 13:52:57,439 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=56588, cols=1.2 M, time=00:00:35 Overall: [keys/s= 1615, latency=610.22 ms] Current: [keys/s=2122, latency=459.96 ms], verified=5755 2018-08-23 13:53:02,440 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=63442, cols=1.3 M, time=00:00:40 Overall: [keys/s= 1584, latency=620.65 ms] Current: [keys/s=1370, latency=706.78 ms], verified=6454 2018-08-23 13:53:07,442 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=72112, cols=1.5 M, time=00:00:45 Overall: [keys/s= 1601, latency=618.87 ms] Current: [keys/s=1734, latency=605.84 ms], verified=7293 2018-08-23 13:53:12,443 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=82197, cols=1.7 M, time=00:00:50 Overall: [keys/s= 1642, latency=598.89 ms] Current: [keys/s=2017, latency=455.99 ms], verified=8272 2018-08-23 13:53:17,444 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=94334, cols=1.9 M, time=00:00:55 Overall: [keys/s= 1714, latency=580.54 ms] Current: [keys/s=2427, latency=456.27 ms], verified=9511 2018-08-23 13:53:22,445 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: RUN SUMMARY: Keys=94334, cols=2.1 M, time=00:00:55 Overall: [keys/s= 1714, latency=580.54 ms] {code} Edit: Just want to know what information should be contained in the summary :) was (Author: colingarcia): [~mdrob] I've been looking at this. Based on the output below, do you just want a final line that's something like the output below? Should I remove the Overall at each step and just put in overall at the end? Thanks in advance :) {code:java} 2018-08-23 13:52:52,435 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=45976, cols=972.4 K, time=00:00:30 Overall: [keys/s= 1531, latency=644.90 ms] Current: [keys/s=2017, latency=492.38 ms], verified=4704 2018-08-23 13:52:57,439 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=56588, cols=1.2 M, time=00:00:35 Overall: [keys/s= 1615, latency=610.22 ms] Current: [keys/s=2122, latency=459.96 ms], verified=5755 2018-08-23 13:53:02,440 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=63442, cols=1.3 M, time=00:00:40 Overall: [keys/s= 1584, latency=620.65 ms] Current: [keys/s=1370, latency=706.78 ms], verified=6454 2018-08-23 13:53:07,442 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=72112, cols=1.5 M, time=00:00:45 Overall: [keys/s= 1601, latency=618.87 ms] Current: [keys/s=1734, latency=605.84 ms], verified=7293 2018-08-23 13:53:12,443 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=82197, cols=1.7 M, time=00:00:50 Overall: [keys/s= 1642, latency=598.89 ms] Current: [keys/s=2017, latency=455.99 ms], verified=8272 2018-08-23 13:53:17,444 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: [R:1000] Keys=94334, cols=1.9 M, time=00:00:55 Overall: [keys/s= 1714, latency=580.54 ms] Current: [keys/s=2427, latency=456.27 ms], verified=9511 2018-08-23 13:53:22,445 INFO [MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction: RUN SUMMARY: Keys=94334, cols=2.1 M, time=00:00:55 Overall: [keys/s= 1714, latency=580.54 ms] {code} > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Wei-Chiu Chuang >Priority: Major > Labels: beginner > > ltt currently prints status as it goes, but doesn't give a nice summary of > what
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590807#comment-16590807 ] Colin Garcia commented on HBASE-20306: -- I'll upload the diff > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Wei-Chiu Chuang >Priority: Major > Labels: beginner > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590806#comment-16590806 ] Colin Garcia commented on HBASE-20306: -- No, I just added the logic locally > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Wei-Chiu Chuang >Priority: Major > Labels: beginner > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20306: - Attachment: HBASE-20306.000.patch > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Wei-Chiu Chuang >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia reassigned HBASE-20306: Assignee: Colin Garcia (was: Wei-Chiu Chuang) > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)