[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16594241#comment-16594241 ] Colin Garcia commented on HBASE-20306: -- Added your suggestions! > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch, HBASE-20306.001.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20306: - Attachment: HBASE-20306.001.patch > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch, HBASE-20306.001.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[
https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16594244#comment-16594244
]
Colin Garcia commented on HBASE-20306:
--
{code:java}
2018-08-27 14:29:33,296 INFO
[MultiThreadedAction-ProgressReporter-1535405323258] util.MultiThreadedAction:
[W:20] Keys=87549, cols=985.4 K, time=00:00:50 Overall: [keys/s= 1749,
latency=11.34 ms] Current: [keys/s=1732, latency=11.47 ms], wroteUpTo=-1
2018-08-27 14:29:38,299 INFO
[MultiThreadedAction-ProgressReporter-1535405323258] util.MultiThreadedAction:
[W:20] Keys=96588, cols=1.1 M, time=00:00:55 Overall: [keys/s= 1754,
latency=11.31 ms] Current: [keys/s=1807, latency=10.98 ms], wroteUpTo=-1
2018-08-27 14:29:43,303 INFO
[MultiThreadedAction-ProgressReporter-1535405323258] util.MultiThreadedAction:
RUN SUMMARY: W Keys=96588, cols=1.1 M, time=00:00:55 Overall: [keys/s= 1754,
latency=11.31 ms] 2018-08-27 14:29:43,324 INFO
[ReadOnlyZKClient-localhost:2181@0x0b6b1987-EventThread] zookeeper.ClientCnxn:
EventThread shut down for session: 0x1657d2d21d30023
{code}
> LoadTestTool does not print summary at end of run
> -
>
> Key: HBASE-20306
> URL: https://issues.apache.org/jira/browse/HBASE-20306
> Project: HBase
> Issue Type: Bug
> Components: tooling
>Reporter: Mike Drob
>Assignee: Colin Garcia
>Priority: Major
> Labels: beginner
> Attachments: HBASE-20306.000.patch, HBASE-20306.001.patch
>
>
> ltt currently prints status as it goes, but doesn't give a nice summary of
> what happened so users have to infer it from the last status line printed.
> Would be nice to print a real summary with statistics about what was run.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20779) Server version is not right when enable TABLES_ON_MASTER
[ https://issues.apache.org/jira/browse/HBASE-20779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20779: - Description: When enble TABLES_ON_MASTER, master will be a region server to carry regions. So it will report to itself, too. And we get server version from rpc call. But master report to itself will skip rpc call. Then ServerManager will record a wrong version 0. (was: When eable TABLES_ON_MASTER, master will be a region server to carry regions. So it will report to itself, too. And we get server version from rpc call. But master report to itself will skip rpc call. Then ServerManager will record a wrong version 0.) > Server version is not right when enable TABLES_ON_MASTER > > > Key: HBASE-20779 > URL: https://issues.apache.org/jira/browse/HBASE-20779 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Priority: Major > > When enble TABLES_ON_MASTER, master will be a region server to carry regions. > So it will report to itself, too. And we get server version from rpc call. > But master report to itself will skip rpc call. Then ServerManager will > record a wrong version 0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20779) Server version is not right when enable TABLES_ON_MASTER
[ https://issues.apache.org/jira/browse/HBASE-20779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20779: - Description: When enable TABLES_ON_MASTER, master will be a region server to carry regions. So it will report to itself, too. And we get server version from rpc call. But master report to itself will skip rpc call. Then ServerManager will record a wrong version 0. (was: When enble TABLES_ON_MASTER, master will be a region server to carry regions. So it will report to itself, too. And we get server version from rpc call. But master report to itself will skip rpc call. Then ServerManager will record a wrong version 0.) > Server version is not right when enable TABLES_ON_MASTER > > > Key: HBASE-20779 > URL: https://issues.apache.org/jira/browse/HBASE-20779 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Priority: Major > > When enable TABLES_ON_MASTER, master will be a region server to carry > regions. So it will report to itself, too. And we get server version from rpc > call. But master report to itself will skip rpc call. Then ServerManager will > record a wrong version 0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20307) LoadTestTool prints too much zookeeper logging
[ https://issues.apache.org/jira/browse/HBASE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597962#comment-16597962 ] Colin Garcia commented on HBASE-20307: -- Bump. Just wondering if you guys had any thoughts about the proposed changes? > LoadTestTool prints too much zookeeper logging > -- > > Key: HBASE-20307 > URL: https://issues.apache.org/jira/browse/HBASE-20307 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20307.000.patch > > > When running ltt there is a ton of ZK related cruft that I probably don't > care about. Hide it behind -verbose flag or point people at log4j > configuration but don't print it by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20307) LoadTestTool prints too much zookeeper logging
[ https://issues.apache.org/jira/browse/HBASE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20307: - Attachment: HBASE-20307.001.patch > LoadTestTool prints too much zookeeper logging > -- > > Key: HBASE-20307 > URL: https://issues.apache.org/jira/browse/HBASE-20307 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20307.000.patch, HBASE-20307.001.patch > > > When running ltt there is a ton of ZK related cruft that I probably don't > care about. Hide it behind -verbose flag or point people at log4j > configuration but don't print it by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612569#comment-16612569 ] Colin Garcia commented on HBASE-20306: -- [~mdrob] I was wondering if you've had a chance to look this over? > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch, HBASE-20306.001.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20306: - Attachment: HBASE-20306.002.patch > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch, HBASE-20306.001.patch, > HBASE-20306.002.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[
https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16614096#comment-16614096
]
Colin Garcia commented on HBASE-20306:
--
Thanks for the comments Andrew. I've updated using the getHistogramReport
functionality. The only concern I have is that the values won't exactly match
the "Overall Status" due to some casting to long from doubles. Here is a
snippet of the output. Let me know what you think :) hoping this is a step in
the right direction
{code:java}
2018-09-13 14:30:52,732 INFO
[MultiThreadedAction-ProgressReporter-1536874207705] util.MultiThreadedAction:
[W:20] Keys=82495, cols=927.8 K, time=00:00:45 Overall: [keys/s= 1832,
latency=10.82 ms] Current: [keys/s=2058, latency=9.64 ms], wroteUpTo=-1
2018-09-13 14:30:57,733 INFO
[MultiThreadedAction-ProgressReporter-1536874207705] util.MultiThreadedAction:
[W:20] Keys=92516, cols=1.0 M, time=00:00:50 Overall: [keys/s= 1849,
latency=10.72 ms] Current: [keys/s=2004, latency=9.89 ms], wroteUpTo=-1
2018-09-13 14:31:02,738 INFO
[MultiThreadedAction-ProgressReporter-1536874207705] util.MultiThreadedAction:
RUN SUMMARY
KEYS PER SECOND:
mean=1849.90, min=1108.00, max=2065.00, stdDev=291.71, 50th=1945.50,
75th=2017.50, 95th=2065.00, 99th=2065.00, 99.9th=2065.00, 99.99th=2065.00,
99.999th=2065.00
LATENCY:
mean=10.60, min=9.00, max=17.00, stdDev=2.41, 50th=10.00, 75th=10.50,
95th=17.00, 99th=17.00, 99.9th=17.00, 99.99th=17.00, 99.999th=17.00
{code}
> LoadTestTool does not print summary at end of run
> -
>
> Key: HBASE-20306
> URL: https://issues.apache.org/jira/browse/HBASE-20306
> Project: HBase
> Issue Type: Bug
> Components: tooling
>Reporter: Mike Drob
>Assignee: Colin Garcia
>Priority: Major
> Labels: beginner
> Attachments: HBASE-20306.000.patch, HBASE-20306.001.patch,
> HBASE-20306.002.patch
>
>
> ltt currently prints status as it goes, but doesn't give a nice summary of
> what happened so users have to infer it from the last status line printed.
> Would be nice to print a real summary with statistics about what was run.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16636241#comment-16636241 ] Colin Garcia commented on HBASE-20306: -- [~apurtell] Thanks! Strangely, I ran with the same input and both threads finished (although it took some time). Could there be a race condition here? I don't see how one could run indefinitely as at some point the worker threads will finish, setting isProgressReporter to false. Can you paste what your output was? Fixing the other pieces you suggested now. > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306-branch-1.000.patch, HBASE-20306.000.patch, > HBASE-20306.001.patch, HBASE-20306.002.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20306: - Attachment: HBASE-20306.003.patch > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306-branch-1.000.patch, HBASE-20306.000.patch, > HBASE-20306.001.patch, HBASE-20306.002.patch, HBASE-20306.003.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16636264#comment-16636264 ] Colin Garcia commented on HBASE-20306: -- I see. But I don't think the old if/else is right then either. If the reader progress was printing the same status, that would have to mean that there were still worker threads running (as they were formerly the same check), so the old if/else wouldn't have terminated either. This problem might be elsewhere, unrelated to the change but having something to do with how these worker threads are exiting. > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306-branch-1.000.patch, HBASE-20306.000.patch, > HBASE-20306.001.patch, HBASE-20306.002.patch, HBASE-20306.003.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16636264#comment-16636264 ] Colin Garcia edited comment on HBASE-20306 at 10/3/18 12:16 AM: I see. But I don't think the old if/else is right then either. If the reader progress was printing the same status, that would have to mean that there were still worker threads running (as they were formerly the same check), so the old if/else wouldn't have terminated either. This problem might be elsewhere, unrelated to the change but having something to do with how these worker threads are exiting. I'm rerunning to see if I can mimic the behavior you describe. was (Author: colingarcia): I see. But I don't think the old if/else is right then either. If the reader progress was printing the same status, that would have to mean that there were still worker threads running (as they were formerly the same check), so the old if/else wouldn't have terminated either. This problem might be elsewhere, unrelated to the change but having something to do with how these worker threads are exiting. > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306-branch-1.000.patch, HBASE-20306.000.patch, > HBASE-20306.001.patch, HBASE-20306.002.patch, HBASE-20306.003.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16636274#comment-16636274 ] Colin Garcia commented on HBASE-20306: -- That's fine, I agree with this assessment. I'm just spitballing to determine what the cause could be. Will keep investigating. > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306-branch-1.000.patch, HBASE-20306.000.patch, > HBASE-20306.001.patch, HBASE-20306.002.patch, HBASE-20306.003.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[
https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16636299#comment-16636299
]
Colin Garcia commented on HBASE-20306:
--
My output from the latest run, for reference:
{code:java}
~/hbase/bin(master*) » ./hbase ltt -read 100:10 -write 10:100:10 -num_keys
100
{code}
{code:java}
2018-10-02 17:42:15,369 INFO
[MultiThreadedAction-ProgressReporter-1538525614665] util.MultiThreadedAction:
[W:11] Keys=993612, cols=10.9 M, time=00:28:40 Overall: [keys/s= 577,
latency=17.27 ms] Current: [keys/s=566, latency=17.62 ms], wroteUpTo=993597,
wroteQSize=4
2018-10-02 17:42:15,423 INFO
[MultiThreadedAction-ProgressReporter-1538525614699] util.MultiThreadedAction:
[R:10] Keys=9144453, cols=155.6 M, time=00:28:40 Overall: [keys/s= 5314,
latency=0.90 ms] Current: [keys/s=5187, latency=0.95 ms], verified=9144453
2018-10-02 17:42:20,369 INFO
[MultiThreadedAction-ProgressReporter-1538525614665] util.MultiThreadedAction:
[W:11] Keys=996682, cols=10.9 M, time=00:28:45 Overall: [keys/s= 577,
latency=17.26 ms] Current: [keys/s=614, latency=16.24 ms], wroteUpTo=996669,
wroteQSize=2
2018-10-02 17:42:20,425 INFO
[MultiThreadedAction-ProgressReporter-1538525614699] util.MultiThreadedAction:
[R:10] Keys=9171711, cols=156.1 M, time=00:28:45 Overall: [keys/s= 5314,
latency=0.90 ms] Current: [keys/s=5451, latency=0.84 ms], verified=9171711
2018-10-02 17:42:25,369 INFO
[MultiThreadedAction-ProgressReporter-1538525614665] util.MultiThreadedAction:
[W:11] Keys=999798, cols=11.0 M, time=00:28:50 Overall: [keys/s= 577,
latency=17.26 ms] Current: [keys/s=623, latency=16.01 ms], wroteUpTo=999780,
wroteQSize=7
2018-10-02 17:42:25,429 INFO
[MultiThreadedAction-ProgressReporter-1538525614699] util.MultiThreadedAction:
[R:10] Keys=9199696, cols=156.5 M, time=00:28:50 Overall: [keys/s= 5315,
latency=0.90 ms] Current: [keys/s=5597, latency=0.81 ms], verified=9199699
2018-10-02 17:42:30,406 INFO
[MultiThreadedAction-ProgressReporter-1538525614665] util.MultiThreadedAction:
RUN SUMMARY
KEYS PER SECOND [W]: mean=577.53, min=322.00, max=729.00, stdDev=65.36,
50th=586.00, 75th=622.00, 95th=663.00, 99th=711.30, 99.9th=729.00,
99.99th=729.00, 99.999th=729.00
LATENCY [W]: mean=17.05, min=13.00, max=30.00, stdDev=2.42, 50th=16.00,
75th=18.00, 95th=22.00, 99th=27.00, 99.9th=30.00, 99.99th=30.00, 99.999th=30.00
2018-10-02 17:42:30,434 INFO
[MultiThreadedAction-ProgressReporter-1538525614699] util.MultiThreadedAction:
RUN SUMMARY
KEYS PER SECOND [R]: mean=5317.31, min=2953.00, max=6900.00, stdDev=639.58,
50th=5366.00, 75th=5727.00, 95th=6248.65, 99th=6598.89, 99.9th=6900.00,
99.99th=6900.00, 99.999th=6900.00
LATENCY [R]: mean=0.28, min=0.00, max=2.00, stdDev=0.46, 50th=0.00, 75th=1.00,
95th=1.00, 99th=1.00, 99.9th=2.00, 99.99th=2.00, 99.999th=2.00
2018-10-02 17:42:30,783 INFO [main] zookeeper.ReadOnlyZKClient: Close
zookeeper connection 0x31aa3ca5 to localhost:2181
Failed to write keys: 0
2018-10-02 17:42:30,794 INFO [main] zookeeper.ReadOnlyZKClient: Close
zookeeper connection 0x6b44435b to localhost:2181{code}
> LoadTestTool does not print summary at end of run
> -
>
> Key: HBASE-20306
> URL: https://issues.apache.org/jira/browse/HBASE-20306
> Project: HBase
> Issue Type: Bug
> Components: tooling
>Reporter: Mike Drob
>Assignee: Colin Garcia
>Priority: Major
> Labels: beginner
> Attachments: HBASE-20306-branch-1.000.patch, HBASE-20306.000.patch,
> HBASE-20306.001.patch, HBASE-20306.002.patch, HBASE-20306.003.patch
>
>
> ltt currently prints status as it goes, but doesn't give a nice summary of
> what happened so users have to infer it from the last status line printed.
> Would be nice to print a real summary with statistics about what was run.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642281#comment-16642281 ] Colin Garcia commented on HBASE-20306: -- Hey Andrew. I was wondering what test env you were using to run the ltt command? I've been running it locally, and haven't been able to replicate. Thanks in advance > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306-branch-1.000.patch, HBASE-20306.000.patch, > HBASE-20306.001.patch, HBASE-20306.002.patch, HBASE-20306.003.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662655#comment-16662655 ] Colin Garcia commented on HBASE-20306: -- Andrew, is it possible for me to use your test cluster? I've ran locally several times and have been unable to replicate the behavior you've described. > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306-branch-1.000.patch, HBASE-20306.000.patch, > HBASE-20306.001.patch, HBASE-20306.002.patch, HBASE-20306.003.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20307) LoadTestTool prints too much zookeeper logging
[ https://issues.apache.org/jira/browse/HBASE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586394#comment-16586394 ] Colin Garcia commented on HBASE-20307: -- Cc: [~liuml07] > LoadTestTool prints too much zookeeper logging > -- > > Key: HBASE-20307 > URL: https://issues.apache.org/jira/browse/HBASE-20307 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Priority: Major > Labels: beginner > > When running ltt there is a ton of ZK related cruft that I probably don't > care about. Hide it behind -verbose flag or point people at log4j > configuration but don't print it by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20307) LoadTestTool prints too much zookeeper logging
[
https://issues.apache.org/jira/browse/HBASE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586392#comment-16586392
]
Colin Garcia commented on HBASE-20307:
--
Hi [~mdrob], new to HBase and was looking to take this on. I've ran the
following command:
{code:java}
~/hbase/bin(master) » ./hbase ltt -tn TestTable -read 10:1000 -num_keys
1000
{code}
And have posted the output below. I was wondering if we're looking to hide the
logs of the form:
[ReadOnlyZKClient-localhost*]? Will ReadOnlyZKClient be called for all the
various inputs?
Output for reference:
{code:java}
~/hbase/bin(master) » ./hbase ltt -tn TestTable -read 10:1000 -num_keys
1000
Key range: [0..999] Multi-gets (value of 1 means no multigets): 1 Percent
of keys to verify: 10 Reader threads: 1000 2018-08-17 14:51:14,391 WARN [main]
util.NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable 2018-08-17 14:51:15,600
INFO [main] zookeeper.ReadOnlyZKClient: Connect 0x7193666c to localhost:2181
with session timeout=9ms, retries 30, retry interval 1000ms,
keepAlive=6ms 2018-08-17 14:51:15,610 INFO
[ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f,
built on 03/23/2017 10:13 GMT 2018-08-17 14:51:15,610 INFO
[ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client
environment:host.name=colingarc-ltm8.internal.salesforce.com 2018-08-17
14:51:15,610 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c]
zookeeper.ZooKeeper: Client environment:java.version=1.8.0_181 2018-08-17
14:51:15,610 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c]
zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
2018-08-17 14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c]
zookeeper.ZooKeeper: Client
environment:java.home=/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre
2018-08-17 14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c]
zookeeper.ZooKeeper:
cia/.m2/repository/javax/validation/validation-api/1.1.0.Final/validation-api-1.1.0.Final.jar:/Users/colin.garcia/hbase/hbase-examples/target/hbase-examples-3.0.0-SNAPSHOT.jar:/Users/colin.garcia/hbase/hbase-endpoint/target/hbase-endpoint-3.0.0-SNAPSHOT.jar:/Users/colin.garcia/.m2/repository/org/apache/curator/curator-framework/4.0.0/curator-framework-4.0.0.jar:/Users/colin.garcia/.m2/repository/org/apache/curator/curator-client/4.0.0/curator-client-4.0.0.jar:/Users/colin.garcia/.m2/repository/org/apache/curator/curator-recipes/4.0.0/curator-recipes-4.0.0.jar:/Users/colin.garcia/hbase/hbase-zookeeper/target/hbase-zookeeper-3.0.0-SNAPSHOT.jar:/Users/colin.garcia/hbase/hbase-rsgroup/target/hbase-rsgroup-3.0.0-SNAPSHOT.jar:/Users/colin.garcia/hbase/hbase-annotations/target/hbase-annotations-3.0.0-SNAPSHOT-tests.jar:/Users/colin.garcia/.m2/repository/org/apache/yetus/audience-annotations/0.5.0/audience-annotations-0.5.0.jar:/Users/colin.garcia/.m2/repository/junit/junit/4.12/junit-4.12.jar
2018-08-17 14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c]
zookeeper.ZooKeeper: Client
environment:java.library.path=/Users/colin.garcia/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
2018-08-17 14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c]
zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/var/folders/ct/l2m2v7h95nb6n_cvp0jf73p1l_c_hb/T/
2018-08-17 14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c]
zookeeper.ZooKeeper: Client environment:java.compiler= 2018-08-17
14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c]
zookeeper.ZooKeeper: Client environment:os.name=Mac OS X 2018-08-17
14:51:15,611 INFO [ReadOnlyZKClient-localhost:2181@0x7193666c]
zookeeper.ZooKeeper: Client environment:os.arch=x86_64 2018-08-17 14:51:15,611
INFO [ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client
environment:os.version=10.13.6 2018-08-17 14:51:15,611 INFO
[ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client
environment:user.name=colin.garcia 2018-08-17 14:51:15,611 INFO
[ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client
environment:user.home=/Users/colin.garcia 2018-08-17 14:51:15,611 INFO
[ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Client
environment:user.dir=/Users/colin.garcia/hbase/bin 2018-08-17 14:51:15,612 INFO
[ReadOnlyZKClient-localhost:2181@0x7193666c] zookeeper.ZooKeeper: Initiating
client connection, connectString
[jira] [Commented] (HBASE-20307) LoadTestTool prints too much zookeeper logging
[
https://issues.apache.org/jira/browse/HBASE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586656#comment-16586656
]
Colin Garcia commented on HBASE-20307:
--
[~mdrob] [~stack], thanks for the responses! Makes sense.
Discussing this with others, we have a workaround proposal and want to see what
you think. I'll post a diff with the changes and a copy of the logs below. We
set the ZK logging level to WARN in the LoadTestTool class, which means right
now it only works for LoadTestTool (currently experimenting to see if it can be
added to ReadOnlyZKClient so that we don't have to add this code to each tool).
Here is an updated look at the logs:
{code:java}
~/hbase/bin(master*) » ./hbase ltt -tn TestTable -read 10:1000 -num_keys
1000
Key range: [0..999]
Multi-gets (value of 1 means no multigets): 1
Percent of keys to verify: 10
Reader threads: 1000
2018-08-20 15:59:08,307 WARN [main] util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
2018-08-20 15:59:08,526 INFO [main] zookeeper.ReadOnlyZKClient: Connect
0x03835c46 to localhost:2181 with session timeout=9ms, retries 30, retry
interval 1000ms, keepAlive=6ms
2018-08-20 15:59:08,574 INFO
[ReadOnlyZKClient-localhost:2181@0x03835c46-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown
error)
2018-08-20 15:59:08,606 INFO
[ReadOnlyZKClient-localhost:2181@0x03835c46-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Socket connection established to
localhost/127.0.0.1:2181, initiating session
2018-08-20 15:59:08,614 INFO
[ReadOnlyZKClient-localhost:2181@0x03835c46-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Session establishment complete on server
localhost/127.0.0.1:2181, sessionid = 0x165599020930005, negotiated timeout =
4
2018-08-20 15:59:13,376 INFO [main] client.RpcRetryingCallerImpl: Call
exception, tries=6, retries=16, started=4634 ms ago, cancelled=false,
msg=org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
2018-08-20 15:59:17,440 INFO [main] hbase.HBaseCommonTestingUtility: Number of
live regionservers: 1, pre-splitting table into 5 regions (regions per server:
5)
2018-08-20 15:59:17,579 WARN [main] hbase.HBaseCommonTestingUtility: Table
TestTable already exists, continuing
2018-08-20 15:59:17,580 INFO [main] client.ConnectionImplementation: Closing
master protocol: MasterService
2018-08-20 15:59:17,580 INFO [main] zookeeper.ReadOnlyZKClient: Close
zookeeper connection 0x03835c46 to localhost:2181
2018-08-20 15:59:17,583 INFO
[ReadOnlyZKClient-localhost:2181@0x03835c46-EventThread] zookeeper.ClientCnxn:
EventThread shut down for session: 0x165599020930005
2018-08-20 15:59:17,597 INFO [main] zookeeper.ReadOnlyZKClient: Connect
0x68fa0ba8 to localhost:2181 with session timeout=9ms, retries 30, retry
interval 1000ms, keepAlive=6ms
2018-08-20 15:59:17,598 INFO
[ReadOnlyZKClient-localhost:2181@0x68fa0ba8-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown
error)
2018-08-20 15:59:17,599 INFO
[ReadOnlyZKClient-localhost:2181@0x68fa0ba8-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Socket connection established to
localhost/127.0.0.1:2181, initiating session
2018-08-20 15:59:17,600 INFO
[ReadOnlyZKClient-localhost:2181@0x68fa0ba8-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Session establishment complete on server
localhost/127.0.0.1:2181, sessionid = 0x165599020930006, negotiated timeout =
4
2018-08-20 15:59:17,621 INFO [main] util.LoadTestTool: Disabling table
TestTable
2018-08-20 15:59:17,622 INFO [main] client.HBaseAdmin: Started disable of
TestTable
2018-08-20 15:59:18,068 INFO [main] client.HBaseAdmin: Operation: DISABLE,
Table Name: default:TestTable, procId: 54 completed
2018-08-20 15:59:18,219 INFO [main] util.LoadTestTool: Enabling table TestTable
2018-08-20 15:59:18,220 INFO [main] client.HBaseAdmin: Started enable of
TestTable
2018-08-20 15:59:18,963 INFO [main] client.HBaseAdmin: Operation: ENABLE,
Table Name: default:TestTable, procId: 61 completed
2018-08-20 15:59:18,963 INFO [main] client.ConnectionImplementation: Closing
master protocol: MasterService
2018-08-20 15:59:18,963 INFO [main] zookeeper.ReadOnlyZKClient: Close
zookeeper connection 0x68fa0ba8 to localhost:2181
2018-08-20 15:59:18,966 INFO
[ReadOnlyZKClient-localhost:2181@0x68fa0ba8-EventThread] zookeeper.ClientCnxn:
EventThread shut down for session: 0x165599020930006
2018-08-20 15:59:18,967 INFO [main] zookeeper.ReadOnlyZKClient: Connect
0x66b7550d to localhost:2181 with session timeout=9ms, retries 30, retry
interval 1000ms,
[jira] [Commented] (HBASE-20307) LoadTestTool prints too much zookeeper logging
[
https://issues.apache.org/jira/browse/HBASE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586670#comment-16586670
]
Colin Garcia commented on HBASE-20307:
--
Can I be added as committer to post the patch? Here is the diff (it's a small
change):
{code:java}
~/hbase/bin(master*) » cat HBASE-20307.000.patch
colin.garcia@colingarc-ltm8 diff --git
a/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java
b/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java
index 41788588ff..0c0023b593 100644 ---
a/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java
+++
b/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java
@@ -36,7 +36,10 @@ import org.apache.hadoop.hbase.HConstants; import
org.apache.hadoop.hbase.TableName; import
org.apache.hadoop.hbase.client.ColumnFamilyDescriptor; import
org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder; +import
org.apache.log4j.Level; +import org.apache.log4j.LogManager; import
org.apache.yetus.audience.InterfaceAudience; +import
org.apache.zookeeper.ZooKeeper; import org.slf4j.Logger; import
org.slf4j.LoggerFactory; import org.apache.hadoop.hbase.client.Admin; @@ -565,6
+568,7 @@ public class LoadTestTool extends AbstractHBaseTool { @Override
protected int doWork() throws IOException { +
LogManager.getLogger(ZooKeeper.class.getName()).setLevel(Level.WARN); if
(numTables > 1) { return parallelLoadTables(); } else {
{code}
Note that we can add another line if we'd like to remove the logs about
ClientCnxn, but that approach doesn't seem optimal. Is there a root ZK Logger
we can set to WARN to remove any logs of Zookeeper.*?
> LoadTestTool prints too much zookeeper logging
> --
>
> Key: HBASE-20307
> URL: https://issues.apache.org/jira/browse/HBASE-20307
> Project: HBase
> Issue Type: Bug
> Components: tooling
>Reporter: Mike Drob
>Priority: Major
> Labels: beginner
>
> When running ltt there is a ton of ZK related cruft that I probably don't
> care about. Hide it behind -verbose flag or point people at log4j
> configuration but don't print it by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20307) LoadTestTool prints too much zookeeper logging
[ https://issues.apache.org/jira/browse/HBASE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20307: - Attachment: HBASE-20307.000.patch > LoadTestTool prints too much zookeeper logging > -- > > Key: HBASE-20307 > URL: https://issues.apache.org/jira/browse/HBASE-20307 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20307.000.patch > > > When running ltt there is a ton of ZK related cruft that I probably don't > care about. Hide it behind -verbose flag or point people at log4j > configuration but don't print it by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587823#comment-16587823 ] Colin Garcia commented on HBASE-20306: -- [~jojochuang] I was wondering what progress you've made? Looking to take a look into this as well :) > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Wei-Chiu Chuang >Priority: Major > Labels: beginner > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[
https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590796#comment-16590796
]
Colin Garcia commented on HBASE-20306:
--
[~mdrob] I've been looking at this. Based on the output below, do you just want
a final line that's something like the output below? Should I remove the
Overall at each step and just put in overall at the end? Thanks in advance :)
{code:java}
2018-08-23 13:52:52,435 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=45976, cols=972.4 K, time=00:00:30 Overall: [keys/s= 1531,
latency=644.90 ms] Current: [keys/s=2017, latency=492.38 ms], verified=4704
2018-08-23 13:52:57,439 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=56588, cols=1.2 M, time=00:00:35 Overall: [keys/s= 1615,
latency=610.22 ms] Current: [keys/s=2122, latency=459.96 ms], verified=5755
2018-08-23 13:53:02,440 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=63442, cols=1.3 M, time=00:00:40 Overall: [keys/s= 1584,
latency=620.65 ms] Current: [keys/s=1370, latency=706.78 ms], verified=6454
2018-08-23 13:53:07,442 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=72112, cols=1.5 M, time=00:00:45 Overall: [keys/s= 1601,
latency=618.87 ms] Current: [keys/s=1734, latency=605.84 ms], verified=7293
2018-08-23 13:53:12,443 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=82197, cols=1.7 M, time=00:00:50 Overall: [keys/s= 1642,
latency=598.89 ms] Current: [keys/s=2017, latency=455.99 ms], verified=8272
2018-08-23 13:53:17,444 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=94334, cols=1.9 M, time=00:00:55 Overall: [keys/s= 1714,
latency=580.54 ms] Current: [keys/s=2427, latency=456.27 ms], verified=9511
2018-08-23 13:53:22,445 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
RUN SUMMARY: Keys=94334, cols=2.1 M, time=00:00:55 Overall: [keys/s= 1714,
latency=580.54 ms]
{code}
> LoadTestTool does not print summary at end of run
> -
>
> Key: HBASE-20306
> URL: https://issues.apache.org/jira/browse/HBASE-20306
> Project: HBase
> Issue Type: Bug
> Components: tooling
>Reporter: Mike Drob
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Labels: beginner
>
> ltt currently prints status as it goes, but doesn't give a nice summary of
> what happened so users have to infer it from the last status line printed.
> Would be nice to print a real summary with statistics about what was run.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20306) LoadTestTool does not print summary at end of run
[
https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590796#comment-16590796
]
Colin Garcia edited comment on HBASE-20306 at 8/23/18 8:56 PM:
---
[~mdrob] I've been looking at this. Based on the output below, do you just want
a final line that's something like the output below? Should I remove the
Overall at each step and just put in overall at the end? Thanks in advance :)
{code:java}
2018-08-23 13:52:52,435 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=45976, cols=972.4 K, time=00:00:30 Overall: [keys/s= 1531,
latency=644.90 ms] Current: [keys/s=2017, latency=492.38 ms], verified=4704
2018-08-23 13:52:57,439 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=56588, cols=1.2 M, time=00:00:35 Overall: [keys/s= 1615,
latency=610.22 ms] Current: [keys/s=2122, latency=459.96 ms], verified=5755
2018-08-23 13:53:02,440 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=63442, cols=1.3 M, time=00:00:40 Overall: [keys/s= 1584,
latency=620.65 ms] Current: [keys/s=1370, latency=706.78 ms], verified=6454
2018-08-23 13:53:07,442 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=72112, cols=1.5 M, time=00:00:45 Overall: [keys/s= 1601,
latency=618.87 ms] Current: [keys/s=1734, latency=605.84 ms], verified=7293
2018-08-23 13:53:12,443 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=82197, cols=1.7 M, time=00:00:50 Overall: [keys/s= 1642,
latency=598.89 ms] Current: [keys/s=2017, latency=455.99 ms], verified=8272
2018-08-23 13:53:17,444 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=94334, cols=1.9 M, time=00:00:55 Overall: [keys/s= 1714,
latency=580.54 ms] Current: [keys/s=2427, latency=456.27 ms], verified=9511
2018-08-23 13:53:22,445 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
RUN SUMMARY: Keys=94334, cols=2.1 M, time=00:00:55 Overall: [keys/s= 1714,
latency=580.54 ms]
{code}
Edit: Just want to know what information should be contained in the summary :)
was (Author: colingarcia):
[~mdrob] I've been looking at this. Based on the output below, do you just want
a final line that's something like the output below? Should I remove the
Overall at each step and just put in overall at the end? Thanks in advance :)
{code:java}
2018-08-23 13:52:52,435 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=45976, cols=972.4 K, time=00:00:30 Overall: [keys/s= 1531,
latency=644.90 ms] Current: [keys/s=2017, latency=492.38 ms], verified=4704
2018-08-23 13:52:57,439 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=56588, cols=1.2 M, time=00:00:35 Overall: [keys/s= 1615,
latency=610.22 ms] Current: [keys/s=2122, latency=459.96 ms], verified=5755
2018-08-23 13:53:02,440 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=63442, cols=1.3 M, time=00:00:40 Overall: [keys/s= 1584,
latency=620.65 ms] Current: [keys/s=1370, latency=706.78 ms], verified=6454
2018-08-23 13:53:07,442 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=72112, cols=1.5 M, time=00:00:45 Overall: [keys/s= 1601,
latency=618.87 ms] Current: [keys/s=1734, latency=605.84 ms], verified=7293
2018-08-23 13:53:12,443 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=82197, cols=1.7 M, time=00:00:50 Overall: [keys/s= 1642,
latency=598.89 ms] Current: [keys/s=2017, latency=455.99 ms], verified=8272
2018-08-23 13:53:17,444 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
[R:1000] Keys=94334, cols=1.9 M, time=00:00:55 Overall: [keys/s= 1714,
latency=580.54 ms] Current: [keys/s=2427, latency=456.27 ms], verified=9511
2018-08-23 13:53:22,445 INFO
[MultiThreadedAction-ProgressReporter-1535057542406] util.MultiThreadedAction:
RUN SUMMARY: Keys=94334, cols=2.1 M, time=00:00:55 Overall: [keys/s= 1714,
latency=580.54 ms]
{code}
> LoadTestTool does not print summary at end of run
> -
>
> Key: HBASE-20306
> URL: https://issues.apache.org/jira/browse/HBASE-20306
> Project: HBase
> Issue Type: Bug
> Components: tooling
>Reporter: Mike Drob
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Labels: beginner
>
> ltt currently prints status as it goes, but doesn't give a nice summary of
> what
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590807#comment-16590807 ] Colin Garcia commented on HBASE-20306: -- I'll upload the diff > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Wei-Chiu Chuang >Priority: Major > Labels: beginner > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590806#comment-16590806 ] Colin Garcia commented on HBASE-20306: -- No, I just added the logic locally > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Wei-Chiu Chuang >Priority: Major > Labels: beginner > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia updated HBASE-20306: - Attachment: HBASE-20306.000.patch > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Wei-Chiu Chuang >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20306) LoadTestTool does not print summary at end of run
[ https://issues.apache.org/jira/browse/HBASE-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Garcia reassigned HBASE-20306: Assignee: Colin Garcia (was: Wei-Chiu Chuang) > LoadTestTool does not print summary at end of run > - > > Key: HBASE-20306 > URL: https://issues.apache.org/jira/browse/HBASE-20306 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20306.000.patch > > > ltt currently prints status as it goes, but doesn't give a nice summary of > what happened so users have to infer it from the last status line printed. > Would be nice to print a real summary with statistics about what was run. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
