Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/9172 )
Change subject: maintenance_manager: log the reason for scheduling each operation ...................................................................... maintenance_manager: log the reason for scheduling each operation This makes it easier to troubleshoot when the maintenance manager appears to be scheduling the "wrong" operation. Example output from a long run of full_stack_insert_scan-test: ...: Scheduling FlushMRSOp(93ded54b4dfb4e1586ff7fe700184f53): under memory pressure (60.30% used, can flush 641875735 bytes) ...: Scheduling LogGCOp(93ded54b4dfb4e1586ff7fe700184f53): free 386079935 bytes of WAL ...: Scheduling UndoDeltaBlockGCOp(93ded54b4dfb4e1586ff7fe700184f53): 29857788 bytes on disk ...: Scheduling FlushMRSOp(93ded54b4dfb4e1586ff7fe700184f53): under memory pressure (60.01% used, can flush 637714199 bytes) ...: Scheduling LogGCOp(93ded54b4dfb4e1586ff7fe700184f53): free 394543122 bytes of WAL ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.281697 ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.280992 ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.280256 ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.060532 ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.060298 ...: Scheduling UndoDeltaBlockGCOp(93ded54b4dfb4e1586ff7fe700184f53): 56855045 bytes on disk ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.054961 ...: Scheduling UndoDeltaBlockGCOp(93ded54b4dfb4e1586ff7fe700184f53): 7202893 bytes on disk ...: Scheduling FlushMRSOp(93ded54b4dfb4e1586ff7fe700184f53): under memory pressure (60.25% used, can flush 633552663 bytes) ...: Scheduling LogGCOp(93ded54b4dfb4e1586ff7fe700184f53): free 394836440 bytes of WAL ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.192819 ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.184820 ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.184674 ...: Scheduling UndoDeltaBlockGCOp(93ded54b4dfb4e1586ff7fe700184f53): 70881575 bytes on disk ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.127476 ...: Scheduling UndoDeltaBlockGCOp(93ded54b4dfb4e1586ff7fe700184f53): 18119656 bytes on disk ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.127334 ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.111677 ...: Scheduling UndoDeltaBlockGCOp(93ded54b4dfb4e1586ff7fe700184f53): 31714242 bytes on disk ...: Scheduling FlushMRSOp(93ded54b4dfb4e1586ff7fe700184f53): under memory pressure (60.06% used, can flush 624189207 bytes) ...: Scheduling LogGCOp(93ded54b4dfb4e1586ff7fe700184f53): free 377818508 bytes of WAL ...: Scheduling UndoDeltaBlockGCOp(93ded54b4dfb4e1586ff7fe700184f53): 29069301 bytes on disk ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.144540 ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.138843 ...: Scheduling UndoDeltaBlockGCOp(93ded54b4dfb4e1586ff7fe700184f53): 36867458 bytes on disk ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.138827 ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.122173 ...: Scheduling UndoDeltaBlockGCOp(93ded54b4dfb4e1586ff7fe700184f53): 31717417 bytes on disk ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.121637 ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.121104 ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.088980 ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.087296 ...: Scheduling UndoDeltaBlockGCOp(93ded54b4dfb4e1586ff7fe700184f53): 54371475 bytes on disk ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.055906 ...: Scheduling UndoDeltaBlockGCOp(93ded54b4dfb4e1586ff7fe700184f53): 9063001 bytes on disk ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.031908 ...: Scheduling UndoDeltaBlockGCOp(93ded54b4dfb4e1586ff7fe700184f53): 6840843 bytes on disk ...: Scheduling FlushMRSOp(93ded54b4dfb4e1586ff7fe700184f53): under memory pressure (60.08% used, can flush 614825751 bytes) ...: Scheduling LogGCOp(93ded54b4dfb4e1586ff7fe700184f53): free 377913596 bytes of WAL ...: Scheduling UndoDeltaBlockGCOp(93ded54b4dfb4e1586ff7fe700184f53): 28612520 bytes on disk ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.113092 ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.110844 ...: Scheduling UndoDeltaBlockGCOp(93ded54b4dfb4e1586ff7fe700184f53): 36863166 bytes on disk ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.109090 ...: Scheduling UndoDeltaBlockGCOp(93ded54b4dfb4e1586ff7fe700184f53): 18129573 bytes on disk ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.108572 ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.108551 ...: Scheduling UndoDeltaBlockGCOp(93ded54b4dfb4e1586ff7fe700184f53): 36251849 bytes on disk ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.105718 ...: Scheduling CompactRowSetsOp(93ded54b4dfb4e1586ff7fe700184f53): perf score=0.065384 Change-Id: I4dcdb863a7a0b0fc2a72757801d5c057fa725c34 Reviewed-on: http://gerrit.cloudera.org:8080/9172 Tested-by: Kudu Jenkins Reviewed-by: Will Berkeley <wdberke...@gmail.com> --- M src/kudu/util/debug/trace_logging.h M src/kudu/util/maintenance_manager-test.cc M src/kudu/util/maintenance_manager.cc M src/kudu/util/maintenance_manager.h 4 files changed, 76 insertions(+), 51 deletions(-) Approvals: Kudu Jenkins: Verified Will Berkeley: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/9172 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I4dcdb863a7a0b0fc2a72757801d5c057fa725c34 Gerrit-Change-Number: 9172 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon <t...@apache.org> Gerrit-Reviewer: Jean-Daniel Cryans <jdcry...@apache.org> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon <t...@apache.org> Gerrit-Reviewer: Will Berkeley <wdberke...@gmail.com>
