Wei Deng created CASSANDRA-12183:
------------------------------------
Summary: compaction_history system table does not capture all
historical compaction sessions
Key: CASSANDRA-12183
URL: https://issues.apache.org/jira/browse/CASSANDRA-12183
Project: Cassandra
Issue Type: Bug
Components: Compaction, Observability
Reporter: Wei Deng
It appears that some compaction sessions are not recorded in
system.compaction_history table after the compaction session successfully
finishes.
The following is an example (test by simply running +cassandra-stress write
n=100000000+):
{noformat}
automaton@wdengdse50google-98425b985-3:~$ nodetool compactionstats
pending tasks: 46
id compaction type keyspace
table completed total unit progress
fa8a4f30-4884-11e6-b916-1dbd340a212f Compaction keyspace1
standard1 4233184044 4774209875 bytes 88.67%
91e30d21-4887-11e6-b916-1dbd340a212f Compaction keyspace1
standard1 836983029 889773060 bytes 94.07%
Active compaction remaining time : 0h00m35s
automaton@wdengdse50google-98425b985-3:~$ nodetool compactionstats
pending tasks: 47
id compaction type keyspace
table completed total unit progress
fa8a4f30-4884-11e6-b916-1dbd340a212f Compaction keyspace1
standard1 4353251539 4774209875 bytes 91.18%
28359094-4888-11e6-b916-1dbd340a212f Compaction keyspace1
standard1 49732274 4071652280 bytes 1.22%
Active compaction remaining time : 0h04m24s
{noformat}
At this point you know the previous compaction session
91e30d21-4887-11e6-b916-1dbd340a212f finished and confirmation can be found
from debug.log
{noformat}
automaton@wdengdse50google-98425b985-3:~$ grep
91e30d21-4887-11e6-b916-1dbd340a212f /var/log/cassandra/debug.log
DEBUG [CompactionExecutor:4] 2016-07-12 23:22:58,674 CompactionTask.java:146 -
Compacting (91e30d21-4887-11e6-b916-1dbd340a212f)
[/var/lib/cassandra/data/keyspace1/standard1-9c02e9c1487c11e6b9161dbd340a212f/mb-290-big-Data.db:level=0,
/var/lib/cassandra/data/keyspace1/standard1-9c02e9c1487c11e6b9161dbd340a212f/mb-279-big-Data.db:level=0,
/var/lib/cassandra/data/keyspace1/standard1-9c02e9c1487c11e6b9161dbd340a212f/mb-281-big-Data.db:level=0,
/var/lib/cassandra/data/keyspace1/standard1-9c02e9c1487c11e6b9161dbd340a212f/mb-280-big-Data.db:level=0,
/var/lib/cassandra/data/keyspace1/standard1-9c02e9c1487c11e6b9161dbd340a212f/mb-284-big-Data.db:level=0,
/var/lib/cassandra/data/keyspace1/standard1-9c02e9c1487c11e6b9161dbd340a212f/mb-283-big-Data.db:level=0,
/var/lib/cassandra/data/keyspace1/standard1-9c02e9c1487c11e6b9161dbd340a212f/mb-287-big-Data.db:level=0,
/var/lib/cassandra/data/keyspace1/standard1-9c02e9c1487c11e6b9161dbd340a212f/mb-292-big-Data.db:level=0,
/var/lib/cassandra/data/keyspace1/standard1-9c02e9c1487c11e6b9161dbd340a212f/mb-286-big-Data.db:level=0,
/var/lib/cassandra/data/keyspace1/standard1-9c02e9c1487c11e6b9161dbd340a212f/mb-289-big-Data.db:level=0,
]
DEBUG [CompactionExecutor:4] 2016-07-12 23:26:56,054 CompactionTask.java:217 -
Compacted (91e30d21-4887-11e6-b916-1dbd340a212f) 10 sstables to
[/var/lib/cassandra/data/keyspace1/standard1-9c02e9c1487c11e6b9161dbd340a212f/mb-293-big,]
to level=0. 889,773,060 bytes to 890,473,350 (~100% of original) in 237,365ms
= 3.577703MB/s. 0 total partitions merged to 3,871,921. Partition merge
counts were {1:3871921, }
{noformat}
However, if you query system.compaction_history table or run "nodetool
compactionhistory | grep 91e30d21-4887-11e6-b916-1dbd340a212f" you will get
nothing:
{noformat}
automaton@wdengdse50google-98425b985-3:~$ cqlsh -u cassandra
Password:
Connected to dse50 at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.0.7.1158 | DSE 5.0.0 | CQL spec 3.4.0 | Native
protocol v4]
Use HELP for help.
cassandra@cqlsh> select * from system.compaction_history where
id=91e30d21-4887-11e6-b916-1dbd340a212f;
id | bytes_in | bytes_out | columnfamily_name | compacted_at | keyspace_name |
rows_merged
----+----------+-----------+-------------------+--------------+---------------+-------------
(0 rows)
automaton@wdengdse50google-98425b985-3:~$ nodetool flush system
automaton@wdengdse50google-98425b985-3:~$ nodetool compactionhistory | grep
91e30d21-4887-11e6-b916-1dbd340a212f
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
