Incorrect override of manifest.json?
Hi all,
In Cassandra 2.1.3 the code below is located in the class
CassandraFamilyStore.java which is called when you do a snapshot. What I can se
happens is that it first creates a snapshot manifest file correctly for a
column family but than the loop returns a index which overrides the data that
was written in the first iteration. Isn´t the code incorrect and the JSONArray
should be located before the outer for-loop or some other solution so it does´t
override content that has once been written?
From the call concatWithIndexes() I see it returns [CFS(Keyspace=’X',
ColumnFamily='tempState'), CFS(Keyspace=’X',
ColumnFamily='tempState.tempState_current_idx’)] and the first iteration writes
correct data to the manifest.json and the second iteration overrides the data
with a empty filesJSONArr”.
Can somebody please confirm if this is a bug, or if I should write one?
//Kind Regards, Daniel
public void snapshotWithoutFlush(String snapshotName, PredicateSSTableReader
predicate)
{
for (ColumnFamilyStore cfs : concatWithIndexes())
{
final JSONArray filesJSONArr = new JSONArray();
try (RefViewFragment currentView =
cfs.selectAndReference(ALL_SSTABLES))
{
for (SSTableReader ssTable : currentView.sstables)
{
if (ssTable.openReason == SSTableReader.OpenReason.EARLY ||
(predicate != null !predicate.apply(ssTable)))
{
continue;
}
File snapshotDirectory =
Directories.getSnapshotDirectory(ssTable.descriptor, snapshotName);
ssTable.createLinks(snapshotDirectory.getPath()); // hard
links
filesJSONArr.add(ssTable.descriptor.relativeFilenameFor(Component.DATA));
if (logger.isDebugEnabled())
logger.debug(Snapshot for {} keyspace data file {}
created in {}, keyspace, ssTable.getFilename(), snapshotDirectory);
}
writeSnapshotManifest(filesJSONArr, snapshotName);
}
}
}
Re: How to stop nodetool repair in 2.1.2?
Will JMX be more readily available by 2100? On 04/15/2015 10:14 PM, Benyi Wang wrote: Using JMX worked. Thanks a lot. On Wed, Apr 15, 2015 at 3:57 PM, Robert Coli rc...@eventbrite.com mailto:rc...@eventbrite.com wrote: On Wed, Apr 15, 2015 at 3:30 PM, Benyi Wang bewang.t...@gmail.com mailto:bewang.t...@gmail.com wrote: It didn't work. I ran the command on all nodes, but I still can see the repair activities. Your input as an operator who wants a nodetool command to trivially stop repairs is welcome here : https://issues.apache.org/jira/browse/CASSANDRA-3486 For now, your two options are : 1) restart all nodes participating in the repair 2) access the JMX endpoint forceTerminateAllRepairSessions on all nodes participating in the repair =Rob
Re: Incorrect override of manifest.json?
On Thu, Apr 16, 2015 at 6:29 AM, Daniel Pyykkö daniel.pyy...@sitevision.se wrote: In Cassandra 2.1.3 the code below is located in the class CassandraFamilyStore.java which is called when you do a snapshot. What I can se happens is that it first creates a snapshot manifest file correctly for a column family but than the loop returns a index which overrides the data that was written in the first iteration. Isn´t the code incorrect and the JSONArray should be located before the outer for-loop or some other solution so it does´t override content that has once been written? In general if you can make a plausible case that a bug might exist, especially if you can point it out in the code, you should inquire in one of three forums, in (in my non-official understanding) descending order of preference : 1) issues.apache.org 2) cassandra-dev@ mailing list 3) #cassandra-dev IRC channel 4) this list =Rob
Re: Drawbacks of Major Compaction now that Automatic Tombstone Compaction Exists
Hi Robert, Any comments or suggestions ? Thanks Anuj Wadehra Sent from Yahoo Mail on Android From:Anuj Wadehra anujw_2...@yahoo.co.in Date:Wed, 15 Apr, 2015 at 8:59 am Subject:Re: Drawbacks of Major Compaction now that Automatic Tombstone Compaction Exists Hi Robert, By automatic tombstone compaction, I am referring to tombstone_threshold sub property under compaction strategy in CQL. It is 0.2 by default. So what I understand from the Datastax documentation is that even if a sstable does not find sstables of similar size (STCS) , an automatic tombstone compaction will trigger on sstable when 20% data is tombstone. This compaction works on single sstable only. How Major compaction is related to automatic tombstone compaction? Earlier we used to say that Major compaction is not recommended because one huge sstable formed after Major compaction will not find any similar size sstables unless huge data is written in new sstables and thus tombstones will be there in the huge sstable unnnecessarily for long time. My understanding is that automatic tombstone compaction will allow tombstone collection on huge sstable formed after major compaction and that should no more be considered a drawback. Please confirm my understanding. Also I want to know are there any other side effects and inefficiences of say 100+ gb sstable? Please refer to my fist email on the issue. We tried splitting sstable using sstablesplit but just because all small sstables are generated of same size, STCS compacted them back to single giant sstable as soon as we started Cassandra. Any other alternatives ? JIRA for the Issue of numerous tiny sstables being generated after repair is still open and we want a confirmation that if we face such issue in prod we can go ahead with one time major compaction. Thanks Anuj Wadehra From:Robert Coli rc...@eventbrite.com Date:Tue, 14 Apr, 2015 at 12:54 am Subject:Re: Drawbacks of Major Compaction now that Automatic Tombstone Compaction Exists On Mon, Apr 13, 2015 at 10:52 AM, Anuj Wadehra anujw_2...@yahoo.co.in wrote: Any comments on side effects of Major compaction especially when sstable generated is 100+ GB? I have no idea how this interacts with the automatic compaction stuff; if you find out, let us know? But if you want to do a major and don't want to deal with One Big SSTable afterwards, stop the node and then run sstable_split utility. =Rob
