[jira] [Updated] (CASSANDRA-7275) Errors in FlushRunnable may leave threads hung
[
https://issues.apache.org/jira/browse/CASSANDRA-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Yaskevich updated CASSANDRA-7275:
---
Attachment: CASSANDRA-7275-flush-info.patch
First take on this problem, I've added Flush info with stores boolean (was
successful or not) per-cfid being flushed and only disregards commit log
per-cfid if the flush was a success.
> Errors in FlushRunnable may leave threads hung
> --
>
> Key: CASSANDRA-7275
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7275
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 2.0.12
>
> Attachments: 0001-Move-latch.countDown-into-finally-block.patch,
> 7252-2.0-v2.txt, CASSANDRA-7275-flush-info.patch
>
>
> In Memtable.FlushRunnable, the CountDownLatch will never be counted down if
> there are errors, which results in hanging any threads that are waiting for
> the flush to complete. For example, an error like this causes the problem:
> {noformat}
> ERROR [FlushWriter:474] 2014-05-20 12:10:31,137 CassandraDaemon.java (line
> 198) Exception in thread Thread[FlushWriter:474,5,main]
> java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Unknown Source)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:64)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:72)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.split(AbstractCompositeType.java:138)
> at
> org.apache.cassandra.io.sstable.ColumnNameHelper.minComponents(ColumnNameHelper.java:103)
> at
> org.apache.cassandra.db.ColumnFamily.getColumnStats(ColumnFamily.java:439)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:194)
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:397)
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:350)
> at
> org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8463) Upgrading 2.0 to 2.1 causes LCS to recompact all files
[ https://issues.apache.org/jira/browse/CASSANDRA-8463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-8463: --- Attachment: 0001-better-logging.patch [~rbranson] could you try running with this patch for a while and posting the logs? It improves compaction logging (ie, what level we take sstables from and where they end up) > Upgrading 2.0 to 2.1 causes LCS to recompact all files > -- > > Key: CASSANDRA-8463 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8463 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Hardware is recent 2-socket, 16-core (x2 Hyperthreaded), > 144G RAM, solid-state storage. > Platform is Linux 3.2.51, Oracle JDK 64-bit 1.7.0_65. > Heap is 32G total, 4G newsize. > 8G/8G on-heap/off-heap memtables, offheap_buffer allocator, 0.5 > memtable_cleanup_threshold > concurrent_compactors: 20 >Reporter: Rick Branson >Assignee: Marcus Eriksson > Fix For: 2.1.3 > > Attachments: 0001-better-logging.patch, log-for-8463.txt > > > It appears that tables configured with LCS will completely re-compact > themselves over some period of time after upgrading from 2.0 to 2.1 (2.0.11 > -> 2.1.2, specifically). It starts out with <10 pending tasks for an hour or > so, then starts building up, now with 50-100 tasks pending across the cluster > after 12 hours. These nodes are under heavy write load, but were easily able > to keep up in 2.0 (they rarely had >5 pending compaction tasks), so I don't > think it's LCS in 2.1 actually being worse, just perhaps some different LCS > behavior that causes the layout of tables from 2.0 to prompt the compactor to > reorganize them? > The nodes flushed ~11MB SSTables under 2.0. They're currently flushing ~36MB > SSTables due to the improved memtable setup in 2.1. Before I upgraded the > entire cluster to 2.1, I noticed the problem and tried several variations on > the flush size, thinking perhaps the larger tables in L0 were causing some > kind of cascading compactions. Even if they're sized roughly like the 2.0 > flushes were, same behavior occurs. I also tried both enabling & disabling > STCS in L0 with no real change other than L0 began to back up faster, so I > left the STCS in L0 enabled. > Tables are configured with 32MB sstable_size_in_mb, which was found to be an > improvement on the 160MB table size for compaction performance. Maybe this is > wrong now? Otherwise, the tables are configured with defaults. Compaction has > been unthrottled to help them catch-up. The compaction threads stay very > busy, with the cluster-wide CPU at 45% "nice" time. No nodes have completely > caught up yet. I'll update JIRA with status about their progress if anything > interesting happens. > From a node around 12 hours ago, around an hour after the upgrade, with 19 > pending compaction tasks: > SSTables in each level: [6/4, 10, 105/100, 268, 0, 0, 0, 0, 0] > SSTables in each level: [6/4, 10, 106/100, 271, 0, 0, 0, 0, 0] > SSTables in each level: [1, 16/10, 105/100, 269, 0, 0, 0, 0, 0] > SSTables in each level: [5/4, 10, 103/100, 272, 0, 0, 0, 0, 0] > SSTables in each level: [4, 11/10, 105/100, 270, 0, 0, 0, 0, 0] > SSTables in each level: [1, 12/10, 105/100, 271, 0, 0, 0, 0, 0] > SSTables in each level: [1, 14/10, 104/100, 267, 0, 0, 0, 0, 0] > SSTables in each level: [9/4, 10, 103/100, 265, 0, 0, 0, 0, 0] > Recently, with 41 pending compaction tasks: > SSTables in each level: [4, 13/10, 106/100, 269, 0, 0, 0, 0, 0] > SSTables in each level: [4, 12/10, 106/100, 273, 0, 0, 0, 0, 0] > SSTables in each level: [5/4, 11/10, 106/100, 271, 0, 0, 0, 0, 0] > SSTables in each level: [4, 12/10, 103/100, 275, 0, 0, 0, 0, 0] > SSTables in each level: [2, 13/10, 106/100, 273, 0, 0, 0, 0, 0] > SSTables in each level: [3, 10, 104/100, 275, 0, 0, 0, 0, 0] > SSTables in each level: [6/4, 11/10, 103/100, 269, 0, 0, 0, 0, 0] > SSTables in each level: [4, 16/10, 105/100, 264, 0, 0, 0, 0, 0] > More information about the use case: writes are roughly uniform across these > tables. The data is "sharded" across these 8 tables by key to improve > compaction parallelism. Each node receives up to 75,000 writes/sec sustained > at peak, and a small number of reads. This is a pre-production cluster that's > being warmed up with new data, so the low volume of reads (~100/sec per node) > is just from automatic sampled data checks, otherwise we'd just use STCS :) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8473) Secondary index support for key-value pairs in CQL3 maps
[
https://issues.apache.org/jira/browse/CASSANDRA-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Samuel Klock updated CASSANDRA-8473:
Attachment: trunk-8473-v2.txt
cassandra-2.1-8473-v2.txt
Attaching new versions of the patch (for 2.1 and trunk) that are intended to
address your feedback. Details:
* {{SingleColumnRelation}}
** bq. For lists, this error message may be confusing:
{{checkTrue(receiver.type instanceof MapType, "Column \"%s\" cannot be used as
a map", receiver.name);}}. For example, if you do {{WHERE mylist\[0\] =
'foo'}}, you're not really trying to use it as a map. You may want to handle
lists specially.
Done: lists get a different error message now. (In the 2.1 patch, this change
is made in {{SelectStatement}}.)
** bq. The {{checkFalse()}} statement below this is starting to get confusing,
I would break it up
Agreed. Refactored the condition into a couple of extracted methods. (In the
2.1 patch, a similar change is made in {{SelectStatement}}.)
** bq. Use curly braces on the "if" clause when they're used on the "else"
Done in trunk patch. No equivalent issue in 2.1 patch.
** bq. I'm not sure that frozen maps are handled correctly here (e.g. WHERE
myfrozenmap\['foo'\] = 'bar'). May want to double-check that.
Added a test. I think trunk is okay (although the error message is imperfect:
"Invalid STRING constant (foo) for "myfrozenmap" of type frozen>"). The 2.1 patch incorrectly gave no error for queries with this
condition; that's been fixed.
* {{SingleColumnRelation.Contains()}}
** bq. Update class-level comment (should be a javadoc) to include map entry
restrictions
Updated the comment in the trunk patch (it was already updated in the 2.1
version). Making it a Javadoc would be inconsistent with the other nested
classes in {{SingleColumnRestriction}} (which have no Javadoc), so I'm
reluctant to do so. But I wouldn't forcefully object.
** bq. entries(): no need for curly braces with a single-line for-loop
Moot because the for-loop is now multiline. No equivalent issue in the 2.1
patch.
* {{CreateIndexStatement}}
** bq. switch on target.type could be clearer if re-organized; also, the error
message about 'keys' is slightly misleading for 'entries' indexes
Corrected both error messages. It's not obvious to me how to reorganize the
{{switch}} to make it clearer (very likely because I wrote it) -- did you have
something specific in mind?
* {{IndexTarget.TargetType.fromIndexOptions()}}
** bq. Should this return FULL if index_values isn't present? Also, no curlies
needed for single-line clauses.
Removed the braces. On the return value: no index options will be present if
the column isn't a non-frozen collection. Even so, this is a correctness
issue: the v1 patch would yield the wrong error message if a user attempted to
create a {{FULL}} index on a frozen map that already had one. Fixed in v2.
* {{ExtendedFilter}}
** bq. {{else if (expr.isContains())}} will always be false (due to the
{{isContains()}} check above).
The block has been removed. Note that it was an attempt to preserve the logic
that existed in the {{MAP}} case previously if {{expr.isContainsKey()}} were
false; since the only kind of expressions apart from {{CONTAINS KEY}} that were
valid for map columns were {{CONTAINS}} relations, this seemed like the right
choice. But your analysis appears to be correct. Is there some other way that
code would have been reachable? (It _looks_ like the code may have been
intended to check {{map\[key\] = value}} conditions, but AFAIK there would have
been no way to trigger its execution.)
* {{CompositesIndex}}
** bq. No nested ternaries, please
Rewritten as {{if}} statements.
* {{CompositesIndexOnCollectionKeyAndValue}}
** bq. makeIndexColumnPrefix(): need to use {{min(count - 1, cellName.size())}}
for loop end (see CASSANDRA-8053 for why)
Fixed by extending {{CompositesIndexOnCollectionKey}}. (Also, did you mean
CASSANDRA-8073?)
** bq. by extending CompositesIndexOnCollectionKey, you could eliminate about
half of the methods
Done, although I'm not positive the abstractions are quite right (is
{{CompositesIndexOnCollectionKeyAndValue}} really a kind of
{{CompositesIndexOnCollectionKey}}? Would it be better to use a common
superclass?). But the reduced duplication is very nice.
** bq. isStale(): instead of building a new composite to compare with the index
entry key, why not compare the cell value with the second item in the index
entry composite? This method could also use a comment or two
Good point -- the implementation no longer builds a new composite for the
comparison. I've also refactored the code for clarity; if you still think it
needs comments, I'll certainly add them. (Also: fixed the return value for
when the cell in the indexed table is dead.)
* {{SecondaryIndexOnMapEntriesTest}}
** bq. Unusued and commented out imports
Removed the co
[jira] [Commented] (CASSANDRA-8316) "Did not get positive replies from all endpoints" error on incremental repair
[ https://issues.apache.org/jira/browse/CASSANDRA-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14246358#comment-14246358 ] Marcus Eriksson commented on CASSANDRA-8316: [~aboudreault] how do you reproduce this? The "Repair failed with error Already repairing"-error is supposed to happen if you start several repairs over the same sstables > "Did not get positive replies from all endpoints" error on incremental repair > -- > > Key: CASSANDRA-8316 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8316 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: cassandra 2.1.2 >Reporter: Loic Lambiel >Assignee: Marcus Eriksson > Fix For: 2.1.3 > > Attachments: 0001-patch.patch, 8316-v2.patch, > CassandraDaemon-2014-11-25-2.snapshot.tar.gz, > CassandraDaemon-2014-12-14.snapshot.tar.gz, test.sh > > > Hi, > I've got an issue with incremental repairs on our production 15 nodes 2.1.2 > (new cluster, not yet loaded, RF=3) > After having successfully performed an incremental repair (-par -inc) on 3 > nodes, I started receiving "Repair failed with error Did not get positive > replies from all endpoints." from nodetool on all remaining nodes : > [2014-11-14 09:12:36,488] Starting repair command #3, repairing 108 ranges > for keyspace (seq=false, full=false) > [2014-11-14 09:12:47,919] Repair failed with error Did not get positive > replies from all endpoints. > All the nodes are up and running and the local system log shows that the > repair commands got started and that's it. > I've also noticed that soon after the repair, several nodes started having > more cpu load indefinitely without any particular reason (no tasks / queries, > nothing in the logs). I then restarted C* on these nodes and retried the > repair on several nodes, which were successful until facing the issue again. > I tried to repro on our 3 nodes preproduction cluster without success > It looks like I'm not the only one having this issue: > http://www.mail-archive.com/user%40cassandra.apache.org/msg39145.html > Any idea? > Thanks > Loic -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8211) Overlapping sstables in L1+
[
https://issues.apache.org/jira/browse/CASSANDRA-8211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14246355#comment-14246355
]
Marcus Eriksson commented on CASSANDRA-8211:
yes it will be in 2.1.3
> Overlapping sstables in L1+
> ---
>
> Key: CASSANDRA-8211
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8211
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 2.0.12, 2.1.3
>
> Attachments: 0001-Avoid-overlaps-in-L1-v2.patch,
> 0001-Avoid-overlaps-in-L1.patch
>
>
> Seems we have a bug that can create overlapping sstables in L1:
> {code}
> WARN [main] 2014-10-28 04:09:42,295 LeveledManifest.java (line 164) At level
> 2, SSTableReader(path='') [DecoratedKey(2838397575996053472, 00
> 10066059b210066059b210400100),
> DecoratedKey(5516674013223138308,
> 001000ff2d161000ff2d160
> 00010400100)] overlaps
> SSTableReader(path='') [DecoratedKey(2839992722300822584,
> 0010
> 00229ad21000229ad210400100),
> DecoratedKey(5532836928694021724,
> 0010034b05a610034b05a6100
> 000400100)]. This could be caused by a bug in
> Cassandra 1.1.0 .. 1.1.3 or due to the fact that you have dropped sstables
> from another node into the data directory. Sending back to L0. If
> you didn't drop in sstables, and have not yet run scrub, you should do so
> since you may also have rows out-of-order within an sstable
> {code}
> Which might manifest itself during compaction with this exception:
> {code}
> ERROR [CompactionExecutor:3152] 2014-10-28 00:24:06,134 CassandraDaemon.java
> (line 199) Exception in thread Thread[CompactionExecutor:3152,1,main]
> java.lang.RuntimeException: Last written key
> DecoratedKey(5516674013223138308,
> 001000ff2d161000ff2d1610400100)
> >= current key DecoratedKey(2839992722300822584,
> 001000229ad21000229ad210400100)
> writing into
> {code}
> since we use LeveledScanner when compacting (the backing sstable scanner
> might go beyond the start of the next sstable scanner)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8211) Overlapping sstables in L1+
[
https://issues.apache.org/jira/browse/CASSANDRA-8211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-8211:
---
Fix Version/s: 2.1.3
> Overlapping sstables in L1+
> ---
>
> Key: CASSANDRA-8211
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8211
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 2.0.12, 2.1.3
>
> Attachments: 0001-Avoid-overlaps-in-L1-v2.patch,
> 0001-Avoid-overlaps-in-L1.patch
>
>
> Seems we have a bug that can create overlapping sstables in L1:
> {code}
> WARN [main] 2014-10-28 04:09:42,295 LeveledManifest.java (line 164) At level
> 2, SSTableReader(path='') [DecoratedKey(2838397575996053472, 00
> 10066059b210066059b210400100),
> DecoratedKey(5516674013223138308,
> 001000ff2d161000ff2d160
> 00010400100)] overlaps
> SSTableReader(path='') [DecoratedKey(2839992722300822584,
> 0010
> 00229ad21000229ad210400100),
> DecoratedKey(5532836928694021724,
> 0010034b05a610034b05a6100
> 000400100)]. This could be caused by a bug in
> Cassandra 1.1.0 .. 1.1.3 or due to the fact that you have dropped sstables
> from another node into the data directory. Sending back to L0. If
> you didn't drop in sstables, and have not yet run scrub, you should do so
> since you may also have rows out-of-order within an sstable
> {code}
> Which might manifest itself during compaction with this exception:
> {code}
> ERROR [CompactionExecutor:3152] 2014-10-28 00:24:06,134 CassandraDaemon.java
> (line 199) Exception in thread Thread[CompactionExecutor:3152,1,main]
> java.lang.RuntimeException: Last written key
> DecoratedKey(5516674013223138308,
> 001000ff2d161000ff2d1610400100)
> >= current key DecoratedKey(2839992722300822584,
> 001000229ad21000229ad210400100)
> writing into
> {code}
> since we use LeveledScanner when compacting (the backing sstable scanner
> might go beyond the start of the next sstable scanner)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8480) Update of primary key should be possible
Jason Kania created CASSANDRA-8480: -- Summary: Update of primary key should be possible Key: CASSANDRA-8480 URL: https://issues.apache.org/jira/browse/CASSANDRA-8480 Project: Cassandra Issue Type: Improvement Components: API Reporter: Jason Kania While attempting to update a column in a row, I encountered the error PRIMARY KEY part thingy found in SET part The error is not helpful as it doesn't state why this is problem so I looked on google and encountered many, many entries from people who have experienced the issue including those with single column table who have to hack to work around this. After looking around further in the documentation, I discovered that it is not possible to update a primary key but I still have not found a good explanation. I suspect that that this is because it would change the indexing location of the record effectively requiring a delete followed by an insert. If the question is one of guaranteeing no update to a deleted row, a client will have the same issue. To me, this really should be handled behind the API because: 1) it is an expected capability in a database to update all columns and having these limitations only puts off potential users especially when they have to discover the limitation after the fact 2) being able to use a column in a WHERE clause requires it to be part of the primary key so what this limitation means is if you can update a column, you can't search for it, or if you can search on a column, you can't update it which leaves a serious gap in handling a wide number of use cases. 3) deleting and inserting a row with an updated primary key will mean sucking in all the data from the row up to the client and sending it all back down even when a single column in the primary key was all that was updated. Why not document the issue but make the interface more usable by supporting the operation? Jason -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8477) CMS GC can not recycle objects
[ https://issues.apache.org/jira/browse/CASSANDRA-8477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philo Yang updated CASSANDRA-8477: -- Attachment: system.log.2014-12-15_1226 log1.txt log2.txt Swap is not been disabled, but it seems that swap is never used. I uploaded 3 files from one node that was unresponsive this morning but recovered by itself after several minutes. log1 and log2 is the file output by -XX:LogFile and -Xloggc > CMS GC can not recycle objects > -- > > Key: CASSANDRA-8477 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8477 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: 2.1.1 or 2.1.2-SNAPSHOT(after CASSANDRA-8459 resolved) >Reporter: Philo Yang > Attachments: cassandra.yaml, histo.txt, jstack.txt, log1.txt, > log2.txt, system.log, system.log.2014-12-15_1226 > > > I have a trouble in my cluster that CMS full gc can not reduce the size of > old gen. Days ago I post this problem to the maillist, people think it will > be solved by tuning the gc setting, however it doesn't work for me. > Then I saw a similar bug in CASSANDRA-8447, but [~benedict] think it is not > related. With the jstack on > https://gist.github.com/yangzhe1991/755ea2a10520be1fe59a, [~benedict] find a > bug and resolved it in CASSANDRA-8459. So I build a latest version on 2.1 > branch and run the SNAPSHOT version on the nodes with gc trouble. > However, there is still the gc issue. So I think opening a new tick and post > more information is a good idea. Thanks for helping me. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8479) Timeout Exception on Node Failure in Remote Data Center
Amit Singh Chowdhery created CASSANDRA-8479: --- Summary: Timeout Exception on Node Failure in Remote Data Center Key: CASSANDRA-8479 URL: https://issues.apache.org/jira/browse/CASSANDRA-8479 Project: Cassandra Issue Type: Bug Components: API, Core, Tools Environment: Unix, Cassandra 2.0.11 Reporter: Amit Singh Chowdhery Priority: Minor Fix For: 2.0.3 Issue Faced : We have a Geo-red setup with 2 Data centers having 3 nodes each. When we bring down a single Cassandra node down in DC2 by kill -9 , reads fail on DC1 with TimedOutException for a brief amount of time (15-20 sec~). Reference : Already a ticket has been opened/resolved and link is provided below : https://issues.apache.org/jira/browse/CASSANDRA-8352 Activity Done as per Resolution Provided : Upgraded to Cassandra 2.0.11 . We have two 3 node clusters in two different DCs and if one or more of the nodes go down in one Data Center , ~5-10% traffic failure is observed on the other. CL: LOCAL_QUORUM RF=3 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6173) Unable to delete multiple entries using In clause on clustering part of compound key
[ https://issues.apache.org/jira/browse/CASSANDRA-6173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14246288#comment-14246288 ] Jason Kania commented on CASSANDRA-6173: Inconsistencies like these, especially when undocumented, are obnoxious because they waste a person's time to figure out. At least document it in the meantime. In general the delete documentation could use more detail as its fairly weak. > Unable to delete multiple entries using In clause on clustering part of > compound key > > > Key: CASSANDRA-6173 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6173 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Ashot Golovenko >Priority: Minor > > I have the following table: > CREATE TABLE user_relation ( > u1 bigint, > u2 bigint, > mf int, > i boolean, > PRIMARY KEY (u1, u2)); > And I'm trying to delete two entries using In clause on clustering part of > compound key and I fail to do so: > cqlsh:bm> DELETE from user_relation WHERE u1 = 755349113 and u2 in > (13404014120, 12537242743); > Bad Request: Invalid operator IN for PRIMARY KEY part u2 > Although the select statement works just fine: > cqlsh:bm> select * from user_relation WHERE u1 = 755349113 and u2 in > (13404014120, 12537242743); > u1| u2 | i| mf > ---+-+--+ > 755349113 | 12537242743 | null | 27 > 755349113 | 13404014120 | null | 0 > (2 rows) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8478) sstableloader NegativeArraySizeException when deserializing to build histograms
[
https://issues.apache.org/jira/browse/CASSANDRA-8478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14246249#comment-14246249
]
Erick Ramirez commented on CASSANDRA-8478:
--
[~yukim] advised there might be possible corruption with the CF metadata. I
have advised customer to temporarily move the {{*-Statistics.db}} files and
attempt to load the data files. This is currently pending.
> sstableloader NegativeArraySizeException when deserializing to build
> histograms
> ---
>
> Key: CASSANDRA-8478
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8478
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Erick Ramirez
> Fix For: 1.2.15
>
>
> When a customer attempts to load sstable data files copied from a production
> cluster, it returns the following exception:
> {code}
> $ sstableloader -d -p -v //
> null
> java.lang.NegativeArraySizeException
> at
> org.apache.cassandra.utils.EstimatedHistogram$EstimatedHistogramSerializer.deserialize(EstimatedHistogram.java:266)
> at
> org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:292)
> at
> org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:282)
> at
> org.apache.cassandra.io.sstable.SSTableReader.openMetadata(SSTableReader.java:234)
> at
> org.apache.cassandra.io.sstable.SSTableReader.openForBatch(SSTableReader.java:162)
> at
> org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:100)
> at java.io.File.list(File.java:1155)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:67)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:121)
> at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:66)
> -pr,--principal kerberos principal
> -k,--keytab keytab location
> --ssl-keystore ssl keystore location
> --ssl-keystore-password ssl keystore password
> --ssl-keystore-type ssl keystore type
> --ssl-truststore ssl truststore location
> --ssl-truststore-password ssl truststore password
> --ssl-truststore-type ssl truststore type
> {code}
> It appears to be failing on this line of code:
> {code}
> public EstimatedHistogram deserialize(DataInput dis) throws
> IOException
> {
> int size = dis.readInt();
> long[] offsets = new long[size - 1]; < here
> {code}
> The same error is returned regardless of which data file is attempted. I
> suspect this may be due to corrupt data files or the way data is written that
> is not compatible with the sstableloader utility.
> NOTE: Both source and target clusters are DSE 3.2.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8478) sstableloader NegativeArraySizeException when deserializing to build histograms
[
https://issues.apache.org/jira/browse/CASSANDRA-8478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Ramirez updated CASSANDRA-8478:
-
Description:
When a customer attempts to load sstable data files copied from a production
cluster, it returns the following exception:
{code}
$ sstableloader -d -p -v //
null
java.lang.NegativeArraySizeException
at
org.apache.cassandra.utils.EstimatedHistogram$EstimatedHistogramSerializer.deserialize(EstimatedHistogram.java:266)
at
org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:292)
at
org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:282)
at
org.apache.cassandra.io.sstable.SSTableReader.openMetadata(SSTableReader.java:234)
at
org.apache.cassandra.io.sstable.SSTableReader.openForBatch(SSTableReader.java:162)
at
org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:100)
at java.io.File.list(File.java:1155)
at
org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:67)
at org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:121)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:66)
-pr,--principal kerberos principal
-k,--keytab keytab location
--ssl-keystore ssl keystore location
--ssl-keystore-password ssl keystore password
--ssl-keystore-type ssl keystore type
--ssl-truststore ssl truststore location
--ssl-truststore-password ssl truststore password
--ssl-truststore-type ssl truststore type
{code}
It appears to be failing on this line of code:
{code}
public EstimatedHistogram deserialize(DataInput dis) throws IOException
{
int size = dis.readInt();
long[] offsets = new long[size - 1]; < here
{code}
The same error is returned regardless of which data file is attempted. I
suspect this may be due to corrupt data files or the way data is written that
is not compatible with the sstableloader utility.
NOTE: Both source and target clusters are DSE 3.2.5.
was:
When a customer attempts to load sstable data files copied from a production
cluster, it returns the following exception:
{code}
$ sstableloader -d -p -v //
null
java.lang.NegativeArraySizeException
at
org.apache.cassandra.utils.EstimatedHistogram$EstimatedHistogramSerializer.deserialize(EstimatedHistogram.java:266)
at
org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:292)
at
org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:282)
at
org.apache.cassandra.io.sstable.SSTableReader.openMetadata(SSTableReader.java:234)
at
org.apache.cassandra.io.sstable.SSTableReader.openForBatch(SSTableReader.java:162)
at
org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:100)
at java.io.File.list(File.java:1155)
at
org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:67)
at org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:121)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:66)
-pr,--principal kerberos principal
-k,--keytab keytab location
--ssl-keystore ssl keystore location
--ssl-keystore-password ssl keystore password
--ssl-keystore-type ssl keystore type
--ssl-truststore ssl truststore location
--ssl-truststore-password ssl truststore password
--ssl-truststore-type ssl truststore type
{code}
It appears to be failing on this line of code:
{code}
public EstimatedHistogram deserialize(DataInput dis) throws IOException
{
int size = dis.readInt();
long[] offsets = new long[size - 1]; < here
{code}
The same error is returned regardless of which data file is attempted. I
suspect this may be due to corrupt data files or the way data is written that
is not compatible with the sstableloader utility.
> sstableloader NegativeArraySizeException when deserializing to build
> histograms
> ---
>
> Key: CASSANDRA-8478
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8478
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Erick Ramirez
> Fix For: 1.2.15
>
>
> When a customer attempts to load sstable data files copied from a production
> cluster, it returns the following exception:
> {code}
> $ sstableloader -d -p -v //
> null
> java.lang.NegativeArraySizeException
> at
> org.apache.cassandra.utils.EstimatedHistogram$EstimatedHistogramSerializer.deserialize(EstimatedHistogram.java:266)
> at
> org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:292)
> at
> org.apache.cassandra.io.sstable.SSTableMetadata$SSTab
[jira] [Created] (CASSANDRA-8478) sstableloader NegativeArraySizeException when deserializing to build histograms
Erick Ramirez created CASSANDRA-8478:
Summary: sstableloader NegativeArraySizeException when
deserializing to build histograms
Key: CASSANDRA-8478
URL: https://issues.apache.org/jira/browse/CASSANDRA-8478
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Erick Ramirez
Fix For: 1.2.15
When a customer attempts to load sstable data files copied from a production
cluster, it returns the following exception:
{code}
$ sstableloader -d -p -v //
null
java.lang.NegativeArraySizeException
at
org.apache.cassandra.utils.EstimatedHistogram$EstimatedHistogramSerializer.deserialize(EstimatedHistogram.java:266)
at
org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:292)
at
org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:282)
at
org.apache.cassandra.io.sstable.SSTableReader.openMetadata(SSTableReader.java:234)
at
org.apache.cassandra.io.sstable.SSTableReader.openForBatch(SSTableReader.java:162)
at
org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:100)
at java.io.File.list(File.java:1155)
at
org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:67)
at org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:121)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:66)
-pr,--principal kerberos principal
-k,--keytab keytab location
--ssl-keystore ssl keystore location
--ssl-keystore-password ssl keystore password
--ssl-keystore-type ssl keystore type
--ssl-truststore ssl truststore location
--ssl-truststore-password ssl truststore password
--ssl-truststore-type ssl truststore type
{code}
It appears to be failing on this line of code:
{code}
public EstimatedHistogram deserialize(DataInput dis) throws IOException
{
int size = dis.readInt();
long[] offsets = new long[size - 1]; < here
{code}
The same error is returned regardless of which data file is attempted. I
suspect this may be due to corrupt data files or the way data is written that
is not compatible with the sstableloader utility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8462) Upgrading a 2.0 to 2.1 breaks CFMetaData on 2.0 nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14246186#comment-14246186
]
Aleksey Yeschenko commented on CASSANDRA-8462:
--
What would be the obvious fix?
That row shouldn't have made it to a 2.0.11 node. No idea how it got there, yet
- unless you've restarted the upgraded 2.1 node as a 2.0 node. This is the
first thing I'd suspect, had I not known the reporter (but please double check,
still - unless someone messed w OTC/ITC code again, we do not exchange schema
between nodes of different majors).
The workaround is obvious though. Just upgrade that node to 2.1.
> Upgrading a 2.0 to 2.1 breaks CFMetaData on 2.0 nodes
> -
>
> Key: CASSANDRA-8462
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8462
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Rick Branson
>Assignee: Aleksey Yeschenko
>
> Added a 2.1.2 node to a cluster running 2.0.11. Didn't make any schema
> changes. When I tried to reboot one of the 2.0 nodes, it failed to boot with
> this exception. Besides an obvious fix, any workarounds for this?
> {noformat}
> java.lang.IllegalArgumentException: No enum constant
> org.apache.cassandra.config.CFMetaData.Caching.{"keys":"ALL",
> "rows_per_partition":"NONE"}
> at java.lang.Enum.valueOf(Enum.java:236)
> at
> org.apache.cassandra.config.CFMetaData$Caching.valueOf(CFMetaData.java:286)
> at
> org.apache.cassandra.config.CFMetaData.fromSchemaNoColumnsNoTriggers(CFMetaData.java:1713)
> at
> org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1793)
> at
> org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:307)
> at
> org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:288)
> at
> org.apache.cassandra.db.DefsTables.loadFromKeyspace(DefsTables.java:131)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:529)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:270)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8316) "Did not get positive replies from all endpoints" error on incremental repair
[
https://issues.apache.org/jira/browse/CASSANDRA-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14246153#comment-14246153
]
Alan Boudreault edited comment on CASSANDRA-8316 at 12/14/14 9:59 PM:
--
[~krummas], unfortunately, I'm still getting the initial issue AND the issue
introduced on multiple nodes with the v2 patch + CASSANDRA-8458. 8 nodes
cluster and cassandra-stress with n=100
{code}
aboudreault@kovarro:~/dev/cstar/8316$ ccm node3 nodetool -- repair -par -inc
[2014-12-14 16:30:48,037] Starting repair command #1, repairing 3 ranges for
keyspace r1 (parallelism=PARALLEL, full=false)
[2014-12-14 16:30:48,040] Repair failed with error Already repairing
SSTableReader(path='/home/aboudreault/.ccm/local/node3/data/r1/Standard1-d11383e083d411e4869ab56034537865/r1-Standard1-ka-38-Data.db'),
can not continue.
{code}
In addition, I noticed that it took 20 minutes restart my cluster this time.
Not sure if it's related to this issue but I've attached a yourkit snapshot.
Let me know if I can anything else.
was (Author: aboudreault):
[~krummas], unfortunately, I'm still getting the same issue introduced on
multiple nodes with the v2 patch + CASSANDRA-8458. 8 nodes cluster and
cassandra-stress with n=100
{code}
aboudreault@kovarro:~/dev/cstar/8316$ ccm node3 nodetool -- repair -par -inc
[2014-12-14 16:30:48,037] Starting repair command #1, repairing 3 ranges for
keyspace r1 (parallelism=PARALLEL, full=false)
[2014-12-14 16:30:48,040] Repair failed with error Already repairing
SSTableReader(path='/home/aboudreault/.ccm/local/node3/data/r1/Standard1-d11383e083d411e4869ab56034537865/r1-Standard1-ka-38-Data.db'),
can not continue.
{code}
In addition, I noticed that it took 20 minutes restart my cluster this time.
Not sure if it's related to this issue but I've attached a yourkit snapshot.
Let me know if I can anything else.
> "Did not get positive replies from all endpoints" error on incremental repair
> --
>
> Key: CASSANDRA-8316
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8316
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: cassandra 2.1.2
>Reporter: Loic Lambiel
>Assignee: Marcus Eriksson
> Fix For: 2.1.3
>
> Attachments: 0001-patch.patch, 8316-v2.patch,
> CassandraDaemon-2014-11-25-2.snapshot.tar.gz,
> CassandraDaemon-2014-12-14.snapshot.tar.gz, test.sh
>
>
> Hi,
> I've got an issue with incremental repairs on our production 15 nodes 2.1.2
> (new cluster, not yet loaded, RF=3)
> After having successfully performed an incremental repair (-par -inc) on 3
> nodes, I started receiving "Repair failed with error Did not get positive
> replies from all endpoints." from nodetool on all remaining nodes :
> [2014-11-14 09:12:36,488] Starting repair command #3, repairing 108 ranges
> for keyspace (seq=false, full=false)
> [2014-11-14 09:12:47,919] Repair failed with error Did not get positive
> replies from all endpoints.
> All the nodes are up and running and the local system log shows that the
> repair commands got started and that's it.
> I've also noticed that soon after the repair, several nodes started having
> more cpu load indefinitely without any particular reason (no tasks / queries,
> nothing in the logs). I then restarted C* on these nodes and retried the
> repair on several nodes, which were successful until facing the issue again.
> I tried to repro on our 3 nodes preproduction cluster without success
> It looks like I'm not the only one having this issue:
> http://www.mail-archive.com/user%40cassandra.apache.org/msg39145.html
> Any idea?
> Thanks
> Loic
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8316) "Did not get positive replies from all endpoints" error on incremental repair
[
https://issues.apache.org/jira/browse/CASSANDRA-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-8316:
---
Attachment: CassandraDaemon-2014-12-14.snapshot.tar.gz
[~krummas], unfortunately, I'm still getting the same issue introduced on
multiple nodes with the v2 patch + CASSANDRA-8458. 8 nodes cluster and
cassandra-stress with n=100
{code}
aboudreault@kovarro:~/dev/cstar/8316$ ccm node3 nodetool -- repair -par -inc
[2014-12-14 16:30:48,037] Starting repair command #1, repairing 3 ranges for
keyspace r1 (parallelism=PARALLEL, full=false)
[2014-12-14 16:30:48,040] Repair failed with error Already repairing
SSTableReader(path='/home/aboudreault/.ccm/local/node3/data/r1/Standard1-d11383e083d411e4869ab56034537865/r1-Standard1-ka-38-Data.db'),
can not continue.
{code}
In addition, I noticed that it took 20 minutes restart my cluster this time.
Not sure if it's related to this issue but I've attached a yourkit snapshot.
Let me know if I can anything else.
> "Did not get positive replies from all endpoints" error on incremental repair
> --
>
> Key: CASSANDRA-8316
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8316
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: cassandra 2.1.2
>Reporter: Loic Lambiel
>Assignee: Marcus Eriksson
> Fix For: 2.1.3
>
> Attachments: 0001-patch.patch, 8316-v2.patch,
> CassandraDaemon-2014-11-25-2.snapshot.tar.gz,
> CassandraDaemon-2014-12-14.snapshot.tar.gz, test.sh
>
>
> Hi,
> I've got an issue with incremental repairs on our production 15 nodes 2.1.2
> (new cluster, not yet loaded, RF=3)
> After having successfully performed an incremental repair (-par -inc) on 3
> nodes, I started receiving "Repair failed with error Did not get positive
> replies from all endpoints." from nodetool on all remaining nodes :
> [2014-11-14 09:12:36,488] Starting repair command #3, repairing 108 ranges
> for keyspace (seq=false, full=false)
> [2014-11-14 09:12:47,919] Repair failed with error Did not get positive
> replies from all endpoints.
> All the nodes are up and running and the local system log shows that the
> repair commands got started and that's it.
> I've also noticed that soon after the repair, several nodes started having
> more cpu load indefinitely without any particular reason (no tasks / queries,
> nothing in the logs). I then restarted C* on these nodes and retried the
> repair on several nodes, which were successful until facing the issue again.
> I tried to repro on our 3 nodes preproduction cluster without success
> It looks like I'm not the only one having this issue:
> http://www.mail-archive.com/user%40cassandra.apache.org/msg39145.html
> Any idea?
> Thanks
> Loic
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8211) Overlapping sstables in L1+
[
https://issues.apache.org/jira/browse/CASSANDRA-8211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14246132#comment-14246132
]
Randy Fradin commented on CASSANDRA-8211:
-
Does this affect 2.1 as well? Will the patch be in 2.1.3?
> Overlapping sstables in L1+
> ---
>
> Key: CASSANDRA-8211
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8211
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 2.0.12
>
> Attachments: 0001-Avoid-overlaps-in-L1-v2.patch,
> 0001-Avoid-overlaps-in-L1.patch
>
>
> Seems we have a bug that can create overlapping sstables in L1:
> {code}
> WARN [main] 2014-10-28 04:09:42,295 LeveledManifest.java (line 164) At level
> 2, SSTableReader(path='') [DecoratedKey(2838397575996053472, 00
> 10066059b210066059b210400100),
> DecoratedKey(5516674013223138308,
> 001000ff2d161000ff2d160
> 00010400100)] overlaps
> SSTableReader(path='') [DecoratedKey(2839992722300822584,
> 0010
> 00229ad21000229ad210400100),
> DecoratedKey(5532836928694021724,
> 0010034b05a610034b05a6100
> 000400100)]. This could be caused by a bug in
> Cassandra 1.1.0 .. 1.1.3 or due to the fact that you have dropped sstables
> from another node into the data directory. Sending back to L0. If
> you didn't drop in sstables, and have not yet run scrub, you should do so
> since you may also have rows out-of-order within an sstable
> {code}
> Which might manifest itself during compaction with this exception:
> {code}
> ERROR [CompactionExecutor:3152] 2014-10-28 00:24:06,134 CassandraDaemon.java
> (line 199) Exception in thread Thread[CompactionExecutor:3152,1,main]
> java.lang.RuntimeException: Last written key
> DecoratedKey(5516674013223138308,
> 001000ff2d161000ff2d1610400100)
> >= current key DecoratedKey(2839992722300822584,
> 001000229ad21000229ad210400100)
> writing into
> {code}
> since we use LeveledScanner when compacting (the backing sstable scanner
> might go beyond the start of the next sstable scanner)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8371) DateTieredCompactionStrategy is always compacting
[
https://issues.apache.org/jira/browse/CASSANDRA-8371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14246079#comment-14246079
]
Jonathan Shook edited comment on CASSANDRA-8371 at 12/14/14 8:16 PM:
-
[~Bj0rn],
The phrase "ideal scheduling" was meant to describe the case in which the data
for each sstable is compacted exactly once per window. In other words, there is
only one coalescing compaction needed for all data in the new interval once a
set of smaller intervals is grouped into a single larger interval. You describe
some of the scenarios which make this more of an ideal than an actuality in
your response above. I understand that the windows are anchored at fixed points
using modulo against the timestamp. The rationale I used above actually depends
on it as an assumption, otherwise you wouldn't be able to achieve ideal
compaction scheduling of "once per interval".
I guess we need to be careful about the terms we use here. I'd favor "fixed
intervals" and "coalescing of fixed intervals". I believe my rationale on
compaction load still makes sense, unless someone has a counter-example or
clarification.
was (Author: jshook):
[~Bj0rn],
The phrase "ideal scheduling" was meant to describe the case in which the data
for each sstable is compacted exactly once per window. In other words, there is
only one coalescing compaction needed for all data once a set of smaller
intervals is grouped into a single larger interval. You describe some of the
scenarios which make this more of an ideal than an actuality in your response
above. I understand that the windows are anchored at fixed points using modulo
against the timestamp. The rationale I used above actually depends on it as an
assumption, otherwise you wouldn't be able to achieve ideal compaction
scheduling of "once per interval".
I guess we need to be careful about the terms we use here. I'd favor "fixed
intervals" and "coalescing of fixed intervals". I believe my rationale on
compaction load still makes sense, unless someone has a counter-example or
clarification.
> DateTieredCompactionStrategy is always compacting
> --
>
> Key: CASSANDRA-8371
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8371
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: mck
>Assignee: Björn Hegerfors
> Labels: compaction, performance
> Attachments: java_gc_counts_rate-month.png,
> read-latency-recommenders-adview.png, read-latency.png,
> sstables-recommenders-adviews.png, sstables.png, vg2_iad-month.png
>
>
> Running 2.0.11 and having switched a table to
> [DTCS|https://issues.apache.org/jira/browse/CASSANDRA-6602] we've seen that
> disk IO and gc count increase, along with the number of reads happening in
> the "compaction" hump of cfhistograms.
> Data, and generally performance, looks good, but compactions are always
> happening, and pending compactions are building up.
> The schema for this is
> {code}CREATE TABLE search (
> loginid text,
> searchid timeuuid,
> description text,
> searchkey text,
> searchurl text,
> PRIMARY KEY ((loginid), searchid)
> );{code}
> We're sitting on about 82G (per replica) across 6 nodes in 4 DCs.
> CQL executed against this keyspace, and traffic patterns, can be seen in
> slides 7+8 of https://prezi.com/b9-aj6p2esft/
> Attached are sstables-per-read and read-latency graphs from cfhistograms, and
> screenshots of our munin graphs as we have gone from STCS, to LCS (week ~44),
> to DTCS (week ~46).
> These screenshots are also found in the prezi on slides 9-11.
> [~pmcfadin], [~Bj0rn],
> Can this be a consequence of occasional deleted rows, as is described under
> (3) in the description of CASSANDRA-6602 ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8371) DateTieredCompactionStrategy is always compacting
[
https://issues.apache.org/jira/browse/CASSANDRA-8371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14246079#comment-14246079
]
Jonathan Shook edited comment on CASSANDRA-8371 at 12/14/14 8:15 PM:
-
[~Bj0rn],
The phrase "ideal scheduling" was meant to describe the case in which the data
for each sstable is compacted exactly once per window. In other words, there is
only one coalescing compaction needed for all data once a set of smaller
intervals is grouped into a single larger interval. You describe some of the
scenarios which make this more of an ideal than an actuality in your response
above. I understand that the windows are anchored at fixed points using modulo
against the timestamp. The rationale I used above actually depends on it as an
assumption, otherwise you wouldn't be able to achieve ideal compaction
scheduling of "once per interval".
I guess we need to be careful about the terms we use here. I'd favor "fixed
intervals" and "coalescing of fixed intervals". I believe my rationale on
compaction load still makes sense, unless someone has a counter-example or
clarification.
was (Author: jshook):
[~Bj0rn],
The phrase "ideal scheduling" was meant to describe the case in which each
sstable is compacted exactly once per window. You describe some of the
scenarios which make this more of an ideal than an actuality in your response
above. I understand that the windows are anchored at fixed points using modulo
against the timestamp. The rationale I used above actually depends on it as an
assumption, otherwise you wouldn't be able to achieve ideal compaction
scheduling of "once per interval".
I guess we need to be careful about the terms we use here. I'd favor "fixed
intervals" and "coalescing of fixed intervals". I believe my rationale on
compaction load still makes sense, unless someone has a counter-example or
clarification.
> DateTieredCompactionStrategy is always compacting
> --
>
> Key: CASSANDRA-8371
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8371
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: mck
>Assignee: Björn Hegerfors
> Labels: compaction, performance
> Attachments: java_gc_counts_rate-month.png,
> read-latency-recommenders-adview.png, read-latency.png,
> sstables-recommenders-adviews.png, sstables.png, vg2_iad-month.png
>
>
> Running 2.0.11 and having switched a table to
> [DTCS|https://issues.apache.org/jira/browse/CASSANDRA-6602] we've seen that
> disk IO and gc count increase, along with the number of reads happening in
> the "compaction" hump of cfhistograms.
> Data, and generally performance, looks good, but compactions are always
> happening, and pending compactions are building up.
> The schema for this is
> {code}CREATE TABLE search (
> loginid text,
> searchid timeuuid,
> description text,
> searchkey text,
> searchurl text,
> PRIMARY KEY ((loginid), searchid)
> );{code}
> We're sitting on about 82G (per replica) across 6 nodes in 4 DCs.
> CQL executed against this keyspace, and traffic patterns, can be seen in
> slides 7+8 of https://prezi.com/b9-aj6p2esft/
> Attached are sstables-per-read and read-latency graphs from cfhistograms, and
> screenshots of our munin graphs as we have gone from STCS, to LCS (week ~44),
> to DTCS (week ~46).
> These screenshots are also found in the prezi on slides 9-11.
> [~pmcfadin], [~Bj0rn],
> Can this be a consequence of occasional deleted rows, as is described under
> (3) in the description of CASSANDRA-6602 ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8371) DateTieredCompactionStrategy is always compacting
[
https://issues.apache.org/jira/browse/CASSANDRA-8371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14246079#comment-14246079
]
Jonathan Shook commented on CASSANDRA-8371:
---
[~Bj0rn],
The phrase "ideal scheduling" was meant to describe the case in which each
sstable is compacted exactly once per window. You describe some of the
scenarios which make this more of an ideal than an actuality in your response
above. I understand that the windows are anchored at fixed points using modulo
against the timestamp. The rationale I used above actually depends on it as an
assumption, otherwise you wouldn't be able to achieve ideal compaction
scheduling of "once per interval".
I guess we need to be careful about the terms we use here. I'd favor "fixed
intervals" and "coalescing of fixed intervals". I believe my rationale on
compaction load still makes sense, unless someone has a counter-example or
clarification.
> DateTieredCompactionStrategy is always compacting
> --
>
> Key: CASSANDRA-8371
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8371
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: mck
>Assignee: Björn Hegerfors
> Labels: compaction, performance
> Attachments: java_gc_counts_rate-month.png,
> read-latency-recommenders-adview.png, read-latency.png,
> sstables-recommenders-adviews.png, sstables.png, vg2_iad-month.png
>
>
> Running 2.0.11 and having switched a table to
> [DTCS|https://issues.apache.org/jira/browse/CASSANDRA-6602] we've seen that
> disk IO and gc count increase, along with the number of reads happening in
> the "compaction" hump of cfhistograms.
> Data, and generally performance, looks good, but compactions are always
> happening, and pending compactions are building up.
> The schema for this is
> {code}CREATE TABLE search (
> loginid text,
> searchid timeuuid,
> description text,
> searchkey text,
> searchurl text,
> PRIMARY KEY ((loginid), searchid)
> );{code}
> We're sitting on about 82G (per replica) across 6 nodes in 4 DCs.
> CQL executed against this keyspace, and traffic patterns, can be seen in
> slides 7+8 of https://prezi.com/b9-aj6p2esft/
> Attached are sstables-per-read and read-latency graphs from cfhistograms, and
> screenshots of our munin graphs as we have gone from STCS, to LCS (week ~44),
> to DTCS (week ~46).
> These screenshots are also found in the prezi on slides 9-11.
> [~pmcfadin], [~Bj0rn],
> Can this be a consequence of occasional deleted rows, as is described under
> (3) in the description of CASSANDRA-6602 ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8261) Clean up schema metadata classes
[ https://issues.apache.org/jira/browse/CASSANDRA-8261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-8261: - Attachment: (was: 8261-isolate-serialization-code-v2.txt) > Clean up schema metadata classes > > > Key: CASSANDRA-8261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8261 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Minor > Fix For: 3.0 > > Attachments: 8261-isolate-hadcoded-system-tables.txt, > 8261-isolate-serialization-code-v2.txt, 8261-isolate-serialization-code.txt, > 8261-isolate-thrift-code.txt > > > While working on CASSANDRA-6717, I've made some general cleanup changes to > schema metadata classes - distracted from the core purpose. Also, being > distracted from it by other things, every time I come back to it gives me a > bit of a rebase hell. > Thus I'm isolating those changes into a separate issue here, hoping to commit > them one by one, before I go back and finalize CASSANDRA-6717. > The changes include: > - moving all the toThrift/fromThrift conversion code to ThriftConversion, > where it belongs > - moving the complied system CFMetaData objects away from CFMetaData (to > SystemKeyspace and TracesKeyspace) > - isolating legacy toSchema/fromSchema code into a separate class > (LegacySchemaTables - former DefsTables) > - refactoring CFMetaData/KSMetaData fields to match CQL CREATE TABLE syntax, > and encapsulating more things in > CompactionOptions/CompressionOptions/ReplicationOptions classes > - moving the definition classes to the new 'schema' package -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8261) Clean up schema metadata classes
[ https://issues.apache.org/jira/browse/CASSANDRA-8261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-8261: - Attachment: 8261-isolate-serialization-code-v2.txt Attaching another rebased version (with aggregates). > Clean up schema metadata classes > > > Key: CASSANDRA-8261 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8261 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Minor > Fix For: 3.0 > > Attachments: 8261-isolate-hadcoded-system-tables.txt, > 8261-isolate-serialization-code-v2.txt, 8261-isolate-serialization-code.txt, > 8261-isolate-thrift-code.txt > > > While working on CASSANDRA-6717, I've made some general cleanup changes to > schema metadata classes - distracted from the core purpose. Also, being > distracted from it by other things, every time I come back to it gives me a > bit of a rebase hell. > Thus I'm isolating those changes into a separate issue here, hoping to commit > them one by one, before I go back and finalize CASSANDRA-6717. > The changes include: > - moving all the toThrift/fromThrift conversion code to ThriftConversion, > where it belongs > - moving the complied system CFMetaData objects away from CFMetaData (to > SystemKeyspace and TracesKeyspace) > - isolating legacy toSchema/fromSchema code into a separate class > (LegacySchemaTables - former DefsTables) > - refactoring CFMetaData/KSMetaData fields to match CQL CREATE TABLE syntax, > and encapsulating more things in > CompactionOptions/CompressionOptions/ReplicationOptions classes > - moving the definition classes to the new 'schema' package -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8477) CMS GC can not recycle objects
[ https://issues.apache.org/jira/browse/CASSANDRA-8477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14245974#comment-14245974 ] Benedict commented on CASSANDRA-8477: - Do you definitely have swap disabled on the box? There were some other slightly less lengthy pauses that were also suspect - anything > 10s when the heap isn't dangerously full is pretty suspect, so if you have some runs with 20s+ pauses with those options enabled, the data could be helpful. > CMS GC can not recycle objects > -- > > Key: CASSANDRA-8477 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8477 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: 2.1.1 or 2.1.2-SNAPSHOT(after CASSANDRA-8459 resolved) >Reporter: Philo Yang > Attachments: cassandra.yaml, histo.txt, jstack.txt, system.log > > > I have a trouble in my cluster that CMS full gc can not reduce the size of > old gen. Days ago I post this problem to the maillist, people think it will > be solved by tuning the gc setting, however it doesn't work for me. > Then I saw a similar bug in CASSANDRA-8447, but [~benedict] think it is not > related. With the jstack on > https://gist.github.com/yangzhe1991/755ea2a10520be1fe59a, [~benedict] find a > bug and resolved it in CASSANDRA-8459. So I build a latest version on 2.1 > branch and run the SNAPSHOT version on the nodes with gc trouble. > However, there is still the gc issue. So I think opening a new tick and post > more information is a good idea. Thanks for helping me. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7708) UDF schema change events/results
[
https://issues.apache.org/jira/browse/CASSANDRA-7708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14245960#comment-14245960
]
Aleksey Yeschenko commented on CASSANDRA-7708:
--
Please hold this back until at least CASSANDRA-8261 is complete. Thanks.
> UDF schema change events/results
>
>
> Key: CASSANDRA-7708
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7708
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Labels: protocolv4
> Fix For: 3.0
>
> Attachments: 7708-1.txt, 7708-2.txt
>
> Time Spent: 2m
> Remaining Estimate: 0h
>
> Schema change notifications for UDF might be interesting for client.
> This covers both - the result of {{CREATE}} + {{DROP}} statements and events.
> Just adding {{FUNCTION}} as a new target for these events breaks previous
> native protocol contract.
> Proposal is to introduce a new target {{FUNCTION}} in native protocol v4.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7124) Use JMX Notifications to Indicate Success/Failure of Long-Running Operations
[
https://issues.apache.org/jira/browse/CASSANDRA-7124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14245940#comment-14245940
]
Rajanarayanan Thottuvaikkatumana commented on CASSANDRA-7124:
-
[~yukim], Please find the commit URL of the Scrub
https://github.com/rnamboodiri/cassandra/commit/8353e5d85ad984a762545c6a8b2958e8ccd14fe3
I have made changes to the test cases and ran them to find success (ant test
-Dtest.name=ScrubTest). Also verified the JMX notifications in the JMX console.
Please review. Thanks
> Use JMX Notifications to Indicate Success/Failure of Long-Running Operations
>
>
> Key: CASSANDRA-7124
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7124
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Tyler Hobbs
>Assignee: Rajanarayanan Thottuvaikkatumana
>Priority: Minor
> Labels: lhf
> Fix For: 3.0
>
> Attachments: 7124-wip.txt, cassandra-trunk-compact-7124.txt,
> cassandra-trunk-decommission-7124.txt
>
>
> If {{nodetool cleanup}} or some other long-running operation takes too long
> to complete, you'll see an error like the one in CASSANDRA-2126, so you can't
> tell if the operation completed successfully or not. CASSANDRA-4767 fixed
> this for repairs with JMX notifications. We should do something similar for
> nodetool cleanup, compact, decommission, move, relocate, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8477) CMS GC can not recycle objects
[ https://issues.apache.org/jira/browse/CASSANDRA-8477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14245935#comment-14245935 ] Philo Yang commented on CASSANDRA-8477: --- The 242970ms pause is very strange and not easy to reproduce. In the last 24 hours, no full gc costs so long like that. Although there are two times that node become unresponsive. And one of them happens when I had disabled autocompaction, so I think disabling autocompaction only slows down the increasing of heap, can not prevent the trouble. > CMS GC can not recycle objects > -- > > Key: CASSANDRA-8477 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8477 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: 2.1.1 or 2.1.2-SNAPSHOT(after CASSANDRA-8459 resolved) >Reporter: Philo Yang > Attachments: cassandra.yaml, histo.txt, jstack.txt, system.log > > > I have a trouble in my cluster that CMS full gc can not reduce the size of > old gen. Days ago I post this problem to the maillist, people think it will > be solved by tuning the gc setting, however it doesn't work for me. > Then I saw a similar bug in CASSANDRA-8447, but [~benedict] think it is not > related. With the jstack on > https://gist.github.com/yangzhe1991/755ea2a10520be1fe59a, [~benedict] find a > bug and resolved it in CASSANDRA-8459. So I build a latest version on 2.1 > branch and run the SNAPSHOT version on the nodes with gc trouble. > However, there is still the gc issue. So I think opening a new tick and post > more information is a good idea. Thanks for helping me. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7985) stress tool doesn't support auth
[ https://issues.apache.org/jira/browse/CASSANDRA-7985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-7985: Attachment: 7985v3.txt Added a v3 patch that adds the auth-provider option allowing 3rd party implementations of com.datastax.driver.core.AuthProvider to be used with the native driver. > stress tool doesn't support auth > > > Key: CASSANDRA-7985 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7985 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Ashic Mahtab >Assignee: T Jake Luciani >Priority: Minor > Fix For: 2.1.3 > > Attachments: 7985.txt, 7985v2.txt, 7985v3.txt > > > stress tool in 2.1 doesn't seem to support username / password authentication > (like cqlsh). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8390) The process cannot access the file because it is being used by another process
[
https://issues.apache.org/jira/browse/CASSANDRA-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14245902#comment-14245902
]
Alexander Radzin commented on CASSANDRA-8390:
-
Joshua McKenzie, thank you for your efforts.
Today I managed to reproduce the issue on other developer's laptop, however
everything worked well on integration machine. Laptops are running windows 7
and 8, integration machine windows 2002, R2.
I played with sys internals and saw that there are 2 processes that act in
cassandra data directory: windows defender and SVN. I stopped both and ran test
again with process monitor from sysinternals and the problem happened again
although the view of process monitor was clear (i.e. only java.exe) was active
in this directory.
Following your suggestion I ran the test again and executed {{hanle}} while
test was running. And here are the results:
{noformat}
C:\Windows\system32>c:\progs\SysinternalsSuite\handle -a
c:\progs\apache-cassandra-2.1.2\data
Handle v3.51
Copyright (C) 1997-2013 Mark Russinovich
Sysinternals - www.sysinternals.com
java.exe pid: 1852 type: File 658:
C:\progs\apache-cassandra-2.1.2\data\commitlog\CommitLog-4-1418558506027.log
java.exe pid: 1852 type: File 6C8:
C:\progs\apache-cassandra-2.1.2\data\commitlog\CommitLog-4-1418558506027.log
java.exe pid: 1852 type: File 740:
C:\progs\apache-cassandra-2.1.2\data\commitlog\CommitLog-4-1418558506028.log
java.exe pid: 1852 type: File 744:
C:\progs\apache-cassandra-2.1.2\data\commitlog\CommitLog-4-1418558506028.log
java.exe pid: 1852 type: File 958:
C:\progs\apache-cassandra-2.1.2\data\data\system\schema_columns-296e9c049bec3085827dc17d3df2122a\system-schema_columns-ka-1-Index.db
java.exe pid: 1852 type: File 968:
C:\progs\apache-cassandra-2.1.2\data\data\system\schema_columns-296e9c049bec3085827dc17d3df2122a\system-schema_columns-ka-1-Data.db
java.exe pid: 1852 type: File 96C:
C:\progs\apache-cassandra-2.1.2\data\data\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-1-Index.db
java.exe pid: 1852 type: File 970:
C:\progs\apache-cassandra-2.1.2\data\data\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-1-Data.db
java.exe pid: 1852 type: File 974:
C:\progs\apache-cassandra-2.1.2\data\data\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-1-Index.db
java.exe pid: 1852 type: File 978:
C:\progs\apache-cassandra-2.1.2\data\data\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-1-Data.db
C:\Windows\system32>
C:\Windows\system32>
C:\Windows\system32>
C:\Windows\system32>
C:\Windows\system32>
C:\Windows\system32>
C:\Windows\system32>
C:\Windows\system32>
C:\Windows\system32>
C:\Windows\system32>c:\progs\SysinternalsSuite\handle -a
c:\progs\apache-cassandra-2.1.2\data
Handle v3.51
Copyright (C) 1997-2013 Mark Russinovich
Sysinternals - www.sysinternals.com
java.exe pid: 1852 type: File 658:
C:\progs\apache-cassandra-2.1.2\data\commitlog\CommitLog-4-1418558506027.log
java.exe pid: 1852 type: File 6C8:
C:\progs\apache-cassandra-2.1.2\data\commitlog\CommitLog-4-1418558506027.log
java.exe pid: 1852 type: File 740:
C:\progs\apache-cassandra-2.1.2\data\commitlog\CommitLog-4-1418558506028.log
java.exe pid: 1852 type: File 744:
C:\progs\apache-cassandra-2.1.2\data\commitlog\CommitLog-4-1418558506028.log
java.exe pid: 1852 type: File 958:
C:\progs\apache-cassandra-2.1.2\data\data\system\schema_columns-296e9c049bec3085827dc17d3df2122a\system-schema_columns-ka-26-Data.db
java.exe pid: 1852 type: File 97C:
C:\progs\apache-cassandra-2.1.2\data\data\system\local-7ad54392bcdd35a684174e047860b377\system-local-ka-5-Data.db
java.exe pid: 1852 type: File B74:
C:\progs\apache-cassandra-2.1.2\data\data\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-29-Data.db
java.exe pid: 1852 type: File B7C:
C:\progs\apache-cassandra-2.1.2\data\data\system\schema_columns-296e9c049bec3085827dc17d3df2122a\system-schema_columns-ka-26-Index.db
java.exe pid: 1852 type: File BA0:
C:\progs\apache-cassandra-2.1.2\data\data\system\schema_columns-296e9c049bec3085827dc17d3df2122a\system-schema_columns-ka-27-Data.db
java.exe pid: 1852 type: File BA4:
C:\progs\apache-cassandra-2.1.2\data\data\system\schema_columns-296e9c049bec3085827dc17d3df2122a\system-schema_columns-ka-25-Data.db
java.exe pid: 1852 type: File B
