[jira] [Commented] (CASSANDRA-5708) Add DELETE ... IF EXISTS to CQL3
[
https://issues.apache.org/jira/browse/CASSANDRA-5708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931506#comment-13931506
]
Sylvain Lebresne commented on CASSANDRA-5708:
-
I would maybe tend with the latter behavior too, as making 'IF EXISTS' mean 'IF
the row EXISTS' in all cases feels a bit more consistent. Besides, we should
probably add non-equal conditions in the short term, at which point the former
behavior will be expressible by:
{noformat}
DELETE v FROM foo WHERE k=0 IF v != null
{noformat}
Add DELETE ... IF EXISTS to CQL3
Key: CASSANDRA-5708
URL: https://issues.apache.org/jira/browse/CASSANDRA-5708
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Tyler Hobbs
Priority: Minor
Fix For: 2.0.7
I've been slightly lazy in CASSANDRA-5443 and didn't added a {{DELETE .. IF
EXISTS}} syntax to CQL because it wasn't immediately clear what was the
correct condition to use for the IF EXISTS. But at least for CQL3 tables,
this is in fact pretty easy to do using the row marker so we should probably
add it.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-6839) Support non equal conditions (for LWT)
Sylvain Lebresne created CASSANDRA-6839:
---
Summary: Support non equal conditions (for LWT)
Key: CASSANDRA-6839
URL: https://issues.apache.org/jira/browse/CASSANDRA-6839
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Tyler Hobbs
Priority: Minor
Fix For: 2.0.7
We currently only support equal conditions in conditional updates, but it would
be relatively trivial to support non-equal ones as well. At the very least we
should support '', '=', '' and '=', though it would probably also make
sense to add a non-equal relation too ('!=').
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-6766) allow now() - uuid compatibility
[ https://issues.apache.org/jira/browse/CASSANDRA-6766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-6766: Assignee: Tyler Hobbs (was: Sylvain Lebresne) allow now() - uuid compatibility - Key: CASSANDRA-6766 URL: https://issues.apache.org/jira/browse/CASSANDRA-6766 Project: Cassandra Issue Type: Bug Components: API Reporter: Jonathan Ellis Assignee: Tyler Hobbs Priority: Minor Fix For: 2.0.7 Bad Request: Type error: cannot assign result of function now (type timeuuid) to id (type uuid) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6774) Cleanup fails with assertion error after stopping previous run
[ https://issues.apache.org/jira/browse/CASSANDRA-6774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931538#comment-13931538 ] Viktor Jevdokimov commented on CASSANDRA-6774: -- Same issue on LCS, but restarting doesn't help. The only way to remove data after decreasing RF is Cleanup, but it fails with the same exception. Is there a way to temporarily disable compactions on LCS? Cleanup fails with assertion error after stopping previous run -- Key: CASSANDRA-6774 URL: https://issues.apache.org/jira/browse/CASSANDRA-6774 Project: Cassandra Issue Type: Bug Components: Core Environment: 2.0.5 Reporter: Keith Wright I am stress testing a new 2.0.5 cluster and did the following: - start decommission during heavy write, moderate read load - trigger cleanup on non-decommissioning node (nodetool cleanup) - Started to see higher GC load stop stopped cleanup via nodetool stop CLEANUP - attempt to launch cleanup now fails with the following message in console. Cassandra log shows: WARN 18:45:38,420 Unable to cancel in-progress compactions for shard_user_lookup. Probably there is an unusually large row in progress somewhere. It is also possible that buggy code left some sstables compacting after it was done with them Exception in thread main java.lang.AssertionError: [SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6129-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6938-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6698-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6884-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6854-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6898-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6522-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6692-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6815-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6677-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6917-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6929-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-7048-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6911-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6876-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-7046-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6762-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6712-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6906-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6886-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-7053-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6696-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6964-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-7043-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6983-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6672-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6714-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6992-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6234-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6971-Data.db'),
[jira] [Assigned] (CASSANDRA-6745) Require specifying rows_per_partition_to_cache
[ https://issues.apache.org/jira/browse/CASSANDRA-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne reassigned CASSANDRA-6745: --- Assignee: Sylvain Lebresne (was: Marcus Eriksson) Require specifying rows_per_partition_to_cache -- Key: CASSANDRA-6745 URL: https://issues.apache.org/jira/browse/CASSANDRA-6745 Project: Cassandra Issue Type: Bug Reporter: Jonathan Ellis Assignee: Sylvain Lebresne Priority: Trivial Fix For: 2.1 beta2 Attachments: 0001-wip-caching-options.patch We should require specifying rows_to_cache_per_partition for new tables or newly ALTERed when row caching is enabled. Pre-upgrade should be grandfathered in as ALL to match existing semantics. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6745) Require specifying rows_per_partition_to_cache
[ https://issues.apache.org/jira/browse/CASSANDRA-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-6745: Assignee: Marcus Eriksson (was: Sylvain Lebresne) Require specifying rows_per_partition_to_cache -- Key: CASSANDRA-6745 URL: https://issues.apache.org/jira/browse/CASSANDRA-6745 Project: Cassandra Issue Type: Bug Reporter: Jonathan Ellis Assignee: Marcus Eriksson Priority: Trivial Fix For: 2.1 beta2 Attachments: 0001-wip-caching-options.patch We should require specifying rows_to_cache_per_partition for new tables or newly ALTERed when row caching is enabled. Pre-upgrade should be grandfathered in as ALL to match existing semantics. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6745) Require specifying rows_per_partition_to_cache
[ https://issues.apache.org/jira/browse/CASSANDRA-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931557#comment-13931557 ] Sylvain Lebresne commented on CASSANDRA-6745: - Mostly look good to me, though one last point is that since the new ''caching syntax beaks the existing one, I think we'd need to still support the old syntax too at least for 2.1 (in CQL3 I mean), with maybe a warning in the log if said old syntax is used. Otherwise this could be annoying for rolling upgrades. Another nit is that I'd leave CQL2 alone. It will be removed in 3.0 so the only thing someone that still use CQL2 in 2.1 is update to CQL3. So changing the syntax of caching there would just be an annoyance, and I wouldn't even bother with supporting the new syntax: again, if you want to use the new caching features and you're on CQL2, it's high time for you to upgrade to CQL3. Require specifying rows_per_partition_to_cache -- Key: CASSANDRA-6745 URL: https://issues.apache.org/jira/browse/CASSANDRA-6745 Project: Cassandra Issue Type: Bug Reporter: Jonathan Ellis Assignee: Marcus Eriksson Priority: Trivial Fix For: 2.1 beta2 Attachments: 0001-wip-caching-options.patch We should require specifying rows_to_cache_per_partition for new tables or newly ALTERed when row caching is enabled. Pre-upgrade should be grandfathered in as ALL to match existing semantics. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6774) Cleanup fails with assertion error after stopping previous run
[ https://issues.apache.org/jira/browse/CASSANDRA-6774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931563#comment-13931563 ] Dmitrij Koniajev commented on CASSANDRA-6774: - Same issue here on LCS - after decreasing RF cleanup fails with the same stacktrace. Restart didn't help, after running nodetool disableautocompaction command and waiting for compaction get finished cleanup fails with the following stacktrace: Error occurred during cleanup java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1 at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:188) at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:227) at org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:265) at org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1115) at org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2152) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75) at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237) at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487) at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420) at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848) at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322) at sun.rmi.transport.Transport$1.run(Transport.java:177) at sun.rmi.transport.Transport$1.run(Transport.java:174) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:173) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1 at java.util.ArrayList.rangeCheck(ArrayList.java:635) at java.util.ArrayList.get(ArrayList.java:411) at org.apache.cassandra.db.compaction.CompactionManager.needsCleanup(CompactionManager.java:502) at org.apache.cassandra.db.compaction.CompactionManager.doCleanupCompaction(CompactionManager.java:540) at org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:62) at org.apache.cassandra.db.compaction.CompactionManager$5.perform(CompactionManager.java:274) at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:222) at java.util.concurrent.FutureTask.run(FutureTask.java:262) ... 3 more Cleanup fails with assertion error after stopping previous
[jira] [Commented] (CASSANDRA-6826) Query returns different number of results depending on fetchsize
[ https://issues.apache.org/jira/browse/CASSANDRA-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931569#comment-13931569 ] Sylvain Lebresne commented on CASSANDRA-6826: - [~wtmitchell3] Did you have a change to test against 2.0.6 yet, and if not, would it be possible for you to give it a shot? I'm wondering if that couldn't be CASSANDRA-6748. Query returns different number of results depending on fetchsize Key: CASSANDRA-6826 URL: https://issues.apache.org/jira/browse/CASSANDRA-6826 Project: Cassandra Issue Type: Bug Components: Core Environment: quad-core Windows 7 x64, single node cluster Cassandra 2.0.5 Reporter: Bill Mitchell Assignee: Sylvain Lebresne I issue a query across the set of partitioned wide rows for one logical row, where s, l, and partition specify the composite primary key for the row: SELECT ec, ea, rd FROM sr WHERE s = ? and partition IN ? and l = ? ALLOW FILTERING; If I set fetchSize to only 1000 when the Cluster is configured, the query sometimes does not return all the results. In the particular case I am chasing, it returns a total of 98586 rows. If I increase the fetchsize to 10, all the 9 actual rows are returned. This suggests there is some problem with fetchsize re-establishing the position on the next segment of the result set, at least when multiple partitions are being accessed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6311) Add CqlRecordReader to take advantage of native CQL pagination
[
https://issues.apache.org/jira/browse/CASSANDRA-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931573#comment-13931573
]
Piotr Kołaczkowski commented on CASSANDRA-6311:
---
Now it looks better, but I don't get why you need this magic constant here:
{noformat}
if (count 2)
{noformat}
Why not just create a list of (origHost, liveRemoteHost1, liveRemoteHost2, ...,
liveRemoteHostN) and return an iterator to it?
To avoid creating a new list just to get an iterator, you can use
Iterators.concat:
{noformat}
return Iterators.concat(Collections.singletonList(origHost).iterator(),
liveRemoteHosts.iterator());
{noformat}
Add CqlRecordReader to take advantage of native CQL pagination
--
Key: CASSANDRA-6311
URL: https://issues.apache.org/jira/browse/CASSANDRA-6311
Project: Cassandra
Issue Type: New Feature
Components: Hadoop
Reporter: Alex Liu
Assignee: Alex Liu
Fix For: 2.0.7
Attachments: 6311-v3-2.0-branch.txt, 6311-v4.txt,
6311-v5-2.0-branch.txt, 6311-v6-2.0-branch.txt, 6311-v7.txt,
6331-2.0-branch.txt, 6331-v2-2.0-branch.txt
Since the latest Cql pagination is done and it should be more efficient, so
we need update CqlPagingRecordReader to use it instead of the custom thrift
paging.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6827) Using static with a counter column is broken
[
https://issues.apache.org/jira/browse/CASSANDRA-6827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931574#comment-13931574
]
Sylvain Lebresne commented on CASSANDRA-6827:
-
+1
Using static with a counter column is broken
Key: CASSANDRA-6827
URL: https://issues.apache.org/jira/browse/CASSANDRA-6827
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Jeremiah Jordan
Assignee: Aleksey Yeschenko
Priority: Minor
Fix For: 2.0.7
Attachments: 6827.txt
From a client:
Looks like there is a problem with serialisation//deserialisation of static
counters.
It reproduces on the 2.0 branch, but doesn't appear to be a problem on trunk.
Here is a small cql script which reproduces the problem:
CREATE KEYSPACE ks1 WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 1};
use ks1;
CREATE COLUMNFAMILY problematic ( stat_cnt counter static, norm_cnt counter,
pk text, ck text, PRIMARY KEY (pk, ck) );
UPDATE problematic SET stat_cnt = stat_cnt + 1 WHERE pk = 'foo';
SELECT * FROM problematic ;
and an example of the error from cqlsh:
cqlsh:ks1 SELECT * FROM problematic ;
pk | ck | stat_cnt | norm_cnt
-+--+--+--
foo | null |
'\x00\x01\x00\x00\xf1$\t`\xa6\x1d\x11\xe3\x82\x9a\x9fqi\x80\x1f\xec\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x01'
| null
(1 rows)
Failed to decode value
'\x00\x01\x00\x00\xf1$\t`\xa6\x1d\x11\xe3\x82\x9a\x9fqi\x80\x1f\xec\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x01'
(for column 'stat_cnt') as counter: unpack requires a string argument of
length 8
It seems like the problem lies in SelectStatement, it doesn't resolve the
counter and ends up sending the entire 36 bytes of raw counter rather than
the 8 bytes.
In select statement we seem to keep the bytebuffer and then in two different
places we end up calling the following method:
Selection.ResultSetBuilder.add(ByteBuffer v )
rather than: add(Column c)
(Which in turn calls value() and invokes the CounterContext)
I suppose the fix is just to ensure that the CounterContext is invoked for
static counters.
I can see various ways and places of putting it in place.
Presumably you'd want to reconcile the static counters just once, earlier and
keep the resolved value around to be ouput in several rows.
I suppose somewhere here either by extending the ternary or changing
getSimpleValue :
staticValues.put(name, name.type.isCollection() ? getCollectionValue(name,
group) : getSimpleValue(name, group));
ColumnGroupMap seems to contain the relevant byte buffers and has methods:
getCollection
getSimple
which could be extended to contain some counter related call wrapped in
SelectStatement similarly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-4165) Generate Digest file for compressed SSTables
[ https://issues.apache.org/jira/browse/CASSANDRA-4165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931589#comment-13931589 ] Marcus Eriksson commented on CASSANDRA-4165: rebased and a fix here: https://github.com/krummas/cassandra/commits/jbellis/4165-3 (the Digest component for compressed files was not written) Generate Digest file for compressed SSTables Key: CASSANDRA-4165 URL: https://issues.apache.org/jira/browse/CASSANDRA-4165 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Marcus Eriksson Assignee: Jonathan Ellis Priority: Minor Labels: performance Fix For: 2.1 beta2 Attachments: 0001-Generate-digest-for-compressed-files-as-well.patch, 0002-dont-do-crc-and-add-digests-for-compressed-files.txt, 4165-rebased.txt We use the generated *Digest.sha1-files to verify backups, would be nice if they were generated for compressed sstables as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6553) Benchmark counter improvements (counters++)
[ https://issues.apache.org/jira/browse/CASSANDRA-6553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931588#comment-13931588 ] Benedict commented on CASSANDRA-6553: - FTR, if you want to run a mixed or read workload with counters, make sure you apply CASSANDRA-6835 Benchmark counter improvements (counters++) --- Key: CASSANDRA-6553 URL: https://issues.apache.org/jira/browse/CASSANDRA-6553 Project: Cassandra Issue Type: Test Reporter: Ryan McGuire Assignee: Russ Hatch Fix For: 2.1 beta2 Benchmark the difference in performance between CASSANDRA-6504 and trunk. * Updating totally unrelated counters (different partitions) * Updating the same counters a lot (same cells in the same partition) * Different cells in the same few partitions (hot counter partition) benchmark: https://github.com/apache/cassandra/tree/1218bcacba7edefaf56cf8440d0aea5794c89a1e (old counters) compared to: https://github.com/apache/cassandra/tree/714c423360c36da2a2b365efaf9c5c4f623ed133 (new counters) So far, the above changes should only affect the write path. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (CASSANDRA-6780) Memtable OffHeap GC Statistics
[ https://issues.apache.org/jira/browse/CASSANDRA-6780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict reassigned CASSANDRA-6780: --- Assignee: (was: Benedict) Memtable OffHeap GC Statistics -- Key: CASSANDRA-6780 URL: https://issues.apache.org/jira/browse/CASSANDRA-6780 Project: Cassandra Issue Type: Improvement Reporter: Benedict Priority: Minor As mentioned in CASSANDRA-6689, it would be nice to expose via JMX some statistics on GC behaviour, instead of just optionally debug logging it (and maybe expand to cover some more things): - Time spent in GC - Amount of memory reclaimed - Number of collections (per CFS?), and average reclaimed per collection -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (CASSANDRA-5978) stressd broken by ClientEncriptionOptions
[
https://issues.apache.org/jira/browse/CASSANDRA-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benedict resolved CASSANDRA-5978.
-
Resolution: Fixed
Fix Version/s: 2.1
stressd broken by ClientEncriptionOptions
-
Key: CASSANDRA-5978
URL: https://issues.apache.org/jira/browse/CASSANDRA-5978
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Jeremiah Jordan
Assignee: Benedict
Priority: Minor
Fix For: 2.1
The ClientEncryptionOptions object added to
org.apache.cassandra.stress.Session is not Serializable. So if you try to
use stress with stressd, the Session can't be serialized to be passed over to
stressd:
{noformat}
Exception in thread main java.io.NotSerializableException:
org.apache.cassandra.config.EncryptionOptions$ClientEncryptionOptions
at java.io.ObjectOutputStream.writeObject0(Unknown Source)
at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source)
at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
at java.io.ObjectOutputStream.writeObject0(Unknown Source)
at java.io.ObjectOutputStream.writeObject(Unknown Source)
at org.apache.cassandra.stress.Stress.main(Unknown Source)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-6840) 2.0.{5,6} node throws EOFExceptions on the Row mutation forwarding path during rolling upgrade from 1.2.15.
Federico Piccinini created CASSANDRA-6840:
-
Summary: 2.0.{5,6} node throws EOFExceptions on the Row mutation
forwarding path during rolling upgrade from 1.2.15.
Key: CASSANDRA-6840
URL: https://issues.apache.org/jira/browse/CASSANDRA-6840
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Federico Piccinini
During a rolling upgrade from 1.2.15 to 2.0.5 nodes running on 2.0.5 throw an
EOFException:
{noformat}
ERROR [MutationStage:12] 2014-03-12 09:46:35,706 RowMutationVerbHandler.java
(line 63) Error in row mutation
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:197)
at
org.apache.cassandra.net.CompactEndpointSerializationHelper.deserialize(CompactEndpointSerializationHelper.java:37)
at
org.apache.cassandra.db.RowMutationVerbHandler.forwardToLocalNodes(RowMutationVerbHandler.java:81)
at
org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:49)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
In this specific context we have a setup with 3 datacenters, 3 nodes in each
datacenter, NetworkTopologyStrategy as placement_strategy with 3 replicas in
each DC. We noticed the issue on the only 2.0.5 node in the ring. All nodes run
on Java7. We have tried to upgrade the node on 2.0.5 to 2.0.6 but that didn't
solve the issue.
At a first glance it seems that the size of the size of the list of forward
addresses in
org.apache.cassandra.db.RowMutationVerbHandler.forwardToLocalNodes() in
inconsistent with the length of the InputStream, which causes the deserializer
to try and read after the end of the InputStream.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6307) Switch cqlsh from cassandra-dbapi2 to python-driver
[ https://issues.apache.org/jira/browse/CASSANDRA-6307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931624#comment-13931624 ] Aleksey Yeschenko commented on CASSANDRA-6307: -- +1. Bump the major on commit to 5.0.0, though, if you don't mind. Switch cqlsh from cassandra-dbapi2 to python-driver --- Key: CASSANDRA-6307 URL: https://issues.apache.org/jira/browse/CASSANDRA-6307 Project: Cassandra Issue Type: Improvement Reporter: Aleksey Yeschenko Assignee: Mikhail Stepura Priority: Minor Fix For: 2.1 beta2 python-driver is hitting 1.0 soon. cassandra-dbapi2 development has stalled. It's time to switch cqlsh to native protocol and cassandra-dbapi2, especially now that 1. Some CQL3 things are not supported by Thrift transport 2. cqlsh no longer has to support CQL2 (dropped in 2.0) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6285) 2.0 HSHA server introduces corrupt data
[ https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931627#comment-13931627 ] Viktor Kuzmin commented on CASSANDRA-6285: -- I've tried my test with beta1 and I can confirm that I was not able to reproduce bug. I think that it will be better to not use disruptor on 2.0.x even with on-heap allocation (we can still reuse buffer in case message will be of equal size when previous). And it should be safe to use disruptor on 2.1 branch. We'll be waiting for 2.1 release, cause it really impressed me over 2.0 2.0 HSHA server introduces corrupt data --- Key: CASSANDRA-6285 URL: https://issues.apache.org/jira/browse/CASSANDRA-6285 Project: Cassandra Issue Type: Bug Components: Core Environment: 4 nodes, shortly updated from 1.2.11 to 2.0.2 Reporter: David Sauer Assignee: Pavel Yaskevich Priority: Critical Fix For: 2.0.6 Attachments: 6285_testnotes1.txt, CASSANDRA-6285-disruptor-heap.patch, cassandra-attack-src.zip, compaction_test.py, disruptor-high-cpu.patch, disruptor-memory-corruption.patch After altering everything to LCS the table OpsCenter.rollups60 amd one other none OpsCenter-Table got stuck with everything hanging around in L0. The compaction started and ran until the logs showed this: ERROR [CompactionExecutor:111] 2013-11-01 19:14:53,865 CassandraDaemon.java (line 187) Exception in thread Thread[CompactionExecutor:111,1,RMI Runtime] java.lang.RuntimeException: Last written key DecoratedKey(1326283851463420237, 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574426c6f6f6d46696c746572537061636555736564) = current key DecoratedKey(954210699457429663, 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574546f74616c4469736b5370616365557365640b0f) writing into /var/lib/cassandra/data/OpsCenter/rollups60/OpsCenter-rollups60-tmp-jb-58656-Data.db at org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:141) at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:164) at org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:160) at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60) at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) at org.apache.cassandra.db.compaction.CompactionManager$6.runMayThrow(CompactionManager.java:296) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Moving back to STC worked to keep the compactions running. Especialy my own Table i would like to move to LCS. After a major compaction with STC the move to LCS fails with the same Exception. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-6841) ConcurrentModificationException in commit-log-writer after localschemareset
Pas created CASSANDRA-6841:
--
Summary: ConcurrentModificationException in commit-log-writer
after localschemareset
Key: CASSANDRA-6841
URL: https://issues.apache.org/jira/browse/CASSANDRA-6841
Project: Cassandra
Issue Type: Bug
Environment: Linux 3.2.0 (Debian Wheezy) Cassandra 2.0.6, Oracle JVM
1.7.0_51
Almost default cassandra.yaml (IPs and cluster name changed)
This is the 2nd node in a 2-node ring. It has ~2500 keyspaces and very low
traffic. (Only new keyspaces see reads and writes.)
Reporter: Pas
Priority: Minor
{code}
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,013
MigrationManager.java (line 329) Starting local schema reset...
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,016
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@394448776(114/1140 serialized/live bytes, 3 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,016 Memtable.java (line 331) Writing
Memtable-local@394448776(114/1140 serialized/live bytes, 3 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,182 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-398-Data.db (145 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33159822)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,185
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@1087210140(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,185 Memtable.java (line 331) Writing
Memtable-local@1087210140(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,357 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-399-Data.db (96 bytes) for
commitlog position ReplayPosition(segmentId=1394620057452, position=33159959)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,361
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@768887091(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,361 Memtable.java (line 331) Writing
Memtable-local@768887091(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,516 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-400-Data.db (96 bytes) for
commitlog position ReplayPosition(segmentId=1394620057452, position=33160096)
INFO [CompactionExecutor:38] 2014-03-12 11:37:54,517 CompactionTask.java (line
115) Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-398-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-400-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-399-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-397-Data.db')]
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,519
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@271993477(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,519 Memtable.java (line 331) Writing
Memtable-local@271993477(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,794 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-401-Data.db (96 bytes) for
commitlog position ReplayPosition(segmentId=1394620057452, position=33160233)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,799
MigrationManager.java (line 357) Local schema reset is complete.
INFO [CompactionExecutor:38] 2014-03-12 11:37:54,848 CompactionTask.java (line
275) Compacted 4 sstables to
[/var/lib/cassandra/data/system/local/system-local-jb-402,]. 6,099 bytes to
5,821 (~95% of original) in 330ms = 0.016822MB/s. 4 total partitions merged to
1. Partition merge counts were {4:1, }
INFO [OptionalTasks:1] 2014-03-12 11:37:55,110 ColumnFamilyStore.java (line
785) Enqueuing flush of Memtable-schema_columnfamilies@106276050(181506/509164
serialized/live bytes, 3276 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:55,110 Memtable.java (line 331) Writing
Memtable-schema_columnfamilies@106276050(181506/509164 serialized/live bytes,
3276 ops)
INFO [OptionalTasks:1] 2014-03-12 11:37:55,110 ColumnFamilyStore.java (line
785) Enqueuing flush of Memtable-schema_columns@252242773(185191/630698
serialized/live bytes, 3614 ops)
ERROR [COMMIT-LOG-WRITER] 2014-03-12 11:37:55,111 CassandraDaemon.java (line
196) Exception in thread Thread[COMMIT-LOG-WRITER,5,main]
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926)
at java.util.HashMap$KeyIterator.next(HashMap.java:960)
at
[jira] [Commented] (CASSANDRA-6146) CQL-native stress
[
https://issues.apache.org/jira/browse/CASSANDRA-6146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931630#comment-13931630
]
Benedict commented on CASSANDRA-6146:
-
Sorry for the glacial response on this [~mishail]. I've setup my JIRA search to
include items awaiting my review so it shouldn't happen again.
First impressions:
1) The installation instructions aren't as obvious as they could be. I tried a
variety of fairly stupid things because it didn't make it clear (to me) exactly
which releases I should be downloading/unpacking where. Would be good to state
unambiguously that you should download the CqlJMeter binary release, and a
JMeter binary release, and that tar -xjf \{CqlJMeter.tgz\} -C \{Unpacked JMeter
Root Directory\} will get everything working.
2) There is no support for prepared statements.
3) If I simply tweak the example write plan, and have it run 10k samples per
thread, it pretty much immediately exhausts C* of its available file handles.
Now, by default on my system the number of available file handles is not very
many, but something is going wrong if I can exhaust them with only the default
10 threads configured (cassandra-stress does not exhaust them with hundreds of
threads).
4) It would be good to have the examples unpack to one of JMeter's
examples/templates directory. You see a mongodb template in there straight away
(bundled), when you click open, so it would be nice to have Cassandra there
without having to go hunting somewhere else for it.
2 and 3 are pretty much show stoppers for me. Once they're sorted, it might be
worth trying to package it with JMeter, since mongodb is.
Personally I also don't find JMeter to be an easy or intuitive way to get
running quickly with stress testing, but that's a completely separate matter,
and your bundled plan does make it much more straight forward than it would
otherwise be. It looks like it is quite powerful and certainly very expressive,
so for serious/complex stress testing and benchmarking it looks like it could
be a great tool.
CQL-native stress
-
Key: CASSANDRA-6146
URL: https://issues.apache.org/jira/browse/CASSANDRA-6146
Project: Cassandra
Issue Type: New Feature
Components: Tools
Reporter: Jonathan Ellis
Fix For: 2.1 beta2
The existing CQL support in stress is not worth discussing. We need to
start over, and we might as well kill two birds with one stone and move to
the native protocol while we're at it.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-6146) CQL-native stress
[
https://issues.apache.org/jira/browse/CASSANDRA-6146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931630#comment-13931630
]
Benedict edited comment on CASSANDRA-6146 at 3/12/14 10:51 AM:
---
Sorry for the glacial response on this [~mishail]. I've setup my JIRA search to
include items awaiting my review so it shouldn't happen again.
First impressions:
# The installation instructions aren't as obvious as they could be. I tried a
variety of fairly stupid things because it didn't make it clear (to me) exactly
which releases I should be downloading/unpacking where. Would be good to state
unambiguously that you should download the CqlJMeter binary release, and a
JMeter binary release, and that tar -xjf \{CqlJMeter.tgz\} -C \{Unpacked JMeter
Root Directory\} will get everything working.
# There is no support for prepared statements.
# If I simply tweak the example write plan, and have it run 10k samples per
thread, it pretty much immediately exhausts C* of its available file handles.
Now, by default on my system the number of available file handles is not very
many, but something is going wrong if I can exhaust them with only the default
10 threads configured (cassandra-stress does not exhaust them with hundreds of
threads).
# It would be good to have the examples unpack to one of JMeter's
examples/templates directory. You see a mongodb template in there straight away
(bundled), when you click open, so it would be nice to have Cassandra there
without having to go hunting somewhere else for it.
2 and 3 are pretty much show stoppers for me. Once they're sorted, it might be
worth trying to package it with JMeter, since mongodb is.
Personally I also don't find JMeter to be an easy or intuitive way to get
running quickly with stress testing, but that's a completely separate matter,
and your bundled plan does make it much more straight forward than it would
otherwise be. It looks like it is quite powerful and certainly very expressive,
so for serious/complex stress testing and benchmarking it looks like it could
be a great tool.
was (Author: benedict):
Sorry for the glacial response on this [~mishail]. I've setup my JIRA search to
include items awaiting my review so it shouldn't happen again.
First impressions:
1) The installation instructions aren't as obvious as they could be. I tried a
variety of fairly stupid things because it didn't make it clear (to me) exactly
which releases I should be downloading/unpacking where. Would be good to state
unambiguously that you should download the CqlJMeter binary release, and a
JMeter binary release, and that tar -xjf \{CqlJMeter.tgz\} -C \{Unpacked JMeter
Root Directory\} will get everything working.
2) There is no support for prepared statements.
3) If I simply tweak the example write plan, and have it run 10k samples per
thread, it pretty much immediately exhausts C* of its available file handles.
Now, by default on my system the number of available file handles is not very
many, but something is going wrong if I can exhaust them with only the default
10 threads configured (cassandra-stress does not exhaust them with hundreds of
threads).
4) It would be good to have the examples unpack to one of JMeter's
examples/templates directory. You see a mongodb template in there straight away
(bundled), when you click open, so it would be nice to have Cassandra there
without having to go hunting somewhere else for it.
2 and 3 are pretty much show stoppers for me. Once they're sorted, it might be
worth trying to package it with JMeter, since mongodb is.
Personally I also don't find JMeter to be an easy or intuitive way to get
running quickly with stress testing, but that's a completely separate matter,
and your bundled plan does make it much more straight forward than it would
otherwise be. It looks like it is quite powerful and certainly very expressive,
so for serious/complex stress testing and benchmarking it looks like it could
be a great tool.
CQL-native stress
-
Key: CASSANDRA-6146
URL: https://issues.apache.org/jira/browse/CASSANDRA-6146
Project: Cassandra
Issue Type: New Feature
Components: Tools
Reporter: Jonathan Ellis
Fix For: 2.1 beta2
The existing CQL support in stress is not worth discussing. We need to
start over, and we might as well kill two birds with one stone and move to
the native protocol while we're at it.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6146) CQL-native stress
[ https://issues.apache.org/jira/browse/CASSANDRA-6146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931632#comment-13931632 ] Benedict commented on CASSANDRA-6146: - Hmm. Also, I didn't realise at all but all of those queries were failing because the keyspace create didn't succeed. Which leads me to question how we're reporting errors back. The litle number in the top-right was incrementing, but I didn't realise this meant there were any errors as the sample output listed all of the runs as a measurement. Also, once I opened the error log it was completely empty (until I tried running again with it open, at which point it starts filling up with errors) This could lead to very wrong utilisation, and no one being any the wiser. Ideally the regular output would discount errors from the samples, and report them separately. CQL-native stress - Key: CASSANDRA-6146 URL: https://issues.apache.org/jira/browse/CASSANDRA-6146 Project: Cassandra Issue Type: New Feature Components: Tools Reporter: Jonathan Ellis Fix For: 2.1 beta2 The existing CQL support in stress is not worth discussing. We need to start over, and we might as well kill two birds with one stone and move to the native protocol while we're at it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6746) Reads have a slow ramp up in speed
[ https://issues.apache.org/jira/browse/CASSANDRA-6746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6746: Attachment: 6746.txt Attaching a trivial patch that changes the default behaviour to TRUE for populate_io_cache_on_flush Reads have a slow ramp up in speed -- Key: CASSANDRA-6746 URL: https://issues.apache.org/jira/browse/CASSANDRA-6746 Project: Cassandra Issue Type: Bug Components: Core Reporter: Ryan McGuire Assignee: Benedict Labels: performance Fix For: 2.1 beta2 Attachments: 2.1_vs_2.0_read.png, 6746.txt, cassandra-2.0-bdplab-trial-fincore.tar.bz2, cassandra-2.1-bdplab-trial-fincore.tar.bz2 On a physical four node cluister I am doing a big write and then a big read. The read takes a long time to ramp up to respectable speeds. !2.1_vs_2.0_read.png! [See data here|http://ryanmcguire.info/ds/graph/graph.html?stats=stats.2.1_vs_2.0_vs_1.2.retry1.jsonmetric=interval_op_rateoperation=stress-readsmoothing=1] -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6689) Partially Off Heap Memtables
[ https://issues.apache.org/jira/browse/CASSANDRA-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931656#comment-13931656 ] Benedict commented on CASSANDRA-6689: - Pushed an updated version with your changes merged, the cassandra.yaml reverted, and a theoretical deadlock risk eliminated in KeysSearcher and CompositesSearcher Partially Off Heap Memtables Key: CASSANDRA-6689 URL: https://issues.apache.org/jira/browse/CASSANDRA-6689 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Benedict Assignee: Benedict Fix For: 2.1 beta2 Attachments: CASSANDRA-6689-small-changes.patch Move the contents of ByteBuffers off-heap for records written to a memtable. (See comments for details) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (CASSANDRA-6557) CommitLogSegment may be duplicated in unlikely race scenario
[ https://issues.apache.org/jira/browse/CASSANDRA-6557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict reassigned CASSANDRA-6557: --- Assignee: Benedict CommitLogSegment may be duplicated in unlikely race scenario Key: CASSANDRA-6557 URL: https://issues.apache.org/jira/browse/CASSANDRA-6557 Project: Cassandra Issue Type: Bug Components: Core Environment: 2.1 Reporter: Benedict Assignee: Benedict Priority: Minor Fix For: 2.1 beta2 In the unlikely event that the thread that switched to a new CLS has not finished executing the cleanup of its switch by the time the CLS has finished being used, it is possible for the same segment to be 'switched' in again. This would be benign except that it is added to the activeSegments queue a second time also, which would permit it to be recycled twice, creating two different CLS objects in memory pointing to the same CLS on disk, after which all bets are off. The issue is highly unlikely to occur, but highly unlikely means it will probably happen eventually. I've fixed this based on my patch for CASSANDRA-5549, using the NonBlockingQueue I introduce there to simplify the logic and make it more obviously correct. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6557) CommitLogSegment may be duplicated in unlikely race scenario
[ https://issues.apache.org/jira/browse/CASSANDRA-6557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6557: Attachment: 6557.2.txt Updated version that uses synchronized block for the swap, instead of CAS CommitLogSegment may be duplicated in unlikely race scenario Key: CASSANDRA-6557 URL: https://issues.apache.org/jira/browse/CASSANDRA-6557 Project: Cassandra Issue Type: Bug Components: Core Environment: 2.1 Reporter: Benedict Assignee: Benedict Priority: Minor Fix For: 2.1 beta2 Attachments: 6557.2.txt In the unlikely event that the thread that switched to a new CLS has not finished executing the cleanup of its switch by the time the CLS has finished being used, it is possible for the same segment to be 'switched' in again. This would be benign except that it is added to the activeSegments queue a second time also, which would permit it to be recycled twice, creating two different CLS objects in memory pointing to the same CLS on disk, after which all bets are off. The issue is highly unlikely to occur, but highly unlikely means it will probably happen eventually. I've fixed this based on my patch for CASSANDRA-5549, using the NonBlockingQueue I introduce there to simplify the logic and make it more obviously correct. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6745) Require specifying rows_per_partition_to_cache
[ https://issues.apache.org/jira/browse/CASSANDRA-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931676#comment-13931676 ] Marcus Eriksson commented on CASSANDRA-6745: pushed a fix on top of the patch: https://github.com/krummas/cassandra/commits/marcuse/6745 cql2 already supports old syntax Require specifying rows_per_partition_to_cache -- Key: CASSANDRA-6745 URL: https://issues.apache.org/jira/browse/CASSANDRA-6745 Project: Cassandra Issue Type: Bug Reporter: Jonathan Ellis Assignee: Marcus Eriksson Priority: Trivial Fix For: 2.1 beta2 Attachments: 0001-wip-caching-options.patch We should require specifying rows_to_cache_per_partition for new tables or newly ALTERed when row caching is enabled. Pre-upgrade should be grandfathered in as ALL to match existing semantics. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6826) Query returns different number of results depending on fetchsize
[ https://issues.apache.org/jira/browse/CASSANDRA-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931677#comment-13931677 ] Bill Mitchell commented on CASSANDRA-6826: -- Sylvain, merci de me l'avoir fait remarquer. La dernière fois que j'ai cherché la nouvelle version, je ne l'ai pas trouvée. Je vais la télécharger sur-le-champ. Sans doute celui-ci c'est le même problème que CASSANDRA-6748. Query returns different number of results depending on fetchsize Key: CASSANDRA-6826 URL: https://issues.apache.org/jira/browse/CASSANDRA-6826 Project: Cassandra Issue Type: Bug Components: Core Environment: quad-core Windows 7 x64, single node cluster Cassandra 2.0.5 Reporter: Bill Mitchell Assignee: Sylvain Lebresne I issue a query across the set of partitioned wide rows for one logical row, where s, l, and partition specify the composite primary key for the row: SELECT ec, ea, rd FROM sr WHERE s = ? and partition IN ? and l = ? ALLOW FILTERING; If I set fetchSize to only 1000 when the Cluster is configured, the query sometimes does not return all the results. In the particular case I am chasing, it returns a total of 98586 rows. If I increase the fetchsize to 10, all the 9 actual rows are returned. This suggests there is some problem with fetchsize re-establishing the position on the next segment of the result set, at least when multiple partitions are being accessed. -- This message was sent by Atlassian JIRA (v6.2#6252)
git commit: Fix static counter columns
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 31cbdfd7b - ed93ad4d8
Fix static counter columns
patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-6827
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ed93ad4d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ed93ad4d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ed93ad4d
Branch: refs/heads/cassandra-2.0
Commit: ed93ad4d8c7ecba8a40ab51b0ecec491629458dd
Parents: 31cbdfd
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Mar 12 14:56:29 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Mar 12 14:56:29 2014 +0300
--
CHANGES.txt | 1 +
.../cql3/statements/SelectStatement.java| 20 +++-
2 files changed, 20 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed93ad4d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d8a348d..906f91f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -2,6 +2,7 @@
* Fix saving triggers to schema (CASSANDRA-6789)
* Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
* Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
+ * Fix static counter columns (CASSANDRA-6827)
2.0.6
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed93ad4d/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 100383f..53b2c05 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -25,6 +25,7 @@ import com.google.common.base.Predicate;
import com.google.common.collect.AbstractIterator;
import com.google.common.collect.Iterables;
import com.google.common.collect.Lists;
+
import org.github.jamm.MemoryMeter;
import org.apache.cassandra.auth.Permission;
@@ -33,6 +34,7 @@ import org.apache.cassandra.transport.messages.ResultMessage;
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.config.ColumnDefinition;
import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.context.CounterContext;
import org.apache.cassandra.db.filter.*;
import org.apache.cassandra.db.marshal.*;
import org.apache.cassandra.dht.*;
@@ -1052,7 +1054,7 @@ public class SelectStatement implements CQLStatement,
MeasurableForPreparedCache
staticValues = new HashMap();
ColumnGroupMap group = builder.firstGroup();
for (CFDefinition.Name name :
Iterables.filter(selection.getColumnsList(), isStaticFilter))
-staticValues.put(name, name.type.isCollection() ?
getCollectionValue(name, group) : getSimpleValue(name, group));
+staticValues.put(name, getValue(name, group));
builder.discardFirst();
// If there was static columns but there is no actual row,
then provided the select was a full
@@ -1180,6 +1182,16 @@ public class SelectStatement implements CQLStatement,
MeasurableForPreparedCache
}
}
+private static ByteBuffer getValue(CFDefinition.Name name, ColumnGroupMap
columns)
+{
+if (name.type.isCollection())
+return getCollectionValue(name, columns);
+else if (name.type.isCommutative())
+return getCounterValue(name, columns);
+
+return getSimpleValue(name, columns);
+}
+
private static ByteBuffer getCollectionValue(CFDefinition.Name name,
ColumnGroupMap columns)
{
ListPairByteBuffer, Column collection =
columns.getCollection(name.name.key);
@@ -1192,6 +1204,12 @@ public class SelectStatement implements CQLStatement,
MeasurableForPreparedCache
return c == null ? null : c.value();
}
+private static ByteBuffer getCounterValue(CFDefinition.Name name,
ColumnGroupMap columns)
+{
+Column c = columns.getSimple(name.name.key);
+return c == null ? null :
CounterColumnType.instance.decompose(CounterContext.instance().total(c.value()));
+}
+
private static boolean isReversedType(CFDefinition.Name name)
{
return name.type instanceof ReversedType;
[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt src/java/org/apache/cassandra/cql3/statements/SelectStatement.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ad5169d2 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ad5169d2 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ad5169d2 Branch: refs/heads/cassandra-2.1 Commit: ad5169d2f8c700799000eb3fafb3cdc606fbafde Parents: 6e03782 ed93ad4 Author: Aleksey Yeschenko alek...@apache.org Authored: Wed Mar 12 15:07:16 2014 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Wed Mar 12 15:07:16 2014 +0300 -- CHANGES.txt | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ad5169d2/CHANGES.txt -- diff --cc CHANGES.txt index 0442b4e,906f91f..d8eb3a1 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -15,6 -2,10 +15,7 @@@ Merged from 2.0 * Fix saving triggers to schema (CASSANDRA-6789) * Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790) * Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838) + * Fix static counter columns (CASSANDRA-6827) - - -2.0.6 * Avoid race-prone second scrub of system keyspace (CASSANDRA-6797) * Pool CqlRecordWriter clients by inetaddress rather than Range (CASSANDRA-6665)
[1/2] git commit: Fix static counter columns
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 6e0378249 - ad5169d2f
Fix static counter columns
patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-6827
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ed93ad4d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ed93ad4d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ed93ad4d
Branch: refs/heads/cassandra-2.1
Commit: ed93ad4d8c7ecba8a40ab51b0ecec491629458dd
Parents: 31cbdfd
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Mar 12 14:56:29 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Mar 12 14:56:29 2014 +0300
--
CHANGES.txt | 1 +
.../cql3/statements/SelectStatement.java| 20 +++-
2 files changed, 20 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed93ad4d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d8a348d..906f91f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -2,6 +2,7 @@
* Fix saving triggers to schema (CASSANDRA-6789)
* Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
* Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
+ * Fix static counter columns (CASSANDRA-6827)
2.0.6
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed93ad4d/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 100383f..53b2c05 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -25,6 +25,7 @@ import com.google.common.base.Predicate;
import com.google.common.collect.AbstractIterator;
import com.google.common.collect.Iterables;
import com.google.common.collect.Lists;
+
import org.github.jamm.MemoryMeter;
import org.apache.cassandra.auth.Permission;
@@ -33,6 +34,7 @@ import org.apache.cassandra.transport.messages.ResultMessage;
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.config.ColumnDefinition;
import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.context.CounterContext;
import org.apache.cassandra.db.filter.*;
import org.apache.cassandra.db.marshal.*;
import org.apache.cassandra.dht.*;
@@ -1052,7 +1054,7 @@ public class SelectStatement implements CQLStatement,
MeasurableForPreparedCache
staticValues = new HashMap();
ColumnGroupMap group = builder.firstGroup();
for (CFDefinition.Name name :
Iterables.filter(selection.getColumnsList(), isStaticFilter))
-staticValues.put(name, name.type.isCollection() ?
getCollectionValue(name, group) : getSimpleValue(name, group));
+staticValues.put(name, getValue(name, group));
builder.discardFirst();
// If there was static columns but there is no actual row,
then provided the select was a full
@@ -1180,6 +1182,16 @@ public class SelectStatement implements CQLStatement,
MeasurableForPreparedCache
}
}
+private static ByteBuffer getValue(CFDefinition.Name name, ColumnGroupMap
columns)
+{
+if (name.type.isCollection())
+return getCollectionValue(name, columns);
+else if (name.type.isCommutative())
+return getCounterValue(name, columns);
+
+return getSimpleValue(name, columns);
+}
+
private static ByteBuffer getCollectionValue(CFDefinition.Name name,
ColumnGroupMap columns)
{
ListPairByteBuffer, Column collection =
columns.getCollection(name.name.key);
@@ -1192,6 +1204,12 @@ public class SelectStatement implements CQLStatement,
MeasurableForPreparedCache
return c == null ? null : c.value();
}
+private static ByteBuffer getCounterValue(CFDefinition.Name name,
ColumnGroupMap columns)
+{
+Column c = columns.getSimple(name.name.key);
+return c == null ? null :
CounterColumnType.instance.decompose(CounterContext.instance().total(c.value()));
+}
+
private static boolean isReversedType(CFDefinition.Name name)
{
return name.type instanceof ReversedType;
[4/5] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt src/java/org/apache/cassandra/cql3/statements/SelectStatement.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ad5169d2 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ad5169d2 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ad5169d2 Branch: refs/heads/trunk Commit: ad5169d2f8c700799000eb3fafb3cdc606fbafde Parents: 6e03782 ed93ad4 Author: Aleksey Yeschenko alek...@apache.org Authored: Wed Mar 12 15:07:16 2014 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Wed Mar 12 15:07:16 2014 +0300 -- CHANGES.txt | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ad5169d2/CHANGES.txt -- diff --cc CHANGES.txt index 0442b4e,906f91f..d8eb3a1 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -15,6 -2,10 +15,7 @@@ Merged from 2.0 * Fix saving triggers to schema (CASSANDRA-6789) * Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790) * Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838) + * Fix static counter columns (CASSANDRA-6827) - - -2.0.6 * Avoid race-prone second scrub of system keyspace (CASSANDRA-6797) * Pool CqlRecordWriter clients by inetaddress rather than Range (CASSANDRA-6665)
[5/5] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a03ac8d0 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a03ac8d0 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a03ac8d0 Branch: refs/heads/trunk Commit: a03ac8d07f04328427020a5dbe68545cf264ce47 Parents: 5023486 ad5169d Author: Aleksey Yeschenko alek...@apache.org Authored: Wed Mar 12 15:08:55 2014 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Wed Mar 12 15:08:55 2014 +0300 -- CHANGES.txt | 2 ++ src/java/org/apache/cassandra/service/FileCacheService.java | 3 +++ 2 files changed, 5 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/a03ac8d0/CHANGES.txt --
[2/5] git commit: merge from 2.0
merge from 2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6e037824 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6e037824 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6e037824 Branch: refs/heads/trunk Commit: 6e0378249a54e8005302aac3e4d4f8b67c6c8f39 Parents: e22d0b1 31cbdfd Author: Jonathan Ellis jbel...@apache.org Authored: Tue Mar 11 23:19:39 2014 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Tue Mar 11 23:19:39 2014 -0500 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/service/FileCacheService.java | 3 +++ 2 files changed, 4 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6e037824/CHANGES.txt -- diff --cc CHANGES.txt index 06331ad,d8a348d..0442b4e --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,19 -1,10 +1,20 @@@ -2.0.7 +2.1.0-beta2 + * Allow cassandra-stress to set compaction strategy options (CASSANDRA-6451) + * Add broadcast_rpc_address option to cassandra.yaml (CASSANDRA-5899) + * Auto reload GossipingPropertyFileSnitch config (CASSANDRA-5897) + * Fix overflow of memtable_total_space_in_mb (CASSANDRA-6573) + * Fix ABTC NPE (CASSANDRA-6692) + * Allow nodetool to use a file or prompt for password (CASSANDRA-6660) + * Fix AIOOBE when concurrently accessing ABSC (CASSANDRA-6742) + * Fix assertion error in ALTER TYPE RENAME (CASSANDRA-6705) + * Scrub should not always clear out repaired status (CASSANDRA-5351) + * Improve handling of range tombstone for wide partitions (CASSANDRA-6446) + * Fix ClassCastException for compact table with composites (CASSANDRA-6738) + * Fix potentially repairing with wrong nodes (CASSANDRA-6808) +Merged from 2.0: * Fix saving triggers to schema (CASSANDRA-6789) * Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790) + * Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838) - - -2.0.6 * Avoid race-prone second scrub of system keyspace (CASSANDRA-6797) * Pool CqlRecordWriter clients by inetaddress rather than Range (CASSANDRA-6665)
[1/5] git commit: Fix accounting in FileCacheService to allow re-using RAR Patch by Benedict Elliott Smith; reviewed by jbellis for CASSANDRA-6838
Repository: cassandra
Updated Branches:
refs/heads/trunk 5023486f4 - a03ac8d07
Fix accounting in FileCacheService to allow re-using RAR
Patch by Benedict Elliott Smith; reviewed by jbellis for CASSANDRA-6838
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/31cbdfd7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/31cbdfd7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/31cbdfd7
Branch: refs/heads/trunk
Commit: 31cbdfd7ba9e7ff2ae5f99f3f0f1a7831cd88147
Parents: fc9cad9
Author: Jonathan Ellis jbel...@apache.org
Authored: Tue Mar 11 23:18:54 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Tue Mar 11 23:18:54 2014 -0500
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/service/FileCacheService.java | 3 +++
2 files changed, 4 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/31cbdfd7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 91037d1..d8a348d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,6 +1,7 @@
2.0.7
* Fix saving triggers to schema (CASSANDRA-6789)
* Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
+ * Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
2.0.6
http://git-wip-us.apache.org/repos/asf/cassandra/blob/31cbdfd7/src/java/org/apache/cassandra/service/FileCacheService.java
--
diff --git a/src/java/org/apache/cassandra/service/FileCacheService.java
b/src/java/org/apache/cassandra/service/FileCacheService.java
index c939a6f..d22763b 100644
--- a/src/java/org/apache/cassandra/service/FileCacheService.java
+++ b/src/java/org/apache/cassandra/service/FileCacheService.java
@@ -91,7 +91,10 @@ public class FileCacheService
QueueRandomAccessReader instances = getCacheFor(path);
RandomAccessReader result = instances.poll();
if (result != null)
+{
metrics.hits.mark();
+memoryUsage.addAndGet(-result.getTotalBufferSize());
+}
return result;
}
[3/5] git commit: Fix static counter columns
Fix static counter columns
patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-6827
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ed93ad4d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ed93ad4d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ed93ad4d
Branch: refs/heads/trunk
Commit: ed93ad4d8c7ecba8a40ab51b0ecec491629458dd
Parents: 31cbdfd
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Mar 12 14:56:29 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Mar 12 14:56:29 2014 +0300
--
CHANGES.txt | 1 +
.../cql3/statements/SelectStatement.java| 20 +++-
2 files changed, 20 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed93ad4d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d8a348d..906f91f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -2,6 +2,7 @@
* Fix saving triggers to schema (CASSANDRA-6789)
* Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
* Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
+ * Fix static counter columns (CASSANDRA-6827)
2.0.6
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed93ad4d/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 100383f..53b2c05 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -25,6 +25,7 @@ import com.google.common.base.Predicate;
import com.google.common.collect.AbstractIterator;
import com.google.common.collect.Iterables;
import com.google.common.collect.Lists;
+
import org.github.jamm.MemoryMeter;
import org.apache.cassandra.auth.Permission;
@@ -33,6 +34,7 @@ import org.apache.cassandra.transport.messages.ResultMessage;
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.config.ColumnDefinition;
import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.context.CounterContext;
import org.apache.cassandra.db.filter.*;
import org.apache.cassandra.db.marshal.*;
import org.apache.cassandra.dht.*;
@@ -1052,7 +1054,7 @@ public class SelectStatement implements CQLStatement,
MeasurableForPreparedCache
staticValues = new HashMap();
ColumnGroupMap group = builder.firstGroup();
for (CFDefinition.Name name :
Iterables.filter(selection.getColumnsList(), isStaticFilter))
-staticValues.put(name, name.type.isCollection() ?
getCollectionValue(name, group) : getSimpleValue(name, group));
+staticValues.put(name, getValue(name, group));
builder.discardFirst();
// If there was static columns but there is no actual row,
then provided the select was a full
@@ -1180,6 +1182,16 @@ public class SelectStatement implements CQLStatement,
MeasurableForPreparedCache
}
}
+private static ByteBuffer getValue(CFDefinition.Name name, ColumnGroupMap
columns)
+{
+if (name.type.isCollection())
+return getCollectionValue(name, columns);
+else if (name.type.isCommutative())
+return getCounterValue(name, columns);
+
+return getSimpleValue(name, columns);
+}
+
private static ByteBuffer getCollectionValue(CFDefinition.Name name,
ColumnGroupMap columns)
{
ListPairByteBuffer, Column collection =
columns.getCollection(name.name.key);
@@ -1192,6 +1204,12 @@ public class SelectStatement implements CQLStatement,
MeasurableForPreparedCache
return c == null ? null : c.value();
}
+private static ByteBuffer getCounterValue(CFDefinition.Name name,
ColumnGroupMap columns)
+{
+Column c = columns.getSimple(name.name.key);
+return c == null ? null :
CounterColumnType.instance.decompose(CounterContext.instance().total(c.value()));
+}
+
private static boolean isReversedType(CFDefinition.Name name)
{
return name.type instanceof ReversedType;
[jira] [Updated] (CASSANDRA-6759) CommitLogSegment should provide madvise DONTNEED hint after syncing a segment
[ https://issues.apache.org/jira/browse/CASSANDRA-6759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6759: Attachment: 6759.txt CommitLogSegment should provide madvise DONTNEED hint after syncing a segment - Key: CASSANDRA-6759 URL: https://issues.apache.org/jira/browse/CASSANDRA-6759 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Priority: Minor Fix For: 3.0 Attachments: 6759.txt This is a really simple change, copying what we do with SequentialWriter, but here we know for sure we don't intend to re-read the region again in the near future so it's definitely wasted file buffer space. I have a patch, but I would like to do some brief performance comparisons before submitting it, and see if providing DONTNEED at file creation like we do for SW is sensible. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6759) CommitLogSegment should provide madvise DONTNEED hint after syncing a segment
[ https://issues.apache.org/jira/browse/CASSANDRA-6759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6759: Attachment: 6759.txt CommitLogSegment should provide madvise DONTNEED hint after syncing a segment - Key: CASSANDRA-6759 URL: https://issues.apache.org/jira/browse/CASSANDRA-6759 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Priority: Minor Fix For: 3.0 Attachments: 6759.txt This is a really simple change, copying what we do with SequentialWriter, but here we know for sure we don't intend to re-read the region again in the near future so it's definitely wasted file buffer space. I have a patch, but I would like to do some brief performance comparisons before submitting it, and see if providing DONTNEED at file creation like we do for SW is sensible. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6759) CommitLogSegment should provide madvise DONTNEED hint after syncing a segment
[ https://issues.apache.org/jira/browse/CASSANDRA-6759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6759: Attachment: (was: 6759.txt) CommitLogSegment should provide madvise DONTNEED hint after syncing a segment - Key: CASSANDRA-6759 URL: https://issues.apache.org/jira/browse/CASSANDRA-6759 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Priority: Minor Fix For: 3.0 This is a really simple change, copying what we do with SequentialWriter, but here we know for sure we don't intend to re-read the region again in the near future so it's definitely wasted file buffer space. I have a patch, but I would like to do some brief performance comparisons before submitting it, and see if providing DONTNEED at file creation like we do for SW is sensible. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6759) CommitLogSegment should provide madvise DONTNEED hint after syncing a segment
[ https://issues.apache.org/jira/browse/CASSANDRA-6759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6759: Attachment: 6759.txt CommitLogSegment should provide madvise DONTNEED hint after syncing a segment - Key: CASSANDRA-6759 URL: https://issues.apache.org/jira/browse/CASSANDRA-6759 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Priority: Minor Fix For: 3.0 This is a really simple change, copying what we do with SequentialWriter, but here we know for sure we don't intend to re-read the region again in the near future so it's definitely wasted file buffer space. I have a patch, but I would like to do some brief performance comparisons before submitting it, and see if providing DONTNEED at file creation like we do for SW is sensible. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6759) CommitLogSegment should provide madvise DONTNEED hint after syncing a segment
[ https://issues.apache.org/jira/browse/CASSANDRA-6759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6759: Attachment: (was: 6759.txt) CommitLogSegment should provide madvise DONTNEED hint after syncing a segment - Key: CASSANDRA-6759 URL: https://issues.apache.org/jira/browse/CASSANDRA-6759 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Priority: Minor Fix For: 3.0 This is a really simple change, copying what we do with SequentialWriter, but here we know for sure we don't intend to re-read the region again in the near future so it's definitely wasted file buffer space. I have a patch, but I would like to do some brief performance comparisons before submitting it, and see if providing DONTNEED at file creation like we do for SW is sensible. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6799) schema_version of newly bootstrapped nodes disagrees with existing nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-6799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931690#comment-13931690
]
Pas commented on CASSANDRA-6799:
Hello!
We're experiencing something very similar, although with a substantially higher
schema traffic and after a few restart-restarts the schemas (on just a 2-node
ring, 2.0.6 both), finally agreed, but after running a few keyspace creates and
drops they drifted again.
{code}
root@test1:/var/log/cassandra# nodetool gossipinfo | grep SCH
SCHEMA:23bf64b8-bfae-3d4a-a238-220731bda1eb
SCHEMA:23bf64b8-bfae-3d4a-a238-220731bda1eb
root@test1:/var/log/cassandra#
root@test1:/var/log/cassandra# nodetool gossipinfo | grep SCH
SCHEMA:fe69b71a-90d7-3c99-a6a7-f5e6b10ed6d1
SCHEMA:fe69b71a-90d7-3c99-a6a7-f5e6b10ed6d1
root@test1:/var/log/cassandra# nodetool gossipinfo | grep SCH
SCHEMA:fe69b71a-90d7-3c99-a6a7-f5e6b10ed6d1
SCHEMA:fe69b71a-90d7-3c99-a6a7-f5e6b10ed6d1
root@test1:/var/log/cassandra# nodetool gossipinfo | grep SCH
SCHEMA:a8f444f9-7860-335c-96a1-1beeebca543f
SCHEMA:15abe898-418f-3fdf-b862-cf06b907704e
root@test1:/var/log/cassandra#
root@test1:/var/log/cassandra# nodetool gossipinfo | grep SCH
SCHEMA:a8f444f9-7860-335c-96a1-1beeebca543f
SCHEMA:15abe898-418f-3fdf-b862-cf06b907704e
{code}
{code}
## this is test2
[default@unknown] create keyspace test13;
fe69b71a-90d7-3c99-a6a7-f5e6b10ed6d1
[default@unknown] drop keyspace test11;
0a96652e-0524-3875-997b-cea2c20291bf
[default@unknown] drop keyspace test12;
c7f7ff34-a70b-3da5-88a8-70ceef91b2e9
[default@unknown] drop keyspace test14;
Keyspace 'test14' not found.
[default@unknown] drop keyspace test13;
a8f444f9-7860-335c-96a1-1beeebca543f
[default@unknown] drop keyspace test13;
Keyspace 'test13' not found.
{code}
schema_version of newly bootstrapped nodes disagrees with existing nodes
Key: CASSANDRA-6799
URL: https://issues.apache.org/jira/browse/CASSANDRA-6799
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: x86_64 ubuntu, java version 1.7.0_45, Cassandra 2.0.5
Reporter: Duncan Sands
Attachments: system.log.gz
After bootstrapping new nodes 172.18.33.23 and 172.18.33.24 last weekend, I
noticed that they have a different schema_version to the existing nodes. The
existing nodes have all been around for a while, saw some schema changes in
the past (eg: timeuuid - timestamp on a column family) but none recently,
and were originally running 1.2 (they were upgraded to 2.0.5). Here you see
the different schema version 0d9173d5-3947-328e-a14d-ce05239f61e0 for the two
nodes:
cqlsh select peer, data_center, host_id, preferred_ip, rack,
release_version, rpc_address, schema_version from system.peers;
peer | data_center | host_id |
preferred_ip | rack | release_version | rpc_address| schema_version
+-+--+--+--+-++--
192.168.21.12 | rdm | 55e4b4b6-2e64-4542-87a4-d8a8e28b5135 |
null | RAC1 | 2.0.5 | 192.168.21.12 |
f673ced0-8cfd-3d69-baba-4f81dc60c5b5
172.18.33.24 | ldn | 6e634206-94b6-4dcf-9cf8-72bfe190feee |
null | RAC1 | 2.0.5 | 172.18.33.24 |
0d9173d5-3947-328e-a14d-ce05239f61e0
172.18.33.22 | ldn | 75c9c81f-b00b-4335-8483-fb7f1bc0be1e |
null | RAC1 | 2.0.5 | 172.18.33.22 |
f673ced0-8cfd-3d69-baba-4f81dc60c5b5
192.168.60.136 | adm | c83d403f-ef0d-4c54-a844-d69730fa54d3 |
null | RAC1 | 2.0.5 | 192.168.60.136 |
f673ced0-8cfd-3d69-baba-4f81dc60c5b5
192.168.60.137 | adm | b12e6d71-e189-4fe8-b00a-8ff2cc9848fd |
null | RAC1 | 2.0.5 | 192.168.60.137 |
f673ced0-8cfd-3d69-baba-4f81dc60c5b5
192.168.21.11 | rdm | dd2e69cb-232f-4236-89f2-b5479669d9f7 |
null | RAC1 | 2.0.5 | 192.168.21.11 |
f673ced0-8cfd-3d69-baba-4f81dc60c5b5
172.18.33.21 | ldn | 6942404c-e512-46b4-977a-243defa48d0f |
null | RAC1 | 2.0.5 | 172.18.33.21 |
f673ced0-8cfd-3d69-baba-4f81dc60c5b5
192.168.60.138 | adm | a229bc0f-201b-479e-8312-66891f37ca85 |
null | RAC1 | 2.0.5 | 192.168.60.138 |
f673ced0-8cfd-3d69-baba-4f81dc60c5b5
192.168.60.134 | adm | 7b860a54-59ea-4a92-9b47-44b52793cc70 |
null | RAC1 | 2.0.5 | 192.168.60.134 |
f673ced0-8cfd-3d69-baba-4f81dc60c5b5
172.18.33.23 | ldn | a08bad62-55bb-492b-be64-7cf5d5073d6d |
null | RAC1 | 2.0.5 | 172.18.33.23 |
0d9173d5-3947-328e-a14d-ce05239f61e0
192.168.60.130 |
[jira] [Updated] (CASSANDRA-5483) Repair tracing
[
https://issues.apache.org/jira/browse/CASSANDRA-5483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lyuben Todorov updated CASSANDRA-5483:
--
Attachment: 5483-full-trunk.txt
I'm attaching a patch that is basically all the patches that have been
committed to the [5483|https://github.com/lyubent/cassandra/commits/5483]
feature branch.
The only change between your last set of patches and this patch is that I added
{{DebuggableThreadPoolExecutor#createWithCachedTheadpool}} which is the new
constructor to allow for a cached pool (just for consistency).
+1 from me.
Repair tracing
--
Key: CASSANDRA-5483
URL: https://issues.apache.org/jira/browse/CASSANDRA-5483
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Yuki Morishita
Assignee: Ben Chan
Priority: Minor
Labels: repair
Attachments: 5483-full-trunk.txt,
5483-v06-04-Allow-tracing-ttl-to-be-configured.patch,
5483-v06-05-Add-a-command-column-to-system_traces.events.patch,
5483-v06-06-Fix-interruption-in-tracestate-propagation.patch,
5483-v07-07-Better-constructor-parameters-for-DebuggableThreadPoolExecutor.patch,
5483-v07-08-Fix-brace-style.patch,
5483-v07-09-Add-trace-option-to-a-more-complete-set-of-repair-functions.patch,
5483-v07-10-Correct-name-of-boolean-repairedAt-to-fullRepair.patch,
ccm-repair-test, test-5483-system_traces-events.txt,
trunk@4620823-5483-v02-0001-Trace-filtering-and-tracestate-propagation.patch,
trunk@4620823-5483-v02-0002-Put-a-few-traces-parallel-to-the-repair-logging.patch,
tr...@8ebeee1-5483-v01-001-trace-filtering-and-tracestate-propagation.txt,
tr...@8ebeee1-5483-v01-002-simple-repair-tracing.txt,
v02p02-5483-v03-0003-Make-repair-tracing-controllable-via-nodetool.patch,
v02p02-5483-v04-0003-This-time-use-an-EnumSet-to-pass-boolean-repair-options.patch,
v02p02-5483-v05-0003-Use-long-instead-of-EnumSet-to-work-with-JMX.patch
I think it would be nice to log repair stats and results like query tracing
stores traces to system keyspace. With it, you don't have to lookup each log
file to see what was the status and how it performed the repair you invoked.
Instead, you can query the repair log with session ID to see the state and
stats of all nodes involved in that repair session.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-6842) Zero-copy reads from off-heap memtables
Benedict created CASSANDRA-6842: --- Summary: Zero-copy reads from off-heap memtables Key: CASSANDRA-6842 URL: https://issues.apache.org/jira/browse/CASSANDRA-6842 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Fix For: 3.0 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-6843) Reclaim overwritten off-heap memtable space early
Benedict created CASSANDRA-6843: --- Summary: Reclaim overwritten off-heap memtable space early Key: CASSANDRA-6843 URL: https://issues.apache.org/jira/browse/CASSANDRA-6843 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Fix For: 3.0 Perform GC on the off-heap memtable arenas -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6745) Require specifying rows_per_partition_to_cache
[ https://issues.apache.org/jira/browse/CASSANDRA-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931716#comment-13931716 ] Sylvain Lebresne commented on CASSANDRA-6745: - +1 Require specifying rows_per_partition_to_cache -- Key: CASSANDRA-6745 URL: https://issues.apache.org/jira/browse/CASSANDRA-6745 Project: Cassandra Issue Type: Bug Reporter: Jonathan Ellis Assignee: Marcus Eriksson Priority: Trivial Fix For: 2.1 beta2 Attachments: 0001-wip-caching-options.patch We should require specifying rows_to_cache_per_partition for new tables or newly ALTERed when row caching is enabled. Pre-upgrade should be grandfathered in as ALL to match existing semantics. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6565) New node refuses to join the ring.
[
https://issues.apache.org/jira/browse/CASSANDRA-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931732#comment-13931732
]
Pas commented on CASSANDRA-6565:
Hello!
Slightly similar issue, I didn't want to further pollute the issue tracker, so:
adding plus one node to a 2-node ring resulted in (all three on 2.0.6):
# grep 192.168.36.171 system.log
{code}
INFO [HANDSHAKE-/192.168.36.173] 2014-03-12 13:45:57,801
OutboundTcpConnection.java (line 386) Handshaking version with /192.168.36.173
INFO [GossipStage:1] 2014-03-12 13:46:00,170 Gossiper.java (line 883) Node
/192.168.36.173 is now part of the cluster
INFO [HANDSHAKE-/192.168.36.173] 2014-03-12 13:46:00,172
OutboundTcpConnection.java (line 386) Handshaking version with /192.168.36.173
INFO [RequestResponseStage:9] 2014-03-12 13:46:00,185 Gossiper.java (line 849)
InetAddress /192.168.36.173 is now UP
INFO [STREAM-INIT-/192.168.36.173:34610] 2014-03-12 13:46:48,608
StreamResultFuture.java (line 121) [Stream
#2e0c6a10-a9e4-11e3-9c59-5340fa68c75f] Received streaming plan for Bootstrap
INFO [STREAM-IN-/192.168.36.173] 2014-03-12 13:47:28,974
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-TStore@119522965(23215181/50011094 serialized/live bytes, 84365 ops)
INFO [STREAM-IN-/192.168.36.173] 2014-03-12 13:47:52,156
StreamResultFuture.java (line 173) [Stream
#2e0c6a10-a9e4-11e3-9c59-5340fa68c75f] Prepare completed. Receiving 0 files(0
bytes), sending 1971 files(6355271334 bytes)
ERROR [STREAM-OUT-/192.168.36.173] 2014-03-12 13:51:21,163 StreamSession.java
(line 418) [Stream #2e0c6a10-a9e4-11e3-9c59-5340fa68c75f] Streaming error
occurred
INFO [STREAM-OUT-/192.168.36.173] 2014-03-12 13:51:21,181
StreamResultFuture.java (line 186) [Stream
#2e0c6a10-a9e4-11e3-9c59-5340fa68c75f] Session with /192.168.36.173 is complete
INFO [STREAM-IN-/192.168.36.173] 2014-03-12 13:51:21,181
StreamResultFuture.java (line 186) [Stream
#2e0c6a10-a9e4-11e3-9c59-5340fa68c75f] Session with /192.168.36.173 is complete
WARN [STREAM-OUT-/192.168.36.173] 2014-03-12 13:51:21,184
StreamResultFuture.java (line 215) [Stream
#2e0c6a10-a9e4-11e3-9c59-5340fa68c75f] Stream failed
WARN [STREAM-IN-/192.168.36.173] 2014-03-12 13:51:21,185
StreamResultFuture.java (line 215) [Stream
#2e0c6a10-a9e4-11e3-9c59-5340fa68c75f] Stream failed
ERROR [STREAM-OUT-/192.168.36.173] 2014-03-12 13:51:21,185 StreamSession.java
(line 418) [Stream #2e0c6a10-a9e4-11e3-9c59-5340fa68c75f] Streaming error
occurred
[]
WARN [STREAM-OUT-/192.168.36.173] 2014-03-12 13:51:21,461
StreamResultFuture.java (line 215) [Stream
#2e0c6a10-a9e4-11e3-9c59-5340fa68c75f] Stream failed
ERROR [STREAM-OUT-/192.168.36.173] 2014-03-12 13:51:21,462 StreamSession.java
(line 418) [Stream #2e0c6a10-a9e4-11e3-9c59-5340fa68c75f] Streaming error
occurred
INFO [STREAM-OUT-/192.168.36.173] 2014-03-12 13:51:21,462
StreamResultFuture.java (line 186) [Stream
#2e0c6a10-a9e4-11e3-9c59-5340fa68c75f] Session with /192.168.36.173 is complete
WARN [STREAM-OUT-/192.168.36.173] 2014-03-12 13:51:21,462
StreamResultFuture.java (line 215) [Stream
#2e0c6a10-a9e4-11e3-9c59-5340fa68c75f] Stream failed
INFO [GossipStage:1] 2014-03-12 13:51:23,158 Gossiper.java (line 863)
InetAddress /192.168.36.173 is now DOWN
INFO [GossipTasks:1] 2014-03-12 13:51:51,522 Gossiper.java (line 633)
FatClient /192.168.36.173 has been silent for 3ms, removing from gossip
{code}
And there is always a Broken Pipe IOException after Streaming error occured
{code}
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
at sun.nio.ch.IOUtil.write(IOUtil.java:65)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:487)
at
org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:44)
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:383)
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:363)
at java.lang.Thread.run(Thread.java:744)
{code}
Whereas on 192.168.36.173 (the new node)
{code}
INFO [main] 2014-03-12 13:45:24,140 StorageService.java (line 985) JOINING:
Starting to bootstrap...
INFO [main] 2014-03-12 13:45:32,319 StreamResultFuture.java (line 87) [Stream
#2e0c6a10-a9e4-11e3-9c59-5340fa68c75f] Executing streaming plan for Bootstrap
INFO [main] 2014-03-12 13:45:32,319 StreamResultFuture.java (line 91) [Stream
#2e0c6a10-a9e4-11e3-9c59-5340fa68c75f] Beginning stream session with
/192.168.36.172
INFO [main] 2014-03-12 13:45:32,319 StreamResultFuture.java (line 91) [Stream
#2e0c6a10-a9e4-11e3-9c59-5340fa68c75f] Beginning stream session with
/192.168.36.171
INFO
git commit: Update syntax for changing caching options.
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 ad5169d2f - da444a690
Update syntax for changing caching options.
Patch by marcuse, reviewed by slebresne for CASSANDRA-6745.
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/da444a69
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/da444a69
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/da444a69
Branch: refs/heads/cassandra-2.1
Commit: da444a6903ddd8d02e7b251c8b38faf11d3a5c93
Parents: ad5169d
Author: Marcus Eriksson marc...@apache.org
Authored: Wed Mar 12 14:04:47 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Mar 12 14:06:05 2014 +0100
--
CHANGES.txt | 1 +
pylib/cqlshlib/cql3handling.py | 13 +-
.../apache/cassandra/cache/CachingOptions.java | 288 +++
.../org/apache/cassandra/cli/CliClient.java | 3 +-
.../org/apache/cassandra/config/CFMetaData.java | 110 ++-
.../cassandra/cql/AlterTableStatement.java | 4 +-
.../cql/CreateColumnFamilyStatement.java| 4 +-
.../cassandra/cql3/statements/CFPropDefs.java | 25 +-
.../apache/cassandra/db/ColumnFamilyStore.java | 18 +-
.../org/apache/cassandra/db/SystemKeyspace.java | 33 ++-
.../cassandra/io/sstable/SSTableReader.java | 6 +-
.../unit/org/apache/cassandra/SchemaLoader.java | 14 +-
12 files changed, 391 insertions(+), 128 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/da444a69/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d8eb3a1..61e17e3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -11,6 +11,7 @@
* Improve handling of range tombstone for wide partitions (CASSANDRA-6446)
* Fix ClassCastException for compact table with composites (CASSANDRA-6738)
* Fix potentially repairing with wrong nodes (CASSANDRA-6808)
+ * Change caching option syntax (CASSANDRA-6745)
Merged from 2.0:
* Fix saving triggers to schema (CASSANDRA-6789)
* Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/da444a69/pylib/cqlshlib/cql3handling.py
--
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index 158e60d..3522d1c 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -62,8 +62,6 @@ class Cql3ParsingRuleSet(CqlParsingRuleSet):
columnfamily_layout_options = (
('bloom_filter_fp_chance', None),
-('caching', None),
-('rows_per_partition_to_cache', None),
('comment', None),
('dclocal_read_repair_chance', 'local_read_repair_chance'),
('gc_grace_seconds', None),
@@ -83,6 +81,8 @@ class Cql3ParsingRuleSet(CqlParsingRuleSet):
('class', 'min_threshold', 'max_threshold')),
('compression', 'compression_parameters',
('sstable_compression', 'chunk_length_kb', 'crc_check_chance')),
+('caching', None,
+('rows_per_partition', 'keys')),
)
obsolete_cf_options = ()
@@ -463,6 +463,8 @@ def cf_prop_val_mapkey_completer(ctxt, cass):
pairsseen = dict(zip(keysseen, valsseen))
if optname == 'compression':
return map(escape_value, set(subopts).difference(keysseen))
+if optname == 'caching':
+return map(escape_value, set(subopts).difference(keysseen))
if optname == 'compaction':
opts = set(subopts)
try:
@@ -488,6 +490,11 @@ def cf_prop_val_mapval_completer(ctxt, cass):
if key == 'sstable_compression':
return map(escape_value, CqlRuleSet.available_compression_classes)
return [Hint('option_value')]
+elif opt == 'caching':
+if key == 'rows_per_partition':
+return [Hint('ALL'), Hint('NONE'), Hint('#rows_per_partition')]
+elif key == 'keys':
+return [Hint('ALL'), Hint('NONE')]
return ()
def cf_prop_val_mapender_completer(ctxt, cass):
@@ -1187,7 +1194,7 @@ class CqlTableDef:
for attr, val in layout.items():
setattr(cf, attr.encode('ascii'), val)
cf.comparator = lookup_casstype(cf.comparator)
-for attr in ('compaction_strategy_options', 'compression_parameters'):
+for attr in ('compaction_strategy_options', 'compression_parameters',
'caching'):
setattr(cf, attr, json.loads(getattr(cf, attr)))
# deal with columns, filter out empty column names (see CASSANDRA-6139)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/da444a69/src/java/org/apache/cassandra/cache/CachingOptions.java
[1/2] git commit: Update syntax for changing caching options.
Repository: cassandra
Updated Branches:
refs/heads/trunk a03ac8d07 - 346d8fb6d
Update syntax for changing caching options.
Patch by marcuse, reviewed by slebresne for CASSANDRA-6745.
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/da444a69
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/da444a69
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/da444a69
Branch: refs/heads/trunk
Commit: da444a6903ddd8d02e7b251c8b38faf11d3a5c93
Parents: ad5169d
Author: Marcus Eriksson marc...@apache.org
Authored: Wed Mar 12 14:04:47 2014 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Mar 12 14:06:05 2014 +0100
--
CHANGES.txt | 1 +
pylib/cqlshlib/cql3handling.py | 13 +-
.../apache/cassandra/cache/CachingOptions.java | 288 +++
.../org/apache/cassandra/cli/CliClient.java | 3 +-
.../org/apache/cassandra/config/CFMetaData.java | 110 ++-
.../cassandra/cql/AlterTableStatement.java | 4 +-
.../cql/CreateColumnFamilyStatement.java| 4 +-
.../cassandra/cql3/statements/CFPropDefs.java | 25 +-
.../apache/cassandra/db/ColumnFamilyStore.java | 18 +-
.../org/apache/cassandra/db/SystemKeyspace.java | 33 ++-
.../cassandra/io/sstable/SSTableReader.java | 6 +-
.../unit/org/apache/cassandra/SchemaLoader.java | 14 +-
12 files changed, 391 insertions(+), 128 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/da444a69/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d8eb3a1..61e17e3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -11,6 +11,7 @@
* Improve handling of range tombstone for wide partitions (CASSANDRA-6446)
* Fix ClassCastException for compact table with composites (CASSANDRA-6738)
* Fix potentially repairing with wrong nodes (CASSANDRA-6808)
+ * Change caching option syntax (CASSANDRA-6745)
Merged from 2.0:
* Fix saving triggers to schema (CASSANDRA-6789)
* Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/da444a69/pylib/cqlshlib/cql3handling.py
--
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index 158e60d..3522d1c 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -62,8 +62,6 @@ class Cql3ParsingRuleSet(CqlParsingRuleSet):
columnfamily_layout_options = (
('bloom_filter_fp_chance', None),
-('caching', None),
-('rows_per_partition_to_cache', None),
('comment', None),
('dclocal_read_repair_chance', 'local_read_repair_chance'),
('gc_grace_seconds', None),
@@ -83,6 +81,8 @@ class Cql3ParsingRuleSet(CqlParsingRuleSet):
('class', 'min_threshold', 'max_threshold')),
('compression', 'compression_parameters',
('sstable_compression', 'chunk_length_kb', 'crc_check_chance')),
+('caching', None,
+('rows_per_partition', 'keys')),
)
obsolete_cf_options = ()
@@ -463,6 +463,8 @@ def cf_prop_val_mapkey_completer(ctxt, cass):
pairsseen = dict(zip(keysseen, valsseen))
if optname == 'compression':
return map(escape_value, set(subopts).difference(keysseen))
+if optname == 'caching':
+return map(escape_value, set(subopts).difference(keysseen))
if optname == 'compaction':
opts = set(subopts)
try:
@@ -488,6 +490,11 @@ def cf_prop_val_mapval_completer(ctxt, cass):
if key == 'sstable_compression':
return map(escape_value, CqlRuleSet.available_compression_classes)
return [Hint('option_value')]
+elif opt == 'caching':
+if key == 'rows_per_partition':
+return [Hint('ALL'), Hint('NONE'), Hint('#rows_per_partition')]
+elif key == 'keys':
+return [Hint('ALL'), Hint('NONE')]
return ()
def cf_prop_val_mapender_completer(ctxt, cass):
@@ -1187,7 +1194,7 @@ class CqlTableDef:
for attr, val in layout.items():
setattr(cf, attr.encode('ascii'), val)
cf.comparator = lookup_casstype(cf.comparator)
-for attr in ('compaction_strategy_options', 'compression_parameters'):
+for attr in ('compaction_strategy_options', 'compression_parameters',
'caching'):
setattr(cf, attr, json.loads(getattr(cf, attr)))
# deal with columns, filter out empty column names (see CASSANDRA-6139)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/da444a69/src/java/org/apache/cassandra/cache/CachingOptions.java
[2/2] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Conflicts: src/java/org/apache/cassandra/cql/AlterTableStatement.java src/java/org/apache/cassandra/cql/CreateColumnFamilyStatement.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/346d8fb6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/346d8fb6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/346d8fb6 Branch: refs/heads/trunk Commit: 346d8fb6dd99c86b632a0b99a368231bbd34b0f7 Parents: a03ac8d da444a6 Author: Marcus Eriksson marc...@apache.org Authored: Wed Mar 12 14:09:16 2014 +0100 Committer: Marcus Eriksson marc...@apache.org Committed: Wed Mar 12 14:09:16 2014 +0100 -- CHANGES.txt | 1 + pylib/cqlshlib/cql3handling.py | 13 +- .../apache/cassandra/cache/CachingOptions.java | 288 +++ .../org/apache/cassandra/cli/CliClient.java | 3 +- .../org/apache/cassandra/config/CFMetaData.java | 110 ++- .../cassandra/cql3/statements/CFPropDefs.java | 25 +- .../apache/cassandra/db/ColumnFamilyStore.java | 18 +- .../org/apache/cassandra/db/SystemKeyspace.java | 33 ++- .../cassandra/io/sstable/SSTableReader.java | 6 +- .../unit/org/apache/cassandra/SchemaLoader.java | 14 +- 10 files changed, 387 insertions(+), 124 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/346d8fb6/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/346d8fb6/src/java/org/apache/cassandra/config/CFMetaData.java --
[jira] [Commented] (CASSANDRA-6745) Require specifying rows_per_partition_to_cache
[ https://issues.apache.org/jira/browse/CASSANDRA-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931736#comment-13931736 ] Marcus Eriksson commented on CASSANDRA-6745: committed, thanks Require specifying rows_per_partition_to_cache -- Key: CASSANDRA-6745 URL: https://issues.apache.org/jira/browse/CASSANDRA-6745 Project: Cassandra Issue Type: Bug Reporter: Jonathan Ellis Assignee: Marcus Eriksson Priority: Trivial Fix For: 2.1 beta2 Attachments: 0001-wip-caching-options.patch We should require specifying rows_to_cache_per_partition for new tables or newly ALTERed when row caching is enabled. Pre-upgrade should be grandfathered in as ALL to match existing semantics. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-6844) Expired cells not converted to deleted cells during cleanup/scrub/compaction (sometimes)
Aleksey Yeschenko created CASSANDRA-6844: Summary: Expired cells not converted to deleted cells during cleanup/scrub/compaction (sometimes) Key: CASSANDRA-6844 URL: https://issues.apache.org/jira/browse/CASSANDRA-6844 Project: Cassandra Issue Type: Bug Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko On read path, we intentionally read expiring cells as expiring cells, never converting them to tombstones, to return consistent results to the clients. For compaction/cleanup/scrub we don't care about that, and are supposed to make the conversion as an optimization. However, somewhere along the way it got lost, and SSTII doesn't always do it now in 2.0 (and never does it in 2.1). SSTI.getColumnFamilyWithColumns() passes the correct expireBefore(), but SSTI.next() does not, and while we use both in 2.0, depending on the some factors, in 2.1 we only use SSTI.next(), and SSTI.getColumnFamilyWithColumns() is actually dead code (I will remove it during the 2.1 merge). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6844) Expired cells not converted to deleted cells during cleanup/scrub/compaction (sometimes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-6844: - Attachment: 6844.txt Expired cells not converted to deleted cells during cleanup/scrub/compaction (sometimes) Key: CASSANDRA-6844 URL: https://issues.apache.org/jira/browse/CASSANDRA-6844 Project: Cassandra Issue Type: Bug Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Fix For: 2.0.7, 2.1 beta2 Attachments: 6844.txt On read path, we intentionally read expiring cells as expiring cells, never converting them to tombstones, to return consistent results to the clients. For compaction/cleanup/scrub we don't care about that, and are supposed to make the conversion as an optimization. However, somewhere along the way it got lost, and SSTII doesn't always do it now in 2.0 (and never does it in 2.1). SSTI.getColumnFamilyWithColumns() passes the correct expireBefore(), but SSTI.next() does not, and while we use both in 2.0, depending on the some factors, in 2.1 we only use SSTI.next(), and SSTI.getColumnFamilyWithColumns() is actually dead code (I will remove it during the 2.1 merge). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6842) Zero-copy reads from off-heap memtables
[ https://issues.apache.org/jira/browse/CASSANDRA-6842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931809#comment-13931809 ] Benedict commented on CASSANDRA-6842: - Patch available [here|https://github.com/belliottsmith/cassandra/tree/iss-6842] See discussion on CASSANDRA-6689 for history. This patch introduces a new NativePool and NativeAllocator implementation, and RefAction, which manages references that escape from memtables, and also introduces new concurrency primitives NonBlockingQueue* and an updated OpOrder, all as described [here|https://issues.apache.org/jira/browse/CASSANDRA-6689?focusedCommentId=13916720page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13916720]. Note that references to GC are not applicable, as this will be introduced separately in CASSANDRA-6843 instead. This patch depends on CASSANDRA-6781. Zero-copy reads from off-heap memtables --- Key: CASSANDRA-6842 URL: https://issues.apache.org/jira/browse/CASSANDRA-6842 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Labels: performance Fix For: 3.0 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (CASSANDRA-6774) Cleanup fails with assertion error after stopping previous run
[ https://issues.apache.org/jira/browse/CASSANDRA-6774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-6774: - Assignee: Marcus Eriksson Cleanup fails with assertion error after stopping previous run -- Key: CASSANDRA-6774 URL: https://issues.apache.org/jira/browse/CASSANDRA-6774 Project: Cassandra Issue Type: Bug Components: Core Environment: 2.0.5 Reporter: Keith Wright Assignee: Marcus Eriksson I am stress testing a new 2.0.5 cluster and did the following: - start decommission during heavy write, moderate read load - trigger cleanup on non-decommissioning node (nodetool cleanup) - Started to see higher GC load stop stopped cleanup via nodetool stop CLEANUP - attempt to launch cleanup now fails with the following message in console. Cassandra log shows: WARN 18:45:38,420 Unable to cancel in-progress compactions for shard_user_lookup. Probably there is an unusually large row in progress somewhere. It is also possible that buggy code left some sstables compacting after it was done with them Exception in thread main java.lang.AssertionError: [SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6129-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6938-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6698-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6884-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6854-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6898-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6522-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6692-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6815-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6677-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6917-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6929-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-7048-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6911-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6876-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-7046-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6762-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6712-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6906-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6886-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-7053-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6696-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6964-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-7043-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6983-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6672-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6714-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6992-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6234-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6971-Data.db'), SSTableReader(path='/data/2/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6826-Data.db'), SSTableReader(path='/data/1/cassandra/data/users/shard_user_lookup/users-shard_user_lookup-jb-6456-Data.db'),
[jira] [Updated] (CASSANDRA-6835) cassandra-stress should support a variable number of counter columns
[ https://issues.apache.org/jira/browse/CASSANDRA-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-6835: -- Reviewer: Pavel Yaskevich Description: Can you review, [~xedin]? cassandra-stress should support a variable number of counter columns Key: CASSANDRA-6835 URL: https://issues.apache.org/jira/browse/CASSANDRA-6835 Project: Cassandra Issue Type: Improvement Reporter: Benedict Assignee: Benedict Priority: Minor Fix For: 2.1 beta2 Can you review, [~xedin]? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6844) Expired cells not converted to deleted cells during cleanup/scrub/compaction (sometimes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931844#comment-13931844 ] Sylvain Lebresne commented on CASSANDRA-6844: - +1, good catch. Expired cells not converted to deleted cells during cleanup/scrub/compaction (sometimes) Key: CASSANDRA-6844 URL: https://issues.apache.org/jira/browse/CASSANDRA-6844 Project: Cassandra Issue Type: Bug Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Fix For: 2.0.7, 2.1 beta2 Attachments: 6844.txt On read path, we intentionally read expiring cells as expiring cells, never converting them to tombstones, to return consistent results to the clients. For compaction/cleanup/scrub we don't care about that, and are supposed to make the conversion as an optimization. However, somewhere along the way it got lost, and SSTII doesn't always do it now in 2.0 (and never does it in 2.1). SSTI.getColumnFamilyWithColumns() passes the correct expireBefore(), but SSTI.next() does not, and while we use both in 2.0, depending on the some factors, in 2.1 we only use SSTI.next(), and SSTI.getColumnFamilyWithColumns() is actually dead code (I will remove it during the 2.1 merge). -- This message was sent by Atlassian JIRA (v6.2#6252)
[2/5] git commit: Switch CRC component to Adler and include it for compressed sstables patch by Marcus Eriksson, Radovan Zvoncek, and jbellis for CASSANDRA-4165
Switch CRC component to Adler and include it for compressed sstables
patch by Marcus Eriksson, Radovan Zvoncek, and jbellis for CASSANDRA-4165
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9405ce0e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9405ce0e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9405ce0e
Branch: refs/heads/cassandra-2.1
Commit: 9405ce0e2d95424544561d088d3167cf328964ec
Parents: da444a6
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Mar 12 10:04:29 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Mar 12 10:04:29 2014 -0500
--
CHANGES.txt | 2 +
.../io/compress/CompressedSequentialWriter.java | 35 -
.../apache/cassandra/io/sstable/Descriptor.java | 3 +
.../cassandra/io/sstable/SSTableWriter.java | 19 +++--
.../io/util/DataIntegrityMetadata.java | 83
.../cassandra/io/util/SequentialWriter.java | 35 +
6 files changed, 79 insertions(+), 98 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9405ce0e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 61e17e3..7a1fc60 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
2.1.0-beta2
+ * Switch CRC component to Adler and include it for compressed sstables
+ (CASSANDRA-4165)
* Allow cassandra-stress to set compaction strategy options (CASSANDRA-6451)
* Add broadcast_rpc_address option to cassandra.yaml (CASSANDRA-5899)
* Auto reload GossipingPropertyFileSnitch config (CASSANDRA-5897)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9405ce0e/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
index 436b6dc..b8a21cc 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
@@ -26,20 +26,15 @@ import java.util.zip.Checksum;
import org.apache.cassandra.io.FSReadError;
import org.apache.cassandra.io.FSWriteError;
import org.apache.cassandra.io.sstable.CorruptSSTableException;
+import org.apache.cassandra.io.sstable.Descriptor;
import org.apache.cassandra.io.sstable.metadata.MetadataCollector;
+import org.apache.cassandra.io.util.DataIntegrityMetadata;
import org.apache.cassandra.io.util.FileMark;
import org.apache.cassandra.io.util.SequentialWriter;
public class CompressedSequentialWriter extends SequentialWriter
{
-public static SequentialWriter open(String dataFilePath,
-String indexFilePath,
-boolean skipIOCache,
-CompressionParameters parameters,
-MetadataCollector
sstableMetadataCollector)
-{
-return new CompressedSequentialWriter(new File(dataFilePath),
indexFilePath, skipIOCache, parameters, sstableMetadataCollector);
-}
+private final DataIntegrityMetadata.ChecksumWriter crcMetadata;
// holds offset in the file where current chunk should be written
// changed only by flush() method where data buffer gets compressed and
stored to the file
@@ -55,14 +50,12 @@ public class CompressedSequentialWriter extends
SequentialWriter
// holds a number of already written chunks
private int chunkCount = 0;
-private final Checksum checksum = new Adler32();
-
private long originalSize = 0, compressedSize = 0;
private final MetadataCollector sstableMetadataCollector;
public CompressedSequentialWriter(File file,
- String indexFilePath,
+ String offsetsPath,
boolean skipIOCache,
CompressionParameters parameters,
MetadataCollector
sstableMetadataCollector)
@@ -74,10 +67,11 @@ public class CompressedSequentialWriter extends
SequentialWriter
compressed = new ICompressor.WrappedArray(new
byte[compressor.initialCompressedBufferLength(buffer.length)]);
/* Index File (-CompressionInfo.db component) and it's header */
-metadataWriter = CompressionMetadata.Writer.open(indexFilePath);
+metadataWriter = CompressionMetadata.Writer.open(offsetsPath);
metadataWriter.writeHeader(parameters);
this.sstableMetadataCollector =
[3/5] git commit: Switch CRC component to Adler and include it for compressed sstables patch by Marcus Eriksson, Radovan Zvoncek, and jbellis for CASSANDRA-4165
Switch CRC component to Adler and include it for compressed sstables
patch by Marcus Eriksson, Radovan Zvoncek, and jbellis for CASSANDRA-4165
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9405ce0e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9405ce0e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9405ce0e
Branch: refs/heads/trunk
Commit: 9405ce0e2d95424544561d088d3167cf328964ec
Parents: da444a6
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Mar 12 10:04:29 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Mar 12 10:04:29 2014 -0500
--
CHANGES.txt | 2 +
.../io/compress/CompressedSequentialWriter.java | 35 -
.../apache/cassandra/io/sstable/Descriptor.java | 3 +
.../cassandra/io/sstable/SSTableWriter.java | 19 +++--
.../io/util/DataIntegrityMetadata.java | 83
.../cassandra/io/util/SequentialWriter.java | 35 +
6 files changed, 79 insertions(+), 98 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9405ce0e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 61e17e3..7a1fc60 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
2.1.0-beta2
+ * Switch CRC component to Adler and include it for compressed sstables
+ (CASSANDRA-4165)
* Allow cassandra-stress to set compaction strategy options (CASSANDRA-6451)
* Add broadcast_rpc_address option to cassandra.yaml (CASSANDRA-5899)
* Auto reload GossipingPropertyFileSnitch config (CASSANDRA-5897)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9405ce0e/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
index 436b6dc..b8a21cc 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressedSequentialWriter.java
@@ -26,20 +26,15 @@ import java.util.zip.Checksum;
import org.apache.cassandra.io.FSReadError;
import org.apache.cassandra.io.FSWriteError;
import org.apache.cassandra.io.sstable.CorruptSSTableException;
+import org.apache.cassandra.io.sstable.Descriptor;
import org.apache.cassandra.io.sstable.metadata.MetadataCollector;
+import org.apache.cassandra.io.util.DataIntegrityMetadata;
import org.apache.cassandra.io.util.FileMark;
import org.apache.cassandra.io.util.SequentialWriter;
public class CompressedSequentialWriter extends SequentialWriter
{
-public static SequentialWriter open(String dataFilePath,
-String indexFilePath,
-boolean skipIOCache,
-CompressionParameters parameters,
-MetadataCollector
sstableMetadataCollector)
-{
-return new CompressedSequentialWriter(new File(dataFilePath),
indexFilePath, skipIOCache, parameters, sstableMetadataCollector);
-}
+private final DataIntegrityMetadata.ChecksumWriter crcMetadata;
// holds offset in the file where current chunk should be written
// changed only by flush() method where data buffer gets compressed and
stored to the file
@@ -55,14 +50,12 @@ public class CompressedSequentialWriter extends
SequentialWriter
// holds a number of already written chunks
private int chunkCount = 0;
-private final Checksum checksum = new Adler32();
-
private long originalSize = 0, compressedSize = 0;
private final MetadataCollector sstableMetadataCollector;
public CompressedSequentialWriter(File file,
- String indexFilePath,
+ String offsetsPath,
boolean skipIOCache,
CompressionParameters parameters,
MetadataCollector
sstableMetadataCollector)
@@ -74,10 +67,11 @@ public class CompressedSequentialWriter extends
SequentialWriter
compressed = new ICompressor.WrappedArray(new
byte[compressor.initialCompressedBufferLength(buffer.length)]);
/* Index File (-CompressionInfo.db component) and it's header */
-metadataWriter = CompressionMetadata.Writer.open(indexFilePath);
+metadataWriter = CompressionMetadata.Writer.open(offsetsPath);
metadataWriter.writeHeader(parameters);
this.sstableMetadataCollector = sstableMetadataCollector;
+
[5/5] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9b123292 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9b123292 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9b123292 Branch: refs/heads/trunk Commit: 9b1232929dea23a067283c6f0df2a12bb7814c77 Parents: 8873afc 9405ce0 Author: Jonathan Ellis jbel...@apache.org Authored: Wed Mar 12 10:04:58 2014 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Wed Mar 12 10:04:58 2014 -0500 -- CHANGES.txt | 2 + .../io/compress/CompressedSequentialWriter.java | 35 - .../apache/cassandra/io/sstable/Descriptor.java | 3 + .../cassandra/io/sstable/SSTableWriter.java | 19 +++-- .../io/util/DataIntegrityMetadata.java | 83 .../cassandra/io/util/SequentialWriter.java | 35 + 6 files changed, 79 insertions(+), 98 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b123292/CHANGES.txt -- diff --cc CHANGES.txt index a722c6c,7a1fc60..76b1942 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,9 -1,6 +1,11 @@@ +3.0 + * Remove CQL2 (CASSANDRA-5918) + * add Thrift get_multi_slice call (CASSANDRA-6757) + + 2.1.0-beta2 + * Switch CRC component to Adler and include it for compressed sstables +(CASSANDRA-4165) * Allow cassandra-stress to set compaction strategy options (CASSANDRA-6451) * Add broadcast_rpc_address option to cassandra.yaml (CASSANDRA-5899) * Auto reload GossipingPropertyFileSnitch config (CASSANDRA-5897)
[1/5] git commit: Merge branch 'cassandra-2.1' into trunk
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 da444a690 - 9405ce0e2 refs/heads/trunk 346d8fb6d - 9b1232929 Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4082e536 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4082e536 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4082e536 Branch: refs/heads/trunk Commit: 4082e5366ca4ac2efd183c53dfc7e976002a2df0 Parents: 2037a8d 6e03782 Author: Jonathan Ellis jbel...@apache.org Authored: Tue Mar 11 23:19:46 2014 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Tue Mar 11 23:19:46 2014 -0500 -- CHANGES.txt | 1 + .../cassandra/service/FileCacheService.java | 3 + .../org/apache/cassandra/tools/NodeTool.java| 217 +-- 3 files changed, 106 insertions(+), 115 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4082e536/CHANGES.txt --
[4/5] git commit: Merge remote-tracking branch 'origin/trunk' into trunk
Merge remote-tracking branch 'origin/trunk' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8873afc4 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8873afc4 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8873afc4 Branch: refs/heads/trunk Commit: 8873afc481f5f315f00fae6efa3854135f16b7b9 Parents: 4082e53 346d8fb Author: Jonathan Ellis jbel...@apache.org Authored: Wed Mar 12 10:04:52 2014 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Wed Mar 12 10:04:52 2014 -0500 -- CHANGES.txt | 2 + pylib/cqlshlib/cql3handling.py | 13 +- .../apache/cassandra/cache/CachingOptions.java | 288 +++ .../org/apache/cassandra/cli/CliClient.java | 3 +- .../org/apache/cassandra/config/CFMetaData.java | 110 ++- .../cassandra/cql3/statements/CFPropDefs.java | 25 +- .../apache/cassandra/db/ColumnFamilyStore.java | 18 +- .../org/apache/cassandra/db/SystemKeyspace.java | 33 ++- .../cassandra/io/sstable/SSTableReader.java | 6 +- .../unit/org/apache/cassandra/SchemaLoader.java | 14 +- 10 files changed, 388 insertions(+), 124 deletions(-) --
[jira] [Commented] (CASSANDRA-4165) Generate Digest file for compressed SSTables
[ https://issues.apache.org/jira/browse/CASSANDRA-4165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931848#comment-13931848 ] Jonathan Ellis commented on CASSANDRA-4165: --- committed Generate Digest file for compressed SSTables Key: CASSANDRA-4165 URL: https://issues.apache.org/jira/browse/CASSANDRA-4165 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Marcus Eriksson Assignee: Jonathan Ellis Priority: Minor Labels: performance Fix For: 2.1 beta2 Attachments: 0001-Generate-digest-for-compressed-files-as-well.patch, 0002-dont-do-crc-and-add-digests-for-compressed-files.txt, 4165-rebased.txt We use the generated *Digest.sha1-files to verify backups, would be nice if they were generated for compressed sstables as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-5483) Repair tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-5483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931858#comment-13931858 ] Jonathan Ellis commented on CASSANDRA-5483: --- Can you post an example trace session? Does this add a verbose flag to trace to show the trace as-it-happens or do I have to manually check the trace tables? Repair tracing -- Key: CASSANDRA-5483 URL: https://issues.apache.org/jira/browse/CASSANDRA-5483 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Yuki Morishita Assignee: Ben Chan Priority: Minor Labels: repair Attachments: 5483-full-trunk.txt, 5483-v06-04-Allow-tracing-ttl-to-be-configured.patch, 5483-v06-05-Add-a-command-column-to-system_traces.events.patch, 5483-v06-06-Fix-interruption-in-tracestate-propagation.patch, 5483-v07-07-Better-constructor-parameters-for-DebuggableThreadPoolExecutor.patch, 5483-v07-08-Fix-brace-style.patch, 5483-v07-09-Add-trace-option-to-a-more-complete-set-of-repair-functions.patch, 5483-v07-10-Correct-name-of-boolean-repairedAt-to-fullRepair.patch, ccm-repair-test, test-5483-system_traces-events.txt, trunk@4620823-5483-v02-0001-Trace-filtering-and-tracestate-propagation.patch, trunk@4620823-5483-v02-0002-Put-a-few-traces-parallel-to-the-repair-logging.patch, tr...@8ebeee1-5483-v01-001-trace-filtering-and-tracestate-propagation.txt, tr...@8ebeee1-5483-v01-002-simple-repair-tracing.txt, v02p02-5483-v03-0003-Make-repair-tracing-controllable-via-nodetool.patch, v02p02-5483-v04-0003-This-time-use-an-EnumSet-to-pass-boolean-repair-options.patch, v02p02-5483-v05-0003-Use-long-instead-of-EnumSet-to-work-with-JMX.patch I think it would be nice to log repair stats and results like query tracing stores traces to system keyspace. With it, you don't have to lookup each log file to see what was the status and how it performed the repair you invoked. Instead, you can query the repair log with session ID to see the state and stats of all nodes involved in that repair session. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-5483) Repair tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-5483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931858#comment-13931858 ] Jonathan Ellis edited comment on CASSANDRA-5483 at 3/12/14 3:10 PM: Can you post an example trace session? Does this add a verbose flag to nodetool repair to show the trace as-it-happens or do I have to manually check the trace tables? was (Author: jbellis): Can you post an example trace session? Does this add a verbose flag to trace to show the trace as-it-happens or do I have to manually check the trace tables? Repair tracing -- Key: CASSANDRA-5483 URL: https://issues.apache.org/jira/browse/CASSANDRA-5483 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Yuki Morishita Assignee: Ben Chan Priority: Minor Labels: repair Attachments: 5483-full-trunk.txt, 5483-v06-04-Allow-tracing-ttl-to-be-configured.patch, 5483-v06-05-Add-a-command-column-to-system_traces.events.patch, 5483-v06-06-Fix-interruption-in-tracestate-propagation.patch, 5483-v07-07-Better-constructor-parameters-for-DebuggableThreadPoolExecutor.patch, 5483-v07-08-Fix-brace-style.patch, 5483-v07-09-Add-trace-option-to-a-more-complete-set-of-repair-functions.patch, 5483-v07-10-Correct-name-of-boolean-repairedAt-to-fullRepair.patch, ccm-repair-test, test-5483-system_traces-events.txt, trunk@4620823-5483-v02-0001-Trace-filtering-and-tracestate-propagation.patch, trunk@4620823-5483-v02-0002-Put-a-few-traces-parallel-to-the-repair-logging.patch, tr...@8ebeee1-5483-v01-001-trace-filtering-and-tracestate-propagation.txt, tr...@8ebeee1-5483-v01-002-simple-repair-tracing.txt, v02p02-5483-v03-0003-Make-repair-tracing-controllable-via-nodetool.patch, v02p02-5483-v04-0003-This-time-use-an-EnumSet-to-pass-boolean-repair-options.patch, v02p02-5483-v05-0003-Use-long-instead-of-EnumSet-to-work-with-JMX.patch I think it would be nice to log repair stats and results like query tracing stores traces to system keyspace. With it, you don't have to lookup each log file to see what was the status and how it performed the repair you invoked. Instead, you can query the repair log with session ID to see the state and stats of all nodes involved in that repair session. -- This message was sent by Atlassian JIRA (v6.2#6252)
[2/3] git commit: add ChecksummedSequentialWriter
add ChecksummedSequentialWriter
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2e00ba8b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2e00ba8b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2e00ba8b
Branch: refs/heads/trunk
Commit: 2e00ba8bfebfa196283b9abbec73d2edd49af541
Parents: 9405ce0
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Mar 12 10:29:42 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Mar 12 10:29:42 2014 -0500
--
.../io/util/ChecksummedSequentialWriter.java| 36
1 file changed, 36 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2e00ba8b/src/java/org/apache/cassandra/io/util/ChecksummedSequentialWriter.java
--
diff --git
a/src/java/org/apache/cassandra/io/util/ChecksummedSequentialWriter.java
b/src/java/org/apache/cassandra/io/util/ChecksummedSequentialWriter.java
new file mode 100644
index 000..3c4c257
--- /dev/null
+++ b/src/java/org/apache/cassandra/io/util/ChecksummedSequentialWriter.java
@@ -0,0 +1,36 @@
+package org.apache.cassandra.io.util;
+
+import java.io.File;
+
+import org.apache.cassandra.io.sstable.Descriptor;
+
+public class ChecksummedSequentialWriter extends SequentialWriter
+{
+private final SequentialWriter crcWriter;
+private final DataIntegrityMetadata.ChecksumWriter crcMetadata;
+
+public ChecksummedSequentialWriter(File file, int bufferSize, boolean
skipIOCache, File crcPath)
+{
+super(file, bufferSize, skipIOCache);
+crcWriter = new SequentialWriter(crcPath, 8 * 1024, true);
+crcMetadata = new
DataIntegrityMetadata.ChecksumWriter(crcWriter.stream);
+crcMetadata.writeChunkSize(buffer.length);
+}
+
+protected void flushData()
+{
+super.flushData();
+crcMetadata.append(buffer, 0, validBufferBytes);
+}
+
+public void writeFullChecksum(Descriptor descriptor)
+{
+crcMetadata.writeFullChecksum(descriptor);
+}
+
+public void close()
+{
+super.close();
+crcWriter.close();
+}
+}
[3/3] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/50afb791 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/50afb791 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/50afb791 Branch: refs/heads/trunk Commit: 50afb791522b12b07dd31502c0de72e0d3536101 Parents: 9b12329 2e00ba8 Author: Jonathan Ellis jbel...@apache.org Authored: Wed Mar 12 10:29:49 2014 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Wed Mar 12 10:29:49 2014 -0500 -- .../io/util/ChecksummedSequentialWriter.java| 36 1 file changed, 36 insertions(+) --
[1/3] git commit: add ChecksummedSequentialWriter
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 9405ce0e2 - 2e00ba8bf
refs/heads/trunk 9b1232929 - 50afb7915
add ChecksummedSequentialWriter
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2e00ba8b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2e00ba8b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2e00ba8b
Branch: refs/heads/cassandra-2.1
Commit: 2e00ba8bfebfa196283b9abbec73d2edd49af541
Parents: 9405ce0
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Mar 12 10:29:42 2014 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Mar 12 10:29:42 2014 -0500
--
.../io/util/ChecksummedSequentialWriter.java| 36
1 file changed, 36 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2e00ba8b/src/java/org/apache/cassandra/io/util/ChecksummedSequentialWriter.java
--
diff --git
a/src/java/org/apache/cassandra/io/util/ChecksummedSequentialWriter.java
b/src/java/org/apache/cassandra/io/util/ChecksummedSequentialWriter.java
new file mode 100644
index 000..3c4c257
--- /dev/null
+++ b/src/java/org/apache/cassandra/io/util/ChecksummedSequentialWriter.java
@@ -0,0 +1,36 @@
+package org.apache.cassandra.io.util;
+
+import java.io.File;
+
+import org.apache.cassandra.io.sstable.Descriptor;
+
+public class ChecksummedSequentialWriter extends SequentialWriter
+{
+private final SequentialWriter crcWriter;
+private final DataIntegrityMetadata.ChecksumWriter crcMetadata;
+
+public ChecksummedSequentialWriter(File file, int bufferSize, boolean
skipIOCache, File crcPath)
+{
+super(file, bufferSize, skipIOCache);
+crcWriter = new SequentialWriter(crcPath, 8 * 1024, true);
+crcMetadata = new
DataIntegrityMetadata.ChecksumWriter(crcWriter.stream);
+crcMetadata.writeChunkSize(buffer.length);
+}
+
+protected void flushData()
+{
+super.flushData();
+crcMetadata.append(buffer, 0, validBufferBytes);
+}
+
+public void writeFullChecksum(Descriptor descriptor)
+{
+crcMetadata.writeFullChecksum(descriptor);
+}
+
+public void close()
+{
+super.close();
+crcWriter.close();
+}
+}
[jira] [Commented] (CASSANDRA-5483) Repair tracing
[
https://issues.apache.org/jira/browse/CASSANDRA-5483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931875#comment-13931875
]
Lyuben Todorov commented on CASSANDRA-5483:
---
It's a good idea to add a verbose option to show tracing as it happens.
Currently to enable logging to the system_traces.events table the {{-tr}} flag
has to be supplied {{./nodetool repair -tr}}, and then to view the traces
{{SELECT * FROM system_traces.events}}
{noformat}
cqlsh SELECT * FROM system_traces.events ;
session_id | event_id |
activity
| command | source| source_elapsed | thread
--+--+---+-+---++---
fedbe970-a9fa-11e3-ab61-2387bbb38302 | fedbe971-a9fa-11e3-ab61-2387bbb38302 |
Starting repair command #3, repairing 2 ranges for keyspace system_traces
(seq=true, full=true) | REPAIR | 127.0.0.1 | 11 |
Thread-16
fedbe970-a9fa-11e3-ab61-2387bbb38302 | fedc5ea0-a9fa-11e3-ab61-2387bbb38302 |
[repair #fedc3790-a9fa-11e3-ab61-2387bbb38302] new session: will sync
/127.0.0.1, /127.0.0.3 on range (-3074457345618258603,3074457345618258602] for
system_traces.[sessions, events] | REPAIR | 127.0.0.1 | 2493 |
AntiEntropySessions:7
fedbe970-a9fa-11e3-ab61-2387bbb38302 | fedef6b0-a9fa-11e3-ab61-2387bbb38302 |
[repair
#fedc3790-a9fa-11e3-ab61-2387bbb38302] requesting merkle trees for sessions (to
[/127.0.0.3, /127.0.0.1]) | REPAIR | 127.0.0.1 | 19687 |
RepairJobTask:2
fedbe970-a9fa-11e3-ab61-2387bbb38302 | fee167b0-a9fa-11e3-b120-2387bbb38302 |
[repair
#fedc3790-a9fa-11e3-ab61-2387bbb38302] Sending completed merkle tree to
/127.0.0.1 for system_traces/sessions | REPAIR | 127.0.0.3 | 31937 |
AntiEntropyStage:1
fedbe970-a9fa-11e3-ab61-2387bbb38302 | fee3b1a0-a9fa-11e3-ab61-2387bbb38302 |
[repair #fedc3790-a9fa-11e3-ab61-2387bbb38302] Received merkle tree for
sessions from /127.0.0.3 | REPAIR | 127.0.0.1 | 50996 |
AntiEntropyStage:1
fedbe970-a9fa-11e3-ab61-2387bbb38302 | fee7d050-a9fa-11e3-ab61-2387bbb38302 |
[repair #fedc3790-a9fa-11e3-ab61-2387bbb38302] Received merkle tree for
sessions from /127.0.0.1 | REPAIR | 127.0.0.1 | 77907 |
AntiEntropyStage:1
fedbe970-a9fa-11e3-ab61-2387bbb38302 | feea1a40-a9fa-11e3-ab61-2387bbb38302 |
[repair
#fedc3790-a9fa-11e3-ab61-2387bbb38302] requesting merkle trees for events (to
[/127.0.0.3, /127.0.0.1]) | REPAIR | 127.0.0.1 | 92358 |
RepairJobTask:2
fedbe970-a9fa-11e3-ab61-2387bbb38302 | feecd960-a9fa-11e3-b120-2387bbb38302 |
[repair
#fedc3790-a9fa-11e3-ab61-2387bbb38302] Sending completed merkle tree to
/127.0.0.1 for system_traces/events | REPAIR | 127.0.0.3 | 107062 |
AntiEntropyStage:1
fedbe970-a9fa-11e3-ab61-2387bbb38302 | feef2350-a9fa-11e3-ab61-2387bbb38302 |
[repair #fedc3790-a9fa-11e3-ab61-2387bbb38302] Received merkle tree for
events from /127.0.0.3 | REPAIR | 127.0.0.1 | 126230 |
AntiEntropyStage:1
fedbe970-a9fa-11e3-ab61-2387bbb38302 | fef3de40-a9fa-11e3-ab61-2387bbb38302 |
[repair #fedc3790-a9fa-11e3-ab61-2387bbb38302] Received merkle tree for
events from /127.0.0.1 | REPAIR | 127.0.0.1 | 157175 |
AntiEntropyStage:1
fedbe970-a9fa-11e3-ab61-2387bbb38302 | fef40551-a9fa-11e3-ab61-2387bbb38302 |
[repair #fef40550-a9fa-11e3-ab61-2387bbb38302] new session: will sync
/127.0.0.1, /127.0.0.2 on range (3074457345618258602,-9223372036854775808] for
system_traces.[sessions, events] | REPAIR | 127.0.0.1 | 158225 |
AntiEntropySessions:8
fedbe970-a9fa-11e3-ab61-2387bbb38302 | fef4c8a0-a9fa-11e3-ab61-2387bbb38302 |
[repair
[jira] [Updated] (CASSANDRA-6841) ConcurrentModificationException in commit-log-writer after local schema reset
[
https://issues.apache.org/jira/browse/CASSANDRA-6841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-6841:
--
Summary: ConcurrentModificationException in commit-log-writer after local
schema reset (was: ConcurrentModificationException in commit-log-writer after
localschemareset)
Can you give more details regarding how you did this?
ConcurrentModificationException in commit-log-writer after local schema reset
-
Key: CASSANDRA-6841
URL: https://issues.apache.org/jira/browse/CASSANDRA-6841
Project: Cassandra
Issue Type: Bug
Environment: Linux 3.2.0 (Debian Wheezy) Cassandra 2.0.6, Oracle JVM
1.7.0_51
Almost default cassandra.yaml (IPs and cluster name changed)
This is the 2nd node in a 2-node ring. It has ~2500 keyspaces and very low
traffic. (Only new keyspaces see reads and writes.)
Reporter: Pas
Priority: Minor
{code}
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,013
MigrationManager.java (line 329) Starting local schema reset...
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,016
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@394448776(114/1140 serialized/live bytes, 3 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,016 Memtable.java (line 331)
Writing Memtable-local@394448776(114/1140 serialized/live bytes, 3 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,182 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-398-Data.db (145 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33159822)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,185
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@1087210140(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,185 Memtable.java (line 331)
Writing Memtable-local@1087210140(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,357 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-399-Data.db (96 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33159959)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,361
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@768887091(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,361 Memtable.java (line 331)
Writing Memtable-local@768887091(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,516 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-400-Data.db (96 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33160096)
INFO [CompactionExecutor:38] 2014-03-12 11:37:54,517 CompactionTask.java
(line 115) Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-398-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-400-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-399-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-397-Data.db')]
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,519
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@271993477(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,519 Memtable.java (line 331)
Writing Memtable-local@271993477(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,794 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-401-Data.db (96 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33160233)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,799
MigrationManager.java (line 357) Local schema reset is complete.
INFO [CompactionExecutor:38] 2014-03-12 11:37:54,848 CompactionTask.java
(line 275) Compacted 4 sstables to
[/var/lib/cassandra/data/system/local/system-local-jb-402,]. 6,099 bytes to
5,821 (~95% of original) in 330ms = 0.016822MB/s. 4 total partitions merged
to 1. Partition merge counts were {4:1, }
INFO [OptionalTasks:1] 2014-03-12 11:37:55,110 ColumnFamilyStore.java (line
785) Enqueuing flush of
Memtable-schema_columnfamilies@106276050(181506/509164 serialized/live bytes,
3276 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:55,110 Memtable.java (line 331)
Writing Memtable-schema_columnfamilies@106276050(181506/509164
serialized/live bytes, 3276 ops)
INFO [OptionalTasks:1] 2014-03-12 11:37:55,110 ColumnFamilyStore.java (line
785)
[jira] [Commented] (CASSANDRA-6841) ConcurrentModificationException in commit-log-writer after local schema reset
[
https://issues.apache.org/jira/browse/CASSANDRA-6841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931897#comment-13931897
]
Benedict commented on CASSANDRA-6841:
-
Without investigating _thoroughly_ to rule out anything else, I would guess the
problem is with Keyspace.cfIdMap, which is not threadsafe. We should probably
create it with two ConcurrentHashMap, instead of the default (plain HashMap)
ConcurrentModificationException in commit-log-writer after local schema reset
-
Key: CASSANDRA-6841
URL: https://issues.apache.org/jira/browse/CASSANDRA-6841
Project: Cassandra
Issue Type: Bug
Environment: Linux 3.2.0 (Debian Wheezy) Cassandra 2.0.6, Oracle JVM
1.7.0_51
Almost default cassandra.yaml (IPs and cluster name changed)
This is the 2nd node in a 2-node ring. It has ~2500 keyspaces and very low
traffic. (Only new keyspaces see reads and writes.)
Reporter: Pas
Priority: Minor
{code}
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,013
MigrationManager.java (line 329) Starting local schema reset...
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,016
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@394448776(114/1140 serialized/live bytes, 3 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,016 Memtable.java (line 331)
Writing Memtable-local@394448776(114/1140 serialized/live bytes, 3 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,182 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-398-Data.db (145 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33159822)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,185
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@1087210140(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,185 Memtable.java (line 331)
Writing Memtable-local@1087210140(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,357 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-399-Data.db (96 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33159959)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,361
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@768887091(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,361 Memtable.java (line 331)
Writing Memtable-local@768887091(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,516 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-400-Data.db (96 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33160096)
INFO [CompactionExecutor:38] 2014-03-12 11:37:54,517 CompactionTask.java
(line 115) Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-398-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-400-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-399-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-397-Data.db')]
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,519
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@271993477(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,519 Memtable.java (line 331)
Writing Memtable-local@271993477(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,794 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-401-Data.db (96 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33160233)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,799
MigrationManager.java (line 357) Local schema reset is complete.
INFO [CompactionExecutor:38] 2014-03-12 11:37:54,848 CompactionTask.java
(line 275) Compacted 4 sstables to
[/var/lib/cassandra/data/system/local/system-local-jb-402,]. 6,099 bytes to
5,821 (~95% of original) in 330ms = 0.016822MB/s. 4 total partitions merged
to 1. Partition merge counts were {4:1, }
INFO [OptionalTasks:1] 2014-03-12 11:37:55,110 ColumnFamilyStore.java (line
785) Enqueuing flush of
Memtable-schema_columnfamilies@106276050(181506/509164 serialized/live bytes,
3276 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:55,110 Memtable.java (line 331)
Writing Memtable-schema_columnfamilies@106276050(181506/509164
serialized/live bytes, 3276 ops)
INFO [OptionalTasks:1] 2014-03-12
[jira] [Commented] (CASSANDRA-6307) Switch cqlsh from cassandra-dbapi2 to python-driver
[ https://issues.apache.org/jira/browse/CASSANDRA-6307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931910#comment-13931910 ] Tyler Hobbs commented on CASSANDRA-6307: +1 as well. Thanks for your excellent work, [~mishail]! Switch cqlsh from cassandra-dbapi2 to python-driver --- Key: CASSANDRA-6307 URL: https://issues.apache.org/jira/browse/CASSANDRA-6307 Project: Cassandra Issue Type: Improvement Reporter: Aleksey Yeschenko Assignee: Mikhail Stepura Priority: Minor Fix For: 2.1 beta2 python-driver is hitting 1.0 soon. cassandra-dbapi2 development has stalled. It's time to switch cqlsh to native protocol and cassandra-dbapi2, especially now that 1. Some CQL3 things are not supported by Thrift transport 2. cqlsh no longer has to support CQL2 (dropped in 2.0) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6845) Cleanup fails with IndexOutOfBoundsException exception
[
https://issues.apache.org/jira/browse/CASSANDRA-6845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-6845:
--
Reviewer: Marcus Eriksson
Labels: (was: newbie patch)
([~krummas] to review)
Cleanup fails with IndexOutOfBoundsException exception
--
Key: CASSANDRA-6845
URL: https://issues.apache.org/jira/browse/CASSANDRA-6845
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: CentOS Linux 2.6.32-431.el6.x86_64
Datastax Cassandra 2.0.5-1
Reporter: Dmitrij Koniajev
Attachments: 0001-fixed-bug-in-CompactionManager.needsCleanup.patch
Originally we had Cassandra cluster with the following configuration:
{quote}
Initial tokens:
10.1.11.51: -9223372036854775808
10.1.11.52: 0
10.2.11.51: -9223372036854775807
10.2.11.52: 1
Endpoint snitch: PropertyFileSnitch
10.1.11.51=DC1:RAC1
10.1.11.52=DC1:RAC1
10.2.11.51=DC2:RAC1
10.2.11.52=DC2:RAC1
{quote}
We have created keyspace with network topology strategy and replication
factor 2:
{quote}
CREATE KEYSPACE our_keyspace WITH replication = {
'class': 'NetworkTopologyStrategy',
'DC2': '2',
'DC1': '2'
};
{quote}
All column families have been created with SizeTiered compaction strategy.
Аfter a while column families have been migrated to LeveledCompactionStrategy
and then replication factor was decreased to 1.
We started cleanup, but on non-seed nodes (10.1.11.52,10.2.11.52) it failed
with assertion error. Assertion stacktrace was the same as described in
[CASSANDRA-6774|https://issues.apache.org/jira/browse/CASSANDRA-6774] issue.
Then we've stopped all compactions, started cleanup again and got another
exception:
{quote}
Error occurred during cleanup
java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException:
Index: 1, Size: 1
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:227)
at
org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:265)
at
org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1115)
at
org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native
[jira] [Created] (CASSANDRA-6845) Cleanup fails with IndexOutOfBoundsException exception
Dmitrij Koniajev created CASSANDRA-6845:
---
Summary: Cleanup fails with IndexOutOfBoundsException exception
Key: CASSANDRA-6845
URL: https://issues.apache.org/jira/browse/CASSANDRA-6845
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: CentOS Linux 2.6.32-431.el6.x86_64
Datastax Cassandra 2.0.5-1
Reporter: Dmitrij Koniajev
Attachments: 0001-fixed-bug-in-CompactionManager.needsCleanup.patch
Originally we had Cassandra cluster with the following configuration:
{quote}
Initial tokens:
10.1.11.51: -9223372036854775808
10.1.11.52: 0
10.2.11.51: -9223372036854775807
10.2.11.52: 1
Endpoint snitch: PropertyFileSnitch
10.1.11.51=DC1:RAC1
10.1.11.52=DC1:RAC1
10.2.11.51=DC2:RAC1
10.2.11.52=DC2:RAC1
{quote}
We have created keyspace with network topology strategy and replication factor
2:
{quote}
CREATE KEYSPACE our_keyspace WITH replication = {
'class': 'NetworkTopologyStrategy',
'DC2': '2',
'DC1': '2'
};
{quote}
All column families have been created with SizeTiered compaction strategy.
Аfter a while column families have been migrated to LeveledCompactionStrategy
and then replication factor was decreased to 1.
We started cleanup, but on non-seed nodes (10.1.11.52,10.2.11.52) it failed
with assertion error. Assertion stacktrace was the same as described in
[CASSANDRA-6774|https://issues.apache.org/jira/browse/CASSANDRA-6774] issue.
Then we've stopped all compactions, started cleanup again and got another
exception:
{quote}
Error occurred during cleanup
java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException:
Index: 1, Size: 1
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:227)
at
org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:265)
at
org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1115)
at
org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
at
[jira] [Assigned] (CASSANDRA-6845) Cleanup fails with IndexOutOfBoundsException exception
[
https://issues.apache.org/jira/browse/CASSANDRA-6845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis reassigned CASSANDRA-6845:
-
Assignee: Dmitrij Koniajev
Cleanup fails with IndexOutOfBoundsException exception
--
Key: CASSANDRA-6845
URL: https://issues.apache.org/jira/browse/CASSANDRA-6845
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: CentOS Linux 2.6.32-431.el6.x86_64
Datastax Cassandra 2.0.5-1
Reporter: Dmitrij Koniajev
Assignee: Dmitrij Koniajev
Attachments: 0001-fixed-bug-in-CompactionManager.needsCleanup.patch
Originally we had Cassandra cluster with the following configuration:
{quote}
Initial tokens:
10.1.11.51: -9223372036854775808
10.1.11.52: 0
10.2.11.51: -9223372036854775807
10.2.11.52: 1
Endpoint snitch: PropertyFileSnitch
10.1.11.51=DC1:RAC1
10.1.11.52=DC1:RAC1
10.2.11.51=DC2:RAC1
10.2.11.52=DC2:RAC1
{quote}
We have created keyspace with network topology strategy and replication
factor 2:
{quote}
CREATE KEYSPACE our_keyspace WITH replication = {
'class': 'NetworkTopologyStrategy',
'DC2': '2',
'DC1': '2'
};
{quote}
All column families have been created with SizeTiered compaction strategy.
Аfter a while column families have been migrated to LeveledCompactionStrategy
and then replication factor was decreased to 1.
We started cleanup, but on non-seed nodes (10.1.11.52,10.2.11.52) it failed
with assertion error. Assertion stacktrace was the same as described in
[CASSANDRA-6774|https://issues.apache.org/jira/browse/CASSANDRA-6774] issue.
Then we've stopped all compactions, started cleanup again and got another
exception:
{quote}
Error occurred during cleanup
java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException:
Index: 1, Size: 1
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:227)
at
org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:265)
at
org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1115)
at
org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
[jira] [Updated] (CASSANDRA-6845) Cleanup fails with IndexOutOfBoundsException exception
[
https://issues.apache.org/jira/browse/CASSANDRA-6845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitrij Koniajev updated CASSANDRA-6845:
Description:
Originally we had Cassandra cluster with the following configuration:
{quote}
Initial tokens:
10.1.11.51: -9223372036854775808
10.1.11.52: 0
10.2.11.51: -9223372036854775807
10.2.11.52: 1
Endpoint snitch: PropertyFileSnitch
10.1.11.51=DC1:RAC1
10.1.11.52=DC1:RAC1
10.2.11.51=DC2:RAC1
10.2.11.52=DC2:RAC1
{quote}
We have created keyspace with network topology strategy and replication factor
2:
{code}
CREATE KEYSPACE our_keyspace WITH replication = {
'class': 'NetworkTopologyStrategy',
'DC2': '2',
'DC1': '2'
};
{code}
All column families have been created with SizeTiered compaction strategy.
Аfter a while column families have been migrated to LeveledCompactionStrategy
and then replication factor was decreased to 1.
We started cleanup, but on non-seed nodes (10.1.11.52,10.2.11.52) it failed
with assertion error. Assertion stacktrace was the same as described in
[CASSANDRA-6774|https://issues.apache.org/jira/browse/CASSANDRA-6774] issue.
Then we've stopped all compactions, started cleanup again and got another
exception:
{quote}
Error occurred during cleanup
java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException:
Index: 1, Size: 1
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:227)
at
org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:265)
at
org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1115)
at
org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at
[jira] [Commented] (CASSANDRA-6843) Reclaim overwritten off-heap memtable space early
[ https://issues.apache.org/jira/browse/CASSANDRA-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931931#comment-13931931 ] Benedict commented on CASSANDRA-6843: - Patch available [here|https://github.com/belliottsmith/cassandra/tree/iss-6843] This patch introduces GC (early reclaim) to the NativePool and NativeAllocator implementations introduced in CASSANDRA-6842, which this patch also depends on. The patch is reasonably simple as the memory management is already largely setup. The main change/addition is a new NativeCleaner and some life cycle management to PoolAllocator. Reclaim overwritten off-heap memtable space early - Key: CASSANDRA-6843 URL: https://issues.apache.org/jira/browse/CASSANDRA-6843 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Labels: performance Fix For: 3.0 Perform GC on the off-heap memtable arenas -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (CASSANDRA-6841) ConcurrentModificationException in commit-log-writer after local schema reset
[
https://issues.apache.org/jira/browse/CASSANDRA-6841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benedict reassigned CASSANDRA-6841:
---
Assignee: Benedict
ConcurrentModificationException in commit-log-writer after local schema reset
-
Key: CASSANDRA-6841
URL: https://issues.apache.org/jira/browse/CASSANDRA-6841
Project: Cassandra
Issue Type: Bug
Environment: Linux 3.2.0 (Debian Wheezy) Cassandra 2.0.6, Oracle JVM
1.7.0_51
Almost default cassandra.yaml (IPs and cluster name changed)
This is the 2nd node in a 2-node ring. It has ~2500 keyspaces and very low
traffic. (Only new keyspaces see reads and writes.)
Reporter: Pas
Assignee: Benedict
Priority: Minor
{code}
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,013
MigrationManager.java (line 329) Starting local schema reset...
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,016
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@394448776(114/1140 serialized/live bytes, 3 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,016 Memtable.java (line 331)
Writing Memtable-local@394448776(114/1140 serialized/live bytes, 3 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,182 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-398-Data.db (145 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33159822)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,185
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@1087210140(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,185 Memtable.java (line 331)
Writing Memtable-local@1087210140(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,357 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-399-Data.db (96 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33159959)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,361
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@768887091(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,361 Memtable.java (line 331)
Writing Memtable-local@768887091(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,516 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-400-Data.db (96 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33160096)
INFO [CompactionExecutor:38] 2014-03-12 11:37:54,517 CompactionTask.java
(line 115) Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-398-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-400-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-399-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-397-Data.db')]
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,519
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@271993477(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,519 Memtable.java (line 331)
Writing Memtable-local@271993477(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,794 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-401-Data.db (96 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33160233)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,799
MigrationManager.java (line 357) Local schema reset is complete.
INFO [CompactionExecutor:38] 2014-03-12 11:37:54,848 CompactionTask.java
(line 275) Compacted 4 sstables to
[/var/lib/cassandra/data/system/local/system-local-jb-402,]. 6,099 bytes to
5,821 (~95% of original) in 330ms = 0.016822MB/s. 4 total partitions merged
to 1. Partition merge counts were {4:1, }
INFO [OptionalTasks:1] 2014-03-12 11:37:55,110 ColumnFamilyStore.java (line
785) Enqueuing flush of
Memtable-schema_columnfamilies@106276050(181506/509164 serialized/live bytes,
3276 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:55,110 Memtable.java (line 331)
Writing Memtable-schema_columnfamilies@106276050(181506/509164
serialized/live bytes, 3276 ops)
INFO [OptionalTasks:1] 2014-03-12 11:37:55,110 ColumnFamilyStore.java (line
785) Enqueuing flush of Memtable-schema_columns@252242773(185191/630698
serialized/live bytes, 3614 ops)
ERROR [COMMIT-LOG-WRITER] 2014-03-12 11:37:55,111 CassandraDaemon.java (line
[jira] [Created] (CASSANDRA-6846) Provide standard interface for deep application server integration
Tupshin Harper created CASSANDRA-6846: - Summary: Provide standard interface for deep application server integration Key: CASSANDRA-6846 URL: https://issues.apache.org/jira/browse/CASSANDRA-6846 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Tupshin Harper Assignee: Tupshin Harper Fix For: 3.0 Instead of creating a pluggable interface for Thrift, I'd like to create a pluggable interface for arbitrary app-server deep integration. Inspired by both the existence of intravert-ug, as well as there being a long history of various parties embedding tomcat or jetty servlet engines inside Cassandra, I'd like to propose the creation an internal somewhat stable (versioned?) interface that could allow any app server to achieve deep integration with Cassandra, and as a result, these servers could 1) host their own apis (REST, for example 2) extend core functionality by having limited (see triggers and wide row scanners) access to the internals of cassandra The hand wavey part comes because while I have been mulling this about for a while, I have not spent any significant time into looking at the actual surface area of intravert-ug's integration. But, using it as a model, and also keeping in minds the general needs of your more traditional servlet/j2ee containers, I believe we could come up with a reasonable interface to allow any jvm app server to be integrated and maintained in or out of the Cassandra tree. This would satisfy the needs that many of us (Both Ed and I, for example) to have a much greater degree of control over server side execution, and to be able to start building much more interestingly (and simply) tiered applications. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6846) Provide standard interface for deep application server integration
[ https://issues.apache.org/jira/browse/CASSANDRA-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931951#comment-13931951 ] Russell Bradberry commented on CASSANDRA-6846: -- :+1: I'd like to take it one step further and even make parts of Cassandra, like CQL, use the interface as well. Something of eating one's own dog food. That way the interface will grow with the features that are added to things like CQL and it won't be a constant battle of Feature X was added to CQL can we please get it exposed in the interface Provide standard interface for deep application server integration -- Key: CASSANDRA-6846 URL: https://issues.apache.org/jira/browse/CASSANDRA-6846 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Tupshin Harper Assignee: Tupshin Harper Fix For: 3.0 Instead of creating a pluggable interface for Thrift, I'd like to create a pluggable interface for arbitrary app-server deep integration. Inspired by both the existence of intravert-ug, as well as there being a long history of various parties embedding tomcat or jetty servlet engines inside Cassandra, I'd like to propose the creation an internal somewhat stable (versioned?) interface that could allow any app server to achieve deep integration with Cassandra, and as a result, these servers could 1) host their own apis (REST, for example 2) extend core functionality by having limited (see triggers and wide row scanners) access to the internals of cassandra The hand wavey part comes because while I have been mulling this about for a while, I have not spent any significant time into looking at the actual surface area of intravert-ug's integration. But, using it as a model, and also keeping in minds the general needs of your more traditional servlet/j2ee containers, I believe we could come up with a reasonable interface to allow any jvm app server to be integrated and maintained in or out of the Cassandra tree. This would satisfy the needs that many of us (Both Ed and I, for example) to have a much greater degree of control over server side execution, and to be able to start building much more interestingly (and simply) tiered applications. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-6846) Provide standard interface for deep application server integration
[ https://issues.apache.org/jira/browse/CASSANDRA-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931951#comment-13931951 ] Russell Bradberry edited comment on CASSANDRA-6846 at 3/12/14 4:34 PM: --- +1 I'd like to take it one step further and even make parts of Cassandra, like CQL, use the interface as well. Something of eating one's own dog food. That way the interface will grow with the features that are added to things like CQL and it won't be a constant battle of Feature X was added to CQL can we please get it exposed in the interface was (Author: devdazed): :+1: I'd like to take it one step further and even make parts of Cassandra, like CQL, use the interface as well. Something of eating one's own dog food. That way the interface will grow with the features that are added to things like CQL and it won't be a constant battle of Feature X was added to CQL can we please get it exposed in the interface Provide standard interface for deep application server integration -- Key: CASSANDRA-6846 URL: https://issues.apache.org/jira/browse/CASSANDRA-6846 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Tupshin Harper Assignee: Tupshin Harper Fix For: 3.0 Instead of creating a pluggable interface for Thrift, I'd like to create a pluggable interface for arbitrary app-server deep integration. Inspired by both the existence of intravert-ug, as well as there being a long history of various parties embedding tomcat or jetty servlet engines inside Cassandra, I'd like to propose the creation an internal somewhat stable (versioned?) interface that could allow any app server to achieve deep integration with Cassandra, and as a result, these servers could 1) host their own apis (REST, for example 2) extend core functionality by having limited (see triggers and wide row scanners) access to the internals of cassandra The hand wavey part comes because while I have been mulling this about for a while, I have not spent any significant time into looking at the actual surface area of intravert-ug's integration. But, using it as a model, and also keeping in minds the general needs of your more traditional servlet/j2ee containers, I believe we could come up with a reasonable interface to allow any jvm app server to be integrated and maintained in or out of the Cassandra tree. This would satisfy the needs that many of us (Both Ed and I, for example) to have a much greater degree of control over server side execution, and to be able to start building much more interestingly (and simply) tiered applications. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6846) Provide standard interface for deep application server integration
[ https://issues.apache.org/jira/browse/CASSANDRA-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931958#comment-13931958 ] Nate McCall commented on CASSANDRA-6846: In hacking on this stuff in the past, what I would like to see here is something(s) wrapping a tighter API around: - StorageProxy - StorageService - Schema - MigrationManager Probably missing a few things, but those are the immediate items. None of them really move around to much, but having a contract on them or encapsulating them would be ideal. Provide standard interface for deep application server integration -- Key: CASSANDRA-6846 URL: https://issues.apache.org/jira/browse/CASSANDRA-6846 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Tupshin Harper Assignee: Tupshin Harper Fix For: 3.0 Instead of creating a pluggable interface for Thrift, I'd like to create a pluggable interface for arbitrary app-server deep integration. Inspired by both the existence of intravert-ug, as well as there being a long history of various parties embedding tomcat or jetty servlet engines inside Cassandra, I'd like to propose the creation an internal somewhat stable (versioned?) interface that could allow any app server to achieve deep integration with Cassandra, and as a result, these servers could 1) host their own apis (REST, for example 2) extend core functionality by having limited (see triggers and wide row scanners) access to the internals of cassandra The hand wavey part comes because while I have been mulling this about for a while, I have not spent any significant time into looking at the actual surface area of intravert-ug's integration. But, using it as a model, and also keeping in minds the general needs of your more traditional servlet/j2ee containers, I believe we could come up with a reasonable interface to allow any jvm app server to be integrated and maintained in or out of the Cassandra tree. This would satisfy the needs that many of us (Both Ed and I, for example) to have a much greater degree of control over server side execution, and to be able to start building much more interestingly (and simply) tiered applications. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6846) Provide standard interface for deep application server integration
[ https://issues.apache.org/jira/browse/CASSANDRA-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-6846: -- Priority: Minor (was: Major) Fix Version/s: (was: 3.0) Provide standard interface for deep application server integration -- Key: CASSANDRA-6846 URL: https://issues.apache.org/jira/browse/CASSANDRA-6846 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Tupshin Harper Assignee: Tupshin Harper Priority: Minor Labels: ponies Instead of creating a pluggable interface for Thrift, I'd like to create a pluggable interface for arbitrary app-server deep integration. Inspired by both the existence of intravert-ug, as well as there being a long history of various parties embedding tomcat or jetty servlet engines inside Cassandra, I'd like to propose the creation an internal somewhat stable (versioned?) interface that could allow any app server to achieve deep integration with Cassandra, and as a result, these servers could 1) host their own apis (REST, for example 2) extend core functionality by having limited (see triggers and wide row scanners) access to the internals of cassandra The hand wavey part comes because while I have been mulling this about for a while, I have not spent any significant time into looking at the actual surface area of intravert-ug's integration. But, using it as a model, and also keeping in minds the general needs of your more traditional servlet/j2ee containers, I believe we could come up with a reasonable interface to allow any jvm app server to be integrated and maintained in or out of the Cassandra tree. This would satisfy the needs that many of us (Both Ed and I, for example) to have a much greater degree of control over server side execution, and to be able to start building much more interestingly (and simply) tiered applications. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6846) Provide standard interface for deep application server integration
[ https://issues.apache.org/jira/browse/CASSANDRA-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-6846: - Labels: (╯°□°)╯︵┻━┻ ponies (was: ponies) Provide standard interface for deep application server integration -- Key: CASSANDRA-6846 URL: https://issues.apache.org/jira/browse/CASSANDRA-6846 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Tupshin Harper Assignee: Tupshin Harper Priority: Minor Labels: (╯°□°)╯︵┻━┻, ponies Instead of creating a pluggable interface for Thrift, I'd like to create a pluggable interface for arbitrary app-server deep integration. Inspired by both the existence of intravert-ug, as well as there being a long history of various parties embedding tomcat or jetty servlet engines inside Cassandra, I'd like to propose the creation an internal somewhat stable (versioned?) interface that could allow any app server to achieve deep integration with Cassandra, and as a result, these servers could 1) host their own apis (REST, for example 2) extend core functionality by having limited (see triggers and wide row scanners) access to the internals of cassandra The hand wavey part comes because while I have been mulling this about for a while, I have not spent any significant time into looking at the actual surface area of intravert-ug's integration. But, using it as a model, and also keeping in minds the general needs of your more traditional servlet/j2ee containers, I believe we could come up with a reasonable interface to allow any jvm app server to be integrated and maintained in or out of the Cassandra tree. This would satisfy the needs that many of us (Both Ed and I, for example) to have a much greater degree of control over server side execution, and to be able to start building much more interestingly (and simply) tiered applications. -- This message was sent by Atlassian JIRA (v6.2#6252)
git commit: Restore expiring-deleted (cell) compaction optimization
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 ed93ad4d8 - 19444ea16
Restore expiring-deleted (cell) compaction optimization
patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-6844
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/19444ea1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/19444ea1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/19444ea1
Branch: refs/heads/cassandra-2.0
Commit: 19444ea1655d76db11a8048c4ecde81f305018c9
Parents: ed93ad4
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Mar 12 19:37:40 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Mar 12 19:37:40 2014 +0300
--
CHANGES.txt| 1 +
.../org/apache/cassandra/io/sstable/SSTableIdentityIterator.java | 2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/19444ea1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 906f91f..42dc4ee 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -3,6 +3,7 @@
* Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
* Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
* Fix static counter columns (CASSANDRA-6827)
+ * Restore expiring-deleted (cell) compaction optimization (CASSANDRA-6844)
2.0.6
http://git-wip-us.apache.org/repos/asf/cassandra/blob/19444ea1/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
--
diff --git
a/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
b/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
index 4e23ac2..52da9bb 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
@@ -102,7 +102,7 @@ public class SSTableIdentityIterator implements
ComparableSSTableIdentityIterat
columnFamily = EmptyColumns.factory.create(metadata);
columnFamily.delete(DeletionTime.serializer.deserialize(in));
columnCount = dataVersion.hasRowSizeAndColumnCount ? in.readInt()
: Integer.MAX_VALUE;
-atomIterator = columnFamily.metadata().getOnDiskIterator(in,
columnCount, dataVersion);
+atomIterator = columnFamily.metadata().getOnDiskIterator(in,
columnCount, flag, expireBefore, dataVersion);
}
catch (IOException e)
{
[jira] [Updated] (CASSANDRA-6846) Provide standard interface for deep application server integration
[ https://issues.apache.org/jira/browse/CASSANDRA-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremiah Jordan updated CASSANDRA-6846: --- Labels: ponies (was: ) Provide standard interface for deep application server integration -- Key: CASSANDRA-6846 URL: https://issues.apache.org/jira/browse/CASSANDRA-6846 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Tupshin Harper Assignee: Tupshin Harper Labels: ponies Fix For: 3.0 Instead of creating a pluggable interface for Thrift, I'd like to create a pluggable interface for arbitrary app-server deep integration. Inspired by both the existence of intravert-ug, as well as there being a long history of various parties embedding tomcat or jetty servlet engines inside Cassandra, I'd like to propose the creation an internal somewhat stable (versioned?) interface that could allow any app server to achieve deep integration with Cassandra, and as a result, these servers could 1) host their own apis (REST, for example 2) extend core functionality by having limited (see triggers and wide row scanners) access to the internals of cassandra The hand wavey part comes because while I have been mulling this about for a while, I have not spent any significant time into looking at the actual surface area of intravert-ug's integration. But, using it as a model, and also keeping in minds the general needs of your more traditional servlet/j2ee containers, I believe we could come up with a reasonable interface to allow any jvm app server to be integrated and maintained in or out of the Cassandra tree. This would satisfy the needs that many of us (Both Ed and I, for example) to have a much greater degree of control over server side execution, and to be able to start building much more interestingly (and simply) tiered applications. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6846) Provide standard interface for deep application server integration
[ https://issues.apache.org/jira/browse/CASSANDRA-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931987#comment-13931987 ] Sylvain Lebresne commented on CASSANDRA-6846: - I'm definitively -1 on putting any type of contract on the internals. They are called internals for a reason, and if rewriting it all entirely tomorrow is best for Cassandra, we should have the possibility to do so. And if we're creating a new abstraction on top of it, well, we're just creating a new API, and well, I really thing we should focus on having just one API to Cassandra and focus efforts there. Provide standard interface for deep application server integration -- Key: CASSANDRA-6846 URL: https://issues.apache.org/jira/browse/CASSANDRA-6846 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Tupshin Harper Assignee: Tupshin Harper Priority: Minor Labels: (╯°□°)╯︵┻━┻, ponies Instead of creating a pluggable interface for Thrift, I'd like to create a pluggable interface for arbitrary app-server deep integration. Inspired by both the existence of intravert-ug, as well as there being a long history of various parties embedding tomcat or jetty servlet engines inside Cassandra, I'd like to propose the creation an internal somewhat stable (versioned?) interface that could allow any app server to achieve deep integration with Cassandra, and as a result, these servers could 1) host their own apis (REST, for example 2) extend core functionality by having limited (see triggers and wide row scanners) access to the internals of cassandra The hand wavey part comes because while I have been mulling this about for a while, I have not spent any significant time into looking at the actual surface area of intravert-ug's integration. But, using it as a model, and also keeping in minds the general needs of your more traditional servlet/j2ee containers, I believe we could come up with a reasonable interface to allow any jvm app server to be integrated and maintained in or out of the Cassandra tree. This would satisfy the needs that many of us (Both Ed and I, for example) to have a much greater degree of control over server side execution, and to be able to start building much more interestingly (and simply) tiered applications. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6841) ConcurrentModificationException in commit-log-writer after local schema reset
[
https://issues.apache.org/jira/browse/CASSANDRA-6841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benedict updated CASSANDRA-6841:
Fix Version/s: 2.0.7
ConcurrentModificationException in commit-log-writer after local schema reset
-
Key: CASSANDRA-6841
URL: https://issues.apache.org/jira/browse/CASSANDRA-6841
Project: Cassandra
Issue Type: Bug
Environment: Linux 3.2.0 (Debian Wheezy) Cassandra 2.0.6, Oracle JVM
1.7.0_51
Almost default cassandra.yaml (IPs and cluster name changed)
This is the 2nd node in a 2-node ring. It has ~2500 keyspaces and very low
traffic. (Only new keyspaces see reads and writes.)
Reporter: Pas
Assignee: Benedict
Priority: Minor
Fix For: 2.0.7
{code}
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,013
MigrationManager.java (line 329) Starting local schema reset...
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,016
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@394448776(114/1140 serialized/live bytes, 3 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,016 Memtable.java (line 331)
Writing Memtable-local@394448776(114/1140 serialized/live bytes, 3 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,182 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-398-Data.db (145 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33159822)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,185
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@1087210140(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,185 Memtable.java (line 331)
Writing Memtable-local@1087210140(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,357 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-399-Data.db (96 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33159959)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,361
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@768887091(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,361 Memtable.java (line 331)
Writing Memtable-local@768887091(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,516 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-400-Data.db (96 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33160096)
INFO [CompactionExecutor:38] 2014-03-12 11:37:54,517 CompactionTask.java
(line 115) Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-398-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-400-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-399-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-397-Data.db')]
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,519
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@271993477(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,519 Memtable.java (line 331)
Writing Memtable-local@271993477(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,794 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-401-Data.db (96 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33160233)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,799
MigrationManager.java (line 357) Local schema reset is complete.
INFO [CompactionExecutor:38] 2014-03-12 11:37:54,848 CompactionTask.java
(line 275) Compacted 4 sstables to
[/var/lib/cassandra/data/system/local/system-local-jb-402,]. 6,099 bytes to
5,821 (~95% of original) in 330ms = 0.016822MB/s. 4 total partitions merged
to 1. Partition merge counts were {4:1, }
INFO [OptionalTasks:1] 2014-03-12 11:37:55,110 ColumnFamilyStore.java (line
785) Enqueuing flush of
Memtable-schema_columnfamilies@106276050(181506/509164 serialized/live bytes,
3276 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:55,110 Memtable.java (line 331)
Writing Memtable-schema_columnfamilies@106276050(181506/509164
serialized/live bytes, 3276 ops)
INFO [OptionalTasks:1] 2014-03-12 11:37:55,110 ColumnFamilyStore.java (line
785) Enqueuing flush of Memtable-schema_columns@252242773(185191/630698
serialized/live bytes, 3614 ops)
ERROR [COMMIT-LOG-WRITER] 2014-03-12 11:37:55,111
[2/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1
Conflicts:
CHANGES.txt
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cf9ae5f9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cf9ae5f9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cf9ae5f9
Branch: refs/heads/trunk
Commit: cf9ae5f9cddd81de6e153eae48357ca0842bf499
Parents: 2e00ba8 19444ea
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Mar 12 19:58:35 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Mar 12 19:58:35 2014 +0300
--
CHANGES.txt | 1 +
.../io/sstable/SSTableIdentityIterator.java | 49 +++-
2 files changed, 8 insertions(+), 42 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf9ae5f9/CHANGES.txt
--
diff --cc CHANGES.txt
index 7a1fc60,42dc4ee..cb51c12
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -19,6 -3,10 +19,7 @@@ Merged from 2.0
* Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
* Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
* Fix static counter columns (CASSANDRA-6827)
+ * Restore expiring-deleted (cell) compaction optimization (CASSANDRA-6844)
-
-
-2.0.6
* Avoid race-prone second scrub of system keyspace (CASSANDRA-6797)
* Pool CqlRecordWriter clients by inetaddress rather than Range
(CASSANDRA-6665)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf9ae5f9/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
index 11f2f71,52da9bb..ce4b670
--- a/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
@@@ -20,33 -20,33 +20,21 @@@ package org.apache.cassandra.io.sstable
import java.io.*;
import java.util.Iterator;
--import org.apache.cassandra.serializers.MarshalException;
--import org.slf4j.Logger;
--import org.slf4j.LoggerFactory;
--
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.db.*;
import org.apache.cassandra.db.columniterator.OnDiskAtomIterator;
import org.apache.cassandra.io.util.RandomAccessReader;
++import org.apache.cassandra.serializers.MarshalException;
public class SSTableIdentityIterator implements
ComparableSSTableIdentityIterator, OnDiskAtomIterator
{
--private static final Logger logger =
LoggerFactory.getLogger(SSTableIdentityIterator.class);
--
private final DecoratedKey key;
private final DataInput in;
public final long dataSize; // we [still] require this so compaction can
tell if it's safe to read the row into memory
public final ColumnSerializer.Flag flag;
private final ColumnFamily columnFamily;
--private final int columnCount;
--
private final IteratorOnDiskAtom atomIterator;
--private final Descriptor.Version dataVersion;
--
--// Used by lazilyCompactedRow, so that we see the same things when
deserializing the first and second time
--private final int expireBefore;
--
private final boolean validateColumns;
private final String filename;
@@@ -56,7 -56,7 +44,6 @@@
* @param file Reading using this file.
* @param key Key of this row.
* @param dataSize length of row data
-- * @throws IOException
*/
public SSTableIdentityIterator(SSTableReader sstable, RandomAccessReader
file, DecoratedKey key, long dataSize)
{
@@@ -92,17 -92,17 +79,18 @@@
this.filename = filename;
this.key = key;
this.dataSize = dataSize;
--this.expireBefore = (int)(System.currentTimeMillis() / 1000);
this.flag = flag;
this.validateColumns = checkData;
--this.dataVersion = sstable == null ? Descriptor.Version.CURRENT :
sstable.descriptor.version;
++
++Descriptor.Version dataVersion = sstable == null ?
Descriptor.Version.CURRENT : sstable.descriptor.version;
++int expireBefore = (int) (System.currentTimeMillis() / 1000);
++columnFamily = ArrayBackedSortedColumns.factory.create(metadata);
try
{
- columnFamily = ArrayBackedSortedColumns.factory.create(metadata);
-columnFamily = EmptyColumns.factory.create(metadata);
columnFamily.delete(DeletionTime.serializer.deserialize(in));
--columnCount = dataVersion.hasRowSizeAndColumnCount ? in.readInt()
: Integer.MAX_VALUE;
- atomIterator = columnFamily.metadata().getOnDiskIterator(in,
columnCount,
[3/3] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c4b2320a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c4b2320a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c4b2320a Branch: refs/heads/trunk Commit: c4b2320aeff1fdbe05d56929c1c8c52ec8ae4c65 Parents: 50afb79 cf9ae5f Author: Aleksey Yeschenko alek...@apache.org Authored: Wed Mar 12 19:59:07 2014 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Wed Mar 12 19:59:07 2014 +0300 -- CHANGES.txt | 1 + .../io/sstable/SSTableIdentityIterator.java | 49 +++- 2 files changed, 8 insertions(+), 42 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c4b2320a/CHANGES.txt --
[1/2] git commit: Restore expiring-deleted (cell) compaction optimization
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 2e00ba8bf - cf9ae5f9c
Restore expiring-deleted (cell) compaction optimization
patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-6844
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/19444ea1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/19444ea1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/19444ea1
Branch: refs/heads/cassandra-2.1
Commit: 19444ea1655d76db11a8048c4ecde81f305018c9
Parents: ed93ad4
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Mar 12 19:37:40 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Mar 12 19:37:40 2014 +0300
--
CHANGES.txt| 1 +
.../org/apache/cassandra/io/sstable/SSTableIdentityIterator.java | 2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/19444ea1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 906f91f..42dc4ee 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -3,6 +3,7 @@
* Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
* Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
* Fix static counter columns (CASSANDRA-6827)
+ * Restore expiring-deleted (cell) compaction optimization (CASSANDRA-6844)
2.0.6
http://git-wip-us.apache.org/repos/asf/cassandra/blob/19444ea1/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
--
diff --git
a/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
b/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
index 4e23ac2..52da9bb 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
@@ -102,7 +102,7 @@ public class SSTableIdentityIterator implements
ComparableSSTableIdentityIterat
columnFamily = EmptyColumns.factory.create(metadata);
columnFamily.delete(DeletionTime.serializer.deserialize(in));
columnCount = dataVersion.hasRowSizeAndColumnCount ? in.readInt()
: Integer.MAX_VALUE;
-atomIterator = columnFamily.metadata().getOnDiskIterator(in,
columnCount, dataVersion);
+atomIterator = columnFamily.metadata().getOnDiskIterator(in,
columnCount, flag, expireBefore, dataVersion);
}
catch (IOException e)
{
[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1
Conflicts:
CHANGES.txt
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cf9ae5f9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cf9ae5f9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cf9ae5f9
Branch: refs/heads/cassandra-2.1
Commit: cf9ae5f9cddd81de6e153eae48357ca0842bf499
Parents: 2e00ba8 19444ea
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Mar 12 19:58:35 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Mar 12 19:58:35 2014 +0300
--
CHANGES.txt | 1 +
.../io/sstable/SSTableIdentityIterator.java | 49 +++-
2 files changed, 8 insertions(+), 42 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf9ae5f9/CHANGES.txt
--
diff --cc CHANGES.txt
index 7a1fc60,42dc4ee..cb51c12
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -19,6 -3,10 +19,7 @@@ Merged from 2.0
* Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
* Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
* Fix static counter columns (CASSANDRA-6827)
+ * Restore expiring-deleted (cell) compaction optimization (CASSANDRA-6844)
-
-
-2.0.6
* Avoid race-prone second scrub of system keyspace (CASSANDRA-6797)
* Pool CqlRecordWriter clients by inetaddress rather than Range
(CASSANDRA-6665)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf9ae5f9/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
index 11f2f71,52da9bb..ce4b670
--- a/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
@@@ -20,33 -20,33 +20,21 @@@ package org.apache.cassandra.io.sstable
import java.io.*;
import java.util.Iterator;
--import org.apache.cassandra.serializers.MarshalException;
--import org.slf4j.Logger;
--import org.slf4j.LoggerFactory;
--
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.db.*;
import org.apache.cassandra.db.columniterator.OnDiskAtomIterator;
import org.apache.cassandra.io.util.RandomAccessReader;
++import org.apache.cassandra.serializers.MarshalException;
public class SSTableIdentityIterator implements
ComparableSSTableIdentityIterator, OnDiskAtomIterator
{
--private static final Logger logger =
LoggerFactory.getLogger(SSTableIdentityIterator.class);
--
private final DecoratedKey key;
private final DataInput in;
public final long dataSize; // we [still] require this so compaction can
tell if it's safe to read the row into memory
public final ColumnSerializer.Flag flag;
private final ColumnFamily columnFamily;
--private final int columnCount;
--
private final IteratorOnDiskAtom atomIterator;
--private final Descriptor.Version dataVersion;
--
--// Used by lazilyCompactedRow, so that we see the same things when
deserializing the first and second time
--private final int expireBefore;
--
private final boolean validateColumns;
private final String filename;
@@@ -56,7 -56,7 +44,6 @@@
* @param file Reading using this file.
* @param key Key of this row.
* @param dataSize length of row data
-- * @throws IOException
*/
public SSTableIdentityIterator(SSTableReader sstable, RandomAccessReader
file, DecoratedKey key, long dataSize)
{
@@@ -92,17 -92,17 +79,18 @@@
this.filename = filename;
this.key = key;
this.dataSize = dataSize;
--this.expireBefore = (int)(System.currentTimeMillis() / 1000);
this.flag = flag;
this.validateColumns = checkData;
--this.dataVersion = sstable == null ? Descriptor.Version.CURRENT :
sstable.descriptor.version;
++
++Descriptor.Version dataVersion = sstable == null ?
Descriptor.Version.CURRENT : sstable.descriptor.version;
++int expireBefore = (int) (System.currentTimeMillis() / 1000);
++columnFamily = ArrayBackedSortedColumns.factory.create(metadata);
try
{
- columnFamily = ArrayBackedSortedColumns.factory.create(metadata);
-columnFamily = EmptyColumns.factory.create(metadata);
columnFamily.delete(DeletionTime.serializer.deserialize(in));
--columnCount = dataVersion.hasRowSizeAndColumnCount ? in.readInt()
: Integer.MAX_VALUE;
- atomIterator = columnFamily.metadata().getOnDiskIterator(in,
columnCount,
[1/3] git commit: Restore expiring-deleted (cell) compaction optimization
Repository: cassandra
Updated Branches:
refs/heads/trunk 50afb7915 - c4b2320ae
Restore expiring-deleted (cell) compaction optimization
patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-6844
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/19444ea1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/19444ea1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/19444ea1
Branch: refs/heads/trunk
Commit: 19444ea1655d76db11a8048c4ecde81f305018c9
Parents: ed93ad4
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Mar 12 19:37:40 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Mar 12 19:37:40 2014 +0300
--
CHANGES.txt| 1 +
.../org/apache/cassandra/io/sstable/SSTableIdentityIterator.java | 2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/19444ea1/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 906f91f..42dc4ee 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -3,6 +3,7 @@
* Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
* Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
* Fix static counter columns (CASSANDRA-6827)
+ * Restore expiring-deleted (cell) compaction optimization (CASSANDRA-6844)
2.0.6
http://git-wip-us.apache.org/repos/asf/cassandra/blob/19444ea1/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
--
diff --git
a/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
b/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
index 4e23ac2..52da9bb 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableIdentityIterator.java
@@ -102,7 +102,7 @@ public class SSTableIdentityIterator implements
ComparableSSTableIdentityIterat
columnFamily = EmptyColumns.factory.create(metadata);
columnFamily.delete(DeletionTime.serializer.deserialize(in));
columnCount = dataVersion.hasRowSizeAndColumnCount ? in.readInt()
: Integer.MAX_VALUE;
-atomIterator = columnFamily.metadata().getOnDiskIterator(in,
columnCount, dataVersion);
+atomIterator = columnFamily.metadata().getOnDiskIterator(in,
columnCount, flag, expireBefore, dataVersion);
}
catch (IOException e)
{
[jira] [Commented] (CASSANDRA-6846) Provide standard interface for deep application server integration
[ https://issues.apache.org/jira/browse/CASSANDRA-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13931997#comment-13931997 ] Peter commented on CASSANDRA-6846: -- In the process of building a temporal database on top of Cassandra, I'd like to see something like this to make it easier to do complex queries. Right now our schema modeler for plain Cassandra uses both Solr and Cassandra so that we can do or disjunction, IN with secondary columns, and like clauses. It would be nicer to have something on the serverside so that we can offload this from the client. Provide standard interface for deep application server integration -- Key: CASSANDRA-6846 URL: https://issues.apache.org/jira/browse/CASSANDRA-6846 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Tupshin Harper Assignee: Tupshin Harper Priority: Minor Labels: (╯°□°)╯︵┻━┻, ponies Instead of creating a pluggable interface for Thrift, I'd like to create a pluggable interface for arbitrary app-server deep integration. Inspired by both the existence of intravert-ug, as well as there being a long history of various parties embedding tomcat or jetty servlet engines inside Cassandra, I'd like to propose the creation an internal somewhat stable (versioned?) interface that could allow any app server to achieve deep integration with Cassandra, and as a result, these servers could 1) host their own apis (REST, for example 2) extend core functionality by having limited (see triggers and wide row scanners) access to the internals of cassandra The hand wavey part comes because while I have been mulling this about for a while, I have not spent any significant time into looking at the actual surface area of intravert-ug's integration. But, using it as a model, and also keeping in minds the general needs of your more traditional servlet/j2ee containers, I believe we could come up with a reasonable interface to allow any jvm app server to be integrated and maintained in or out of the Cassandra tree. This would satisfy the needs that many of us (Both Ed and I, for example) to have a much greater degree of control over server side execution, and to be able to start building much more interestingly (and simply) tiered applications. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6841) ConcurrentModificationException in commit-log-writer after local schema reset
[
https://issues.apache.org/jira/browse/CASSANDRA-6841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932005#comment-13932005
]
Pas commented on CASSANDRA-6841:
To give more context. There was a single node ring used for running test of an
application that uses, among other stores, Cassandra as a backend. It has a lot
of small keyspaces (each with 1 or 2 CFs - supercolumn based), total data on
the node was/is less than 300G. We are in the process of migrating away from
supercolumns, so we're moving to CQL and composite primary keys and whatnot, so
there are a few keyspaces already with this new schema.
Then we tried to add a new node, setting its seed property to the old one. It
failed to bootstrap with errors (
https://issues.apache.org/jira/browse/CASSANDRA-6565?focusedCommentId=13931732page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13931732
), but finally after a few tries it started and went from BOOT to NORMAL. Then
we noticed that the new node has a different schema UUID (as indicated by
gossipinfo), and tried to do resetlocalschema. (Which happened, according to
system.log, but then we have observed no improvement, so we tried it again then
the gossip stage appeared to have been deadlocked; but a restart solved it. See
also
https://issues.apache.org/jira/browse/CASSANDRA-6799?focusedCommentId=13931690page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13931690
)
ConcurrentModificationException in commit-log-writer after local schema reset
-
Key: CASSANDRA-6841
URL: https://issues.apache.org/jira/browse/CASSANDRA-6841
Project: Cassandra
Issue Type: Bug
Environment: Linux 3.2.0 (Debian Wheezy) Cassandra 2.0.6, Oracle JVM
1.7.0_51
Almost default cassandra.yaml (IPs and cluster name changed)
This is the 2nd node in a 2-node ring. It has ~2500 keyspaces and very low
traffic. (Only new keyspaces see reads and writes.)
Reporter: Pas
Assignee: Benedict
Priority: Minor
Fix For: 2.0.7
{code}
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,013
MigrationManager.java (line 329) Starting local schema reset...
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,016
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@394448776(114/1140 serialized/live bytes, 3 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,016 Memtable.java (line 331)
Writing Memtable-local@394448776(114/1140 serialized/live bytes, 3 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,182 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-398-Data.db (145 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33159822)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,185
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@1087210140(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,185 Memtable.java (line 331)
Writing Memtable-local@1087210140(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,357 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-399-Data.db (96 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33159959)
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,361
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@768887091(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,361 Memtable.java (line 331)
Writing Memtable-local@768887091(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,516 Memtable.java (line 371)
Completed flushing
/var/lib/cassandra/data/system/local/system-local-jb-400-Data.db (96 bytes)
for commitlog position ReplayPosition(segmentId=1394620057452,
position=33160096)
INFO [CompactionExecutor:38] 2014-03-12 11:37:54,517 CompactionTask.java
(line 115) Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-398-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-400-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-399-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-jb-397-Data.db')]
INFO [RMI TCP Connection(38)-192.168.36.171] 2014-03-12 11:37:54,519
ColumnFamilyStore.java (line 785) Enqueuing flush of
Memtable-local@271993477(62/620 serialized/live bytes, 1 ops)
INFO [FlushWriter:6] 2014-03-12 11:37:54,519 Memtable.java (line 331)
Writing Memtable-local@271993477(62/620 serialized/live bytes, 1
[jira] [Commented] (CASSANDRA-6759) CommitLogSegment should provide madvise DONTNEED hint after syncing a segment
[ https://issues.apache.org/jira/browse/CASSANDRA-6759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13932012#comment-13932012 ] Jonathan Ellis commented on CASSANDRA-6759: --- committed CommitLogSegment should provide madvise DONTNEED hint after syncing a segment - Key: CASSANDRA-6759 URL: https://issues.apache.org/jira/browse/CASSANDRA-6759 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Priority: Minor Fix For: 2.1 beta2 Attachments: 6759.txt This is a really simple change, copying what we do with SequentialWriter, but here we know for sure we don't intend to re-read the region again in the near future so it's definitely wasted file buffer space. I have a patch, but I would like to do some brief performance comparisons before submitting it, and see if providing DONTNEED at file creation like we do for SW is sensible. -- This message was sent by Atlassian JIRA (v6.2#6252)
git commit: fixed bug in CompactionManager.needsCleanup
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 19444ea16 - 05da045bb
fixed bug in CompactionManager.needsCleanup
Patch by Dmitrij Koniajev, reviewed by marcuse for CASSANDRA-6845.
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/05da045b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/05da045b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/05da045b
Branch: refs/heads/cassandra-2.0
Commit: 05da045bbfe29bf472982d3c5da90050176202df
Parents: 19444ea
Author: Dmitrij Koniajev dimchan...@gmail.com
Authored: Wed Mar 12 17:05:31 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Mar 12 18:14:59 2014 +0100
--
CHANGES.txt| 2 +-
src/java/org/apache/cassandra/db/compaction/CompactionManager.java | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/05da045b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 42dc4ee..3531737 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -4,7 +4,7 @@
* Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
* Fix static counter columns (CASSANDRA-6827)
* Restore expiring-deleted (cell) compaction optimization (CASSANDRA-6844)
-
+ * Fix CompactionManager.needsCleanup (CASSANDRA-6845)
2.0.6
* Avoid race-prone second scrub of system keyspace (CASSANDRA-6797)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/05da045b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 02cf433..453176e 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -493,7 +493,7 @@ public class CompactionManager implements
CompactionManagerMBean
return false;
}
-if (i == (ownedRanges.size() - 1))
+if (i == (sortedRanges.size() - 1))
{
// we're at the last range and we found a key beyond the end
of the range
return true;
[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/019e3386 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/019e3386 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/019e3386 Branch: refs/heads/cassandra-2.1 Commit: 019e338670ecbf84fab9ae9814db0c5fcf2f1dff Parents: cf9ae5f 05da045 Author: Marcus Eriksson marc...@apache.org Authored: Wed Mar 12 18:17:22 2014 +0100 Committer: Marcus Eriksson marc...@apache.org Committed: Wed Mar 12 18:17:22 2014 +0100 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/db/compaction/CompactionManager.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/019e3386/CHANGES.txt -- diff --cc CHANGES.txt index cb51c12,3531737..00cf087 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -20,6 -4,9 +20,7 @@@ Merged from 2.0 * Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838) * Fix static counter columns (CASSANDRA-6827) * Restore expiring-deleted (cell) compaction optimization (CASSANDRA-6844) + * Fix CompactionManager.needsCleanup (CASSANDRA-6845) - -2.0.6 * Avoid race-prone second scrub of system keyspace (CASSANDRA-6797) * Pool CqlRecordWriter clients by inetaddress rather than Range (CASSANDRA-6665) http://git-wip-us.apache.org/repos/asf/cassandra/blob/019e3386/src/java/org/apache/cassandra/db/compaction/CompactionManager.java --
[1/2] git commit: fixed bug in CompactionManager.needsCleanup
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 cf9ae5f9c - 019e33867
fixed bug in CompactionManager.needsCleanup
Patch by Dmitrij Koniajev, reviewed by marcuse for CASSANDRA-6845.
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/05da045b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/05da045b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/05da045b
Branch: refs/heads/cassandra-2.1
Commit: 05da045bbfe29bf472982d3c5da90050176202df
Parents: 19444ea
Author: Dmitrij Koniajev dimchan...@gmail.com
Authored: Wed Mar 12 17:05:31 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Mar 12 18:14:59 2014 +0100
--
CHANGES.txt| 2 +-
src/java/org/apache/cassandra/db/compaction/CompactionManager.java | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/05da045b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 42dc4ee..3531737 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -4,7 +4,7 @@
* Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
* Fix static counter columns (CASSANDRA-6827)
* Restore expiring-deleted (cell) compaction optimization (CASSANDRA-6844)
-
+ * Fix CompactionManager.needsCleanup (CASSANDRA-6845)
2.0.6
* Avoid race-prone second scrub of system keyspace (CASSANDRA-6797)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/05da045b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 02cf433..453176e 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -493,7 +493,7 @@ public class CompactionManager implements
CompactionManagerMBean
return false;
}
-if (i == (ownedRanges.size() - 1))
+if (i == (sortedRanges.size() - 1))
{
// we're at the last range and we found a key beyond the end
of the range
return true;
[1/3] git commit: fixed bug in CompactionManager.needsCleanup
Repository: cassandra
Updated Branches:
refs/heads/trunk c4b2320ae - 4f525b3a8
fixed bug in CompactionManager.needsCleanup
Patch by Dmitrij Koniajev, reviewed by marcuse for CASSANDRA-6845.
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/05da045b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/05da045b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/05da045b
Branch: refs/heads/trunk
Commit: 05da045bbfe29bf472982d3c5da90050176202df
Parents: 19444ea
Author: Dmitrij Koniajev dimchan...@gmail.com
Authored: Wed Mar 12 17:05:31 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Mar 12 18:14:59 2014 +0100
--
CHANGES.txt| 2 +-
src/java/org/apache/cassandra/db/compaction/CompactionManager.java | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/05da045b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 42dc4ee..3531737 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -4,7 +4,7 @@
* Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
* Fix static counter columns (CASSANDRA-6827)
* Restore expiring-deleted (cell) compaction optimization (CASSANDRA-6844)
-
+ * Fix CompactionManager.needsCleanup (CASSANDRA-6845)
2.0.6
* Avoid race-prone second scrub of system keyspace (CASSANDRA-6797)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/05da045b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 02cf433..453176e 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -493,7 +493,7 @@ public class CompactionManager implements
CompactionManagerMBean
return false;
}
-if (i == (ownedRanges.size() - 1))
+if (i == (sortedRanges.size() - 1))
{
// we're at the last range and we found a key beyond the end
of the range
return true;
