[jira] [Commented] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406865#comment-13406865 ] Jonathan Ellis commented on CASSANDRA-1123: --- Sure. > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: 1123-3.patch.gz > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4321) stackoverflow building interval tree & possible sstable corruptions
[ https://issues.apache.org/jira/browse/CASSANDRA-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406844#comment-13406844 ] Anton Winter commented on CASSANDRA-4321: - New issue raised as requested: CASSANDRA-4411 > stackoverflow building interval tree & possible sstable corruptions > --- > > Key: CASSANDRA-4321 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4321 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.1.1 >Reporter: Anton Winter >Assignee: Sylvain Lebresne > Fix For: 1.1.2 > > Attachments: 0001-Fix-overlapping-computation-v7.txt, > 0002-Scrub-detects-and-repair-outOfOrder-rows-v7.txt, > 0003-Create-standalone-scrub-v7.txt, > 0004-Add-manifest-integrity-check-v7.txt, cleanup.txt, > ooyala-hastur-stacktrace.txt > > > After upgrading to 1.1.1 (from 1.1.0) I have started experiencing > StackOverflowError's resulting in compaction backlog and failure to restart. > The ring currently consists of 6 DC's and 22 nodes using LCS & compression. > This issue was first noted on 2 nodes in one DC and then appears to have > spread to various other nodes in the other DC's. > When the first occurrence of this was found I restarted the instance but it > failed to start so I cleared its data and treated it as a replacement node > for the token it was previously responsible for. This node successfully > streamed all the relevant data back but failed again a number of hours later > with the same StackOverflowError and again was unable to restart. > The initial stack overflow error on a running instance looks like this: > ERROR [CompactionExecutor:314] 2012-06-07 09:59:43,017 > AbstractCassandraDaemon.java (line 134) Exception in thread > Thread[CompactionExecutor:314,1,main] > java.lang.StackOverflowError > at java.util.Arrays.mergeSort(Arrays.java:1157) > at java.util.Arrays.sort(Arrays.java:1092) > at java.util.Collections.sort(Collections.java:134) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.findMinMedianMax(IntervalNode.java:114) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:49) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow. Compactions stop from this point > onwards] > I restarted this failing instance with DEBUG logging enabled and it throws > the following exception part way through startup: > ERROR 11:37:51,046 Exception in thread Thread[OptionalTasks:1,5,main] > java.lang.StackOverflowError > at > org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:307) > at > org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:276) > at > org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:230) > at > org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:124) > at > org.slf4j.impl.Log4jLoggerAdapter.debug(Log4jLoggerAdapter.java:228) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:45) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow] > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalTree.(IntervalTree.java:39) > at > org.apache.cassandra.db.DataTracker.buildIntervalTree(DataTra
[jira] [Created] (CASSANDRA-4411) Assertion with LCS compaction
Anton Winter created CASSANDRA-4411:
---
Summary: Assertion with LCS compaction
Key: CASSANDRA-4411
URL: https://issues.apache.org/jira/browse/CASSANDRA-4411
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.1.2
Reporter: Anton Winter
As instructed in CASSANDRA-4321 I have raised this issue as a continuation of
that issue as it appears the problem still exists.
I have repeatedly run sstablescrub across all my nodes after the 1.1.2 upgrade
until sstablescrub shows no errors. The exceptions described in CASSANDRA-4321
do not occur as frequently now but the integrity check still throws exceptions
on a number of nodes. Once those exceptions occur compactionstats shows a
large number of pending tasks with no progression afterwards.
{code}
ERROR [CompactionExecutor:150] 2012-07-05 04:26:15,570
AbstractCassandraDaemon.java (line 134) Exception in thread
Thread[CompactionExecutor:150,1,main]
java.lang.AssertionError
at
org.apache.cassandra.db.compaction.LeveledManifest.promote(LeveledManifest.java:214)
at
org.apache.cassandra.db.compaction.LeveledCompactionStrategy.handleNotification(LeveledCompactionStrategy.java:158)
at
org.apache.cassandra.db.DataTracker.notifySSTablesChanged(DataTracker.java:531)
at
org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:254)
at
org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:978)
at
org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:200)
at
org.apache.cassandra.db.compaction.LeveledCompactionTask.execute(LeveledCompactionTask.java:50)
at
org.apache.cassandra.db.compaction.CompactionManager$1.runMayThrow(CompactionManager.java:150)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406836#comment-13406836 ] David Alves commented on CASSANDRA-1123: was about to start tracing with StopWatch, but it's only available from 10.0. upgrade? > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: 1123-3.patch.gz > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3862) RowCache misses Updates
[ https://issues.apache.org/jira/browse/CASSANDRA-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406783#comment-13406783 ] Hudson commented on CASSANDRA-3862: --- Integrated in Cassandra #1646 (See [https://builds.apache.org/job/Cassandra/1646/]) restore pre-CASSANDRA-3862 approach to removing expired tombstones during compaction (Revision fbb5ec0374e1a5f1b24680f1604b6e9201fb535f) fix build - re-add CompactionController.removeDeletedInCache for commit fbb5ec0374e1a5f1b24680f1604b6e9201fb535f restore pre-CASSANDRA-3862 approach to removing expired tombstones during compaction (Revision 086c06ad7fb211de6be877c3c1ea2ee4f86c6d7e) Result = ABORTED jbellis : Files : * src/java/org/apache/cassandra/db/compaction/CompactionIterable.java * CHANGES.txt dbrosius : Files : * src/java/org/apache/cassandra/db/compaction/CompactionController.java > RowCache misses Updates > --- > > Key: CASSANDRA-3862 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3862 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.6 >Reporter: Daniel Doubleday >Assignee: Sylvain Lebresne > Fix For: 1.1.0 > > Attachments: 3862-7.txt, 3862-cleanup.txt, 3862-v2.patch, > 3862-v4.patch, 3862-v5.txt, 3862-v6.txt, 3862-v8.txt, 3862.patch, > 3862_v3.patch, 3862_v8_addon.txt, include_memtables_in_rowcache_read.patch > > > While performing stress tests to find any race problems for CASSANDRA-2864 I > guess I (re-)found one for the standard on-heap row cache. > During my stress test I hava lots of threads running with some of them only > reading other writing and re-reading the value. > This seems to happen: > - Reader tries to read row A for the first time doing a getTopLevelColumns > - Row A which is not in the cache yet is updated by Writer. The row is not > eagerly read during write (because we want fast writes) so the writer cannot > perform a cache update > - Reader puts the row in the cache which is now missing the update > I already asked this some time ago on the mailing list but unfortunately > didn't dig after I got no answer since I assumed that I just missed > something. In a way I still do but haven't found any locking mechanism that > makes sure that this should not happen. > The problem can be reproduced with every run of my stress test. When I > restart the server the expected column is there. It's just missing from the > cache. > To test I have created a patch that merges memtables with the row cache. With > the patch the problem is gone. > I can also reproduce in 0.8. Haven't checked 1.1 but I haven't found any > relevant change their either so I assume the same aplies there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4404) tombstone estimation needs to avoid using global partitioner against index sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406782#comment-13406782 ] Hudson commented on CASSANDRA-4404: --- Integrated in Cassandra #1646 (See [https://builds.apache.org/job/Cassandra/1646/]) use proper partitioner for Range; patch by yukim, reviewed by jbellis for CASSANDRA-4404 (Revision d2b60f28935466f6e37fc9d64a44c5c81bc14fb4) Result = ABORTED yukim : Files : * src/java/org/apache/cassandra/db/compaction/SizeTieredCompactionStrategy.java > tombstone estimation needs to avoid using global partitioner against index > sstables > --- > > Key: CASSANDRA-4404 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4404 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.2 >Reporter: Jonathan Ellis >Assignee: Yuki Morishita > Labels: compaction > Fix For: 1.2 > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4285) Atomic, eventually-consistent batches
[ https://issues.apache.org/jira/browse/CASSANDRA-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406777#comment-13406777 ] Jonathan Ellis commented on CASSANDRA-4285: --- We can break up implementation as follows: # Add an atomic_batch_mutate method (with the same parameters as batch_mutate) using batchlog/SimpleStrategy in CassandraServer + StorageProxy # Implement batchlog replay # Implement a custom BatchlogStrategy that ensures redundancy at RF=1 # Add CQL3 support 3 and 4 appear straightforward. 1 and 2 have some hairier corners: For the batchlog write: - We don't want to make the write path more fragile (in the sense that atomic writes will fail, where non-atomic ones would succeed). But, the batchlog shard will probably be on a different machine than the replicas. If that machine is down, we could raise UnavailableException... but better would be to try different shards until we find one whose owner is up. - Part of the goal here is to avoid forcing the client to retry on TimedOutException. So if we attempt a batchlog write that times out, we should also retry to another shard instead of propagating TOE to the client. - Corollary: we don't need to worry about batchlog hints. - Finally, once the batchlog write succeeds, we shouldn't have to make the client retry for timeouts writing to the replicas either; we can do the retry server-side. But, we can't just return success, since that would imply that we'd achieved the requested ConsistencyLevel and the data is available to be read. Instead, we should introduce a new exception (InProgressException?) to indicate that the data isn't available to read yet, but the client does not need to retry. (We could use this exception as well for normal reads, where we have at least one replica acknowledge the update in time.) - What about RF and CL for batchlog? If it's convenient, we can allow users to customize batchlog RF, but we should always use CL=1 for read and writes. (If we need to go lower level though instead of re-using the normal write path, I'm fine with hardcoding RF=1.) We don't care about "latest versions," since we're append only and if we don't see an entry on one replay attempt we'll see it on the next, and we really don't need more durability than one replica since it's only a "staging area" until it's sent out to the replicas. The main alternative would be to use the CL for the batch, in the batchlog write. I don't like that though because that's going to introduce extra latency for the batchlog write that you don't want 99% of the time. For replay: - The main difficulty is that the batchlog shard owners can't be assumed to be alive when we restart. So, we'll need to track replay status for each shard: check on startup and retry periodically until we're successful. (One advantage of restricting batchlog to RF=1 is, when a read succeeds we know we're done replaying. But if we have RF>1, then we need to retry the read indefinitely in case another replica recovered that had additional entries. > Atomic, eventually-consistent batches > - > > Key: CASSANDRA-4285 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4285 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis > > I discussed this in the context of triggers (CASSANDRA-1311) but it's useful > as a standalone feature as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4400) Correctly catch exception when Snappy cannot be loaded
[
https://issues.apache.org/jira/browse/CASSANDRA-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406762#comment-13406762
]
Andy Cobley commented on CASSANDRA-4400:
Sylvian,
I've done a little digging into the snappy source code, looks like they are not
catching a java.lang.reflect.InvocationTargetException at line 319 of
SnappyLoader:
loadMethod.invoke(null, nativeLib.getAbsolutePath());
I'll try and raise a ticket with them.
Are you happy to commit your changes for 1.1.3?
> Correctly catch exception when Snappy cannot be loaded
> --
>
> Key: CASSANDRA-4400
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4400
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.1.1
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Minor
> Fix For: 1.1.3
>
> Attachments: 4400.txt
>
>
> From the mailing list, on C* 1.1.1:
> {noformat}
> INFO 14:22:07,600 Global memtable threshold is enabled at 35MB
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at
> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> at org.xerial.snappy.Snappy.(Snappy.java:44)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:45)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:55)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.(SnappyCompressor.java:37)
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:76)
> at
> org.apache.cassandra.config.KSMetaData.systemKeyspace(KSMetaData.java:79)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadYaml(DatabaseDescriptor.java:439)
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:118)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:126)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353)
> at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106)
> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1681)
> at java.lang.Runtime.loadLibrary0(Runtime.java:840)
> at java.lang.System.loadLibrary(System.java:1047)
> at
> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
> ... 17 more
> ERROR 14:22:09,934 Exception encountered during startup
> org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] null
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229)
> at org.xerial.snappy.Snappy.(Snappy.java:44)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:45)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:55)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.(SnappyCompressor.java:37)
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:76)
> at
> org.apache.cassandra.config.KSMetaData.systemKeyspace(KSMetaData.java:79)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadYaml(DatabaseDescriptor.java:439)
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:118)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:126)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353)
> at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106)
> org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] null
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229)
> at org.xerial.snappy.Snappy.(Snappy.java:44)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:45)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:55)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.(SnappyCompressor.java:37)
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:76)
> at
> org.apache.cassandra.config.KSMetaData.syst
[jira] [Assigned] (CASSANDRA-3772) Evaluate Murmur3-based partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-3772: - Assignee: Pavel Yaskevich (was: Dave Brosius) Pavel, can you see if you can reproduce md5 bottlenecks in 2I reads, and see if this partitioner makes a meaningful difference? > Evaluate Murmur3-based partitioner > -- > > Key: CASSANDRA-3772 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3772 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: Pavel Yaskevich > Fix For: 1.2 > > Attachments: 0001-CASSANDRA-3772-Test.patch, MumPartitionerTest.docx, > hashed_partitioner.diff, hashed_partitioner_3.diff, try_murmur3.diff, > try_murmur3_2.diff > > > MD5 is a relatively heavyweight hash to use when we don't need cryptographic > qualities, just a good output distribution. Let's see how much overhead we > can save by using Murmur3 instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-4038) Investigate improving the dynamic snitch with reservoir sampling
[ https://issues.apache.org/jira/browse/CASSANDRA-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-4038: - Assignee: Pavel Yaskevich (was: Brandon Williams) Pavel, take a look at this and see if it's worth pursuing. > Investigate improving the dynamic snitch with reservoir sampling > > > Key: CASSANDRA-4038 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4038 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Brandon Williams >Assignee: Pavel Yaskevich > Fix For: 1.2 > > > Dsnitch's UPDATES_PER_INTERVAL and WINDOW_SIZE are chosen somewhat > arbitrarily. A better fit may be something similar to Metric's > ExponentiallyDecayingSample, where more recent information is weighted > heavier than past information, and reservoir sampling would also be an > efficient way of keeping a statistically significant sample rather than > refusing updates after UPDATES_PER_INTERVAL and only keeping WINDOW_SIZE > amount. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3930) CommitLogSegment uses RandomAccessFile which doesnt have fadvice
[ https://issues.apache.org/jira/browse/CASSANDRA-3930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406680#comment-13406680 ] Jonathan Ellis commented on CASSANDRA-3930: --- What we want to do is have the CLS writes not evict other data from the page cache... Looking closer I agree that SW is not a good interface for what we want to do in CLS (assuming again that we make progress on 3578). Maybe the right thing to do is to do 3578 first, then we will have a better idea of the requirements here. > CommitLogSegment uses RandomAccessFile which doesnt have fadvice > > > Key: CASSANDRA-3930 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3930 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.0.7 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.1.3 > > > Wondering if we even need MMap file in this case as it is always sequential. > If yes, we need it we might want to replace > logFileAccessor = new RandomAccessFile(logFile, "rw"); > to > logFileAccessor = SequentialWriter.open(logFile, true); > in CommitLogSegment -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3930) CommitLogSegment uses RandomAccessFile which doesnt have fadvice
[ https://issues.apache.org/jira/browse/CASSANDRA-3930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406681#comment-13406681 ] Jonathan Ellis commented on CASSANDRA-3930: --- bq. we should probably look into "madvice" direction if yoi want to control over CLS mappings Also a good idea. > CommitLogSegment uses RandomAccessFile which doesnt have fadvice > > > Key: CASSANDRA-3930 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3930 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.0.7 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.1.3 > > > Wondering if we even need MMap file in this case as it is always sequential. > If yes, we need it we might want to replace > logFileAccessor = new RandomAccessFile(logFile, "rw"); > to > logFileAccessor = SequentialWriter.open(logFile, true); > in CommitLogSegment -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4275) Oracle Java 1.7 u4 does not allow Xss128k
[ https://issues.apache.org/jira/browse/CASSANDRA-4275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406678#comment-13406678 ] Jonathan Ellis commented on CASSANDRA-4275: --- No. You can tweak your 1.0.x startup script manually if you really want to run it on java 7 (not really recommended, they're still shaking the bugs out). > Oracle Java 1.7 u4 does not allow Xss128k > - > > Key: CASSANDRA-4275 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4275 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.0.9, 1.1.0 >Reporter: Edward Capriolo >Assignee: Sylvain Lebresne > Fix For: 1.1.2 > > Attachments: 4275.txt, trunk-cassandra-4275.1.patch.txt, > v1-0001-CASSANDRA-4275-Use-JVM-s-reported-minimum-stack-size-o.txt > > > Problem: This happens when you try to start it with default Xss setting of > 128k > === > The stack size specified is too small, Specify at least 160k > Error: Could not create the Java Virtual Machine. > Error: A fatal exception has occurred. Program will exit. > Solution > === > Set -Xss to 256k > Problem: This happens when you try to start it with Xss = 160k > > ERROR [Thrift:14] 2012-05-22 14:42:40,479 AbstractCassandraDaemon.java (line > 139) Fatal exception in thread Thread[Thrift:14,5,main] > java.lang.StackOverflowError > Solution > === > Set -Xss to 256k -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-3930) CommitLogSegment uses RandomAccessFile which doesnt have fadvice
[ https://issues.apache.org/jira/browse/CASSANDRA-3930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406655#comment-13406655 ] Pavel Yaskevich edited comment on CASSANDRA-3930 at 7/4/12 6:21 PM: How do you mean active? mmap uses the same page cache as normal access does so the same page out applies, I don't see how we dould be able to avoid using buffer for CLS if we to use SW. Edit: forgot to mention, we should probably look into "madvice" direction if yoi want to control over CLS mappings, instead of trying to use SW was (Author: xedin): How do you mean active? mmap uses the same page cache as normal access does so the same page out applies, I don't see how we dould be able to avoid using buffer for CLS if we to use SW. > CommitLogSegment uses RandomAccessFile which doesnt have fadvice > > > Key: CASSANDRA-3930 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3930 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.0.7 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.1.3 > > > Wondering if we even need MMap file in this case as it is always sequential. > If yes, we need it we might want to replace > logFileAccessor = new RandomAccessFile(logFile, "rw"); > to > logFileAccessor = SequentialWriter.open(logFile, true); > in CommitLogSegment -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3930) CommitLogSegment uses RandomAccessFile which doesnt have fadvice
[ https://issues.apache.org/jira/browse/CASSANDRA-3930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406655#comment-13406655 ] Pavel Yaskevich commented on CASSANDRA-3930: How do you mean active? mmap uses the same page cache as normal access does so the same page out applies, I don't see how we dould be able to avoid using buffer for CLS if we to use SW. > CommitLogSegment uses RandomAccessFile which doesnt have fadvice > > > Key: CASSANDRA-3930 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3930 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.0.7 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.1.3 > > > Wondering if we even need MMap file in this case as it is always sequential. > If yes, we need it we might want to replace > logFileAccessor = new RandomAccessFile(logFile, "rw"); > to > logFileAccessor = SequentialWriter.open(logFile, true); > in CommitLogSegment -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3930) CommitLogSegment uses RandomAccessFile which doesnt have fadvice
[ https://issues.apache.org/jira/browse/CASSANDRA-3930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406651#comment-13406651 ] Vijay commented on CASSANDRA-3930: -- Hi Pavel, Agree, but keeping 4 GB of memory active (via file cache) doesn't sound right either... If we agree i can work on the patch. Let me know Thanks! > CommitLogSegment uses RandomAccessFile which doesnt have fadvice > > > Key: CASSANDRA-3930 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3930 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.0.7 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.1.3 > > > Wondering if we even need MMap file in this case as it is always sequential. > If yes, we need it we might want to replace > logFileAccessor = new RandomAccessFile(logFile, "rw"); > to > logFileAccessor = SequentialWriter.open(logFile, true); > in CommitLogSegment -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4275) Oracle Java 1.7 u4 does not allow Xss128k
[ https://issues.apache.org/jira/browse/CASSANDRA-4275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406650#comment-13406650 ] Shawn Bissell commented on CASSANDRA-4275: -- Will this fix be back ported to 1.0.x? > Oracle Java 1.7 u4 does not allow Xss128k > - > > Key: CASSANDRA-4275 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4275 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.0.9, 1.1.0 >Reporter: Edward Capriolo >Assignee: Sylvain Lebresne > Fix For: 1.1.2 > > Attachments: 4275.txt, trunk-cassandra-4275.1.patch.txt, > v1-0001-CASSANDRA-4275-Use-JVM-s-reported-minimum-stack-size-o.txt > > > Problem: This happens when you try to start it with default Xss setting of > 128k > === > The stack size specified is too small, Specify at least 160k > Error: Could not create the Java Virtual Machine. > Error: A fatal exception has occurred. Program will exit. > Solution > === > Set -Xss to 256k > Problem: This happens when you try to start it with Xss = 160k > > ERROR [Thrift:14] 2012-05-22 14:42:40,479 AbstractCassandraDaemon.java (line > 139) Fatal exception in thread Thread[Thrift:14,5,main] > java.lang.StackOverflowError > Solution > === > Set -Xss to 256k -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: Only consider whole row tombstone in collation controller
Updated Branches:
refs/heads/trunk e7d323009 -> 45ad66895
Only consider whole row tombstone in collation controller
patch by slebresne; reviewed by jbellis for CASSANDRA-4409
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/45ad6689
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/45ad6689
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/45ad6689
Branch: refs/heads/trunk
Commit: 45ad66895d0e5b99feaa74ec061b887c1a9688a3
Parents: e7d3230
Author: Sylvain Lebresne
Authored: Wed Jul 4 19:35:03 2012 +0200
Committer: Sylvain Lebresne
Committed: Wed Jul 4 19:35:03 2012 +0200
--
.../apache/cassandra/db/CollationController.java |4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/45ad6689/src/java/org/apache/cassandra/db/CollationController.java
--
diff --git a/src/java/org/apache/cassandra/db/CollationController.java
b/src/java/org/apache/cassandra/db/CollationController.java
index c10caef..193d892 100644
--- a/src/java/org/apache/cassandra/db/CollationController.java
+++ b/src/java/org/apache/cassandra/db/CollationController.java
@@ -126,7 +126,7 @@ public class CollationController
if (cf.isMarkedForDelete())
{
// track the most recent row level tombstone we
encounter
-mostRecentRowTombstone =
cf.deletionInfo().maxTimestamp();
+mostRecentRowTombstone =
cf.deletionInfo().getTopLevelDeletion().markedForDeleteAt;
}
container.delete(cf);
@@ -257,7 +257,7 @@ public class CollationController
{
ColumnFamily cf = iter.getColumnFamily();
if (cf.isMarkedForDelete())
-mostRecentRowTombstone =
cf.deletionInfo().maxTimestamp();
+mostRecentRowTombstone =
cf.deletionInfo().getTopLevelDeletion().markedForDeleteAt;
returnCF.delete(cf);
sstablesIterated++;
[jira] [Commented] (CASSANDRA-3930) CommitLogSegment uses RandomAccessFile which doesnt have fadvice
[ https://issues.apache.org/jira/browse/CASSANDRA-3930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406627#comment-13406627 ] Pavel Yaskevich commented on CASSANDRA-3930: It's not that simple :) I see that at least to methods require buffer seeking in CommitLongSegment: recycle() and write(RowMutation) - the second uses buffer.position for housekeeping at the end of the method plus there is file pre-allocation which couldn't be done by means of SW. I guess RAF + mmap there is the most flexible solution here. > CommitLogSegment uses RandomAccessFile which doesnt have fadvice > > > Key: CASSANDRA-3930 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3930 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.0.7 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.1.3 > > > Wondering if we even need MMap file in this case as it is always sequential. > If yes, we need it we might want to replace > logFileAccessor = new RandomAccessFile(logFile, "rw"); > to > logFileAccessor = SequentialWriter.open(logFile, true); > in CommitLogSegment -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4324) Implement Lucene FST in for key index
[ https://issues.apache.org/jira/browse/CASSANDRA-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406607#comment-13406607 ] Yuki Morishita commented on CASSANDRA-4324: --- Manually put jar file inside lib directory (as well as adding dependency for maven in build.xml as you do). I've noticed you added 4.0-SNAPSHOT to build.xml, but I think 4.0 is still alpha, so does lucene-core 3.6 work for this? > Implement Lucene FST in for key index > - > > Key: CASSANDRA-4324 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4324 > Project: Cassandra > Issue Type: Improvement >Reporter: Jason Rutherglen >Assignee: Jason Rutherglen >Priority: Minor > Fix For: 1.2 > > Attachments: CASSANDRA-4324.patch > > > The Lucene FST data structure offers a compact and fast system for indexing > Cassandra keys. More keys may be loaded which in turn should seeks faster. > * Update the IndexSummary class to make use of the Lucene FST, overriding the > serialization mechanism. > * Alter SSTableReader to make use of the FST seek mechanism -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4265) Limit total open connections (HSHA server)
[ https://issues.apache.org/jira/browse/CASSANDRA-4265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-4265: -- Reviewer: brandon.williams Assignee: Pavel Yaskevich > Limit total open connections (HSHA server) > -- > > Key: CASSANDRA-4265 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4265 > Project: Cassandra > Issue Type: Improvement > Components: API >Affects Versions: 0.8.3 > Environment: Ubuntu 10.04, 64bit, Oracle 1.6.0_32 and OpenJDK 1.6.0_20 > Connecting with Hector 1.1-0 >Reporter: James Kearney >Assignee: Pavel Yaskevich >Priority: Minor > Labels: thrift > Fix For: 1.1.3 > > Attachments: 0001-Limit-HSHA-open-connections.patch > > > When using the rpc_server_type: hsha there seems to be no limit on the number > of open connections that cassandra accepts / on the total memory consumed by > them. This can lead to OOM errors since the HSHA server assigns a FrameBuffer > per connection which is only cleaned up when the connection is closed. > Setup: > I wrote a simple test App using Hector which iterated through my rows > retrieving data. If my Hector connection pool size was set high (in this case > 100) then after a while cassandra would crash with OOM. The program was > sequential so only 1 connection was actually in use at any one time but from > what I can tell (and from MAT analysis) all the open connections were > consuming memory as well (the FrameBuffers). > At the moment all the solutions to this OOM problem seem to rest with the > client. The memory consumed (on a node) is equal to [open connections]*[max > request/response size] (it is max because of > https://issues.apache.org/jira/browse/THRIFT-1457). > This means the client needs to know how much memory each node has spare for > it to use up with its connection pool. If you have a distributed set of > clients then they would have to co-ordinate on how many open connections they > have per node. > I was just testing on a dev machine with small heap sizes (512mb-2GB) but the > memory consumed as stated is based off connections and buffer size so this > problem would scale for larger heap sizes. > Solutions: > The simplest would be a limit on the number of connections the HSHA server > accepts. I only started looking into cassandra a few days ago but I tried a > very simple connection limit mechanism (I will attach the patch) which seemed > to work. I'm sure it can be done much cleaner than my version. > This means the client or clients only have to have a hard limit on their max > request size (lets say 2mb). Then for each node you know that allowing 100 > connections to this node would potentially use up 200mb of memory. You can > then tune this number per node. (This isn't perfect since clients can't > always tell exactly how big a serialised response will be so you can go above > the 200mb) > A more complex solution might be able to remove the burden from the client > completely. Thrift doesn't have streaming support but I assume when cassandra > reads data from disk / memtables streaming can be done at that point. If this > is the case then you can monitor how much memory client connections are > consuming in total. If a request comes in and buffering the request / > buffering the response would push cassandra over the limit then you can send > an error back instead of servicing the request. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-4410) BulkOutputFormat and CompositeColumn results in java.lang.AssertionError: Added column does not sort as the last column
[ https://issues.apache.org/jira/browse/CASSANDRA-4410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-4410: - Assignee: Pavel Yaskevich > BulkOutputFormat and CompositeColumn results in java.lang.AssertionError: > Added column does not sort as the last column > --- > > Key: CASSANDRA-4410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4410 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Affects Versions: 1.1.1 > Environment: hadoop 0.20 > cassandra 1.1.1 > cql 3.0.0 >Reporter: Henning Kropp >Assignee: Pavel Yaskevich > Labels: bulkloader, compositeColumns, cql3, hadoop > Fix For: 1.1.3 > > Attachments: CompositeColumnsBulkOutputDriver.java, > CompositeColumnsBulkOutputMapper.java, > CompositeColumnsBulkOutputReducer.java, HectorInsertTest.java, input.csv, > keyspace.cli, pom.xml, schema.cql, schema_hector.cql > > > Using the org.apache.cassandra.hadoop.BulkOutputFormat.class with > CompositeColumns results in the following AsertionError during a Select query > with WHERE ID='id' or using an ID in any way in the where clause. > ERROR [ReadStage:34] 2012-07-04 14:33:27,376 AbstractCassandraDaemon.java > (line 134) Exception in thread Thread[ReadStage:34,5,main] > java.lang.AssertionError: Added column does not sort as the last column > at > org.apache.cassandra.db.ArrayBackedSortedColumns.addColumn(ArrayBackedSortedColumns.java:130) > at > org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:107) > at > org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:102) > at > org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:141) > at > org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:139) > at > org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:283) > at > org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:63) > at > org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1321) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1183) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1118) > at org.apache.cassandra.db.Table.getRow(Table.java:374) > at > org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:69) > at > org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:816) > at > org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1250) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4410) BulkOutputFormat and CompositeColumn results in java.lang.AssertionError: Added column does not sort as the last column
[ https://issues.apache.org/jira/browse/CASSANDRA-4410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-4410: -- Reviewer: slebresne Priority: Minor (was: Major) > BulkOutputFormat and CompositeColumn results in java.lang.AssertionError: > Added column does not sort as the last column > --- > > Key: CASSANDRA-4410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4410 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Affects Versions: 1.1.1 > Environment: hadoop 0.20 > cassandra 1.1.1 > cql 3.0.0 >Reporter: Henning Kropp >Assignee: Pavel Yaskevich >Priority: Minor > Labels: bulkloader, compositeColumns, cql3, hadoop > Fix For: 1.1.3 > > Attachments: CompositeColumnsBulkOutputDriver.java, > CompositeColumnsBulkOutputMapper.java, > CompositeColumnsBulkOutputReducer.java, HectorInsertTest.java, input.csv, > keyspace.cli, pom.xml, schema.cql, schema_hector.cql > > > Using the org.apache.cassandra.hadoop.BulkOutputFormat.class with > CompositeColumns results in the following AsertionError during a Select query > with WHERE ID='id' or using an ID in any way in the where clause. > ERROR [ReadStage:34] 2012-07-04 14:33:27,376 AbstractCassandraDaemon.java > (line 134) Exception in thread Thread[ReadStage:34,5,main] > java.lang.AssertionError: Added column does not sort as the last column > at > org.apache.cassandra.db.ArrayBackedSortedColumns.addColumn(ArrayBackedSortedColumns.java:130) > at > org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:107) > at > org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:102) > at > org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:141) > at > org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:139) > at > org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:283) > at > org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:63) > at > org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1321) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1183) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1118) > at org.apache.cassandra.db.Table.getRow(Table.java:374) > at > org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:69) > at > org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:816) > at > org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1250) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4324) Implement Lucene FST in for key index
[ https://issues.apache.org/jira/browse/CASSANDRA-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406603#comment-13406603 ] Jason Rutherglen commented on CASSANDRA-4324: - An easy approach is to generate an ascending set of keys and positions, apply the MD5 hash and add them to both data structures and compare the RAM usage. > Implement Lucene FST in for key index > - > > Key: CASSANDRA-4324 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4324 > Project: Cassandra > Issue Type: Improvement >Reporter: Jason Rutherglen >Assignee: Jason Rutherglen >Priority: Minor > Fix For: 1.2 > > Attachments: CASSANDRA-4324.patch > > > The Lucene FST data structure offers a compact and fast system for indexing > Cassandra keys. More keys may be loaded which in turn should seeks faster. > * Update the IndexSummary class to make use of the Lucene FST, overriding the > serialization mechanism. > * Alter SSTableReader to make use of the FST seek mechanism -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4324) Implement Lucene FST in for key index
[ https://issues.apache.org/jira/browse/CASSANDRA-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406604#comment-13406604 ] Jason Rutherglen commented on CASSANDRA-4324: - Also what is the best way to include the Lucene libraries? > Implement Lucene FST in for key index > - > > Key: CASSANDRA-4324 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4324 > Project: Cassandra > Issue Type: Improvement >Reporter: Jason Rutherglen >Assignee: Jason Rutherglen >Priority: Minor > Fix For: 1.2 > > Attachments: CASSANDRA-4324.patch > > > The Lucene FST data structure offers a compact and fast system for indexing > Cassandra keys. More keys may be loaded which in turn should seeks faster. > * Update the IndexSummary class to make use of the Lucene FST, overriding the > serialization mechanism. > * Alter SSTableReader to make use of the FST seek mechanism -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-3930) CommitLogSegment uses RandomAccessFile which doesnt have fadvice
[ https://issues.apache.org/jira/browse/CASSANDRA-3930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406602#comment-13406602 ] Jonathan Ellis edited comment on CASSANDRA-3930 at 7/4/12 3:54 PM: --- Changing from RAF to SW looks reasonable to me. Do you want to run any tests, Pavel? was (Author: jbellis): Looks reasonable to me. Do you want to run any tests, Pavel? > CommitLogSegment uses RandomAccessFile which doesnt have fadvice > > > Key: CASSANDRA-3930 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3930 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.0.7 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.1.3 > > > Wondering if we even need MMap file in this case as it is always sequential. > If yes, we need it we might want to replace > logFileAccessor = new RandomAccessFile(logFile, "rw"); > to > logFileAccessor = SequentialWriter.open(logFile, true); > in CommitLogSegment -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3930) CommitLogSegment uses RandomAccessFile which doesnt have fadvice
[ https://issues.apache.org/jira/browse/CASSANDRA-3930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3930: -- Reviewer: xedin Looks reasonable to me. Do you want to run any tests, Pavel? > CommitLogSegment uses RandomAccessFile which doesnt have fadvice > > > Key: CASSANDRA-3930 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3930 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.0.7 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.1.3 > > > Wondering if we even need MMap file in this case as it is always sequential. > If yes, we need it we might want to replace > logFileAccessor = new RandomAccessFile(logFile, "rw"); > to > logFileAccessor = SequentialWriter.open(logFile, true); > in CommitLogSegment -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2489) better not close the out/error stream from CassandraDaemon
[
https://issues.apache.org/jira/browse/CASSANDRA-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-2489.
---
Resolution: Not A Problem
Fix Version/s: (was: 1.1.3)
It sounds like the motivation here is just to be able to use kill -3, but you
can always use jstack instead to get the same information. So closing as not a
problem.
> better not close the out/error stream from CassandraDaemon
> --
>
> Key: CASSANDRA-2489
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2489
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jackson Chung
>Assignee: paul cannon
>Priority: Minor
> Labels: lhf
>
> When cassandra is started in the background (-p /tmp/pidfile.pid), the
> System.out System.err is closed. That may not be suitable in case where
> capturing thread dump via traditional sigquit is preferred (kill -3 ).
> This could be useful especially when jmx collection is failing, or ability to
> do some little script to continuously capture thread dump via a script in the
> background.
> closing System.err could also potentially be swallowing fatal error that
> crashing the jvm.
> I would suggest make change to the stocked cassandra start script, when the
> intends is not running in foreground, (ie running in background) then
> redirect the standard output/error to a file, eg:
> /var/log/cassandra/cassandra.out 2>&1 &
> currently i apply a workaround to the script by faking the
> cassandra-foreground flag as -Dcassandra-foreground=no since the check is
> simply look for null:
> {code:title=AbstracCassandraDaemon.activate()}
>if (System.getProperty("cassandra-foreground") == null)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4327) Sorting results when using IN()
[
https://issues.apache.org/jira/browse/CASSANDRA-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-4327:
--
Reviewer: (was: xedin)
Assignee: Pavel Yaskevich
Pavel, go ahead and see if this would be a quick fix but if not let's table it
for now.
> Sorting results when using IN()
>
>
> Key: CASSANDRA-4327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4327
> Project: Cassandra
> Issue Type: Improvement
>Affects Versions: 1.1.1
>Reporter: Stephen Powis
>Assignee: Pavel Yaskevich
>Priority: Minor
> Labels: cql3, patch, sort
> Fix For: 1.1.3
>
> Attachments: trunk-4327.txt
>
>
> Using the following test schema:
> CREATE TABLE test (
> my_id varchar,
> time_id uuid,
> value int,
> PRIMARY KEY (my_id, time_id)
> );
> When you issue a CQL3 query like:
> select * from test where my_id in('key1', 'key2') order by time_id;
> You receive the error:
> "Ordering is only supported if the first part of the PRIMARY KEY is
> restricted by an Equal"
> I'm including a patch I put together after spending an hour or two poking
> thru the code base that sorts the results for these types of queries. I'm
> hoping someone with a deeper understanding of Cassandra's code base can take
> a look at it, clean it up or use it as a starting place, and include it in an
> upcoming release.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4327) Sorting results when using IN()
[
https://issues.apache.org/jira/browse/CASSANDRA-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406596#comment-13406596
]
Jonathan Ellis commented on CASSANDRA-4327:
---
To clarify, the error is because we want to provide ordering based on the
clustering provided by the column name
(www.datastax.com/dev/blog/schema-in-cassandra-1-1). However, it does seem
reasonable that we could provide a "merge" step for the IN case as well.
> Sorting results when using IN()
>
>
> Key: CASSANDRA-4327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4327
> Project: Cassandra
> Issue Type: Improvement
>Affects Versions: 1.1.1
>Reporter: Stephen Powis
>Priority: Minor
> Labels: cql3, patch, sort
> Fix For: 1.1.3
>
> Attachments: trunk-4327.txt
>
>
> Using the following test schema:
> CREATE TABLE test (
> my_id varchar,
> time_id uuid,
> value int,
> PRIMARY KEY (my_id, time_id)
> );
> When you issue a CQL3 query like:
> select * from test where my_id in('key1', 'key2') order by time_id;
> You receive the error:
> "Ordering is only supported if the first part of the PRIMARY KEY is
> restricted by an Equal"
> I'm including a patch I put together after spending an hour or two poking
> thru the code base that sorts the results for these types of queries. I'm
> hoping someone with a deeper understanding of Cassandra's code base can take
> a look at it, clean it up or use it as a starting place, and include it in an
> upcoming release.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4324) Implement Lucene FST in for key index
[ https://issues.apache.org/jira/browse/CASSANDRA-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406591#comment-13406591 ] Jonathan Ellis commented on CASSANDRA-4324: --- bq. In general the big win with the FST is the amount of RAM consumed should be far less. That is fairly easy to measure by generating N keys and comparing the RAM usage I think that's what yuki meant by "micro benchmark [of] memory." > Implement Lucene FST in for key index > - > > Key: CASSANDRA-4324 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4324 > Project: Cassandra > Issue Type: Improvement >Reporter: Jason Rutherglen >Assignee: Jason Rutherglen >Priority: Minor > Fix For: 1.2 > > Attachments: CASSANDRA-4324.patch > > > The Lucene FST data structure offers a compact and fast system for indexing > Cassandra keys. More keys may be loaded which in turn should seeks faster. > * Update the IndexSummary class to make use of the Lucene FST, overriding the > serialization mechanism. > * Alter SSTableReader to make use of the FST seek mechanism -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4409) Only consider whole row tombstone in collation controller
[ https://issues.apache.org/jira/browse/CASSANDRA-4409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406586#comment-13406586 ] Jonathan Ellis commented on CASSANDRA-4409: --- +1 > Only consider whole row tombstone in collation controller > - > > Key: CASSANDRA-4409 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4409 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.2 >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne >Priority: Trivial > Fix For: 1.2 > > Attachments: > 0001-Use-only-top-level-row-deletion-to-avoid-sstable-durin.txt > > > CollationController has that optimization that if it has already seen a row > tombstone more recent that a sstable max timestamp, it skips the sstable. > However, this was not updated correctly while introducing range tombstone and > currently the code might skip a sstable based on the timestamp of a tombstone > that does not cover the full row. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2698) Instrument repair to be able to assess it's efficiency (precision)
[ https://issues.apache.org/jira/browse/CASSANDRA-2698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406548#comment-13406548 ] Jason Wee commented on CASSANDRA-2698: -- Hello Jonathan, I read through this ticket and trying to understand the context. Unfortunately, at this stage, I am clueless where to start. However, attach is the output of nodetool repair and nodetool cfhistogram which may help in this ticket context imo. Please let me know if the attachment helps or I should probably go to trivial bug first so that I can have more time to understand the code as well as the development culture and procedures here. > Instrument repair to be able to assess it's efficiency (precision) > -- > > Key: CASSANDRA-2698 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2698 > Project: Cassandra > Issue Type: Improvement >Reporter: Sylvain Lebresne >Priority: Minor > Labels: lhf > Attachments: nodetool_repair_and_cfhistogram.tar.gz > > > Some reports indicate that repair sometime transfer huge amounts of data. One > hypothesis is that the merkle tree precision may deteriorate too much at some > data size. To check this hypothesis, it would be reasonably to gather > statistic during the merkle tree building of how many rows each merkle tree > range account for (and the size that this represent). It is probably an > interesting statistic to have anyway. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2698) Instrument repair to be able to assess it's efficiency (precision)
[ https://issues.apache.org/jira/browse/CASSANDRA-2698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Wee updated CASSANDRA-2698: - Attachment: nodetool_repair_and_cfhistogram.tar.gz output of nodetool repair on one of the node. output of nodetool cfhistogram > Instrument repair to be able to assess it's efficiency (precision) > -- > > Key: CASSANDRA-2698 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2698 > Project: Cassandra > Issue Type: Improvement >Reporter: Sylvain Lebresne >Priority: Minor > Labels: lhf > Attachments: nodetool_repair_and_cfhistogram.tar.gz > > > Some reports indicate that repair sometime transfer huge amounts of data. One > hypothesis is that the merkle tree precision may deteriorate too much at some > data size. To check this hypothesis, it would be reasonably to gather > statistic during the merkle tree building of how many rows each merkle tree > range account for (and the size that this represent). It is probably an > interesting statistic to have anyway. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4121) TokenMetadata supports multiple tokens per host
[
https://issues.apache.org/jira/browse/CASSANDRA-4121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406515#comment-13406515
]
Sam Overton commented on CASSANDRA-4121:
bq. If we do go with these wrappers I'd prefer to keep it as thin as possible –
no internal synchronization, no copies on inverse(). (I think the existing
synchronizied wrapper around the bootstrapTokens BiMap saves us exactly one use
of the explicit lock, in pendingRangeChanges. I'm fine with giving that up.)
I think given the changes to synchronization in TokenMetadata for
CASSANDRA-3881 I will revisit synchronization in these BMVMaps - they can
probably just rely on the ReadWriteLock in TMD. Re: "no copies on inverse()" -
unmodifiableMultimap just makes the returned inverse map immutable, there is no
copy.
bq. I think the code reads better with, than without them, but I can understand
the argument for keeping them "thin". Sam?
I do think these wrappers are useful. It cuts down on the size of the interface
that TMD has to provide. Looking up the endpoint to tokens mapping in both
directions is useful and we already use the reverse mapping of bootstrapTokens
in SS.calculatePendingRanges(). I would be tempted to make
TMD.getEndpointToTokenMapForReading return the BMVMap too (currently it copies
into a Multimap).
> TokenMetadata supports multiple tokens per host
> ---
>
> Key: CASSANDRA-4121
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4121
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
>Reporter: Sam Overton
>Assignee: Sam Overton
> Labels: vnodes
> Fix For: 1.2
>
>
> _Edit0: Append patch information._
> h3. Patches
> ||Compare||Raw diff||Description||
> |[01_support_multiple_tokens_per_host|https://github.com/acunu/cassandra/compare/top-bases/p/4121/01_support_multiple_tokens_per_host...p/4121/01_support_multiple_tokens_per_host]|[01_support_multiple_tokens_per_host.patch|https://github.com/acunu/cassandra/compare/top-bases/p/4121/01_support_multiple_tokens_per_host...p/4121/01_support_multiple_tokens_per_host.diff]|Support
> associating more than one token per node|
>
> _Note: These are branches managed with TopGit. If you are applying the patch
> output manually, you will either need to filter the TopGit metadata files
> (i.e. {{wget -O - | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}),
> or remove them afterward ({{rm .topmsg .topdeps}})._
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4179) Add more general support for composites (to row key, column value)
[
https://issues.apache.org/jira/browse/CASSANDRA-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406514#comment-13406514
]
Sylvain Lebresne commented on CASSANDRA-4179:
-
Looking at that more closely, there is a few details on which to decide. At
least:
* Do we care about allowing to add new columns to a composite row key? I guess
since we decided to not support it for composite comparators, it's consistent
to just say we don't support it here. But I'll just note that if we want to
support it that's not trivial because row key with one element won't be
composite underneath so we won't be able to "extend" them.
* What about thrift? This require changing key_alias to be key_aliases. Should
we just say that from thrift you can only see/set the first key alias even if
there are more than one? If we do that, working with those table from thrift
will be more difficult though. That is really the same problem that discussed
in CASSANDRA-4377, do we care than thrift client might not have enough
information to work with CQL3 table correctly?
> Add more general support for composites (to row key, column value)
> --
>
> Key: CASSANDRA-4179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4179
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Minor
>
> Currently CQL3 have a nice syntax for using composites in the column name
> (it's more than that in fact, it creates a whole new abstraction but let's
> say I'm talking implementation here). There is however 2 other place where
> composites could be used (again implementation wise): the row key and the
> column value. This ticket proposes to explore which of those make sense for
> CQL3 and how.
> For the row key, I really think that CQL support makes sense. It's very
> common (and useful) to want to stuff composite information in a row key.
> Sharding a time serie (CASSANDRA-4176) is probably the best example but there
> is other.
> For the column value it is less clear. CQL3 makes it very transparent and
> convenient to store multiple related values into multiple columns so maybe
> composites in a column value is much less needed. I do still see two cases
> for which it could be handy:
> # to save some disk/memory space, if you do know it makes no sense to
> insert/read two value separatly.
> # if you want to enforce that two values should not be inserted separatly.
> I.e. to enforce a form of "constraint" to avoid programatic error.
> Those are not widely useful things, but my reasoning is that if whatever
> syntax we come up for "grouping" row key in a composite trivially extends to
> column values, why not support it.
> As for syntax I have 3 suggestions (that are just that, suggestions):
> # If we only care about allowing grouping for row keys:
> {noformat}
> CREATE TABLE timeline (
> name text,
> month int,
> ts timestamp,
> value text,
> PRIMARY KEY ((name, month), ts)
> )
> {noformat}
> # A syntax that could work for both grouping in row key and colum value:
> {noformat}
> CREATE TABLE timeline (
> name text,
> month int,
> ts timestamp,
> value1 text,
> value2 text,
> GROUP (name, month) as key,
> GROUP (value1, value2),
> PRIMARY KEY (key, ts)
> )
> {noformat}
> # An alternative to the preceding one:
> {noformat}
> CREATE TABLE timeline (
> name text,
> month int,
> ts timestamp,
> value1 text,
> value2 text,
> GROUP (name, month) as key,
> GROUP (value1, value2),
> PRIMARY KEY (key, ts)
> ) WITH GROUP (name, month) AS key
>AND GROUP (value1, value2)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-4179) Add more general support for composites (to row key, column value)
[
https://issues.apache.org/jira/browse/CASSANDRA-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne reassigned CASSANDRA-4179:
---
Assignee: Sylvain Lebresne
> Add more general support for composites (to row key, column value)
> --
>
> Key: CASSANDRA-4179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4179
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Minor
>
> Currently CQL3 have a nice syntax for using composites in the column name
> (it's more than that in fact, it creates a whole new abstraction but let's
> say I'm talking implementation here). There is however 2 other place where
> composites could be used (again implementation wise): the row key and the
> column value. This ticket proposes to explore which of those make sense for
> CQL3 and how.
> For the row key, I really think that CQL support makes sense. It's very
> common (and useful) to want to stuff composite information in a row key.
> Sharding a time serie (CASSANDRA-4176) is probably the best example but there
> is other.
> For the column value it is less clear. CQL3 makes it very transparent and
> convenient to store multiple related values into multiple columns so maybe
> composites in a column value is much less needed. I do still see two cases
> for which it could be handy:
> # to save some disk/memory space, if you do know it makes no sense to
> insert/read two value separatly.
> # if you want to enforce that two values should not be inserted separatly.
> I.e. to enforce a form of "constraint" to avoid programatic error.
> Those are not widely useful things, but my reasoning is that if whatever
> syntax we come up for "grouping" row key in a composite trivially extends to
> column values, why not support it.
> As for syntax I have 3 suggestions (that are just that, suggestions):
> # If we only care about allowing grouping for row keys:
> {noformat}
> CREATE TABLE timeline (
> name text,
> month int,
> ts timestamp,
> value text,
> PRIMARY KEY ((name, month), ts)
> )
> {noformat}
> # A syntax that could work for both grouping in row key and colum value:
> {noformat}
> CREATE TABLE timeline (
> name text,
> month int,
> ts timestamp,
> value1 text,
> value2 text,
> GROUP (name, month) as key,
> GROUP (value1, value2),
> PRIMARY KEY (key, ts)
> )
> {noformat}
> # An alternative to the preceding one:
> {noformat}
> CREATE TABLE timeline (
> name text,
> month int,
> ts timestamp,
> value1 text,
> value2 text,
> GROUP (name, month) as key,
> GROUP (value1, value2),
> PRIMARY KEY (key, ts)
> ) WITH GROUP (name, month) AS key
>AND GROUP (value1, value2)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4340) Cassandra upgrade to 1.1.1 resulted in slow query issue
[
https://issues.apache.org/jira/browse/CASSANDRA-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Yaskevich resolved CASSANDRA-4340.
Resolution: Not A Problem
> Cassandra upgrade to 1.1.1 resulted in slow query issue
> ---
>
> Key: CASSANDRA-4340
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4340
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 1.1.1
> Environment: Ubuntu Linux, Java 7, Hector 1.0-1
>Reporter: Ivan Ganza
>Assignee: Pavel Yaskevich
> Fix For: 1.1.3
>
> Attachments: CassandraIssue.java
>
>
> We have recently introduced Cassandra at the Globe and Mail here in Toronto,
> Canada. We are processing and storing the North American stock-market feed.
> We have found it to work very quickly and things have been looking very good.
> Recently we upgraded to version 1.1.1 and then we have noticed some issues
> occurring.
> I will try to describe it for you here. Basically one operation that we very
> often perform and is very critical is the ability to 'get the latest quote'.
> This would return to you the latest Quote adjusted against exchange delay
> rules. With Cassandra version 1.0.3 we could get a Quote in around 2ms.
> After update we are looking at time of at least 2-3 seconds.
> The way we query the quote is using a REVERSED SuperSliceQuery with
> start=now, end=00:00:00.000 (beginning of day) LIMITED to 1.
> Our investigation leads us to suspect that, since upgrade, Cassandra seems to
> be reading the sstable from disk even when we request a small range of day
> only 5 seconds back. If you look at the output below you can see that the
> query does NOT get slower as the lookback increases from 5 sec, 60 sec, 15
> min, 60 min, and 24 hours.
> We also noticed that the query was very fast for the first five minutes of
> trading, apparently until the first sstable was flushed to disk. After that
> we go into query times of 1-2 seconds or so.
> Query time[lookback=5]:[1711ms]
> Query time[lookback=60]:[1592ms]
> Query time[lookback=900]:[1520ms]
> Query time[lookback=3600]:[1294ms]
> Query time[lookback=86400]:[1391ms]
> We would really appreciate input or help on this.
> Cassandra version: 1.1.1
> Hector version: 1.0-1
> ---
> public void testCassandraIssue() {
> try {
> int[] seconds = new int[]{ 5, 60, 60 * 15, 60 * 60, 60 * 60
> * 24};
> for(int sec : seconds) {
> DateTime start = new DateTime();
> SuperSliceQuery
> superSliceQuery = HFactory.createSuperSliceQuery(keyspaceOperator,
> StringSerializer.get(), StringSerializer.get(), StringSerializer.get(),
> StringSerializer.get());
> superSliceQuery.setKey("101390" + "." +
> testFormatter.print(start));
> superSliceQuery.setColumnFamily("Quotes");
>
> superSliceQuery.setRange(superKeyFormatter.print(start),
>
> superKeyFormatter.print(start.minusSeconds(sec)),
> true,
> 1);
> long theStart = System.currentTimeMillis();
> QueryResult>
> result = superSliceQuery.execute();
> long end = System.currentTimeMillis();
> System.out.println("Query time[lookback=" + sec +
> "]:[" + (end - theStart) + "ms]");
> }
> } catch(Exception e) {
> e.printStackTrace();
> fail(e.getMessage());
> }
> }
> ---
> create column family Quotes
> with column_type = Super
> and comparator = BytesType
> and subcomparator = BytesType
> and keys_cached = 7000
> and rows_cached = 0
> and row_cache_save_period = 0
> and key_cache_save_period = 3600
> and memtable_throughput = 255
> and memtable_operations = 0.29
> AND compression_options={sstable_compression:SnappyCompressor,
> chunk_length_kb:64};
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4410) BulkOutputFormat and CompositeColumn results in java.lang.AssertionError: Added column does not sort as the last column
[ https://issues.apache.org/jira/browse/CASSANDRA-4410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-4410: Fix Version/s: 1.1.3 > BulkOutputFormat and CompositeColumn results in java.lang.AssertionError: > Added column does not sort as the last column > --- > > Key: CASSANDRA-4410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4410 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Affects Versions: 1.1.1 > Environment: hadoop 0.20 > cassandra 1.1.1 > cql 3.0.0 >Reporter: Henning Kropp > Labels: bulkloader, compositeColumns, cql3, hadoop > Fix For: 1.1.3 > > Attachments: CompositeColumnsBulkOutputDriver.java, > CompositeColumnsBulkOutputMapper.java, > CompositeColumnsBulkOutputReducer.java, HectorInsertTest.java, input.csv, > keyspace.cli, pom.xml, schema.cql, schema_hector.cql > > > Using the org.apache.cassandra.hadoop.BulkOutputFormat.class with > CompositeColumns results in the following AsertionError during a Select query > with WHERE ID='id' or using an ID in any way in the where clause. > ERROR [ReadStage:34] 2012-07-04 14:33:27,376 AbstractCassandraDaemon.java > (line 134) Exception in thread Thread[ReadStage:34,5,main] > java.lang.AssertionError: Added column does not sort as the last column > at > org.apache.cassandra.db.ArrayBackedSortedColumns.addColumn(ArrayBackedSortedColumns.java:130) > at > org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:107) > at > org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:102) > at > org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:141) > at > org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:139) > at > org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:283) > at > org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:63) > at > org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1321) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1183) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1118) > at org.apache.cassandra.db.Table.getRow(Table.java:374) > at > org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:69) > at > org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:816) > at > org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1250) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4324) Implement Lucene FST in for key index
[ https://issues.apache.org/jira/browse/CASSANDRA-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406499#comment-13406499 ] Jason Rutherglen commented on CASSANDRA-4324: - The range portion of the patch is not completed as per the initial patch posted. So I'm not sure tools/stress will work? However if tools/stress has a way to generate keys that would be useful. > Implement Lucene FST in for key index > - > > Key: CASSANDRA-4324 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4324 > Project: Cassandra > Issue Type: Improvement >Reporter: Jason Rutherglen >Assignee: Jason Rutherglen >Priority: Minor > Fix For: 1.2 > > Attachments: CASSANDRA-4324.patch > > > The Lucene FST data structure offers a compact and fast system for indexing > Cassandra keys. More keys may be loaded which in turn should seeks faster. > * Update the IndexSummary class to make use of the Lucene FST, overriding the > serialization mechanism. > * Alter SSTableReader to make use of the FST seek mechanism -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of "ArticlesAndPresentations" by RobertoAmor
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "ArticlesAndPresentations" page has been changed by RobertoAmor: http://wiki.apache.org/cassandra/ArticlesAndPresentations?action=diff&rev1=135&rev2=136 Comment: Add spanish presentation * [[http://www.emtg.net78.net/2011/10/21/cassandra_hector.html|Cassandra y Hector]], Spanish, October 2011 = Presentations = + * [[http://www.roviyo.com/documents/NoSQL-Apache_Cassandra.pdf|Bases de Datos NoSQL - Apache Cassandra (in spanish)]] - Roberto Amor, April 2012 * [[http://www.jaxio.com/2012/01/06/introduction-a-cassandra-nosql-video.html|Introduction à Cassandra (in french)]] - Nicolas Romanetti, January 2012 * [[http://www.slideshare.net/jsevellec/cassandra-pour-les-dveloppeurs-java|Cassandra pour les (ch'tis) Développeurs Java]] - Jérémy Sevellec, December 2011 * [[http://www.slideshare.net/mattdennis/cassandra-data-modeling|Cassandra Data Modeling Workshop]] - Cassandra SF, Matthew F. Dennis, July 2011
[jira] [Commented] (CASSANDRA-4324) Implement Lucene FST in for key index
[ https://issues.apache.org/jira/browse/CASSANDRA-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406496#comment-13406496 ] Sylvain Lebresne commented on CASSANDRA-4324: - I guess we mostly want to make sure this patch is not clearly slower that with the current index (and if it's faster then that's even better). Using tools/stress (with enough keys inserted) should be a good start. > Implement Lucene FST in for key index > - > > Key: CASSANDRA-4324 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4324 > Project: Cassandra > Issue Type: Improvement >Reporter: Jason Rutherglen >Assignee: Jason Rutherglen >Priority: Minor > Fix For: 1.2 > > Attachments: CASSANDRA-4324.patch > > > The Lucene FST data structure offers a compact and fast system for indexing > Cassandra keys. More keys may be loaded which in turn should seeks faster. > * Update the IndexSummary class to make use of the Lucene FST, overriding the > serialization mechanism. > * Alter SSTableReader to make use of the FST seek mechanism -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4410) BulkOutputFormat and CompositeColumn results in java.lang.AssertionError: Added column does not sort as the last column
[ https://issues.apache.org/jira/browse/CASSANDRA-4410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henning Kropp updated CASSANDRA-4410: - Attachment: HectorInsertTest.java schema_hector.cql To rule out it's because of the use of the Composite class form hector. Query "select * from user_logs_hector where user_id='user1';" after running HectorInsertTest.java works just fine. > BulkOutputFormat and CompositeColumn results in java.lang.AssertionError: > Added column does not sort as the last column > --- > > Key: CASSANDRA-4410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4410 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Affects Versions: 1.1.1 > Environment: hadoop 0.20 > cassandra 1.1.1 > cql 3.0.0 >Reporter: Henning Kropp > Labels: bulkloader, compositeColumns, cql3, hadoop > Attachments: CompositeColumnsBulkOutputDriver.java, > CompositeColumnsBulkOutputMapper.java, > CompositeColumnsBulkOutputReducer.java, HectorInsertTest.java, input.csv, > keyspace.cli, pom.xml, schema.cql, schema_hector.cql > > > Using the org.apache.cassandra.hadoop.BulkOutputFormat.class with > CompositeColumns results in the following AsertionError during a Select query > with WHERE ID='id' or using an ID in any way in the where clause. > ERROR [ReadStage:34] 2012-07-04 14:33:27,376 AbstractCassandraDaemon.java > (line 134) Exception in thread Thread[ReadStage:34,5,main] > java.lang.AssertionError: Added column does not sort as the last column > at > org.apache.cassandra.db.ArrayBackedSortedColumns.addColumn(ArrayBackedSortedColumns.java:130) > at > org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:107) > at > org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:102) > at > org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:141) > at > org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:139) > at > org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:283) > at > org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:63) > at > org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1321) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1183) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1118) > at org.apache.cassandra.db.Table.getRow(Table.java:374) > at > org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:69) > at > org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:816) > at > org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1250) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4324) Implement Lucene FST in for key index
[ https://issues.apache.org/jira/browse/CASSANDRA-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406487#comment-13406487 ] Jason Rutherglen commented on CASSANDRA-4324: - The benchmark idea is interesting, however it will not take into account the fact that the FST will be able to store more keys and use less RAM. With greater key granularity, a seek to a given value will be faster? Is there an existing benchmark framework that will for example generate the keys? In general the big win with the FST is the amount of RAM consumed should be far less. That is fairly easy to measure by generating N keys and comparing the RAM usage, which with the existing IndexSummary will include object pointers. This article describes the improvements seen using Wikipedia using the FST, up to 52% less RAM used, and 22% faster. Though we need to perform our own benchmarks because an MD5 key is different than a dictionary of words. http://blog.mikemccandless.com/2011/01/finite-state-transducers-part-2.html > Implement Lucene FST in for key index > - > > Key: CASSANDRA-4324 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4324 > Project: Cassandra > Issue Type: Improvement >Reporter: Jason Rutherglen >Assignee: Jason Rutherglen >Priority: Minor > Fix For: 1.2 > > Attachments: CASSANDRA-4324.patch > > > The Lucene FST data structure offers a compact and fast system for indexing > Cassandra keys. More keys may be loaded which in turn should seeks faster. > * Update the IndexSummary class to make use of the Lucene FST, overriding the > serialization mechanism. > * Alter SSTableReader to make use of the FST seek mechanism -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4340) Cassandra upgrade to 1.1.1 resulted in slow query issue
[
https://issues.apache.org/jira/browse/CASSANDRA-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406486#comment-13406486
]
Ivan Ganza commented on CASSANDRA-4340:
---
So far so good. We can probably close this issue for now. We will re-open if
issue re-asserts itself. However for now seems good.
Thanks.
> Cassandra upgrade to 1.1.1 resulted in slow query issue
> ---
>
> Key: CASSANDRA-4340
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4340
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 1.1.1
> Environment: Ubuntu Linux, Java 7, Hector 1.0-1
>Reporter: Ivan Ganza
>Assignee: Pavel Yaskevich
> Fix For: 1.1.3
>
> Attachments: CassandraIssue.java
>
>
> We have recently introduced Cassandra at the Globe and Mail here in Toronto,
> Canada. We are processing and storing the North American stock-market feed.
> We have found it to work very quickly and things have been looking very good.
> Recently we upgraded to version 1.1.1 and then we have noticed some issues
> occurring.
> I will try to describe it for you here. Basically one operation that we very
> often perform and is very critical is the ability to 'get the latest quote'.
> This would return to you the latest Quote adjusted against exchange delay
> rules. With Cassandra version 1.0.3 we could get a Quote in around 2ms.
> After update we are looking at time of at least 2-3 seconds.
> The way we query the quote is using a REVERSED SuperSliceQuery with
> start=now, end=00:00:00.000 (beginning of day) LIMITED to 1.
> Our investigation leads us to suspect that, since upgrade, Cassandra seems to
> be reading the sstable from disk even when we request a small range of day
> only 5 seconds back. If you look at the output below you can see that the
> query does NOT get slower as the lookback increases from 5 sec, 60 sec, 15
> min, 60 min, and 24 hours.
> We also noticed that the query was very fast for the first five minutes of
> trading, apparently until the first sstable was flushed to disk. After that
> we go into query times of 1-2 seconds or so.
> Query time[lookback=5]:[1711ms]
> Query time[lookback=60]:[1592ms]
> Query time[lookback=900]:[1520ms]
> Query time[lookback=3600]:[1294ms]
> Query time[lookback=86400]:[1391ms]
> We would really appreciate input or help on this.
> Cassandra version: 1.1.1
> Hector version: 1.0-1
> ---
> public void testCassandraIssue() {
> try {
> int[] seconds = new int[]{ 5, 60, 60 * 15, 60 * 60, 60 * 60
> * 24};
> for(int sec : seconds) {
> DateTime start = new DateTime();
> SuperSliceQuery
> superSliceQuery = HFactory.createSuperSliceQuery(keyspaceOperator,
> StringSerializer.get(), StringSerializer.get(), StringSerializer.get(),
> StringSerializer.get());
> superSliceQuery.setKey("101390" + "." +
> testFormatter.print(start));
> superSliceQuery.setColumnFamily("Quotes");
>
> superSliceQuery.setRange(superKeyFormatter.print(start),
>
> superKeyFormatter.print(start.minusSeconds(sec)),
> true,
> 1);
> long theStart = System.currentTimeMillis();
> QueryResult>
> result = superSliceQuery.execute();
> long end = System.currentTimeMillis();
> System.out.println("Query time[lookback=" + sec +
> "]:[" + (end - theStart) + "ms]");
> }
> } catch(Exception e) {
> e.printStackTrace();
> fail(e.getMessage());
> }
> }
> ---
> create column family Quotes
> with column_type = Super
> and comparator = BytesType
> and subcomparator = BytesType
> and keys_cached = 7000
> and rows_cached = 0
> and row_cache_save_period = 0
> and key_cache_save_period = 3600
> and memtable_throughput = 255
> and memtable_operations = 0.29
> AND compression_options={sstable_compression:SnappyCompressor,
> chunk_length_kb:64};
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3647) Support set and map value types in CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406485#comment-13406485 ] Pavel Yaskevich commented on CASSANDRA-3647: I think I put my position on both points as clear as I could. Somebody else would have to take a look at this as well, I am not convinced that (at least) approuch taken for collection representation such as term/value/literal interconnection is a correct way to go. Code also adds few new types to o.a.c.db.marshal that make AbstractType even worse interface with new "execute" operations added. > Support set and map value types in CQL > -- > > Key: CASSANDRA-3647 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3647 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Jonathan Ellis >Assignee: Sylvain Lebresne > Labels: cql > Fix For: 1.2 > > > Composite columns introduce the ability to have arbitrarily nested data in a > Cassandra row. We should expose this through CQL. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4410) BulkOutputFormat and CompositeColumn results in java.lang.AssertionError: Added column does not sort as the last column
[ https://issues.apache.org/jira/browse/CASSANDRA-4410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henning Kropp updated CASSANDRA-4410: - Attachment: pom.xml schema.cql keyspace.cli input.csv CompositeColumnsBulkOutputReducer.java CompositeColumnsBulkOutputMapper.java CompositeColumnsBulkOutputDriver.java Setup to reproduce the problem. Use query "select * from user_logs where user_id='user1';" to see error. > BulkOutputFormat and CompositeColumn results in java.lang.AssertionError: > Added column does not sort as the last column > --- > > Key: CASSANDRA-4410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4410 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Affects Versions: 1.1.1 > Environment: hadoop 0.20 > cassandra 1.1.1 > cql 3.0.0 >Reporter: Henning Kropp > Labels: bulkloader, compositeColumns, cql3, hadoop > Attachments: CompositeColumnsBulkOutputDriver.java, > CompositeColumnsBulkOutputMapper.java, > CompositeColumnsBulkOutputReducer.java, input.csv, keyspace.cli, pom.xml, > schema.cql > > > Using the org.apache.cassandra.hadoop.BulkOutputFormat.class with > CompositeColumns results in the following AsertionError during a Select query > with WHERE ID='id' or using an ID in any way in the where clause. > ERROR [ReadStage:34] 2012-07-04 14:33:27,376 AbstractCassandraDaemon.java > (line 134) Exception in thread Thread[ReadStage:34,5,main] > java.lang.AssertionError: Added column does not sort as the last column > at > org.apache.cassandra.db.ArrayBackedSortedColumns.addColumn(ArrayBackedSortedColumns.java:130) > at > org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:107) > at > org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:102) > at > org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:141) > at > org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:139) > at > org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:283) > at > org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:63) > at > org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1321) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1183) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1118) > at org.apache.cassandra.db.Table.getRow(Table.java:374) > at > org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:69) > at > org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:816) > at > org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1250) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4410) BulkOutputFormat and CompositeColumn results in java.lang.AssertionError: Added column does not sort as the last column
Henning Kropp created CASSANDRA-4410: Summary: BulkOutputFormat and CompositeColumn results in java.lang.AssertionError: Added column does not sort as the last column Key: CASSANDRA-4410 URL: https://issues.apache.org/jira/browse/CASSANDRA-4410 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.1.1 Environment: hadoop 0.20 cassandra 1.1.1 cql 3.0.0 Reporter: Henning Kropp Using the org.apache.cassandra.hadoop.BulkOutputFormat.class with CompositeColumns results in the following AsertionError during a Select query with WHERE ID='id' or using an ID in any way in the where clause. ERROR [ReadStage:34] 2012-07-04 14:33:27,376 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[ReadStage:34,5,main] java.lang.AssertionError: Added column does not sort as the last column at org.apache.cassandra.db.ArrayBackedSortedColumns.addColumn(ArrayBackedSortedColumns.java:130) at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:107) at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:102) at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:141) at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:139) at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:283) at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:63) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1321) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1183) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1118) at org.apache.cassandra.db.Table.getRow(Table.java:374) at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:69) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:816) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1250) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4400) Correctly catch exception when Snappy cannot be loaded
[
https://issues.apache.org/jira/browse/CASSANDRA-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406480#comment-13406480
]
Andy Cobley commented on CASSANDRA-4400:
let me confirm thats the problem if thats the case, yes a ticket should be
opened.
> Correctly catch exception when Snappy cannot be loaded
> --
>
> Key: CASSANDRA-4400
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4400
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.1.1
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Minor
> Fix For: 1.1.3
>
> Attachments: 4400.txt
>
>
> From the mailing list, on C* 1.1.1:
> {noformat}
> INFO 14:22:07,600 Global memtable threshold is enabled at 35MB
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at
> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> at org.xerial.snappy.Snappy.(Snappy.java:44)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:45)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:55)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.(SnappyCompressor.java:37)
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:76)
> at
> org.apache.cassandra.config.KSMetaData.systemKeyspace(KSMetaData.java:79)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadYaml(DatabaseDescriptor.java:439)
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:118)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:126)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353)
> at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106)
> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1681)
> at java.lang.Runtime.loadLibrary0(Runtime.java:840)
> at java.lang.System.loadLibrary(System.java:1047)
> at
> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
> ... 17 more
> ERROR 14:22:09,934 Exception encountered during startup
> org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] null
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229)
> at org.xerial.snappy.Snappy.(Snappy.java:44)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:45)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:55)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.(SnappyCompressor.java:37)
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:76)
> at
> org.apache.cassandra.config.KSMetaData.systemKeyspace(KSMetaData.java:79)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadYaml(DatabaseDescriptor.java:439)
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:118)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:126)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353)
> at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106)
> org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] null
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229)
> at org.xerial.snappy.Snappy.(Snappy.java:44)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:45)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:55)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.(SnappyCompressor.java:37)
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:76)
> at
> org.apache.cassandra.config.KSMetaData.systemKeyspace(KSMetaData.java:79)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadYaml(DatabaseDescriptor.java:439)
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:118)
>
[jira] [Commented] (CASSANDRA-4400) Correctly catch exception when Snappy cannot be loaded
[
https://issues.apache.org/jira/browse/CASSANDRA-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406475#comment-13406475
]
Sylvain Lebresne commented on CASSANDRA-4400:
-
:(
So I guess the current output is the best we can do and we can open a ticket on
Snappy-Java to have them remove that.
> Correctly catch exception when Snappy cannot be loaded
> --
>
> Key: CASSANDRA-4400
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4400
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.1.1
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Minor
> Fix For: 1.1.3
>
> Attachments: 4400.txt
>
>
> From the mailing list, on C* 1.1.1:
> {noformat}
> INFO 14:22:07,600 Global memtable threshold is enabled at 35MB
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at
> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> at org.xerial.snappy.Snappy.(Snappy.java:44)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:45)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:55)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.(SnappyCompressor.java:37)
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:76)
> at
> org.apache.cassandra.config.KSMetaData.systemKeyspace(KSMetaData.java:79)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadYaml(DatabaseDescriptor.java:439)
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:118)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:126)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353)
> at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106)
> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1681)
> at java.lang.Runtime.loadLibrary0(Runtime.java:840)
> at java.lang.System.loadLibrary(System.java:1047)
> at
> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
> ... 17 more
> ERROR 14:22:09,934 Exception encountered during startup
> org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] null
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229)
> at org.xerial.snappy.Snappy.(Snappy.java:44)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:45)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:55)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.(SnappyCompressor.java:37)
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:76)
> at
> org.apache.cassandra.config.KSMetaData.systemKeyspace(KSMetaData.java:79)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadYaml(DatabaseDescriptor.java:439)
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:118)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:126)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353)
> at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106)
> org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] null
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229)
> at org.xerial.snappy.Snappy.(Snappy.java:44)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:45)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:55)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.(SnappyCompressor.java:37)
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:76)
> at
> org.apache.cassandra.config.KSMetaData.systemKeyspace(KSMetaData.java:79)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadYaml(DatabaseDescriptor.java:439)
> at
> org.apache.cassandra.config.DatabaseDes
[jira] [Commented] (CASSANDRA-3647) Support set and map value types in CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406474#comment-13406474 ] Sylvain Lebresne commented on CASSANDRA-3647: - bq. If the most common thing we do after cast with CompositeType is get "types" field then we could add Collection> getComponentTypes() method So 100% of the time, when we access the "types" field the code is specific to compositeType, so we would still have to test if the type is composite and we would never call the getComponentTypes for non-composite type. Hence the only thing that would change is that we would replace "if (type instanceof CompositeType)" by "if (type.isCompositeType())" and "((CompositeType)type).types" by "type.getComponentTypes()" which is imho *exactly* the same thing (allergic reaction to instanceof/casts put aside). Except that in the meantime you'll have polluted the interface of AbstractType with 2 new methods. And it is less safe because if someone writes a custom type that happens to change the implementation of those methods, it will almost surely be a bug. So I'm not convinced at all by that idea. But don't get me wrong, if I were to redesign C* from the ground up, I would probably make composite a more "native" notion and everything would be a composite (with potentially only 1 component) and composites wouldn't be a subclass of AbstractType at all. But this is not really an option, at least not on the short term. On a more general "design" level, in the current code, an AbstractType in general don't have "components types" so I don't think we should add a getComponentTypes() method. But since all that has nothing to do with this ticket, if you think it would be a good idea, I can only encourage you to open a separate ticket on which we could have a more focused discussion. bq. And why do we need the common ancestor Because we want Operation.Set and 'List columnValues' in UpdateStatement to be able to contain both Term and collection Literal (i.e. Value is a marker interface basically). bq. List/Set/Map don't have anything in common with Term That's not what I said. And in fact Value defines the asList() method that both Term and List/Set/Map implement. What I said is that List/Set/Map also have most of their methods that don't apply to Term and Term have most of his method that don't apply to List/Set/Map. Having the intersection non-empty don't mean one is contain in the other. Note that I'm not saying it's the only design. We probably could duplicate Operation.Set to have one applying to Term and one applying to collection literals and split UpdateStatement.columnValues into two lists, but imho having the Value interface to avoid the code duplication is cleaner. > Support set and map value types in CQL > -- > > Key: CASSANDRA-3647 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3647 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Jonathan Ellis >Assignee: Sylvain Lebresne > Labels: cql > Fix For: 1.2 > > > Composite columns introduce the ability to have arbitrarily nested data in a > Cassandra row. We should expose this through CQL. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3647) Support set and map value types in CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406454#comment-13406454 ] Pavel Yaskevich commented on CASSANDRA-3647: bq. Ok fine. I don't agree (more precisely I don't think perfect OO design at all cost is necessarily superior) but I'm not interested in that debate, at least not on that ticket. So do you have concrete proposition for that ticket? If not, I suggest we consider it good enough for now, and feel free to open a follow up ticket with a clean refactor that follow OO design to the letter. This is what I wrote in the previous ticket - If the most common thing we do after cast with CompositeType is get "types" field then we could add Collection> getComponentTypes() method. Simple types would return just one component - themselves (or throw an exception saying that type is not complex to be sure that method is not misused) and complex ones would be able to return immutable list of their component types, in combination with isComposite() method that would eliminate instanceof/downcasts completely. bq. Now the fact is that we need a class to represent those collection literals and we need a common ancestor to that class and the Term class. Making the literals being of class Term (or a subclass) would be ugly because none of the method of Term apply to the (collection) literals. But the literals also have methods that make no sense for Term (namely validateType, isEmpty and constructFunction), so we don't want to put those methods in the interface shared with Term either. And why do we need the common ancestor so much if List/Set/Map don't have anything in common with Term except those classes are it's containers? > Support set and map value types in CQL > -- > > Key: CASSANDRA-3647 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3647 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Jonathan Ellis >Assignee: Sylvain Lebresne > Labels: cql > Fix For: 1.2 > > > Composite columns introduce the ability to have arbitrarily nested data in a > Cassandra row. We should expose this through CQL. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4329) CQL3: Always use composite types by default
[ https://issues.apache.org/jira/browse/CASSANDRA-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406451#comment-13406451 ] Sylvain Lebresne commented on CASSANDRA-4329: - Committed (with nit fixed). Thanks. > CQL3: Always use composite types by default > > > Key: CASSANDRA-4329 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4329 > Project: Cassandra > Issue Type: Improvement >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Labels: cql3 > Fix For: 1.2 > > Attachments: 4329.txt, 4329_fix.txt > > > Currently, when defining a table with a single (non-composite) PRIMARY KEY, > we don't use a CompositeType in the underlying comparator. This is however a > problem for CASSANDRA-3647 as this means those tables cannot use collections. > So this ticket suggests to change that default behavior, and to always use > (by default at least, see below) a composite comparator underneath. I'll note > that doing so will mean an overhead of 3 bytes per column for non-composite > columns, but I believe getting collection is well worth it. > Of course the suggestion above apply to the default behavior and this ticket > would also add an option to table creation to get back to the current > behavior of not using a composite comparator (if ony for backward > compatibility sake). And I believe that we can actually reuse 'COMPACT > STORAGE' for that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[4/4] git commit: implement addAllWithSizeDelta for ThreadSafeSortedColumns (used in Memtable for supercolumns); see #4399
implement addAllWithSizeDelta for ThreadSafeSortedColumns (used in Memtable for
supercolumns); see #4399
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e0f4c7cc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e0f4c7cc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e0f4c7cc
Branch: refs/heads/trunk
Commit: e0f4c7cccff023140b33ae2223fbb2f36e20265f
Parents: be96989
Author: Jonathan Ellis
Authored: Tue Jul 3 12:29:24 2012 -0500
Committer: Jonathan Ellis
Committed: Tue Jul 3 12:29:24 2012 -0500
--
.../db/AbstractThreadUnsafeSortedColumns.java | 10 +
.../cassandra/db/ArrayBackedSortedColumns.java |3 +-
.../apache/cassandra/db/AtomicSortedColumns.java | 19 ++---
.../cassandra/db/ThreadSafeSortedColumns.java | 30 ---
.../cassandra/db/TreeMapBackedSortedColumns.java |3 +-
5 files changed, 33 insertions(+), 32 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e0f4c7cc/src/java/org/apache/cassandra/db/AbstractThreadUnsafeSortedColumns.java
--
diff --git
a/src/java/org/apache/cassandra/db/AbstractThreadUnsafeSortedColumns.java
b/src/java/org/apache/cassandra/db/AbstractThreadUnsafeSortedColumns.java
index 1360336..b50dda5 100644
--- a/src/java/org/apache/cassandra/db/AbstractThreadUnsafeSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/AbstractThreadUnsafeSortedColumns.java
@@ -89,21 +89,13 @@ public abstract class AbstractThreadUnsafeSortedColumns
implements ISortedColumn
}
}
-// Implementations should implement this rather than addAll to avoid
-// having to care about the deletion infos
-protected abstract void addAllColumns(ISortedColumns columns, Allocator
allocator, Function transformation);
-
public long addAllWithSizeDelta(ISortedColumns cm, Allocator allocator,
Function transformation)
{
// sizeDelta is only needed by memtable updates which should not be
using thread-unsafe containers
throw new UnsupportedOperationException();
}
-public void addAll(ISortedColumns columns, Allocator allocator,
Function transformation)
-{
-addAllColumns(columns, allocator, transformation);
-delete(columns.getDeletionInfo());
-}
+public abstract void addAll(ISortedColumns columns, Allocator allocator,
Function transformation);
public boolean isEmpty()
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e0f4c7cc/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
index 246b133..1ce3aac 100644
--- a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
@@ -201,8 +201,9 @@ public class ArrayBackedSortedColumns extends
AbstractThreadUnsafeSortedColumns
return -mid - (result < 0 ? 1 : 2);
}
-protected void addAllColumns(ISortedColumns cm, Allocator allocator,
Function transformation)
+public void addAll(ISortedColumns cm, Allocator allocator,
Function transformation)
{
+delete(cm.getDeletionInfo());
if (cm.isEmpty())
return;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e0f4c7cc/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
b/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
index 9cb44d2..d421d56 100644
--- a/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/AtomicSortedColumns.java
@@ -342,41 +342,30 @@ public class AtomicSortedColumns implements ISortedColumns
long addColumn(IColumn column, Allocator allocator)
{
ByteBuffer name = column.name();
-IColumn oldColumn;
-long sizeDelta = 0;
while (true)
{
-oldColumn = map.putIfAbsent(name, column);
+IColumn oldColumn = map.putIfAbsent(name, column);
if (oldColumn == null)
-{
-sizeDelta += column.serializedSize();
-break;
-}
+return column.serializedSize();
if (oldColumn instanceof SuperColumn)
{
assert column instanceof SuperColumn;
long previousSize = oldColumn.serializedSize();
((Su
[1/4] git commit: Fix #4329
Updated Branches:
refs/heads/trunk bbfab669f -> e7d323009
Fix #4329
patch by slebresne; reviewed by xedin for CASSANDRA-4329
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e7d32300
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e7d32300
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e7d32300
Branch: refs/heads/trunk
Commit: e7d323009526fd3ef7347537eddb168809cbc5a6
Parents: a0c9a01
Author: Sylvain Lebresne
Authored: Wed Jul 4 13:23:52 2012 +0200
Committer: Sylvain Lebresne
Committed: Wed Jul 4 13:23:52 2012 +0200
--
.../org/apache/cassandra/service/StorageProxy.java | 10 +-
1 files changed, 9 insertions(+), 1 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e7d32300/src/java/org/apache/cassandra/service/StorageProxy.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageProxy.java
b/src/java/org/apache/cassandra/service/StorageProxy.java
index b087b3d..8ae063b 100644
--- a/src/java/org/apache/cassandra/service/StorageProxy.java
+++ b/src/java/org/apache/cassandra/service/StorageProxy.java
@@ -587,6 +587,14 @@ public class StorageProxy implements StorageProxyMBean
};
}
+private static boolean systemTableQuery(List cmds)
+{
+for (ReadCommand cmd : cmds)
+if (!cmd.table.equals(Table.SYSTEM_TABLE))
+return false;
+return true;
+}
+
/**
* Performs the actual reading of a row out of the StorageService, fetching
* a specific set of column names from a given column family.
@@ -594,7 +602,7 @@ public class StorageProxy implements StorageProxyMBean
public static List read(List commands, ConsistencyLevel
consistency_level)
throws IOException, UnavailableException, TimeoutException,
InvalidRequestException
{
-if (StorageService.instance.isBootstrapMode())
+if (StorageService.instance.isBootstrapMode() &&
!systemTableQuery(commands))
{
ClientRequestMetrics.readUnavailables.inc();
throw new UnavailableException();
[2/4] git commit: Merge branch 'cassandra-1.1' into trunk
Merge branch 'cassandra-1.1' into trunk Conflicts: src/java/org/apache/cassandra/db/AtomicSortedColumns.java src/java/org/apache/cassandra/db/ThreadSafeSortedColumns.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a0c9a01e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a0c9a01e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a0c9a01e Branch: refs/heads/trunk Commit: a0c9a01e8b022f69bce7568a01e48727b8f7dfcd Parents: bbfab66 b8ca84c Author: Sylvain Lebresne Authored: Wed Jul 4 12:06:43 2012 +0200 Committer: Sylvain Lebresne Committed: Wed Jul 4 12:06:43 2012 +0200 -- --
[3/4] git commit: log mx4j absence at debug
log mx4j absence at debug
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b8ca84c8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b8ca84c8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b8ca84c8
Branch: refs/heads/trunk
Commit: b8ca84c8a024f9ad7860600020f9edd108250e6f
Parents: e0f4c7c
Author: Jonathan Ellis
Authored: Tue Jul 3 17:14:13 2012 -0500
Committer: Jonathan Ellis
Committed: Tue Jul 3 17:14:13 2012 -0500
--
src/java/org/apache/cassandra/utils/Mx4jTool.java |2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8ca84c8/src/java/org/apache/cassandra/utils/Mx4jTool.java
--
diff --git a/src/java/org/apache/cassandra/utils/Mx4jTool.java
b/src/java/org/apache/cassandra/utils/Mx4jTool.java
index 63cdff8..5953cb3 100644
--- a/src/java/org/apache/cassandra/utils/Mx4jTool.java
+++ b/src/java/org/apache/cassandra/utils/Mx4jTool.java
@@ -69,7 +69,7 @@ public class Mx4jTool
}
catch (ClassNotFoundException e)
{
-logger.info("Will not load MX4J, mx4j-tools.jar is not in the
classpath");
+logger.debug("Will not load MX4J, mx4j-tools.jar is not in the
classpath");
}
catch(Exception e)
{
[jira] [Commented] (CASSANDRA-3647) Support set and map value types in CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406447#comment-13406447 ] Sylvain Lebresne commented on CASSANDRA-3647: - bq. Every time instanceof is used it indicates that components' OO design is broken. Ok fine. I don't agree (more precisely I don't think perfect OO design at all cost is necessarily superior) but I'm not interested in that debate, at least not on that ticket. So do you have concrete proposition for that ticket? If not, I suggest we consider it good enough for now, and feel free to open a follow up ticket with a clean refactor that follow OO design to the letter. bq. That means that you are confusing semantics of the "literal" Please Pavel, let's stop with that. I don't fucking care about wikipedia definitions and thanks but I'm not confused by any semantic. So yes, when I say Literal it's a name for the class used to represent the collection literals. If you find the naming confusing, then fine, we can call the class CollectionLiteral (that's even a good idea). Now the fact is that we need a class to represent those collection literals and we *need* a common ancestor to that class and the Term class. Making the literals being of class Term (or a subclass) would be ugly because none of the method of Term apply to the (collection) literals. But the literals also have methods that make no sense for Term (namely validateType, isEmpty and constructFunction), so we don't want to put those methods in the interface shared with Term either. So I don't see a cleaner to having Term, Value and Literal (though again, I'm fine changing the naming) and I'm not confused about that. And if we're being pedantic on naming, Term should not be renamed to UnaryLiteral because it does not only represent literals, but also bind variables. > Support set and map value types in CQL > -- > > Key: CASSANDRA-3647 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3647 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Jonathan Ellis >Assignee: Sylvain Lebresne > Labels: cql > Fix For: 1.2 > > > Composite columns introduce the ability to have arbitrarily nested data in a > Cassandra row. We should expose this through CQL. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4329) CQL3: Always use composite types by default
[ https://issues.apache.org/jira/browse/CASSANDRA-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406435#comment-13406435 ] Pavel Yaskevich commented on CASSANDRA-4329: +1 with nit - instead of "system" in systemTableQuery(...) would be better to use Table.SYSTEM_TABLE > CQL3: Always use composite types by default > > > Key: CASSANDRA-4329 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4329 > Project: Cassandra > Issue Type: Improvement >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Labels: cql3 > Fix For: 1.2 > > Attachments: 4329.txt, 4329_fix.txt > > > Currently, when defining a table with a single (non-composite) PRIMARY KEY, > we don't use a CompositeType in the underlying comparator. This is however a > problem for CASSANDRA-3647 as this means those tables cannot use collections. > So this ticket suggests to change that default behavior, and to always use > (by default at least, see below) a composite comparator underneath. I'll note > that doing so will mean an overhead of 3 bytes per column for non-composite > columns, but I believe getting collection is well worth it. > Of course the suggestion above apply to the default behavior and this ticket > would also add an option to table creation to get back to the current > behavior of not using a composite comparator (if ony for backward > compatibility sake). And I believe that we can actually reuse 'COMPACT > STORAGE' for that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-3647) Support set and map value types in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406428#comment-13406428
]
Pavel Yaskevich edited comment on CASSANDRA-3647 at 7/4/12 10:35 AM:
-
bq. We test 6 times for "instanceof CollectionType": one time is to call
serializeForThrift() and 2 other times are to call the execute() methods...
I wouldn't actually judge this by *current* usages of instanceof/casts because
what they show bad tendency where users are forced to use downcasts to get
things which are natural to the complex types but not reflected in the common
AbstractType interface. Every time instanceof is used it indicates that
components' OO design is broken.
If the most common thing we do after cast with CompositeType is get "types"
field then we could add Collection> getComponentTypes() method.
Simple types would return just one component - themselves (or throw an
exception saying that type is not complex to be sure that method is not
misused) and complex ones would be able to return immutable list of their
component types, in combination with isComposite() method that would eliminate
instanceof/downcasts completely.
I don't currently clearly see how to be with CollectionType.execute{Function}
methods but I will think about it.
bq. We cannot do that because Term implements Value. The point of Value is to
be a Term or a Literal.
That means that you are confusing semantics of the "literal" e.g.
https://en.wikipedia.org/wiki/Literal_(computer_programming) , so Term actually
should become {Unary | Single}Literal and implement Literal where {List, Set,
Map} classes would have UnaryLiteral as base element, there is no need to
invent hierarchy like Value.Literal just to make it fit with current Term
implementation.
was (Author: xedin):
bq. We test 6 times for "instanceof CollectionType": one time is to call
serializeForThrift() and 2 other times are to call the execute() methods...
I wouldn't actually judge this by *current* usages of instanceof/casts because
what they show bad tendency where users are forced to use downcasts to get
things which are natural to the complex types but not reflected in the common
AbstractType interface. Every time instanceof is used it indicates that
components' OO design is broken.
If the most common thing we do after cast with CompositeType is get "types"
field then we could add Collection> getComponentTypes() method.
Simple types would return just one component - themselves and complex ones
would be able to return immutable list of their component types, in combination
with isComposite() method that would eliminate instanceof/downcasts completely.
I don't currently clearly see how to be with CollectionType.execute{Function}
methods but I will think about it.
bq. We cannot do that because Term implements Value. The point of Value is to
be a Term or a Literal.
That means that you are confusing semantics of the "literal" e.g.
https://en.wikipedia.org/wiki/Literal_(computer_programming) , so Term actually
should become {Unary | Single}Literal and implement Literal where {List, Set,
Map} classes would have UnaryLiteral as base element, there is no need to
invent hierarchy like Value.Literal just to make it fit with current Term
implementation.
> Support set and map value types in CQL
> --
>
> Key: CASSANDRA-3647
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3647
> Project: Cassandra
> Issue Type: New Feature
> Components: API, Core
>Reporter: Jonathan Ellis
>Assignee: Sylvain Lebresne
> Labels: cql
> Fix For: 1.2
>
>
> Composite columns introduce the ability to have arbitrarily nested data in a
> Cassandra row. We should expose this through CQL.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3647) Support set and map value types in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406428#comment-13406428
]
Pavel Yaskevich commented on CASSANDRA-3647:
bq. We test 6 times for "instanceof CollectionType": one time is to call
serializeForThrift() and 2 other times are to call the execute() methods...
I wouldn't actually judge this by *current* usages of instanceof/casts because
what they show bad tendency where users are forced to use downcasts to get
things which are natural to the complex types but not reflected in the common
AbstractType interface. Every time instanceof is used it indicates that
components' OO design is broken.
If the most common thing we do after cast with CompositeType is get "types"
field then we could add Collection> getComponentTypes() method.
Simple types would return just one component - themselves and complex ones
would be able to return immutable list of their component types, in combination
with isComposite() method that would eliminate instanceof/downcasts completely.
I don't currently clearly see how to be with CollectionType.execute{Function}
methods but I will think about it.
bq. We cannot do that because Term implements Value. The point of Value is to
be a Term or a Literal.
That means that you are confusing semantics of the "literal" e.g.
https://en.wikipedia.org/wiki/Literal_(computer_programming) , so Term actually
should become {Unary | Single}Literal and implement Literal where {List, Set,
Map} classes would have UnaryLiteral as base element, there is no need to
invent hierarchy like Value.Literal just to make it fit with current Term
implementation.
> Support set and map value types in CQL
> --
>
> Key: CASSANDRA-3647
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3647
> Project: Cassandra
> Issue Type: New Feature
> Components: API, Core
>Reporter: Jonathan Ellis
>Assignee: Sylvain Lebresne
> Labels: cql
> Fix For: 1.2
>
>
> Composite columns introduce the ability to have arbitrarily nested data in a
> Cassandra row. We should expose this through CQL.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4400) Correctly catch exception when Snappy cannot be loaded
[
https://issues.apache.org/jira/browse/CASSANDRA-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406426#comment-13406426
]
Andy Cobley commented on CASSANDRA-4400:
I think it's being printed by :
static {
try {
impl = SnappyLoader.load();
}
catch (Exception e) {
e.printStackTrace();
}
in Snappy.java line 47. I can experiment and confirm
> Correctly catch exception when Snappy cannot be loaded
> --
>
> Key: CASSANDRA-4400
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4400
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.1.1
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Minor
> Fix For: 1.1.3
>
> Attachments: 4400.txt
>
>
> From the mailing list, on C* 1.1.1:
> {noformat}
> INFO 14:22:07,600 Global memtable threshold is enabled at 35MB
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at
> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> at org.xerial.snappy.Snappy.(Snappy.java:44)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:45)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:55)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.(SnappyCompressor.java:37)
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:76)
> at
> org.apache.cassandra.config.KSMetaData.systemKeyspace(KSMetaData.java:79)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadYaml(DatabaseDescriptor.java:439)
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:118)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:126)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353)
> at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106)
> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1681)
> at java.lang.Runtime.loadLibrary0(Runtime.java:840)
> at java.lang.System.loadLibrary(System.java:1047)
> at
> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
> ... 17 more
> ERROR 14:22:09,934 Exception encountered during startup
> org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] null
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229)
> at org.xerial.snappy.Snappy.(Snappy.java:44)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:45)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:55)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.(SnappyCompressor.java:37)
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:76)
> at
> org.apache.cassandra.config.KSMetaData.systemKeyspace(KSMetaData.java:79)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadYaml(DatabaseDescriptor.java:439)
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:118)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:126)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353)
> at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106)
> org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] null
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229)
> at org.xerial.snappy.Snappy.(Snappy.java:44)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:45)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:55)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.(SnappyCompressor.java:37)
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:76)
> at
> org.apache.cassandra.config.KSMetaData.systemKeyspace(KSMetaData.java:79)
> at
> org.apache.cassandra.config.DatabaseD
[jira] [Updated] (CASSANDRA-4329) CQL3: Always use composite types by default
[ https://issues.apache.org/jira/browse/CASSANDRA-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-4329: Attachment: 4329_fix.txt Fix attached. > CQL3: Always use composite types by default > > > Key: CASSANDRA-4329 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4329 > Project: Cassandra > Issue Type: Improvement >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Labels: cql3 > Fix For: 1.2 > > Attachments: 4329.txt, 4329_fix.txt > > > Currently, when defining a table with a single (non-composite) PRIMARY KEY, > we don't use a CompositeType in the underlying comparator. This is however a > problem for CASSANDRA-3647 as this means those tables cannot use collections. > So this ticket suggests to change that default behavior, and to always use > (by default at least, see below) a composite comparator underneath. I'll note > that doing so will mean an overhead of 3 bytes per column for non-composite > columns, but I believe getting collection is well worth it. > Of course the suggestion above apply to the default behavior and this ticket > would also add an option to table creation to get back to the current > behavior of not using a composite comparator (if ony for backward > compatibility sake). And I believe that we can actually reuse 'COMPACT > STORAGE' for that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4400) Correctly catch exception when Snappy cannot be loaded
[
https://issues.apache.org/jira/browse/CASSANDRA-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406410#comment-13406410
]
Sylvain Lebresne commented on CASSANDRA-4400:
-
Actually, I'd like to get rid of that exception at startup but I'm not sure why
that InvocationTargetException is not caught by the 'catch (Exception e)' in
SnappyCompressor.isAvailable(). Unless it's printed in the log by the snappy
library itself.
> Correctly catch exception when Snappy cannot be loaded
> --
>
> Key: CASSANDRA-4400
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4400
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.1.1
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Minor
> Fix For: 1.1.3
>
> Attachments: 4400.txt
>
>
> From the mailing list, on C* 1.1.1:
> {noformat}
> INFO 14:22:07,600 Global memtable threshold is enabled at 35MB
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at
> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> at org.xerial.snappy.Snappy.(Snappy.java:44)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:45)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:55)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.(SnappyCompressor.java:37)
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:76)
> at
> org.apache.cassandra.config.KSMetaData.systemKeyspace(KSMetaData.java:79)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadYaml(DatabaseDescriptor.java:439)
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:118)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:126)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353)
> at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106)
> Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1681)
> at java.lang.Runtime.loadLibrary0(Runtime.java:840)
> at java.lang.System.loadLibrary(System.java:1047)
> at
> org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
> ... 17 more
> ERROR 14:22:09,934 Exception encountered during startup
> org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] null
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229)
> at org.xerial.snappy.Snappy.(Snappy.java:44)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:45)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:55)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.(SnappyCompressor.java:37)
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:76)
> at
> org.apache.cassandra.config.KSMetaData.systemKeyspace(KSMetaData.java:79)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadYaml(DatabaseDescriptor.java:439)
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:118)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:126)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353)
> at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106)
> org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] null
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229)
> at org.xerial.snappy.Snappy.(Snappy.java:44)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:45)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:55)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.(SnappyCompressor.java:37)
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:76)
> at
> org.apache.cassandra.config.KSMetaData.systemKeyspace(KSMetaData.java:79)
> at
> org.apache
[jira] [Updated] (CASSANDRA-4409) Only consider whole row tombstone in collation controller
[ https://issues.apache.org/jira/browse/CASSANDRA-4409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-4409: Attachment: 0001-Use-only-top-level-row-deletion-to-avoid-sstable-durin.txt Patch attached to fix. I will note that in practice this was not really a bug because the deletionInfo used were those of the columnIterator, and at that point those deletionInfo should only contain whole row tombstone, not range tombstone. Yet, the code was misleading and could have trigger a bug if the code change underneath. > Only consider whole row tombstone in collation controller > - > > Key: CASSANDRA-4409 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4409 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.2 >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne >Priority: Trivial > Fix For: 1.2 > > Attachments: > 0001-Use-only-top-level-row-deletion-to-avoid-sstable-durin.txt > > > CollationController has that optimization that if it has already seen a row > tombstone more recent that a sstable max timestamp, it skips the sstable. > However, this was not updated correctly while introducing range tombstone and > currently the code might skip a sstable based on the timestamp of a tombstone > that does not cover the full row. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4409) Only consider whole row tombstone in collation controller
Sylvain Lebresne created CASSANDRA-4409: --- Summary: Only consider whole row tombstone in collation controller Key: CASSANDRA-4409 URL: https://issues.apache.org/jira/browse/CASSANDRA-4409 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Priority: Trivial Fix For: 1.2 CollationController has that optimization that if it has already seen a row tombstone more recent that a sstable max timestamp, it skips the sstable. However, this was not updated correctly while introducing range tombstone and currently the code might skip a sstable based on the timestamp of a tombstone that does not cover the full row. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3647) Support set and map value types in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406402#comment-13406402
]
Sylvain Lebresne commented on CASSANDRA-3647:
-
bq. that we need a casts/instanceof checks which is a chronic problem of type
hierarchy of o.a.c.db.marshal package related to Composite and Collection types
We test 6 times for "instanceof CollectionType": one time is to call
serializeForThrift() and 2 other times are to call the execute() methods. Both
of those are very specific to CollectionType currently and I'm not convainced
at all moving these "functionality" to AbstractType would make sense/be
cleaner. One other time, we cast to CollectionType in order to construct a
ColumnToCollectionType so we'll have to do that cast anyway. Remains 2
occurences were we could indeed replace the 'instanceof' by a 'isCollection()'
method (we don't cast to CollectionType after the instanceof test), but again
I'm not sure there is a point? It won't be measurably faster and it won't even
save characters (since we'll have to define the isCollection method).
As for CompositeType, there is (only) 4 occurrences of "instanceof
CompositeType", all of which are followed by a cast so that we can access the
'types' field of CompositeType. I don't see any sane way to move that
functionality into AbstractType.
So honestly I think we actually do a reasonably good job of 'reflecting their
most commonly used functionality in AbstractType' and in practice I don't
really see how to do better.
bq. remove (or rename Value to) Literal as all of the classes implement only
that one interface, so it would be someting like Literal.{List, Set, Map} and
you would be able to "assert instance of Literal" in UpdateStatement?
We cannot do that because Term implements Value. The point of Value is to be a
Term or a Literal.
bq. There are couple of same typos in the UpdateStatement left - lines _341_,
_349_ and _373_
I'll update those.
> Support set and map value types in CQL
> --
>
> Key: CASSANDRA-3647
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3647
> Project: Cassandra
> Issue Type: New Feature
> Components: API, Core
>Reporter: Jonathan Ellis
>Assignee: Sylvain Lebresne
> Labels: cql
> Fix For: 1.2
>
>
> Composite columns introduce the ability to have arbitrarily nested data in a
> Cassandra row. We should expose this through CQL.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4321) stackoverflow building interval tree & possible sstable corruptions
[ https://issues.apache.org/jira/browse/CASSANDRA-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406395#comment-13406395 ] Sylvain Lebresne commented on CASSANDRA-4321: - Damn. Ok, since this has been released with 1.1.2 already, would you mind opening a new one? > stackoverflow building interval tree & possible sstable corruptions > --- > > Key: CASSANDRA-4321 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4321 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.1.1 >Reporter: Anton Winter >Assignee: Sylvain Lebresne > Fix For: 1.1.2 > > Attachments: 0001-Fix-overlapping-computation-v7.txt, > 0002-Scrub-detects-and-repair-outOfOrder-rows-v7.txt, > 0003-Create-standalone-scrub-v7.txt, > 0004-Add-manifest-integrity-check-v7.txt, cleanup.txt, > ooyala-hastur-stacktrace.txt > > > After upgrading to 1.1.1 (from 1.1.0) I have started experiencing > StackOverflowError's resulting in compaction backlog and failure to restart. > The ring currently consists of 6 DC's and 22 nodes using LCS & compression. > This issue was first noted on 2 nodes in one DC and then appears to have > spread to various other nodes in the other DC's. > When the first occurrence of this was found I restarted the instance but it > failed to start so I cleared its data and treated it as a replacement node > for the token it was previously responsible for. This node successfully > streamed all the relevant data back but failed again a number of hours later > with the same StackOverflowError and again was unable to restart. > The initial stack overflow error on a running instance looks like this: > ERROR [CompactionExecutor:314] 2012-06-07 09:59:43,017 > AbstractCassandraDaemon.java (line 134) Exception in thread > Thread[CompactionExecutor:314,1,main] > java.lang.StackOverflowError > at java.util.Arrays.mergeSort(Arrays.java:1157) > at java.util.Arrays.sort(Arrays.java:1092) > at java.util.Collections.sort(Collections.java:134) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.findMinMedianMax(IntervalNode.java:114) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:49) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow. Compactions stop from this point > onwards] > I restarted this failing instance with DEBUG logging enabled and it throws > the following exception part way through startup: > ERROR 11:37:51,046 Exception in thread Thread[OptionalTasks:1,5,main] > java.lang.StackOverflowError > at > org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:307) > at > org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:276) > at > org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:230) > at > org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:124) > at > org.slf4j.impl.Log4jLoggerAdapter.debug(Log4jLoggerAdapter.java:228) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:45) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > [snip - this repeats until stack overflow] > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:64) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalNode.(IntervalNode.java:62) > at > org.apache.cassandra.utils.IntervalTree.IntervalTree.(IntervalTree.java:39) > at > org.a
