[jira] [Updated] (CASSANDRA-10109) Windows dtest 3.0: ttl_test.py failures
[
https://issues.apache.org/jira/browse/CASSANDRA-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefania updated CASSANDRA-10109:
-
Fix Version/s: (was: 3.0.x)
3.0.0 rc1
> Windows dtest 3.0: ttl_test.py failures
> ---
>
> Key: CASSANDRA-10109
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10109
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Joshua McKenzie
>Assignee: Stefania
> Labels: Windows
> Fix For: 3.0.0 rc1
>
>
> ttl_test.py:TestTTL.update_column_ttl_with_default_ttl_test2
> ttl_test.py:TestTTL.update_multiple_columns_ttl_test
> ttl_test.py:TestTTL.update_single_column_ttl_test
> Errors locally are different than CI from yesterday. Yesterday on CI we have
> timeouts and general node hangs. Today on all 3 tests when run locally I see:
> {noformat}
> Traceback (most recent call last):
> File "c:\src\cassandra-dtest\dtest.py", line 532, in tearDown
> raise AssertionError('Unexpected error in %s node log: %s' % (node.name,
> errors))
> AssertionError: Unexpected error in node1 node log: ['ERROR [main] 2015-08-17
> 16:53:43,120 NoSpamLogger.java:97 - This platform does not support atomic
> directory streams (SecureDirectoryStream); race conditions when loading
> sstable files could occurr']
> {noformat}
> This traces back to the commit for CASSANDRA-7066 today by [~Stefania] and
> [~benedict]. Stefania - care to take this ticket and also look further into
> whether or not we're going to have issues with 7066 on Windows? That error
> message certainly *sounds* like it's not a good thing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10035) Windows utest 3.0: TransactionLogsTest failure
[
https://issues.apache.org/jira/browse/CASSANDRA-10035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708767#comment-14708767
]
Stefania edited comment on CASSANDRA-10035 at 8/24/15 4:41 AM:
---
Since CASSANDRA-7066 was committed, the tests have been failing less but we
still have problems. The first utest build on 3.0 with the patch committed was
build #15. Since then:
* TransactionLogTest.testObsoletedDataFileUpdateTimeChanged failed on build 15
* TransactionLogTest.testWrongChecksumLastLineMissingFile failed on build 19
* TransactionLogTest.testRemoveUnfinishedLeftovers_abort and
testRemoveUnfinishedLeftovers_commit failed on build 22
The failure in build 15 is odd, it seems we did not pick up the modification of
the last updated time, we probably just need to release the sstables earlier.
The remaining failures are all related to listing temporary files, so I suspect
the root cause is CASSANDRA-10109.
was (Author: stefania):
Since CASSANDRA-7066 was committed, the tests have been failing less but we
still have problems. The first utest build on 3.0 with the patch committed was
build #15. Since there:
* TransactionLogTest.testObsoletedDataFileUpdateTimeChanged failed on build 15
* TransactionLogTest.testWrongChecksumLastLineMissingFile failed on build 19
* TransactionLogTest.testRemoveUnfinishedLeftovers_abort and
testRemoveUnfinishedLeftovers_commit failed on build 22
The failure in build 15 is odd, it seems we did not pick up the modification of
the last updated time, we probably just need to release the sstables earlier.
The remaining failures are all related to listing temporary files, so I suspect
the root cause is CASSANDRA-10109.
> Windows utest 3.0: TransactionLogsTest failure
> --
>
> Key: CASSANDRA-10035
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10035
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Joshua McKenzie
>Assignee: Stefania
> Labels: Windows
> Fix For: 3.0.0 rc1
>
>
> {noformat}
> [junit] Testcase:
> testUntrack(org.apache.cassandra.db.lifecycle.TransactionLogsTest):
> FAILED
> [junit]
> build\test\cassandra\data;0\TransactionLogsTest\mockcf4-6742d2103f8911e58737831ef068609c\ma-1-big-Data.db
> [junit] junit.framework.AssertionFailedError:
> build\test\cassandra\data;0\TransactionLogsTest\mockcf4-6742d2103f8911e58737831ef068609c\ma-1-big-Data.db
> [junit] at
> org.apache.cassandra.db.lifecycle.TransactionLogsTest.assertFiles(TransactionLogsTest.java:575)
> [junit] at
> org.apache.cassandra.db.lifecycle.TransactionLogsTest.testUntrack(TransactionLogsTest.java:215)
> [junit]
> [junit]
> [junit] Testcase:
> testAbortOnlyNew(org.apache.cassandra.db.lifecycle.TransactionLogsTest):
> FAILED
> [junit]
> build\test\cassandra\data;0\TransactionLogsTest\mockcf5-67456a203f8911e58737831ef068609c\ma-0-big-Data.db
> [junit] junit.framework.AssertionFailedError:
> build\test\cassandra\data;0\TransactionLogsTest\mockcf5-67456a203f8911e58737831ef068609c\ma-0-big-Data.db
> [junit] at
> org.apache.cassandra.db.lifecycle.TransactionLogsTest.assertFiles(TransactionLogsTest.java:575)
> [junit] at
> org.apache.cassandra.db.lifecycle.TransactionLogsTest.testAbortOnlyNew(TransactionLogsTest.java:312)
> [junit]
> [junit]
> [junit] Testcase:
> testNoPrepare(org.apache.cassandra.db.lifecycle.TransactionLogsTest):
> FAILED
> [junit]
> build\test\cassandra\data;0\TransactionLogsTest\mockcf13-675681203f8911e58737831ef068609c\ma-1-big-Data.db
> [junit] junit.framework.AssertionFailedError:
> build\test\cassandra\data;0\TransactionLogsTest\mockcf13-675681203f8911e58737831ef068609c\ma-1-big-Data.db
> [junit] at
> org.apache.cassandra.db.lifecycle.TransactionLogsTest.assertFiles(TransactionLogsTest.java:575)
> [junit] at
> org.apache.cassandra.db.lifecycle.TransactionLogsTest.access$200(TransactionLogsTest.java:56)
> [junit] at
> org.apache.cassandra.db.lifecycle.TransactionLogsTest$TxnTest$Transaction.assertAborted(TransactionLogsTest.java:143)
> [junit] at
> org.apache.cassandra.db.lifecycle.TransactionLogsTest$TxnTest.assertAborted(TransactionLogsTest.java:191)
> [junit] at
> org.apache.cassandra.utils.concurrent.AbstractTransactionalTest.testNoPrepare(AbstractTransactionalTest.java:40)
> {noformat}
> taking the testPrepare case as an example: looks like it never passed on
> Windows
> ([history|http://cassci.datastax.com/view/trunk/job/trunk_utest_win32/lastCompletedBuild/testReport/org.apache.cassandra.db.lifecycle/TransactionLogsTest/testPrepare/history/])
> [~stefania_alborghetti]: annotate points to you on these tests. Care to take
> a look at this?
--
This mes
[jira] [Commented] (CASSANDRA-10035) Windows utest 3.0: TransactionLogsTest failure
[
https://issues.apache.org/jira/browse/CASSANDRA-10035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708767#comment-14708767
]
Stefania commented on CASSANDRA-10035:
--
Since CASSANDRA-7066 was committed, the tests have been failing less but we
still have problems. The first utest build on 3.0 with the patch committed was
build #15. Since there:
* TransactionLogTest.testObsoletedDataFileUpdateTimeChanged failed on build 15
* TransactionLogTest.testWrongChecksumLastLineMissingFile failed on build 19
* TransactionLogTest.testRemoveUnfinishedLeftovers_abort and
testRemoveUnfinishedLeftovers_commit failed on build 22
The failure in build 15 is odd, it seems we did not pick up the modification of
the last updated time, we probably just need to release the sstables earlier.
The remaining failures are all related to listing temporary files, so I suspect
the root cause is CASSANDRA-10109.
> Windows utest 3.0: TransactionLogsTest failure
> --
>
> Key: CASSANDRA-10035
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10035
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Joshua McKenzie
>Assignee: Stefania
> Labels: Windows
> Fix For: 3.0.0 rc1
>
>
> {noformat}
> [junit] Testcase:
> testUntrack(org.apache.cassandra.db.lifecycle.TransactionLogsTest):
> FAILED
> [junit]
> build\test\cassandra\data;0\TransactionLogsTest\mockcf4-6742d2103f8911e58737831ef068609c\ma-1-big-Data.db
> [junit] junit.framework.AssertionFailedError:
> build\test\cassandra\data;0\TransactionLogsTest\mockcf4-6742d2103f8911e58737831ef068609c\ma-1-big-Data.db
> [junit] at
> org.apache.cassandra.db.lifecycle.TransactionLogsTest.assertFiles(TransactionLogsTest.java:575)
> [junit] at
> org.apache.cassandra.db.lifecycle.TransactionLogsTest.testUntrack(TransactionLogsTest.java:215)
> [junit]
> [junit]
> [junit] Testcase:
> testAbortOnlyNew(org.apache.cassandra.db.lifecycle.TransactionLogsTest):
> FAILED
> [junit]
> build\test\cassandra\data;0\TransactionLogsTest\mockcf5-67456a203f8911e58737831ef068609c\ma-0-big-Data.db
> [junit] junit.framework.AssertionFailedError:
> build\test\cassandra\data;0\TransactionLogsTest\mockcf5-67456a203f8911e58737831ef068609c\ma-0-big-Data.db
> [junit] at
> org.apache.cassandra.db.lifecycle.TransactionLogsTest.assertFiles(TransactionLogsTest.java:575)
> [junit] at
> org.apache.cassandra.db.lifecycle.TransactionLogsTest.testAbortOnlyNew(TransactionLogsTest.java:312)
> [junit]
> [junit]
> [junit] Testcase:
> testNoPrepare(org.apache.cassandra.db.lifecycle.TransactionLogsTest):
> FAILED
> [junit]
> build\test\cassandra\data;0\TransactionLogsTest\mockcf13-675681203f8911e58737831ef068609c\ma-1-big-Data.db
> [junit] junit.framework.AssertionFailedError:
> build\test\cassandra\data;0\TransactionLogsTest\mockcf13-675681203f8911e58737831ef068609c\ma-1-big-Data.db
> [junit] at
> org.apache.cassandra.db.lifecycle.TransactionLogsTest.assertFiles(TransactionLogsTest.java:575)
> [junit] at
> org.apache.cassandra.db.lifecycle.TransactionLogsTest.access$200(TransactionLogsTest.java:56)
> [junit] at
> org.apache.cassandra.db.lifecycle.TransactionLogsTest$TxnTest$Transaction.assertAborted(TransactionLogsTest.java:143)
> [junit] at
> org.apache.cassandra.db.lifecycle.TransactionLogsTest$TxnTest.assertAborted(TransactionLogsTest.java:191)
> [junit] at
> org.apache.cassandra.utils.concurrent.AbstractTransactionalTest.testNoPrepare(AbstractTransactionalTest.java:40)
> {noformat}
> taking the testPrepare case as an example: looks like it never passed on
> Windows
> ([history|http://cassci.datastax.com/view/trunk/job/trunk_utest_win32/lastCompletedBuild/testReport/org.apache.cassandra.db.lifecycle/TransactionLogsTest/testPrepare/history/])
> [~stefania_alborghetti]: annotate points to you on these tests. Care to take
> a look at this?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10086) Add a "CLEAR" cqlsh command to clear the console
[ https://issues.apache.org/jira/browse/CASSANDRA-10086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708745#comment-14708745 ] Stefania commented on CASSANDRA-10086: -- Thanks for the patch, +1. [~jbellis] could you commit on 3.0? The tests for cqlsh are located in _cqlsh_tests/cqlsh_tests.py_. We are missing some commands but we've started adding tests whenever we make any changes to existing commands or add new ones. A good example would be the DESCRIBE command, for which you find some tests in this file. The test is actually quite simple, we just run cqlsh and check that there is no error in stderr and that stdout contains one of the clear screen ANSI escape sequences. Have a look at the top commit [here|https://github.com/stef1927/cassandra-dtest/commits/10086]. It works on Linux. Are you by any chance running Windows or Mac OS, to see if it works there too? If not, don't worry about it, just review the test and confirm you are happy with it. > Add a "CLEAR" cqlsh command to clear the console > > > Key: CASSANDRA-10086 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10086 > Project: Cassandra > Issue Type: Improvement >Reporter: Paul O'Fallon >Priority: Trivial > Labels: cqlsh, doc-impacting > Attachments: 10086.txt, 10086v2.txt > > > It would be very helpful to have a "CLEAR" command to clear the cqlsh > console. I learned (after researching a patch for this) that lowercase > CTRL+L will clear the screen, but having a discrete command would make that > more obvious. To match the expectations of Windows users, an alias to "CLS" > would be nice as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8630) Faster sequential IO (on compaction, streaming, etc)
[
https://issues.apache.org/jira/browse/CASSANDRA-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708687#comment-14708687
]
Stefania commented on CASSANDRA-8630:
-
I've fixed {{ChecksummedDataInput}} so that the slow path in
{{RebufferingInputStream}} is no longer required.
I've added a comment regarding copying of {{MmappedRegions}} and thread safety
and filed a follow-up ticket: CASSANDRA-10158.
[~aweisberg], ready for another round of review.
> Faster sequential IO (on compaction, streaming, etc)
>
>
> Key: CASSANDRA-8630
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8630
> Project: Cassandra
> Issue Type: Improvement
> Components: Core, Tools
>Reporter: Oleg Anastasyev
>Assignee: Stefania
> Labels: compaction, performance
> Fix For: 3.x
>
> Attachments: 8630-FasterSequencialReadsAndWrites.txt, cpu_load.png,
> flight_recorder_001_files.tar.gz, flight_recorder_002_files.tar.gz,
> mmaped_uncomp_hotspot.png
>
>
> When node is doing a lot of sequencial IO (streaming, compacting, etc) a lot
> of CPU is lost in calls to RAF's int read() and DataOutputStream's write(int).
> This is because default implementations of readShort,readLong, etc as well as
> their matching write* are implemented with numerous calls of byte by byte
> read and write.
> This makes a lot of syscalls as well.
> A quick microbench shows than just reimplementation of these methods in
> either way gives 8x speed increase.
> A patch attached implements RandomAccessReader.read and
> SequencialWriter.write methods in more efficient way.
> I also eliminated some extra byte copies in CompositeType.split and
> ColumnNameHelper.maxComponents, which were on my profiler's hotspot method
> list during tests.
> A stress tests on my laptop show that this patch makes compaction 25-30%
> faster on uncompressed sstables and 15% faster for compressed ones.
> A deployment to production shows much less CPU load for compaction.
> (I attached a cpu load graph from one of our production, orange is niced CPU
> load - i.e. compaction; yellow is user - i.e. not compaction related tasks)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10158) Rationalize implementations of DataInputPlus and DataOutputPlus
Stefania created CASSANDRA-10158:
Summary: Rationalize implementations of DataInputPlus and
DataOutputPlus
Key: CASSANDRA-10158
URL: https://issues.apache.org/jira/browse/CASSANDRA-10158
Project: Cassandra
Issue Type: Improvement
Reporter: Stefania
Assignee: Stefania
Following CASSANDRA-8630, we can further improve implementations of
{{DataInputPlus}} and {{DataOutputPlus}} as follows:
* In {{MmappedRegions}}, compact the mmap ranges, at least on the final opening
of the file
* Use the mmap extension logic for compressed files
* Consider renaming classes to more appropriate names, moving them in their own
package and making their relations more simply defined.
* Consider unifying {{ChecksummedDataInput}} and
{{ChecksummedRandomAccessReader}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10155) 2i key cache load fails
[
https://issues.apache.org/jira/browse/CASSANDRA-10155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708635#comment-14708635
]
Daniel Chia commented on CASSANDRA-10155:
-
It looks like switching to use cfId exclusively in CASSANDRA-7784 introduced
this regression, so it will be found 2.1 as well.
Looks like the right fix is to re-introduce pathInfo - what do people think?
> 2i key cache load fails
> ---
>
> Key: CASSANDRA-10155
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10155
> Project: Cassandra
> Issue Type: Bug
>Reporter: Robert Stupp
> Fix For: 3.0.0 rc1
>
>
> CASSANDRA-9265 changed how key cache content is serialized to disk. It uses
> {{UUID cfId}} to generate the file path for each {{ColumnFamilyStore}}.
> Since {{cfId}} of a secondary index is the same as for the base table, the
> key-cache files for 2i's and the base are the same. This will/may lead to
> deserialization failures on restart for tables with at least one 2i.
> /cc [~aweisberg] [~danchia]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10155) 2i key cache load fails
[
https://issues.apache.org/jira/browse/CASSANDRA-10155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708617#comment-14708617
]
Daniel Chia commented on CASSANDRA-10155:
-
Ah good call. That was actually introduced in CASSANDRA-7784, I'll investigate
further.
> 2i key cache load fails
> ---
>
> Key: CASSANDRA-10155
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10155
> Project: Cassandra
> Issue Type: Bug
>Reporter: Robert Stupp
> Fix For: 3.0.0 rc1
>
>
> CASSANDRA-9265 changed how key cache content is serialized to disk. It uses
> {{UUID cfId}} to generate the file path for each {{ColumnFamilyStore}}.
> Since {{cfId}} of a secondary index is the same as for the base table, the
> key-cache files for 2i's and the base are the same. This will/may lead to
> deserialization failures on restart for tables with at least one 2i.
> /cc [~aweisberg] [~danchia]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10061) Only use batchlog when paired view replica is remote
[ https://issues.apache.org/jira/browse/CASSANDRA-10061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708484#comment-14708484 ] Aleksey Yeschenko commented on CASSANDRA-10061: --- What happens if a node dies between the apply() calls? We'll still recover all right? > Only use batchlog when paired view replica is remote > > > Key: CASSANDRA-10061 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10061 > Project: Cassandra > Issue Type: Improvement >Reporter: T Jake Luciani >Assignee: T Jake Luciani > Fix For: 3.0 beta 2 > > > As described in the MV design doc the base and view replicas are paired one > to one. > If the replica selected for the view is the local node itself there is no > need to create a local batchlog and we can simply apply the view mutations > locally. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9973) java.lang.IllegalStateException: Unable to compute when histogram overflowed
[
https://issues.apache.org/jira/browse/CASSANDRA-9973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708448#comment-14708448
]
Mark Manley commented on CASSANDRA-9973:
Do we have an ETA for the release date of 2.2.1? My ring was again crippled
this morning when several of my nodes spewed out hundreds of these errors a
minute. It corresponds to the time that they stopped answering requests
reliably. If there is a workaround for 2.2.0, I am all ears.
Thanks!
> java.lang.IllegalStateException: Unable to compute when histogram overflowed
>
>
> Key: CASSANDRA-9973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9973
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Mark Manley
>Assignee: T Jake Luciani
> Fix For: 2.2.1
>
> Attachments: 9973.txt
>
>
> I recently, and probably mistakenly, upgraded one of my production C*
> clusters to 2.2.0. I am seeing these errors in the logs, followed by an
> intense period of garbage collection until the node, then the ring, becomes
> crippled:
> {code}
> ERROR [OptionalTasks:1] 2015-08-04 03:24:56,057 CassandraDaemon.java:182 -
> Exception in thread Thread[OptionalTasks:1,5,main]
> java.lang.IllegalStateException: Unable to compute when histogram overflowed
> at
> org.apache.cassandra.utils.EstimatedHistogram.percentile(EstimatedHistogram.java:179)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.metrics.EstimatedHistogramReservoir$HistogramSnapshot.getValue(EstimatedHistogramReservoir.java:84)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.ColumnFamilyStore$3.run(ColumnFamilyStore.java:405)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_45]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_45]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> {code}
> I am not sure if the GC instability is this or something else, but I though
> this histogram overflow issue was fixed in 2.1.3? Anyway, reporting now as a
> possible regression. Please let me know what I can provide in terms of
> information to help with this. Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10157) SchemaKeyspace.addAggregateToSchemaMutation keeps previous on CREATE OR REPLACE AGGREGATE
Robert Stupp created CASSANDRA-10157: Summary: SchemaKeyspace.addAggregateToSchemaMutation keeps previous on CREATE OR REPLACE AGGREGATE Key: CASSANDRA-10157 URL: https://issues.apache.org/jira/browse/CASSANDRA-10157 Project: Cassandra Issue Type: Bug Reporter: Robert Stupp Assignee: Robert Stupp Fix For: 3.0 beta 2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9738) Migrate key-cache to be fully off-heap
[
https://issues.apache.org/jira/browse/CASSANDRA-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708431#comment-14708431
]
Robert Stupp commented on CASSANDRA-9738:
-
Branch is ready for another review round.
It's rebased against current 3.0 and trunk.
Changes are:
* updated to ohc 0.4.2 (to get rid of {{ByteBuffer.order()}} calls and the
try-finally-clauses)
* no more duplicate serialization effort in {{serializedSize()}} +
{{serialize()}} - uses a thread-local serialization buffer. Can be further
improved with CASSANDRA-9929.
* added/moved string serialization methods to {{ByteBufferUtil}}, which has
distinct code paths for 7-bit ASCII and strings with wider chars
> Migrate key-cache to be fully off-heap
> --
>
> Key: CASSANDRA-9738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9738
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Fix For: 3.0 beta 2
>
>
> Key cache still uses a concurrent map on-heap. This could go to off-heap and
> feels doable now after CASSANDRA-8099.
> Evaluation should be done in advance based on a POC to prove that pure
> off-heap counter cache buys a performance and/or gc-pressure improvement.
> In theory, elimination of on-heap management of the map should buy us some
> benefit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6018) Add option to encrypt commitlog
[
https://issues.apache.org/jira/browse/CASSANDRA-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708426#comment-14708426
]
Jason Brown commented on CASSANDRA-6018:
bq. the way the header parameters are used by the descriptor
You are correct - the only thing I'm using the headerParameters field for is to
stash the IV, so that we can get it at read-time (when replaying the commit
log). I was hoping to keep EncyptionContext/TransparentDataEncryptionOptions a
bit more general, and not stuff file-specific details in those classes, but
perhaps somewhere in those classes is the best for the IV to go, after all.
I'll think about that some more.
bq. Since you are compressing to ON_HEAP, I think it will help to use the same
buffer type for the input as well
Will do. As a side note, I originally wrote this code using the byte array APIs
of Cipher, so thus everything needed to be on-heap. After I discovered the
ByteBuffer versions of the Cipher methods, I switched to those and honestly
didn't think about the on/off heap question again. I can give a shot to trying
everything off-heap vs. on-heap, but not sure there's a lot of gain in the
off-heap model as, after we compress to off-heap, we would need to pull those
bytes back into local process memory to encrypt. Any opinions, [~benedict]?
bq. {{doFinal}} is copy-safe
Huh, somehow I've always been using two buffers (separate in/out) with
encryption operations, and never saw that clause in the API :). I'll give that
a shot and see what happens - thanks for pointing it out!
Will fix the other items, as well, in next update.
> Add option to encrypt commitlog
>
>
> Key: CASSANDRA-6018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6018
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
>Reporter: Jason Brown
>Assignee: Jason Brown
> Labels: commit_log, encryption, security
> Fix For: 3.x
>
>
> We are going to start using cassandra for a billing system, and while I can
> encrypt sstables at rest (via Datastax Enterprise), commit logs are more or
> less plain text. Thus, an attacker would be able to easily read, for example,
> credit card numbers in the clear text commit log (if the calling app does not
> encrypt the data itself before sending it to cassandra).
> I want to allow the option of encrypting the commit logs, most likely
> controlled by a property in the yaml.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-6018) Add option to encrypt commitlog
[
https://issues.apache.org/jira/browse/CASSANDRA-6018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708426#comment-14708426
]
Jason Brown edited comment on CASSANDRA-6018 at 8/23/15 4:26 PM:
-
bq. the way the header parameters are used by the descriptor
You are correct - the only thing I'm using the headerParameters field for is to
stash the IV, so that we can get it at read-time (when replaying the commit
log). I was hoping to keep EncyptionContext/TransparentDataEncryptionOptions a
bit more general, and not stuff file-specific details in those classes, but
perhaps somewhere in those classes is the best for the IV to go, after all.
I'll think about that some more.
bq. Since you are compressing to ON_HEAP, I think it will help to use the same
buffer type for the input as well
Will do. As a side note, I originally wrote this code using the byte array APIs
of Cipher, so thus everything needed to be on-heap. After I discovered the
ByteBuffer versions of the Cipher methods, I switched to those and honestly
didn't think about the on/off heap question again. I can give a shot to trying
everything off-heap vs. on-heap, but not sure there's a lot of gain in the
off-heap model as, after we compress to off-heap, we would need to pull those
bytes back into local process memory to encrypt. Any opinions, [~benedict]?
bq. {{doFinal}} is copy-safe
Huh, somehow I've always been using two buffers (separate in/out) with
encryption operations with Cipher, and never saw that clause in the API :).
I'll give that a shot and see what happens - thanks for pointing it out!
Will fix the other items, as well, in next update.
was (Author: jasobrown):
bq. the way the header parameters are used by the descriptor
You are correct - the only thing I'm using the headerParameters field for is to
stash the IV, so that we can get it at read-time (when replaying the commit
log). I was hoping to keep EncyptionContext/TransparentDataEncryptionOptions a
bit more general, and not stuff file-specific details in those classes, but
perhaps somewhere in those classes is the best for the IV to go, after all.
I'll think about that some more.
bq. Since you are compressing to ON_HEAP, I think it will help to use the same
buffer type for the input as well
Will do. As a side note, I originally wrote this code using the byte array APIs
of Cipher, so thus everything needed to be on-heap. After I discovered the
ByteBuffer versions of the Cipher methods, I switched to those and honestly
didn't think about the on/off heap question again. I can give a shot to trying
everything off-heap vs. on-heap, but not sure there's a lot of gain in the
off-heap model as, after we compress to off-heap, we would need to pull those
bytes back into local process memory to encrypt. Any opinions, [~benedict]?
bq. {{doFinal}} is copy-safe
Huh, somehow I've always been using two buffers (separate in/out) with
encryption operations, and never saw that clause in the API :). I'll give that
a shot and see what happens - thanks for pointing it out!
Will fix the other items, as well, in next update.
> Add option to encrypt commitlog
>
>
> Key: CASSANDRA-6018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6018
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
>Reporter: Jason Brown
>Assignee: Jason Brown
> Labels: commit_log, encryption, security
> Fix For: 3.x
>
>
> We are going to start using cassandra for a billing system, and while I can
> encrypt sstables at rest (via Datastax Enterprise), commit logs are more or
> less plain text. Thus, an attacker would be able to easily read, for example,
> credit card numbers in the clear text commit log (if the calling app does not
> encrypt the data itself before sending it to cassandra).
> I want to allow the option of encrypting the commit logs, most likely
> controlled by a property in the yaml.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9673) Improve batchlog write path
[
https://issues.apache.org/jira/browse/CASSANDRA-9673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708398#comment-14708398
]
Aleksey Yeschenko commented on CASSANDRA-9673:
--
Rebased against most recent trunk and made some minor changes, here:
https://github.com/iamaleksey/cassandra/tree/9673-3.0. Didn't go very far.
Removed {{BATCHLOG_MUTATION}} Stage that Carl and I think is no longer needed
(but we'll bring it back if that's a mistake), and did some renames and minor
cleanup of unused methods.
Good work. There is only one real issue so far, with that fixed, we could
commit as is, but I have a bunch of nits that'd be nice to address: we are not
properly handling old-format batchlog mutations sent live from 2.1/2.2 nodes,
during live upgrades.
You need to modify {{MutationVerbHandler}} and add special treatment to
mutations for the old batchlog table.
Nits:
1. I would prefer {{BatchStoreMessage}} and {{BatchRemoveMessage}} to be dumb
and only be concerned about serialization/deserialization. This means no
{{getMutation()}} in either. For {{BatchRemoveMessage}}, the actual logic
should be in {{BatchRemoveVerbHandler}}.
2. I would like to separate {{Batch}} and {{BatchStoreMessage}} and put actual
saving to batchlog logic into a new {{Batch}} class - to be used both for local
(MV) writes and proper BL remote writes.
3. If moved to {{Batch}} (and if not, too), I would prefer the mutation
collections in them to only be set in the constructor.
4. Somehow we are calling {{LegacyBatchMigrator.convertBatchEntries()}} on
every replay, reverting Branimir's changes, instead of explicitly only doing it
once, upon first replay. I'm presuming that this is done as a workaround for
the live upgrade issue. I would prefer that we moved the migration to
{{CassandraDaemon.setup()}}, where legacy schema and hints migration already
get called.
> Improve batchlog write path
> ---
>
> Key: CASSANDRA-9673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9673
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Stefania
> Labels: performance
> Fix For: 3.0 beta 2
>
> Attachments: 9673_001.tar.gz, 9673_004.tar.gz,
> gc_times_first_node_patched_004.png, gc_times_first_node_trunk_004.png
>
>
> Currently we allocate an on-heap {{ByteBuffer}} to serialize the batched
> mutations into, before sending it to a distant node, generating unnecessary
> garbage (potentially a lot of it).
> With materialized views using the batchlog, it would be nice to optimise the
> write path:
> - introduce a new verb ({{Batch}})
> - introduce a new message ({{BatchMessage}}) that would encapsulate the
> mutations, expiration, and creation time (similar to {{HintMessage}} in
> CASSANDRA-6230)
> - have MS serialize it directly instead of relying on an intermediate buffer
> To avoid merely shifting the temp buffer to the receiving side(s) we should
> change the structure of the batchlog table to use a list or a map of
> individual mutations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10151) Fix TrackerTest.testTryModify
[
https://issues.apache.org/jira/browse/CASSANDRA-10151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708349#comment-14708349
]
Benedict commented on CASSANDRA-10151:
--
Thanks. Committed as 941a5dd8a1ca9b759e2091ca151bb339bc947c58
> Fix TrackerTest.testTryModify
> -
>
> Key: CASSANDRA-10151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10151
> Project: Cassandra
> Issue Type: Test
>Reporter: Stefania
>Assignee: Benedict
> Fix For: 2.2.1, 3.0 beta 2
>
>
> TrackerTest.testTryModify has started failing on trunk and 3.0:
> http://cassci.datastax.com/job/cassandra-3.0_testall/53/testReport/org.apache.cassandra.db.lifecycle/TrackerTest/testTryModify/
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.utils.concurrent.Ref$State.assertNotReleased(Ref.java:165)
> at org.apache.cassandra.utils.concurrent.Ref.ref(Ref.java:121)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$GlobalTidy.get(SSTableReader.java:2251)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.setup(SSTableReader.java:2099)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.setup(SSTableReader.java:2040)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.internalOpen(SSTableReader.java:575)
> at org.apache.cassandra.MockSchema.sstable(MockSchema.java:124)
> at org.apache.cassandra.MockSchema.sstable(MockSchema.java:79)
> at org.apache.cassandra.MockSchema.sstable(MockSchema.java:74)
> at
> org.apache.cassandra.db.lifecycle.TrackerTest.testTryModify(TrackerTest.java:86)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[6/6] cassandra git commit: Merge branch 'cassandra-3.0' into trunk
Merge branch 'cassandra-3.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/62f20789 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/62f20789 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/62f20789 Branch: refs/heads/trunk Commit: 62f20789b14b516e29790365a857087644e494d9 Parents: b1fb6b9 29d3d44 Author: Benedict Elliott Smith Authored: Sun Aug 23 11:51:17 2015 +0100 Committer: Benedict Elliott Smith Committed: Sun Aug 23 11:51:17 2015 +0100 -- test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) --
[4/6] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/29d3d44d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/29d3d44d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/29d3d44d Branch: refs/heads/trunk Commit: 29d3d44d2056e5ab84a2a5357cd782344683fc8e Parents: 342e33f 941a5dd Author: Benedict Elliott Smith Authored: Sun Aug 23 11:51:10 2015 +0100 Committer: Benedict Elliott Smith Committed: Sun Aug 23 11:51:10 2015 +0100 -- test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/29d3d44d/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java --
[2/6] cassandra git commit: 10151
10151
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/941a5dd8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/941a5dd8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/941a5dd8
Branch: refs/heads/cassandra-3.0
Commit: 941a5dd8a1ca9b759e2091ca151bb339bc947c58
Parents: a6d6540
Author: Benedict Elliott Smith
Authored: Fri Aug 21 10:50:36 2015 +0100
Committer: Benedict Elliott Smith
Committed: Sun Aug 23 11:50:53 2015 +0100
--
test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/941a5dd8/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
b/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
index dbd5287..93b88b4 100644
--- a/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
+++ b/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
@@ -82,7 +82,7 @@ public class TrackerTest
{
ColumnFamilyStore cfs = MockSchema.newCFS();
Tracker tracker = new Tracker(cfs, false);
-List readers = ImmutableList.of(MockSchema.sstable(0,
cfs), MockSchema.sstable(1, cfs), MockSchema.sstable(2, cfs));
+List readers = ImmutableList.of(MockSchema.sstable(0,
true, cfs), MockSchema.sstable(1, cfs), MockSchema.sstable(2, cfs));
tracker.addInitialSSTables(copyOf(readers));
Assert.assertNull(tracker.tryModify(ImmutableList.of(MockSchema.sstable(0,
cfs)), OperationType.COMPACTION));
try (LifecycleTransaction txn = tracker.tryModify(readers.get(0),
OperationType.COMPACTION);)
@@ -97,6 +97,7 @@ public class TrackerTest
Assert.assertNotNull(txn);
Assert.assertEquals(0, txn.originals().size());
}
+readers.get(0).selfRef().release();
}
@Test
[5/6] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/29d3d44d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/29d3d44d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/29d3d44d Branch: refs/heads/cassandra-3.0 Commit: 29d3d44d2056e5ab84a2a5357cd782344683fc8e Parents: 342e33f 941a5dd Author: Benedict Elliott Smith Authored: Sun Aug 23 11:51:10 2015 +0100 Committer: Benedict Elliott Smith Committed: Sun Aug 23 11:51:10 2015 +0100 -- test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/29d3d44d/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java --
[1/6] cassandra git commit: 10151
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 a6d654044 -> 941a5dd8a
refs/heads/cassandra-3.0 342e33ffb -> 29d3d44d2
refs/heads/trunk b1fb6b939 -> 62f20789b
10151
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/941a5dd8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/941a5dd8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/941a5dd8
Branch: refs/heads/cassandra-2.2
Commit: 941a5dd8a1ca9b759e2091ca151bb339bc947c58
Parents: a6d6540
Author: Benedict Elliott Smith
Authored: Fri Aug 21 10:50:36 2015 +0100
Committer: Benedict Elliott Smith
Committed: Sun Aug 23 11:50:53 2015 +0100
--
test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/941a5dd8/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
b/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
index dbd5287..93b88b4 100644
--- a/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
+++ b/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
@@ -82,7 +82,7 @@ public class TrackerTest
{
ColumnFamilyStore cfs = MockSchema.newCFS();
Tracker tracker = new Tracker(cfs, false);
-List readers = ImmutableList.of(MockSchema.sstable(0,
cfs), MockSchema.sstable(1, cfs), MockSchema.sstable(2, cfs));
+List readers = ImmutableList.of(MockSchema.sstable(0,
true, cfs), MockSchema.sstable(1, cfs), MockSchema.sstable(2, cfs));
tracker.addInitialSSTables(copyOf(readers));
Assert.assertNull(tracker.tryModify(ImmutableList.of(MockSchema.sstable(0,
cfs)), OperationType.COMPACTION));
try (LifecycleTransaction txn = tracker.tryModify(readers.get(0),
OperationType.COMPACTION);)
@@ -97,6 +97,7 @@ public class TrackerTest
Assert.assertNotNull(txn);
Assert.assertEquals(0, txn.originals().size());
}
+readers.get(0).selfRef().release();
}
@Test
[3/6] cassandra git commit: 10151
10151
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/941a5dd8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/941a5dd8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/941a5dd8
Branch: refs/heads/trunk
Commit: 941a5dd8a1ca9b759e2091ca151bb339bc947c58
Parents: a6d6540
Author: Benedict Elliott Smith
Authored: Fri Aug 21 10:50:36 2015 +0100
Committer: Benedict Elliott Smith
Committed: Sun Aug 23 11:50:53 2015 +0100
--
test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/941a5dd8/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
b/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
index dbd5287..93b88b4 100644
--- a/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
+++ b/test/unit/org/apache/cassandra/db/lifecycle/TrackerTest.java
@@ -82,7 +82,7 @@ public class TrackerTest
{
ColumnFamilyStore cfs = MockSchema.newCFS();
Tracker tracker = new Tracker(cfs, false);
-List readers = ImmutableList.of(MockSchema.sstable(0,
cfs), MockSchema.sstable(1, cfs), MockSchema.sstable(2, cfs));
+List readers = ImmutableList.of(MockSchema.sstable(0,
true, cfs), MockSchema.sstable(1, cfs), MockSchema.sstable(2, cfs));
tracker.addInitialSSTables(copyOf(readers));
Assert.assertNull(tracker.tryModify(ImmutableList.of(MockSchema.sstable(0,
cfs)), OperationType.COMPACTION));
try (LifecycleTransaction txn = tracker.tryModify(readers.get(0),
OperationType.COMPACTION);)
@@ -97,6 +97,7 @@ public class TrackerTest
Assert.assertNotNull(txn);
Assert.assertEquals(0, txn.originals().size());
}
+readers.get(0).selfRef().release();
}
@Test
[09/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/342e33ff Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/342e33ff Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/342e33ff Branch: refs/heads/trunk Commit: 342e33ffb14d10c29d040fa12dbaf48d6f16e410 Parents: 6e48573 a6d6540 Author: Benedict Elliott Smith Authored: Sun Aug 23 11:30:57 2015 +0100 Committer: Benedict Elliott Smith Committed: Sun Aug 23 11:30:57 2015 +0100 -- .../apache/cassandra/stress/StressProfile.java | 3 +- .../stress/generate/values/BigDecimals.java | 39 2 files changed, 41 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/342e33ff/tools/stress/src/org/apache/cassandra/stress/StressProfile.java --
[04/10] cassandra git commit: Support BigDecimal in cassanra-stress
Support BigDecimal in cassanra-stress
patch by Sebastian Estevez; reviewed by Benedict for CASSANDRA-10048
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5bee617f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5bee617f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5bee617f
Branch: refs/heads/trunk
Commit: 5bee617fea68e7b745c18d14b9b7076ab6fe219b
Parents: df9e798
Author: phact
Authored: Fri Aug 21 09:50:43 2015 +0100
Committer: Benedict Elliott Smith
Committed: Sun Aug 23 11:27:16 2015 +0100
--
.../apache/cassandra/stress/StressProfile.java | 3 +-
.../stress/generate/values/BigDecimals.java | 39
2 files changed, 41 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bee617f/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
index 49c4682..192b535 100644
--- a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
+++ b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
@@ -502,6 +502,7 @@ public class StressProfile implements Serializable
case BOOLEAN:
return new Booleans(name, config);
case DECIMAL:
+return new BigDecimals(name, config);
case DOUBLE:
return new Doubles(name, config);
case FLOAT:
@@ -523,7 +524,7 @@ public class StressProfile implements Serializable
case LIST:
return new Lists(name, getGenerator(name,
type.getTypeArguments().get(0), config), config);
default:
-throw new UnsupportedOperationException();
+throw new UnsupportedOperationException("Because of this
name: "+name+" if you removed it from the yaml and are still seeing this, make
sure to drop table");
}
}
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bee617f/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
--
diff --git
a/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
b/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
new file mode 100644
index 000..42758ed
--- /dev/null
+++
b/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
@@ -0,0 +1,39 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+package org.apache.cassandra.stress.generate.values;
+
+import org.apache.cassandra.db.marshal.DecimalType;
+
+import java.math.BigDecimal;
+
+public class BigDecimals extends Generator
+{
+public BigDecimals(String name, GeneratorConfig config)
+{
+super(DecimalType.instance, config, name, BigDecimal.class);
+}
+
+@Override
+public BigDecimal generate()
+{
+return BigDecimal.valueOf(identityDistribution.next());
+}
+}
[01/10] cassandra git commit: Support BigDecimal in cassanra-stress
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 df9e798de -> 5bee617fe
refs/heads/cassandra-2.2 dd19a7fcf -> a6d654044
refs/heads/cassandra-3.0 6e48573be -> 342e33ffb
refs/heads/trunk 7aa49acf6 -> b1fb6b939
Support BigDecimal in cassanra-stress
patch by Sebastian Estevez; reviewed by Benedict for CASSANDRA-10048
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5bee617f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5bee617f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5bee617f
Branch: refs/heads/cassandra-2.1
Commit: 5bee617fea68e7b745c18d14b9b7076ab6fe219b
Parents: df9e798
Author: phact
Authored: Fri Aug 21 09:50:43 2015 +0100
Committer: Benedict Elliott Smith
Committed: Sun Aug 23 11:27:16 2015 +0100
--
.../apache/cassandra/stress/StressProfile.java | 3 +-
.../stress/generate/values/BigDecimals.java | 39
2 files changed, 41 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bee617f/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
index 49c4682..192b535 100644
--- a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
+++ b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
@@ -502,6 +502,7 @@ public class StressProfile implements Serializable
case BOOLEAN:
return new Booleans(name, config);
case DECIMAL:
+return new BigDecimals(name, config);
case DOUBLE:
return new Doubles(name, config);
case FLOAT:
@@ -523,7 +524,7 @@ public class StressProfile implements Serializable
case LIST:
return new Lists(name, getGenerator(name,
type.getTypeArguments().get(0), config), config);
default:
-throw new UnsupportedOperationException();
+throw new UnsupportedOperationException("Because of this
name: "+name+" if you removed it from the yaml and are still seeing this, make
sure to drop table");
}
}
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bee617f/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
--
diff --git
a/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
b/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
new file mode 100644
index 000..42758ed
--- /dev/null
+++
b/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
@@ -0,0 +1,39 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+package org.apache.cassandra.stress.generate.values;
+
+import org.apache.cassandra.db.marshal.DecimalType;
+
+import java.math.BigDecimal;
+
+public class BigDecimals extends Generator
+{
+public BigDecimals(String name, GeneratorConfig config)
+{
+super(DecimalType.instance, config, name, BigDecimal.class);
+}
+
+@Override
+public BigDecimal generate()
+{
+return BigDecimal.valueOf(identityDistribution.next());
+}
+}
[02/10] cassandra git commit: Support BigDecimal in cassanra-stress
Support BigDecimal in cassanra-stress
patch by Sebastian Estevez; reviewed by Benedict for CASSANDRA-10048
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5bee617f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5bee617f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5bee617f
Branch: refs/heads/cassandra-2.2
Commit: 5bee617fea68e7b745c18d14b9b7076ab6fe219b
Parents: df9e798
Author: phact
Authored: Fri Aug 21 09:50:43 2015 +0100
Committer: Benedict Elliott Smith
Committed: Sun Aug 23 11:27:16 2015 +0100
--
.../apache/cassandra/stress/StressProfile.java | 3 +-
.../stress/generate/values/BigDecimals.java | 39
2 files changed, 41 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bee617f/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
index 49c4682..192b535 100644
--- a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
+++ b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
@@ -502,6 +502,7 @@ public class StressProfile implements Serializable
case BOOLEAN:
return new Booleans(name, config);
case DECIMAL:
+return new BigDecimals(name, config);
case DOUBLE:
return new Doubles(name, config);
case FLOAT:
@@ -523,7 +524,7 @@ public class StressProfile implements Serializable
case LIST:
return new Lists(name, getGenerator(name,
type.getTypeArguments().get(0), config), config);
default:
-throw new UnsupportedOperationException();
+throw new UnsupportedOperationException("Because of this
name: "+name+" if you removed it from the yaml and are still seeing this, make
sure to drop table");
}
}
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bee617f/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
--
diff --git
a/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
b/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
new file mode 100644
index 000..42758ed
--- /dev/null
+++
b/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
@@ -0,0 +1,39 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+package org.apache.cassandra.stress.generate.values;
+
+import org.apache.cassandra.db.marshal.DecimalType;
+
+import java.math.BigDecimal;
+
+public class BigDecimals extends Generator
+{
+public BigDecimals(String name, GeneratorConfig config)
+{
+super(DecimalType.instance, config, name, BigDecimal.class);
+}
+
+@Override
+public BigDecimal generate()
+{
+return BigDecimal.valueOf(identityDistribution.next());
+}
+}
[08/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/342e33ff Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/342e33ff Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/342e33ff Branch: refs/heads/cassandra-3.0 Commit: 342e33ffb14d10c29d040fa12dbaf48d6f16e410 Parents: 6e48573 a6d6540 Author: Benedict Elliott Smith Authored: Sun Aug 23 11:30:57 2015 +0100 Committer: Benedict Elliott Smith Committed: Sun Aug 23 11:30:57 2015 +0100 -- .../apache/cassandra/stress/StressProfile.java | 3 +- .../stress/generate/values/BigDecimals.java | 39 2 files changed, 41 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/342e33ff/tools/stress/src/org/apache/cassandra/stress/StressProfile.java --
[03/10] cassandra git commit: Support BigDecimal in cassanra-stress
Support BigDecimal in cassanra-stress
patch by Sebastian Estevez; reviewed by Benedict for CASSANDRA-10048
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5bee617f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5bee617f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5bee617f
Branch: refs/heads/cassandra-3.0
Commit: 5bee617fea68e7b745c18d14b9b7076ab6fe219b
Parents: df9e798
Author: phact
Authored: Fri Aug 21 09:50:43 2015 +0100
Committer: Benedict Elliott Smith
Committed: Sun Aug 23 11:27:16 2015 +0100
--
.../apache/cassandra/stress/StressProfile.java | 3 +-
.../stress/generate/values/BigDecimals.java | 39
2 files changed, 41 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bee617f/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
index 49c4682..192b535 100644
--- a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
+++ b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
@@ -502,6 +502,7 @@ public class StressProfile implements Serializable
case BOOLEAN:
return new Booleans(name, config);
case DECIMAL:
+return new BigDecimals(name, config);
case DOUBLE:
return new Doubles(name, config);
case FLOAT:
@@ -523,7 +524,7 @@ public class StressProfile implements Serializable
case LIST:
return new Lists(name, getGenerator(name,
type.getTypeArguments().get(0), config), config);
default:
-throw new UnsupportedOperationException();
+throw new UnsupportedOperationException("Because of this
name: "+name+" if you removed it from the yaml and are still seeing this, make
sure to drop table");
}
}
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5bee617f/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
--
diff --git
a/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
b/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
new file mode 100644
index 000..42758ed
--- /dev/null
+++
b/tools/stress/src/org/apache/cassandra/stress/generate/values/BigDecimals.java
@@ -0,0 +1,39 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+package org.apache.cassandra.stress.generate.values;
+
+import org.apache.cassandra.db.marshal.DecimalType;
+
+import java.math.BigDecimal;
+
+public class BigDecimals extends Generator
+{
+public BigDecimals(String name, GeneratorConfig config)
+{
+super(DecimalType.instance, config, name, BigDecimal.class);
+}
+
+@Override
+public BigDecimal generate()
+{
+return BigDecimal.valueOf(identityDistribution.next());
+}
+}
[05/10] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a6d65404 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a6d65404 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a6d65404 Branch: refs/heads/cassandra-3.0 Commit: a6d65404467117c5b69b982c3576afb7e583db55 Parents: dd19a7f 5bee617 Author: Benedict Elliott Smith Authored: Sun Aug 23 11:30:43 2015 +0100 Committer: Benedict Elliott Smith Committed: Sun Aug 23 11:30:43 2015 +0100 -- .../apache/cassandra/stress/StressProfile.java | 3 +- .../stress/generate/values/BigDecimals.java | 39 2 files changed, 41 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/a6d65404/tools/stress/src/org/apache/cassandra/stress/StressProfile.java --
[06/10] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a6d65404 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a6d65404 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a6d65404 Branch: refs/heads/cassandra-2.2 Commit: a6d65404467117c5b69b982c3576afb7e583db55 Parents: dd19a7f 5bee617 Author: Benedict Elliott Smith Authored: Sun Aug 23 11:30:43 2015 +0100 Committer: Benedict Elliott Smith Committed: Sun Aug 23 11:30:43 2015 +0100 -- .../apache/cassandra/stress/StressProfile.java | 3 +- .../stress/generate/values/BigDecimals.java | 39 2 files changed, 41 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/a6d65404/tools/stress/src/org/apache/cassandra/stress/StressProfile.java --
[07/10] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a6d65404 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a6d65404 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a6d65404 Branch: refs/heads/trunk Commit: a6d65404467117c5b69b982c3576afb7e583db55 Parents: dd19a7f 5bee617 Author: Benedict Elliott Smith Authored: Sun Aug 23 11:30:43 2015 +0100 Committer: Benedict Elliott Smith Committed: Sun Aug 23 11:30:43 2015 +0100 -- .../apache/cassandra/stress/StressProfile.java | 3 +- .../stress/generate/values/BigDecimals.java | 39 2 files changed, 41 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/a6d65404/tools/stress/src/org/apache/cassandra/stress/StressProfile.java --
[10/10] cassandra git commit: Merge branch 'cassandra-3.0' into trunk
Merge branch 'cassandra-3.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b1fb6b93 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b1fb6b93 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b1fb6b93 Branch: refs/heads/trunk Commit: b1fb6b939ac892432ace5e7f5c7597bd573b669c Parents: 7aa49ac 342e33f Author: Benedict Elliott Smith Authored: Sun Aug 23 11:31:03 2015 +0100 Committer: Benedict Elliott Smith Committed: Sun Aug 23 11:31:03 2015 +0100 -- .../apache/cassandra/stress/StressProfile.java | 3 +- .../stress/generate/values/BigDecimals.java | 39 2 files changed, 41 insertions(+), 1 deletion(-) --
[jira] [Comment Edited] (CASSANDRA-10155) 2i key cache load fails
[
https://issues.apache.org/jira/browse/CASSANDRA-10155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708338#comment-14708338
]
Robert Stupp edited comment on CASSANDRA-10155 at 8/23/15 10:19 AM:
Right, cfName is different for base table and 2i.
But the only parameter to
{{AutoSavingCache#getCacheDataPath}}/{{getCacheCrcPath}} is cfId - so
{{DatabaseDescriptor.getSerializedCachePath}} can only return the file name for
the base table. I.e. a {{AutoSavingCache#loadSaved}} will access the same file
for base table and 2i {{ColumnFamilyStore}}s.
EDIT: I'm in favor of making cfId (or some other identifier) unique
(CASSANDRA-10028) since there's a lot of special casing for 2i names.
was (Author: snazy):
Right, cfName is different for base table and 2i.
But the only parameter to
{{AutoSavingCache#getCacheDataPath}}/{{getCacheCrcPath}} is cfId - so
{{DatabaseDescriptor.getSerializedCachePath}} can only return the file name for
the base table. I.e. a {{AutoSavingCache#loadSaved}} will access the same file
for base table and 2i {{ColumnFamilyStore}}s.
> 2i key cache load fails
> ---
>
> Key: CASSANDRA-10155
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10155
> Project: Cassandra
> Issue Type: Bug
>Reporter: Robert Stupp
> Fix For: 3.0.0 rc1
>
>
> CASSANDRA-9265 changed how key cache content is serialized to disk. It uses
> {{UUID cfId}} to generate the file path for each {{ColumnFamilyStore}}.
> Since {{cfId}} of a secondary index is the same as for the base table, the
> key-cache files for 2i's and the base are the same. This will/may lead to
> deserialization failures on restart for tables with at least one 2i.
> /cc [~aweisberg] [~danchia]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10155) 2i key cache load fails
[
https://issues.apache.org/jira/browse/CASSANDRA-10155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708338#comment-14708338
]
Robert Stupp commented on CASSANDRA-10155:
--
Right, cfName is different for base table and 2i.
But the only parameter to
{{AutoSavingCache#getCacheDataPath}}/{{getCacheCrcPath}} is cfId - so
{{DatabaseDescriptor.getSerializedCachePath}} can only return the file name for
the base table. I.e. a {{AutoSavingCache#loadSaved}} will access the same file
for base table and 2i {{ColumnFamilyStore}}s.
> 2i key cache load fails
> ---
>
> Key: CASSANDRA-10155
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10155
> Project: Cassandra
> Issue Type: Bug
>Reporter: Robert Stupp
> Fix For: 3.0.0 rc1
>
>
> CASSANDRA-9265 changed how key cache content is serialized to disk. It uses
> {{UUID cfId}} to generate the file path for each {{ColumnFamilyStore}}.
> Since {{cfId}} of a secondary index is the same as for the base table, the
> key-cache files for 2i's and the base are the same. This will/may lead to
> deserialization failures on restart for tables with at least one 2i.
> /cc [~aweisberg] [~danchia]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9459) SecondaryIndex API redesign
[
https://issues.apache.org/jira/browse/CASSANDRA-9459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708330#comment-14708330
]
Benedict commented on CASSANDRA-9459:
-
A nit question relating to this ticket: In each of the main Transaction
implementations we seem to be making unnecessary use of streams. These are a
"hot" code paths on systems using indexes, and I'm not convinced we should be
incurring the GC burden, especially when clarity isn't meaningfully improved.
Compare
{code}
try (OpOrder.Group opGroup = Keyspace.writeOrder.start())
{
Index.Indexer[] indexers = Arrays.stream(indexes)
.map(i -> i.indexerFor(key,
nowInSec, opGroup, Type.CLEANUP))
.toArray(Index.Indexer[]::new);
Arrays.stream(indexers).forEach(Index.Indexer::begin);
if (partitionDelete != null)
Arrays.stream(indexers).forEach(indexer ->
indexer.partitionDelete(partitionDelete));
if (row != null)
Arrays.stream(indexers).forEach(indexer ->
indexer.removeRow(row));
Arrays.stream(indexers).forEach(Index.Indexer::finish);
}
{code}
with
{code}
try (OpOrder.Group opGroup = Keyspace.writeOrder.start())
{
for (Index index : indexes)
{
Index.Indexer indexer = index.indexerFor(key, nowInSec,
opGroup, Type.CLEANUP);
indexer.begin();
if (partitionDelete != null)
indexer.partitionDelete(partitionDelete);
if (row != null)
indexer.removeRow(row);
indexer.finish();
}
}
{code}
I'm pretty convinced the latter is clearer, and in the former I count at least
8 unnecessary allocations for the first stream, and 4 for each of the rest. A
few of these allocations are >= 64Kb, and I estimate total allocation per row
on the order of (but probably a little less than) 1Kb. Conversely, the
old-style code performs no allocations besides that of the {{Indexer}}.
> SecondaryIndex API redesign
> ---
>
> Key: CASSANDRA-9459
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9459
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
> Fix For: 3.0 beta 1
>
>
> For some time now the index subsystem has been a pain point and in large part
> this is due to the way that the APIs and principal classes have grown
> organically over the years. It would be a good idea to conduct a wholesale
> review of the area and see if we can come up with something a bit more
> coherent.
> A few starting points:
> * There's a lot in AbstractPerColumnSecondaryIndex & its subclasses which
> could be pulled up into SecondaryIndexSearcher (note that to an extent, this
> is done in CASSANDRA-8099).
> * SecondayIndexManager is overly complex and several of its functions should
> be simplified/re-examined. The handling of which columns are indexed and
> index selection on both the read and write paths are somewhat dense and
> unintuitive.
> * The SecondaryIndex class hierarchy is rather convoluted and could use some
> serious rework.
> There are a number of outstanding tickets which we should be able to roll
> into this higher level one as subtasks (but I'll defer doing that until
> getting into the details of the redesign):
> * CASSANDRA-7771
> * CASSANDRA-8103
> * CASSANDRA-9041
> * CASSANDRA-4458
> * CASSANDRA-8505
> Whilst they're not hard dependencies, I propose that this be done on top of
> both CASSANDRA-8099 and CASSANDRA-6717. The former largely because the
> storage engine changes may facilitate a friendlier index API, but also
> because of the changes to SIS mentioned above. As for 6717, the changes to
> schema tables there will help facilitate CASSANDRA-7771.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
