[jira] [Commented] (CASSANDRA-9893) Fix upgrade tests from #9704 that are still failing
[
https://issues.apache.org/jira/browse/CASSANDRA-9893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728454#comment-14728454
]
Blake Eggleston commented on CASSANDRA-9893:
Here's an updated branch: https://github.com/bdeggleston/cassandra/tree/9893-4
(and the dtest updates:
https://github.com/bdeggleston/cassandra-dtest/tree/8099-backwards-compat)
I reverted the change to the DataRange.toPaging call, and
{{composite_index_collections_test}} is now consistently passing for me too, so
I'll just leave it alone. I also took out the superfluous {{columns}} in
{{UnfilteredRowIterators.noRowIterator}}, and made the ColumnFilter.Builder
constructor private.
{{TestCQL.static_columns_with_distinct_test}} will occasionally fail because of
CASSANDRA-9857, but it mostly fails because of a 2.1 bug, which I've opened
CASSANDRA-10254 for.
> Fix upgrade tests from #9704 that are still failing
> ---
>
> Key: CASSANDRA-9893
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9893
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Blake Eggleston
> Fix For: 3.0 beta 2
>
>
> The first things to do on this ticket would be to commit Tyler's branch
> (https://github.com/thobbs/cassandra-dtest/tree/8099-backwards-compat) to the
> dtests so cassci run them. I've had to do a few minor modifications to have
> them run locally so someone which access to cassci should do it and make sure
> it runs properly.
> Once we have that, we should fix any test that isn't passing. I've ran the
> tests locally and I had 8 failures. for 2 of them, it sounds plausible that
> they'll get fixed by the patch of CASSANDRA-9775, though that's just a guess.
> The rest where test that timeouted without a particular error in the log,
> and running some of them individually, they passed. So we'll have to see if
> it's just my machine being overly slow when running them all.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10254) invalid client PagingState makes it past validation
Blake Eggleston created CASSANDRA-10254:
---
Summary: invalid client PagingState makes it past validation
Key: CASSANDRA-10254
URL: https://issues.apache.org/jira/browse/CASSANDRA-10254
Project: Cassandra
Issue Type: Bug
Reporter: Blake Eggleston
When running the upgrade dtest {{TestCQL.static_columns_with_distinct_test}},
the client appears to be passing in an invalid paging state (no cell name).
Instead of catching the problem in validation and returning an error to the
client, the select statement tries to run it, causing this exception:
{noformat}
ERROR QueryMessage.java:135 - Unexpected error during query
java.lang.ClassCastException:
org.apache.cassandra.db.composites.Composites$EmptyComposite cannot be cast to
org.apache.cassandra.db.composites.CellName
at
org.apache.cassandra.service.pager.SliceQueryPager.(SliceQueryPager.java:64)
~[main/:na]
at
org.apache.cassandra.service.pager.MultiPartitionPager.makePager(MultiPartitionPager.java:93)
~[main/:na]
at
org.apache.cassandra.service.pager.MultiPartitionPager.(MultiPartitionPager.java:75)
~[main/:na]
at
org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:102)
~[main/:na]
at
org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:126)
~[main/:na]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:228)
~[main/:na]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:67)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:238)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:260)
~[main/:na]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:122)
~[main/:na]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
[main/:na]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
[main/:na]
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_40]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
[main/:na]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[main/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
{noformat}
Running {code} UPGRADE_MODE=none nosetests
upgrade_tests/cql_tests.py:TestCQL.static_columns_with_distinct_test{code}
should cause the error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9304) COPY TO improvements
[
https://issues.apache.org/jira/browse/CASSANDRA-9304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728422#comment-14728422
]
Stefania commented on CASSANDRA-9304:
-
The {{RateLimiter}} still seems a bit off. It looked kind of wrong before as
you pointed out. It's not terribly important but I think this line
{{self.current_rate = (self.current_rate + new_rate) / 2.0}} was meant as an
average between the current rate and the new one. So the first time, when
{{current_rate}} is zero, it should not divide by 2 or else we report half the
rate. Secondly, when we calculate the new rate as {{n / difference}}, we may
miss records because {{n}} is the number of records passed to every call whilst
{{difference}} is the time elapsed since the last time we logged. I wouldn't
calculate the rate every time either, but only when logging it. If
{{current_record}} cannot be reset to zero after logging it (maybe this was the
initial intention of the existing code), then we need a new counter which gives
the number of records accumulated between each log point.
It's great we now test for all partitioners but we are only exporting 1 record
in {{test_all_datatypes_round_trip}} so a better candidate would have been
{{test_round_trip}}, where at least we export 10K records. So would you mind
adapting {{test_round_trip}} to also run with every partitioner?
In fact it would be good to have a bulk round-trip test as well (only for the
default partitioner) where we export and import 1M records? We would need to
use cassandra stress to write the records. Then we just check the counts. This
is just a suggestion.
I had problems when running the cqlsh_tests locally:
{code}
nosetests -s cqlsh_tests
{code}
{code}
==
ERROR: test_source_glass (cqlsh_tests.cqlsh_tests.TestCqlsh)
--
Traceback (most recent call last):
File "/home/stefania/git/cstar/cassandra-dtest/tools.py", line 252, in wrapped
f(obj)
File "/home/stefania/git/cstar/cassandra-dtest/cqlsh_tests/cqlsh_tests.py",
line 341, in test_source_glass
self.verify_glass(node1)
File "/home/stefania/git/cstar/cassandra-dtest/cqlsh_tests/cqlsh_tests.py",
line 102, in verify_glass
'I can eat glass and it does not hurt me': 'Is'
File "/home/stefania/git/cstar/cassandra-dtest/cqlsh_tests/cqlsh_tests.py",
line 95, in verify_varcharmap
got = {k.encode("utf-8"): v for k, v in rows[0][0].iteritems()}
IndexError: list index out of range
>> begin captured logging <<
dtest: DEBUG: cluster ccm directory: /tmp/dtest-Ldxvcq
- >> end captured logging << -
==
FAIL: test_all_datatypes_read (cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest)
--
Traceback (most recent call last):
File
"/home/stefania/git/cstar/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py",
line 690, in test_all_datatypes_read
self.assertCsvResultEqual(self.tempfile.name, results)
File
"/home/stefania/git/cstar/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py",
line 153, in assertCsvResultEqual
raise e
AssertionError: Element counts were not equal:
First has 1, Second has 0: ['ascii', '1099511627776', '0xbeef', 'True',
'3.140124344978758017532527446746826171875', '2.444', '1.1',
'127.0.0.1', '25',
'\xe3\x83\xbd(\xc2\xb4\xe3\x83\xbc\xef\xbd\x80)\xe3\x83\x8e', '2005-07-14
12:30:00', '2b4e32ce-51de-11e5-85b7-0050b67e8b2f',
'830bc4cd-a790-4ac2-85f9-648b0a71306b', 'asdf', '36893488147419103232']
First has 0, Second has 1: ['ascii', '1099511627776', '0xbeef', 'True',
'3.140124344978758017532527446746826171875', '2.444', '1.1',
'127.0.0.1', '25',
'\xe3\x83\xbd(\xc2\xb4\xe3\x83\xbc\xef\xbd\x80)\xe3\x83\x8e', '2005-07-14
04:30:00', '2b4e32ce-51de-11e5-85b7-0050b67e8b2f',
'830bc4cd-a790-4ac2-85f9-648b0a71306b', 'asdf', '36893488147419103232']
>> begin captured logging <<
dtest: DEBUG: cluster ccm directory: /tmp/dtest-cSohP9
dtest: DEBUG: Importing from csv file: /tmp/tmpJgdPJc
dtest: WARNING: Mismatch at index: 10
dtest: WARNING: Value in csv: 2005-07-14 12:30:00
dtest: WARNING: Value in result: 2005-07-14 04:30:00
- >> end captured logging << -
--
Ran 69 tests in 1161.775s
FAILED (SKIP=5, errors=1, failures=1)
{code}
I scheduled new CI jobs on my view:
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9304-testall/
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9304-dtest/
Let's see if they too report the problems I had locally.
> COPY TO
[jira] [Commented] (CASSANDRA-10195) TWCS experiments and improvement proposals
[ https://issues.apache.org/jira/browse/CASSANDRA-10195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728408#comment-14728408 ] Jonathan Ellis commented on CASSANDRA-10195: Very promising for TWCS! > TWCS experiments and improvement proposals > -- > > Key: CASSANDRA-10195 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10195 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Antti Nissinen > Fix For: 2.1.x, 2.2.x > > Attachments: 20150814_1027_compaction_hierarchy.txt, > node0_20150727_1250_time_graph.txt, node0_20150810_1017_time_graph.txt, > node0_20150812_1531_time_graph.txt, node0_20150813_0835_time_graph.txt, > node0_20150814_1054_time_graph.txt, node1_20150727_1250_time_graph.txt, > node1_20150810_1017_time_graph.txt, node1_20150812_1531_time_graph.txt, > node1_20150813_0835_time_graph.txt, node1_20150814_1054_time_graph.txt, > node2_20150727_1250_time_graph.txt, node2_20150810_1017_time_graph.txt, > node2_20150812_1531_time_graph.txt, node2_20150813_0835_time_graph.txt, > node2_20150814_1054_time_graph.txt, sstable_count_figure1.png, > sstable_count_figure2.png > > > This JIRA item describes experiments with DateTieredCompactionStartegy (DTCS) > and TimeWindowCompactionStrategy (TWCS) and proposes modifications to the > TWCS. In a test system several crashes were caused intentionally (and > unintentionally) and repair operations were executed leading to flood of > small SSTables. Target was to be able compact those files are release disk > space reserved by duplicate data. Setup is following: > - Three nodes > - DateTieredCompactionStrategy, max_sstable_age_days = 5 > Cassandra 2.1.2 > The setup and data format has been documented in detailed here > https://issues.apache.org/jira/browse/CASSANDRA-9644. > The test was started by dumping few days worth of data to the database for > 100 000 signals. Time graphs of SStables from different nodes indicates that > the DTCS has been working as expected and SStables are nicely ordered in time > wise. > See files: > node0_20150727_1250_time_graph.txt > node1_20150727_1250_time_graph.txt > node2_20150727_1250_time_graph.txt > Status=Up/Down > |/ State=Normal/Leaving/Joining/Moving > -- AddressLoad Tokens OwnsHost ID > Rack > UN 139.66.43.170 188.87 GB 256 ? > dfc29863-c935-4909-9d7f-c59a47eda03d rack1 > UN 139.66.43.169 198.37 GB 256 ? > 12e7628b-7f05-48f6-b7e4-35a82010021a rack1 > UN 139.66.43.168 191.88 GB 256 ? > 26088392-f803-4d59-9073-c75f857fb332 rack1 > All nodes crashed due to power failure (know beforehand) and repair > operations were started for each node one at the time. Below is the behavior > of SSTable count on different nodes. New data was dumped simultaneously with > repair operation. > SEE FIGURE: sstable_count_figure1.png > Vertical lines indicate following events. > 1) Cluster was down due to power shutdown and was restarted. At the first > vertical line the repair operation (nodetool repair -pr) was started for the > first node > 2) Repair for the second repair operation was started after the first node > was successfully repaired. > 3) Repair for the third repair operation was started > 4) Third repair operation was finished > 5) One of the nodes crashed (unknown reason in OS level) > 6) Repair operation (nodetool repair -pr) was started for the first node > 7) Repair operation for the second node was started > 8) Repair operation for the third node was started > 9) Repair operations finished > These repair operations are leading to huge amount of small SSTables covering > the whole time span of the data. The compaction horizon of DTCS was limited > to 5 days (max_sstable_age_days) due to the size of the SStables on the disc. > Therefore, small SStables won't be compacted. Below are the time graphs from > SSTables after the second round of repairs. > Status=Up/Down > |/ State=Normal/Leaving/Joining/Moving > -- AddressLoad Tokens OwnsHost ID > Rack > UN xx.xx.xx.170 663.61 GB 256 ? > dfc29863-c935-4909-9d7f-c59a47eda03d rack1 > UN xx.xx.xx.169 763.52 GB 256 ? > 12e7628b-7f05-48f6-b7e4-35a82010021a rack1 > UN xx.xx.xx.168 651.59 GB 256 ? > 26088392-f803-4d59-9073-c75f857fb332 rack1 > See files: > node0_20150810_1017_time_graph.txt > node1_20150810_1017_time_graph.txt > node2_20150810_1017_time_graph.txt > To get rid of the SStables the TimeWindowCompactionStrategy was taken into > use. Window size was set to 5 days. Cassandra version was updated to 2.1.8. > Below figure shows the behavior of SStable count. TWCS was taken into use > 10.8.2015 at 13:10. The
[jira] [Commented] (CASSANDRA-9664) Allow MV's select statements to be more complex
[
https://issues.apache.org/jira/browse/CASSANDRA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728376#comment-14728376
]
Tyler Hobbs commented on CASSANDRA-9664:
I have a [branch|https://github.com/thobbs/cassandra/tree/CASSANDRA-9664] with
an implementation that's ready for review.
As noted in my previous comment, this only allows restricting columns that are
in the base table's primary key. When handling individual mutations this
filtering can be performed prior to the read-before-write, so write-path work
for unselected rows is minimal. When performing the initial build of the MV,
we don't yet take full advantage of the select statement's restrictions. I
would like to improve single-partition builds and builds that can be assisted
by secondary indexes, but if it's okay with the reviewer, that feels best left
to another ticket.
One of the trickiest parts of this ticket was representing the WHERE clause
restrictions in the MV's schema. This needs to support multi-column relations,
single-column relations, and any operator (including IN, which expects multiple
values). The schema I settled on was this:
{noformat}
where_clause frozen, int, list>>>
{noformat}
Roughly speaking, this is a list of tuples, but with
lists for ids and values to support multi-column relations. I know the nesting
is a little crazy there, but that allows us to represent everything that we
need. I also considered storing a single string of the WHERE clause, but this
presents difficulties when loading the MV from the schema. In particular, we
don't have a good way to use the parser only for the {{whereClause}} rule. If
somebody has a better idea, I'm open to suggestions.
Last, this implementation is nearly restricted to what normal SELECT statements
allow. In some cases those restrictions don't make much sense for MVs, where
we don't need to execute an efficient query. For the most part I haven't
changed anything here. The one modification I did make is to allow filtering
on clustering columns when the SELECT is being built for use by an MV. As an
example, if the base primary key is (a, b, c), the MV can do "WHERE c = 0"
without restricting column b. Normally this is only allowed if column c is
indexed, but for MV purposes, this is efficient to filter.
Pending CI tests:
* [3.0
testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-9664-testall/]
* [3.0
dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-9664-dtest/]
After the first round of review, I'll also run CI tests on trunk.
> Allow MV's select statements to be more complex
> ---
>
> Key: CASSANDRA-9664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9664
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Carl Yeksigian
> Fix For: 3.x
>
>
> [Materialized Views|https://issues.apache.org/jira/browse/CASSANDRA-6477] add
> support for a syntax which includes a {{SELECT}} statement, but only allows
> selection of direct columns, and does not allow any filtering to take place.
> We should add support to the MV {{SELECT}} statement to bring better parity
> with the normal CQL {{SELECT}} statement, specifically simple functions in
> the selected columns, as well as specifying a {{WHERE}} clause.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10222) Periodically attempt to delete failed snapshot deletions on Windows
[ https://issues.apache.org/jira/browse/CASSANDRA-10222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728332#comment-14728332 ] Stefania commented on CASSANDRA-10222: -- LGTM, thanks. +1 once the CI jobs you've set-up are OK. No need to repeat dtests on 3.0. > Periodically attempt to delete failed snapshot deletions on Windows > --- > > Key: CASSANDRA-10222 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10222 > Project: Cassandra > Issue Type: Improvement >Reporter: Joshua McKenzie >Assignee: Joshua McKenzie > Labels: Windows > Fix For: 2.2.2 > > > The changes in CASSANDRA-9658 leave us in a position where a node on Windows > will have to be restarted to clear out snapshots that cannot be deleted at > request time due to sstables still being mapped, thus preventing deletions of > hard links. A simple periodic task to categorize failed snapshot deletions > and retry them would help prevent node disk utilization from growing > unbounded by snapshots as compaction will eventually make these snapshot > files deletable. > Given that hard links to files in NTFS don't take up any extra space on disk > so long as the original file still exists, the only limitation for users from > this approach will be the inability to 'move' a snapshot file to another > drive share. They will be copyable, however, so it's a minor platform > difference. > This goes directly against the goals of CASSANDRA-8271 and will likely be > built on top of that code. Until such time as we get buffered performance > in-line with memory-mapped, this is an interim necessity for production > roll-outs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8630) Faster sequential IO (on compaction, streaming, etc)
[ https://issues.apache.org/jira/browse/CASSANDRA-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728305#comment-14728305 ] Stefania commented on CASSANDRA-8630: - Looking at the first read operation, 8630 is top in the first two runs and second in the third run. Anyway, good enough for sure. Windows utests are also OK, 2 failures same as unpatched branch. dtests still running. Pushed one more rebased version to run CI one more time on Linux, a failing unit test cropped up but seems unrelated and it passed locally. > Faster sequential IO (on compaction, streaming, etc) > > > Key: CASSANDRA-8630 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8630 > Project: Cassandra > Issue Type: Improvement > Components: Core, Tools >Reporter: Oleg Anastasyev >Assignee: Stefania > Labels: compaction, performance > Fix For: 3.x > > Attachments: 8630-FasterSequencialReadsAndWrites.txt, cpu_load.png, > flight_recorder_001_files.tar.gz, flight_recorder_002_files.tar.gz, > mmaped_uncomp_hotspot.png > > > When node is doing a lot of sequencial IO (streaming, compacting, etc) a lot > of CPU is lost in calls to RAF's int read() and DataOutputStream's write(int). > This is because default implementations of readShort,readLong, etc as well as > their matching write* are implemented with numerous calls of byte by byte > read and write. > This makes a lot of syscalls as well. > A quick microbench shows than just reimplementation of these methods in > either way gives 8x speed increase. > A patch attached implements RandomAccessReader.read and > SequencialWriter.write methods in more efficient way. > I also eliminated some extra byte copies in CompositeType.split and > ColumnNameHelper.maxComponents, which were on my profiler's hotspot method > list during tests. > A stress tests on my laptop show that this patch makes compaction 25-30% > faster on uncompressed sstables and 15% faster for compressed ones. > A deployment to production shows much less CPU load for compaction. > (I attached a cpu load graph from one of our production, orange is niced CPU > load - i.e. compaction; yellow is user - i.e. not compaction related tasks) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9857) Deal with backward compatibilty issue of broken AbstractBounds serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-9857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728306#comment-14728306
]
Blake Eggleston commented on CASSANDRA-9857:
Implementation looks good to me, and the failing upgrade tests are now passing
(with the expected exception of static_columns_with_distinct_test) . Could you
just add a unit test?
> Deal with backward compatibilty issue of broken AbstractBounds serialization
>
>
> Key: CASSANDRA-9857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9857
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0 beta 2
>
>
> This ticket is related to CASSANDRA-9856 and CASSANDRA-9775. Even if the
> broken/incomplete serialization of {{AbstractBounds}} is not a problem per-se
> for pre-3.0 versions, it's still a problem for trunk and even though it's
> fixed by CASSANDRA-9775 for 3.0 nodes, it might be a problem for 3.0 nodes
> talking to older nodes.
> As the paging tests where those that exposed the problem (on trunk) in the
> first place, it would be nice to modify said paging tests to work on mixed
> version clustering so we can valid if it is a problem. If it is, then we'll
> probably have to add redundant checks on trunk so they ignore anything the
> 3.0 node sends incorrectly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9830) Option to disable bloom filter in highest level of LCS sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-9830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728275#comment-14728275 ] Paulo Motta commented on CASSANDRA-9830: [~JoshuaMcKenzie] ping :-) (just in case you missed this one, please ignore if it's already in your to-review list - given this is not priority) > Option to disable bloom filter in highest level of LCS sstables > --- > > Key: CASSANDRA-9830 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9830 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: Paulo Motta >Priority: Minor > Labels: performance > Fix For: 3.x > > > We expect about 90% of data to be in the highest level of LCS in a fully > populated series. (See also CASSANDRA-9829.) > Thus if the user is primarily asking for data (partitions) that has actually > been inserted, the bloom filter on the highest level only helps reject > sstables about 10% of the time. > We should add an option that suppresses bloom filter creation on top-level > sstables. This will dramatically reduce memory usage for LCS and may even > improve performance as we no longer check a low-value filter. > (This is also an idea from RocksDB.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8741) Running a drain before a decommission apparently the wrong thing to do
[
https://issues.apache.org/jira/browse/CASSANDRA-8741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728259#comment-14728259
]
Stefania commented on CASSANDRA-8741:
-
[~jbellis] we need a committer.
> Running a drain before a decommission apparently the wrong thing to do
> --
>
> Key: CASSANDRA-8741
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8741
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Ubuntu 14.04; Cassandra 2.0.11.82 (Datastax Enterprise
> 4.5.3)
>Reporter: Casey Marshall
>Assignee: Jan Karlsson
>Priority: Trivial
> Labels: lhf
> Fix For: 2.1.x, 2.0.x
>
> Attachments: 8741.txt
>
>
> This might simply be a documentation issue. It appears that running "nodetool
> drain" is a very wrong thing to do before running a "nodetool decommission".
> The idea was that I was going to safely shut off writes and flush everything
> to disk before beginning the decommission. What happens is the "decommission"
> call appears to fail very early on after starting, and afterwards, the node
> in question is stuck in state LEAVING, but all other nodes in the ring see
> that node as NORMAL, but down. No streams are ever sent from the node being
> decommissioned to other nodes.
> The drain command does indeed shut down the "BatchlogTasks" executor
> (org/apache/cassandra/service/StorageService.java, line 3445 in git tag
> "cassandra-2.0.11") but the decommission process tries using that executor
> when calling the "startBatchlogReplay" function
> (org/apache/cassandra/db/BatchlogManager.java, line 123) called through
> org.apache.cassandra.service.StorageService.unbootstrap (see the stack trace
> pasted below).
> This also failed in a similar way on Cassandra 1.2.13-ish (DSE 3.2.4).
> So, either something is wrong with the drain/decommission commands, or it's
> very wrong to run a drain before a decommission. What's worse, there seems to
> be no way to recover this node once it is in this state; you need to shut it
> down and run "removenode".
> My terminal output:
> {code}
> ubuntu@x:~$ nodetool drain
> ubuntu@x:~$ tail /var/log/^C
> ubuntu@x:~$ nodetool decommission
> Exception in thread "main" java.util.concurrent.RejectedExecutionException:
> Task
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@3008fa33
> rejected from
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor@1d6242e8[Terminated,
> pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 52]
> at
> java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:325)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:530)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.submit(ScheduledThreadPoolExecutor.java:629)
> at
> org.apache.cassandra.db.BatchlogManager.startBatchlogReplay(BatchlogManager.java:123)
> at
> org.apache.cassandra.service.StorageService.unbootstrap(StorageService.java:2966)
> at
> org.apache.cassandra.service.StorageService.decommission(StorageService.java:2934)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.inv
[jira] [Comment Edited] (CASSANDRA-9839) Move crc_check_chance out of compressions options
[
https://issues.apache.org/jira/browse/CASSANDRA-9839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728236#comment-14728236
]
Paulo Motta edited comment on CASSANDRA-9839 at 9/2/15 11:51 PM:
-
Attaching 3.0+ patch for review.
Relevant changes described below:
* Moved {{crc_check_chance}} out of compression options, turning it into a
top-level table option.
* Accepting {{crc_check_chance}} as a compression option on {{CREATE TABLE}}
and {{ALTER TABLE}} statements temporarily, for backwards compatibility.
** If that's the case, a warning is logged and sent to the client explaining
the change.
* Performing schema migration of {{crc_check_chance}} property on upgrade from
2.2- clusters to 3.0+
* Modification of {{crc_check_chance}} via JMX no longer changes {{CFMetadata}}
directly on {{ColumnFamilyStore}}.
* Similar to the options {{min_compaction_treshold}} and
{{max_compaction_treshold}}, the JMX change will persist during the session,
being reset to the value of {{CFMetadata}} on next restart.
* Introduced interface {{ICheckSummedFiled}}, implemented by {{SegmentedFile}}
and extended by {{ICompressedFiled}}
** This interface allows {{CompressedRandomAccessReader}} to consume the
current value of {{ColumnFamilyStore.crcCheckChance}} by transparently
accessing {{ICheckSummedFile.getCrcCheckChanceSupplier()}}.
*** A default supplier of {{ICheckSummedFile.getCrcCheckChanceSupplier()}} with
a value of 1.0 is provided by {{SegmentedFile}} for use in offline tools.
*** In live clusters, during {{SStableReader}} initialization it calls
{{ICheckSummedFile.setCrcCheckChanceSupplier(ColumnfamilyStore::getCrcCheckChance)}}
to expose the real time value of {{crc_check_chance}} in a transparent way.
* Updated/fixed {{CrcCheckChanceTest}} unit tests
* Updated {{NEWS.txt}}
* Fixed dtests and added a new dtest for correctly verifying schema migration
of {{crc_check_chance}} option from 2.2-: [cassandra-dtest
PR|https://github.com/riptano/cassandra-dtest/pull/528]
Tests (will be available shortly):
* [3.0
dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-9839-dtest/]
* [3.0
testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-9839-testall/]
* [trunk
dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-9839-dtest/]
* [trunk
testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-9839-testall/]
was (Author: pauloricardomg):
Attaching 3.0+ patch for review.
Relevant changes described below:
* Moved {{crc_check_chance}} out of compression options, turning it into a
top-level table option.
* Accepting {{crc_check_chance}} as a compression option on {{CREATE TABLE}}
and {{ALTER TABLE}} statements temporarily, for backwards compatibility.
** If that's the case, a warning is logged and sent to the client explaining
the change.
* Performing schema migration of {{crc_check_chance}} property on upgrade from
2.2- clusters to 3.0+
* Modification of {{crc_check_chance}} via JMX no longer changes {{CFMetadata}}
directly on {{ColumnFamilyStore}}.
* Similar to the options {{min_compaction_treshold}} and
{{max_compaction_treshold}}, the JMX change will persist during the session,
being reset to the value of {{CFMetadata}} on next restart.
* Introduced interface {{ICheckSummedFiled}}, implemented by {{SegmentedFile}}
and extended by {{ICompressedFiled}}
** This interface allows {{CompressedRandomAccessReader}} to consume the
current value of {{ColumnFamilyStore.crcCheckChance}} by transparently
accessing {{ICheckSummedFile.getCrcCheckChanceSupplier()}}.
*** A default supplier of {{ICheckSummedFile.getCrcCheckChanceSupplier()}} with
a value of 1.0 is provided by {{SegmentedFile}} for use in offline tools.
*** In live clusters, during {{SStableReader}} initialization it calls
{{ICheckSummedFile.setCrcCheckChanceSupplier(ColumnfamilyStore::getCrcCheckChance)}}
to expose the current value of {{crc_check_chance}} in a transparent way.
* Updated/fixed {{CrcCheckChanceTest}} unit tests
* Updated {{NEWS.txt}}
* Fixed dtests and added a new dtest for correctly verifying schema migration
of {{crc_check_chance}} option from 2.2-: [cassandra-dtest
PR|https://github.com/riptano/cassandra-dtest/pull/528]
Tests (will be available shortly):
* [3.0
dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-9839-dtest/]
* [3.0
testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-9839-testall/]
* [trunk
dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-9839-dtest/]
* [trunk
testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-9839-testall/]
> Move crc_check_chance out of compressions options
> -
>
>
[jira] [Commented] (CASSANDRA-9839) Move crc_check_chance out of compressions options
[
https://issues.apache.org/jira/browse/CASSANDRA-9839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728236#comment-14728236
]
Paulo Motta commented on CASSANDRA-9839:
Attaching 3.0+ patch for review.
Relevant changes described below:
* Moved {{crc_check_chance}} out of compression options, turning it into a
top-level table option.
* Accepting {{crc_check_chance}} as a compression option on {{CREATE TABLE}}
and {{ALTER TABLE}} statements temporarily, for backwards compatibility.
** If that's the case, a warning is logged and sent to the client explaining
the change.
* Performing schema migration of {{crc_check_chance}} property on upgrade from
2.2- clusters to 3.0+
* Modification of {{crc_check_chance}} via JMX no longer changes {{CFMetadata}}
directly on {{ColumnFamilyStore}}.
* Similar to the options {{min_compaction_treshold}} and
{{max_compaction_treshold}}, the JMX change will persist during the session,
being reset to the value of {{CFMetadata}} on next restart.
* Introduced interface {{ICheckSummedFiled}}, implemented by {{SegmentedFile}}
and extended by {{ICompressedFiled}}
** This interface allows {{CompressedRandomAccessReader}} to consume the
current value of {{ColumnFamilyStore.crcCheckChance}} by transparently
accessing {{ICheckSummedFile.getCrcCheckChanceSupplier()}}.
*** A default supplier of {{ICheckSummedFile.getCrcCheckChanceSupplier()}} with
a value of 1.0 is provided by {{SegmentedFile}} for use in offline tools.
*** In live clusters, during {{SStableReader}} initialization it calls
{{ICheckSummedFile.setCrcCheckChanceSupplier(ColumnfamilyStore::getCrcCheckChance)}}
to expose the current value of {{crc_check_chance}} in a transparent way.
* Updated/fixed {{CrcCheckChanceTest}} unit tests
* Updated {{NEWS.txt}}
* Fixed dtests and added a new dtest for correctly verifying schema migration
of {{crc_check_chance}} option from 2.2-: [cassandra-dtest
PR|https://github.com/riptano/cassandra-dtest/pull/528]
Tests (will be available shortly):
* [3.0
dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-9839-dtest/]
* [3.0
testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-9839-testall/]
* [trunk
dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-9839-dtest/]
* [trunk
testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-9839-testall/]
> Move crc_check_chance out of compressions options
> -
>
> Key: CASSANDRA-9839
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9839
> Project: Cassandra
> Issue Type: Bug
>Reporter: Aleksey Yeschenko
>Assignee: Paulo Motta
>Priority: Minor
> Labels: client-impacting, docs-impacting
> Fix For: 3.0.0 rc1
>
>
> Follow up to CASSANDRA-8384. The option doesn't belong to compression params
> - it doesn't affect compression, itself, and isn't passed to compressors upon
> initialization.
> While it's true that it is (currently) only being honored when reading
> compressed sstables, it still doesn't belong to compression params (and is
> causing CASSANDRA-7978 -like issues).
> [~tjake] suggested we should make it an option of its own, and I think we
> should.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10195) TWCS experiments and improvement proposals
[ https://issues.apache.org/jira/browse/CASSANDRA-10195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728221#comment-14728221 ] Philip Thompson commented on CASSANDRA-10195: - DTCS: Total bytes compacted: 2019124775954 TWCS Total bytes compacted: 1260627615892 > TWCS experiments and improvement proposals > -- > > Key: CASSANDRA-10195 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10195 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Antti Nissinen > Fix For: 2.1.x, 2.2.x > > Attachments: 20150814_1027_compaction_hierarchy.txt, > node0_20150727_1250_time_graph.txt, node0_20150810_1017_time_graph.txt, > node0_20150812_1531_time_graph.txt, node0_20150813_0835_time_graph.txt, > node0_20150814_1054_time_graph.txt, node1_20150727_1250_time_graph.txt, > node1_20150810_1017_time_graph.txt, node1_20150812_1531_time_graph.txt, > node1_20150813_0835_time_graph.txt, node1_20150814_1054_time_graph.txt, > node2_20150727_1250_time_graph.txt, node2_20150810_1017_time_graph.txt, > node2_20150812_1531_time_graph.txt, node2_20150813_0835_time_graph.txt, > node2_20150814_1054_time_graph.txt, sstable_count_figure1.png, > sstable_count_figure2.png > > > This JIRA item describes experiments with DateTieredCompactionStartegy (DTCS) > and TimeWindowCompactionStrategy (TWCS) and proposes modifications to the > TWCS. In a test system several crashes were caused intentionally (and > unintentionally) and repair operations were executed leading to flood of > small SSTables. Target was to be able compact those files are release disk > space reserved by duplicate data. Setup is following: > - Three nodes > - DateTieredCompactionStrategy, max_sstable_age_days = 5 > Cassandra 2.1.2 > The setup and data format has been documented in detailed here > https://issues.apache.org/jira/browse/CASSANDRA-9644. > The test was started by dumping few days worth of data to the database for > 100 000 signals. Time graphs of SStables from different nodes indicates that > the DTCS has been working as expected and SStables are nicely ordered in time > wise. > See files: > node0_20150727_1250_time_graph.txt > node1_20150727_1250_time_graph.txt > node2_20150727_1250_time_graph.txt > Status=Up/Down > |/ State=Normal/Leaving/Joining/Moving > -- AddressLoad Tokens OwnsHost ID > Rack > UN 139.66.43.170 188.87 GB 256 ? > dfc29863-c935-4909-9d7f-c59a47eda03d rack1 > UN 139.66.43.169 198.37 GB 256 ? > 12e7628b-7f05-48f6-b7e4-35a82010021a rack1 > UN 139.66.43.168 191.88 GB 256 ? > 26088392-f803-4d59-9073-c75f857fb332 rack1 > All nodes crashed due to power failure (know beforehand) and repair > operations were started for each node one at the time. Below is the behavior > of SSTable count on different nodes. New data was dumped simultaneously with > repair operation. > SEE FIGURE: sstable_count_figure1.png > Vertical lines indicate following events. > 1) Cluster was down due to power shutdown and was restarted. At the first > vertical line the repair operation (nodetool repair -pr) was started for the > first node > 2) Repair for the second repair operation was started after the first node > was successfully repaired. > 3) Repair for the third repair operation was started > 4) Third repair operation was finished > 5) One of the nodes crashed (unknown reason in OS level) > 6) Repair operation (nodetool repair -pr) was started for the first node > 7) Repair operation for the second node was started > 8) Repair operation for the third node was started > 9) Repair operations finished > These repair operations are leading to huge amount of small SSTables covering > the whole time span of the data. The compaction horizon of DTCS was limited > to 5 days (max_sstable_age_days) due to the size of the SStables on the disc. > Therefore, small SStables won't be compacted. Below are the time graphs from > SSTables after the second round of repairs. > Status=Up/Down > |/ State=Normal/Leaving/Joining/Moving > -- AddressLoad Tokens OwnsHost ID > Rack > UN xx.xx.xx.170 663.61 GB 256 ? > dfc29863-c935-4909-9d7f-c59a47eda03d rack1 > UN xx.xx.xx.169 763.52 GB 256 ? > 12e7628b-7f05-48f6-b7e4-35a82010021a rack1 > UN xx.xx.xx.168 651.59 GB 256 ? > 26088392-f803-4d59-9073-c75f857fb332 rack1 > See files: > node0_20150810_1017_time_graph.txt > node1_20150810_1017_time_graph.txt > node2_20150810_1017_time_graph.txt > To get rid of the SStables the TimeWindowCompactionStrategy was taken into > use. Window size was set to 5 days. Cassandra version was updated to 2.1.8. > Below figure shows the behavior of SSta
[jira] [Updated] (CASSANDRA-10253) Incrimental repairs not working as expected with DTCS
[
https://issues.apache.org/jira/browse/CASSANDRA-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-10253:
Assignee: Marcus Eriksson
> Incrimental repairs not working as expected with DTCS
> -
>
> Key: CASSANDRA-10253
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10253
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Pre-prod
>Reporter: vijay
>Assignee: Marcus Eriksson
> Fix For: 2.1.x
>
> Attachments: systemfiles 2.zip
>
>
> HI,
> we are ingesting data 6 million records every 15 mins into one DTCS table and
> relaying on Cassandra for purging the data.Table Schema given below, Issue 1:
> we are expecting to see table sstable created on day d1 will not be compacted
> after d1 how we are not seeing this, how ever i see some data being purged at
> random intervals
> Issue 2: when we run incremental repair using "nodetool repair keyspace table
> -inc -pr" each sstable is splitting up to multiple smaller SStables and
> increasing the total storage.This behavior is same running repairs on any
> node and any number of times
> There are mutation drop's in the cluster
> Table:
> {code}
> CREATE TABLE TableA (
> F1 text,
> F2 int,
> createts bigint,
> stats blob,
> PRIMARY KEY ((F1,F2), createts)
> ) WITH CLUSTERING ORDER BY (createts DESC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'min_threshold': '12', 'max_sstable_age_days': '1',
> 'base_time_seconds': '50', 'class':
> 'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy'}
> AND compression = {'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.0
> AND default_time_to_live = 93600
> AND gc_grace_seconds = 3600
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {code}
> Thanks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10079) LEAK DETECTED, after nodetool drain
[
https://issues.apache.org/jira/browse/CASSANDRA-10079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728186#comment-14728186
]
Yuki Morishita commented on CASSANDRA-10079:
Reproduced with the following:
# Create SSTables using {{cassaandra-stress}} with size tiered compaction
disabled.
# {{nodetool compact}}
# During major compaction, trigger drain.
# Force GC
# LEAK DETECTED
I found that compaction keeps running after drain, and it can get stuck when
updating {{compaction_in_progress}} because commit log is stopped during drain.
Stack trace where compaction is hanging is:
{code}
CompactionExecutor:1 [WAITING] [DAEMON]
sun.misc.Unsafe.park(boolean, long) Unsafe.java (native)
java.util.concurrent.locks.LockSupport.park() LockSupport.java:304
org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUninterruptibly()
WaitQueue.java:283
org.apache.cassandra.db.commitlog.PeriodicCommitLogService.maybeWaitForSync(CommitLogSegment$Allocation)
PeriodicCommitLogService.java:44
org.apache.cassandra.db.commitlog.AbstractCommitLogService.finishWriteFor(CommitLogSegment$Allocation)
AbstractCommitLogService.java:152
org.apache.cassandra.db.commitlog.CommitLog.add(Mutation) CommitLog.java:252
org.apache.cassandra.db.Keyspace.apply(Mutation, boolean, boolean)
Keyspace.java:383
org.apache.cassandra.db.Keyspace.apply(Mutation, boolean) Keyspace.java:363
org.apache.cassandra.db.Mutation.apply() Mutation.java:214
org.apache.cassandra.cql3.statements.ModificationStatement.executeInternalWithoutCondition(QueryState,
QueryOptions) ModificationStatement.java:641
org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(QueryState,
QueryOptions) ModificationStatement.java:631
org.apache.cassandra.cql3.QueryProcessor.executeInternal(String, Object[])
QueryProcessor.java:317
org.apache.cassandra.db.SystemKeyspace.finishCompaction(UUID)
SystemKeyspace.java:241
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow()
CompactionTask.java:233
org.apache.cassandra.utils.WrappedRunnable.run() WrappedRunnable.java:28
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionManager$CompactionExecutorStatsCollector)
CompactionTask.java:73
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(CompactionManager$CompactionExecutorStatsCollector)
AbstractCompactionTask.java:59
org.apache.cassandra.db.compaction.CompactionManager$7.runMayThrow()
CompactionManager.java:510
org.apache.cassandra.utils.WrappedRunnable.run() WrappedRunnable.java:28
java.util.concurrent.Executors$RunnableAdapter.call() Executors.java:511
java.util.concurrent.FutureTask.run() FutureTask.java:266
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker)
ThreadPoolExecutor.java:1142
java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java:617
java.lang.Thread.run() Thread.java:745
{code}
Since compaction task relies on accessing system tables
({{compaction_in_progress}} (not in 3.0 anymore) and {{compaction_history}}) we
should shutdown compaction before shutting down commit log.
> LEAK DETECTED, after nodetool drain
> ---
>
> Key: CASSANDRA-10079
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10079
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sebastian Estevez
>Assignee: Yuki Morishita
> Fix For: 2.1.x
>
>
> 6 node cluster running 2.1.8
> Sequence of events:
> {quote}
> 2015-08-14 13:37:07,049 - Drain the node
> 2015-08-14 13:37:11,943 - Drained
> 2015-08-14 13:37:37,055 Ref.java:179 - LEAK DETECTED:
> {quote}
> {code}
> ERROR [Reference-Reaper:1] 2015-08-14 13:37:37,055 Ref.java:179 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@5534701) to class
> org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@194296283:[[OffHeapBitSet]]
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-08-14 13:37:37,057 Ref.java:179 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@fab2c71) to class
> org.apache.cassandra.io.util.MmappedSegmentedFile$Cleanup@1252635616:/var/lib/cassandra/data/metric/metric-811fa5402a3b11e5a2c0870545c0f352/metric-metric-ka-6883-Index.db
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-08-14 13:37:37,057 Ref.java:179 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@555d8efb) to class
> org.apache.cassandra.io.util.MmappedSegmentedFile$Cleanup@1252635616:/var/lib/cassandra/data/metric/metric-811fa5402a3b11e5a2c0870545c0f352/metric-metric-ka-6883-Index.db
> was not released before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2015-08-14 13:37:37,057 Ref.java:179 - LEAK
> DETECTED:
[jira] [Updated] (CASSANDRA-10253) Incrimental repairs not working as expected with DTCS
[
https://issues.apache.org/jira/browse/CASSANDRA-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vijay updated CASSANDRA-10253:
--
Attachment: (was: systemfiles 2.zip)
> Incrimental repairs not working as expected with DTCS
> -
>
> Key: CASSANDRA-10253
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10253
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Pre-prod
>Reporter: vijay
> Fix For: 2.1.x
>
> Attachments: systemfiles 2.zip
>
>
> HI,
> we are ingesting data 6 million records every 15 mins into one DTCS table and
> relaying on Cassandra for purging the data.Table Schema given below, Issue 1:
> we are expecting to see table sstable created on day d1 will not be compacted
> after d1 how we are not seeing this, how ever i see some data being purged at
> random intervals
> Issue 2: when we run incremental repair using "nodetool repair keyspace table
> -inc -pr" each sstable is splitting up to multiple smaller SStables and
> increasing the total storage.This behavior is same running repairs on any
> node and any number of times
> There are mutation drop's in the cluster
> Table:
> {code}
> CREATE TABLE TableA (
> F1 text,
> F2 int,
> createts bigint,
> stats blob,
> PRIMARY KEY ((F1,F2), createts)
> ) WITH CLUSTERING ORDER BY (createts DESC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'min_threshold': '12', 'max_sstable_age_days': '1',
> 'base_time_seconds': '50', 'class':
> 'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy'}
> AND compression = {'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.0
> AND default_time_to_live = 93600
> AND gc_grace_seconds = 3600
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {code}
> Thanks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10253) Incrimental repairs not working as expected with DTCS
[
https://issues.apache.org/jira/browse/CASSANDRA-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vijay updated CASSANDRA-10253:
--
Attachment: systemfiles 2.zip
> Incrimental repairs not working as expected with DTCS
> -
>
> Key: CASSANDRA-10253
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10253
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Pre-prod
>Reporter: vijay
> Fix For: 2.1.x
>
> Attachments: systemfiles 2.zip, systemfiles 2.zip
>
>
> HI,
> we are ingesting data 6 million records every 15 mins into one DTCS table and
> relaying on Cassandra for purging the data.Table Schema given below, Issue 1:
> we are expecting to see table sstable created on day d1 will not be compacted
> after d1 how we are not seeing this, how ever i see some data being purged at
> random intervals
> Issue 2: when we run incremental repair using "nodetool repair keyspace table
> -inc -pr" each sstable is splitting up to multiple smaller SStables and
> increasing the total storage.This behavior is same running repairs on any
> node and any number of times
> There are mutation drop's in the cluster
> Table:
> {code}
> CREATE TABLE TableA (
> F1 text,
> F2 int,
> createts bigint,
> stats blob,
> PRIMARY KEY ((F1,F2), createts)
> ) WITH CLUSTERING ORDER BY (createts DESC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'min_threshold': '12', 'max_sstable_age_days': '1',
> 'base_time_seconds': '50', 'class':
> 'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy'}
> AND compression = {'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.0
> AND default_time_to_live = 93600
> AND gc_grace_seconds = 3600
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {code}
> Thanks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10253) Incrimental repairs not working as expected with DTCS
[
https://issues.apache.org/jira/browse/CASSANDRA-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vijay updated CASSANDRA-10253:
--
Attachment: systemfiles 2.zip
Attached system.log files from all 6 nodes
> Incrimental repairs not working as expected with DTCS
> -
>
> Key: CASSANDRA-10253
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10253
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Pre-prod
>Reporter: vijay
> Fix For: 2.1.x
>
> Attachments: systemfiles 2.zip
>
>
> HI,
> we are ingesting data 6 million records every 15 mins into one DTCS table and
> relaying on Cassandra for purging the data.Table Schema given below, Issue 1:
> we are expecting to see table sstable created on day d1 will not be compacted
> after d1 how we are not seeing this, how ever i see some data being purged at
> random intervals
> Issue 2: when we run incremental repair using "nodetool repair keyspace table
> -inc -pr" each sstable is splitting up to multiple smaller SStables and
> increasing the total storage.This behavior is same running repairs on any
> node and any number of times
> There are mutation drop's in the cluster
> Table:
> {code}
> CREATE TABLE TableA (
> F1 text,
> F2 int,
> createts bigint,
> stats blob,
> PRIMARY KEY ((F1,F2), createts)
> ) WITH CLUSTERING ORDER BY (createts DESC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'min_threshold': '12', 'max_sstable_age_days': '1',
> 'base_time_seconds': '50', 'class':
> 'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy'}
> AND compression = {'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.0
> AND default_time_to_live = 93600
> AND gc_grace_seconds = 3600
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {code}
> Thanks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10195) TWCS experiments and improvement proposals
[
https://issues.apache.org/jira/browse/CASSANDRA-10195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728151#comment-14728151
]
Jeff Jirsa edited comment on CASSANDRA-10195 at 9/2/15 10:36 PM:
-
[~philipthompson], thanks!
Can you run this quick script on both (only looks at latest logfiles, if logs
have rotated, adjust accordingly):
{code}
export bytes_out=0
for i in `grep Compacted /var/log/cassandra/system.log | grep -v system | awk
'{ print $15 }' | sed -e 's/,//g'`
do
bytes_out=`echo $bytes_out + $i | bc`
done
echo "Total bytes compacted: $bytes_out"
{code}
(Basically just counting the total bytes compacted while running, excludes
system keyspace)
was (Author: jjirsa):
[~philipthompson], thanks!
Can you run this quick script on both:
{code}
export bytes_out=0
for i in `grep Compacted /var/log/cassandra/system.log | grep -v system | awk
'{ print $15 }' | sed -e 's/,//g'`
do
bytes_out=`echo $bytes_out + $i | bc`
done
echo "Total bytes compacted: $bytes_out"
{code}
(Basically just counting the total bytes compacted while running, excludes
system keyspace)
> TWCS experiments and improvement proposals
> --
>
> Key: CASSANDRA-10195
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10195
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Antti Nissinen
> Fix For: 2.1.x, 2.2.x
>
> Attachments: 20150814_1027_compaction_hierarchy.txt,
> node0_20150727_1250_time_graph.txt, node0_20150810_1017_time_graph.txt,
> node0_20150812_1531_time_graph.txt, node0_20150813_0835_time_graph.txt,
> node0_20150814_1054_time_graph.txt, node1_20150727_1250_time_graph.txt,
> node1_20150810_1017_time_graph.txt, node1_20150812_1531_time_graph.txt,
> node1_20150813_0835_time_graph.txt, node1_20150814_1054_time_graph.txt,
> node2_20150727_1250_time_graph.txt, node2_20150810_1017_time_graph.txt,
> node2_20150812_1531_time_graph.txt, node2_20150813_0835_time_graph.txt,
> node2_20150814_1054_time_graph.txt, sstable_count_figure1.png,
> sstable_count_figure2.png
>
>
> This JIRA item describes experiments with DateTieredCompactionStartegy (DTCS)
> and TimeWindowCompactionStrategy (TWCS) and proposes modifications to the
> TWCS. In a test system several crashes were caused intentionally (and
> unintentionally) and repair operations were executed leading to flood of
> small SSTables. Target was to be able compact those files are release disk
> space reserved by duplicate data. Setup is following:
> - Three nodes
> - DateTieredCompactionStrategy, max_sstable_age_days = 5
> Cassandra 2.1.2
> The setup and data format has been documented in detailed here
> https://issues.apache.org/jira/browse/CASSANDRA-9644.
> The test was started by dumping few days worth of data to the database for
> 100 000 signals. Time graphs of SStables from different nodes indicates that
> the DTCS has been working as expected and SStables are nicely ordered in time
> wise.
> See files:
> node0_20150727_1250_time_graph.txt
> node1_20150727_1250_time_graph.txt
> node2_20150727_1250_time_graph.txt
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- AddressLoad Tokens OwnsHost ID
> Rack
> UN 139.66.43.170 188.87 GB 256 ?
> dfc29863-c935-4909-9d7f-c59a47eda03d rack1
> UN 139.66.43.169 198.37 GB 256 ?
> 12e7628b-7f05-48f6-b7e4-35a82010021a rack1
> UN 139.66.43.168 191.88 GB 256 ?
> 26088392-f803-4d59-9073-c75f857fb332 rack1
> All nodes crashed due to power failure (know beforehand) and repair
> operations were started for each node one at the time. Below is the behavior
> of SSTable count on different nodes. New data was dumped simultaneously with
> repair operation.
> SEE FIGURE: sstable_count_figure1.png
> Vertical lines indicate following events.
> 1) Cluster was down due to power shutdown and was restarted. At the first
> vertical line the repair operation (nodetool repair -pr) was started for the
> first node
> 2) Repair for the second repair operation was started after the first node
> was successfully repaired.
> 3) Repair for the third repair operation was started
> 4) Third repair operation was finished
> 5) One of the nodes crashed (unknown reason in OS level)
> 6) Repair operation (nodetool repair -pr) was started for the first node
> 7) Repair operation for the second node was started
> 8) Repair operation for the third node was started
> 9) Repair operations finished
> These repair operations are leading to huge amount of small SSTables covering
> the whole time span of the data. The compaction horizon of DTCS was limited
> to 5 days (max_sstable_age_days) due to the size of the SStables on the disc.
> Therefore, small SS
[jira] [Commented] (CASSANDRA-10195) TWCS experiments and improvement proposals
[
https://issues.apache.org/jira/browse/CASSANDRA-10195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728151#comment-14728151
]
Jeff Jirsa commented on CASSANDRA-10195:
[~philipthompson], thanks!
Can you run this quick script on both:
{code}
export bytes_out=0
for i in `grep Compacted /var/log/cassandra/system.log | grep -v system | awk
'{ print $15 }' | sed -e 's/,//g'`
do
bytes_out=`echo $bytes_out + $i | bc`
done
echo "Total bytes compacted: $bytes_out"
{code}
(Basically just counting the total bytes compacted while running, excludes
system keyspace)
> TWCS experiments and improvement proposals
> --
>
> Key: CASSANDRA-10195
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10195
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Antti Nissinen
> Fix For: 2.1.x, 2.2.x
>
> Attachments: 20150814_1027_compaction_hierarchy.txt,
> node0_20150727_1250_time_graph.txt, node0_20150810_1017_time_graph.txt,
> node0_20150812_1531_time_graph.txt, node0_20150813_0835_time_graph.txt,
> node0_20150814_1054_time_graph.txt, node1_20150727_1250_time_graph.txt,
> node1_20150810_1017_time_graph.txt, node1_20150812_1531_time_graph.txt,
> node1_20150813_0835_time_graph.txt, node1_20150814_1054_time_graph.txt,
> node2_20150727_1250_time_graph.txt, node2_20150810_1017_time_graph.txt,
> node2_20150812_1531_time_graph.txt, node2_20150813_0835_time_graph.txt,
> node2_20150814_1054_time_graph.txt, sstable_count_figure1.png,
> sstable_count_figure2.png
>
>
> This JIRA item describes experiments with DateTieredCompactionStartegy (DTCS)
> and TimeWindowCompactionStrategy (TWCS) and proposes modifications to the
> TWCS. In a test system several crashes were caused intentionally (and
> unintentionally) and repair operations were executed leading to flood of
> small SSTables. Target was to be able compact those files are release disk
> space reserved by duplicate data. Setup is following:
> - Three nodes
> - DateTieredCompactionStrategy, max_sstable_age_days = 5
> Cassandra 2.1.2
> The setup and data format has been documented in detailed here
> https://issues.apache.org/jira/browse/CASSANDRA-9644.
> The test was started by dumping few days worth of data to the database for
> 100 000 signals. Time graphs of SStables from different nodes indicates that
> the DTCS has been working as expected and SStables are nicely ordered in time
> wise.
> See files:
> node0_20150727_1250_time_graph.txt
> node1_20150727_1250_time_graph.txt
> node2_20150727_1250_time_graph.txt
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- AddressLoad Tokens OwnsHost ID
> Rack
> UN 139.66.43.170 188.87 GB 256 ?
> dfc29863-c935-4909-9d7f-c59a47eda03d rack1
> UN 139.66.43.169 198.37 GB 256 ?
> 12e7628b-7f05-48f6-b7e4-35a82010021a rack1
> UN 139.66.43.168 191.88 GB 256 ?
> 26088392-f803-4d59-9073-c75f857fb332 rack1
> All nodes crashed due to power failure (know beforehand) and repair
> operations were started for each node one at the time. Below is the behavior
> of SSTable count on different nodes. New data was dumped simultaneously with
> repair operation.
> SEE FIGURE: sstable_count_figure1.png
> Vertical lines indicate following events.
> 1) Cluster was down due to power shutdown and was restarted. At the first
> vertical line the repair operation (nodetool repair -pr) was started for the
> first node
> 2) Repair for the second repair operation was started after the first node
> was successfully repaired.
> 3) Repair for the third repair operation was started
> 4) Third repair operation was finished
> 5) One of the nodes crashed (unknown reason in OS level)
> 6) Repair operation (nodetool repair -pr) was started for the first node
> 7) Repair operation for the second node was started
> 8) Repair operation for the third node was started
> 9) Repair operations finished
> These repair operations are leading to huge amount of small SSTables covering
> the whole time span of the data. The compaction horizon of DTCS was limited
> to 5 days (max_sstable_age_days) due to the size of the SStables on the disc.
> Therefore, small SStables won't be compacted. Below are the time graphs from
> SSTables after the second round of repairs.
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- AddressLoad Tokens OwnsHost ID
> Rack
> UN xx.xx.xx.170 663.61 GB 256 ?
> dfc29863-c935-4909-9d7f-c59a47eda03d rack1
> UN xx.xx.xx.169 763.52 GB 256 ?
> 12e7628b-7f05-48f6-b7e4-35a82010021a rack1
> UN xx.xx.xx.168 651.59 GB 256 ?
> 26088392-f803-4d59-9073-c75f857fb332 rack1
> See files:
[jira] [Commented] (CASSANDRA-10169) Test regression for consistency_test.TestAccuracy
[
https://issues.apache.org/jira/browse/CASSANDRA-10169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728143#comment-14728143
]
Blake Eggleston commented on CASSANDRA-10169:
-
{{consistency_test.TestAccuracy.test_simple_strategy_users}} is failing with
{noformat}
ReadFailure: code=1300 [Replica(s) failed to execute read] message="Operation
failed - received 1 responses and 2 failures" info={'failures': 2,
'received_responses': 1, 'required_responses': 2, 'consistency': 'QUORUM'}
{noformat}
Looking at the logs of the nodes with failures, I'm seeing this exception:
{noformat}
WARN [SharedPool-Worker-5] 2015-09-02 13:15:49,703
AbstractTracingAwareExecutorService.java:169 - Uncaught exception on thread
Thread[SharedPool-Worker-5,5,main]: {}
java.lang.IllegalStateException: [age firstname lastname] is not a subset of
[age]
at
org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:532)
~[main/:na]
at
org.apache.cassandra.db.Columns$Serializer.serializeSubset(Columns.java:466)
~[main/:na]
at
org.apache.cassandra.db.SerializationHeader$Serializer.serializeForMessaging(SerializationHeader.java:370)
~[main/:na]
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:117)
~[main/:na]
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:86)
~[main/:na]
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
~[main/:na]
at
org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:381)
~[main/:na]
at
org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:135)
~[main/:na]
at
org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:127)
~[main/:na]
at
org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:122)
~[main/:na]
at
org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:64)
~[main/:na]
at
org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:287)
~[main/:na]
at
org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:53)
~[main/:na]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_40]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
~[main/:na]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[main/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
{noformat}
which is being caused by the column bitmap serialization introduced in
CASSANDRA-9894 being fed columns it's not expecting. This appears to be related
to the table in question using compact storage. The PartitionColumns instance
in the column filter also seems to think that age is a static column, which it
is not.
> Test regression for consistency_test.TestAccuracy
> -
>
> Key: CASSANDRA-10169
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10169
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Sylvain Lebresne
>Assignee: Blake Eggleston
> Fix For: 3.0 beta 2
>
>
> We have 2 tests failures:
> {noformat}
> consistency_test.TestAccuracy.test_network_topology_strategy_users
> consistency_test.TestAccuracy.test_simple_strategy_users
> {noformat}
> and looking at [the
> history|http://cassci.datastax.com/job/cassandra-3.0_dtest/90/testReport/junit/consistency_test/TestAccuracy/test_simple_strategy_users/history/?start=50],
> CASSANDRA-9913 appears to be the culprit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10233) IndexOutOfBoundsException in HintedHandOffManager
[
https://issues.apache.org/jira/browse/CASSANDRA-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728125#comment-14728125
]
Omri Iluz commented on CASSANDRA-10233:
---
Upgraded from 2.1.8. I upgraded today to 2.2.1 and still see the issue.
I am not sure which table is causing this issue and we have quite a few tables
so pasting the whole schema would be impossible. If you have any pointers on
how to narrow to a single table (or even a single keyspace) I'd be happy to
provide it's schema.
The only changes to cassandra.yaml are increasing the write timeout and
increasing the size of out of heap memtable to 6GB.
> IndexOutOfBoundsException in HintedHandOffManager
> -
>
> Key: CASSANDRA-10233
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10233
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.2.0
>Reporter: Omri Iluz
>
> After upgrading our cluster to 2.2.0, the following error started showing
> exectly every 10 minutes on every server in the cluster:
> {noformat}
> INFO [CompactionExecutor:1381] 2015-08-31 18:31:55,506
> CompactionTask.java:142 - Compacting (8e7e1520-500e-11e5-b1e3-e95897ba4d20)
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-540-big-Data.db:level=0,
> ]
> INFO [CompactionExecutor:1381] 2015-08-31 18:31:55,599
> CompactionTask.java:224 - Compacted (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 1
> sstables to
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-541-big,]
> to level=0. 1,544,495 bytes to 1,544,495 (~100% of original) in 93ms =
> 15.838121MB/s. 0 total partitions merged to 4. Partition merge counts were
> {1:4, }
> ERROR [HintedHandoff:1] 2015-08-31 18:31:55,600 CassandraDaemon.java:182 -
> Exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_79]
> at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:410)
> ~[na:1.7.0_79]
> at org.apache.cassandra.utils.UUIDGen.getUUID(UUIDGen.java:106)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:515)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:88)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:168)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_79]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> [na:1.7.0_79]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> [na:1.7.0_79]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [na:1.7.0_79]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_79]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_79]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10240) sstableexpiredblockers can throw FileNotFound exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-10240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728124#comment-14728124
]
Brandon Williams commented on CASSANDRA-10240:
--
+1, I guess we'll have to run it offline to be certain the results are correct.
> sstableexpiredblockers can throw FileNotFound exceptions
>
>
> Key: CASSANDRA-10240
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10240
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Brandon Williams
>Assignee: Marcus Eriksson
> Fix For: 2.1.x, 2.0.x, 2.2.x, 3.0.x
>
>
> {noformat}
> Exception in thread "main" java.lang.RuntimeException:
> java.io.FileNotFoundException:
> /path/to/data/system/sstable_activity/system-sstable_activity-jb-85002-Data.db
> (No such file or directory)
> at
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:47)
> at
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile.createReader(CompressedPoolingSegmentedFile.java:48)
> at
> org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:39)
> at
> org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:1242)
> at
> org.apache.cassandra.db.columniterator.SimpleSliceReader.(SimpleSliceReader.java:57)
> at
> org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:65)
> at
> org.apache.cassandra.db.columniterator.SSTableSliceIterator.(SSTableSliceIterator.java:42)
> at
> org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:173)
> at
> org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62)
> at
> org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:250)
> at
> org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1599)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1418)
> at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:330)
> at
> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.readLocally(SelectStatement.java:296)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:315)
> at
> org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:64)
> at
> org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208)
> at
> org.apache.cassandra.db.SystemKeyspace.getSSTableReadMeter(SystemKeyspace.java:957)
> at
> org.apache.cassandra.io.sstable.SSTableReader.(SSTableReader.java:340)
> at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:196)
> at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:184)
> at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:150)
> at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:142)
> at
> org.apache.cassandra.tools.SSTableExpiredBlockers.main(SSTableExpiredBlockers.java:77)
> {noformat}
> It appears this may be caused by sstable_activity getting compacted away
> while this is running. I'm not sure if sstableexpiredblockers should just
> try again or what, but it should probably at least handle the exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10229) bad read performance after a major compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-10229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728089#comment-14728089
]
Alan Boudreault commented on CASSANDRA-10229:
-
So, just finished to test again this on ec2 (i2.2xlarge).
Before major compaction: 37500 op/s.
After major compaction run 1: 27000 op/s.
After major compaction run 2: 28800 op/s.
After major compaction run 3: 29000 op/s.
After major compaction run 4: 28700 op/s.
So, its better than the 3x regression I got the other day. but curious
where this 37k comes from if running it after multiple times doesn't help.
[~enigmacurry] [~jjordan] I know this ticket seems strange and/or invalid...
but I would like to understand what's going on. Should we investigate more or
close this ticket?
[~mambocab] want to give it a try on ec2?
> bad read performance after a major compaction
> -
>
> Key: CASSANDRA-10229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10229
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
> Labels: perfomance
> Attachments: users-caching.yaml
>
>
> I am trying to understand what I am seeing. My scenario is very basic, it's a
> simple users table with key cache and row cache disabled. I write 50M then
> read 5M random elements. The read performance is not that bad BEFORE a major
> compaction of the data. I see a ~3x performance regression AFTER I run a
> major compaction.
> Here's the read performance numbers for my scenario:
> {code}
> 3.0 before a major compaction (Key cache and row cache disabled), note that
> this is the numbers from 50M, I see the same with 5M
> ==
> Results:
> op rate : 9149 [read:9149]
> partition rate: 9149 [read:9149]
> row rate : 9149 [read:9149]
> latency mean : 32.8 [read:32.8]
> latency median: 31.2 [read:31.2]
> latency 95th percentile : 47.2 [read:47.2]
> latency 99th percentile : 55.0 [read:55.0]
> latency 99.9th percentile : 66.4 [read:66.4]
> latency max : 305.4 [read:305.4]
> Total partitions : 5000 [read:5000]
> Total errors : 0 [read:0]
> total gc count: 0
> total gc mb : 0
> total gc time (s) : 0
> avg gc time(ms) : NaN
> stdev gc time(ms) : 0
> Total operation time : 01:31:05
> END
> -rw-rw-r-- 1 aboudreault aboudreault 4.7G Aug 31 08:51 ma-1024-big-Data.db
> -rw-rw-r-- 1 aboudreault aboudreault 4.9G Aug 31 09:08 ma-1077-big-Data.db
> 3.0 after a major compaction (Key cache and row cache disabled), note that
> this is the numbers from 50M, I see the same with 5M
>
> Results:
> op rate : 3275 [read:3275]
> partition rate: 3275 [read:3275]
> row rate : 3275 [read:3275]
> latency mean : 91.6 [read:91.6]
> latency median: 88.8 [read:88.8]
> latency 95th percentile : 107.2 [read:107.2]
> latency 99th percentile : 116.0 [read:116.0]
> latency 99.9th percentile : 125.5 [read:125.5]
> latency max : 249.0 [read:249.0]
> Total partitions : 5000 [read:5000]
> Total errors : 0 [read:0]
> total gc count: 0
> total gc mb : 0
> total gc time (s) : 0
> avg gc time(ms) : NaN
> stdev gc time(ms) : 0
> Total operation time : 04:14:26
> END
> -rw-rw-r-- 1 aboudreault aboudreault 9.5G Aug 31 09:40 ma-1079-big-Data.db
> 2.1 before major compaction (Key cache and row cache disabled)
> ==
> Results:
> op rate : 21348 [read:21348]
> partition rate: 21348 [read:21348]
> row rate : 21348 [read:21348]
> latency mean : 14.1 [read:14.1]
> latency median: 8.0 [read:8.0]
> latency 95th percentile : 38.5 [read:38.5]
> latency 99th percentile : 60.8 [read:60.8]
> latency 99.9th percentile : 99.2 [read:99.2]
> latency max : 229.2 [read:229.2]
> Total partitions : 500 [read:500]
> Total errors : 0 [read:0]
> total gc count: 0
> total gc mb : 0
> total gc time (s) : 0
> avg gc time(ms) : NaN
> stdev gc time(ms) : 0
> Total operation time : 00:03:54
> END
> 2.1 after major compaction (Key cache and row cache disabled)
> =
> Results:
> op rate : 5262 [read:5262]
> partition rate: 5262 [read:5262]
> row rate : 5262
[jira] [Commented] (CASSANDRA-10253) Incrimental repairs not working as expected with DTCS
[
https://issues.apache.org/jira/browse/CASSANDRA-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728078#comment-14728078
]
Philip Thompson commented on CASSANDRA-10253:
-
Can you attach the system.log file from one of the nodes, please?
> Incrimental repairs not working as expected with DTCS
> -
>
> Key: CASSANDRA-10253
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10253
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Pre-prod
>Reporter: vijay
> Fix For: 2.1.x
>
>
> HI,
> we are ingesting data 6 million records every 15 mins into one DTCS table and
> relaying on Cassandra for purging the data.Table Schema given below, Issue 1:
> we are expecting to see table sstable created on day d1 will not be compacted
> after d1 how we are not seeing this, how ever i see some data being purged at
> random intervals
> Issue 2: when we run incremental repair using "nodetool repair keyspace table
> -inc -pr" each sstable is splitting up to multiple smaller SStables and
> increasing the total storage.This behavior is same running repairs on any
> node and any number of times
> There are mutation drop's in the cluster
> Table:
> {code}
> CREATE TABLE TableA (
> F1 text,
> F2 int,
> createts bigint,
> stats blob,
> PRIMARY KEY ((F1,F2), createts)
> ) WITH CLUSTERING ORDER BY (createts DESC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'min_threshold': '12', 'max_sstable_age_days': '1',
> 'base_time_seconds': '50', 'class':
> 'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy'}
> AND compression = {'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.0
> AND default_time_to_live = 93600
> AND gc_grace_seconds = 3600
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {code}
> Thanks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10253) Incrimental repairs not working as expected with DTCS
[
https://issues.apache.org/jira/browse/CASSANDRA-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-10253:
Reproduced In: 2.1.8
Fix Version/s: 2.1.x
> Incrimental repairs not working as expected with DTCS
> -
>
> Key: CASSANDRA-10253
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10253
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Pre-prod
>Reporter: vijay
> Fix For: 2.1.x
>
>
> HI,
> we are ingesting data 6 million records every 15 mins into one DTCS table and
> relaying on Cassandra for purging the data.Table Schema given below, Issue 1:
> we are expecting to see table sstable created on day d1 will not be compacted
> after d1 how we are not seeing this, how ever i see some data being purged at
> random intervals
> Issue 2: when we run incremental repair using "nodetool repair keyspace table
> -inc -pr" each sstable is splitting up to multiple smaller SStables and
> increasing the total storage.This behavior is same running repairs on any
> node and any number of times
> There are mutation drop's in the cluster
> Table:
> {code}
> CREATE TABLE TableA (
> F1 text,
> F2 int,
> createts bigint,
> stats blob,
> PRIMARY KEY ((F1,F2), createts)
> ) WITH CLUSTERING ORDER BY (createts DESC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'min_threshold': '12', 'max_sstable_age_days': '1',
> 'base_time_seconds': '50', 'class':
> 'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy'}
> AND compression = {'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.0
> AND default_time_to_live = 93600
> AND gc_grace_seconds = 3600
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {code}
> Thanks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10253) Incrimental repairs not working as expected with DTCS
[
https://issues.apache.org/jira/browse/CASSANDRA-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728076#comment-14728076
]
vijay commented on CASSANDRA-10253:
---
HI we are running cassandra 2.1.8
> Incrimental repairs not working as expected with DTCS
> -
>
> Key: CASSANDRA-10253
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10253
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Pre-prod
>Reporter: vijay
>
> HI,
> we are ingesting data 6 million records every 15 mins into one DTCS table and
> relaying on Cassandra for purging the data.Table Schema given below, Issue 1:
> we are expecting to see table sstable created on day d1 will not be compacted
> after d1 how we are not seeing this, how ever i see some data being purged at
> random intervals
> Issue 2: when we run incremental repair using "nodetool repair keyspace table
> -inc -pr" each sstable is splitting up to multiple smaller SStables and
> increasing the total storage.This behavior is same running repairs on any
> node and any number of times
> There are mutation drop's in the cluster
> Table:
> {code}
> CREATE TABLE TableA (
> F1 text,
> F2 int,
> createts bigint,
> stats blob,
> PRIMARY KEY ((F1,F2), createts)
> ) WITH CLUSTERING ORDER BY (createts DESC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'min_threshold': '12', 'max_sstable_age_days': '1',
> 'base_time_seconds': '50', 'class':
> 'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy'}
> AND compression = {'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.0
> AND default_time_to_live = 93600
> AND gc_grace_seconds = 3600
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {code}
> Thanks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9667) strongly consistent membership and ownership
[ https://issues.apache.org/jira/browse/CASSANDRA-9667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728069#comment-14728069 ] Jason Brown commented on CASSANDRA-9667: updated manual join protocol: We require two "consensus transactions"(1) to become a full token-owner in the cluster: one transaction to become a member, and another to take ownership of some tokens(2). So, the protocol would look like this: # new node comes up, and performs consensus transaction to become a member. Will probably need to contact a seed to perform transaction, and should happen automatically, without operator intervention. ## transaction adds new node to the member set, as well as incrementing a logical clock(2) ## commit the transaction (retry on failure) ## updated member set/logical clock is gossiped - perhaps also directly message the joining node, as well (sending it the total member set and metadata). ## node is allowed to participate in gossip after becoming a member. # Repeat for each new node to be added to the cluster. # operator can inoke 'nodetool show-pending-nodes' (or whatever we call the command) to see a plan of the current nodes waiting to be assigned tokens (and become part of the ring). Operator can confirm that everything looks as expected. # Operator invokes 'nodetool joinall' to start the next consensus transaction, to make new nodes owners of tokens: ## proposer of transaction auto-generates tokens for nodes that did not declare any (like a node replace), ## updates the member set with tokens for each node, and increments the logical clock ## commit the transaction (retry on failure) ## updated member set/increments is gossiped - perhaps also directly message the transitioning nodes, as well. ## transitioning nodes can change their status themselves, and start bootstrapping (if necessary) Note: if we generate token in the ownership step (as I think we should, for optiminzing the token selection), then we cannot show the 'pending ranges/size to transfer' in the 'nodetool show-pending-nodes' command output (as we don't know all the nodes/tokens yet) as [~tjake] proposed. However, we might be able to get away with displaying it after the owner consensus transaction is complete. Alternatively, we could run the same alg that generates the tokens to 'predict' what the tokens will look like, and display that as a potentially non-exact (but close enough) approximation of what the cluster will look like before executing the transaction. 1) I'm not completely sure of using LWT for the necessary linearizability operation (for safe cluster state transitions), so I'll use the stand-in "consensus transaction" for now. 2) Tokens may be assigned manually by an operator, derived from another node that is being replaced, or auto-generated by the a proposer of the consensus transaction (think proposer in Paxos). 3) Logical clock - an monotonic counter that indicates the linearized changes to the member set (either adding/removing nodes, or changes to tokens of a member). Basically, an integer that is incremented on every modification. > strongly consistent membership and ownership > > > Key: CASSANDRA-9667 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9667 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jason Brown >Assignee: Jason Brown > Labels: LWT, membership, ownership > Fix For: 3.x > > > Currently, there is advice to users to "wait two minutes between adding new > nodes" in order for new node tokens, et al, to propagate. Further, as there's > no coordination amongst joining node wrt token selection, new nodes can end > up selecting ranges that overlap with other joining nodes. This causes a lot > of duplicate streaming from the existing source nodes as they shovel out the > bootstrap data for those new nodes. > This ticket proposes creating a mechanism that allows strongly consistent > membership and ownership changes in cassandra such that changes are performed > in a linearizable and safe manner. The basic idea is to use LWT operations > over a global system table, and leverage the linearizability of LWT for > ensuring the safety of cluster membership/ownership state changes. This work > is inspired by Riak's claimant module. > The existing workflows for node join, decommission, remove, replace, and > range move (there may be others I'm not thinking of) will need to be modified > to participate in this scheme, as well as changes to nodetool to enable them. > Note: we distinguish between membership and ownership in the following ways: > for membership we mean "a host in this cluster and it's state". For > ownership, we mean "what tokens (or ranges) does each node own"; these nod
[jira] [Commented] (CASSANDRA-10253) Incrimental repairs not working as expected with DTCS
[
https://issues.apache.org/jira/browse/CASSANDRA-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728061#comment-14728061
]
Philip Thompson commented on CASSANDRA-10253:
-
What Cassandra version are you running?
> Incrimental repairs not working as expected with DTCS
> -
>
> Key: CASSANDRA-10253
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10253
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Pre-prod
>Reporter: vijay
>
> HI,
> we are ingesting data 6 million records every 15 mins into one DTCS table and
> relaying on Cassandra for purging the data.Table Schema given below, Issue 1:
> we are expecting to see table sstable created on day d1 will not be compacted
> after d1 how we are not seeing this, how ever i see some data being purged at
> random intervals
> Issue 2: when we run incremental repair using "nodetool repair keyspace table
> -inc -pr" each sstable is splitting up to multiple smaller SStables and
> increasing the total storage.This behavior is same running repairs on any
> node and any number of times
> There are mutation drop's in the cluster
> Table:
> {code}
> CREATE TABLE TableA (
> F1 text,
> F2 int,
> createts bigint,
> stats blob,
> PRIMARY KEY ((F1,F2), createts)
> ) WITH CLUSTERING ORDER BY (createts DESC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'min_threshold': '12', 'max_sstable_age_days': '1',
> 'base_time_seconds': '50', 'class':
> 'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy'}
> AND compression = {'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.0
> AND default_time_to_live = 93600
> AND gc_grace_seconds = 3600
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {code}
> Thanks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10253) Incrimental repairs not working as expected with DTCS
[
https://issues.apache.org/jira/browse/CASSANDRA-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-10253:
Description:
HI,
we are ingesting data 6 million records every 15 mins into one DTCS table and
relaying on Cassandra for purging the data.Table Schema given below, Issue 1:
we are expecting to see table sstable created on day d1 will not be compacted
after d1 how we are not seeing this, how ever i see some data being purged at
random intervals
Issue 2: when we run incremental repair using "nodetool repair keyspace table
-inc -pr" each sstable is splitting up to multiple smaller SStables and
increasing the total storage.This behavior is same running repairs on any node
and any number of times
There are mutation drop's in the cluster
Table:
{code}
CREATE TABLE TableA (
F1 text,
F2 int,
createts bigint,
stats blob,
PRIMARY KEY ((F1,F2), createts)
) WITH CLUSTERING ORDER BY (createts DESC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = {'min_threshold': '12', 'max_sstable_age_days': '1',
'base_time_seconds': '50', 'class':
'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy'}
AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 93600
AND gc_grace_seconds = 3600
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
{code}
Thanks
was:
HI,
we are ingesting data 6 million records every 15 mins into one DTCS table and
relaying on Cassandra for purging the data.Table Schema given below, Issue 1:
we are expecting to see table sstable created on day d1 will not be compacted
after d1 how we are not seeing this, how ever i see some data being purged at
random intervals
Issue 2: when we run incremental repair using "nodetool repair keyspace table
-inc -pr" each sstable is splitting up to multiple smaller SStables and
increasing the total storage.This behavior is same running repairs on any node
and any number of times
There are mutation drop's in the cluster
Table:
CREATE TABLE TableA (
F1 text,
F2 int,
createts bigint,
stats blob,
PRIMARY KEY ((F1,F2), createts)
) WITH CLUSTERING ORDER BY (createts DESC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = {'min_threshold': '12', 'max_sstable_age_days': '1',
'base_time_seconds': '50', 'class':
'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy'}
AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 93600
AND gc_grace_seconds = 3600
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
Thanks
> Incrimental repairs not working as expected with DTCS
> -
>
> Key: CASSANDRA-10253
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10253
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Pre-prod
>Reporter: vijay
>
> HI,
> we are ingesting data 6 million records every 15 mins into one DTCS table and
> relaying on Cassandra for purging the data.Table Schema given below, Issue 1:
> we are expecting to see table sstable created on day d1 will not be compacted
> after d1 how we are not seeing this, how ever i see some data being purged at
> random intervals
> Issue 2: when we run incremental repair using "nodetool repair keyspace table
> -inc -pr" each sstable is splitting up to multiple smaller SStables and
> increasing the total storage.This behavior is same running repairs on any
> node and any number of times
> There are mutation drop's in the cluster
> Table:
> {code}
> CREATE TABLE TableA (
> F1 text,
> F2 int,
> createts bigint,
> stats blob,
> PRIMARY KEY ((F1,F2), createts)
> ) WITH CLUSTERING ORDER BY (createts DESC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'min_threshold': '12', 'max_sstable_age_days': '1',
> 'base_time_seconds': '50', 'class':
> 'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy'}
> AND compression = {'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.0
> AND default_time_to_live = 93
[jira] [Commented] (CASSANDRA-10229) bad read performance after a major compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-10229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728054#comment-14728054
]
Alan Boudreault commented on CASSANDRA-10229:
-
Currently testing this it again on ec2. Not sure if 2.1 vs 3.0 is fair since
they have been run on a different day and not the same cluster. Things are
strange sometime on ec2.
> bad read performance after a major compaction
> -
>
> Key: CASSANDRA-10229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10229
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
> Labels: perfomance
> Attachments: users-caching.yaml
>
>
> I am trying to understand what I am seeing. My scenario is very basic, it's a
> simple users table with key cache and row cache disabled. I write 50M then
> read 5M random elements. The read performance is not that bad BEFORE a major
> compaction of the data. I see a ~3x performance regression AFTER I run a
> major compaction.
> Here's the read performance numbers for my scenario:
> {code}
> 3.0 before a major compaction (Key cache and row cache disabled), note that
> this is the numbers from 50M, I see the same with 5M
> ==
> Results:
> op rate : 9149 [read:9149]
> partition rate: 9149 [read:9149]
> row rate : 9149 [read:9149]
> latency mean : 32.8 [read:32.8]
> latency median: 31.2 [read:31.2]
> latency 95th percentile : 47.2 [read:47.2]
> latency 99th percentile : 55.0 [read:55.0]
> latency 99.9th percentile : 66.4 [read:66.4]
> latency max : 305.4 [read:305.4]
> Total partitions : 5000 [read:5000]
> Total errors : 0 [read:0]
> total gc count: 0
> total gc mb : 0
> total gc time (s) : 0
> avg gc time(ms) : NaN
> stdev gc time(ms) : 0
> Total operation time : 01:31:05
> END
> -rw-rw-r-- 1 aboudreault aboudreault 4.7G Aug 31 08:51 ma-1024-big-Data.db
> -rw-rw-r-- 1 aboudreault aboudreault 4.9G Aug 31 09:08 ma-1077-big-Data.db
> 3.0 after a major compaction (Key cache and row cache disabled), note that
> this is the numbers from 50M, I see the same with 5M
>
> Results:
> op rate : 3275 [read:3275]
> partition rate: 3275 [read:3275]
> row rate : 3275 [read:3275]
> latency mean : 91.6 [read:91.6]
> latency median: 88.8 [read:88.8]
> latency 95th percentile : 107.2 [read:107.2]
> latency 99th percentile : 116.0 [read:116.0]
> latency 99.9th percentile : 125.5 [read:125.5]
> latency max : 249.0 [read:249.0]
> Total partitions : 5000 [read:5000]
> Total errors : 0 [read:0]
> total gc count: 0
> total gc mb : 0
> total gc time (s) : 0
> avg gc time(ms) : NaN
> stdev gc time(ms) : 0
> Total operation time : 04:14:26
> END
> -rw-rw-r-- 1 aboudreault aboudreault 9.5G Aug 31 09:40 ma-1079-big-Data.db
> 2.1 before major compaction (Key cache and row cache disabled)
> ==
> Results:
> op rate : 21348 [read:21348]
> partition rate: 21348 [read:21348]
> row rate : 21348 [read:21348]
> latency mean : 14.1 [read:14.1]
> latency median: 8.0 [read:8.0]
> latency 95th percentile : 38.5 [read:38.5]
> latency 99th percentile : 60.8 [read:60.8]
> latency 99.9th percentile : 99.2 [read:99.2]
> latency max : 229.2 [read:229.2]
> Total partitions : 500 [read:500]
> Total errors : 0 [read:0]
> total gc count: 0
> total gc mb : 0
> total gc time (s) : 0
> avg gc time(ms) : NaN
> stdev gc time(ms) : 0
> Total operation time : 00:03:54
> END
> 2.1 after major compaction (Key cache and row cache disabled)
> =
> Results:
> op rate : 5262 [read:5262]
> partition rate: 5262 [read:5262]
> row rate : 5262 [read:5262]
> latency mean : 57.0 [read:57.0]
> latency median: 55.5 [read:55.5]
> latency 95th percentile : 69.4 [read:69.4]
> latency 99th percentile : 83.3 [read:83.3]
> latency 99.9th percentile : 197.4 [read:197.4]
> latency max : 1169.0 [read:1169.0]
> Total partitions : 500 [read:500]
> Total errors : 0 [read:0]
> total gc count: 0
> total gc mb : 0
>
[jira] [Commented] (CASSANDRA-8630) Faster sequential IO (on compaction, streaming, etc)
[ https://issues.apache.org/jira/browse/CASSANDRA-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728053#comment-14728053 ] Benedict commented on CASSANDRA-8630: - Well, I don't know if it's random chance or what, but now it's consistently better (across three runs), which is good enough for me. > Faster sequential IO (on compaction, streaming, etc) > > > Key: CASSANDRA-8630 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8630 > Project: Cassandra > Issue Type: Improvement > Components: Core, Tools >Reporter: Oleg Anastasyev >Assignee: Stefania > Labels: compaction, performance > Fix For: 3.x > > Attachments: 8630-FasterSequencialReadsAndWrites.txt, cpu_load.png, > flight_recorder_001_files.tar.gz, flight_recorder_002_files.tar.gz, > mmaped_uncomp_hotspot.png > > > When node is doing a lot of sequencial IO (streaming, compacting, etc) a lot > of CPU is lost in calls to RAF's int read() and DataOutputStream's write(int). > This is because default implementations of readShort,readLong, etc as well as > their matching write* are implemented with numerous calls of byte by byte > read and write. > This makes a lot of syscalls as well. > A quick microbench shows than just reimplementation of these methods in > either way gives 8x speed increase. > A patch attached implements RandomAccessReader.read and > SequencialWriter.write methods in more efficient way. > I also eliminated some extra byte copies in CompositeType.split and > ColumnNameHelper.maxComponents, which were on my profiler's hotspot method > list during tests. > A stress tests on my laptop show that this patch makes compaction 25-30% > faster on uncompressed sstables and 15% faster for compressed ones. > A deployment to production shows much less CPU load for compaction. > (I attached a cpu load graph from one of our production, orange is niced CPU > load - i.e. compaction; yellow is user - i.e. not compaction related tasks) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10109) Windows dtest 3.0: ttl_test.py failures
[
https://issues.apache.org/jira/browse/CASSANDRA-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728051#comment-14728051
]
Benedict commented on CASSANDRA-10109:
--
So, I have pushed a version
[here|github.com/belliottsmith/cassandra/tree/10109-3.0] with some largeish
changes. These are all *suggestions*, and I would like to emphasize that you
should challenge anything you think is a retrograde step for clarity. I just
felt the code was likely to be tricky for anybody besides the two of us to
maintain as it stood, and I hope you don't mind my having a go at (hopefully)
making it more approachable to a non-author. To summarise, I have:
* Combined both atomic/non-atomic {{DirectoryStream}} paths
* Split out all of the inner classes into their own files
* Merged the {{TransactionFile}} and {{TransactionData}} classes into
{{LogFile}}, and moved the resource management out into the {{TransactionLog}}
(which is all that needs it)
* Merged the checksum portion of the log file lines with the {{Record}}
functionality, so all parsing is in one place
* Moved all of the validation logic into one place, including checksum
verification
* Made validation throw no exceptions, only log and return the result, for
consideration of the caller
* Made extensive use of streams where possible and clarity was helped, since
these are all low traffic codepaths
I haven't quite finished looking it all over and thinking about the behaviour.
It's worth noting that four of your unit tests now fail, however looking at
them it appears that the correct behaviour is to fail - although perhaps not if
we are just listing. That requires some consideration.
Let me know what you think.
> Windows dtest 3.0: ttl_test.py failures
> ---
>
> Key: CASSANDRA-10109
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10109
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Joshua McKenzie
>Assignee: Stefania
> Labels: Windows
> Fix For: 3.0.0 rc1
>
>
> ttl_test.py:TestTTL.update_column_ttl_with_default_ttl_test2
> ttl_test.py:TestTTL.update_multiple_columns_ttl_test
> ttl_test.py:TestTTL.update_single_column_ttl_test
> Errors locally are different than CI from yesterday. Yesterday on CI we have
> timeouts and general node hangs. Today on all 3 tests when run locally I see:
> {noformat}
> Traceback (most recent call last):
> File "c:\src\cassandra-dtest\dtest.py", line 532, in tearDown
> raise AssertionError('Unexpected error in %s node log: %s' % (node.name,
> errors))
> AssertionError: Unexpected error in node1 node log: ['ERROR [main] 2015-08-17
> 16:53:43,120 NoSpamLogger.java:97 - This platform does not support atomic
> directory streams (SecureDirectoryStream); race conditions when loading
> sstable files could occurr']
> {noformat}
> This traces back to the commit for CASSANDRA-7066 today by [~Stefania] and
> [~benedict]. Stefania - care to take this ticket and also look further into
> whether or not we're going to have issues with 7066 on Windows? That error
> message certainly *sounds* like it's not a good thing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10229) bad read performance after a major compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-10229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728047#comment-14728047
]
Jeremiah Jordan edited comment on CASSANDRA-10229 at 9/2/15 9:26 PM:
-
The fact that the 3.0 op rate is 50% of the 2.1 op rate pre compaction is more
concerning to me than the fact that after a restart and major compaction things
are slower. The restart+major compaction probably trashed all of the OS caches.
was (Author: jjordan):
The fact that the 3.0 op rate is 50% of the 2.1 op rate pre compaction is more
concerning to me than the fact that after a major compaction things are slower.
The major compaction probably trashed all of the OS caches.
> bad read performance after a major compaction
> -
>
> Key: CASSANDRA-10229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10229
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
> Labels: perfomance
> Attachments: users-caching.yaml
>
>
> I am trying to understand what I am seeing. My scenario is very basic, it's a
> simple users table with key cache and row cache disabled. I write 50M then
> read 5M random elements. The read performance is not that bad BEFORE a major
> compaction of the data. I see a ~3x performance regression AFTER I run a
> major compaction.
> Here's the read performance numbers for my scenario:
> {code}
> 3.0 before a major compaction (Key cache and row cache disabled), note that
> this is the numbers from 50M, I see the same with 5M
> ==
> Results:
> op rate : 9149 [read:9149]
> partition rate: 9149 [read:9149]
> row rate : 9149 [read:9149]
> latency mean : 32.8 [read:32.8]
> latency median: 31.2 [read:31.2]
> latency 95th percentile : 47.2 [read:47.2]
> latency 99th percentile : 55.0 [read:55.0]
> latency 99.9th percentile : 66.4 [read:66.4]
> latency max : 305.4 [read:305.4]
> Total partitions : 5000 [read:5000]
> Total errors : 0 [read:0]
> total gc count: 0
> total gc mb : 0
> total gc time (s) : 0
> avg gc time(ms) : NaN
> stdev gc time(ms) : 0
> Total operation time : 01:31:05
> END
> -rw-rw-r-- 1 aboudreault aboudreault 4.7G Aug 31 08:51 ma-1024-big-Data.db
> -rw-rw-r-- 1 aboudreault aboudreault 4.9G Aug 31 09:08 ma-1077-big-Data.db
> 3.0 after a major compaction (Key cache and row cache disabled), note that
> this is the numbers from 50M, I see the same with 5M
>
> Results:
> op rate : 3275 [read:3275]
> partition rate: 3275 [read:3275]
> row rate : 3275 [read:3275]
> latency mean : 91.6 [read:91.6]
> latency median: 88.8 [read:88.8]
> latency 95th percentile : 107.2 [read:107.2]
> latency 99th percentile : 116.0 [read:116.0]
> latency 99.9th percentile : 125.5 [read:125.5]
> latency max : 249.0 [read:249.0]
> Total partitions : 5000 [read:5000]
> Total errors : 0 [read:0]
> total gc count: 0
> total gc mb : 0
> total gc time (s) : 0
> avg gc time(ms) : NaN
> stdev gc time(ms) : 0
> Total operation time : 04:14:26
> END
> -rw-rw-r-- 1 aboudreault aboudreault 9.5G Aug 31 09:40 ma-1079-big-Data.db
> 2.1 before major compaction (Key cache and row cache disabled)
> ==
> Results:
> op rate : 21348 [read:21348]
> partition rate: 21348 [read:21348]
> row rate : 21348 [read:21348]
> latency mean : 14.1 [read:14.1]
> latency median: 8.0 [read:8.0]
> latency 95th percentile : 38.5 [read:38.5]
> latency 99th percentile : 60.8 [read:60.8]
> latency 99.9th percentile : 99.2 [read:99.2]
> latency max : 229.2 [read:229.2]
> Total partitions : 500 [read:500]
> Total errors : 0 [read:0]
> total gc count: 0
> total gc mb : 0
> total gc time (s) : 0
> avg gc time(ms) : NaN
> stdev gc time(ms) : 0
> Total operation time : 00:03:54
> END
> 2.1 after major compaction (Key cache and row cache disabled)
> =
> Results:
> op rate : 5262 [read:5262]
> partition rate: 5262 [read:5262]
> row rate : 5262 [read:5262]
> latency mean : 57.0 [read:57.0]
> latency median: 55.5 [read:55.
[jira] [Commented] (CASSANDRA-10229) bad read performance after a major compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-10229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728047#comment-14728047
]
Jeremiah Jordan commented on CASSANDRA-10229:
-
The fact that the 3.0 op rate is 50% of the 2.1 op rate pre compaction is more
concerning to me than the fact that after a major compaction things are slower.
The major compaction probably trashed all of the OS caches.
> bad read performance after a major compaction
> -
>
> Key: CASSANDRA-10229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10229
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alan Boudreault
> Labels: perfomance
> Attachments: users-caching.yaml
>
>
> I am trying to understand what I am seeing. My scenario is very basic, it's a
> simple users table with key cache and row cache disabled. I write 50M then
> read 5M random elements. The read performance is not that bad BEFORE a major
> compaction of the data. I see a ~3x performance regression AFTER I run a
> major compaction.
> Here's the read performance numbers for my scenario:
> {code}
> 3.0 before a major compaction (Key cache and row cache disabled), note that
> this is the numbers from 50M, I see the same with 5M
> ==
> Results:
> op rate : 9149 [read:9149]
> partition rate: 9149 [read:9149]
> row rate : 9149 [read:9149]
> latency mean : 32.8 [read:32.8]
> latency median: 31.2 [read:31.2]
> latency 95th percentile : 47.2 [read:47.2]
> latency 99th percentile : 55.0 [read:55.0]
> latency 99.9th percentile : 66.4 [read:66.4]
> latency max : 305.4 [read:305.4]
> Total partitions : 5000 [read:5000]
> Total errors : 0 [read:0]
> total gc count: 0
> total gc mb : 0
> total gc time (s) : 0
> avg gc time(ms) : NaN
> stdev gc time(ms) : 0
> Total operation time : 01:31:05
> END
> -rw-rw-r-- 1 aboudreault aboudreault 4.7G Aug 31 08:51 ma-1024-big-Data.db
> -rw-rw-r-- 1 aboudreault aboudreault 4.9G Aug 31 09:08 ma-1077-big-Data.db
> 3.0 after a major compaction (Key cache and row cache disabled), note that
> this is the numbers from 50M, I see the same with 5M
>
> Results:
> op rate : 3275 [read:3275]
> partition rate: 3275 [read:3275]
> row rate : 3275 [read:3275]
> latency mean : 91.6 [read:91.6]
> latency median: 88.8 [read:88.8]
> latency 95th percentile : 107.2 [read:107.2]
> latency 99th percentile : 116.0 [read:116.0]
> latency 99.9th percentile : 125.5 [read:125.5]
> latency max : 249.0 [read:249.0]
> Total partitions : 5000 [read:5000]
> Total errors : 0 [read:0]
> total gc count: 0
> total gc mb : 0
> total gc time (s) : 0
> avg gc time(ms) : NaN
> stdev gc time(ms) : 0
> Total operation time : 04:14:26
> END
> -rw-rw-r-- 1 aboudreault aboudreault 9.5G Aug 31 09:40 ma-1079-big-Data.db
> 2.1 before major compaction (Key cache and row cache disabled)
> ==
> Results:
> op rate : 21348 [read:21348]
> partition rate: 21348 [read:21348]
> row rate : 21348 [read:21348]
> latency mean : 14.1 [read:14.1]
> latency median: 8.0 [read:8.0]
> latency 95th percentile : 38.5 [read:38.5]
> latency 99th percentile : 60.8 [read:60.8]
> latency 99.9th percentile : 99.2 [read:99.2]
> latency max : 229.2 [read:229.2]
> Total partitions : 500 [read:500]
> Total errors : 0 [read:0]
> total gc count: 0
> total gc mb : 0
> total gc time (s) : 0
> avg gc time(ms) : NaN
> stdev gc time(ms) : 0
> Total operation time : 00:03:54
> END
> 2.1 after major compaction (Key cache and row cache disabled)
> =
> Results:
> op rate : 5262 [read:5262]
> partition rate: 5262 [read:5262]
> row rate : 5262 [read:5262]
> latency mean : 57.0 [read:57.0]
> latency median: 55.5 [read:55.5]
> latency 95th percentile : 69.4 [read:69.4]
> latency 99th percentile : 83.3 [read:83.3]
> latency 99.9th percentile : 197.4 [read:197.4]
> latency max : 1169.0 [read:1169.0]
> Total partitions : 500 [read:500]
> Total errors : 0 [read:0]
> total gc count
[jira] [Comment Edited] (CASSANDRA-10195) TWCS experiments and improvement proposals
[
https://issues.apache.org/jira/browse/CASSANDRA-10195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728032#comment-14728032
]
Philip Thompson edited comment on CASSANDRA-10195 at 9/2/15 9:23 PM:
-
I have been doing some single node testing on i2.2xlarge instance with both
DTCS and TWCS. The DTCS node was running 2.1-HEAD at sha df9e798de6eb4. The
TWCS node was running Jeff's twcs-2.1 branch at git sha 6119d119c21a. Using
cassandra stress, I loaded 4.5B rows into each node, resulting in about 1TB of
data each.
The stress output from loading was too large to graph, but here are the
summaries:
DTCS:
{code}
Results:
op rate : 16010 [insert:16010]
partition rate: 16010 [insert:16010]
row rate : 16010 [insert:16010]
latency mean : 15.6 [insert:15.6]
latency median: 4.0 [insert:4.0]
latency 95th percentile : 21.2 [insert:21.2]
latency 99th percentile : 59.1 [insert:59.1]
latency 99.9th percentile : 366.1 [insert:366.1]
latency max : 12100.5 [insert:12100.5]
{code}
TWCS:
{code}
Results:
op rate : 16397 [insert:16397]
partition rate: 16397 [insert:16397]
row rate : 16397 [insert:16397]
latency mean : 15.2 [insert:15.2]
latency median: 3.9 [insert:3.9]
latency 95th percentile : 22.0 [insert:22.0]
latency 99th percentile : 66.9 [insert:66.9]
latency 99.9th percentile : 1494.0 [insert:1494.0]
latency max : 24187.3 [insert:24187.3]
{code}
The stress yaml profiles I used are available here:
https://gist.github.com/ptnapoleon/6e3f16049c756ba1e53a
https://gist.github.com/ptnapoleon/bdc9390edb0b004691bd
After loading the data, I gave the cluster some time to finish compactions. I
then began a long-running mixed workload that I terminated after a day, in
favor of a shorter, 5M op mixed workload, at a 1:3 insert/read ratio. Here are
the results of that test, which show better performance from TWCS:
http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=twcs_dtcs.brief.json&metric=op_rate&operation=1_MIXED&smoothing=1&show_aggregates=true&xmin=0&xmax=30877.88&ymin=0&ymax=761.2
The only non-default configuration change I made, was to increase
concurrent_compactors to 8 on each node.
I still have both clusters up, and have a lot of log data collected. What
additional compaction specific tunings, or workloads should be tested?
was (Author: philipthompson):
I have been doing some single node testing on i2.2xlarge instance with both
DTCS and TWCS. The DTCS node was running 2.1-HEAD at sha df9e798de6eb4. The
TWCS node was running Jeff's twcs-2.1 branch at git sha 6119d119c21a. Using
cassandra stress, I loaded 4.5B rows into each node, resulting in about 1TB of
data each.
The stress yaml profiles I used are available here:
https://gist.github.com/ptnapoleon/6e3f16049c756ba1e53a
https://gist.github.com/ptnapoleon/bdc9390edb0b004691bd
After loading the data, I gave the cluster some time to finish compactions. I
then began a long-running mixed workload that I terminated after a day, in
favor of a shorter, 5M op mixed workload, at a 1:3 insert/read ratio. Here are
the results of that test, which show better performance from TWCS:
http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=twcs_dtcs.brief.json&metric=op_rate&operation=1_MIXED&smoothing=1&show_aggregates=true&xmin=0&xmax=30877.88&ymin=0&ymax=761.2
The only non-default configuration change I made, was to increase
concurrent_compactors to 8 on each node.
I still have both clusters up, and have a lot of log data collected. What
additional compaction specific tunings, or workloads should be tested?
> TWCS experiments and improvement proposals
> --
>
> Key: CASSANDRA-10195
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10195
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Antti Nissinen
> Fix For: 2.1.x, 2.2.x
>
> Attachments: 20150814_1027_compaction_hierarchy.txt,
> node0_20150727_1250_time_graph.txt, node0_20150810_1017_time_graph.txt,
> node0_20150812_1531_time_graph.txt, node0_20150813_0835_time_graph.txt,
> node0_20150814_1054_time_graph.txt, node1_20150727_1250_time_graph.txt,
> node1_20150810_1017_time_graph.txt, node1_20150812_1531_time_graph.txt,
> node1_20150813_0835_time_graph.txt, node1_20150814_1054_time_graph.txt,
> node2_20150727_1250_time_graph.txt, node2_20150810_1017_time_graph.txt,
> node2_20150812_1531_time_graph.txt, node2_20150813_0835_time_graph.txt,
> node2_20150814_1054_time_graph.txt, sstable_count_figure1.png,
> sstable_count_figure2.png
>
>
> This JIRA item describes experiments with DateTieredCompactionStarte
[jira] [Updated] (CASSANDRA-10229) bad read performance after a major compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-10229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-10229:
Description:
I am trying to understand what I am seeing. My scenario is very basic, it's a
simple users table with key cache and row cache disabled. I write 50M then read
5M random elements. The read performance is not that bad BEFORE a major
compaction of the data. I see a ~3x performance regression AFTER I run a major
compaction.
Here's the read performance numbers for my scenario:
{code}
3.0 before a major compaction (Key cache and row cache disabled), note that
this is the numbers from 50M, I see the same with 5M
==
Results:
op rate : 9149 [read:9149]
partition rate: 9149 [read:9149]
row rate : 9149 [read:9149]
latency mean : 32.8 [read:32.8]
latency median: 31.2 [read:31.2]
latency 95th percentile : 47.2 [read:47.2]
latency 99th percentile : 55.0 [read:55.0]
latency 99.9th percentile : 66.4 [read:66.4]
latency max : 305.4 [read:305.4]
Total partitions : 5000 [read:5000]
Total errors : 0 [read:0]
total gc count: 0
total gc mb : 0
total gc time (s) : 0
avg gc time(ms) : NaN
stdev gc time(ms) : 0
Total operation time : 01:31:05
END
-rw-rw-r-- 1 aboudreault aboudreault 4.7G Aug 31 08:51 ma-1024-big-Data.db
-rw-rw-r-- 1 aboudreault aboudreault 4.9G Aug 31 09:08 ma-1077-big-Data.db
3.0 after a major compaction (Key cache and row cache disabled), note that this
is the numbers from 50M, I see the same with 5M
Results:
op rate : 3275 [read:3275]
partition rate: 3275 [read:3275]
row rate : 3275 [read:3275]
latency mean : 91.6 [read:91.6]
latency median: 88.8 [read:88.8]
latency 95th percentile : 107.2 [read:107.2]
latency 99th percentile : 116.0 [read:116.0]
latency 99.9th percentile : 125.5 [read:125.5]
latency max : 249.0 [read:249.0]
Total partitions : 5000 [read:5000]
Total errors : 0 [read:0]
total gc count: 0
total gc mb : 0
total gc time (s) : 0
avg gc time(ms) : NaN
stdev gc time(ms) : 0
Total operation time : 04:14:26
END
-rw-rw-r-- 1 aboudreault aboudreault 9.5G Aug 31 09:40 ma-1079-big-Data.db
2.1 before major compaction (Key cache and row cache disabled)
==
Results:
op rate : 21348 [read:21348]
partition rate: 21348 [read:21348]
row rate : 21348 [read:21348]
latency mean : 14.1 [read:14.1]
latency median: 8.0 [read:8.0]
latency 95th percentile : 38.5 [read:38.5]
latency 99th percentile : 60.8 [read:60.8]
latency 99.9th percentile : 99.2 [read:99.2]
latency max : 229.2 [read:229.2]
Total partitions : 500 [read:500]
Total errors : 0 [read:0]
total gc count: 0
total gc mb : 0
total gc time (s) : 0
avg gc time(ms) : NaN
stdev gc time(ms) : 0
Total operation time : 00:03:54
END
2.1 after major compaction (Key cache and row cache disabled)
=
Results:
op rate : 5262 [read:5262]
partition rate: 5262 [read:5262]
row rate : 5262 [read:5262]
latency mean : 57.0 [read:57.0]
latency median: 55.5 [read:55.5]
latency 95th percentile : 69.4 [read:69.4]
latency 99th percentile : 83.3 [read:83.3]
latency 99.9th percentile : 197.4 [read:197.4]
latency max : 1169.0 [read:1169.0]
Total partitions : 500 [read:500]
Total errors : 0 [read:0]
total gc count: 0
total gc mb : 0
total gc time (s) : 0
avg gc time(ms) : NaN
stdev gc time(ms) : 0
Total operation time : 00:15:50
END
{code}
I can reproduce that read performance regression on EC2 and locally. To
reproduce:
1. Launch a 1 node cluster (2.1, 2.2 or 3.0)
2. Set the compaction thoughput at 0. (need a restart IIRC)
3. Write 50M elements (so we get the same sstable size for the test). The yaml
profile is attached in this ticket. Ensure you are using stress from
apache/cassandra-3.0, trunk is broken at the moment.
{code}
cassandra-stress user profile=`pwd`/users-caching.yaml ops\(insert=1\) n=50M
-rate threads=100
{code}
4. Wait for the auto-compaction to finish and also flush the memtables. You
should get around 2-6 sstables when it's done.
5. Read 5M element
[jira] [Created] (CASSANDRA-10253) Incrimental repairs not working as expected with DTCS
vijay created CASSANDRA-10253:
-
Summary: Incrimental repairs not working as expected with DTCS
Key: CASSANDRA-10253
URL: https://issues.apache.org/jira/browse/CASSANDRA-10253
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Pre-prod
Reporter: vijay
HI,
we are ingesting data 6 million records every 15 mins into one DTCS table and
relaying on Cassandra for purging the data.Table Schema given below, Issue 1:
we are expecting to see table sstable created on day d1 will not be compacted
after d1 how we are not seeing this, how ever i see some data being purged at
random intervals
Issue 2: when we run incremental repair using "nodetool repair keyspace table
-inc -pr" each sstable is splitting up to multiple smaller SStables and
increasing the total storage.This behavior is same running repairs on any node
and any number of times
There are mutation drop's in the cluster
Table:
CREATE TABLE TableA (
F1 text,
F2 int,
createts bigint,
stats blob,
PRIMARY KEY ((F1,F2), createts)
) WITH CLUSTERING ORDER BY (createts DESC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = {'min_threshold': '12', 'max_sstable_age_days': '1',
'base_time_seconds': '50', 'class':
'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy'}
AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 93600
AND gc_grace_seconds = 3600
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
Thanks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10195) TWCS experiments and improvement proposals
[ https://issues.apache.org/jira/browse/CASSANDRA-10195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728032#comment-14728032 ] Philip Thompson commented on CASSANDRA-10195: - I have been doing some single node testing on i2.2xlarge instance with both DTCS and TWCS. The DTCS node was running 2.1-HEAD at sha df9e798de6eb4. The TWCS node was running Jeff's twcs-2.1 branch at git sha 6119d119c21a. Using cassandra stress, I loaded 4.5B rows into each node, resulting in about 1TB of data each. The stress yaml profiles I used are available here: https://gist.github.com/ptnapoleon/6e3f16049c756ba1e53a https://gist.github.com/ptnapoleon/bdc9390edb0b004691bd After loading the data, I gave the cluster some time to finish compactions. I then began a long-running mixed workload that I terminated after a day, in favor of a shorter, 5M op mixed workload, at a 1:3 insert/read ratio. Here are the results of that test, which show better performance from TWCS: http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=twcs_dtcs.brief.json&metric=op_rate&operation=1_MIXED&smoothing=1&show_aggregates=true&xmin=0&xmax=30877.88&ymin=0&ymax=761.2 The only non-default configuration change I made, was to increase concurrent_compactors to 8 on each node. I still have both clusters up, and have a lot of log data collected. What additional compaction specific tunings, or workloads should be tested? > TWCS experiments and improvement proposals > -- > > Key: CASSANDRA-10195 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10195 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Antti Nissinen > Fix For: 2.1.x, 2.2.x > > Attachments: 20150814_1027_compaction_hierarchy.txt, > node0_20150727_1250_time_graph.txt, node0_20150810_1017_time_graph.txt, > node0_20150812_1531_time_graph.txt, node0_20150813_0835_time_graph.txt, > node0_20150814_1054_time_graph.txt, node1_20150727_1250_time_graph.txt, > node1_20150810_1017_time_graph.txt, node1_20150812_1531_time_graph.txt, > node1_20150813_0835_time_graph.txt, node1_20150814_1054_time_graph.txt, > node2_20150727_1250_time_graph.txt, node2_20150810_1017_time_graph.txt, > node2_20150812_1531_time_graph.txt, node2_20150813_0835_time_graph.txt, > node2_20150814_1054_time_graph.txt, sstable_count_figure1.png, > sstable_count_figure2.png > > > This JIRA item describes experiments with DateTieredCompactionStartegy (DTCS) > and TimeWindowCompactionStrategy (TWCS) and proposes modifications to the > TWCS. In a test system several crashes were caused intentionally (and > unintentionally) and repair operations were executed leading to flood of > small SSTables. Target was to be able compact those files are release disk > space reserved by duplicate data. Setup is following: > - Three nodes > - DateTieredCompactionStrategy, max_sstable_age_days = 5 > Cassandra 2.1.2 > The setup and data format has been documented in detailed here > https://issues.apache.org/jira/browse/CASSANDRA-9644. > The test was started by dumping few days worth of data to the database for > 100 000 signals. Time graphs of SStables from different nodes indicates that > the DTCS has been working as expected and SStables are nicely ordered in time > wise. > See files: > node0_20150727_1250_time_graph.txt > node1_20150727_1250_time_graph.txt > node2_20150727_1250_time_graph.txt > Status=Up/Down > |/ State=Normal/Leaving/Joining/Moving > -- AddressLoad Tokens OwnsHost ID > Rack > UN 139.66.43.170 188.87 GB 256 ? > dfc29863-c935-4909-9d7f-c59a47eda03d rack1 > UN 139.66.43.169 198.37 GB 256 ? > 12e7628b-7f05-48f6-b7e4-35a82010021a rack1 > UN 139.66.43.168 191.88 GB 256 ? > 26088392-f803-4d59-9073-c75f857fb332 rack1 > All nodes crashed due to power failure (know beforehand) and repair > operations were started for each node one at the time. Below is the behavior > of SSTable count on different nodes. New data was dumped simultaneously with > repair operation. > SEE FIGURE: sstable_count_figure1.png > Vertical lines indicate following events. > 1) Cluster was down due to power shutdown and was restarted. At the first > vertical line the repair operation (nodetool repair -pr) was started for the > first node > 2) Repair for the second repair operation was started after the first node > was successfully repaired. > 3) Repair for the third repair operation was started > 4) Third repair operation was finished > 5) One of the nodes crashed (unknown reason in OS level) > 6) Repair operation (nodetool repair -pr) was started for the first node > 7) Repair operation for the second node was started > 8) Repair operation for the third node was s
[jira] [Commented] (CASSANDRA-9667) strongly consistent membership and ownership
[ https://issues.apache.org/jira/browse/CASSANDRA-9667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727993#comment-14727993 ] Jason Brown commented on CASSANDRA-9667: bq. Do you think this approach can be extended to allow more consistent schema changes like changing RF or altering a table? I think that would be more of a function of the underlying paxos/LWT/consensus alg (which may or may not be the existing LWT, still considering and debating), more so than the overall membership changes. But I would hope the consensus alg work here would apply to other efforts, as well! bq. Also, for your manual join, what kind of information can we give to users to allow them to evaluate the pending transaction? Initially I was only thinking of showing the minimal info: IP addr (or other host info) and possibly any token info (like if the node is replace another, or the operator is explicitly setting tokens). That being said, we could display any amount of info we choose - the initial set was only bounded by my imagination :). However, I really do like your idea about being able to determine the amount of data to be streamed to the new node - something like that should be a reasonably simple calculation and certainly helpful for operators. Note: I'm still ironing out the protocol and transition points, but let me post the updates in a short while. > strongly consistent membership and ownership > > > Key: CASSANDRA-9667 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9667 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jason Brown >Assignee: Jason Brown > Labels: LWT, membership, ownership > Fix For: 3.x > > > Currently, there is advice to users to "wait two minutes between adding new > nodes" in order for new node tokens, et al, to propagate. Further, as there's > no coordination amongst joining node wrt token selection, new nodes can end > up selecting ranges that overlap with other joining nodes. This causes a lot > of duplicate streaming from the existing source nodes as they shovel out the > bootstrap data for those new nodes. > This ticket proposes creating a mechanism that allows strongly consistent > membership and ownership changes in cassandra such that changes are performed > in a linearizable and safe manner. The basic idea is to use LWT operations > over a global system table, and leverage the linearizability of LWT for > ensuring the safety of cluster membership/ownership state changes. This work > is inspired by Riak's claimant module. > The existing workflows for node join, decommission, remove, replace, and > range move (there may be others I'm not thinking of) will need to be modified > to participate in this scheme, as well as changes to nodetool to enable them. > Note: we distinguish between membership and ownership in the following ways: > for membership we mean "a host in this cluster and it's state". For > ownership, we mean "what tokens (or ranges) does each node own"; these nodes > must already be a member to be assigned tokens. > A rough draft sketch of how the 'add new node' workflow might look like is: > new nodes would no longer create tokens themselves, but instead contact a > member of a Paxos cohort (via a seed). The cohort member will generate the > tokens and execute a LWT transaction, ensuring a linearizable change to the > membership/ownership state. The updated state will then be disseminated via > the existing gossip. > As for joining specifically, I think we could support two modes: auto-mode > and manual-mode. Auto-mode is for adding a single new node per LWT operation, > and would require no operator intervention (much like today). In manual-mode, > however, multiple new nodes could (somehow) signal their their intent to join > to the cluster, but will wait until an operator executes a nodetool command > that will trigger the token generation and LWT operation for all pending new > nodes. This will allow us better range partitioning and will make the > bootstrap streaming more efficient as we won't have overlapping range > requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10055) High CPU load for Cassandra 2.1.8
[ https://issues.apache.org/jira/browse/CASSANDRA-10055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727979#comment-14727979 ] vijay commented on CASSANDRA-10055: --- HI, Benedict controlling the number of coreConnPerHostLocal and maxConnPerHostLocal helped in cpu load issue we are facing. > High CPU load for Cassandra 2.1.8 > - > > Key: CASSANDRA-10055 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10055 > Project: Cassandra > Issue Type: Bug > Components: Config > Environment: Prod >Reporter: vijay > Labels: triaged > Fix For: 2.1.x > > Attachments: dstst-lcdn.log, dstst-lcdn2.log, dstst-lcdn3.log, > dstst-lcdn4.log, dstst-lcdn5.log, dstst-lcdn6.log, js.log, js2.log, js3.log, > js4.log, js5.log, js6.log, top-bHn1-2.log, top-bHn1-3.log, top-bHn1-4.log, > top-bHn1-5.log, top-bHn1-6.log, top-bHn1.log > > > We are seeing High CPU Load about 80% to 100% in Cassandra 2.1.8 when doing > Data ingest, we did not had this issue in 2.0.x version of Cassandra > we tested this in different Cloud platforms and results are same. > CPU: Tested with M3 2xlarge AWS instances > Ingest rate: Injecting 1 million Inserts each insert is of 1000bytes > no other Operations are happening except inserts in Cassandra > let me know if more info is needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10232) Small optimizations in index entry serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-10232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727948#comment-14727948 ] Ariel Weisberg commented on CASSANDRA-10232: bq. I suppose you mean it's not done yet. Can you point to the ticket that will do that for the records. Otherwise, is my understanding correct that you're just suggesting the comment may be removed? [The addition of checksums to saved cache files changed the cache version.|https://issues.apache.org/jira/browse/CASSANDRA-9265] > Small optimizations in index entry serialization > > > Key: CASSANDRA-10232 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10232 > Project: Cassandra > Issue Type: Improvement >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Fix For: 3.0.0 rc1 > > > While we should improve the data structure we use for our on-disk index in > future versions, it occurred to me that we had a few _very_ low hanging fruit > optimization (as in, for 3.0) we could do for the serialization of our > current entries, like using vint encodings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6237) Allow range deletions in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727915#comment-14727915
]
Joshua McKenzie commented on CASSANDRA-6237:
Rebase looks good as do subsequent patches and CI.
While testing some combinations and poking around at the patch, I ran into the
following:
{code:title=schema}
create table test3(a int, b float, c text, primary key(a,b));
{code}
{code:title=command and error}
cqlsh:deltest> update test3 set c = 'three' where a = 3 and b < 4;
ServerError:
{code}
Server error:
{noformat}
ERROR 19:49:09 Unexpected error during query
java.lang.UnsupportedOperationException: null
at
org.apache.cassandra.cql3.restrictions.SingleColumnRestriction$SliceRestriction.appendTo(SingleColumnRestriction.java:343)
~[main/:na]
at
org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.appendTo(PrimaryKeyRestrictionSet.java:189)
~[main/:na]
at
org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.valuesAsClustering(PrimaryKeyRestrictionSet.java:181)
~[main/:na]
at
org.apache.cassandra.cql3.restrictions.StatementRestrictions.getClusteringColumns(StatementRestrictions.java:580)
~[main/:na]
at
org.apache.cassandra.cql3.statements.ModificationStatement.createClustering(ModificationStatement.java:294)
~[main/:na]
at
org.apache.cassandra.cql3.statements.ModificationStatement.addUpdates(ModificationStatement.java:646)
~[main/:na]
at
org.apache.cassandra.cql3.statements.ModificationStatement.getMutations(ModificationStatement.java:606)
~[main/:na]
at
org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:413)
~[main/:na]
at
org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:401)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:204)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:235)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:220)
~[main/:na]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:123)
~[main/:na]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
[main/:na]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
[main/:na]
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_45]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
[main/:na]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[main/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
ERROR 19:49:09 Unexpected exception during request
java.lang.UnsupportedOperationException: null
at
org.apache.cassandra.cql3.restrictions.SingleColumnRestriction$SliceRestriction.appendTo(SingleColumnRestriction.java:343)
~[main/:na]
at
org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.appendTo(PrimaryKeyRestrictionSet.java:189)
~[main/:na]
at
org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.valuesAsClustering(PrimaryKeyRestrictionSet.java:181)
~[main/:na]
at
org.apache.cassandra.cql3.restrictions.StatementRestrictions.getClusteringColumns(StatementRestrictions.java:580)
~[main/:na]
at
org.apache.cassandra.cql3.statements.ModificationStatement.createClustering(ModificationStatement.java:294)
~[main/:na]
at
org.apache.cassandra.cql3.statements.ModificationStatement.addUpdates(ModificationStatement.java:646)
~[main/:na]
at
org.apache.cassandra.cql3.statements.ModificationStatement.getMutations(ModificationStatement.java:606)
~[main/:na]
at
org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:413)
~[main/:na]
at
org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:401)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:20
[jira] [Commented] (CASSANDRA-10136) startup error after upgrade to 3.0
[
https://issues.apache.org/jira/browse/CASSANDRA-10136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727891#comment-14727891
]
Branimir Lambov commented on CASSANDRA-10136:
-
Code looks good, with a couple of nits:
[CompositeType.lastEOC|https://github.com/pcmanus/cassandra/commit/a04c881cc1a73b2bc2e53da425168370aa535dbc#diff-c24fe7f68164af00f4e94f6408e62e1fR205]
doesn't look right: {{get(int)}} ignores the position, yet you're using
{{remaining()}} which includes it. I see the position is supposed to be fixed
as 0, but even so it makes sense to use {{limit()}} instead to avoid
brittleness and confusing people reading the code.
[IndexState.updateBlock|https://github.com/pcmanus/cassandra/commit/4c4ca413e716267112f2d901f3276742a8cbca81#diff-fb1874f891c1a014fb57f8b4e42b5247R466]:
In addition to being more efficient to execute, fewer branches make the code
easier to follow and understand; I'd rephrase that construct as
{code}
if (startOfBlock == currentFilePointer)
mark = reader.file.mark();
else
{
reader.seekToPosition(startOfBlock);
mark = reader.file.mark();
reader.seekToPosition(currentFilePointer);
}
{code}
> startup error after upgrade to 3.0
> --
>
> Key: CASSANDRA-10136
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10136
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Russ Hatch
>Assignee: Sylvain Lebresne
> Fix For: 3.0 beta 2
>
>
> Encountering this error after a node is upgraded to 3.0 HEAD.
> This is a rolling upgrade test, where a second node (of three) has been
> upgraded to 3.0.
> {noformat}
> ERROR [main] 2015-08-19 17:30:16,153 CassandraDaemon.java:635 - Exception
> encountered during startup
> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> java.lang.UnsupportedOperationException
> at
> org.apache.cassandra.hints.LegacyHintsMigrator.forceCompaction(LegacyHintsMigrator.java:119)
> ~[main/:na]
> at
> org.apache.cassandra.hints.LegacyHintsMigrator.compactLegacyHints(LegacyHintsMigrator.java:108)
> ~[main/:na]
> at
> org.apache.cassandra.hints.LegacyHintsMigrator.migrate(LegacyHintsMigrator.java:92)
> ~[main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:281)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:516)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:622)
> [main/:na]
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.UnsupportedOperationException
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> ~[na:1.8.0_45]
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> ~[na:1.8.0_45]
> at
> org.apache.cassandra.hints.LegacyHintsMigrator.forceCompaction(LegacyHintsMigrator.java:115)
> ~[main/:na]
> ... 5 common frames omitted
> Caused by: java.lang.UnsupportedOperationException: null
> at
> org.apache.cassandra.db.Serializers.clusteringPrefixSerializer(Serializers.java:52)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.IndexHelper$IndexInfo$Serializer.deserialize(IndexHelper.java:171)
> ~[main/:na]
> at
> org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:150)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:286)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:260)
> ~[main/:na]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableScanner.hasNext(BigTableScanner.java:240)
> ~[main/:na]
> at
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:439)
> ~[main/:na]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[main/:na]
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$5.hasNext(UnfilteredPartitionIterators.java:234)
> ~[main/:na]
> at
> org.apache.cassandra.db.partitions.WrappingUnfilteredPartitionIterator.prepareNext(WrappingUnfilteredPartitionIterator.java:71)
> ~[main/:na]
> at
> org.apache.cassandra.db.partitions.WrappingUnfilteredPartitionIterator.hasNext(WrappingUnfilteredPartitionIterator.java:55)
> ~[main/:na]
> at
> org.apache.cassandra.db.partitions.PurgingPartitionIterator.hasNext(PurgingPartition
[jira] [Comment Edited] (CASSANDRA-9590) Support for both encrypted and unencrypted native transport connections
[
https://issues.apache.org/jira/browse/CASSANDRA-9590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727870#comment-14727870
]
Robert Stupp edited comment on CASSANDRA-9590 at 9/2/15 7:10 PM:
-
Patch and tests look good so far.
Some notes:
* Can you add the option {{native_transport_port_ssl}} to
{{conf/cassandra.yaml}} (commented out, but with some words describing its
meaning and how it relates to {{native_transport_port}})? You can use {{9142}}
as the (commented out) standard port. Maybe also a note that it's beneficial to
install the _Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction
Policy Files_?
* Let startup fail, if both {{native_transport_port}} and
{{native_transport_port_ssl}} are set but {{client_encryption_options}} is not
enabled. It is a configuration failure. At the moment it silently just not
starts SSL at all.
* The unit tests look good, but never start NetworkTransportService with SSL
enabled - but that's ok as there are dtests.
* dtests unfortunately don't work on my machine. Is the {{keystone.jks}} file
mentioned in the test source missing? (Ping me, if you need some logs or so.)
I tested the stuff manually using a self-signed cert with cqlsh and it works
(with JCE policy files).
EDIT: Forgot to mention: please open a PR for the dtest as soon as it is
running. Just put a note in the comment not to merge before this ticket is
committed. You can remove the {{@require}} annotation.
was (Author: snazy):
Patch and tests look good so far.
Some notes:
* Can you add the option {{native_transport_port_ssl}} to
{{conf/cassandra.yaml}} (commented out, but with some words describing its
meaning and how it relates to {{native_transport_port}})? You can use {{9142}}
as the (commented out) standard port. Maybe also a note that it's beneficial to
install the _Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction
Policy Files_?
* Let startup fail, if both {{native_transport_port}} and
{{native_transport_port_ssl}} are set but {{client_encryption_options}} is not
enabled. It is a configuration failure. At the moment it silently just not
starts SSL at all.
* The unit tests look good, but never start NetworkTransportService with SSL
enabled - but that's ok as there are dtests.
* dtests unfortunately don't work on my machine. Is the {{keystone.jks}} file
mentioned in the test source missing? (Ping me, if you need some logs or so.)
I tested the stuff manually using a self-signed cert with cqlsh and it works
(with JCE policy files).
> Support for both encrypted and unencrypted native transport connections
> ---
>
> Key: CASSANDRA-9590
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9590
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 2.1.x
>
>
> Enabling encryption for native transport currently turns SSL exclusively on
> or off for the opened socket. Migrating from plain to encrypted requires to
> migrate all native clients as well and redeploy all of them at the same time
> after starting the SSL enabled Cassandra nodes.
> This patch would allow to start Cassandra with both an unencrypted and ssl
> enabled native port. Clients can connect to either, based whether they
> support ssl or not.
> This has been implemented by introducing a new {{native_transport_port_ssl}}
> config option.
> There would be three scenarios:
> * client encryption disabled, {{native_transport_port}} unencrypted,
> {{native_transport_port_ssl}} not used
> * client encryption enabled, {{native_transport_port_ssl}} not set,
> {{native_transport_port}} encrypted
> * client encryption enabled, {{native_transport_port_ssl}} set,
> {{native_transport_port}} unencrypted, {{native_transport_port_ssl}} encrypted
> This approach would keep configuration behavior fully backwards compatible.
> Patch proposal:
> [Branch|https://github.com/spodkowinski/cassandra/tree/cassandra-9590], [Diff
> cassandra-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...spodkowinski:cassandra-9590],
> [Patch against
> cassandra-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...spodkowinski:cassandra-9590.patch]
> DTest:
> [Branch|https://github.com/spodkowinski/cassandra-dtest/tree/cassandra-9590],
> [Diff
> master|https://github.com/riptano/cassandra-dtest/compare/master...spodkowinski:cassandra-9590]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9590) Support for both encrypted and unencrypted native transport connections
[
https://issues.apache.org/jira/browse/CASSANDRA-9590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727870#comment-14727870
]
Robert Stupp commented on CASSANDRA-9590:
-
Patch and tests look good so far.
Some notes:
* Can you add the option {{native_transport_port_ssl}} to
{{conf/cassandra.yaml}} (commented out, but with some words describing its
meaning and how it relates to {{native_transport_port}})? You can use {{9142}}
as the (commented out) standard port. Maybe also a note that it's beneficial to
install the _Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction
Policy Files_?
* Let startup fail, if both {{native_transport_port}} and
{{native_transport_port_ssl}} are set but {{client_encryption_options}} is not
enabled. It is a configuration failure. At the moment it silently just not
starts SSL at all.
* The unit tests look good, but never start NetworkTransportService with SSL
enabled - but that's ok as there are dtests.
* dtests unfortunately don't work on my machine. Is the {{keystone.jks}} file
mentioned in the test source missing? (Ping me, if you need some logs or so.)
I tested the stuff manually using a self-signed cert with cqlsh and it works
(with JCE policy files).
> Support for both encrypted and unencrypted native transport connections
> ---
>
> Key: CASSANDRA-9590
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9590
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 2.1.x
>
>
> Enabling encryption for native transport currently turns SSL exclusively on
> or off for the opened socket. Migrating from plain to encrypted requires to
> migrate all native clients as well and redeploy all of them at the same time
> after starting the SSL enabled Cassandra nodes.
> This patch would allow to start Cassandra with both an unencrypted and ssl
> enabled native port. Clients can connect to either, based whether they
> support ssl or not.
> This has been implemented by introducing a new {{native_transport_port_ssl}}
> config option.
> There would be three scenarios:
> * client encryption disabled, {{native_transport_port}} unencrypted,
> {{native_transport_port_ssl}} not used
> * client encryption enabled, {{native_transport_port_ssl}} not set,
> {{native_transport_port}} encrypted
> * client encryption enabled, {{native_transport_port_ssl}} set,
> {{native_transport_port}} unencrypted, {{native_transport_port_ssl}} encrypted
> This approach would keep configuration behavior fully backwards compatible.
> Patch proposal:
> [Branch|https://github.com/spodkowinski/cassandra/tree/cassandra-9590], [Diff
> cassandra-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...spodkowinski:cassandra-9590],
> [Patch against
> cassandra-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...spodkowinski:cassandra-9590.patch]
> DTest:
> [Branch|https://github.com/spodkowinski/cassandra-dtest/tree/cassandra-9590],
> [Diff
> master|https://github.com/riptano/cassandra-dtest/compare/master...spodkowinski:cassandra-9590]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-9952) UDF with no parameters prevents cqlsh DESCRIBE from working
[ https://issues.apache.org/jira/browse/CASSANDRA-9952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727860#comment-14727860 ] Ryan McGuire edited comment on CASSANDRA-9952 at 9/2/15 6:56 PM: - This looks to be a python driver only issue. Attached is a test that exemplifies it outside of cqlsh. /cc [~aholmber] if he thinks I'm correct and if there's a known bug opened for this driver side. was (Author: enigmacurry): This looks to be a python driver only issue. Attached is a test that exemplifies it outside of cqlsh. > UDF with no parameters prevents cqlsh DESCRIBE from working > --- > > Key: CASSANDRA-9952 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9952 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: ubuntu 64 bit, using ccm tool with a one node cluster, > release 2.2.0 >Reporter: Jim Meyer >Priority: Minor > Fix For: 2.2.x > > Attachments: test_9952.py > > > If I create a user defined function that takes no parameters like this: > cqlsh:test> CREATE FUNCTION no_parm() CALLED ON NULL INPUT RETURNS bigint > LANGUAGE java AS 'return System.currentTimeMillis() / 1000L;'; > The function works fine in queries, but in cqlsh the describe command stops > working: > cqlsh:test> DESC KEYSPACE test; > izip argument #1 must support iteration > If I drop the function, then describe starts working normally again. It > appears describe assumes there is at least one argument for UDF's, but > creating and using the functions does not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9952) UDF with no parameters prevents cqlsh DESCRIBE from working
[ https://issues.apache.org/jira/browse/CASSANDRA-9952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McGuire updated CASSANDRA-9952: Attachment: test_9952.py This looks to be a python driver only issue. Attached is a test that exemplifies it outside of cqlsh. > UDF with no parameters prevents cqlsh DESCRIBE from working > --- > > Key: CASSANDRA-9952 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9952 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: ubuntu 64 bit, using ccm tool with a one node cluster, > release 2.2.0 >Reporter: Jim Meyer >Priority: Minor > Fix For: 2.2.x > > Attachments: test_9952.py > > > If I create a user defined function that takes no parameters like this: > cqlsh:test> CREATE FUNCTION no_parm() CALLED ON NULL INPUT RETURNS bigint > LANGUAGE java AS 'return System.currentTimeMillis() / 1000L;'; > The function works fine in queries, but in cqlsh the describe command stops > working: > cqlsh:test> DESC KEYSPACE test; > izip argument #1 must support iteration > If I drop the function, then describe starts working normally again. It > appears describe assumes there is at least one argument for UDF's, but > creating and using the functions does not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9459) SecondaryIndex API redesign
[
https://issues.apache.org/jira/browse/CASSANDRA-9459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727859#comment-14727859
]
Andrew Hust commented on CASSANDRA-9459:
When creating separate indexes on both the key and value of a map column the
ddl for the table in cqlsh only contains the index on the value. Both indexes
are functional and queries return expected results. When querying metadata
from the python driver (3.0.0a2) both indexes are present and using the
function as_cql_query produces the correct ddl. This might just be an out of
date python lib in cqlsh.
Tested on C*: {{66b0e1d7889d0858753c6e364e77d86fe278eee4}}
Can be reproduced with the following shell commands and ccm:
{code}
ccm remove 2i_test
ccm create -n 1 -v git:cassandra-3.0 -s 2i_test
ccm start
cat << EOF | ccm node1 cqlsh
CREATE KEYSPACE index_test_ks WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 1};
USE index_test_ks;
CREATE TABLE tbl1 (id uuid primary key, ds map, c1 int);
INSERT INTO tbl1 (id, ds, c1) values (uuid(), {'foo': 1, 'bar': 2}, 1);
INSERT INTO tbl1 (id, ds, c1) values (uuid(), {'faz': 1, 'baz': 2}, 2);
CREATE INDEX ix_tbl1_map_values ON tbl1(ds);
CREATE INDEX ix_tbl1_map_keys ON tbl1(keys(ds));
SELECT * FROM tbl1 where ds contains 1;
SELECT * FROM tbl1 where ds contains key 'foo';
// ***
// DDL only has ix_tbl1_map_values present
// ***
DESC TABLE tbl1;
// ***
// system_schema.indexes is correct
// ***
SELECT * FROM system_schema.indexes;
EOF
ccm stop
{code}
Example output:
{code}
CREATE TABLE index_test_ks.tbl1 (
id uuid PRIMARY KEY,
c1 int,
ds map
) WITH bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
CREATE INDEX ix_tbl1_map_values ON index_test_ks.tbl1 (ds);
keyspace_name | table_name | index_name | index_type | options
| target_columns | target_type
---++++--++-
index_test_ks | tbl1 | ix_tbl1_map_keys | COMPOSITES |
{'index_keys': ''} | {'ds'} | COLUMN
index_test_ks | tbl1 | ix_tbl1_map_values | COMPOSITES |
{'index_values': ''} | {'ds'} | COLUMN
{code}
> SecondaryIndex API redesign
> ---
>
> Key: CASSANDRA-9459
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9459
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
> Fix For: 3.0 beta 1
>
>
> For some time now the index subsystem has been a pain point and in large part
> this is due to the way that the APIs and principal classes have grown
> organically over the years. It would be a good idea to conduct a wholesale
> review of the area and see if we can come up with something a bit more
> coherent.
> A few starting points:
> * There's a lot in AbstractPerColumnSecondaryIndex & its subclasses which
> could be pulled up into SecondaryIndexSearcher (note that to an extent, this
> is done in CASSANDRA-8099).
> * SecondayIndexManager is overly complex and several of its functions should
> be simplified/re-examined. The handling of which columns are indexed and
> index selection on both the read and write paths are somewhat dense and
> unintuitive.
> * The SecondaryIndex class hierarchy is rather convoluted and could use some
> serious rework.
> There are a number of outstanding tickets which we should be able to roll
> into this higher level one as subtasks (but I'll defer doing that until
> getting into the details of the redesign):
> * CASSANDRA-7771
> * CASSANDRA-8103
> * CASSANDRA-9041
> * CASSANDRA-4458
> * CASSANDRA-8505
> Whilst they're not hard dependencies, I propose that this be done on top of
> both CASSANDRA-8099 and CASSANDRA-6717. The former largely because the
> storage engine changes may facilitate a friendlier index API, but also
> because of the changes to SIS mentioned above. As for 6717, the changes to
> schema tables there will help facilitate CASSANDRA-7771.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10232) Small optimizations in index entry serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-10232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727821#comment-14727821 ] Ariel Weisberg commented on CASSANDRA-10232: +1. Our process does specify we wait until CASSANDRA-10136 is done so I would prefer doing that unless there a reason waiting would be a problem. > Small optimizations in index entry serialization > > > Key: CASSANDRA-10232 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10232 > Project: Cassandra > Issue Type: Improvement >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Fix For: 3.0.0 rc1 > > > While we should improve the data structure we use for our on-disk index in > future versions, it occurred to me that we had a few _very_ low hanging fruit > optimization (as in, for 3.0) we could do for the serialization of our > current entries, like using vint encodings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10147) Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation
[
https://issues.apache.org/jira/browse/CASSANDRA-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727771#comment-14727771
]
Carl Yeksigian commented on CASSANDRA-10147:
Pushed an update with a test.
> Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation
> ---
>
> Key: CASSANDRA-10147
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10147
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Jonathan Ellis
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 3.0.0 rc1
>
>
> {code}
> CREATE TABLE users (
> id uuid PRIMARY KEY,
> username text,
> email text,
> age int
> );
> CREATE MATERIALIZED VIEW users_by_username AS SELECT * FROM users WHERE
> username IS NOT NULL PRIMARY KEY (username, id);
> InvalidRequest: code=2200 [Invalid query] message="Primary key column 'id' is
> required to be filtered by 'IS NOT NULL'"
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
cassandra git commit: Fix SSTableLoaderTest on Windows
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.0 6a86d221a -> 66b0e1d78
Fix SSTableLoaderTest on Windows
Patch by pmotta; reviewed by jmckenzie for CASSANDRA-10210
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/66b0e1d7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/66b0e1d7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/66b0e1d7
Branch: refs/heads/cassandra-3.0
Commit: 66b0e1d7889d0858753c6e364e77d86fe278eee4
Parents: 6a86d22
Author: Paulo Motta
Authored: Wed Sep 2 14:20:00 2015 -0400
Committer: Joshua McKenzie
Committed: Wed Sep 2 14:20:00 2015 -0400
--
.../cassandra/io/sstable/SSTableLoaderTest.java | 14 +-
1 file changed, 13 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/66b0e1d7/test/unit/org/apache/cassandra/io/sstable/SSTableLoaderTest.java
--
diff --git a/test/unit/org/apache/cassandra/io/sstable/SSTableLoaderTest.java
b/test/unit/org/apache/cassandra/io/sstable/SSTableLoaderTest.java
index 3370e56..4eebdeb 100644
--- a/test/unit/org/apache/cassandra/io/sstable/SSTableLoaderTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/SSTableLoaderTest.java
@@ -38,6 +38,7 @@ import org.apache.cassandra.db.partitions.*;
import org.apache.cassandra.db.marshal.AsciiType;
import org.apache.cassandra.dht.Range;
import org.apache.cassandra.dht.Token;
+import org.apache.cassandra.io.FSWriteError;
import org.apache.cassandra.io.util.FileUtils;
import org.apache.cassandra.schema.KeyspaceParams;
import org.apache.cassandra.service.StorageService;
@@ -80,7 +81,18 @@ public class SSTableLoaderTest
@After
public void cleanup()
{
-FileUtils.deleteRecursive(tmpdir);
+try {
+FileUtils.deleteRecursive(tmpdir);
+} catch (FSWriteError e) {
+/**
+ * Windows does not allow a mapped file to be deleted, so we
probably forgot to clean the buffers somewhere.
+ * We force a GC here to force buffer deallocation, and then try
deleting the directory again.
+ * For more information, see:
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=4715154
+ * If this is not the problem, the exception will be rethrown
anyway.
+ */
+System.gc();
+FileUtils.deleteRecursive(tmpdir);
+}
}
private static final class TestClient extends SSTableLoader.Client
cassandra git commit: Fix SSTableLoaderTest on Windows
Repository: cassandra
Updated Branches:
refs/heads/trunk f66730d61 -> 966bea74b
Fix SSTableLoaderTest on Windows
Patch by pmotta; reviewed by jmckenzie for CASSANDRA-10210
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/966bea74
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/966bea74
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/966bea74
Branch: refs/heads/trunk
Commit: 966bea74bd031f92f953fa66bb1d6da3867b06b9
Parents: f66730d
Author: Paulo Motta
Authored: Wed Sep 2 14:20:00 2015 -0400
Committer: Joshua McKenzie
Committed: Wed Sep 2 14:21:03 2015 -0400
--
.../cassandra/io/sstable/SSTableLoaderTest.java | 14 +-
1 file changed, 13 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/966bea74/test/unit/org/apache/cassandra/io/sstable/SSTableLoaderTest.java
--
diff --git a/test/unit/org/apache/cassandra/io/sstable/SSTableLoaderTest.java
b/test/unit/org/apache/cassandra/io/sstable/SSTableLoaderTest.java
index 3370e56..4eebdeb 100644
--- a/test/unit/org/apache/cassandra/io/sstable/SSTableLoaderTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/SSTableLoaderTest.java
@@ -38,6 +38,7 @@ import org.apache.cassandra.db.partitions.*;
import org.apache.cassandra.db.marshal.AsciiType;
import org.apache.cassandra.dht.Range;
import org.apache.cassandra.dht.Token;
+import org.apache.cassandra.io.FSWriteError;
import org.apache.cassandra.io.util.FileUtils;
import org.apache.cassandra.schema.KeyspaceParams;
import org.apache.cassandra.service.StorageService;
@@ -80,7 +81,18 @@ public class SSTableLoaderTest
@After
public void cleanup()
{
-FileUtils.deleteRecursive(tmpdir);
+try {
+FileUtils.deleteRecursive(tmpdir);
+} catch (FSWriteError e) {
+/**
+ * Windows does not allow a mapped file to be deleted, so we
probably forgot to clean the buffers somewhere.
+ * We force a GC here to force buffer deallocation, and then try
deleting the directory again.
+ * For more information, see:
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=4715154
+ * If this is not the problem, the exception will be rethrown
anyway.
+ */
+System.gc();
+FileUtils.deleteRecursive(tmpdir);
+}
}
private static final class TestClient extends SSTableLoader.Client
[jira] [Commented] (CASSANDRA-10222) Periodically attempt to delete failed snapshot deletions on Windows
[
https://issues.apache.org/jira/browse/CASSANDRA-10222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727734#comment-14727734
]
Joshua McKenzie commented on CASSANDRA-10222:
-
The redundant run call was an artifact of me changing that interface and then
not actually reading back through it again from that perspective - bad form on
my part. I've refactored the constructor on SnapshotDeletingTask a bit and
broke out the task creation to an {{addFailedSnapshot}} method - I think it
cleans that interface up quite a bit; let me know what you think on that front.
I'm pretty sure all compactions go through the CompactionExecutor; this change
actually gets us a tiny bit of *over* deletion attempts as
{{CompactionManager.ValidationExecutor}} and
{{CompactionManager.CacheCleanupExecutor}} are both going to rely on
{{CompactionExecutor.afterExecute}}, running the
{{SnapshotDeletingTask.rescheduleFailedTasks}}, but I think the cost of
refactoring those classes isn't worth it just to try and eliminate rare
potential no-op task removal/re-add on a snapshot deletion that's not ready yet.
I've gone ahead and manually set up some CI jobs to run on Windows:
[2.2
utest|http://cassci.datastax.com/view/Dev/view/josh-mckenzie/job/10222_2.2_utest_win32/]
[2.2
dtest|http://cassci.datastax.com/view/Dev/view/josh-mckenzie/job/10222_2.2_dtest_win32/]
[3.0
utest|http://cassci.datastax.com/view/Dev/view/josh-mckenzie/job/10222_3.0_utest_win32/]
As dtest runs on the platform are currently 10+ hours, I've limited us to 2.2
only at this time. I can create and run a 3.0 job if you're concerned about it,
however with Windows-specific changes like this (and 3.0 being in beta) I tend
to be a *little* less stringent on running the full CI gamut than I would
otherwise be.
> Periodically attempt to delete failed snapshot deletions on Windows
> ---
>
> Key: CASSANDRA-10222
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10222
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Joshua McKenzie
>Assignee: Joshua McKenzie
> Labels: Windows
> Fix For: 2.2.2
>
>
> The changes in CASSANDRA-9658 leave us in a position where a node on Windows
> will have to be restarted to clear out snapshots that cannot be deleted at
> request time due to sstables still being mapped, thus preventing deletions of
> hard links. A simple periodic task to categorize failed snapshot deletions
> and retry them would help prevent node disk utilization from growing
> unbounded by snapshots as compaction will eventually make these snapshot
> files deletable.
> Given that hard links to files in NTFS don't take up any extra space on disk
> so long as the original file still exists, the only limitation for users from
> this approach will be the inability to 'move' a snapshot file to another
> drive share. They will be copyable, however, so it's a minor platform
> difference.
> This goes directly against the goals of CASSANDRA-8271 and will likely be
> built on top of that code. Until such time as we get buffered performance
> in-line with memory-mapped, this is an interim necessity for production
> roll-outs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10161) Composite case-sensitive primary key: first item is not quoted in DESCRIBE TABLE
[
https://issues.apache.org/jira/browse/CASSANDRA-10161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727708#comment-14727708
]
T Jake Luciani commented on CASSANDRA-10161:
Let's wait till the 3.0 driver to have the fix /cc [~aholmber]
> Composite case-sensitive primary key: first item is not quoted in DESCRIBE
> TABLE
>
>
> Key: CASSANDRA-10161
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10161
> Project: Cassandra
> Issue Type: Bug
>Reporter: Julien Moreau
>Assignee: Carl Yeksigian
>Priority: Minor
> Labels: cqlsh
> Fix For: 2.1.x
>
>
> A table is created with case-sensitive composite primary key:
> {code}
> CREATE TABLE foo (
> "Key1" text,
> "Key2" text,
> "Value" text
> PRIMARY KEY ("Key1", "Key2")
> );
> {code}
> In the result of {{DESCRIBE TABLE foo;}}:
> {code}
> CREATE TABLE foo (
> "Key1" text,
> "Key2" text,
> "Value" text
> PRIMARY KEY (Key1, "Key2")
> );
> {code}
> {{Key1}} is not quoted.
> When trying to re-create the table with this description, there is an error:
> {code}
> InvalidRequest: code=2200 [Invalid query] message="Unknown definition key1
> referenced in PRIMARY KEY"
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10249) Reduce over-read for standard disk io by 16x
[ https://issues.apache.org/jira/browse/CASSANDRA-10249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727703#comment-14727703 ] Jonathan Shook commented on CASSANDRA-10249: I'm not so sure that this is a niche. Compression is not a default win, and I'd prefer that it be "unset" and require users to pick "compressed" or "uncompressed" in the DDL. But we don't do that. So, compressed is a default. Still, uncompressed is not quite a niche. I'm less sure about the buffered IO angle. If these are reasonable options for some scenarios, then I don't feel quite right calling them niche. One persons niche is another's standard. For those that need these settings to get the most of their current hardware, the large minimum read size is, in fact, a deoptimization from normal. > Reduce over-read for standard disk io by 16x > > > Key: CASSANDRA-10249 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10249 > Project: Cassandra > Issue Type: Improvement >Reporter: Albert P Tobey > Fix For: 2.1.x > > Attachments: patched-2.1.9-dstat-lvn10.png, > stock-2.1.9-dstat-lvn10.png, yourkit-screenshot.png > > > On read workloads, Cassandra 2.1 reads drastically more data than it emits > over the network. This causes problems throughput the system by wasting disk > IO and causing unnecessary GC. > I have reproduce the issue on clusters and locally with a single instance. > The only requirement to reproduce the issue is enough data to blow through > the page cache. The default schema and data size with cassandra-stress is > sufficient for exposing the issue. > With stock 2.1.9 I regularly observed anywhere from 300:1 to 500 > disk:network ratio. That is to say, for 1MB/s of network IO, Cassandra was > doing 300-500MB/s of disk reads, saturating the drive. > After applying this patch for standard IO mode > https://gist.github.com/tobert/10c307cf3709a585a7cf the ratio fell to around > 100:1 on my local test rig. Latency improved considerably and GC became a lot > less frequent. > I tested with 512 byte reads as well, but got the same performance, which > makes sense since all HDD and SSD made in the last few years have a 4K block > size (many of them lie and say 512). > I'm re-running the numbers now and will post them tomorrow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10127) Make naming for secondary indexes consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-10127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727701#comment-14727701
]
T Jake Luciani commented on CASSANDRA-10127:
+1 Let's just add a note on the nodetool change in NEWS.txt
> Make naming for secondary indexes consistent
>
>
> Key: CASSANDRA-10127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10127
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sam Tunnicliffe
>Assignee: Benjamin Lerer
> Fix For: 3.0.0 rc1
>
> Attachments: 10127.txt
>
>
> We have a longstanding mismatch between the name of an index as defined in
> schema and what gets returned from {{SecondaryIndex#getIndexName()}}, which
> for the builtin index impls is the name of the underlying index CFS, of the
> form {{.}}.
> This mismatch causes a number of UI inconsistencies:
> {code}nodetool rebuild_index {code}
> {{}} must be qualified, i.e. include the redundant table name as without
> it, the rebuild silently fails
> {{system.IndexInfo}} (which is also exposed over JMX) uses the form
> {{.}}
> {code}cqlsh> describe index [.]{code}
> here, qualifying {{}} with the base table name is an error.
> Generally, anything CQL related uses the index name directly, whereas anthing
> concerned with building or rebuiling requires the version based on an
> underlying backing table name.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10147) Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation
[
https://issues.apache.org/jira/browse/CASSANDRA-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727668#comment-14727668
]
T Jake Luciani edited comment on CASSANDRA-10147 at 9/2/15 5:24 PM:
LGTM. Just add a test in case the logic isn't undone.
was (Author: tjake):
LGTM. Just add a test in case the logic is undone.
> Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation
> ---
>
> Key: CASSANDRA-10147
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10147
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Jonathan Ellis
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 3.0.0 rc1
>
>
> {code}
> CREATE TABLE users (
> id uuid PRIMARY KEY,
> username text,
> email text,
> age int
> );
> CREATE MATERIALIZED VIEW users_by_username AS SELECT * FROM users WHERE
> username IS NOT NULL PRIMARY KEY (username, id);
> InvalidRequest: code=2200 [Invalid query] message="Primary key column 'id' is
> required to be filtered by 'IS NOT NULL'"
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10147) Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation
[
https://issues.apache.org/jira/browse/CASSANDRA-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727668#comment-14727668
]
T Jake Luciani commented on CASSANDRA-10147:
LGTM. Just add a test in case the logic is undone.
> Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation
> ---
>
> Key: CASSANDRA-10147
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10147
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Jonathan Ellis
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 3.0.0 rc1
>
>
> {code}
> CREATE TABLE users (
> id uuid PRIMARY KEY,
> username text,
> email text,
> age int
> );
> CREATE MATERIALIZED VIEW users_by_username AS SELECT * FROM users WHERE
> username IS NOT NULL PRIMARY KEY (username, id);
> InvalidRequest: code=2200 [Invalid query] message="Primary key column 'id' is
> required to be filtered by 'IS NOT NULL'"
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8630) Faster sequential IO (on compaction, streaming, etc)
[ https://issues.apache.org/jira/browse/CASSANDRA-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727634#comment-14727634 ] Philip Thompson edited comment on CASSANDRA-8630 at 9/2/15 4:57 PM: Windows CI: Dtest http://cassci.datastax.com/view/win32/job/stef1927-8630_dtest_win32/ Utest http://cassci.datastax.com/view/win32/job/stef1927-8630_utest_win32/ I based these on https://github.com/stef1927/cassandra/tree/8630-3.0 was (Author: philipthompson): Windows CI: Dtest http://cassci.datastax.com/view/win32/job/stef1927-8630_dtest_win32/ Utest http://cassci.datastax.com/view/win32/job/stef1927-8630_utest_win32/ > Faster sequential IO (on compaction, streaming, etc) > > > Key: CASSANDRA-8630 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8630 > Project: Cassandra > Issue Type: Improvement > Components: Core, Tools >Reporter: Oleg Anastasyev >Assignee: Stefania > Labels: compaction, performance > Fix For: 3.x > > Attachments: 8630-FasterSequencialReadsAndWrites.txt, cpu_load.png, > flight_recorder_001_files.tar.gz, flight_recorder_002_files.tar.gz, > mmaped_uncomp_hotspot.png > > > When node is doing a lot of sequencial IO (streaming, compacting, etc) a lot > of CPU is lost in calls to RAF's int read() and DataOutputStream's write(int). > This is because default implementations of readShort,readLong, etc as well as > their matching write* are implemented with numerous calls of byte by byte > read and write. > This makes a lot of syscalls as well. > A quick microbench shows than just reimplementation of these methods in > either way gives 8x speed increase. > A patch attached implements RandomAccessReader.read and > SequencialWriter.write methods in more efficient way. > I also eliminated some extra byte copies in CompositeType.split and > ColumnNameHelper.maxComponents, which were on my profiler's hotspot method > list during tests. > A stress tests on my laptop show that this patch makes compaction 25-30% > faster on uncompressed sstables and 15% faster for compressed ones. > A deployment to production shows much less CPU load for compaction. > (I attached a cpu load graph from one of our production, orange is niced CPU > load - i.e. compaction; yellow is user - i.e. not compaction related tasks) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8630) Faster sequential IO (on compaction, streaming, etc)
[ https://issues.apache.org/jira/browse/CASSANDRA-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727634#comment-14727634 ] Philip Thompson commented on CASSANDRA-8630: Windows CI: Dtest http://cassci.datastax.com/view/win32/job/stef1927-8630_dtest_win32/ Utest http://cassci.datastax.com/view/win32/job/stef1927-8630_utest_win32/ > Faster sequential IO (on compaction, streaming, etc) > > > Key: CASSANDRA-8630 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8630 > Project: Cassandra > Issue Type: Improvement > Components: Core, Tools >Reporter: Oleg Anastasyev >Assignee: Stefania > Labels: compaction, performance > Fix For: 3.x > > Attachments: 8630-FasterSequencialReadsAndWrites.txt, cpu_load.png, > flight_recorder_001_files.tar.gz, flight_recorder_002_files.tar.gz, > mmaped_uncomp_hotspot.png > > > When node is doing a lot of sequencial IO (streaming, compacting, etc) a lot > of CPU is lost in calls to RAF's int read() and DataOutputStream's write(int). > This is because default implementations of readShort,readLong, etc as well as > their matching write* are implemented with numerous calls of byte by byte > read and write. > This makes a lot of syscalls as well. > A quick microbench shows than just reimplementation of these methods in > either way gives 8x speed increase. > A patch attached implements RandomAccessReader.read and > SequencialWriter.write methods in more efficient way. > I also eliminated some extra byte copies in CompositeType.split and > ColumnNameHelper.maxComponents, which were on my profiler's hotspot method > list during tests. > A stress tests on my laptop show that this patch makes compaction 25-30% > faster on uncompressed sstables and 15% faster for compressed ones. > A deployment to production shows much less CPU load for compaction. > (I attached a cpu load graph from one of our production, orange is niced CPU > load - i.e. compaction; yellow is user - i.e. not compaction related tasks) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10109) Windows dtest 3.0: ttl_test.py failures
[
https://issues.apache.org/jira/browse/CASSANDRA-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727622#comment-14727622
]
Philip Thompson commented on CASSANDRA-10109:
-
Windows CI:
Dtest http://cassci.datastax.com/view/win32/job/stef1927-10109_dtest_win32/
Utest http://cassci.datastax.com/view/win32/job/stef1927-10109_utest_win32/
> Windows dtest 3.0: ttl_test.py failures
> ---
>
> Key: CASSANDRA-10109
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10109
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Joshua McKenzie
>Assignee: Stefania
> Labels: Windows
> Fix For: 3.0.0 rc1
>
>
> ttl_test.py:TestTTL.update_column_ttl_with_default_ttl_test2
> ttl_test.py:TestTTL.update_multiple_columns_ttl_test
> ttl_test.py:TestTTL.update_single_column_ttl_test
> Errors locally are different than CI from yesterday. Yesterday on CI we have
> timeouts and general node hangs. Today on all 3 tests when run locally I see:
> {noformat}
> Traceback (most recent call last):
> File "c:\src\cassandra-dtest\dtest.py", line 532, in tearDown
> raise AssertionError('Unexpected error in %s node log: %s' % (node.name,
> errors))
> AssertionError: Unexpected error in node1 node log: ['ERROR [main] 2015-08-17
> 16:53:43,120 NoSpamLogger.java:97 - This platform does not support atomic
> directory streams (SecureDirectoryStream); race conditions when loading
> sstable files could occurr']
> {noformat}
> This traces back to the commit for CASSANDRA-7066 today by [~Stefania] and
> [~benedict]. Stefania - care to take this ticket and also look further into
> whether or not we're going to have issues with 7066 on Windows? That error
> message certainly *sounds* like it's not a good thing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9952) UDF with no parameters prevents cqlsh DESCRIBE from working
[ https://issues.apache.org/jira/browse/CASSANDRA-9952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727588#comment-14727588 ] Ryan McGuire commented on CASSANDRA-9952: - I can reproduce this problem on cassandra-2.2 HEAD. It appears to have been fixed in the cassandra-3.0 branch. > UDF with no parameters prevents cqlsh DESCRIBE from working > --- > > Key: CASSANDRA-9952 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9952 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: ubuntu 64 bit, using ccm tool with a one node cluster, > release 2.2.0 >Reporter: Jim Meyer >Priority: Minor > Fix For: 2.2.x > > > If I create a user defined function that takes no parameters like this: > cqlsh:test> CREATE FUNCTION no_parm() CALLED ON NULL INPUT RETURNS bigint > LANGUAGE java AS 'return System.currentTimeMillis() / 1000L;'; > The function works fine in queries, but in cqlsh the describe command stops > working: > cqlsh:test> DESC KEYSPACE test; > izip argument #1 must support iteration > If I drop the function, then describe starts working normally again. It > appears describe assumes there is at least one argument for UDF's, but > creating and using the functions does not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9857) Deal with backward compatibilty issue of broken AbstractBounds serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-9857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-9857:
--
Reviewer: Blake Eggleston
[~bdeggleston] to review
> Deal with backward compatibilty issue of broken AbstractBounds serialization
>
>
> Key: CASSANDRA-9857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9857
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0 beta 2
>
>
> This ticket is related to CASSANDRA-9856 and CASSANDRA-9775. Even if the
> broken/incomplete serialization of {{AbstractBounds}} is not a problem per-se
> for pre-3.0 versions, it's still a problem for trunk and even though it's
> fixed by CASSANDRA-9775 for 3.0 nodes, it might be a problem for 3.0 nodes
> talking to older nodes.
> As the paging tests where those that exposed the problem (on trunk) in the
> first place, it would be nice to modify said paging tests to work on mixed
> version clustering so we can valid if it is a problem. If it is, then we'll
> probably have to add redundant checks on trunk so they ignore anything the
> 3.0 node sends incorrectly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6237) Allow range deletions in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727567#comment-14727567
]
Joshua McKenzie commented on CASSANDRA-6237:
bq. all the validation errors are returned as InvalidRequestException
Enough said. Leave it.
I'll start taking a look at the rebase - thanks or the quick turnaround.
> Allow range deletions in CQL
>
>
> Key: CASSANDRA-6237
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6237
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Benjamin Lerer
>Priority: Minor
> Labels: cql, docs
> Fix For: 3.0 beta 2
>
> Attachments: CASSANDRA-6237.txt
>
>
> We uses RangeTombstones internally in a number of places, but we could expose
> more directly too. Typically, given a table like:
> {noformat}
> CREATE TABLE events (
> id text,
> created_at timestamp,
> content text,
> PRIMARY KEY (id, created_at)
> )
> {noformat}
> we could allow queries like:
> {noformat}
> DELETE FROM events WHERE id='someEvent' AND created_at < 'Jan 3, 2013';
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10239) Failure to launch on Windows with spaces in directory.
[ https://issues.apache.org/jira/browse/CASSANDRA-10239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-10239: Attachment: 10239_v1.txt Looks like this was broken by both CASSANDRA-7838 and CASSANDRA-7874. I've fixed the sigar path on both .bat and .ps1 and removed the duplicate "" around the classpath from the SetJsr223Env method in cassandra-env.ps1. Launches with both .bat and .ps1 with a space in dir name after applying patch for me. > Failure to launch on Windows with spaces in directory. > -- > > Key: CASSANDRA-10239 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10239 > Project: Cassandra > Issue Type: Bug > Components: Packaging > Environment: Windows Server 2012 > Oracle JDK 1.7.79 >Reporter: vyacheslav zaslavskiy >Assignee: Joshua McKenzie > Fix For: 2.2.x > > Attachments: 10239_v1.txt > > > Repro: > Download cassandra 2.2.0 > Extra to C:\Program Files > Execute with cassandra, cassandra.bat, powershell /file cassandra.ps1 > The only way I was able to get this to work was by setting contrast_home to > Progra~1. I have seen multiple resolved tickets for similar issues, but that > doesn't appear to be the case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-3810) reconsider rack awareness
[
https://issues.apache.org/jira/browse/CASSANDRA-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727556#comment-14727556
]
Jonathan Ellis commented on CASSANDRA-3810:
---
The problem is as Peter pointed out that you really only have as much capacity
as your smallest rack. So we are telling people that they must sacrifice
capacity for fault tolerance, which doesn't sound like the right trade in most
situations.
> reconsider rack awareness
> -
>
> Key: CASSANDRA-3810
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3810
> Project: Cassandra
> Issue Type: Task
>Reporter: Peter Schuller
>Assignee: Peter Schuller
>Priority: Minor
>
> We believed we wanted to be rack aware because we want to ensure that loosing
> a rack only affects a single replica of any given row key.
> When using rack awareness, the first problem you encounter immediately if you
> aren't careful is that you induce hotspots as a result of rack aware replica
> selection. Using the format {{rackname-nodename}}, consider a part of the
> ring that looks like this:
> {code}
> ...
> r1-n1
> r1-n2
> r1-n3
> r2-n1
> r3-n1
> r4-n1
> ...
> {code}
> Due to the rack awareness, {{r2-n1}} will be the second replica for all data
> whose primary replica is on {{r1-n1}}, {{r1-n2}} and {{r1-n3}} since they
> would all be forced to skip over any identical racks.
> The way we end up allocating nodes in a cluster is to satisfy this criteria:
> * Any node in rack {{r}} in a cluster of a replication factor of {{rf}}, must
> not have another node in {{r}} within {{rf-1}} steps in the ring in either
> direction.
> Any violation of this criteria implies the induction of hotspots due to rack
> awareness.
> The realization however, that I had a few days ago, is that *the
> rackawareness is not actually changing replica placement* when using this
> ring topology. In other words, *the way you have to use* rack awareness is to
> construct the ring such that *the rack awareness is a NOOP*.
> So, questions:
> * Is there any non-hotspot inducing use-case where rack awareness can be used
> ("used" in the sense that it actually changes the placement relative to
> non-awareness) effectively without satisfying the criteria above?
> * Is it misleading and counter-productive to teach people (via documentation
> for example) to rely on rack awareness in their rings instead of just giving
> them the rule above for ring topology?
> * Would it be a better service to the user to provide an easy way to *ensure*
> that the ring topology adheres to this criteria (such as refusing to
> bootstrap a new node if rack awareness is requested, and taking it into
> consideration on automatic token selection (does anyone use that?)), than to
> "silently" generate hotspots by altering the replication strategy? (The
> "silence" problem is magnified by the fact that {{nodetool ring}} doesn't
> reflect this; so the user must take into account both the RF *and* the racks
> when interpreting {{nodetool ring}} output.)
> FWIW, internally we just go with the criteria outlined above, and we have a
> separate tool which will print the *actual* ownership percentage of a node in
> the ring (based on the thrift {{describe_ring}} call). Any ring that has node
> selections that causes a violation of the criteria is effectively a
> bug/mis-configured ring, so only in the event of mistakes are we "using" the
> rack awareness (using the definition of "use" above).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10005) Streaming not enough bytes error
[
https://issues.apache.org/jira/browse/CASSANDRA-10005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan McGuire updated CASSANDRA-10005:
-
Labels: triaged (was: )
> Streaming not enough bytes error
>
>
> Key: CASSANDRA-10005
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10005
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Chris Moos
>Priority: Minor
> Labels: triaged
> Fix For: 2.2.x
>
> Attachments: deadlock.txt, errors.txt
>
>
> I'm adding a new node to the cluster and I'm seeing a bunch of the errors
> below and the node never joins. It looks like a deadlock.
> After looking through the code it looks like IncomingFileMessage will tell
> the session to retry on Exceptions (except IOException) but the
> CompressedInputStream thread is still running and then the retry happens and
> the deadlock ensues. It might be best to close the StreamReader (and stop the
> thread) if an Exception happens before retrying.
> I'm not sure why I am getting this error to begin with though, might it have
> something to do with not being able to upgrade my SSTables after going from
> 2.1.2 -> 2.2.0?
> {code}
> error: null
> -- StackTrace --
> java.lang.AssertionError
> at
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.checkUnused(LifecycleTransaction.java:428)
> at
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.split(LifecycleTransaction.java:408)
> at
> org.apache.cassandra.db.compaction.CompactionManager.parallelAllSSTableOperation(CompactionManager.java:268)
> at
> org.apache.cassandra.db.compaction.CompactionManager.performSSTableRewrite(CompactionManager.java:373)
> at
> org.apache.cassandra.db.ColumnFamilyStore.sstablesRewrite(ColumnFamilyStore.java:1524)
> at
> org.apache.cassandra.service.StorageService.upgradeSSTables(StorageService.java:2521)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10055) High CPU load for Cassandra 2.1.8
[ https://issues.apache.org/jira/browse/CASSANDRA-10055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McGuire updated CASSANDRA-10055: - Labels: triaged (was: ) > High CPU load for Cassandra 2.1.8 > - > > Key: CASSANDRA-10055 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10055 > Project: Cassandra > Issue Type: Bug > Components: Config > Environment: Prod >Reporter: vijay > Labels: triaged > Fix For: 2.1.x > > Attachments: dstst-lcdn.log, dstst-lcdn2.log, dstst-lcdn3.log, > dstst-lcdn4.log, dstst-lcdn5.log, dstst-lcdn6.log, js.log, js2.log, js3.log, > js4.log, js5.log, js6.log, top-bHn1-2.log, top-bHn1-3.log, top-bHn1-4.log, > top-bHn1-5.log, top-bHn1-6.log, top-bHn1.log > > > We are seeing High CPU Load about 80% to 100% in Cassandra 2.1.8 when doing > Data ingest, we did not had this issue in 2.0.x version of Cassandra > we tested this in different Cloud platforms and results are same. > CPU: Tested with M3 2xlarge AWS instances > Ingest rate: Injecting 1 million Inserts each insert is of 1000bytes > no other Operations are happening except inserts in Cassandra > let me know if more info is needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10084) Very slow performance streaming a large query from a single CF
[
https://issues.apache.org/jira/browse/CASSANDRA-10084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan McGuire updated CASSANDRA-10084:
-
Labels: triaged (was: )
> Very slow performance streaming a large query from a single CF
> --
>
> Key: CASSANDRA-10084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10084
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra 2.1.8
> 12GB EC2 instance
> 12 node cluster
> 32 concurrent reads
> 32 concurrent writes
> 6GB heap space
>Reporter: Brent Haines
> Labels: triaged
> Attachments: cassandra.yaml, node1.txt, node2.txt, node3.txt
>
>
> We have a relatively simple column family that we use to track event data
> from different providers. We have been utilizing it for some time. Here is
> what it looks like:
> {code}
> CREATE TABLE data.stories_by_text (
> ref_id timeuuid,
> second_type text,
> second_value text,
> object_type text,
> field_name text,
> value text,
> story_id timeuuid,
> data map,
> PRIMARY KEY ((ref_id, second_type, second_value, object_type,
> field_name), value, story_id)
> ) WITH CLUSTERING ORDER BY (value ASC, story_id ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = 'Searchable fields and actions in a story are indexed by
> ref id which corresponds to a brand, app, app instance, or user.'
> AND compaction = {'min_threshold': '4', 'cold_reads_to_omit': '0.0',
> 'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32'}
> AND compression = {'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {code}
> We will, on a daily basis pull a query of the complete data for a given
> index, it will look like this:
> {code}
> select * from stories_by_text where ref_id =
> f0124740-2f5a-11e5-a113-03cdf3f3c6dc and second_type = 'Day' and second_value
> = '20150812' and object_type = 'booshaka:user' and field_name = 'hashedEmail';
> {code}
> In the past, we have been able to pull millions of records out of the CF in a
> few seconds. We recently added the data column so that we could filter on
> event data and provide more detailed analysis of activity for our reports.
> The data map, declared with 'data map' is very small; only 2 or 3
> name/value pairs.
> Since we have added this column, our streaming query performance has gone
> straight to hell. I just ran the above query and it took 46 minutes to read
> 86K rows and then it timed out.
> I am uncertain what other data you need to see in order to diagnose this. We
> are using STCS and are considering a change to Leveled Compaction. The table
> is repaired nightly and the updates, which are at a very fast clip will only
> impact the partition key for today, while the queries are for previous days
> only.
> To my knowledge these queries no longer finish ever. They time out, even
> though I put a 60 second timeout on the read for the cluster. I can watch it
> pause for 30 to 50 seconds many times during the stream.
> Again, this only started happening when we added the data column.
> Please let me know what else you need for this. It is having a very big
> impact on our system.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10233) IndexOutOfBoundsException in HintedHandOffManager
[
https://issues.apache.org/jira/browse/CASSANDRA-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727558#comment-14727558
]
Ryan McGuire commented on CASSANDRA-10233:
--
We haven't seen this in our upgrade suite. Can you provide any further
information as to your environment, what version did you upgrade from? Any
non-default cassandra yaml settings? Can you procide your (sanitized) schema,
and any other relevant logs?
/cc [~krummas] if he has any hint as to the problem.
> IndexOutOfBoundsException in HintedHandOffManager
> -
>
> Key: CASSANDRA-10233
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10233
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.2.0
>Reporter: Omri Iluz
>
> After upgrading our cluster to 2.2.0, the following error started showing
> exectly every 10 minutes on every server in the cluster:
> {noformat}
> INFO [CompactionExecutor:1381] 2015-08-31 18:31:55,506
> CompactionTask.java:142 - Compacting (8e7e1520-500e-11e5-b1e3-e95897ba4d20)
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-540-big-Data.db:level=0,
> ]
> INFO [CompactionExecutor:1381] 2015-08-31 18:31:55,599
> CompactionTask.java:224 - Compacted (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 1
> sstables to
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-541-big,]
> to level=0. 1,544,495 bytes to 1,544,495 (~100% of original) in 93ms =
> 15.838121MB/s. 0 total partitions merged to 4. Partition merge counts were
> {1:4, }
> ERROR [HintedHandoff:1] 2015-08-31 18:31:55,600 CassandraDaemon.java:182 -
> Exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_79]
> at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:410)
> ~[na:1.7.0_79]
> at org.apache.cassandra.utils.UUIDGen.getUUID(UUIDGen.java:106)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:515)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:88)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:168)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_79]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> [na:1.7.0_79]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> [na:1.7.0_79]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [na:1.7.0_79]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_79]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_79]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-3810) reconsider rack awareness
[
https://issues.apache.org/jira/browse/CASSANDRA-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727559#comment-14727559
]
Jonathan Ellis commented on CASSANDRA-3810:
---
(Configuring replica count per rack makes it more complicated but not actually
much more flexible. Now if you want 3 copies in a DC you either need 3
balanced racks, or two with exactly 2x as many nodes as the other.)
> reconsider rack awareness
> -
>
> Key: CASSANDRA-3810
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3810
> Project: Cassandra
> Issue Type: Task
>Reporter: Peter Schuller
>Assignee: Peter Schuller
>Priority: Minor
>
> We believed we wanted to be rack aware because we want to ensure that loosing
> a rack only affects a single replica of any given row key.
> When using rack awareness, the first problem you encounter immediately if you
> aren't careful is that you induce hotspots as a result of rack aware replica
> selection. Using the format {{rackname-nodename}}, consider a part of the
> ring that looks like this:
> {code}
> ...
> r1-n1
> r1-n2
> r1-n3
> r2-n1
> r3-n1
> r4-n1
> ...
> {code}
> Due to the rack awareness, {{r2-n1}} will be the second replica for all data
> whose primary replica is on {{r1-n1}}, {{r1-n2}} and {{r1-n3}} since they
> would all be forced to skip over any identical racks.
> The way we end up allocating nodes in a cluster is to satisfy this criteria:
> * Any node in rack {{r}} in a cluster of a replication factor of {{rf}}, must
> not have another node in {{r}} within {{rf-1}} steps in the ring in either
> direction.
> Any violation of this criteria implies the induction of hotspots due to rack
> awareness.
> The realization however, that I had a few days ago, is that *the
> rackawareness is not actually changing replica placement* when using this
> ring topology. In other words, *the way you have to use* rack awareness is to
> construct the ring such that *the rack awareness is a NOOP*.
> So, questions:
> * Is there any non-hotspot inducing use-case where rack awareness can be used
> ("used" in the sense that it actually changes the placement relative to
> non-awareness) effectively without satisfying the criteria above?
> * Is it misleading and counter-productive to teach people (via documentation
> for example) to rely on rack awareness in their rings instead of just giving
> them the rule above for ring topology?
> * Would it be a better service to the user to provide an easy way to *ensure*
> that the ring topology adheres to this criteria (such as refusing to
> bootstrap a new node if rack awareness is requested, and taking it into
> consideration on automatic token selection (does anyone use that?)), than to
> "silently" generate hotspots by altering the replication strategy? (The
> "silence" problem is magnified by the fact that {{nodetool ring}} doesn't
> reflect this; so the user must take into account both the RF *and* the racks
> when interpreting {{nodetool ring}} output.)
> FWIW, internally we just go with the criteria outlined above, and we have a
> separate tool which will print the *actual* ownership percentage of a node in
> the ring (based on the thrift {{describe_ring}} call). Any ring that has node
> selections that causes a violation of the criteria is effectively a
> bug/mis-configured ring, so only in the event of mistakes are we "using" the
> rack awareness (using the definition of "use" above).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9465) No client warning on tombstone threshold
[ https://issues.apache.org/jira/browse/CASSANDRA-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-9465: -- Reviewer: Joshua McKenzie [~JoshuaMcKenzie] to review > No client warning on tombstone threshold > > > Key: CASSANDRA-9465 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9465 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Adam Holmberg >Assignee: Carl Yeksigian >Priority: Minor > Fix For: 2.2.x > > > It appears that a client warning is not coming back for the tombstone > threshold case. The batch warning works. > Repro: > Create a data condition with tombstone_warn_threshold < tombstones < > tombstone_failure_threshold > Query the row > Expected: > Warning in server log, warning returned to client > I'm basing this expectation on what I see > [here|https://github.com/apache/cassandra/blob/68722e7e594d228b4bf14c8cd8cbee19b50835ec/src/java/org/apache/cassandra/db/filter/SliceQueryFilter.java#L235-L247] > Observed: > Warning in server log, no warning flag in response message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10249) Reduce over-read for standard disk io by 16x
[ https://issues.apache.org/jira/browse/CASSANDRA-10249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727546#comment-14727546 ] Jonathan Ellis commented on CASSANDRA-10249: As Benedict noted over [here|https://issues.apache.org/jira/browse/CASSANDRA-8894?focusedCommentId=14726943&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14726943], this only affects users who have # disabled compression # switched to buffered i/o from mmap'd Should we really be making changes in 2.1 to optimize a niche path? > Reduce over-read for standard disk io by 16x > > > Key: CASSANDRA-10249 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10249 > Project: Cassandra > Issue Type: Improvement >Reporter: Albert P Tobey > Fix For: 2.1.x > > Attachments: patched-2.1.9-dstat-lvn10.png, > stock-2.1.9-dstat-lvn10.png, yourkit-screenshot.png > > > On read workloads, Cassandra 2.1 reads drastically more data than it emits > over the network. This causes problems throughput the system by wasting disk > IO and causing unnecessary GC. > I have reproduce the issue on clusters and locally with a single instance. > The only requirement to reproduce the issue is enough data to blow through > the page cache. The default schema and data size with cassandra-stress is > sufficient for exposing the issue. > With stock 2.1.9 I regularly observed anywhere from 300:1 to 500 > disk:network ratio. That is to say, for 1MB/s of network IO, Cassandra was > doing 300-500MB/s of disk reads, saturating the drive. > After applying this patch for standard IO mode > https://gist.github.com/tobert/10c307cf3709a585a7cf the ratio fell to around > 100:1 on my local test rig. Latency improved considerably and GC became a lot > less frequent. > I tested with 512 byte reads as well, but got the same performance, which > makes sense since all HDD and SSD made in the last few years have a 4K block > size (many of them lie and say 512). > I'm re-running the numbers now and will post them tomorrow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9893) Fix upgrade tests from #9704 that are still failing
[
https://issues.apache.org/jira/browse/CASSANDRA-9893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727541#comment-14727541
]
Sylvain Lebresne commented on CASSANDRA-9893:
-
Thanks for the rebase.
bq. CASSANDRA-9554 fixed edge_2i_on_complex_pk_test and indexed_with_eq_test.
That's a little bit surprising to be honest, but as long as none of us see
failures with those tests, I'll take it.
bq. {{composite_index_collections_test}} is still failing for me on
cassandra-3.0 though
This is weird because I'm still not seeing this (I just ran this test 5 times
without issue).
I also remember why we pass {{false}} to the {{DataRange.forPaging()}} call in
that case: that's because what 2.1 sends for the {{startBound}} will be the
name of the _last_ cell (let's call it {{C}}) for the last row (let's call it
{{R}}) that was returned. On 3.0 however, we transform that cell name into a
row clustering, so if we include it, we'll select the whole row {{R}}. When we
send it back to 2.1 however, the node will expect to see {{C}} as first thing
in the result, but instead they will get the first cell of {{R}} and we'll
re-include {{R}} even though it was previously returned, hence duplicating a
result. So as far as I can tell, using {{false}} unconditionally is the right
thing to do here.
I'll add that I'm not sure how the change of the first commit could ever fix
{{composite_index_collections_test}}. The query of that test doesn't have any
condition on the clustering columns and all results will be returned in the
first page (since there is so few of them). So the (only) query sent from the
2.1 node will have its {{start}} be empty (see
{{RangeSliceQueryPager.queryNextPage()}} in 2.1/2.2. On the first page,
{{start}} is assigned the start of the {{SliceQueryFilter}}, and because there
is not clustering columns condition, this will be {{Composites.EMPTY}}). As a
result, when deserializing that command on the 3.0 node (in
{{ReadCommand.LegacyPagedRangeCommandSerializer.deserialize}}), the
{{startBound}} variable will be {{LegacyLayout.LegacyBound.BOTTOM}} (or it's a
bug) and the line the first commit changes won't even be used.
bq. {{cql_tests:TestCQL.static_columns_cas_test}} was failing because calls to
UnfilteredRowIterators.noRowsIterator discard the static Columns data, which
causes problems deserializing the static row.
Right, but the fix for that was sneaked into CASSANDRA-10045. Currently, if you
look at {{UnfilteredRowIterators.noRowIterator}} in your branch, it defines
twice {{columns}}.
bq. The call to noRowsIterator from SelectNoSlices.makeSliceIterator was also
discarding the static row entirely.
That change makes sense however.
bq. \[...\] are also caused by CASSANDRA-9857
I've pushed a patch to that ticket. If you could validate that it does fix
those tests for you, that would be awesome.
bq. I've added a 'maybeDisableFetchAll' method to the column filter builder,
which is called from the legacy slice command deserializer, and turns off
isFetchAll if the command isn't selecting all regular/static columns.
I think what you want here is just to use {{ColumnFilter.selectionBuilder()}}
to create the builder in
{{ReadCommand.LegacyReadCommandSerializer.SinglePartitionSliceCommand}}. In
fact, I'd say it's a bug that the ctor of {{ColumnFilter.Builder}} is public:
it should be private and one should always use one of the 2 static method
{{selectionBuilder}} or {{allColumnsBuilder()}} (so if you could also switch
the ctor to private visibility in your patch, that would be great).
> Fix upgrade tests from #9704 that are still failing
> ---
>
> Key: CASSANDRA-9893
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9893
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Blake Eggleston
> Fix For: 3.0 beta 2
>
>
> The first things to do on this ticket would be to commit Tyler's branch
> (https://github.com/thobbs/cassandra-dtest/tree/8099-backwards-compat) to the
> dtests so cassci run them. I've had to do a few minor modifications to have
> them run locally so someone which access to cassci should do it and make sure
> it runs properly.
> Once we have that, we should fix any test that isn't passing. I've ran the
> tests locally and I had 8 failures. for 2 of them, it sounds plausible that
> they'll get fixed by the patch of CASSANDRA-9775, though that's just a guess.
> The rest where test that timeouted without a particular error in the log,
> and running some of them individually, they passed. So we'll have to see if
> it's just my machine being overly slow when running them all.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10250) Executing lots of schema alters concurrently can lead to dropped alters
[
https://issues.apache.org/jira/browse/CASSANDRA-10250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-10250:
--
Reproduced In: 3.0 beta 1, 2.2.1, 2.1.9 (was: 2.1.9, 2.2.1, 3.0 beta 1)
Reviewer: Aleksey Yeschenko
> Executing lots of schema alters concurrently can lead to dropped alters
> ---
>
> Key: CASSANDRA-10250
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10250
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andrew Hust
> Attachments: concurrent_schema_changes.py, node1.log, node2.log,
> node3.log
>
>
> A recently added
> [dtest|http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/132/testReport/junit/concurrent_schema_changes_test/TestConcurrentSchemaChanges/create_lots_of_schema_churn_test/]
> has been flapping on cassci and has exposed an issue with running lots of
> schema alterations concurrently. The failures occur on healthy clusters but
> seem to occur at higher rates when 1 node is down during the alters.
> The test executes the following – 440 total commands:
> - Create 20 new tables
> - Drop 7 columns one at time across 20 tables
> - Add 7 columns one at time across 20 tables
> - Add one column index on each of the 7 columns on 20 tables
> Outcome is random. Majority of the failures are dropped columns still being
> present, but new columns and indexes have been observed to be incorrect. The
> logs are don’t have exceptions and the columns/indexes that are incorrect
> don’t seem to follow a pattern. Running a {{nodetool describecluster}} on
> each node shows the same schema id on all nodes.
> Attached is a python script extracted from the dtest. Running against a
> local 3 node cluster will reproduce the issue (with enough runs – fails ~20%
> on my machine).
> Also attached is the node logs from a run with when a dropped column
> (alter_me_7 table, column s1) is still present. Checking the system_schema
> tables for this case shows the s1 column in both the columns and drop_columns
> tables.
> This has been flapping on cassci on versions 2+ and doesn’t seem to be
> related to changes in 3.0. More testing needs to be done though.
> //cc [~enigmacurry]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10165) Query fails when batch_size_warn_threshold_in_kb is not set on cassandra.yaml
[
https://issues.apache.org/jira/browse/CASSANDRA-10165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan McGuire updated CASSANDRA-10165:
-
Labels: triaged (was: )
> Query fails when batch_size_warn_threshold_in_kb is not set on cassandra.yaml
> -
>
> Key: CASSANDRA-10165
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10165
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: C* 2.1.5
>Reporter: Jose Martinez Poblete
> Labels: triaged
> Fix For: 2.1.x
>
>
> Jobs failed with the following error:
> {noformat}
> ERROR [SharedPool-Worker-1] 2015-08-21 18:06:42,759 ErrorMessage.java:244 -
> Unexpected exception during request
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.config.DatabaseDescriptor.getBatchSizeWarnThreshold(DatabaseDescriptor.java:855)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.cql3.statements.BatchStatement.verifyBatchSize(BatchStatement.java:239)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.cql3.statements.BatchStatement.executeWithoutConditions(BatchStatement.java:311)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.cql3.statements.BatchStatement.execute(BatchStatement.java:296)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.cql3.statements.BatchStatement.execute(BatchStatement.java:282)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.cql3.QueryProcessor.processBatch(QueryProcessor.java:503)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> com.datastax.bdp.cassandra.cql3.DseQueryHandler$BatchStatementExecution.execute(DseQueryHandler.java:327)
> ~[dse.jar:4.7.0]
> at
> com.datastax.bdp.cassandra.cql3.DseQueryHandler$Operation.executeWithTiming(DseQueryHandler.java:223)
> ~[dse.jar:4.7.0]
> at
> com.datastax.bdp.cassandra.cql3.DseQueryHandler$Operation.executeWithAuditLogging(DseQueryHandler.java:259)
> ~[dse.jar:4.7.0]
> at
> com.datastax.bdp.cassandra.cql3.DseQueryHandler.processBatch(DseQueryHandler.java:110)
> ~[dse.jar:4.7.0]
> at
> org.apache.cassandra.transport.messages.BatchMessage.execute(BatchMessage.java:215)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
> [cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
> [cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
> [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
> [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
> [netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_75]
> at
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
> [cassandra-all-2.1.5.469.jar:2.1.5.469]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [cassandra-all-2.1.5.469.jar:2.1.5.469]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
> {noformat}
> It turns there was no entry for *batch_size_warn_threshold_in_kb* on
> cassandra.yaml
> Once we set that parameter on the file, the error went away
> Can we please have C* assume this setting assumes the default without
> prejudice on the job if it's not specified on the yaml file?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10250) Executing lots of schema alters concurrently can lead to dropped alters
[
https://issues.apache.org/jira/browse/CASSANDRA-10250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727531#comment-14727531
]
Aleksey Yeschenko commented on CASSANDRA-10250:
---
Fair enough. In that case I'll review, but not before 3.0 is released. If you
want a faster review will need to find someone else, sorry.
> Executing lots of schema alters concurrently can lead to dropped alters
> ---
>
> Key: CASSANDRA-10250
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10250
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andrew Hust
> Attachments: concurrent_schema_changes.py, node1.log, node2.log,
> node3.log
>
>
> A recently added
> [dtest|http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/132/testReport/junit/concurrent_schema_changes_test/TestConcurrentSchemaChanges/create_lots_of_schema_churn_test/]
> has been flapping on cassci and has exposed an issue with running lots of
> schema alterations concurrently. The failures occur on healthy clusters but
> seem to occur at higher rates when 1 node is down during the alters.
> The test executes the following – 440 total commands:
> - Create 20 new tables
> - Drop 7 columns one at time across 20 tables
> - Add 7 columns one at time across 20 tables
> - Add one column index on each of the 7 columns on 20 tables
> Outcome is random. Majority of the failures are dropped columns still being
> present, but new columns and indexes have been observed to be incorrect. The
> logs are don’t have exceptions and the columns/indexes that are incorrect
> don’t seem to follow a pattern. Running a {{nodetool describecluster}} on
> each node shows the same schema id on all nodes.
> Attached is a python script extracted from the dtest. Running against a
> local 3 node cluster will reproduce the issue (with enough runs – fails ~20%
> on my machine).
> Also attached is the node logs from a run with when a dropped column
> (alter_me_7 table, column s1) is still present. Checking the system_schema
> tables for this case shows the s1 column in both the columns and drop_columns
> tables.
> This has been flapping on cassci on versions 2+ and doesn’t seem to be
> related to changes in 3.0. More testing needs to be done though.
> //cc [~enigmacurry]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10250) Executing lots of schema alters concurrently can lead to dropped alters
[
https://issues.apache.org/jira/browse/CASSANDRA-10250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727530#comment-14727530
]
Ryan McGuire commented on CASSANDRA-10250:
--
Thanks for the confirmation that this is a known issue. I think it's good to
include this in dtest now, even though it will fail. We are working on a better
reporting mechanism with multiple views, one of which would be to hide "known"
failures like this.
> Executing lots of schema alters concurrently can lead to dropped alters
> ---
>
> Key: CASSANDRA-10250
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10250
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andrew Hust
> Attachments: concurrent_schema_changes.py, node1.log, node2.log,
> node3.log
>
>
> A recently added
> [dtest|http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/132/testReport/junit/concurrent_schema_changes_test/TestConcurrentSchemaChanges/create_lots_of_schema_churn_test/]
> has been flapping on cassci and has exposed an issue with running lots of
> schema alterations concurrently. The failures occur on healthy clusters but
> seem to occur at higher rates when 1 node is down during the alters.
> The test executes the following – 440 total commands:
> - Create 20 new tables
> - Drop 7 columns one at time across 20 tables
> - Add 7 columns one at time across 20 tables
> - Add one column index on each of the 7 columns on 20 tables
> Outcome is random. Majority of the failures are dropped columns still being
> present, but new columns and indexes have been observed to be incorrect. The
> logs are don’t have exceptions and the columns/indexes that are incorrect
> don’t seem to follow a pattern. Running a {{nodetool describecluster}} on
> each node shows the same schema id on all nodes.
> Attached is a python script extracted from the dtest. Running against a
> local 3 node cluster will reproduce the issue (with enough runs – fails ~20%
> on my machine).
> Also attached is the node logs from a run with when a dropped column
> (alter_me_7 table, column s1) is still present. Checking the system_schema
> tables for this case shows the s1 column in both the columns and drop_columns
> tables.
> This has been flapping on cassci on versions 2+ and doesn’t seem to be
> related to changes in 3.0. More testing needs to be done though.
> //cc [~enigmacurry]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10250) Executing lots of schema alters concurrently can lead to dropped alters
[
https://issues.apache.org/jira/browse/CASSANDRA-10250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727525#comment-14727525
]
Aleksey Yeschenko commented on CASSANDRA-10250:
---
CASSANDRA-8387 one of them, and that one can be reproduced using just a single
node. Worse with multiple ones.
This would be a good kind of test to have for CASSANDRA-9424 though.
> Executing lots of schema alters concurrently can lead to dropped alters
> ---
>
> Key: CASSANDRA-10250
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10250
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andrew Hust
> Attachments: concurrent_schema_changes.py, node1.log, node2.log,
> node3.log
>
>
> A recently added
> [dtest|http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/132/testReport/junit/concurrent_schema_changes_test/TestConcurrentSchemaChanges/create_lots_of_schema_churn_test/]
> has been flapping on cassci and has exposed an issue with running lots of
> schema alterations concurrently. The failures occur on healthy clusters but
> seem to occur at higher rates when 1 node is down during the alters.
> The test executes the following – 440 total commands:
> - Create 20 new tables
> - Drop 7 columns one at time across 20 tables
> - Add 7 columns one at time across 20 tables
> - Add one column index on each of the 7 columns on 20 tables
> Outcome is random. Majority of the failures are dropped columns still being
> present, but new columns and indexes have been observed to be incorrect. The
> logs are don’t have exceptions and the columns/indexes that are incorrect
> don’t seem to follow a pattern. Running a {{nodetool describecluster}} on
> each node shows the same schema id on all nodes.
> Attached is a python script extracted from the dtest. Running against a
> local 3 node cluster will reproduce the issue (with enough runs – fails ~20%
> on my machine).
> Also attached is the node logs from a run with when a dropped column
> (alter_me_7 table, column s1) is still present. Checking the system_schema
> tables for this case shows the s1 column in both the columns and drop_columns
> tables.
> This has been flapping on cassci on versions 2+ and doesn’t seem to be
> related to changes in 3.0. More testing needs to be done though.
> //cc [~enigmacurry]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10250) Executing lots of schema alters concurrently can lead to dropped alters
[
https://issues.apache.org/jira/browse/CASSANDRA-10250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727523#comment-14727523
]
Aleksey Yeschenko commented on CASSANDRA-10250:
---
It's a good test to have, but the issue is known. We have several JIRAs open.
Hopefully to be fixed in 3.X line, but not really fixable before then.
I can review the test, but we aren't going to commit one that fails to dtests,
are we?
> Executing lots of schema alters concurrently can lead to dropped alters
> ---
>
> Key: CASSANDRA-10250
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10250
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andrew Hust
> Attachments: concurrent_schema_changes.py, node1.log, node2.log,
> node3.log
>
>
> A recently added
> [dtest|http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/132/testReport/junit/concurrent_schema_changes_test/TestConcurrentSchemaChanges/create_lots_of_schema_churn_test/]
> has been flapping on cassci and has exposed an issue with running lots of
> schema alterations concurrently. The failures occur on healthy clusters but
> seem to occur at higher rates when 1 node is down during the alters.
> The test executes the following – 440 total commands:
> - Create 20 new tables
> - Drop 7 columns one at time across 20 tables
> - Add 7 columns one at time across 20 tables
> - Add one column index on each of the 7 columns on 20 tables
> Outcome is random. Majority of the failures are dropped columns still being
> present, but new columns and indexes have been observed to be incorrect. The
> logs are don’t have exceptions and the columns/indexes that are incorrect
> don’t seem to follow a pattern. Running a {{nodetool describecluster}} on
> each node shows the same schema id on all nodes.
> Attached is a python script extracted from the dtest. Running against a
> local 3 node cluster will reproduce the issue (with enough runs – fails ~20%
> on my machine).
> Also attached is the node logs from a run with when a dropped column
> (alter_me_7 table, column s1) is still present. Checking the system_schema
> tables for this case shows the s1 column in both the columns and drop_columns
> tables.
> This has been flapping on cassci on versions 2+ and doesn’t seem to be
> related to changes in 3.0. More testing needs to be done though.
> //cc [~enigmacurry]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10250) Executing lots of schema alters concurrently can lead to dropped alters
[
https://issues.apache.org/jira/browse/CASSANDRA-10250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727517#comment-14727517
]
Ryan McGuire commented on CASSANDRA-10250:
--
[~iamaleksey] can you review this test?
> Executing lots of schema alters concurrently can lead to dropped alters
> ---
>
> Key: CASSANDRA-10250
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10250
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andrew Hust
> Attachments: concurrent_schema_changes.py, node1.log, node2.log,
> node3.log
>
>
> A recently added
> [dtest|http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/132/testReport/junit/concurrent_schema_changes_test/TestConcurrentSchemaChanges/create_lots_of_schema_churn_test/]
> has been flapping on cassci and has exposed an issue with running lots of
> schema alterations concurrently. The failures occur on healthy clusters but
> seem to occur at higher rates when 1 node is down during the alters.
> The test executes the following – 440 total commands:
> - Create 20 new tables
> - Drop 7 columns one at time across 20 tables
> - Add 7 columns one at time across 20 tables
> - Add one column index on each of the 7 columns on 20 tables
> Outcome is random. Majority of the failures are dropped columns still being
> present, but new columns and indexes have been observed to be incorrect. The
> logs are don’t have exceptions and the columns/indexes that are incorrect
> don’t seem to follow a pattern. Running a {{nodetool describecluster}} on
> each node shows the same schema id on all nodes.
> Attached is a python script extracted from the dtest. Running against a
> local 3 node cluster will reproduce the issue (with enough runs – fails ~20%
> on my machine).
> Also attached is the node logs from a run with when a dropped column
> (alter_me_7 table, column s1) is still present. Checking the system_schema
> tables for this case shows the s1 column in both the columns and drop_columns
> tables.
> This has been flapping on cassci on versions 2+ and doesn’t seem to be
> related to changes in 3.0. More testing needs to be done though.
> //cc [~enigmacurry]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10250) Executing lots of schema alters concurrently can lead to dropped alters
[
https://issues.apache.org/jira/browse/CASSANDRA-10250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan McGuire updated CASSANDRA-10250:
-
Reproduced In: 3.0 beta 1, 2.2.1, 2.1.9 (was: 2.1.9, 2.2.1, 3.0 alpha 1)
> Executing lots of schema alters concurrently can lead to dropped alters
> ---
>
> Key: CASSANDRA-10250
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10250
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andrew Hust
> Attachments: concurrent_schema_changes.py, node1.log, node2.log,
> node3.log
>
>
> A recently added
> [dtest|http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/132/testReport/junit/concurrent_schema_changes_test/TestConcurrentSchemaChanges/create_lots_of_schema_churn_test/]
> has been flapping on cassci and has exposed an issue with running lots of
> schema alterations concurrently. The failures occur on healthy clusters but
> seem to occur at higher rates when 1 node is down during the alters.
> The test executes the following – 440 total commands:
> - Create 20 new tables
> - Drop 7 columns one at time across 20 tables
> - Add 7 columns one at time across 20 tables
> - Add one column index on each of the 7 columns on 20 tables
> Outcome is random. Majority of the failures are dropped columns still being
> present, but new columns and indexes have been observed to be incorrect. The
> logs are don’t have exceptions and the columns/indexes that are incorrect
> don’t seem to follow a pattern. Running a {{nodetool describecluster}} on
> each node shows the same schema id on all nodes.
> Attached is a python script extracted from the dtest. Running against a
> local 3 node cluster will reproduce the issue (with enough runs – fails ~20%
> on my machine).
> Also attached is the node logs from a run with when a dropped column
> (alter_me_7 table, column s1) is still present. Checking the system_schema
> tables for this case shows the s1 column in both the columns and drop_columns
> tables.
> This has been flapping on cassci on versions 2+ and doesn’t seem to be
> related to changes in 3.0. More testing needs to be done though.
> //cc [~enigmacurry]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10250) Executing lots of schema alters concurrently can lead to dropped alters
[
https://issues.apache.org/jira/browse/CASSANDRA-10250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan McGuire updated CASSANDRA-10250:
-
Reproduced In: 3.0 alpha 1, 2.2.1, 2.1.9 (was: 3.0 alpha 1)
> Executing lots of schema alters concurrently can lead to dropped alters
> ---
>
> Key: CASSANDRA-10250
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10250
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andrew Hust
> Attachments: concurrent_schema_changes.py, node1.log, node2.log,
> node3.log
>
>
> A recently added
> [dtest|http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/132/testReport/junit/concurrent_schema_changes_test/TestConcurrentSchemaChanges/create_lots_of_schema_churn_test/]
> has been flapping on cassci and has exposed an issue with running lots of
> schema alterations concurrently. The failures occur on healthy clusters but
> seem to occur at higher rates when 1 node is down during the alters.
> The test executes the following – 440 total commands:
> - Create 20 new tables
> - Drop 7 columns one at time across 20 tables
> - Add 7 columns one at time across 20 tables
> - Add one column index on each of the 7 columns on 20 tables
> Outcome is random. Majority of the failures are dropped columns still being
> present, but new columns and indexes have been observed to be incorrect. The
> logs are don’t have exceptions and the columns/indexes that are incorrect
> don’t seem to follow a pattern. Running a {{nodetool describecluster}} on
> each node shows the same schema id on all nodes.
> Attached is a python script extracted from the dtest. Running against a
> local 3 node cluster will reproduce the issue (with enough runs – fails ~20%
> on my machine).
> Also attached is the node logs from a run with when a dropped column
> (alter_me_7 table, column s1) is still present. Checking the system_schema
> tables for this case shows the s1 column in both the columns and drop_columns
> tables.
> This has been flapping on cassci on versions 2+ and doesn’t seem to be
> related to changes in 3.0. More testing needs to be done though.
> //cc [~enigmacurry]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
cassandra git commit: Remove dead code
Repository: cassandra Updated Branches: refs/heads/cassandra-3.0 796b0f26a -> 6a86d221a Remove dead code Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6a86d221 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6a86d221 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6a86d221 Branch: refs/heads/cassandra-3.0 Commit: 6a86d221a5b4dc11784a022917d0e4c6845e03aa Parents: 796b0f2 Author: Sylvain Lebresne Authored: Wed Sep 2 17:22:53 2015 +0200 Committer: Sylvain Lebresne Committed: Wed Sep 2 17:22:53 2015 +0200 -- src/java/org/apache/cassandra/db/ReadCommand.java | 9 - 1 file changed, 9 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6a86d221/src/java/org/apache/cassandra/db/ReadCommand.java -- diff --git a/src/java/org/apache/cassandra/db/ReadCommand.java b/src/java/org/apache/cassandra/db/ReadCommand.java index 0bc8cea..0ccd229 100644 --- a/src/java/org/apache/cassandra/db/ReadCommand.java +++ b/src/java/org/apache/cassandra/db/ReadCommand.java @@ -971,21 +971,12 @@ public abstract class ReadCommand implements ReadQuery LegacyLayout.LegacyBound startBound = LegacyLayout.decodeBound(metadata, ByteBufferUtil.readWithShortLength(in), true); ByteBufferUtil.readWithShortLength(in); // the composite "stop", which isn't actually needed -// pre-3.0 nodes will sometimes use a clustering prefix for the Command-level start and stop, but in all -// cases this should also be represented by the ClusteringIndexFilter, so we can ignore them -Clustering startClustering; -if (startBound == LegacyLayout.LegacyBound.BOTTOM || startBound.bound.size() < metadata.comparator.size()) -startClustering = Clustering.EMPTY; -else -startClustering = startBound.getAsClustering(metadata); - ColumnFilter selection = LegacyRangeSliceCommandSerializer.getColumnSelectionForSlice(filter, compositesToGroup, metadata); RowFilter rowFilter = LegacyRangeSliceCommandSerializer.deserializeRowFilter(in, metadata); int maxResults = in.readInt(); in.readBoolean(); // countCQL3Rows - boolean selectsStatics = (!selection.fetchedColumns().statics.isEmpty() || filter.selects(Clustering.STATIC_CLUSTERING)); boolean isDistinct = compositesToGroup == -2 || (perPartitionLimit == 1 && selectsStatics); DataLimits limits;
[jira] [Commented] (CASSANDRA-8630) Faster sequential IO (on compaction, streaming, etc)
[ https://issues.apache.org/jira/browse/CASSANDRA-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727483#comment-14727483 ] Joshua McKenzie commented on CASSANDRA-8630: In regards to the interrelationship of both. It's really not hard to get correct (from an NTFS perspective), just that if it's not on our radar (and we don't run tests on Windows) we can come up with some surprises. My thought is that any time we change either a) file rename/deletion code or b) mmap code, we should run the tests on Windows just as a sanity check. Better to find out in advance than after a commit and who knows what other little "gifts" the platform has for us in the future? :) > Faster sequential IO (on compaction, streaming, etc) > > > Key: CASSANDRA-8630 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8630 > Project: Cassandra > Issue Type: Improvement > Components: Core, Tools >Reporter: Oleg Anastasyev >Assignee: Stefania > Labels: compaction, performance > Fix For: 3.x > > Attachments: 8630-FasterSequencialReadsAndWrites.txt, cpu_load.png, > flight_recorder_001_files.tar.gz, flight_recorder_002_files.tar.gz, > mmaped_uncomp_hotspot.png > > > When node is doing a lot of sequencial IO (streaming, compacting, etc) a lot > of CPU is lost in calls to RAF's int read() and DataOutputStream's write(int). > This is because default implementations of readShort,readLong, etc as well as > their matching write* are implemented with numerous calls of byte by byte > read and write. > This makes a lot of syscalls as well. > A quick microbench shows than just reimplementation of these methods in > either way gives 8x speed increase. > A patch attached implements RandomAccessReader.read and > SequencialWriter.write methods in more efficient way. > I also eliminated some extra byte copies in CompositeType.split and > ColumnNameHelper.maxComponents, which were on my profiler's hotspot method > list during tests. > A stress tests on my laptop show that this patch makes compaction 25-30% > faster on uncompressed sstables and 15% faster for compressed ones. > A deployment to production shows much less CPU load for compaction. > (I attached a cpu load graph from one of our production, orange is niced CPU > load - i.e. compaction; yellow is user - i.e. not compaction related tasks) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8741) Running a drain before a decommission apparently the wrong thing to do
[
https://issues.apache.org/jira/browse/CASSANDRA-8741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727475#comment-14727475
]
Jan Karlsson commented on CASSANDRA-8741:
-
Took it for a test spin.
+1
> Running a drain before a decommission apparently the wrong thing to do
> --
>
> Key: CASSANDRA-8741
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8741
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Ubuntu 14.04; Cassandra 2.0.11.82 (Datastax Enterprise
> 4.5.3)
>Reporter: Casey Marshall
>Assignee: Jan Karlsson
>Priority: Trivial
> Labels: lhf
> Fix For: 2.1.x, 2.0.x
>
> Attachments: 8741.txt
>
>
> This might simply be a documentation issue. It appears that running "nodetool
> drain" is a very wrong thing to do before running a "nodetool decommission".
> The idea was that I was going to safely shut off writes and flush everything
> to disk before beginning the decommission. What happens is the "decommission"
> call appears to fail very early on after starting, and afterwards, the node
> in question is stuck in state LEAVING, but all other nodes in the ring see
> that node as NORMAL, but down. No streams are ever sent from the node being
> decommissioned to other nodes.
> The drain command does indeed shut down the "BatchlogTasks" executor
> (org/apache/cassandra/service/StorageService.java, line 3445 in git tag
> "cassandra-2.0.11") but the decommission process tries using that executor
> when calling the "startBatchlogReplay" function
> (org/apache/cassandra/db/BatchlogManager.java, line 123) called through
> org.apache.cassandra.service.StorageService.unbootstrap (see the stack trace
> pasted below).
> This also failed in a similar way on Cassandra 1.2.13-ish (DSE 3.2.4).
> So, either something is wrong with the drain/decommission commands, or it's
> very wrong to run a drain before a decommission. What's worse, there seems to
> be no way to recover this node once it is in this state; you need to shut it
> down and run "removenode".
> My terminal output:
> {code}
> ubuntu@x:~$ nodetool drain
> ubuntu@x:~$ tail /var/log/^C
> ubuntu@x:~$ nodetool decommission
> Exception in thread "main" java.util.concurrent.RejectedExecutionException:
> Task
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@3008fa33
> rejected from
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor@1d6242e8[Terminated,
> pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 52]
> at
> java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:325)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:530)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.submit(ScheduledThreadPoolExecutor.java:629)
> at
> org.apache.cassandra.db.BatchlogManager.startBatchlogReplay(BatchlogManager.java:123)
> at
> org.apache.cassandra.service.StorageService.unbootstrap(StorageService.java:2966)
> at
> org.apache.cassandra.service.StorageService.decommission(StorageService.java:2934)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer
[jira] [Commented] (CASSANDRA-8970) Allow custom time_format on cqlsh COPY TO
[
https://issues.apache.org/jira/browse/CASSANDRA-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727468#comment-14727468
]
Carl Yeksigian commented on CASSANDRA-8970:
---
Sorry I missed this ticket. This looks good, I created CASSANDRA-10252 as the
follow up to use the same property for both {{COPY TO}} and {{COPY FROM}}.
> Allow custom time_format on cqlsh COPY TO
> -
>
> Key: CASSANDRA-8970
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8970
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Aaron Ploetz
>Priority: Trivial
> Labels: cqlsh
> Fix For: 2.1.x
>
> Attachments: CASSANDRA-8970.patch
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> When executing a COPY TO from cqlsh, the user is currently has no control
> over the format of exported timestamp columns. If the user has indicated a
> {{time_format}} in their cqlshrc file, that format will be used. Otherwise,
> the system default format will be used.
> The problem comes into play when the timestamp format used on a COPY TO, is
> not valid when the data is sent back into Cassandra with a COPY FROM.
> For instance, if a user has {{time_format = %Y-%m-%d %H:%M:%S%Z}} specified
> in their cqlshrc, COPY TO will format timestamp columns like this:
> {{userid|posttime|postcontent}}
> {{0|2015-03-14 14:59:00CDT|rtyeryerweh}}
> {{0|2015-03-14 14:58:00CDT|sdfsdfsdgfjdsgojr}}
> {{0|2015-03-12 14:27:00CDT|sdgfjdsgojr}}
> Executing a COPY FROM on that same file will produce an "unable to coerce to
> formatted date(long)" error.
> Right now, the only way to change the way timestamps are formatted is to exit
> cqlsh, modify the {{time_format}} property in cqlshrc, and restart cqlsh.
> The ability to specify a COPY option of TIME_FORMAT with a Python strftime
> format, would allow the user to quickly alter the timestamp format for
> export, without reconfiguring cqlsh.
> {{aploetz@cqlsh:stackoverflow> COPY posts1 TO '/home/aploetz/posts1.csv' WITH
> DELIMITER='|' AND HEADER=true AND TIME_FORMAT='%Y-%m-%d %H:%M:%S%z;}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10252) COPY FROM should respect time_format from cqlshrc
Carl Yeksigian created CASSANDRA-10252:
--
Summary: COPY FROM should respect time_format from cqlshrc
Key: CASSANDRA-10252
URL: https://issues.apache.org/jira/browse/CASSANDRA-10252
Project: Cassandra
Issue Type: Bug
Reporter: Carl Yeksigian
Fix For: 2.1.x
Currently, {{COPY FROM}} doesn't respect the {{time_format}} property in
cqlshrc, we only use that for {{COPY TO}}. We should use the same property for
both to prevent issues like CASSANDRA-8970.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-10242) Validate rack information on startup
[ https://issues.apache.org/jira/browse/CASSANDRA-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Yeksigian reassigned CASSANDRA-10242: -- Assignee: Carl Yeksigian > Validate rack information on startup > > > Key: CASSANDRA-10242 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10242 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Carl Yeksigian > Fix For: 2.1.x > > > Moving to a new rack means that different data should be stored on a node. > We already persist rack information in a system table; we should fail startup > if this doesn't match what the snitch thinks it should be. (Either the > snitch is wrong, and needs to be fixed, or the machine has been moved and > needs to be rebootstrapped.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10209) Missing role manager in cassandra.yaml causes unexpected behaviour
[
https://issues.apache.org/jira/browse/CASSANDRA-10209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-10209:
---
Fix Version/s: (was: 3.0 beta 2)
3.0.0 rc1
> Missing role manager in cassandra.yaml causes unexpected behaviour
> --
>
> Key: CASSANDRA-10209
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10209
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Fix For: 2.2.x, 3.0.0 rc1
>
>
> On upgrading to 2.2+, if the new {{role_manager}} option is not added to
> {{cassandra.yaml}}, an instance of the default {{CassandraRoleManager}} is
> created during initialization of {{DatabaseDescriptor}}. This is a problem as
> the set of role options supported by {{CRM}} depends on the configured
> {{IAuthenticator}}, which at that point in time is always
> {{AllowAllAuthenticator}}.
> This StackOverflow post describes the problem; the configured authenticator
> is {{PasswordAuthenticator}}, the role manager should allow roles to be
> created using the {{PASSWORD}} option, but it does not.
> http://stackoverflow.com/questions/31820914/in-cassandra-2-2-unable-to-create-role-containing-password
> The simple workaround is to ensure that yaml contains the role manager option
> {code}
> role_manager: CassandraRoleManager
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10209) Missing role manager in cassandra.yaml causes unexpected behaviour
[
https://issues.apache.org/jira/browse/CASSANDRA-10209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727433#comment-14727433
]
Jonathan Ellis commented on CASSANDRA-10209:
Reverted.
Carl proposes this as a fix:
https://github.com/carlyeks/cassandra/commit/204f7bdd8ea0a18d5c642cb7d42104749b82a62b
Which looks reasonable but I'm not sure what other resources should be excluded
from client mode, and I'd rather not do it halfway.
> Missing role manager in cassandra.yaml causes unexpected behaviour
> --
>
> Key: CASSANDRA-10209
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10209
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Fix For: 2.2.x, 3.0 beta 2
>
>
> On upgrading to 2.2+, if the new {{role_manager}} option is not added to
> {{cassandra.yaml}}, an instance of the default {{CassandraRoleManager}} is
> created during initialization of {{DatabaseDescriptor}}. This is a problem as
> the set of role options supported by {{CRM}} depends on the configured
> {{IAuthenticator}}, which at that point in time is always
> {{AllowAllAuthenticator}}.
> This StackOverflow post describes the problem; the configured authenticator
> is {{PasswordAuthenticator}}, the role manager should allow roles to be
> created using the {{PASSWORD}} option, but it does not.
> http://stackoverflow.com/questions/31820914/in-cassandra-2-2-unable-to-create-role-containing-password
> The simple workaround is to ensure that yaml contains the role manager option
> {code}
> role_manager: CassandraRoleManager
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[5/6] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/796b0f26 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/796b0f26 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/796b0f26 Branch: refs/heads/cassandra-3.0 Commit: 796b0f26a9ebc061fe30b86f089237e82ec90199 Parents: 53a177a f7644f2 Author: Jonathan Ellis Authored: Wed Sep 2 09:36:02 2015 -0500 Committer: Jonathan Ellis Committed: Wed Sep 2 09:36:02 2015 -0500 -- .../org/apache/cassandra/config/DatabaseDescriptor.java | 9 +++-- 1 file changed, 3 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/796b0f26/src/java/org/apache/cassandra/config/DatabaseDescriptor.java --
[4/6] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/796b0f26 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/796b0f26 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/796b0f26 Branch: refs/heads/trunk Commit: 796b0f26a9ebc061fe30b86f089237e82ec90199 Parents: 53a177a f7644f2 Author: Jonathan Ellis Authored: Wed Sep 2 09:36:02 2015 -0500 Committer: Jonathan Ellis Committed: Wed Sep 2 09:36:02 2015 -0500 -- .../org/apache/cassandra/config/DatabaseDescriptor.java | 9 +++-- 1 file changed, 3 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/796b0f26/src/java/org/apache/cassandra/config/DatabaseDescriptor.java --
[1/6] cassandra git commit: Revert "Defer initialization of default RoleManager until IAuthenticator is set"
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 45dd5d178 -> f7644f265
refs/heads/cassandra-3.0 53a177a91 -> 796b0f26a
refs/heads/trunk 79628dd70 -> f66730d61
Revert "Defer initialization of default RoleManager until IAuthenticator is set"
This reverts commit 0d2f4038fc2564c695d0d8c4b1ad523ec1afaceb.
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f7644f26
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f7644f26
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f7644f26
Branch: refs/heads/cassandra-2.2
Commit: f7644f265aec236d1daf825a7d55e9695c06b73d
Parents: 45dd5d1
Author: Jonathan Ellis
Authored: Wed Sep 2 09:35:45 2015 -0500
Committer: Jonathan Ellis
Committed: Wed Sep 2 09:35:45 2015 -0500
--
.../org/apache/cassandra/config/DatabaseDescriptor.java | 9 +++--
1 file changed, 3 insertions(+), 6 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f7644f26/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 9cf3959..b7e3eaa 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -87,9 +87,7 @@ public class DatabaseDescriptor
private static IAuthenticator authenticator = new AllowAllAuthenticator();
private static IAuthorizer authorizer = new AllowAllAuthorizer();
-// Don't initialize the role manager until applying config. The options
supported by CassandraRoleManager
-// depend on the configured IAuthenticator, so defer creating it until
that's been set.
-private static IRoleManager roleManager;
+private static IRoleManager roleManager = new CassandraRoleManager();
private static IRequestScheduler requestScheduler;
private static RequestSchedulerId requestSchedulerId;
@@ -325,9 +323,8 @@ public class DatabaseDescriptor
if (authenticator instanceof AllowAllAuthenticator && !(authorizer
instanceof AllowAllAuthorizer))
throw new ConfigurationException("AllowAllAuthenticator can't be
used with " + conf.authorizer, false);
-roleManager = conf.role_manager == null
-? new CassandraRoleManager()
-: FBUtilities.newRoleManager(conf.role_manager);
+if (conf.role_manager != null)
+roleManager = FBUtilities.newRoleManager(conf.role_manager);
if (authenticator instanceof PasswordAuthenticator && !(roleManager
instanceof CassandraRoleManager))
throw new ConfigurationException("CassandraRoleManager must be
used with PasswordAuthenticator", false);
[2/6] cassandra git commit: Revert "Defer initialization of default RoleManager until IAuthenticator is set"
Revert "Defer initialization of default RoleManager until IAuthenticator is set"
This reverts commit 0d2f4038fc2564c695d0d8c4b1ad523ec1afaceb.
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f7644f26
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f7644f26
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f7644f26
Branch: refs/heads/cassandra-3.0
Commit: f7644f265aec236d1daf825a7d55e9695c06b73d
Parents: 45dd5d1
Author: Jonathan Ellis
Authored: Wed Sep 2 09:35:45 2015 -0500
Committer: Jonathan Ellis
Committed: Wed Sep 2 09:35:45 2015 -0500
--
.../org/apache/cassandra/config/DatabaseDescriptor.java | 9 +++--
1 file changed, 3 insertions(+), 6 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f7644f26/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 9cf3959..b7e3eaa 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -87,9 +87,7 @@ public class DatabaseDescriptor
private static IAuthenticator authenticator = new AllowAllAuthenticator();
private static IAuthorizer authorizer = new AllowAllAuthorizer();
-// Don't initialize the role manager until applying config. The options
supported by CassandraRoleManager
-// depend on the configured IAuthenticator, so defer creating it until
that's been set.
-private static IRoleManager roleManager;
+private static IRoleManager roleManager = new CassandraRoleManager();
private static IRequestScheduler requestScheduler;
private static RequestSchedulerId requestSchedulerId;
@@ -325,9 +323,8 @@ public class DatabaseDescriptor
if (authenticator instanceof AllowAllAuthenticator && !(authorizer
instanceof AllowAllAuthorizer))
throw new ConfigurationException("AllowAllAuthenticator can't be
used with " + conf.authorizer, false);
-roleManager = conf.role_manager == null
-? new CassandraRoleManager()
-: FBUtilities.newRoleManager(conf.role_manager);
+if (conf.role_manager != null)
+roleManager = FBUtilities.newRoleManager(conf.role_manager);
if (authenticator instanceof PasswordAuthenticator && !(roleManager
instanceof CassandraRoleManager))
throw new ConfigurationException("CassandraRoleManager must be
used with PasswordAuthenticator", false);
[6/6] cassandra git commit: Merge branch 'cassandra-3.0' into trunk
Merge branch 'cassandra-3.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f66730d6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f66730d6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f66730d6 Branch: refs/heads/trunk Commit: f66730d612b3e828c8b26f8016314effb1c39d02 Parents: 79628dd 796b0f2 Author: Jonathan Ellis Authored: Wed Sep 2 09:36:15 2015 -0500 Committer: Jonathan Ellis Committed: Wed Sep 2 09:36:15 2015 -0500 -- .../org/apache/cassandra/config/DatabaseDescriptor.java | 9 +++-- 1 file changed, 3 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f66730d6/src/java/org/apache/cassandra/config/DatabaseDescriptor.java --
[3/6] cassandra git commit: Revert "Defer initialization of default RoleManager until IAuthenticator is set"
Revert "Defer initialization of default RoleManager until IAuthenticator is set"
This reverts commit 0d2f4038fc2564c695d0d8c4b1ad523ec1afaceb.
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f7644f26
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f7644f26
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f7644f26
Branch: refs/heads/trunk
Commit: f7644f265aec236d1daf825a7d55e9695c06b73d
Parents: 45dd5d1
Author: Jonathan Ellis
Authored: Wed Sep 2 09:35:45 2015 -0500
Committer: Jonathan Ellis
Committed: Wed Sep 2 09:35:45 2015 -0500
--
.../org/apache/cassandra/config/DatabaseDescriptor.java | 9 +++--
1 file changed, 3 insertions(+), 6 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f7644f26/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 9cf3959..b7e3eaa 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -87,9 +87,7 @@ public class DatabaseDescriptor
private static IAuthenticator authenticator = new AllowAllAuthenticator();
private static IAuthorizer authorizer = new AllowAllAuthorizer();
-// Don't initialize the role manager until applying config. The options
supported by CassandraRoleManager
-// depend on the configured IAuthenticator, so defer creating it until
that's been set.
-private static IRoleManager roleManager;
+private static IRoleManager roleManager = new CassandraRoleManager();
private static IRequestScheduler requestScheduler;
private static RequestSchedulerId requestSchedulerId;
@@ -325,9 +323,8 @@ public class DatabaseDescriptor
if (authenticator instanceof AllowAllAuthenticator && !(authorizer
instanceof AllowAllAuthorizer))
throw new ConfigurationException("AllowAllAuthenticator can't be
used with " + conf.authorizer, false);
-roleManager = conf.role_manager == null
-? new CassandraRoleManager()
-: FBUtilities.newRoleManager(conf.role_manager);
+if (conf.role_manager != null)
+roleManager = FBUtilities.newRoleManager(conf.role_manager);
if (authenticator instanceof PasswordAuthenticator && !(roleManager
instanceof CassandraRoleManager))
throw new ConfigurationException("CassandraRoleManager must be
used with PasswordAuthenticator", false);
[jira] [Commented] (CASSANDRA-9857) Deal with backward compatibilty issue of broken AbstractBounds serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-9857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727408#comment-14727408
]
Sylvain Lebresne commented on CASSANDRA-9857:
-
Btw, CI results with be at
[utests|http://cassci.datastax.com/job/pcmanus-9857-testall/] and
[dtests|http://cassci.datastax.com/job/pcmanus-9857-dtest/] as soon as cassci
stops timeouting when fetching from github.
> Deal with backward compatibilty issue of broken AbstractBounds serialization
>
>
> Key: CASSANDRA-9857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9857
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0 beta 2
>
>
> This ticket is related to CASSANDRA-9856 and CASSANDRA-9775. Even if the
> broken/incomplete serialization of {{AbstractBounds}} is not a problem per-se
> for pre-3.0 versions, it's still a problem for trunk and even though it's
> fixed by CASSANDRA-9775 for 3.0 nodes, it might be a problem for 3.0 nodes
> talking to older nodes.
> As the paging tests where those that exposed the problem (on trunk) in the
> first place, it would be nice to modify said paging tests to work on mixed
> version clustering so we can valid if it is a problem. If it is, then we'll
> probably have to add redundant checks on trunk so they ignore anything the
> 3.0 node sends incorrectly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9857) Deal with backward compatibilty issue of broken AbstractBounds serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-9857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727405#comment-14727405
]
Sylvain Lebresne commented on CASSANDRA-9857:
-
So, as pointed by [~bdeggleston] in CASSANDRA-9893, the backward compatibility
tests from CASSANDRA-9704 does allow to expose this problem. So I've pushed a
branch [here|https://github.com/pcmanus/cassandra/commits/9857] with a fix for
this (which turns out not too hard). I've validated locally that this was
fixing the {{large_count_test}} from Tyler's CASSANDRA-9704 tests in particular.
As a side note, on CASSANDRA-9893, Blake attributes the failure of
{{static_columns_with_distinct_test}} to this, but I don't think that it's the
case. I'm investigating what the real cause is and will followup on
CASSANDRA-9393 but in any case, the patch here does fix the legitimate bug this
ticket is about.
> Deal with backward compatibilty issue of broken AbstractBounds serialization
>
>
> Key: CASSANDRA-9857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9857
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0 beta 2
>
>
> This ticket is related to CASSANDRA-9856 and CASSANDRA-9775. Even if the
> broken/incomplete serialization of {{AbstractBounds}} is not a problem per-se
> for pre-3.0 versions, it's still a problem for trunk and even though it's
> fixed by CASSANDRA-9775 for 3.0 nodes, it might be a problem for 3.0 nodes
> talking to older nodes.
> As the paging tests where those that exposed the problem (on trunk) in the
> first place, it would be nice to modify said paging tests to work on mixed
> version clustering so we can valid if it is a problem. If it is, then we'll
> probably have to add redundant checks on trunk so they ignore anything the
> 3.0 node sends incorrectly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
