[jira] [Commented] (CASSANDRA-8591) Tunable bootstrapping
[
https://issues.apache.org/jira/browse/CASSANDRA-8591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273410#comment-14273410
]
Sylvain Lebresne commented on CASSANDRA-8591:
-
The thing is, tunable consistency doesn't mean that C* is free to randomly fail
the consistency guarantees that you ask. If we were to allow this, that is what
would happen: some queries woud fail the consistency level you ask it to do. I
have a hard time seeing that as an acceptable behavior, unless maybe (maybe)
the option is very clearly labeled as
{{I_willingly_authorise_cassandra_to_give_up_all_consistency_guarantee_from_now_on_and_until_I_ve_run_a_full_repair}}.
Open to see other opinions, but _a priori_ I feel reticent about the concept.
Tunable bootstrapping
-
Key: CASSANDRA-8591
URL: https://issues.apache.org/jira/browse/CASSANDRA-8591
Project: Cassandra
Issue Type: Improvement
Reporter: Donald Smith
Priority: Minor
Fix For: 3.0
Often bootstrapping fails due to errors like unable to find sufficient
sources for streaming range. But cassandra is supposed to be fault tolerant,
and it's supposed to have tunable consistency.
If it can't find sources for some ranges, it should allow bootstrapping to
continue and should print out a report about what ranges were missing.
Allow the bootstrap to be tunable, under control of parameters (allow up to
100 failures, for example).
For many apps, it's far better to bootstrap what's available then to fail
flat.
Same with rebuilds.
We were doing maintenance on some disks, and when we started cassandra back
up, some nodes ran out of disk space, due to operator miscalculation.
Thereafter, we've been unable to bootstrap new nodes, due to unable to find
sufficient sources for streaming range. But bootstrapping with partial
success would be far better than being unable to bootstrap at all, and
cheaper than a repair. Our consistency requirements aren't high but we prefer
as much consistency as cassandra can give us.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8303) Provide strict mode for CQL Queries
[
https://issues.apache.org/jira/browse/CASSANDRA-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273438#comment-14273438
]
Sam Tunnicliffe commented on CASSANDRA-8303:
Agreed, the yaml overrides would be clunky at best so I'm definitely +1 on
doing it via authz exclusively.
Also, location most definitely does not fit into the resource hierarchy, so +1
to Aleksey's points there.
I'd question whether UNPREPARED_STATEMENTS was really necessary. parsing
regular statements certainly doesn't have anything near the potential impact of
ALLOW FILTERING or multipartition queries, so I'd be tempted to leave that out
for version 1 and if it proves to be a genuine issue we can always consider
adding it later.
I've also been thinking about alternatives for syntax. One option could be to
attach the restrictions to the (existing) permission that they affect, rather
than simply making new permissions.
{code}
GRANT SELECT ON resource TO user WITH RESTRICTION ON MULTIPARTITION_QUERY;
GRANT SELECT ON resource TO user WITH RESTRICTION ON ALLOW_FILTERING;
GRANT SELECT ON resource TO user WITH RESTRICTION ON INDEX_USAGE;
GRANT ALTER ON resource TO user WITH RESTRICTION ON INDEX_CREATION;
GRANT MODIFY ON resource TO user WITH RESTRICTION ON LOGGED_BATCH;
GRANT MODIFY ON resource TO user WITH RESTRICTION ON UNLOGGED_BATCH;
{code}
Behaviour syntax for unrestricted GRANTS would remain unchanged backwards
compatible.
FTR, I'm not advocating this syntax yet as the version suggested by Aleksey is
certainly more SQL-ish, but just wanted to put it up for discussion. Adding
restrictions as a first class concept in the language *may* simplify things
conceptually and prevent a proliferation of permissions.
Provide strict mode for CQL Queries
-
Key: CASSANDRA-8303
URL: https://issues.apache.org/jira/browse/CASSANDRA-8303
Project: Cassandra
Issue Type: Improvement
Reporter: Anupam Arora
Fix For: 3.0
Please provide a strict mode option in cassandra that will kick out any CQL
queries that are expensive, e.g. any query with ALLOWS FILTERING,
multi-partition queries, secondary index queries, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8580) AssertionErrors after activating unchecked_tombstone_compaction with leveled compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Björn Hachmann updated CASSANDRA-8580:
--
Attachment: system.log
Please find attached a 12h extract from our log files from the 11.Dec.
The error suddenly occurs repeatedly (starting from 11:15). After restarting
the node (at 14:45) the error disappears, but later on the following day (not
shown in the log file) reappears.
AssertionErrors after activating unchecked_tombstone_compaction with leveled
compaction
---
Key: CASSANDRA-8580
URL: https://issues.apache.org/jira/browse/CASSANDRA-8580
Project: Cassandra
Issue Type: Bug
Reporter: Björn Hachmann
Assignee: Marcus Eriksson
Fix For: 2.1.3
Attachments: system.log
During our upgrade of Cassandra from version 2.0.7 to 2.1.2 we experienced a
serious problem regarding the setting unchecked_tombstone_compaction in
combination with leveled compaction strategy.
In order to prevent tombstone-threshold-warnings we activated the setting for
a specific table after the upgrade. Some time after that we observed new
errors in our log files:
{code}
INFO [CompactionExecutor:184] 2014-12-11 12:36:06,597
CompactionTask.java:136 - Compacting
[SSTableReader(path='/data/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ka-1848-Data.db'),
SSTableReader(path='/
data/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ka-1847-Data.db'),
SSTableReader(path='/data/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ka-1845-Data.db'),
SSTableReader
(path='/data/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ka-1846-Data.db')]
ERROR [CompactionExecutor:183] 2014-12-11 12:36:06,613
CassandraDaemon.java:153 - Exception in thread
Thread[CompactionExecutor:183,1,main]
java.lang.AssertionError:
/data/cassandra/data/metrigo_prod/new_user_data/metrigo_prod-new_user_data-tmplink-ka-705732-Data.db
at
org.apache.cassandra.io.sstable.SSTableReader.getApproximateKeyCount(SSTableReader.java:243)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:146)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:232)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
{code}
Obviously that error aborted the compaction and after some time the number of
pending compactions became very high on every node. Of course, this in turn
had a negative impact on several other metrics.
After reverting the setting we had to restart all nodes. After that
compactions could finish again and the pending compactions could be worked
off.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8595) Emit timeouts per endpoint
sankalp kohli created CASSANDRA-8595: Summary: Emit timeouts per endpoint Key: CASSANDRA-8595 URL: https://issues.apache.org/jira/browse/CASSANDRA-8595 Project: Cassandra Issue Type: Improvement Reporter: sankalp kohli Priority: Minor We currently emit number of timeouts experienced by a co-ordinator while doing reads and writes. This does not tell us which replica or endpoint is responsible for the timeouts. We can keep a map of endpoint to number of timeouts which could be emitted via JMX. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8562) Fix checking available disk space before compaction starts
[ https://issues.apache.org/jira/browse/CASSANDRA-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273482#comment-14273482 ] Marcus Eriksson edited comment on CASSANDRA-8562 at 1/12/15 11:32 AM: -- [~JoshuaMcKenzie] attaching the 2.1-patch, seems we have a bug in CompactionTask#reduceForLimitedScope since CASSANDRA-6916 - we remove the biggest sstable from the sstables we compact, but we never unmark it as compacting. Could you have a look at my fix? was (Author: krummas): [~JoshuaMcKenzie] attaching the 2.1-patch, seems we have a bug in CompactionTask#reduceForLimitedScope since CASSANDRA-6916 - we remove the biggest sstable from the sstables we compact, but we never unmark them as compacting. Could you have a look at my fix? Fix checking available disk space before compaction starts -- Key: CASSANDRA-8562 URL: https://issues.apache.org/jira/browse/CASSANDRA-8562 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 2.0.12, 2.1.3 Attachments: 0001-Check-for-available-disk-space-before-starting-compa.patch, 8562-2.1-v2.txt When starting a compaction we check if there is enough disk space available to start it, otherwise we might (for STCS) reduce the compaction so that the result could fit. Now (since CASSANDRA-8329) we check for the directory to write to a lot later and this can reduce the compaction after we have created the scanners. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8303) Provide strict mode for CQL Queries
[
https://issues.apache.org/jira/browse/CASSANDRA-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273580#comment-14273580
]
Sylvain Lebresne commented on CASSANDRA-8303:
-
bq. it doesn't fit anywhere in the hierarchy
Which hierarchy are we talking about? At the very list in Sam's proposal of
syntax, I don't see what would be the problem of having such restrictions. If
this doesn't fit the hierachy of restrictions-as-permissions, then as said
above I think this might point to the lack of flexibility of that approach more
than anything.
bq. we call checkAccess() on an already prepared statement. Would have to add
another check to prepare() itself, which is a mess
Fair enough, I can buy that argument for not including it as of this v1. Even
though adding a check in prepare() feels hardly like a bit deal to me if I'm
being honest.
bq. This is just abuse of the authorization subsystem
I really fail to see how. Authorization is about allowing configurable limits
on what people can and cannot do and {{UNPREPARED_STATEMENTS}} fits that as
much as any other restriction we're discussing here as far as I can tell.
Granted, it's a restriction on query preparations instead of query execution,
but I don't see why limiting execution would be fine but limiting preparation
would be an horrible abuse of the system.
Provide strict mode for CQL Queries
-
Key: CASSANDRA-8303
URL: https://issues.apache.org/jira/browse/CASSANDRA-8303
Project: Cassandra
Issue Type: Improvement
Reporter: Anupam Arora
Fix For: 3.0
Please provide a strict mode option in cassandra that will kick out any CQL
queries that are expensive, e.g. any query with ALLOWS FILTERING,
multi-partition queries, secondary index queries, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7653) Add role based access control to Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273610#comment-14273610 ] Sam Tunnicliffe commented on CASSANDRA-7653: Ok, that's a fair enough point. Incidentally, the same applies to LIST USERS as it's become an alias for LIST ROLES which has a pretty different set of metadata so I'll add the deprecated columns there too. Just to note that this will break the auth_test dtest, as it verifies resultsets for those queries using ordinals not names. I'll include changes to handle this in my dtests PR. Add role based access control to Cassandra -- Key: CASSANDRA-7653 URL: https://issues.apache.org/jira/browse/CASSANDRA-7653 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Mike Adamson Assignee: Sam Tunnicliffe Fix For: 3.0 Attachments: 7653.patch, CQLSmokeTest.java, cql_smoke_test.py The current authentication model supports granting permissions to individual users. While this is OK for small or medium organizations wanting to implement authorization, it does not work well in large organizations because of the overhead of having to maintain the permissions for each user. Introducing roles into the authentication model would allow sets of permissions to be controlled in one place as a role and then the role granted to users. Roles should also be able to be granted to other roles to allow hierarchical sets of permissions to be built up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8303) Provide strict mode for CQL Queries
[ https://issues.apache.org/jira/browse/CASSANDRA-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273662#comment-14273662 ] Sylvain Lebresne commented on CASSANDRA-8303: - bq. So to me it both doesn't fit conceptually and is rather useless in itself. Well, I continue to find your conceptual distinction a bit arbitrary but I can agree that the usefulness part is important. And on that topic, I happen to be of the opinion that it's useful and would be used but I could be wrong and I'm fine with leaving it out of v1 until we have more inputs on the question. Provide strict mode for CQL Queries - Key: CASSANDRA-8303 URL: https://issues.apache.org/jira/browse/CASSANDRA-8303 Project: Cassandra Issue Type: Improvement Reporter: Anupam Arora Fix For: 3.0 Please provide a strict mode option in cassandra that will kick out any CQL queries that are expensive, e.g. any query with ALLOWS FILTERING, multi-partition queries, secondary index queries, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8303) Provide strict mode for CQL Queries
[ https://issues.apache.org/jira/browse/CASSANDRA-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273638#comment-14273638 ] Aleksey Yeschenko commented on CASSANDRA-8303: -- bq. Authorization is about allowing configurable limits on what people can and cannot do and UNPREPARED_STATEMENTS fits that as much as any other restriction we're discussing here as far as I can tell. IMO authorization is about limiting access to resources (tables and functions). Adding limits on different ways to access them (filtering, indexes, multigets, batches) is a stretch and a hack, but I'm willing to let it happen, simply because of their destructive potential. Prepared vs. unprepared is almost a protocol level detail, is not a distinction that the authorization subsystem should ever be aware of. That there was a conceptual issue. I also highly doubt the usefulness of it - non-prepared statements don't have a destructive potential, and you often want to use them (from cqlsh, for one, and for one-off queries that you don't reuse anyway). So to me it both doesn't fit conceptually and is rather useless in itself. Provide strict mode for CQL Queries - Key: CASSANDRA-8303 URL: https://issues.apache.org/jira/browse/CASSANDRA-8303 Project: Cassandra Issue Type: Improvement Reporter: Anupam Arora Fix For: 3.0 Please provide a strict mode option in cassandra that will kick out any CQL queries that are expensive, e.g. any query with ALLOWS FILTERING, multi-partition queries, secondary index queries, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6237) Allow range deletions in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273654#comment-14273654
]
Benedict commented on CASSANDRA-6237:
-
I was just going by the ticket description, which didn't mention any syntax
like this, and the related conversation about extending the functionality
focused on multiple clustering column relations (which I also agree should be
introduced). If what I'm suggesting is already covered by AND syntax and is
pencilled for inclusion, ignore me! :)
Allow range deletions in CQL
Key: CASSANDRA-6237
URL: https://issues.apache.org/jira/browse/CASSANDRA-6237
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Benjamin Lerer
Priority: Minor
Labels: cql, docs
Fix For: 3.0
Attachments: CASSANDRA-6237.txt
We uses RangeTombstones internally in a number of places, but we could expose
more directly too. Typically, given a table like:
{noformat}
CREATE TABLE events (
id text,
created_at timestamp,
content text,
PRIMARY KEY (id, created_at)
)
{noformat}
we could allow queries like:
{noformat}
DELETE FROM events WHERE id='someEvent' AND created_at 'Jan 3, 2013';
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8580) AssertionErrors after activating unchecked_tombstone_compaction with leveled compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273584#comment-14273584
]
Marcus Eriksson commented on CASSANDRA-8580:
Are you sure you are only seeing this when enabling the
unchecked_tombstone_compaction flag?
AssertionErrors after activating unchecked_tombstone_compaction with leveled
compaction
---
Key: CASSANDRA-8580
URL: https://issues.apache.org/jira/browse/CASSANDRA-8580
Project: Cassandra
Issue Type: Bug
Reporter: Björn Hachmann
Assignee: Marcus Eriksson
Fix For: 2.1.3
Attachments: system.log
During our upgrade of Cassandra from version 2.0.7 to 2.1.2 we experienced a
serious problem regarding the setting unchecked_tombstone_compaction in
combination with leveled compaction strategy.
In order to prevent tombstone-threshold-warnings we activated the setting for
a specific table after the upgrade. Some time after that we observed new
errors in our log files:
{code}
INFO [CompactionExecutor:184] 2014-12-11 12:36:06,597
CompactionTask.java:136 - Compacting
[SSTableReader(path='/data/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ka-1848-Data.db'),
SSTableReader(path='/
data/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ka-1847-Data.db'),
SSTableReader(path='/data/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ka-1845-Data.db'),
SSTableReader
(path='/data/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ka-1846-Data.db')]
ERROR [CompactionExecutor:183] 2014-12-11 12:36:06,613
CassandraDaemon.java:153 - Exception in thread
Thread[CompactionExecutor:183,1,main]
java.lang.AssertionError:
/data/cassandra/data/metrigo_prod/new_user_data/metrigo_prod-new_user_data-tmplink-ka-705732-Data.db
at
org.apache.cassandra.io.sstable.SSTableReader.getApproximateKeyCount(SSTableReader.java:243)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:146)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:232)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
{code}
Obviously that error aborted the compaction and after some time the number of
pending compactions became very high on every node. Of course, this in turn
had a negative impact on several other metrics.
After reverting the setting we had to restart all nodes. After that
compactions could finish again and the pending compactions could be worked
off.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8580) AssertionErrors after activating unchecked_tombstone_compaction with leveled compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273602#comment-14273602
]
Björn Hachmann commented on CASSANDRA-8580:
---
Yes, I am quite sure, at least it seemed so.
The errors started to occur some time after I have enabled the setting. After
some futile investigations I disabled again. Then I wondered why the rollback
did not seem to have the desired effect, but after restarting every node, the
errors disappeared.
Of course it could have been an unfortunate coincidence.
AssertionErrors after activating unchecked_tombstone_compaction with leveled
compaction
---
Key: CASSANDRA-8580
URL: https://issues.apache.org/jira/browse/CASSANDRA-8580
Project: Cassandra
Issue Type: Bug
Reporter: Björn Hachmann
Assignee: Marcus Eriksson
Fix For: 2.1.3
Attachments: system.log
During our upgrade of Cassandra from version 2.0.7 to 2.1.2 we experienced a
serious problem regarding the setting unchecked_tombstone_compaction in
combination with leveled compaction strategy.
In order to prevent tombstone-threshold-warnings we activated the setting for
a specific table after the upgrade. Some time after that we observed new
errors in our log files:
{code}
INFO [CompactionExecutor:184] 2014-12-11 12:36:06,597
CompactionTask.java:136 - Compacting
[SSTableReader(path='/data/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ka-1848-Data.db'),
SSTableReader(path='/
data/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ka-1847-Data.db'),
SSTableReader(path='/data/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ka-1845-Data.db'),
SSTableReader
(path='/data/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ka-1846-Data.db')]
ERROR [CompactionExecutor:183] 2014-12-11 12:36:06,613
CassandraDaemon.java:153 - Exception in thread
Thread[CompactionExecutor:183,1,main]
java.lang.AssertionError:
/data/cassandra/data/metrigo_prod/new_user_data/metrigo_prod-new_user_data-tmplink-ka-705732-Data.db
at
org.apache.cassandra.io.sstable.SSTableReader.getApproximateKeyCount(SSTableReader.java:243)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:146)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:232)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
{code}
Obviously that error aborted the compaction and after some time the number of
pending compactions became very high on every node. Of course, this in turn
had a negative impact on several other metrics.
After reverting the setting we had to restart all nodes. After that
compactions could finish again and the pending compactions could be worked
off.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8303) Provide strict mode for CQL Queries
[ https://issues.apache.org/jira/browse/CASSANDRA-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273646#comment-14273646 ] Aleksey Yeschenko commented on CASSANDRA-8303: -- It's an alternative syntax, but I don't see how it's any different other than cosmetically. It still produces the same number of new permissions in the end - you just split its definition in two parts. What would the new IAuthorizer look like? How would you properly compose restricted/unrestricted permissions from all/keyspace/table? I know it's doable, but it doesn't make things any simpler conceptually, and in fact makes the code and more importantly the APIs, more complex. Provide strict mode for CQL Queries - Key: CASSANDRA-8303 URL: https://issues.apache.org/jira/browse/CASSANDRA-8303 Project: Cassandra Issue Type: Improvement Reporter: Anupam Arora Fix For: 3.0 Please provide a strict mode option in cassandra that will kick out any CQL queries that are expensive, e.g. any query with ALLOWS FILTERING, multi-partition queries, secondary index queries, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6237) Allow range deletions in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273645#comment-14273645
]
Sylvain Lebresne commented on CASSANDRA-6237:
-
How would {{x BETWEEN y AND z}} differs from {{x = y AND x = z}} (not
necessarily vetoing it as syntaxic sugar but that's not really linked to that
issue (it's not limited to deletes))?
Allow range deletions in CQL
Key: CASSANDRA-6237
URL: https://issues.apache.org/jira/browse/CASSANDRA-6237
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Benjamin Lerer
Priority: Minor
Labels: cql, docs
Fix For: 3.0
Attachments: CASSANDRA-6237.txt
We uses RangeTombstones internally in a number of places, but we could expose
more directly too. Typically, given a table like:
{noformat}
CREATE TABLE events (
id text,
created_at timestamp,
content text,
PRIMARY KEY (id, created_at)
)
{noformat}
we could allow queries like:
{noformat}
DELETE FROM events WHERE id='someEvent' AND created_at 'Jan 3, 2013';
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7653) Add role based access control to Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273600#comment-14273600 ] Sylvain Lebresne commented on CASSANDRA-7653: - bq. I think it's probably acceptable for us to change metadata like this in a major release, particularly as the semantics and structure of the resultset is unchanged The problem is that it makes it really hard during rolling upgrade, because a given client will talk to server in both version and so you'd need code just for the migration to test if a column or the other is present. So I agree that we need to have both column for at least one version. It's not perfect, some will wonder if there is a difference between the two, but it's better than the alternative. Add role based access control to Cassandra -- Key: CASSANDRA-7653 URL: https://issues.apache.org/jira/browse/CASSANDRA-7653 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Mike Adamson Assignee: Sam Tunnicliffe Fix For: 3.0 Attachments: 7653.patch, CQLSmokeTest.java, cql_smoke_test.py The current authentication model supports granting permissions to individual users. While this is OK for small or medium organizations wanting to implement authorization, it does not work well in large organizations because of the overhead of having to maintain the permissions for each user. Introducing roles into the authentication model would allow sets of permissions to be controlled in one place as a role and then the role granted to users. Roles should also be able to be granted to other roles to allow hierarchical sets of permissions to be built up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8558) deleted row still can be selected out
[
https://issues.apache.org/jira/browse/CASSANDRA-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-8558:
Attachment: 8558-v2_2.1.txt
8558-v2_2.0.txt
bq. If they insert a range tombstone that spans a non-singleton range, and
query a slice starting in the middle of that range...?
You're right. I looked at your previous comment and focused on the EOC thing
because that's something I've wanted (and forgot) to change for a long time,
but that's actually not the problem. It's inconsistent and I think we should
fix it at least in 2.1 but the fact it fixes the test above is incidental and a
slightly modified version of the test (including in the patch) exhibits the bug
of this ticket even with the EOC fix. I was further lead astray by the fix
version: the underlying bug does affect 2.0 as well, it's just that particular
example that does not.
The problem is indeed that IndexSliceReader don't include those range
tombstones that does intersect with the queried slices but don't start within
said slice. Attaching v2 patches to fix that. The 2.0 patch is smaller because:
# 2.1 is a little bit more careful to not create cell objects for stuff it
doesn't need to and as a consequence require a bit more code.
# the 2.1 patch includes a unit test (a small variation on the test above but
such that the range tombstone starts strictly before the queried slice to avoid
having the EOC business shadow the important issue). 2.0 doesn't have CQLTester
so I'll push the dtest when this is committed.
# I've included the v1 fixes in the 2.1 patch because it's the right thing to
do. I've left 2.0 alone however since it's less inconsistent than 2.1 on that
point.
# For some reason the indexed case of SSTablesNamesIterator has the same
problem. 2.0 and the non-indexed case do the right thing however so it's an
oversight.
deleted row still can be selected out
-
Key: CASSANDRA-8558
URL: https://issues.apache.org/jira/browse/CASSANDRA-8558
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: 2.1.2
java version 1.7.0_55
Reporter: zhaoyan
Assignee: Sylvain Lebresne
Priority: Blocker
Fix For: 2.1.3
Attachments: 8558-v2_2.0.txt, 8558-v2_2.1.txt, 8558.txt
first
{code}CREATE KEYSPACE space1 WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 3};
CREATE TABLE space1.table3(a int, b int, c text,primary key(a,b));
CREATE KEYSPACE space2 WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 3};{code}
second
{code}CREATE TABLE space2.table1(a int, b int, c int, primary key(a,b));
CREATE TABLE space2.table2(a int, b int, c int, primary key(a,b));
INSERT INTO space1.table3(a,b,c) VALUES(1,1,'1');
drop table space2.table1;
DELETE FROM space1.table3 where a=1 and b=1;
drop table space2.table2;
select * from space1.table3 where a=1 and b=1;{code}
you will find that the row (a=1 and b=1) in space1.table3 is not deleted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8303) Provide strict mode for CQL Queries
[
https://issues.apache.org/jira/browse/CASSANDRA-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273468#comment-14273468
]
Sylvain Lebresne commented on CASSANDRA-8303:
-
bq. FTR, I'm not advocating this syntax yet as the version suggested by Aleksey
is certainly more SQL-ish, but just wanted to put it up for discussion
I don't yet have a definitive opinion on this either, but while the fact that
Aleksey's suggestion doesn't add any new concept is seducing, I do also suspect
that treating this as restrictions of other permissions might be clearer
conceptually and more future proof (it's more clear to me what happen when we
add new permission in the resctriction-version than if {{SELECT}} is just an
alias for a bunch of permissions. And syntax-wise, something like {{GRANT
INDEXING ON}} could become a problem if we ever allow the use of index in
updates (which I don't want to do but my point is just that future proofing
everything int the syntax might turn uglier than with Sam's {{WITH
RESTRICTION}} alternative)).
bq. I'd question whether UNPREPARED_STATEMENTS was really necessary
I agree that it's probably less useful than other restrictions, but I can see
it as useful for large organizations that want to enforce this easily. And
since it seems easy enough to include and without any real downside, I'd be
personally keen on including it from the start.
Provide strict mode for CQL Queries
-
Key: CASSANDRA-8303
URL: https://issues.apache.org/jira/browse/CASSANDRA-8303
Project: Cassandra
Issue Type: Improvement
Reporter: Anupam Arora
Fix For: 3.0
Please provide a strict mode option in cassandra that will kick out any CQL
queries that are expensive, e.g. any query with ALLOWS FILTERING,
multi-partition queries, secondary index queries, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8562) Fix checking available disk space before compaction starts
[ https://issues.apache.org/jira/browse/CASSANDRA-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-8562: --- Attachment: (was: 8562-v2.patch) Fix checking available disk space before compaction starts -- Key: CASSANDRA-8562 URL: https://issues.apache.org/jira/browse/CASSANDRA-8562 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 2.0.12, 2.1.3 Attachments: 0001-Check-for-available-disk-space-before-starting-compa.patch When starting a compaction we check if there is enough disk space available to start it, otherwise we might (for STCS) reduce the compaction so that the result could fit. Now (since CASSANDRA-8329) we check for the directory to write to a lot later and this can reduce the compaction after we have created the scanners. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8562) Fix checking available disk space before compaction starts
[ https://issues.apache.org/jira/browse/CASSANDRA-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-8562: --- Attachment: 8562-2.1-v2.txt Fix checking available disk space before compaction starts -- Key: CASSANDRA-8562 URL: https://issues.apache.org/jira/browse/CASSANDRA-8562 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 2.0.12, 2.1.3 Attachments: 0001-Check-for-available-disk-space-before-starting-compa.patch, 8562-2.1-v2.txt When starting a compaction we check if there is enough disk space available to start it, otherwise we might (for STCS) reduce the compaction so that the result could fit. Now (since CASSANDRA-8329) we check for the directory to write to a lot later and this can reduce the compaction after we have created the scanners. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8562) Fix checking available disk space before compaction starts
[ https://issues.apache.org/jira/browse/CASSANDRA-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-8562: --- Attachment: (was: 8562-2.1-v2.txt) Fix checking available disk space before compaction starts -- Key: CASSANDRA-8562 URL: https://issues.apache.org/jira/browse/CASSANDRA-8562 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 2.0.12, 2.1.3 Attachments: 0001-Check-for-available-disk-space-before-starting-compa.patch, 8562-2.1-v2.txt When starting a compaction we check if there is enough disk space available to start it, otherwise we might (for STCS) reduce the compaction so that the result could fit. Now (since CASSANDRA-8329) we check for the directory to write to a lot later and this can reduce the compaction after we have created the scanners. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8562) Fix checking available disk space before compaction starts
[ https://issues.apache.org/jira/browse/CASSANDRA-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273287#comment-14273287 ] Marcus Eriksson edited comment on CASSANDRA-8562 at 1/12/15 10:18 AM: -- bq. Delete default no-op implementation of reduceScopeForLimitedSpace from DiskAwareRunnable as it's no longer used fixed bq. is throwing an RTE the best thing for us to do when we don't have sufficient disk space for a node to compact files? Perhaps not, using something like a compaction_failure_policy might be good, problem is though that in the compaction case there is a chance that we might actually recover - we could have 10 compactions going eating up disk space but once those are done we will have plenty of room on the drive. Perhaps if we do something like if we fail finding space to compact to for X minutes, die? I wanted a quick fix in for 2.0.12 though, as we broke this in CASSANDRA-8329 (which is not yet released) edit: oops misread, thought you suggested adding compaction_failure_policy in this ticket, will get it committed was (Author: krummas): bq. Delete default no-op implementation of reduceScopeForLimitedSpace from DiskAwareRunnable as it's no longer used fixed bq. is throwing an RTE the best thing for us to do when we don't have sufficient disk space for a node to compact files? Perhaps not, using something like a compaction_failure_policy might be good, problem is though that in the compaction case there is a chance that we might actually recover - we could have 10 compactions going eating up disk space but once those are done we will have plenty of room on the drive. Perhaps if we do something like if we fail finding space to compact to for X minutes, die? I wanted a quick fix in for 2.0.12 though, as we broke this in CASSANDRA-8329 (which is not yet released) Fix checking available disk space before compaction starts -- Key: CASSANDRA-8562 URL: https://issues.apache.org/jira/browse/CASSANDRA-8562 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 2.0.12, 2.1.3 Attachments: 0001-Check-for-available-disk-space-before-starting-compa.patch, 8562-v2.patch When starting a compaction we check if there is enough disk space available to start it, otherwise we might (for STCS) reduce the compaction so that the result could fit. Now (since CASSANDRA-8329) we check for the directory to write to a lot later and this can reduce the compaction after we have created the scanners. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7705) Safer Resource Management
[ https://issues.apache.org/jira/browse/CASSANDRA-7705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273698#comment-14273698 ] Benedict commented on CASSANDRA-7705: - I've pushed a rebased-to-2.1 version [here|https://github.com/belliottsmith/cassandra/tree/7705-2.1] Safer Resource Management - Key: CASSANDRA-7705 URL: https://issues.apache.org/jira/browse/CASSANDRA-7705 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Fix For: 3.0 We've had a spate of bugs recently with bad reference counting. these can have potentially dire consequences, generally either randomly deleting data or giving us infinite loops. Since in 2.1 we only reference count resources that are relatively expensive and infrequently managed (or in places where this safety is probably not as necessary, e.g. SerializingCache), we could without any negative consequences (and only slight code complexity) introduce a safer resource management scheme for these more expensive/infrequent actions. Basically, I propose when we want to acquire a resource we allocate an object that manages the reference. This can only be released once; if it is released twice, we fail immediately at the second release, reporting where the bug is (rather than letting it continue fine until the next correct release corrupts the count). The reference counter remains the same, but we obtain guarantees that the reference count itself is never badly maintained, although code using it could mistakenly release its own handle early (typically this is only an issue when cleaning up after a failure, in which case under the new scheme this would be an innocuous error) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8061) tmplink files are not removed
[
https://issues.apache.org/jira/browse/CASSANDRA-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273701#comment-14273701
]
Benedict commented on CASSANDRA-8061:
-
[~markncooper]: I've pushed a rebased-to-latest 2.1 version of CASSANDRA-7705.
It won't likely apply cleanly to 2.1.2, but 2.1.3 should hopefully be released
shortly.
tmplink files are not removed
-
Key: CASSANDRA-8061
URL: https://issues.apache.org/jira/browse/CASSANDRA-8061
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Linux
Reporter: Gianluca Borello
Assignee: Joshua McKenzie
Priority: Critical
Fix For: 2.1.3
Attachments: 8061_v1.txt, 8248-thread_dump.txt
After installing 2.1.0, I'm experiencing a bunch of tmplink files that are
filling my disk. I found https://issues.apache.org/jira/browse/CASSANDRA-7803
and that is very similar, and I confirm it happens both on 2.1.0 as well as
from the latest commit on the cassandra-2.1 branch
(https://github.com/apache/cassandra/commit/aca80da38c3d86a40cc63d9a122f7d45258e4685
from the cassandra-2.1)
Even starting with a clean keyspace, after a few hours I get:
{noformat}
$ sudo find /raid0 | grep tmplink | xargs du -hs
2.7G
/raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Data.db
13M
/raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Index.db
1.8G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Data.db
12M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Index.db
5.2M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Index.db
822M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Data.db
7.3M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Index.db
1.2G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Data.db
6.7M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Index.db
1.1G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Data.db
11M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Index.db
1.7G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Data.db
812K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Index.db
122M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-208-Data.db
744K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-739-Index.db
660K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Index.db
796K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Index.db
137M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Data.db
161M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Data.db
139M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Data.db
940K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Index.db
936K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Index.db
161M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Data.db
672K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-197-Index.db
113M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Data.db
116M
[jira] [Commented] (CASSANDRA-6237) Allow range deletions in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273741#comment-14273741
]
Sylvain Lebresne commented on CASSANDRA-6237:
-
Yeah, the plan here is to allow the same restrictions we allow on {{SELECT}}
for clustering columns, which includes that kind of queries.
Allow range deletions in CQL
Key: CASSANDRA-6237
URL: https://issues.apache.org/jira/browse/CASSANDRA-6237
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Benjamin Lerer
Priority: Minor
Labels: cql, docs
Fix For: 3.0
Attachments: CASSANDRA-6237.txt
We uses RangeTombstones internally in a number of places, but we could expose
more directly too. Typically, given a table like:
{noformat}
CREATE TABLE events (
id text,
created_at timestamp,
content text,
PRIMARY KEY (id, created_at)
)
{noformat}
we could allow queries like:
{noformat}
DELETE FROM events WHERE id='someEvent' AND created_at 'Jan 3, 2013';
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8597) Stress: make simple things simple
Jonathan Ellis created CASSANDRA-8597: - Summary: Stress: make simple things simple Key: CASSANDRA-8597 URL: https://issues.apache.org/jira/browse/CASSANDRA-8597 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jonathan Ellis Assignee: T Jake Luciani Fix For: 2.1.3 Some of the trouble people have with stress is a documentation problem, but some is functional. Comments from [~iamaleksey]: # 3 clustering columns, make a million cells in a single partition, should be simple, but it's not. have to tweak 'clustering' on the three columns just right to make stress work at all. w/ some values it'd just gets stuck forever computing batches # for others, it generates huge, megabyte-size batches, utterly disrespecting 'select' clause in 'insert' # I want a sequential generator too, to be able to predict deterministic result sets. uniform() only gets you so far # impossible to simulate a time series workload /cc [~jshook] [~aweisberg] [~benedict] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8580) AssertionErrors after activating unchecked_tombstone_compaction with leveled compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273754#comment-14273754
]
Benedict commented on CASSANDRA-8580:
-
I managed to reproduce this with a slight bug consistently locally todat with
ColumnFaimlyStoreTest. Unfortunately it seems to have been a wild goose chase
tracking down the cause; testRemoveUnfinishedCompactionLeftovers deletes a
stats file, and this file was then consistently missing. Strangely, though, it
only occurred if DataTracker.releaseReferences did NOT release the final
reference to sstables, but I have stopped looking further after spending quite
a bit of time on it, since this was a deliberate corruption of the files for
the unit test. I post here only in case it helps understand the cause, given
the reference release interplay, although I expect it won't.
AssertionErrors after activating unchecked_tombstone_compaction with leveled
compaction
---
Key: CASSANDRA-8580
URL: https://issues.apache.org/jira/browse/CASSANDRA-8580
Project: Cassandra
Issue Type: Bug
Reporter: Björn Hachmann
Assignee: Marcus Eriksson
Fix For: 2.1.3
Attachments: system.log
During our upgrade of Cassandra from version 2.0.7 to 2.1.2 we experienced a
serious problem regarding the setting unchecked_tombstone_compaction in
combination with leveled compaction strategy.
In order to prevent tombstone-threshold-warnings we activated the setting for
a specific table after the upgrade. Some time after that we observed new
errors in our log files:
{code}
INFO [CompactionExecutor:184] 2014-12-11 12:36:06,597
CompactionTask.java:136 - Compacting
[SSTableReader(path='/data/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ka-1848-Data.db'),
SSTableReader(path='/
data/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ka-1847-Data.db'),
SSTableReader(path='/data/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ka-1845-Data.db'),
SSTableReader
(path='/data/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-ka-1846-Data.db')]
ERROR [CompactionExecutor:183] 2014-12-11 12:36:06,613
CassandraDaemon.java:153 - Exception in thread
Thread[CompactionExecutor:183,1,main]
java.lang.AssertionError:
/data/cassandra/data/metrigo_prod/new_user_data/metrigo_prod-new_user_data-tmplink-ka-705732-Data.db
at
org.apache.cassandra.io.sstable.SSTableReader.getApproximateKeyCount(SSTableReader.java:243)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:146)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:232)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
{code}
Obviously that error aborted the compaction and after some time the number of
pending compactions became very high on every node. Of course, this in turn
had a negative impact on several other metrics.
After reverting the setting we had to restart all nodes. After that
compactions could finish again and the pending compactions could be worked
off.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[ https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274005#comment-14274005 ] Ariel Weisberg edited comment on CASSANDRA-7438 at 1/12/15 7:25 PM: If you go all the way down the JMH rabbit hole you don't need to do any of your own timing and JMH will actually do some smart things to give you accurate timing and ameliorate the impact of non-scalable/expensive timing measurement. Metrics uses System.nanoTime() internally so it isn't really any better as far as I can tell. System.nanoTime() on Linux is pretty scalable http://shipilev.net/blog/2014/nanotrusting-nanotime/. When I tested it in JMH it actually seemed to be linearly scalable, but JMH will solve that for you even on platforms where nanoTime is finicky. The C* integration looks good. I'm glad it was easy. When it comes to exposing configuration parameters less is more. I would prefer not to expose anything new because once people start using them they don't like to have the options taken away (or disabled). We should make an effort to set them automatically (or good enough) and if that fails we can add user visible configuration. My preference is to make the options accessible via properties as an escape hatch in production, and then add them to config if we really can't set them automatically. The stress tool when used without workload profiles does some validation. It checks that values are there and that the contents are correct. Did not know about the JNA synchronized block. That is surprising, but I am glad to hear it is getting fixed. For access to jemalloc I recommend using unsafe and LD_PRELOAD jemalloc. I think that would be the recommended approach and the one you should benchmark against and JNA would be there as a fallback. That gives you a JNI call for allocation/deallocation. I am trying out the JMH benchmark and looking at the new linked implementation right now. How are you starting the JMH benchmark? was (Author: aweisberg): If you go all the way down the JMH rabbit hole you don't need to do any of your own timing and JMH will actually do some smart things to give you accurate timing and ameliorate the impact of non-scalable/expensive timing measurement. Metrics uses System.nanoTime() internally so it isn't really any better as far as I can tell. System.nanoTime() on Linux is pretty scalable http://shipilev.net/blog/2014/nanotrusting-nanotime/. When I tested it in JMH it actually seemed to be linearly scalable, but JMH will solve that for you even on platforms where nanoTime is finicky. The C* integration looks good. I'm glad it was easy. When it comes to exposing configuration parameters less is more The stress tool when used without workload profiles does some validation. It checks that values are there and that the contents are correct. Did not know about the JNA synchronized block. That is surprising, but I am glad to hear it is getting fixed. For access to jemalloc I recommend using unsafe and LD_PRELOAD jemalloc. I think that would be the recommended approach and the one you should benchmark against and JNA would be there as a fallback. That gives you a JNI call for allocation/deallocation. I am trying out the JMH benchmark and looking at the new linked implementation right now. How are you starting the JMH benchmark? Serializing Row cache alternative (Fully off heap) -- Key: CASSANDRA-7438 URL: https://issues.apache.org/jira/browse/CASSANDRA-7438 Project: Cassandra Issue Type: Improvement Components: Core Environment: Linux Reporter: Vijay Assignee: Robert Stupp Labels: performance Fix For: 3.0 Attachments: 0001-CASSANDRA-7438.patch, tests.zip Currently SerializingCache is partially off heap, keys are still stored in JVM heap as BB, * There is a higher GC costs for a reasonably big cache. * Some users have used the row cache efficiently in production for better results, but this requires careful tunning. * Overhead in Memory for the cache entries are relatively high. So the proposal for this ticket is to move the LRU cache logic completely off heap and use JNI to interact with cache. We might want to ensure that the new implementation match the existing API's (ICache), and the implementation needs to have safe memory access, low overhead in memory and less memcpy's (As much as possible). We might also want to make this cache configurable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[ https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274005#comment-14274005 ] Ariel Weisberg commented on CASSANDRA-7438: --- If you go all the way down the JMH rabbit hole you don't need to do any of your own timing and JMH will actually do some smart things to give you accurate timing and ameliorate the impact of non-scalable/expensive timing measurement. Metrics uses System.nanoTime() internally so it isn't really any better as far as I can tell. System.nanoTime() on Linux is pretty scalable http://shipilev.net/blog/2014/nanotrusting-nanotime/. When I tested it in JMH it actually seemed to be linearly scalable, but JMH will solve that for you even on platforms where nanoTime is finicky. The C* integration looks good. I'm glad it was easy. When it comes to exposing configuration parameters less is more The stress tool when used without workload profiles does some validation. It checks that values are there and that the contents are correct. Did not know about the JNA synchronized block. That is surprising, but I am glad to hear it is getting fixed. For access to jemalloc I recommend using unsafe and LD_PRELOAD jemalloc. I think that would be the recommended approach and the one you should benchmark against and JNA would be there as a fallback. That gives you a JNI call for allocation/deallocation. I am trying out the JMH benchmark and looking at the new linked implementation right now. How are you starting the JMH benchmark? Serializing Row cache alternative (Fully off heap) -- Key: CASSANDRA-7438 URL: https://issues.apache.org/jira/browse/CASSANDRA-7438 Project: Cassandra Issue Type: Improvement Components: Core Environment: Linux Reporter: Vijay Assignee: Robert Stupp Labels: performance Fix For: 3.0 Attachments: 0001-CASSANDRA-7438.patch, tests.zip Currently SerializingCache is partially off heap, keys are still stored in JVM heap as BB, * There is a higher GC costs for a reasonably big cache. * Some users have used the row cache efficiently in production for better results, but this requires careful tunning. * Overhead in Memory for the cache entries are relatively high. So the proposal for this ticket is to move the LRU cache logic completely off heap and use JNI to interact with cache. We might want to ensure that the new implementation match the existing API's (ICache), and the implementation needs to have safe memory access, low overhead in memory and less memcpy's (As much as possible). We might also want to make this cache configurable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8597) Stress: make simple things simple
[ https://issues.apache.org/jira/browse/CASSANDRA-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273998#comment-14273998 ] Sebastian Estevez commented on CASSANDRA-8597: -- Feel free to add issues / ideas: https://github.com/phact/CASTableSizer/issues Stress: make simple things simple - Key: CASSANDRA-8597 URL: https://issues.apache.org/jira/browse/CASSANDRA-8597 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jonathan Ellis Assignee: T Jake Luciani Fix For: 2.1.3 Some of the trouble people have with stress is a documentation problem, but some is functional. Comments from [~iamaleksey]: # 3 clustering columns, make a million cells in a single partition, should be simple, but it's not. have to tweak 'clustering' on the three columns just right to make stress work at all. w/ some values it'd just gets stuck forever computing batches # for others, it generates huge, megabyte-size batches, utterly disrespecting 'select' clause in 'insert' # I want a sequential generator too, to be able to predict deterministic result sets. uniform() only gets you so far # impossible to simulate a time series workload /cc [~jshook] [~aweisberg] [~benedict] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[ https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274005#comment-14274005 ] Ariel Weisberg edited comment on CASSANDRA-7438 at 1/12/15 7:28 PM: If you go all the way down the JMH rabbit hole you don't need to do any of your own timing and JMH will actually do some smart things to give you accurate timing and ameliorate the impact of non-scalable/expensive timing measurement. Metrics uses System.nanoTime() internally so it isn't really any better as far as I can tell. System.nanoTime() on Linux is pretty scalable http://shipilev.net/blog/2014/nanotrusting-nanotime/. When I tested it in JMH it actually seemed to be linearly scalable, but JMH will solve that for you even on platforms where nanoTime is finicky. The C* integration looks good. I'm glad it was easy. When it comes to exposing configuration parameters less is more. I would prefer not to expose anything new because once people start using them they don't like to have the options taken away (or disabled). We should make an effort to set them automatically (or good enough) and if that fails we can add user visible configuration. My preference is to make the options accessible via properties as an escape hatch in production, and then add them to config if we really can't set them automatically. Can you prefix any System properties you have with a classname/package or something that makes it clear they are part of OHC? The stress tool when used without workload profiles does some validation. It checks that values are there and that the contents are correct. Did not know about the JNA synchronized block. That is surprising, but I am glad to hear it is getting fixed. For access to jemalloc I recommend using unsafe and LD_PRELOAD jemalloc. I think that would be the recommended approach and the one you should benchmark against and JNA would be there as a fallback. That gives you a JNI call for allocation/deallocation. I am trying out the JMH benchmark and looking at the new linked implementation right now. How are you starting the JMH benchmark? was (Author: aweisberg): If you go all the way down the JMH rabbit hole you don't need to do any of your own timing and JMH will actually do some smart things to give you accurate timing and ameliorate the impact of non-scalable/expensive timing measurement. Metrics uses System.nanoTime() internally so it isn't really any better as far as I can tell. System.nanoTime() on Linux is pretty scalable http://shipilev.net/blog/2014/nanotrusting-nanotime/. When I tested it in JMH it actually seemed to be linearly scalable, but JMH will solve that for you even on platforms where nanoTime is finicky. The C* integration looks good. I'm glad it was easy. When it comes to exposing configuration parameters less is more. I would prefer not to expose anything new because once people start using them they don't like to have the options taken away (or disabled). We should make an effort to set them automatically (or good enough) and if that fails we can add user visible configuration. My preference is to make the options accessible via properties as an escape hatch in production, and then add them to config if we really can't set them automatically. The stress tool when used without workload profiles does some validation. It checks that values are there and that the contents are correct. Did not know about the JNA synchronized block. That is surprising, but I am glad to hear it is getting fixed. For access to jemalloc I recommend using unsafe and LD_PRELOAD jemalloc. I think that would be the recommended approach and the one you should benchmark against and JNA would be there as a fallback. That gives you a JNI call for allocation/deallocation. I am trying out the JMH benchmark and looking at the new linked implementation right now. How are you starting the JMH benchmark? Serializing Row cache alternative (Fully off heap) -- Key: CASSANDRA-7438 URL: https://issues.apache.org/jira/browse/CASSANDRA-7438 Project: Cassandra Issue Type: Improvement Components: Core Environment: Linux Reporter: Vijay Assignee: Robert Stupp Labels: performance Fix For: 3.0 Attachments: 0001-CASSANDRA-7438.patch, tests.zip Currently SerializingCache is partially off heap, keys are still stored in JVM heap as BB, * There is a higher GC costs for a reasonably big cache. * Some users have used the row cache efficiently in production for better results, but this requires careful tunning. * Overhead in Memory for the cache entries are relatively high. So the proposal for this ticket is to move the LRU cache logic completely off heap and use
[jira] [Updated] (CASSANDRA-7933) Update cassandra-stress README
[ https://issues.apache.org/jira/browse/CASSANDRA-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson updated CASSANDRA-7933: --- Reviewer: (was: Philip Thompson) Update cassandra-stress README -- Key: CASSANDRA-7933 URL: https://issues.apache.org/jira/browse/CASSANDRA-7933 Project: Cassandra Issue Type: Task Reporter: Benedict Assignee: Philip Thompson Priority: Minor Fix For: 2.1.3 Attachments: CASSANDRA-7933.txt There is a README in the tools/stress directory. It is completely out of date. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8599) Refactor or fix CqlStorage
[ https://issues.apache.org/jira/browse/CASSANDRA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274163#comment-14274163 ] Brandon Williams commented on CASSANDRA-8599: - That's fine, but I don't want to remove something in a minor, so let's make CS wrap CNS and emit a deprecation warning, then we'll remove it in 3.0. Refactor or fix CqlStorage -- Key: CASSANDRA-8599 URL: https://issues.apache.org/jira/browse/CASSANDRA-8599 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Brandon Williams Assignee: Alex Liu Fix For: 2.1.3 In CASSANDRA-8541, for 2.1, I ultimately replace the non-existent CPIF references with CIF, since CNS was broken otherwise. But this means CS no longer works since it's not a simple drop in replacement (but have CNS work is better than having them both broken by a class that doesn't exist.) We can't just deprecate and remove CS either, because CNS extends it. We either need to fix CS to work with CIF, or we need to refactor CNS so that we can just remove CS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8599) Refactor or fix CqlStorage
[ https://issues.apache.org/jira/browse/CASSANDRA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274134#comment-14274134 ] Alex Liu commented on CASSANDRA-8599: - I am removing CS and refactoring CNS. Refactor or fix CqlStorage -- Key: CASSANDRA-8599 URL: https://issues.apache.org/jira/browse/CASSANDRA-8599 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Brandon Williams Assignee: Alex Liu Fix For: 2.1.3 In CASSANDRA-8541, for 2.1, I ultimately replace the non-existent CPIF references with CIF, since CNS was broken otherwise. But this means CS no longer works since it's not a simple drop in replacement (but have CNS work is better than having them both broken by a class that doesn't exist.) We can't just deprecate and remove CS either, because CNS extends it. We either need to fix CS to work with CIF, or we need to refactor CNS so that we can just remove CS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8600) Refactor stress options to reduce prominence of legacy modes (counter_read, etc.)
Benedict created CASSANDRA-8600: --- Summary: Refactor stress options to reduce prominence of legacy modes (counter_read, etc.) Key: CASSANDRA-8600 URL: https://issues.apache.org/jira/browse/CASSANDRA-8600 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Benedict Priority: Minor There;s an asymmetry to how we treat these, with all four legacy commands (read, write, counter_read, counter_write) getting their own full-blown command, whereas the user command is comparatively almost hidden. There are also two option sets that apply only to the legacy commands (schema and cols) which isn't at all apparent. We should clean this up, by e.g. renaming user to profile or something more obvious, and dropping the four old commands and renaming mixed to simple - this would give simple a similar spec style to profile. We should then make it clear that cols and schema only apply to the simple mode, and should fail if they are supplied otherwise. We can of course continue to accept all of the old ways of specifying commands if we want, and just not print them out in the help. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8601) cassandra-stress yaml profiles should support all -pop and -insert options
Benedict created CASSANDRA-8601: --- Summary: cassandra-stress yaml profiles should support all -pop and -insert options Key: CASSANDRA-8601 URL: https://issues.apache.org/jira/browse/CASSANDRA-8601 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Benedict The yaml file currently supports most -insert options, but for simplicity and uniformity it makes sense to offer all of them and the -pop options as well, with any command line options simply overriding the settings provided in the yaml (since many of these do make more sense to specify on the command line, for easy scripting). It might be nice, even, to permit the command ratios to be defined, and for multiple different sets of all of the above parameters to be defined in a profiles section, so that you could define a populate profile, and different kinds of read workload profiles, which can each easily be specified on the command line to define the complete workload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError
[
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274142#comment-14274142
]
Sebastian Estevez commented on CASSANDRA-8548:
--
The following procedure gets the sstable back in order (thanks Marcus):
1) Restart the node, this will release the SSTable which was not released
probably due to the SSTable corruption.
2) run nodetool scrub on the culprit sstables on that node to fix the
corruption.
3) Run cleanup.
Nodetool Cleanup - java.lang.AssertionError
---
Key: CASSANDRA-8548
URL: https://issues.apache.org/jira/browse/CASSANDRA-8548
Project: Cassandra
Issue Type: Bug
Reporter: Sebastian Estevez
Assignee: Marcus Eriksson
Fix For: 2.0.12
Attachments: 0001-make-sure-we-unmark-compacting.patch
Needed to free up some space on a node but getting the dump below when
running nodetool cleanup.
Tried turning on debug to try to obtain additional details in the logs but
nothing gets added to the logs when running cleanup. Added:
log4j.logger.org.apache.cassandra.db=DEBUG
in log4j-server.properties
See the stack trace below:
root@cassandra-019:~# nodetool cleanup
{code}Error occurred during cleanup
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:228)
at
org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:266)
at
org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1112)
at
org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2162)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
[jira] [Commented] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError
[
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274192#comment-14274192
]
Andrei Ivanov commented on CASSANDRA-8548:
--
What's the right procedure to restart the node? ( I ran disablebinary,
disablethrift, disablegossip, setcompactionthroughput 0 and then enabled
everything back).
Tried running scrub after that and I still get the same error.
Nodetool Cleanup - java.lang.AssertionError
---
Key: CASSANDRA-8548
URL: https://issues.apache.org/jira/browse/CASSANDRA-8548
Project: Cassandra
Issue Type: Bug
Reporter: Sebastian Estevez
Assignee: Marcus Eriksson
Fix For: 2.0.12
Attachments: 0001-make-sure-we-unmark-compacting.patch
Needed to free up some space on a node but getting the dump below when
running nodetool cleanup.
Tried turning on debug to try to obtain additional details in the logs but
nothing gets added to the logs when running cleanup. Added:
log4j.logger.org.apache.cassandra.db=DEBUG
in log4j-server.properties
See the stack trace below:
root@cassandra-019:~# nodetool cleanup
{code}Error occurred during cleanup
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:228)
at
org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:266)
at
org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1112)
at
org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2162)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by:
[jira] [Commented] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError
[
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274221#comment-14274221
]
Sebastian Estevez commented on CASSANDRA-8548:
--
[~aivanov93] stop the actual cassandra process and start it up again. Before
doing this ensure you have the right CL/RF settings to prevent downtime.
Nodetool Cleanup - java.lang.AssertionError
---
Key: CASSANDRA-8548
URL: https://issues.apache.org/jira/browse/CASSANDRA-8548
Project: Cassandra
Issue Type: Bug
Reporter: Sebastian Estevez
Assignee: Marcus Eriksson
Fix For: 2.0.12
Attachments: 0001-make-sure-we-unmark-compacting.patch
Needed to free up some space on a node but getting the dump below when
running nodetool cleanup.
Tried turning on debug to try to obtain additional details in the logs but
nothing gets added to the logs when running cleanup. Added:
log4j.logger.org.apache.cassandra.db=DEBUG
in log4j-server.properties
See the stack trace below:
root@cassandra-019:~# nodetool cleanup
{code}Error occurred during cleanup
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:228)
at
org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:266)
at
org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1112)
at
org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2162)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.IllegalArgumentException
at
Git Push Summary
Repository: cassandra Updated Tags: refs/tags/2.0.12-tentative [deleted] df1f5ead0
[jira] [Updated] (CASSANDRA-8558) deleted row still can be selected out
[
https://issues.apache.org/jira/browse/CASSANDRA-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-8558:
-
Fix Version/s: 2.0.12
deleted row still can be selected out
-
Key: CASSANDRA-8558
URL: https://issues.apache.org/jira/browse/CASSANDRA-8558
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: 2.1.2
java version 1.7.0_55
Reporter: zhaoyan
Assignee: Sylvain Lebresne
Priority: Blocker
Fix For: 2.0.12, 2.1.3
Attachments: 8558-v2_2.0.txt, 8558-v2_2.1.txt, 8558.txt
first
{code}CREATE KEYSPACE space1 WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 3};
CREATE TABLE space1.table3(a int, b int, c text,primary key(a,b));
CREATE KEYSPACE space2 WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 3};{code}
second
{code}CREATE TABLE space2.table1(a int, b int, c int, primary key(a,b));
CREATE TABLE space2.table2(a int, b int, c int, primary key(a,b));
INSERT INTO space1.table3(a,b,c) VALUES(1,1,'1');
drop table space2.table1;
DELETE FROM space1.table3 where a=1 and b=1;
drop table space2.table2;
select * from space1.table3 where a=1 and b=1;{code}
you will find that the row (a=1 and b=1) in space1.table3 is not deleted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8602) ArithmethicException: Divide by zero in agent (cassandra)
Catalin Alexandru Zamfir created CASSANDRA-8602:
---
Summary: ArithmethicException: Divide by zero in agent (cassandra)
Key: CASSANDRA-8602
URL: https://issues.apache.org/jira/browse/CASSANDRA-8602
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Catalin Alexandru Zamfir
We got the following exception and no data is currently showing on the graphs
in OpsCenter. From the datastax-agent logs:
{{
ERROR [jmx-metrics-2] 2015-01-11 03:55:00,000 Error getting CF metrics
java.lang.ArithmeticException: Divide by zero
at clojure.lang.Numbers.divide(Numbers.java:156)
at opsagent.rollup$transform_value.invoke(rollup.clj:43)
at opsagent.rollup$add_value.invoke(rollup.clj:132)
at opsagent.rollup$add_value.invoke(rollup.clj:150)
at opsagent.rollup$add_value.invoke(rollup.clj:150)
at opsagent.rollup$process_keypair$fn__701.invoke(rollup.clj:211)
at
opsagent.cache$update_cache_value_default$fn__481$fn__482.invoke(cache.clj:23)
at clojure.lang.AFn.applyToHelper(AFn.java:161)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.lang.Ref.alter(Ref.java:174)
at clojure.core$alter.doInvoke(core.clj:2244)
at clojure.lang.RestFn.invoke(RestFn.java:425)
at
opsagent.cache$update_cache_value_default$fn__481.invoke(cache.clj:23)
at clojure.lang.AFn.call(AFn.java:18)
at clojure.lang.LockingTransaction.run(LockingTransaction.java:263)
at
clojure.lang.LockingTransaction.runInTransaction(LockingTransaction.java:231)
at opsagent.cache$update_cache_value_default.invoke(cache.clj:22)
at opsagent.rollup$process_keypair.invoke(rollup.clj:211)
at opsagent.rollup$process_metric_map.invoke(rollup.clj:217)
at
opsagent.metrics.jmx$start_jmx_metric_collection$send_metrics__5266.invoke(jmx.clj:200)
at opsagent.metrics.jmx$cf_metric_helper.invoke(jmx.clj:92)
at opsagent.metrics.jmx$start_pool$fn__5238.invoke(jmx.clj:148)
at clojure.lang.AFn.run(AFn.java:24)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
ERROR [jmx-metrics-2] 2015-01-11 14:27:00,000 Error getting CF metrics
java.lang.ArithmeticException: Divide by zero
at clojure.lang.Numbers.divide(Numbers.java:156)
at opsagent.rollup$transform_value.invoke(rollup.clj:43)
at opsagent.rollup$add_value.invoke(rollup.clj:132)
at opsagent.rollup$add_value.invoke(rollup.clj:150)
at opsagent.rollup$add_value.invoke(rollup.clj:150)
at opsagent.rollup$process_keypair$fn__701.invoke(rollup.clj:211)
at
opsagent.cache$update_cache_value_default$fn__481$fn__482.invoke(cache.clj:23)
at clojure.lang.AFn.applyToHelper(AFn.java:161)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.lang.Ref.alter(Ref.java:174)
at clojure.core$alter.doInvoke(core.clj:2244)
at clojure.lang.RestFn.invoke(RestFn.java:425)
at
opsagent.cache$update_cache_value_default$fn__481.invoke(cache.clj:23)
at clojure.lang.AFn.call(AFn.java:18)
at clojure.lang.LockingTransaction.run(LockingTransaction.java:263)
at
clojure.lang.LockingTransaction.runInTransaction(LockingTransaction.java:231)
at opsagent.cache$update_cache_value_default.invoke(cache.clj:22)
at opsagent.rollup$process_keypair.invoke(rollup.clj:211)
at opsagent.rollup$process_metric_map.invoke(rollup.clj:217)
at
opsagent.metrics.jmx$start_jmx_metric_collection$send_metrics__5266.invoke(jmx.clj:200)
at opsagent.metrics.jmx$cf_metric_helper.invoke(jmx.clj:92)
at opsagent.metrics.jmx$start_pool$fn__5238.invoke(jmx.clj:148)
at clojure.lang.AFn.run(AFn.java:24)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
[jira] [Commented] (CASSANDRA-8558) deleted row still can be selected out
[
https://issues.apache.org/jira/browse/CASSANDRA-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274235#comment-14274235
]
Aleksey Yeschenko commented on CASSANDRA-8558:
--
This is blocking the 2.0.12 release now.
deleted row still can be selected out
-
Key: CASSANDRA-8558
URL: https://issues.apache.org/jira/browse/CASSANDRA-8558
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: 2.1.2
java version 1.7.0_55
Reporter: zhaoyan
Assignee: Sylvain Lebresne
Priority: Blocker
Fix For: 2.0.12, 2.1.3
Attachments: 8558-v2_2.0.txt, 8558-v2_2.1.txt, 8558.txt
first
{code}CREATE KEYSPACE space1 WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 3};
CREATE TABLE space1.table3(a int, b int, c text,primary key(a,b));
CREATE KEYSPACE space2 WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 3};{code}
second
{code}CREATE TABLE space2.table1(a int, b int, c int, primary key(a,b));
CREATE TABLE space2.table2(a int, b int, c int, primary key(a,b));
INSERT INTO space1.table3(a,b,c) VALUES(1,1,'1');
drop table space2.table1;
DELETE FROM space1.table3 where a=1 and b=1;
drop table space2.table2;
select * from space1.table3 where a=1 and b=1;{code}
you will find that the row (a=1 and b=1) in space1.table3 is not deleted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8599) Refactor or fix CqlStorage
[ https://issues.apache.org/jira/browse/CASSANDRA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated CASSANDRA-8599: Attachment: 8166_2.1_branch.txt v1 patch on 2.1 branch is attached which wrap CqlStorage on CqlNativeStorage to retain backward compatibility. Refactor or fix CqlStorage -- Key: CASSANDRA-8599 URL: https://issues.apache.org/jira/browse/CASSANDRA-8599 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Brandon Williams Assignee: Alex Liu Fix For: 2.1.3 Attachments: 8166_2.1_branch.txt In CASSANDRA-8541, for 2.1, I ultimately replace the non-existent CPIF references with CIF, since CNS was broken otherwise. But this means CS no longer works since it's not a simple drop in replacement (but have CNS work is better than having them both broken by a class that doesn't exist.) We can't just deprecate and remove CS either, because CNS extends it. We either need to fix CS to work with CIF, or we need to refactor CNS so that we can just remove CS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8599) Refactor or fix CqlStorage
[ https://issues.apache.org/jira/browse/CASSANDRA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated CASSANDRA-8599: Attachment: 8599-2.1-branch.txt Refactor or fix CqlStorage -- Key: CASSANDRA-8599 URL: https://issues.apache.org/jira/browse/CASSANDRA-8599 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Brandon Williams Assignee: Alex Liu Fix For: 2.1.3 Attachments: 8599-2.1-branch.txt In CASSANDRA-8541, for 2.1, I ultimately replace the non-existent CPIF references with CIF, since CNS was broken otherwise. But this means CS no longer works since it's not a simple drop in replacement (but have CNS work is better than having them both broken by a class that doesn't exist.) We can't just deprecate and remove CS either, because CNS extends it. We either need to fix CS to work with CIF, or we need to refactor CNS so that we can just remove CS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Git Push Summary
Repository: cassandra Updated Tags: refs/tags/2.0.12-tentative [created] 9e5a4fad7
Git Push Summary
Repository: cassandra Updated Tags: refs/tags/2.0.12-tentative [deleted] 9e5a4fad7
[jira] [Updated] (CASSANDRA-8602) ArithmethicException: Divide by zero in agent (cassandra)
[
https://issues.apache.org/jira/browse/CASSANDRA-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Catalin Alexandru Zamfir updated CASSANDRA-8602:
Description:
We got the following exception and no data is currently showing on the graphs
in OpsCenter. From the datastax-agent logs:
{code}
ERROR [jmx-metrics-2] 2015-01-11 03:55:00,000 Error getting CF metrics
java.lang.ArithmeticException: Divide by zero
at clojure.lang.Numbers.divide(Numbers.java:156)
at opsagent.rollup$transform_value.invoke(rollup.clj:43)
at opsagent.rollup$add_value.invoke(rollup.clj:132)
at opsagent.rollup$add_value.invoke(rollup.clj:150)
at opsagent.rollup$add_value.invoke(rollup.clj:150)
at opsagent.rollup$process_keypair$fn__701.invoke(rollup.clj:211)
at
opsagent.cache$update_cache_value_default$fn__481$fn__482.invoke(cache.clj:23)
at clojure.lang.AFn.applyToHelper(AFn.java:161)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.lang.Ref.alter(Ref.java:174)
at clojure.core$alter.doInvoke(core.clj:2244)
at clojure.lang.RestFn.invoke(RestFn.java:425)
at
opsagent.cache$update_cache_value_default$fn__481.invoke(cache.clj:23)
at clojure.lang.AFn.call(AFn.java:18)
at clojure.lang.LockingTransaction.run(LockingTransaction.java:263)
at
clojure.lang.LockingTransaction.runInTransaction(LockingTransaction.java:231)
at opsagent.cache$update_cache_value_default.invoke(cache.clj:22)
at opsagent.rollup$process_keypair.invoke(rollup.clj:211)
at opsagent.rollup$process_metric_map.invoke(rollup.clj:217)
at
opsagent.metrics.jmx$start_jmx_metric_collection$send_metrics__5266.invoke(jmx.clj:200)
at opsagent.metrics.jmx$cf_metric_helper.invoke(jmx.clj:92)
at opsagent.metrics.jmx$start_pool$fn__5238.invoke(jmx.clj:148)
at clojure.lang.AFn.run(AFn.java:24)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
ERROR [jmx-metrics-2] 2015-01-11 14:27:00,000 Error getting CF metrics
java.lang.ArithmeticException: Divide by zero
at clojure.lang.Numbers.divide(Numbers.java:156)
at opsagent.rollup$transform_value.invoke(rollup.clj:43)
at opsagent.rollup$add_value.invoke(rollup.clj:132)
at opsagent.rollup$add_value.invoke(rollup.clj:150)
at opsagent.rollup$add_value.invoke(rollup.clj:150)
at opsagent.rollup$process_keypair$fn__701.invoke(rollup.clj:211)
at
opsagent.cache$update_cache_value_default$fn__481$fn__482.invoke(cache.clj:23)
at clojure.lang.AFn.applyToHelper(AFn.java:161)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.lang.Ref.alter(Ref.java:174)
at clojure.core$alter.doInvoke(core.clj:2244)
at clojure.lang.RestFn.invoke(RestFn.java:425)
at
opsagent.cache$update_cache_value_default$fn__481.invoke(cache.clj:23)
at clojure.lang.AFn.call(AFn.java:18)
at clojure.lang.LockingTransaction.run(LockingTransaction.java:263)
at
clojure.lang.LockingTransaction.runInTransaction(LockingTransaction.java:231)
at opsagent.cache$update_cache_value_default.invoke(cache.clj:22)
at opsagent.rollup$process_keypair.invoke(rollup.clj:211)
at opsagent.rollup$process_metric_map.invoke(rollup.clj:217)
at
opsagent.metrics.jmx$start_jmx_metric_collection$send_metrics__5266.invoke(jmx.clj:200)
at opsagent.metrics.jmx$cf_metric_helper.invoke(jmx.clj:92)
at opsagent.metrics.jmx$start_pool$fn__5238.invoke(jmx.clj:148)
at clojure.lang.AFn.run(AFn.java:24)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744
{code}
was:
We got the following exception and no
[jira] [Updated] (CASSANDRA-8558) deleted row still can be selected out
[
https://issues.apache.org/jira/browse/CASSANDRA-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-8558:
-
Reviewer: Tyler Hobbs
deleted row still can be selected out
-
Key: CASSANDRA-8558
URL: https://issues.apache.org/jira/browse/CASSANDRA-8558
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: 2.1.2
java version 1.7.0_55
Reporter: zhaoyan
Assignee: Sylvain Lebresne
Priority: Blocker
Fix For: 2.0.12, 2.1.3
Attachments: 8558-v2_2.0.txt, 8558-v2_2.1.txt, 8558.txt
first
{code}CREATE KEYSPACE space1 WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 3};
CREATE TABLE space1.table3(a int, b int, c text,primary key(a,b));
CREATE KEYSPACE space2 WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 3};{code}
second
{code}CREATE TABLE space2.table1(a int, b int, c int, primary key(a,b));
CREATE TABLE space2.table2(a int, b int, c int, primary key(a,b));
INSERT INTO space1.table3(a,b,c) VALUES(1,1,'1');
drop table space2.table1;
DELETE FROM space1.table3 where a=1 and b=1;
drop table space2.table2;
select * from space1.table3 where a=1 and b=1;{code}
you will find that the row (a=1 and b=1) in space1.table3 is not deleted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8599) Refactor or fix CqlStorage
[ https://issues.apache.org/jira/browse/CASSANDRA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated CASSANDRA-8599: Attachment: (was: 8166_2.1_branch.txt) Refactor or fix CqlStorage -- Key: CASSANDRA-8599 URL: https://issues.apache.org/jira/browse/CASSANDRA-8599 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Brandon Williams Assignee: Alex Liu Fix For: 2.1.3 In CASSANDRA-8541, for 2.1, I ultimately replace the non-existent CPIF references with CIF, since CNS was broken otherwise. But this means CS no longer works since it's not a simple drop in replacement (but have CNS work is better than having them both broken by a class that doesn't exist.) We can't just deprecate and remove CS either, because CNS extends it. We either need to fix CS to work with CIF, or we need to refactor CNS so that we can just remove CS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8599) Refactor or fix CqlStorage
[ https://issues.apache.org/jira/browse/CASSANDRA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated CASSANDRA-8599: Attachment: (was: 8599-2.1-branch.txt) Refactor or fix CqlStorage -- Key: CASSANDRA-8599 URL: https://issues.apache.org/jira/browse/CASSANDRA-8599 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Brandon Williams Assignee: Alex Liu Fix For: 2.1.3 Attachments: 8599-2.1-branch.txt In CASSANDRA-8541, for 2.1, I ultimately replace the non-existent CPIF references with CIF, since CNS was broken otherwise. But this means CS no longer works since it's not a simple drop in replacement (but have CNS work is better than having them both broken by a class that doesn't exist.) We can't just deprecate and remove CS either, because CNS extends it. We either need to fix CS to work with CIF, or we need to refactor CNS so that we can just remove CS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8599) Refactor or fix CqlStorage
[ https://issues.apache.org/jira/browse/CASSANDRA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated CASSANDRA-8599: Attachment: 8599-2.1-branch.txt Refactor or fix CqlStorage -- Key: CASSANDRA-8599 URL: https://issues.apache.org/jira/browse/CASSANDRA-8599 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Brandon Williams Assignee: Alex Liu Fix For: 2.1.3 Attachments: 8599-2.1-branch.txt In CASSANDRA-8541, for 2.1, I ultimately replace the non-existent CPIF references with CIF, since CNS was broken otherwise. But this means CS no longer works since it's not a simple drop in replacement (but have CNS work is better than having them both broken by a class that doesn't exist.) We can't just deprecate and remove CS either, because CNS extends it. We either need to fix CS to work with CIF, or we need to refactor CNS so that we can just remove CS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8577) Values of set types not loading correctly into Pig
[
https://issues.apache.org/jira/browse/CASSANDRA-8577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274266#comment-14274266
]
Oksana Danylyshyn commented on CASSANDRA-8577:
--
With the patch applied, I am getting values of set types successfully loaded as
expected.
Thanks.
Values of set types not loading correctly into Pig
--
Key: CASSANDRA-8577
URL: https://issues.apache.org/jira/browse/CASSANDRA-8577
Project: Cassandra
Issue Type: Bug
Reporter: Oksana Danylyshyn
Assignee: Brandon Williams
Fix For: 2.1.3
Attachments: cassandra-2.1-8577.txt
Values of set types are not loading correctly from Cassandra (cql3 table,
Native protocol v3) into Pig using CqlNativeStorage.
When using Cassandra version 2.1.0 only empty values are loaded, and for
newer versions (2.1.1 and 2.1.2) the following error is received:
org.apache.cassandra.serializers.MarshalException: Unexpected extraneous
bytes after set value
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:94)
Steps to reproduce:
{code}cqlsh:socialdata CREATE TABLE test (
key varchar PRIMARY KEY,
tags setvarchar
);
cqlsh:socialdata insert into test (key, tags) values ('key', {'Running',
'onestep4red', 'running'});
cqlsh:socialdata select * from test;
key | tags
-+---
key | {'Running', 'onestep4red', 'running'}
(1 rows){code}
With version 2.1.0:
{code}grunt data = load 'cql://socialdata/test' using
org.apache.cassandra.hadoop.pig.CqlNativeStorage();
grunt dump data;
(key,()){code}
With version 2.1.2:
{code}grunt data = load 'cql://socialdata/test' using
org.apache.cassandra.hadoop.pig.CqlNativeStorage();
grunt dump data;
org.apache.cassandra.serializers.MarshalException: Unexpected extraneous
bytes after set value
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:94)
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:27)
at
org.apache.cassandra.hadoop.pig.AbstractCassandraStorage.cassandraToObj(AbstractCassandraStorage.java:796)
at
org.apache.cassandra.hadoop.pig.CqlStorage.cqlColumnToObj(CqlStorage.java:195)
at
org.apache.cassandra.hadoop.pig.CqlNativeStorage.getNext(CqlNativeStorage.java:106)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212){code}
Expected result:
{code}(key,(Running,onestep4red,running)){code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8599) Refactor or fix CqlStorage
[ https://issues.apache.org/jira/browse/CASSANDRA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274300#comment-14274300 ] Brandon Williams commented on CASSANDRA-8599: - We should still emit a warning that CS is deprecated and they should use CNS instead so we can eventually remove CS. Also a lot of the imports could be combined (generally past 3, we just import *) Refactor or fix CqlStorage -- Key: CASSANDRA-8599 URL: https://issues.apache.org/jira/browse/CASSANDRA-8599 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Brandon Williams Assignee: Alex Liu Fix For: 2.1.3 Attachments: 8599-2.1-branch.txt In CASSANDRA-8541, for 2.1, I ultimately replace the non-existent CPIF references with CIF, since CNS was broken otherwise. But this means CS no longer works since it's not a simple drop in replacement (but have CNS work is better than having them both broken by a class that doesn't exist.) We can't just deprecate and remove CS either, because CNS extends it. We either need to fix CS to work with CIF, or we need to refactor CNS so that we can just remove CS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (CASSANDRA-8577) Values of set types not loading correctly into Pig
[
https://issues.apache.org/jira/browse/CASSANDRA-8577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams reopened CASSANDRA-8577:
-
Reproduced In: 2.1.2, 2.1.1 (was: 2.1.1, 2.1.2)
Values of set types not loading correctly into Pig
--
Key: CASSANDRA-8577
URL: https://issues.apache.org/jira/browse/CASSANDRA-8577
Project: Cassandra
Issue Type: Bug
Reporter: Oksana Danylyshyn
Assignee: Brandon Williams
Fix For: 2.1.3
Attachments: cassandra-2.1-8577.txt
Values of set types are not loading correctly from Cassandra (cql3 table,
Native protocol v3) into Pig using CqlNativeStorage.
When using Cassandra version 2.1.0 only empty values are loaded, and for
newer versions (2.1.1 and 2.1.2) the following error is received:
org.apache.cassandra.serializers.MarshalException: Unexpected extraneous
bytes after set value
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:94)
Steps to reproduce:
{code}cqlsh:socialdata CREATE TABLE test (
key varchar PRIMARY KEY,
tags setvarchar
);
cqlsh:socialdata insert into test (key, tags) values ('key', {'Running',
'onestep4red', 'running'});
cqlsh:socialdata select * from test;
key | tags
-+---
key | {'Running', 'onestep4red', 'running'}
(1 rows){code}
With version 2.1.0:
{code}grunt data = load 'cql://socialdata/test' using
org.apache.cassandra.hadoop.pig.CqlNativeStorage();
grunt dump data;
(key,()){code}
With version 2.1.2:
{code}grunt data = load 'cql://socialdata/test' using
org.apache.cassandra.hadoop.pig.CqlNativeStorage();
grunt dump data;
org.apache.cassandra.serializers.MarshalException: Unexpected extraneous
bytes after set value
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:94)
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:27)
at
org.apache.cassandra.hadoop.pig.AbstractCassandraStorage.cassandraToObj(AbstractCassandraStorage.java:796)
at
org.apache.cassandra.hadoop.pig.CqlStorage.cqlColumnToObj(CqlStorage.java:195)
at
org.apache.cassandra.hadoop.pig.CqlNativeStorage.getNext(CqlNativeStorage.java:106)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212){code}
Expected result:
{code}(key,(Running,onestep4red,running)){code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8577) Values of set types not loading correctly into Pig
[
https://issues.apache.org/jira/browse/CASSANDRA-8577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274309#comment-14274309
]
Brandon Williams commented on CASSANDRA-8577:
-
[~Oksana Danylyshyn] are you replacing the jar as Artem described? I notice
your description says native V3, but we don't ship with
cassandra-driver-core-2.1.
Values of set types not loading correctly into Pig
--
Key: CASSANDRA-8577
URL: https://issues.apache.org/jira/browse/CASSANDRA-8577
Project: Cassandra
Issue Type: Bug
Reporter: Oksana Danylyshyn
Assignee: Brandon Williams
Fix For: 2.1.3
Attachments: cassandra-2.1-8577.txt
Values of set types are not loading correctly from Cassandra (cql3 table,
Native protocol v3) into Pig using CqlNativeStorage.
When using Cassandra version 2.1.0 only empty values are loaded, and for
newer versions (2.1.1 and 2.1.2) the following error is received:
org.apache.cassandra.serializers.MarshalException: Unexpected extraneous
bytes after set value
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:94)
Steps to reproduce:
{code}cqlsh:socialdata CREATE TABLE test (
key varchar PRIMARY KEY,
tags setvarchar
);
cqlsh:socialdata insert into test (key, tags) values ('key', {'Running',
'onestep4red', 'running'});
cqlsh:socialdata select * from test;
key | tags
-+---
key | {'Running', 'onestep4red', 'running'}
(1 rows){code}
With version 2.1.0:
{code}grunt data = load 'cql://socialdata/test' using
org.apache.cassandra.hadoop.pig.CqlNativeStorage();
grunt dump data;
(key,()){code}
With version 2.1.2:
{code}grunt data = load 'cql://socialdata/test' using
org.apache.cassandra.hadoop.pig.CqlNativeStorage();
grunt dump data;
org.apache.cassandra.serializers.MarshalException: Unexpected extraneous
bytes after set value
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:94)
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:27)
at
org.apache.cassandra.hadoop.pig.AbstractCassandraStorage.cassandraToObj(AbstractCassandraStorage.java:796)
at
org.apache.cassandra.hadoop.pig.CqlStorage.cqlColumnToObj(CqlStorage.java:195)
at
org.apache.cassandra.hadoop.pig.CqlNativeStorage.getNext(CqlNativeStorage.java:106)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212){code}
Expected result:
{code}(key,(Running,onestep4red,running)){code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7933) Update cassandra-stress README
[ https://issues.apache.org/jira/browse/CASSANDRA-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson updated CASSANDRA-7933: --- Reviewer: Benedict Attachment: 7933-2.txt I've given the README a quick update, just so that it no longer contains only correct information accurate for the 2.1 stress. I agree with adding everything [~aweisberg] suggested, but I don't have experience with new stress beyond the very basic read/write commands. If the README should contain more than a basic overview of options, then it would probably be best to reassign this ticket. However, I do feel that shipping 2.1.3 with a completely useless README is unacceptable, given that the README will have been out of date since 2.1.0. Given that, I would like to suggest opening a new ticket for 'Adding advanced use guide to cassandra-stress README' to ensure that the minor, but necessary changes I have made make it into the impending 2.1.3 release, giving Ariel or Benedict time to add more advanced documentation. Update cassandra-stress README -- Key: CASSANDRA-7933 URL: https://issues.apache.org/jira/browse/CASSANDRA-7933 Project: Cassandra Issue Type: Task Reporter: Benedict Assignee: Philip Thompson Priority: Minor Fix For: 2.1.3 Attachments: 7933-2.txt, CASSANDRA-7933.txt There is a README in the tools/stress directory. It is completely out of date. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7933) Update cassandra-stress README
[ https://issues.apache.org/jira/browse/CASSANDRA-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274326#comment-14274326 ] Philip Thompson edited comment on CASSANDRA-7933 at 1/12/15 10:51 PM: -- I've given the README a quick update, just so that it no longer contains only correct information accurate for the 2.1 stress. I agree with adding everything [~aweisberg] suggested, but I don't have experience with new stress beyond the very basic read/write commands. If the README should contain more than a basic overview of options, then it would probably be best to reassign this ticket. However, I do feel that shipping 2.1.3 with a completely useless README is unacceptable, given that the README will have been out of date since 2.1.0. Given that, I would like to suggest opening a new ticket for 'Adding advanced use guide to cassandra-stress README' to ensure that the minor, but necessary changes I have made make it into the impending 2.1.3 release as part of this ticket, giving Ariel or Benedict time to add more advanced documentation. was (Author: philipthompson): I've given the README a quick update, just so that it no longer contains only correct information accurate for the 2.1 stress. I agree with adding everything [~aweisberg] suggested, but I don't have experience with new stress beyond the very basic read/write commands. If the README should contain more than a basic overview of options, then it would probably be best to reassign this ticket. However, I do feel that shipping 2.1.3 with a completely useless README is unacceptable, given that the README will have been out of date since 2.1.0. Given that, I would like to suggest opening a new ticket for 'Adding advanced use guide to cassandra-stress README' to ensure that the minor, but necessary changes I have made make it into the impending 2.1.3 release, giving Ariel or Benedict time to add more advanced documentation. Update cassandra-stress README -- Key: CASSANDRA-7933 URL: https://issues.apache.org/jira/browse/CASSANDRA-7933 Project: Cassandra Issue Type: Task Reporter: Benedict Assignee: Philip Thompson Priority: Minor Fix For: 2.1.3 Attachments: 7933-2.txt, CASSANDRA-7933.txt There is a README in the tools/stress directory. It is completely out of date. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8602) ArithmethicException: Divide by zero in agent (cassandra)
[
https://issues.apache.org/jira/browse/CASSANDRA-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Bailey resolved CASSANDRA-8602.
Resolution: Invalid
So this is just the issue tracker for Apache Cassandra doesn't include tracking
issues for OpsCenter, so I'm going to close this here.
The best way to report issues like this for OpsCenter is via the feedback form
in the OpsCenter interface.
For this particular bug, we are tracking this internally and it should be fixed
in the next major release of OpsCenter. In the meantime, the bug should be
mostly harmless, except for the alarming logging. Thanks for the report though.
ArithmethicException: Divide by zero in agent (cassandra)
-
Key: CASSANDRA-8602
URL: https://issues.apache.org/jira/browse/CASSANDRA-8602
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Catalin Alexandru Zamfir
We got the following exception and no data is currently showing on the graphs
in OpsCenter. From the datastax-agent logs:
{code}
ERROR [jmx-metrics-2] 2015-01-11 03:55:00,000 Error getting CF metrics
java.lang.ArithmeticException: Divide by zero
at clojure.lang.Numbers.divide(Numbers.java:156)
at opsagent.rollup$transform_value.invoke(rollup.clj:43)
at opsagent.rollup$add_value.invoke(rollup.clj:132)
at opsagent.rollup$add_value.invoke(rollup.clj:150)
at opsagent.rollup$add_value.invoke(rollup.clj:150)
at opsagent.rollup$process_keypair$fn__701.invoke(rollup.clj:211)
at
opsagent.cache$update_cache_value_default$fn__481$fn__482.invoke(cache.clj:23)
at clojure.lang.AFn.applyToHelper(AFn.java:161)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.lang.Ref.alter(Ref.java:174)
at clojure.core$alter.doInvoke(core.clj:2244)
at clojure.lang.RestFn.invoke(RestFn.java:425)
at
opsagent.cache$update_cache_value_default$fn__481.invoke(cache.clj:23)
at clojure.lang.AFn.call(AFn.java:18)
at clojure.lang.LockingTransaction.run(LockingTransaction.java:263)
at
clojure.lang.LockingTransaction.runInTransaction(LockingTransaction.java:231)
at opsagent.cache$update_cache_value_default.invoke(cache.clj:22)
at opsagent.rollup$process_keypair.invoke(rollup.clj:211)
at opsagent.rollup$process_metric_map.invoke(rollup.clj:217)
at
opsagent.metrics.jmx$start_jmx_metric_collection$send_metrics__5266.invoke(jmx.clj:200)
at opsagent.metrics.jmx$cf_metric_helper.invoke(jmx.clj:92)
at opsagent.metrics.jmx$start_pool$fn__5238.invoke(jmx.clj:148)
at clojure.lang.AFn.run(AFn.java:24)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
ERROR [jmx-metrics-2] 2015-01-11 14:27:00,000 Error getting CF metrics
java.lang.ArithmeticException: Divide by zero
at clojure.lang.Numbers.divide(Numbers.java:156)
at opsagent.rollup$transform_value.invoke(rollup.clj:43)
at opsagent.rollup$add_value.invoke(rollup.clj:132)
at opsagent.rollup$add_value.invoke(rollup.clj:150)
at opsagent.rollup$add_value.invoke(rollup.clj:150)
at opsagent.rollup$process_keypair$fn__701.invoke(rollup.clj:211)
at
opsagent.cache$update_cache_value_default$fn__481$fn__482.invoke(cache.clj:23)
at clojure.lang.AFn.applyToHelper(AFn.java:161)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.lang.Ref.alter(Ref.java:174)
at clojure.core$alter.doInvoke(core.clj:2244)
at clojure.lang.RestFn.invoke(RestFn.java:425)
at
opsagent.cache$update_cache_value_default$fn__481.invoke(cache.clj:23)
at clojure.lang.AFn.call(AFn.java:18)
at clojure.lang.LockingTransaction.run(LockingTransaction.java:263)
at
clojure.lang.LockingTransaction.runInTransaction(LockingTransaction.java:231)
at opsagent.cache$update_cache_value_default.invoke(cache.clj:22)
at opsagent.rollup$process_keypair.invoke(rollup.clj:211)
at opsagent.rollup$process_metric_map.invoke(rollup.clj:217)
at
[jira] [Updated] (CASSANDRA-8599) Refactor or fix CqlStorage
[ https://issues.apache.org/jira/browse/CASSANDRA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated CASSANDRA-8599: Attachment: 8599-v2-2.1-branch.txt Refactor or fix CqlStorage -- Key: CASSANDRA-8599 URL: https://issues.apache.org/jira/browse/CASSANDRA-8599 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Brandon Williams Assignee: Alex Liu Fix For: 2.1.3 Attachments: 8599-2.1-branch.txt, 8599-v2-2.1-branch.txt In CASSANDRA-8541, for 2.1, I ultimately replace the non-existent CPIF references with CIF, since CNS was broken otherwise. But this means CS no longer works since it's not a simple drop in replacement (but have CNS work is better than having them both broken by a class that doesn't exist.) We can't just deprecate and remove CS either, because CNS extends it. We either need to fix CS to work with CIF, or we need to refactor CNS so that we can just remove CS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8414) Avoid loops over array backed iterators that call iter.remove()
[ https://issues.apache.org/jira/browse/CASSANDRA-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274593#comment-14274593 ] Richard Low commented on CASSANDRA-8414: Only minor nit is that the BitSet can be initialized with size rather than cells.length, but otherwise +1. Avoid loops over array backed iterators that call iter.remove() --- Key: CASSANDRA-8414 URL: https://issues.apache.org/jira/browse/CASSANDRA-8414 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Richard Low Assignee: Jimmy Mårdell Labels: performance Fix For: 2.1.3 Attachments: cassandra-2.0-8414-1.txt, cassandra-2.0-8414-2.txt, cassandra-2.0-8414-3.txt, cassandra-2.0-8414-4.txt, cassandra-2.0-8414-5.txt, cassandra-2.1-8414-5.txt I noticed from sampling that sometimes compaction spends almost all of its time in iter.remove() in ColumnFamilyStore.removeDeletedStandard. It turns out that the cf object is using ArrayBackedSortedColumns, so deletes are from an ArrayList. If the majority of your columns are GCable tombstones then this is O(n^2). The data structure should be changed or a copy made to avoid this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8599) Refactor or fix CqlStorage
[ https://issues.apache.org/jira/browse/CASSANDRA-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274370#comment-14274370 ] Alex Liu commented on CASSANDRA-8599: - V2 is attached to add deprecation warning message and combine imports Refactor or fix CqlStorage -- Key: CASSANDRA-8599 URL: https://issues.apache.org/jira/browse/CASSANDRA-8599 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Brandon Williams Assignee: Alex Liu Fix For: 2.1.3 Attachments: 8599-2.1-branch.txt, 8599-v2-2.1-branch.txt In CASSANDRA-8541, for 2.1, I ultimately replace the non-existent CPIF references with CIF, since CNS was broken otherwise. But this means CS no longer works since it's not a simple drop in replacement (but have CNS work is better than having them both broken by a class that doesn't exist.) We can't just deprecate and remove CS either, because CNS extends it. We either need to fix CS to work with CIF, or we need to refactor CNS so that we can just remove CS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8594) pidfile is never filled by cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274570#comment-14274570 ] Cyril Scetbon commented on CASSANDRA-8594: -- bin/cassandra only sets the dedicated cassandra parameter [here|https://github.com/cscetbon/cassandra/blob/cassandra-2.0.10/bin/cassandra#L139] which doesn't write the pid at all. In 1.2, jsvc was used and it was this tool that [was setting the pid|https://github.com/apache/cassandra/blob/cassandra-1.2.13/debian/init#L136-L139]. But now that we don't use it (2.0+) we use [start-stop-daemon|https://github.com/cscetbon/cassandra/blob/cassandra-2.0.10/debian/init#L96] without asking it to create the pidfile. And as bin/cassandra says that it [stores the pid|https://github.com/cscetbon/cassandra/blob/cassandra-2.0.10/bin/cassandra#L21] if we use -p pidfile, I think it's time to implement it in cassandra. pidfile is never filled by cassandra Key: CASSANDRA-8594 URL: https://issues.apache.org/jira/browse/CASSANDRA-8594 Project: Cassandra Issue Type: Bug Reporter: Cyril Scetbon Assignee: Cyril Scetbon Fix For: 2.0.13 The pid file is never filled by cassandra. there is only a File object that is created with [those lines|https://github.com/cscetbon/cassandra/blob/cassandra-2.0.10/src/java/org/apache/cassandra/service/CassandraDaemon.java#L498-L501] Here is a [fix|https://github.com/cscetbon/cassandra/commit/d0c5e0c9be00e48e6d0cd0de208c53274f1919c0.patch] that writes the current PID into the pidfile -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[ https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274409#comment-14274409 ] Ariel Weisberg commented on CASSANDRA-7438: --- I did another review. The additional test coverage looks great. Don’t throw Error, throw runtime exceptions on things like serialization issues. The only place it make sense to throw error is when allocating memory fails. That would match the behavior of ByteBuffer.allocateDirect. I don’t see failure to allocate from the heap allocator as recoverable even in the context of the cache. IOError is thrown from one place in the entire JDK (Console) so it's an odd choice. freeCapacity should reall be a field inside each segment and full/not full and eviction decisions should be made inside each segment independently. In practice inside C* it’s probably fine as just an AtomicLong, but I want to see OHC be all it can be. Rehash test could validate the data. After the rehash. It could also validate the rehash under concurrent access, say have a reader thread that is randomly accessing values the already inserted value.I can’t tell if the crosscheck test inserts enough values to trigger rehashing. Inlining the murmur3 changes makes me a little uncomfortable. It’s good see see some test coverage comparing with another implementation, but it’s over a small set of data. It seems like the Unsigned stuff necessary to perfectly mimic the native version of murmur3 is missing? Add 2-4 byte coed points for the UTF-8 tests. FastUtil is a 17 megabyte dependency all to get one array list. The cross checking implementation is really nice. Looking at the AbstractKeyIterator, I don’t see how it can do the right thing when a segment rehashes. It will point to a random spot in the segment after a rehash right? In practice maybe this doesn’t matter since they should size up promptly and it’s just an optimization that we dump this stuff at all. I can understand what the current code does so I lean towards keeping it. There are a couple of places (serializeForPut, putInternal, maybe others) where there are two exception handlers that each de-allocate the same piece of memory. The deallocation could go in a finally instead of the exception handlers since it always happens. Serializing Row cache alternative (Fully off heap) -- Key: CASSANDRA-7438 URL: https://issues.apache.org/jira/browse/CASSANDRA-7438 Project: Cassandra Issue Type: Improvement Components: Core Environment: Linux Reporter: Vijay Assignee: Robert Stupp Labels: performance Fix For: 3.0 Attachments: 0001-CASSANDRA-7438.patch, tests.zip Currently SerializingCache is partially off heap, keys are still stored in JVM heap as BB, * There is a higher GC costs for a reasonably big cache. * Some users have used the row cache efficiently in production for better results, but this requires careful tunning. * Overhead in Memory for the cache entries are relatively high. So the proposal for this ticket is to move the LRU cache logic completely off heap and use JNI to interact with cache. We might want to ensure that the new implementation match the existing API's (ICache), and the implementation needs to have safe memory access, low overhead in memory and less memcpy's (As much as possible). We might also want to make this cache configurable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8598) Windows - SSTableRewriterTest fails on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-8598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-8598:
---
Reviewer: Marcus Eriksson
Windows - SSTableRewriterTest fails on trunk
Key: CASSANDRA-8598
URL: https://issues.apache.org/jira/browse/CASSANDRA-8598
Project: Cassandra
Issue Type: Bug
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
Priority: Minor
Labels: Windows
Fix For: 3.0
Attachments: 8598_v1.txt
Right at the top of the test, we see:
{noformat}
[junit] ERROR 18:15:05 Unable to delete
build\test\cassandra\data;0\SSTableRewriterTest\Standard1-e63f49c09a8611e4bebb8ff5e6ab1035\tmplink-la-27-big-Data.db
(it will be removed on server restart; we'll also retry after GC)
[junit] ERROR 18:15:05 Unable to delete
build\test\cassandra\data;0\SSTableRewriterTest\Standard1-e63f49c09a8611e4bebb8ff5e6ab1035\tmplink-la-27-big-Data.db
(it will be removed on server restart; we'll also retry after GC)
[junit] - ---
[junit] Testcase:
testFileRemoval(org.apache.cassandra.io.sstable.SSTableRewriterTest):
FAILED
[junit] expected:0 but was:2
[junit] junit.framework.AssertionFailedError: expected:0 but was:2
[junit] at
org.apache.cassandra.io.sstable.SSTableRewriterTest.assertFileCounts(SSTableRewriterTest.java:758)
[junit] at
org.apache.cassandra.io.sstable.SSTableRewriterTest.testFileRemoval(SSTableRewriterTest.java:229)
{noformat}
The rest cascade after that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8577) Values of set types not loading correctly into Pig
[
https://issues.apache.org/jira/browse/CASSANDRA-8577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273865#comment-14273865
]
Philip Thompson edited comment on CASSANDRA-8577 at 1/12/15 6:05 PM:
-
Running the reproduction steps provided by [~Oksana Danylyshyn], I successfully
see the expected result {code}(key,(Running,onestep4red,running)){code} on
2.1-HEAD
was (Author: philipthompson):
Running the reproduction steps provided by [~Oksana Danylyshyn], I successfully
see the expected result {{(key,(Running,onestep4red,running))}}
Values of set types not loading correctly into Pig
--
Key: CASSANDRA-8577
URL: https://issues.apache.org/jira/browse/CASSANDRA-8577
Project: Cassandra
Issue Type: Bug
Reporter: Oksana Danylyshyn
Assignee: Brandon Williams
Fix For: 2.1.3
Attachments: cassandra-2.1-8577.txt
Values of set types are not loading correctly from Cassandra (cql3 table,
Native protocol v3) into Pig using CqlNativeStorage.
When using Cassandra version 2.1.0 only empty values are loaded, and for
newer versions (2.1.1 and 2.1.2) the following error is received:
org.apache.cassandra.serializers.MarshalException: Unexpected extraneous
bytes after set value
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:94)
Steps to reproduce:
{code}cqlsh:socialdata CREATE TABLE test (
key varchar PRIMARY KEY,
tags setvarchar
);
cqlsh:socialdata insert into test (key, tags) values ('key', {'Running',
'onestep4red', 'running'});
cqlsh:socialdata select * from test;
key | tags
-+---
key | {'Running', 'onestep4red', 'running'}
(1 rows){code}
With version 2.1.0:
{code}grunt data = load 'cql://socialdata/test' using
org.apache.cassandra.hadoop.pig.CqlNativeStorage();
grunt dump data;
(key,()){code}
With version 2.1.2:
{code}grunt data = load 'cql://socialdata/test' using
org.apache.cassandra.hadoop.pig.CqlNativeStorage();
grunt dump data;
org.apache.cassandra.serializers.MarshalException: Unexpected extraneous
bytes after set value
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:94)
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:27)
at
org.apache.cassandra.hadoop.pig.AbstractCassandraStorage.cassandraToObj(AbstractCassandraStorage.java:796)
at
org.apache.cassandra.hadoop.pig.CqlStorage.cqlColumnToObj(CqlStorage.java:195)
at
org.apache.cassandra.hadoop.pig.CqlNativeStorage.getNext(CqlNativeStorage.java:106)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212){code}
Expected result:
{code}(key,(Running,onestep4red,running)){code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8577) Values of set types not loading correctly into Pig
[
https://issues.apache.org/jira/browse/CASSANDRA-8577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273865#comment-14273865
]
Philip Thompson edited comment on CASSANDRA-8577 at 1/12/15 6:07 PM:
-
Running the reproduction steps provided by [~Oksana Danylyshyn], I successfully
see the expected result {code}(key,(Running,onestep4red,running)){code} on
2.1-HEAD and on 2.1.2.
was (Author: philipthompson):
Running the reproduction steps provided by [~Oksana Danylyshyn], I successfully
see the expected result {code}(key,(Running,onestep4red,running)){code} on
2.1-HEAD
Values of set types not loading correctly into Pig
--
Key: CASSANDRA-8577
URL: https://issues.apache.org/jira/browse/CASSANDRA-8577
Project: Cassandra
Issue Type: Bug
Reporter: Oksana Danylyshyn
Assignee: Brandon Williams
Fix For: 2.1.3
Attachments: cassandra-2.1-8577.txt
Values of set types are not loading correctly from Cassandra (cql3 table,
Native protocol v3) into Pig using CqlNativeStorage.
When using Cassandra version 2.1.0 only empty values are loaded, and for
newer versions (2.1.1 and 2.1.2) the following error is received:
org.apache.cassandra.serializers.MarshalException: Unexpected extraneous
bytes after set value
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:94)
Steps to reproduce:
{code}cqlsh:socialdata CREATE TABLE test (
key varchar PRIMARY KEY,
tags setvarchar
);
cqlsh:socialdata insert into test (key, tags) values ('key', {'Running',
'onestep4red', 'running'});
cqlsh:socialdata select * from test;
key | tags
-+---
key | {'Running', 'onestep4red', 'running'}
(1 rows){code}
With version 2.1.0:
{code}grunt data = load 'cql://socialdata/test' using
org.apache.cassandra.hadoop.pig.CqlNativeStorage();
grunt dump data;
(key,()){code}
With version 2.1.2:
{code}grunt data = load 'cql://socialdata/test' using
org.apache.cassandra.hadoop.pig.CqlNativeStorage();
grunt dump data;
org.apache.cassandra.serializers.MarshalException: Unexpected extraneous
bytes after set value
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:94)
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:27)
at
org.apache.cassandra.hadoop.pig.AbstractCassandraStorage.cassandraToObj(AbstractCassandraStorage.java:796)
at
org.apache.cassandra.hadoop.pig.CqlStorage.cqlColumnToObj(CqlStorage.java:195)
at
org.apache.cassandra.hadoop.pig.CqlNativeStorage.getNext(CqlNativeStorage.java:106)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212){code}
Expected result:
{code}(key,(Running,onestep4red,running)){code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7404) Use direct i/o for sequential operations (compaction/streaming)
[ https://issues.apache.org/jira/browse/CASSANDRA-7404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273870#comment-14273870 ] Ariel Weisberg commented on CASSANDRA-7404: --- We are trying to use direct IO for reads to reduce the impact of reads against the page cache. For writes we benefit from allowing the page cache and IO scheduler to do their thing to an extent. It's not clear yet whether the page cache needs any help in determining hot vs cold. I am going to circle back to this and see if I can find a benchmark that benefits. Use direct i/o for sequential operations (compaction/streaming) --- Key: CASSANDRA-7404 URL: https://issues.apache.org/jira/browse/CASSANDRA-7404 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jason Brown Assignee: Ariel Weisberg Labels: performance Fix For: 3.0 Investigate using linux's direct i/o for operations where we read sequentially through a file (repair and bootstrap streaming, compaction reads, and so on). Direct i/o does not go through the kernel page page, so it should leave the hot cache pages used for live reads unaffected. Note: by using direct i/o, we will probably take a performance hit on reading the file we're sequentially scanning through (that is, compactions may get slower), but the goal of this ticket is to limit the impact of these background tasks on the main read/write functionality. Of course, I'll measure any perf hit that is incurred, and see if there's any mechanisms to mitigate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8598) Windows - SSTableRewriterTest fails on trunk
Joshua McKenzie created CASSANDRA-8598:
--
Summary: Windows - SSTableRewriterTest fails on trunk
Key: CASSANDRA-8598
URL: https://issues.apache.org/jira/browse/CASSANDRA-8598
Project: Cassandra
Issue Type: Bug
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
Priority: Minor
Fix For: 3.0
Right at the top of the test, we see:
{noformat}
[junit] ERROR 18:15:05 Unable to delete
build\test\cassandra\data;0\SSTableRewriterTest\Standard1-e63f49c09a8611e4bebb8ff5e6ab1035\tmplink-la-27-big-Data.db
(it will be removed on server restart; we'll also retry after GC)
[junit] ERROR 18:15:05 Unable to delete
build\test\cassandra\data;0\SSTableRewriterTest\Standard1-e63f49c09a8611e4bebb8ff5e6ab1035\tmplink-la-27-big-Data.db
(it will be removed on server restart; we'll also retry after GC)
[junit] - ---
[junit] Testcase:
testFileRemoval(org.apache.cassandra.io.sstable.SSTableRewriterTest): FAILED
[junit] expected:0 but was:2
[junit] junit.framework.AssertionFailedError: expected:0 but was:2
[junit] at
org.apache.cassandra.io.sstable.SSTableRewriterTest.assertFileCounts(SSTableRewriterTest.java:758)
[junit] at
org.apache.cassandra.io.sstable.SSTableRewriterTest.testFileRemoval(SSTableRewriterTest.java:229)
{noformat}
The rest cascade after that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8598) Windows - SSTableRewriterTest fails on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-8598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-8598:
---
Attachment: 8598_v1.txt
SSTableRewriterTest.testFileRemoval was failing and leaving files sitting
around causing other tests to fail. It looks like the root cause is that after
the call to s.releaseReference(), we're expecting the tmplink files to be
removed and asserting to that effect. This fails on Windows since
SequentialWriter.out is a RandomAccessFile, meaning it doesn't have the
FILE_SHARE_DELETE flag so files can't be deleted while still open. Indeed, the
Unable to delete SSTableDeletingTask message pops up in the logs during that
unit test right before assertion failure.
I've attached a v1 that changes some of the logic on the test (marks s2 as
replacing s1) and removes the assertion that expects the tmplink file to be
gone before the writer is closed or aborted and the test now passes on trunk in
Windows. Ultimately the test is a little less granular and we lose the
confirmation that the releaseReferences deletes the files before the writer is
closed (since that differs based on platform), however after CASSANDRA-8535 and
CASSANDRA-8551 this problem will likely be addressed.
[~krummas]: Care to review? Should be trivial and you're quite familiar with
the test code in question.
Windows - SSTableRewriterTest fails on trunk
Key: CASSANDRA-8598
URL: https://issues.apache.org/jira/browse/CASSANDRA-8598
Project: Cassandra
Issue Type: Bug
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
Priority: Minor
Labels: Windows
Fix For: 3.0
Attachments: 8598_v1.txt
Right at the top of the test, we see:
{noformat}
[junit] ERROR 18:15:05 Unable to delete
build\test\cassandra\data;0\SSTableRewriterTest\Standard1-e63f49c09a8611e4bebb8ff5e6ab1035\tmplink-la-27-big-Data.db
(it will be removed on server restart; we'll also retry after GC)
[junit] ERROR 18:15:05 Unable to delete
build\test\cassandra\data;0\SSTableRewriterTest\Standard1-e63f49c09a8611e4bebb8ff5e6ab1035\tmplink-la-27-big-Data.db
(it will be removed on server restart; we'll also retry after GC)
[junit] - ---
[junit] Testcase:
testFileRemoval(org.apache.cassandra.io.sstable.SSTableRewriterTest):
FAILED
[junit] expected:0 but was:2
[junit] junit.framework.AssertionFailedError: expected:0 but was:2
[junit] at
org.apache.cassandra.io.sstable.SSTableRewriterTest.assertFileCounts(SSTableRewriterTest.java:758)
[junit] at
org.apache.cassandra.io.sstable.SSTableRewriterTest.testFileRemoval(SSTableRewriterTest.java:229)
{noformat}
The rest cascade after that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8597) Stress: make simple things simple
[ https://issues.apache.org/jira/browse/CASSANDRA-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273883#comment-14273883 ] Aleksey Yeschenko commented on CASSANDRA-8597: -- bq. i'm not sure what's meant here? it's a deterministic workload if you use the -pop seq=1..N, except for thread interleavings and ancillary chances like select. Do you mean a deterministic non-uniform distribution? Deterministic select behaviour? Can't use in a user profile. Use case here: want exactly 1M cells in one partition w/ 3 clustering columns (as in https://issues.apache.org/jira/browse/CASSANDRA-6694?focusedCommentId=13908198page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13908198). Can use uniform and get close, but overall w/ issues 1,2,3 it's very PITA. Stress: make simple things simple - Key: CASSANDRA-8597 URL: https://issues.apache.org/jira/browse/CASSANDRA-8597 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jonathan Ellis Assignee: T Jake Luciani Fix For: 2.1.3 Some of the trouble people have with stress is a documentation problem, but some is functional. Comments from [~iamaleksey]: # 3 clustering columns, make a million cells in a single partition, should be simple, but it's not. have to tweak 'clustering' on the three columns just right to make stress work at all. w/ some values it'd just gets stuck forever computing batches # for others, it generates huge, megabyte-size batches, utterly disrespecting 'select' clause in 'insert' # I want a sequential generator too, to be able to predict deterministic result sets. uniform() only gets you so far # impossible to simulate a time series workload /cc [~jshook] [~aweisberg] [~benedict] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8303) Provide strict mode for CQL Queries
[ https://issues.apache.org/jira/browse/CASSANDRA-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273892#comment-14273892 ] Jeremiah Jordan commented on CASSANDRA-8303: I definitely think a new interface is the right way to go here if we do this. I think think this fits neatly into the existing hierarchy. Provide strict mode for CQL Queries - Key: CASSANDRA-8303 URL: https://issues.apache.org/jira/browse/CASSANDRA-8303 Project: Cassandra Issue Type: Improvement Reporter: Anupam Arora Fix For: 3.0 Please provide a strict mode option in cassandra that will kick out any CQL queries that are expensive, e.g. any query with ALLOWS FILTERING, multi-partition queries, secondary index queries, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8303) Provide strict mode for CQL Queries
[ https://issues.apache.org/jira/browse/CASSANDRA-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273892#comment-14273892 ] Jeremiah Jordan edited comment on CASSANDRA-8303 at 1/12/15 6:25 PM: - I definitely think a new interface is the right way to go here if we do this. I don't think this fits neatly into the existing hierarchy. was (Author: jjordan): I definitely think a new interface is the right way to go here if we do this. I think think this fits neatly into the existing hierarchy. Provide strict mode for CQL Queries - Key: CASSANDRA-8303 URL: https://issues.apache.org/jira/browse/CASSANDRA-8303 Project: Cassandra Issue Type: Improvement Reporter: Anupam Arora Fix For: 3.0 Please provide a strict mode option in cassandra that will kick out any CQL queries that are expensive, e.g. any query with ALLOWS FILTERING, multi-partition queries, secondary index queries, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8597) Stress: make simple things simple
[ https://issues.apache.org/jira/browse/CASSANDRA-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273894#comment-14273894 ] Benedict commented on CASSANDRA-8597: - There is a FIXED distribution - if you want exactly 1M, why not use this? With a depth of 3, as stated, FIXED(100) for each clustering column would do this trick. If we reenvisage the way we define the distribution, as I alluded to in #2, you could define the total number of rows you want in the partition. But then conceptualising how those rows are distributed amongst the clustering columns becomes hard and a different PITA. You'd need two knobs per clustering column: the share of fan-out they should adopt, and the variance between each value. Understanding how these interplayed with each other (both intra-tier and inter-tier) would be really quite difficult for people to think about, which is why I originally chose to let it be configured by clustering column. It does, however, also solve your problem #2. It's a more powerful way of specifying, but I'm concerned that stress is already considered difficult to understand. Stress: make simple things simple - Key: CASSANDRA-8597 URL: https://issues.apache.org/jira/browse/CASSANDRA-8597 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jonathan Ellis Assignee: T Jake Luciani Fix For: 2.1.3 Some of the trouble people have with stress is a documentation problem, but some is functional. Comments from [~iamaleksey]: # 3 clustering columns, make a million cells in a single partition, should be simple, but it's not. have to tweak 'clustering' on the three columns just right to make stress work at all. w/ some values it'd just gets stuck forever computing batches # for others, it generates huge, megabyte-size batches, utterly disrespecting 'select' clause in 'insert' # I want a sequential generator too, to be able to predict deterministic result sets. uniform() only gets you so far # impossible to simulate a time series workload /cc [~jshook] [~aweisberg] [~benedict] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[
https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273734#comment-14273734
]
Leonid Shalupov commented on CASSANDRA-8535:
Nope, it's not about 8544. 8544 is on our current production, 2.1.1.
This bug is in 2.1-branch, after 2.1.1.
I've seen it in production, so it's not test-only. I'll try to reproduce it and
report.
java.lang.RuntimeException: Failed to rename XXX to YYY
---
Key: CASSANDRA-8535
URL: https://issues.apache.org/jira/browse/CASSANDRA-8535
Project: Cassandra
Issue Type: Bug
Environment: Windows 2008 X64
Reporter: Leonid Shalupov
Assignee: Joshua McKenzie
Fix For: 2.1.3
{code}
java.lang.RuntimeException: Failed to rename
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
to
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170)
~[main/:na]
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200)
~[main/:na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
~[main/:na]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: java.nio.file.FileSystemException:
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
-
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db:
The process cannot access the file because it is being used by another
process.
at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
~[na:1.7.0_45]
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
~[na:1.7.0_45]
at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301)
~[na:1.7.0_45]
at
sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
~[na:1.7.0_45]
at java.nio.file.Files.move(Files.java:1345) ~[na:1.7.0_45]
at
org.apache.cassandra.io.util.FileUtils.atomicMoveWithFallback(FileUtils.java:184)
~[main/:na]
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166)
~[main/:na]
... 18 common frames omitted
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8053) Support for user defined aggregate functions
[
https://issues.apache.org/jira/browse/CASSANDRA-8053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-8053:
-
Labels: client-impacting cql udf (was: cql udf)
Support for user defined aggregate functions
Key: CASSANDRA-8053
URL: https://issues.apache.org/jira/browse/CASSANDRA-8053
Project: Cassandra
Issue Type: New Feature
Reporter: Robert Stupp
Assignee: Robert Stupp
Labels: client-impacting, cql, udf
Fix For: 3.0
Attachments: 8053-final.txt, 8053v1.txt, 8053v2.txt
CASSANDRA-4914 introduces aggregate functions.
This ticket is about to decide how we can support user defined aggregate
functions. UD aggregate functions should be supported for all UDF flavors
(class, java, jsr223).
Things to consider:
* Special implementations for each scripting language should be omitted
* No exposure of internal APIs (e.g. {{AggregateFunction}} interface)
* No need for users to deal with serializers / codecs
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-6572) Workload recording / playback
[ https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremiah Jordan updated CASSANDRA-6572: --- Fix Version/s: (was: 2.1.3) 3.0 Workload recording / playback - Key: CASSANDRA-6572 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572 Project: Cassandra Issue Type: New Feature Components: Core, Tools Reporter: Jonathan Ellis Assignee: Carl Yeksigian Fix For: 3.0 Attachments: 6572-trunk.diff Write sample mode gets us part way to testing new versions against a real world workload, but we need an easy way to test the query side as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8594) pidfile is never filled by cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273831#comment-14273831 ] Joshua McKenzie commented on CASSANDRA-8594: The pidfile is written by bin/cassandra and bin/cassandra.ps1 on linux and windows, respectively. The code inside the daemon serves to auto-delete the pidfile on server shutdown. The logic to perform that on linux is pretty old (from 2011 on the linux side); are you seeing circumstances where the pidfile isn't being written? I'd expect a write permissions issue before code-problem since this is both 1) old and 2) widely tested. pidfile is never filled by cassandra Key: CASSANDRA-8594 URL: https://issues.apache.org/jira/browse/CASSANDRA-8594 Project: Cassandra Issue Type: Bug Reporter: Cyril Scetbon Assignee: Cyril Scetbon Fix For: 2.0.13 The pid file is never filled by cassandra. there is only a File object that is created with [those lines|https://github.com/cscetbon/cassandra/blob/cassandra-2.0.10/src/java/org/apache/cassandra/service/CassandraDaemon.java#L498-L501] Here is a [fix|https://github.com/cscetbon/cassandra/commit/d0c5e0c9be00e48e6d0cd0de208c53274f1919c0.patch] that writes the current PID into the pidfile -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: Check for available disk space before starting a compaction.
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 e4fc39524 - c20d41583
Check for available disk space before starting a compaction.
Patch by marcuse; reviewed by JoshuaMcKenzie for CASSANDRA-8562
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c20d4158
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c20d4158
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c20d4158
Branch: refs/heads/cassandra-2.0
Commit: c20d415833b785bfa5f49d3dd2a7468e111fb5d0
Parents: e4fc395
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Jan 12 11:22:23 2015 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Jan 12 11:35:53 2015 +0100
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/db/Directories.java | 18 ++
.../cassandra/db/compaction/CompactionTask.java | 16 +++-
.../cassandra/io/util/DiskAwareRunnable.java | 17 +
4 files changed, 35 insertions(+), 17 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c20d4158/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fc43dfa..9b20a06 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.12:
+ * Check for available disk space before starting a compaction (CASSANDRA-8562)
* Fix DISTINCT queries with LIMITs or paging when some partitions
contain only tombstones (CASSANDRA-8490)
* Introduce background cache refreshing to permissions cache
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c20d4158/src/java/org/apache/cassandra/db/Directories.java
--
diff --git a/src/java/org/apache/cassandra/db/Directories.java
b/src/java/org/apache/cassandra/db/Directories.java
index 69c7a06..8fd1762 100644
--- a/src/java/org/apache/cassandra/db/Directories.java
+++ b/src/java/org/apache/cassandra/db/Directories.java
@@ -308,6 +308,24 @@ public class Directories
Collections.sort(candidates);
}
+public boolean hasAvailableDiskSpace(long estimatedSSTables, long
expectedTotalWriteSize)
+{
+long writeSize = expectedTotalWriteSize / estimatedSSTables;
+long totalAvailable = 0L;
+
+for (DataDirectory dataDir : dataFileLocations)
+{
+if
(BlacklistedDirectories.isUnwritable(getLocationForDisk(dataDir)))
+ continue;
+DataDirectoryCandidate candidate = new
DataDirectoryCandidate(dataDir);
+// exclude directory if its total writeSize does not fit to data
directory
+if (candidate.availableSpace writeSize)
+continue;
+totalAvailable += candidate.availableSpace;
+}
+return totalAvailable expectedTotalWriteSize;
+}
+
public static File getSnapshotDirectory(Descriptor desc, String
snapshotName)
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c20d4158/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 38de8a9..6c6d3a2 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@ -100,6 +100,11 @@ public class CompactionTask extends AbstractCompactionTask
if (DatabaseDescriptor.isSnapshotBeforeCompaction())
cfs.snapshotWithoutFlush(System.currentTimeMillis() + -compact-
+ cfs.name);
+// note that we need to do a rough estimate early if we can fit the
compaction on disk - this is pessimistic, but
+// since we might remove sstables from the compaction in
checkAvailableDiskSpace it needs to be done here
+long earlySSTableEstimate = Math.max(1,
cfs.getExpectedCompactedFileSize(toCompact, compactionType) /
strategy.getMaxSSTableBytes());
+checkAvailableDiskSpace(earlySSTableEstimate);
+
// sanity check: all sstables must belong to the same cfs
for (SSTableReader sstable : toCompact)
assert sstable.descriptor.cfname.equals(cfs.name);
@@ -118,7 +123,7 @@ public class CompactionTask extends AbstractCompactionTask
long totalkeysWritten = 0;
long estimatedTotalKeys = Math.max(cfs.metadata.getIndexInterval(),
SSTableReader.getApproximateKeyCount(actuallyCompact, cfs.metadata));
-long estimatedSSTables = Math.max(1,
SSTable.getTotalBytes(actuallyCompact) / strategy.getMaxSSTableBytes());
+long estimatedSSTables = Math.max(1,
[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt src/java/org/apache/cassandra/db/Directories.java src/java/org/apache/cassandra/db/compaction/CompactionTask.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/75378c20 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/75378c20 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/75378c20 Branch: refs/heads/cassandra-2.1 Commit: 75378c204c30f5d5f679009885e2ace105793a67 Parents: 5364083 c20d415 Author: Marcus Eriksson marc...@apache.org Authored: Mon Jan 12 18:43:47 2015 +0100 Committer: Marcus Eriksson marc...@apache.org Committed: Mon Jan 12 18:43:47 2015 +0100 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/Directories.java| 18 .../cassandra/db/compaction/CompactionTask.java | 29 .../cassandra/io/util/DiskAwareRunnable.java| 17 +--- 4 files changed, 43 insertions(+), 22 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/75378c20/CHANGES.txt -- diff --cc CHANGES.txt index 55ca55d,9b20a06..f2e25c4 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,56 -1,5 +1,57 @@@ -2.0.12: +2.1.3 + * Don't reuse the same cleanup strategy for all sstables (CASSANDRA-8537) + * Fix case-sensitivity of index name on CREATE and DROP INDEX + statements (CASSANDRA-8365) + * Better detection/logging for corruption in compressed sstables (CASSANDRA-8192) + * Use the correct repairedAt value when closing writer (CASSANDRA-8570) + * (cqlsh) Handle a schema mismatch being detected on startup (CASSANDRA-8512) + * Properly calculate expected write size during compaction (CASSANDRA-8532) + * Invalidate affected prepared statements when a table's columns + are altered (CASSANDRA-7910) + * Stress - user defined writes should populate sequentally (CASSANDRA-8524) + * Fix regression in SSTableRewriter causing some rows to become unreadable + during compaction (CASSANDRA-8429) + * Run major compactions for repaired/unrepaired in parallel (CASSANDRA-8510) + * (cqlsh) Fix compression options in DESCRIBE TABLE output when compression + is disabled (CASSANDRA-8288) + * (cqlsh) Fix DESCRIBE output after keyspaces are altered (CASSANDRA-7623) + * Make sure we set lastCompactedKey correctly (CASSANDRA-8463) + * (cqlsh) Fix output of CONSISTENCY command (CASSANDRA-8507) + * (cqlsh) Fixed the handling of LIST statements (CASSANDRA-8370) + * Make sstablescrub check leveled manifest again (CASSANDRA-8432) + * Check first/last keys in sstable when giving out positions (CASSANDRA-8458) + * Disable mmap on Windows (CASSANDRA-6993) + * Add missing ConsistencyLevels to cassandra-stress (CASSANDRA-8253) + * Add auth support to cassandra-stress (CASSANDRA-7985) + * Fix ArrayIndexOutOfBoundsException when generating error message + for some CQL syntax errors (CASSANDRA-8455) + * Scale memtable slab allocation logarithmically (CASSANDRA-7882) + * cassandra-stress simultaneous inserts over same seed (CASSANDRA-7964) + * Reduce cassandra-stress sampling memory requirements (CASSANDRA-7926) + * Ensure memtable flush cannot expire commit log entries from its future (CASSANDRA-8383) + * Make read defrag async to reclaim memtables (CASSANDRA-8459) + * Remove tmplink files for offline compactions (CASSANDRA-8321) + * Reduce maxHintsInProgress (CASSANDRA-8415) + * BTree updates may call provided update function twice (CASSANDRA-8018) + * Release sstable references after anticompaction (CASSANDRA-8386) + * Handle abort() in SSTableRewriter properly (CASSANDRA-8320) + * Fix high size calculations for prepared statements (CASSANDRA-8231) + * Centralize shared executors (CASSANDRA-8055) + * Fix filtering for CONTAINS (KEY) relations on frozen collection + clustering columns when the query is restricted to a single + partition (CASSANDRA-8203) + * Do more aggressive entire-sstable TTL expiry checks (CASSANDRA-8243) + * Add more log info if readMeter is null (CASSANDRA-8238) + * add check of the system wall clock time at startup (CASSANDRA-8305) + * Support for frozen collections (CASSANDRA-7859) + * Fix overflow on histogram computation (CASSANDRA-8028) + * Have paxos reuse the timestamp generation of normal queries (CASSANDRA-7801) + * Fix incremental repair not remove parent session on remote (CASSANDRA-8291) + * Improve JBOD disk utilization (CASSANDRA-7386) + * Log failed host when preparing incremental repair (CASSANDRA-8228) + * Force config client mode in CQLSSTableWriter (CASSANDRA-8281) +Merged from 2.0: + * Check for available disk space before starting a compaction (CASSANDRA-8562) * Fix
[1/3] cassandra git commit: Check for available disk space before starting a compaction.
Repository: cassandra
Updated Branches:
refs/heads/trunk c04c50c95 - caeef1740
Check for available disk space before starting a compaction.
Patch by marcuse; reviewed by JoshuaMcKenzie for CASSANDRA-8562
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c20d4158
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c20d4158
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c20d4158
Branch: refs/heads/trunk
Commit: c20d415833b785bfa5f49d3dd2a7468e111fb5d0
Parents: e4fc395
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Jan 12 11:22:23 2015 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Jan 12 11:35:53 2015 +0100
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/db/Directories.java | 18 ++
.../cassandra/db/compaction/CompactionTask.java | 16 +++-
.../cassandra/io/util/DiskAwareRunnable.java | 17 +
4 files changed, 35 insertions(+), 17 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c20d4158/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fc43dfa..9b20a06 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.12:
+ * Check for available disk space before starting a compaction (CASSANDRA-8562)
* Fix DISTINCT queries with LIMITs or paging when some partitions
contain only tombstones (CASSANDRA-8490)
* Introduce background cache refreshing to permissions cache
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c20d4158/src/java/org/apache/cassandra/db/Directories.java
--
diff --git a/src/java/org/apache/cassandra/db/Directories.java
b/src/java/org/apache/cassandra/db/Directories.java
index 69c7a06..8fd1762 100644
--- a/src/java/org/apache/cassandra/db/Directories.java
+++ b/src/java/org/apache/cassandra/db/Directories.java
@@ -308,6 +308,24 @@ public class Directories
Collections.sort(candidates);
}
+public boolean hasAvailableDiskSpace(long estimatedSSTables, long
expectedTotalWriteSize)
+{
+long writeSize = expectedTotalWriteSize / estimatedSSTables;
+long totalAvailable = 0L;
+
+for (DataDirectory dataDir : dataFileLocations)
+{
+if
(BlacklistedDirectories.isUnwritable(getLocationForDisk(dataDir)))
+ continue;
+DataDirectoryCandidate candidate = new
DataDirectoryCandidate(dataDir);
+// exclude directory if its total writeSize does not fit to data
directory
+if (candidate.availableSpace writeSize)
+continue;
+totalAvailable += candidate.availableSpace;
+}
+return totalAvailable expectedTotalWriteSize;
+}
+
public static File getSnapshotDirectory(Descriptor desc, String
snapshotName)
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c20d4158/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 38de8a9..6c6d3a2 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@ -100,6 +100,11 @@ public class CompactionTask extends AbstractCompactionTask
if (DatabaseDescriptor.isSnapshotBeforeCompaction())
cfs.snapshotWithoutFlush(System.currentTimeMillis() + -compact-
+ cfs.name);
+// note that we need to do a rough estimate early if we can fit the
compaction on disk - this is pessimistic, but
+// since we might remove sstables from the compaction in
checkAvailableDiskSpace it needs to be done here
+long earlySSTableEstimate = Math.max(1,
cfs.getExpectedCompactedFileSize(toCompact, compactionType) /
strategy.getMaxSSTableBytes());
+checkAvailableDiskSpace(earlySSTableEstimate);
+
// sanity check: all sstables must belong to the same cfs
for (SSTableReader sstable : toCompact)
assert sstable.descriptor.cfname.equals(cfs.name);
@@ -118,7 +123,7 @@ public class CompactionTask extends AbstractCompactionTask
long totalkeysWritten = 0;
long estimatedTotalKeys = Math.max(cfs.metadata.getIndexInterval(),
SSTableReader.getApproximateKeyCount(actuallyCompact, cfs.metadata));
-long estimatedSSTables = Math.max(1,
SSTable.getTotalBytes(actuallyCompact) / strategy.getMaxSSTableBytes());
+long estimatedSSTables = Math.max(1,
[1/2] cassandra git commit: Check for available disk space before starting a compaction.
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 536408380 - 75378c204
Check for available disk space before starting a compaction.
Patch by marcuse; reviewed by JoshuaMcKenzie for CASSANDRA-8562
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c20d4158
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c20d4158
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c20d4158
Branch: refs/heads/cassandra-2.1
Commit: c20d415833b785bfa5f49d3dd2a7468e111fb5d0
Parents: e4fc395
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Jan 12 11:22:23 2015 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Jan 12 11:35:53 2015 +0100
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/db/Directories.java | 18 ++
.../cassandra/db/compaction/CompactionTask.java | 16 +++-
.../cassandra/io/util/DiskAwareRunnable.java | 17 +
4 files changed, 35 insertions(+), 17 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c20d4158/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index fc43dfa..9b20a06 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.12:
+ * Check for available disk space before starting a compaction (CASSANDRA-8562)
* Fix DISTINCT queries with LIMITs or paging when some partitions
contain only tombstones (CASSANDRA-8490)
* Introduce background cache refreshing to permissions cache
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c20d4158/src/java/org/apache/cassandra/db/Directories.java
--
diff --git a/src/java/org/apache/cassandra/db/Directories.java
b/src/java/org/apache/cassandra/db/Directories.java
index 69c7a06..8fd1762 100644
--- a/src/java/org/apache/cassandra/db/Directories.java
+++ b/src/java/org/apache/cassandra/db/Directories.java
@@ -308,6 +308,24 @@ public class Directories
Collections.sort(candidates);
}
+public boolean hasAvailableDiskSpace(long estimatedSSTables, long
expectedTotalWriteSize)
+{
+long writeSize = expectedTotalWriteSize / estimatedSSTables;
+long totalAvailable = 0L;
+
+for (DataDirectory dataDir : dataFileLocations)
+{
+if
(BlacklistedDirectories.isUnwritable(getLocationForDisk(dataDir)))
+ continue;
+DataDirectoryCandidate candidate = new
DataDirectoryCandidate(dataDir);
+// exclude directory if its total writeSize does not fit to data
directory
+if (candidate.availableSpace writeSize)
+continue;
+totalAvailable += candidate.availableSpace;
+}
+return totalAvailable expectedTotalWriteSize;
+}
+
public static File getSnapshotDirectory(Descriptor desc, String
snapshotName)
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c20d4158/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 38de8a9..6c6d3a2 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@ -100,6 +100,11 @@ public class CompactionTask extends AbstractCompactionTask
if (DatabaseDescriptor.isSnapshotBeforeCompaction())
cfs.snapshotWithoutFlush(System.currentTimeMillis() + -compact-
+ cfs.name);
+// note that we need to do a rough estimate early if we can fit the
compaction on disk - this is pessimistic, but
+// since we might remove sstables from the compaction in
checkAvailableDiskSpace it needs to be done here
+long earlySSTableEstimate = Math.max(1,
cfs.getExpectedCompactedFileSize(toCompact, compactionType) /
strategy.getMaxSSTableBytes());
+checkAvailableDiskSpace(earlySSTableEstimate);
+
// sanity check: all sstables must belong to the same cfs
for (SSTableReader sstable : toCompact)
assert sstable.descriptor.cfname.equals(cfs.name);
@@ -118,7 +123,7 @@ public class CompactionTask extends AbstractCompactionTask
long totalkeysWritten = 0;
long estimatedTotalKeys = Math.max(cfs.metadata.getIndexInterval(),
SSTableReader.getApproximateKeyCount(actuallyCompact, cfs.metadata));
-long estimatedSSTables = Math.max(1,
SSTable.getTotalBytes(actuallyCompact) / strategy.getMaxSSTableBytes());
+long estimatedSSTables = Math.max(1,
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk
Conflicts:
src/java/org/apache/cassandra/db/compaction/CompactionTask.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/caeef174
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/caeef174
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/caeef174
Branch: refs/heads/trunk
Commit: caeef1740703a309a28c8621e5b4e8107dc1edb7
Parents: c04c50c 75378c2
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Jan 12 18:45:22 2015 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Jan 12 18:45:22 2015 +0100
--
CHANGES.txt | 1 +
.../org/apache/cassandra/db/Directories.java| 18
.../cassandra/db/compaction/CompactionTask.java | 29
.../cassandra/io/util/DiskAwareRunnable.java| 17 +---
4 files changed, 43 insertions(+), 22 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/caeef174/CHANGES.txt
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/caeef174/src/java/org/apache/cassandra/db/Directories.java
--
diff --cc src/java/org/apache/cassandra/db/Directories.java
index 4c4078c,6af3082..fa9b320
--- a/src/java/org/apache/cassandra/db/Directories.java
+++ b/src/java/org/apache/cassandra/db/Directories.java
@@@ -345,21 -342,27 +345,39 @@@ public class Directorie
Collections.sort(candidates);
}
+ public boolean hasAvailableDiskSpace(long estimatedSSTables, long
expectedTotalWriteSize)
+ {
+ long writeSize = expectedTotalWriteSize / estimatedSSTables;
+ long totalAvailable = 0L;
+
+ for (DataDirectory dataDir : dataDirectories)
+ {
+ if
(BlacklistedDirectories.isUnwritable(getLocationForDisk(dataDir)))
+ continue;
+ DataDirectoryCandidate candidate = new
DataDirectoryCandidate(dataDir);
+ // exclude directory if its total writeSize does not fit to data
directory
+ if (candidate.availableSpace writeSize)
+ continue;
+ totalAvailable += candidate.availableSpace;
+ }
+ return totalAvailable expectedTotalWriteSize;
+ }
+
public static File getSnapshotDirectory(Descriptor desc, String
snapshotName)
{
-return getOrCreate(desc.directory, SNAPSHOT_SUBDIR, snapshotName);
+return getSnapshotDirectory(desc.directory, snapshotName);
+}
+
+public static File getSnapshotDirectory(File location, String
snapshotName)
+{
+if (location.getName().startsWith(SECONDARY_INDEX_NAME_SEPARATOR))
+{
+return getOrCreate(location.getParentFile(), SNAPSHOT_SUBDIR,
snapshotName, location.getName());
+}
+else
+{
+return getOrCreate(location, SNAPSHOT_SUBDIR, snapshotName);
+}
}
public File getSnapshotManifestFile(String snapshotName)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/caeef174/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 8b5058b,eda09c0..dfbdc22
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@@ -19,7 -19,9 +19,8 @@@ package org.apache.cassandra.db.compact
import java.io.File;
import java.io.IOException;
+ import java.util.Arrays;
import java.util.Collection;
-import java.util.Collections;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
@@@ -149,10 -151,8 +157,10 @@@ public class CompactionTask extends Abs
SetSSTableReader actuallyCompact = Sets.difference(sstables,
controller.getFullyExpiredSSTables());
long estimatedTotalKeys =
Math.max(cfs.metadata.getMinIndexInterval(),
SSTableReader.getApproximateKeyCount(actuallyCompact));
- long estimatedSSTables = Math.max(1,
SSTableReader.getTotalBytes(actuallyCompact) / strategy.getMaxSSTableBytes());
+ long estimatedSSTables = Math.max(1,
cfs.getExpectedCompactedFileSize(actuallyCompact, compactionType) /
strategy.getMaxSSTableBytes());
long keysPerSSTable = (long) Math.ceil((double)
estimatedTotalKeys / estimatedSSTables);
+SSTableFormat.Type sstableFormat = getFormatType(sstables);
+
long expectedSSTableSize = Math.min(getExpectedWriteSize(),
strategy.getMaxSSTableBytes());
logger.debug(Expected bloom
[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt src/java/org/apache/cassandra/db/Directories.java src/java/org/apache/cassandra/db/compaction/CompactionTask.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/75378c20 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/75378c20 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/75378c20 Branch: refs/heads/trunk Commit: 75378c204c30f5d5f679009885e2ace105793a67 Parents: 5364083 c20d415 Author: Marcus Eriksson marc...@apache.org Authored: Mon Jan 12 18:43:47 2015 +0100 Committer: Marcus Eriksson marc...@apache.org Committed: Mon Jan 12 18:43:47 2015 +0100 -- CHANGES.txt | 1 + .../org/apache/cassandra/db/Directories.java| 18 .../cassandra/db/compaction/CompactionTask.java | 29 .../cassandra/io/util/DiskAwareRunnable.java| 17 +--- 4 files changed, 43 insertions(+), 22 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/75378c20/CHANGES.txt -- diff --cc CHANGES.txt index 55ca55d,9b20a06..f2e25c4 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,56 -1,5 +1,57 @@@ -2.0.12: +2.1.3 + * Don't reuse the same cleanup strategy for all sstables (CASSANDRA-8537) + * Fix case-sensitivity of index name on CREATE and DROP INDEX + statements (CASSANDRA-8365) + * Better detection/logging for corruption in compressed sstables (CASSANDRA-8192) + * Use the correct repairedAt value when closing writer (CASSANDRA-8570) + * (cqlsh) Handle a schema mismatch being detected on startup (CASSANDRA-8512) + * Properly calculate expected write size during compaction (CASSANDRA-8532) + * Invalidate affected prepared statements when a table's columns + are altered (CASSANDRA-7910) + * Stress - user defined writes should populate sequentally (CASSANDRA-8524) + * Fix regression in SSTableRewriter causing some rows to become unreadable + during compaction (CASSANDRA-8429) + * Run major compactions for repaired/unrepaired in parallel (CASSANDRA-8510) + * (cqlsh) Fix compression options in DESCRIBE TABLE output when compression + is disabled (CASSANDRA-8288) + * (cqlsh) Fix DESCRIBE output after keyspaces are altered (CASSANDRA-7623) + * Make sure we set lastCompactedKey correctly (CASSANDRA-8463) + * (cqlsh) Fix output of CONSISTENCY command (CASSANDRA-8507) + * (cqlsh) Fixed the handling of LIST statements (CASSANDRA-8370) + * Make sstablescrub check leveled manifest again (CASSANDRA-8432) + * Check first/last keys in sstable when giving out positions (CASSANDRA-8458) + * Disable mmap on Windows (CASSANDRA-6993) + * Add missing ConsistencyLevels to cassandra-stress (CASSANDRA-8253) + * Add auth support to cassandra-stress (CASSANDRA-7985) + * Fix ArrayIndexOutOfBoundsException when generating error message + for some CQL syntax errors (CASSANDRA-8455) + * Scale memtable slab allocation logarithmically (CASSANDRA-7882) + * cassandra-stress simultaneous inserts over same seed (CASSANDRA-7964) + * Reduce cassandra-stress sampling memory requirements (CASSANDRA-7926) + * Ensure memtable flush cannot expire commit log entries from its future (CASSANDRA-8383) + * Make read defrag async to reclaim memtables (CASSANDRA-8459) + * Remove tmplink files for offline compactions (CASSANDRA-8321) + * Reduce maxHintsInProgress (CASSANDRA-8415) + * BTree updates may call provided update function twice (CASSANDRA-8018) + * Release sstable references after anticompaction (CASSANDRA-8386) + * Handle abort() in SSTableRewriter properly (CASSANDRA-8320) + * Fix high size calculations for prepared statements (CASSANDRA-8231) + * Centralize shared executors (CASSANDRA-8055) + * Fix filtering for CONTAINS (KEY) relations on frozen collection + clustering columns when the query is restricted to a single + partition (CASSANDRA-8203) + * Do more aggressive entire-sstable TTL expiry checks (CASSANDRA-8243) + * Add more log info if readMeter is null (CASSANDRA-8238) + * add check of the system wall clock time at startup (CASSANDRA-8305) + * Support for frozen collections (CASSANDRA-7859) + * Fix overflow on histogram computation (CASSANDRA-8028) + * Have paxos reuse the timestamp generation of normal queries (CASSANDRA-7801) + * Fix incremental repair not remove parent session on remote (CASSANDRA-8291) + * Improve JBOD disk utilization (CASSANDRA-7386) + * Log failed host when preparing incremental repair (CASSANDRA-8228) + * Force config client mode in CQLSSTableWriter (CASSANDRA-8281) +Merged from 2.0: + * Check for available disk space before starting a compaction (CASSANDRA-8562) * Fix DISTINCT
[2/6] cassandra git commit: Make sstablemetadata work outside install dir
Make sstablemetadata work outside install dir patch by Jimmy MÃ¥rdell; reviewed by yukim for CASSANDRA-8579 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9e5a4fad Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9e5a4fad Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9e5a4fad Branch: refs/heads/cassandra-2.1 Commit: 9e5a4fad7d8421837cb32aabd643b6ac7f9e62b6 Parents: c20d415 Author: Jimmy MÃ¥rdell ya...@spotify.com Authored: Fri Jan 9 16:35:13 2015 -0600 Committer: Yuki Morishita yu...@apache.org Committed: Mon Jan 12 11:50:25 2015 -0600 -- tools/bin/sstablemetadata | 37 - 1 file changed, 20 insertions(+), 17 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e5a4fad/tools/bin/sstablemetadata -- diff --git a/tools/bin/sstablemetadata b/tools/bin/sstablemetadata index 5fe8cc4..5e7c26a 100755 --- a/tools/bin/sstablemetadata +++ b/tools/bin/sstablemetadata @@ -16,29 +16,32 @@ # See the License for the specific language governing permissions and # limitations under the License. -if [ x$CLASSPATH = x ]; then - -# execute from the build dir. -if [ -d `dirname $0`/../../build/classes ]; then -for directory in `dirname $0`/../../build/classes/*; do -CLASSPATH=$CLASSPATH:$directory -done -else -if [ -f `dirname $0`/../lib/stress.jar ]; then -CLASSPATH=`dirname $0`/../lib/stress.jar +if [ x$CASSANDRA_INCLUDE = x ]; then +for include in `dirname $0`/cassandra.in.sh \ + $HOME/.cassandra.in.sh \ + /usr/share/cassandra/cassandra.in.sh \ + /usr/local/share/cassandra/cassandra.in.sh \ + /opt/cassandra/cassandra.in.sh; do +if [ -r $include ]; then +. $include +break fi -fi - -for jar in `dirname $0`/../../lib/*.jar; do -CLASSPATH=$CLASSPATH:$jar done +elif [ -r $CASSANDRA_INCLUDE ]; then +. $CASSANDRA_INCLUDE fi + # Use JAVA_HOME if set, otherwise look for java in PATH -if [ -x $JAVA_HOME/bin/java ]; then -JAVA=$JAVA_HOME/bin/java +if [ -x $JAVA_HOME/bin/java ]; then +JAVA=$JAVA_HOME/bin/java else -JAVA=`which java` +JAVA=`which java` +fi + +if [ -z $CLASSPATH ]; then +echo You must set the CLASSPATH var 2 +exit 1 fi $JAVA -cp $CLASSPATH \
[5/6] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f136bacb Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f136bacb Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f136bacb Branch: refs/heads/cassandra-2.1 Commit: f136bacb925722cfad4a08dc3bbead4ae6ae2c09 Parents: 75378c2 9e5a4fa Author: Yuki Morishita yu...@apache.org Authored: Mon Jan 12 11:51:00 2015 -0600 Committer: Yuki Morishita yu...@apache.org Committed: Mon Jan 12 11:51:00 2015 -0600 -- tools/bin/sstablemetadata | 37 - 1 file changed, 20 insertions(+), 17 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f136bacb/tools/bin/sstablemetadata --
[3/6] cassandra git commit: Make sstablemetadata work outside install dir
Make sstablemetadata work outside install dir patch by Jimmy MÃ¥rdell; reviewed by yukim for CASSANDRA-8579 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9e5a4fad Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9e5a4fad Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9e5a4fad Branch: refs/heads/trunk Commit: 9e5a4fad7d8421837cb32aabd643b6ac7f9e62b6 Parents: c20d415 Author: Jimmy MÃ¥rdell ya...@spotify.com Authored: Fri Jan 9 16:35:13 2015 -0600 Committer: Yuki Morishita yu...@apache.org Committed: Mon Jan 12 11:50:25 2015 -0600 -- tools/bin/sstablemetadata | 37 - 1 file changed, 20 insertions(+), 17 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e5a4fad/tools/bin/sstablemetadata -- diff --git a/tools/bin/sstablemetadata b/tools/bin/sstablemetadata index 5fe8cc4..5e7c26a 100755 --- a/tools/bin/sstablemetadata +++ b/tools/bin/sstablemetadata @@ -16,29 +16,32 @@ # See the License for the specific language governing permissions and # limitations under the License. -if [ x$CLASSPATH = x ]; then - -# execute from the build dir. -if [ -d `dirname $0`/../../build/classes ]; then -for directory in `dirname $0`/../../build/classes/*; do -CLASSPATH=$CLASSPATH:$directory -done -else -if [ -f `dirname $0`/../lib/stress.jar ]; then -CLASSPATH=`dirname $0`/../lib/stress.jar +if [ x$CASSANDRA_INCLUDE = x ]; then +for include in `dirname $0`/cassandra.in.sh \ + $HOME/.cassandra.in.sh \ + /usr/share/cassandra/cassandra.in.sh \ + /usr/local/share/cassandra/cassandra.in.sh \ + /opt/cassandra/cassandra.in.sh; do +if [ -r $include ]; then +. $include +break fi -fi - -for jar in `dirname $0`/../../lib/*.jar; do -CLASSPATH=$CLASSPATH:$jar done +elif [ -r $CASSANDRA_INCLUDE ]; then +. $CASSANDRA_INCLUDE fi + # Use JAVA_HOME if set, otherwise look for java in PATH -if [ -x $JAVA_HOME/bin/java ]; then -JAVA=$JAVA_HOME/bin/java +if [ -x $JAVA_HOME/bin/java ]; then +JAVA=$JAVA_HOME/bin/java else -JAVA=`which java` +JAVA=`which java` +fi + +if [ -z $CLASSPATH ]; then +echo You must set the CLASSPATH var 2 +exit 1 fi $JAVA -cp $CLASSPATH \
[jira] [Commented] (CASSANDRA-8597) Stress: make simple things simple
[ https://issues.apache.org/jira/browse/CASSANDRA-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273835#comment-14273835 ] Ariel Weisberg commented on CASSANDRA-8597: --- My use case was simple so I didn't run into capability issues. My comments are almost all about documentation. The embedded help is very good although concepts can be hard. It's the higher level docs I had trouble with. The first was that I didn't find the right stress docs because I got 2.0 when I had to go specifically look for 2.1 docs. Embedded help providing a doc URL would be nice. Flag formatting is tricky both due to how positional they are, but also due to the escaping required in the shell for parentheses. When you are working through it the first time there are small things like that breaking your flow. There was also the fact that distributions don't accept spaces between parameters. The last bit is the lack of recipe style documentation. Most flags are part of some recipe so it's helpful to see them in action. There are two dimensions data distribution and access distribution and knowing all the knobs for describing data distribution (# partition, # cells/rows, #size of cells) and access distribution (across partitions, across cells within partitions, random vs sequential strides, # of rows/cells to select) would be helpful. I think some times these parameters are linked as well. It's also worth mentioning in the distribution section that you can get the tool to print a summary of distributions so you know what your parameters are doing. Stress: make simple things simple - Key: CASSANDRA-8597 URL: https://issues.apache.org/jira/browse/CASSANDRA-8597 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jonathan Ellis Assignee: T Jake Luciani Fix For: 2.1.3 Some of the trouble people have with stress is a documentation problem, but some is functional. Comments from [~iamaleksey]: # 3 clustering columns, make a million cells in a single partition, should be simple, but it's not. have to tweak 'clustering' on the three columns just right to make stress work at all. w/ some values it'd just gets stuck forever computing batches # for others, it generates huge, megabyte-size batches, utterly disrespecting 'select' clause in 'insert' # I want a sequential generator too, to be able to predict deterministic result sets. uniform() only gets you so far # impossible to simulate a time series workload /cc [~jshook] [~aweisberg] [~benedict] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[6/6] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7bd6d56b Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7bd6d56b Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7bd6d56b Branch: refs/heads/trunk Commit: 7bd6d56b5acbece45f31d38b2bc51132388d2c20 Parents: caeef17 f136bac Author: Yuki Morishita yu...@apache.org Authored: Mon Jan 12 11:52:13 2015 -0600 Committer: Yuki Morishita yu...@apache.org Committed: Mon Jan 12 11:52:13 2015 -0600 -- tools/bin/sstablemetadata | 37 - 1 file changed, 20 insertions(+), 17 deletions(-) --
[1/6] cassandra git commit: Make sstablemetadata work outside install dir
Repository: cassandra Updated Branches: refs/heads/cassandra-2.0 c20d41583 - 9e5a4fad7 refs/heads/cassandra-2.1 75378c204 - f136bacb9 refs/heads/trunk caeef1740 - 7bd6d56b5 Make sstablemetadata work outside install dir patch by Jimmy MÃ¥rdell; reviewed by yukim for CASSANDRA-8579 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9e5a4fad Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9e5a4fad Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9e5a4fad Branch: refs/heads/cassandra-2.0 Commit: 9e5a4fad7d8421837cb32aabd643b6ac7f9e62b6 Parents: c20d415 Author: Jimmy MÃ¥rdell ya...@spotify.com Authored: Fri Jan 9 16:35:13 2015 -0600 Committer: Yuki Morishita yu...@apache.org Committed: Mon Jan 12 11:50:25 2015 -0600 -- tools/bin/sstablemetadata | 37 - 1 file changed, 20 insertions(+), 17 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e5a4fad/tools/bin/sstablemetadata -- diff --git a/tools/bin/sstablemetadata b/tools/bin/sstablemetadata index 5fe8cc4..5e7c26a 100755 --- a/tools/bin/sstablemetadata +++ b/tools/bin/sstablemetadata @@ -16,29 +16,32 @@ # See the License for the specific language governing permissions and # limitations under the License. -if [ x$CLASSPATH = x ]; then - -# execute from the build dir. -if [ -d `dirname $0`/../../build/classes ]; then -for directory in `dirname $0`/../../build/classes/*; do -CLASSPATH=$CLASSPATH:$directory -done -else -if [ -f `dirname $0`/../lib/stress.jar ]; then -CLASSPATH=`dirname $0`/../lib/stress.jar +if [ x$CASSANDRA_INCLUDE = x ]; then +for include in `dirname $0`/cassandra.in.sh \ + $HOME/.cassandra.in.sh \ + /usr/share/cassandra/cassandra.in.sh \ + /usr/local/share/cassandra/cassandra.in.sh \ + /opt/cassandra/cassandra.in.sh; do +if [ -r $include ]; then +. $include +break fi -fi - -for jar in `dirname $0`/../../lib/*.jar; do -CLASSPATH=$CLASSPATH:$jar done +elif [ -r $CASSANDRA_INCLUDE ]; then +. $CASSANDRA_INCLUDE fi + # Use JAVA_HOME if set, otherwise look for java in PATH -if [ -x $JAVA_HOME/bin/java ]; then -JAVA=$JAVA_HOME/bin/java +if [ -x $JAVA_HOME/bin/java ]; then +JAVA=$JAVA_HOME/bin/java else -JAVA=`which java` +JAVA=`which java` +fi + +if [ -z $CLASSPATH ]; then +echo You must set the CLASSPATH var 2 +exit 1 fi $JAVA -cp $CLASSPATH \
[4/6] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f136bacb Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f136bacb Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f136bacb Branch: refs/heads/trunk Commit: f136bacb925722cfad4a08dc3bbead4ae6ae2c09 Parents: 75378c2 9e5a4fa Author: Yuki Morishita yu...@apache.org Authored: Mon Jan 12 11:51:00 2015 -0600 Committer: Yuki Morishita yu...@apache.org Committed: Mon Jan 12 11:51:00 2015 -0600 -- tools/bin/sstablemetadata | 37 - 1 file changed, 20 insertions(+), 17 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f136bacb/tools/bin/sstablemetadata --
[1/2] cassandra git commit: fix errant CPIF references
Repository: cassandra
Updated Branches:
refs/heads/trunk 7bd6d56b5 - 3b945483b
fix errant CPIF references
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f6d0f436
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f6d0f436
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f6d0f436
Branch: refs/heads/trunk
Commit: f6d0f436b39f78f2b4fee5e84190a53259ff677b
Parents: 75378c2
Author: Brandon Williams brandonwilli...@apache.org
Authored: Mon Jan 12 11:57:23 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Mon Jan 12 11:57:23 2015 -0600
--
examples/hadoop_cql3_word_count/src/WordCount.java | 3 +--
examples/hadoop_cql3_word_count/src/WordCountCounters.java | 3 +--
src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java | 2 +-
3 files changed, 3 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f6d0f436/examples/hadoop_cql3_word_count/src/WordCount.java
--
diff --git a/examples/hadoop_cql3_word_count/src/WordCount.java
b/examples/hadoop_cql3_word_count/src/WordCount.java
index 519a98f..6a2f846 100644
--- a/examples/hadoop_cql3_word_count/src/WordCount.java
+++ b/examples/hadoop_cql3_word_count/src/WordCount.java
@@ -26,7 +26,6 @@ import org.apache.cassandra.hadoop.cql3.CqlOutputFormat;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
import org.apache.cassandra.hadoop.cql3.CqlInputFormat;
import org.apache.cassandra.hadoop.ConfigHelper;
import org.apache.cassandra.utils.ByteBufferUtil;
@@ -247,7 +246,7 @@ public class WordCount extends Configured implements Tool
else
{
job.setMapperClass(TokenizerMapper.class);
-job.setInputFormatClass(CqlPagingInputFormat.class);
+job.setInputFormatClass(CqlInputFormat.class);
ConfigHelper.setInputRpcPort(job.getConfiguration(), 9160);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f6d0f436/examples/hadoop_cql3_word_count/src/WordCountCounters.java
--
diff --git a/examples/hadoop_cql3_word_count/src/WordCountCounters.java
b/examples/hadoop_cql3_word_count/src/WordCountCounters.java
index 74de9ab..150d18d 100644
--- a/examples/hadoop_cql3_word_count/src/WordCountCounters.java
+++ b/examples/hadoop_cql3_word_count/src/WordCountCounters.java
@@ -25,7 +25,6 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.hadoop.cql3.CqlConfigHelper;
-import org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
import org.apache.cassandra.hadoop.cql3.CqlInputFormat;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
@@ -156,7 +155,7 @@ public class WordCountCounters extends Configured
implements Tool
else
{
job.setMapperClass(SumMapper.class);
-job.setInputFormatClass(CqlPagingInputFormat.class);
+job.setInputFormatClass(CqlInputFormat.class);
ConfigHelper.setInputRpcPort(job.getConfiguration(), 9160);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f6d0f436/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
index 2ba4dbf..08926fa 100644
--- a/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
+++ b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
@@ -76,7 +76,7 @@ public class CqlStorage extends AbstractCassandraStorage
{
super();
this.pageSize = pageSize;
-DEFAULT_INPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
+DEFAULT_INPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlInputFormat;
DEFAULT_OUTPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlOutputFormat;
}
[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3b945483 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3b945483 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3b945483 Branch: refs/heads/trunk Commit: 3b945483bdb486487201923258421e3baa4a8b2a Parents: 7bd6d56 f6d0f43 Author: Brandon Williams brandonwilli...@apache.org Authored: Mon Jan 12 11:57:36 2015 -0600 Committer: Brandon Williams brandonwilli...@apache.org Committed: Mon Jan 12 11:57:36 2015 -0600 -- examples/hadoop_cql3_word_count/src/WordCount.java | 3 +-- examples/hadoop_cql3_word_count/src/WordCountCounters.java | 3 +-- src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java | 2 +- 3 files changed, 3 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3b945483/examples/hadoop_cql3_word_count/src/WordCount.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3b945483/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java --
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f1475244 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f1475244 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f1475244 Branch: refs/heads/trunk Commit: f1475244f5522ab27f504b5ebb6492b02a7ec9a9 Parents: 3b94548 0757dc7 Author: Brandon Williams brandonwilli...@apache.org Authored: Mon Jan 12 11:58:03 2015 -0600 Committer: Brandon Williams brandonwilli...@apache.org Committed: Mon Jan 12 11:58:03 2015 -0600 -- --
[2/3] cassandra git commit: fix errant CPIF references
fix errant CPIF references
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0757dc72
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0757dc72
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0757dc72
Branch: refs/heads/trunk
Commit: 0757dc72f270c1f0efd212400e9a7040dc744c9b
Parents: f136bac
Author: Brandon Williams brandonwilli...@apache.org
Authored: Mon Jan 12 11:57:23 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Mon Jan 12 11:57:53 2015 -0600
--
examples/hadoop_cql3_word_count/src/WordCount.java | 3 +--
examples/hadoop_cql3_word_count/src/WordCountCounters.java | 3 +--
src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java | 2 +-
3 files changed, 3 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0757dc72/examples/hadoop_cql3_word_count/src/WordCount.java
--
diff --git a/examples/hadoop_cql3_word_count/src/WordCount.java
b/examples/hadoop_cql3_word_count/src/WordCount.java
index 519a98f..6a2f846 100644
--- a/examples/hadoop_cql3_word_count/src/WordCount.java
+++ b/examples/hadoop_cql3_word_count/src/WordCount.java
@@ -26,7 +26,6 @@ import org.apache.cassandra.hadoop.cql3.CqlOutputFormat;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
import org.apache.cassandra.hadoop.cql3.CqlInputFormat;
import org.apache.cassandra.hadoop.ConfigHelper;
import org.apache.cassandra.utils.ByteBufferUtil;
@@ -247,7 +246,7 @@ public class WordCount extends Configured implements Tool
else
{
job.setMapperClass(TokenizerMapper.class);
-job.setInputFormatClass(CqlPagingInputFormat.class);
+job.setInputFormatClass(CqlInputFormat.class);
ConfigHelper.setInputRpcPort(job.getConfiguration(), 9160);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0757dc72/examples/hadoop_cql3_word_count/src/WordCountCounters.java
--
diff --git a/examples/hadoop_cql3_word_count/src/WordCountCounters.java
b/examples/hadoop_cql3_word_count/src/WordCountCounters.java
index 74de9ab..150d18d 100644
--- a/examples/hadoop_cql3_word_count/src/WordCountCounters.java
+++ b/examples/hadoop_cql3_word_count/src/WordCountCounters.java
@@ -25,7 +25,6 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.hadoop.cql3.CqlConfigHelper;
-import org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
import org.apache.cassandra.hadoop.cql3.CqlInputFormat;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
@@ -156,7 +155,7 @@ public class WordCountCounters extends Configured
implements Tool
else
{
job.setMapperClass(SumMapper.class);
-job.setInputFormatClass(CqlPagingInputFormat.class);
+job.setInputFormatClass(CqlInputFormat.class);
ConfigHelper.setInputRpcPort(job.getConfiguration(), 9160);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0757dc72/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
index 2ba4dbf..08926fa 100644
--- a/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
+++ b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
@@ -76,7 +76,7 @@ public class CqlStorage extends AbstractCassandraStorage
{
super();
this.pageSize = pageSize;
-DEFAULT_INPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
+DEFAULT_INPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlInputFormat;
DEFAULT_OUTPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlOutputFormat;
}
[1/3] cassandra git commit: fix errant CPIF references
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 f136bacb9 - 0757dc72f
refs/heads/trunk 3b945483b - f1475244f
fix errant CPIF references
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0757dc72
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0757dc72
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0757dc72
Branch: refs/heads/cassandra-2.1
Commit: 0757dc72f270c1f0efd212400e9a7040dc744c9b
Parents: f136bac
Author: Brandon Williams brandonwilli...@apache.org
Authored: Mon Jan 12 11:57:23 2015 -0600
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Mon Jan 12 11:57:53 2015 -0600
--
examples/hadoop_cql3_word_count/src/WordCount.java | 3 +--
examples/hadoop_cql3_word_count/src/WordCountCounters.java | 3 +--
src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java | 2 +-
3 files changed, 3 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0757dc72/examples/hadoop_cql3_word_count/src/WordCount.java
--
diff --git a/examples/hadoop_cql3_word_count/src/WordCount.java
b/examples/hadoop_cql3_word_count/src/WordCount.java
index 519a98f..6a2f846 100644
--- a/examples/hadoop_cql3_word_count/src/WordCount.java
+++ b/examples/hadoop_cql3_word_count/src/WordCount.java
@@ -26,7 +26,6 @@ import org.apache.cassandra.hadoop.cql3.CqlOutputFormat;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
import org.apache.cassandra.hadoop.cql3.CqlInputFormat;
import org.apache.cassandra.hadoop.ConfigHelper;
import org.apache.cassandra.utils.ByteBufferUtil;
@@ -247,7 +246,7 @@ public class WordCount extends Configured implements Tool
else
{
job.setMapperClass(TokenizerMapper.class);
-job.setInputFormatClass(CqlPagingInputFormat.class);
+job.setInputFormatClass(CqlInputFormat.class);
ConfigHelper.setInputRpcPort(job.getConfiguration(), 9160);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0757dc72/examples/hadoop_cql3_word_count/src/WordCountCounters.java
--
diff --git a/examples/hadoop_cql3_word_count/src/WordCountCounters.java
b/examples/hadoop_cql3_word_count/src/WordCountCounters.java
index 74de9ab..150d18d 100644
--- a/examples/hadoop_cql3_word_count/src/WordCountCounters.java
+++ b/examples/hadoop_cql3_word_count/src/WordCountCounters.java
@@ -25,7 +25,6 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.hadoop.cql3.CqlConfigHelper;
-import org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
import org.apache.cassandra.hadoop.cql3.CqlInputFormat;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
@@ -156,7 +155,7 @@ public class WordCountCounters extends Configured
implements Tool
else
{
job.setMapperClass(SumMapper.class);
-job.setInputFormatClass(CqlPagingInputFormat.class);
+job.setInputFormatClass(CqlInputFormat.class);
ConfigHelper.setInputRpcPort(job.getConfiguration(), 9160);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0757dc72/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
index 2ba4dbf..08926fa 100644
--- a/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
+++ b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
@@ -76,7 +76,7 @@ public class CqlStorage extends AbstractCassandraStorage
{
super();
this.pageSize = pageSize;
-DEFAULT_INPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
+DEFAULT_INPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlInputFormat;
DEFAULT_OUTPUT_FORMAT =
org.apache.cassandra.hadoop.cql3.CqlOutputFormat;
}
[jira] [Commented] (CASSANDRA-8541) References to non-existent/deprecated CqlPagingInputFormat in code
[ https://issues.apache.org/jira/browse/CASSANDRA-8541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273858#comment-14273858 ] Brandon Williams commented on CASSANDRA-8541: - Not sure what happened here, but I changed everything back in 2.1+ because CPIF doesn't exist. References to non-existent/deprecated CqlPagingInputFormat in code -- Key: CASSANDRA-8541 URL: https://issues.apache.org/jira/browse/CASSANDRA-8541 Project: Cassandra Issue Type: Bug Components: Hadoop Reporter: Rekha Joshi Assignee: Rekha Joshi Labels: hadoop Fix For: 2.0.12 Attachments: CASSANDRA-8541.txt On Mac 10.9.5, Java 1.7, latest cassandra trunk - References to non-existent/deprecated CqlPagingInputFormat in code. As per Changes.txt/7570 both CqlPagingInputFormat and CqlPagingRecordReader are removed, but lingering references in WordCount,CqlStorage.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8303) Provide strict mode for CQL Queries
[ https://issues.apache.org/jira/browse/CASSANDRA-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273851#comment-14273851 ] Aleksey Yeschenko commented on CASSANDRA-8303: -- After some IRC discussion we came up with another suggestion. As before, ignore the naming and the syntax, these are purely illustrative. The proposed new restrictions, while useful, do not conceptually fit with the concept of authorization. Authorization is about what data a particular role can access and/of modify, not about the particular query details. As far as authorization is concerned, there should be no distinction between writing a single partition to table 'foo' or two, in a batch - all that matters is that data in table 'foo' is being modified. I will strongly insist on it staying this way. There is also broad agreement that we need a way to broadly limit users' ability to harm cluster's performance with indexes, filtering, and batches (among other things). Given that we have roles now (or, rather, will have very soon - likely to be committed today or tomorrow) it seems natural to reuse the concept of roles for this broad capability management. It doesn't mean, however, that we should use authorization for this (which I'm strongly -1 on). The way Sam and Mike designed CASSANDRA-7653, roles, authentication, and authorization - are three separate concepts, and three separate APIs (IRoleManager, IAuthenticator, IAuthorizer). So we can take roles and authentication, leave out authorization, and add an extra API (say, ICapabilityManager) that would manage roles' restrictions on operation types a role can perform, not tied to any particular keyspace or table. On CQL side, we'd have something like this (syntax intentionally silly to prevent bikeshedding at this early stage): TAKE AWAY FILTERING FROM 'foo'; GIVE BACK BATCHES TO 'bar'; TAKE AWAY TRUNCATE FROM 'baz'; (I believe TRUNCATE belongs here, not to permissions - there we already have MODIFY) Shouldn't reuse GRANT/REVOKE because authorization/permission are conceptually a whitelist. A role has none unless it's granted some. Capability restrictions work as a black list - you can do any of those operations until someone puts a restriction on you. Permissions and restrictions are also inherited differently from parent roles. What does this buy us? 1. Being a new API, there is no rush for implementing it by 3.0.0 - we are now in aggressive feature scope cut mode. 2. If you don't care about authorization (which can be expensive, too), you can still enable broad restrictions, since they wouldn't rely on IAuthorizer 3. If you don't care about capability restrictions, but only about authorization, you can only enable IAuthorizer. The two APIs are nicely orthogonal. 4. IAuthorizer stays simple - both conceptually and implementation-wise. You don't want your security code be complex, for obvious reasons, but you also don't want the model to be confusing, which it would be if you tried to mix restrictions on batches and filtering with stuff like MODIFY and SELECT. 5. It can actually happen without being vetoed. Provide strict mode for CQL Queries - Key: CASSANDRA-8303 URL: https://issues.apache.org/jira/browse/CASSANDRA-8303 Project: Cassandra Issue Type: Improvement Reporter: Anupam Arora Fix For: 3.0 Please provide a strict mode option in cassandra that will kick out any CQL queries that are expensive, e.g. any query with ALLOWS FILTERING, multi-partition queries, secondary index queries, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8597) Stress: make simple things simple
[ https://issues.apache.org/jira/browse/CASSANDRA-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273863#comment-14273863 ] Benedict commented on CASSANDRA-8597: - I completely agree about documentation concerns, but these should probably be addressed to either the existing tickets or to new ones where they don't overlap. The parsing issue is something I've toyed with fixing off-and-on for a while, but felt I couldn't justify filing a ticket. Now stress is being used widely, it is probably worthwhile. It's actually pretty easy to achieve: we simply erase spaces after a bracket and before/after a comma in a pre-process, and the existing parser will work. Documenting that surrounding your entire command with double quotes may be useful is probably a good idea as well, to avoid having to escape all your options. Stress: make simple things simple - Key: CASSANDRA-8597 URL: https://issues.apache.org/jira/browse/CASSANDRA-8597 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jonathan Ellis Assignee: T Jake Luciani Fix For: 2.1.3 Some of the trouble people have with stress is a documentation problem, but some is functional. Comments from [~iamaleksey]: # 3 clustering columns, make a million cells in a single partition, should be simple, but it's not. have to tweak 'clustering' on the three columns just right to make stress work at all. w/ some values it'd just gets stuck forever computing batches # for others, it generates huge, megabyte-size batches, utterly disrespecting 'select' clause in 'insert' # I want a sequential generator too, to be able to predict deterministic result sets. uniform() only gets you so far # impossible to simulate a time series workload /cc [~jshook] [~aweisberg] [~benedict] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8577) Values of set types not loading correctly into Pig
[
https://issues.apache.org/jira/browse/CASSANDRA-8577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273865#comment-14273865
]
Philip Thompson commented on CASSANDRA-8577:
Running the reproduction steps provided by [~Oksana Danylyshyn], I successfully
see the expected result {{(key,(Running,onestep4red,running))}}
Values of set types not loading correctly into Pig
--
Key: CASSANDRA-8577
URL: https://issues.apache.org/jira/browse/CASSANDRA-8577
Project: Cassandra
Issue Type: Bug
Reporter: Oksana Danylyshyn
Assignee: Brandon Williams
Fix For: 2.1.3
Attachments: cassandra-2.1-8577.txt
Values of set types are not loading correctly from Cassandra (cql3 table,
Native protocol v3) into Pig using CqlNativeStorage.
When using Cassandra version 2.1.0 only empty values are loaded, and for
newer versions (2.1.1 and 2.1.2) the following error is received:
org.apache.cassandra.serializers.MarshalException: Unexpected extraneous
bytes after set value
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:94)
Steps to reproduce:
{code}cqlsh:socialdata CREATE TABLE test (
key varchar PRIMARY KEY,
tags setvarchar
);
cqlsh:socialdata insert into test (key, tags) values ('key', {'Running',
'onestep4red', 'running'});
cqlsh:socialdata select * from test;
key | tags
-+---
key | {'Running', 'onestep4red', 'running'}
(1 rows){code}
With version 2.1.0:
{code}grunt data = load 'cql://socialdata/test' using
org.apache.cassandra.hadoop.pig.CqlNativeStorage();
grunt dump data;
(key,()){code}
With version 2.1.2:
{code}grunt data = load 'cql://socialdata/test' using
org.apache.cassandra.hadoop.pig.CqlNativeStorage();
grunt dump data;
org.apache.cassandra.serializers.MarshalException: Unexpected extraneous
bytes after set value
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:94)
at
org.apache.cassandra.serializers.SetSerializer.deserializeForNativeProtocol(SetSerializer.java:27)
at
org.apache.cassandra.hadoop.pig.AbstractCassandraStorage.cassandraToObj(AbstractCassandraStorage.java:796)
at
org.apache.cassandra.hadoop.pig.CqlStorage.cqlColumnToObj(CqlStorage.java:195)
at
org.apache.cassandra.hadoop.pig.CqlNativeStorage.getNext(CqlNativeStorage.java:106)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212){code}
Expected result:
{code}(key,(Running,onestep4red,running)){code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8597) Stress: make simple things simple
[ https://issues.apache.org/jira/browse/CASSANDRA-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273799#comment-14273799 ] Benedict commented on CASSANDRA-8597: - # see CASSANDRA-7980 # This is a known problem with heavily skewed distributions, and is challenging to resolve, largely because we don't know how many values we will generate for any lower tier when deciding if we will descend from an upper tier (tier being clustering column prefix); this would be even worse with 7980. I've given this a little thought in the past, but since stress hasn't been considered a major priority have left on the back burner to try and resolve. One possibility is, instead of generating the number of values on a per tier, we _could_ instead generate a total number of values for all tiers, then generate a distribution for ratio of adoption for each tier, and each part of the tier. This is pretty difficult to conceptualise though, and implement. There are some other possibilities but they don't avoid similar problems. For instance, we could visit all of the lower tiers with the defined select chance, but since the upper tier may be filtered out with higher chance than it deserves, these rows will be visited with much lower likelihood. TL;DR: this is a complex ticket of its own, and requires a mini-research project to improve. # i'm not sure what's meant here? it's a deterministic workload if you use the -pop seq=1..N, except for thread interleavings and ancillary chances like select. Do you mean a deterministic non-uniform distribution? Deterministic select behaviour? # With 7980, we can simulate a workload very similar to a time-series one, by generating giant partitions with a temporal component and visiting their contents in ascending order. _Exactly_ simulating one requires some thought as to how to best define, model and deliver it though. The TODO in generator.Dates helps, but is probably not the best avenue; permitting expressions for ranges based on the partition seed might be a better route. I have idly wondered if, generally, we shouldn't permit some arbitrary javascript with a couple of predefined inputs to generate values, or the value ranges since this would be the most elegant and general way of supporting this. Again, not trivial though. Stress: make simple things simple - Key: CASSANDRA-8597 URL: https://issues.apache.org/jira/browse/CASSANDRA-8597 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Jonathan Ellis Assignee: T Jake Luciani Fix For: 2.1.3 Some of the trouble people have with stress is a documentation problem, but some is functional. Comments from [~iamaleksey]: # 3 clustering columns, make a million cells in a single partition, should be simple, but it's not. have to tweak 'clustering' on the three columns just right to make stress work at all. w/ some values it'd just gets stuck forever computing batches # for others, it generates huge, megabyte-size batches, utterly disrespecting 'select' clause in 'insert' # I want a sequential generator too, to be able to predict deterministic result sets. uniform() only gets you so far # impossible to simulate a time series workload /cc [~jshook] [~aweisberg] [~benedict] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
