[jira] [Updated] (CASSANDRA-13867) Make PartitionUpdate and Mutation immutable

2018-01-08 Thread Marcus Eriksson (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-13867:

   Resolution: Fixed
Fix Version/s: (was: 4.x)
   4.0
   Status: Resolved  (was: Ready to Commit)

Committed with the comments fixed as 
{{de7c24b395265ff619c622ed6be8d88453f158ac}} - thanks! The fixes are in 
separate commits 
[here|https://github.com/krummas/cassandra/commits/marcuse/13867-trunk-v2]

> Make PartitionUpdate and Mutation immutable
> ---
>
> Key: CASSANDRA-13867
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13867
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.0
>
>
> To avoid issues like CASSANDRA-13619 we should make PartitionUpdate and 
> Mutation immutable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[2/2] cassandra git commit: Make PartitionUpdate and Mutation immutable

2018-01-08 Thread marcuse 
Make PartitionUpdate and Mutation immutable

Patch by marcuse; reviewed by Aleksey Yeschenko for CASSANDRA-13867


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/de7c24b3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/de7c24b3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/de7c24b3

Branch: refs/heads/trunk
Commit: de7c24b395265ff619c622ed6be8d88453f158ac
Parents: 45c7c45
Author: Marcus Eriksson 
Authored: Wed Sep 13 15:57:50 2017 +0200
Committer: Marcus Eriksson 
Committed: Tue Jan 9 08:10:23 2018 +0100

--
 CHANGES.txt |   1 +
 .../cql3/statements/BatchStatement.java |   4 +-
 .../cql3/statements/BatchUpdatesCollector.java  | 231 ++
 .../cql3/statements/CQL3CasRequest.java |  26 +-
 .../cql3/statements/DeleteStatement.java|  18 +-
 .../cql3/statements/ModificationStatement.java  |  33 +-
 .../statements/SingleTableUpdatesCollector.java | 101 +
 .../cql3/statements/UpdateStatement.java|  12 +-
 .../cql3/statements/UpdatesCollector.java   | 127 +-
 .../apache/cassandra/db/CounterMutation.java|   7 +-
 src/java/org/apache/cassandra/db/IMutation.java |   6 +
 src/java/org/apache/cassandra/db/Mutation.java  | 137 +++---
 .../org/apache/cassandra/db/SimpleBuilders.java |  25 +-
 .../org/apache/cassandra/db/SystemKeyspace.java |   9 +-
 .../db/commitlog/CommitLogReplayer.java |  20 +-
 .../db/partitions/PartitionUpdate.java  | 448 +--
 .../apache/cassandra/db/view/TableViews.java|  16 +-
 .../cassandra/db/view/ViewUpdateGenerator.java  |  20 +-
 .../io/sstable/AbstractSSTableSimpleWriter.java |   4 +-
 .../io/sstable/SSTableSimpleUnsortedWriter.java |  22 +-
 .../io/sstable/SSTableSimpleWriter.java |  10 +-
 .../apache/cassandra/schema/SchemaKeyspace.java |  14 +-
 .../apache/cassandra/service/DataResolver.java  |   8 +-
 .../apache/cassandra/service/paxos/Commit.java  |   4 +-
 .../cassandra/db/ColumnFamilyStoreTest.java |   4 +-
 .../cassandra/db/CounterMutationTest.java   |   8 +-
 .../cassandra/db/DeletePartitionTest.java   |   3 +-
 test/unit/org/apache/cassandra/db/RowTest.java  |   8 +-
 .../apache/cassandra/db/RowUpdateBuilder.java   |  28 +-
 .../db/commitlog/CommitLogReaderTest.java   |   4 +-
 .../db/compaction/CompactionsPurgeTest.java |  11 +-
 .../db/partition/PartitionUpdateTest.java   |  15 +
 .../rows/RowAndDeletionMergeIteratorTest.java   |   6 +-
 .../cassandra/index/sasi/SASIIndexTest.java | 142 +++---
 .../cassandra/net/WriteCallbackInfoTest.java|   6 +-
 .../cassandra/triggers/TriggerExecutorTest.java |  18 +-
 36 files changed, 912 insertions(+), 644 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/de7c24b3/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ba0f842..623bf8a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0
+ * Make PartitionUpdate and Mutation immutable (CASSANDRA-13867)
  * Fix CommitLogReplayer exception for CDC data (CASSANDRA-14066)
  * Fix cassandra-stress startup failure (CASSANDRA-14106)
  * Remove initialDirectories from CFS (CASSANDRA-13928)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/de7c24b3/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
index 7497f47..b54e3a0 100644
--- a/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
@@ -224,7 +224,7 @@ public class BatchStatement implements CQLStatement
 throws RequestExecutionException, RequestValidationException
 {
 Set tablesWithZeroGcGs = null;
-UpdatesCollector collector = new UpdatesCollector(updatedColumns, 
updatedRows());
+BatchUpdatesCollector collector = new 
BatchUpdatesCollector(updatedColumns, updatedRows());
 for (int i = 0; i < statements.size(); i++)
 {
 ModificationStatement statement = statements.get(i);
@@ -247,8 +247,6 @@ public class BatchStatement implements CQLStatement
 
ClientWarn.instance.warn(MessageFormatter.arrayFormat(LOGGED_BATCH_LOW_GCGS_WARNING,
 new Object[] { suffix, tablesWithZeroGcGs })
  .getMessage());
 }
-
-collector.validateIndexedColumns();
 return collector.toMutations();
 }

[1/2] cassandra git commit: Make PartitionUpdate and Mutation immutable

2018-01-08 Thread marcuse 
Repository: cassandra
Updated Branches:
  refs/heads/trunk 45c7c4561 -> de7c24b39


http://git-wip-us.apache.org/repos/asf/cassandra/blob/de7c24b3/src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java 
b/src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java
index a663051..530a03b 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java
@@ -39,7 +39,7 @@ import org.apache.cassandra.schema.TableMetadataRef;
 class SSTableSimpleWriter extends AbstractSSTableSimpleWriter
 {
 protected DecoratedKey currentKey;
-protected PartitionUpdate update;
+protected PartitionUpdate.Builder update;
 
 private SSTableTxnWriter writer;
 
@@ -56,7 +56,7 @@ class SSTableSimpleWriter extends AbstractSSTableSimpleWriter
 return writer;
 }
 
-PartitionUpdate getUpdateFor(DecoratedKey key) throws IOException
+PartitionUpdate.Builder getUpdateFor(DecoratedKey key) throws IOException
 {
 assert key != null;
 
@@ -65,9 +65,9 @@ class SSTableSimpleWriter extends AbstractSSTableSimpleWriter
 if (!key.equals(currentKey))
 {
 if (update != null)
-writePartition(update);
+writePartition(update.build());
 currentKey = key;
-update = new PartitionUpdate(metadata.get(), currentKey, columns, 
4);
+update = new PartitionUpdate.Builder(metadata.get(), currentKey, 
columns, 4);
 }
 
 assert update != null;
@@ -79,7 +79,7 @@ class SSTableSimpleWriter extends AbstractSSTableSimpleWriter
 try
 {
 if (update != null)
-writePartition(update);
+writePartition(update.build());
 if (writer != null)
 writer.finish(false);
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/de7c24b3/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java 
b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
index cd03a40..9d86feb 100644
--- a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
+++ b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
@@ -21,6 +21,7 @@ import java.nio.ByteBuffer;
 import java.nio.charset.CharacterCodingException;
 import java.util.*;
 import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
 
 import com.google.common.collect.*;
 import com.google.common.collect.Maps;
@@ -356,15 +357,15 @@ public final class SchemaKeyspace
 
 static Collection convertSchemaToMutations()
 {
-Map mutationMap = new HashMap<>();
+Map mutationMap = new 
HashMap<>();
 
 for (String table : ALL)
 convertSchemaToMutations(mutationMap, table);
 
-return mutationMap.values();
+return 
mutationMap.values().stream().map(Mutation.PartitionUpdateCollector::build).collect(Collectors.toList());
 }
 
-private static void convertSchemaToMutations(Map 
mutationMap, String schemaTableName)
+private static void convertSchemaToMutations(Map mutationMap, String schemaTableName)
 {
 ReadCommand cmd = getReadCommandForTableSchema(schemaTableName);
 try (ReadExecutionController executionController = 
cmd.executionController();
@@ -378,9 +379,8 @@ public final class SchemaKeyspace
 continue;
 
 DecoratedKey key = partition.partitionKey();
-Mutation mutation = mutationMap.computeIfAbsent(key, k -> 
new Mutation(SchemaConstants.SCHEMA_KEYSPACE_NAME, key));
-
-mutation.add(makeUpdateForSchema(partition, 
cmd.columnFilter()));
+Mutation.PartitionUpdateCollector puCollector = 
mutationMap.computeIfAbsent(key, k -> new 
Mutation.PartitionUpdateCollector(SchemaConstants.SCHEMA_KEYSPACE_NAME, key));
+puCollector.add(makeUpdateForSchema(partition, 
cmd.columnFilter()));
 }
 }
 }
@@ -423,7 +423,7 @@ public final class SchemaKeyspace
 @SuppressWarnings("unchecked")
 private static DecoratedKey decorate(TableMetadata metadata, Object value)
 {
-return 
metadata.partitioner.decorateKey(((AbstractType)metadata.partitionKeyType).decompose(value));
+return metadata.partitioner.decorateKey(((AbstractType) 
metadata.partitionKeyType).decompose(value));
 }
 
 static Mutation.SimpleBuilder makeCreateKeyspaceMutation(String name, 
KeyspaceParams params, long timestamp)

[jira] [Updated] (CASSANDRA-14128) Prune dead links from cassandra.apache.org

2018-01-08 Thread Kurt Greaves (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kurt Greaves updated CASSANDRA-14128:
-
Attachment: 14128-site.patch

> Prune dead links from cassandra.apache.org
> --
>
> Key: CASSANDRA-14128
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14128
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation and Website
>Reporter: Jeff Jirsa
>Assignee: Kurt Greaves
>Priority: Trivial
>  Labels: lhf
> Attachments: 14128-site.patch
>
>
> Lots of dead links on the site, including the homepage. Should do some 
> pruning cleanup.
> Site repo is [here|https://svn.apache.org/repos/asf/cassandra/site/] (in svn) 
> for anyone who wishes to give this a shot.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14128) Prune dead links from cassandra.apache.org

2018-01-08 Thread Kurt Greaves (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kurt Greaves updated CASSANDRA-14128:
-
Status: Patch Available  (was: In Progress)

So, initial changes:
Removed a bunch of dead links from the homepage. Fixed some typo's. Makes it 
look a bit bare, but as I said we can add more content/references later on.

Updated any links pointing to the old wiki to point to our current docs. In the 
case of the links for hints and read repair I've also got a patch 
[here|https://github.com/apache/cassandra/compare/trunk...kgreav:14128-trunk] 
which has some very basic docs copied from the old wiki and a few tidbits I've 
added (needs to be checked for correctness).

I've also moved the thirdparty support page over to the new website, however we 
need to decide whether this is something we still want to have. Personally, my 
biased opinion says it is useful, as the first place people are going to look 
for support is on the C* website. It also encourages people to use C* if they 
can clearly see that there are backers where they can get support.
Cons are it will be slightly more difficult for companies to "add themselves" 
to the new page, as it will require a JIRA and a patch, and thus require 
reviewing, but we can document this and it's not like it will be much of a task 
to review additions. There is also some onus on us to keep the page up to date, 
or at least actually review any tickets that want to make changes to the page 
within some reasonable timeframe (or we'll be blamed for endorsing/not 
endorsing companies). All in all this is probably a question for the dev ML, 
WDYT [~jjirsa]?

In my current changes I've shamelessly re-ordered and removed some companies 
where their links didn't work (palominodb and Venarc) and URImagination, who 
I've contacted to see whether their link is still relevant, as they don't have 
a mention of C* on their website at all anymore.
I've also left Datastax out for the moment, as their advert obviously has some 
problems since their departure from Apache Cassandra. Will need someone from DS 
to confirm what they want to do here.
If we're going to keep this page it would also make sense to add others to the 
list who are currently missing as well. If we decide to keep it I'll poke some 
people.

> Prune dead links from cassandra.apache.org
> --
>
> Key: CASSANDRA-14128
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14128
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation and Website
>Reporter: Jeff Jirsa
>Assignee: Kurt Greaves
>Priority: Trivial
>  Labels: lhf
>
> Lots of dead links on the site, including the homepage. Should do some 
> pruning cleanup.
> Site repo is [here|https://svn.apache.org/repos/asf/cassandra/site/] (in svn) 
> for anyone who wishes to give this a shot.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-14155) [TRUNK] Gossiper somewhat frequently hitting an NPE on node startup with dtests at org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)

2018-01-08 Thread Jason Brown (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown reassigned CASSANDRA-14155:
---

Assignee: Jason Brown

> [TRUNK] Gossiper somewhat frequently hitting an NPE on node startup with 
> dtests at 
> org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
> 
>
> Key: CASSANDRA-14155
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14155
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Kjellman
>Assignee: Jason Brown
>
> Gossiper is somewhat frequently hitting an NPE on node startup with dtests at 
> org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
> {code}
> test teardown failure
> Unexpected error found in node logs (see stdout for full details). Errors: 
> [ERROR [main] 2018-01-08 21:41:01,832 CassandraDaemon.java:675 - Exception 
> encountered during startup
> java.lang.NullPointerException: null
> at 
> org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:511)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:761)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:621)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:568)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:360) 
> [main/:na]
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:569)
>  [main/:na]
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:658) 
> [main/:na], ERROR [main] 2018-01-08 21:41:01,832 CassandraDaemon.java:675 - 
> Exception encountered during startup
> java.lang.NullPointerException: null
> at 
> org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769) 
> ~[main/:na]
> at 
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:511)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:761)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:621)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:568)
>  ~[main/:na]
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:360) 
> [main/:na]
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:569)
>  [main/:na]
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:658) 
> [main/:na]]
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13391) nodetool clearsnapshot should require --all to clear all snapshots

2018-01-08 Thread Jon Haddad (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317321#comment-16317321
 ] 

Jon Haddad commented on CASSANDRA-13391:


[~zznate] Good call, I had added a note to CHANGES.txt but didn't have anything 
in NEWS.txt.  Pushed up 45c7c45616.

> nodetool clearsnapshot should require --all to clear all snapshots
> --
>
> Key: CASSANDRA-13391
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13391
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jon Haddad
> Fix For: 4.0
>
>
> Deleting all snapshots by default is insanely dangerous.  It would be like if 
> rm's default was -rf /.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

cassandra git commit: Updated NEWS.txt with details on change to nodetool snapshot.

2018-01-08 Thread rustyrazorblade 
Repository: cassandra
Updated Branches:
  refs/heads/trunk 21be9d2f5 -> 45c7c4561


Updated NEWS.txt with details on change to nodetool snapshot.


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/45c7c456
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/45c7c456
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/45c7c456

Branch: refs/heads/trunk
Commit: 45c7c456167e77cb7e8cdf161bc4350deba08aef
Parents: 21be9d2
Author: Jon Haddad 
Authored: Mon Jan 8 15:19:37 2018 -0800
Committer: Jon Haddad 
Committed: Mon Jan 8 15:19:37 2018 -0800

--
 NEWS.txt | 2 ++
 1 file changed, 2 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/45c7c456/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 259507e..06d73ea 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -101,6 +101,8 @@ Upgrading
  Batch mode remains a valid commit log mode, however.
- There is a new commit log mode, group, which is similar to batch mode
  but blocks for up to a configurable number of milliseconds between 
disk flushes.
+   - nodetool clearsnapshot now required the --all flag to remove all 
snapshots.
+ Previous behavior would delete all snapshots by default.
 
 Materialized Views
 ---


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Issue Comment Deleted] (CASSANDRA-13391) nodetool clearsnapshot should require --all to clear all snapshots

2018-01-08 Thread Jon Haddad (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Haddad updated CASSANDRA-13391:
---
Comment: was deleted

(was: [~zznate] I did as a followup commit in e2e28a02e8, forgot to make a note 
here.  )

> nodetool clearsnapshot should require --all to clear all snapshots
> --
>
> Key: CASSANDRA-13391
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13391
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jon Haddad
> Fix For: 4.0
>
>
> Deleting all snapshots by default is insanely dangerous.  It would be like if 
> rm's default was -rf /.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13391) nodetool clearsnapshot should require --all to clear all snapshots

2018-01-08 Thread Jon Haddad (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317312#comment-16317312
 ] 

Jon Haddad commented on CASSANDRA-13391:


[~zznate] I did as a followup commit in e2e28a02e8, forgot to make a note here. 
 

> nodetool clearsnapshot should require --all to clear all snapshots
> --
>
> Key: CASSANDRA-13391
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13391
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jon Haddad
> Fix For: 4.0
>
>
> Deleting all snapshots by default is insanely dangerous.  It would be like if 
> rm's default was -rf /.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13391) nodetool clearsnapshot should require --all to clear all snapshots

2018-01-08 Thread Nate McCall (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317307#comment-16317307
 ] 

Nate McCall commented on CASSANDRA-13391:
-

[~rustyrazorblade] Can you add line to NEWS.txt for 4.0 with this change? I 
know there are folks out there relying on this behavior in scripts and such and 
we are changing a default. 

> nodetool clearsnapshot should require --all to clear all snapshots
> --
>
> Key: CASSANDRA-13391
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13391
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jon Haddad
> Fix For: 4.0
>
>
> Deleting all snapshots by default is insanely dangerous.  It would be like if 
> rm's default was -rf /.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14155) [TRUNK] Gossiper somewhat frequently hitting an NPE on node startup with dtests at org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)

2018-01-08 Thread Michael Kjellman (JIRA) 
Michael Kjellman created CASSANDRA-14155:


 Summary: [TRUNK] Gossiper somewhat frequently hitting an NPE on 
node startup with dtests at 
org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)
 Key: CASSANDRA-14155
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14155
 Project: Cassandra
  Issue Type: Bug
Reporter: Michael Kjellman


Gossiper is somewhat frequently hitting an NPE on node startup with dtests at 
org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769)

{code}
test teardown failure
Unexpected error found in node logs (see stdout for full details). Errors: 
[ERROR [main] 2018-01-08 21:41:01,832 CassandraDaemon.java:675 - Exception 
encountered during startup
java.lang.NullPointerException: null
at 
org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769) 
~[main/:na]
at 
org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:511)
 ~[main/:na]
at 
org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:761)
 ~[main/:na]
at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:621) 
~[main/:na]
at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:568) 
~[main/:na]
at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:360) 
[main/:na]
at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:569) 
[main/:na]
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:658) 
[main/:na], ERROR [main] 2018-01-08 21:41:01,832 CassandraDaemon.java:675 - 
Exception encountered during startup
java.lang.NullPointerException: null
at 
org.apache.cassandra.gms.Gossiper.isSafeForStartup(Gossiper.java:769) 
~[main/:na]
at 
org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:511)
 ~[main/:na]
at 
org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:761)
 ~[main/:na]
at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:621) 
~[main/:na]
at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:568) 
~[main/:na]
at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:360) 
[main/:na]
at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:569) 
[main/:na]
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:658) 
[main/:na]]
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-12763) Compaction performance issues when a table has a lot of sstables

2018-01-08 Thread Marcus Eriksson (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-12763:

   Labels:   (was: lcs)
Fix Version/s: 4.x
   3.11.x
   3.0.x
   Status: Patch Available  (was: Open)

patch that pre-creates the deletion log records and does not recreate the 
records when checking if the log transaction contains operations for the same 
sstables:

https://github.com/krummas/cassandra/commits/marcuse/12763
https://circleci.com/gh/krummas/cassandra/tree/marcuse%2F12763
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/468/console

> Compaction performance issues when a table has a lot of sstables
> 
>
> Key: CASSANDRA-12763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12763
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Tom van der Woerdt
>Assignee: Marcus Eriksson
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> An issue with a script flooded my cluster with sstables. There is now a table 
> with 100k sstables, all on the order of KBytes, and it's taking a long time 
> (ETA 20 days) to compact, even though the table is only ~30GB.
> Stack trace :
> {noformat}
> "CompactionExecutor:269" #7536 daemon prio=1 os_prio=4 tid=0x7f4acd40fc00 
> nid=0x14f8 runnable [0x7f4798436000]
>java.lang.Thread.State: RUNNABLE
>   at java.io.UnixFileSystem.list(Native Method)
>   at java.io.File.list(File.java:1122)
>   at java.io.File.listFiles(File.java:1248)
>   at 
> org.apache.cassandra.db.lifecycle.LogRecord.getExistingFiles(LogRecord.java:268)
>   at org.apache.cassandra.db.lifecycle.LogRecord.make(LogRecord.java:150)
>   at 
> org.apache.cassandra.db.lifecycle.LogFile.makeRecord(LogFile.java:293)
>   at org.apache.cassandra.db.lifecycle.LogFile.add(LogFile.java:283)
>   at 
> org.apache.cassandra.db.lifecycle.LogTransaction.obsoleted(LogTransaction.java:158)
>   at 
> org.apache.cassandra.db.lifecycle.Helpers.prepareForObsoletion(Helpers.java:134)
>   at 
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.doPrepare(LifecycleTransaction.java:193)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.io.sstable.SSTableRewriter.doPrepare(SSTableRewriter.java:376)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.doPrepare(CompactionAwareWriter.java:84)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish(Transactional.java:184)
>   at 
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.finish(CompactionAwareWriter.java:94)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:194)
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>   at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
>   at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}
> listFiles is being called over and over, apparently scaling with the number 
> of files in the compaction.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14154) `ant javadoc` task broken due to UTF-8 characters in multiple source files

2018-01-08 Thread Johannes Grassler (JIRA) 
Johannes Grassler created CASSANDRA-14154:
-

 Summary: `ant javadoc` task broken due to UTF-8 characters in 
multiple source files
 Key: CASSANDRA-14154
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14154
 Project: Cassandra
  Issue Type: Bug
  Components: Build
 Environment: Built on OpenSUSE Tumbleweed. I used the following java 
packages when building:

{quote}
titan% rpm -qa | grep java
javapackages-local-4.7.0+git20170331.ef4057e7-1.5.x86_64
java-1_8_0-openjdk-devel-1.8.0.151-1.3.x86_64
python3-javapackages-4.7.0+git20170331.ef4057e7-1.5.x86_64
java-1_8_0-openjdk-headless-1.8.0.151-1.3.x86_64
timezone-java-2017c-1.3.noarch
java-1_8_0-openjdk-1.8.0.151-1.3.x86_64
libjavascriptcoregtk-1_0-0-2.4.11-7.6.x86_64
libjavascriptcoregtk-4_0-18-2.18.4-1.1.x86_64
javapackages-tools-4.7.0+git20170331.ef4057e7-1.5.x86_64
{quote}
Reporter: Johannes Grassler
Priority: Minor
 Attachments: build.log, javadoc-encoding.patch

Several source files contain UTF-8 characters in String literals. When building 
the {{javadoc}} target with ant ({{ant javadoc}}), these will trip up javadoc, 
which defaults to ASCII encoding. See the {{build.log}} for what I did and the 
resulting output.

I created a patch that will fix the problem ({{javadoc-encoding.patch}}), which 
is attached.

I encountered this problem in 3.11.1, but I haven't checked whether other 
versions are affected as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14145) Detecting data resurrection during read

2018-01-08 Thread Stefan Podkowinski (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316132#comment-16316132
 ] 

Stefan Podkowinski commented on CASSANDRA-14145:


I agree with the problem description and that we should detect inconsistencies 
in the repaired set when reading data. But, as described in CASSANDRA-13912, 
I'm not convinced that correcting data as part of read repairs is the way to go 
for already repaired data. 

There could be many different reasons for inconsistencies. Corrupted or missing 
sstables are relatively easy to reason about in this context. What I'm more 
concerned of are internal consistency issues due to rare edge cases, e.g. 
during upgrade paths or race conditions. Returning inconsistent results in rare 
cases might be less of an issue, compared to risking greater harm by running 
repair code which was created with other situations in mind.

I'm wondering if we shouldn't just start by logging inconsistencies, allowing 
operators to further investigate. If there's some serious issue with a node, it 
needs to be replaced anyways. If the hardware is fine and there are no external 
causes that lead to inconsistencies, we may have a bigger problem to fix.


>  Detecting data resurrection during read
> 
>
> Key: CASSANDRA-14145
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14145
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Priority: Minor
>
> We have seen several bugs in which deleted data gets resurrected. We should 
> try to see if we can detect this on the read path and possibly fix it. Here 
> are a few examples which brought back data
> A replica lost an sstable on startup which caused one replica to lose the 
> tombstone and not the data. This tombstone was past gc grace which means this 
> could resurrect data. We can detect such invalid states by looking at other 
> replicas. 
> If we are running incremental repair, Cassandra will keep repaired and 
> non-repaired data separate. Every-time incremental repair will run, it will 
> move the data from non-repaired to repaired. Repaired data across all 
> replicas should be 100% consistent. 
> Here is an example of how we can detect and mitigate the issue in most cases. 
> Say we have 3 machines, A,B and C. All these machines will have data split 
> b/w repaired and non-repaired. 
> 1. Machine A due to some bug bring backs data D. This data D is in repaired 
> dataset. All other replicas will have data D and tombstone T 
> 2. Read for data D comes from application which involve replicas A and B. The 
> data being read involves data which is in repaired state.  A will respond 
> back to co-ordinator with data D and B will send nothing as tombstone is past 
> gc grace. This will cause digest mismatch. 
> 3. This patch will only kick in when there is a digest mismatch. Co-ordinator 
> will ask both replicas to send back all data like we do today but with this 
> patch, replicas will respond back what data it is returning is coming from 
> repaired vs non-repaired. If data coming from repaired does not match, we 
> know there is a something wrong!! At this time, co-ordinator cannot determine 
> if replica A has resurrected some data or replica B has lost some data. We 
> can still log error in the logs saying we hit an invalid state.
> 4. Besides the log, we can take this further and even correct the response to 
> the query. After logging an invalid state, we can ask replica A and B (and 
> also C if alive) to send back all data for this including gcable tombstones. 
> If any machine returns a tombstone which is after this data, we know we 
> cannot return this data. This way we can avoid returning data which has been 
> deleted. 
> Some Challenges with this 
> 1. When data will be moved from non-repaired to repaired, there could be a 
> race here. We can look at which incremental repairs have promoted things on 
> which replica to avoid false positives.  
> 2. If the third replica is down and live replica does not have any tombstone, 
> we wont be able to break the tie in deciding whether data was actually 
> deleted or resurrected. 
> 3. If the read is for latest data only, we wont be able to detect it as the 
> read will be served from non-repaired data. 
> 4. If the replica where we lose a tombstone is the last replica to compact 
> the tombstone, we wont be able to decide if data is coming back or rest of 
> the replicas has lost that data. But we will still detect something is wrong. 
> 5. We wont affect 99.9% of the read queries as we only do extra work during 
> digest mismatch.
> 6. CL.ONE reads will not be able to detect this. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To

[jira] [Updated] (CASSANDRA-8527) Account for range tombstones wherever we account for tombstones

2018-01-08 Thread Alexander Dejanovski (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Dejanovski updated CASSANDRA-8527:

Attachment: CASSANDRA-8527-4.x.diff

> Account for range tombstones wherever we account for tombstones
> ---
>
> Key: CASSANDRA-8527
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8527
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Alexander Dejanovski
> Fix For: 4.x
>
> Attachments: CASSANDRA-8527-4.x.diff
>
>
> As discussed in CASSANDRA-8477, we should make sure the tombstone thresholds 
> also apply to range tombstones, since they poses the same problems than cell 
> tombstones.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8527) Account for range tombstones wherever we account for tombstones

2018-01-08 Thread Alexander Dejanovski (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16315931#comment-16315931
 ] 

Alexander Dejanovski commented on CASSANDRA-8527:
-

[~rustyrazorblade], 

I've simplified the patch a lot following the discussions on the dev ML. 
Counting row tombstones as "deleted rows" creates a confusion that rows 
shadowed by multi row range tombstones would be counted too.
Since it's not the case (rows shadowed by RT get merged away before the 
ReadCommand class and only 2 tombstone cells get counted for each RT), I've 
moved back to counting row tombstones as part of the current tombstone cell 
counter.

Here's the updated code : 
https://github.com/apache/cassandra/compare/trunk...thelastpickle:CASSANDRA-8527-trace-range-tombstones

We will deal with shadowed rows in CASSANDRA-14149.

> Account for range tombstones wherever we account for tombstones
> ---
>
> Key: CASSANDRA-8527
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8527
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Alexander Dejanovski
> Fix For: 4.x
>
>
> As discussed in CASSANDRA-8477, we should make sure the tombstone thresholds 
> also apply to range tombstones, since they poses the same problems than cell 
> tombstones.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-8527) Account for range tombstones wherever we account for tombstones

2018-01-08 Thread Alexander Dejanovski (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Dejanovski updated CASSANDRA-8527:

Status: Patch Available  (was: Open)

> Account for range tombstones wherever we account for tombstones
> ---
>
> Key: CASSANDRA-8527
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8527
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Alexander Dejanovski
> Fix For: 4.x
>
> Attachments: CASSANDRA-8527-4.x.diff
>
>
> As discussed in CASSANDRA-8477, we should make sure the tombstone thresholds 
> also apply to range tombstones, since they poses the same problems than cell 
> tombstones.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-8527) Account for range tombstones wherever we account for tombstones

2018-01-08 Thread Alexander Dejanovski (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Dejanovski updated CASSANDRA-8527:

Status: Open  (was: Patch Available)

> Account for range tombstones wherever we account for tombstones
> ---
>
> Key: CASSANDRA-8527
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8527
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Alexander Dejanovski
> Fix For: 4.x
>
>
> As discussed in CASSANDRA-8477, we should make sure the tombstone thresholds 
> also apply to range tombstones, since they poses the same problems than cell 
> tombstones.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-8527) Account for range tombstones wherever we account for tombstones

2018-01-08 Thread Alexander Dejanovski (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Dejanovski updated CASSANDRA-8527:

Attachment: (was: CASSANDRA-8527.diff)

> Account for range tombstones wherever we account for tombstones
> ---
>
> Key: CASSANDRA-8527
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8527
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Alexander Dejanovski
> Fix For: 4.x
>
>
> As discussed in CASSANDRA-8477, we should make sure the tombstone thresholds 
> also apply to range tombstones, since they poses the same problems than cell 
> tombstones.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org