[jira] [Assigned] (CASSANDRA-14725) calculatePendingRanges does not handle full<->transient transition implied by add/remove another node

2019-04-16 Thread Benedict (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict reassigned CASSANDRA-14725:


Assignee: Benedict

> calculatePendingRanges does not handle full<->transient transition implied by 
> add/remove another node
> -
>
> Key: CASSANDRA-14725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14725
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Coordination
>Reporter: Benedict
>Assignee: Benedict
>Priority: Normal
>  Labels: correctness, transient-replication
> Fix For: 4.0
>
>
> We only implemented handling those implied directly by a node changing its 
> own status, but a node entering or leaving the ring can imply such a movement 
> on another node, and these need to be handled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Assigned] (CASSANDRA-14725) calculatePendingRanges does not handle full<->transient transition implied by add/remove another node

2019-04-16 Thread Benedict (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict reassigned CASSANDRA-14725:


Assignee: (was: Benedict)

> calculatePendingRanges does not handle full<->transient transition implied by 
> add/remove another node
> -
>
> Key: CASSANDRA-14725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14725
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Coordination
>Reporter: Benedict
>Priority: Normal
>  Labels: correctness, transient-replication
> Fix For: 4.0
>
>
> We only implemented handling those implied directly by a node changing its 
> own status, but a node entering or leaving the ring can imply such a movement 
> on another node, and these need to be handled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14725) calculatePendingRanges does not handle full<->transient transition implied by add/remove another node

2019-04-16 Thread Benedict (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-14725:
-
Severity: Critical  (was: Normal)

> calculatePendingRanges does not handle full<->transient transition implied by 
> add/remove another node
> -
>
> Key: CASSANDRA-14725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14725
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Coordination
>Reporter: Benedict
>Assignee: Benedict
>Priority: Urgent
>  Labels: correctness, transient-replication
> Fix For: 4.0
>
>
> We only implemented handling those implied directly by a node changing its 
> own status, but a node entering or leaving the ring can imply such a movement 
> on another node, and these need to be handled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14725) calculatePendingRanges does not handle full<->transient transition implied by add/remove another node

2019-04-16 Thread Benedict (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-14725:
-
Priority: Urgent  (was: Normal)

> calculatePendingRanges does not handle full<->transient transition implied by 
> add/remove another node
> -
>
> Key: CASSANDRA-14725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14725
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Coordination
>Reporter: Benedict
>Assignee: Benedict
>Priority: Urgent
>  Labels: correctness, transient-replication
> Fix For: 4.0
>
>
> We only implemented handling those implied directly by a node changing its 
> own status, but a node entering or leaving the ring can imply such a movement 
> on another node, and these need to be handled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14725) calculatePendingRanges does not handle full<->transient transition implied by add/remove another node

2019-04-16 Thread Benedict (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-14725:
-
Severity: Normal  (was: Critical)

> calculatePendingRanges does not handle full<->transient transition implied by 
> add/remove another node
> -
>
> Key: CASSANDRA-14725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14725
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Coordination
>Reporter: Benedict
>Assignee: Benedict
>Priority: Normal
>  Labels: correctness, transient-replication
> Fix For: 4.0
>
>
> We only implemented handling those implied directly by a node changing its 
> own status, but a node entering or leaving the ring can imply such a movement 
> on another node, and these need to be handled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14725) calculatePendingRanges does not handle full<->transient transition implied by add/remove another node

2019-04-16 Thread Benedict (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-14725:
-
Priority: Normal  (was: Urgent)

> calculatePendingRanges does not handle full<->transient transition implied by 
> add/remove another node
> -
>
> Key: CASSANDRA-14725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14725
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Coordination
>Reporter: Benedict
>Assignee: Benedict
>Priority: Normal
>  Labels: correctness, transient-replication
> Fix For: 4.0
>
>
> We only implemented handling those implied directly by a node changing its 
> own status, but a node entering or leaving the ring can imply such a movement 
> on another node, and these need to be handled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14725) calculatePendingRanges does not handle full<->transient transition implied by add/remove another node

2019-04-16 Thread Benedict (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-14725:
-
Severity: Critical  (was: Normal)

> calculatePendingRanges does not handle full<->transient transition implied by 
> add/remove another node
> -
>
> Key: CASSANDRA-14725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14725
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Coordination
>Reporter: Benedict
>Assignee: Benedict
>Priority: Urgent
>  Labels: correctness, transient-replication
> Fix For: 4.0
>
>
> We only implemented handling those implied directly by a node changing its 
> own status, but a node entering or leaving the ring can imply such a movement 
> on another node, and these need to be handled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14725) calculatePendingRanges does not handle full<->transient transition implied by add/remove another node

2019-04-16 Thread Benedict (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-14725:
-
Severity: Normal  (was: Critical)

> calculatePendingRanges does not handle full<->transient transition implied by 
> add/remove another node
> -
>
> Key: CASSANDRA-14725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14725
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Coordination
>Reporter: Benedict
>Assignee: Benedict
>Priority: Normal
>  Labels: correctness, transient-replication
> Fix For: 4.0
>
>
> We only implemented handling those implied directly by a node changing its 
> own status, but a node entering or leaving the ring can imply such a movement 
> on another node, and these need to be handled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14725) calculatePendingRanges does not handle full<->transient transition implied by add/remove another node

2019-04-16 Thread Benedict (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-14725:
-
Severity: Low  (was: Normal)

> calculatePendingRanges does not handle full<->transient transition implied by 
> add/remove another node
> -
>
> Key: CASSANDRA-14725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14725
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Coordination
>Reporter: Benedict
>Assignee: Benedict
>Priority: Low
>  Labels: correctness, transient-replication
> Fix For: 4.0
>
>
> We only implemented handling those implied directly by a node changing its 
> own status, but a node entering or leaving the ring can imply such a movement 
> on another node, and these need to be handled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14725) calculatePendingRanges does not handle full<->transient transition implied by add/remove another node

2019-04-16 Thread Benedict (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-14725:
-
Severity:   (was: Low)

> calculatePendingRanges does not handle full<->transient transition implied by 
> add/remove another node
> -
>
> Key: CASSANDRA-14725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14725
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Coordination
>Reporter: Benedict
>Assignee: Benedict
>Priority: Low
>  Labels: correctness, transient-replication
> Fix For: 4.0
>
>
> We only implemented handling those implied directly by a node changing its 
> own status, but a node entering or leaving the ring can imply such a movement 
> on another node, and these need to be handled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14725) calculatePendingRanges does not handle full<->transient transition implied by add/remove another node

2019-04-16 Thread Benedict (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-14725:
-
Severity: Normal

> calculatePendingRanges does not handle full<->transient transition implied by 
> add/remove another node
> -
>
> Key: CASSANDRA-14725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14725
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Coordination
>Reporter: Benedict
>Assignee: Benedict
>Priority: Normal
>  Labels: correctness, transient-replication
> Fix For: 4.0
>
>
> We only implemented handling those implied directly by a node changing its 
> own status, but a node entering or leaving the ring can imply such a movement 
> on another node, and these need to be handled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-14412) Restore automatic snapshot of system keyspace during upgrade

2019-04-16 Thread Sam Tunnicliffe (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16818943#comment-16818943
 ] 

Sam Tunnicliffe commented on CASSANDRA-14412:
-

Thanks [~tommy_s], I've added extra checks to the test, rebased and pushed:

||branch||CI||
|[14412-trunk|https://github.com/beobal/cassandra/tree/14412-trunk]|[circle|https://circleci.com/gh/beobal/workflows/cassandra/tree/cci%2F14412-trunk]|

> Restore automatic snapshot of system keyspace during upgrade
> 
>
> Key: CASSANDRA-14412
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14412
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Startup and Shutdown
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0
>
>
> Since 2.2, the installed version is compared with the version persisted in 
> system.local (if any) at startup. If these versions differ, the system 
> keyspace is snapshotted before proceeding in order to enable a rollback if 
> any other issue prevents startup from completing. Although the method to 
> perform this check & snapshot is still present in {{SystemKeyspace}}, its 
> only callsite was mistakenly removed from {{CassandraDaemon}} in 
> CASSANDRA-12716.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-14983) Local reads potentially blocking remote reads

2019-04-16 Thread Benedict (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16818948#comment-16818948
 ] 

Benedict commented on CASSANDRA-14983:
--

So, thinking about it a bit more.  It's possible this could actually be seen as 
a feature for speculative retry.  Speculative retry is invoked by a really 
inadequate metric, and a slow local query might much more strongly correlate 
with a legitimately slow query than anything else, and prevent unnecessary 
speculation.

I actually wonder if it anyway makes more sense to use the local query 
evaluation time when deciding if we should speculate, only doing so after some 
delta has elapsed over the local query time.

Unfortunately I cannot remember what, if any, consideration was given to this 
when the work initially landed.

> Local reads potentially blocking remote reads
> -
>
> Key: CASSANDRA-14983
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14983
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
>Priority: Low
> Attachments: graph_local_read.html, graph_local_read_trunk.html, 
> local_read_trace.log
>
>
> Since CASSANDRA-4718 there is a fast path allowing local requests to continue 
> to [work in the same 
> thread|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/reads/AbstractReadExecutor.java#L157]
>  rather than being sent over to the read stage.
> Based on the comment
> {code:java}
> // We delay the local (potentially blocking) read till the end to avoid 
> stalling remote requests.
> {code}
> it seems like this should be performed last in the chain to avoid blocking 
> remote requests but that does not seem to be the case when the local request 
> is a data request. The digest request(s) are sent after the data requests are 
> sent (and now the transient replica requests as well). When the fast path is 
> used for local data/transient data requests this will block the next type of 
> request from being sent away until the local read is finished and add 
> additional latency to the request.
> In addition to this it seems like local requests are *always* data requests 
> (might not be a problem), but the log message can say either ["digest" or 
> "data"|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/reads/AbstractReadExecutor.java#L156]
>  as the type of request.
> I have tried to run performance measurements to see the impact of this in 3.0 
> (by moving local requests to the end of ARE#executeAsync()) but I haven't 
> seen any big difference yet. I'll continue to run some more tests to see if I 
> can find a use case affected by this.
> Attaching a trace (3.0) where this happens. Reproduction:
>  # Create a three node CCM cluster
>  # Provision data with stress (rf=3)
>  # In parallel:
>  ## Start stress read run
>  ## Run multiple manual read queries in cqlsh with tracing on and 
> local_quorum (as this does not always happen)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-15085) Cassandra query raise error " max() arg is an empty sequence"

2019-04-16 Thread dong tan (JIRA)
dong tan created CASSANDRA-15085:


 Summary: Cassandra query raise error " max() arg is an empty 
sequence"
 Key: CASSANDRA-15085
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15085
 Project: Cassandra
  Issue Type: Bug
Reporter: dong tan


select * from rules where type='role' and id='test'' and 9437=9437--'; 

when i execute this query it raise error "max() arg is an empty sequence"; 
version 2.2.5

but verion 2.2.5-2 can successfull.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-15086) Illegal column names make legacy sstables unreadable in 3.0/3.x

2019-04-16 Thread Sam Tunnicliffe (JIRA)
Sam Tunnicliffe created CASSANDRA-15086:
---

 Summary: Illegal column names make legacy sstables unreadable in 
3.0/3.x
 Key: CASSANDRA-15086
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15086
 Project: Cassandra
  Issue Type: Bug
  Components: Local/SSTable
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe


CASSANDRA-10608 adds extra validation when decoding a bytebuffer representing a 
legacy cellname. If the table is not COMPACT and the column name component of 
the cellname refers to a primary key column, an IllegalArgumentException is 
thrown. It looks like the original intent of 10608 was to prevent Thrift writes 
from inserting these invalid cells, but the same code path is exercised on the 
read path. The problem is that this kind of cells may exist in pre-3.0 
sstables, either due to Thrift writes or through side loading of externally 
generated SSTables. Following an upgrade to 3.0, these partitions become 
unreadable, breaking both the read and compaction paths (and so also 
upgradesstables). Scrub in 2.1 does not help here as it blindly reproduces the 
invalid cells.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15086) Illegal column names make legacy sstables unreadable in 3.0/3.x

2019-04-16 Thread Sam Tunnicliffe (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-15086:

 Severity: Normal
   Complexity: Normal
Discovered By: Adhoc Test
 Bug Category: Parent values: Availability(12983)Level 1 values: Response 
Crash(12991)
   Status: Open  (was: Triage Needed)

> Illegal column names make legacy sstables unreadable in 3.0/3.x
> ---
>
> Key: CASSANDRA-15086
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15086
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/SSTable
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
>
> CASSANDRA-10608 adds extra validation when decoding a bytebuffer representing 
> a legacy cellname. If the table is not COMPACT and the column name component 
> of the cellname refers to a primary key column, an IllegalArgumentException 
> is thrown. It looks like the original intent of 10608 was to prevent Thrift 
> writes from inserting these invalid cells, but the same code path is 
> exercised on the read path. The problem is that this kind of cells may exist 
> in pre-3.0 sstables, either due to Thrift writes or through side loading of 
> externally generated SSTables. Following an upgrade to 3.0, these partitions 
> become unreadable, breaking both the read and compaction paths (and so also 
> upgradesstables). Scrub in 2.1 does not help here as it blindly reproduces 
> the invalid cells.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15086) Illegal column names make legacy sstables unreadable in 3.0/3.x

2019-04-16 Thread Sam Tunnicliffe (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-15086:

Fix Version/s: 3.11.x
   3.0.x

> Illegal column names make legacy sstables unreadable in 3.0/3.x
> ---
>
> Key: CASSANDRA-15086
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15086
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/SSTable
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> CASSANDRA-10608 adds extra validation when decoding a bytebuffer representing 
> a legacy cellname. If the table is not COMPACT and the column name component 
> of the cellname refers to a primary key column, an IllegalArgumentException 
> is thrown. It looks like the original intent of 10608 was to prevent Thrift 
> writes from inserting these invalid cells, but the same code path is 
> exercised on the read path. The problem is that this kind of cells may exist 
> in pre-3.0 sstables, either due to Thrift writes or through side loading of 
> externally generated SSTables. Following an upgrade to 3.0, these partitions 
> become unreadable, breaking both the read and compaction paths (and so also 
> upgradesstables). Scrub in 2.1 does not help here as it blindly reproduces 
> the invalid cells.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15086) Illegal column names make legacy sstables unreadable in 3.0/3.x

2019-04-16 Thread Sam Tunnicliffe (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-15086:

Test and Documentation Plan: Add additional test to LegacySSTableTest. Run 
upgrade tests and compare against baseline 3.0/3.11 runs
 Status: Patch Available  (was: In Progress)

This changes the behaviour from throwing IllegalArgumentException to throw a 
new IllegalLegacyColumnException when such a cell is encountered which allows 
the exception handling to be dependent on the context. On the read path, 
including compaction reads of SSTables, we can safely skip the illegal atoms, 
which is equivalent to the pre-3.0 behaviour. In the thrift layer, we can 
forcefully reject any writes or reads with filters or predicates containing 
illegal cells.

The bulk of the patch is mechanical changes to extend either method signatures 
or catch blocks to accommodate the new exception. 

||branch||CI||
|[15086-3.0|https://github.com/beobal/cassandra/tree/15086-3.0]|[circle|https://circleci.com/gh/beobal/workflows/cassandra/tree/cci%2F15086-3.0]|
|[15086-3.11|https://github.com/beobal/cassandra/tree/15086-3.11]|[circle|https://circleci.com/gh/beobal/workflows/cassandra/tree/cci%2F15086-3.11]|

> Illegal column names make legacy sstables unreadable in 3.0/3.x
> ---
>
> Key: CASSANDRA-15086
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15086
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/SSTable
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> CASSANDRA-10608 adds extra validation when decoding a bytebuffer representing 
> a legacy cellname. If the table is not COMPACT and the column name component 
> of the cellname refers to a primary key column, an IllegalArgumentException 
> is thrown. It looks like the original intent of 10608 was to prevent Thrift 
> writes from inserting these invalid cells, but the same code path is 
> exercised on the read path. The problem is that this kind of cells may exist 
> in pre-3.0 sstables, either due to Thrift writes or through side loading of 
> externally generated SSTables. Following an upgrade to 3.0, these partitions 
> become unreadable, breaking both the read and compaction paths (and so also 
> upgradesstables). Scrub in 2.1 does not help here as it blindly reproduces 
> the invalid cells.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Assigned] (CASSANDRA-15050) JDK7 compatibility broken in cassandra-2.2 by CASSANDRA-14821

2019-04-16 Thread Ted Petersson (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Petersson reassigned CASSANDRA-15050:
-

Assignee: Ted Petersson

> JDK7 compatibility broken in cassandra-2.2 by CASSANDRA-14821
> -
>
> Key: CASSANDRA-15050
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15050
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Shuler
>Assignee: Ted Petersson
>Priority: Normal
> Fix For: 2.2.x
>
>
> CASSANDRA-14821 introduced some errors on JDK7. The build fails as follows:
> {noformat}
> (cassandra-2.2)mshuler@mana:~/git/cassandra$ java -version
> java version "1.7.0_80"
> Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
> Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)
> (cassandra-2.2)mshuler@mana:~/git/cassandra$ ant jar
> Buildfile: /home/mshuler/git/cassandra/build.xml
> init:
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/classes/main
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/classes/thrift
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/test/lib
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/test/classes
>    [mkdir] Created dir: /home/mshuler/git/cassandra/src/gen-java
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/lib
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/jacoco
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/jacoco/partials
> maven-ant-tasks-localrepo:
> [copy] Copying 1 file to /home/mshuler/git/cassandra/build
> maven-ant-tasks-download:
> maven-ant-tasks-init:
> maven-declare-dependencies:
> maven-ant-tasks-retrieve-build:
> [artifact:dependencies] Building ant file: 
> /home/mshuler/git/cassandra/build/build-dependencies.xml
> [artifact:dependencies] Building ant file: 
> /home/mshuler/git/cassandra/build/build-dependencies-sources.xml
> [copy] Copying 65 files to /home/mshuler/git/cassandra/build/lib/jars
> [copy] Copying 41 files to /home/mshuler/git/cassandra/build/lib/sources
> [copy] Copying 25 files to /home/mshuler/git/cassandra/build/lib/jars
>    [unzip] Expanding: 
> /home/mshuler/git/cassandra/build/lib/jars/org.jacoco.agent-0.7.5.201505241946.jar
>  into /home/mshu
> ler/git/cassandra/build/lib/jars
> check-gen-cql3-grammar:
> gen-cql3-grammar:
> [echo] Building Grammar 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/cql3/Cql.g  ...
> generate-cql-html:
> build-project:
> [echo] apache-cassandra: /home/mshuler/git/cassandra/build.xml
>    [javac] Compiling 45 source files to 
> /home/mshuler/git/cassandra/build/classes/thrift
>    [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.Benchmark
> Processor' less than -source '1.7'
>    [javac] Note: 
> /home/mshuler/git/cassandra/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
>  uses o
> r overrides a deprecated API.
>    [javac] Note: Recompile with -Xlint:deprecation for details.
>    [javac] Note: Some input files use unchecked or unsafe operations.
>    [javac] Note: Recompile with -Xlint:unchecked for details.
>    [javac] 1 warning
>    [javac] Compiling 1171 source files to 
> /home/mshuler/git/cassandra/build/classes/main
>    [javac] Note: Processing compiler hints annotations
>    [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.Benchmark
> Processor' less than -source '1.7'
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:22:
>  error: package java.uti
> l.function does not exist
>    [javac] import java.util.function.Consumer;
>    [javac]  ^
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:203:
>  error: cannot find sym
> bol
>    [javac] private Consumer handler;
>    [javac] ^
>    [javac]   symbol:   class Consumer
>    [javac]   location: class OnException
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:204:
>  error: cannot find sym
> bol
>    [javac] OnException(Consumer handler)
>    [javac] ^
>    [javac]   symbol:   class Consumer
>    [javac]   location: class OnException
>    [javac] Note: Processing compiler hints annotations
>    [javac] Note: Writing compiler command file at META-INF/hotspot_compiler
>    [javac] Note: Done processing compiler hints annotations
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:22:
>  error: package java.uti
> l.function does not exist
>    [javac] import java.util.function.Consumer;
>    [javac]  ^
>

[jira] [Updated] (CASSANDRA-15050) JDK7 compatibility broken in cassandra-2.2 by CASSANDRA-14821

2019-04-16 Thread Ted Petersson (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Petersson updated CASSANDRA-15050:
--
Attachment: 15050-2.2.txt

> JDK7 compatibility broken in cassandra-2.2 by CASSANDRA-14821
> -
>
> Key: CASSANDRA-15050
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15050
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Shuler
>Assignee: Ted Petersson
>Priority: Normal
> Fix For: 2.2.x
>
> Attachments: 15050-2.2.txt
>
>
> CASSANDRA-14821 introduced some errors on JDK7. The build fails as follows:
> {noformat}
> (cassandra-2.2)mshuler@mana:~/git/cassandra$ java -version
> java version "1.7.0_80"
> Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
> Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)
> (cassandra-2.2)mshuler@mana:~/git/cassandra$ ant jar
> Buildfile: /home/mshuler/git/cassandra/build.xml
> init:
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/classes/main
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/classes/thrift
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/test/lib
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/test/classes
>    [mkdir] Created dir: /home/mshuler/git/cassandra/src/gen-java
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/lib
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/jacoco
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/jacoco/partials
> maven-ant-tasks-localrepo:
> [copy] Copying 1 file to /home/mshuler/git/cassandra/build
> maven-ant-tasks-download:
> maven-ant-tasks-init:
> maven-declare-dependencies:
> maven-ant-tasks-retrieve-build:
> [artifact:dependencies] Building ant file: 
> /home/mshuler/git/cassandra/build/build-dependencies.xml
> [artifact:dependencies] Building ant file: 
> /home/mshuler/git/cassandra/build/build-dependencies-sources.xml
> [copy] Copying 65 files to /home/mshuler/git/cassandra/build/lib/jars
> [copy] Copying 41 files to /home/mshuler/git/cassandra/build/lib/sources
> [copy] Copying 25 files to /home/mshuler/git/cassandra/build/lib/jars
>    [unzip] Expanding: 
> /home/mshuler/git/cassandra/build/lib/jars/org.jacoco.agent-0.7.5.201505241946.jar
>  into /home/mshu
> ler/git/cassandra/build/lib/jars
> check-gen-cql3-grammar:
> gen-cql3-grammar:
> [echo] Building Grammar 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/cql3/Cql.g  ...
> generate-cql-html:
> build-project:
> [echo] apache-cassandra: /home/mshuler/git/cassandra/build.xml
>    [javac] Compiling 45 source files to 
> /home/mshuler/git/cassandra/build/classes/thrift
>    [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.Benchmark
> Processor' less than -source '1.7'
>    [javac] Note: 
> /home/mshuler/git/cassandra/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
>  uses o
> r overrides a deprecated API.
>    [javac] Note: Recompile with -Xlint:deprecation for details.
>    [javac] Note: Some input files use unchecked or unsafe operations.
>    [javac] Note: Recompile with -Xlint:unchecked for details.
>    [javac] 1 warning
>    [javac] Compiling 1171 source files to 
> /home/mshuler/git/cassandra/build/classes/main
>    [javac] Note: Processing compiler hints annotations
>    [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.Benchmark
> Processor' less than -source '1.7'
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:22:
>  error: package java.uti
> l.function does not exist
>    [javac] import java.util.function.Consumer;
>    [javac]  ^
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:203:
>  error: cannot find sym
> bol
>    [javac] private Consumer handler;
>    [javac] ^
>    [javac]   symbol:   class Consumer
>    [javac]   location: class OnException
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:204:
>  error: cannot find sym
> bol
>    [javac] OnException(Consumer handler)
>    [javac] ^
>    [javac]   symbol:   class Consumer
>    [javac]   location: class OnException
>    [javac] Note: Processing compiler hints annotations
>    [javac] Note: Writing compiler command file at META-INF/hotspot_compiler
>    [javac] Note: Done processing compiler hints annotations
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:22:
>  error: package java.uti
> l.function does not exist
>    [javac] import java.util.function.Consumer;
>    [j

[jira] [Commented] (CASSANDRA-15050) JDK7 compatibility broken in cassandra-2.2 by CASSANDRA-14821

2019-04-16 Thread Ted Petersson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819042#comment-16819042
 ] 

Ted Petersson commented on CASSANDRA-15050:
---

Could not find any "Submit Patch" button to click so I just attached the patch 
file.

This is a simple patch that removes the usage of java.util.function.Consumer, 
which was introduced in Java 8.

One alternative was to introduce a new interface that was similar to 
Consumer... But I thought it was cleaner to just put the 1-line logic on the 
Enums instead.

> JDK7 compatibility broken in cassandra-2.2 by CASSANDRA-14821
> -
>
> Key: CASSANDRA-15050
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15050
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Michael Shuler
>Assignee: Ted Petersson
>Priority: Normal
> Fix For: 2.2.x
>
> Attachments: 15050-2.2.txt
>
>
> CASSANDRA-14821 introduced some errors on JDK7. The build fails as follows:
> {noformat}
> (cassandra-2.2)mshuler@mana:~/git/cassandra$ java -version
> java version "1.7.0_80"
> Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
> Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)
> (cassandra-2.2)mshuler@mana:~/git/cassandra$ ant jar
> Buildfile: /home/mshuler/git/cassandra/build.xml
> init:
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/classes/main
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/classes/thrift
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/test/lib
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/test/classes
>    [mkdir] Created dir: /home/mshuler/git/cassandra/src/gen-java
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/lib
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/jacoco
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/jacoco/partials
> maven-ant-tasks-localrepo:
> [copy] Copying 1 file to /home/mshuler/git/cassandra/build
> maven-ant-tasks-download:
> maven-ant-tasks-init:
> maven-declare-dependencies:
> maven-ant-tasks-retrieve-build:
> [artifact:dependencies] Building ant file: 
> /home/mshuler/git/cassandra/build/build-dependencies.xml
> [artifact:dependencies] Building ant file: 
> /home/mshuler/git/cassandra/build/build-dependencies-sources.xml
> [copy] Copying 65 files to /home/mshuler/git/cassandra/build/lib/jars
> [copy] Copying 41 files to /home/mshuler/git/cassandra/build/lib/sources
> [copy] Copying 25 files to /home/mshuler/git/cassandra/build/lib/jars
>    [unzip] Expanding: 
> /home/mshuler/git/cassandra/build/lib/jars/org.jacoco.agent-0.7.5.201505241946.jar
>  into /home/mshu
> ler/git/cassandra/build/lib/jars
> check-gen-cql3-grammar:
> gen-cql3-grammar:
> [echo] Building Grammar 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/cql3/Cql.g  ...
> generate-cql-html:
> build-project:
> [echo] apache-cassandra: /home/mshuler/git/cassandra/build.xml
>    [javac] Compiling 45 source files to 
> /home/mshuler/git/cassandra/build/classes/thrift
>    [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.Benchmark
> Processor' less than -source '1.7'
>    [javac] Note: 
> /home/mshuler/git/cassandra/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
>  uses o
> r overrides a deprecated API.
>    [javac] Note: Recompile with -Xlint:deprecation for details.
>    [javac] Note: Some input files use unchecked or unsafe operations.
>    [javac] Note: Recompile with -Xlint:unchecked for details.
>    [javac] 1 warning
>    [javac] Compiling 1171 source files to 
> /home/mshuler/git/cassandra/build/classes/main
>    [javac] Note: Processing compiler hints annotations
>    [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.Benchmark
> Processor' less than -source '1.7'
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:22:
>  error: package java.uti
> l.function does not exist
>    [javac] import java.util.function.Consumer;
>    [javac]  ^
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:203:
>  error: cannot find sym
> bol
>    [javac] private Consumer handler;
>    [javac] ^
>    [javac]   symbol:   class Consumer
>    [javac]   location: class OnException
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:204:
>  error: cannot find sym
> bol
>    [javac] OnException(Consumer handler)
>    [javac] ^
>    [javac]   symbol:   class Consumer
>    [javac]   location: class OnException
>    [javac] Note: P

[jira] [Updated] (CASSANDRA-15050) JDK7 compatibility broken in cassandra-2.2 by CASSANDRA-14821

2019-04-16 Thread Michael Shuler (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler updated CASSANDRA-15050:
---
   Complexity: Normal
Discovered By: User Report
 Bug Category: Parent values: Correctness(12982)
  Component/s: Build
   Status: Open  (was: Triage Needed)

> JDK7 compatibility broken in cassandra-2.2 by CASSANDRA-14821
> -
>
> Key: CASSANDRA-15050
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15050
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: Michael Shuler
>Assignee: Ted Petersson
>Priority: Normal
> Fix For: 2.2.x
>
> Attachments: 15050-2.2.txt
>
>
> CASSANDRA-14821 introduced some errors on JDK7. The build fails as follows:
> {noformat}
> (cassandra-2.2)mshuler@mana:~/git/cassandra$ java -version
> java version "1.7.0_80"
> Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
> Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)
> (cassandra-2.2)mshuler@mana:~/git/cassandra$ ant jar
> Buildfile: /home/mshuler/git/cassandra/build.xml
> init:
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/classes/main
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/classes/thrift
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/test/lib
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/test/classes
>    [mkdir] Created dir: /home/mshuler/git/cassandra/src/gen-java
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/lib
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/jacoco
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/jacoco/partials
> maven-ant-tasks-localrepo:
> [copy] Copying 1 file to /home/mshuler/git/cassandra/build
> maven-ant-tasks-download:
> maven-ant-tasks-init:
> maven-declare-dependencies:
> maven-ant-tasks-retrieve-build:
> [artifact:dependencies] Building ant file: 
> /home/mshuler/git/cassandra/build/build-dependencies.xml
> [artifact:dependencies] Building ant file: 
> /home/mshuler/git/cassandra/build/build-dependencies-sources.xml
> [copy] Copying 65 files to /home/mshuler/git/cassandra/build/lib/jars
> [copy] Copying 41 files to /home/mshuler/git/cassandra/build/lib/sources
> [copy] Copying 25 files to /home/mshuler/git/cassandra/build/lib/jars
>    [unzip] Expanding: 
> /home/mshuler/git/cassandra/build/lib/jars/org.jacoco.agent-0.7.5.201505241946.jar
>  into /home/mshu
> ler/git/cassandra/build/lib/jars
> check-gen-cql3-grammar:
> gen-cql3-grammar:
> [echo] Building Grammar 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/cql3/Cql.g  ...
> generate-cql-html:
> build-project:
> [echo] apache-cassandra: /home/mshuler/git/cassandra/build.xml
>    [javac] Compiling 45 source files to 
> /home/mshuler/git/cassandra/build/classes/thrift
>    [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.Benchmark
> Processor' less than -source '1.7'
>    [javac] Note: 
> /home/mshuler/git/cassandra/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
>  uses o
> r overrides a deprecated API.
>    [javac] Note: Recompile with -Xlint:deprecation for details.
>    [javac] Note: Some input files use unchecked or unsafe operations.
>    [javac] Note: Recompile with -Xlint:unchecked for details.
>    [javac] 1 warning
>    [javac] Compiling 1171 source files to 
> /home/mshuler/git/cassandra/build/classes/main
>    [javac] Note: Processing compiler hints annotations
>    [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.Benchmark
> Processor' less than -source '1.7'
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:22:
>  error: package java.uti
> l.function does not exist
>    [javac] import java.util.function.Consumer;
>    [javac]  ^
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:203:
>  error: cannot find sym
> bol
>    [javac] private Consumer handler;
>    [javac] ^
>    [javac]   symbol:   class Consumer
>    [javac]   location: class OnException
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:204:
>  error: cannot find sym
> bol
>    [javac] OnException(Consumer handler)
>    [javac] ^
>    [javac]   symbol:   class Consumer
>    [javac]   location: class OnException
>    [javac] Note: Processing compiler hints annotations
>    [javac] Note: Writing compiler command file at META-INF/hotspot_compiler
>    [javac] Note: Done processing compiler hints annotations
>    [javac] 
> /home/mshuler/g

[jira] [Updated] (CASSANDRA-15050) JDK7 compatibility broken in cassandra-2.2 by CASSANDRA-14821

2019-04-16 Thread Michael Shuler (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler updated CASSANDRA-15050:
---
Test and Documentation Plan: .
 Status: Patch Available  (was: Open)

> JDK7 compatibility broken in cassandra-2.2 by CASSANDRA-14821
> -
>
> Key: CASSANDRA-15050
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15050
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: Michael Shuler
>Assignee: Ted Petersson
>Priority: Normal
> Fix For: 2.2.x
>
> Attachments: 15050-2.2.txt
>
>
> CASSANDRA-14821 introduced some errors on JDK7. The build fails as follows:
> {noformat}
> (cassandra-2.2)mshuler@mana:~/git/cassandra$ java -version
> java version "1.7.0_80"
> Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
> Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)
> (cassandra-2.2)mshuler@mana:~/git/cassandra$ ant jar
> Buildfile: /home/mshuler/git/cassandra/build.xml
> init:
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/classes/main
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/classes/thrift
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/test/lib
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/test/classes
>    [mkdir] Created dir: /home/mshuler/git/cassandra/src/gen-java
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/lib
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/jacoco
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/jacoco/partials
> maven-ant-tasks-localrepo:
> [copy] Copying 1 file to /home/mshuler/git/cassandra/build
> maven-ant-tasks-download:
> maven-ant-tasks-init:
> maven-declare-dependencies:
> maven-ant-tasks-retrieve-build:
> [artifact:dependencies] Building ant file: 
> /home/mshuler/git/cassandra/build/build-dependencies.xml
> [artifact:dependencies] Building ant file: 
> /home/mshuler/git/cassandra/build/build-dependencies-sources.xml
> [copy] Copying 65 files to /home/mshuler/git/cassandra/build/lib/jars
> [copy] Copying 41 files to /home/mshuler/git/cassandra/build/lib/sources
> [copy] Copying 25 files to /home/mshuler/git/cassandra/build/lib/jars
>    [unzip] Expanding: 
> /home/mshuler/git/cassandra/build/lib/jars/org.jacoco.agent-0.7.5.201505241946.jar
>  into /home/mshu
> ler/git/cassandra/build/lib/jars
> check-gen-cql3-grammar:
> gen-cql3-grammar:
> [echo] Building Grammar 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/cql3/Cql.g  ...
> generate-cql-html:
> build-project:
> [echo] apache-cassandra: /home/mshuler/git/cassandra/build.xml
>    [javac] Compiling 45 source files to 
> /home/mshuler/git/cassandra/build/classes/thrift
>    [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.Benchmark
> Processor' less than -source '1.7'
>    [javac] Note: 
> /home/mshuler/git/cassandra/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
>  uses o
> r overrides a deprecated API.
>    [javac] Note: Recompile with -Xlint:deprecation for details.
>    [javac] Note: Some input files use unchecked or unsafe operations.
>    [javac] Note: Recompile with -Xlint:unchecked for details.
>    [javac] 1 warning
>    [javac] Compiling 1171 source files to 
> /home/mshuler/git/cassandra/build/classes/main
>    [javac] Note: Processing compiler hints annotations
>    [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.Benchmark
> Processor' less than -source '1.7'
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:22:
>  error: package java.uti
> l.function does not exist
>    [javac] import java.util.function.Consumer;
>    [javac]  ^
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:203:
>  error: cannot find sym
> bol
>    [javac] private Consumer handler;
>    [javac] ^
>    [javac]   symbol:   class Consumer
>    [javac]   location: class OnException
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:204:
>  error: cannot find sym
> bol
>    [javac] OnException(Consumer handler)
>    [javac] ^
>    [javac]   symbol:   class Consumer
>    [javac]   location: class OnException
>    [javac] Note: Processing compiler hints annotations
>    [javac] Note: Writing compiler command file at META-INF/hotspot_compiler
>    [javac] Note: Done processing compiler hints annotations
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:22:
>  error: pac

[jira] [Commented] (CASSANDRA-15050) JDK7 compatibility broken in cassandra-2.2 by CASSANDRA-14821

2019-04-16 Thread Michael Shuler (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819165#comment-16819165
 ] 

Michael Shuler commented on CASSANDRA-15050:


Bug needs to be "opened", then can be set to "patch available".

> JDK7 compatibility broken in cassandra-2.2 by CASSANDRA-14821
> -
>
> Key: CASSANDRA-15050
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15050
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: Michael Shuler
>Assignee: Ted Petersson
>Priority: Normal
> Fix For: 2.2.x
>
> Attachments: 15050-2.2.txt
>
>
> CASSANDRA-14821 introduced some errors on JDK7. The build fails as follows:
> {noformat}
> (cassandra-2.2)mshuler@mana:~/git/cassandra$ java -version
> java version "1.7.0_80"
> Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
> Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)
> (cassandra-2.2)mshuler@mana:~/git/cassandra$ ant jar
> Buildfile: /home/mshuler/git/cassandra/build.xml
> init:
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/classes/main
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/classes/thrift
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/test/lib
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/test/classes
>    [mkdir] Created dir: /home/mshuler/git/cassandra/src/gen-java
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/lib
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/jacoco
>    [mkdir] Created dir: /home/mshuler/git/cassandra/build/jacoco/partials
> maven-ant-tasks-localrepo:
> [copy] Copying 1 file to /home/mshuler/git/cassandra/build
> maven-ant-tasks-download:
> maven-ant-tasks-init:
> maven-declare-dependencies:
> maven-ant-tasks-retrieve-build:
> [artifact:dependencies] Building ant file: 
> /home/mshuler/git/cassandra/build/build-dependencies.xml
> [artifact:dependencies] Building ant file: 
> /home/mshuler/git/cassandra/build/build-dependencies-sources.xml
> [copy] Copying 65 files to /home/mshuler/git/cassandra/build/lib/jars
> [copy] Copying 41 files to /home/mshuler/git/cassandra/build/lib/sources
> [copy] Copying 25 files to /home/mshuler/git/cassandra/build/lib/jars
>    [unzip] Expanding: 
> /home/mshuler/git/cassandra/build/lib/jars/org.jacoco.agent-0.7.5.201505241946.jar
>  into /home/mshu
> ler/git/cassandra/build/lib/jars
> check-gen-cql3-grammar:
> gen-cql3-grammar:
> [echo] Building Grammar 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/cql3/Cql.g  ...
> generate-cql-html:
> build-project:
> [echo] apache-cassandra: /home/mshuler/git/cassandra/build.xml
>    [javac] Compiling 45 source files to 
> /home/mshuler/git/cassandra/build/classes/thrift
>    [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.Benchmark
> Processor' less than -source '1.7'
>    [javac] Note: 
> /home/mshuler/git/cassandra/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java
>  uses o
> r overrides a deprecated API.
>    [javac] Note: Recompile with -Xlint:deprecation for details.
>    [javac] Note: Some input files use unchecked or unsafe operations.
>    [javac] Note: Recompile with -Xlint:unchecked for details.
>    [javac] 1 warning
>    [javac] Compiling 1171 source files to 
> /home/mshuler/git/cassandra/build/classes/main
>    [javac] Note: Processing compiler hints annotations
>    [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.Benchmark
> Processor' less than -source '1.7'
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:22:
>  error: package java.uti
> l.function does not exist
>    [javac] import java.util.function.Consumer;
>    [javac]  ^
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:203:
>  error: cannot find sym
> bol
>    [javac] private Consumer handler;
>    [javac] ^
>    [javac]   symbol:   class Consumer
>    [javac]   location: class OnException
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapper.java:204:
>  error: cannot find sym
> bol
>    [javac] OnException(Consumer handler)
>    [javac] ^
>    [javac]   symbol:   class Consumer
>    [javac]   location: class OnException
>    [javac] Note: Processing compiler hints annotations
>    [javac] Note: Writing compiler command file at META-INF/hotspot_compiler
>    [javac] Note: Done processing compiler hints annotations
>    [javac] 
> /home/mshuler/git/cassandra/src/java/org/apache/cassandra/utils/MBeanWrapp

[jira] [Updated] (CASSANDRA-14412) Restore automatic snapshot of system keyspace during upgrade

2019-04-16 Thread Aleksey Yeschenko (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-14412:
--
Status: Review In Progress  (was: Patch Available)

> Restore automatic snapshot of system keyspace during upgrade
> 
>
> Key: CASSANDRA-14412
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14412
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Startup and Shutdown
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0
>
>
> Since 2.2, the installed version is compared with the version persisted in 
> system.local (if any) at startup. If these versions differ, the system 
> keyspace is snapshotted before proceeding in order to enable a rollback if 
> any other issue prevents startup from completing. Although the method to 
> perform this check & snapshot is still present in {{SystemKeyspace}}, its 
> only callsite was mistakenly removed from {{CassandraDaemon}} in 
> CASSANDRA-12716.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14412) Restore automatic snapshot of system keyspace during upgrade

2019-04-16 Thread Aleksey Yeschenko (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-14412:
--
Reviewers: Aleksey Yeschenko, Tommy Stendahl

> Restore automatic snapshot of system keyspace during upgrade
> 
>
> Key: CASSANDRA-14412
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14412
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Startup and Shutdown
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0
>
>
> Since 2.2, the installed version is compared with the version persisted in 
> system.local (if any) at startup. If these versions differ, the system 
> keyspace is snapshotted before proceeding in order to enable a rollback if 
> any other issue prevents startup from completing. Although the method to 
> perform this check & snapshot is still present in {{SystemKeyspace}}, its 
> only callsite was mistakenly removed from {{CassandraDaemon}} in 
> CASSANDRA-12716.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15013) Message Flusher queue can grow unbounded, potentially running JVM out of memory

2019-04-16 Thread Sumanth Pasupuleti (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819299#comment-16819299
 ] 

Sumanth Pasupuleti commented on CASSANDRA-15013:


[~benedict] here is the PR that includes per-endpoint limit 
https://github.com/apache/cassandra/pull/311/commits/a03cd0550118e3c1dc98694c0dc0ed84824853d1
This patch has a drawback of a slow leak (we never remove an endpoint from the 
endpoint inflight map) - looking for advice.

> Message Flusher queue can grow unbounded, potentially running JVM out of 
> memory
> ---
>
> Key: CASSANDRA-15013
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15013
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Client
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0, 3.0.x, 3.11.x
>
> Attachments: BlockedEpollEventLoopFromHeapDump.png, 
> BlockedEpollEventLoopFromThreadDump.png, RequestExecutorQueueFull.png, heap 
> dump showing each ImmediateFlusher taking upto 600MB.png
>
>
> This is a follow-up ticket out of CASSANDRA-14855, to make the Flusher queue 
> bounded, since, in the current state, items get added to the queue without 
> any checks on queue size, nor with any checks on netty outbound buffer to 
> check the isWritable state.
> We are seeing this issue hit our production 3.0 clusters quite often.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15013) Message Flusher queue can grow unbounded, potentially running JVM out of memory

2019-04-16 Thread Sumanth Pasupuleti (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumanth Pasupuleti updated CASSANDRA-15013:
---
Status: In Progress  (was: Changes Suggested)

> Message Flusher queue can grow unbounded, potentially running JVM out of 
> memory
> ---
>
> Key: CASSANDRA-15013
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15013
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Client
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0, 3.0.x, 3.11.x
>
> Attachments: BlockedEpollEventLoopFromHeapDump.png, 
> BlockedEpollEventLoopFromThreadDump.png, RequestExecutorQueueFull.png, heap 
> dump showing each ImmediateFlusher taking upto 600MB.png
>
>
> This is a follow-up ticket out of CASSANDRA-14855, to make the Flusher queue 
> bounded, since, in the current state, items get added to the queue without 
> any checks on queue size, nor with any checks on netty outbound buffer to 
> check the isWritable state.
> We are seeing this issue hit our production 3.0 clusters quite often.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15013) Message Flusher queue can grow unbounded, potentially running JVM out of memory

2019-04-16 Thread Sumanth Pasupuleti (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumanth Pasupuleti updated CASSANDRA-15013:
---
Test and Documentation Plan: 
Passing UTs: https://circleci.com/gh/sumanth-pasupuleti/cassandra/288
Passing jvm dtests: https://circleci.com/gh/sumanth-pasupuleti/cassandra/287
DTests with 2 failures (seems unrelated): 
https://circleci.com/gh/sumanth-pasupuleti/cassandra/290#tests/containers/81, 
https://circleci.com/gh/sumanth-pasupuleti/cassandra/289#tests/containers/66

  was:
Pull Request: https://github.com/apache/cassandra/pull/308
Passing UTs: https://circleci.com/gh/sumanth-pasupuleti/cassandra/261
Passing JVM DTests: https://circleci.com/gh/sumanth-pasupuleti/cassandra/260
Passing DTests with vnodes: 
https://circleci.com/gh/sumanth-pasupuleti/cassandra/262
DTests without vnodes with two failures: 
https://circleci.com/gh/sumanth-pasupuleti/cassandra/263 (looking into the two 
failures)

 Status: Patch Available  (was: In Progress)

> Message Flusher queue can grow unbounded, potentially running JVM out of 
> memory
> ---
>
> Key: CASSANDRA-15013
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15013
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Client
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0, 3.0.x, 3.11.x
>
> Attachments: BlockedEpollEventLoopFromHeapDump.png, 
> BlockedEpollEventLoopFromThreadDump.png, RequestExecutorQueueFull.png, heap 
> dump showing each ImmediateFlusher taking upto 600MB.png
>
>
> This is a follow-up ticket out of CASSANDRA-14855, to make the Flusher queue 
> bounded, since, in the current state, items get added to the queue without 
> any checks on queue size, nor with any checks on netty outbound buffer to 
> check the isWritable state.
> We are seeing this issue hit our production 3.0 clusters quite often.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15013) Message Flusher queue can grow unbounded, potentially running JVM out of memory

2019-04-16 Thread Benedict (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819331#comment-16819331
 ] 

Benedict commented on CASSANDRA-15013:
--

[~sumanth.pasupuleti] thanks for the update.  As to addressing the slow leak, 
there are a number of similar approaches, but I would propose the following:

* Instead of a {{Map}} in {{Dispatcher}} have a 
{{Map}} in {{Server.Initializer}}
* Add a reference count to the {{Dispatcher}} as well as an atomic 
{{bytesInFlight}}
* In {{Server.Initializer.initChannel}}, lookup the socket InetAddress:
*# If there is no {{Dispatcher}} create one 
*# If the {{Dispatcher}} cannot increment its reference count, remove it from 
the map and goto (1)
*# Otherwise we've taken ownership of the {{Dispatcher}} and can use it
* Then, in the {{Dispatcher}}, override {{channelInactive}} to decrement our 
reference count and remove ourselves from the map if we've been freed

This also marginally reduces the per-message cost of enforcing these 
constraints.  WDYT?

> Message Flusher queue can grow unbounded, potentially running JVM out of 
> memory
> ---
>
> Key: CASSANDRA-15013
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15013
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Client
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0, 3.0.x, 3.11.x
>
> Attachments: BlockedEpollEventLoopFromHeapDump.png, 
> BlockedEpollEventLoopFromThreadDump.png, RequestExecutorQueueFull.png, heap 
> dump showing each ImmediateFlusher taking upto 600MB.png
>
>
> This is a follow-up ticket out of CASSANDRA-14855, to make the Flusher queue 
> bounded, since, in the current state, items get added to the queue without 
> any checks on queue size, nor with any checks on netty outbound buffer to 
> check the isWritable state.
> We are seeing this issue hit our production 3.0 clusters quite often.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-15013) Message Flusher queue can grow unbounded, potentially running JVM out of memory

2019-04-16 Thread Benedict (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819331#comment-16819331
 ] 

Benedict edited comment on CASSANDRA-15013 at 4/16/19 5:44 PM:
---

[~sumanth.pasupuleti] thanks for the update.  As to addressing the slow leak, I 
would propose the following:

* Instead of a {{Map}} in {{Dispatcher}} have a 
{{Map}} in {{Server.Initializer}}
* Add a reference count to the {{Dispatcher}} as well as an atomic 
{{bytesInFlight}}
* In {{Server.Initializer.initChannel}}, lookup the socket InetAddress:
*# If there is no {{Dispatcher}} create one 
*# If the {{Dispatcher}} cannot increment its reference count, remove it from 
the map and goto (1)
*# Otherwise we've taken ownership of the {{Dispatcher}} and can use it
* Then, in the {{Dispatcher}}, override {{channelInactive}} to decrement our 
reference count and remove ourselves from the map if we've been freed

This also marginally reduces the per-message cost of enforcing these 
constraints.  WDYT?


was (Author: benedict):
[~sumanth.pasupuleti] thanks for the update.  As to addressing the slow leak, 
there are a number of similar approaches, but I would propose the following:

* Instead of a {{Map}} in {{Dispatcher}} have a 
{{Map}} in {{Server.Initializer}}
* Add a reference count to the {{Dispatcher}} as well as an atomic 
{{bytesInFlight}}
* In {{Server.Initializer.initChannel}}, lookup the socket InetAddress:
*# If there is no {{Dispatcher}} create one 
*# If the {{Dispatcher}} cannot increment its reference count, remove it from 
the map and goto (1)
*# Otherwise we've taken ownership of the {{Dispatcher}} and can use it
* Then, in the {{Dispatcher}}, override {{channelInactive}} to decrement our 
reference count and remove ourselves from the map if we've been freed

This also marginally reduces the per-message cost of enforcing these 
constraints.  WDYT?

> Message Flusher queue can grow unbounded, potentially running JVM out of 
> memory
> ---
>
> Key: CASSANDRA-15013
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15013
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Client
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0, 3.0.x, 3.11.x
>
> Attachments: BlockedEpollEventLoopFromHeapDump.png, 
> BlockedEpollEventLoopFromThreadDump.png, RequestExecutorQueueFull.png, heap 
> dump showing each ImmediateFlusher taking upto 600MB.png
>
>
> This is a follow-up ticket out of CASSANDRA-14855, to make the Flusher queue 
> bounded, since, in the current state, items get added to the queue without 
> any checks on queue size, nor with any checks on netty outbound buffer to 
> check the isWritable state.
> We are seeing this issue hit our production 3.0 clusters quite often.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-15087) Find a place for the professional services page

2019-04-16 Thread Jon Haddad (JIRA)
Jon Haddad created CASSANDRA-15087:
--

 Summary: Find a place for the professional services page
 Key: CASSANDRA-15087
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15087
 Project: Cassandra
  Issue Type: Improvement
  Components: Legacy/Documentation and Website
Reporter: Jon Haddad


We've got a link to our old wiki on the Cassandra homepage for third party 
support.  We link to the old Wiki, which will eventually be sunset.  We should 
migrate this information to a page in the static site and link to it there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15083) Apache NetBeans "Run", "Debug", "Profile" IDE actions

2019-04-16 Thread mck (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mck updated CASSANDRA-15083:

Status: Triage Needed  (was: Awaiting Feedback)

The {{`JAVA8_HOME`}} issue I don't think we should try and workaround. We're 
trying not to re-invent the weheel, and instead to do everything from the 
command line. That means {{`JAVA8_HOME`}} is a requirement.

Will fix the non-existant javadoc folder issue.

> Apache NetBeans "Run", "Debug", "Profile" IDE actions
> -
>
> Key: CASSANDRA-15083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15083
> Project: Cassandra
>  Issue Type: Task
>  Components: Build
>Reporter: mck
>Assignee: mck
>Priority: Low
> Fix For: 4.x
>
> Attachments: nb-e1.png, nb-e2.png
>
>
> Add to the NetBeans project configuration the action to run Cassandra from 
> the IDE.
> This should be an ant task that executes the {{`bin/cassandra`}} script.
> The ant task should be written into {{ide/nbproject/nbjdk.xml}} (which should 
> probably be renamed to something as it no longer relates to nb's jdk).
> This follows on from the discussion in the original ticket for the NetBeans 
> project configuration in 
> https://issues.apache.org/jira/browse/CASSANDRA-15073?focusedCommentId=16815843&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16815843
> It should be easy to attach either the debugger or profiler to this then 
> running Cassandra.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15083) Apache NetBeans "Run", "Debug", "Profile" IDE actions

2019-04-16 Thread mck (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mck updated CASSANDRA-15083:

Change Category: Quality Assurance
 Status: Open  (was: Triage Needed)

> Apache NetBeans "Run", "Debug", "Profile" IDE actions
> -
>
> Key: CASSANDRA-15083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15083
> Project: Cassandra
>  Issue Type: Task
>  Components: Build
>Reporter: mck
>Assignee: mck
>Priority: Low
> Fix For: 4.x
>
> Attachments: nb-e1.png, nb-e2.png
>
>
> Add to the NetBeans project configuration the action to run Cassandra from 
> the IDE.
> This should be an ant task that executes the {{`bin/cassandra`}} script.
> The ant task should be written into {{ide/nbproject/nbjdk.xml}} (which should 
> probably be renamed to something as it no longer relates to nb's jdk).
> This follows on from the discussion in the original ticket for the NetBeans 
> project configuration in 
> https://issues.apache.org/jira/browse/CASSANDRA-15073?focusedCommentId=16815843&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16815843
> It should be easy to attach either the debugger or profiler to this then 
> running Cassandra.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-15083) Apache NetBeans "Run", "Debug", "Profile" IDE actions

2019-04-16 Thread mck (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819474#comment-16819474
 ] 

mck edited comment on CASSANDRA-15083 at 4/16/19 8:44 PM:
--

The {{`JAVA8_HOME`}} issue I don't think we should try and workaround. We're 
trying not to re-invent the wheel, and instead to do everything from the 
command line. That means {{`JAVA8_HOME`}} is a requirement.

Will fix the non-existant javadoc folder issue.


was (Author: michaelsembwever):
The {{`JAVA8_HOME`}} issue I don't think we should try and workaround. We're 
trying not to re-invent the weheel, and instead to do everything from the 
command line. That means {{`JAVA8_HOME`}} is a requirement.

Will fix the non-existant javadoc folder issue.

> Apache NetBeans "Run", "Debug", "Profile" IDE actions
> -
>
> Key: CASSANDRA-15083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15083
> Project: Cassandra
>  Issue Type: Task
>  Components: Build
>Reporter: mck
>Assignee: mck
>Priority: Low
> Fix For: 4.x
>
> Attachments: nb-e1.png, nb-e2.png
>
>
> Add to the NetBeans project configuration the action to run Cassandra from 
> the IDE.
> This should be an ant task that executes the {{`bin/cassandra`}} script.
> The ant task should be written into {{ide/nbproject/nbjdk.xml}} (which should 
> probably be renamed to something as it no longer relates to nb's jdk).
> This follows on from the discussion in the original ticket for the NetBeans 
> project configuration in 
> https://issues.apache.org/jira/browse/CASSANDRA-15073?focusedCommentId=16815843&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16815843
> It should be easy to attach either the debugger or profiler to this then 
> running Cassandra.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15083) Apache NetBeans "Run", "Debug", "Profile" IDE actions

2019-04-16 Thread mck (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819487#comment-16819487
 ] 

mck commented on CASSANDRA-15083:
-

Improvements added…
 * fail-fast error message for when JAVA8_HOME is not defined
 * ignore failure when javadoc or doc folder is missing (as it is possible to 
skip their generation)

> Apache NetBeans "Run", "Debug", "Profile" IDE actions
> -
>
> Key: CASSANDRA-15083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15083
> Project: Cassandra
>  Issue Type: Task
>  Components: Build
>Reporter: mck
>Assignee: mck
>Priority: Low
> Fix For: 4.x
>
> Attachments: nb-e1.png, nb-e2.png
>
>
> Add to the NetBeans project configuration the action to run Cassandra from 
> the IDE.
> This should be an ant task that executes the {{`bin/cassandra`}} script.
> The ant task should be written into {{ide/nbproject/nbjdk.xml}} (which should 
> probably be renamed to something as it no longer relates to nb's jdk).
> This follows on from the discussion in the original ticket for the NetBeans 
> project configuration in 
> https://issues.apache.org/jira/browse/CASSANDRA-15073?focusedCommentId=16815843&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16815843
> It should be easy to attach either the debugger or profiler to this then 
> running Cassandra.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-12735) org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out

2019-04-16 Thread Shannon Carey (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-12735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819544#comment-16819544
 ] 

Shannon Carey commented on CASSANDRA-12735:
---

Adjusting the cache validity period and refresh rate only helps so much. A 
configuration should be exposed to allow control over the timeout duration for 
this. If the timeout is too small, then it repeatedly tries reading the data, 
making it even less likely to succeed.

> org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out
> -
>
> Key: CASSANDRA-12735
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12735
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Core, Local/Config
> Environment: Python 2.7.11, Datastax Cassandra 3.7.0  
>Reporter: Rajesh Radhakrishnan
>Priority: Normal
> Fix For: 3.11.x
>
>
> We got a cluster of two nodes running Cassandra.3.7.0 and using client 
> running Python 2.7.11 injecting lot of data from maybe 100 or so jobs. 
> --
> Cache setting can be seen from system.log:
> INFO  [main] 2016-09-30 15:12:50,002 AuthCache.java:172 - (Re)initializing 
> CredentialsCache (validity period/update interval/max entries) 
> (2000/2000/1000)
> INFO  [SharedPool-Worker-1] 2016-09-30 15:15:09,561 AuthCache.java:172 - 
> (Re)initializing PermissionsCache (validity period/update interval/max 
> entries) (1/1/1000)
> INFO  [SharedPool-Worker-1] 2016-09-30 15:15:24,319 AuthCache.java:172 - 
> (Re)initializing RolesCache (validity period/update interval/max entries) 
> (5000/5000/1000)
> ===
> But I am getting the following exception :
> ERROR [SharedPool-Worker-90] 2016-09-30 15:17:20,883 ErrorMessage.java:338 - 
> Unexpected exception during request
> com.google.common.util.concurrent.UncheckedExecutionException: 
> com.google.common.util.concurrent.UncheckedExecutionException: 
> java.lang.RuntimeException: 
> org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - 
> received only 0 responses.
>   at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2203) 
> ~[guava-18.0.jar:na]
>   at com.google.common.cache.LocalCache.get(LocalCache.java:3937) 
> ~[guava-18.0.jar:na]
>   at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941) 
> ~[guava-18.0.jar:na]
>   at 
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)
>  ~[guava-18.0.jar:na]
>   at org.apache.cassandra.auth.AuthCache.get(AuthCache.java:108) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
>   at 
> org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:45)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
>   at 
> org.apache.cassandra.auth.AuthenticatedUser.getPermissions(AuthenticatedUser.java:104)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
>   at 
> org.apache.cassandra.service.ClientState.authorize(ClientState.java:375) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
>   at 
> org.apache.cassandra.service.ClientState.checkPermissionOnResourceChain(ClientState.java:308)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
>   at 
> org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:285)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
>   at 
> org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:272) 
> ~[apache-cassandra-3.7.0.jar:3.7.0]
>   at 
> org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:256)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.checkAccess(ModificationStatement.java:211)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
>   at 
> org.apache.cassandra.cql3.statements.BatchStatement.checkAccess(BatchStatement.java:137)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processBatch(QueryProcessor.java:502)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processBatch(QueryProcessor.java:495)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
>   at 
> org.apache.cassandra.transport.messages.BatchMessage.execute(BatchMessage.java:217)
>  ~[apache-cassandra-3.7.0.jar:3.7.0]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [apache-cassandra-3.7.0.jar:3.7.0]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [apache-cassandra-3.7.0.jar:3.7.0]
>   at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.36.Final.jar:4.0.36.Final]
>  

[jira] [Commented] (CASSANDRA-10735) Support netty openssl (netty-tcnative) for client encryption

2019-04-16 Thread Niten Aggarwal (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-10735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819595#comment-16819595
 ] 

Niten Aggarwal commented on CASSANDRA-10735:


Hi @Jason,

I tried upgrading my driver to 4.0 with SSL and getting below exception. 
Truststore is valid becaise if i change password, it gives me authentication 
error. I believe either it needs socket timeout setting?

Another question on this topic.. This Jira was suppose to solve " Support netty 
openssl (netty-tcnative) for client encryption" but as per configuration it 
only provides Default JDK ssl. How to enable SSL for openSSL?? If we have to 
write our custom SSL handler, I believe that's not the intent of Jira??

 

16:22:04.624 [s0-admin-1] WARN  c.d.o.d.i.c.c.ControlConnection - [s0] Error 
connecting to idpcsbmdevdpl002.vci.att.com/135.198.127.60:7011, trying next node
com.datastax.oss.driver.api.core.connection.ConnectionInitException: 
[s0|control|id: 0x8f1cc575, L:/135.165.156.72:62524 - 
R:idpcsbmdevdpl002.vci.att.com/135.198.127.60:7011] init query STARTUP: error 
writing 
    at 
com.datastax.oss.driver.internal.core.channel.ProtocolInitHandler$InitRequest.fail(ProtocolInitHandler.java:297)
    at 
com.datastax.oss.driver.internal.core.channel.ChannelHandlerRequest.writeListener(ChannelHandlerRequest.java:74)
    at 
io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:502)
    at 
io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:495)
    at 
io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:474)
    at 
io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:415)
    at 
io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:540)
    at 
io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:533)
    at 
io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:114)
    at 
io.netty.util.internal.PromiseNotificationUtil.tryFailure(PromiseNotificationUtil.java:64)
    at 
io.netty.channel.DelegatingChannelPromiseNotifier.operationComplete(DelegatingChannelPromiseNotifier.java:57)
    at 
io.netty.channel.DelegatingChannelPromiseNotifier.operationComplete(DelegatingChannelPromiseNotifier.java:31)
    at 
io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:502)
    at 
io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:476)
    at 
io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:415)
    at 
io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:540)
    at 
io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:533)
    at 
io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:114)
    at 
io.netty.util.internal.PromiseNotificationUtil.tryFailure(PromiseNotificationUtil.java:64)
    at 
io.netty.channel.DelegatingChannelPromiseNotifier.operationComplete(DelegatingChannelPromiseNotifier.java:57)
    at 
io.netty.channel.DelegatingChannelPromiseNotifier.operationComplete(DelegatingChannelPromiseNotifier.java:31)
    at 
io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:502)
    at 
io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:476)
    at 
io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:415)
    at 
io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:540)
    at 
io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:533)
    at 
io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:114)
    at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:847)
    at io.netty.handler.ssl.SslHandler.wrapAndFlush(SslHandler.java:810)
    at 
io.netty.handler.ssl.SslHandler.handleUnwrapThrowable(SslHandler.java:1255)
    at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1231)
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1272)
    at 
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502)
    at 
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441)
    at 
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278)
    at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
    at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345)
    at 
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337)
    at 
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408)
    at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
    at 
io.netty.channe