[jira] [Commented] (CASSANDRA-13579) Request to change the accessibility of `RowCacheSerializer` needed for CAPi-Flash Enablement.

2017-06-07 Thread Jeff Jirsa (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16042232#comment-16042232
 ] 

Jeff Jirsa commented on CASSANDRA-13579:


Let's change it to:

{{Public; used by external Row Cache plugins}}

Just so future contributions don't accidentally set it to private thinking it's 
only used by tests.



> Request to change the accessibility of `RowCacheSerializer` needed for 
> CAPi-Flash Enablement.
> -
>
> Key: CASSANDRA-13579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13579
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: Amitkumar Ghatwal
>Assignee: Amitkumar Ghatwal
>  Labels: patch
> Fix For: 4.x
>
> Attachments: cassandra_change_RowCacheSerializer_accessibility.patch
>
>
> Hi All,
> [~ReiOdaira] , and I are working on creating a plugin for vendor specific 
> (CAPI-Flash ) implementation. As for our CAPI Flash enablement code, we are 
> now working on the plugin approach .And for creating the same we would need a 
> change in the accessibility of `RowCacheSerializer` . 
> Please refer to jira ticket/dev mails for reference : 
> 1) https://issues.apache.org/jira/browse/CASSANDRA-13486
> 2) http://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html
> [~spo...@gmail.com] [~jjirsa] : FYI . Adding a patch to change the 
> accessibility. Let me know once the change will be merged in the mainline.
> PR raised : https://github.com/apache/cassandra/pull/117
> Thanks !!!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-13579) Request to change the accessibility of `RowCacheSerializer` needed for CAPi-Flash Enablement.

2017-06-07 Thread Amitkumar Ghatwal (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16042219#comment-16042219
 ] 

Amitkumar Ghatwal commented on CASSANDRA-13579:
---

[~jjirsa] -  Thanks for your review . I can change the comment from 
// Package protected for tests
to
// Package public for tests
Yes we are targeting trunk/4.0/ and 3.x. [~ReiOdaira] - please confirm this as 
well.

> Request to change the accessibility of `RowCacheSerializer` needed for 
> CAPi-Flash Enablement.
> -
>
> Key: CASSANDRA-13579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13579
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: Amitkumar Ghatwal
>Assignee: Amitkumar Ghatwal
>  Labels: patch
> Fix For: 4.x
>
> Attachments: cassandra_change_RowCacheSerializer_accessibility.patch
>
>
> Hi All,
> [~ReiOdaira] , and I are working on creating a plugin for vendor specific 
> (CAPI-Flash ) implementation. As for our CAPI Flash enablement code, we are 
> now working on the plugin approach .And for creating the same we would need a 
> change in the accessibility of `RowCacheSerializer` . 
> Please refer to jira ticket/dev mails for reference : 
> 1) https://issues.apache.org/jira/browse/CASSANDRA-13486
> 2) http://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html
> [~spo...@gmail.com] [~jjirsa] : FYI . Adding a patch to change the 
> accessibility. Let me know once the change will be merged in the mainline.
> PR raised : https://github.com/apache/cassandra/pull/117
> Thanks !!!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-13584) Inclusion of cassandra-dtest project from code donation

2017-06-07 Thread Nate McCall (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16042148#comment-16042148
 ] 

Nate McCall commented on CASSANDRA-13584:
-

Message sent to {{gene...@incubator.apache.org}}:
https://lists.apache.org/thread.html/4bc7d591a03c1d84bd53828c23ae1059845c9f16e00ff3cac75d64cc@%3Cgeneral.incubator.apache.org%3E


> Inclusion of cassandra-dtest project from code donation
> ---
>
> Key: CASSANDRA-13584
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13584
> Project: Cassandra
>  Issue Type: Task
>Reporter: Nate McCall
>Assignee: Nate McCall
> Attachments: DataStax_software-grant.pdf
>
>
> The issue is for tracking legal and incubator acceptance of cassandra-dtest. 
> IP clearance template is located here:
> https://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/cassandra-dtest.xml
> The dtest project is located here:
> https://github.com/riptano/cassandra-dtest 
> For convienience, vote thread is here:
> https://lists.apache.org/thread.html/d9e694ba8eaac8e8c70cbfd3f6ee249d43f8c67279882ffc65e56cac@%3Cdev.cassandra.apache.org%3E
> Software grant from DataStax is attached. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-13584) Inclusion of cassandra-dtest project from code donation

2017-06-07 Thread Nate McCall (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nate McCall updated CASSANDRA-13584:

Description: 
The issue is for tracking legal and incubator acceptance of cassandra-dtest. 

IP clearance template is located here:
https://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/cassandra-dtest.xml

The dtest project is located here:
https://github.com/riptano/cassandra-dtest 

For convienience, vote thread is here:
https://lists.apache.org/thread.html/d9e694ba8eaac8e8c70cbfd3f6ee249d43f8c67279882ffc65e56cac@%3Cdev.cassandra.apache.org%3E

Software grant from DataStax is attached. 



  was:
The issue is for tracking legal and incubator acceptance of cassandra-dtest. 

IP clearance template is located here:
https://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/cassandra-dtest.xml

The dtest project is located here:
https://github.com/riptano/cassandra-dtest (uploaded tarball is from: 
6f7caba9c59daa949e67efc28f75e7de4c5b9fa7)

For convienience, vote thread is here:
https://lists.apache.org/thread.html/d9e694ba8eaac8e8c70cbfd3f6ee249d43f8c67279882ffc65e56cac@%3Cdev.cassandra.apache.org%3E

Software grant from DataStax is attached. 




> Inclusion of cassandra-dtest project from code donation
> ---
>
> Key: CASSANDRA-13584
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13584
> Project: Cassandra
>  Issue Type: Task
>Reporter: Nate McCall
>Assignee: Nate McCall
> Attachments: DataStax_software-grant.pdf
>
>
> The issue is for tracking legal and incubator acceptance of cassandra-dtest. 
> IP clearance template is located here:
> https://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/cassandra-dtest.xml
> The dtest project is located here:
> https://github.com/riptano/cassandra-dtest 
> For convienience, vote thread is here:
> https://lists.apache.org/thread.html/d9e694ba8eaac8e8c70cbfd3f6ee249d43f8c67279882ffc65e56cac@%3Cdev.cassandra.apache.org%3E
> Software grant from DataStax is attached. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-13584) Inclusion of cassandra-dtest project from code donation

2017-06-07 Thread Nate McCall (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nate McCall updated CASSANDRA-13584:

Description: 
The issue is for tracking legal and incubator acceptance of cassandra-dtest. 

IP clearance template is located here:
https://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/cassandra-dtest.xml

The dtest project is located here:
https://github.com/riptano/cassandra-dtest (uploaded tarball is from: 
6f7caba9c59daa949e67efc28f75e7de4c5b9fa7)

For convienience, vote thread is here:
https://lists.apache.org/thread.html/d9e694ba8eaac8e8c70cbfd3f6ee249d43f8c67279882ffc65e56cac@%3Cdev.cassandra.apache.org%3E

Software grant from DataStax is attached. 



  was:
The issue is for tracking legal and incubator acceptance of cassandra-dtest. 

IP clearance template is located here:
https://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/cassandra-dtest.xml

The dtest project is located here:
https://github.com/riptano/cassandra-dtest

For convienience, vote thread is here:
https://lists.apache.org/thread.html/d9e694ba8eaac8e8c70cbfd3f6ee249d43f8c67279882ffc65e56cac@%3Cdev.cassandra.apache.org%3E

Software grant from DataStax is attached. 




> Inclusion of cassandra-dtest project from code donation
> ---
>
> Key: CASSANDRA-13584
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13584
> Project: Cassandra
>  Issue Type: Task
>Reporter: Nate McCall
>Assignee: Nate McCall
> Attachments: DataStax_software-grant.pdf
>
>
> The issue is for tracking legal and incubator acceptance of cassandra-dtest. 
> IP clearance template is located here:
> https://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/cassandra-dtest.xml
> The dtest project is located here:
> https://github.com/riptano/cassandra-dtest (uploaded tarball is from: 
> 6f7caba9c59daa949e67efc28f75e7de4c5b9fa7)
> For convienience, vote thread is here:
> https://lists.apache.org/thread.html/d9e694ba8eaac8e8c70cbfd3f6ee249d43f8c67279882ffc65e56cac@%3Cdev.cassandra.apache.org%3E
> Software grant from DataStax is attached. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-13584) Inclusion of cassandra-dtest project from code donation

2017-06-07 Thread Nate McCall (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nate McCall updated CASSANDRA-13584:

Description: 
The issue is for tracking legal and incubator acceptance of cassandra-dtest. 

IP clearance template is located here:
https://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/cassandra-dtest.xml

The dtest project is located here:
https://github.com/riptano/cassandra-dtest

For convienience, vote thread is here:
https://lists.apache.org/thread.html/d9e694ba8eaac8e8c70cbfd3f6ee249d43f8c67279882ffc65e56cac@%3Cdev.cassandra.apache.org%3E

Software grant from DataStax is attached. 



  was:
The issue is for tracking legal and incubator acceptance of cassandra-dtest. 

IP clearance template is located here:
https://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/cassandra-dtest.xml

The dtest project is located here:
https://github.com/riptano/cassandra-dtest

For convienience, vote thread is here:
https://lists.apache.org/list.html?d...@cassandra.apache.org:lte=9M:vote%20cassandra-dtest

Software grant from DataStax is attached. 




> Inclusion of cassandra-dtest project from code donation
> ---
>
> Key: CASSANDRA-13584
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13584
> Project: Cassandra
>  Issue Type: Task
>Reporter: Nate McCall
>Assignee: Nate McCall
> Attachments: DataStax_software-grant.pdf
>
>
> The issue is for tracking legal and incubator acceptance of cassandra-dtest. 
> IP clearance template is located here:
> https://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/cassandra-dtest.xml
> The dtest project is located here:
> https://github.com/riptano/cassandra-dtest
> For convienience, vote thread is here:
> https://lists.apache.org/thread.html/d9e694ba8eaac8e8c70cbfd3f6ee249d43f8c67279882ffc65e56cac@%3Cdev.cassandra.apache.org%3E
> Software grant from DataStax is attached. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-13584) Inclusion of cassandra-dtest project from code donation

2017-06-07 Thread Nate McCall (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16042096#comment-16042096
 ] 

Nate McCall commented on CASSANDRA-13584:
-

Message was acknowledged as received from secretary@ via a CC to private@c.a.o 
so we have it on record. 

> Inclusion of cassandra-dtest project from code donation
> ---
>
> Key: CASSANDRA-13584
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13584
> Project: Cassandra
>  Issue Type: Task
>Reporter: Nate McCall
>Assignee: Nate McCall
> Attachments: DataStax_software-grant.pdf
>
>
> The issue is for tracking legal and incubator acceptance of cassandra-dtest. 
> IP clearance template is located here:
> https://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/cassandra-dtest.xml
> The dtest project is located here:
> https://github.com/riptano/cassandra-dtest
> For convienience, vote thread is here:
> https://lists.apache.org/list.html?d...@cassandra.apache.org:lte=9M:vote%20cassandra-dtest
> Software grant from DataStax is attached. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-13577) Fix dynamic endpoint snitch for sub-millisecond use case

2017-06-07 Thread Simon Zhou (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16042055#comment-16042055
 ] 

Simon Zhou commented on CASSANDRA-13577:


Here are the patches. Not sure if we need one for 3.11.

|3.0.x |[patch | 
https://github.com/szhou1234/cassandra/commit/50a0a081f976d94b2d6f7883e28d4c427baa120c]|
|4.0 |[patch | 
https://github.com/szhou1234/cassandra/commit/73a3ff467a852eec7993efb0133945416bad4e46]|


> Fix dynamic endpoint snitch for sub-millisecond use case
> 
>
> Key: CASSANDRA-13577
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13577
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Simon Zhou
>Assignee: Simon Zhou
> Fix For: 3.0.x
>
>
> This is a follow up of https://issues.apache.org/jira/browse/CASSANDRA-6908. 
> After disabling severity (CASSANDRA-11737/CASSANDRA-11738) in a few 
> production clusters, I observed that the scores for all the endpoints are 
> mostly 0.0. Through debugging, I found this is caused by that these clusters 
> have p50 latency well below 1ms and the network latency is also <0.1ms (round 
> trip). Be noted that we use p50 sampled read latency and millisecond as time 
> unit. That means, if the latency is mostly below 1ms, the score will be 0. 
> This is definitely not something we want. To make DES work for these 
> sub-millisecond use cases, we should change the timeunit to at least 
> microsecond, or even nanosecond. I'll provide a patch soon.
> Evidence of the p50 latency:
> {code}
> nodetool tablehistograms  
> Percentile  SSTables Write Latency  Read LatencyPartition Size
> Cell Count
>   (micros)  (micros)   (bytes)
>   
> 50% 2.00 35.43454.83 20501
>  3
> 75% 2.00 42.51654.95 29521
>  3
> 95% 3.00182.79943.13 61214
>  3
> 98% 4.00263.21   1131.75 73457
>  3
> 99% 4.00315.85   1358.10 88148
>  3
> Min 0.00  9.89 11.8761
>  3
> Max 5.00654.95 129557.75943127
>  3
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-13577) Fix dynamic endpoint snitch for sub-millisecond use case

2017-06-07 Thread Simon Zhou (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Zhou updated CASSANDRA-13577:
---
Status: Patch Available  (was: Open)

> Fix dynamic endpoint snitch for sub-millisecond use case
> 
>
> Key: CASSANDRA-13577
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13577
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Simon Zhou
>Assignee: Simon Zhou
> Fix For: 3.0.x
>
>
> This is a follow up of https://issues.apache.org/jira/browse/CASSANDRA-6908. 
> After disabling severity (CASSANDRA-11737/CASSANDRA-11738) in a few 
> production clusters, I observed that the scores for all the endpoints are 
> mostly 0.0. Through debugging, I found this is caused by that these clusters 
> have p50 latency well below 1ms and the network latency is also <0.1ms (round 
> trip). Be noted that we use p50 sampled read latency and millisecond as time 
> unit. That means, if the latency is mostly below 1ms, the score will be 0. 
> This is definitely not something we want. To make DES work for these 
> sub-millisecond use cases, we should change the timeunit to at least 
> microsecond, or even nanosecond. I'll provide a patch soon.
> Evidence of the p50 latency:
> {code}
> nodetool tablehistograms  
> Percentile  SSTables Write Latency  Read LatencyPartition Size
> Cell Count
>   (micros)  (micros)   (bytes)
>   
> 50% 2.00 35.43454.83 20501
>  3
> 75% 2.00 42.51654.95 29521
>  3
> 95% 3.00182.79943.13 61214
>  3
> 98% 4.00263.21   1131.75 73457
>  3
> 99% 4.00315.85   1358.10 88148
>  3
> Min 0.00  9.89 11.8761
>  3
> Max 5.00654.95 129557.75943127
>  3
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-13584) Inclusion of cassandra-dtest project from code donation

2017-06-07 Thread Nate McCall (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16041791#comment-16041791
 ] 

Nate McCall commented on CASSANDRA-13584:
-

Email sent to secretary@ with the attached software grant. Will update once a 
reply is received. 

Next step is submission to {{gene...@incubator.apache.org}} for a lazy 
consensus. 

> Inclusion of cassandra-dtest project from code donation
> ---
>
> Key: CASSANDRA-13584
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13584
> Project: Cassandra
>  Issue Type: Task
>Reporter: Nate McCall
>Assignee: Nate McCall
> Attachments: DataStax_software-grant.pdf
>
>
> The issue is for tracking legal and incubator acceptance of cassandra-dtest. 
> IP clearance template is located here:
> https://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/cassandra-dtest.xml
> The dtest project is located here:
> https://github.com/riptano/cassandra-dtest
> For convienience, vote thread is here:
> https://lists.apache.org/list.html?d...@cassandra.apache.org:lte=9M:vote%20cassandra-dtest
> Software grant from DataStax is attached. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-13584) Inclusion of cassandra-dtest project from code donation

2017-06-07 Thread Nate McCall (JIRA)
Nate McCall created CASSANDRA-13584:
---

 Summary: Inclusion of cassandra-dtest project from code donation
 Key: CASSANDRA-13584
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13584
 Project: Cassandra
  Issue Type: Task
Reporter: Nate McCall
Assignee: Nate McCall
 Attachments: DataStax_software-grant.pdf

The issue is for tracking legal and incubator acceptance of cassandra-dtest. 

IP clearance template is located here:
https://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/cassandra-dtest.xml

The dtest project is located here:
https://github.com/riptano/cassandra-dtest

For convienience, vote thread is here:
https://lists.apache.org/list.html?d...@cassandra.apache.org:lte=9M:vote%20cassandra-dtest

Software grant from DataStax is attached. 





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-13583) test failure in rebuild_test.TestRebuild.disallow_rebuild_from_nonreplica_test

2017-06-07 Thread Michael Hamm (JIRA)
Michael Hamm created CASSANDRA-13583:


 Summary: test failure in 
rebuild_test.TestRebuild.disallow_rebuild_from_nonreplica_test
 Key: CASSANDRA-13583
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13583
 Project: Cassandra
  Issue Type: Bug
Reporter: Michael Hamm
 Attachments: node1_debug.log, node1_gc.log, node1.log, 
node2_debug.log, node2_gc.log, node2.log, node3_debug.log, node3_gc.log, 
node3.log

example failure:

http://cassci.datastax.com/job/trunk_novnode_dtest/524/testReport/rebuild_test/TestRebuild/disallow_rebuild_from_nonreplica_test

{noformat}
Error Message

ToolError not raised
 >> begin captured logging << 
dtest: DEBUG: Python driver version in use: 3.10
dtest: DEBUG: cluster ccm directory: /tmp/dtest-0tUjhX
dtest: DEBUG: Done setting configuration options:
{   'num_tokens': None,
'phi_convict_threshold': 5,
'range_request_timeout_in_ms': 1,
'read_request_timeout_in_ms': 1,
'request_timeout_in_ms': 1,
'truncate_request_timeout_in_ms': 1,
'write_request_timeout_in_ms': 1}
cassandra.cluster: INFO: New Cassandra host  discovered
cassandra.cluster: INFO: New Cassandra host  discovered
- >> end captured logging << -
{noformat}


{noformat}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/tools/decorators.py", line 48, in 
wrappedtestrebuild
f(obj)
  File "/home/automaton/cassandra-dtest/rebuild_test.py", line 357, in 
disallow_rebuild_from_nonreplica_test
node1.nodetool('rebuild -ks ks1 -ts (%s,%s] -s %s' % (node3_token, 
node1_token, node3_address))
  File "/usr/lib/python2.7/unittest/case.py", line 116, in __exit__
"{0} not raised".format(exc_name))
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-13582) test failure in upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_AllVersions_EndsAt_Trunk_HEAD.rolling_upgrade_test

2017-06-07 Thread Michael Hamm (JIRA)
Michael Hamm created CASSANDRA-13582:


 Summary: test failure in 
upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_AllVersions_EndsAt_Trunk_HEAD.rolling_upgrade_test
 Key: CASSANDRA-13582
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13582
 Project: Cassandra
  Issue Type: Bug
Reporter: Michael Hamm
 Attachments: node1_debug.log, node1_gc.log, node1.log, 
node2_debug.log, node2_gc.log, node2.log, node3_debug.log, node3_gc.log, 
node3.log

example failure:

http://cassci.datastax.com/job/cassandra-3.11_large_dtest/39/testReport/upgrade_tests.upgrade_through_versions_test/ProtoV3Upgrade_AllVersions_EndsAt_Trunk_HEAD/rolling_upgrade_test


{noformat}
Error Message

Subprocess ['nodetool', '-h', 'localhost', '-p', '7100', ['upgradesstables', 
'-a']] exited with non-zero status; exit status: 2; 
stderr: error: null
-- StackTrace --
java.lang.AssertionError
at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70)
at 
org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
at 
org.apache.cassandra.db.transform.BaseRows.applyOne(BaseRows.java:116)
at org.apache.cassandra.db.transform.BaseRows.add(BaseRows.java:107)
at 
org.apache.cassandra.db.transform.UnfilteredRows.add(UnfilteredRows.java:41)
at 
org.apache.cassandra.db.transform.Transformation.add(Transformation.java:156)
at 
org.apache.cassandra.db.transform.Transformation.apply(Transformation.java:122)
at 
org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:147)
at 
org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
at 
org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:57)
at 
org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
at 
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:195)
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at 
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
at 
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
at 
org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:415)
at 
org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:307)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at 
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
at java.lang.Thread.run(Thread.java:748)
{noformat}



{noformat}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File 
"/home/automaton/cassandra-dtest/upgrade_tests/upgrade_through_versions_test.py",
 line 279, in rolling_upgrade_test
self.upgrade_scenario(rolling=True)
  File 
"/home/automaton/cassandra-dtest/upgrade_tests/upgrade_through_versions_test.py",
 line 345, in upgrade_scenario
self.upgrade_to_version(version_meta, partial=True, nodes=(node,))
  File 
"/home/automaton/cassandra-dtest/upgrade_tests/upgrade_through_versions_test.py",
 line 446, in upgrade_to_version
node.nodetool('upgradesstables -a')
  File "/home/automaton/venv/local/lib/python2.7/site-packages/ccmlib/node.py", 
line 792, in nodetool
return handle_external_tool_process(p, ['nodetool', '-h', 'localhost', 
'-p', str(self.jmx_port), cmd.split()])
  File "/home/automaton/venv/local/lib/python2.7/site-packages/ccmlib/node.py", 
line 2018, in handle_external_tool_process
raise ToolError(cmd_args, rc, out, err)
'Subprocess [\'nodetool\', \'-h\', \'localhost\', \'-p\', \'7100\', 
[\'upgradesstables\', \'-a\']] exited with non-zero status; exit status: 2; 
\nstderr: error: null\n-- StackTrace --\njava.lang.AssertionError\n\tat 
org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70)\n\tat 
org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)\n\tat
 org.apache.cassandra.db.transform.BaseRows.applyOne(BaseRows.java:116)\n\tat 
org.apache.cassandra.db.transform.BaseRows.add(BaseRows.java:107)\n\tat 
org.apache.cassandra.db.transform.UnfilteredRows.add(UnfilteredRows.java:41)\n\tat
 
org.apache.cassandra.db.transform.Transformation.add(Transformation.java:156)\n\tat
 
org.apache.

[jira] [Commented] (CASSANDRA-13418) Allow TWCS to ignore overlaps when dropping fully expired sstables

2017-06-07 Thread Jonathan Owens (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16041465#comment-16041465
 ] 

Jonathan Owens commented on CASSANDRA-13418:


I also should mention that our TWCS compaction window length on this system is 
12 hours, but our partitions are 1 month, so partitions overlap at best 60 
tables. 

I think TWCS recommendations are to align compaction window length to partition 
width, which would result in few-to-no overlapping partitions across tables. We 
didn't know that at the time. I'm not sure how that interacts here.

> Allow TWCS to ignore overlaps when dropping fully expired sstables
> --
>
> Key: CASSANDRA-13418
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13418
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Corentin Chary
>  Labels: twcs
>
> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If 
> you really want read-repairs you're going to have sstables blocking the 
> expiration of other fully expired SSTables because they overlap.
> You can set unchecked_tombstone_compaction = true or tombstone_threshold to a 
> very low value and that will purge the blockers of old data that should 
> already have expired, thus removing the overlaps and allowing the other 
> SSTables to expire.
> The thing is that this is rather CPU intensive and not optimal. If you have 
> time series, you might not care if all your data doesn't exactly expire at 
> the right time, or if data re-appears for some time, as long as it gets 
> deleted as soon as it can. And in this situation I believe it would be really 
> beneficial to allow users to simply ignore overlapping SSTables when looking 
> for fully expired ones.
> To the question: why would you need read-repairs ?
> - Full repairs basically take longer than the TTL of the data on my dataset, 
> so this isn't really effective.
> - Even with a 10% chances of doing a repair, we found out that this would be 
> enough to greatly reduce entropy of the most used data (and if you have 
> timeseries, you're likely to have a dashboard doing the same important 
> queries over and over again).
> - LOCAL_QUORUM is too expensive (need >3 replicas), QUORUM is too slow.
> I'll try to come up with a patch demonstrating how this would work, try it on 
> our system and report the effects.
> cc: [~adejanovski], [~rgerard] as I know you worked on similar issues already.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-13579) Request to change the accessibility of `RowCacheSerializer` needed for CAPi-Flash Enablement.

2017-06-07 Thread Jeff Jirsa (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16041359#comment-16041359
 ] 

Jeff Jirsa commented on CASSANDRA-13579:


Seems reasonable, though the comment above your change should be modified to 
match the new behavior.

What versions are you targeting here? Just trunk/4.0 ? 


> Request to change the accessibility of `RowCacheSerializer` needed for 
> CAPi-Flash Enablement.
> -
>
> Key: CASSANDRA-13579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13579
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: Amitkumar Ghatwal
>Assignee: Amitkumar Ghatwal
>  Labels: patch
> Fix For: 4.x
>
> Attachments: cassandra_change_RowCacheSerializer_accessibility.patch
>
>
> Hi All,
> [~ReiOdaira] , and I are working on creating a plugin for vendor specific 
> (CAPI-Flash ) implementation. As for our CAPI Flash enablement code, we are 
> now working on the plugin approach .And for creating the same we would need a 
> change in the accessibility of `RowCacheSerializer` . 
> Please refer to jira ticket/dev mails for reference : 
> 1) https://issues.apache.org/jira/browse/CASSANDRA-13486
> 2) http://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html
> [~spo...@gmail.com] [~jjirsa] : FYI . Adding a patch to change the 
> accessibility. Let me know once the change will be merged in the mainline.
> PR raised : https://github.com/apache/cassandra/pull/117
> Thanks !!!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-11577) Traces persist for longer than 24 hours

2017-06-07 Thread Randy Fradin (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16041007#comment-16041007
 ] 

Randy Fradin commented on CASSANDRA-11577:
--

I am seeing the same issue, though my definitions for system_traces.* do have a 
default_time_to_live of 24 hours. I think there are 2 issues here:
(1) the definitions of the two tables in system_traces had default_time_to_live 
properties added to them, but these are only applied if the keyspace didn't 
exist in the first place, so if you upgrade from an earlier version then the 
definitions of the tables are never updated. I haven't tested this but it looks 
like that's what's going on in StorageService.joinTokenRing().
(2) Regardless of whether default_time_to_live is set or not, it's not actually 
being applied. Tracing.java uses CFRowAdder to create the cells for insertion 
into the tables, but the way it constructs the cells bypasses 
AbstractCell.create() which would have actually applied the defaultTimeToLive 
from the table definition, and instead directly constructs a BufferCell with no 
ttl.

>From a glance it looks like this is probably fixed in Cassandra 2.2+, possibly 
>by accident while implementing having different TTLs for different types of 
>tracing. But in Cassandra 2.1 as near as I can tell there's no way to enable 
>any tracing and have a ttl applied to the data.

> Traces persist for longer than 24 hours
> ---
>
> Key: CASSANDRA-11577
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11577
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Josh Wickman
>Priority: Minor
>
> My deployment currently has clusters on both Cassandra 1.2 (1.2.19) and 2.1 
> (2.1.11) with tracing on.  On 2.1, the trace records persist for longer than 
> the [documented 24 
> hours|https://docs.datastax.com/en/cql/3.3/cql/cql_reference/tracing_r.html]:
> {noformat}
> cqlsh> select started_at from system_traces.sessions limit 10;
>  started_at
> --
>  2016-03-11 23:28:40+
>  2016-03-14 21:09:07+
>  2016-03-14 16:42:25+
>  2016-03-14 16:13:13+
>  2016-03-14 19:12:11+
>  2016-03-14 21:25:57+
>  2016-03-29 22:45:28+
>  2016-03-14 19:56:27+
>  2016-03-09 23:31:41+
>  2016-03-10 23:08:44+
> (10 rows)
> {noformat}
> My systems on 1.2 do not exhibit this problem:
> {noformat}
> cqlsh> select started_at from system_traces.sessions limit 10;
>  started_at
> --
>  2016-04-13 22:49:31+
>  2016-04-14 18:06:45+
>  2016-04-14 07:57:00+
>  2016-04-14 04:35:05+
>  2016-04-14 03:54:20+
>  2016-04-14 10:54:38+
>  2016-04-14 18:34:04+
>  2016-04-14 12:56:57+
>  2016-04-14 01:57:20+
>  2016-04-13 21:36:01+
> {noformat}
> The event records also persist alongside the session records, for example:
> {noformat}
> cqlsh> select session_id, dateOf(event_id) from system_traces.events where 
> session_id = fc8c1e80-e7e0-11e5-a2fb-1968ff3c067b;
>  session_id   | dateOf(event_id)
> --+--
>  fc8c1e80-e7e0-11e5-a2fb-1968ff3c067b | 2016-03-11 23:28:40+
> {noformat}
> Between these versions, the table parameter {{default_time_to_live}} was 
> introduced.  The {{system_traces}} tables report the default value of 0:
> {noformat}
> cqlsh> desc table system_traces.sessions
> CREATE TABLE system_traces.sessions (
> session_id uuid PRIMARY KEY,
> coordinator inet,
> duration int,
> parameters map,
> request text,
> started_at timestamp
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = 'traced sessions'
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
> AND compression = {'sstable_compression': 
> 'org.apache.cassandra.io.compress.SnappyCompressor'}
> AND dclocal_read_repair_chance = 0.0
> AND default_time_to_live = 0
> AND gc_grace_seconds = 0
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {noformat}
> I suspect that {{default_time_to_live}} is superseding the mechanism used in 
> 1.2 to expire the trace records.  Evidently I cannot change this parameter 
> for this table:
> {noformat}
> cqlsh> alter table system_traces.sessions with default_time_to_live = 86400;
> Unauthorized: code=2100 [Unauthorized] message="Cannot ALTER  system_traces.sessions>"
> {noformat}
> I realize Cassandra 1.2 is no longer supported, but the problem is being 
> manifested in Cassandra 2.1 for me (I included 1.2 only for comparison).  
> Since I couldn't find an existin

[jira] [Created] (CASSANDRA-13581) Adding plugins support to Cassandra's webpage

2017-06-07 Thread Amitkumar Ghatwal (JIRA)
Amitkumar Ghatwal created CASSANDRA-13581:
-

 Summary: Adding plugins support to Cassandra's webpage
 Key: CASSANDRA-13581
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13581
 Project: Cassandra
  Issue Type: Improvement
  Components: Documentation and Website
Reporter: Amitkumar Ghatwal
 Fix For: 4.x


Hi [~spo...@gmail.com],

As was suggested here : 
http://www.mail-archive.com/dev@cassandra.apache.org/msg11183.html .  Have 
created the necessary *.rst file to create "plugins" link here : 
https://cassandra.apache.org/doc/latest/.

Have followed the steps here : 
https://cassandra.apache.org/doc/latest/development/documentation.html  and 
raised a PR : https://github.com/apache/cassandra/pull/118 for introducing 
plugins support on Cassandra's Webpage.

Let me know your review comments and if i have not done things correctly to 
make changes to cassandra's website i can rectify the same.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-13581) Adding plugins support to Cassandra's webpage

2017-06-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16040806#comment-16040806
 ] 

ASF GitHub Bot commented on CASSANDRA-13581:


Github user ghatwala commented on the issue:

https://github.com/apache/cassandra/pull/118
  
JIRA ticket created : https://issues.apache.org/jira/browse/CASSANDRA-13581


> Adding plugins support to Cassandra's webpage
> -
>
> Key: CASSANDRA-13581
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13581
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Amitkumar Ghatwal
>  Labels: documentation
> Fix For: 4.x
>
>
> Hi [~spo...@gmail.com],
> As was suggested here : 
> http://www.mail-archive.com/dev@cassandra.apache.org/msg11183.html .  Have 
> created the necessary *.rst file to create "plugins" link here : 
> https://cassandra.apache.org/doc/latest/.
> Have followed the steps here : 
> https://cassandra.apache.org/doc/latest/development/documentation.html  and 
> raised a PR : https://github.com/apache/cassandra/pull/118 for introducing 
> plugins support on Cassandra's Webpage.
> Let me know your review comments and if i have not done things correctly to 
> make changes to cassandra's website i can rectify the same.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-13580) Readonly datacenter support

2017-06-07 Thread Marcus Eriksson (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-13580:

Status: Patch Available  (was: Open)

https://github.com/krummas/cassandra/commits/marcuse/readonlydcs is my 
suggestion for this - needs a bit more work, for example it should be possible 
to override the readonly-flag to do a full repair

> Readonly datacenter support
> ---
>
> Key: CASSANDRA-13580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13580
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> Some setups include datacenters where only reads are performed (example could 
> be datacenter dedicated for taking backups).
> We could use this information during repair to make sure that we never stream 
> out of a read only dc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-13580) Readonly datacenter support

2017-06-07 Thread Marcus Eriksson (JIRA)
Marcus Eriksson created CASSANDRA-13580:
---

 Summary: Readonly datacenter support
 Key: CASSANDRA-13580
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13580
 Project: Cassandra
  Issue Type: Bug
Reporter: Marcus Eriksson
Assignee: Marcus Eriksson
 Fix For: 4.x


Some setups include datacenters where only reads are performed (example could 
be datacenter dedicated for taking backups).

We could use this information during repair to make sure that we never stream 
out of a read only dc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-8272) 2ndary indexes can return stale data

2017-06-07 Thread JIRA

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16039285#comment-16039285
 ] 

Andrés de la Peña edited comment on CASSANDRA-8272 at 6/7/17 11:21 AM:
---

Here is a new version of the patch for 3.11 and trunk:

||[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...adelapena:454617607063bfb554b841f0d891798404faf0b1]|[utests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-8272-3.11-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-8272-3.11-dtest/]|
||[trunk|https://github.com/apache/cassandra/compare/trunk...adelapena:1416d9b082d7f93b187cbf67abd9a917735c4804]|[utests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-8272-trunk-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/adelapena/job/adelapena-8272-trunk-dtest/]|

bq. No, I don't think we have to return all the rows not satisfying the index. 
I believe only returning those that are before the {{n}} th "valid" entry is 
enough. I don't think it's different from how we handle tombstones here: we 
don't return all tombstones, just the ones before the {{n}} th live results.
bq. Note that both with those new "invalid" entries and with tombstones, it's 
possible that post-resolution on the coordinator we end up being short on 
results. That is, a "valid" result from A is canceled by a tombstone/"invalid" 
result of B and vice-versa and we end up with less results than requested. But 
that's where the short-read protection from {{DataResolver}} kicks in.

Indeed, short-read protection solves the problem, so I have left the 
{{DataLimits.Counter}} as a stopping transformation. I have added [some 
dtests|https://github.com/adelapena/cassandra-dtest/blob/CASSANDRA-8272/secondary_indexes_test.py#L1205-L1343]
 checking these scenarios with indexes.

bq. As an aside, had a very very quick scan of the patch, and I'll also note 
that in {{StorageProxy}} and 
{{SinglePartitionReadCommand.Group.executeInternal}}, using only the 
post-processor of the 1st command would break if the index actually makes 
assumption based on the command it's passed on, so it feels dodgy and I think 
we sould make sure it's applied to each command result individually.

Yes, it is dodgy. I have changed it to apply the post processing to each 
command in the group. 

As we said, the patch for 3.11 only contains the changes in the coordinator 
side. I have added [a 
test|https://github.com/adelapena/cassandra/blob/8272-3.11/test/unit/org/apache/cassandra/index/CustomIndexTest.java#L804-L871]
 in {{CustomIndexTest}} that uses [a custom index 
implementation|https://github.com/adelapena/cassandra/blob/8272-3.11/test/unit/org/apache/cassandra/index/CustomIndexTest.java#L1180-L1256]
 to validate coordinator side filtering.

The patch for trunk also modifies regular secondary indexes to send stale rows. 
SASI don't uses the mechanism because of the aforementioned problem with 
expressions evaluation and text analysis, I think we should fix this in a 
separate ticket.

Please let me know what do you think.


was (Author: adelapena):
Here is a new version of the patch for 3.11 and trunk:

||[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...adelapena:454617607063bfb554b841f0d891798404faf0b1]||[trunk|https://github.com/apache/cassandra/compare/trunk...adelapena:1416d9b082d7f93b187cbf67abd9a917735c4804]||[dtests|https://github.com/riptano/cassandra-dtest/compare/master...adelapena:CASSANDRA-8272]||

Unfortunately cassci service is not working right now, I'll add the test 
results as soon as it comes back to live.

bq. No, I don't think we have to return all the rows not satisfying the index. 
I believe only returning those that are before the {{n}} th "valid" entry is 
enough. I don't think it's different from how we handle tombstones here: we 
don't return all tombstones, just the ones before the {{n}} th live results.
bq. Note that both with those new "invalid" entries and with tombstones, it's 
possible that post-resolution on the coordinator we end up being short on 
results. That is, a "valid" result from A is canceled by a tombstone/"invalid" 
result of B and vice-versa and we end up with less results than requested. But 
that's where the short-read protection from {{DataResolver}} kicks in.

Indeed, short-read protection solves the problem, so I have left the 
{{DataLimits.Counter}} as a stopping transformation. I have added [some 
dtests|https://github.com/adelapena/cassandra-dtest/blob/CASSANDRA-8272/secondary_indexes_test.py#L1205-L1343]
 checking these scenarios with indexes.

bq. As an aside, had a very very quick scan of the patch, and I'll also note 
that in {{StorageProxy}} and 
{{SinglePartitionReadCommand.Group.executeInternal}}, using only the 
post-processor of the 1st command would break if the index actually mak

[jira] [Updated] (CASSANDRA-13579) Request to change the accessibility of `RowCacheSerializer` needed for CAPi-Flash Enablement.

2017-06-07 Thread Stefan Podkowinski (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-13579:
---
Reviewer:   (was: Stefan Podkowinski)

> Request to change the accessibility of `RowCacheSerializer` needed for 
> CAPi-Flash Enablement.
> -
>
> Key: CASSANDRA-13579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13579
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: Amitkumar Ghatwal
>Assignee: Amitkumar Ghatwal
>  Labels: patch
> Fix For: 4.x
>
> Attachments: cassandra_change_RowCacheSerializer_accessibility.patch
>
>
> Hi All,
> [~ReiOdaira] , and I are working on creating a plugin for vendor specific 
> (CAPI-Flash ) implementation. As for our CAPI Flash enablement code, we are 
> now working on the plugin approach .And for creating the same we would need a 
> change in the accessibility of `RowCacheSerializer` . 
> Please refer to jira ticket/dev mails for reference : 
> 1) https://issues.apache.org/jira/browse/CASSANDRA-13486
> 2) http://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html
> [~spo...@gmail.com] [~jjirsa] : FYI . Adding a patch to change the 
> accessibility. Let me know once the change will be merged in the mainline.
> PR raised : https://github.com/apache/cassandra/pull/117
> Thanks !!!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-13579) Request to change the accessibility of `RowCacheSerializer` needed for CAPi-Flash Enablement.

2017-06-07 Thread Stefan Podkowinski (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-13579:
---
Component/s: (was: Build)
 (was: Core)
 (was: Packaging)
 Local Write-Read Paths

> Request to change the accessibility of `RowCacheSerializer` needed for 
> CAPi-Flash Enablement.
> -
>
> Key: CASSANDRA-13579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13579
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: Amitkumar Ghatwal
>Assignee: Amitkumar Ghatwal
>  Labels: patch
> Fix For: 4.x
>
> Attachments: cassandra_change_RowCacheSerializer_accessibility.patch
>
>
> Hi All,
> [~ReiOdaira] , and I are working on creating a plugin for vendor specific 
> (CAPI-Flash ) implementation. As for our CAPI Flash enablement code, we are 
> now working on the plugin approach .And for creating the same we would need a 
> change in the accessibility of `RowCacheSerializer` . 
> Please refer to jira ticket/dev mails for reference : 
> 1) https://issues.apache.org/jira/browse/CASSANDRA-13486
> 2) http://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html
> [~spo...@gmail.com] [~jjirsa] : FYI . Adding a patch to change the 
> accessibility. Let me know once the change will be merged in the mainline.
> PR raised : https://github.com/apache/cassandra/pull/117
> Thanks !!!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-13579) Request to change the accessibility of `RowCacheSerializer` needed for CAPi-Flash Enablement.

2017-06-07 Thread Stefan Podkowinski (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-13579:
---
Fix Version/s: (was: 3.11.x)

> Request to change the accessibility of `RowCacheSerializer` needed for 
> CAPi-Flash Enablement.
> -
>
> Key: CASSANDRA-13579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13579
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: Amitkumar Ghatwal
>Assignee: Amitkumar Ghatwal
>  Labels: patch
> Fix For: 4.x
>
> Attachments: cassandra_change_RowCacheSerializer_accessibility.patch
>
>
> Hi All,
> [~ReiOdaira] , and I are working on creating a plugin for vendor specific 
> (CAPI-Flash ) implementation. As for our CAPI Flash enablement code, we are 
> now working on the plugin approach .And for creating the same we would need a 
> change in the accessibility of `RowCacheSerializer` . 
> Please refer to jira ticket/dev mails for reference : 
> 1) https://issues.apache.org/jira/browse/CASSANDRA-13486
> 2) http://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html
> [~spo...@gmail.com] [~jjirsa] : FYI . Adding a patch to change the 
> accessibility. Let me know once the change will be merged in the mainline.
> PR raised : https://github.com/apache/cassandra/pull/117
> Thanks !!!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-13579) Request to change the accessibility of `RowCacheSerializer` needed for CAPi-Flash Enablement.

2017-06-07 Thread Stefan Podkowinski (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski updated CASSANDRA-13579:
---
Flags: Patch  (was: Patch,Important)

> Request to change the accessibility of `RowCacheSerializer` needed for 
> CAPi-Flash Enablement.
> -
>
> Key: CASSANDRA-13579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13579
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: Amitkumar Ghatwal
>Assignee: Amitkumar Ghatwal
>  Labels: patch
> Fix For: 4.x
>
> Attachments: cassandra_change_RowCacheSerializer_accessibility.patch
>
>
> Hi All,
> [~ReiOdaira] , and I are working on creating a plugin for vendor specific 
> (CAPI-Flash ) implementation. As for our CAPI Flash enablement code, we are 
> now working on the plugin approach .And for creating the same we would need a 
> change in the accessibility of `RowCacheSerializer` . 
> Please refer to jira ticket/dev mails for reference : 
> 1) https://issues.apache.org/jira/browse/CASSANDRA-13486
> 2) http://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html
> [~spo...@gmail.com] [~jjirsa] : FYI . Adding a patch to change the 
> accessibility. Let me know once the change will be merged in the mainline.
> PR raised : https://github.com/apache/cassandra/pull/117
> Thanks !!!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-13579) Request to change the accessibility of `RowCacheSerializer` needed for CAPi-Flash Enablement.

2017-06-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16040550#comment-16040550
 ] 

ASF GitHub Bot commented on CASSANDRA-13579:


Github user ghatwala commented on the issue:

https://github.com/apache/cassandra/pull/117
  
JIRA ticket raised : https://issues.apache.org/jira/browse/CASSANDRA-13579


> Request to change the accessibility of `RowCacheSerializer` needed for 
> CAPi-Flash Enablement.
> -
>
> Key: CASSANDRA-13579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13579
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Build, Core, Packaging
>Reporter: Amitkumar Ghatwal
>Assignee: Amitkumar Ghatwal
>  Labels: patch
> Fix For: 3.11.x, 4.x
>
> Attachments: cassandra_change_RowCacheSerializer_accessibility.patch
>
>
> Hi All,
> [~ReiOdaira] , and I are working on creating a plugin for vendor specific 
> (CAPI-Flash ) implementation. As for our CAPI Flash enablement code, we are 
> now working on the plugin approach .And for creating the same we would need a 
> change in the accessibility of `RowCacheSerializer` . 
> Please refer to jira ticket/dev mails for reference : 
> 1) https://issues.apache.org/jira/browse/CASSANDRA-13486
> 2) http://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html
> [~spo...@gmail.com] [~jjirsa] : FYI . Adding a patch to change the 
> accessibility. Let me know once the change will be merged in the mainline.
> PR raised : https://github.com/apache/cassandra/pull/117
> Thanks !!!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-13579) Request to change the accessibility of `RowCacheSerializer` needed for CAPi-Flash Enablement.

2017-06-07 Thread Amitkumar Ghatwal (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amitkumar Ghatwal updated CASSANDRA-13579:
--
Flags: Patch,Important  (was: Patch)
 Reviewer: Stefan Podkowinski
Fix Version/s: 3.11.x
  Description: 
Hi All,

[~ReiOdaira] , and I are working on creating a plugin for vendor specific 
(CAPI-Flash ) implementation. As for our CAPI Flash enablement code, we are now 
working on the plugin approach .And for creating the same we would need a 
change in the accessibility of `RowCacheSerializer` . 

Please refer to jira ticket/dev mails for reference : 
1) https://issues.apache.org/jira/browse/CASSANDRA-13486
2) http://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html

[~spo...@gmail.com] [~jjirsa] : FYI . Adding a patch to change the 
accessibility. Let me know once the change will be merged in the mainline.

PR raised : https://github.com/apache/cassandra/pull/117

Thanks !!!

  was:
Hi All,

[~ReiOdaira] , and I are working on creating a plugin for vendor specific 
(CAPI-Flash ) implementation. As for our CAPI Flash enablement code, we are now 
working on the plugin approach .And for creating the same we would need a 
change in the accessibility of `RowCacheSerializer` . 

Please refer to jira ticket/dev mails for reference : 
1) https://issues.apache.org/jira/browse/CASSANDRA-13486
2) http://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html

[~spo...@gmail.com] [~jjirsa] : FYI . Adding a patch to change the 
accessibility. Let me know once the change will be merged in the mainline.

Thanks !!!

  Component/s: (was: Local Write-Read Paths)
   Packaging
   Core
   Build
  Summary: Request to change the accessibility of `RowCacheSerializer` 
needed for CAPi-Flash Enablement.  (was: Change accessibility of 
`RowCacheSerializer` for CAPi-Flash enablement)

> Request to change the accessibility of `RowCacheSerializer` needed for 
> CAPi-Flash Enablement.
> -
>
> Key: CASSANDRA-13579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13579
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Build, Core, Packaging
>Reporter: Amitkumar Ghatwal
>Assignee: Amitkumar Ghatwal
>  Labels: patch
> Fix For: 3.11.x, 4.x
>
> Attachments: cassandra_change_RowCacheSerializer_accessibility.patch
>
>
> Hi All,
> [~ReiOdaira] , and I are working on creating a plugin for vendor specific 
> (CAPI-Flash ) implementation. As for our CAPI Flash enablement code, we are 
> now working on the plugin approach .And for creating the same we would need a 
> change in the accessibility of `RowCacheSerializer` . 
> Please refer to jira ticket/dev mails for reference : 
> 1) https://issues.apache.org/jira/browse/CASSANDRA-13486
> 2) http://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html
> [~spo...@gmail.com] [~jjirsa] : FYI . Adding a patch to change the 
> accessibility. Let me know once the change will be merged in the mainline.
> PR raised : https://github.com/apache/cassandra/pull/117
> Thanks !!!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Assigned] (CASSANDRA-13579) Change accessibility of `RowCacheSerializer` for CAPi-Flash enablement

2017-06-07 Thread Stefan Podkowinski (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Podkowinski reassigned CASSANDRA-13579:
--

 Assignee: Amitkumar Ghatwal
Flags: Patch  (was: Patch,Important)
 Reviewer:   (was: Stefan Podkowinski)
  Summary: Change accessibility of `RowCacheSerializer` for CAPi-Flash 
enablement  (was: Request to change the accessibility of `RowCacheSerializer` 
needed for CAPi-Flash Enablement.)
Fix Version/s: (was: 3.11.x)
  Component/s: (was: Build)
   (was: Core)
   (was: Packaging)
   Local Write-Read Paths

> Change accessibility of `RowCacheSerializer` for CAPi-Flash enablement
> --
>
> Key: CASSANDRA-13579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13579
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Build, Core, Packaging
>Reporter: Amitkumar Ghatwal
>Assignee: Amitkumar Ghatwal
>  Labels: patch
> Fix For: 3.11.x, 4.x
>
> Attachments: cassandra_change_RowCacheSerializer_accessibility.patch
>
>
> Hi All,
> [~ReiOdaira] , and I are working on creating a plugin for vendor specific 
> (CAPI-Flash ) implementation. As for our CAPI Flash enablement code, we are 
> now working on the plugin approach .And for creating the same we would need a 
> change in the accessibility of `RowCacheSerializer` . 
> Please refer to jira ticket/dev mails for reference : 
> 1) https://issues.apache.org/jira/browse/CASSANDRA-13486
> 2) http://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html
> [~spo...@gmail.com] [~jjirsa] : FYI . Adding a patch to change the 
> accessibility. Let me know once the change will be merged in the mainline.
> Thanks !!!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-13547) Filtered materialized views missing data

2017-06-07 Thread Krishna Dattu Koneru (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krishna Dattu Koneru updated CASSANDRA-13547:
-
Status: Patch Available  (was: In Progress)

> Filtered materialized views missing data
> 
>
> Key: CASSANDRA-13547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13547
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
> Environment: Official Cassandra 3.10 Docker image (ID 154b919bf8ce).
>Reporter: Craig Nicholson
>Assignee: Krishna Dattu Koneru
>Priority: Blocker
>  Labels: materializedviews
> Fix For: 3.11.x
>
>
> When creating a materialized view against a base table the materialized view 
> does not always reflect the correct data.
> Using the following test schema:
> {code:title=Schema|language=sql}
> DROP KEYSPACE IF EXISTS test;
> CREATE KEYSPACE test
>   WITH REPLICATION = { 
>'class' : 'SimpleStrategy', 
>'replication_factor' : 1 
>   };
> CREATE TABLE test.table1 (
> id int,
> name text,
> enabled boolean,
> foo text,
> PRIMARY KEY (id, name));
> CREATE MATERIALIZED VIEW test.table1_mv1 AS SELECT id, name, foo
> FROM test.table1
> WHERE id IS NOT NULL 
> AND name IS NOT NULL 
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> CREATE MATERIALIZED VIEW test.table1_mv2 AS SELECT id, name, foo, enabled
> FROM test.table1
> WHERE id IS NOT NULL 
> AND name IS NOT NULL 
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> {code}
> When I insert a row into the base table the materialized views are updated 
> appropriately. (+)
> {code:title=Insert row|language=sql}
> cqlsh> INSERT INTO test.table1 (id, name, enabled, foo) VALUES (1, 'One', 
> TRUE, 'Bar');
> cqlsh> SELECT * FROM test.table1;
>  id | name | enabled | foo
> +--+-+-
>   1 |  One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
>  name | id | foo
> --++-
>   One |  1 | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
>  name | id | enabled | foo
> --++-+-
>   One |  1 |True | Bar
> (1 rows)
> {code}
> Updating the record in the base table and setting enabled to FALSE will 
> filter the record from both materialized views. (+)
> {code:title=Disable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = FALSE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
>  id | name | enabled | foo
> +--+-+-
>   1 |  One |   False | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
>  name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
>  name | id | enabled | foo
> --++-+-
> (0 rows)
> {code}
> However a further update to the base table setting enabled to TRUE should 
> include the record in both materialzed views, however only one view 
> (table1_mv2) gets updated. (-)
> It appears that only the view (table1_mv2) that returns the filtered column 
> (enabled) is updated. (-)
> Additionally columns that are not part of the partiion or clustering key are 
> not updated. You can see that the foo column has a null value in table1_mv2. 
> (-)
> {code:title=Enable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
>  id | name | enabled | foo
> +--+-+-
>   1 |  One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
>  name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
>  name | id | enabled | foo
> --++-+--
>   One |  1 |True | null
> (1 rows)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-13547) Filtered materialized views missing data

2017-06-07 Thread Krishna Dattu Koneru (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16040527#comment-16040527
 ] 

Krishna Dattu Koneru commented on CASSANDRA-13547:
--

I have uploaded a patch (with unittest) for problem 1 in above comment @ 
https://github.com/apache/cassandra/compare/cassandra-3.11...krishna-koneru:cassandra-3.11-13547

below problem addressed by this patch  (from the description)
{code}
cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
cqlsh> SELECT * FROM test.table1;

 id | name | enabled | foo
+--+-+-
  1 |  One |True | Bar

(1 rows)
cqlsh> SELECT * FROM test.table1_mv1;

 name | id | foo
--++-
{code}

This happens because view metadata does not have column "enabled" .
The decision "should this base table update, update view as well?" is made by 
looking at view metadata (among other things).
Due to this, some updates to base table do not result in updates to MV even if 
update qualifies .

This patch disallows usage of any non PK columns in WHERE clause if they are 
not in SELECT list.

Comments are appreciated !


> Filtered materialized views missing data
> 
>
> Key: CASSANDRA-13547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13547
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
> Environment: Official Cassandra 3.10 Docker image (ID 154b919bf8ce).
>Reporter: Craig Nicholson
>Assignee: Krishna Dattu Koneru
>Priority: Blocker
>  Labels: materializedviews
> Fix For: 3.11.x
>
>
> When creating a materialized view against a base table the materialized view 
> does not always reflect the correct data.
> Using the following test schema:
> {code:title=Schema|language=sql}
> DROP KEYSPACE IF EXISTS test;
> CREATE KEYSPACE test
>   WITH REPLICATION = { 
>'class' : 'SimpleStrategy', 
>'replication_factor' : 1 
>   };
> CREATE TABLE test.table1 (
> id int,
> name text,
> enabled boolean,
> foo text,
> PRIMARY KEY (id, name));
> CREATE MATERIALIZED VIEW test.table1_mv1 AS SELECT id, name, foo
> FROM test.table1
> WHERE id IS NOT NULL 
> AND name IS NOT NULL 
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> CREATE MATERIALIZED VIEW test.table1_mv2 AS SELECT id, name, foo, enabled
> FROM test.table1
> WHERE id IS NOT NULL 
> AND name IS NOT NULL 
> AND enabled = TRUE
> PRIMARY KEY ((name), id);
> {code}
> When I insert a row into the base table the materialized views are updated 
> appropriately. (+)
> {code:title=Insert row|language=sql}
> cqlsh> INSERT INTO test.table1 (id, name, enabled, foo) VALUES (1, 'One', 
> TRUE, 'Bar');
> cqlsh> SELECT * FROM test.table1;
>  id | name | enabled | foo
> +--+-+-
>   1 |  One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
>  name | id | foo
> --++-
>   One |  1 | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
>  name | id | enabled | foo
> --++-+-
>   One |  1 |True | Bar
> (1 rows)
> {code}
> Updating the record in the base table and setting enabled to FALSE will 
> filter the record from both materialized views. (+)
> {code:title=Disable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = FALSE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
>  id | name | enabled | foo
> +--+-+-
>   1 |  One |   False | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
>  name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
>  name | id | enabled | foo
> --++-+-
> (0 rows)
> {code}
> However a further update to the base table setting enabled to TRUE should 
> include the record in both materialzed views, however only one view 
> (table1_mv2) gets updated. (-)
> It appears that only the view (table1_mv2) that returns the filtered column 
> (enabled) is updated. (-)
> Additionally columns that are not part of the partiion or clustering key are 
> not updated. You can see that the foo column has a null value in table1_mv2. 
> (-)
> {code:title=Enable the row|language=sql}
> cqlsh> UPDATE test.table1 SET enabled = TRUE WHERE id = 1 AND name = 'One';
> cqlsh> SELECT * FROM test.table1;
>  id | name | enabled | foo
> +--+-+-
>   1 |  One |True | Bar
> (1 rows)
> cqlsh> SELECT * FROM test.table1_mv1;
>  name | id | foo
> --++-
> (0 rows)
> cqlsh> SELECT * FROM test.table1_mv2;
>  name | id | enabled | foo
> --++-+-

[jira] [Commented] (CASSANDRA-13569) Schedule schema pulls just once per endpoint

2017-06-07 Thread Stefan Podkowinski (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16040476#comment-16040476
 ] 

Stefan Podkowinski commented on CASSANDRA-13569:


Ideally there would only be a single in-flight migration request for any schema 
ID and from there we'd go on and do the next request if any schema IDs still 
don't match. But this would require some more significant changes compared to 
the initially proposed patch, as we'd have to track pending requests based on 
schema ID and handle timeout/retry semantics. As I understand, [~iamaleksey] is 
going to implement consistent ordering for schema changes anyways in 
CASSANDRA-10699.

I'd really leave the patch as is for now to focus on avoiding scheduling schema 
pulls from the same endpoint multiple times.

> Schedule schema pulls just once per endpoint
> 
>
> Key: CASSANDRA-13569
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13569
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Distributed Metadata
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Schema mismatches detected through gossip will get resolved by calling 
> {{MigrationManager.maybeScheduleSchemaPull}}. This method may decide to 
> schedule execution of {{MigrationTask}}, but only after using a 
> {{MIGRATION_DELAY_IN_MS = 6}} delay (for reasons unclear to me). 
> Meanwhile, as long as the migration task hasn't been executed, we'll continue 
> to have schema mismatches reported by gossip and will have corresponding 
> {{maybeScheduleSchemaPull}} calls, which will schedule further tasks with the 
> mentioned delay. Some local testing shows that dozens of tasks for the same 
> endpoint will eventually be executed and causing the same, stormy behavior 
> for this very endpoints.
> My proposal would be to simply not schedule new tasks for the same endpoint, 
> in case we still have pending tasks waiting for execution after 
> {{MIGRATION_DELAY_IN_MS}}.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org