[
https://issues.apache.org/jira/browse/CASSANDRA-10621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14981810#comment-14981810
]
Joel Knighton edited comment on CASSANDRA-10621 at 10/30/15 3:22 AM:
---------------------------------------------------------------------
Thanks, Alan.
This error is thrown when a node is bootstrapping and is streamed a mutation
that affects a MV base table and the token of the view partition key is not
owned by the bootstrapping node.
It was not caught in dtests because the base table and view table always had
the same value for the partition key in writes, so there was always a pending
node for the view token.
I've pushed a fix as well as changes to the dtests to better detect similar
issues. I've also added a dtest for write survey mode. I'll update once CassCi
has results.
----
* [dtest
commit|https://github.com/jkni/cassandra-dtest/tree/74125f3f218a6eb7ff6403e8d21e91c48bfff2ab]
* [Cassandra 3.0
commit|https://github.com/jkni/cassandra/tree/b21a19b3131de49b01c615953f04b7092f03f638]
* [cassci-dtest|http://cassci.datastax.com/job/jkni-10621-dtest/]
* [cassci-testall|http://cassci.datastax.com/job/jkni-10621-testall/]
was (Author: jkni):
Thanks, Alan.
This error is thrown when a node is bootstrapping and is streamed a mutation
that affects a MV base table and the token of the view partition key is not
owned by the bootstrapping node.
It was not caught in dtests because the base table and view table always had
the same value for the partition key in writes, so there was always a pending
node for the view token.
I've pushed a fix as well as changes to the dtests to better detect similar
issues. I've also added a dtest for write survey mode.
----
* [dtest
commit|https://github.com/jkni/cassandra-dtest/tree/74125f3f218a6eb7ff6403e8d21e91c48bfff2ab]
* [Cassandra 3.0
commit|https://github.com/jkni/cassandra/tree/b21a19b3131de49b01c615953f04b7092f03f638]
* [cassci-dtest|http://cassci.datastax.com/job/jkni-10621-dtest/]
* [cassci-testall|http://cassci.datastax.com/job/jkni-10621-testall/]
> Error while bootstraping a new node with Materialized Views
> -----------------------------------------------------------
>
> Key: CASSANDRA-10621
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10621
> Project: Cassandra
> Issue Type: Bug
> Reporter: Alan Boudreault
> Assignee: Joel Knighton
> Priority: Critical
> Fix For: 3.0.0
>
> Attachments: system.log
>
>
> If I try to add a new node in a cluster that has materialized views, I get
> the following error:
> {code}
> ERROR 19:05:15 Unknown exception caught while attempting to update
> MaterializedView! mview.user_playlists
> java.lang.RuntimeException: Trying to get the view natural endpoint on a
> non-data replica
> at
> org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
> ~[main/:na]
> at
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482)
> [main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387)
> [main/:na]
> at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215)
> [main/:na]
> at
> org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
> [main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 19:05:15 Error applying streamed sstable:
> java.lang.RuntimeException: Trying to get the view natural endpoint on a
> non-data replica
> at
> org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
> ~[main/:na]
> at
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387)
> ~[main/:na]
> at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
> ~[main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> WARN 19:05:18 Writing large partition mview/genre_to_user:genre_3 (116986142
> bytes)
> WARN 19:05:21 Writing large partition mview/genre_to_user:genre_2 (116985746
> bytes)
> WARN 19:05:24 Writing large partition mview/genre_to_user:genre_5 (116986337
> bytes)
> ERROR 19:05:33 Unknown exception caught while attempting to update
> MaterializedView! mview.user_playlists
> java.lang.RuntimeException: Trying to get the view natural endpoint on a
> non-data replica
> at
> org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
> ~[main/:na]
> at
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482)
> [main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387)
> [main/:na]
> at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215)
> [main/:na]
> at
> org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
> [main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 19:05:33 Error applying streamed sstable:
> java.lang.RuntimeException: Trying to get the view natural endpoint on a
> non-data replica
> at
> org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
> ~[main/:na]
> at
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387)
> ~[main/:na]
> at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
> ~[main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 19:05:40 Unknown exception caught while attempting to update
> MaterializedView! mview.user_playlists
> java.lang.RuntimeException: Trying to get the view natural endpoint on a
> non-data replica
> at
> org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
> ~[main/:na]
> at
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482)
> [main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387)
> [main/:na]
> at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215)
> [main/:na]
> at
> org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
> [main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 19:05:40 Error applying streamed sstable:
> java.lang.RuntimeException: Trying to get the view natural endpoint on a
> non-data replica
> at
> org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
> ~[main/:na]
> at
> org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482)
> ~[main/:na]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387)
> ~[main/:na]
> at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215)
> ~[main/:na]
> at
> org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
> ~[main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
