[ https://issues.apache.org/jira/browse/CASSANDRA-11583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15843131#comment-15843131 ]
Cristian P edited comment on CASSANDRA-11583 at 1/27/17 5:09 PM: ----------------------------------------------------------------- I think I hit the same problem (for apache cassandra 2.1.13). I have the exact same error. What I noticed is that if I avoid the -i flag (ignore hosts) it is working as expected. ERROR 17:02:14 [Stream #fdaa4e80-e4b1-11e6-aae3-8b496c707234] Streaming error occurred java.lang.AssertionError: null at org.apache.cassandra.io.sstable.SSTableLoader.releaseReferences(SSTableLoader.java:208) ~[apache-cassandra-2.1.13.jar:2.1.13] at org.apache.cassandra.io.sstable.SSTableLoader.onSuccess(SSTableLoader.java:193) ~[apache-cassandra-2.1.13.jar:2.1.13] at org.apache.cassandra.io.sstable.SSTableLoader.onSuccess(SSTableLoader.java:48) ~[apache-cassandra-2.1.13.jar:2.1.13] at com.google.common.util.concurrent.Futures$4.run(Futures.java:1181) ~[guava-16.0.jar:na] at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297) ~[guava-16.0.jar:na] at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156) ~[guava-16.0.jar:na] at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145) ~[guava-16.0.jar:na] at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:185) ~[guava-16.0.jar:na] at org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:213) ~[apache-cassandra-2.1.13.jar:2.1.13] at org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:184) ~[apache-cassandra-2.1.13.jar:2.1.13] at org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:415) ~[apache-cassandra-2.1.13.jar:2.1.13] at org.apache.cassandra.streaming.StreamSession.complete(StreamSession.java:607) ~[apache-cassandra-2.1.13.jar:2.1.13] at org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:471) ~[apache-cassandra-2.1.13.jar:2.1.13] at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:256) ~[apache-cassandra-2.1.13.jar:2.1.13] was (Author: cmposto): I think I hit the same problem. I have the exact same error. What I noticed is that if I avoid the -i flag (ignore hosts) it is working as expected. ERROR 17:02:14 [Stream #fdaa4e80-e4b1-11e6-aae3-8b496c707234] Streaming error occurred java.lang.AssertionError: null at org.apache.cassandra.io.sstable.SSTableLoader.releaseReferences(SSTableLoader.java:208) ~[apache-cassandra-2.1.13.jar:2.1.13] at org.apache.cassandra.io.sstable.SSTableLoader.onSuccess(SSTableLoader.java:193) ~[apache-cassandra-2.1.13.jar:2.1.13] at org.apache.cassandra.io.sstable.SSTableLoader.onSuccess(SSTableLoader.java:48) ~[apache-cassandra-2.1.13.jar:2.1.13] at com.google.common.util.concurrent.Futures$4.run(Futures.java:1181) ~[guava-16.0.jar:na] at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297) ~[guava-16.0.jar:na] at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156) ~[guava-16.0.jar:na] at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145) ~[guava-16.0.jar:na] at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:185) ~[guava-16.0.jar:na] at org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:213) ~[apache-cassandra-2.1.13.jar:2.1.13] at org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:184) ~[apache-cassandra-2.1.13.jar:2.1.13] at org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:415) ~[apache-cassandra-2.1.13.jar:2.1.13] at org.apache.cassandra.streaming.StreamSession.complete(StreamSession.java:607) ~[apache-cassandra-2.1.13.jar:2.1.13] at org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:471) ~[apache-cassandra-2.1.13.jar:2.1.13] at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:256) ~[apache-cassandra-2.1.13.jar:2.1.13] > Exception when streaming sstables using `sstableloader` > ------------------------------------------------------- > > Key: CASSANDRA-11583 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11583 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: $ uname -a > Linux bigdb-100 3.2.0-99-virtual #139-Ubuntu SMP Mon Feb 1 23:52:21 UTC 2016 > x86_64 x86_64 x86_64 GNU/Linux > I am using Datastax Enterprise 4.7.8-1 which is based on 2.1.13. > Reporter: Jens Rantil > > This bug came out of CASSANDRA-11562. > I have have a keyspace snapshotted from a 2.1.11 (DSE 4.7.5-1) node. When I'm > running the `sstableloader` I get the following output/exception: > {noformat} > # sstableloader --nodes X.X.X.20 --username YYY --password ZZZ --ignore XXX > /var/lib/cassandra/data/XXX/ZZZ-f7ebdf0daa3a3062828fddebc109a3b2 > Established connection to initial hosts > Opening sstables and calculating sections to stream > Streaming relevant part of > /var/lib/cassandra/data/XXX/ZZZ-f7ebdf0daa3a3062828fddebc109a3b2/XXX-ZZZ-ka-6463-Data.db > > /var/lib/cassandra/data/XXX/ZZZ-f7ebdf0daa3a3062828fddebc109a3b2/tink-ZZZ-ka-6464-Data.db > to [/X.X.X.33, /X.X.X.113, /X.X.X.32, /X.X.X.20, /X.X.X.122, /X.X.X.176, > /X.X.X.143, /X.X.X.172, /X.X.X.50, /X.X.X.51, /X.X.X.52, /X.X.X.71, > /X.X.X.53, /X.X.X.54, /X.X.X.47, /X.X.X.31, /X.X.X.8] > progress: [/X.X.X.113]0:0/2 0 % [/X.X.X.143]0:0/2 0 % [/X.X.X.172]0:0/2 0 > % [/X.X.X.20]0:0/2 0 % [/X.X.X.71]0:0/2 0 % [/X.X.X.122]0:0/2 0 % > [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 0 % [/X.X.X.143]0:0/2 0 % > [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:0/2 0 % > [/X.X.X.122]0:0/2 0 % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 0 % > [/X.X.X.143]0:0/2 1 % [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 % > [/X.X.X.71]0:0/2 0 % [/X.X.X.122]0:0/2 0 % [/X.X.X.47]0:0/2 progress: > [/X.X.X.113]0:0/2 0 % [/X.X.X.143]0:1/2 1 % [/X.X.X.172]0:0/2 0 % > [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:0/2 0 % [/X.X.X.122]0:0/2 0 % > [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 0 % [/X.X.X.143]0:1/2 1 % > [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:1/2 1 % > [/X.X.X.122]0:0/2 0 % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 0 % > [/X.X.X.143]0:1/2 1 % [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 % > [/X.X.X.71]0:1/2 1 % [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 progress: > [/X.X.X.113]0:0/2 0 % [/X.X.X.143]0:1/2 1 % [/X.X.X.172]0:0/2 0 % > [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:1/2 1 % [/X.X.X.122]0:1/2 1 % > [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 7 % [/X.X.X.143]0:1/2 1 % > [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:1/2 1 % > [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 7 % > [/X.X.X.143]0:1/2 6 % [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 % > [/X.X.X.71]0:1/2 1 % [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 progress: > [/X.X.X.113]0:0/2 12 % [/X.X.X.143]0:1/2 6 % [/X.X.X.172]0:0/2 0 % > [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:1/2 1 % [/X.X.X.122]0:1/2 1 % > [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 12 % [/X.X.X.143]0:1/2 11 % > [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:1/2 1 % > [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 19 % > [/X.X.X.143]0:1/2 11 % [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 % > [/X.X.X.71]0:1/2 1 % [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 progress: > [/X.X.X.113]0:0/2 19 % [/X.X.X.143]0:1/2 15 % [/X.X.X.172]0:0/2 0 % > [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:1/2 1 % [/X.X.X.122]0:1/2 1 % > [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 26 % [/X.X.X.143]0:1/2 15 % > [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:1/2 1 % > [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 26 % > [/X.X.X.143]0:1/2 20 % [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 % > [/X.X.X.71]0:1/2 1 % [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 progress: > [/X.X.X.113]0:0/2 26 % [/X.X.X.143]0:1/2 21 % [/X.X.X.172]0:0/2 0 % > [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:1/2 1 % [/X.X.X.122]0:1/2 1 % > [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 26 % [/X.X.X.143]0:1/2 21 % > [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 3 % [/X.X.X.71]0:1/2 1 % > [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 42 % > [/X.X.X.143]0:1/2 27 % [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 3 % > [/X.X.X.71]0:1/2 6 % [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 > [...] > progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 78 > % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:1/2 97 % > [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% > [/X.X.X.172]0:0/2 78 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% > [/X.X.X.122]0:1/2 97 % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2 100% > [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 86 % [/X.X.X.20]0:2/2 100% > [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:0/2 progress: > [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 86 % > [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% > [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% > [/X.X.X.172]0:0/2 86 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% > [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2 100% > [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 86 % [/X.X.X.20]0:2/2 100% > [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:0/2 progress: > [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 86 % > [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% > [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% > [/X.X.X.172]0:0/2 95 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% > [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2 100% > [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 95 % [/X.X.X.20]0:2/2 100% > [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:1/2 progress: > [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 95 % > [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% > [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% > [/X.X.X.172]0:0/2 95 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% > [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% > [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:1/2 98 % [/X.X.X.20]0:2/2 100% > [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: > [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:1/2 99 % > [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% > [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% > [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% > [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% > [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% > [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: > [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100% > [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% > [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% > [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% > [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% > [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% > [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: > [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100% > [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% > [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% > [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% > [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% > [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% > [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: > [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100% > [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% > [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% > [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% > [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 100% [/X.X.X.176]0:2/2 100% > [/X.X.X.8]0:2/2 100% total: 100% 0 MB/s(avg: 23 MB/s) > Summary statistics: > Connections per host: : 1 > Total files transferred: : 14 > Total bytes transferred: : 142070241 > Total duration (ms): : 5823 > Average transfer rate (MB/s): : 23 > Peak transfer rate (MB/s): : 33 > ERROR 09:07:12 [Stream #6e345de0-02e9-11e6-b944-1fee10beb91b] Streaming error > occurred > java.lang.AssertionError: null > at > org.apache.cassandra.io.sstable.SSTableLoader.releaseReferences(SSTableLoader.java:208) > ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218] > at > org.apache.cassandra.io.sstable.SSTableLoader.onSuccess(SSTableLoader.java:193) > ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218] > at > org.apache.cassandra.io.sstable.SSTableLoader.onSuccess(SSTableLoader.java:48) > ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218] > at com.google.common.util.concurrent.Futures$4.run(Futures.java:1181) > ~[guava-16.0.1.jar:na] > at > com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297) > ~[guava-16.0.1.jar:na] > at > com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156) > ~[guava-16.0.1.jar:na] > at > com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145) > ~[guava-16.0.1.jar:na] > at > com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:185) > ~[guava-16.0.1.jar:na] > at > org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:213) > ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218] > at > org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:184) > ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218] > at > org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:415) > ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218] > at > org.apache.cassandra.streaming.StreamSession.complete(StreamSession.java:607) > ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218] > at > org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:471) > ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218] > at > org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:263) > ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80] > {noformat} > Question: Is it an issue that I dumped the sstables using a _slightly_ older > Cassandra node (2.1.11 instead of 2.1.13)? Is streaming from 2.1.13 to 2.1.11 > incompatible? > Let me know if you need anything else from me. Also, know of any possible for > workaround? -- This message was sent by Atlassian JIRA (v6.3.4#6332)