More finding on the problem:
1. the problem present itself when using nodetool status
$ nodetool status
error: Unknown keyspace system_traces
-- StackTrace --
java.lang.AssertionError: Unknown keyspace system_traces
at org.apache.cassandra.db.Keyspace.<init>(Keyspace.java:270)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:119)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:96)
...
2. the problem disappear when I create an empty keyspace
cqlsh> create keyspace temp WITH REPLICATION = { 'class' :
'SimpleStrategy', 'replication_factor' : 2 };
$ nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID
Rack
UN 172.31.44.38 118.31 KB 256 ?
e5e97978-b048-46d8-936b-88b544459856 rack1
UN 172.31.44.39 106.48 KB 256 ?
6b021cd9-2fd0-44f4-afea-d01f0b64c45c rack1
My guess is system_traces initialization complete only after any data
insertion.
Before it does, any attempt to read from it either from nodetool, cqlsh or
streaming to a new node will fail.
On Mon, May 18, 2015 at 3:33 PM, Tzach Livyatan <tz...@cloudius-systems.com>
wrote:
> I have a dev cluster of two Cassandra 2.12 servers on EC2
> When adding a new server, I get a
> "Streaming error occurred java.lang.AssertionError: Unknown keyspace
> system_traces"
> exception on the cluster (not the new) server (full log below).
>
> Indeed, when I cqlsh to the cluster server, I see the following:
> cqlsh> DESCRIBE KEYSPACES;
>
> system_traces system
>
> cqlsh> use system_traces;
> code=2200 [Invalid query] message="Keyspace 'system_traces' does not exist"
>
> While
> cqlsh> DESCRIBE KEYSPACE system_traces;
> Do works!
>
> Is it a bug? feature?
>
> Thanks
> Tzach
>
>
>
> Full log from adding a node:
> INFO [STREAM-INIT-/172.31.19.130:48054] 2015-05-18 11:36:17,734
> StreamResultFuture.java:109 - [Stream #18f25dc0-fd52-11e4-a00a-c1b81a3b68f5
> ID#0] Creating new streaming plan for Bootst
> rap
> INFO [STREAM-INIT-/172.31.19.130:48054] 2015-05-18 11:36:17,735
> StreamResultFuture.java:116 - [Stream
> #18f25dc0-fd52-11e4-a00a-c1b81a3b68f5, ID#0] Received streaming plan for
> Bootstrap
> INFO [STREAM-INIT-/172.31.19.130:48055] 2015-05-18 11:36:17,736
> StreamResultFuture.java:116 - [Stream
> #18f25dc0-fd52-11e4-a00a-c1b81a3b68f5, ID#0] Received streaming plan for
> Bootstrap
> ERROR [STREAM-IN-/172.31.19.130] 2015-05-18 11:36:17,777
> StreamSession.java:472 - [Stream #18f25dc0-fd52-11e4-a00a-c1b81a3b68f5]
> Streaming error occurred
> java.lang.AssertionError: Unknown keyspace system_traces
> at org.apache.cassandra.db.Keyspace.<init>(Keyspace.java:273)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at org.apache.cassandra.db.Keyspace.open(Keyspace.java:122)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at org.apache.cassandra.db.Keyspace.open(Keyspace.java:99)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.streaming.StreamSession.getColumnFamilyStores(StreamSession.java:280)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.streaming.StreamSession.addTransferRanges(StreamSession.java:257)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.streaming.StreamSession.prepare(StreamSession.java:488)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:420)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:251)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
>
