[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690138#comment-13690138
]
Sylvain Lebresne commented on CASSANDRA-5668:
-
I don't think this patch can be committed as is: in both TracingExecutorService
and ExpiredTracingState, the package declaration is before the license header,
and I can't let that slide.
But with that fixed, +1.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-logs.tar.gz,
5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690328#comment-13690328
]
Jonathan Ellis commented on CASSANDRA-5668:
---
Fixed and committed. :)
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-logs.tar.gz,
5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690391#comment-13690391
]
Sylvain Lebresne commented on CASSANDRA-5668:
-
+1 on that follow up.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-followup.txt,
5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690401#comment-13690401
]
Jonathan Ellis commented on CASSANDRA-5668:
---
committed.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-followup.txt,
5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690403#comment-13690403
]
Ryan McGuire commented on CASSANDRA-5668:
-
The NullPointerException is gone, however, I see an unexpected change in the
trace after this commit.
Prior to the patch, I do this:
{code}
$ ccm create 5668
Current cluster is now: 5668
$ ccm populate -n 2:2
$ ccm start
$ ccm node1 cqlsh
Connected to 5668 at 127.0.0.1:9160.
[cqlsh 4.0.0 | Cassandra 2.0-SNAPSHOT | CQL spec 3.1.0 | Thrift protocol
19.37.0]
Use HELP for help.
cqlsh CREATE KEYSPACE test WITH replication = {'class':
'NetworkTopologyStrategy', 'dc1':2, 'dc2':2};
cqlsh CREATE TABLE test.test (id int PRIMARY KEY, value text);
cqlsh tracing on;
Now tracing requests.
cqlsh INSERT INTO test.test (id, value) VALUES ( 5, 'asdf');
Tracing session: 5d3bdaa0-da87-11e2-9eaa-35db2404c433
activity | timestamp
| source| source_elapsed
+--+---+
execute_cql3_query | 11:29:29,423
| 127.0.0.1 | 0
Parsing INSERT INTO test.test (id, value) VALUES ( 5, 'asdf'); | 11:29:29,424
| 127.0.0.1 | 1082
Peparing statement | 11:29:29,425
| 127.0.0.1 | 2241
Determining replicas for mutation | 11:29:29,425
| 127.0.0.1 | 2617
Sending message to /127.0.0.2 | 11:29:29,432
| 127.0.0.1 | 9321
Acquiring switchLock read lock | 11:29:29,433
| 127.0.0.1 | 10376
Appending to commitlog | 11:29:29,433
| 127.0.0.1 | 10424
Adding to test memtable | 11:29:29,433
| 127.0.0.1 | 10544
Sending message to /127.0.0.3 | 11:29:29,435
| 127.0.0.1 | 12658
Message received from /127.0.0.1 | 11:29:29,436
| 127.0.0.2 |102
Message received from /127.0.0.1 | 11:29:29,438
| 127.0.0.3 |103
Enqueuing forwarded write to /127.0.0.4 | 11:29:29,441
| 127.0.0.3 | 3596
Acquiring switchLock read lock | 11:29:29,441
| 127.0.0.3 | 3686
Appending to commitlog | 11:29:29,441
| 127.0.0.3 | 3704
Acquiring switchLock read lock | 11:29:29,442
| 127.0.0.2 | 5768
Adding to test memtable | 11:29:29,442
| 127.0.0.3 | 3759
Appending to commitlog | 11:29:29,442
| 127.0.0.2 | 5821
Sending message to /127.0.0.4 | 11:29:29,442
| 127.0.0.3 | 3834
Adding to test memtable | 11:29:29,442
| 127.0.0.2 | 5896
Enqueuing response to /127.0.0.1 | 11:29:29,442
| 127.0.0.3 | 4307
Enqueuing response to /127.0.0.1 | 11:29:29,443
| 127.0.0.2 | 6602
Message received from /127.0.0.3 | 11:29:29,444
| 127.0.0.4 | 92
Sending message to /127.0.0.1 | 11:29:29,444
| 127.0.0.2 | 8037
Message received from /127.0.0.2 | 11:29:29,445
| 127.0.0.1 | 36
Processing response from /127.0.0.2 | 11:29:29,445
| 127.0.0.1 |147
Acquiring switchLock read lock | 11:29:29,452
| 127.0.0.4 | 8791
Appending to commitlog | 11:29:29,452
| 127.0.0.4 | 8835
Adding to test memtable | 11:29:29,452
| 127.0.0.4 | 8897
Enqueuing response to /127.0.0.1 | 11:29:29,453
| 127.0.0.4 | 9542
Sending message to /127.0.0.1 | 11:29:29,454
| 127.0.0.4 | 10021
Message received from /127.0.0.4 | 11:29:29,454
| 127.0.0.1 | 9660
Processing response from /127.0.0.4 | 11:29:29,455
| 127.0.0.1 | 9804
Request complete | 11:29:29,435
| 127.0.0.1 | 12668
{code}
That's a 2x2 multi-dc cluster with a RF of 2 in each dc. I write a single
INSERT and I see four nodes append to their commit log. All is well.
After the patch, I see this trace:
{code}
Tracing session:
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690406#comment-13690406
]
Jonathan Ellis commented on CASSANDRA-5668:
---
Did you get the followup patch in that? Should have addressed it.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-followup.txt,
5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690411#comment-13690411
]
Ryan McGuire commented on CASSANDRA-5668:
-
I've retested with the second patch, it's the same.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-followup.txt,
5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690434#comment-13690434
]
Jonathan Ellis commented on CASSANDRA-5668:
---
fix pushed in 110d283afd780774a44368b17177b5e8e781e37f
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-followup.txt,
5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690441#comment-13690441
]
Ryan McGuire commented on CASSANDRA-5668:
-
That works in cassandra-1.2 - if you commit to trunk, I'll test there too.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-followup.txt,
5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13690537#comment-13690537
]
Jonathan Ellis commented on CASSANDRA-5668:
---
Merged to trunk.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Fix For: 1.2.6
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-followup.txt,
5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql, 5668.txt, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13689015#comment-13689015
]
Sylvain Lebresne commented on CASSANDRA-5668:
-
For what it's worth, I think that for the 2nd problem, another option might be
to make Tracing.initializeMessage behave slightly differently depending on the
message type. So if the state doesn't exist but the message type is a
REQUEST_RESPONSE, we could create the state and set it in the threadLocal, but
not save it in the global state map.
It's a bit of a hack though, but it slightly bother me to leave this to
expiration either so
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6, 2.0 beta 1
Reporter: Ryan McGuire
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-logs.tar.gz,
5668_npe_ddl.cql, 5668_npe_insert.cql, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13688345#comment-13688345
]
Ryan McGuire commented on CASSANDRA-5668:
-
I've tried to introduce sleeps in between INSERTs, even with 30s pauses between
them I still see the same (yet fewer) in the logs.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6, 2.0 beta 1
Reporter: Ryan McGuire
Attachments: 5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13688353#comment-13688353
]
Ryan McGuire commented on CASSANDRA-5668:
-
This is what one the traces looks like that failed:
{code}
127.0.0.1 78 Parsing statement Thrift:1
127.0.0.1 606 Peparing statement Thrift:1
127.0.0.1 949 Determining replicas for mutation Thrift:1
127.0.0.1 3577Acquiring switchLock read lock MutationStage:1
127.0.0.1 3622Appending to commitlog MutationStage:1
127.0.0.1 3754Adding to test memtable MutationStage:1
127.0.0.1 4936Sending message to /127.0.0.2 WRITE-/127.0.0.2
127.0.0.2 37 Message received from /127.0.0.1Thread-4
127.0.0.2 562 Acquiring switchLock read lock MutationStage:1
127.0.0.2 583 Appending to commitlog MutationStage:1
127.0.0.1 32 Message received from /127.0.0.2Thread-7
127.0.0.2 634 Adding to test memtable MutationStage:1
127.0.0.2 1033Enqueuing response to /127.0.0.1MutationStage:1
127.0.0.2 1217Sending message to /127.0.0.1 WRITE-/127.0.0.1
127.0.0.1 132 Processing response from /127.0.0.2
RequestResponseStage:3
{code}
With a replication factor of 3 it should be writing to all three nodes
(127.0.0.1, 127.0.0.2, and 127.0.0.3) - 127.0.0.3 is conspicuously missing from
that trace.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6, 2.0 beta 1
Reporter: Ryan McGuire
Attachments: 5668-logs.tar.gz, 5668_npe_ddl.cql, 5668_npe_insert.cql
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13688530#comment-13688530
]
Ryan McGuire commented on CASSANDRA-5668:
-
I ran with the patch, but I did not see any assertion errors.
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6, 2.0 beta 1
Reporter: Ryan McGuire
Attachments: 5668-assert.txt, 5668-logs.tar.gz, 5668_npe_ddl.cql,
5668_npe_insert.cql
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13688569#comment-13688569
]
Jonathan Ellis commented on CASSANDRA-5668:
---
So, .2 is sending two messages for this session, and the first is deleting the
session when it's done. (We know that the session is getting created correctly
since the first patch, that checks at message send time, works fine.)
The part I don't understand is, why two messages from .2? There should only be
one (responding to .1).
(This could be problematic for cross-dc replication, although so far tracing
seems to be working for CASSANDRA-5632. But for same-dc, one message per
replica should be straightforward.)
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6, 2.0 beta 1
Reporter: Ryan McGuire
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-logs.tar.gz,
5668_npe_ddl.cql, 5668_npe_insert.cql, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13688581#comment-13688581
]
Jonathan Ellis commented on CASSANDRA-5668:
---
Are we actually creating multiple sessions?
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6, 2.0 beta 1
Reporter: Ryan McGuire
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-logs.tar.gz,
5668_npe_ddl.cql, 5668_npe_insert.cql, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5668) NPE in net.OutputTcpConnection when tracing is enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13688866#comment-13688866
]
Jonathan Ellis commented on CASSANDRA-5668:
---
Okay, here's what's happening:
{noformat}
INFO [Thrift:1] 2013-06-19 23:36:51,719 Tracing.java (line 176) session
0702a620-d963-11e2-832d-53376523a4a2 is complete
java.lang.AssertionError: Asked to trace TYPE:MUTATION VERB:MUTATION for
session 0702a620-d963-11e2-832d-53376523a4a2 but that state does not exist
{noformat}
cqlsh is requesting QUORUM CL (or ONE?) so once that's achieved the coordinator
sends success to the client and closes the tracing session.
if other messages have not yet gone out, then we error.
But it gets worse...
Once the coordinator's state is discarded, any late-arriving replies will
create a new, non-local session. Since the coordinator will not send any
messages again for this session -- which is the trigger we use on replicas to
indicate we're done -- the nonlocal session will persist indefinitely,
leaking memory.
I think we can solve both of these:
# Make a static TraceState method that only needs the sessionid to be passed in
to log an event. OTC can use this to avoid having to look up tracestate at
all; if it's cleared out, not a problem.
# Make Tracing.sessions an expiring map so sessions we don't clean up manually
still get removed
Alternatively we could just go with #2 by itself and not try to cleanup
manually at all. Average case memory used will be worse, but maybe that is
okay since we assume only a tiny fraction of requests are traced at all.
What do you think [~slebresne]?
NPE in net.OutputTcpConnection when tracing is enabled
--
Key: CASSANDRA-5668
URL: https://issues.apache.org/jira/browse/CASSANDRA-5668
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6, 2.0 beta 1
Reporter: Ryan McGuire
Attachments: 5668-assert-2.txt, 5668-assert.txt, 5668-logs.tar.gz,
5668_npe_ddl.cql, 5668_npe_insert.cql, system.log
I get multiple NullPointerException when trying to trace INSERT statements.
To reproduce:
{code}
$ ccm create -v git:trunk
$ ccm populate -n 3
$ ccm start
$ ccm node1 cqlsh 5668_npe_ddl.cql
$ ccm node1 cqlsh 5668_npe_insert.cql
{code}
And see many exceptions like this in the logs of node1:
{code}
ERROR [WRITE-/127.0.0.3] 2013-06-19 14:54:35,885 OutboundTcpConnection.java
(line 197) error writing to /127.0.0.3
java.lang.NullPointerException
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:182)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:144)
{code}
This is similar to CASSANDRA-5658 and is the reason that npe_ddl and
npe_insert are separate files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
