[ https://issues.apache.org/jira/browse/CASSANDRA-16533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17309635#comment-17309635 ]
Alexandre Dutra commented on CASSANDRA-16533: --------------------------------------------- If that's all it takes, then the PR is ready. CircleCI doesn't love me much (and likewise) but I think I've got the jobs running here: * [CircleCI java8|https://app.circleci.com/pipelines/github/adutra/cassandra/6/workflows/ff748295-a629-473f-8658-a6327ec9023a] * [CircleCI java11|https://app.circleci.com/pipelines/github/adutra/cassandra/6/workflows/5ef9459f-387c-4488-a06c-80f4a06c36ee] > The two tests in NetstatsBootstrapWithEntireSSTablesCompressionStreamingTest > are flaky > --------------------------------------------------------------------------------------- > > Key: CASSANDRA-16533 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16533 > Project: Cassandra > Issue Type: Bug > Components: CI > Reporter: Ekaterina Dimitrova > Assignee: Alexandre Dutra > Priority: Normal > Fix For: 4.0-rc > > > The issue is when there are interleaving logs, then the filter > [here|https://github.com/apache/cassandra/blob/trunk/test/distributed/org/apache/cassandra/distributed/test/AbstractNetstatsStreaming.java#L142-L148] > doesn't help. > I managed to reproduce the issue only once. > But I found logs from old CircleCI > [run|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/705/workflows/c3534f6a-071e-428c-9a4d-0a1e08f23257/jobs/3866/artifacts]. > Please see > [logs/org.apache.cassandra.distributed.test.NetstatsBootstrapWithoutEntireSSTablesCompressionStreamingTest/<main>/<main>/system.log|https://3866-218559411-gh.circle-artifacts.com/0/logs/org.apache.cassandra.distributed.test.NetstatsBootstrapWithoutEntireSSTablesCompressionStreamingTest/%3Cmain%3E/%3Cmain%3E/system.log] > The excerpt that shows what I am talking about: > > {code:java} > DEBUG [node1_NettyStreaming-Outbound-/127.0.0.2.7012:1] node1 2021-03-17 > 22:41:21,057 CassandraCompressedStreamWriter.java:102 - [Stream > #e3d93810-8771-11eb-a0f1-f756ef192fe7] Finished streaming file > /tmp/dtests9118120876279144235/node1/data1/netstats_test/test_table-c0f7eda0877111ebbfc147ef604f6105/na-11-big-Data.db > to /127.0.0.2:7012, bytesTransferred = 66.646KiB, totalSize = 66.646KiB > Bootstrap e3d93810-8771-11eb-a0f1-f756ef192fe7 > /127.0.0.2 > Sending 15 files, 1339433 bytes total. Already sent 3 files (20.00%), 197286 > bytes total (14.73%) > DEBUG [Stream-Deserializer-/127.0.0.1:7012-2bcb5810] node2 2021-03-17 > 22:41:21,057 StreamingInboundHandler.java:187 - [Stream > #e3d93810-8771-11eb-a0f1-f756ef192fe7 channel: 2bcb5810] Received > IncomingStreamMessage{header=Header (tableId: > c0f7eda0-8771-11eb-bfc1-47ef604f6105, #1, repairedAt: 0, pendingRepair: null, > sendByFollower: true), > stream=CassandraIncomingFile{sstable=netstats_test/test_table}} > DEBUG [Stream-Deserializer-/127.0.0.1:7012-2bcb5810] node2 2021-03-17 > 22:41:21,058 NettyStreamingMessageSender.java:258 - [Stream > #e3d93810-8771-11eb-a0f1-f756ef192fe7 channel: 09fdf87e] Sending Received > (c0f7eda0-8771-11eb-bfc1-47ef604f6105, #1) > > /tmp/dtests9118120876279144235/node1/data1/system_auth/roles-5bc52802de2535edaeab188eecebb090/na-2-big-Data.db > 102/102 bytes (100%) sent to idx:0/127.0.0.2 > > /tmp/dtests9118120876279144235/node1/data1/netstats_test/test_table-c0f7eda0877111ebbfc147ef604f6105/na-11-big-Data.db > 65536/68246 bytes (96%) sent to idx:0/127.0.0.2 > > /tmp/dtests9118120876279144235/node1/data1/netstats_test/test_table-c0f7eda0877111ebbfc147ef604f6105/na-20-big-Data.db > 62836/62836 bytes (100%) sent to idx:0/127.0.0.2 > > /tmp/dtests9118120876279144235/node1/data1/netstats_test/test_table-c0f7eda0877111ebbfc147ef604f6105/na-17-big-Data.db > 68812/68812 bytes (100%) sent to idx:0/127.0.0.2 > Read Repair Statistics: > Attempted: 0 > Mismatch (Blocking): 0 > Mismatch (Background): 0 > Pool Name DEBUG [node1_NettyStreaming-Outbound-/127.0.0.2.7012:1] node1 > 2021-03-17 22:41:21,058 CassandraCompressedStreamWriter.java:64 - [Stream > #e3d93810-8771-11eb-a0f1-f756ef192fe7] Start streaming file > /tmp/dtests9118120876279144235/node1/data1/netstats_test/test_table-c0f7eda0877111ebbfc147ef604f6105/na-8-big-Data.db > to /127.0.0.2:7012, repairedAt = 0, totalSize = 67504 > Active PendingDEBUG [Stream-Deserializer-/127.0.0.1:7012-2bcb5810] node2 > 2021-03-17 22:41:21,058 StreamReceiveTask.java:88 - received 2 of 14 total > files, 131648 of total bytes 68812 > Completed Dropped > Large messages n/a 0 0 0 > Small messages n/a 0 1 0 > DEBUG [node1_NettyStreaming-Outbound-/127.0.0.2.7012:1] node1 2021-03-17 > 22:41:21,058 CassandraCompressedStreamWriter.java:81 - [Stream > #e3d93810-8771-11eb-a0f1-f756ef192fe7] Writing section 0 with length 67504 to > stream. > Gossip messages n/a 0 54 0 > INFO [pool-2-thread-2] <main> 2021-03-17 22:41:21,567 /127.0.0.1:7012 Mode: > NORMAL > > {code} > The line _Pool Name Active Pending Completed Dropped_ is split into three > parts due to DEBUG logs interleaving and the filter doesn't remove _Completed > Dropped_ which is added to the results > [here|https://github.com/apache/cassandra/blob/trunk/test/distributed/org/apache/cassandra/distributed/test/AbstractNetstatsStreaming.java#L179] > which leads to the following [exception > |https://github.com/apache/cassandra/blob/d656f8ac012f4577d22ed7bd3db94c15ae8eb5a9/test/distributed/org/apache/cassandra/distributed/test/AbstractNetstatsStreaming.java#L46] > CC [~yifanc] as he was working on CASSANDRA-16057 > and [~stefan.miklosovic] as the original author of these tests, if you guys > know something I am missing here > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org