[jira] [Updated] (CASSANDRA-14747) Test Messaging Refactor with: 200 node, compression=none, encryption=none, coalescing=off
[ https://issues.apache.org/jira/browse/CASSANDRA-14747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joey Lynch updated CASSANDRA-14747: --- Fix Version/s: (was: 4.0-beta3) 4.0-rc > Test Messaging Refactor with: 200 node, compression=none, encryption=none, > coalescing=off > -- > > Key: CASSANDRA-14747 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14747 > Project: Cassandra > Issue Type: Sub-task > Components: Legacy/Testing >Reporter: Joey Lynch >Assignee: Joey Lynch >Priority: Normal > Fix For: 4.0-rc > > Attachments: 3.0.17-QPS.png, 4.0.1-QPS.png, > 4.0.11-after-jolynch-tweaks.svg, 4.0.12-after-unconditional-flush.svg, > 4.0.15-after-sndbuf-fix.svg, 4.0.7-before-my-changes.svg, > 4.0_errors_showing_heap_pressure.txt, > 4.0_heap_histogram_showing_many_MessageOuts.txt, > i-0ed2acd2dfacab7c1-after-looping-fixes.svg, > trunk_14503_v2_cpuflamegraph.svg, trunk_vs_3.0.17_latency_under_load.png, > ttop_NettyOutbound-Thread_spinning.txt, > useast1c-i-0e1ddfe8b2f769060-mutation-flame.svg, > useast1e-i-08635fa1631601538_flamegraph_96node.svg, > useast1e-i-08635fa1631601538_ttop_netty_outbound_threads_96nodes, > useast1e-i-08635fa1631601538_uninlinedcpuflamegraph.0_96node_60sec_profile.svg > > > Tracks evaluating a 200 node cluster with all internode settings off (no > compression, no encryption, no coalescing). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14764) Test Messaging Refactor with: 12 Node Breaking Point, compression=none, encryption=none, coalescing=off
[
https://issues.apache.org/jira/browse/CASSANDRA-14764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joey Lynch updated CASSANDRA-14764:
---
Fix Version/s: (was: 4.0.x)
4.0-rc
> Test Messaging Refactor with: 12 Node Breaking Point, compression=none,
> encryption=none, coalescing=off
> ---
>
> Key: CASSANDRA-14764
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14764
> Project: Cassandra
> Issue Type: Sub-task
> Components: Legacy/Streaming and Messaging
>Reporter: Joey Lynch
>Assignee: Vinay Chella
>Priority: Normal
> Fix For: 4.0-rc
>
> Attachments: i-03341e1c52de6ea3e-after-queue-change.svg,
> i-07cd92e844d66d801-after-queue-bound.svg, i-07cd92e844d66d801-hint-play.svg,
> i-07cd92e844d66d801-uninlined-with-jvm-methods.svg, ttop.txt
>
>
> *Setup:*
> * Cassandra: 12 (2*6) node i3.xlarge AWS instance (4 cpu cores, 30GB ram)
> running cassandra trunk off of jasobrown/14503 jdd7ec5a2 (Jasons patched
> internode messaging branch) vs the same footprint running 3.0.17
> * Two datacenters with 100ms latency between them
> * No compression, encryption, or coalescing turned on
> *Test #1:*
> ndbench sent 1.5k QPS at a coordinator level to one datacenter (RF=3*2 = 6 so
> 3k global replica QPS) of 4kb single partition BATCH mutations at LOCAL_ONE.
> This represents about 250 QPS per coordinator in the first datacenter or 60
> QPS per core. The goal was to observe P99 write and read latencies under
> various QPS.
> *Result:*
> The good news is since the CASSANDRA-14503 changes, instead of keeping the
> mutations on heap we put the message into hints instead and don't run out of
> memory. The bad news is that the {{MessagingService-NettyOutbound-Thread's}}
> would occasionally enter a degraded state where they would just spin on a
> core. I've attached flame graphs showing the CPU state as [~jasobrown]
> applied fixes to the {{OutboundMessagingConnection}} class.
> *Follow Ups:*
> [~jasobrown] has committed a number of fixes onto his
> {{jasobrown/14503-collab}} branch including:
> 1. Limiting the amount of time spent dequeuing messages if they are expired
> (previously if messages entered the queue faster than we could dequeue them
> we'd just inifinte loop on the consumer side)
> 2. Don't call {{dequeueMessages}} from within {{dequeueMessages}} created
> callbacks.
> We're continuing to use CPU flamegraphs to figure out where we're looping and
> fixing bugs as we find them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14747) Test Messaging Refactor with: 200 node, compression=none, encryption=none, coalescing=off
[ https://issues.apache.org/jira/browse/CASSANDRA-14747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joey Lynch updated CASSANDRA-14747: --- Fix Version/s: (was: 4.0.x) 4.0-beta > Test Messaging Refactor with: 200 node, compression=none, encryption=none, > coalescing=off > -- > > Key: CASSANDRA-14747 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14747 > Project: Cassandra > Issue Type: Sub-task > Components: Legacy/Testing >Reporter: Joey Lynch >Assignee: Joey Lynch >Priority: Normal > Fix For: 4.0-beta > > Attachments: 3.0.17-QPS.png, 4.0.1-QPS.png, > 4.0.11-after-jolynch-tweaks.svg, 4.0.12-after-unconditional-flush.svg, > 4.0.15-after-sndbuf-fix.svg, 4.0.7-before-my-changes.svg, > 4.0_errors_showing_heap_pressure.txt, > 4.0_heap_histogram_showing_many_MessageOuts.txt, > i-0ed2acd2dfacab7c1-after-looping-fixes.svg, > trunk_14503_v2_cpuflamegraph.svg, trunk_vs_3.0.17_latency_under_load.png, > ttop_NettyOutbound-Thread_spinning.txt, > useast1c-i-0e1ddfe8b2f769060-mutation-flame.svg, > useast1e-i-08635fa1631601538_flamegraph_96node.svg, > useast1e-i-08635fa1631601538_ttop_netty_outbound_threads_96nodes, > useast1e-i-08635fa1631601538_uninlinedcpuflamegraph.0_96node_60sec_profile.svg > > > Tracks evaluating a 200 node cluster with all internode settings off (no > compression, no encryption, no coalescing). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14747) Test Messaging Refactor with: 200 node, compression=none, encryption=none, coalescing=off
[ https://issues.apache.org/jira/browse/CASSANDRA-14747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joey Lynch updated CASSANDRA-14747: --- Fix Version/s: (was: 4.0-beta) 4.0-beta3 > Test Messaging Refactor with: 200 node, compression=none, encryption=none, > coalescing=off > -- > > Key: CASSANDRA-14747 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14747 > Project: Cassandra > Issue Type: Sub-task > Components: Legacy/Testing >Reporter: Joey Lynch >Assignee: Joey Lynch >Priority: Normal > Fix For: 4.0-beta3 > > Attachments: 3.0.17-QPS.png, 4.0.1-QPS.png, > 4.0.11-after-jolynch-tweaks.svg, 4.0.12-after-unconditional-flush.svg, > 4.0.15-after-sndbuf-fix.svg, 4.0.7-before-my-changes.svg, > 4.0_errors_showing_heap_pressure.txt, > 4.0_heap_histogram_showing_many_MessageOuts.txt, > i-0ed2acd2dfacab7c1-after-looping-fixes.svg, > trunk_14503_v2_cpuflamegraph.svg, trunk_vs_3.0.17_latency_under_load.png, > ttop_NettyOutbound-Thread_spinning.txt, > useast1c-i-0e1ddfe8b2f769060-mutation-flame.svg, > useast1e-i-08635fa1631601538_flamegraph_96node.svg, > useast1e-i-08635fa1631601538_ttop_netty_outbound_threads_96nodes, > useast1e-i-08635fa1631601538_uninlinedcpuflamegraph.0_96node_60sec_profile.svg > > > Tracks evaluating a 200 node cluster with all internode settings off (no > compression, no encryption, no coalescing). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211482#comment-17211482
]
Caleb Rackliffe edited comment on CASSANDRA-15299 at 10/10/20, 12:28 AM:
-
[~samt] Here are the detailed/tactical points from my first pass at review.
Some of these items, especially the testing bits, are things I could
potentially branch and work on a bit myself, so let me know if you'd like a
hand. I still want to make one more top-down pass at things Monday now that the
smaller details are clearer.
*Correctness*
- {{AbstractMessageHandler}}
-- Are we incrementing {{receivedCount}} both on the first frame of a large
message (in {{processFirstFrameOfLargeMessage()}}) and on the last frame (in
{{processSubsequentFrameOfLargeMessage()}})?
-- {{forceOverAllocation()}} only allocates enough to hit the limit, but is it
possible for the subsequent release to de-allocate the full frame size?
- {{Flusher}} uses a {{PayloadAllocator}} from either the CRC or LZ4 encoder.
CRC includes the header and trailer size when it hits the {{BufferPool}}, while
LZ4 uses the {{Payload}} constructor that doesn't take those into account. Is
this correct?
- {{PostV5ExceptionHandler}} - in {{exceptionCaught()}}, does
{{payload.release()}} need to be in a {{finally}} block?
- {{CQLMessageHandler}} - should {{processOneContainedMessage()}} release the
frame in a {{finally}} block?
- It isn't really a correctness issue per-se, but {{FrameSet}} would feel safer
if we used composition rather than inheritance. (Was the motivator not creating
the additional object?)
*Testing*
- {{CQLTester}} - It might be a little awkward to follow the semantics of
"require" -> "prepare" -> "initialize". Maybe we should just inline
{{prepareNetwork()}}.
- The large frame accumulation state machine might be a good unit test target.
Lowest level might be a test-only subclass of
{{AbstractMessageHandler.LargeMessage}} that simulates the cases we're
interested in. Another would be to go one level up and test the
{{CQLMessageHandler.LargeMessage}}, but that pulls all of {{CQLMessageHandler}}
in, as it's not a static class. (We could make it static, create a little
interface to abstract {{CQLMessageHandler}} away, etc.)
- It might be worth pulling up {{WaitQueue}} (which is already static) as a
top-level class, then throw a few unit-level tests around it it.
- {{StartupMessage}} - There's enough logic that it might be nice to unit test
execute(). (Mocking {{Connection}} and checking replays, etc.)
*Documentation*
- Should describe {{native_transport_receive_queue_capacity_in_bytes}} in
{{cassandra.yaml}} and mention it in the Jira docs section?
- Is the plan to just merge the {{ql_protocol_V5_framing.asc}} content into
{{native_protocol_v5.spec}} when things solidify?
- {{cql_protocol_V5_framing.asc}} - worth mentioning somewhere there (or if
this ends up in the official V5 spec) that non-self-contained frames still only
hold part of one message and can't contain the end of one large message and the
start of another one
- {{OutboundMessageQueue}} - The race from CASSANDRA-15958 is fixed? (Noticed
the comment was removed...)
- {{InboundMessageHandler}} - The class-level JavaDoc is now almost identical
to {{AbstractMessageHandler}}, so we might want to narrow down to the places
where the former specializes. (ex. Only it uses, {{ProcessMessage}}.) The same
thing goes for the {{LargeMessage}} abstract class and its two sub-classes.
- {{CQLMessageHandler}} - could use brief class-level JavaDoc on how it extends
{{AbstractMessageHandler}}
- {{transport.Frame}} - brief class-level JavaDoc might help clarify its
scope/usage (even if it's renamed)
- Would class-level JavaDoc for {{transport.Dispatcher}} and
{{transport.Flusher}} be helpful. (ex. How do they interact with each other?)
- {{Payload#finish()}} - It might be useful to have a bit of detail in the
method JavaDoc explaining when we're expected to call it in the {{Payload}}
lifecycle.
- Do we need a deprecation plan for {{native_transport_frame_block_size_in_kb}}
...or not so much, because it was only added for 4.0 anyway?
*Zombies*
- {{transport.Frame.Header.BODY_LENGTH_SIZE}} is unused
- {{PasswordAuthenticatorTest}} - {{import java.net.InetSocketAddress}} is
unused
- {{OutboundConnection}} - {{import java.util.stream.Stream}} is unused
- {{StressSettings}} - {{import com.datastax.driver.core.Metadata}} and
{{import com.google.common.collect.ImmutableMap}} are unused
- {{FrameCompressor.LZ4Compressor}} - {{compress()}} doesn't need the
{{throws}} clause, and {{outputOffset + 0}} could be simplified
- {{ProtocolNegotiationTest}} - can remove the dead {{throws}} clauses from a
few tests
- {{ProtocolErrorTest}} - {{testErrorMessageWithNullString()}} doesn't need
throws clause
- {{CQLMessageHandler}} - {{UNKNOWN_STREAM_ID}} and
[jira] [Comment Edited] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211482#comment-17211482
]
Caleb Rackliffe edited comment on CASSANDRA-15299 at 10/10/20, 12:27 AM:
-
[~samt] Here are the detailed/tactical points from my first pass at review.
Some of these items, especially the testing bits, are things I could
potentially branch and work on a bit myself, so let me know if you'd like a
hand. I still want to make one more top-down pass at things Monday now that the
smaller details are clearer.
*Correctness*
- {{AbstractMessageHandler}}
-- Are we incrementing {{receivedCount}} both on the first frame of a large
message (in {{processFirstFrameOfLargeMessage()}}) and on the last frame (in
{{processSubsequentFrameOfLargeMessage()}})?
-- {{forceOverAllocation()}} only allocates enough to hit the limit, but is it
possible for the subsequent release to de-allocate the full frame size?
- {{Flusher}} uses a {{PayloadAllocator}} from either the CRC or LZ4 encoder.
CRC includes the header and trailer size when it hits the {{BufferPool}}, while
LZ4 uses the {{Payload}} constructor that doesn't take those into account. Is
this correct?
- {{PostV5ExceptionHandler}} - in {{exceptionCaught()}}, does
{{payload.release()}} need to be in a {{finally}} block?
- {{CQLMessageHandler}} - should {{processOneContainedMessage()}} release the
frame in a {{finally}} block?
- It isn't really a correctness issue per-se, but {{FrameSet}} would feel safer
if we used composition rather than inheritance. (Was the motivator not creating
the additional object?)
*Testing*
- {{CQLTester}} - It might be a little awkward to follow the semantics of
"require" -> "prepare" -> "initialize". Maybe we should just inline
{{prepareNetwork()}}.
- The large frame accumulation state machine might be a good unit test target.
Lowest level might be a test-only subclass of
{{AbstractMessageHandler.LargeMessage}} that simulates the cases we're
interested in. Another would be to go one level up and test the
{{CQLMessageHandler.LargeMessage}}, but that pulls all of {{CQLMessageHandler}}
in, as it's not a static class. (We could make it static, create a little
interface to abstract {{CQLMessageHandler}} away, etc.)
- It might be worth pulling up {{WaitQueue}} (which is already static) as a
top-level class, then throw a few unit-level tests around it it.
- {{StartupMessage}} - There's enough logic that it might be nice to unit test
execute(). (Mocking {{Connection}} and checking replays, etc.)
*Documentation*
- Should describe {{native_transport_receive_queue_capacity_in_bytes}} in
{{cassandra.yaml}} and mention it in the Jira docs section?
- Is the plan to just merge the {{ql_protocol_V5_framing.asc}} content into
{{native_protocol_v5.spec}} when things solidify?
- {{cql_protocol_V5_framing.asc}} - worth mentioning somewhere there (or if
this ends up in the official V5 spec) that non-self-contained frames still only
hold part of one message and can't contain the end of one large message and the
start of another one
- {{OutboundMessageQueue}} - The race from CASSANDRA-15958 is fixed? (Noticed
the comment was removed...)
- {{InboundMessageHandler}} - The class-level JavaDoc is now almost identical
to {{AbstractMessageHandler}}, so we might want to narrow down to the places
where the former specializes. (ex. Only it uses, {{ProcessMessage}}.) The same
thing goes for the {{LargeMessage}} abstract class and its two sub-classes.
- {{CQLMessageHandler}} - could use brief class-level JavaDoc on how it extends
{{AbstractMessageHandler}}
- {{transport.Frame}} - brief class-level JavaDoc might help clarify its
scope/usage (even if it's renamed)
- Would class-level JavaDoc for {{transport.Dispatcher}} and
{{transport.Flusher}} be helpful. (ex. How do they interact with each other?)
- {{Payload#finish()}} - It might be useful to have a bit of detail in the
method JavaDoc explaining when we're expected to call it in the {{Payload}}
lifecycle.
- Do we need a deprecation plan for {{native_transport_frame_block_size_in_kb}}
...or not so much, because it was only added for 4.0 anyway?
*Zombies*
- {{transport.Frame.Header.BODY_LENGTH_SIZE}} is unused
- {{PasswordAuthenticatorTest}} - {{import java.net.InetSocketAddress}} is
unused
- {{OutboundConnection}} - {{import java.util.stream.Stream}} is unused
- {{StressSettings}} - {{import com.datastax.driver.core.Metadata}} and
{{import com.google.common.collect.ImmutableMap}} are unused
- {{FrameCompressor.LZ4Compressor}} - {{compress()}} doesn't need the
{{throws}} clause, and {{outputOffset + 0}} could be simplified
- {{ProtocolNegotiationTest}} - can remove the dead {{throws}} clauses from a
few tests
- {{ProtocolErrorTest}} - {{testErrorMessageWithNullString()}} doesn't need
throws clause
- {{CQLMessageHandler}} - {{UNKNOWN_STREAM_ID}} and
[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211482#comment-17211482
]
Caleb Rackliffe commented on CASSANDRA-15299:
-
[~samt] Here are the detailed/tactical points from my first pass at review.
Some of these items, especially the testing bits, are things I could
potentially branch and work on a bit myself, so let me know if you'd like a
hand. I still want to make one more top-down pass at things Monday now that the
smaller details are clearer.
*Correctness*
- {{AbstractMessageHandler}}
-- Are we incrementing {{receivedCount}} both on the first frame of a large
message (in {{processFirstFrameOfLargeMessage()}}) and on the last frame (in
{{processSubsequentFrameOfLargeMessage()}})?
-- {{forceOverAllocation()}} only allocates enough to hit the limit, but is it
possible for the subsequent release to de-allocate the full frame size?
- {{Flusher}} uses a {{PayloadAllocator}} from either the CRC or LZ4 encoder.
CRC includes the header and trailer size when it hits the {{BufferPool}}, while
LZ4 uses the {{Payload}} constructor that doesn't take those into account. Is
this correct?
- {{PostV5ExceptionHandler}} - in {{exceptionCaught()}}, does
{{payload.release()}} need to be in a {{finally}} block?
- {{CQLMessageHandler}} - should {{processOneContainedMessage()}} release the
frame in a {{finally}} block?
- It isn't really a correctness issue per-se, but {{FrameSet}} would feel safer
if we used composition rather than inheritance. (Was the motivator not creating
the additional object?)
*Testing*
- {{CQLTester}} - It might be a little awkward to follow the semantics of
"require" -> "prepare" -> "initialize". Maybe we should just inline
{{prepareNetwork()}}.
- The large frame accumulation state machine might be a good unit test target.
Lowest level might be a test-only subclass of
{{AbstractMessageHandler.LargeMessage}} that simulates the cases we're
interested in. Another would be to go one level up and test the
{{CQLMessageHandler.LargeMessage}, but that pulls all of {{CQLMessageHandler}}
in, as it's not a static class. (We could make it static, create a little
interface to abstract {{CQLMessageHandler}} away, etc.)
- It might be worth pulling up {{WaitQueue}} (which is already static) as a
top-level class, then throw a few unit-level tests around it it.
- {{StartupMessage}} - There's enough logic that it might be nice to unit test
execute(). (Mocking {{Connection}} and checking replays, etc.)
*Documentation*
- Should describe {{native_transport_receive_queue_capacity_in_bytes}} in
{{cassandra.yaml}} and mention it in the Jira docs section?
- Is the plan to just merge the {{ql_protocol_V5_framing.asc}} content into
{{native_protocol_v5.spec}} when things solidify?
- {{cql_protocol_V5_framing.asc}} - worth mentioning somewhere there (or if
this ends up in the official V5 spec) that non-self-contained frames still only
hold part of one message and can't contain the end of one large message and the
start of another one
- {{OutboundMessageQueue}} - The race from CASSANDRA-15958 is fixed? (Noticed
the comment was removed...)
- {{InboundMessageHandler}} - The class-level JavaDoc is now almost identical
to {{AbstractMessageHandler}}, so we might want to narrow down to the places
where the former specializes. (ex. Only it uses, {{ProcessMessage}}.) The same
thing goes for the {{LargeMessage}} abstract class and its two sub-classes.
- {{CQLMessageHandler}} - could use brief class-level JavaDoc on how it extends
{{AbstractMessageHandler}}
- {{transport.Frame}} - brief class-level JavaDoc might help clarify its
scope/usage (even if it's renamed)
- Would class-level JavaDoc for {{transport.Dispatcher}} and
{{transport.Flusher}} be helpful. (ex. How do they interact with each other?)
- {{Payload#finish()}} - It might be useful to have a bit of detail in the
method JavaDoc explaining when we're expected to call it in the {{Payload}}
lifecycle.
- Do we need a deprecation plan for {{native_transport_frame_block_size_in_kb}}
...or not so much, because it was only added for 4.0 anyway?
*Zombies*
- {{transport.Frame.Header.BODY_LENGTH_SIZE}} is unused
- {{PasswordAuthenticatorTest}} - {{import java.net.InetSocketAddress}} is
unused
- {{OutboundConnection}} - {{import java.util.stream.Stream}} is unused
- {{StressSettings}} - {{import com.datastax.driver.core.Metadata}} and
{{import com.google.common.collect.ImmutableMap}} are unused
- {{FrameCompressor.LZ4Compressor}} - {{compress()}} doesn't need the
{{throws}} clause, and {{outputOffset + 0}} could be simplified
- {{ProtocolNegotiationTest}} - can remove the dead {{throws}} clauses from a
few tests
- {{ProtocolErrorTest}} - {{testErrorMessageWithNullString()}} doesn't need
throws clause
- {{CQLMessageHandler}} - {{UNKNOWN_STREAM_ID}} and {{import
org.apache.cassandra.net.Crc} are unused
-
[jira] [Commented] (CASSANDRA-15158) Wait for schema agreement rather than in flight schema requests when bootstrapping
[ https://issues.apache.org/jira/browse/CASSANDRA-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211469#comment-17211469 ] Stefan Miklosovic commented on CASSANDRA-15158: --- Thanks for this a lot! I ll review right at the beginning of the next week. > Wait for schema agreement rather than in flight schema requests when > bootstrapping > -- > > Key: CASSANDRA-15158 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15158 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip, Cluster/Schema >Reporter: Vincent White >Assignee: Blake Eggleston >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > Currently when a node is bootstrapping we use a set of latches > (org.apache.cassandra.service.MigrationTask#inflightTasks) to keep track of > in-flight schema pull requests, and we don't proceed with > bootstrapping/stream until all the latches are released (or we timeout > waiting for each one). One issue with this is that if we have a large schema, > or the retrieval of the schema from the other nodes was unexpectedly slow > then we have no explicit check in place to ensure we have actually received a > schema before we proceed. > While it's possible to increase "migration_task_wait_in_seconds" to force the > node to wait on each latche longer, there are cases where this doesn't help > because the callbacks for the schema pull requests have expired off the > messaging service's callback map > (org.apache.cassandra.net.MessagingService#callbacks) after > request_timeout_in_ms (default 10 seconds) before the other nodes were able > to respond to the new node. > This patch checks for schema agreement between the bootstrapping node and the > rest of the live nodes before proceeding with bootstrapping. It also adds a > check to prevent the new node from flooding existing nodes with simultaneous > schema pull requests as can happen in large clusters. > Removing the latch system should also prevent new nodes in large clusters > getting stuck for extended amounts of time as they wait > `migration_task_wait_in_seconds` on each of the latches left orphaned by the > timed out callbacks. > > ||3.11|| > |[PoC|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:check_for_schema]| > |[dtest|https://github.com/apache/cassandra-dtest/compare/master...vincewhite:wait_for_schema_agreement]| > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16144) TLS connections to the storage port on a node without server encryption configured causes java.io.IOException accessing missing keystore
[
https://issues.apache.org/jira/browse/CASSANDRA-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-16144:
--
Reviewers: David Capwell, Dinesh Joshi (was: Dinesh Joshi)
> TLS connections to the storage port on a node without server encryption
> configured causes java.io.IOException accessing missing keystore
>
>
> Key: CASSANDRA-16144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16144
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> If a TLS connection is requested against a node with all encryption disabled
> by configuration,
> configured with
> {code}
> server_encryption_options: {optional:false, internode_encryption: none}
> {code}
> it logs the following error if no keystore exists for the node.
> {code}
> INFO [Messaging-EventLoop-3-3] 2020-09-15T14:30:02,952 : -
> 127.0.0.1:7000->127.0.1.1:7000-URGENT_MESSAGES-[no-channel] failed to connect
> io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
> refused: local1-i1/127.0.1.1:7000
> Caused by: java.net.ConnectException: Connection refused
>at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
>at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779) ~[?:?]
>at
> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at java.lang.Thread.run(Thread.java:834) [?:?]
> WARN [Messaging-EventLoop-3-9] 2020-09-15T14:30:06,375 : - Failed to
> initialize a channel. Closing: [id: 0x0746c157, L:/127.0.0.1:7000 -
> R:/127.0.0.1:59623]
> java.io.IOException: failed to build trust manager store for secure
> connections
>at
> org.apache.cassandra.security.SSLFactory.buildKeyManagerFactory(SSLFactory.java:232)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.createNettySslContext(SSLFactory.java:300)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:276)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:257)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.net.InboundConnectionInitiator$Initializer.initChannel(InboundConnectionInitiator.java:107)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> org.apache.cassandra.net.InboundConnectionInitiator$Initializer.initChannel(InboundConnectionInitiator.java:71)
> ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at
> io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:129)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:112)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded(AbstractChannelHandlerContext.java:938)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
> io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:609)
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at
>
[jira] [Commented] (CASSANDRA-16127) NullPointerException when calling nodetool enablethrift
[
https://issues.apache.org/jira/browse/CASSANDRA-16127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211454#comment-17211454
]
David Capwell commented on CASSANDRA-16127:
---
Starting commit
CI Results (pending):
Branch: cassandra-2.2
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16127-cassandra-2.2-806833FE-D6C3-4556-A63B-8E1E7AC387F0
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/89/
Branch: cassandra-3.0
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16127-cassandra-3.0-806833FE-D6C3-4556-A63B-8E1E7AC387F0
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/90/
Branch: cassandra-3.11
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16127-cassandra-3.11-806833FE-D6C3-4556-A63B-8E1E7AC387F0
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/91/
Branch: trunk
Circle:
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16127-trunk-806833FE-D6C3-4556-A63B-8E1E7AC387F0
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/92/
> NullPointerException when calling nodetool enablethrift
> ---
>
> Key: CASSANDRA-16127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16127
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Thrift
>Reporter: Tibor Repasi
>Assignee: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x
>
>
> Having thrift disabled, it's impossible to enable it again without restarting
> the node:
> {code}
> $ nodetool statusthrift
> not running
> $ nodetool enablethrift
> error: null
> -- StackTrace --
> java.lang.NullPointerException
> at
> org.apache.cassandra.service.StorageService.startRPCServer(StorageService.java:392)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
> at java.security.AccessController.doPrivileged(Native Method)
> at
>
[jira] [Commented] (CASSANDRA-16152) In-JVM dtest - modify schema with stopped nodes and use yaml fragments for config
[
https://issues.apache.org/jira/browse/CASSANDRA-16152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211447#comment-17211447
]
David Capwell commented on CASSANDRA-16152:
---
For more context, here is the proposed change to the interface
{code}
.withConfig(c -> c.set("server_encryption_options", ImmutableMap.of(
"keystore", newKeystore,
"truststore", newTruststore)))
{code}
This aligns with the changes [~e.dimitrova] was working on in CASSANDRA-15234
and makes it so dtest and CassandraDaemon use the same config logic (which also
means config migration).
> In-JVM dtest - modify schema with stopped nodes and use yaml fragments for
> config
> -
>
> Key: CASSANDRA-16152
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16152
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
>
> Some convenience improvements to in-JVM dtest that are useful across versions
> that I needed while working on CASSANDRA-16144
> * Add support for changing schema with stopped nodes.
> * Make it simpler to modify nested configuration items by specifying Yaml
> fragments
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16152) In-JVM dtest - modify schema with stopped nodes and use yaml fragments for config
[ https://issues.apache.org/jira/browse/CASSANDRA-16152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211445#comment-17211445 ] David Capwell commented on CASSANDRA-16152: --- Review * https://github.com/apache/cassandra/compare/cassandra-2.2...jonmeredith:C16152-2.2#diff-e398a00672550f1911eb13e4d4aa86cbR486 - `coordinator(instance.config().num())` can be simplified to `instance.coordinator()` * the config changes I am against as they get in the way of other changes we have talked about, so I propose we just do the config change that was talked about before in the context of CASSANDRA-15234 I spoke to [~jmeredithco] about the second comment as its bigger, and worked on a patch to do it but it depends the the Shared annotation added in CASSANDRA-16127 > In-JVM dtest - modify schema with stopped nodes and use yaml fragments for > config > - > > Key: CASSANDRA-16152 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16152 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > > Some convenience improvements to in-JVM dtest that are useful across versions > that I needed while working on CASSANDRA-16144 > * Add support for changing schema with stopped nodes. > * Make it simpler to modify nested configuration items by specifying Yaml > fragments -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16152) In-JVM dtest - modify schema with stopped nodes and use yaml fragments for config
[ https://issues.apache.org/jira/browse/CASSANDRA-16152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16152: -- Status: Changes Suggested (was: Ready to Commit) > In-JVM dtest - modify schema with stopped nodes and use yaml fragments for > config > - > > Key: CASSANDRA-16152 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16152 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > > Some convenience improvements to in-JVM dtest that are useful across versions > that I needed while working on CASSANDRA-16144 > * Add support for changing schema with stopped nodes. > * Make it simpler to modify nested configuration items by specifying Yaml > fragments -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15158) Wait for schema agreement rather than in flight schema requests when bootstrapping
[ https://issues.apache.org/jira/browse/CASSANDRA-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211409#comment-17211409 ] Blake Eggleston commented on CASSANDRA-15158: - Ok, I've fixed all the dtest issues and ported the 3.11 fix to 3.0 and trunk. The 3.11 branch with misc fixes is here: https://github.com/bdeggleston/cassandra/tree/15158-3.11 Most of the fixes are self explanatory, the less obvious ones are: * wait for gossip to settle before waiting on schemas. The original patch accidentally removed the wait on the schema version to be non-empty, so this both a fix and a change. Waiting for gossip makes it more likely that we've seen all current schema versions before we begin waiting instead of just waiting for the first schema to be received, which was the effect of waiting on Schema.instance.isEmpty * don't check liveness in shouldPullFromEndpoint. There were cases where the node wasn't considered alive until after `reportEndpointVersion` had been called (because it happens as part of the current gossip update). * move migration start inside StorageService.joinRing because in-jvm dtests don't use CassandraDaemon * don't fail maybePullSchema if version info has no endpoints. We automatically call that after completing a pull, so a version will eventually have no endpoints left on it The squashed ports are here: | [3.0|https://github.com/bdeggleston/cassandra/tree/15158-3.0] | [circle|https://app.circleci.com/pipelines/github/bdeggleston/cassandra?branch=15158-3.0] | | [3.11|https://github.com/bdeggleston/cassandra/tree/15158-3.11-squashed] | [circle|https://app.circleci.com/pipelines/github/bdeggleston/cassandra?branch=15158-3.11-squashed] | | [trunk|https://github.com/bdeggleston/cassandra/tree/15158-trunk] | [circle|https://app.circleci.com/pipelines/github/bdeggleston/cassandra?branch=15158-trunk] | > Wait for schema agreement rather than in flight schema requests when > bootstrapping > -- > > Key: CASSANDRA-15158 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15158 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip, Cluster/Schema >Reporter: Vincent White >Assignee: Blake Eggleston >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > Currently when a node is bootstrapping we use a set of latches > (org.apache.cassandra.service.MigrationTask#inflightTasks) to keep track of > in-flight schema pull requests, and we don't proceed with > bootstrapping/stream until all the latches are released (or we timeout > waiting for each one). One issue with this is that if we have a large schema, > or the retrieval of the schema from the other nodes was unexpectedly slow > then we have no explicit check in place to ensure we have actually received a > schema before we proceed. > While it's possible to increase "migration_task_wait_in_seconds" to force the > node to wait on each latche longer, there are cases where this doesn't help > because the callbacks for the schema pull requests have expired off the > messaging service's callback map > (org.apache.cassandra.net.MessagingService#callbacks) after > request_timeout_in_ms (default 10 seconds) before the other nodes were able > to respond to the new node. > This patch checks for schema agreement between the bootstrapping node and the > rest of the live nodes before proceeding with bootstrapping. It also adds a > check to prevent the new node from flooding existing nodes with simultaneous > schema pull requests as can happen in large clusters. > Removing the latch system should also prevent new nodes in large clusters > getting stuck for extended amounts of time as they wait > `migration_task_wait_in_seconds` on each of the latches left orphaned by the > timed out callbacks. > > ||3.11|| > |[PoC|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:check_for_schema]| > |[dtest|https://github.com/apache/cassandra-dtest/compare/master...vincewhite:wait_for_schema_agreement]| > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16115) New Cassandra website design, content and layout to work with Antora
[ https://issues.apache.org/jira/browse/CASSANDRA-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211401#comment-17211401 ] Melissa Logan edited comment on CASSANDRA-16115 at 10/9/20, 9:04 PM: - [~flightc] Great to hear, glad you like. Doc should have been public, sorry about that (I run into trouble w/ this sometimes). Try this one: [https://docs.google.com/document/d/1O8Q61qIBUCse29cEb3OAfR258VJtPV62zkSgsIhsWXk/edit?usp=sharing] was (Author: mklogan): [~flightc] Great to hear, glad you like. Doc should have been public, sorry about that (I run into trouble w/ this sometimes). Try this one: [https://docs.google.com/document/d/e/2PACX-1vRmbzg6spuou11epVyG9NdK0XXr4Q5oqtfVD0nIQLNKeOY0PhK53mCebcWlV66strFqlywplc3lyOav/pub|https://docs.google.com/document/d/e/2PACX-1vRmbzg6spuou11epVyG9NdK0XXr4Q5oqtfVD0nIQLNKeOY0PhK53mCebcWlV66strFqlywplc3lyOav/pub.] > New Cassandra website design, content and layout to work with Antora > > > Key: CASSANDRA-16115 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16115 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Melissa Logan >Assignee: Melissa Logan >Priority: Normal > Fix For: 4.0-rc, 4.0-triage > > Attachments: Screen Shot 2020-09-03 at 09.48.53.png > > > This task is related to CASSANDRA-16066 (Update and rework the > cassandra-website material to work with Antora). The goal is to update the > front-end of the C* website (design, IA and content) to work with Antora to > help modernize the website as discussed on the [mailing > list|https://www.mail-archive.com/dev@cassandra.apache.org/msg15537.html]. > *Design Concepts:* A minimum of two homepage design concepts will be created > and shared for input, which will help standardize a brand palette for C* and > a design language for the site. This may include custom iconography and > graphics. The chosen design language will be used to develop the remaining > templates. > *Template Design*: It's estimated that 7 template designs will be needed > including the creation of several new pages: > * Homepage template > * Toplevel template - e.g. Community. > * General template - Mostly textual with some images, e.g. Intro, Quickstart > * “Library” template - A library of assets (links, downloads, logos etc) > that are sortable by metadata, e.g Resources, or Kafka's Powered By page). > * Blog landing template > * Blog single template > * Docs template > *Website Content:* Along with new design will be a need for new or updated > content to fit the new page layouts. The intention is to use as much as > possible from existing content, and augment with new content where needed. > *Template Development:* This includes the frontend development, such as any > HTML markup to achieve designs. HTML would be crafted so as to preserve any > backend/API calls, such that content is pulled in as designed. The majority > of the frontend work would come in the form of crafting CSS to bring the > designs to life, plus any minor Javascript to add subtle delights to key > pages. > *Style Guide*: Once all is complete, a Style Guide be added to GitHub for > contributors. > The [cassandra-website|https://github.com/apache/cassandra-website] > repository would need to be modified. Specific changes to be determined. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16115) New Cassandra website design, content and layout to work with Antora
[ https://issues.apache.org/jira/browse/CASSANDRA-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211401#comment-17211401 ] Melissa Logan edited comment on CASSANDRA-16115 at 10/9/20, 9:03 PM: - [~flightc] Great to hear, glad you like. Doc should have been public, sorry about that (I run into trouble w/ this sometimes). Try this one: [https://docs.google.com/document/d/e/2PACX-1vRmbzg6spuou11epVyG9NdK0XXr4Q5oqtfVD0nIQLNKeOY0PhK53mCebcWlV66strFqlywplc3lyOav/pub|https://docs.google.com/document/d/e/2PACX-1vRmbzg6spuou11epVyG9NdK0XXr4Q5oqtfVD0nIQLNKeOY0PhK53mCebcWlV66strFqlywplc3lyOav/pub.] was (Author: mklogan): [~flightc] Great to hear, glad you like. Doc should have been public, sorry about that (I run into trouble w/ this sometimes). Try this one: [https://docs.google.com/document/d/e/2PACX-1vRmbzg6spuou11epVyG9NdK0XXr4Q5oqtfVD0nIQLNKeOY0PhK53mCebcWlV66strFqlywplc3lyOav/pub|https://docs.google.com/document/d/e/2PACX-1vRmbzg6spuou11epVyG9NdK0XXr4Q5oqtfVD0nIQLNKeOY0PhK53mCebcWlV66strFqlywplc3lyOav/pub.] > New Cassandra website design, content and layout to work with Antora > > > Key: CASSANDRA-16115 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16115 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Melissa Logan >Assignee: Melissa Logan >Priority: Normal > Fix For: 4.0-rc, 4.0-triage > > Attachments: Screen Shot 2020-09-03 at 09.48.53.png > > > This task is related to CASSANDRA-16066 (Update and rework the > cassandra-website material to work with Antora). The goal is to update the > front-end of the C* website (design, IA and content) to work with Antora to > help modernize the website as discussed on the [mailing > list|https://www.mail-archive.com/dev@cassandra.apache.org/msg15537.html]. > *Design Concepts:* A minimum of two homepage design concepts will be created > and shared for input, which will help standardize a brand palette for C* and > a design language for the site. This may include custom iconography and > graphics. The chosen design language will be used to develop the remaining > templates. > *Template Design*: It's estimated that 7 template designs will be needed > including the creation of several new pages: > * Homepage template > * Toplevel template - e.g. Community. > * General template - Mostly textual with some images, e.g. Intro, Quickstart > * “Library” template - A library of assets (links, downloads, logos etc) > that are sortable by metadata, e.g Resources, or Kafka's Powered By page). > * Blog landing template > * Blog single template > * Docs template > *Website Content:* Along with new design will be a need for new or updated > content to fit the new page layouts. The intention is to use as much as > possible from existing content, and augment with new content where needed. > *Template Development:* This includes the frontend development, such as any > HTML markup to achieve designs. HTML would be crafted so as to preserve any > backend/API calls, such that content is pulled in as designed. The majority > of the frontend work would come in the form of crafting CSS to bring the > designs to life, plus any minor Javascript to add subtle delights to key > pages. > *Style Guide*: Once all is complete, a Style Guide be added to GitHub for > contributors. > The [cassandra-website|https://github.com/apache/cassandra-website] > repository would need to be modified. Specific changes to be determined. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16115) New Cassandra website design, content and layout to work with Antora
[ https://issues.apache.org/jira/browse/CASSANDRA-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211401#comment-17211401 ] Melissa Logan commented on CASSANDRA-16115: --- [~flightc] Great to hear, glad you like. Doc should have been public, sorry about that (I run into trouble w/ this sometimes). Try this one: [https://docs.google.com/document/d/e/2PACX-1vRmbzg6spuou11epVyG9NdK0XXr4Q5oqtfVD0nIQLNKeOY0PhK53mCebcWlV66strFqlywplc3lyOav/pub|https://docs.google.com/document/d/e/2PACX-1vRmbzg6spuou11epVyG9NdK0XXr4Q5oqtfVD0nIQLNKeOY0PhK53mCebcWlV66strFqlywplc3lyOav/pub.] > New Cassandra website design, content and layout to work with Antora > > > Key: CASSANDRA-16115 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16115 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Melissa Logan >Assignee: Melissa Logan >Priority: Normal > Fix For: 4.0-rc, 4.0-triage > > Attachments: Screen Shot 2020-09-03 at 09.48.53.png > > > This task is related to CASSANDRA-16066 (Update and rework the > cassandra-website material to work with Antora). The goal is to update the > front-end of the C* website (design, IA and content) to work with Antora to > help modernize the website as discussed on the [mailing > list|https://www.mail-archive.com/dev@cassandra.apache.org/msg15537.html]. > *Design Concepts:* A minimum of two homepage design concepts will be created > and shared for input, which will help standardize a brand palette for C* and > a design language for the site. This may include custom iconography and > graphics. The chosen design language will be used to develop the remaining > templates. > *Template Design*: It's estimated that 7 template designs will be needed > including the creation of several new pages: > * Homepage template > * Toplevel template - e.g. Community. > * General template - Mostly textual with some images, e.g. Intro, Quickstart > * “Library” template - A library of assets (links, downloads, logos etc) > that are sortable by metadata, e.g Resources, or Kafka's Powered By page). > * Blog landing template > * Blog single template > * Docs template > *Website Content:* Along with new design will be a need for new or updated > content to fit the new page layouts. The intention is to use as much as > possible from existing content, and augment with new content where needed. > *Template Development:* This includes the frontend development, such as any > HTML markup to achieve designs. HTML would be crafted so as to preserve any > backend/API calls, such that content is pulled in as designed. The majority > of the frontend work would come in the form of crafting CSS to bring the > designs to life, plus any minor Javascript to add subtle delights to key > pages. > *Style Guide*: Once all is complete, a Style Guide be added to GitHub for > contributors. > The [cassandra-website|https://github.com/apache/cassandra-website] > repository would need to be modified. Specific changes to be determined. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16113) Consolidate dead nodes check in force repair
[ https://issues.apache.org/jira/browse/CASSANDRA-16113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-16113: Source Control Link: https://github.com/apache/cassandra/commit/1728da30e4e7858d30178ef74350af3e690adf0c Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed, thanks [~yifanc]! > Consolidate dead nodes check in force repair > > > Key: CASSANDRA-16113 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16113 > Project: Cassandra > Issue Type: Improvement > Components: Local/Other >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 4h 10m > Remaining Estimate: 0h > > The check for dead nodes during force repair is duplicated in the normal and > incremental repair. We could consolidate those 2 checks to make the code more > dry. > The check should throw a more meaningful error message to indicate that all > neighbor nodes are down, instead of "java.lang.IllegalArgumentException: > Endpoints can not be empty" -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Consolidate node liveness check for forced repair
This is an automated email from the ASF dual-hosted git repository.
bdeggleston pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 1728da3 Consolidate node liveness check for forced repair
1728da3 is described below
commit 1728da30e4e7858d30178ef74350af3e690adf0c
Author: yifan-c
AuthorDate: Tue Sep 8 18:23:30 2020 -0700
Consolidate node liveness check for forced repair
Patch by Yifan Cai; Reviewed by Blake Eggleston for CASSANDRA-16113
---
CHANGES.txt| 1 +
.../org/apache/cassandra/repair/CommonRange.java | 21 ++--
.../apache/cassandra/repair/RepairRunnable.java| 121 +++--
.../org/apache/cassandra/repair/RepairSession.java | 38 +--
.../cassandra/service/ActiveRepairService.java | 3 +-
.../distributed/test/DistributedRepairUtils.java | 33 --
.../cassandra/distributed/test/RepairTest.java | 92 +++-
...nnableTest.java => NeighborsAndRangesTest.java} | 25 +++--
.../org/apache/cassandra/repair/RepairJobTest.java | 7 +-
.../apache/cassandra/repair/RepairSessionTest.java | 2 +-
10 files changed, 185 insertions(+), 158 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 289d4e8..412b336 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0-beta3
+ * Consolidate node liveness check for forced repair (CASSANDRA-16113)
* Use unsigned short in ValueAccessor.sliceWithShortLength (CASSANDRA-16147)
* Abort repairs when getting a truncation request (CASSANDRA-15854)
* Remove bad assert when getting active compactions for an sstable
(CASSANDRA-15457)
diff --git a/src/java/org/apache/cassandra/repair/CommonRange.java
b/src/java/org/apache/cassandra/repair/CommonRange.java
index dab77c5..5eb8061 100644
--- a/src/java/org/apache/cassandra/repair/CommonRange.java
+++ b/src/java/org/apache/cassandra/repair/CommonRange.java
@@ -21,6 +21,7 @@ package org.apache.cassandra.repair;
import java.util.ArrayList;
import java.util.Collection;
+import java.util.Objects;
import java.util.Set;
import com.google.common.base.Preconditions;
@@ -38,9 +39,15 @@ public class CommonRange
public final ImmutableSet endpoints;
public final ImmutableSet transEndpoints;
public final Collection> ranges;
+public final boolean hasSkippedReplicas;
public CommonRange(Set endpoints,
Set transEndpoints, Collection> ranges)
{
+this(endpoints, transEndpoints, ranges, false);
+}
+
+public CommonRange(Set endpoints,
Set transEndpoints, Collection> ranges,
boolean hasSkippedReplicas)
+{
Preconditions.checkArgument(endpoints != null && !endpoints.isEmpty(),
"Endpoints can not be empty");
Preconditions.checkArgument(transEndpoints != null, "Transient
endpoints can not be null");
Preconditions.checkArgument(endpoints.containsAll(transEndpoints),
"transEndpoints must be a subset of endpoints");
@@ -49,6 +56,7 @@ public class CommonRange
this.endpoints = ImmutableSet.copyOf(endpoints);
this.transEndpoints = ImmutableSet.copyOf(transEndpoints);
this.ranges = new ArrayList<>(ranges);
+this.hasSkippedReplicas = hasSkippedReplicas;
}
public boolean matchesEndpoints(Set endpoints,
Set transEndpoints)
@@ -64,17 +72,15 @@ public class CommonRange
CommonRange that = (CommonRange) o;
-if (!endpoints.equals(that.endpoints)) return false;
-if (!transEndpoints.equals(that.transEndpoints)) return false;
-return ranges.equals(that.ranges);
+return Objects.equals(endpoints, that.endpoints)
+ && Objects.equals(transEndpoints, that.transEndpoints)
+ && Objects.equals(ranges, that.ranges)
+ && hasSkippedReplicas == that.hasSkippedReplicas;
}
public int hashCode()
{
-int result = endpoints.hashCode();
-result = 31 * result + transEndpoints.hashCode();
-result = 31 * result + ranges.hashCode();
-return result;
+return Objects.hash(endpoints, transEndpoints, ranges,
hasSkippedReplicas);
}
public String toString()
@@ -83,6 +89,7 @@ public class CommonRange
"endpoints=" + endpoints +
", transEndpoints=" + transEndpoints +
", ranges=" + ranges +
+ ", hasSkippedReplicas=" + hasSkippedReplicas +
'}';
}
}
diff --git a/src/java/org/apache/cassandra/repair/RepairRunnable.java
b/src/java/org/apache/cassandra/repair/RepairRunnable.java
index f6aa6d1..5d8e945 100644
--- a/src/java/org/apache/cassandra/repair/RepairRunnable.java
+++ b/src/java/org/apache/cassandra/repair/RepairRunnable.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.repair;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.ArrayList;
+import
[jira] [Commented] (CASSANDRA-16205) Offline token allocation strategy generator tool
[
https://issues.apache.org/jira/browse/CASSANDRA-16205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211389#comment-17211389
]
Michael Semb Wever commented on CASSANDRA-16205:
Patch at
https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_16205
Unit tests soon…
> Offline token allocation strategy generator tool
>
>
> Key: CASSANDRA-16205
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16205
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config, Local/Scripts
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>
> A command line tool to generate tokens (using the
> allocate_tokens_for_local_replication_factor algorithm) for pre-configuration
> of {{initial_tokens}} in cassandra.yaml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16113) Consolidate dead nodes check in force repair
[ https://issues.apache.org/jira/browse/CASSANDRA-16113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-16113: Status: Ready to Commit (was: Changes Suggested) > Consolidate dead nodes check in force repair > > > Key: CASSANDRA-16113 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16113 > Project: Cassandra > Issue Type: Improvement > Components: Local/Other >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 4h 10m > Remaining Estimate: 0h > > The check for dead nodes during force repair is duplicated in the normal and > incremental repair. We could consolidate those 2 checks to make the code more > dry. > The check should throw a more meaningful error message to indicate that all > neighbor nodes are down, instead of "java.lang.IllegalArgumentException: > Endpoints can not be empty" -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211382#comment-17211382 ] Ekaterina Dimitrova commented on CASSANDRA-15897: - My personal preference would be to keep the work incremental - review and commit this work and open a new ticket on the other part. I can rebase and push CI on this one on Monday? > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Sylvain Lebresne >Priority: Normal > Fix For: 4.0-beta3 > > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-2848) Make the Client API support passing down timeouts
[ https://issues.apache.org/jira/browse/CASSANDRA-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-2848: - Status: Open (was: Patch Available) > Make the Client API support passing down timeouts > - > > Key: CASSANDRA-2848 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2848 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client, Messaging/Internode >Reporter: Chris Goffinet >Assignee: Yifan Cai >Priority: Low > Labels: protocolv5 > Fix For: 4.0-beta, 4.0-triage > > Attachments: 2848-trunk-v2.txt, 2848-trunk.txt > > > Having a max server RPC timeout is good for worst case, but many applications > that have middleware in front of Cassandra, might have higher timeout > requirements. In a fail fast environment, if my application starting at say > the front-end, only has 20ms to process a request, and it must connect to X > services down the stack, by the time it hits Cassandra, we might only have > 10ms. I propose we provide the ability to specify the timeout on each call we > do optionally. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14607) Explore optimizations in AbstractBtreePartiton, java 11 variant
[
https://issues.apache.org/jira/browse/CASSANDRA-14607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14607:
--
Complexity: Normal
> Explore optimizations in AbstractBtreePartiton, java 11 variant
> ---
>
> Key: CASSANDRA-14607
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14607
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Jason Brown
>Assignee: Blake Eggleston
>Priority: Normal
> Labels: Java11
> Fix For: 4.0, 4.0-alpha1
>
>
> In CASSANDRA-9608, we discussed some way to optimize the java 11
> implementation of {{AbstractBTreePartition}}. This ticket serves that
> purpose, as well as a "note to selves" to ensure the java 11 version does not
> have a performance regression.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13994) Remove dead compact storage code before 4.0 release
[
https://issues.apache.org/jira/browse/CASSANDRA-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-13994:
--
Complexity: Normal
> Remove dead compact storage code before 4.0 release
> ---
>
> Key: CASSANDRA-13994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13994
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Local Write-Read Paths
>Reporter: Alex Petrov
>Assignee: Ekaterina Dimitrova
>Priority: Low
> Fix For: 4.0-beta2
>
>
> 4.0 comes without thrift (after [CASSANDRA-5]) and COMPACT STORAGE (after
> [CASSANDRA-10857]), and since Compact Storage flags are now disabled, all of
> the related functionality is useless.
> There are still some things to consider:
> 1. One of the system tables (built indexes) was compact. For now, we just
> added {{value}} column to it to make sure it's backwards-compatible, but we
> might want to make sure it's just a "normal" table and doesn't have redundant
> columns.
> 2. Compact Tables were building indexes in {{KEYS}} mode. Removing it is
> trivial, but this would mean that all built indexes will be defunct. We could
> log a warning for now and ask users to migrate off those for now and
> completely remove it from future releases. It's just a couple of classes
> though.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14939) fix some operational holes in incremental repair
[ https://issues.apache.org/jira/browse/CASSANDRA-14939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-14939: -- Complexity: Normal > fix some operational holes in incremental repair > > > Key: CASSANDRA-14939 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14939 > Project: Cassandra > Issue Type: Bug >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 4.0 > > > Incremental repair has a few operational rough spots that make it more > difficult to fully automate and operate at scale than it should be. > * Visibility into whether pending repair data exists for a given token range. > * Ability to force promotion/demotion of data for completed sessions instead > of waiting for compaction. > * Get the most recent repairedAt timestamp for a given token range. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13935) Indexes and UDTs creation should have IF NOT EXISTS on its String representation
[
https://issues.apache.org/jira/browse/CASSANDRA-13935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-13935:
--
Complexity: Normal
> Indexes and UDTs creation should have IF NOT EXISTS on its String
> representation
>
>
> Key: CASSANDRA-13935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13935
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index, Legacy/CQL
> Environment: Ubuntu 16.04.2 LTS
> java version "1.8.0_144"
> Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
>Reporter: Javier Canillas
>Assignee: Stefan Miklosovic
>Priority: Low
> Fix For: 3.0.23, 3.11.9, 4.0-beta3
>
> Attachments: 13935-3.0.txt, 13935-3.11.txt, 13935-trunk.txt
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> I came across something that bothers me a lot. I'm using snapshots to backup
> data from my Cassandra cluster in case something really bad happens (like
> dropping a table or a keyspace).
> Exercising the recovery actions from those backups, I discover that the
> schema put on the file "schema.cql" as a result of the snapshot has the
> "CREATE IF NOT EXISTS" for the table, but not for the indexes.
> When restoring from snapshots, and relying on the execution of these schemas
> to build up the table structure, everything seems fine for tables without
> secondary indexes, but for the ones that make use of them, the execution of
> these statements fail miserably.
> Here I paste a generated schema.cql content for a table with indexes:
> CREATE TABLE IF NOT EXISTS keyspace1.table1 (
> id text PRIMARY KEY,
> content text,
> last_update_date date,
> last_update_date_time timestamp)
> WITH ID = f1045fc0-2f59-11e7-95ec-295c3c064920
> AND bloom_filter_fp_chance = 0.01
> AND dclocal_read_repair_chance = 0.1
> AND crc_check_chance = 1.0
> AND default_time_to_live = 864
> AND gc_grace_seconds = 864000
> AND min_index_interval = 128
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE'
> AND caching = { 'keys': 'NONE', 'rows_per_partition': 'NONE' }
> AND compaction = { 'max_threshold': '32', 'min_threshold': '4',
> 'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy' }
> AND compression = { 'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor' }
> AND cdc = false
> AND extensions = { };
> CREATE INDEX table1_last_update_date_idx ON keyspace1.table1
> (last_update_date);
> I think the last part should be:
> CREATE INDEX IF NOT EXISTS table1_last_update_date_idx ON keyspace1.table1
> (last_update_date);
> // edit by Stefan Miklosovic
> PR: https://github.com/apache/cassandra/pull/731
> I have added UDTs as part of this patch as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14801) calculatePendingRanges no longer safe for multiple adjacent range movements
[
https://issues.apache.org/jira/browse/CASSANDRA-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14801:
--
Complexity: Normal
> calculatePendingRanges no longer safe for multiple adjacent range movements
> ---
>
> Key: CASSANDRA-14801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14801
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Coordination, Legacy/Distributed Metadata
>Reporter: Benedict Elliott Smith
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0, 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Correctness depended upon the narrowing to a {{Set}},
> which we no longer do - we maintain a collection of all {{Replica}}. Our
> {{RangesAtEndpoint}} collection built by {{getPendingRanges}} can as a result
> contain the same endpoint multiple times; and our {{EndpointsForToken}}
> obtained by {{TokenMetadata.pendingEndpointsFor}} may fail to be constructed,
> resulting in cluster-wide failures for writes to the affected token ranges
> for the duration of the range movement.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14157) [DTEST] [TRUNK] test_tracing_does_not_interfere_with_digest_calculation - cql_tracing_test.TestCqlTracing failed once : AssertionError: assert 0 == 1
[
https://issues.apache.org/jira/browse/CASSANDRA-14157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14157:
--
Complexity: Normal
> [DTEST] [TRUNK] test_tracing_does_not_interfere_with_digest_calculation -
> cql_tracing_test.TestCqlTracing failed once : AssertionError: assert 0 == 1
> -
>
> Key: CASSANDRA-14157
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14157
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Michael Kjellman
>Assignee: Adam Holmberg
>Priority: Normal
> Labels: dtest
> Fix For: 4.0-beta3
>
>
> test_tracing_does_not_interfere_with_digest_calculation -
> cql_tracing_test.TestCqlTracing failed it's assertion once today in a
> circleci run. the dtests were running against trunk.
> Although it has failed once so far, a quick read of the comments in the test
> seems to indicate that the assertion failing this way might mean that
> CASSANDRA-13964 didn't fully fix the issue.
> {code:python}
> if jmx.has_mbean(rr_count):
> # expect 0 digest mismatches
> > assert 0 == jmx.read_attribute(rr_count, 'Count')
> E AssertionError: assert 0 == 1
> E+ where 1 = 0x7f62d4156898>>('org.apache.cassandra.metrics:type=ReadRepair,name=RepairedBlocking',
> 'Count')
> E+where > =
> .read_attribute
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14103) Fix potential race during compaction strategy reload
[
https://issues.apache.org/jira/browse/CASSANDRA-14103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14103:
--
Complexity: Normal
> Fix potential race during compaction strategy reload
>
>
> Key: CASSANDRA-14103
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14103
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Paulo Motta
>Assignee: Marcus Eriksson
>Priority: Urgent
> Fix For: 3.11.9, 4.0-beta3
>
> Attachments: 3.11-14103-dtest.png, 3.11-14103-testall.png,
> trunk-14103-dtest.png, trunk-14103-testall.png
>
>
> When the compaction strategies are reloaded after disk boundary changes
> (CASSANDRA-13948), it's possible that a recently finished SSTable is added
> twice to the compaction strategy: once when the compaction strategies are
> reloaded due to the disk boundary change ({{maybeReloadDiskBoundarie}}), and
> another when the {{CompactionStrategyManager}} is processing the
> {{SSTableAddedNotification}}.
> This should be quite unlikely because a compaction must finish as soon as the
> disk boundary changes, and even if it happens most compaction strategies
> would not be affected by it since they deduplicate sstables internally, but
> we should protect against such scenario.
> For more context see [this
> comment|https://issues.apache.org/jira/browse/CASSANDRA-13948?focusedCommentId=16280448=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16280448]
> from Marcus.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14806:
--
Complexity: Normal
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, CI, Legacy/Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Normal
> Labels: CI
> Fix For: 2.2.15, 3.0.19, 3.11.5, 4.0, 4.0-alpha1
>
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211237#comment-17211237 ] David Capwell edited comment on CASSANDRA-16150 at 10/9/20, 8:03 PM: - CI Results: Yellow, known broken tests only Branch: trunk Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16150-trunk-89A170D2-1BB1-4D7D-ADA0-7D9942489E9C Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/88/ was (Author: dcapwell): Starting commit CI Results (pending): Branch: trunk Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16150-trunk-89A170D2-1BB1-4D7D-ADA0-7D9942489E9C Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/88/ > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.0-beta3 > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16150: -- Fix Version/s: (was: 4.0-beta) 4.0-beta3 Since Version: 4.0-beta1 Source Control Link: https://github.com/apache/cassandra/commit/7648588b65a5801c1e8af026b6459cd65799eace Resolution: Fixed Status: Resolved (was: Ready to Commit) I also validated the md5 of the lib matches maven central. > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.0-beta3 > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new e37f766 Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix e37f766 is described below commit e37f766403e6911e5d965a211758387c6ef4c587 Author: Rahul Nandi AuthorDate: Fri Oct 9 10:56:55 2020 -0700 Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix patch by Rahul Nandi; reviewed by Alex Petrov, David Capwell for CASSANDRA-16150 --- CHANGES.txt| 1 + build.xml | 3 +-- lib/snakeyaml-1.23.jar | Bin 301298 -> 0 bytes lib/snakeyaml-1.26.jar | Bin 0 -> 309001 bytes 4 files changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index a990fb0..289d4e8 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -18,6 +18,7 @@ * NPE thrown while updating speculative execution time if keyspace is removed during task execution (CASSANDRA-15949) * Show the progress of data streaming and index build (CASSANDRA-15406) * Add flag to disable chunk cache and disable by default (CASSANDRA-16036) + * Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix (CASSANDRA-16150) Merged from 3.11: * Fix memory leak in CompressedChunkReader (CASSANDRA-15880) * Don't attempt value skipping with mixed version cluster (CASSANDRA-15833) diff --git a/build.xml b/build.xml index 6a3eb1e..e026630 100644 --- a/build.xml +++ b/build.xml @@ -583,8 +583,7 @@ - - + diff --git a/lib/snakeyaml-1.23.jar b/lib/snakeyaml-1.23.jar deleted file mode 100644 index adcef4f..000 Binary files a/lib/snakeyaml-1.23.jar and /dev/null differ diff --git a/lib/snakeyaml-1.26.jar b/lib/snakeyaml-1.26.jar new file mode 100644 index 000..8f301fd Binary files /dev/null and b/lib/snakeyaml-1.26.jar differ - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15348) Harry: generator library and extensible framework for fuzz testing Apache Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-15348: -- Complexity: Challenging > Harry: generator library and extensible framework for fuzz testing Apache > Cassandra > --- > > Key: CASSANDRA-15348 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15348 > Project: Cassandra > Issue Type: New Feature >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Fix For: 4.0-beta > > > h2. Description: > This ticket introduces Harry, a component for fuzz testing and verification > of the Apache Cassandra clusters at scale. > h2. Motivation: > Current testing tooling largely tests for common- and edge-cases, and most of > the tests use predefined datasets. Property-based tests can help explore a > broader range of states, but often require either a complex model or a large > state to test against. > h2. What problems Harry solves: > Harry allows to run tests that are able to validate state of both dense nodes > (to test local read-write path) and large clusters (to test distributed > read-write path), and do it efficiently. Main goals, and what sets it apart > from the other testing tools is: > * The state required for verification should remain as compact as possible. > * The verification process itself should be as performant as possible. > * Ideally, we'd want a way to verify database state while _continuing_ > running state change queries against it. > h2. What Harry does: > To achieve this, Harry defines a model that holds the state of the database, > generators that produce reproducible, pseudo-random schemas, mutations, and > queries, and a validator that asserts the correctness of the model following > execution of generated traffic. > h2. Harry consists of multiple reusable components: > * Generator library: how to create a library of invertible, order-preserving > generators for simple and composite data types. > * Model and checker: how to use the properties of generators to validate the > output of an eventually-consistent database in a linear time. > * Runner library: how to create a scheme for reproducible runs, despite the > concurrent nature of database and fuzzer itself. > h2. Short and somewhat approximate description of how Harry achieves this: > Generation and validation define strict mathematical relations between the > generated values and pseudorandom numbers they were generated from. Using > these properties, we can store minimal state and check if these properties > hold during validation. > Since Cassandra stores data in rows, we should be able to "inflate" data to > insert a row into the database from a single number we call _descriptor_. > Each value in the row read from the database can be "deflated" back to the > descriptor it was generated from. This way, to precisely verify the state of > the row, we only need to know the descriptor it was generated from and a > timestamp at which it was inserted. > Similarly, keys for the inserted row can be "inflated" from a single 64-bit > integer, and then "deflated" back to it. To efficiently search for keys, > while allowing range scans, our generation scheme preserves the order of the > original 64-bit integer. Every pair of keys generated from two 64-bit > integers would sort the same way as these integers. > This way, in order to validate a state of the range of rows queried from the > database, it is sufficient to "deflate" its key and data values, use deflated > 64-bit key representation to find all descriptors these rows were generated > from, and ensure that the given sequence of descriptors could have resulted > in the state that database has responded with. > Using this scheme, we keep a minimum possible amount of data per row, can > efficiently generate the data, and backtrack values to the numbers they were > generated from. Most of the time, we operate on 64-bit integer values and > only use "inflated" objects when running queries against database state, > minimizing the amount of required memory. > h2. Name: > Harry (verb). > According to Marriam-Webster: > * to torment by or as if by constant attack > * persistently carry out attacks on (an enemy or an enemy's territory) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16057) Should update in-jvm dtest to expose stdout and stderr for nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-16057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211322#comment-17211322 ] David Capwell commented on CASSANDRA-16057: --- code mostly LGTM, basically the same as the trunk patch. Can you run python dtest and jvm upgrade tests? Also 3.11 fails to build. > Should update in-jvm dtest to expose stdout and stderr for nodetool > --- > > Key: CASSANDRA-16057 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16057 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: David Capwell >Assignee: Yifan Cai >Priority: Normal > Fix For: NA > > Time Spent: 2h 20m > Remaining Estimate: 0h > > Many nodetool commands output to stdout or stderr so running nodetool using > in-jvm dtest should expose that to tests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16057) Should update in-jvm dtest to expose stdout and stderr for nodetool
[
https://issues.apache.org/jira/browse/CASSANDRA-16057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211311#comment-17211311
]
Yifan Cai commented on CASSANDRA-16057:
---
Listing the files that are still using {{System.out | System.err}}. Nodetool
commands are no longer using the system out/err directly.
In 2.2
{code:java}
12:30:07 in cassandra on b/CASSANDRA-16057-2.2
➜ egrep -r 'System.out|System.err' src/java/org/apache/cassandra/tools | awk
{'print $1'} | sort | uniq -u
src/java/org/apache/cassandra/tools/GetVersion.java:
src/java/org/apache/cassandra/tools/Output.java:
src/java/org/apache/cassandra/tools/SSTableExpiredBlockers.java:
src/java/org/apache/cassandra/tools/SSTableMetadataViewer.java:
{code}
In 3.0
{code:java}
12:32:28 in cassandra on b/CASSANDRA-16057-3.0
➜ egrep -r 'System.out|System.err' src/java/org/apache/cassandra/tools | awk
{'print $1'} | sort | uniq -u
src/java/org/apache/cassandra/tools/GetVersion.java:
src/java/org/apache/cassandra/tools/Output.java:
src/java/org/apache/cassandra/tools/SSTableExpiredBlockers.java:
src/java/org/apache/cassandra/tools/SSTableMetadataViewer.java:
{code}
In 3.11
{code:java}
12:33:20 in cassandra on b/CASSANDRA-16057-3.11
➜ egrep -r 'System.out|System.err' src/java/org/apache/cassandra/tools | awk
{'print $1'} | sort | uniq -u
src/java/org/apache/cassandra/tools/GetVersion.java:
src/java/org/apache/cassandra/tools/Output.java:
src/java/org/apache/cassandra/tools/SSTableExpiredBlockers.java:
src/java/org/apache/cassandra/tools/SSTableMetadataViewer.java:
src/java/org/apache/cassandra/tools/nodetool/formatter/TableBuilder.java: //
It has System.out in the comment.
{code}
> Should update in-jvm dtest to expose stdout and stderr for nodetool
> ---
>
> Key: CASSANDRA-16057
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16057
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: NA
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Many nodetool commands output to stdout or stderr so running nodetool using
> in-jvm dtest should expose that to tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14793) Improve system table handling when losing a disk when using JBOD
[ https://issues.apache.org/jira/browse/CASSANDRA-14793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-14793: -- Complexity: Normal > Improve system table handling when losing a disk when using JBOD > > > Key: CASSANDRA-14793 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14793 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core >Reporter: Marcus Eriksson >Assignee: Benjamin Lerer >Priority: Normal > Fix For: 4.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > We should improve the way we handle disk failures when losing a disk in a > JBOD setup > One way could be to pin the system tables to a special data directory. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14973) Bring v5 driver out of beta, introduce v6 before 4.0 release is cut
[ https://issues.apache.org/jira/browse/CASSANDRA-14973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-14973: -- Complexity: Normal > Bring v5 driver out of beta, introduce v6 before 4.0 release is cut > --- > > Key: CASSANDRA-14973 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14973 > Project: Cassandra > Issue Type: Task >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Urgent > Labels: protocolv5 > Fix For: 4.0-beta > > > In http://issues.apache.org/jira/browse/CASSANDRA-12142, we’ve introduced > Beta flag for v5 protocol. However, up till now, v5 is in beta both in > [Cassandra|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/transport/ProtocolVersion.java#L46] > and in > [java-driver|https://github.com/datastax/java-driver/blob/3.x/driver-core/src/main/java/com/datastax/driver/core/ProtocolVersion.java#L35]. > > Before the final 4.0 release is cut, we need to bring v5 out of beta and > finalise native protocol spec, and start bringing all new changes to v6 > protocol, which will be in beta. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14788) Add test coverage workflows to CircleCI config
[ https://issues.apache.org/jira/browse/CASSANDRA-14788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-14788: -- Complexity: Normal > Add test coverage workflows to CircleCI config > -- > > Key: CASSANDRA-14788 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14788 > Project: Cassandra > Issue Type: Improvement > Components: Build, CI >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Low > Labels: CI, pull-request-available > Fix For: 4.0, 4.0-triage > > Time Spent: 1h 10m > Remaining Estimate: 0h > > To support 4.0 testing efforts it's helpful to know how much of the code is > being exercised by unit tests and dtests. > Add support for running the unit tests and dtests instrumented for test > coverage on CircleCI and then combine the results of all tests (unit, dtest > with vnodes, dtest without vnodes) into a single coverage report. > All of the hard work of getting JaCoCo to work with unit tests and dtests has > already been done, it just needs wiring up. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13649) Uncaught exceptions in Netty pipeline
[
https://issues.apache.org/jira/browse/CASSANDRA-13649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-13649:
--
Complexity: Normal
> Uncaught exceptions in Netty pipeline
> -
>
> Key: CASSANDRA-13649
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13649
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Streaming and Messaging, Legacy/Testing
>Reporter: Stefan Podkowinski
>Assignee: Norman Maurer
>Priority: Normal
> Labels: patch
> Fix For: 4.0, 2.2.x, 3.0.x, 3.11.x
>
> Attachments:
> 0001-CASSANDRA-13649-Ensure-all-exceptions-are-correctly-.patch,
> test_stdout.txt
>
>
> I've noticed some netty related errors in trunk in [some of the dtest
> results|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/106/#showFailuresLink].
> Just want to make sure that we don't have to change anything related to the
> exception handling in our pipeline and that this isn't a netty issue.
> Actually if this causes flakiness but is otherwise harmless, we should do
> something about it, even if it's just on the dtest side.
> {noformat}
> WARN [epollEventLoopGroup-2-9] 2017-06-28 17:23:49,699 Slf4JLogger.java:151
> - An exceptionCaught() event was fired, and it reached at the tail of the
> pipeline. It usually means the last handler in the pipeline did not handle
> the exception.
> io.netty.channel.unix.Errors$NativeIoException: syscall:read(...)() failed:
> Connection reset by peer
> at io.netty.channel.unix.FileDescriptor.readAddress(...)(Unknown
> Source) ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> {noformat}
> And again in another test:
> {noformat}
> WARN [epollEventLoopGroup-2-8] 2017-06-29 02:27:31,300 Slf4JLogger.java:151
> - An exceptionCaught() event was fired, and it reached at the tail of the
> pipeline. It usually means the last handler in the pipeline did not handle
> the exception.
> io.netty.channel.unix.Errors$NativeIoException: syscall:read(...)() failed:
> Connection reset by peer
> at io.netty.channel.unix.FileDescriptor.readAddress(...)(Unknown
> Source) ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> {noformat}
> Edit:
> The {{io.netty.channel.unix.Errors$NativeIoException: syscall:read(...)()
> failed}} error also causes tests to fail for 3.0 and 3.11.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13047) Point cqlsh help to the new doc
[ https://issues.apache.org/jira/browse/CASSANDRA-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-13047: -- Complexity: Normal > Point cqlsh help to the new doc > --- > > Key: CASSANDRA-13047 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13047 > Project: Cassandra > Issue Type: Bug > Components: Legacy/CQL >Reporter: Sylvain Lebresne >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0, 3.11.x > > Time Spent: 0.5h > Remaining Estimate: 0h > > Cqlsh still points to the "old" textile CQL doc, but that's not really > maintain anymore now that we have the new doc (which include everything the > old doc had and more). We should update cqlsh to point to the new doc so we > can remove the old one completely. > Any takers? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-12126) CAS Reads Inconsistencies
[ https://issues.apache.org/jira/browse/CASSANDRA-12126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-12126: -- Complexity: Normal > CAS Reads Inconsistencies > -- > > Key: CASSANDRA-12126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12126 > Project: Cassandra > Issue Type: Bug > Components: Feature/Lightweight Transactions, Legacy/Coordination >Reporter: Sankalp Kohli >Assignee: Sylvain Lebresne >Priority: Normal > Labels: LWT, pull-request-available > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > While looking at the CAS code in Cassandra, I found a potential issue with > CAS Reads. Here is how it can happen with RF=3 > 1) You issue a CAS Write and it fails in the propose phase. A machine replies > true to a propose and saves the commit in accepted filed. The other two > machines B and C does not get to the accept phase. > Current state is that machine A has this commit in paxos table as accepted > but not committed and B and C does not. > 2) Issue a CAS Read and it goes to only B and C. You wont be able to read the > value written in step 1. This step is as if nothing is inflight. > 3) Issue another CAS Read and it goes to A and B. Now we will discover that > there is something inflight from A and will propose and commit it with the > current ballot. Now we can read the value written in step 1 as part of this > CAS read. > If we skip step 3 and instead run step 4, we will never learn about value > written in step 1. > 4. Issue a CAS Write and it involves only B and C. This will succeed and > commit a different value than step 1. Step 1 value will never be seen again > and was never seen before. > If you read the Lamport “paxos made simple” paper and read section 2.3. It > talks about this issue which is how learners can find out if majority of the > acceptors have accepted the proposal. > In step 3, it is correct that we propose the value again since we dont know > if it was accepted by majority of acceptors. When we ask majority of > acceptors, and more than one acceptors but not majority has something in > flight, we have no way of knowing if it is accepted by majority of acceptors. > So this behavior is correct. > However we need to fix step 2, since it caused reads to not be linearizable > with respect to writes and other reads. In this case, we know that majority > of acceptors have no inflight commit which means we have majority that > nothing was accepted by majority. I think we should run a propose step here > with empty commit and that will cause write written in step 1 to not be > visible ever after. > With this fix, we will either see data written in step 1 on next serial read > or will never see it which is what we want. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13701) Lower default num_tokens
[ https://issues.apache.org/jira/browse/CASSANDRA-13701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-13701: -- Complexity: Normal > Lower default num_tokens > > > Key: CASSANDRA-13701 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13701 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Chris Lohfink >Assignee: Alexander Dejanovski >Priority: Low > Fix For: 4.0-alpha > > > For reasons highlighted in CASSANDRA-7032, the high number of vnodes is not > necessary. It is very expensive for operations processes and scanning. Its > come up a lot and its pretty standard and known now to always reduce the > num_tokens within the community. We should just lower the defaults. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14694) add latency sample for speculative read repair writes
[ https://issues.apache.org/jira/browse/CASSANDRA-14694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-14694: -- Complexity: Normal > add latency sample for speculative read repair writes > - > > Key: CASSANDRA-14694 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14694 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Coordination, Observability/Metrics >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 4.0, 4.0-triage > > > Speculative read repair mutations shouldn't use read latencies to determine > when to send a speculative mutation. It should have it's own value based on > mutation latencies. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14747) Test Messaging Refactor with: 200 node, compression=none, encryption=none, coalescing=off
[ https://issues.apache.org/jira/browse/CASSANDRA-14747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-14747: -- Complexity: Normal > Test Messaging Refactor with: 200 node, compression=none, encryption=none, > coalescing=off > -- > > Key: CASSANDRA-14747 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14747 > Project: Cassandra > Issue Type: Sub-task > Components: Legacy/Testing >Reporter: Joey Lynch >Assignee: Joey Lynch >Priority: Normal > Fix For: 4.0-beta, 4.0-triage > > Attachments: 3.0.17-QPS.png, 4.0.1-QPS.png, > 4.0.11-after-jolynch-tweaks.svg, 4.0.12-after-unconditional-flush.svg, > 4.0.15-after-sndbuf-fix.svg, 4.0.7-before-my-changes.svg, > 4.0_errors_showing_heap_pressure.txt, > 4.0_heap_histogram_showing_many_MessageOuts.txt, > i-0ed2acd2dfacab7c1-after-looping-fixes.svg, > trunk_14503_v2_cpuflamegraph.svg, trunk_vs_3.0.17_latency_under_load.png, > ttop_NettyOutbound-Thread_spinning.txt, > useast1c-i-0e1ddfe8b2f769060-mutation-flame.svg, > useast1e-i-08635fa1631601538_flamegraph_96node.svg, > useast1e-i-08635fa1631601538_ttop_netty_outbound_threads_96nodes, > useast1e-i-08635fa1631601538_uninlinedcpuflamegraph.0_96node_60sec_profile.svg > > > Tracks evaluating a 200 node cluster with all internode settings off (no > compression, no encryption, no coalescing). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-2848) Make the Client API support passing down timeouts
[ https://issues.apache.org/jira/browse/CASSANDRA-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-2848: - Complexity: Normal > Make the Client API support passing down timeouts > - > > Key: CASSANDRA-2848 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2848 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client, Messaging/Internode >Reporter: Chris Goffinet >Assignee: Yifan Cai >Priority: Low > Labels: protocolv5 > Fix For: 4.0-beta, 4.0-triage > > Attachments: 2848-trunk-v2.txt, 2848-trunk.txt > > > Having a max server RPC timeout is good for worst case, but many applications > that have middleware in front of Cassandra, might have higher timeout > requirements. In a fail fast environment, if my application starting at say > the front-end, only has 20ms to process a request, and it must connect to X > services down the stack, by the time it hits Cassandra, we might only have > 10ms. I propose we provide the ability to specify the timeout on each call we > do optionally. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14608) Confirm correctness of windows scripts post-9608
[ https://issues.apache.org/jira/browse/CASSANDRA-14608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-14608: -- Complexity: Normal > Confirm correctness of windows scripts post-9608 > > > Key: CASSANDRA-14608 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14608 > Project: Cassandra > Issue Type: Task > Environment: Windows >Reporter: Jason Brown >Priority: Urgent > Fix For: 4.0-rc, 4.0-triage > > Time Spent: 10m > Remaining Estimate: 0h > > In CASSANDRA-9608, we chose to defer making all the changes to Windows > scripts. This ticket is to ensure that we do that work. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14753) Document incremental repair session timeouts and repair_admin usage
[ https://issues.apache.org/jira/browse/CASSANDRA-14753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-14753: -- Complexity: Normal > Document incremental repair session timeouts and repair_admin usage > --- > > Key: CASSANDRA-14753 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14753 > Project: Cassandra > Issue Type: Task > Components: Legacy/Documentation and Website >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Low > Fix For: 4.0 > > > As seen in CASSANDRA-14685, the behavior of incremental repair sessions with > failed streams is not obvious and appears to be a bug (although it's working > as expected). The incremental repair documentation should be updated to > describe what happens if an incremental repair session fails mid-stream, the > session timeouts, and how and when to use nodetool repair_admin. The sstable > acquisition error should also be updated to mention this as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14688) Update protocol spec and class level doc with protocol checksumming details
[
https://issues.apache.org/jira/browse/CASSANDRA-14688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14688:
--
Complexity: Normal
> Update protocol spec and class level doc with protocol checksumming details
> ---
>
> Key: CASSANDRA-14688
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14688
> Project: Cassandra
> Issue Type: Task
> Components: Legacy/Documentation and Website
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-rc
>
>
> CASSANDRA-13304 provides an option to add checksumming to the frame body of
> native protocol messages. The native protocol spec needs to be updated to
> reflect this ASAP. We should also verify that the javadoc comments describing
> the on-wire format in
> {{o.a.c.transport.frame.checksum.ChecksummingTransformer}} are up to date.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14764) Test Messaging Refactor with: 12 Node Breaking Point, compression=none, encryption=none, coalescing=off
[
https://issues.apache.org/jira/browse/CASSANDRA-14764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14764:
--
Complexity: Normal
> Test Messaging Refactor with: 12 Node Breaking Point, compression=none,
> encryption=none, coalescing=off
> ---
>
> Key: CASSANDRA-14764
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14764
> Project: Cassandra
> Issue Type: Sub-task
> Components: Legacy/Streaming and Messaging
>Reporter: Joey Lynch
>Assignee: Vinay Chella
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
> Attachments: i-03341e1c52de6ea3e-after-queue-change.svg,
> i-07cd92e844d66d801-after-queue-bound.svg, i-07cd92e844d66d801-hint-play.svg,
> i-07cd92e844d66d801-uninlined-with-jvm-methods.svg, ttop.txt
>
>
> *Setup:*
> * Cassandra: 12 (2*6) node i3.xlarge AWS instance (4 cpu cores, 30GB ram)
> running cassandra trunk off of jasobrown/14503 jdd7ec5a2 (Jasons patched
> internode messaging branch) vs the same footprint running 3.0.17
> * Two datacenters with 100ms latency between them
> * No compression, encryption, or coalescing turned on
> *Test #1:*
> ndbench sent 1.5k QPS at a coordinator level to one datacenter (RF=3*2 = 6 so
> 3k global replica QPS) of 4kb single partition BATCH mutations at LOCAL_ONE.
> This represents about 250 QPS per coordinator in the first datacenter or 60
> QPS per core. The goal was to observe P99 write and read latencies under
> various QPS.
> *Result:*
> The good news is since the CASSANDRA-14503 changes, instead of keeping the
> mutations on heap we put the message into hints instead and don't run out of
> memory. The bad news is that the {{MessagingService-NettyOutbound-Thread's}}
> would occasionally enter a degraded state where they would just spin on a
> core. I've attached flame graphs showing the CPU state as [~jasobrown]
> applied fixes to the {{OutboundMessagingConnection}} class.
> *Follow Ups:*
> [~jasobrown] has committed a number of fixes onto his
> {{jasobrown/14503-collab}} branch including:
> 1. Limiting the amount of time spent dequeuing messages if they are expired
> (previously if messages entered the queue faster than we could dequeue them
> we'd just inifinte loop on the consumer side)
> 2. Don't call {{dequeueMessages}} from within {{dequeueMessages}} created
> callbacks.
> We're continuing to use CPU flamegraphs to figure out where we're looping and
> fixing bugs as we find them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15804) system_schema keyspace complain of schema mismatch during upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-15804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-15804:
--
Complexity: Normal
> system_schema keyspace complain of schema mismatch during upgrade
> -
>
> Key: CASSANDRA-15804
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15804
> Project: Cassandra
> Issue Type: Bug
>Reporter: Pedro Gordo
>Priority: Low
> Fix For: 3.11.x, 4.0-beta, 4.0-triage
>
>
> When upgrading from 3.11.4 to 3.11.6, we got the following error:
> {code:Plain Text}
> ERROR [MessagingService-Incoming-/10.20.11.59] 2020-05-07 13:53:52,627
> CassandraDaemon.java:228 - Exception in thread
> Thread[MessagingService-Incoming-/10.20.11.59,5,main]
> java.lang.RuntimeException: Unknown column kind during deserialization
> at
> org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:464)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:419)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:195)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:851)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:839)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:425)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:434)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:675)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:658)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at org.apache.cassandra.net.MessageIn.read(MessageIn.java:123)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:192)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:180)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:94)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> {code}
> I've noticed that system_schema.dropped_columns has a new column called
> "kind".
> No issues arise from this error message, and the error disappeared after
> upgrading all nodes. But it still caused concerns due to the ERROR logging
> level, although "nodetool describecluster" reported only one schema version.
> It makes sense for the system keyspaces to not be included for the
> "describecluster" schema version check, but it seems to me that these
> internal schema mismatches should be ignored if the versions are different
> between the nodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14606) Add documentation for java 11 support
[ https://issues.apache.org/jira/browse/CASSANDRA-14606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-14606: -- Complexity: Normal > Add documentation for java 11 support > - > > Key: CASSANDRA-14606 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14606 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Documentation and Website >Reporter: Jason Brown >Priority: Low > Labels: Java11 > Fix For: 4.0, 4.0-triage > > > Let's add some documentation for operators around the java 11 support that > was introduced in CASSANDRA-9608. Also, we should point out changes in the > scripts that might affect automation that operators have in place. > Parking on [~snazy] just 'cuz ;) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15249) Add documentation on release lifecycle
[ https://issues.apache.org/jira/browse/CASSANDRA-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-15249: -- Complexity: Normal > Add documentation on release lifecycle > -- > > Key: CASSANDRA-15249 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15249 > Project: Cassandra > Issue Type: Improvement > Components: Documentation/Website >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > Fix For: 4.0, 4.0-triage > > Attachments: release_lifecycle.patch > > > Relevant dev list mail thread: > https://lists.apache.org/thread.html/1a768d057d1af5a0f373c4c399a23e65cb04c61bbfff612634b9437c@%3Cdev.cassandra.apache.org%3E > Cassandra wiki on release lifecycle - > https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle > [Legacy - this google doc content is now moved to above cwiki; keeping google > doc link here for preserving comments history] Google doc with community > collaboration on documenting release lifecycle > https://docs.google.com/document/d/1bS6sr-HSrHFjZb0welife6Qx7u3ZDgRiAoENMLYlfz8/edit. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13951) Release Java and Python drivers and re-enable build.xml driver dep
[
https://issues.apache.org/jira/browse/CASSANDRA-13951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-13951:
--
Complexity: Normal
> Release Java and Python drivers and re-enable build.xml driver dep
> --
>
> Key: CASSANDRA-13951
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13951
> Project: Cassandra
> Issue Type: Task
> Components: Build, Dependencies
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Urgent
> Fix For: 4.0, 4.x
>
>
> During [CASSANDRA-10786], driver changes were introduced that required a
> snapshot release of both drivers.
> When the stable driver version is published, we have to update the driver
> dependencies in {{lib}} and un-comment {{build.xml}} sections with
> {{java-driver}} information for correct publish of 4.0. This manipulation was
> required in order to account for the chicken-or-egg problem between Cassandra
> itself and the drivers.
> This absolutely has to be done before 4.0 & there should be no release
> without making sure right drivers are in place.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14404) Transient Replication & Cheap Quorums: Decouple storage requirements from consensus group size using incremental repair
[ https://issues.apache.org/jira/browse/CASSANDRA-14404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-14404: -- Complexity: Normal > Transient Replication & Cheap Quorums: Decouple storage requirements from > consensus group size using incremental repair > --- > > Key: CASSANDRA-14404 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14404 > Project: Cassandra > Issue Type: New Feature > Components: Consistency/Hints, Consistency/Repair, Feature/2i Index, > Feature/Materialized Views, Legacy/Coordination, Legacy/Core, Legacy/CQL, > Legacy/Distributed Metadata, Legacy/Local Write-Read Paths, Legacy/Testing, > Legacy/Tools >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Normal > Fix For: 4.0, 4.0-triage > > > Transient Replication is an implementation of [Witness > Replicas|http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.146.3429=rep1=pdf] > that leverages incremental repair to make full replicas consistent with > transient replicas that don't store the entire data set. Witness replicas are > used in real world systems such as Megastore and Spanner to increase > availability inexpensively without having to commit to more full copies of > the database. Transient replicas implement functionality similar to > upgradable and temporary replicas from the paper. > With transient replication the replication factor is increased beyond the > desired level of data redundancy by adding replicas that only store data when > sufficient full replicas are unavailable to store the data. These replicas > are called transient replicas. When incremental repair runs transient > replicas stream any data they have received to full replicas and once the > data is fully replicated it is dropped at the transient replicas. > Cheap quorums are a further set of optimizations on the write path to avoid > writing to transient replicas unless sufficient full replicas are available > as well as optimizations on the read path to prefer reading from transient > replicas. When writing at quorum to a table configured to use transient > replication the quorum will always prefer available full replicas over > transient replicas so that transient replicas don't have to process writes. > Rapid write protection (similar to rapid read protection) reduces tail > latency when full replicas are slow/unavailable to respond by sending writes > to additional replicas if necessary. > Transient replicas can generally service reads faster because they don't have > to do anything beyond bloom filter checks if they have no data. With vnodes > and larger size clusters they will not have a large quantity of data even in > failure cases where transient replicas start to serve a steady amount of > write traffic for some of their transiently replicated ranges. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14746) Ensure Netty Internode Messaging Refactor is Solid
[ https://issues.apache.org/jira/browse/CASSANDRA-14746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-14746: -- Complexity: Normal > Ensure Netty Internode Messaging Refactor is Solid > -- > > Key: CASSANDRA-14746 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14746 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Streaming and Messaging >Reporter: Joey Lynch >Assignee: Joey Lynch >Priority: Normal > Labels: 4.0-QA > Fix For: 4.0-beta > > > Before we release 4.0 let's ensure that the internode messaging refactor is > 100% solid. As internode messaging is naturally used in many code paths and > widely configurable we have a large number of cluster configurations and test > configurations that must be vetted. > We plan to vary the following: > * Version of Cassandra 3.0.17 vs 4.0-alpha > * Cluster sizes with *multi-dc* deployments ranging from 6 - 100 nodes > * Client request rates varying between 1k QPS and 100k QPS of varying sizes > and shapes (BATCH, INSERT, SELECT point, SELECT range, etc ...) > * Internode compression > * Internode SSL (as well as openssl vs jdk) > * Internode Coalescing options > We are looking to measure the following as appropriate: > * Latency distributions of reads and writes (lower is better) > * Scaling limit, aka maximum throughput before violating p99 latency > deadline of 10ms @ LOCAL_QUORUM, on a fixed hardware deployment for 100% > writes, 100% reads and 50-50 writes+reads (higher is better) > * Thread counts (lower is better) > * Context switches (lower is better) > * On-CPU time of tasks (higher periods without context switch is better) > * GC allocation rates / throughput for a fixed size heap (lower allocation > better) > * Streaming recovery time for a single node failure, i.e. can Cassandra > saturate the NIC > > The goal is that 4.0 should have better latency, more throughput, fewer > threads, fewer context switches, less GC allocation, and faster recovery > time. I'm putting Jason Brown as the reviewer since he implemented most of > the internode refactor. > Current collaborators driving this QA task: Dinesh Joshi, Jordan West, Joey > Lynch (Netflix), Vinay Chella (Netflix) > Owning committer(s): Jason Brown -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14361) Allow SimpleSeedProvider to resolve multiple IPs per DNS name
[ https://issues.apache.org/jira/browse/CASSANDRA-14361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-14361: -- Complexity: Normal > Allow SimpleSeedProvider to resolve multiple IPs per DNS name > - > > Key: CASSANDRA-14361 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14361 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ben Bromhead >Assignee: Ben Bromhead >Priority: Low > Fix For: 4.0, 4.0-triage > > Time Spent: 20m > Remaining Estimate: 0h > > Currently SimpleSeedProvider can accept a comma separated string of IPs or > hostnames as the set of Cassandra seeds. hostnames are resolved via > InetAddress.getByName, which will only return the first IP associated with an > A, or CNAME record. > By changing to InetAddress.getAllByName, existing behavior is preserved, but > now Cassandra can discover multiple IP address per record, allowing seed > discovery by DNS to be a little easier. > Some examples of improved workflows with this change include: > * specify the DNS name of a headless service in Kubernetes which will > resolve to all IP addresses of pods within that service. > * seed discovery for multi-region clusters via AWS route53, AzureDNS etc > * Other common DNS service discovery mechanisms. > The only behavior this is likely to impact would be where users are relying > on the fact that getByName only returns a single IP address. > I can't imagine any scenario where that is a sane choice. Even when that > choice has been made, it only impacts the first startup of Cassandra and > would not be on any critical path. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14834) Avoid keeping StreamingTombstoneHistogramBuilder.Spool in memory during the whole compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-14834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14834:
--
Complexity: Normal
> Avoid keeping StreamingTombstoneHistogramBuilder.Spool in memory during the
> whole compaction
>
>
> Key: CASSANDRA-14834
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14834
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Low
> Fix For: 4.0, 4.0-beta
>
>
> Since CASSANDRA-13444 {{StreamingTombstoneHistogramBuilder.Spool}} is
> allocated to keep around an array with 131072 * 2 * 2 integers *per written
> sstable* during the whole compaction. With LCS at times creating 1000s of
> sstables during a compaction it kills the node.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14607) Explore optimizations in AbstractBtreePartiton, java 11 variant
[
https://issues.apache.org/jira/browse/CASSANDRA-14607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14607:
--
Status: Open (was: Resolved)
> Explore optimizations in AbstractBtreePartiton, java 11 variant
> ---
>
> Key: CASSANDRA-14607
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14607
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Jason Brown
>Assignee: Blake Eggleston
>Priority: Normal
> Labels: Java11
> Fix For: 4.0, 4.0-alpha1
>
>
> In CASSANDRA-9608, we discussed some way to optimize the java 11
> implementation of {{AbstractBTreePartition}}. This ticket serves that
> purpose, as well as a "note to selves" to ensure the java 11 version does not
> have a performance regression.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14607) Explore optimizations in AbstractBtreePartiton, java 11 variant
[
https://issues.apache.org/jira/browse/CASSANDRA-14607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14607:
--
Fix Version/s: (was: 4.0-triage)
> Explore optimizations in AbstractBtreePartiton, java 11 variant
> ---
>
> Key: CASSANDRA-14607
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14607
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Jason Brown
>Assignee: Blake Eggleston
>Priority: Normal
> Labels: Java11
> Fix For: 4.0, 4.0-alpha1
>
>
> In CASSANDRA-9608, we discussed some way to optimize the java 11
> implementation of {{AbstractBTreePartition}}. This ticket serves that
> purpose, as well as a "note to selves" to ensure the java 11 version does not
> have a performance regression.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14607) Explore optimizations in AbstractBtreePartiton, java 11 variant
[
https://issues.apache.org/jira/browse/CASSANDRA-14607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14607:
--
Resolution: Fixed
Status: Resolved (was: Open)
> Explore optimizations in AbstractBtreePartiton, java 11 variant
> ---
>
> Key: CASSANDRA-14607
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14607
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Jason Brown
>Assignee: Blake Eggleston
>Priority: Normal
> Labels: Java11
> Fix For: 4.0, 4.0-alpha1
>
>
> In CASSANDRA-9608, we discussed some way to optimize the java 11
> implementation of {{AbstractBTreePartition}}. This ticket serves that
> purpose, as well as a "note to selves" to ensure the java 11 version does not
> have a performance regression.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14806:
--
Status: Open (was: Resolved)
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, CI, Legacy/Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Normal
> Labels: CI
> Fix For: 2.2.15, 3.0.19, 3.11.5, 4.0, 4.0-alpha1
>
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14806:
--
Resolution: Fixed
Status: Resolved (was: Open)
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, CI, Legacy/Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Normal
> Labels: CI
> Fix For: 2.2.15, 3.0.19, 3.11.5, 4.0, 4.0-alpha1
>
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-14806:
--
Fix Version/s: (was: 4.0-triage)
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, CI, Legacy/Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Normal
> Labels: CI
> Fix For: 2.2.15, 3.0.19, 3.11.5, 4.0, 4.0-alpha1
>
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15587) 4.0 quality testing: Platforms and Runtimes
[
https://issues.apache.org/jira/browse/CASSANDRA-15587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-15587:
--
Complexity: Challenging
> 4.0 quality testing: Platforms and Runtimes
> ---
>
> Key: CASSANDRA-15587
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15587
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/java, Test/dtest/python
>Reporter: Josh McKenzie
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Reference [doc from
> NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
> for context.
> *Shepherd: {color:#ff}NONE{color}*
> CASSANDRA-9608 introduces support for Java 11. We'll want to verify that
> Cassandra under Java 11 meets expectations of stability.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16180) 4.0 quality testing: Coordination
[ https://issues.apache.org/jira/browse/CASSANDRA-16180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-16180: -- Complexity: Challenging > 4.0 quality testing: Coordination > - > > Key: CASSANDRA-16180 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16180 > Project: Cassandra > Issue Type: Task > Components: Test/unit >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 4.0 > > > This is a subtask of CASSANDRA-15579 focusing on coordination. > I think that the main reference dtest for this is > [consistency_test.py|https://github.com/apache/cassandra-dtest/blob/master/consistency_test.py]. > We should identify which other tests cover this and identify what should be > extended, similarly to what has been done with CASSANDRA-15977. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16181) 4.0 quality testing: Replication
[ https://issues.apache.org/jira/browse/CASSANDRA-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-16181: -- Complexity: Challenging > 4.0 quality testing: Replication > > > Key: CASSANDRA-16181 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16181 > Project: Cassandra > Issue Type: Task > Components: Test/unit >Reporter: Andres de la Peña >Priority: Normal > Fix For: 4.0 > > > This is a subtask of CASSANDRA-15579 focusing on replication. > I think that the main reference dtest for this is > [replication_test.py|https://github.com/apache/cassandra-dtest/blob/master/replication_test.py]. > We should identify which other tests cover this and identify what should be > extended, similarly to what has been done with CASSANDRA-15977. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15588) 4.0 quality testing: Cluster Upgrade
[ https://issues.apache.org/jira/browse/CASSANDRA-15588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-15588: -- Complexity: Challenging > 4.0 quality testing: Cluster Upgrade > > > Key: CASSANDRA-15588 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15588 > Project: Cassandra > Issue Type: Task > Components: Test/dtest/java, Test/dtest/python >Reporter: Josh McKenzie >Priority: Normal > Fix For: 4.0-beta > > > We've historically had numerous bugs concerning upgrading clusters from one > version to the other. Let's establish the supported upgrade path and ensure > that users can safely perform the upgrades in production. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16205) Offline token allocation strategy generator tool
[
https://issues.apache.org/jira/browse/CASSANDRA-16205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-16205:
---
Change Category: Operability
Complexity: Normal
Status: Open (was: Triage Needed)
> Offline token allocation strategy generator tool
>
>
> Key: CASSANDRA-16205
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16205
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config, Local/Scripts
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>
> A command line tool to generate tokens (using the
> allocate_tokens_for_local_replication_factor algorithm) for pre-configuration
> of {{initial_tokens}} in cassandra.yaml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16205) Offline token allocation strategy generator tool
[
https://issues.apache.org/jira/browse/CASSANDRA-16205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-16205:
---
Description:
A command line tool to generate tokens (using the
allocate_tokens_for_local_replication_factor algorithm) for pre-configuration
of {{initial_tokens}} in cassandra.yaml.
was:A command line tool to generate tokens (using the
allocate_tokens_for_local_replication_factor algorithm) for pre-configuration
of {{initial_tokens}} in cassandra.yaml.
> Offline token allocation strategy generator tool
>
>
> Key: CASSANDRA-16205
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16205
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config, Local/Scripts
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>
> A command line tool to generate tokens (using the
> allocate_tokens_for_local_replication_factor algorithm) for pre-configuration
> of {{initial_tokens}} in cassandra.yaml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16205) Offline token allocation strategy generator tool
Michael Semb Wever created CASSANDRA-16205:
--
Summary: Offline token allocation strategy generator tool
Key: CASSANDRA-16205
URL: https://issues.apache.org/jira/browse/CASSANDRA-16205
Project: Cassandra
Issue Type: Improvement
Components: Local/Config, Local/Scripts
Reporter: Michael Semb Wever
Assignee: Michael Semb Wever
A command line tool to generate tokens (using the
allocate_tokens_for_local_replication_factor algorithm) for pre-configuration
of {{initial_tokens}} in cassandra.yaml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16152) In-JVM dtest - modify schema with stopped nodes and use yaml fragments for config
[ https://issues.apache.org/jira/browse/CASSANDRA-16152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211298#comment-17211298 ] Jon Meredith commented on CASSANDRA-16152: -- I've rebased all the branches against recent versions of the cassandra-x.x branches and trunk. > In-JVM dtest - modify schema with stopped nodes and use yaml fragments for > config > - > > Key: CASSANDRA-16152 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16152 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > > Some convenience improvements to in-JVM dtest that are useful across versions > that I needed while working on CASSANDRA-16144 > * Add support for changing schema with stopped nodes. > * Make it simpler to modify nested configuration items by specifying Yaml > fragments -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15584) 4.0 quality testing: Tooling - External Ecosystem
[
https://issues.apache.org/jira/browse/CASSANDRA-15584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-15584:
--
Description:
Reference [doc from
NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
for context.
*Shepherd: Benjamin Lerer*
Many users of Apache Cassandra employ open source tooling to automate Cassandra
configuration, runtime management, and repair scheduling. Prior to release, we
need to confirm that popular third-party tools function properly.
Current list of tools:
|| Name || Status || Contact ||
| [Priam|http://netflix.github.io/Priam/] |{color:#00875A} *DONE WITH
ALPHA*{color} (need to be tested with beta) | [~sumanth.pasupuleti]|
| [sstabletools|https://github.com/instaclustr/cassandra-sstable-tools] | *NOT
STARTED* | [~stefan.miklosovic]|
| [cassandra-exporter|https://github.com/instaclustr/cassandra-exporter]| *NOT
STARTED* | [~stefan.miklosovic]|
| [Instaclustr Cassandra
operator|https://github.com/instaclustr/cassandra-operator]|
{color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
| [Instaclustr Esop |
https://github.com/instaclustr/instaclustr-esop]|{color:#00875A}*DONE*{color} |
[~stefan.miklosovic]|
| [Instaclustr Icarus |
https://github.com/instaclustr/instaclustr-icarus]|{color:#00875A}*DONE*{color}
| [~stefan.miklosovic]|
| [Cassandra SSTable generator |
https://github.com/instaclustr/cassandra-sstable-generator]|{color:#00875A}*DONE*{color}|
[~stefan.miklosovic]|
| [Cassandra TTL Remover | https://github.com/instaclustr/TTLRemover] |
{color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
| [Cassandra Everywhere Strategy |
https://github.com/instaclustr/cassandra-everywhere-strategy] |
{color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
| [Reaper|http://cassandra-reaper.io/]| {color:#00875A}*AUTOMATIC*{color} |
[~adejanovski]|
| [Medusa|https://github.com/thelastpickle/cassandra-medusa]| *IN PROGRESS*|
[~adejanovski]|
| [Casskop|https://orange-opensource.github.io/casskop/]| *NOT STARTED*| Franck
Dehay|
|
[spark-cassandra-connector|https://github.com/datastax/spark-cassandra-connector]|
{color:#00875A}*DONE*{color}| [~jtgrabowski]|
| [cass operator|https://github.com/datastax/cass-operator]|
{color:#00875A}*DONE*{color}| [~jimdickinson]|
| [metric
collector|https://github.com/datastax/metric-collector-for-apache-cassandra]|
{color:#00875A}*DONE*{color}| [~tjake]|
| [managment
API|https://github.com/datastax/management-api-for-apache-cassandra]|
{color:#00875A}*DONE*{color}| [~tjake]|
Columns descriptions:
* *Name*: Name and link to the tool official page
* *Status*: {{NOT STARTED}}, {{IN PROGRESS}}, {{BLOCKED}} if you hit any issue
and have to wait for it to be solved, {{DONE}}, {{AUTOMATIC}} if testing 4.0 is
part of your CI process.
* *Contact*: The person acting as the contact point for that tool.
was:
Reference [doc from
NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
for context.
*Shepherd: Benjamin Lerer*
Many users of Apache Cassandra employ open source tooling to automate Cassandra
configuration, runtime management, and repair scheduling. Prior to release, we
need to confirm that popular third-party tools function properly.
Current list of tools:
|| Name || Status || Contact ||
| [Priam|http://netflix.github.io/Priam/] |{color:#00875A} *DONE WITH
ALPHA*{color} (need to be tested with beta) | [~sumanth.pasupuleti]|
| [sstabletools|https://github.com/instaclustr/cassandra-sstable-tools] | *NOT
STARTED* | [~stefan.miklosovic]|
| [cassandra-exporter|https://github.com/instaclustr/cassandra-exporter]| *NOT
STARTED* | [~stefan.miklosovic]|
| [Instaclustr Cassandra
operator|https://github.com/instaclustr/cassandra-operator]|
{color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
| [Instaclustr Cassandra Backup Restore |
https://github.com/instaclustr/cassandra-backup]|{color:#00875A}*DONE*{color} |
[~stefan.miklosovic]|
| [Instaclustr Cassandra Sidecar |
https://github.com/instaclustr/cassandra-sidecar]|{color:#00875A}*DONE*{color}
| [~stefan.miklosovic]|
| [Cassandra SSTable generator |
https://github.com/instaclustr/cassandra-sstable-generator]|{color:#00875A}*DONE*{color}|
[~stefan.miklosovic]|
| [Cassandra TTL Remover | https://github.com/instaclustr/TTLRemover] |
{color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
| [Cassandra Everywhere Strategy |
https://github.com/instaclustr/cassandra-everywhere-strategy] |
{color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
| [Reaper|http://cassandra-reaper.io/]| {color:#00875A}*AUTOMATIC*{color} |
[~adejanovski]|
| [Medusa|https://github.com/thelastpickle/cassandra-medusa]| *IN PROGRESS*|
[~adejanovski]|
| [Casskop|https://orange-opensource.github.io/casskop/]| *NOT STARTED*| Franck
Dehay|
|
[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211259#comment-17211259
]
Adam Holmberg commented on CASSANDRA-15299:
---
Thanks for the additional detail.
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> particular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed,
> and that we do it before native protocol v5 graduates from beta. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
> and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16152) In-JVM dtest - modify schema with stopped nodes and use yaml fragments for config
[ https://issues.apache.org/jira/browse/CASSANDRA-16152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211243#comment-17211243 ] Jon Meredith commented on CASSANDRA-16152: -- I think that would be a nice improvement. I backed off doing the dtest-api release dance as I didn't see enough use-cases in the config to be worth delaying the work backed up behind this. > In-JVM dtest - modify schema with stopped nodes and use yaml fragments for > config > - > > Key: CASSANDRA-16152 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16152 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > > Some convenience improvements to in-JVM dtest that are useful across versions > that I needed while working on CASSANDRA-16144 > * Add support for changing schema with stopped nodes. > * Make it simpler to modify nested configuration items by specifying Yaml > fragments -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211237#comment-17211237 ] David Capwell commented on CASSANDRA-16150: --- Starting commit CI Results (pending): Branch: trunk Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16150-trunk-89A170D2-1BB1-4D7D-ADA0-7D9942489E9C Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/88/ > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.0-beta > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16146) Node state incorrectly set to NORMAL after nodetool disablegossip and enablegossip during bootstrap
[
https://issues.apache.org/jira/browse/CASSANDRA-16146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211232#comment-17211232
]
Brandon Williams commented on CASSANDRA-16146:
--
+1
> Node state incorrectly set to NORMAL after nodetool disablegossip and
> enablegossip during bootstrap
> ---
>
> Key: CASSANDRA-16146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16146
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta3
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> At high level, {{StorageService#setGossipTokens}} set the gossip state to
> {{NORMAL}} blindly. Therefore, re-enabling gossip (stop and start gossip)
> overrides the actual gossip state.
>
> It could happen in the below scenario.
> # Bootstrap failed. The gossip state remains in {{BOOT}} / {{JOINING}} and
> code execution exits StorageService#initServer.
> # Operator runs nodetool to stop and re-start gossip. The gossip state gets
> flipped to {{NORMAL}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211229#comment-17211229 ] Brandon Williams commented on CASSANDRA-16150: -- Nope, go for it. > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.0-beta > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16155) ByteBufferAccessor cast exceptions are thrown when trying to query a virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-16155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211227#comment-17211227
]
David Capwell commented on CASSANDRA-16155:
---
[~ifesdjeen] this is for test table test_virtual_ks.vt3, was hoping we could
have a test for the table that actually broke which triggered this. The
comment was more that a public table failed and we didn't have test coverage to
detect it, so would be good to have a basic test that makes sure it mostly
works. I am 100% fine with the test you have, more asking for an additional
one.
bq. David Capwell you've also mentioned in review comment "for me: the issue
was that the method didn't take it, but takes Object...", what do you think can
be an alternative here? Unfortunately we do not have a more specific type to
prevent something similar happening in the future.
When it comes to Object... that gets tricky as it traps everything... not sure
how we could prevent this from happening with that signature... The best hack
I can think of is a static analyzer tool (custom to us) that checks these
things, nothing else is coming to mind.
bq. David Capwell you've also mentioned in review comment "for me:
Since reviews can go back-and-forth over multiple days, I tend to leave "for
me" comments to remind me later of stuff I knew while reviewing as I will
likely forget this the next time I need to look at it; these comments can
mostly be ignored as they are just review documentation.
> ByteBufferAccessor cast exceptions are thrown when trying to query a virtual
> table
> --
>
> Key: CASSANDRA-16155
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16155
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Virtual Tables
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
>
> Start a fresh trunk node, and try to run
> SELECT * FROM system_views.local_read_latency ;
> You’ll get:
> {code:java}
> ERROR [Native-Transport-Requests-1] 2020-09-30 09:44:45,099
> ErrorMessage.java:457 - Unexpected exception during request
> java.lang.ClassCastException:
> org.apache.cassandra.db.marshal.ByteBufferAccessor cannot be cast to
> java.lang.String
> at
> org.apache.cassandra.serializers.AbstractTextSerializer.serialize(AbstractTextSerializer.java:29)
> at
> org.apache.cassandra.db.marshal.AbstractType.decompose(AbstractType.java:131)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16150: -- Status: Ready to Commit (was: Review In Progress) > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.x > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211219#comment-17211219 ] David Capwell commented on CASSANDRA-16150: --- [~brandon.williams] have any issues with 1.26, else I can merge this today. > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.x > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16150: -- Fix Version/s: (was: 4.x) 4.0-beta > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.0-beta > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16127) NullPointerException when calling nodetool enablethrift
[
https://issues.apache.org/jira/browse/CASSANDRA-16127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211179#comment-17211179
]
David Capwell commented on CASSANDRA-16127:
---
Doh, I knew about this but forgot to update the test... the error is ok to
ignore
{code}
if self.cluster.version() >= LooseVersion('4.0'):
> self.assert_log_had_msg(node2, 'Not starting client transports as
> bootstrap has not completed')
{code}
that log was added in 4.0 and made the logic more complex as it copy/pasted the
validation logic just to rename the string that was used before. I felt it was
best to keep the branches in-sync so removed that log in favor of the previous
log, here is the code that the jvm-dtest checks for and the python version also
checks for pre-4.0
{code}
throw new IllegalStateException("Node is not yet bootstrapped completely. Use
nodetool to check bootstrap" +
" state and resume. For
more, see `nodetool help bootstrap`");
{code}
> NullPointerException when calling nodetool enablethrift
> ---
>
> Key: CASSANDRA-16127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16127
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Thrift
>Reporter: Tibor Repasi
>Assignee: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x
>
>
> Having thrift disabled, it's impossible to enable it again without restarting
> the node:
> {code}
> $ nodetool statusthrift
> not running
> $ nodetool enablethrift
> error: null
> -- StackTrace --
> java.lang.NullPointerException
> at
> org.apache.cassandra.service.StorageService.startRPCServer(StorageService.java:392)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
[jira] [Commented] (CASSANDRA-16127) NullPointerException when calling nodetool enablethrift
[
https://issues.apache.org/jira/browse/CASSANDRA-16127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211174#comment-17211174
]
David Capwell commented on CASSANDRA-16127:
---
Yep, will check the test failure; the test failed in trunk for Circle and
Jenkins, so doesn't feel like a 1-off flaky test.
[~e.dimitrova] I will take [~adelapena] version in slack and see how it works,
this will also make it easier to document as my current document is hidden in a
random function.
> NullPointerException when calling nodetool enablethrift
> ---
>
> Key: CASSANDRA-16127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16127
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Thrift
>Reporter: Tibor Repasi
>Assignee: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x
>
>
> Having thrift disabled, it's impossible to enable it again without restarting
> the node:
> {code}
> $ nodetool statusthrift
> not running
> $ nodetool enablethrift
> error: null
> -- StackTrace --
> java.lang.NullPointerException
> at
> org.apache.cassandra.service.StorageService.startRPCServer(StorageService.java:392)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211167#comment-17211167
]
Sam Tunnicliffe commented on CASSANDRA-15299:
-
[~aholmber] review, plus I'm running some stress tests, which should be done
soon. After those two things are finished, there's just some internal renaming
to do, which I held off doing until now to avoid polluting the changeset. Not
to preempt [~maedhroz] 's review, but I think the wire format is pretty stable
now, and the drivers are working well. I've used the python driver branch that
[~aboudreault] referenced to re-enable the v5 dtests, which seems fine (there
are a handful of failures, but I think those are unrelated).
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> particular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed,
> and that we do it before native protocol v5 graduates from beta. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
> and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16191) Add tests for Repair metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fábio Takeo Ueno reassigned CASSANDRA-16191:
Assignee: Fábio Takeo Ueno
> Add tests for Repair metrics
>
>
> Key: CASSANDRA-16191
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16191
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: Benjamin Lerer
>Assignee: Fábio Takeo Ueno
>Priority: Normal
> Fix For: 4.0-beta
>
>
> We do not seems to have some tests for the {{RepairMetrics.previewFailures}}
> counter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fábio Takeo Ueno reassigned CASSANDRA-16192: Assignee: (was: Fábio Takeo Ueno) > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211165#comment-17211165
]
Adam Holmberg commented on CASSANDRA-15299:
---
Great to hear. Thanks for checking in. Not trying to hassle, just looking for
assurance we're not stuck.
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> particular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed,
> and that we do it before native protocol v5 graduates from beta. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
> and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16146) Node state incorrectly set to NORMAL after nodetool disablegossip and enablegossip during bootstrap
[
https://issues.apache.org/jira/browse/CASSANDRA-16146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-16146:
-
Reviewers: Blake Eggleston, Brandon Williams, Brandon Williams (was: Blake
Eggleston, Brandon Williams)
Blake Eggleston, Brandon Williams, Brandon Williams (was: Blake
Eggleston)
Status: Review In Progress (was: Patch Available)
> Node state incorrectly set to NORMAL after nodetool disablegossip and
> enablegossip during bootstrap
> ---
>
> Key: CASSANDRA-16146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16146
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta3
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> At high level, {{StorageService#setGossipTokens}} set the gossip state to
> {{NORMAL}} blindly. Therefore, re-enabling gossip (stop and start gossip)
> overrides the actual gossip state.
>
> It could happen in the below scenario.
> # Bootstrap failed. The gossip state remains in {{BOOT}} / {{JOINING}} and
> code execution exits StorageService#initServer.
> # Operator runs nodetool to stop and re-start gossip. The gossip state gets
> flipped to {{NORMAL}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16204) PicklingError: Can't pickle : attribute lookup video_encoding on cqlshlib.copyutil failed
[
https://issues.apache.org/jira/browse/CASSANDRA-16204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phil Miesle updated CASSANDRA-16204:
Description:
This seems to be a different issue than Cassandra-14982 :
$ cqlsh --version
cqlsh 6.8.0
Following Datastax Academy course DS220, in the Denormalization exercise. I
received
{code:java}
PicklingError: Can't pickle :
attribute lookup video_encoding on cqlshlib.copyutil failed{code}
on the COPY command into the following table:
{code:java}
CREATE TABLE videos_by_actor (
actor text,
added_date timestamp,
video_id timeuuid,
character_name text,
description text,
encoding frozen,
tags set,
title text,
user_id uuid,
PRIMARY KEY ( (actor), added_date, video_id )
) WITH CLUSTERING ORDER BY ( added_date desc, video_id asc);
COPY
videos_by_actor(actor,added_date,video_id,character_name,description,encoding,tags,title,user_id)
FROM 'videos_by_actor.csv' WITH HEADER = true{code}
Now, as it turns out my PRIMARY KEY was non-unique (noted when I failed to load
as many records as were in the file), and when I changed to:
{code:java}
PRIMARY KEY ((actor), added_date, video_id, character_name){code}
the command worked. BUT the following options also worked (though they both
dropped records):
{code:java}
WITH HEADER = true AND MINBATCHSIZE=1 AND MAXBATCHSIZE=1 AND PAGESIZE=10;{code}
and
{code:java}
WITH HEADER = true AND NUMPROCESSES=1;{code}
So this seems to be a problem of multi-threading and user-defined TYPEs?
I'll note that I'm running inside a WSL2 Docker Container based on
ubuntu:bionic:
$ uname -a
Linux node-1 4.19.104-microsoft-standard #1 SMP Wed Feb 19 06:37:35 UTC 2020
x86_64 x86_64 x86_64 GNU/Linux
was:
This seems to be a different issue than Cassandra-14982 :
$ cqlsh --version
cqlsh 6.8.0
Following Datastax Academy course DS220, in the Denormalization exercise. I
received
{code:java}
PicklingError: Can't pickle :
attribute lookup video_encoding on cqlshlib.copyutil failed{code}
on the COPY command into the following table:
{code:java}
CREATE TABLE videos_by_actor (
actor text,
added_date timestamp,
video_id timeuuid,
character_name text,
description text,
encoding frozen,
tags set,
title text,
user_id uuid,
PRIMARY KEY ( (actor), added_date, video_id )
) WITH CLUSTERING ORDER BY ( added_date desc, video_id asc);
COPY
videos_by_actor(actor,added_date,video_id,character_name,description,encoding,tags,title,user_id)
FROM 'videos_by_actor.csv' WITH HEADER = true{code}
Now, as it turns out my PRIMARY KEY was non-unique (noted when I failed to load
as many records as were in the file), and when I changed to:
{code:java}
PRIMARY KEY ((actor), added_date, video_id, character_name){code}
the command worked. BUT the following options also worked (though they both
dropped records):
{code:java}
WITH HEADER = true AND MINBATCHSIZE=1 AND MAXBATCHSIZE=1 AND PAGESIZE=10;{code}
and
{code:java}
WITH HEADER = true AND NUMPROCESSES=1;{code}
So this seems to be a problem of multi-threading and user-defined TYPEs?
> PicklingError: Can't pickle :
> attribute lookup video_encoding on cqlshlib.copyutil failed
> ---
>
> Key: CASSANDRA-16204
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16204
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Phil Miesle
>Priority: Normal
> Attachments: videos_by_actor.csv.gz
>
>
> This seems to be a different issue than Cassandra-14982 :
> $ cqlsh --version
> cqlsh 6.8.0
> Following Datastax Academy course DS220, in the Denormalization exercise. I
> received
> {code:java}
> PicklingError: Can't pickle :
> attribute lookup video_encoding on cqlshlib.copyutil failed{code}
> on the COPY command into the following table:
> {code:java}
> CREATE TABLE videos_by_actor (
> actor text,
> added_date timestamp,
> video_id timeuuid,
> character_name text,
> description text,
> encoding frozen,
> tags set,
> title text,
> user_id uuid,
> PRIMARY KEY ( (actor), added_date, video_id )
> ) WITH CLUSTERING ORDER BY ( added_date desc, video_id asc);
> COPY
> videos_by_actor(actor,added_date,video_id,character_name,description,encoding,tags,title,user_id)
> FROM 'videos_by_actor.csv' WITH HEADER = true{code}
> Now, as it turns out my PRIMARY KEY was non-unique (noted when I failed to
> load as many records as were in the file), and when I changed to:
> {code:java}
> PRIMARY KEY ((actor), added_date, video_id, character_name){code}
> the command worked. BUT the following options also worked (though they both
> dropped records):
> {code:java}
> WITH HEADER = true AND MINBATCHSIZE=1 AND MAXBATCHSIZE=1 AND
> PAGESIZE=10;{code}
> and
> {code:java}
> WITH HEADER = true AND NUMPROCESSES=1;{code}
> So this
[jira] [Created] (CASSANDRA-16204) PicklingError: Can't pickle : attribute lookup video_encoding on cqlshlib.copyutil failed
Phil Miesle created CASSANDRA-16204:
---
Summary: PicklingError: Can't pickle : attribute lookup video_encoding on
cqlshlib.copyutil failed
Key: CASSANDRA-16204
URL: https://issues.apache.org/jira/browse/CASSANDRA-16204
Project: Cassandra
Issue Type: Bug
Components: CQL/Interpreter
Reporter: Phil Miesle
Attachments: videos_by_actor.csv.gz
This seems to be a different issue than Cassandra-14982 :
$ cqlsh --version
cqlsh 6.8.0
Following Datastax Academy course DS220, in the Denormalization exercise. I
received
{code:java}
PicklingError: Can't pickle :
attribute lookup video_encoding on cqlshlib.copyutil failed{code}
on the COPY command into the following table:
{code:java}
CREATE TABLE videos_by_actor (
actor text,
added_date timestamp,
video_id timeuuid,
character_name text,
description text,
encoding frozen,
tags set,
title text,
user_id uuid,
PRIMARY KEY ( (actor), added_date, video_id )
) WITH CLUSTERING ORDER BY ( added_date desc, video_id asc);
COPY
videos_by_actor(actor,added_date,video_id,character_name,description,encoding,tags,title,user_id)
FROM 'videos_by_actor.csv' WITH HEADER = true{code}
Now, as it turns out my PRIMARY KEY was non-unique (noted when I failed to load
as many records as were in the file), and when I changed to:
{code:java}
PRIMARY KEY ((actor), added_date, video_id, character_name){code}
the command worked. BUT the following options also worked (though they both
dropped records):
{code:java}
WITH HEADER = true AND MINBATCHSIZE=1 AND MAXBATCHSIZE=1 AND PAGESIZE=10;{code}
and
{code:java}
WITH HEADER = true AND NUMPROCESSES=1;{code}
So this seems to be a problem of multi-threading and user-defined TYPEs?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211093#comment-17211093
]
Caleb Rackliffe commented on CASSANDRA-15299:
-
[~aholmber] I'm _trying_ to finalize my review today.
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> particular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed,
> and that we do it before native protocol v5 graduates from beta. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
> and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211090#comment-17211090
]
Adam Holmberg commented on CASSANDRA-15299:
---
Is this waiting on further work from you Sam, or more review?
Just wanted to touch and make sure it's not deadlocked somehow by missed
assumptions. There are drivers and server stuff that can be broken loose when
this is finalized.
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> particular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed,
> and that we do it before native protocol v5 graduates from beta. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
> and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16155) ByteBufferAccessor cast exceptions are thrown when trying to query a virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-16155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211089#comment-17211089
]
Caleb Rackliffe commented on CASSANDRA-16155:
-
bq. do you have any objections to committing a test that's similar to the rest
of VirtualTableTest tests?
[~ifesdjeen] That's fine w/ me. (The idea behind my {{SimpleDataSetTest}} being
so isolated is that it should cover all current and _future_ usages of that
path in {{SimpleDataSet}}.)
> ByteBufferAccessor cast exceptions are thrown when trying to query a virtual
> table
> --
>
> Key: CASSANDRA-16155
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16155
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Virtual Tables
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
>
> Start a fresh trunk node, and try to run
> SELECT * FROM system_views.local_read_latency ;
> You’ll get:
> {code:java}
> ERROR [Native-Transport-Requests-1] 2020-09-30 09:44:45,099
> ErrorMessage.java:457 - Unexpected exception during request
> java.lang.ClassCastException:
> org.apache.cassandra.db.marshal.ByteBufferAccessor cannot be cast to
> java.lang.String
> at
> org.apache.cassandra.serializers.AbstractTextSerializer.serialize(AbstractTextSerializer.java:29)
> at
> org.apache.cassandra.db.marshal.AbstractType.decompose(AbstractType.java:131)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16127) NullPointerException when calling nodetool enablethrift
[
https://issues.apache.org/jira/browse/CASSANDRA-16127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211054#comment-17211054
]
Ekaterina Dimitrova commented on CASSANDRA-16127:
-
Unfortunately, I found this dtest failure in Jenkins for trunk:
[https://ci-cassandra.apache.org/job/Cassandra-devbranch/87/testReport/junit/dtest.bootstrap_test/TestBootstrap/test_bootstrap_binary_disabled/]
Please check it
> NullPointerException when calling nodetool enablethrift
> ---
>
> Key: CASSANDRA-16127
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16127
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Thrift
>Reporter: Tibor Repasi
>Assignee: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x
>
>
> Having thrift disabled, it's impossible to enable it again without restarting
> the node:
> {code}
> $ nodetool statusthrift
> not running
> $ nodetool enablethrift
> error: null
> -- StackTrace --
> java.lang.NullPointerException
> at
> org.apache.cassandra.service.StorageService.startRPCServer(StorageService.java:392)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15865) Flaky dtest hintedhandoff_test.py::TestHintedHandoffConfig::test_hintedhandoff_setmaxwindow
[
https://issues.apache.org/jira/browse/CASSANDRA-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211050#comment-17211050
]
Brandon Williams commented on CASSANDRA-15865:
--
{code}
test_hintedhandoff_setmaxwindow failed and was not selected for rerun.
Expected [Row(c1='value1', c2='value2')] to have length 0, but instead
is of length 1
[, , , ]
{code}
> Flaky dtest
> hintedhandoff_test.py::TestHintedHandoffConfig::test_hintedhandoff_setmaxwindow
> ---
>
> Key: CASSANDRA-15865
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15865
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Sam Tunnicliffe
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-beta
>
>
> I've seen this fail a couple of times under JDK11, when it doesn't appear to
> be related to the changes under test.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16012) sstablesplit unit test hardening
[
https://issues.apache.org/jira/browse/CASSANDRA-16012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-16012:
---
Fix Version/s: (was: 4.0-beta)
4.0-beta3
Since Version: 1.2.9
Source Control Link:
https://github.com/apache/cassandra/commit/4564e102684dc5f66ec73de1fc836f97a1fa33c9
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> sstablesplit unit test hardening
>
>
> Key: CASSANDRA-16012
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16012
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Labels: low-hanging-fruit
> Fix For: 4.0-beta3
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
>
> During CASSANDRA-15883 / CASSANDRA-15991 it was detected unit test coverage
> for this tool is minimal. There is a unit test to enhance upon under
> {{test/unit/org/apache/cassandra/tools}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables
[
https://issues.apache.org/jira/browse/CASSANDRA-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211027#comment-17211027
]
Andres de la Peña commented on CASSANDRA-16063:
---
{quote}My suggestion is to not add it back for now and continue that work in
the other ticket. I am almost 99% sure we will utilize his test at the end.
{quote}
Fine with me.
Running a final round of CI upgrade tests before commit:
||[3.0|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest-upgrade/6/]||[3.11|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest-upgrade/7/]||[trunk|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest-upgrade/8/]||
> Fix user experience when upgrading to 4.0 with compact tables
> -
>
> Key: CASSANDRA-16063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16063
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Sylvain Lebresne
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: Compact_storage_upgrade_tests.txt
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The code to handle compact tables has been removed from 4.0, and the intended
> upgrade path to 4.0 for users having compact tables on 3.x is that they must
> execute {{ALTER ... DROP COMPACT STORAGE}} on all of their compact tables
> *before* attempting the upgrade.
> Obviously, some users won't read the upgrade instructions (or miss a table)
> and may try upgrading despite still having compact tables. If they do so, the
> intent is that the node will _not_ start, with a message clearly indicating
> the pre-upgrade step the user has missed. The user will then downgrade back
> the node(s) to 3.x, run the proper {{ALTER ... DROP COMPACT STORAGE}}, and
> then upgrade again.
> But while 4.0 does currently fail startup when finding any compact tables
> with a decent message, I believe the check is done too late during startup.
> Namely, that check is done as we read the tables schema, so within
> [{{Schema.instance.loadFromDisk()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L241].
> But by then, we've _at least_ called
> {{SystemKeyspace.persistLocalMetadata()}}} and
> {{SystemKeyspaceMigrator40.migrate()}}, which will get into the commit log,
> and even possibly flush new {{na}} format sstables. As a results, a user
> might not be able to seemlessly restart the node on 3.x (to drop compact
> storage on the appropriate tables).
> Basically, we should make sure the check for compact tables done at 4.0
> startup is done as a {{StartupCheck}}, before the node does anything.
> We should also add a test for this (checking that if you try upgrading to 4.0
> with compact storage, you can downgrade back with no intervention whatsoever).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: sstablesplit tool unit testing
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 4564e10 sstablesplit tool unit testing
4564e10 is described below
commit 4564e102684dc5f66ec73de1fc836f97a1fa33c9
Author: Bereng
AuthorDate: Thu Oct 1 10:56:17 2020 +0200
sstablesplit tool unit testing
patch by Berenguer Blasi; reviewed by Yifan Cai, Mick Semb Wever for
CASSANDRA-16012
---
.../apache/cassandra/tools/StandaloneSplitter.java | 6 +-
src/java/org/apache/cassandra/tools/Util.java | 1 +
.../tools/StandaloneSplitterWithCQLTesterTest.java | 188 +
3 files changed, 194 insertions(+), 1 deletion(-)
diff --git a/src/java/org/apache/cassandra/tools/StandaloneSplitter.java
b/src/java/org/apache/cassandra/tools/StandaloneSplitter.java
index 7a60b6f..e15e5bc 100644
--- a/src/java/org/apache/cassandra/tools/StandaloneSplitter.java
+++ b/src/java/org/apache/cassandra/tools/StandaloneSplitter.java
@@ -26,6 +26,7 @@ import org.apache.cassandra.schema.Schema;
import org.apache.cassandra.io.sstable.format.SSTableReader;
import org.apache.commons.cli.*;
+import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.db.ColumnFamilyStore;
import org.apache.cassandra.db.compaction.OperationType;
import org.apache.cassandra.db.Directories;
@@ -51,7 +52,10 @@ public class StandaloneSplitter
public static void main(String args[])
{
Options options = Options.parseArgs(args);
-Util.initDatabaseDescriptor();
+if (Boolean.getBoolean(Util.ALLOW_TOOL_REINIT_FOR_TEST))
+DatabaseDescriptor.toolInitialization(false); //Necessary for
testing
+else
+Util.initDatabaseDescriptor();
try
{
diff --git a/src/java/org/apache/cassandra/tools/Util.java
b/src/java/org/apache/cassandra/tools/Util.java
index db664aa..7411858 100644
--- a/src/java/org/apache/cassandra/tools/Util.java
+++ b/src/java/org/apache/cassandra/tools/Util.java
@@ -54,6 +54,7 @@ import com.google.common.collect.Lists;
@SuppressWarnings("serial")
public final class Util
{
+static final String ALLOW_TOOL_REINIT_FOR_TEST = Util.class.getName() +
"ALLOW_TOOL_REINIT_FOR_TEST"; // Necessary for testing
static final String RESET = "\u001B[0m";
static final String BLUE = "\u001B[34m";
static final String CYAN = "\u001B[36m";
diff --git
a/test/unit/org/apache/cassandra/tools/StandaloneSplitterWithCQLTesterTest.java
b/test/unit/org/apache/cassandra/tools/StandaloneSplitterWithCQLTesterTest.java
new file mode 100644
index 000..9785d84
--- /dev/null
+++
b/test/unit/org/apache/cassandra/tools/StandaloneSplitterWithCQLTesterTest.java
@@ -0,0 +1,188 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.tools;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import com.google.common.io.Files;
+
+import org.junit.Test;
+import org.junit.runner.RunWith;
+
+import org.apache.cassandra.OrderedJUnit4ClassRunner;
+import org.apache.cassandra.cql3.CQLTester;
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.io.sstable.format.SSTableReader;
+import org.apache.cassandra.tools.ToolRunner.ToolResult;
+import org.assertj.core.api.Assertions;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+
+@RunWith(OrderedJUnit4ClassRunner.class)
+public class StandaloneSplitterWithCQLTesterTest extends CQLTester
+{
+private static String sstableFileName;
+private static File sstablesDir;
+private static File sstablesBackupDir;
+private static List origSstables;
+
+// CQLTester post test method cleanup needs to be avoided by overriding as
it'd clean all sstables, env, etc.
+@Override
+public void afterTest() throws Throwable
+{
+}
+
+@Test
+public void setupEnv() throws Throwable
+{
+// Stop the server after
