[jira] [Updated] (CASSANDRA-15937) JMX output inconsistencies from CASSANDRA-7544 storage-port-configurable-per-node
[
https://issues.apache.org/jira/browse/CASSANDRA-15937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Meredith updated CASSANDRA-15937:
-
Bug Category: Parent values: Code(13163)
Complexity: Normal
Discovered By: Code Inspection
Fix Version/s: 4.0-beta
Severity: Low
Status: Open (was: Triage Needed)
> JMX output inconsistencies from CASSANDRA-7544
> storage-port-configurable-per-node
> -
>
> Key: CASSANDRA-15937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15937
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/JMX
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta
>
>
> CASSANDRA-7544 introduced changes to allow the storage port number to be
> configured per-node. As part of that work it introduces new MBeans for
> MessagingService, FailureDetector providing new 'WithPort' versions that
> include the new port information, however there are some mistakes and
> inconsistencies.
> {code:java}
> 3.11.6 trunk trunk
> w/Port Notes
>
> AllEndpointStates /127.0.0.1\n... /127.0.0.3\n...
> 127.0.0.3:7000\n (trunk /w port different)
> SimpleStates /127.0.0.2=UP /127.0.0.2=UP
> 127.0.0.3:7000=UP (trunk /w port different)
> LargeMessagePendingTasks /127.0.0.1=0 /127.0.0.1=0
> 127.0.0.3:7000=0 (trunk /w port different)
> TimeoutsPerHost 127.0.0.1=0 /127.0.0.1=0
> 127.0.0.3:7000=0 3.0/3.11.6 & trunk differ.
> BackPressurePerHost 127.0.0.1=Infinity /127.0.0.2=Infinity
> /127.0.0.2=Infinity 3.11 & trunk differ, missing port number for
> BackPressurePerHostWithPort
> SchemaVersions {...=[127.0.0.1,...]} {...=[127.0.0.1,...]}
> {...=[127.0.0.1:7000,...]
>
> TokenToEndpointMap {-92...8=127.0.0.1, -92...8=127.0.0.1
> -92..8=127.0.0.1:7000
> HostIdMap 127.0.0.1=1ee..6f0af 127.0.0.1=e06...7e MISSING
> Deprecated for EndpointToHostId
> EndpointToHostId 127.0.0.1=1ee..6f0a 127.0.0.1=e06...7e
> 127.0.0.1:7000=e0..7e
> HostIdToEndpoint 1ee..6f0a=127.0.0.1 e06..7e=127.0.0.1
> e06..7e=127.0.0.1:7000
> LoadMap 127.0.0.1=185.01 KiB 127.0.0.1:7000=106.08 KiB
> 127.0.0.1=106.08 Ki LoadMap and LoadMapWithPort are flipped.
> LiveNodes 127.0.0.1 127.0.0.1
> 127.0.0.1:7000
> Ownership /127.0.0.1=0.33 /127.0.0.1=0.33
> 127.0.0.1:7000=0.33
> Scores /127.0.0.1=0.0 /127.0.0.1=0.0
> 127.0.0.1:7000=0.0
> {code}
>
> Proposed changes
>
> 1) AllEndpointStats, SimpleStates, Connection message tracking,
> TimeoutsPerHost - include the host/ip:port in the WithPort version
> 2) Add port number to BackPressurePerHostWithPort
> 3) Correct LoadMap to omit port / LoadMapWithPort to include port
> 4) Ownership - update with port to host/ip:port version
> 5) Scores - update with port to host/ip:port version
>
>
> Additionally while dumping out all of the JMX info with `sjk mxdump`
>
> 6) DynamicEndpointSnitch.getScoresWithPort now returns an InetAddressAndPort
> which should just be a String
> 7) ClientMetrics.clientsByProtocolVersion returns a Guava Immutable map
> 8) StorageService.getIdealConsistencyLevel fails if none set (as we try and
> call ConsistencyLevel.toString on a null pointer).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15937) JMX output inconsistencies from CASSANDRA-7544 storage-port-configurable-per-node
Jon Meredith created CASSANDRA-15937:
Summary: JMX output inconsistencies from CASSANDRA-7544
storage-port-configurable-per-node
Key: CASSANDRA-15937
URL: https://issues.apache.org/jira/browse/CASSANDRA-15937
Project: Cassandra
Issue Type: Bug
Components: Observability/JMX
Reporter: Jon Meredith
Assignee: Jon Meredith
CASSANDRA-7544 introduced changes to allow the storage port number to be
configured per-node. As part of that work it introduces new MBeans for
MessagingService, FailureDetector providing new 'WithPort' versions that
include the new port information, however there are some mistakes and
inconsistencies.
{code:java}
3.11.6 trunk trunk
w/Port Notes
AllEndpointStates /127.0.0.1\n... /127.0.0.3\n...
127.0.0.3:7000\n (trunk /w port different)
SimpleStates /127.0.0.2=UP /127.0.0.2=UP
127.0.0.3:7000=UP (trunk /w port different)
LargeMessagePendingTasks /127.0.0.1=0 /127.0.0.1=0
127.0.0.3:7000=0 (trunk /w port different)
TimeoutsPerHost 127.0.0.1=0 /127.0.0.1=0
127.0.0.3:7000=0 3.0/3.11.6 & trunk differ.
BackPressurePerHost 127.0.0.1=Infinity /127.0.0.2=Infinity
/127.0.0.2=Infinity 3.11 & trunk differ, missing port number for
BackPressurePerHostWithPort
SchemaVersions {...=[127.0.0.1,...]} {...=[127.0.0.1,...]}
{...=[127.0.0.1:7000,...]
TokenToEndpointMap {-92...8=127.0.0.1, -92...8=127.0.0.1
-92..8=127.0.0.1:7000
HostIdMap 127.0.0.1=1ee..6f0af 127.0.0.1=e06...7e MISSING
Deprecated for EndpointToHostId
EndpointToHostId 127.0.0.1=1ee..6f0a 127.0.0.1=e06...7e
127.0.0.1:7000=e0..7e
HostIdToEndpoint 1ee..6f0a=127.0.0.1 e06..7e=127.0.0.1
e06..7e=127.0.0.1:7000
LoadMap 127.0.0.1=185.01 KiB 127.0.0.1:7000=106.08 KiB
127.0.0.1=106.08 Ki LoadMap and LoadMapWithPort are flipped.
LiveNodes 127.0.0.1 127.0.0.1
127.0.0.1:7000
Ownership /127.0.0.1=0.33 /127.0.0.1=0.33
127.0.0.1:7000=0.33
Scores /127.0.0.1=0.0 /127.0.0.1=0.0
127.0.0.1:7000=0.0
{code}
Proposed changes
1) AllEndpointStats, SimpleStates, Connection message tracking,
TimeoutsPerHost - include the host/ip:port in the WithPort version
2) Add port number to BackPressurePerHostWithPort
3) Correct LoadMap to omit port / LoadMapWithPort to include port
4) Ownership - update with port to host/ip:port version
5) Scores - update with port to host/ip:port version
Additionally while dumping out all of the JMX info with `sjk mxdump`
6) DynamicEndpointSnitch.getScoresWithPort now returns an InetAddressAndPort
which should just be a String
7) ClientMetrics.clientsByProtocolVersion returns a Guava Immutable map
8) StorageService.getIdealConsistencyLevel fails if none set (as we try and
call ConsistencyLevel.toString on a null pointer).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15809) ASF CI builds for JDK11
[ https://issues.apache.org/jira/browse/CASSANDRA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15809: --- Reviewers: Michael Semb Wever, Michael Semb Wever (was: Michael Semb Wever) Michael Semb Wever, Michael Semb Wever (was: Michael Semb Wever) Status: Review In Progress (was: Patch Available) > ASF CI builds for JDK11 > --- > > Key: CASSANDRA-15809 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15809 > Project: Cassandra > Issue Type: Task > Components: Build, CI >Reporter: Michael Semb Wever >Assignee: shylaja kokoori >Priority: Normal > Fix For: 4.0-beta > > Attachments: Screenshot 2020-05-13 at 09.39.56.png, > build_with_multiple_jdk.patch > > > ASF CI builds today only run on JDK1.8 > On the Jenkins cluster JDKs from 1.4 through to 15 are available. See > attached screenshot for naming specifics. > This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for > parity to CircleCI's workflows. (There is also the question about > testing/running on Cassandra-2.2 which needs to support JDK1.7, though > Cassandra-2.2 is nearing EOL.) > > The JDK is specified in the groovy DSL: > [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11] > > > Some examples: > * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with > both JDK1.8 and JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8] > * CircleCI JDK11 workflow example. This builds and tests only with JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306] > * Jenkins Cqlshlib tests showing matrix builds: > ** > [https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/] > > Background > thread:[https://lists.apache.org/thread.html/rbaeb960901fa53b50227b37d64e5a456b68f749f4229c6e3e086ff85%40%3Cdev.cassandra.apache.org%3E] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15859) Avoid per-host hinted-handoff throttle being rounded to 0 in large cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-15859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15859:
---
Fix Version/s: (was: 3.11.x)
(was: 4.x)
(was: 3.0.x)
4.0-alpha5
4.0
3.11.7
3.0.21
> Avoid per-host hinted-handoff throttle being rounded to 0 in large cluster
> --
>
> Key: CASSANDRA-15859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15859
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0, 4.0-alpha5
>
>
> When "hinted_handoff_throttle_in_kb" is sufficiently small or num of nodes in
> the cluster is sufficiently large, the per-host throttle will be rounded to
> 0, aka. unthrottled.
>
> {code:java|title=HintsDispatchExecutor.java}
> int throttleInKB = DatabaseDescriptor.getHintedHandoffThrottleInKB() /
> nodesCount;
> this.rateLimiter = RateLimiter.create(throttleInKB == 0 ? Double.MAX_VALUE :
> throttleInKB * 1024);
> {code}
> [trunk-patch|https://github.com/apache/cassandra/pull/616]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15924) Avoid emitting empty range tombstones from RangeTombstoneList
[
https://issues.apache.org/jira/browse/CASSANDRA-15924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15924:
---
Fix Version/s: (was: 4.0-alpha)
4.0-alpha5
4.0
> Avoid emitting empty range tombstones from RangeTombstoneList
> -
>
> Key: CASSANDRA-15924
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15924
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0, 4.0-alpha5
>
>
> In {{RangeTombstoneList#iterator}} there is a chance we emit empty range
> tombstones depending on the slice passed in. This can happen during read
> repair with either an empty slice or with paging and the final page being
> empty.
> This creates problems in RTL if we try to insert a new range tombstone which
> covers the empty ones;
> {code}
> Caused by: java.lang.AssertionError
> at
> org.apache.cassandra.db.RangeTombstoneList.insertFrom(RangeTombstoneList.java:541)
> at
> org.apache.cassandra.db.RangeTombstoneList.addAll(RangeTombstoneList.java:217)
> at
> org.apache.cassandra.db.MutableDeletionInfo.add(MutableDeletionInfo.java:141)
> at
> org.apache.cassandra.db.partitions.AtomicBTreePartition.addAllWithSizeDelta(AtomicBTreePartition.java:137)
> at org.apache.cassandra.db.Memtable.put(Memtable.java:254)
> at
> org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1210)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:573)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:421)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:210)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:215)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:224)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternalWithoutCondition(ModificationStatement.java:582)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:572)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15857) Frozen RawTuple is not annotated with frozen in the toString method
[
https://issues.apache.org/jira/browse/CASSANDRA-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155680#comment-17155680
]
Yifan Cai commented on CASSANDRA-15857:
---
bq. As tuples cannot be frozen at all
Is it a typo? According to the document, tuple is always frozen.
https://docs.datastax.com/en/dse/6.8/cql/cql/cql_reference/refDataTypes.html?hl=tuple%2Cfrozen#refDataTypes__tuple
The parsing rules defined at {{Parser.g}} only freeze the type when there is a
frozen keyword.
Are you suggesting to freeze the RawTuple when constructing the object? So we
can get rid of the {{frozen}} param.
> Frozen RawTuple is not annotated with frozen in the toString method
> ---
>
> Key: CASSANDRA-15857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15857
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0, 3.11.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> All raw types (e.g. RawCollection, RawUT) that supports freezing wraps the
> type name with 'frozen<>' in the toString method, except RawTuple.
> Therefore, the RawTuple::toString output misses the frozen wrapper.
> Tuple is always frozen. However since CASSANDRA-15035, it throws when the
> inner tuple is not explicitly wrapped with frozen within a collection.
> The method, CQL3Type.Raw::toString, is referenced at multiple places in the
> source. For example, referenced in CreateTypeStatement.Raw and involved in
> CQLSSTableWriter. Another example is that it is called to produce the
> SchemaChange at several AlterSchemaStatement implementations.
> A test can prove that missing the frozen wrapper causes exception when
> building CQLSSTableWriter for user types defined like below. Note that the
> inner tuple is wrapped with frozen in the initial CQL statement.
> {code:java}
> CREATE TYPE ks.fooType ( f list>> )
> {code}
> {code:java}
> org.apache.cassandra.exceptions.InvalidRequestException: Non-frozen tuples
> are not allowed inside collections: list>
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.throwNestedNonFrozenError(CQL3Type.java:710)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepare(CQL3Type.java:669)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepareInternal(CQL3Type.java:661)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.lambda$prepare$1(Types.java:341)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.prepare(Types.java:342)
> at org.apache.cassandra.schema.Types$RawBuilder.build(Types.java:291)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.createTypes(CQLSSTableWriter.java:551)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.build(CQLSSTableWriter.java:527)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155676#comment-17155676
]
Sam Tunnicliffe edited comment on CASSANDRA-15299 at 7/10/20, 7:12 PM:
---
[~omichallat] I absolutely take on board all of your arguments. Now the changes
are getting to a somewhat more decent shape, lets take another look and see if
it a large scale rename is absolutely necessary. Maybe it's just Stockholm
Syndrome, but I'm finding that the clashes between the {{o.a.c.net}} and
{{o.a.c.transport}} classes are less and less of a problem. Perhaps we can live
with things as they are after all.
Unfortunately, I've been a bit short of time to focus on this since the last
update, but I've made some changes to the server configuration to improve
testability and added a first meaningful(ish) test. Fleshing out the test
coverage should now be a bit easier, which I'll try to do next week.
As I mentioned, I've been running end to end tests with the
[java2772|https://github.com/datastax/java-driver/tree/java2772] branch of the
java driver and have bundled a build from there for now.
What would be really useful would be some experimental level of support in the
python driver, for cqlsh and python dtests.
|branch|[15299-trunk|https://github.com/beobal/cassandra/tree/15299-trunk]|
|dtests|[15299|https://github.com/beobal/cassandra-dtest/tree/15299]|
|ci|[circle|https://app.circleci.com/pipelines/github/beobal/cassandra?branch=15299-trunk]|
was (Author: beobal):
[~omichallat] I absolutely take on board all of your arguments. Now the changes
are getting to a somewhat more decent shape, lets take another look and see if
it a large scale rename is absolutely necessary. Maybe it's just Stockholm
Syndrome, but I'm finding that the clashes between the {{o.a.c.net}} and
{{o.a.c.transport}} classes are less and less of a problem. Perhaps we can live
with things as they are after all.
Unfortunately, I've been a bit short of time to focus on this since the last
update, but I've made some changes to the server configuration to improve
testability and added a first meaningful(ish) test. Fleshing out the test
coverage should now be a bit easier, which I'll try to do next week.
As I mentioned, I've been running end to end tests with the
[java2772|https://github.com/datastax/java-driver/tree/java2772] branch of the
java driver and have bundled a build from there for now.
What would be really useful would be some experimental level of support in the
python driver, for cqlsh and python dtests.
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> particular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a l
[jira] [Comment Edited] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[ https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17139920#comment-17139920 ] Sam Tunnicliffe edited comment on CASSANDRA-15299 at 7/10/20, 7:11 PM: --- Pushed an updated branch where the protocol changes are pretty stable. I've been testing this with the [java2772|https://github.com/datastax/java-driver/tree/java2772] branch of the driver and with debug-cql and everything is working pretty much as expected. Due to current lack of support in the python driver, I've had to use a modified [dtest branch|https://github.com/beobal/cassandra-dtest/tree/15299] and make a temporary hack to cqlsh, but the tests that are running are pretty much green. Obviously, those are not covering anything from v5 now, but my primary concern was to make sure there's no regressions for v4 clients. Although I think this is ready for some more eyes on it, there's still a non-trivial amount of work to be done. Items outstanding include: * Comprehensive unit and in-jvm tests - in progress * Metrics * Python driver support (doesn't have to be fully implemented, but a basic level is needed for pytests and cqlsh) * Documentation * Renaming existing classes. There are a number of slightly confusing conflicts in naming now. These should be simple to resolve, just automated renaming mostly, but I've held off doing them for now because they'll make the patch much bigger and probably harder to read. The patch is also not quite a massive at it might appear at first. A large proportion is the revert of CASSANDRA-13304, and the rest is largely additive. I've tried hard not touch code on the v4 path, even where it could clearly be refactored, to minimize the delta & the risk there. So the patch largely consists of some v5 specific additions, moving a few existing classes/methods around, and changing modifiers on previously private/package-private things. I'm still actively working on the tests, but I don't think that (nor the renaming) need hold up review any more. |branch|[15299-trunk|https://github.com/beobal/cassandra/tree/15299-trunk]| |dtests|[15299|https://github.com/beobal/cassandra-dtest/tree/15299]| |ci|[circle|https://app.circleci.com/pipelines/github/beobal/cassandra?branch=15299-trunk]| was (Author: beobal): Pushed an updated branch where the protocol changes are pretty stable. I've been testing this with the [java2772|https://github.com/datastax/java-driver/tree/java2772] branch of the driver and with debug-cql and everything is working pretty much as expected. Due to current lack of support in the python driver, I've had to use a modified [dtest branch|https://github.com/beobal/cassandra-dtest/tree/15299] and make a temporary hack to cqlsh, but the tests that are running are pretty much green. Obviously, those are not covering anything from v5 now, but my primary concern was to make sure there's no regressions for v4 clients. Although I think this is ready for some more eyes on it, there's still a non-trivial amount of work to be done. Items outstanding include: * Comprehensive unit and in-jvm tests - in progress * Metrics * Python driver support (doesn't have to be fully implemented, but a basic level is needed for pytests and cqlsh) * Documentation * Renaming existing classes. There are a number of slightly confusing conflicts in naming now. These should be simple to resolve, just automated renaming mostly, but I've held off doing them for now because they'll make the patch much bigger and probably harder to read. The patch is also not quite a massive at it might appear at first. A large proportion is the revert of CASSANDRA-13304, and the rest is largely additive. I've tried hard not touch code on the v4 path, even where it could clearly be refactored, to minimize the delta & the risk there. So the patch largely consists of some v5 specific additions, moving a few existing classes/methods around, and changing modifiers on previously private/package-private things. I'm still actively working on the tests, but I don't think that (nor the renaming) need hold up review any more. |branch|[15299-trunk|https://github.com/beobal/cassandra/tree/15299-trunk]| |dtests|[15299|https://github.com/beobal/cassandra-dtest/tree/15299]| |ci|[circle|https://app.circleci.com/pipelines/github/beobal/cassandra?branch=15299-trunk]| > CASSANDRA-13304 follow-up: improve checksumming and compression in protocol > v5-beta > --- > > Key: CASSANDRA-15299 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15299 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client >Reporter: Aleksey Yeschenko >Assignee: Sam Tunnicliffe >Priority: N
[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155676#comment-17155676
]
Sam Tunnicliffe commented on CASSANDRA-15299:
-
[~omichallat] I absolutely take on board all of your arguments. Now the changes
are getting to a somewhat more decent shape, lets take another look and see if
it a large scale rename is absolutely necessary. Maybe it's just Stockholm
Syndrome, but I'm finding that the clashes between the {{o.a.c.net}} and
{{o.a.c.transport}} classes are less and less of a problem. Perhaps we can live
with things as they are after all.
Unfortunately, I've been a bit short of time to focus on this since the last
update, but I've made some changes to the server configuration to improve
testability and added a first meaningful(ish) test. Fleshing out the test
coverage should now be a bit easier, which I'll try to do next week.
As I mentioned, I've been running end to end tests with the
[java2772|https://github.com/datastax/java-driver/tree/java2772] branch of the
java driver and have bundled a build from there for now.
What would be really useful would be some experimental level of support in the
python driver, for cqlsh and python dtests.
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> particular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed,
> and that we do it before native protocol v5 graduates from beta. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
> and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15936) Extend IInternodeAuthenticator to inspect existing connections
Yifan Cai created CASSANDRA-15936:
-
Summary: Extend IInternodeAuthenticator to inspect existing
connections
Key: CASSANDRA-15936
URL: https://issues.apache.org/jira/browse/CASSANDRA-15936
Project: Cassandra
Issue Type: Improvement
Components: Messaging/Internode
Reporter: Yifan Cai
Assignee: Yifan Cai
The internode authenticator ({{IInternodeAuthenticator}}) should support
inspecting the existing connections and close the connections if no longer
permitted.
It would be useful and necessary for the dynamic internode authenticator
implementations, which may stop authenticating connections between some nodes
at a later time point.
The current interface only authenticates on a connection during tcp/nodes
handshake. It leaves the existing connection alive and the nodes are still able
to communicate, after the authenticator has been updated to not permit
connections between the nodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15809) ASF CI builds for JDK11
[ https://issues.apache.org/jira/browse/CASSANDRA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shylaja kokoori updated CASSANDRA-15809: Test and Documentation Plan: With this patch, Jenkins will display 2 JDK configurations jdk=JDK 1.8 (latest) & jdk=JDK 11 (latest) for cassandra-trunk-artifacts & cassandra-devbranch-artifacts. A build issued will (depending on the availability of executors) start both the builds parallelly. I have tested the build artifacts on trunk. [^build_with_multiple_jdk.patch] Status: Patch Available (was: In Progress) Currently only JDK8 & 11 have been enabled based on Mick's suggestion. I can add others as needed. Like David mentioned JDK14 will need GC change. Removal of Nashorn Javascript Engine might cause a build problem JDK15 onwards > ASF CI builds for JDK11 > --- > > Key: CASSANDRA-15809 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15809 > Project: Cassandra > Issue Type: Task > Components: Build, CI >Reporter: Michael Semb Wever >Assignee: shylaja kokoori >Priority: Normal > Fix For: 4.0-beta > > Attachments: Screenshot 2020-05-13 at 09.39.56.png, > build_with_multiple_jdk.patch > > > ASF CI builds today only run on JDK1.8 > On the Jenkins cluster JDKs from 1.4 through to 15 are available. See > attached screenshot for naming specifics. > This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for > parity to CircleCI's workflows. (There is also the question about > testing/running on Cassandra-2.2 which needs to support JDK1.7, though > Cassandra-2.2 is nearing EOL.) > > The JDK is specified in the groovy DSL: > [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11] > > > Some examples: > * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with > both JDK1.8 and JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8] > * CircleCI JDK11 workflow example. This builds and tests only with JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306] > * Jenkins Cqlshlib tests showing matrix builds: > ** > [https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/] > > Background > thread:[https://lists.apache.org/thread.html/rbaeb960901fa53b50227b37d64e5a456b68f749f4229c6e3e086ff85%40%3Cdev.cassandra.apache.org%3E] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-10968) When taking snapshot, manifest.json contains incorrect or no files when column family has secondary indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-10968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-10968: -- Fix Version/s: (was: 3.11.x) (was: 3.0.x) (was: 2.2.x) 4.0-alpha5 2.1.x 3.11.7 3.0.21 2.2.17 Since Version: 0.7 beta 1 Source Control Link: https://github.com/apache/cassandra/commit/976096abd2ba786f747774ee5160c4cba6fefce2 Resolution: Fixed Status: Resolved (was: Ready to Commit) > When taking snapshot, manifest.json contains incorrect or no files when > column family has secondary indexes > --- > > Key: CASSANDRA-10968 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10968 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Fred A >Assignee: Aleksandr Sorokoumov >Priority: Normal > Labels: lhf > Fix For: 2.2.17, 3.0.21, 3.11.7, 2.1.x, 4.0-alpha5 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > xNoticed indeterminate behaviour when taking snapshot on column families that > has secondary indexes setup. The created manifest.json created when doing > snapshot, sometimes contains no file names at all and sometimes some file > names. > I don't know if this post is related but that was the only thing I could find: > http://www.mail-archive.com/user%40cassandra.apache.org/msg42019.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10968) When taking snapshot, manifest.json contains incorrect or no files when column family has secondary indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-10968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155634#comment-17155634 ] Andres de la Peña commented on CASSANDRA-10968: --- Committed to cassandra-2.1 branch as [976096abd2ba786f747774ee5160c4cba6fefce2|https://github.com/apache/cassandra/commit/976096abd2ba786f747774ee5160c4cba6fefce2] and merged up to trunk. > When taking snapshot, manifest.json contains incorrect or no files when > column family has secondary indexes > --- > > Key: CASSANDRA-10968 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10968 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Fred A >Assignee: Aleksandr Sorokoumov >Priority: Normal > Labels: lhf > Fix For: 2.2.x, 3.0.x, 3.11.x > > Time Spent: 1h 40m > Remaining Estimate: 0h > > xNoticed indeterminate behaviour when taking snapshot on column families that > has secondary indexes setup. The created manifest.json created when doing > snapshot, sometimes contains no file names at all and sometimes some file > names. > I don't know if this post is related but that was the only thing I could find: > http://www.mail-archive.com/user%40cassandra.apache.org/msg42019.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 4f11cb9feebdb0017de1e1063cbf72b840567518 Merge: 979a71b 257fb03 Author: adelapena AuthorDate: Fri Jul 10 18:04:18 2020 +0100 Merge branch 'cassandra-2.2' into cassandra-3.0 # Conflicts: # CHANGES.txt # src/java/org/apache/cassandra/db/ColumnFamilyStore.java # test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java CHANGES.txt| 3 ++ .../org/apache/cassandra/db/ColumnFamilyStore.java | 12 +++ src/java/org/apache/cassandra/db/Directories.java | 23 - .../apache/cassandra/io/sstable/Descriptor.java| 7 +++- .../apache/cassandra/db/ColumnFamilyStoreTest.java | 40 ++ 5 files changed, 69 insertions(+), 16 deletions(-) diff --cc CHANGES.txt index 16974c4,02de7c1..5681007 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -26,71 -4,24 +26,74 @@@ Merged from 2.2 * Remove duplicated tools binaries from tarballs (CASSANDRA-15768) * Duplicate results with DISTINCT queries in mixed mode (CASSANDRA-15501) * Disable JMX rebinding (CASSANDRA-15653) - * Fix Commit log replays when static column clustering keys are collections (CASSANDRA-14365) - * Fix Red Hat init script on newer systemd versions (CASSANDRA-15273) - * Allow EXTRA_CLASSPATH to work on tar/source installations (CASSANDRA-15567) Merged from 2.1: + * Fix writing of snapshot manifest when the table has table-backed secondary indexes (CASSANDRA-10968) * Fix parse error in cqlsh COPY FROM and formatting for map of blobs (CASSANDRA-15679) + * Fix Commit log replays when static column clustering keys are collections (CASSANDRA-14365) + * Fix Red Hat init script on newer systemd versions (CASSANDRA-15273) + * Allow EXTRA_CLASSPATH to work on tar/source installations (CASSANDRA-15567) + -2.2.16 +3.0.20 + * Run in-jvm upgrade dtests in circleci (CASSANDRA-15506) + * Include updates to static column in mutation size calculations (CASSANDRA-15293) + * Fix point-in-time recoevery ignoring timestamp of updates to static columns (CASSANDRA-15292) + * GC logs are also put under $CASSANDRA_LOG_DIR (CASSANDRA-14306) + * Fix sstabledump's position key value when partitions have multiple rows (CASSANDRA-14721) + * Avoid over-scanning data directories in LogFile.verify() (CASSANDRA-15364) + * Bump generations and document changes to system_distributed and system_traces in 3.0, 3.11 + (CASSANDRA-15441) + * Fix system_traces creation timestamp; optimise system keyspace upgrades (CASSANDRA-15398) + * Fix various data directory prefix matching issues (CASSANDRA-13974) + * Minimize clustering values in metadata collector (CASSANDRA-15400) + * Avoid over-trimming of results in mixed mode clusters (CASSANDRA-15405) + * validate value sizes in LegacyLayout (CASSANDRA-15373) + * Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default (CASSANDRA-15385) + * Make sure index summary redistribution does not start when compactions are paused (CASSANDRA-15265) + * Ensure legacy rows have primary key livenessinfo when they contain illegal cells (CASSANDRA-15365) + * Fix race condition when setting bootstrap flags (CASSANDRA-14878) + * Fix NativeLibrary.tryOpenDirectory callers for Windows (CASSANDRA-15426) +Merged from 2.2: * Fix SELECT JSON output for empty blobs (CASSANDRA-15435) * In-JVM DTest: Set correct internode message version for upgrade test (CASSANDRA-15371) - * In-JVM DTest: Support NodeTool in dtest + * In-JVM DTest: Support NodeTool in dtest (CASSANDRA-15429) + -2.2.15 +3.0.19 + * Add ability to cap max negotiable protocol version (CASSANDRA-15193) + * Gossip tokens on startup if available (CASSANDRA-15335) + * Fix resource leak in CompressedSequentialWriter (CASSANDRA-15340) + * Fix merge which reverted CASSANDRA-14993 (CASSANDRA-15289) + * Fix LegacyLayout RangeTombstoneList IndexOutOfBoundsException when upgrading and RangeTombstone bounds are asymmetric (CASSANDRA-15172) + * Fix NPE when using allocate_tokens_for_keyspace on new DC/rack (CASSANDRA-14952) + * Filter sstables earlier when running cleanup (CASSANDRA-15100) + * Use mean row count instead of mean column count for index selectivity calculation (CASSANDRA-15259) + * Avoid updating unchanged gossip states (CASSANDRA-15097) + * Prevent recreation of previously dropped columns with a different kind (CASSANDRA-14948) + * Prevent client requests from blocking on executor task queue (CASSANDRA-15013) + * Toughen up column drop/recreate type validations (CASSANDRA-15204) + * LegacyLayout should handle paging states that cross a collection column (CASSANDRA-15201) + * Prevent RuntimeException when username or password is empty/null (CASSANDRA-15198) + * Multiget thrift query returns null records after digest
[cassandra] branch cassandra-3.11 updated (e819f9b -> f3bee1a)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from e819f9b Merge branch 'cassandra-3.0' into cassandra-3.11 new 976096a Fix writing of snapshot manifest when the table has table-backed secondary indexes patch by Aleksandr Sorokoumov; reviewed by Andrés de la Peña for CASSANDRA-10968 new 257fb03 Merge branch 'cassandra-2.1' into cassandra-2.2 new 4f11cb9 Merge branch 'cassandra-2.2' into cassandra-3.0 new f3bee1a Merge branch 'cassandra-3.0' into cassandra-3.11 The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../org/apache/cassandra/db/ColumnFamilyStore.java | 11 +++--- src/java/org/apache/cassandra/db/Directories.java | 23 - .../apache/cassandra/io/sstable/Descriptor.java| 7 +++- .../apache/cassandra/db/ColumnFamilyStoreTest.java | 39 ++ .../cassandra/io/sstable/DescriptorTest.java | 8 + 6 files changed, 74 insertions(+), 15 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (979a71b -> 4f11cb9)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 979a71b Avoid hinted handoff per-host throttle being arounded to 0 in large cluster new 976096a Fix writing of snapshot manifest when the table has table-backed secondary indexes patch by Aleksandr Sorokoumov; reviewed by Andrés de la Peña for CASSANDRA-10968 new 257fb03 Merge branch 'cassandra-2.1' into cassandra-2.2 new 4f11cb9 Merge branch 'cassandra-2.2' into cassandra-3.0 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 3 ++ .../org/apache/cassandra/db/ColumnFamilyStore.java | 12 +++ src/java/org/apache/cassandra/db/Directories.java | 23 - .../apache/cassandra/io/sstable/Descriptor.java| 7 +++- .../apache/cassandra/db/ColumnFamilyStoreTest.java | 40 ++ 5 files changed, 69 insertions(+), 16 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.1' into cassandra-2.2
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch cassandra-2.2 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 257fb0377343cbfdb58327da17f31d4eaed940f5 Merge: f79d105 976096a Author: adelapena AuthorDate: Fri Jul 10 17:52:23 2020 +0100 Merge branch 'cassandra-2.1' into cassandra-2.2 # Conflicts: # CHANGES.txt # src/java/org/apache/cassandra/db/ColumnFamilyStore.java # test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java CHANGES.txt| 11 +++--- src/java/org/apache/cassandra/config/Config.java | 2 +- .../org/apache/cassandra/db/ColumnFamilyStore.java | 5 ++- src/java/org/apache/cassandra/db/Directories.java | 23 - .../apache/cassandra/io/sstable/Descriptor.java| 7 +++- .../apache/cassandra/db/ColumnFamilyStoreTest.java | 39 ++ 6 files changed, 68 insertions(+), 19 deletions(-) diff --cc CHANGES.txt index b10a057,0a6268c..02de7c1 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,61 -1,8 +1,62 @@@ - 2.2.18 -2.1.21 ++2.2.17 + * Fix nomenclature of allow and deny lists (CASSANDRA-15862) + * Remove generated files from source artifact (CASSANDRA-15849) + * Remove duplicated tools binaries from tarballs (CASSANDRA-15768) + * Duplicate results with DISTINCT queries in mixed mode (CASSANDRA-15501) + * Disable JMX rebinding (CASSANDRA-15653) - Merged from 2.1: - * Fix parse error in cqlsh COPY FROM and formatting for map of blobs (CASSANDRA-15679) - - 2.2.17 + * Fix Commit log replays when static column clustering keys are collections (CASSANDRA-14365) + * Fix Red Hat init script on newer systemd versions (CASSANDRA-15273) + * Allow EXTRA_CLASSPATH to work on tar/source installations (CASSANDRA-15567) ++Merged from 2.1: + * Fix writing of snapshot manifest when the table has table-backed secondary indexes (CASSANDRA-10968) + * Fix parse error in cqlsh COPY FROM and formatting for map of blobs (CASSANDRA-15679) ++ + +2.2.16 + * Fix SELECT JSON output for empty blobs (CASSANDRA-15435) + * In-JVM DTest: Set correct internode message version for upgrade test (CASSANDRA-15371) + * In-JVM DTest: Support NodeTool in dtest + ++ +2.2.15 + * Catch non-IOException in FileUtils.close to make sure that all resources are closed (CASSANDRA-15225) + * Handle exceptions during authentication/authorization (CASSANDRA-15041) + * Fix JDK7 compatibility broken in cassandra-2.2 (CASSANDRA-15050) + * Support cross version messaging in in-jvm upgrade dtests (CASSANDRA-15078) + * Fix index summary redistribution cancellation (CASSANDRA-15045) + * Refactor Circle CI configuration (CASSANDRA-14806) + * Fixing invalid CQL in security documentation (CASSANDRA-15020) + * Make tools/bin/token-generator py2/3 compatible (CASSANDRA-15012) + * Multi-version in-JVM dtests (CASSANDRA-14937) + * Allow instance class loaders to be garbage collected for inJVM dtest (CASSANDRA-15170) + * Add support for network topology and query tracing for inJVM dtest (CASSANDRA-15319) + + +2.2.14 + * CircleCI docker image should bake in more dependencies (CASSANDRA-14985) + * Don't enable client transports when bootstrap is pending (CASSANDRA-14525) + * MigrationManager attempts to pull schema from different major version nodes (CASSANDRA-14928) + * Don't skip entire sstables when reading backwards with mixed clustering column order + (CASSANDRA-14910) + * Cannot perform slice reads in reverse direction against tables with clustering columns + in mixed order (CASSANDRA-14899) + * Fix incorrect cqlsh results when selecting same columns multiple times (CASSANDRA-13262) + * Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377) +Merged from 2.1: * Paged Range Slice queries with DISTINCT can drop rows from results (CASSANDRA-14956) * Update release checksum algorithms to SHA-256, SHA-512 (CASSANDRA-14970) + + +2.2.13 + * Fix bug that prevented compaction of SSTables after full repairs (CASSANDRA-14423) + * Incorrect counting of pending messages in OutboundTcpConnection (CASSANDRA-11551) + * Fix compaction failure caused by reading un-flushed data (CASSANDRA-12743) + * Use Bounds instead of Range for sstables in anticompaction (CASSANDRA-14411) + * Fix JSON queries with IN restrictions and ORDER BY clause (CASSANDRA-14286) + * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891) + * Fix query pager DEBUG log leak causing hit in paged reads throughput (CASSANDRA-14318) + * Backport circleci yaml (CASSANDRA-14240) +Merged from 2.1: * Check checksum before decompressing data (CASSANDRA-14284) * CVE-2017-5929 Security vulnerability in Logback warning in NEWS.txt (CASSANDRA-14183) diff --cc src/java/org/apache/cassandra/config/Config.java index 7836396a,2c5e36a..ab79a08 --- a/src/java/org/apache/cassandra/config/Config.java +++ b/src/java/org/apache/cassand
[cassandra] branch cassandra-2.2 updated (f79d105 -> 257fb03)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch cassandra-2.2 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from f79d105 Add bytebuddy support for in-jvm dtest new 976096a Fix writing of snapshot manifest when the table has table-backed secondary indexes patch by Aleksandr Sorokoumov; reviewed by Andrés de la Peña for CASSANDRA-10968 new 257fb03 Merge branch 'cassandra-2.1' into cassandra-2.2 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 11 +++--- src/java/org/apache/cassandra/config/Config.java | 2 +- .../org/apache/cassandra/db/ColumnFamilyStore.java | 5 ++- src/java/org/apache/cassandra/db/Directories.java | 23 - .../apache/cassandra/io/sstable/Descriptor.java| 7 +++- .../apache/cassandra/db/ColumnFamilyStoreTest.java | 39 ++ 6 files changed, 68 insertions(+), 19 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit f3bee1aecd3ee6b8a7cded866c0357031c861a05
Merge: e819f9b 4f11cb9
Author: adelapena
AuthorDate: Fri Jul 10 18:14:01 2020 +0100
Merge branch 'cassandra-3.0' into cassandra-3.11
# Conflicts:
# CHANGES.txt
# src/java/org/apache/cassandra/db/ColumnFamilyStore.java
CHANGES.txt| 1 +
.../org/apache/cassandra/db/ColumnFamilyStore.java | 11 +++---
src/java/org/apache/cassandra/db/Directories.java | 23 -
.../apache/cassandra/io/sstable/Descriptor.java| 7 +++-
.../apache/cassandra/db/ColumnFamilyStoreTest.java | 39 ++
.../cassandra/io/sstable/DescriptorTest.java | 8 +
6 files changed, 74 insertions(+), 15 deletions(-)
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index fa00e5b,70c14c0..99cac7c
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@@ -1812,12 -1655,13 +1812,13 @@@ public class ColumnFamilyStore implemen
logger.trace("Snapshot for {} keyspace data file {}
created in {}", keyspace, ssTable.getFilename(), snapshotDirectory);
snapshottedSSTables.add(ssTable);
}
-
- writeSnapshotManifest(filesJSONArr, snapshotName);
- if (!SchemaConstants.isLocalSystemKeyspace(metadata.ksName)
&& !SchemaConstants.isReplicatedSystemKeyspace(metadata.ksName))
- writeSnapshotSchema(snapshotName);
}
}
-writeSnapshotManifest(filesJSONArr, snapshotName);
+
-if (!Schema.isLocalSystemKeyspace(metadata.ksName) &&
!Schema.isReplicatedSystemKeyspace(metadata.ksName))
++writeSnapshotManifest(filesJSONArr, snapshotName);
++if (!SchemaConstants.isLocalSystemKeyspace(metadata.ksName) &&
!SchemaConstants.isReplicatedSystemKeyspace(metadata.ksName))
+ writeSnapshotSchema(snapshotName);
+
if (ephemeral)
createEphemeralSnapshotMarkerFile(snapshotName);
return snapshottedSSTables;
diff --cc test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
index 8463a1f,f7152ff..a3564bb
--- a/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
+++ b/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
@@@ -514,6 -518,40 +520,39 @@@ public class ColumnFamilyStoreTes
}
@Test
+ public void testSnapshotWithoutFlushWithSecondaryIndexes() throws
Exception
+ {
+ Keyspace keyspace = Keyspace.open(KEYSPACE1);
+ ColumnFamilyStore cfs = keyspace.getColumnFamilyStore(CF_INDEX1);
+ cfs.truncateBlocking();
+
+ UpdateBuilder builder = UpdateBuilder.create(cfs.metadata, "key")
+ .newRow()
+ .add("birthdate", 1L)
+ .add("notbirthdate", 2L);
+ new Mutation(builder.build()).applyUnsafe();
+ cfs.forceBlockingFlush();
+
+ String snapshotName = "newSnapshot";
+ cfs.snapshotWithoutFlush(snapshotName);
+
+ File snapshotManifestFile =
cfs.getDirectories().getSnapshotManifestFile(snapshotName);
+ JSONParser parser = new JSONParser();
+ JSONObject manifest = (JSONObject) parser.parse(new
FileReader(snapshotManifestFile));
+ JSONArray files = (JSONArray) manifest.get("files");
+
+ // Keyspace1-Indexed1 and the corresponding index
+ assert files.size() == 2;
+
+ // Snapshot of the secondary index is stored in the subfolder with
the same file name
+ String baseTableFile = (String) files.get(0);
+ String indexTableFile = (String) files.get(1);
+ assert !baseTableFile.equals(indexTableFile);
+ assert Directories.isSecondaryIndexFolder(new
File(indexTableFile).getParentFile());
+ assert indexTableFile.endsWith(baseTableFile);
+ }
+
-
+ @Test
public void testScrubDataDirectories() throws Throwable
{
ColumnFamilyStore cfs =
Keyspace.open(KEYSPACE1).getColumnFamilyStore(CF_STANDARD1);
diff --cc test/unit/org/apache/cassandra/io/sstable/DescriptorTest.java
index 64367dc,184d637..9c1dc84
--- a/test/unit/org/apache/cassandra/io/sstable/DescriptorTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/DescriptorTest.java
@@@ -23,8 -23,8 +23,10 @@@ import java.util.UUID
import org.apache.commons.lang3.StringUtils;
import org.junit.Assert;
++import org.junit.BeforeClass;
import org.junit.Test;
++import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.db.Directories;
import org.apache.cassandra.io.sstable.format.SSTableFormat;
import org.apache.cassa
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 8330aa171bf0178e1373a4e102438ccac01f2bbb Merge: 5d44a50 f3bee1a Author: adelapena AuthorDate: Fri Jul 10 18:24:00 2020 +0100 Merge branch 'cassandra-3.11' into trunk # Conflicts: # src/java/org/apache/cassandra/db/ColumnFamilyStore.java # src/java/org/apache/cassandra/io/sstable/Descriptor.java CHANGES.txt| 2 ++ .../org/apache/cassandra/db/ColumnFamilyStore.java | 11 +++--- src/java/org/apache/cassandra/db/Directories.java | 23 - .../apache/cassandra/io/sstable/Descriptor.java| 7 +++- .../apache/cassandra/db/ColumnFamilyStoreTest.java | 40 +- .../cassandra/io/sstable/DescriptorTest.java | 8 + 6 files changed, 75 insertions(+), 16 deletions(-) diff --cc CHANGES.txt index c2b3d69,9463403..60d1e22 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -82,58 -30,9 +82,60 @@@ Merged from 2.2 * Remove generated files from source artifact (CASSANDRA-15849) * Remove duplicated tools binaries from tarballs (CASSANDRA-15768) * Duplicate results with DISTINCT queries in mixed mode (CASSANDRA-15501) - * Disable JMX rebinding (CASSANDRA-15653) + Merged from 2.1: + * Fix writing of snapshot manifest when the table has table-backed secondary indexes (CASSANDRA-10968) + +4.0-alpha4 + * Add client request size server metrics (CASSANDRA-15704) + * Add additional logging around FileUtils and compaction leftover cleanup (CASSANDRA-15705) + * Mark system_views/system_virtual_schema as non-alterable keyspaces in cqlsh (CASSANDRA-15711) + * Fail incremental repair if an old version sstable is involved (CASSANDRA-15612) + * Fix overflows on StreamingTombstoneHistogramBuilder produced by large deletion times (CASSANDRA-14773) + * Mark system_views/system_virtual_schema as system keyspaces in cqlsh (CASSANDRA-15706) + * Avoid unnecessary collection/iterator allocations during btree construction (CASSANDRA-15390) + * Repair history tables should have TTL and TWCS (CASSANDRA-12701) + * Fix cqlsh erroring out on Python 3.7 due to webbrowser module being absent (CASSANDRA-15572) + * Fix IMH#acquireCapacity() to return correct Outcome when endpoint reserve runs out (CASSANDRA-15607) + * Fix nodetool describering output (CASSANDRA-15682) + * Only track ideal CL failure when request CL met (CASSANDRA-15696) + * Fix flaky CoordinatorMessagingTest and docstring in OutboundSink and ConsistentSession (CASSANDRA-15672) + * Fix force compaction of wrapping ranges (CASSANDRA-15664) + * Expose repair streaming metrics (CASSANDRA-15656) + * Set now in seconds in the future for validation repairs (CASSANDRA-15655) + * Emit metric on preview repair failure (CASSANDRA-15654) + * Use more appropriate logging levels (CASSANDRA-15661) + * Fixed empty check in TrieMemIndex due to potential state inconsistency in ConcurrentSkipListMap (CASSANDRA-15526) + * Added UnleveledSSTables global and table level metric (CASSANDRA-15620) + * Added Virtual Table exposing Cassandra relevant system properties (CASSANDRA-15616, CASSANDRA-15643) + * Improve the algorithmic token allocation in case racks = RF (CASSANDRA-15600) + * Fix ConnectionTest.testAcquireReleaseOutbound (CASSANDRA-15308) + * Include finalized pending sstables in preview repair (CASSANDRA-15553) + * Reverted to the original behavior of CLUSTERING ORDER on CREATE TABLE (CASSANDRA-15271) + * Correct inaccurate logging message (CASSANDRA-15549) + * Unset GREP_OPTIONS (CASSANDRA-14487) + * Update to Python driver 3.21 for cqlsh (CASSANDRA-14872) + * Fix missing Keyspaces in cqlsh describe output (CASSANDRA-15576) + * Fix multi DC nodetool status output (CASSANDRA-15305) + * updateCoordinatorWriteLatencyTableMetric can produce misleading metrics (CASSANDRA-15569) + * Make cqlsh and cqlshlib Python 2 & 3 compatible (CASSANDRA-10190) + * Improve the description of nodetool listsnapshots command (CASSANDRA-14587) + * allow embedded cassandra launched from a one-jar or uno-jar (CASSANDRA-15494) + * Update hppc library to version 0.8.1 (CASSANDRA-12995) + * Limit the dependencies used by UDFs/UDAs (CASSANDRA-14737) + * Make native_transport_max_concurrent_requests_in_bytes updatable (CASSANDRA-15519) + * Cleanup and improvements to IndexInfo/ColumnIndex (CASSANDRA-15469) + * Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB & Bytes (CASSANDRA-15470) +Merged from 3.11: + * Allow sstableloader to use SSL on the native port (CASSANDRA-14904) +Merged from 3.0: + * cqlsh return non-zero status when STDIN CQL fails (CASSANDRA-15623) + * Don't skip sstables in slice queries based only on local min/max/deletion timestamp (CASSANDRA-15690) + * Memtable memory allocations may deadlock (CASSANDRA-15367) + * Run evictFromMembers
[cassandra] branch cassandra-2.1 updated: Fix writing of snapshot manifest when the table has table-backed secondary indexes patch by Aleksandr Sorokoumov; reviewed by Andrés de la Peña for CASSANDRA-
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-2.1
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-2.1 by this push:
new 976096a Fix writing of snapshot manifest when the table has
table-backed secondary indexes patch by Aleksandr Sorokoumov; reviewed by
Andrés de la Peña for CASSANDRA-10968
976096a is described below
commit 976096abd2ba786f747774ee5160c4cba6fefce2
Author: Aleksandr Sorokoumov
AuthorDate: Fri Jul 10 16:57:40 2020 +0100
Fix writing of snapshot manifest when the table has table-backed secondary
indexes
patch by Aleksandr Sorokoumov; reviewed by Andrés de la Peña for
CASSANDRA-10968
---
CHANGES.txt| 1 +
.../org/apache/cassandra/db/ColumnFamilyStore.java | 5 ++--
.../apache/cassandra/db/ColumnFamilyStoreTest.java | 34 ++
3 files changed, 37 insertions(+), 3 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index d0d0993..0a6268c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.21
+ * Fix writing of snapshot manifest when the table has table-backed secondary
indexes (CASSANDRA-10968)
* Fix parse error in cqlsh COPY FROM and formatting for map of blobs
(CASSANDRA-15679)
* Paged Range Slice queries with DISTINCT can drop rows from results
(CASSANDRA-14956)
* Update release checksum algorithms to SHA-256, SHA-512 (CASSANDRA-14970)
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 2989b9d..df9b046 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -2335,9 +2335,9 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
*/
public void snapshotWithoutFlush(String snapshotName,
Predicate predicate, boolean ephemeral)
{
+final JSONArray filesJSONArr = new JSONArray();
for (ColumnFamilyStore cfs : concatWithIndexes())
{
-final JSONArray filesJSONArr = new JSONArray();
try (RefViewFragment currentView =
cfs.selectAndReference(CANONICAL_SSTABLES))
{
for (SSTableReader ssTable : currentView.sstables)
@@ -2352,10 +2352,9 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
if (logger.isDebugEnabled())
logger.debug("Snapshot for {} keyspace data file {}
created in {}", keyspace, ssTable.getFilename(), snapshotDirectory);
}
-
-writeSnapshotManifest(filesJSONArr, snapshotName);
}
}
+writeSnapshotManifest(filesJSONArr, snapshotName);
if (ephemeral)
createEphemeralSnapshotMarkerFile(snapshotName);
}
diff --git a/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
b/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
index 35814f0..1fc2029 100644
--- a/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
+++ b/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
@@ -19,6 +19,7 @@
package org.apache.cassandra.db;
import java.io.File;
+import java.io.FileReader;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.charset.CharacterCodingException;
@@ -57,6 +58,9 @@ import org.apache.cassandra.thrift.SlicePredicate;
import org.apache.cassandra.thrift.SliceRange;
import org.apache.cassandra.thrift.ThriftValidation;
import org.apache.cassandra.utils.*;
+import org.json.simple.JSONArray;
+import org.json.simple.JSONObject;
+import org.json.simple.parser.JSONParser;
import static org.apache.cassandra.Util.cellname;
import static org.apache.cassandra.Util.column;
@@ -2233,4 +2237,34 @@ public class ColumnFamilyStoreTest extends SchemaLoader
PerRowSecondaryIndexTest.TestIndex.reset();
}
+
+@Test
+public void testSnapshotWithoutFlushWithSecondaryIndexes() throws Exception
+{
+String keyspaceName = "Keyspace1";
+String cfName = "Indexed1";
+Keyspace keyspace = Keyspace.open(keyspaceName);
+ColumnFamilyStore cfs = keyspace.getColumnFamilyStore(cfName);
+cfs.truncateBlocking();
+
+List rms = new LinkedList<>();
+Mutation rm;
+
+rm = new Mutation(keyspaceName, ByteBufferUtil.bytes("k1"));
+rm.add(cfName, cellname("birthdate"), ByteBufferUtil.bytes(1L), 0);
+rm.add(cfName, cellname("nobirthdate"), ByteBufferUtil.bytes(1L), 0);
+rms.add(rm);
+Util.writeColumnFamily(rms);
+
+String snapshotName = "newSnapshot";
+cfs.snapshotWithoutFlush(snapshotName);
+
+File snapshotManifestFile =
cfs.directories.getSnapshotManifestFile(snapshotName);
+JSONParser parser = new JSONParser();
+JSONObject manifest = (JSONObject) parser.
[cassandra] branch trunk updated (5d44a50 -> 8330aa1)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 5d44a50 Merge branch 'cassandra-3.11' into trunk new 976096a Fix writing of snapshot manifest when the table has table-backed secondary indexes patch by Aleksandr Sorokoumov; reviewed by Andrés de la Peña for CASSANDRA-10968 new 257fb03 Merge branch 'cassandra-2.1' into cassandra-2.2 new 4f11cb9 Merge branch 'cassandra-2.2' into cassandra-3.0 new f3bee1a Merge branch 'cassandra-3.0' into cassandra-3.11 new 8330aa1 Merge branch 'cassandra-3.11' into trunk The 5 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 2 ++ .../org/apache/cassandra/db/ColumnFamilyStore.java | 11 +++--- src/java/org/apache/cassandra/db/Directories.java | 23 - .../apache/cassandra/io/sstable/Descriptor.java| 7 +++- .../apache/cassandra/db/ColumnFamilyStoreTest.java | 40 +- .../cassandra/io/sstable/DescriptorTest.java | 8 + 6 files changed, 75 insertions(+), 16 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15809) ASF CI builds for JDK11
[ https://issues.apache.org/jira/browse/CASSANDRA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shylaja kokoori updated CASSANDRA-15809: Attachment: build_with_multiple_jdk.patch > ASF CI builds for JDK11 > --- > > Key: CASSANDRA-15809 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15809 > Project: Cassandra > Issue Type: Task > Components: Build, CI >Reporter: Michael Semb Wever >Assignee: shylaja kokoori >Priority: Normal > Fix For: 4.0-beta > > Attachments: Screenshot 2020-05-13 at 09.39.56.png, > build_with_multiple_jdk.patch > > > ASF CI builds today only run on JDK1.8 > On the Jenkins cluster JDKs from 1.4 through to 15 are available. See > attached screenshot for naming specifics. > This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for > parity to CircleCI's workflows. (There is also the question about > testing/running on Cassandra-2.2 which needs to support JDK1.7, though > Cassandra-2.2 is nearing EOL.) > > The JDK is specified in the groovy DSL: > [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11] > > > Some examples: > * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with > both JDK1.8 and JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8] > * CircleCI JDK11 workflow example. This builds and tests only with JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306] > * Jenkins Cqlshlib tests showing matrix builds: > ** > [https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/] > > Background > thread:[https://lists.apache.org/thread.html/rbaeb960901fa53b50227b37d64e5a456b68f749f4229c6e3e086ff85%40%3Cdev.cassandra.apache.org%3E] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15933) Forbid adding new fields to UDTs used in partition key columns
[
https://issues.apache.org/jira/browse/CASSANDRA-15933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155598#comment-17155598
]
Aleksey Yeschenko edited comment on CASSANDRA-15933 at 7/10/20, 4:59 PM:
-
Closing, as this is only an issue in 3.0, and fixed in 3.11 and 4.0 since
CASSANDRA-7423. The issue in 3.0: altering a type used in a PK of a table
causes lookups for previously written data to fail. This will be addressed by
backporting {{TupleType}} changes from 3.11 to 3.0.
EDIT: in all versions this is a problem if a UDT used frozen in a partition key
is altered, since we primarily treat partition keys as opaque byte buffers
internally, and change in serialization makes previously written data
inaccessible.
was (Author: iamaleksey):
Closing, as this is only an issue in 3.0, and fixed in 3.11 and 4.0 since
CASSANDRA-7423. The issue in 3.0: altering a type used in a PK of a table
causes lookups for previously written data to fail. This will be addressed by
backporting {{TupleType}} changes from 3.11 to 3.0.
> Forbid adding new fields to UDTs used in partition key columns
> --
>
> Key: CASSANDRA-15933
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15933
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 4.0, 3.0.x, 3.11.x
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15933) Forbid adding new fields to UDTs used in partition key columns
[ https://issues.apache.org/jira/browse/CASSANDRA-15933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15933: -- Summary: Forbid adding new fields to UDTs used in partition key columns (was: Forbid adding new fields to UDTs used in primary key columns) > Forbid adding new fields to UDTs used in partition key columns > -- > > Key: CASSANDRA-15933 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15933 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.x > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15933) Forbid adding new fields to UDTs used in partition key columns
[ https://issues.apache.org/jira/browse/CASSANDRA-15933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15933: -- Fix Version/s: 3.11.x 4.0 > Forbid adding new fields to UDTs used in partition key columns > -- > > Key: CASSANDRA-15933 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15933 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 4.0, 3.0.x, 3.11.x > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15933) Forbid adding new fields to UDTs used in primary key columns
[ https://issues.apache.org/jira/browse/CASSANDRA-15933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15933: -- Status: Open (was: Resolved) > Forbid adding new fields to UDTs used in primary key columns > > > Key: CASSANDRA-15933 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15933 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.x > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15933) Forbid adding new fields to UDTs used in primary key columns
[
https://issues.apache.org/jira/browse/CASSANDRA-15933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15933:
--
Resolution: Won't Fix
Status: Resolved (was: Open)
Closing, as this is only an issue in 3.0, and fixed in 3.11 and 4.0 since
CASSANDRA-7423. The issue in 3.0: altering a type used in a PK of a table
causes lookups for previously written data to fail. This will be addressed by
backporting {{TupleType}} changes from 3.11 to 3.0.
> Forbid adding new fields to UDTs used in primary key columns
>
>
> Key: CASSANDRA-15933
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15933
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15933) Forbid adding new fields to UDTs used in primary key columns
[ https://issues.apache.org/jira/browse/CASSANDRA-15933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15933: -- Fix Version/s: (was: 4.0-beta) (was: 3.11.x) > Forbid adding new fields to UDTs used in primary key columns > > > Key: CASSANDRA-15933 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15933 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.x > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15873) Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues)
[ https://issues.apache.org/jira/browse/CASSANDRA-15873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1714#comment-1714 ] Matt Davis commented on CASSANDRA-15873: Thanks [~brandon.williams] for running this. I don't have bandwidth right now to investigate the failures, so I'll leave this open for others to look into. > Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues) > --- > > Key: CASSANDRA-15873 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15873 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Matt Davis >Assignee: Matt Davis >Priority: Normal > Fix For: 3.11.x > > Attachments: dependency-check-report.html, unittest_netty.log > > > See https://issues.apache.org/jira/browse/CASSANDRA-15868 for the same issue > on 4.0 / trunk. Attached is an OWASP dependency report for Netty 4.0.44, > which identifies 3 of the same vulnerabilities as above. > > Additionally, 4.1.50 contains aarch64 native libraries which can improve > performance on ARM processors. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15873) Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues)
[ https://issues.apache.org/jira/browse/CASSANDRA-15873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Davis reassigned CASSANDRA-15873: -- Assignee: (was: Matt Davis) > Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues) > --- > > Key: CASSANDRA-15873 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15873 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Matt Davis >Priority: Normal > Fix For: 3.11.x > > Attachments: dependency-check-report.html, unittest_netty.log > > > See https://issues.apache.org/jira/browse/CASSANDRA-15868 for the same issue > on 4.0 / trunk. Attached is an OWASP dependency report for Netty 4.0.44, > which identifies 3 of the same vulnerabilities as above. > > Additionally, 4.1.50 contains aarch64 native libraries which can improve > performance on ARM processors. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15933) Forbid adding new fields to UDTs used in primary key columns
[ https://issues.apache.org/jira/browse/CASSANDRA-15933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15933: -- Bug Category: Parent values: Correctness(12982)Level 1 values: Recoverable Corruption / Loss(12986) Complexity: Normal Discovered By: Code Inspection Fix Version/s: 4.0-beta 3.11.x 3.0.x Severity: Normal Status: Open (was: Triage Needed) > Forbid adding new fields to UDTs used in primary key columns > > > Key: CASSANDRA-15933 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15933 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15933) Forbid adding new fields to UDTs used in primary key columns
[ https://issues.apache.org/jira/browse/CASSANDRA-15933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15933: -- Summary: Forbid adding new fields to UDTs used in primary key columns (was: TBD (unexciting)) > Forbid adding new fields to UDTs used in primary key columns > > > Key: CASSANDRA-15933 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15933 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15935) Improve machinery for testing consistency in presence of range movements
Alex Petrov created CASSANDRA-15935: --- Summary: Improve machinery for testing consistency in presence of range movements Key: CASSANDRA-15935 URL: https://issues.apache.org/jira/browse/CASSANDRA-15935 Project: Cassandra Issue Type: Improvement Components: Test/dtest Reporter: Alex Petrov Assignee: Alex Petrov Currently, we can test range movements only by adding and bootstrapping a new node. This is both inefficient and insufficient for large-scale tests. We need a possibility to dynamically change ring ownership over the lifetime of cluster, with a flexibility to changing gossip status of the node from perspective of other participants, adding and removing nodes from other nodes' views on demand. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-in-jvm-dtest-api] branch release updated: Add entries for 0.0.4 to the changelog
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a commit to branch release in repository https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git The following commit(s) were added to refs/heads/release by this push: new 7e1e250 Add entries for 0.0.4 to the changelog 7e1e250 is described below commit 7e1e250185408db1286aceb40e59252f17394fa2 Author: Alex Petrov AuthorDate: Fri Jul 10 16:12:11 2020 +0200 Add entries for 0.0.4 to the changelog --- CHANGES.txt | 4 1 file changed, 4 insertions(+) diff --git a/CHANGES.txt b/CHANGES.txt index a80d4f5..27ce824 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,3 +1,7 @@ +# 0.0.4 + +CASSANDRA-15920: Make SimpleQueryResult a container for client warnings, and expose those warnings via QueryResult + # 0.0.3 CASSANDRA-15851: Add instance initializer - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-in-jvm-dtest-api] branch release updated: [maven-release-plugin] prepare for next development iteration
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a commit to branch release in repository https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git The following commit(s) were added to refs/heads/release by this push: new b481c85 [maven-release-plugin] prepare for next development iteration b481c85 is described below commit b481c85f9ce49e17b47765800ca5c2b396b1fa73 Author: Alex Petrov AuthorDate: Fri Jul 10 16:10:14 2020 +0200 [maven-release-plugin] prepare for next development iteration --- pom.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pom.xml b/pom.xml index ff59ec0..5e36e66 100644 --- a/pom.xml +++ b/pom.xml @@ -27,7 +27,7 @@ 4.0.0 org.apache.cassandra dtest-api -0.0.4 +0.0.5-SNAPSHOT In JVM Test API In JVM Test API @@ -136,7 +136,7 @@ scm:git:https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git scm:git:https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git - 0.0.4 + HEAD - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-in-jvm-dtest-api] 01/01: [maven-release-plugin] prepare release 0.0.4
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a commit to branch release in repository https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git commit a9f30a3d69de8e1949154221de3f255dfa8b2e9a Author: Alex Petrov AuthorDate: Fri Jul 10 16:10:03 2020 +0200 [maven-release-plugin] prepare release 0.0.4 --- pom.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pom.xml b/pom.xml index fe2baff..ff59ec0 100644 --- a/pom.xml +++ b/pom.xml @@ -27,7 +27,7 @@ 4.0.0 org.apache.cassandra dtest-api -0.0.4-SNAPSHOT +0.0.4 In JVM Test API In JVM Test API @@ -136,7 +136,7 @@ scm:git:https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git scm:git:https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git - HEAD + 0.0.4 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-in-jvm-dtest-api] annotated tag 0.0.4 created (now e25852c)
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a change to annotated tag 0.0.4 in repository https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git. at e25852c (tag) tagging a9f30a3d69de8e1949154221de3f255dfa8b2e9a (commit) replaces 0.0.3 by Alex Petrov on Fri Jul 10 16:10:09 2020 +0200 - Log - [maven-release-plugin] copy for tag 0.0.4 --- No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-in-jvm-dtest-api] branch release updated (3f01a17 -> a9f30a3)
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a change to branch release in repository https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git. from 3f01a17 [maven-release-plugin] prepare for next development iteration add fb4cc70 Add entries for 0.0.3 to the changelog add 327c598 make SimpleQueryResult a container for client warnings, and expose those warnings via QueryResult new a9f30a3 [maven-release-plugin] prepare release 0.0.4 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 4 +++ pom.xml| 4 +-- .../cassandra/distributed/api/QueryResult.java | 2 ++ .../cassandra/distributed/api/QueryResults.java| 24 +++-- .../distributed/api/SimpleQueryResult.java | 14 .../cassandra/distributed/api/QueryResultTest.java | 40 ++ 6 files changed, 84 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-in-jvm-dtest-api] branch master updated: make SimpleQueryResult a container for client warnings, and expose those warnings via QueryResult
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git
The following commit(s) were added to refs/heads/master by this push:
new 327c598 make SimpleQueryResult a container for client warnings, and
expose those warnings via QueryResult
327c598 is described below
commit 327c5980b21b1d7c47e0935ef5cb0c0c4c90d3ed
Author: Caleb Rackliffe
AuthorDate: Tue Jul 7 16:21:25 2020 -0500
make SimpleQueryResult a container for client warnings, and expose those
warnings via QueryResult
---
.../cassandra/distributed/api/QueryResult.java | 2 ++
.../cassandra/distributed/api/QueryResults.java| 24 +++--
.../distributed/api/SimpleQueryResult.java | 14
.../cassandra/distributed/api/QueryResultTest.java | 40 ++
4 files changed, 78 insertions(+), 2 deletions(-)
diff --git
a/src/main/java/org/apache/cassandra/distributed/api/QueryResult.java
b/src/main/java/org/apache/cassandra/distributed/api/QueryResult.java
index 9281794..90cc83f 100644
--- a/src/main/java/org/apache/cassandra/distributed/api/QueryResult.java
+++ b/src/main/java/org/apache/cassandra/distributed/api/QueryResult.java
@@ -58,6 +58,8 @@ import java.util.function.Predicate;
public interface QueryResult extends Iterator
{
List names();
+
+List warnings();
default QueryResult filter(Predicate fn)
{
diff --git
a/src/main/java/org/apache/cassandra/distributed/api/QueryResults.java
b/src/main/java/org/apache/cassandra/distributed/api/QueryResults.java
index 80202eb..081d06a 100644
--- a/src/main/java/org/apache/cassandra/distributed/api/QueryResults.java
+++ b/src/main/java/org/apache/cassandra/distributed/api/QueryResults.java
@@ -80,7 +80,8 @@ public final class QueryResults
private int numColumns = UNSET;
private String[] names;
-private List results = new ArrayList<>();
+private final List results = new ArrayList<>();
+private final List warnings = new ArrayList<>();
public Builder columns(String... columns)
{
@@ -110,6 +111,12 @@ public final class QueryResults
return this;
}
+public Builder warning(String message)
+{
+warnings.add(message);
+return this;
+}
+
public SimpleQueryResult build()
{
if (names == null)
@@ -120,7 +127,8 @@ public final class QueryResults
for (int i = 0; i < numColumns; i++)
names[i] = "unknown";
}
-return new SimpleQueryResult(names,
results.stream().toArray(Object[][]::new));
+
+return new SimpleQueryResult(names, results.toArray(new
Object[0][]), warnings);
}
}
@@ -147,6 +155,12 @@ public final class QueryResults
}
@Override
+public List warnings()
+{
+throw new UnsupportedOperationException("Warnings are not yet
supported for " + getClass().getSimpleName());
+}
+
+@Override
public boolean hasNext()
{
return iterator.hasNext();
@@ -176,6 +190,12 @@ public final class QueryResults
{
return delegate.names();
}
+
+@Override
+public List warnings()
+{
+return delegate.warnings();
+}
@Override
public boolean hasNext()
diff --git
a/src/main/java/org/apache/cassandra/distributed/api/SimpleQueryResult.java
b/src/main/java/org/apache/cassandra/distributed/api/SimpleQueryResult.java
index a44411d..04509e2 100644
--- a/src/main/java/org/apache/cassandra/distributed/api/SimpleQueryResult.java
+++ b/src/main/java/org/apache/cassandra/distributed/api/SimpleQueryResult.java
@@ -63,14 +63,21 @@ public class SimpleQueryResult implements QueryResult
{
private final String[] names;
private final Object[][] results;
+private final List warnings;
private final Predicate filter;
private final Row row;
private int offset = -1;
public SimpleQueryResult(String[] names, Object[][] results)
{
+this(names, results, Collections.emptyList());
+}
+
+public SimpleQueryResult(String[] names, Object[][] results, List
warnings)
+{
this.names = Objects.requireNonNull(names, "names");
this.results = results;
+this.warnings = Objects.requireNonNull(warnings, "warnings");
this.row = new Row(names);
this.filter = ignore -> true;
}
@@ -79,6 +86,7 @@ public class SimpleQueryResult implements QueryResult
{
this.names = names;
this.results = results;
+this.warnings = Collections.emptyList();
this.filter = filter;
this.offset = offset;
this.row = new Row(names);
@@ -89,6 +97,12 @@ public class SimpleQueryResul
[jira] [Updated] (CASSANDRA-15859) Avoid per-host hinted-handoff throttle being rounded to 0 in large cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-15859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15859:
-
Reviewers: Brandon Williams, Robert Stupp (was: Robert Stupp)
> Avoid per-host hinted-handoff throttle being rounded to 0 in large cluster
> --
>
> Key: CASSANDRA-15859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15859
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> When "hinted_handoff_throttle_in_kb" is sufficiently small or num of nodes in
> the cluster is sufficiently large, the per-host throttle will be rounded to
> 0, aka. unthrottled.
>
> {code:java|title=HintsDispatchExecutor.java}
> int throttleInKB = DatabaseDescriptor.getHintedHandoffThrottleInKB() /
> nodesCount;
> this.rateLimiter = RateLimiter.create(throttleInKB == 0 ? Double.MAX_VALUE :
> throttleInKB * 1024);
> {code}
> [trunk-patch|https://github.com/apache/cassandra/pull/616]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15859) Avoid per-host hinted-handoff throttle being rounded to 0 in large cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-15859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15859:
-
Since Version: 1.2.6
Source Control Link:
https://github.com/apache/cassandra/commit/5d44a5080acacfe93aae3f19b7788035dd04cd03
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed, thanks.
> Avoid per-host hinted-handoff throttle being rounded to 0 in large cluster
> --
>
> Key: CASSANDRA-15859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15859
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> When "hinted_handoff_throttle_in_kb" is sufficiently small or num of nodes in
> the cluster is sufficiently large, the per-host throttle will be rounded to
> 0, aka. unthrottled.
>
> {code:java|title=HintsDispatchExecutor.java}
> int throttleInKB = DatabaseDescriptor.getHintedHandoffThrottleInKB() /
> nodesCount;
> this.rateLimiter = RateLimiter.create(throttleInKB == 0 ? Double.MAX_VALUE :
> throttleInKB * 1024);
> {code}
> [trunk-patch|https://github.com/apache/cassandra/pull/616]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15859) Avoid per-host hinted-handoff throttle being rounded to 0 in large cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-15859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15859:
-
Status: Ready to Commit (was: Review In Progress)
> Avoid per-host hinted-handoff throttle being rounded to 0 in large cluster
> --
>
> Key: CASSANDRA-15859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15859
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> When "hinted_handoff_throttle_in_kb" is sufficiently small or num of nodes in
> the cluster is sufficiently large, the per-host throttle will be rounded to
> 0, aka. unthrottled.
>
> {code:java|title=HintsDispatchExecutor.java}
> int throttleInKB = DatabaseDescriptor.getHintedHandoffThrottleInKB() /
> nodesCount;
> this.rateLimiter = RateLimiter.create(throttleInKB == 0 ? Double.MAX_VALUE :
> throttleInKB * 1024);
> {code}
> [trunk-patch|https://github.com/apache/cassandra/pull/616]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (ac28927 -> e819f9b)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from ac28927 Merge branch 'cassandra-3.0' into cassandra-3.11 new 979a71b Avoid hinted handoff per-host throttle being arounded to 0 in large cluster new e819f9b Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + src/java/org/apache/cassandra/hints/HintsDispatchExecutor.java | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated: Avoid hinted handoff per-host throttle being arounded to 0 in large cluster
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-3.0 by this push: new 979a71b Avoid hinted handoff per-host throttle being arounded to 0 in large cluster 979a71b is described below commit 979a71b13af0fac643a6cd6957e74313139ed17d Author: Zhao Yang AuthorDate: Sun Jun 7 01:14:56 2020 +0800 Avoid hinted handoff per-host throttle being arounded to 0 in large cluster Patch by Zhao Yang, reviewed by snazy and brandonwilliams for CASSANDRA-15859 --- CHANGES.txt| 1 + src/java/org/apache/cassandra/hints/HintsDispatchExecutor.java | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index 631b329..16974c4 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.21 + * Avoid hinted handoff per-host throttle being arounded to 0 in large cluster (CASSANDRA-15859) * Avoid emitting empty range tombstones from RangeTombstoneList (CASSANDRA-15924) * Avoid thread starvation, and improve compare-and-swap performance, in the slab allocators (CASSANDRA-15922) * Fix broken KEYS 2i queries after DROP COMPACT STORAGE (CASSANDRA-15906) diff --git a/src/java/org/apache/cassandra/hints/HintsDispatchExecutor.java b/src/java/org/apache/cassandra/hints/HintsDispatchExecutor.java index c562dd0..eda4179 100644 --- a/src/java/org/apache/cassandra/hints/HintsDispatchExecutor.java +++ b/src/java/org/apache/cassandra/hints/HintsDispatchExecutor.java @@ -206,8 +206,8 @@ final class HintsDispatchExecutor // not total outgoing hints traffic from this node - this is why the rate limiter is not shared between // all the dispatch tasks (as there will be at most one dispatch task for a particular host id at a time). int nodesCount = Math.max(1, StorageService.instance.getTokenMetadata().getAllEndpoints().size() - 1); -int throttleInKB = DatabaseDescriptor.getHintedHandoffThrottleInKB() / nodesCount; -this.rateLimiter = RateLimiter.create(throttleInKB == 0 ? Double.MAX_VALUE : throttleInKB * 1024); +double throttleInBytes = DatabaseDescriptor.getHintedHandoffThrottleInKB() * 1024.0 / nodesCount; +this.rateLimiter = RateLimiter.create(throttleInBytes == 0 ? Double.MAX_VALUE : throttleInBytes); } public void run() - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (b6ce3b8 -> 5d44a50)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from b6ce3b8 Merge branch 'cassandra-3.11' into trunk new 979a71b Avoid hinted handoff per-host throttle being arounded to 0 in large cluster new e819f9b Merge branch 'cassandra-3.0' into cassandra-3.11 new 5d44a50 Merge branch 'cassandra-3.11' into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + src/java/org/apache/cassandra/hints/HintsDispatchExecutor.java | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit e819f9b69a151063f3d5770f57c4144a66cba409 Merge: ac28927 979a71b Author: Brandon Williams AuthorDate: Fri Jul 10 08:42:35 2020 -0500 Merge branch 'cassandra-3.0' into cassandra-3.11 CHANGES.txt| 1 + src/java/org/apache/cassandra/hints/HintsDispatchExecutor.java | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --cc CHANGES.txt index 10234d8,16974c4..d4ce6dc --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,11 -1,8 +1,12 @@@ -3.0.21 +3.11.7 + * Fix cqlsh output when fetching all rows in batch mode (CASSANDRA-15905) + * Upgrade Jackson to 2.9.10 (CASSANDRA-15867) + * Fix CQL formatting of read command restrictions for slow query log (CASSANDRA-15503) + * Allow sstableloader to use SSL on the native port (CASSANDRA-14904) +Merged from 3.0: + * Avoid hinted handoff per-host throttle being arounded to 0 in large cluster (CASSANDRA-15859) * Avoid emitting empty range tombstones from RangeTombstoneList (CASSANDRA-15924) * Avoid thread starvation, and improve compare-and-swap performance, in the slab allocators (CASSANDRA-15922) - * Fix broken KEYS 2i queries after DROP COMPACT STORAGE (CASSANDRA-15906) * Add token to tombstone warning and error messages (CASSANDRA-15890) * Fixed range read concurrency factor computation and capped as 10 times tpc cores (CASSANDRA-15752) * Catch exception on bootstrap resume and init native transport (CASSANDRA-15863) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 5d44a5080acacfe93aae3f19b7788035dd04cd03 Merge: b6ce3b8 e819f9b Author: Brandon Williams AuthorDate: Fri Jul 10 08:46:09 2020 -0500 Merge branch 'cassandra-3.11' into trunk CHANGES.txt| 1 + src/java/org/apache/cassandra/hints/HintsDispatchExecutor.java | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-) diff --cc CHANGES.txt index 0e0ad17,d4ce6dc..c2b3d69 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -58,7 -2,9 +58,8 @@@ Merged from 3.11 * Fix cqlsh output when fetching all rows in batch mode (CASSANDRA-15905) * Upgrade Jackson to 2.9.10 (CASSANDRA-15867) * Fix CQL formatting of read command restrictions for slow query log (CASSANDRA-15503) - * Allow sstableloader to use SSL on the native port (CASSANDRA-14904) Merged from 3.0: + * Avoid hinted handoff per-host throttle being arounded to 0 in large cluster (CASSANDRA-15859) * Avoid emitting empty range tombstones from RangeTombstoneList (CASSANDRA-15924) * Avoid thread starvation, and improve compare-and-swap performance, in the slab allocators (CASSANDRA-15922) * Add token to tombstone warning and error messages (CASSANDRA-15890) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15873) Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues)
[ https://issues.apache.org/jira/browse/CASSANDRA-15873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155476#comment-17155476 ] Brandon Williams commented on CASSANDRA-15873: -- I ran CI here: [https://ci-cassandra.apache.org/job/Cassandra-devbranch/206/] and here: [https://app.circleci.com/pipelines/github/driftx/cassandra/51/workflows/4680a5da-8e32-474e-b01c-059728a30b6f] I believe the teardown failures like [https://app.circleci.com/pipelines/github/driftx/cassandra/51/workflows/4680a5da-8e32-474e-b01c-059728a30b6f/jobs/595] are new. > Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues) > --- > > Key: CASSANDRA-15873 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15873 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Matt Davis >Assignee: Matt Davis >Priority: Normal > Fix For: 3.11.x > > Attachments: dependency-check-report.html, unittest_netty.log > > > See https://issues.apache.org/jira/browse/CASSANDRA-15868 for the same issue > on 4.0 / trunk. Attached is an OWASP dependency report for Netty 4.0.44, > which identifies 3 of the same vulnerabilities as above. > > Additionally, 4.1.50 contains aarch64 native libraries which can improve > performance on ARM processors. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14793) Improve system table handling when losing a disk when using JBOD
[ https://issues.apache.org/jira/browse/CASSANDRA-14793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155415#comment-17155415 ] Benjamin Lerer commented on CASSANDRA-14793: The patch ensure that the data of the local system keyspaces is stored by default within the first data directory (at the exception of the paxos table) to allow the node to tolerate a failure of the disks associated with the other directories. The patch also allow users to configure a different directory for the system keyspaces. This would allow people to use a disk providing redundancy to support the lost of any of the disks used to store the data. On startup existing system keyspace data will be automatically migrated to support 4.0 upgrades or configuration changes (use of a separate disk for the system keyspaces). > Improve system table handling when losing a disk when using JBOD > > > Key: CASSANDRA-14793 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14793 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core >Reporter: Marcus Eriksson >Assignee: Benjamin Lerer >Priority: Normal > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > We should improve the way we handle disk failures when losing a disk in a > JBOD setup > One way could be to pin the system tables to a special data directory. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14793) Improve system table handling when losing a disk when using JBOD
[ https://issues.apache.org/jira/browse/CASSANDRA-14793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-14793: --- Test and Documentation Plan: * Run of JUnit tests with separate directory for local system keyspaces * New unit tests for some of the new code. Status: Patch Available (was: Open) > Improve system table handling when losing a disk when using JBOD > > > Key: CASSANDRA-14793 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14793 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core >Reporter: Marcus Eriksson >Assignee: Benjamin Lerer >Priority: Normal > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > We should improve the way we handle disk failures when losing a disk in a > JBOD setup > One way could be to pin the system tables to a special data directory. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15909) Make Table/Keyspace Metric Names Consistent With Each Other
[
https://issues.apache.org/jira/browse/CASSANDRA-15909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155406#comment-17155406
]
Stephen Mallette commented on CASSANDRA-15909:
--
I've beat this one back and forth a bunch in terms of how to best implement
this. Ultimately I chose to just keep the changes fairly focused in on where
the "deprecation" occurred with the metrics so as not to overhaul too much
existing code. I imagine it would have been nicer to build the concept of
"metric deprecation" into the {{CassandraMetricsRegistry}} a bit more as a
first class feature of it, but given the proximity to release of 4.0 it is
perhaps best to save such larger changes for a future date.
My changes can be reviewed here:
https://github.com/spmallette/cassandra/pull/2
As I alluded to earlier, when this change merges I will rebase CASSANDRA-15821
and make the appropriate documentation changes there. Those documentation
changes will probably be necessary to explain the slightly differently named
metrics that now appear in JMX.
> Make Table/Keyspace Metric Names Consistent With Each Other
> ---
>
> Key: CASSANDRA-15909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15909
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Metrics
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
> Fix For: 4.0-beta
>
>
> As part of CASSANDRA-15821 it became apparent that certain metric names found
> in keyspace and tables had different names but were in fact the same metric -
> they are as follows:
> * Table.SyncTime == Keyspace.RepairSyncTime
> * Table.RepairedDataTrackingOverreadRows == Keyspace.RepairedOverreadRows
> * Table.RepairedDataTrackingOverreadTime == Keyspace.RepairedOverreadTime
> * Table.AllMemtablesHeapSize == Keyspace.AllMemtablesOnHeapDataSize
> * Table.AllMemtablesOffHeapSize == Keyspace.AllMemtablesOffHeapDataSize
> * Table.MemtableOnHeapSize == Keyspace.MemtableOnHeapDataSize
> * Table.MemtableOffHeapSize == Keyspace.MemtableOffHeapDataSize
> Also, client metrics are the only metrics to start with a lower case letter.
> Change those to upper case to match all the other metrics.
> Unifying this naming would help make metrics more consistent as part of
> CASSANDRA-15582
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15859) Avoid per-host hinted-handoff throttle being rounded to 0 in large cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-15859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155307#comment-17155307
]
ZhaoYang commented on CASSANDRA-15859:
--
|patch|circle|
| [trunk|https://github.com/apache/cassandra/pull/616/files] |
[ci|https://circleci.com/workflow-run/63f19a49-568a-4350-b368-9c33eeaa17de] |
| [3.11|https://github.com/apache/cassandra/pull/674/files] |
[ci|https://circleci.com/workflow-run/f18b7afa-36c7-4d7b-a5a3-9792528cc963] |
| [3.0|https://github.com/apache/cassandra/pull/673/files] |
[ci|https://circleci.com/workflow-run/e2b22eef-f0b2-4752-a4b6-f1b5766e170c] |
ported to 3.0 and 3.11..
> Avoid per-host hinted-handoff throttle being rounded to 0 in large cluster
> --
>
> Key: CASSANDRA-15859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15859
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> When "hinted_handoff_throttle_in_kb" is sufficiently small or num of nodes in
> the cluster is sufficiently large, the per-host throttle will be rounded to
> 0, aka. unthrottled.
>
> {code:java|title=HintsDispatchExecutor.java}
> int throttleInKB = DatabaseDescriptor.getHintedHandoffThrottleInKB() /
> nodesCount;
> this.rateLimiter = RateLimiter.create(throttleInKB == 0 ? Double.MAX_VALUE :
> throttleInKB * 1024);
> {code}
> [trunk-patch|https://github.com/apache/cassandra/pull/616]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15857) Frozen RawTuple is not annotated with frozen in the toString method
[
https://issues.apache.org/jira/browse/CASSANDRA-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155304#comment-17155304
]
Robert Stupp commented on CASSANDRA-15857:
--
As tuples cannot be frozen at all, it seems more convenient to completely omit
the {{frozen<>}} bracket. I.e. removing the {{frozen}} param from
{{RawTuple.}}, removing the {{frozen}} handling in
{{CQL3Type.Tuple.toString()}} (remove {{frozen<>}} entirely from the string)
and adopting the frozen-validation-logic in
{{CQL3Type.Raw.RawCollection.prepare()}}.
Honestly, the handling of frozen isn't actually great in the code base, but
cleaning that up is quite a bit of work (too much for a patch release and even
for 4.0 now). E.g. it should be ensured that all subtypes of a type are always
frozen no matter what, the whole partition key must always be frozen,
clustering key elements must be frozen, a bunch of code constructs like "if
(some-check) freeze()" should be simplified and generalized. Things become a
bit complicated when freezable types are nested, for example a non-frozen-UDT
in a non-frozen-collection in a UDT, which is legal.
> Frozen RawTuple is not annotated with frozen in the toString method
> ---
>
> Key: CASSANDRA-15857
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15857
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0, 3.11.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> All raw types (e.g. RawCollection, RawUT) that supports freezing wraps the
> type name with 'frozen<>' in the toString method, except RawTuple.
> Therefore, the RawTuple::toString output misses the frozen wrapper.
> Tuple is always frozen. However since CASSANDRA-15035, it throws when the
> inner tuple is not explicitly wrapped with frozen within a collection.
> The method, CQL3Type.Raw::toString, is referenced at multiple places in the
> source. For example, referenced in CreateTypeStatement.Raw and involved in
> CQLSSTableWriter. Another example is that it is called to produce the
> SchemaChange at several AlterSchemaStatement implementations.
> A test can prove that missing the frozen wrapper causes exception when
> building CQLSSTableWriter for user types defined like below. Note that the
> inner tuple is wrapped with frozen in the initial CQL statement.
> {code:java}
> CREATE TYPE ks.fooType ( f list>> )
> {code}
> {code:java}
> org.apache.cassandra.exceptions.InvalidRequestException: Non-frozen tuples
> are not allowed inside collections: list>
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.throwNestedNonFrozenError(CQL3Type.java:710)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepare(CQL3Type.java:669)
> at
> org.apache.cassandra.cql3.CQL3Type$Raw$RawCollection.prepareInternal(CQL3Type.java:661)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.lambda$prepare$1(Types.java:341)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.cassandra.schema.Types$RawBuilder$RawUDT.prepare(Types.java:342)
> at org.apache.cassandra.schema.Types$RawBuilder.build(Types.java:291)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.createTypes(CQLSSTableWriter.java:551)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.build(CQLSSTableWriter.java:527)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15851) Add bytebuddy support for in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15851: Fix Version/s: (was: 4.0) 4.0-alpha 3.11.7 3.0.21 2.2.17 Source Control Link: https://github.com/apache/cassandra/commit/f79d1059287cc7370004e64175942bd50f2c Resolution: Fixed Status: Resolved (was: Ready to Commit) and committed, thanks! > Add bytebuddy support for in-jvm dtests > --- > > Key: CASSANDRA-15851 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15851 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Labels: pull-request-available > Fix For: 2.2.17, 3.0.21, 3.11.7, 4.0-alpha > > > Old python dtests support byteman, but that is quite horrible to work with, > [bytebuddy|https://bytebuddy.net/#/] is much better, so we should add support > for that in the in-jvm dtests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (52ce48b -> b6ce3b8)
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 52ce48b Merge branch 'cassandra-3.11' into trunk new f79d105 Add bytebuddy support for in-jvm dtest new fad5a09 Merge branch 'cassandra-2.2' into cassandra-3.0 new ac28927 Merge branch 'cassandra-3.0' into cassandra-3.11 new b6ce3b8 Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: build.xml | 7 +- .../distributed/impl/AbstractCluster.java | 5 + .../distributed/test/ByteBuddyExamples.java| 123 + 3 files changed, 134 insertions(+), 1 deletion(-) create mode 100644 test/distributed/org/apache/cassandra/distributed/test/ByteBuddyExamples.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15851) Add bytebuddy support for in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15851: Status: Ready to Commit (was: Review In Progress) > Add bytebuddy support for in-jvm dtests > --- > > Key: CASSANDRA-15851 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15851 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > > Old python dtests support byteman, but that is quite horrible to work with, > [bytebuddy|https://bytebuddy.net/#/] is much better, so we should add support > for that in the in-jvm dtests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit fad5a09c9f2ca84e4e030061fb02ca823128c896 Merge: ab6a87b f79d105 Author: Marcus Eriksson AuthorDate: Fri Jul 10 10:30:46 2020 +0200 Merge branch 'cassandra-2.2' into cassandra-3.0 build.xml | 5 +- .../distributed/impl/AbstractCluster.java | 6 + .../distributed/test/ByteBuddyExamples.java| 123 + 3 files changed, 133 insertions(+), 1 deletion(-) diff --cc build.xml index ba6d817,8bed062..d50007e --- a/build.xml +++ b/build.xml @@@ -415,11 -419,19 +416,13 @@@ - - - - + + + + + + - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (ab6a87b -> fad5a09)
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from ab6a87b Avoid emitting empty range tombstones from RangeTombstoneList new f79d105 Add bytebuddy support for in-jvm dtest new fad5a09 Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: build.xml | 5 +- .../distributed/impl/AbstractCluster.java | 6 + .../distributed/test/ByteBuddyExamples.java| 123 + 3 files changed, 133 insertions(+), 1 deletion(-) create mode 100644 test/distributed/org/apache/cassandra/distributed/test/ByteBuddyExamples.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit b6ce3b8091aced43adc27ff86b54a5622cdc59d2
Merge: 52ce48b ac28927
Author: Marcus Eriksson
AuthorDate: Fri Jul 10 10:37:40 2020 +0200
Merge branch 'cassandra-3.11' into trunk
build.xml | 7 +-
.../distributed/impl/AbstractCluster.java | 5 +
.../distributed/test/ByteBuddyExamples.java| 123 +
3 files changed, 134 insertions(+), 1 deletion(-)
diff --cc build.xml
index 15d4276,9932604..43ec135
--- a/build.xml
+++ b/build.xml
@@@ -115,19 -119,10 +115,20 @@@
-
+
+
+
+
+
+
+
-
+
+
+
+
+
+
@@@ -544,14 -412,19 +545,15 @@@
-
+
-
-
-
-
-
-
-
-
+
+
+
-
+
++
diff --cc
test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
index 0c8421a,3cb8dac..a28c935
---
a/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
+++
b/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
@@@ -30,9 -30,9 +30,10 @@@ import java.util.Set
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
+ import java.util.function.BiConsumer;
import java.util.function.Consumer;
import java.util.stream.Collectors;
+import java.util.stream.IntStream;
import java.util.stream.Stream;
import com.google.common.collect.Sets;
@@@ -118,30 -117,10 +119,31 @@@ public abstract class AbstractCluster<
// mutated by user-facing API
private final MessageFilters filters;
+private final INodeProvisionStrategy.Strategy nodeProvisionStrategy;
+ private final BiConsumer instanceInitializer;
-
private volatile Thread.UncaughtExceptionHandler previousHandler = null;
+/**
+ * Common builder, add methods that are applicable to both Cluster and
Upgradable cluster here.
+ */
+public static abstract class AbstractBuilder>
+extends org.apache.cassandra.distributed.shared.AbstractBuilder
+{
+private INodeProvisionStrategy.Strategy nodeProvisionStrategy =
INodeProvisionStrategy.Strategy.MultipleNetworkInterfaces;
+
+public AbstractBuilder(Factory factory)
+{
+super(factory);
+}
+
+public B withNodeProvisionStrategy(INodeProvisionStrategy.Strategy
nodeProvisionStrategy)
+{
+this.nodeProvisionStrategy = nodeProvisionStrategy;
+return (B) this;
+}
+}
+
+
protected class Wrapper extends DelegatingInvokableInstance implements
IUpgradeableInstance
{
private final int generation;
@@@ -176,8 -155,10 +178,10 @@@
private IInvokableInstance newInstance(int generation)
{
ClassLoader classLoader = new InstanceClassLoader(generation,
config.num(), version.classpath, sharedClassLoader);
+ if (instanceInitializer != null)
+ instanceInitializer.accept(classLoader, config.num());
-return
Instance.transferAdhoc((SerializableBiFunction)Instance::new, classLoader)
-.apply(config, classLoader);
+return
Instance.transferAdhoc((SerializableBiFunction)Instance::new, classLoader)
+
.apply(config.forVersion(version.major), classLoader);
}
public IInstanceConfig config()
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (ca2b281 -> ac28927)
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from ca2b281 Merge branch 'cassandra-3.0' into cassandra-3.11 new f79d105 Add bytebuddy support for in-jvm dtest new fad5a09 Merge branch 'cassandra-2.2' into cassandra-3.0 new ac28927 Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: build.xml | 5 +- .../distributed/impl/AbstractCluster.java | 6 + .../distributed/test/ByteBuddyExamples.java| 123 + 3 files changed, 133 insertions(+), 1 deletion(-) create mode 100644 test/distributed/org/apache/cassandra/distributed/test/ByteBuddyExamples.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit ac289270f2bb3bb7251319f7f71d6c66a4272db4
Merge: ca2b281 fad5a09
Author: Marcus Eriksson
AuthorDate: Fri Jul 10 10:32:37 2020 +0200
Merge branch 'cassandra-3.0' into cassandra-3.11
build.xml | 5 +-
.../distributed/impl/AbstractCluster.java | 6 +
.../distributed/test/ByteBuddyExamples.java| 123 +
3 files changed, 133 insertions(+), 1 deletion(-)
diff --cc build.xml
index 25a4733,d50007e..9932604
--- a/build.xml
+++ b/build.xml
@@@ -447,9 -421,11 +448,11 @@@
+
+
-
-
+
+
diff --cc
test/distributed/org/apache/cassandra/distributed/test/ByteBuddyExamples.java
index 000,b63b3e1..b49572d
mode 00,100644..100644
---
a/test/distributed/org/apache/cassandra/distributed/test/ByteBuddyExamples.java
+++
b/test/distributed/org/apache/cassandra/distributed/test/ByteBuddyExamples.java
@@@ -1,0 -1,123 +1,123 @@@
+ /*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+ package org.apache.cassandra.distributed.test;
+
+ import java.io.IOException;
+ import java.util.concurrent.Callable;
+ import java.util.concurrent.atomic.AtomicInteger;
+
+ import org.junit.Test;
+
+ import net.bytebuddy.ByteBuddy;
+ import net.bytebuddy.dynamic.loading.ClassLoadingStrategy;
+ import net.bytebuddy.implementation.MethodDelegation;
+ import net.bytebuddy.implementation.bind.annotation.SuperCall;
+ import org.apache.cassandra.cql3.QueryOptions;
+ import org.apache.cassandra.cql3.statements.ModificationStatement;
+ import org.apache.cassandra.cql3.statements.SelectStatement;
+ import org.apache.cassandra.distributed.Cluster;
+ import org.apache.cassandra.distributed.api.ConsistencyLevel;
+ import org.apache.cassandra.service.QueryState;
+ import org.apache.cassandra.transport.messages.ResultMessage;
+
+ import static net.bytebuddy.matcher.ElementMatchers.named;
+ import static net.bytebuddy.matcher.ElementMatchers.takesArguments;
+ import static org.junit.Assert.assertEquals;
+ import static org.junit.Assert.fail;
+
+ public class ByteBuddyExamples extends TestBaseImpl
+ {
+ @Test
+ public void writeFailureTest() throws Throwable
+ {
+ try(Cluster cluster = init(Cluster.build(1)
+
.withInstanceInitializer(BBFailHelper::install)
+ .start()))
+ {
+ cluster.schemaChange("create table " + KEYSPACE + ".tbl (id int
primary key, t int)");
+ try
+ {
+ cluster.coordinator(1).execute("insert into " + KEYSPACE +
".tbl (id, t) values (1, 1)", ConsistencyLevel.ALL);
+ fail("Should fail");
+ }
+ catch (RuntimeException e)
+ {
+ // expected
+ }
+ }
+ }
+
+ public static class BBFailHelper
+ {
+ static void install(ClassLoader cl, int nodeNumber)
+ {
+ new ByteBuddy().redefine(ModificationStatement.class)
+.method(named("execute"))
+.intercept(MethodDelegation.to(BBFailHelper.class))
+.make()
+.load(cl, ClassLoadingStrategy.Default.INJECTION);
+ }
+ public static ResultMessage execute()
+ {
+ throw new RuntimeException();
+ }
+ }
+
+ @Test
+ public void countTest() throws IOException
+ {
+ try(Cluster cluster = init(Cluster.build(2)
+
.withInstanceInitializer(BBCountHelper::install)
+ .start()))
+ {
+ cluster.schemaChange("create table " + KEYSPACE + ".tbl (id int
primary key, t int)");
+ cluster.coordinator(1).execute("select * from " + KEYSPACE +
".tbl;", ConsistencyLevel.ALL);
+
[cassandra] branch cassandra-2.2 updated: Add bytebuddy support for in-jvm dtest
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-2.2 by this push:
new f79d105 Add bytebuddy support for in-jvm dtest
f79d105 is described below
commit f79d1059287cc7370004e64175942bd50f2c
Author: Marcus Eriksson
AuthorDate: Wed Jun 10 10:05:54 2020 +0200
Add bytebuddy support for in-jvm dtest
Patch by marcuse; reviewed by David Capwell for CASSANDRA-15851
---
build.xml | 5 +-
.../distributed/impl/AbstractCluster.java | 6 +
.../distributed/test/ByteBuddyExamples.java| 123 +
3 files changed, 133 insertions(+), 1 deletion(-)
diff --git a/build.xml b/build.xml
index 83fb674..8bed062 100644
--- a/build.xml
+++ b/build.xml
@@ -123,6 +123,7 @@
+
@@ -395,7 +396,7 @@
-
+
@@ -429,6 +430,8 @@
+
+
diff --git
a/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
b/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
index 28abece..0085f1c 100644
---
a/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
+++
b/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
@@ -30,6 +30,7 @@ import java.util.Set;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
+import java.util.function.BiConsumer;
import java.util.function.Consumer;
import java.util.stream.Collectors;
import java.util.stream.Stream;
@@ -116,6 +117,8 @@ public abstract class AbstractCluster
implements ICluster instanceInitializer;
+
private volatile Thread.UncaughtExceptionHandler previousHandler = null;
protected class Wrapper extends DelegatingInvokableInstance implements
IUpgradeableInstance
@@ -152,6 +155,8 @@ public abstract class AbstractCluster
implements ICluster)Instance::new, classLoader)
.apply(config, classLoader);
}
@@ -261,6 +266,7 @@ public abstract class AbstractCluster
implements ICluster();
this.initialVersion = builder.getVersion();
this.filters = new MessageFilters();
+this.instanceInitializer = builder.getInstanceInitializer();
int generation = GENERATION.incrementAndGet();
for (int i = 0; i < builder.getNodeCount(); ++i)
diff --git
a/test/distributed/org/apache/cassandra/distributed/test/ByteBuddyExamples.java
b/test/distributed/org/apache/cassandra/distributed/test/ByteBuddyExamples.java
new file mode 100644
index 000..b63b3e1
--- /dev/null
+++
b/test/distributed/org/apache/cassandra/distributed/test/ByteBuddyExamples.java
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.distributed.test;
+
+import java.io.IOException;
+import java.util.concurrent.Callable;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import org.junit.Test;
+
+import net.bytebuddy.ByteBuddy;
+import net.bytebuddy.dynamic.loading.ClassLoadingStrategy;
+import net.bytebuddy.implementation.MethodDelegation;
+import net.bytebuddy.implementation.bind.annotation.SuperCall;
+import org.apache.cassandra.cql3.QueryOptions;
+import org.apache.cassandra.cql3.statements.ModificationStatement;
+import org.apache.cassandra.cql3.statements.SelectStatement;
+import org.apache.cassandra.distributed.Cluster;
+import org.apache.cassandra.distributed.api.ConsistencyLevel;
+import org.apache.cassandra.service.QueryState;
+import org.apache.cassandra.transport.messages.ResultMessage;
+
+import static net.bytebuddy.matcher.ElementMatchers.named;
+import static net.bytebuddy.matcher.ElementMatchers.takesArguments;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.fail;
+
+public class ByteBuddyExamples extends TestBaseImpl
+{
+@Test
+public void writeFailureTest(
