[jira] [Commented] (CASSANDRA-18398) CEP-25: Trie-indexed SSTable format
[
https://issues.apache.org/jira/browse/CASSANDRA-18398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715208#comment-17715208
]
Caleb Rackliffe commented on CASSANDRA-18398:
-
Just ran the branch through some Harry fuzz testing w/ the {{bti}} format
active. The configuration I'm using basically looks like this:
{noformat}
new Configuration.ConfigurationBuilder()
.setSeed(1L)
.setSchemaProvider(new
Configuration.DefaultSchemaProviderConfiguration())
.setDropSchema(false)
.setCreateSchema(true)
.setTruncateTable(false)
.setClock(new
Configuration.ApproximateMonotonicClockConfiguration(7300, 1, TimeUnit.SECONDS))
.setSUT(new InJvmSutConfiguration(3, 10, ...))
.setPartitionDescriptorSelector(new
Configuration.DefaultPDSelectorConfiguration(10, 100))
.setClusteringDescriptorSelector(
new Configuration.DefaultCDSelectorConfiguration(
new Configuration.ConstantDistributionConfig(4),
new Configuration.ConstantDistributionConfig(2),
1000,
ImmutableMap.builder()
.put(OpSelectors.OperationKind.DELETE_RANGE, 1)
.put(OpSelectors.OperationKind.DELETE_SLICE, 1)
.put(OpSelectors.OperationKind.DELETE_ROW, 1)
.put(OpSelectors.OperationKind.DELETE_COLUMN, 1)
.put(OpSelectors.OperationKind.DELETE_PARTITION, 1)
.put(OpSelectors.OperationKind.DELETE_COLUMN_WITH_STATICS, 1)
.put(OpSelectors.OperationKind.INSERT_WITH_STATICS, 50)
.put(OpSelectors.OperationKind.INSERT,
50)
.put(OpSelectors.OperationKind.UPDATE_WITH_STATICS, 50)
.put(OpSelectors.OperationKind.UPDATE,
50)
.build(),
null))
.setRunner(new
Configuration.SequentialRunnerConfig(ImmutableList.of(
new
Configuration.LoggingVisitorConfiguration(MutatingRowVisitor::new),
new
ParallelRecentValidator.ParallelRecentValidatorConfig(100, 20, 10_000,
new
Configuration.QuiescentCheckerConfig(),
QueryLogger.NoOpQueryLogger::new),
new
Configuration.AllPartitionsValidatorConfiguration(20,
new
Configuration.QuiescentCheckerConfig(),
QueryLogger.NoOpQueryLogger::new)),
60,
TimeUnit.MINUTES))
.build();
{noformat}
I managed to get through that hour run without detecting any problems, which is
a good sign. Going to press on and finish the actual review...
> CEP-25: Trie-indexed SSTable format
> ---
>
> Key: CASSANDRA-18398
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18398
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Branimir Lambov
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 14h
> Remaining Estimate: 0h
>
> Implementation of Big Trie-Indexed (BTI) SSTable format, per
> [CEP-25|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-25%3A+Trie-indexed+SSTable+format].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (bc21a268 -> cabdd75b)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard bc21a268 generate docs for eb5e2e7b new cabdd75b generate docs for eb5e2e7b This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (bc21a268) \ N -- N -- N refs/heads/asf-staging (cabdd75b) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/search-index.js | 2 +- site-ui/build/ui-bundle.zip | Bin 4796900 -> 4796900 bytes 2 files changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18475) Formatting looks broken on the metrics page
Yury Vidineev created CASSANDRA-18475: - Summary: Formatting looks broken on the metrics page Key: CASSANDRA-18475 URL: https://issues.apache.org/jira/browse/CASSANDRA-18475 Project: Cassandra Issue Type: Bug Components: Documentation/Website Reporter: Yury Vidineev [https://cassandra.apache.org/doc/latest/cassandra/operating/metrics.html] Things like "[cols=",,",options="header",]" look odd, and I believe is a part of some format setting (row HTML?) and isn't supposed to be displayed. I'm willing to help if there is a place where I can send a merge request. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18474) Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequest
[
https://issues.apache.org/jira/browse/CASSANDRA-18474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715196#comment-17715196
]
Jon Meredith commented on CASSANDRA-18474:
--
sigh... managed to add review notes at the test plan. Leaving them there.
> Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequest
> ---
>
> Key: CASSANDRA-18474
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18474
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Repair, Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{SyncRequest}} message assumes the initiator/src/dst are all the IP
> address
> type as an optimization when calculating serialized size. It needs to handle
> mixed address families.
> {code}
> /1.2.3.4:7000->/[::1]:7000-SMALL_MESSAGES-f83ce0bc dropping message of type
> SYNC_REQ due to error"
> {code}
> {Exception}
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 158, actual size at least 170, for verb SYNC_REQ
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18428) Implement/override equals and hashCode methods in the ServerEncryptionOptions class
[
https://issues.apache.org/jira/browse/CASSANDRA-18428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715192#comment-17715192
]
Maulin Vasavada commented on CASSANDRA-18428:
-
[~edimitrova] Does this sound like a straightforward change to be considered
for a quick review/merge?
> Implement/override equals and hashCode methods in the ServerEncryptionOptions
> class
> ---
>
> Key: CASSANDRA-18428
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18428
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Maulin Vasavada
>Assignee: Maulin Vasavada
>Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We have {{equals and hashCode}} methods in
> [EncryptionOptions|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/EncryptionOptions.java#L551]
> object but not (overridden/extended) in
> [ServerEncryptionOptions.|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/EncryptionOptions.java#L600]
> Code is using the EncryptionOptions as the key in the
> [ConcurrentHashMap|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/security/SSLFactory.java#L84]
> in the SSLFactory.java. Hence technically we must have a equals/hashCode
> override in the ServerEncryptionOptions to account for fields that matter
> additionally (e.g. outbound_keystore/password).
> We discussed this over the [cassandra-dev slack
> channel|https://the-asf.slack.com/archives/CK23JSY2K/p1680118849081399] and
> it seems agreeable to make this change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18474) Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequest
[
https://issues.apache.org/jira/browse/CASSANDRA-18474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-18474:
Reviewers: Caleb Rackliffe
> Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequest
> ---
>
> Key: CASSANDRA-18474
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18474
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Repair, Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{SyncRequest}} message assumes the initiator/src/dst are all the IP
> address
> type as an optimization when calculating serialized size. It needs to handle
> mixed address families.
> {code}
> /1.2.3.4:7000->/[::1]:7000-SMALL_MESSAGES-f83ce0bc dropping message of type
> SYNC_REQ due to error"
> {code}
> {Exception}
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 158, actual size at least 170, for verb SYNC_REQ
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (7aa0a507 -> bc21a268)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git omit 7aa0a507 generate docs for eb5e2e7b new bc21a268 generate docs for eb5e2e7b This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (7aa0a507) \ N -- N -- N refs/heads/asf-staging (bc21a268) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: site-ui/build/ui-bundle.zip | Bin 4796900 -> 4796900 bytes 1 file changed, 0 insertions(+), 0 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18472) Docker images can no longer be built due to python2.7
[
https://issues.apache.org/jira/browse/CASSANDRA-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715187#comment-17715187
]
Dinesh Joshi commented on CASSANDRA-18472:
--
It would have been great if we would've had cqlsh as a separate repo /
sub-project. It would be easy to avoid issues such as these. Having Python 2
dependency for 3.0 and 3.11 is not great.
> Docker images can no longer be built due to python2.7
> -
>
> Key: CASSANDRA-18472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
>
> {noformat}
> => [linux/amd64 35/56] WORKDIR /home/cassandra
>
> 0.1s
> => [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
> /home/cassandra/.bashrc && echo 'export
> JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
> /home/cassandra/.b 0.2s
> => ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
>
> 0.5s
> --
> > [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
> #100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
> python_spec='python2.7'
> --
> ubuntu2004_j11.docker:128
>
> 126 | # included in the base image, the compiled objects are not updated
> by pip at run time, which can
> 127 | # cause errors if the tests rely on new driver functionality or
> bug fixes.
> 128 | >>> RUN virtualenv --python=python2.7 env2.7
> 129 | RUN chmod +x env2.7/bin/activate
> 130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
> CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
> --upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
>
> error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
> virtualenv --python=python2.7 env2.7" did not complete successfully: exit
> code: 1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18474) Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequest
[
https://issues.apache.org/jira/browse/CASSANDRA-18474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Meredith updated CASSANDRA-18474:
-
Test and Documentation Plan:
Change is the same for all branches. Just made a single PR here.
https://github.com/apache/cassandra/pull/2288
CI Results (pending):
||Branch||Source||Circle CI||Jenkins||
|cassandra-4.0|[branch|https://github.com/jonmeredith/cassandra/tree/commit_remote_branch/CASSANDRA-18474-cassandra-4.0-47323483-B211-4278-931D-4297C88B10D9]|[build|https://app.circleci.com/pipelines/github/jonmeredith/cassandra?branch=commit_remote_branch%2FCASSANDRA-18474-cassandra-4.0-47323483-B211-4278-931D-4297C88B10D9]|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/2434/]|
|cassandra-4.1|[branch|https://github.com/jonmeredith/cassandra/tree/commit_remote_branch/CASSANDRA-18474-cassandra-4.1-47323483-B211-4278-931D-4297C88B10D9]|[build|https://app.circleci.com/pipelines/github/jonmeredith/cassandra?branch=commit_remote_branch%2FCASSANDRA-18474-cassandra-4.1-47323483-B211-4278-931D-4297C88B10D9]|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/2435/]|
|trunk|[branch|https://github.com/jonmeredith/cassandra/tree/commit_remote_branch/CASSANDRA-18474-trunk-47323483-B211-4278-931D-4297C88B10D9]|[build|https://app.circleci.com/pipelines/github/jonmeredith/cassandra?branch=commit_remote_branch%2FCASSANDRA-18474-trunk-47323483-B211-4278-931D-4297C88B10D9]|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/2436/]|
Status: Patch Available (was: Open)
> Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequest
> ---
>
> Key: CASSANDRA-18474
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18474
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Repair, Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{SyncRequest}} message assumes the initiator/src/dst are all the IP
> address
> type as an optimization when calculating serialized size. It needs to handle
> mixed address families.
> {code}
> /1.2.3.4:7000->/[::1]:7000-SMALL_MESSAGES-f83ce0bc dropping message of type
> SYNC_REQ due to error"
> {code}
> {Exception}
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 158, actual size at least 170, for verb SYNC_REQ
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18474) Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequest
[
https://issues.apache.org/jira/browse/CASSANDRA-18474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Meredith updated CASSANDRA-18474:
-
Summary: Incremental repairs fail on mixed IPv4/v6 addresses serializing
SyncRequest (was: Incremental repairs fail on mixed IPv4/v6 addresses
serializing SyncRequest)
> Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequest
> ---
>
> Key: CASSANDRA-18474
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18474
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Repair, Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0
>
>
> The {{SyncRequest}} message assumes the initiator/src/dst are all the IP
> address
> type as an optimization when calculating serialized size. It needs to handle
> mixed address families.
> {code}
> /1.2.3.4:7000->/[::1]:7000-SMALL_MESSAGES-f83ce0bc dropping message of type
> SYNC_REQ due to error"
> {code}
> {Exception}
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 158, actual size at least 170, for verb SYNC_REQ
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18474) Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequest
[
https://issues.apache.org/jira/browse/CASSANDRA-18474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Meredith updated CASSANDRA-18474:
-
Summary: Incremental repairs fail on mixed IPv4/v6 addresses serializing
SyncRequest (was: Incremental repairs fail on mixed IPv4/v6 addresses
serializing SyncRequst)
> Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequest
>
>
> Key: CASSANDRA-18474
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18474
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Repair, Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0
>
>
> The {{SyncRequest}} message assumes the initiator/src/dst are all the IP
> address
> type as an optimization when calculating serialized size. It needs to handle
> mixed address families.
> {code}
> /1.2.3.4:7000->/[::1]:7000-SMALL_MESSAGES-f83ce0bc dropping message of type
> SYNC_REQ due to error"
> {code}
> {Exception}
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 158, actual size at least 170, for verb SYNC_REQ
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18474) Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequst
[
https://issues.apache.org/jira/browse/CASSANDRA-18474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Meredith updated CASSANDRA-18474:
-
Change Category: Operability
Complexity: Normal
Component/s: Consistency/Repair
Messaging/Internode
Fix Version/s: 4.0.x
4.1.x
5.0
Status: Open (was: Triage Needed)
> Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequst
> ---
>
> Key: CASSANDRA-18474
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18474
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Repair, Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0
>
>
> The {{SyncRequest}} message assumes the initiator/src/dst are all the IP
> address
> type as an optimization when calculating serialized size. It needs to handle
> mixed address families.
> {code}
> /1.2.3.4:7000->/[::1]:7000-SMALL_MESSAGES-f83ce0bc dropping message of type
> SYNC_REQ due to error"
> {code}
> {Exception}
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
> size; expected 158, actual size at least 170, for verb SYNC_REQ
> at
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
> at
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18474) Incremental repairs fail on mixed IPv4/v6 addresses serializing SyncRequst
Jon Meredith created CASSANDRA-18474:
Summary: Incremental repairs fail on mixed IPv4/v6 addresses
serializing SyncRequst
Key: CASSANDRA-18474
URL: https://issues.apache.org/jira/browse/CASSANDRA-18474
Project: Cassandra
Issue Type: Improvement
Reporter: Jon Meredith
Assignee: Jon Meredith
The {{SyncRequest}} message assumes the initiator/src/dst are all the IP address
type as an optimization when calculating serialized size. It needs to handle
mixed address families.
{code}
/1.2.3.4:7000->/[::1]:7000-SMALL_MESSAGES-f83ce0bc dropping message of type
SYNC_REQ due to error"
{code}
{Exception}
{code}
org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized
size; expected 158, actual size at least 170, for verb SYNC_REQ
at
org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
at
org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
at
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
at
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18472) Docker images can no longer be built due to python2.7
[
https://issues.apache.org/jira/browse/CASSANDRA-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18472:
-
Status: Open (was: Patch Available)
> Docker images can no longer be built due to python2.7
> -
>
> Key: CASSANDRA-18472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
>
> {noformat}
> => [linux/amd64 35/56] WORKDIR /home/cassandra
>
> 0.1s
> => [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
> /home/cassandra/.bashrc && echo 'export
> JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
> /home/cassandra/.b 0.2s
> => ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
>
> 0.5s
> --
> > [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
> #100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
> python_spec='python2.7'
> --
> ubuntu2004_j11.docker:128
>
> 126 | # included in the base image, the compiled objects are not updated
> by pip at run time, which can
> 127 | # cause errors if the tests rely on new driver functionality or
> bug fixes.
> 128 | >>> RUN virtualenv --python=python2.7 env2.7
> 129 | RUN chmod +x env2.7/bin/activate
> 130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
> CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
> --upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
>
> error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
> virtualenv --python=python2.7 env2.7" did not complete successfully: exit
> code: 1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18472) Docker images can no longer be built due to python2.7
[
https://issues.apache.org/jira/browse/CASSANDRA-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715168#comment-17715168
]
Brandon Williams commented on CASSANDRA-18472:
--
Yes, 3.0 and 3.11 still need python2 for cqlsh. But the dtests don't, and this
is getting [those
dependencies|https://github.com/apache/cassandra-builds/blob/trunk/docker/testing/ubuntu2004_j11.docker#L74]
from the dtests. I guess I will see about making it only get what is needed,
then do an image upload to my account and run the test gamut to make sure it
works.
> Docker images can no longer be built due to python2.7
> -
>
> Key: CASSANDRA-18472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
>
> {noformat}
> => [linux/amd64 35/56] WORKDIR /home/cassandra
>
> 0.1s
> => [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
> /home/cassandra/.bashrc && echo 'export
> JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
> /home/cassandra/.b 0.2s
> => ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
>
> 0.5s
> --
> > [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
> #100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
> python_spec='python2.7'
> --
> ubuntu2004_j11.docker:128
>
> 126 | # included in the base image, the compiled objects are not updated
> by pip at run time, which can
> 127 | # cause errors if the tests rely on new driver functionality or
> bug fixes.
> 128 | >>> RUN virtualenv --python=python2.7 env2.7
> 129 | RUN chmod +x env2.7/bin/activate
> 130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
> CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
> --upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
>
> error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
> virtualenv --python=python2.7 env2.7" did not complete successfully: exit
> code: 1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18472) Docker images can no longer be built due to python2.7
[
https://issues.apache.org/jira/browse/CASSANDRA-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715167#comment-17715167
]
Michael Semb Wever commented on CASSANDRA-18472:
do older cassandra branches need 2.7 in any part of their building/testing?
(ant jar/artifacts, cqlshlib, ?)
> Docker images can no longer be built due to python2.7
> -
>
> Key: CASSANDRA-18472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
>
> {noformat}
> => [linux/amd64 35/56] WORKDIR /home/cassandra
>
> 0.1s
> => [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
> /home/cassandra/.bashrc && echo 'export
> JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
> /home/cassandra/.b 0.2s
> => ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
>
> 0.5s
> --
> > [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
> #100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
> python_spec='python2.7'
> --
> ubuntu2004_j11.docker:128
>
> 126 | # included in the base image, the compiled objects are not updated
> by pip at run time, which can
> 127 | # cause errors if the tests rely on new driver functionality or
> bug fixes.
> 128 | >>> RUN virtualenv --python=python2.7 env2.7
> 129 | RUN chmod +x env2.7/bin/activate
> 130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
> CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
> --upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
>
> error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
> virtualenv --python=python2.7 env2.7" did not complete successfully: exit
> code: 1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18472) Docker images can no longer be built due to python2.7
[
https://issues.apache.org/jira/browse/CASSANDRA-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715133#comment-17715133
]
Brandon Williams commented on CASSANDRA-18472:
--
Now we have the problem of 2.7 being unable to find a pytest version high
enough to meet the cassandra-dtest requirement:
{noformat}
#101 11.45 ERROR: Could not find a version that satisfies the requirement
pytest>=6.5.0 (from -r /opt/requirements.txt (line 18)) (from versions: 2.0.0,
2.0.1, 2.0.2, 2.0.3, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.2.0, 2.2.1, 2.2.2, 2.2.3,
2.2.4, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.4.0, 2.4.1, 2.4.2, 2.5.0,
2.5.1, 2.5.2, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.7.0, 2.7.1, 2.7.2, 2.7.3,
2.8.0, 2.8.1, 2.8.2, 2.8.3, 2.8.4, 2.8.5, 2.8.6, 2.8.7, 2.9.0, 2.9.1, 2.9.2,
3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.0.4, 3.0.5, 3.0.6, 3.0.7, 3.1.0, 3.1.1, 3.1.2,
3.1.3, 3.2.0, 3.2.1, 3.2.2, 3.2.3, 3.2.4, 3.2.5, 3.3.0, 3.3.1, 3.3.2, 3.4.0,
3.4.1, 3.4.2, 3.5.0, 3.5.1, 3.6.0, 3.6.1, 3.6.2, 3.6.3, 3.6.4, 3.7.0, 3.7.1,
3.7.2, 3.7.3, 3.7.4, 3.8.0, 3.8.1, 3.8.2, 3.9.1, 3.9.2, 3.9.3, 3.10.0, 3.10.1,
4.0.0, 4.0.1, 4.0.2, 4.1.0, 4.1.1, 4.2.0, 4.2.1, 4.3.0, 4.3.1, 4.4.0, 4.4.1,
4.4.2, 4.5.0, 4.6.0, 4.6.1, 4.6.2, 4.6.3, 4.6.4, 4.6.5, 4.6.6, 4.6.7, 4.6.8,
4.6.9, 4.6.10, 4.6.11)
#101 11.45 ERROR: No matching distribution found for pytest>=6.5.0 (from -r
/opt/requirements.txt (line 18))
{noformat}
I bumped this in CASSANDRA-18121 when adding 3.11 support, and there probably
is not a version higher than 4.6.11 that will work with python2. I don't think
there should be any reason python2 needs to track the dtest requirements
though, since we aren't running python2 there. I'm actually not sure what we
need python2.7 for here, [~mck] ?
> Docker images can no longer be built due to python2.7
> -
>
> Key: CASSANDRA-18472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
>
> {noformat}
> => [linux/amd64 35/56] WORKDIR /home/cassandra
>
> 0.1s
> => [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
> /home/cassandra/.bashrc && echo 'export
> JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
> /home/cassandra/.b 0.2s
> => ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
>
> 0.5s
> --
> > [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
> #100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
> python_spec='python2.7'
> --
> ubuntu2004_j11.docker:128
>
> 126 | # included in the base image, the compiled objects are not updated
> by pip at run time, which can
> 127 | # cause errors if the tests rely on new driver functionality or
> bug fixes.
> 128 | >>> RUN virtualenv --python=python2.7 env2.7
> 129 | RUN chmod +x env2.7/bin/activate
> 130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
> CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
> --upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
>
> error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
> virtualenv --python=python2.7 env2.7" did not complete successfully: exit
> code: 1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18105) TRUNCATED data come back after a restart or upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715126#comment-17715126
]
Caleb Rackliffe commented on CASSANDRA-18105:
-
LGTM, w/ a couple small nits left inline.
Just let me know when the other branches are up...
> TRUNCATED data come back after a restart or upgrade
> ---
>
> Key: CASSANDRA-18105
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18105
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Ke Han
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When we use the TRUNCATE command to delete all data in the table, the deleted
> data come back after a node restart or upgrade. This problem happens at the
> latest releases (2.2.19, 3.0.28, or 4.0.7)
> h1. Steps to reproduce
> h2. To reproduce it at release (3.0.28 or 4.0.7)
> Start up a single Cassandra node. Using the default configuration and execute
> the following cqlsh commands.
> {code:java}
> CREATE KEYSPACE IF NOT EXISTS ks WITH REPLICATION = { 'class' :
> 'SimpleStrategy', 'replication_factor' : 1 };
> CREATE TABLE ks.tb (c3 TEXT,c4 TEXT,c2 INT,c1 TEXT, PRIMARY KEY (c1, c2, c3
> ));
> INSERT INTO ks.tb (c3, c1, c2) VALUES ('val1','val2',1);
> CREATE INDEX IF NOT EXISTS tb ON ks.tb ( c3);
> TRUNCATE TABLE ks.tb;
> DROP INDEX IF EXISTS ks.tb; {code}
> Execute a read command
> {code:java}
> cqlsh> SELECT c2 FROM ks.tb;
> c2
>
> (0 rows) {code}
> Then, we flush the node and kill the Cassandra daemon by
> {code:java}
> bin/nodetool flush
> pgrep -f cassandra | xargs kill -9 {code}
> We restart the node. When the node has started, perform the same read, and
> the deleted data comes back again.
> {code:java}
> cqlsh> SELECT c2 FROM ks.tb;
> c2
>
> 1
> (1 rows) {code}
> h2. To reproduce it at release (2.2.19)
> We don't need to kill the Cassandra daemon. Use bin/nodetool stopdaemon is
> enough. The other steps are the same as reproducing it at 4.0.7 or 3.0.28.
> {code:java}
> bin/nodetool -h :::127.0.0.1 flush
> bin/nodetool -h :::127.0.0.1 stopdaemon{code}
>
> I have put the full log to reproduce it for release 4.0.7 and 2.2.19 in the
> comments.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18472) Docker images can no longer be built due to python2.7
[
https://issues.apache.org/jira/browse/CASSANDRA-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715125#comment-17715125
]
Ekaterina Dimitrova commented on CASSANDRA-18472:
-
Great, as you already built an image to test this, can you, please, push it to
docker hub for testing to ensure there are no other surprises?
Thanks in advance!
> Docker images can no longer be built due to python2.7
> -
>
> Key: CASSANDRA-18472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
>
> {noformat}
> => [linux/amd64 35/56] WORKDIR /home/cassandra
>
> 0.1s
> => [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
> /home/cassandra/.bashrc && echo 'export
> JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
> /home/cassandra/.b 0.2s
> => ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
>
> 0.5s
> --
> > [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
> #100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
> python_spec='python2.7'
> --
> ubuntu2004_j11.docker:128
>
> 126 | # included in the base image, the compiled objects are not updated
> by pip at run time, which can
> 127 | # cause errors if the tests rely on new driver functionality or
> bug fixes.
> 128 | >>> RUN virtualenv --python=python2.7 env2.7
> 129 | RUN chmod +x env2.7/bin/activate
> 130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
> CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
> --upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
>
> error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
> virtualenv --python=python2.7 env2.7" did not complete successfully: exit
> code: 1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18443) Deadlock updating sstable metadata if disk boundaries need reloading
[
https://issues.apache.org/jira/browse/CASSANDRA-18443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Meredith updated CASSANDRA-18443:
-
Fix Version/s: 4.1.2
4.0.10
(was: 4.0.x)
(was: 4.1.x)
Source Control Link:
https://github.com/apache/cassandra/commit/cd9bed0aeadd94136a8a6c6ed284cc4684b0666c
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Deadlock updating sstable metadata if disk boundaries need reloading
>
>
> Key: CASSANDRA-18443
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18443
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction, Local/Memtable, Local/SSTable
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 5.0, 4.1.2, 4.0.10
>
>
> {{CompactionStrategyManager.handleNotification}} holds the read lock while
> processing notifications. When handling metadata changed notifications, an
> extra call is made to maybeReloadDiskBoundaries which tries to grab the write
> lock and deadlocks the thread.
> Partial stacktrace
> {code}
> at jdk.internal.misc.Unsafe.park(java.base@11.0.16/Native Method)
> - parking to wait for <0x0005cc78> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire
> at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.maybeReloadDiskBoundaries(CompactionStrategyManager.java:495)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.getCompactionStrategyFor(CompactionStrategyManager.java:343)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleMetadataChangedNotification(CompactionStrategyManager.java:796)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleNotification(CompactionStrategyManager.java:838)
> at
> org.apache.cassandra.db.lifecycle.Tracker.notifySSTableMetadataChanged(Tracker.java:482)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleNotification(CompactionStrategyManager.java:838)
> {code}
> Deadlocking with the read lock held blocks the SlabpoolCleaner while
> notifying ColumnFamilyStore so memtables are prevented from being flushed and
> recycled, causing any thread applying a mutation to the database (at least
> GossipStage and MutationStage) to be considered down by peers and/or back up
> with pending requests.
> All the cases investigated were during single sstable upleveling by
> {{org.apache.cassandra.db.compaction.SingleSSTableLCSTask}} added in
> CASSANDRA-12526.
> Other less critical work was also affected, JMX calls to get estimated
> remaining compaction tasks, the index summary manager redistributing
> summaries, the StatusLogger trying to log dropped messages, and the
> ValidationManager.
> Workaround is to reboot the affected host.
> The fix is to just remove the redundant disk boundary reload check on that
> path.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-4.0' into cassandra-4.1
This is an automated email from the ASF dual-hosted git repository. jonmeredith pushed a commit to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 4c13df58cb3c5bea102b3a026a60574763d4dafd Merge: 2208235ce9 cd9bed0aea Author: Jon Meredith AuthorDate: Fri Apr 21 10:47:59 2023 -0600 Merge branch 'cassandra-4.0' into cassandra-4.1 CHANGES.txt| 1 + .../org/apache/cassandra/db/compaction/CompactionStrategyManager.java | 3 +-- 2 files changed, 2 insertions(+), 2 deletions(-) diff --cc CHANGES.txt index 852e4f9c91,35a0fc16d6..1c9a38a6aa --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,13 -1,15 +1,14 @@@ -4.0.10 +4.1.2 + * Allow keystore and trustrore passwords to be nullable (CASSANDRA-18124) + * Return snapshots with dots in their name in nodetool listsnapshots (CASSANDRA-18371) + * Fix NPE when loading snapshots and data directory is one directory from root (CASSANDRA-18359) + * Do not submit hints when hinted_handoff_enabled=false (CASSANDRA-18304) + * Fix COPY ... TO STDOUT behavior in cqlsh (CASSANDRA-18353) + * Remove six and Py2SaferScanner merge cruft (CASSANDRA-18354) +Merged from 4.0: + * Deadlock updating sstable metadata if disk boundaries need reloading (CASSANDRA-18443) * Fix nested selection of reversed collections (CASSANDRA-17913) - -4.0.9 * Update zstd-jni library to version 1.5.5 (CASSANDRA-18429) - * Backport CASSANDRA-17205 to 4.0 branch - Remove self-reference in SSTableTidier (CASSANDRA-18332) - * Avoid loading the preferred IP for BulkLoader streaming (CASSANDRA-18370) - * Fix BufferPool incorrect memoryInUse when putUnusedPortion is used (CASSANDRA-18311) - * Improve memtable allocator accounting when updating AtomicBTreePartition (CASSANDRA-18125) - * Update zstd-jni to version 1.5.4-1 (CASSANDRA-18259) - * Split and order IDEA workspace template VM_PARAMETERS (CASSANDRA-18242) Merged from 3.11: * Fix sstable_count metric missing from tablestats json/yaml output (CASSANDRA-18448) * Suppress CVE-2022-45688 (CASSANDRA-18389) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-4.1' into trunk
This is an automated email from the ASF dual-hosted git repository. jonmeredith pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 8df072e104f6ae5391de12d58e9f973f08cb2c57 Merge: 976b8395f9 4c13df58cb Author: Jon Meredith AuthorDate: Fri Apr 21 10:57:29 2023 -0600 Merge branch 'cassandra-4.1' into trunk CHANGES.txt| 2 ++ .../org/apache/cassandra/db/compaction/CompactionStrategyManager.java | 3 +-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --cc CHANGES.txt index 2d57dd07a0,1c9a38a6aa..4e3192321a --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,133 -1,3 +1,135 @@@ +5.0 + * Upgrade commons-io to 2.11.0 (CASSANDRA-17364) + * Node Draining Should Abort All Current SSTables Imports (CASSANDRA-18373) + * Use snake case for the names of CQL native functions (CASSANDRA-18037) + * Use jdk-dependent checkstyle version to check the source code (CASSANDRA-18262) + * Provide summary of failed SessionInfo's in StreamResultFuture (CASSANDRA-17199) + * CEP-20: Dynamic Data Masking (CASSANDRA-17940) + * Add system_views.snapshots virtual table (CASSANDRA-18102) + * Update OpenHFT dependencies (chronicle-queue, chronicle-core, chronicle-bytes, chronicle-wire, chronicle-threads) (CASSANDRA-18049) + * Remove org.apache.cassandra.hadoop code (CASSANDRA-18323) + * Remove deprecated CQL functions dateOf and unixTimestampOf (CASSANDRA-18328) + * Remove DateTieredCompactionStrategy (CASSANDRA-18043) + * Add system_views.max_sstable_size and system_views.max_sstable_duration tables (CASSANDRA-18333) + * Extend implicit allow-filtering for virtual tables to clustering columns (CASSANDRA-18331) + * Upgrade maven-shade-plugin to 3.4.1 to fix shaded dtest JAR build (CASSANDRA-18136) + * Upgrade to Opcodes.ASM9 (CASSANDRA-17971) + * Add MaxSSTableSize and MaxSSTableDuration metrics and propagate them together with local read/write ratio to tablestats (CASSANDRA-18283) + * Add more logging around CompactionManager operations (CASSANDRA-18268) + * Reduce memory allocations of calls to ByteBufer.duplicate() made in org.apache.cassandra.transport.CBUtil#writeValue (CASSANDRA-18212) + * CEP-17: SSTable API (CASSANDRA-17056) + * Gossip stateMapOrdering does not have correct ordering when both EndpointState are in the bootstrapping set (CASSANDRA-18292) + * Snapshot only sstables containing mismatching ranges on preview repair mismatch (CASSANDRA-17561) + * More accurate skipping of sstables in read path (CASSANDRA-18134) + * Prepare for JDK17 experimental support (CASSANDRA-18179, CASSANDRA-18258) + * Remove Scripted UDFs internals; hooks to be added later in CASSANDRA-17281 (CASSANDRA-18252) + * Update JNA to 5.13.0 (CASSANDRA-18050) + * Make virtual tables decide if they implicitly enable ALLOW FILTERING (CASSANDRA-18238) + * Add row, tombstone, and sstable count to nodetool profileload (CASSANDRA-18022) + * Coordinator level metrics for read response and mutation row and column counts (CASSANDRA-18155) + * Add CQL functions for dynamic data masking (CASSANDRA-17941) + * Print friendly error when nodetool attempts to connect to uninitialized server (CASSANDRA-11537) + * Use G1GC by default, and update default G1GC settings (CASSANDRA-18027) + * SimpleSeedProvider can resolve multiple IP addresses per DNS record (CASSANDRA-14361) + * Remove mocking in InternalNodeProbe spying on StorageServiceMBean (CASSANDRA-18152) + * Add compaction_properties column to system.compaction_history table and nodetool compactionhistory command (CASSANDRA-18061) + * Remove ProtocolVersion entirely from the CollectionSerializer ecosystem (CASSANDRA-18114) + * Fix serialization error in new getsstables --show-levels option (CASSANDRA-18140) + * Use checked casts when reading vints as ints (CASSANDRA-18099) + * Add Mutation Serialization Caching (CASSANDRA-17998) + * Only reload compaction strategies if disk boundaries change (CASSANDRA-17874) + * CEP-10: Simulator Java11 Support (CASSANDRA-17178) + * Set the major compaction type correctly for compactionstats (CASSANDRA-18055) + * Print exception message without stacktrace when nodetool commands fail on probe.getOwnershipWithPort() (CASSANDRA-18079) + * Add option to print level in nodetool getsstables output (CASSANDRA-18023) + * Implement a guardrail for not having zero default_time_to_live on tables with TWCS (CASSANDRA-18042) + * Add CQL scalar functions for collection aggregation (CASSANDRA-18060) + * Make cassandra.replayList property for CommitLogReplayer possible to react on keyspaces only (CASSANDRA-18044) + * Add Mathematical functions (CASSANDRA-17221) + * Make incremental backup configurable per table (CASSANDRA-15402) + * Change shebangs of Python scripts to resolve Python 3 from env command (CASSANDRA-17832) + * Add reasons to guardrail messages and consider guardrails in the error message for
[cassandra] branch trunk updated (976b8395f9 -> 8df072e104)
This is an automated email from the ASF dual-hosted git repository. jonmeredith pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 976b8395f9 Merge branch 'cassandra-4.1' into trunk new cd9bed0aea Deadlock updating sstable metadata if disk boundaries need reloading new 4c13df58cb Merge branch 'cassandra-4.0' into cassandra-4.1 new 8df072e104 Merge branch 'cassandra-4.1' into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 2 ++ .../org/apache/cassandra/db/compaction/CompactionStrategyManager.java | 3 +-- 2 files changed, 3 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.1 updated (2208235ce9 -> 4c13df58cb)
This is an automated email from the ASF dual-hosted git repository. jonmeredith pushed a change to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 2208235ce9 Merge branch 'cassandra-4.0' into cassandra-4.1 new cd9bed0aea Deadlock updating sstable metadata if disk boundaries need reloading new 4c13df58cb Merge branch 'cassandra-4.0' into cassandra-4.1 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../org/apache/cassandra/db/compaction/CompactionStrategyManager.java | 3 +-- 2 files changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated: Deadlock updating sstable metadata if disk boundaries need reloading
This is an automated email from the ASF dual-hosted git repository.
jonmeredith pushed a commit to branch cassandra-4.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-4.0 by this push:
new cd9bed0aea Deadlock updating sstable metadata if disk boundaries need
reloading
cd9bed0aea is described below
commit cd9bed0aeadd94136a8a6c6ed284cc4684b0666c
Author: Jon Meredith
AuthorDate: Thu Apr 20 14:17:36 2023 -0600
Deadlock updating sstable metadata if disk boundaries need reloading
patch by Jon Meredith; reviewed by Marcus Eriksson for CASSANDRA-18443
---
CHANGES.txt| 1 +
.../org/apache/cassandra/db/compaction/CompactionStrategyManager.java | 3 +--
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index cf208d7eb9..35a0fc16d6 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0.10
+ * Deadlock updating sstable metadata if disk boundaries need reloading
(CASSANDRA-18443)
* Fix nested selection of reversed collections (CASSANDRA-17913)
4.0.9
diff --git
a/src/java/org/apache/cassandra/db/compaction/CompactionStrategyManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionStrategyManager.java
index deece30d45..99e2ce996f 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionStrategyManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionStrategyManager.java
@@ -724,8 +724,7 @@ public class CompactionStrategyManager implements
INotificationConsumer
*/
private void handleMetadataChangedNotification(SSTableReader sstable,
StatsMetadata oldMetadata)
{
-AbstractCompactionStrategy acs = getCompactionStrategyFor(sstable);
-acs.metadataChanged(oldMetadata, sstable);
+compactionStrategyFor(sstable).metadataChanged(oldMetadata, sstable);
}
/**
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18105) TRUNCATED data come back after a restart or upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-18105:
Reviewers: Caleb Rackliffe, maxwellguo, Caleb Rackliffe (was: maxwellguo)
Caleb Rackliffe, maxwellguo, Caleb Rackliffe (was: Caleb
Rackliffe, maxwellguo)
Status: Review In Progress (was: Patch Available)
> TRUNCATED data come back after a restart or upgrade
> ---
>
> Key: CASSANDRA-18105
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18105
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Ke Han
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When we use the TRUNCATE command to delete all data in the table, the deleted
> data come back after a node restart or upgrade. This problem happens at the
> latest releases (2.2.19, 3.0.28, or 4.0.7)
> h1. Steps to reproduce
> h2. To reproduce it at release (3.0.28 or 4.0.7)
> Start up a single Cassandra node. Using the default configuration and execute
> the following cqlsh commands.
> {code:java}
> CREATE KEYSPACE IF NOT EXISTS ks WITH REPLICATION = { 'class' :
> 'SimpleStrategy', 'replication_factor' : 1 };
> CREATE TABLE ks.tb (c3 TEXT,c4 TEXT,c2 INT,c1 TEXT, PRIMARY KEY (c1, c2, c3
> ));
> INSERT INTO ks.tb (c3, c1, c2) VALUES ('val1','val2',1);
> CREATE INDEX IF NOT EXISTS tb ON ks.tb ( c3);
> TRUNCATE TABLE ks.tb;
> DROP INDEX IF EXISTS ks.tb; {code}
> Execute a read command
> {code:java}
> cqlsh> SELECT c2 FROM ks.tb;
> c2
>
> (0 rows) {code}
> Then, we flush the node and kill the Cassandra daemon by
> {code:java}
> bin/nodetool flush
> pgrep -f cassandra | xargs kill -9 {code}
> We restart the node. When the node has started, perform the same read, and
> the deleted data comes back again.
> {code:java}
> cqlsh> SELECT c2 FROM ks.tb;
> c2
>
> 1
> (1 rows) {code}
> h2. To reproduce it at release (2.2.19)
> We don't need to kill the Cassandra daemon. Use bin/nodetool stopdaemon is
> enough. The other steps are the same as reproducing it at 4.0.7 or 3.0.28.
> {code:java}
> bin/nodetool -h :::127.0.0.1 flush
> bin/nodetool -h :::127.0.0.1 stopdaemon{code}
>
> I have put the full log to reproduce it for release 4.0.7 and 2.2.19 in the
> comments.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18443) Deadlock updating sstable metadata if disk boundaries need reloading
[
https://issues.apache.org/jira/browse/CASSANDRA-18443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715090#comment-17715090
]
Jon Meredith commented on CASSANDRA-18443:
--
Test failures are timeouts/unrelated.
> Deadlock updating sstable metadata if disk boundaries need reloading
>
>
> Key: CASSANDRA-18443
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18443
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction, Local/Memtable, Local/SSTable
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0
>
>
> {{CompactionStrategyManager.handleNotification}} holds the read lock while
> processing notifications. When handling metadata changed notifications, an
> extra call is made to maybeReloadDiskBoundaries which tries to grab the write
> lock and deadlocks the thread.
> Partial stacktrace
> {code}
> at jdk.internal.misc.Unsafe.park(java.base@11.0.16/Native Method)
> - parking to wait for <0x0005cc78> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire
> at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.maybeReloadDiskBoundaries(CompactionStrategyManager.java:495)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.getCompactionStrategyFor(CompactionStrategyManager.java:343)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleMetadataChangedNotification(CompactionStrategyManager.java:796)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleNotification(CompactionStrategyManager.java:838)
> at
> org.apache.cassandra.db.lifecycle.Tracker.notifySSTableMetadataChanged(Tracker.java:482)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleNotification(CompactionStrategyManager.java:838)
> {code}
> Deadlocking with the read lock held blocks the SlabpoolCleaner while
> notifying ColumnFamilyStore so memtables are prevented from being flushed and
> recycled, causing any thread applying a mutation to the database (at least
> GossipStage and MutationStage) to be considered down by peers and/or back up
> with pending requests.
> All the cases investigated were during single sstable upleveling by
> {{org.apache.cassandra.db.compaction.SingleSSTableLCSTask}} added in
> CASSANDRA-12526.
> Other less critical work was also affected, JMX calls to get estimated
> remaining compaction tasks, the index summary manager redistributing
> summaries, the StatusLogger trying to log dropped messages, and the
> ValidationManager.
> Workaround is to reboot the affected host.
> The fix is to just remove the redundant disk boundary reload check on that
> path.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18216) Allow sharding of the SAI in-memory index
[ https://issues.apache.org/jira/browse/CASSANDRA-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18216: Epic Link: CASSANDRA-18473 (was: CASSANDRA-16052) > Allow sharding of the SAI in-memory index > -- > > Key: CASSANDRA-18216 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18216 > Project: Cassandra > Issue Type: Improvement > Components: Feature/SAI >Reporter: Mike Adamson >Priority: Normal > Labels: SAI > > The Memtable implementation allows it to be split into a number of shards. > This reduces contention during writes. > The MemtableIndex in SAI should follow this practice. It currently does not > support sharding so all writes hit the same synchronized write block. The > in-memory index search can also use the sharding to reduce the number of > indexes that it searches based on the key range passed to the search. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18280) Investigate initial size of GrowableByteArrayDataOutput in RAMIndexOutput
[ https://issues.apache.org/jira/browse/CASSANDRA-18280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18280: Epic Link: CASSANDRA-18473 (was: CASSANDRA-16052) > Investigate initial size of GrowableByteArrayDataOutput in RAMIndexOutput > - > > Key: CASSANDRA-18280 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18280 > Project: Cassandra > Issue Type: Task > Components: Feature/SAI >Reporter: Mike Adamson >Priority: Normal > > The GrowableByteArrayDataOutput in RAMIndexOutput is currently initialized > with a size of 128 bytes. There is no explanation as to why this size was > chosen. > The GrowableByteArrayDataOutput does not lazily allocate memory but only ever > allocates enough for each write operation. This can lead to a lot of fresh > allocations and calls to System.arrayCopy. > Since RAMIndexOutput is used to build the on-disk postings in SAI it is > likely that the size of the in-memory array is going to grow considerably > more that 128 bytes. > We should investigate changing this initial value to something higher and > possibly changing the GrowableByteArrayDataOutput class to allocate in blocks > rather than write increments. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18167) Bypass row-awareness for small partitions
[ https://issues.apache.org/jira/browse/CASSANDRA-18167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18167: Epic Link: CASSANDRA-18473 (was: CASSANDRA-16052) > Bypass row-awareness for small partitions > - > > Key: CASSANDRA-18167 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18167 > Project: Cassandra > Issue Type: Improvement > Components: Feature/SAI >Reporter: Mike Adamson >Priority: Normal > Labels: SAI > > SAI supports row-awareness in that it indexes both the partition key and the > clustering key of a row. This improves query performance significantly for > wide partitions with many rows but it can impact performance for small > partitions where it could make sense to bypass row-awareness post-filter the > results (read the whole partition) or batch rows for a single partition. > However this is achieved it would be necessary for the index to have an idea > of the size of the partition being read and to be aware of whether reading > the whole partition is likely to improve read performance. > SAI is aware of partition sizes during indexing so one option would be feed > these sizes into a histogram in the index metadata and apply a set of rules > to this metadata to decide whether we should attempt any optimisation. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18166) Improve the code model around IndexContext
[ https://issues.apache.org/jira/browse/CASSANDRA-18166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18166: Epic Link: CASSANDRA-18473 (was: CASSANDRA-16052) > Improve the code model around IndexContext > -- > > Key: CASSANDRA-18166 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18166 > Project: Cassandra > Issue Type: Improvement > Components: Feature/SAI >Reporter: Mike Adamson >Priority: Normal > Labels: SAI > > We currently have a situation where we need to create an IndexContext that is > for a non-indexed column and therefore is never going to be used for indexing > or searching. This results in the IndexContext having to check for this at > points in the code with assertions. The reason for this that, even when the > column is non-indexed, we need to have information about the column for the > purpose of post-filtering. > It would make sense to split out the column / index information needed for > filtering from the indexing / searching requirements such that we could avoid > unnecessary assertions in the code. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18473) Storage Attached Indexes (Phase 2)
[ https://issues.apache.org/jira/browse/CASSANDRA-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18473: Description: At the completion of CASSANDRA-16052, we should be able to release the core capabilities of SAI in a stable, production-ready package. Once that begins to gain traction, we'll be able to make improvements and add features for the next major release. The major initial theme of this epic is likely to be performance, but it will likely expand to include features like basic text analysis, etc. (was: At the completion of CASSANDRA-16052, we should be able to release the core capabilities of SAI in a stable, production-ready package. Once that begins to gain traction, we'll be able to make improvements and add features for the next major release.) > Storage Attached Indexes (Phase 2) > -- > > Key: CASSANDRA-18473 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18473 > Project: Cassandra > Issue Type: Epic > Components: Feature/2i Index >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > > At the completion of CASSANDRA-16052, we should be able to release the core > capabilities of SAI in a stable, production-ready package. Once that begins > to gain traction, we'll be able to make improvements and add features for the > next major release. The major initial theme of this epic is likely to be > performance, but it will likely expand to include features like basic text > analysis, etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18165) Investigate removing PriorityQueue usage from KeyRangeConcatIterator
[
https://issues.apache.org/jira/browse/CASSANDRA-18165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-18165:
Epic Link: CASSANDRA-18473 (was: CASSANDRA-16052)
> Investigate removing PriorityQueue usage from KeyRangeConcatIterator
>
>
> Key: CASSANDRA-18165
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18165
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/SAI
>Reporter: Mike Adamson
>Priority: Normal
> Labels: SAI
>
> It has been identified during the review of CASSANDRA-18058 that the
> KeyRangeConcatIterator could potentially stop using a PriorityQueue to
> maintain it's active list of sorted KeyRangeIterators.
> The code suggested by [~maedhroz] is as follows:
> {code:java}
> private int i = 0;
> ...
> protected void performSkipTo(PrimaryKey primaryKey)
> {
> while (i < toRelease.size())
> {
> RangeIterator currentIterator = toRelease.get(i);
> if (currentIterator.getCurrent().compareTo(primaryKey) >= 0)
> break;
> if (currentIterator.getMaximum().compareTo(primaryKey) >= 0)
> {
> currentIterator.skipTo(primaryKey);
> break;
> }
> i++;
> }
> }
> ...
> protected PrimaryKey computeNext()
> {
> while (i < toRelease.size())
> {
> RangeIterator currentIterator = toRelease.get(i);
>
> if (currentIterator.hasNext())
> return currentIterator.next();
>
> i++;
> }
> return endOfData();
> }
> {code}
> It was decided that this change would need performance and correctness
> testing in it's own right would not be included in the original SAI CEP
> ticket.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18473) Storage Attached Indexes (Phase 2)
[ https://issues.apache.org/jira/browse/CASSANDRA-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18473: Change Category: Performance Complexity: Normal Component/s: Feature/2i Index Status: Open (was: Triage Needed) > Storage Attached Indexes (Phase 2) > -- > > Key: CASSANDRA-18473 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18473 > Project: Cassandra > Issue Type: Epic > Components: Feature/2i Index >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > > At the completion of CASSANDRA-16052, we should be able to release the core > capabilities of SAI in a stable, production-ready package. Once that begins > to gain traction, we'll be able to make improvements and add features for the > next major release. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18443) Deadlock updating sstable metadata if disk boundaries need reloading
[
https://issues.apache.org/jira/browse/CASSANDRA-18443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715088#comment-17715088
]
Jon Meredith commented on CASSANDRA-18443:
--
Starting commit
CI Results (pending):
||Branch||Source||Circle CI||Jenkins||
|cassandra-4.0|[branch|https://github.com/jonmeredith/cassandra/tree/commit_remote_branch/CASSANDRA-18443-cassandra-4.0-03ECF093-C946-410B-85D2-BF4C5F6ACFB4]|[build|https://app.circleci.com/pipelines/github/jonmeredith/cassandra?branch=commit_remote_branch%2FCASSANDRA-18443-cassandra-4.0-03ECF093-C946-410B-85D2-BF4C5F6ACFB4]|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/2431/]|
|cassandra-4.1|[branch|https://github.com/jonmeredith/cassandra/tree/commit_remote_branch/CASSANDRA-18443-cassandra-4.1-03ECF093-C946-410B-85D2-BF4C5F6ACFB4]|[build|https://app.circleci.com/pipelines/github/jonmeredith/cassandra?branch=commit_remote_branch%2FCASSANDRA-18443-cassandra-4.1-03ECF093-C946-410B-85D2-BF4C5F6ACFB4]|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/2432/]|
|trunk|[branch|https://github.com/jonmeredith/cassandra/tree/commit_remote_branch/CASSANDRA-18443-trunk-03ECF093-C946-410B-85D2-BF4C5F6ACFB4]|[build|https://app.circleci.com/pipelines/github/jonmeredith/cassandra?branch=commit_remote_branch%2FCASSANDRA-18443-trunk-03ECF093-C946-410B-85D2-BF4C5F6ACFB4]|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/2433/]|
> Deadlock updating sstable metadata if disk boundaries need reloading
>
>
> Key: CASSANDRA-18443
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18443
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction, Local/Memtable, Local/SSTable
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0
>
>
> {{CompactionStrategyManager.handleNotification}} holds the read lock while
> processing notifications. When handling metadata changed notifications, an
> extra call is made to maybeReloadDiskBoundaries which tries to grab the write
> lock and deadlocks the thread.
> Partial stacktrace
> {code}
> at jdk.internal.misc.Unsafe.park(java.base@11.0.16/Native Method)
> - parking to wait for <0x0005cc78> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire
> at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.maybeReloadDiskBoundaries(CompactionStrategyManager.java:495)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.getCompactionStrategyFor(CompactionStrategyManager.java:343)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleMetadataChangedNotification(CompactionStrategyManager.java:796)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleNotification(CompactionStrategyManager.java:838)
> at
> org.apache.cassandra.db.lifecycle.Tracker.notifySSTableMetadataChanged(Tracker.java:482)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleNotification(CompactionStrategyManager.java:838)
> {code}
> Deadlocking with the read lock held blocks the SlabpoolCleaner while
> notifying ColumnFamilyStore so memtables are prevented from being flushed and
> recycled, causing any thread applying a mutation to the database (at least
> GossipStage and MutationStage) to be considered down by peers and/or back up
> with pending requests.
> All the cases investigated were during single sstable upleveling by
> {{org.apache.cassandra.db.compaction.SingleSSTableLCSTask}} added in
> CASSANDRA-12526.
> Other less critical work was also affected, JMX calls to get estimated
> remaining compaction tasks, the index summary manager redistributing
> summaries, the StatusLogger trying to log dropped messages, and the
> ValidationManager.
> Workaround is to reboot the affected host.
> The fix is to just remove the redundant disk boundary reload check on that
> path.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18473) Storage Attached Indexes (Phase 2)
Caleb Rackliffe created CASSANDRA-18473: --- Summary: Storage Attached Indexes (Phase 2) Key: CASSANDRA-18473 URL: https://issues.apache.org/jira/browse/CASSANDRA-18473 Project: Cassandra Issue Type: Epic Reporter: Caleb Rackliffe Assignee: Caleb Rackliffe At the completion of CASSANDRA-16052, we should be able to release the core capabilities of SAI in a stable, production-ready package. Once that begins to gain traction, we'll be able to make improvements and add features for the next major release. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16052) CEP-7 Storage Attached Indexes (Phase 1)
[ https://issues.apache.org/jira/browse/CASSANDRA-16052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-16052: Summary: CEP-7 Storage Attached Indexes (Phase 1) (was: CEP-7 Storage Attached Indexes) > CEP-7 Storage Attached Indexes (Phase 1) > > > Key: CASSANDRA-16052 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16052 > Project: Cassandra > Issue Type: Epic > Components: Feature/2i Index, Feature/SAI >Reporter: Zhao Yang >Assignee: Caleb Rackliffe >Priority: Normal > Labels: SAI > Fix For: 5.x > > Time Spent: 50m > Remaining Estimate: 0h > > [CEP|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-7%3A+Storage+Attached+Index] > - A new index implementation, called Storage > Attached Index(SAI), based on the advancement made by SASI. > * disk usage by sharing of common data between multiple column indexes on > the same table and better compression of on-disk structures. > * numeric range query performance with modified KDTree and collection type > support. > * compaction performance and stability for larger data set. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18018) List command output not correct for super user, after grant command
[
https://issues.apache.org/jira/browse/CASSANDRA-18018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Chanturiay updated CASSANDRA-18018:
-
Test and Documentation Plan: (was: There is an automated test located at
{code:java}
/test/unit/org/apache/cassandra/cql3/statements/ListPermissionsTest.java
{code}
It covers the following scenarios:
* all permissions are listed for the superuser role.
* a single permission is listed for the superuser role.
* a number of permissions are listed for the superuser role.
* the role is removed superuser status and permissions are listed in accordance
with explicit grant.
Until pull request is approved, the source code for the test file is available
at https://github.com/apache/cassandra/pull/2121
)
> List command output not correct for super user, after grant command
> ---
>
> Key: CASSANDRA-18018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18018
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Authorization
>Reporter: Shailaja Koppu
>Priority: Normal
> Labels: lhf
> Fix For: 4.1.x
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Running local Cassandra with below config:
> {noformat}
> authenticator: PasswordAuthenticator
> authorizer: CassandraAuthorizer
> role_manager: CassandraRoleManager
> network_authorizer: CassandraNetworkAuthorizer{noformat}
> Created a super user and then ran *Grant select* command on a keyspace.
> {noformat}
> shaadmin1@cqlsh> CREATE USER 'shaadmin1c1' WITH PASSWORD 'shaadmin1c1'
> SUPERUSER;
> shaadmin1@cqlsh:system_auth> grant select on testk1.t1 to shaadmin1c1;
> shaadmin1@cqlsh:system_auth> alter role shaadmin1c1 with access to all
> datacenters;
> {noformat}
>
> After this, list permissions command showing only select permission for that
> role on the resource.
> {noformat}
> shaadmin1c1@cqlsh> list all permissions of shaadmin1c1;
> role | username | resource | permission
> +---
> shaadmin1c1 | shaadmin1c1 | | SELECT
> {noformat}
>
> Row in role_permissions table:
> {noformat}
> role | resource | permissions
> --
> shaadmin1c1 | data/testk1/t1 | {'SELECT'}{noformat}
> But insert command by that role on the resource is successful because role is
> a super user
> {noformat}
> shaadmin1c1@cqlsh> insert into testk1.t1 (c1, c2) values ('a', 1);
> shaadmin1c1@cqlsh> select * from testk1.t1 ;
> c1 | c2
> ---+---
> a | 1
> (1 rows)
> {noformat}
>
> The problem is, output of list permissions command, which indicates only
> select permission on the resource, is misleading. I think list command need
> to be fixed to show all permissions super user has on the resource. Also
> grant command for a super user can be either a no-op or throw error, because
> the role already have requested permissions.
>
> Documentation also misleading:
> {quote}True automatically grants AUTHORIZE, CREATE and DROP permission on ALL
> ROLES.
> Superusers can only manage roles by default. To manage other resources,
> {color:#ff}you must grant the permission set to that resource. **
> {color}For example, to allow access management for all keyspaces: {{{}GRANT
> ALL PERMISSIONS ON ALL KEYSPACES TO }}\{{{}{*}role_name{*}{}}}.
> {quote}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18018) List command output not correct for super user, after grant command
[
https://issues.apache.org/jira/browse/CASSANDRA-18018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Chanturiay reassigned CASSANDRA-18018:
Assignee: (was: Maxim Chanturiay)
> List command output not correct for super user, after grant command
> ---
>
> Key: CASSANDRA-18018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18018
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Authorization
>Reporter: Shailaja Koppu
>Priority: Normal
> Labels: lhf
> Fix For: 4.1.x
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Running local Cassandra with below config:
> {noformat}
> authenticator: PasswordAuthenticator
> authorizer: CassandraAuthorizer
> role_manager: CassandraRoleManager
> network_authorizer: CassandraNetworkAuthorizer{noformat}
> Created a super user and then ran *Grant select* command on a keyspace.
> {noformat}
> shaadmin1@cqlsh> CREATE USER 'shaadmin1c1' WITH PASSWORD 'shaadmin1c1'
> SUPERUSER;
> shaadmin1@cqlsh:system_auth> grant select on testk1.t1 to shaadmin1c1;
> shaadmin1@cqlsh:system_auth> alter role shaadmin1c1 with access to all
> datacenters;
> {noformat}
>
> After this, list permissions command showing only select permission for that
> role on the resource.
> {noformat}
> shaadmin1c1@cqlsh> list all permissions of shaadmin1c1;
> role | username | resource | permission
> +---
> shaadmin1c1 | shaadmin1c1 | | SELECT
> {noformat}
>
> Row in role_permissions table:
> {noformat}
> role | resource | permissions
> --
> shaadmin1c1 | data/testk1/t1 | {'SELECT'}{noformat}
> But insert command by that role on the resource is successful because role is
> a super user
> {noformat}
> shaadmin1c1@cqlsh> insert into testk1.t1 (c1, c2) values ('a', 1);
> shaadmin1c1@cqlsh> select * from testk1.t1 ;
> c1 | c2
> ---+---
> a | 1
> (1 rows)
> {noformat}
>
> The problem is, output of list permissions command, which indicates only
> select permission on the resource, is misleading. I think list command need
> to be fixed to show all permissions super user has on the resource. Also
> grant command for a super user can be either a no-op or throw error, because
> the role already have requested permissions.
>
> Documentation also misleading:
> {quote}True automatically grants AUTHORIZE, CREATE and DROP permission on ALL
> ROLES.
> Superusers can only manage roles by default. To manage other resources,
> {color:#ff}you must grant the permission set to that resource. **
> {color}For example, to allow access management for all keyspaces: {{{}GRANT
> ALL PERMISSIONS ON ALL KEYSPACES TO }}\{{{}{*}role_name{*}{}}}.
> {quote}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18018) List command output not correct for super user, after grant command
[
https://issues.apache.org/jira/browse/CASSANDRA-18018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715055#comment-17715055
]
Maxim Chanturiay commented on CASSANDRA-18018:
--
[~samt] I've throughly searched the web regarding definitions for "permission"
and "superuser privilege".
First of all, I did not fully grasp that they are not the same "tool" and each
one is intended for a different case.
I am sorry for wasting our time here :(.
I'll close my pull request as it doesn't make sense to couple between distinct
concepts ¯\_(ツ)_/¯
[~skoppu] I've looked over the way database team is managing superuser
privilege for roles at my place of work.
Our developers and customers (which can be looked as typical users) do not find
themselves interacting with superuser roles in any shape or form.
It is actually a security breach when such a role is gone outside of the
database team premises.
And database team members are fully aware of the superuser privilege
capabilities. They do not need a command to point that out - they know ahead
what role has it (even before connecting to the database).
> List command output not correct for super user, after grant command
> ---
>
> Key: CASSANDRA-18018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18018
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Authorization
>Reporter: Shailaja Koppu
>Assignee: Maxim Chanturiay
>Priority: Normal
> Labels: lhf
> Fix For: 4.1.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Running local Cassandra with below config:
> {noformat}
> authenticator: PasswordAuthenticator
> authorizer: CassandraAuthorizer
> role_manager: CassandraRoleManager
> network_authorizer: CassandraNetworkAuthorizer{noformat}
> Created a super user and then ran *Grant select* command on a keyspace.
> {noformat}
> shaadmin1@cqlsh> CREATE USER 'shaadmin1c1' WITH PASSWORD 'shaadmin1c1'
> SUPERUSER;
> shaadmin1@cqlsh:system_auth> grant select on testk1.t1 to shaadmin1c1;
> shaadmin1@cqlsh:system_auth> alter role shaadmin1c1 with access to all
> datacenters;
> {noformat}
>
> After this, list permissions command showing only select permission for that
> role on the resource.
> {noformat}
> shaadmin1c1@cqlsh> list all permissions of shaadmin1c1;
> role | username | resource | permission
> +---
> shaadmin1c1 | shaadmin1c1 | | SELECT
> {noformat}
>
> Row in role_permissions table:
> {noformat}
> role | resource | permissions
> --
> shaadmin1c1 | data/testk1/t1 | {'SELECT'}{noformat}
> But insert command by that role on the resource is successful because role is
> a super user
> {noformat}
> shaadmin1c1@cqlsh> insert into testk1.t1 (c1, c2) values ('a', 1);
> shaadmin1c1@cqlsh> select * from testk1.t1 ;
> c1 | c2
> ---+---
> a | 1
> (1 rows)
> {noformat}
>
> The problem is, output of list permissions command, which indicates only
> select permission on the resource, is misleading. I think list command need
> to be fixed to show all permissions super user has on the resource. Also
> grant command for a super user can be either a no-op or throw error, because
> the role already have requested permissions.
>
> Documentation also misleading:
> {quote}True automatically grants AUTHORIZE, CREATE and DROP permission on ALL
> ROLES.
> Superusers can only manage roles by default. To manage other resources,
> {color:#ff}you must grant the permission set to that resource. **
> {color}For example, to allow access management for all keyspaces: {{{}GRANT
> ALL PERMISSIONS ON ALL KEYSPACES TO }}\{{{}{*}role_name{*}{}}}.
> {quote}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18472) Docker images can no longer be built due to python2.7
[
https://issues.apache.org/jira/browse/CASSANDRA-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18472:
-
Test and Documentation Plan: we just go for it with this stuff.
Status: Patch Available (was: Open)
> Docker images can no longer be built due to python2.7
> -
>
> Key: CASSANDRA-18472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
>
> {noformat}
> => [linux/amd64 35/56] WORKDIR /home/cassandra
>
> 0.1s
> => [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
> /home/cassandra/.bashrc && echo 'export
> JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
> /home/cassandra/.b 0.2s
> => ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
>
> 0.5s
> --
> > [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
> #100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
> python_spec='python2.7'
> --
> ubuntu2004_j11.docker:128
>
> 126 | # included in the base image, the compiled objects are not updated
> by pip at run time, which can
> 127 | # cause errors if the tests rely on new driver functionality or
> bug fixes.
> 128 | >>> RUN virtualenv --python=python2.7 env2.7
> 129 | RUN chmod +x env2.7/bin/activate
> 130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
> CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
> --upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
>
> error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
> virtualenv --python=python2.7 env2.7" did not complete successfully: exit
> code: 1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18472) Docker images can no longer be built due to python2.7
[
https://issues.apache.org/jira/browse/CASSANDRA-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715048#comment-17715048
]
Brandon Williams commented on CASSANDRA-18472:
--
Indeed, creating the venv by forcing 2.7 to be the virtualenv backend solves
it. Patch
[here|https://github.com/driftx/cassandra-builds/tree/CASSANDRA-18472] that
does that.
> Docker images can no longer be built due to python2.7
> -
>
> Key: CASSANDRA-18472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
>
> {noformat}
> => [linux/amd64 35/56] WORKDIR /home/cassandra
>
> 0.1s
> => [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
> /home/cassandra/.bashrc && echo 'export
> JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
> /home/cassandra/.b 0.2s
> => ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
>
> 0.5s
> --
> > [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
> #100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
> python_spec='python2.7'
> --
> ubuntu2004_j11.docker:128
>
> 126 | # included in the base image, the compiled objects are not updated
> by pip at run time, which can
> 127 | # cause errors if the tests rely on new driver functionality or
> bug fixes.
> 128 | >>> RUN virtualenv --python=python2.7 env2.7
> 129 | RUN chmod +x env2.7/bin/activate
> 130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
> CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
> --upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
>
> error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
> virtualenv --python=python2.7 env2.7" did not complete successfully: exit
> code: 1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18472) Docker images can no longer be built due to python2.7
[
https://issues.apache.org/jira/browse/CASSANDRA-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715045#comment-17715045
]
Brandon Williams commented on CASSANDRA-18472:
--
I think adding debugging to venv has revealed something:
{noformat}
#99 0.208 80 failed to query /usr/bin/python2.7 with code 1 err: ' File
"/usr/local/lib/python3.11/dist-packages/virtualenv/discovery/py_info.py", line
152\nos.path.join(base_dir, exe) for exe in (f"python{major}",
f"python{major}.{minor}")\n
^\nSyntaxError: invalid syntax\n' [INFO cached_py_info:35]
{noformat}
So the problem seems to be using python 3.11 to discover python2.7 (which is
indeed at that location.)
> Docker images can no longer be built due to python2.7
> -
>
> Key: CASSANDRA-18472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
>
> {noformat}
> => [linux/amd64 35/56] WORKDIR /home/cassandra
>
> 0.1s
> => [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
> /home/cassandra/.bashrc && echo 'export
> JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
> /home/cassandra/.b 0.2s
> => ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
>
> 0.5s
> --
> > [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
> #100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
> python_spec='python2.7'
> --
> ubuntu2004_j11.docker:128
>
> 126 | # included in the base image, the compiled objects are not updated
> by pip at run time, which can
> 127 | # cause errors if the tests rely on new driver functionality or
> bug fixes.
> 128 | >>> RUN virtualenv --python=python2.7 env2.7
> 129 | RUN chmod +x env2.7/bin/activate
> 130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
> CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
> --upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
>
> error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
> virtualenv --python=python2.7 env2.7" did not complete successfully: exit
> code: 1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18401) Investigate preloading ccm repositories in the docker image
[ https://issues.apache.org/jira/browse/CASSANDRA-18401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715029#comment-17715029 ] Ekaterina Dimitrova edited comment on CASSANDRA-18401 at 4/21/23 2:14 PM: -- For the record - CASSANDRA-18472 wha created as we see issues with Python 2.7. Blocking this ticket on that one was (Author: e.dimitrova): For the record - CASSANDRA-18401 wha created as we see issues with Python 2.7. Blocking this ticket on that one > Investigate preloading ccm repositories in the docker image > --- > > Key: CASSANDRA-18401 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18401 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 5.x > > > In CASSANDRA-18391 it was discovered that to skip some upgrade tests, the ccm > repository first needed to be populated with older versions. While that case > was solved, it may still be beneficial to preload the ccm repositories in the > docker image so they don't need to be fetched at all. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18401) Investigate preloading ccm repositories in the docker image
[ https://issues.apache.org/jira/browse/CASSANDRA-18401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715029#comment-17715029 ] Ekaterina Dimitrova edited comment on CASSANDRA-18401 at 4/21/23 2:14 PM: -- For the record - CASSANDRA-18472 was created as we see issues with Python 2.7. Blocking this ticket on that one was (Author: e.dimitrova): For the record - CASSANDRA-18472 wha created as we see issues with Python 2.7. Blocking this ticket on that one > Investigate preloading ccm repositories in the docker image > --- > > Key: CASSANDRA-18401 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18401 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 5.x > > > In CASSANDRA-18391 it was discovered that to skip some upgrade tests, the ccm > repository first needed to be populated with older versions. While that case > was solved, it may still be beneficial to preload the ccm repositories in the > docker image so they don't need to be fetched at all. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18401) Investigate preloading ccm repositories in the docker image
[ https://issues.apache.org/jira/browse/CASSANDRA-18401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715029#comment-17715029 ] Ekaterina Dimitrova commented on CASSANDRA-18401: - For the record - CASSANDRA-18401 wha created as we see issues with Python 2.7. Blocking this ticket on that one > Investigate preloading ccm repositories in the docker image > --- > > Key: CASSANDRA-18401 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18401 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 5.x > > > In CASSANDRA-18391 it was discovered that to skip some upgrade tests, the ccm > repository first needed to be populated with older versions. While that case > was solved, it may still be beneficial to preload the ccm repositories in the > docker image so they don't need to be fetched at all. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18472) Docker images can no longer be built due to python2.7
[
https://issues.apache.org/jira/browse/CASSANDRA-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18472:
-
Fix Version/s: 3.0.x
3.11.x
4.0.x
4.1.x
5.x
> Docker images can no longer be built due to python2.7
> -
>
> Key: CASSANDRA-18472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
>
> {noformat}
> => [linux/amd64 35/56] WORKDIR /home/cassandra
>
> 0.1s
> => [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
> /home/cassandra/.bashrc && echo 'export
> JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
> /home/cassandra/.b 0.2s
> => ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
>
> 0.5s
> --
> > [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
> #100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
> python_spec='python2.7'
> --
> ubuntu2004_j11.docker:128
>
> 126 | # included in the base image, the compiled objects are not updated
> by pip at run time, which can
> 127 | # cause errors if the tests rely on new driver functionality or
> bug fixes.
> 128 | >>> RUN virtualenv --python=python2.7 env2.7
> 129 | RUN chmod +x env2.7/bin/activate
> 130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
> CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
> --upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
>
> error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
> virtualenv --python=python2.7 env2.7" did not complete successfully: exit
> code: 1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18472) Docker images can no longer be built due to python2.7
[
https://issues.apache.org/jira/browse/CASSANDRA-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18472:
-
Bug Category: Parent values: Degradation(12984)Level 1 values: Other
Exception(12998)
Complexity: Normal
Component/s: CI
Discovered By: User Report
Severity: Normal
Assignee: Brandon Williams
Status: Open (was: Triage Needed)
> Docker images can no longer be built due to python2.7
> -
>
> Key: CASSANDRA-18472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
>
> {noformat}
> => [linux/amd64 35/56] WORKDIR /home/cassandra
>
> 0.1s
> => [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
> /home/cassandra/.bashrc && echo 'export
> JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
> /home/cassandra/.b 0.2s
> => ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
>
> 0.5s
> --
> > [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
> #100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
> python_spec='python2.7'
> --
> ubuntu2004_j11.docker:128
>
> 126 | # included in the base image, the compiled objects are not updated
> by pip at run time, which can
> 127 | # cause errors if the tests rely on new driver functionality or
> bug fixes.
> 128 | >>> RUN virtualenv --python=python2.7 env2.7
> 129 | RUN chmod +x env2.7/bin/activate
> 130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
> CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
> --upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
>
> error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
> virtualenv --python=python2.7 env2.7" did not complete successfully: exit
> code: 1
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18472) Docker images can no longer be built due to python2.7
Brandon Williams created CASSANDRA-18472:
Summary: Docker images can no longer be built due to python2.7
Key: CASSANDRA-18472
URL: https://issues.apache.org/jira/browse/CASSANDRA-18472
Project: Cassandra
Issue Type: Bug
Reporter: Brandon Williams
{noformat}
=> [linux/amd64 35/56] WORKDIR /home/cassandra
0.1s
=> [linux/amd64 36/56] RUN echo 'export ANT_HOME=/usr/share/ant' >>
/home/cassandra/.bashrc && echo 'export
JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >>
/home/cassandra/.b 0.2s
=> ERROR [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7
0.5s
--
> [linux/amd64 37/56] RUN virtualenv --python=python2.7 env2.7:
#100 0.424 RuntimeError: failed to find interpreter for Builtin discover of
python_spec='python2.7'
--
ubuntu2004_j11.docker:128
126 | # included in the base image, the compiled objects are not updated
by pip at run time, which can
127 | # cause errors if the tests rely on new driver functionality or bug
fixes.
128 | >>> RUN virtualenv --python=python2.7 env2.7
129 | RUN chmod +x env2.7/bin/activate
130 | RUN /bin/bash -c "export CASS_DRIVER_NO_CYTHON=1
CASS_DRIVER_NO_EXTENSIONS=1 && source ~/env2.7/bin/activate && pip2 install
--upgrade pip && pip2 install -r /opt/requirements.txt && pip2 freeze --user"
error: failed to solve: rpc error: code = Unknown desc = process "/bin/sh -c
virtualenv --python=python2.7 env2.7" did not complete successfully: exit code:
1
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] (CASSANDRA-12937) Default setting (yaml) for SSTable compression
[ https://issues.apache.org/jira/browse/CASSANDRA-12937 ]
Claude Warren deleted comment on CASSANDRA-12937:
---
was (Author: claudenw):
Code rebased to trunk
> Default setting (yaml) for SSTable compression
> --
>
> Key: CASSANDRA-12937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12937
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Michael Semb Wever
>Assignee: Claude Warren
>Priority: Low
> Labels: AdventCalendar2021, lhf
> Fix For: 5.x
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> In many situations the choice of compression for sstables is more relevant to
> the disks attached than to the schema and data.
> This issue is to add to cassandra.yaml a default value for sstable
> compression that new tables will inherit (instead of the defaults found in
> {{CompressionParams.DEFAULT}}.
> Examples where this can be relevant are filesystems that do on-the-fly
> compression (btrfs, zfs) or specific disk configurations or even specific C*
> versions (see CASSANDRA-10995 ).
> +Additional information for newcomers+
> Some new fields need to be added to {{cassandra.yaml}} to allow specifying
> the field required for defining the default compression parameters. In
> {{DatabaseDescriptor}} a new {{CompressionParams}} field should be added for
> the default compression. This field should be initialized in
> {{DatabaseDescriptor.applySimpleConfig()}}. At the different places where
> {{CompressionParams.DEFAULT}} was used the code should call
> {{DatabaseDescriptor#getDefaultCompressionParams}} that should return some
> copy of configured {{CompressionParams}}.
> Some unit test using {{OverrideConfigurationLoader}} should be used to test
> that the table schema use the new default when a new table is created (see
> CreateTest for some example).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18217) Allow CQL queries on multiple indexes without ALLOW FILTERING
[ https://issues.apache.org/jira/browse/CASSANDRA-18217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña reassigned CASSANDRA-18217: - Assignee: Andres de la Peña > Allow CQL queries on multiple indexes without ALLOW FILTERING > - > > Key: CASSANDRA-18217 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18217 > Project: Cassandra > Issue Type: Improvement > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Andres de la Peña >Priority: Normal > Labels: SAI > > The Index Group index was added by > [CASSANDRA-16092|https://issues.apache.org/jira/browse/CASSANDRA-16092] to > allow indexes to be grouped and have more than one index expression in a > query. This did not make any changes to StatementRestrictions so CQL queries > using multiple index expression still need to add ALLOW FILTERING to the > query to be valid. > This restriction should be removed such that any number of index expressions > can be used on a CQL query without needing ALLOW FILTERING. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18217) Allow CQL queries on multiple indexes without ALLOW FILTERING
[ https://issues.apache.org/jira/browse/CASSANDRA-18217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-18217: -- Change Category: Semantic Complexity: Normal Status: Open (was: Triage Needed) > Allow CQL queries on multiple indexes without ALLOW FILTERING > - > > Key: CASSANDRA-18217 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18217 > Project: Cassandra > Issue Type: Improvement > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Andres de la Peña >Priority: Normal > Labels: SAI > > The Index Group index was added by > [CASSANDRA-16092|https://issues.apache.org/jira/browse/CASSANDRA-16092] to > allow indexes to be grouped and have more than one index expression in a > query. This did not make any changes to StatementRestrictions so CQL queries > using multiple index expression still need to add ALLOW FILTERING to the > query to be valid. > This restriction should be removed such that any number of index expressions > can be used on a CQL query without needing ALLOW FILTERING. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-12937) Default setting (yaml) for SSTable compression
[
https://issues.apache.org/jira/browse/CASSANDRA-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17714856#comment-17714856
]
Claude Warren edited comment on CASSANDRA-12937 at 4/21/23 1:57 PM:
[~smiklosovic], [~mck]
Please review [Github Pull Request
#2254|https://github.com/apache/cassandra/pull/2254] there are a number of
changes.
* switched to ParameterizedClass for yaml configuration
* ensured processing to support CQL compression parameters
* added extensive yaml file documentation.
* moved methods used only in testing from CompressionParams to a new
TestingCompressionParamsFactory class.
* updated tests to use TestingCompressionFactory.
* added extensive testing to ensure that all combinations of Map and
ParameterizedClass construct the same CompressionParams.
* added additional testing coverage for pre-existing methods
* unified error reporting so that the same error on different paths reports
with the same text.
was (Author: claudenw):
[~smiklosovic], [~mck]
Please review [Github Pull Request
#2254|https://github.com/apache/cassandra/pull/2254] there are a number of
changes.
* switched to ParameterizedClass for yaml configuration
* ensured processing to support CQL compression parameters
* added extensive yaml file documentation.
* moved methods used only in testing from CompressionParams to a new
TestingCompressionParamsFactory class.
* updated tests to use TestingCompressionFactory.
* added extensive testing to ensure that all combinations of Map and
ParameterixedClass construct the same CompressionParams.
* added additional testing coverage or pre-existing methods
* unified error reporting so that the same error on different paths reports
with the same text.
> Default setting (yaml) for SSTable compression
> --
>
> Key: CASSANDRA-12937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12937
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Michael Semb Wever
>Assignee: Claude Warren
>Priority: Low
> Labels: AdventCalendar2021, lhf
> Fix For: 5.x
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> In many situations the choice of compression for sstables is more relevant to
> the disks attached than to the schema and data.
> This issue is to add to cassandra.yaml a default value for sstable
> compression that new tables will inherit (instead of the defaults found in
> {{CompressionParams.DEFAULT}}.
> Examples where this can be relevant are filesystems that do on-the-fly
> compression (btrfs, zfs) or specific disk configurations or even specific C*
> versions (see CASSANDRA-10995 ).
> +Additional information for newcomers+
> Some new fields need to be added to {{cassandra.yaml}} to allow specifying
> the field required for defining the default compression parameters. In
> {{DatabaseDescriptor}} a new {{CompressionParams}} field should be added for
> the default compression. This field should be initialized in
> {{DatabaseDescriptor.applySimpleConfig()}}. At the different places where
> {{CompressionParams.DEFAULT}} was used the code should call
> {{DatabaseDescriptor#getDefaultCompressionParams}} that should return some
> copy of configured {{CompressionParams}}.
> Some unit test using {{OverrideConfigurationLoader}} should be used to test
> that the table schema use the new default when a new table is created (see
> CreateTest for some example).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18449) Integer(int), Long(long), Float(double) were deprecated in JDK9
[ https://issues.apache.org/jira/browse/CASSANDRA-18449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715020#comment-17715020 ] Brandon Williams commented on CASSANDRA-18449: -- Added Byte, Short, and Double [here|https://github.com/driftx/cassandra/commit/06ecaeddad6cb1802d36af0f13a8f1fc528dc486]. > Integer(int), Long(long), Float(double) were deprecated in JDK9 > --- > > Key: CASSANDRA-18449 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18449 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Brandon Williams >Priority: Normal > Fix For: 5.x > > > Now when we are moving with Cassandra 5.0 to 11+17, it is good to declutter > at least a bit our build log which contains 76 deprecation warnings. > Half of them are for deprecation of Integer(int), Long(long) and > Float(double). > Oracle advises to use factory methods which are likely to yield significantly > better space and time performance too. The factory methods are also marked > with > @IntrinsicCandidate > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16718) Changing listen_address with prefer_local may lead to issues
[ https://issues.apache.org/jira/browse/CASSANDRA-16718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715009#comment-17715009 ] Maciej Sokol commented on CASSANDRA-16718: -- You are right, it feels strange to break existing functionality in an older branch. [~brandon.williams] could you rebase and run CI when you get the time? Or are we waiting for something? > Changing listen_address with prefer_local may lead to issues > > > Key: CASSANDRA-16718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16718 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Jan Karlsson >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.11.x, 4.0.x, 4.1.x, 5.x > > > Many container based solution function by assigning new listen_addresses when > nodes are stopped. Changing the listen_address is usually as simple as > turning off the node and changing the yaml file. > However, if prefer_local is enabled, I observed that nodes were unable to > join the cluster and fail with 'Unable to gossip with any seeds'. > Trace shows that the changing node will try to communicate with the existing > node but the response is never received. I assume it is because the existing > node attempts to communicate with the local address during the shadow round. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18470) Average of "decimal" values rounds the average if all inputs are integers
[
https://issues.apache.org/jira/browse/CASSANDRA-18470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715001#comment-17715001
]
Branimir Lambov commented on CASSANDRA-18470:
-
{{BigDecimal}} behaves too much like an integer for {{avg = avg + (value - avg)
/ count}} to work. For this type it makes full sense to just calculate the sum
and divide by count.
However, there's a bigger problem (and a potential attack vector) for these
methods. If one manages to sneak e.g. {{1e-1}} in a field, it would
make sum and avg take forever.
The precision of {{decimal}} was dealt with in CASSANDRA-15232 and
CASSANDRA-17221. Looks like it's time we did this for the aggregate function as
well.
> Average of "decimal" values rounds the average if all inputs are integers
> -
>
> Key: CASSANDRA-18470
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18470
> Project: Cassandra
> Issue Type: Bug
>Reporter: Nadav Har'El
>Priority: Normal
>
> When running the AVG aggregator on "decimal" values, each value is an
> arbitrary-precision number which may be an integer or fractional, but it is
> expected that the average would be, in general, fractional. But it turns out
> that if all the values are integer *without* a ".0", the aggregator sums them
> up as integers and the final division returns an integer too instead of the
> fractional response expected from a "decimal" value.
> For example:
> # AVG of {{decimal}} values 1.0 and 2.0 returns 1.5, as expected.
> # AVG of 1.0 and 2 or 1 and 2.0 also return 1.5.
> # But AVG of 1 and 2 returns... 1. This is wrong. The user asked for the
> average to be a "decimal", not a "varint", so there is no reason why it
> should be rounded up to be an integer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18470) Average of "decimal" values rounds the average if all inputs are integers
[
https://issues.apache.org/jira/browse/CASSANDRA-18470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17714905#comment-17714905
]
Benedict Elliott Smith commented on CASSANDRA-18470:
Oof, that is a pretty serious bug IMO, and probably deserves its own ticket.
[~ifesdjeen], [~blambov], [~blerer]: this appears to have been introduced by
CASSANDRA-12417, would any of you like to have a look?
> Average of "decimal" values rounds the average if all inputs are integers
> -
>
> Key: CASSANDRA-18470
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18470
> Project: Cassandra
> Issue Type: Bug
>Reporter: Nadav Har'El
>Priority: Normal
>
> When running the AVG aggregator on "decimal" values, each value is an
> arbitrary-precision number which may be an integer or fractional, but it is
> expected that the average would be, in general, fractional. But it turns out
> that if all the values are integer *without* a ".0", the aggregator sums them
> up as integers and the final division returns an integer too instead of the
> fractional response expected from a "decimal" value.
> For example:
> # AVG of {{decimal}} values 1.0 and 2.0 returns 1.5, as expected.
> # AVG of 1.0 and 2 or 1 and 2.0 also return 1.5.
> # But AVG of 1 and 2 returns... 1. This is wrong. The user asked for the
> average to be a "decimal", not a "varint", so there is no reason why it
> should be rounded up to be an integer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12937) Default setting (yaml) for SSTable compression
[
https://issues.apache.org/jira/browse/CASSANDRA-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17714856#comment-17714856
]
Claude Warren commented on CASSANDRA-12937:
---
[~smiklosovic], [~mck]
Please review [Github Pull Request
#2254|https://github.com/apache/cassandra/pull/2254] there are a number of
changes.
* switched to ParameterizedClass for yaml configuration
* ensured processing to support CQL compression parameters
* added extensive yaml file documentation.
* moved methods used only in testing from CompressionParams to a new
TestingCompressionParamsFactory class.
* updated tests to use TestingCompressionFactory.
* added extensive testing to ensure that all combinations of Map and
ParameterixedClass construct the same CompressionParams.
* added additional testing coverage or pre-existing methods
* unified error reporting so that the same error on different paths reports
with the same text.
> Default setting (yaml) for SSTable compression
> --
>
> Key: CASSANDRA-12937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12937
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Michael Semb Wever
>Assignee: Claude Warren
>Priority: Low
> Labels: AdventCalendar2021, lhf
> Fix For: 5.x
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> In many situations the choice of compression for sstables is more relevant to
> the disks attached than to the schema and data.
> This issue is to add to cassandra.yaml a default value for sstable
> compression that new tables will inherit (instead of the defaults found in
> {{CompressionParams.DEFAULT}}.
> Examples where this can be relevant are filesystems that do on-the-fly
> compression (btrfs, zfs) or specific disk configurations or even specific C*
> versions (see CASSANDRA-10995 ).
> +Additional information for newcomers+
> Some new fields need to be added to {{cassandra.yaml}} to allow specifying
> the field required for defining the default compression parameters. In
> {{DatabaseDescriptor}} a new {{CompressionParams}} field should be added for
> the default compression. This field should be initialized in
> {{DatabaseDescriptor.applySimpleConfig()}}. At the different places where
> {{CompressionParams.DEFAULT}} was used the code should call
> {{DatabaseDescriptor#getDefaultCompressionParams}} that should return some
> copy of configured {{CompressionParams}}.
> Some unit test using {{OverrideConfigurationLoader}} should be used to test
> that the table schema use the new default when a new table is created (see
> CreateTest for some example).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12937) Default setting (yaml) for SSTable compression
[
https://issues.apache.org/jira/browse/CASSANDRA-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17714848#comment-17714848
]
Claude Warren commented on CASSANDRA-12937:
---
Code rebased to trunk
> Default setting (yaml) for SSTable compression
> --
>
> Key: CASSANDRA-12937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12937
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Michael Semb Wever
>Assignee: Claude Warren
>Priority: Low
> Labels: AdventCalendar2021, lhf
> Fix For: 5.x
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> In many situations the choice of compression for sstables is more relevant to
> the disks attached than to the schema and data.
> This issue is to add to cassandra.yaml a default value for sstable
> compression that new tables will inherit (instead of the defaults found in
> {{CompressionParams.DEFAULT}}.
> Examples where this can be relevant are filesystems that do on-the-fly
> compression (btrfs, zfs) or specific disk configurations or even specific C*
> versions (see CASSANDRA-10995 ).
> +Additional information for newcomers+
> Some new fields need to be added to {{cassandra.yaml}} to allow specifying
> the field required for defining the default compression parameters. In
> {{DatabaseDescriptor}} a new {{CompressionParams}} field should be added for
> the default compression. This field should be initialized in
> {{DatabaseDescriptor.applySimpleConfig()}}. At the different places where
> {{CompressionParams.DEFAULT}} was used the code should call
> {{DatabaseDescriptor#getDefaultCompressionParams}} that should return some
> copy of configured {{CompressionParams}}.
> Some unit test using {{OverrideConfigurationLoader}} should be used to test
> that the table schema use the new default when a new table is created (see
> CreateTest for some example).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
