Re: [PR] Updating README JIRA link to ASF JIRA [cassandra-java-driver]
michaelsembwever commented on code in PR #1921: URL: https://github.com/apache/cassandra-java-driver/pull/1921#discussion_r1535104581 ## README.md: ## @@ -80,7 +80,7 @@ See the [Cassandra error handling done right blog](https://www.datastax.com/blog * [FAQ] [API docs]: https://docs.datastax.com/en/drivers/java/4.17 -[JIRA]: https://datastax-oss.atlassian.net/browse/JAVA +[JIRA]: https://issues.apache.org/jira/projects/CASSANDRA/issues/ Review Comment: ```suggestion [JIRA]: https://issues.apache.org/jira/issues/?jql=project%20%3D%20CASSANDRA%20AND%20component%20%3D%20%22Client%2Fjava-driver%22%20ORDER%20BY%20key%20DESC ``` ? there's no way to assign the component when creating a new issue by url or template… -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829749#comment-17829749
]
Berenguer Blasi commented on CASSANDRA-19484:
-
The new code lgtm. We just need to make sure running it per branch now shows no
new CVEs. Do I understand correctly that is the case now (clean runs) now than
you 'ant clean' before compiling each branch? If so +1 from me.
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] Updating README JIRA link to ASF JIRA [cassandra-java-driver]
absurdfarce commented on code in PR #1921: URL: https://github.com/apache/cassandra-java-driver/pull/1921#discussion_r1535063321 ## README.md: ## @@ -80,7 +80,7 @@ See the [Cassandra error handling done right blog](https://www.datastax.com/blog * [FAQ] [API docs]: https://docs.datastax.com/en/drivers/java/4.17 -[JIRA]: https://datastax-oss.atlassian.net/browse/JAVA +[JIRA]: https://issues.apache.org/jira/projects/CASSANDRA/issues/ Review Comment: Mentioned in separate conversation by @ekaterinadimitrova2: we should mention the "Client/java-driver" component we've been using for Java driver tickets here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] Updating README JIRA link to ASF JIRA [cassandra-java-driver]
absurdfarce commented on code in PR #1921: URL: https://github.com/apache/cassandra-java-driver/pull/1921#discussion_r1535063553 ## README.md: ## @@ -80,7 +80,7 @@ See the [Cassandra error handling done right blog](https://www.datastax.com/blog * [FAQ] [API docs]: https://docs.datastax.com/en/drivers/java/4.17 -[JIRA]: https://datastax-oss.atlassian.net/browse/JAVA +[JIRA]: https://issues.apache.org/jira/projects/CASSANDRA/issues/ Review Comment: I'm kinda wondering if there isn't some way we could pre-fill in that component by customizing this link a bit, maybe with some query params or extra URL args... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18762) Repair triggers OOM with direct buffer memory
[
https://issues.apache.org/jira/browse/CASSANDRA-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829743#comment-17829743
]
Manish Khandelwal commented on CASSANDRA-18762:
---
I think reason for getting OOM here is related to same reasoning as mentioned
in https://issues.apache.org/jira/browse/CASSANDRA-19336. I applied the patch
for https://issues.apache.org/jira/browse/CASSANDRA-19336 and all full repairs
with -pr on keyspace were successful.
As without this patch in one repair we can see almost 240 sessions triggered (
vnode:256, 11*11 cluster), resulting in 240*6 merkle tree requests for one
table. For a keywpace with 3 tables this number was astonishing 240*6*3
resulting in direct byte buffer within a minute of running.
After applying the patch repairs ran without issue also no memory pressue.
> Repair triggers OOM with direct buffer memory
> -
>
> Key: CASSANDRA-18762
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18762
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair
>Reporter: Brad Schoening
>Priority: Normal
> Labels: OutOfMemoryError
> Attachments: Cluster-dm-metrics-1.PNG,
> image-2023-12-06-15-28-05-459.png, image-2023-12-06-15-29-31-491.png,
> image-2023-12-06-15-58-55-007.png
>
>
> We are seeing repeated failures of nodes with 16GB of heap on a VM with 32GB
> of physical RAM due to direct memory. This seems to be related to
> CASSANDRA-15202 which moved Merkel trees off-heap in 4.0. Using Cassandra
> 4.0.6 with Java 11.
> {noformat}
> 2023-08-09 04:30:57,470 [INFO ] [AntiEntropyStage:1] cluster_id=101
> ip_address=169.0.0.1 RepairSession.java:202 - [repair
> #5e55a3b0-366d-11ee-a644-d91df26add5e] Received merkle tree for table_a from
> /169.102.200.241:7000
> 2023-08-09 04:30:57,567 [INFO ] [AntiEntropyStage:1] cluster_id=101
> ip_address=169.0.0.1 RepairSession.java:202 - [repair
> #5e0d2900-366d-11ee-a644-d91df26add5e] Received merkle tree for table_b from
> /169.93.192.29:7000
> 2023-08-09 04:30:57,568 [INFO ] [AntiEntropyStage:1] cluster_id=101
> ip_address=169.0.0.1 RepairSession.java:202 - [repair
> #5e1dcad0-366d-11ee-a644-d91df26add5e] Received merkle tree for table_c from
> /169.104.171.134:7000
> 2023-08-09 04:30:57,591 [INFO ] [AntiEntropyStage:1] cluster_id=101
> ip_address=169.0.0.1 RepairSession.java:202 - [repair
> #5e69a0e0-366d-11ee-a644-d91df26add5e] Received merkle tree for table_b from
> /169.79.232.67:7000
> 2023-08-09 04:30:57,876 [INFO ] [Service Thread] cluster_id=101
> ip_address=169.0.0.1 GCInspector.java:294 - G1 Old Generation GC in 282ms.
> Compressed Class Space: 8444560 -> 8372152; G1 Eden Space: 7809794048 -> 0;
> G1 Old Gen: 1453478400 -> 820942800; G1 Survivor Space: 419430400 -> 0;
> Metaspace: 80411136 -> 80176528
> 2023-08-09 04:30:58,387 [ERROR] [AntiEntropyStage:1] cluster_id=101
> ip_address=169.0.0.1 JVMStabilityInspector.java:102 - OutOfMemory error
> letting the JVM handle the error:
> java.lang.OutOfMemoryError: Direct buffer memory
> at java.base/java.nio.Bits.reserveMemory(Bits.java:175)
> at java.base/java.nio.DirectByteBuffer.(DirectByteBuffer.java:118)
> at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:318)
> at org.apache.cassandra.utils.MerkleTree.allocate(MerkleTree.java:742)
> at
> org.apache.cassandra.utils.MerkleTree.deserializeOffHeap(MerkleTree.java:780)
> at org.apache.cassandra.utils.MerkleTree.deserializeTree(MerkleTree.java:751)
> at org.apache.cassandra.utils.MerkleTree.deserialize(MerkleTree.java:720)
> at org.apache.cassandra.utils.MerkleTree.deserialize(MerkleTree.java:698)
> at
> org.apache.cassandra.utils.MerkleTrees$MerkleTreesSerializer.deserialize(MerkleTrees.java:416)
> at
> org.apache.cassandra.repair.messages.ValidationResponse$1.deserialize(ValidationResponse.java:100)
> at
> org.apache.cassandra.repair.messages.ValidationResponse$1.deserialize(ValidationResponse.java:84)
> at
> org.apache.cassandra.net.Message$Serializer.deserializePost40(Message.java:782)
> at org.apache.cassandra.net.Message$Serializer.deserialize(Message.java:642)
> at
> org.apache.cassandra.net.InboundMessageHandler$LargeMessage.deserialize(InboundMessageHandler.java:364)
> at
> org.apache.cassandra.net.InboundMessageHandler$LargeMessage.access$1100(InboundMessageHandler.java:317)
> at
> org.apache.cassandra.net.InboundMessageHandler$ProcessLargeMessage.provideMessage(InboundMessageHandler.java:504)
> at
> org.apache.cassandra.net.InboundMessageHandler$ProcessMessage.run(InboundMessageHandler.java:429)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolEx
[jira] [Updated] (CASSANDRA-19461) SAI does not index empty bytes even for types that allow empty bytes as a valid input
[
https://issues.apache.org/jira/browse/CASSANDRA-19461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-19461:
Status: Ready to Commit (was: Review In Progress)
Trunk CI results are similarly unremarkable. Moving to commit...
> SAI does not index empty bytes even for types that allow empty bytes as a
> valid input
> -
>
> Key: CASSANDRA-19461
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19461
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Attachments: ci_summary.html, result_details.tar.gz
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> This is easy to reproduce with a test that looks something like this:
> {noformat}
> @Test
> public void testEmptyString()
> {
> createTable("CREATE TABLE %s (k TEXT PRIMARY KEY, v text)");
> createIndex(String.format(CREATE_INDEX_TEMPLATE, 'v'));
> execute("INSERT INTO %s (k, v) VALUES ('0', '')");
> execute("INSERT INTO %s (k) VALUES ('1')");
>
> // flush(); < there is not always a memtable index involved, a fix
> will have to pay attention to this
> List rows = executeNet("SELECT * FROM %s WHERE v = ''").all();
> assertEquals(1, rows.size()); <— FAILS! No matches...
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13855) Implement Http Seed provider
[ https://issues.apache.org/jira/browse/CASSANDRA-13855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829709#comment-17829709 ] Stefan Miklosovic edited comment on CASSANDRA-13855 at 3/21/24 11:58 PM: - The problem I see is that if we e.g. talk about that EC2SeedProvider as in that link above, it uses some AWS libraries. If we wanted to ship it with Cassandra, we would have to depend on this library and this would be done for each such cloud. That opens a can of worms ... Are licences compatible? What version of that library are we going to use? Who is going to update it and when? Are these libraries even compatible between themselves when we put them on the class path? They might depend on various libraries themselves, are we going to ship all of that with the project too? The task here is to code up a "HTTP seed provider" -> HTTP meaning it would read some external data in a predefined format which would be parsed and it is responsibility of a user to provide such HTTP endpoint to read the metadata from. Cloud snitches are modelled like that too - they are just using HTTP to call cloud metadata services just in different ways. Would it be possible to not depend on cloud libraries but to code cloud seed providers in such a way that they would be still using HTTP under the hood? Secondly, the point is to return, from a seed provider, list of addresses which are supposed to be seeds. I just do not see how putting a snitch into a seed provider achieves this. EDIT: I think your goal was to not bring all libs into the project after all, as I read it for the second time ... But I just still dont see how it would be achieved. Please move it to a separate ticket, that would be great. was (Author: smiklosovic): The problem I see is that if we e.g. talk about that EC2SeedProvider as in that link above, it uses some AWS libraries. If we wanted to ship it with Cassandra, we would have to depend on this library and this would be done for each such cloud. That opens a can of worms ... Are licences compatible? What version of that library are we going to use? Who is going to update it and when? Are these libraries even compatible between themselves when we put them on the class path? They might depend on various libraries themselves, are we going to ship all of that with the project too? The task here is to code up a "HTTP seed provider" -> HTTP meaning it would read some external data in a predefined format which would be parsed and it is responsibility of a user to provide such HTTP endpoint to read the metadata from. Cloud snitches are modelled like that too - they are just using HTTP to call cloud metadata services just in different ways. Would it be possible to not depend on cloud libraries but to code cloud seed providers in such a way that they would be still using HTTP under the hood? Secondly, the point is to return, from a seed provider, list of addresses which are supposed to be seeds. I just do not see how putting a snitch into a seed provider achieves this. > Implement Http Seed provider > > > Key: CASSANDRA-13855 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13855 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Coordination, Legacy/Core >Reporter: Jon Haddad >Assignee: Claude Warren >Priority: Low > Labels: lhf > Fix For: 5.x > > Attachments: 0001-Add-URL-Seed-Provider-trunk.txt, signature.asc, > signature.asc, signature.asc > > Time Spent: 0.5h > Remaining Estimate: 0h > > Seems like including a dead simple seed provider that can fetch from a URL, 1 > line per seed, would be useful. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13855) Implement Http Seed provider
[ https://issues.apache.org/jira/browse/CASSANDRA-13855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829709#comment-17829709 ] Stefan Miklosovic edited comment on CASSANDRA-13855 at 3/21/24 11:54 PM: - The problem I see is that if we e.g. talk about that EC2SeedProvider as in that link above, it uses some AWS libraries. If we wanted to ship it with Cassandra, we would have to depend on this library and this would be done for each such cloud. That opens a can of worms ... Are licences compatible? What version of that library are we going to use? Who is going to update it and when? Are these libraries even compatible between themselves when we put them on the class path? They might depend on various libraries themselves, are we going to ship all of that with the project too? The task here is to code up a "HTTP seed provider" -> HTTP meaning it would read some external data in a predefined format which would be parsed and it is responsibility of a user to provide such HTTP endpoint to read the metadata from. Cloud snitches are modelled like that too - they are just using HTTP to call cloud metadata services just in different ways. Would it be possible to not depend on cloud libraries but to code cloud seed providers in such a way that they would be still using HTTP under the hood? Secondly, the point is to return, from a seed provider, list of addresses which are supposed to be seeds. I just do not see how putting a snitch into a seed provider achieves this. was (Author: smiklosovic): The problem I see is that if we e.g. talk about that EC2SeedProvider as in that link above, it uses some AWS libraries. If we wanted to ship it with Cassandra, we would have to depend on this library and this would be done for each such cloud. That opens a can of worms ... Are licences compatible? What version of that library are we going to use? Who is going to update it and when? Are these libraries even compatible between themselves when we put them on the class path? They might depend on various libraries themselves, are we going to ship all of that with the project too? The task here is to code up a "HTTP seed provider" -> HTTP meaning it would read some external data in a predefined format which would be parsed and it is responsibility of a user to provide such HTTP endpoint to read the metadata from. Cloud snitches are modelled like that too - they are just using HTTP to call cloud metadata services just in different ways. Would it be possible to not depend on cloud libraries but to code cloud snitches in such a way that they would be still using HTTP under the hood? Secondly, the point is to return, from a seed provider, list of addresses which are supposed to be seeds. I just do not see how putting a snitch into a seed provider achieves this. > Implement Http Seed provider > > > Key: CASSANDRA-13855 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13855 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Coordination, Legacy/Core >Reporter: Jon Haddad >Assignee: Claude Warren >Priority: Low > Labels: lhf > Fix For: 5.x > > Attachments: 0001-Add-URL-Seed-Provider-trunk.txt, signature.asc, > signature.asc, signature.asc > > Time Spent: 0.5h > Remaining Estimate: 0h > > Seems like including a dead simple seed provider that can fetch from a URL, 1 > line per seed, would be useful. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13855) Implement Http Seed provider
[ https://issues.apache.org/jira/browse/CASSANDRA-13855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829709#comment-17829709 ] Stefan Miklosovic edited comment on CASSANDRA-13855 at 3/21/24 11:53 PM: - The problem I see is that if we e.g. talk about that EC2SeedProvider as in that link above, it uses some AWS libraries. If we wanted to ship it with Cassandra, we would have to depend on this library and this would be done for each such cloud. That opens a can of worms ... Are licences compatible? What version of that library are we going to use? Who is going to update it and when? Are these libraries even compatible between themselves when we put them on the class path? They might depend on various libraries themselves, are we going to ship all of that with the project too? The task here is to code up a "HTTP seed provider" -> HTTP meaning it would read some external data in a predefined format which would be parsed and it is responsibility of a user to provide such HTTP endpoint to read the metadata from. Cloud snitches are modelled like that too - they are just using HTTP to call cloud metadata services just in different ways. Would it be possible to not depend on cloud libraries but to code cloud snitches in such a way that they would be still using HTTP under the hood? Secondly, the point is to return, from a seed provider, list of addresses which are supposed to be seeds. I just do not see how putting a snitch into a seed provider achieves this. was (Author: smiklosovic): The problem I see is that if we e.g. talk about that EC2SeedProvider as in that link above, it uses some AWS libraries. If we wanted to ship it with Cassandra, we would have to depend on this library and this would be done for each such cloud. That opens a can of worms ... Are licences compatible? What version of that library are we going to use? Who is going to update it and when? Are these libraries even compatible between themselves when we put the on the class path? They might depend on various libraries themselves, are we going to ship all of that with the project too? The task here is to code up a "HTTP seed provider" -> HTTP meaning it would read some external data in a predefined format which would be parsed and it is responsibility of a user to provide such HTTP endpoint to read the metadata from. Cloud snitches are modelled like that too - they are just using HTTP to call cloud metadata services just in different ways. It would be possible to not depend on cloud libraries but to code cloud snitches in such a way that they would be still using HTTP under the hood? Secondly, the point is to return, from a seed provider, list of addresses which are supposed to be seeds. I just do not see how putting a snitch into a seed provider achieves this. > Implement Http Seed provider > > > Key: CASSANDRA-13855 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13855 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Coordination, Legacy/Core >Reporter: Jon Haddad >Assignee: Claude Warren >Priority: Low > Labels: lhf > Fix For: 5.x > > Attachments: 0001-Add-URL-Seed-Provider-trunk.txt, signature.asc, > signature.asc, signature.asc > > Time Spent: 0.5h > Remaining Estimate: 0h > > Seems like including a dead simple seed provider that can fetch from a URL, 1 > line per seed, would be useful. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13855) Implement Http Seed provider
[ https://issues.apache.org/jira/browse/CASSANDRA-13855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829709#comment-17829709 ] Stefan Miklosovic commented on CASSANDRA-13855: --- The problem I see is that if we e.g. talk about that EC2SeedProvider as in that link above, it uses some AWS libraries. If we wanted to ship it with Cassandra, we would have to depend on this library and this would be done for each such cloud. That opens a can of worms ... Are licences compatible? What version of that library are we going to use? Who is going to update it and when? Are these libraries even compatible between themselves when we put the on the class path? They might depend on various libraries themselves, are we going to ship all of that with the project too? The task here is to code up a "HTTP seed provider" -> HTTP meaning it would read some external data in a predefined format which would be parsed and it is responsibility of a user to provide such HTTP endpoint to read the metadata from. Cloud snitches are modelled like that too - they are just using HTTP to call cloud metadata services just in different ways. It would be possible to not depend on cloud libraries but to code cloud snitches in such a way that they would be still using HTTP under the hood? Secondly, the point is to return, from a seed provider, list of addresses which are supposed to be seeds. I just do not see how putting a snitch into a seed provider achieves this. > Implement Http Seed provider > > > Key: CASSANDRA-13855 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13855 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Coordination, Legacy/Core >Reporter: Jon Haddad >Assignee: Claude Warren >Priority: Low > Labels: lhf > Fix For: 5.x > > Attachments: 0001-Add-URL-Seed-Provider-trunk.txt, signature.asc, > signature.asc, signature.asc > > Time Spent: 0.5h > Remaining Estimate: 0h > > Seems like including a dead simple seed provider that can fetch from a URL, 1 > line per seed, would be useful. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-java-driver) branch 4.x updated: Replace uses of AttributeKey.newInstance
This is an automated email from the ASF dual-hosted git repository.
absurdfarce pushed a commit to branch 4.x
in repository https://gitbox.apache.org/repos/asf/cassandra-java-driver.git
The following commit(s) were added to refs/heads/4.x by this push:
new 4c7133c72 Replace uses of AttributeKey.newInstance
4c7133c72 is described below
commit 4c7133c72e136d23dbcea795e0041df764568931
Author: Andy Tolbert <6889771+tolber...@users.noreply.github.com>
AuthorDate: Tue Jan 23 10:21:02 2024 -0600
Replace uses of AttributeKey.newInstance
The java driver uses netty channel attributes to decorate a
connection's channel with the cluster name (returned from
the system.local table) and the map from the OPTIONS response, both
of which are obtained on connection initialization.
There's an issue here that I wouldn't expect to see in practice in
that the AttributeKey's used are created using
AttributeKey.newInstance, which throws an exception if an
AttributeKey of that name is defined anywhere else in evaluated code.
This change attempts to resolve this issue by changing AttributeKey
initialiation in DriverChannel from newInstance to valueOf, which avoids
throwing an exception if an AttributeKey of the same name was previously
instantiated.
patch by Andy Tolbert; reviewed by Bret McGuire, Alexandre Dutra, Abe
Ratnofsky for CASSANDRA-19290
---
.../com/datastax/oss/driver/internal/core/channel/DriverChannel.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git
a/core/src/main/java/com/datastax/oss/driver/internal/core/channel/DriverChannel.java
b/core/src/main/java/com/datastax/oss/driver/internal/core/channel/DriverChannel.java
index 50932bed8..e40aa6f30 100644
---
a/core/src/main/java/com/datastax/oss/driver/internal/core/channel/DriverChannel.java
+++
b/core/src/main/java/com/datastax/oss/driver/internal/core/channel/DriverChannel.java
@@ -49,9 +49,9 @@ import net.jcip.annotations.ThreadSafe;
@ThreadSafe
public class DriverChannel {
- static final AttributeKey CLUSTER_NAME_KEY =
AttributeKey.newInstance("cluster_name");
+ static final AttributeKey CLUSTER_NAME_KEY =
AttributeKey.valueOf("cluster_name");
static final AttributeKey>> OPTIONS_KEY =
- AttributeKey.newInstance("options");
+ AttributeKey.valueOf("options");
@SuppressWarnings("RedundantStringConstructorCall")
static final Object GRACEFUL_CLOSE_MESSAGE = new
String("GRACEFUL_CLOSE_MESSAGE");
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19290: Replace uses of AttributeKey.newInstance [cassandra-java-driver]
absurdfarce merged PR #1908: URL: https://github.com/apache/cassandra-java-driver/pull/1908 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-107) Improve logging for slice restore task
[ https://issues.apache.org/jira/browse/CASSANDRASC-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRASC-107: --- Fix Version/s: 1.0 Source Control Link: https://github.com/apache/cassandra-sidecar/commit/f848cd063e5e1671c84807615f5eae809253971d Resolution: Fixed Status: Resolved (was: Ready to Commit) > Improve logging for slice restore task > -- > > Key: CASSANDRASC-107 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-107 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Francisco Guerrero >Priority: Normal > Labels: low-hanging-fruit, pull-request-available > Fix For: 1.0 > > > I want to propose logging improvements. > Add more logs to the individual steps during the restore task, i.e. > RestoreSliceTask and StorageClient. > In other places like retrying to poll the object existence, the stack trace > can be omitted, as it provides no additional knowledge than object not found. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-sidecar) branch trunk updated: CASSANDRASC-107: Improve logging for slice restore task (#108)
This is an automated email from the ASF dual-hosted git repository.
frankgh pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-sidecar.git
The following commit(s) were added to refs/heads/trunk by this push:
new f848cd06 CASSANDRASC-107: Improve logging for slice restore task (#108)
f848cd06 is described below
commit f848cd063e5e1671c84807615f5eae809253971d
Author: Francisco Guerrero
AuthorDate: Thu Mar 21 15:26:06 2024 -0700
CASSANDRASC-107: Improve logging for slice restore task (#108)
Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRASC-107
---
CHANGES.txt| 3 +-
.../cassandra/sidecar/restore/RestoreJobUtil.java | 8 +-
.../sidecar/restore/RestoreSliceTask.java | 229 ++---
.../cassandra/sidecar/restore/StorageClient.java | 40 +++-
4 files changed, 143 insertions(+), 137 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index ce39b66c..638ee316 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
1.0.0
-
+ * Improve logging for slice restore task (CASSANDRASC-107)
* Add restore task watcher to report long running tasks (CASSANDRASC-106)
* RestoreSliceTask could be stuck due to missing exception handling
(CASSANDRASC-105)
* Make hash algorithm implementation pluggable (CASSANDRASC-114)
@@ -82,4 +83,4 @@
* Add integration tests task (CASSANDRA-15031)
* Add support for SSL and bindable address (CASSANDRA-15030)
* Autogenerate API docs for sidecar (CASSANDRA-15028)
- * C* Management process (CASSANDRA-14395)
\ No newline at end of file
+ * C* Management process (CASSANDRA-14395)
diff --git
a/src/main/java/org/apache/cassandra/sidecar/restore/RestoreJobUtil.java
b/src/main/java/org/apache/cassandra/sidecar/restore/RestoreJobUtil.java
index 8651d535..be488ae4 100644
--- a/src/main/java/org/apache/cassandra/sidecar/restore/RestoreJobUtil.java
+++ b/src/main/java/org/apache/cassandra/sidecar/restore/RestoreJobUtil.java
@@ -78,9 +78,9 @@ public class RestoreJobUtil
{
try (ZipInputStream zis = new
ZipInputStream(Files.newInputStream(zipFile.toPath(
{
-ZipEntry zipEntry = zis.getNextEntry();
+ZipEntry zipEntry;
-while (zipEntry != null)
+while ((zipEntry = zis.getNextEntry()) != null)
{
// Encounters a directory inside the zip file
// It is not expected. The zip file should have the directory
depth of 1.
@@ -92,8 +92,6 @@ public class RestoreJobUtil
File targetFile = newProtectedTargetFile(zipEntry, targetDir);
Files.copy(zis, targetFile.toPath(),
StandardCopyOption.REPLACE_EXISTING);
-
-zipEntry = zis.getNextEntry();
}
zis.closeEntry();
}
@@ -161,7 +159,7 @@ public class RestoreJobUtil
}
catch (IOException e)
{
-LOGGER.error("Unexpected error occurred while cleaning
directory {}, ", path, e);
+LOGGER.error("Unexpected error occurred while cleaning
directory {}", path, e);
throw new RuntimeException(e);
}
});
diff --git
a/src/main/java/org/apache/cassandra/sidecar/restore/RestoreSliceTask.java
b/src/main/java/org/apache/cassandra/sidecar/restore/RestoreSliceTask.java
index ed85d3f9..8c828d89 100644
--- a/src/main/java/org/apache/cassandra/sidecar/restore/RestoreSliceTask.java
+++ b/src/main/java/org/apache/cassandra/sidecar/restore/RestoreSliceTask.java
@@ -21,7 +21,6 @@ package org.apache.cassandra.sidecar.restore;
import java.io.File;
import java.nio.file.Files;
import java.util.Map;
-import java.util.concurrent.CompletableFuture;
import com.google.common.annotations.VisibleForTesting;
import org.slf4j.Logger;
@@ -48,6 +47,7 @@ import
software.amazon.awssdk.core.exception.ApiCallTimeoutException;
import software.amazon.awssdk.services.s3.model.NoSuchKeyException;
import software.amazon.awssdk.services.s3.model.S3Exception;
+import static io.vertx.core.Future.fromCompletionStage;
import static
org.apache.cassandra.sidecar.utils.AsyncFileSystemUtils.ensureSufficientStorage;
/**
@@ -55,7 +55,7 @@ import static
org.apache.cassandra.sidecar.utils.AsyncFileSystemUtils.ensureSuff
* and imports SSTables into Cassandra.
* It the execution ever fails, the cause should only be
* {@link org.apache.cassandra.sidecar.exceptions.RestoreJobException}
- *
+ *
* Note that the class is package private, and it is not intended to be
referenced by other packages.
*/
public class RestoreSliceTask implements RestoreSliceHandler
@@ -130,25 +130,17 @@ public class RestoreSliceTask implements
RestoreSliceHandler
}
// 1. check object existence and validate eTag / checksum
-CompletableFuture fut = checkObjectExis
[jira] [Updated] (CASSANDRASC-107) Improve logging for slice restore task
[ https://issues.apache.org/jira/browse/CASSANDRASC-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRASC-107: --- Status: Ready to Commit (was: Review In Progress) > Improve logging for slice restore task > -- > > Key: CASSANDRASC-107 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-107 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Francisco Guerrero >Priority: Normal > Labels: low-hanging-fruit, pull-request-available > > I want to propose logging improvements. > Add more logs to the individual steps during the restore task, i.e. > RestoreSliceTask and StorageClient. > In other places like retrying to poll the object existence, the stack trace > can be omitted, as it provides no additional knowledge than object not found. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-107) Improve logging for slice restore task
[ https://issues.apache.org/jira/browse/CASSANDRASC-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRASC-107: -- Reviewers: Yifan Cai Status: Review In Progress (was: Patch Available) > Improve logging for slice restore task > -- > > Key: CASSANDRASC-107 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-107 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Francisco Guerrero >Priority: Normal > Labels: low-hanging-fruit, pull-request-available > > I want to propose logging improvements. > Add more logs to the individual steps during the restore task, i.e. > RestoreSliceTask and StorageClient. > In other places like retrying to poll the object existence, the stack trace > can be omitted, as it provides no additional knowledge than object not found. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19418) [Analytics] Report additional bulk analytics job stats for instrumentation
[ https://issues.apache.org/jira/browse/CASSANDRA-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829663#comment-17829663 ] Arjun Ashok edited comment on CASSANDRA-19418 at 3/21/24 9:34 PM: -- CI is green: https://app.circleci.com/pipelines/github/arjunashok/cassandra-analytics/78/workflows/4cf445d1-1a4c-430a-a498-54341f754996 was (Author: JIRAUSER301082): CI is green: https://app.circleci.com/pipelines/github/arjunashok/cassandra-analytics/77/workflows/f87b3605-f5b2-4f50-a039-b55822b43e2c > [Analytics] Report additional bulk analytics job stats for instrumentation > -- > > Key: CASSANDRA-19418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19418 > Project: Cassandra > Issue Type: Task > Components: Analytics Library >Reporter: Arjun Ashok >Assignee: Arjun Ashok >Priority: Normal > Time Spent: 3h > Remaining Estimate: 0h > > Currently, the Cassandra bulk analytics library supports a "dialHome" API to > publish some initial job metadata, which in its current form, is redirected > to a log. The intention behind this is to allow custom implementations that > can utilize these summarized stats for instrumentation or reporting of client > behavior. > This task is meant to enhance this API to allow for additional job metadata > to be published both at the Spark executor level and at the task levels to > gather stats such as "success/failure", "number of rows written/read", > "failure reason" etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[PR] Updating README JIRA link to ASF JIRA [cassandra-java-driver]
absurdfarce opened a new pull request, #1921: URL: https://github.com/apache/cassandra-java-driver/pull/1921 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19352) 4.x Java driver support for native_port_ssl and native_transport_port_ssl
[ https://issues.apache.org/jira/browse/CASSANDRA-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bret McGuire updated CASSANDRA-19352: - Resolution: Fixed Status: Resolved (was: Triage Needed) > 4.x Java driver support for native_port_ssl and native_transport_port_ssl > - > > Key: CASSANDRA-19352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19352 > Project: Cassandra > Issue Type: Improvement > Components: Client/java-driver >Reporter: absurdfarce#1 >Assignee: Bret McGuire >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > DSE 6.8 added a "native_transport_port_ssl" column to peers_v2 to indicate > when peers were making use of "native_transport_port_ssl" in cassandra.yaml. > Similar functionality (with slightly different column names) was brought to > OSS Cassandra with CASSANDRA-16999. 3.x Java driver support for these > columns has been added (or is in the process of being added) in > [JAVA-2967|https://datastax-oss.atlassian.net/browse/JAVA-2967]. This ticket > represents the work to implement similar functionality for the 4.x Java > driver. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-19487) Rework json testing
[ https://issues.apache.org/jira/browse/CASSANDRA-19487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Ganesh reassigned CASSANDRA-19487: --- Assignee: Arun Ganesh > Rework json testing > --- > > Key: CASSANDRA-19487 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19487 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Arun Ganesh >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > The only real json testing we have is in [this > dtest|https://github.com/apache/cassandra-dtest/blob/trunk/json_test.py] > which is full of magic like running from the tests' docstrings which makes > other things very difficult, like adding any kind of version-based variation. > I propose rewriting this, either as sane python or a java dtest. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13855) Implement Http Seed provider
[ https://issues.apache.org/jira/browse/CASSANDRA-13855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829693#comment-17829693 ] Jon Haddad edited comment on CASSANDRA-13855 at 3/21/24 8:58 PM: - At a high level, I like the ability to use Cloud provider tags to mark a node as a seed, but I don't think it has to be an either-or situation. The http based solution I proposed is simple, straightforward, and works everywhere. It sounds to me like there's a case to be made for multiple solutions, whether it be etcd, consul, tags, ZK, etc. I don't think we should turn this into a choice between them. Can you please open another JIRA with your suggestion? was (Author: rustyrazorblade): At a high level, I like the ability to use Cloud provider tags to mark a node as a seed, but I don't think it has to be an either-or situation. The http based solution I proposed is simple, straightforward, and works everywhere. It sounds to me like there's a case to be made for multiple solutions, whether it be etcd, consul, tags, ZK, etc. I don't think need to turn this into a choice between them. Can you please open another JIRA with your suggestion? > Implement Http Seed provider > > > Key: CASSANDRA-13855 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13855 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Coordination, Legacy/Core >Reporter: Jon Haddad >Assignee: Claude Warren >Priority: Low > Labels: lhf > Fix For: 5.x > > Attachments: 0001-Add-URL-Seed-Provider-trunk.txt, signature.asc, > signature.asc, signature.asc > > Time Spent: 0.5h > Remaining Estimate: 0h > > Seems like including a dead simple seed provider that can fetch from a URL, 1 > line per seed, would be useful. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13855) Implement Http Seed provider
[ https://issues.apache.org/jira/browse/CASSANDRA-13855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829693#comment-17829693 ] Jon Haddad commented on CASSANDRA-13855: At a high level, I like the ability to use Cloud provider tags to mark a node as a seed, but I don't think it has to be an either-or situation. The http based solution I proposed is simple, straightforward, and works everywhere. It sounds to me like there's a case to be made for multiple solutions, whether it be etcd, consul, tags, ZK, etc. I don't think need to turn this into a choice between them. Can you please open another JIRA with your suggestion? > Implement Http Seed provider > > > Key: CASSANDRA-13855 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13855 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Coordination, Legacy/Core >Reporter: Jon Haddad >Assignee: Claude Warren >Priority: Low > Labels: lhf > Fix For: 5.x > > Attachments: 0001-Add-URL-Seed-Provider-trunk.txt, signature.asc, > signature.asc, signature.asc > > Time Spent: 0.5h > Remaining Estimate: 0h > > Seems like including a dead simple seed provider that can fetch from a URL, 1 > line per seed, would be useful. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19418) [Analytics] Report additional bulk analytics job stats for instrumentation
[ https://issues.apache.org/jira/browse/CASSANDRA-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829692#comment-17829692 ] Francisco Guerrero commented on CASSANDRA-19418: +1 Thanks for the patch and for addressing all the comments. > [Analytics] Report additional bulk analytics job stats for instrumentation > -- > > Key: CASSANDRA-19418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19418 > Project: Cassandra > Issue Type: Task > Components: Analytics Library >Reporter: Arjun Ashok >Assignee: Arjun Ashok >Priority: Normal > Time Spent: 3h > Remaining Estimate: 0h > > Currently, the Cassandra bulk analytics library supports a "dialHome" API to > publish some initial job metadata, which in its current form, is redirected > to a log. The intention behind this is to allow custom implementations that > can utilize these summarized stats for instrumentation or reporting of client > behavior. > This task is meant to enhance this API to allow for additional job metadata > to be published both at the Spark executor level and at the task levels to > gather stats such as "success/failure", "number of rows written/read", > "failure reason" etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19418) [Analytics] Report additional bulk analytics job stats for instrumentation
[ https://issues.apache.org/jira/browse/CASSANDRA-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRA-19418: --- Reviewers: Doug Rohrer, Francisco Guerrero Status: Review In Progress (was: Patch Available) > [Analytics] Report additional bulk analytics job stats for instrumentation > -- > > Key: CASSANDRA-19418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19418 > Project: Cassandra > Issue Type: Task > Components: Analytics Library >Reporter: Arjun Ashok >Assignee: Arjun Ashok >Priority: Normal > Time Spent: 3h > Remaining Estimate: 0h > > Currently, the Cassandra bulk analytics library supports a "dialHome" API to > publish some initial job metadata, which in its current form, is redirected > to a log. The intention behind this is to allow custom implementations that > can utilize these summarized stats for instrumentation or reporting of client > behavior. > This task is meant to enhance this API to allow for additional job metadata > to be published both at the Spark executor level and at the task levels to > gather stats such as "success/failure", "number of rows written/read", > "failure reason" etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13855) Implement Http Seed provider
[
https://issues.apache.org/jira/browse/CASSANDRA-13855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829689#comment-17829689
]
Maxim Muzafarov commented on CASSANDRA-13855:
-
Hey,
I've checked the implementation and I'd like to throw a new idea on the table.
I have nothing against the HttpSeedProvider itself, but I guess for the end
user it is more interesting to have cloud-related stuff that they are already
configured with (the same as we provide for the snitches - Azure, EC2, Google).
Here are the SeedProviders that can be found on the GitHub:
https://github.com/search?q=SeedProvider&type=repositories
So, I guess we don't need to create a parallel hierarchy of classes for the
SeedProvider interface and can do the following:
# Either extend the IEndpointSnitch or create a new interface new a new :
{code}
public String getRack(InetAddressAndPort endpoint);
public String getDatacenter(InetAddressAndPort endpoint);
/** {@code true} if the endpoint is the seed node. */
default public boolean isSeed(InetAddressAndPort endpoint) {};
{code}
# Implement a new {{isSeed}} method for each cloud adapter we are interested in
e.g. EC2, Azure, Google etc. that fetches the seed status from a tag.
https://github.com/kryptnostic/ec2-cassandra-seedprovider/blob/master/src/main/java/com/kryptnostic/Ec2SeedProvider.java#L35
# Create a new adapter that implements the SeedProvider interface and wraps the
{{IEndpointSnitch}} e.g. new CloudSeedProvider(IEndpointSnitch snitch).
WDYT?
> Implement Http Seed provider
>
>
> Key: CASSANDRA-13855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13855
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Coordination, Legacy/Core
>Reporter: Jon Haddad
>Assignee: Claude Warren
>Priority: Low
> Labels: lhf
> Fix For: 5.x
>
> Attachments: 0001-Add-URL-Seed-Provider-trunk.txt, signature.asc,
> signature.asc, signature.asc
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Seems like including a dead simple seed provider that can fetch from a URL, 1
> line per seed, would be useful.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-java-driver) branch 4.x updated: CASSANDRA-19352: Support native_transport_(address|port) + native_transport_port_ssl for DSE 6.8 (4.x edition)
This is an automated email from the ASF dual-hosted git repository.
absurdfarce pushed a commit to branch 4.x
in repository https://gitbox.apache.org/repos/asf/cassandra-java-driver.git
The following commit(s) were added to refs/heads/4.x by this push:
new 7e2c6579a CASSANDRA-19352: Support native_transport_(address|port) +
native_transport_port_ssl for DSE 6.8 (4.x edition)
7e2c6579a is described below
commit 7e2c6579af564be6d1b161ec4159ecf517c190b4
Author: Bret McGuire
AuthorDate: Tue Feb 6 15:18:59 2024 -0600
CASSANDRA-19352: Support native_transport_(address|port) +
native_transport_port_ssl for DSE 6.8 (4.x edition)
patch by absurdfarce; reviewed by absurdfarce and adutra for CASSANDRA-19352
---
.../core/metadata/DefaultTopologyMonitor.java | 76 +++--
.../core/metadata/DefaultTopologyMonitorTest.java | 180 +++--
2 files changed, 223 insertions(+), 33 deletions(-)
diff --git
a/core/src/main/java/com/datastax/oss/driver/internal/core/metadata/DefaultTopologyMonitor.java
b/core/src/main/java/com/datastax/oss/driver/internal/core/metadata/DefaultTopologyMonitor.java
index 87008b05c..f3dc988cf 100644
---
a/core/src/main/java/com/datastax/oss/driver/internal/core/metadata/DefaultTopologyMonitor.java
+++
b/core/src/main/java/com/datastax/oss/driver/internal/core/metadata/DefaultTopologyMonitor.java
@@ -34,6 +34,7 @@ import
com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures;
import
com.datastax.oss.driver.shaded.guava.common.annotations.VisibleForTesting;
import com.datastax.oss.driver.shaded.guava.common.collect.ImmutableMap;
import com.datastax.oss.driver.shaded.guava.common.collect.ImmutableSet;
+import com.datastax.oss.driver.shaded.guava.common.collect.Iterators;
import com.datastax.oss.protocol.internal.ProtocolConstants;
import com.datastax.oss.protocol.internal.response.Error;
import edu.umd.cs.findbugs.annotations.NonNull;
@@ -69,6 +70,10 @@ public class DefaultTopologyMonitor implements
TopologyMonitor {
// Assume topology queries never need paging
private static final int INFINITE_PAGE_SIZE = -1;
+ // A few system.peers columns which get special handling below
+ private static final String NATIVE_PORT = "native_port";
+ private static final String NATIVE_TRANSPORT_PORT = "native_transport_port";
+
private final String logPrefix;
private final InternalDriverContext context;
private final ControlConnection controlConnection;
@@ -494,28 +499,65 @@ public class DefaultTopologyMonitor implements
TopologyMonitor {
@Nullable
protected InetSocketAddress getBroadcastRpcAddress(
@NonNull AdminRow row, @NonNull EndPoint localEndPoint) {
-// in system.peers or system.local
-InetAddress broadcastRpcInetAddress = row.getInetAddress("rpc_address");
+
+InetAddress broadcastRpcInetAddress = null;
+Iterator addrCandidates =
+Iterators.forArray(
+// in system.peers_v2 (Cassandra >= 4.0)
+"native_address",
+// DSE 6.8 introduced native_transport_address and
native_transport_port for the
+// listen address.
+"native_transport_address",
+// in system.peers or system.local
+"rpc_address");
+
+while (broadcastRpcInetAddress == null && addrCandidates.hasNext())
+ broadcastRpcInetAddress = row.getInetAddress(addrCandidates.next());
+// This could only happen if system tables are corrupted, but handle
gracefully
if (broadcastRpcInetAddress == null) {
- // in system.peers_v2 (Cassandra >= 4.0)
- broadcastRpcInetAddress = row.getInetAddress("native_address");
- if (broadcastRpcInetAddress == null) {
-// This could only happen if system tables are corrupted, but handle
gracefully
-return null;
+ LOG.warn(
+ "[{}] Unable to determine broadcast RPC IP address, returning null.
"
+ + "This is likely due to a misconfiguration or invalid system
tables. "
+ + "Please validate the contents of system.local and/or {}.",
+ logPrefix,
+ getPeerTableName());
+ return null;
+}
+
+Integer broadcastRpcPort = null;
+Iterator portCandidates =
+Iterators.forArray(
+// in system.peers_v2 (Cassandra >= 4.0)
+NATIVE_PORT,
+// DSE 6.8 introduced native_transport_address and
native_transport_port for the
+// listen address.
+NATIVE_TRANSPORT_PORT,
+// system.local for Cassandra >= 4.0
+"rpc_port");
+
+while ((broadcastRpcPort == null || broadcastRpcPort == 0) &&
portCandidates.hasNext()) {
+
+ String colName = portCandidates.next();
+ broadcastRpcPort = row.getInteger(colName);
+ // Support override for SSL port (if enabled) in DSE
+ if (NATIVE_TRANSPORT_PORT.equals(colName) &&
context.getSslEngineFactory().isPresent()) {
+
+String sslColName = colName + "_ssl";
+
Re: [PR] CASSANDRA-19352: Support native_transport_(address|port) + native_transport_port_ssl for DSE 6.8 (4.x edition) [cassandra-java-driver]
absurdfarce merged PR #1913: URL: https://github.com/apache/cassandra-java-driver/pull/1913 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-website) branch asf-staging updated (618ca889f -> b8c4937c9)
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a change to branch asf-staging
in repository https://gitbox.apache.org/repos/asf/cassandra-website.git
discard 618ca889f generate docs for fd550e9c
new b8c4937c9 generate docs for fd550e9c
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (618ca889f)
\
N -- N -- N refs/heads/asf-staging (b8c4937c9)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../managing/configuration/cass_yaml_file.html | 51 +-
.../doc/5.1/cassandra/developing/cql/security.html | 16 +
.../managing/configuration/cass_yaml_file.html | 51 +-
.../cassandra/managing/tools/nodetool/cms.html}| 62 +-
.../managing/tools/nodetool/describecms.html | 1112 ---
.../cassandra/managing/tools/nodetool/import.html |6 +-
.../managing/tools/nodetool/nodetool.html | 12 +-
.../managing/tools/nodetool/reconfigurecms.html| 1132
.../reference/cql-commands/commands-toc.html |4 +
.../managing/configuration/cass_yaml_file.html | 51 +-
.../trunk/cassandra/developing/cql/security.html | 16 +
.../managing/configuration/cass_yaml_file.html | 51 +-
.../cassandra/managing/tools/nodetool/cms.html}| 62 +-
.../managing/tools/nodetool/describecms.html | 1112 ---
.../cassandra/managing/tools/nodetool/import.html |6 +-
.../managing/tools/nodetool/nodetool.html | 12 +-
.../managing/tools/nodetool/reconfigurecms.html| 1132
.../reference/cql-commands/commands-toc.html |4 +
content/search-index.js|2 +-
site-ui/build/ui-bundle.zip| Bin 4883646 -> 4883646
bytes
20 files changed, 335 insertions(+), 4559 deletions(-)
rename content/doc/{trunk/cassandra/managing/tools/nodetool/initializecms.html
=> 5.1/cassandra/managing/tools/nodetool/cms.html} (94%)
delete mode 100644
content/doc/5.1/cassandra/managing/tools/nodetool/describecms.html
delete mode 100644
content/doc/5.1/cassandra/managing/tools/nodetool/reconfigurecms.html
rename content/doc/{5.1/cassandra/managing/tools/nodetool/initializecms.html
=> trunk/cassandra/managing/tools/nodetool/cms.html} (94%)
delete mode 100644
content/doc/trunk/cassandra/managing/tools/nodetool/describecms.html
delete mode 100644
content/doc/trunk/cassandra/managing/tools/nodetool/reconfigurecms.html
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-107) Improve logging for slice restore task
[ https://issues.apache.org/jira/browse/CASSANDRASC-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRASC-107: --- Test and Documentation Plan: Improved logging Status: Patch Available (was: In Progress) PR: https://github.com/apache/cassandra-sidecar/pull/108 CI: https://app.circleci.com/pipelines/github/frankgh/cassandra-sidecar?branch=CASSANDRASC-107 > Improve logging for slice restore task > -- > > Key: CASSANDRASC-107 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-107 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Francisco Guerrero >Priority: Normal > Labels: low-hanging-fruit, pull-request-available > > I want to propose logging improvements. > Add more logs to the individual steps during the restore task, i.e. > RestoreSliceTask and StorageClient. > In other places like retrying to poll the object existence, the stack trace > can be omitted, as it provides no additional knowledge than object not found. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19428) Clean up KeyRangeIterator classes
[ https://issues.apache.org/jira/browse/CASSANDRA-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829677#comment-17829677 ] Ekaterina Dimitrova commented on CASSANDRA-19428: - Alright, marked as a 5.0 blocker then, thanks! > Clean up KeyRangeIterator classes > - > > Key: CASSANDRA-19428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19428 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 5.0-rc, 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > Remove KeyRangeIterator.current and simplify -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19428) Clean up KeyRangeIterator classes
[ https://issues.apache.org/jira/browse/CASSANDRA-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-19428: Fix Version/s: 5.0-rc (was: 5.0) (was: 5.0.x) > Clean up KeyRangeIterator classes > - > > Key: CASSANDRA-19428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19428 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 5.0-rc, 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > Remove KeyRangeIterator.current and simplify -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19428) Clean up KeyRangeIterator classes
[ https://issues.apache.org/jira/browse/CASSANDRA-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-19428: Fix Version/s: 5.0 > Clean up KeyRangeIterator classes > - > > Key: CASSANDRA-19428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19428 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 5.0, 5.0.x, 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > Remove KeyRangeIterator.current and simplify -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19418) [Analytics] Report additional bulk analytics job stats for instrumentation
[ https://issues.apache.org/jira/browse/CASSANDRA-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Ashok updated CASSANDRA-19418: Test and Documentation Plan: Validated unit/integration tests and on a test cluster. CI is green Status: Patch Available (was: Open) > [Analytics] Report additional bulk analytics job stats for instrumentation > -- > > Key: CASSANDRA-19418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19418 > Project: Cassandra > Issue Type: Task > Components: Analytics Library >Reporter: Arjun Ashok >Assignee: Arjun Ashok >Priority: Normal > Time Spent: 3h > Remaining Estimate: 0h > > Currently, the Cassandra bulk analytics library supports a "dialHome" API to > publish some initial job metadata, which in its current form, is redirected > to a log. The intention behind this is to allow custom implementations that > can utilize these summarized stats for instrumentation or reporting of client > behavior. > This task is meant to enhance this API to allow for additional job metadata > to be published both at the Spark executor level and at the task levels to > gather stats such as "success/failure", "number of rows written/read", > "failure reason" etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19418) [Analytics] Report additional bulk analytics job stats for instrumentation
[ https://issues.apache.org/jira/browse/CASSANDRA-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829663#comment-17829663 ] Arjun Ashok edited comment on CASSANDRA-19418 at 3/21/24 7:07 PM: -- CI is green: https://app.circleci.com/pipelines/github/arjunashok/cassandra-analytics/77/workflows/f87b3605-f5b2-4f50-a039-b55822b43e2c was (Author: JIRAUSER301082): CI: [https://app.circleci.com/pipelines/github/arjunashok/cassandra-analytics/76/workflows/662c438a-7541-4a53-a9e9-08b8d472e6df|https://app.circleci.com/pipelines/github/arjunashok/cassandra-analytics/77/workflows/fc6fd747-80ea-425e-bb2c-3998647f7a42] > [Analytics] Report additional bulk analytics job stats for instrumentation > -- > > Key: CASSANDRA-19418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19418 > Project: Cassandra > Issue Type: Task > Components: Analytics Library >Reporter: Arjun Ashok >Assignee: Arjun Ashok >Priority: Normal > Time Spent: 3h > Remaining Estimate: 0h > > Currently, the Cassandra bulk analytics library supports a "dialHome" API to > publish some initial job metadata, which in its current form, is redirected > to a log. The intention behind this is to allow custom implementations that > can utilize these summarized stats for instrumentation or reporting of client > behavior. > This task is meant to enhance this API to allow for additional job metadata > to be published both at the Spark executor level and at the task levels to > gather stats such as "success/failure", "number of rows written/read", > "failure reason" etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-107) Improve logging for slice restore task
[ https://issues.apache.org/jira/browse/CASSANDRASC-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CASSANDRASC-107: --- Labels: low-hanging-fruit pull-request-available (was: low-hanging-fruit) > Improve logging for slice restore task > -- > > Key: CASSANDRASC-107 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-107 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Francisco Guerrero >Priority: Normal > Labels: low-hanging-fruit, pull-request-available > > I want to propose logging improvements. > Add more logs to the individual steps during the restore task, i.e. > RestoreSliceTask and StorageClient. > In other places like retrying to poll the object existence, the stack trace > can be omitted, as it provides no additional knowledge than object not found. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19461) SAI does not index empty bytes even for types that allow empty bytes as a valid input
[
https://issues.apache.org/jira/browse/CASSANDRA-19461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-19461:
Authors: Caleb Rackliffe, David Capwell (was: Caleb Rackliffe)
> SAI does not index empty bytes even for types that allow empty bytes as a
> valid input
> -
>
> Key: CASSANDRA-19461
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19461
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Attachments: ci_summary.html, result_details.tar.gz
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> This is easy to reproduce with a test that looks something like this:
> {noformat}
> @Test
> public void testEmptyString()
> {
> createTable("CREATE TABLE %s (k TEXT PRIMARY KEY, v text)");
> createIndex(String.format(CREATE_INDEX_TEMPLATE, 'v'));
> execute("INSERT INTO %s (k, v) VALUES ('0', '')");
> execute("INSERT INTO %s (k) VALUES ('1')");
>
> // flush(); < there is not always a memtable index involved, a fix
> will have to pay attention to this
> List rows = executeNet("SELECT * FROM %s WHERE v = ''").all();
> assertEquals(1, rows.size()); <— FAILS! No matches...
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19418) [Analytics] Report additional bulk analytics job stats for instrumentation
[ https://issues.apache.org/jira/browse/CASSANDRA-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829663#comment-17829663 ] Arjun Ashok edited comment on CASSANDRA-19418 at 3/21/24 6:25 PM: -- CI: [https://app.circleci.com/pipelines/github/arjunashok/cassandra-analytics/76/workflows/662c438a-7541-4a53-a9e9-08b8d472e6df|https://app.circleci.com/pipelines/github/arjunashok/cassandra-analytics/77/workflows/fc6fd747-80ea-425e-bb2c-3998647f7a42] was (Author: JIRAUSER301082): CI: https://app.circleci.com/pipelines/github/arjunashok/cassandra-analytics/76/workflows/662c438a-7541-4a53-a9e9-08b8d472e6df > [Analytics] Report additional bulk analytics job stats for instrumentation > -- > > Key: CASSANDRA-19418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19418 > Project: Cassandra > Issue Type: Task > Components: Analytics Library >Reporter: Arjun Ashok >Assignee: Arjun Ashok >Priority: Normal > Time Spent: 3h > Remaining Estimate: 0h > > Currently, the Cassandra bulk analytics library supports a "dialHome" API to > publish some initial job metadata, which in its current form, is redirected > to a log. The intention behind this is to allow custom implementations that > can utilize these summarized stats for instrumentation or reporting of client > behavior. > This task is meant to enhance this API to allow for additional job metadata > to be published both at the Spark executor level and at the task levels to > gather stats such as "success/failure", "number of rows written/read", > "failure reason" etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19418) [Analytics] Report additional bulk analytics job stats for instrumentation
[ https://issues.apache.org/jira/browse/CASSANDRA-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arjun Ashok updated CASSANDRA-19418: Complexity: Normal Status: Open (was: Triage Needed) > [Analytics] Report additional bulk analytics job stats for instrumentation > -- > > Key: CASSANDRA-19418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19418 > Project: Cassandra > Issue Type: Task > Components: Analytics Library >Reporter: Arjun Ashok >Assignee: Arjun Ashok >Priority: Normal > Time Spent: 3h > Remaining Estimate: 0h > > Currently, the Cassandra bulk analytics library supports a "dialHome" API to > publish some initial job metadata, which in its current form, is redirected > to a log. The intention behind this is to allow custom implementations that > can utilize these summarized stats for instrumentation or reporting of client > behavior. > This task is meant to enhance this API to allow for additional job metadata > to be published both at the Spark executor level and at the task levels to > gather stats such as "success/failure", "number of rows written/read", > "failure reason" etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19418) [Analytics] Report additional bulk analytics job stats for instrumentation
[ https://issues.apache.org/jira/browse/CASSANDRA-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829663#comment-17829663 ] Arjun Ashok commented on CASSANDRA-19418: - CI: https://app.circleci.com/pipelines/github/arjunashok/cassandra-analytics/76/workflows/662c438a-7541-4a53-a9e9-08b8d472e6df > [Analytics] Report additional bulk analytics job stats for instrumentation > -- > > Key: CASSANDRA-19418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19418 > Project: Cassandra > Issue Type: Task > Components: Analytics Library >Reporter: Arjun Ashok >Assignee: Arjun Ashok >Priority: Normal > Time Spent: 3h > Remaining Estimate: 0h > > Currently, the Cassandra bulk analytics library supports a "dialHome" API to > publish some initial job metadata, which in its current form, is redirected > to a log. The intention behind this is to allow custom implementations that > can utilize these summarized stats for instrumentation or reporting of client > behavior. > This task is meant to enhance this API to allow for additional job metadata > to be published both at the Spark executor level and at the task levels to > gather stats such as "success/failure", "number of rows written/read", > "failure reason" etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19428) Clean up KeyRangeIterator classes
[ https://issues.apache.org/jira/browse/CASSANDRA-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829661#comment-17829661 ] Caleb Rackliffe commented on CASSANDRA-19428: - [~e.dimitrova] I think we want this to be consistent across 5.0 and trunk. (We have better tooling to test on trunk, but that's fine. The codebases are still so similar I don't think it matters if we don't literally run them on 5.0.) I'll jump on review ASAP... > Clean up KeyRangeIterator classes > - > > Key: CASSANDRA-19428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19428 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 5.0.x, 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > Remove KeyRangeIterator.current and simplify -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17808) Optionally avoid hint transfer during decommission
[
https://issues.apache.org/jira/browse/CASSANDRA-17808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829659#comment-17829659
]
Caleb Rackliffe commented on CASSANDRA-17808:
-
Let me see if I can find some time to back-port this to 4.0 and 4.1 this week
or next...
> Optionally avoid hint transfer during decommission
> --
>
> Key: CASSANDRA-17808
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17808
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-alpha1, 5.0
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Both because they aren’t strictly necessary to maintain consistency, and
> because throttling induced by their rate-limiter (see
> {{hinted_handoff_throttle}}) may stall progress, transferring hints during
> decommission (specifically unbootstrap) rather than just pausing, disabling,
> and truncating them probably doesn’t make sense. The only other concern would
> be the BatchLog, which nominally depends on hint delivery to maintain its
> "guarantees". However, during BatchLog replay on unbootstrap,
> {{ReplayingBatch}} ignores batches older the gcgs anyway.
> Here's a proposal from [~aleksey] that might strike a reasonable balance:
> 1.) We continue to transfer hints by default during decommission, but at a
> higher rate. We could, for instance, stop having {{DispatchHintsTask}} divide
> its effective rate by the number of nodes in the cluster.
> {noformat}
> int nodesCount = Math.max(1,
> StorageService.instance.getTokenMetadata().getAllEndpoints().size() - 1);
> double throttleInBytes = DatabaseDescriptor.getHintedHandoffThrottleInKiB() *
> 1024.0 / nodesCount;
> this.rateLimiter = RateLimiter.create(throttleInBytes == 0 ? Double.MAX_VALUE
> : throttleInBytes);
> {noformat}
> 2.) We provide an option to simply avoid transferring hints during
> unbootstrap. Even this would only take the BatchLog from "best effort" to
> "slightly less effort" ;)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19428) Clean up KeyRangeIterator classes
[ https://issues.apache.org/jira/browse/CASSANDRA-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-19428: Test and Documentation Plan: 5.0 patch PR - [https://github.com/apache/cassandra/pull/3194] Fully green CI - [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=19428-5.0-3] I tested locally the patch on top of the trunk - running the new fuzz tests added, and they all passed, too. I can propagate and run full CI when we confirm the final patch. Clean up KeyRangeIterator classes * replace KeyRangeConcatIterator's PriorityQeueu with List * remove KeyRangeIterator.current and simplify * remove injected exception and tests - not relevant to the current implementation * expand randomized testing * inline getCurrent() -> peek(); rename getCount to getMaxKeys * redefine skipTo contract to not return a value (which saves unnecessary work when skipTo is called multiple times in a row) * calling hasNext in skipTo is a pessimization; if the iterator is in DONE state, then skipTo will see it and avoid further effort; if it is not, then we are computing a next value that we're just going to throw away * Fix a bug in QueryController#getIndexResults error handling Status: Patch Available (was: In Progress) > Clean up KeyRangeIterator classes > - > > Key: CASSANDRA-19428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19428 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 5.0.x, 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > Remove KeyRangeIterator.current and simplify -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19428) Clean up KeyRangeIterator classes
[ https://issues.apache.org/jira/browse/CASSANDRA-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829658#comment-17829658 ] Ekaterina Dimitrova commented on CASSANDRA-19428: - The failing part turned out to be an edge case bug in Cassandra, which became apparent after the refactoring. Fixed. 5.0 patch PR - https://github.com/apache/cassandra/pull/3194 Fully green CI - [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=19428-5.0-3] I tested locally the patch on top of trunk - running the new fuzz tests added, they all pass too. I can propagate and run full CI when we confirm the final patch. [~maedhroz] , can you review? Also, I will rely on your expertise to advise whether to push this to 5.0 and trunk or only trunk. I am still learning that area of the code. Clean up KeyRangeIterator classes * replace KeyRangeConcatIterator's PriorityQeueu with List * remove KeyRangeIterator.current and simplify * remove injected exception and tests - not relevant to the current implementation * expand randomized testing * inline getCurrent() -> peek(); rename getCount to getMaxKeys * redefine skipTo contract to not return a value (which saves unnecessary work when skipTo is called multiple times in a row) * calling hasNext in skipTo is a pessimization; if the iterator is in DONE state, then skipTo will see it and avoid further effort; if it is not, then we are computing a next value that we're just going to throw away * Fix a bug in QueryController#getIndexResults error handling > Clean up KeyRangeIterator classes > - > > Key: CASSANDRA-19428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19428 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 5.0.x, 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > Remove KeyRangeIterator.current and simplify -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-107) Improve logging for slice restore task
[ https://issues.apache.org/jira/browse/CASSANDRASC-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRASC-107: --- Labels: low-hanging-fruit (was: ) > Improve logging for slice restore task > -- > > Key: CASSANDRASC-107 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-107 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement >Reporter: Yifan Cai >Assignee: Francisco Guerrero >Priority: Normal > Labels: low-hanging-fruit > > I want to propose logging improvements. > Add more logs to the individual steps during the restore task, i.e. > RestoreSliceTask and StorageClient. > In other places like retrying to poll the object existence, the stack trace > can be omitted, as it provides no additional knowledge than object not found. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-107) Improve logging for slice restore task
[ https://issues.apache.org/jira/browse/CASSANDRASC-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRASC-107: --- Change Category: Operability Complexity: Low Hanging Fruit Component/s: Rest API Status: Open (was: Triage Needed) > Improve logging for slice restore task > -- > > Key: CASSANDRASC-107 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-107 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Francisco Guerrero >Priority: Normal > Labels: low-hanging-fruit > > I want to propose logging improvements. > Add more logs to the individual steps during the restore task, i.e. > RestoreSliceTask and StorageClient. > In other places like retrying to poll the object existence, the stack trace > can be omitted, as it provides no additional knowledge than object not found. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829629#comment-17829629
]
Ariel Weisberg edited comment on CASSANDRA-19484 at 3/21/24 5:43 PM:
-
*edit* Removed a bunch of incorrectly generated dependencies with CVEs to
shorten the comment thread.
was (Author: aweisberg):
3.0
{noformat}
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-core/pom.xml:
CVE-2010-0538
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-mapping/pom.xml:
CVE-2010-0538
jackson-databind-2.13.2.2.jar: CVE-2023-35116, CVE-2022-42003, CVE-2022-42004
snappy-java-1.1.8.4.jar: CVE-2023-34455, CVE-2023-34454, CVE-2023-34453,
CVE-2023-43642
{noformat}
3.11
{noformat}
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-core/pom.xml:
CVE-2010-0538
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-mapping/pom.xml:
CVE-2010-0538
jackson-mapper-asl-1.9.2.jar: CVE-2017-7525, CVE-2019-10172
snakeyaml-1.11.jar: CVE-2017-18640
snappy-java-1.1.8.4.jar: CVE-2023-34455, CVE-2023-34454, CVE-2023-34453,
CVE-2023-43642
{noformat}
4.0
{noformat}
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-core/pom.xml:
CVE-2010-0538
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-mapping/pom.xml:
CVE-2010-0538
guava-18.0.jar: CVE-2018-10237
jackson-mapper-asl-1.9.2.jar: CVE-2017-7525, CVE-2019-10172
libthrift-0.9.2.jar: CVE-2016-5397, CVE-2018-1320, CVE-2015-3254,
CVE-2018-11798, CVE-2019-0205
netty-all-4.0.44.Final.jar: CVE-2019-16869, CVE-2019-20445, CVE-2019-20444,
CVE-2020-7238
snakeyaml-1.11.jar: CVE-2017-18640
snappy-java-1.1.8.4.jar: CVE-2023-34455, CVE-2023-34454, CVE-2023-34453,
CVE-2023-43642
thrift-server-0.3.7.jar: CVE-2016-5397, CVE-2015-3254, CVE-2019-0205
{noformat}
4.1
{noformat}
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-core/pom.xml:
CVE-2010-0538
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-mapping/pom.xml:
CVE-2010-0538
guava-18.0.jar: CVE-2018-10237
jackson-mapper-asl-1.9.2.jar: CVE-2017-7525, CVE-2019-10172
libthrift-0.9.2.jar: CVE-2016-5397, CVE-2018-1320, CVE-2015-3254,
CVE-2018-11798, CVE-2019-0205
netty-all-4.0.44.Final.jar: CVE-2019-16869, CVE-2019-20445, CVE-2019-20444,
CVE-2020-7238
snakeyaml-1.11.jar: CVE-2017-18640
snappy-java-1.1.8.4.jar: CVE-2023-34455, CVE-2023-34454, CVE-2023-34453,
CVE-2023-43642
thrift-server-0.3.7.jar: CVE-2016-5397, CVE-2015-3254, CVE-2019-0205
{noformat}
5.0
{noformat}
guava-18.0.jar: CVE-2020-8908, CVE-2018-10237, CVE-2023-2976
guava-27.0-jre.jar: CVE-2020-8908, CVE-2023-2976
jackson-mapper-asl-1.9.2.jar: CVE-2017-7525, CVE-2019-10172
libthrift-0.9.2.jar: CVE-2016-5397, CVE-2018-1320, CVE-2015-3254,
CVE-2018-11798, CVE-2019-0205
netty-all-4.0.44.Final.jar: CVE-2021-43797, CVE-2019-16869, CVE-2021-37136,
CVE-2021-37137, CVE-2019-20445, CVE-2019-20444, CVE-2021-21295, CVE-2023-34462,
CVE-2021-21290, CVE-2022-24823, CVE-2022-41881, CVE-2021-21409, CVE-2020-7238
netty-all-4.1.58.Final.jar: CVE-2021-43797, CVE-2021-37136, CVE-2021-37137,
CVE-2022-24823, CVE-2022-41881, CVE-2021-21295, CVE-2021-21409, CVE-2023-34462,
CVE-2021-21290
snakeyaml-1.11.jar: CVE-2017-18640
snappy-java-1.1.8.4.jar: CVE-2023-34455, CVE-2023-34454, CVE-2023-34453,
CVE-2023-43642
thrift-server-0.3.7.jar: CVE-2016-5397, CVE-2015-3254, CVE-2019-0205
{noformat}
trunk
{noformat}
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-core/pom.xml:
CVE-2010-0538
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-mapping/pom.xml:
CVE-2010-0538
guava-18.0.jar: CVE-2020-8908, CVE-2018-10237, CVE-2023-2976
guava-27.0-jre.jar: CVE-2020-8908, CVE-2023-2976
jackson-databind-2.13.2.2.jar: CVE-2022-42003, CVE-2022-42004
jackson-mapper-asl-1.9.2.jar: CVE-2017-7525, CVE-2019-10172
libthrift-0.9.2.jar: CVE-2016-5397, CVE-2018-1320, CVE-2015-3254,
CVE-2018-11798, CVE-2019-0205
netty-all-4.0.44.Final.jar: CVE-2021-43797, CVE-2019-16869, CVE-2021-37136,
CVE-2021-37137, CVE-2019-20445, CVE-2019-20444, CVE-2021-21295, CVE-2023-34462,
CVE-2021-21290, CVE-2022-24823, CVE-2022-41881, CVE-2021-21409, CVE-2020-7238
netty-all-4.1.58.Final.jar: CVE-2021-43797, CVE-2021-37136, CVE-2021-37137,
CVE-2022-24823, CVE-2022-41881, CVE-2021-21295, CVE-2021-21409, CVE-2023-34462,
CVE-2021-21290
snakeyaml-1.11.jar: CVE-2017-18640, CVE-2022-38752, CVE-2022-38751,
CVE-2022-38750, CVE-2022-41854, CVE-2022-25857, CVE-2022-38749, CVE-2022-1471
snakeyaml-1.26.jar: CVE-2022-38752, CVE-2022-38751, CVE-2022-38750,
CVE-2022-41854, CVE-2022-25857, CVE-2022-38749, CVE-2022-1471
snappy-java-1.1.8.4.jar: CVE-2023-
Re: [PR] CASSANDRA-19418 - Changes to report additional bulk analytics job stats for instrumentation [cassandra-analytics]
arjunashok commented on code in PR #41: URL: https://github.com/apache/cassandra-analytics/pull/41#discussion_r1534349869 ## cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/RingInstance.java: ## @@ -130,6 +130,22 @@ private void writeObject(ObjectOutputStream out) throws IOException out.writeUTF(ringEntry.fqdn()); out.writeUTF(ringEntry.status()); out.writeUTF(ringEntry.state()); +if (ringEntry.rack() != null) Review Comment: Addressed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-106) Add restore task watcher to report long running tasks
[ https://issues.apache.org/jira/browse/CASSANDRASC-106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRASC-106: --- Fix Version/s: 1.0 Source Control Link: https://github.com/apache/cassandra-sidecar/commit/c824d112de2c92d180a90a1830108d225f45dc23 Resolution: Fixed Status: Resolved (was: Ready to Commit) > Add restore task watcher to report long running tasks > - > > Key: CASSANDRASC-106 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-106 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Doug Rohrer >Priority: Normal > Labels: pull-request-available > Fix For: 1.0 > > > Having a watcher to report the long running restore slice task can provide > better insights. > The watcher can live inside the RestoreProcessor and periodically examine the > futures of the running tasks. > Ideally, it signals the task to log the current stack trace. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-sidecar) branch trunk updated: CASSANDRASC-106: Add restore task watcher to report long running tasks (#104)
This is an automated email from the ASF dual-hosted git repository. frankgh pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-sidecar.git The following commit(s) were added to refs/heads/trunk by this push: new c824d11 CASSANDRASC-106: Add restore task watcher to report long running tasks (#104) c824d11 is described below commit c824d112de2c92d180a90a1830108d225f45dc23 Author: Doug Rohrer <682821+jeetkund...@users.noreply.github.com> AuthorDate: Thu Mar 21 13:27:31 2024 -0400 CASSANDRASC-106: Add restore task watcher to report long running tasks (#104) Patch by Doug Rohrer; Reviewed by Yifan Cai, Francisco Guerrero for CASSANDRASC-106 --- CHANGES.txt| 3 +- checkstyle.xml | 4 +- .../cassandra/sidecar/client/RequestExecutor.java | 1 + .../selection/OrderedInstanceSelectionPolicy.java | 1 + .../sidecar/cluster/CQLSessionProviderImpl.java| 1 + .../sidecar/cluster/CassandraAdapterDelegate.java | 2 + .../sidecar/config/RestoreJobConfiguration.java| 5 ++ .../config/yaml/DriverConfigurationImpl.java | 3 + .../sidecar/config/yaml/JmxConfigurationImpl.java | 2 + .../config/yaml/RestoreJobConfigurationImpl.java | 32 ++ .../apache/cassandra/sidecar/db/RestoreSlice.java | 29 + .../sidecar/db/schema/RestoreSlicesSchema.java | 1 + .../cassandra/sidecar/restore/RestoreJobUtil.java | 10 ++- .../sidecar/restore/RestoreProcessor.java | 56 ++--- .../sidecar/restore/RestoreSliceHandler.java} | 28 + .../sidecar/restore/RestoreSliceTask.java | 66 +++- .../cassandra/sidecar/stats/RestoreJobStats.java | 23 ++- .../cluster/SidecarLoadBalancingPolicyTest.java| 1 + .../sidecar/testing/IntegrationTestModule.java | 2 + .../testing/SharedExecutorNettyOptions.java| 3 + .../cassandra/testing/SimpleCassandraVersion.java | 1 + .../cassandra/sidecar/HealthServiceSslTest.java| 1 + .../cassandra/sidecar/HealthServiceTest.java | 1 + .../org/apache/cassandra/sidecar/TestModule.java | 1 + .../sidecar/restore/RestoreProcessorTest.java | 72 +- .../sidecar/restore/RestoreSliceTaskTest.java | 38 +--- .../sstableuploads/BaseUploadsHandlerTest.java | 1 + .../sidecar/stats/TestRestoreJobStats.java | 9 +++ 28 files changed, 343 insertions(+), 54 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index e625cde..ce39b66 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,5 +1,6 @@ 1.0.0 - + * Add restore task watcher to report long running tasks (CASSANDRASC-106) * RestoreSliceTask could be stuck due to missing exception handling (CASSANDRASC-105) * Make hash algorithm implementation pluggable (CASSANDRASC-114) * Fix ClosedChannelException when downloading from S3 (CASSANDRASC-112) @@ -81,4 +82,4 @@ * Add integration tests task (CASSANDRA-15031) * Add support for SSL and bindable address (CASSANDRA-15030) * Autogenerate API docs for sidecar (CASSANDRA-15028) - * C* Management process (CASSANDRA-14395) + * C* Management process (CASSANDRA-14395) \ No newline at end of file diff --git a/checkstyle.xml b/checkstyle.xml index fd7c1b0..cea9a21 100644 --- a/checkstyle.xml +++ b/checkstyle.xml @@ -244,8 +244,8 @@ page at http://checkstyle.sourceforge.net/config.html --> - - + +
[jira] [Updated] (CASSANDRASC-106) Add restore task watcher to report long running tasks
[ https://issues.apache.org/jira/browse/CASSANDRASC-106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRASC-106: --- Status: Ready to Commit (was: Review In Progress) > Add restore task watcher to report long running tasks > - > > Key: CASSANDRASC-106 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-106 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Doug Rohrer >Priority: Normal > Labels: pull-request-available > > Having a watcher to report the long running restore slice task can provide > better insights. > The watcher can live inside the RestoreProcessor and periodically examine the > futures of the running tasks. > Ideally, it signals the task to log the current stack trace. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829641#comment-17829641
]
Brandon Williams commented on CASSANDRA-19471:
--
Thank you for further suggestions, [~blambov], I have implemented them
[here|https://github.com/driftx/cassandra/commit/3659730ac1cfcadada46677b19b5e714531a2984]
but it's not entirely clear to me if this is supposed to only fix the IAE (it
does) or that and also the commitlog being written when durable writes are
disabled (it does not.)
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, and the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829631#comment-17829631
]
Ariel Weisberg commented on CASSANDRA-19484:
PEBKAC, I didn't know you need a clean build before running the dependency
check.
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829630#comment-17829630
]
Brandon Williams edited comment on CASSANDRA-19484 at 3/21/24 4:29 PM:
---
Let's isolate to 3.0 first. Where is jackson-databind-2.13.2.2.jar or
snappy-java-1.1.8.4.jar present? 3.0 doesn't use databind, and snappy is at
1.1.10.4
thrift-server-0.3.7.jar in recent versions is clearly incorrect.
was (Author: brandon.williams):
Let's isolate to 3.0 first. Where is jackson-databind-2.13.2.2.jar or
snappy-java-1.1.8.4.jar present? 3.0 doesn't use databind, and snappy is at
1.1.10.4
thrift-server-0.3.7.jar in recent versions in clearly incorrect.
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19418 - Changes to report additional bulk analytics job stats for instrumentation [cassandra-analytics]
frankgh commented on code in PR #41:
URL:
https://github.com/apache/cassandra-analytics/pull/41#discussion_r1534250572
##
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/RingInstance.java:
##
@@ -130,6 +130,22 @@ private void writeObject(ObjectOutputStream out) throws
IOException
out.writeUTF(ringEntry.fqdn());
out.writeUTF(ringEntry.status());
out.writeUTF(ringEntry.state());
+if (ringEntry.rack() != null)
Review Comment:
I think this is not sufficient, I pinged you offline for a suggestion
##
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/common/stats/JobStatsImpl.java:
##
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.cassandra.spark.common.stats;
+
+import java.util.HashMap;
+import java.util.Map;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Implementation of {@link JobStats} that is used to record stats through the
course of the
+ * Spark job execution and publish them. This implementation logs the stats
when published.
+ */
+public class JobStatsImpl implements JobStats
+{
+private static final Logger LOGGER =
LoggerFactory.getLogger(JobStatsImpl.class);
+private final transient Map jobStats = new HashMap<>();
+public void recordJobStats(Map stats)
+{
+jobStats.putAll(stats);
+}
+
+public void publishJobStats()
Review Comment:
can we add override annotations?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829630#comment-17829630
]
Brandon Williams edited comment on CASSANDRA-19484 at 3/21/24 4:28 PM:
---
Let's isolate to 3.0 first. Where is jackson-databind-2.13.2.2.jar or
snappy-java-1.1.8.4.jar present? 3.0 doesn't use databind, and snappy is at
1.1.10.4
thrift-server-0.3.7.jar in recent versions in clearly incorrect.
was (Author: brandon.williams):
Let's isolate to 3.0 first. Where is jackson-databind-2.13.2.2.jar or
snappy-java-1.1.8.4.jar present? 3.0 doesn't use databind, and snappy is at
1.1.10.4
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829630#comment-17829630
]
Brandon Williams commented on CASSANDRA-19484:
--
Let's isolate to 3.0 first. Where is jackson-databind-2.13.2.2.jar or
snappy-java-1.1.8.4.jar present? 3.0 doesn't use databind, and snappy is at
1.1.10.4
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19487) Rework json testing
[ https://issues.apache.org/jira/browse/CASSANDRA-19487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19487: - Change Category: Semantic Complexity: Normal Component/s: Test/dtest/python Fix Version/s: 3.0.x 3.11.x 4.0.x 4.1.x 5.0.x 5.x Status: Open (was: Triage Needed) > Rework json testing > --- > > Key: CASSANDRA-19487 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19487 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/python >Reporter: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > The only real json testing we have is in [this > dtest|https://github.com/apache/cassandra-dtest/blob/trunk/json_test.py] > which is full of magic like running from the tests' docstrings which makes > other things very difficult, like adding any kind of version-based variation. > I propose rewriting this, either as sane python or a java dtest. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19487) Rework json testing
Brandon Williams created CASSANDRA-19487: Summary: Rework json testing Key: CASSANDRA-19487 URL: https://issues.apache.org/jira/browse/CASSANDRA-19487 Project: Cassandra Issue Type: Improvement Reporter: Brandon Williams The only real json testing we have is in [this dtest|https://github.com/apache/cassandra-dtest/blob/trunk/json_test.py] which is full of magic like running from the tests' docstrings which makes other things very difficult, like adding any kind of version-based variation. I propose rewriting this, either as sane python or a java dtest. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19418 - Changes to report additional bulk analytics job stats for instrumentation [cassandra-analytics]
arjunashok commented on code in PR #41: URL: https://github.com/apache/cassandra-analytics/pull/41#discussion_r1534232804 ## cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/BulkWriterContext.java: ## @@ -21,7 +21,9 @@ import java.io.Serializable; -public interface BulkWriterContext extends Serializable +import org.apache.cassandra.spark.common.JobStats; + +public interface BulkWriterContext extends Serializable, JobStats Review Comment: Makes sense. Incorporated this change -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829629#comment-17829629
]
Ariel Weisberg commented on CASSANDRA-19484:
3.0
{noformat}
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-core/pom.xml:
CVE-2010-0538
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-mapping/pom.xml:
CVE-2010-0538
jackson-databind-2.13.2.2.jar: CVE-2023-35116, CVE-2022-42003, CVE-2022-42004
snappy-java-1.1.8.4.jar: CVE-2023-34455, CVE-2023-34454, CVE-2023-34453,
CVE-2023-43642
{noformat}
3.11
{noformat}
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-core/pom.xml:
CVE-2010-0538
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-mapping/pom.xml:
CVE-2010-0538
jackson-mapper-asl-1.9.2.jar: CVE-2017-7525, CVE-2019-10172
snakeyaml-1.11.jar: CVE-2017-18640
snappy-java-1.1.8.4.jar: CVE-2023-34455, CVE-2023-34454, CVE-2023-34453,
CVE-2023-43642
{noformat}
4.0
{noformat}
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-core/pom.xml:
CVE-2010-0538
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-mapping/pom.xml:
CVE-2010-0538
guava-18.0.jar: CVE-2018-10237
jackson-mapper-asl-1.9.2.jar: CVE-2017-7525, CVE-2019-10172
libthrift-0.9.2.jar: CVE-2016-5397, CVE-2018-1320, CVE-2015-3254,
CVE-2018-11798, CVE-2019-0205
netty-all-4.0.44.Final.jar: CVE-2019-16869, CVE-2019-20445, CVE-2019-20444,
CVE-2020-7238
snakeyaml-1.11.jar: CVE-2017-18640
snappy-java-1.1.8.4.jar: CVE-2023-34455, CVE-2023-34454, CVE-2023-34453,
CVE-2023-43642
thrift-server-0.3.7.jar: CVE-2016-5397, CVE-2015-3254, CVE-2019-0205
{noformat}
4.1
{noformat}
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-core/pom.xml:
CVE-2010-0538
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-mapping/pom.xml:
CVE-2010-0538
guava-18.0.jar: CVE-2018-10237
jackson-mapper-asl-1.9.2.jar: CVE-2017-7525, CVE-2019-10172
libthrift-0.9.2.jar: CVE-2016-5397, CVE-2018-1320, CVE-2015-3254,
CVE-2018-11798, CVE-2019-0205
netty-all-4.0.44.Final.jar: CVE-2019-16869, CVE-2019-20445, CVE-2019-20444,
CVE-2020-7238
snakeyaml-1.11.jar: CVE-2017-18640
snappy-java-1.1.8.4.jar: CVE-2023-34455, CVE-2023-34454, CVE-2023-34453,
CVE-2023-43642
thrift-server-0.3.7.jar: CVE-2016-5397, CVE-2015-3254, CVE-2019-0205
{noformat}
5.0
{noformat}
guava-18.0.jar: CVE-2020-8908, CVE-2018-10237, CVE-2023-2976
guava-27.0-jre.jar: CVE-2020-8908, CVE-2023-2976
jackson-mapper-asl-1.9.2.jar: CVE-2017-7525, CVE-2019-10172
libthrift-0.9.2.jar: CVE-2016-5397, CVE-2018-1320, CVE-2015-3254,
CVE-2018-11798, CVE-2019-0205
netty-all-4.0.44.Final.jar: CVE-2021-43797, CVE-2019-16869, CVE-2021-37136,
CVE-2021-37137, CVE-2019-20445, CVE-2019-20444, CVE-2021-21295, CVE-2023-34462,
CVE-2021-21290, CVE-2022-24823, CVE-2022-41881, CVE-2021-21409, CVE-2020-7238
netty-all-4.1.58.Final.jar: CVE-2021-43797, CVE-2021-37136, CVE-2021-37137,
CVE-2022-24823, CVE-2022-41881, CVE-2021-21295, CVE-2021-21409, CVE-2023-34462,
CVE-2021-21290
snakeyaml-1.11.jar: CVE-2017-18640
snappy-java-1.1.8.4.jar: CVE-2023-34455, CVE-2023-34454, CVE-2023-34453,
CVE-2023-43642
thrift-server-0.3.7.jar: CVE-2016-5397, CVE-2015-3254, CVE-2019-0205
{noformat}
trunk
{noformat}
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-core/pom.xml:
CVE-2010-0538
cassandra-client-4.0.35.jar/META-INF/maven/com.apple.pie.cassandra/pie-cassandra-driver-mapping/pom.xml:
CVE-2010-0538
guava-18.0.jar: CVE-2020-8908, CVE-2018-10237, CVE-2023-2976
guava-27.0-jre.jar: CVE-2020-8908, CVE-2023-2976
jackson-databind-2.13.2.2.jar: CVE-2022-42003, CVE-2022-42004
jackson-mapper-asl-1.9.2.jar: CVE-2017-7525, CVE-2019-10172
libthrift-0.9.2.jar: CVE-2016-5397, CVE-2018-1320, CVE-2015-3254,
CVE-2018-11798, CVE-2019-0205
netty-all-4.0.44.Final.jar: CVE-2021-43797, CVE-2019-16869, CVE-2021-37136,
CVE-2021-37137, CVE-2019-20445, CVE-2019-20444, CVE-2021-21295, CVE-2023-34462,
CVE-2021-21290, CVE-2022-24823, CVE-2022-41881, CVE-2021-21409, CVE-2020-7238
netty-all-4.1.58.Final.jar: CVE-2021-43797, CVE-2021-37136, CVE-2021-37137,
CVE-2022-24823, CVE-2022-41881, CVE-2021-21295, CVE-2021-21409, CVE-2023-34462,
CVE-2021-21290
snakeyaml-1.11.jar: CVE-2017-18640, CVE-2022-38752, CVE-2022-38751,
CVE-2022-38750, CVE-2022-41854, CVE-2022-25857, CVE-2022-38749, CVE-2022-1471
snakeyaml-1.26.jar: CVE-2022-38752, CVE-2022-38751, CVE-2022-38750,
CVE-2022-41854, CVE-2022-25857, CVE-2022-38749, CVE-2022-1471
snappy-java-1.1.8.4.jar: CVE-2023-34455, CVE-2023-34454, CVE-2023-34453,
CVE-2023-43642
thrift-server-0.3.7.jar: CVE-2016-5397, CVE-2015-3254, CVE-2019-0205
{noformat}
> Add support for providing nvdDatafeedUrl
Re: [PR] CASSANDRA-19418 - Changes to report additional bulk analytics job stats for instrumentation [cassandra-analytics]
arjunashok commented on code in PR #41: URL: https://github.com/apache/cassandra-analytics/pull/41#discussion_r1534231920 ## cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/CassandraBulkWriterContext.java: ## @@ -56,10 +57,12 @@ public class CassandraBulkWriterContext implements BulkWriterContext, KryoSerial private final CassandraClusterInfo clusterInfo; private final SchemaInfo schemaInfo; -private CassandraBulkWriterContext(@NotNull BulkSparkConf conf, - @NotNull CassandraClusterInfo clusterInfo, - @NotNull StructType dfSchema, - SparkContext sparkContext) +private final Map jobStats = new HashMap<>(); Review Comment: Addressed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19418 - Changes to report additional bulk analytics job stats for instrumentation [cassandra-analytics]
arjunashok commented on code in PR #41: URL: https://github.com/apache/cassandra-analytics/pull/41#discussion_r1534231627 ## cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/WriteResult.java: ## @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.cassandra.spark.bulkwriter; + +import java.util.List; + +import scala.Serializable; Review Comment: Correct. fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17808) Optionally avoid hint transfer during decommission
[
https://issues.apache.org/jira/browse/CASSANDRA-17808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829621#comment-17829621
]
Paul Chandler commented on CASSANDRA-17808:
---
[~smiklosovic] and [~maedhroz] I can see that there is a PR for this change in
4.1, but it has not been committed. Are there any plans to do that.?
We would like to port it to 4.0 as we are having the same problem for a large
4.0 cluster, so we would be able to provide a patch for 4.0, but I wanted to
check there were no issues with the 4.1 version first that are stopping it
being committed?
> Optionally avoid hint transfer during decommission
> --
>
> Key: CASSANDRA-17808
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17808
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-alpha1, 5.0
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Both because they aren’t strictly necessary to maintain consistency, and
> because throttling induced by their rate-limiter (see
> {{hinted_handoff_throttle}}) may stall progress, transferring hints during
> decommission (specifically unbootstrap) rather than just pausing, disabling,
> and truncating them probably doesn’t make sense. The only other concern would
> be the BatchLog, which nominally depends on hint delivery to maintain its
> "guarantees". However, during BatchLog replay on unbootstrap,
> {{ReplayingBatch}} ignores batches older the gcgs anyway.
> Here's a proposal from [~aleksey] that might strike a reasonable balance:
> 1.) We continue to transfer hints by default during decommission, but at a
> higher rate. We could, for instance, stop having {{DispatchHintsTask}} divide
> its effective rate by the number of nodes in the cluster.
> {noformat}
> int nodesCount = Math.max(1,
> StorageService.instance.getTokenMetadata().getAllEndpoints().size() - 1);
> double throttleInBytes = DatabaseDescriptor.getHintedHandoffThrottleInKiB() *
> 1024.0 / nodesCount;
> this.rateLimiter = RateLimiter.create(throttleInBytes == 0 ? Double.MAX_VALUE
> : throttleInBytes);
> {noformat}
> 2.) We provide an option to simply avoid transferring hints during
> unbootstrap. Even this would only take the BatchLog from "best effort" to
> "slightly less effort" ;)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829620#comment-17829620
]
Brandon Williams commented on CASSANDRA-19484:
--
Is that running across all branches? We have per-branch suppressions in
`.build/owasp/dependency-check-suppressions.xml`
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[ https://issues.apache.org/jira/browse/CASSANDRA-19484 ]
Brandon Williams deleted comment on CASSANDRA-19484:
--
was (Author: brandon.williams):
I think they are present
[here|https://github.com/apache/cassandra/blob/trunk/.build/owasp/dependency-check-suppressions.xml]?
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829619#comment-17829619
]
Brandon Williams commented on CASSANDRA-19484:
--
I think they are present
[here|https://github.com/apache/cassandra/blob/trunk/.build/owasp/dependency-check-suppressions.xml]?
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829617#comment-17829617
]
Ariel Weisberg commented on CASSANDRA-19484:
[~bereng] finished updating. If you are still +1 on the new version I will
merge.
I noticed there are lot of unsuppressed CVEs.
{code:java}
guava-18.0.jar: CVE-2020-8908, CVE-2018-10237, CVE-2023-2976
guava-27.0-jre.jar: CVE-2020-8908, CVE-2023-2976
jackson-databind-2.13.2.2.jar: CVE-2022-42003, CVE-2022-42004
jackson-mapper-asl-1.9.2.jar: CVE-2017-7525, CVE-2019-10172
libthrift-0.9.2.jar: CVE-2016-5397, CVE-2018-1320, CVE-2015-3254,
CVE-2018-11798, CVE-2019-0205
netty-all-4.0.44.Final.jar: CVE-2021-43797, CVE-2019-16869, CVE-2021-37136,
CVE-2021-37137, CVE-2019-20445, CVE-2019-20444, CVE-2021-21295, CVE-2023-34462,
CVE-2021-21290, CVE-2022-24823, CVE-2022-41881, CVE-2021-21409, CVE-2020-7238
netty-all-4.1.58.Final.jar: CVE-2021-43797, CVE-2021-37136, CVE-2021-37137,
CVE-2022-24823, CVE-2022-41881, CVE-2021-21295, CVE-2021-21409, CVE-2023-34462,
CVE-2021-21290
snakeyaml-1.11.jar: CVE-2017-18640, CVE-2022-38752, CVE-2022-38751,
CVE-2022-38750, CVE-2022-41854, CVE-2022-25857, CVE-2022-38749, CVE-2022-1471
snakeyaml-1.26.jar: CVE-2022-38752, CVE-2022-38751, CVE-2022-38750,
CVE-2022-41854, CVE-2022-25857, CVE-2022-38749, CVE-2022-1471
thrift-server-0.3.7.jar: CVE-2016-5397, CVE-2015-3254, CVE-2019-0205
{code}
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829608#comment-17829608
]
Ariel Weisberg commented on CASSANDRA-19484:
Ah, I tried that and it didn't work because in `ant` when you reference a
property that isn't set it doesn't resolve to the empty string it resolves to
the name of the property! Will update.
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19341) Relation and Restriction hierachies are too complex and error prone
[
https://issues.apache.org/jira/browse/CASSANDRA-19341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19341:
-
Description:
The {{Relation}} and {{Restriction}} hierarchy have been designed when C* was
only supporting a limited amount of operators and columns expressions (single
column, multi-column and token expressions). Over time they have grown in
complexity making the code harder to understand and modify and error prone.
Their design is also resulting in unnecessary limitations that could be easily
lifted, like the ability to accept different predicates on the same column.
Today adding a new operator requires the addition of a lot of glue code and
surgical changes accross the CQL layer. Making patch for features such as
CASSANDRA-18584 much complex than it should be.
The goal of this ticket is to simplify the {{Relation}} and {{Restriction}}
hierarchies and modify operator class so that adding new operators requires
only changes to the {{Operator}} class and ANTLR file.
was:
The {{Relation}} and {{Restriction}} hierarchy have been designed when C* was
only supporting a limited amount of operators and columns expressions (single
column, multi-column and token expressions). Over time they have grown in
complexity making the code harder to understand and modify and error prone.
Their design is also resulting in unnecessary limitations that could be easily
lifted, like the ability to accept different predicates on the same column.
Today adding a new operator requires the addition of a lot of glue code and
chirurgical changes accross the CQL layer. Making patch for features such as
CASSANDRA-18584 much complex than it should be.
The goal of this ticket is to simplify the {{Relation}} and {{Restriction}}
hierarchies and modify operator class so that adding new operators requires
only changes to the {{Operator}} class and ANTLR file.
> Relation and Restriction hierachies are too complex and error prone
> ---
>
> Key: CASSANDRA-19341
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19341
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL/Interpreter
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{Relation}} and {{Restriction}} hierarchy have been designed when C* was
> only supporting a limited amount of operators and columns expressions (single
> column, multi-column and token expressions). Over time they have grown in
> complexity making the code harder to understand and modify and error prone.
> Their design is also resulting in unnecessary limitations that could be
> easily lifted, like the ability to accept different predicates on the same
> column.
> Today adding a new operator requires the addition of a lot of glue code and
> surgical changes accross the CQL layer. Making patch for features such as
> CASSANDRA-18584 much complex than it should be.
> The goal of this ticket is to simplify the {{Relation}} and {{Restriction}}
> hierarchies and modify operator class so that adding new operators requires
> only changes to the {{Operator}} class and ANTLR file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19341) Relation and Restriction hierachies are too complex and error prone
[
https://issues.apache.org/jira/browse/CASSANDRA-19341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-19341:
---
Change Category: Code Clarity
Complexity: Normal
Status: Open (was: Triage Needed)
> Relation and Restriction hierachies are too complex and error prone
> ---
>
> Key: CASSANDRA-19341
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19341
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL/Interpreter
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{Relation}} and {{Restriction}} hierarchy have been designed when C* was
> only supporting a limited amount of operators and columns expressions (single
> column, multi-column and token expressions). Over time they have grown in
> complexity making the code harder to understand and modify and error prone.
> Their design is also resulting in unnecessary limitations that could be
> easily lifted, like the ability to accept different predicates on the same
> column.
> Today adding a new operator requires the addition of a lot of glue code and
> chirurgical changes accross the CQL layer. Making patch for features such as
> CASSANDRA-18584 much complex than it should be.
> The goal of this ticket is to simplify the {{Relation}} and {{Restriction}}
> hierarchies and modify operator class so that adding new operators requires
> only changes to the {{Operator}} class and ANTLR file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19352) 4.x Java driver support for native_port_ssl and native_transport_port_ssl
[ https://issues.apache.org/jira/browse/CASSANDRA-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bret McGuire updated CASSANDRA-19352: - Component/s: Client/java-driver > 4.x Java driver support for native_port_ssl and native_transport_port_ssl > - > > Key: CASSANDRA-19352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19352 > Project: Cassandra > Issue Type: Improvement > Components: Client/java-driver >Reporter: absurdfarce#1 >Priority: Normal > > DSE 6.8 added a "native_transport_port_ssl" column to peers_v2 to indicate > when peers were making use of "native_transport_port_ssl" in cassandra.yaml. > Similar functionality (with slightly different column names) was brought to > OSS Cassandra with CASSANDRA-16999. 3.x Java driver support for these > columns has been added (or is in the process of being added) in > [JAVA-2967|https://datastax-oss.atlassian.net/browse/JAVA-2967]. This ticket > represents the work to implement similar functionality for the 4.x Java > driver. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-19352) 4.x Java driver support for native_port_ssl and native_transport_port_ssl
[ https://issues.apache.org/jira/browse/CASSANDRA-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bret McGuire reassigned CASSANDRA-19352: Assignee: Bret McGuire > 4.x Java driver support for native_port_ssl and native_transport_port_ssl > - > > Key: CASSANDRA-19352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19352 > Project: Cassandra > Issue Type: Improvement > Components: Client/java-driver >Reporter: absurdfarce#1 >Assignee: Bret McGuire >Priority: Normal > > DSE 6.8 added a "native_transport_port_ssl" column to peers_v2 to indicate > when peers were making use of "native_transport_port_ssl" in cassandra.yaml. > Similar functionality (with slightly different column names) was brought to > OSS Cassandra with CASSANDRA-16999. 3.x Java driver support for these > columns has been added (or is in the process of being added) in > [JAVA-2967|https://datastax-oss.atlassian.net/browse/JAVA-2967]. This ticket > represents the work to implement similar functionality for the 4.x Java > driver. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19352) 4.x Java driver support for native_port_ssl and native_transport_port_ssl
[ https://issues.apache.org/jira/browse/CASSANDRA-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829592#comment-17829592 ] Bret McGuire commented on CASSANDRA-19352: -- Per the discussion in CASSANDRA-16999 we won't be adding support for "native_transport_port_ssl" to OSS Cassandra. The Java driver does still need to add this support in order to work properly with DSE 6.8+, however, so this ticket will address that change. > 4.x Java driver support for native_port_ssl and native_transport_port_ssl > - > > Key: CASSANDRA-19352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19352 > Project: Cassandra > Issue Type: Improvement >Reporter: absurdfarce#1 >Priority: Normal > > DSE 6.8 added a "native_transport_port_ssl" column to peers_v2 to indicate > when peers were making use of "native_transport_port_ssl" in cassandra.yaml. > Similar functionality (with slightly different column names) was brought to > OSS Cassandra with CASSANDRA-16999. 3.x Java driver support for these > columns has been added (or is in the process of being added) in > [JAVA-2967|https://datastax-oss.atlassian.net/browse/JAVA-2967]. This ticket > represents the work to implement similar functionality for the 4.x Java > driver. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19477) Do not go to disk to get HintsStore.getTotalFileSize
[
https://issues.apache.org/jira/browse/CASSANDRA-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829583#comment-17829583
]
Stefan Miklosovic commented on CASSANDRA-19477:
---
the work and reviews are done, I just need to test this and Jon should
perf-test it. Ideally this should be committed very early next week.
> Do not go to disk to get HintsStore.getTotalFileSize
>
>
> Key: CASSANDRA-19477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19477
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
>Reporter: Jon Haddad
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.1.x, 5.0-rc, 5.x
>
> Attachments: flamegraph.cpu.html
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> When testing a cluster with more requests than it could handle, I noticed
> significant CPU time (25%) spent in HintsStore.getTotalFileSize. Here's what
> I'm seeing from profiling:
> 10% of CPU time spent in HintsDescriptor.fileName which only does this:
>
> {noformat}
> return String.format("%s-%s-%s.hints", hostId, timestamp, version);{noformat}
> At a bare minimum here we should create this string up front with the host
> and version and eliminate 2 of the 3 substitutions, but I think it's probably
> faster to use a StringBuilder and avoid the underlying regular expression
> altogether.
> 12% of the time is spent in org.apache.cassandra.io.util.File.length. It
> looks like this is called once for each hint file on disk for each host we're
> hinting to. In the case of an overloaded cluster, this is significant. It
> would be better if we were to track the file size in memory for each hint
> file and reference that rather than go to the filesystem.
> These fairly small changes should make Cassandra more reliable when under
> load spikes.
> CPU Flame graph attached.
> I only tested this in 4.1 but it looks like this is present up to trunk.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19463) Test failure: org.apache.cassandra.fuzz.ring.ConsistentBootstrapTest.coordinatorIsBehindTest
[
https://issues.apache.org/jira/browse/CASSANDRA-19463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-19463:
Resolution: Duplicate
Status: Resolved (was: Triage Needed)
> Test failure:
> org.apache.cassandra.fuzz.ring.ConsistentBootstrapTest.coordinatorIsBehindTest
>
>
> Key: CASSANDRA-19463
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19463
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Priority: Normal
>
> Looks like this was broken by CASSANDRA-18275
> Timeouts in the log and then:
> {code}
> java.lang.IllegalStateException: Can't use shutdown instances, delegate is
> null
> at
> org.apache.cassandra.distributed.impl.AbstractCluster$Wrapper.delegate(AbstractCluster.java:283)
> at
> org.apache.cassandra.distributed.impl.DelegatingInvokableInstance.transfer(DelegatingInvokableInstance.java:49)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runsOnInstance(IInvokableInstance.java:45)
> at
> org.apache.cassandra.distributed.api.IInvokableInstance.runOnInstance(IInvokableInstance.java:46)
> at
> org.apache.cassandra.distributed.shared.ClusterUtils.unpauseCommits(ClusterUtils.java:548)
> at
> org.apache.cassandra.fuzz.ring.ConsistentBootstrapTest.coordinatorIsBehindTest(ConsistentBootstrapTest.java:227)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
> at
> com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
> at
> com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
> at
> com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
> at
> com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
> at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19461) SAI does not index empty bytes even for types that allow empty bytes as a valid input
[
https://issues.apache.org/jira/browse/CASSANDRA-19461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829558#comment-17829558
]
Ekaterina Dimitrova commented on CASSANDRA-19461:
-
[~dcapwell] already reviewed it so I will move away :)
> SAI does not index empty bytes even for types that allow empty bytes as a
> valid input
> -
>
> Key: CASSANDRA-19461
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19461
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Attachments: ci_summary.html, result_details.tar.gz
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> This is easy to reproduce with a test that looks something like this:
> {noformat}
> @Test
> public void testEmptyString()
> {
> createTable("CREATE TABLE %s (k TEXT PRIMARY KEY, v text)");
> createIndex(String.format(CREATE_INDEX_TEMPLATE, 'v'));
> execute("INSERT INTO %s (k, v) VALUES ('0', '')");
> execute("INSERT INTO %s (k) VALUES ('1')");
>
> // flush(); < there is not always a memtable index involved, a fix
> will have to pay attention to this
> List rows = executeNet("SELECT * FROM %s WHERE v = ''").all();
> assertEquals(1, rows.size()); <— FAILS! No matches...
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19461) SAI does not index empty bytes even for types that allow empty bytes as a valid input
[
https://issues.apache.org/jira/browse/CASSANDRA-19461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-19461:
Reviewers: David Capwell (was: David Capwell, Ekaterina Dimitrova)
> SAI does not index empty bytes even for types that allow empty bytes as a
> valid input
> -
>
> Key: CASSANDRA-19461
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19461
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Attachments: ci_summary.html, result_details.tar.gz
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> This is easy to reproduce with a test that looks something like this:
> {noformat}
> @Test
> public void testEmptyString()
> {
> createTable("CREATE TABLE %s (k TEXT PRIMARY KEY, v text)");
> createIndex(String.format(CREATE_INDEX_TEMPLATE, 'v'));
> execute("INSERT INTO %s (k, v) VALUES ('0', '')");
> execute("INSERT INTO %s (k) VALUES ('1')");
>
> // flush(); < there is not always a memtable index involved, a fix
> will have to pay attention to this
> List rows = executeNet("SELECT * FROM %s WHERE v = ''").all();
> assertEquals(1, rows.size()); <— FAILS! No matches...
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19185) Vector search tests are failing on recall accuracy
[ https://issues.apache.org/jira/browse/CASSANDRA-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-19185: - Fix Version/s: 5.0-beta2 (was: 5.x) (was: 5.0.x) Since Version: 5.0-alpha1 Source Control Link: https://github.com/apache/cassandra/commit/7aab61b06357ce0b59977715f82fed1ad24474b4 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as https://github.com/apache/cassandra/commit/7aab61b06357ce0b59977715f82fed1ad24474b4 > Vector search tests are failing on recall accuracy > -- > > Key: CASSANDRA-19185 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19185 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Normal > Fix For: 5.0-beta2 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Vector tests are failing randomly because they do not meet recall assertion > values. Currently, the following tests have been reported as failing: > VectorSegmentationTest.testMultipleSegmentsForCompaction > VectorDistributedTest.rangeRestrictedTest > VectorDistributedTest.testPartitionRestrictedVectorSearch > Since the vector searches are approximate and the vectors used in the tests > are random, it is unlikely that they will always meet a high recall. The > recall assertions are looking for recall values of 0.9 and above. Part of > this issue is related to the use of random values in the vectors being > tested. We have seen, with other tests, that the vector search performs > better with non-random generated datasets like the Glove datasets. As such, > there are the following available to fix these tests. > # Downgrade the assertions to a value that is likely to always pass. The > problem is that there is no guarantee that a test will always pass any recall > value we give it. > # Use generated datasets for these tests to see if that improves the recall > results. > # Remove the recall assertions unless they are specifically asked for. We > could use a system property to enable recall testing for targeted vector > testing. > I don't think option 1 is a viable long-term solution as we can never be > certain that it will always work. Option 2 has more promise but it could > still result in failures because of the approximate nature of the vector > searches. As such, option 3 seems the only viable solution here but means > that, in most cases, we are only really testing that we are returning results > from the search, not how accurate those results are. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams reassigned CASSANDRA-19471:
Assignee: Brandon Williams
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, and the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-5.0 updated (c3a18e36d1 -> 7aab61b063)
This is an automated email from the ASF dual-hosted git repository. mikea pushed a change to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from c3a18e36d1 Minor bugs in generate.sh -d add 7aab61b063 Use glove vectors instead of random vectors in vector tests - avoid randomisation to make tests more consistent - use heap_buffers for VectorDistributedTest for consistency with other tests No new revisions were added by this update. Summary of changes: .../test/sai/VectorDistributedTest.java| 23 - .../cassandra/index/sai/cql/VectorLocalTest.java | 17 + .../index/sai/cql/VectorSegmentationTest.java | 29 +++--- .../cassandra/index/sai/cql/VectorTester.java | 10 .../index/sai/cql/VectorUpdateDeleteTest.java | 1 - .../apache/cassandra/index/sai/utils/Glove.java| 5 6 files changed, 35 insertions(+), 50 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. mikea pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit e6df9f0e3e23448cca04e48a7ba632358bb40902 Merge: 32b146dc63 7aab61b063 Author: Mike Adamson AuthorDate: Thu Mar 21 13:00:39 2024 + Merge branch 'cassandra-5.0' into trunk .../test/sai/VectorDistributedTest.java| 23 - .../cassandra/index/sai/cql/VectorLocalTest.java | 17 + .../index/sai/cql/VectorSegmentationTest.java | 29 +++--- .../cassandra/index/sai/cql/VectorTester.java | 10 .../index/sai/cql/VectorUpdateDeleteTest.java | 1 - .../apache/cassandra/index/sai/utils/Glove.java| 5 6 files changed, 35 insertions(+), 50 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch trunk updated (32b146dc63 -> e6df9f0e3e)
This is an automated email from the ASF dual-hosted git repository. mikea pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 32b146dc63 Merge branch 'cassandra-5.0' into trunk add 7aab61b063 Use glove vectors instead of random vectors in vector tests - avoid randomisation to make tests more consistent - use heap_buffers for VectorDistributedTest for consistency with other tests new e6df9f0e3e Merge branch 'cassandra-5.0' into trunk The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../test/sai/VectorDistributedTest.java| 23 - .../cassandra/index/sai/cql/VectorLocalTest.java | 17 + .../index/sai/cql/VectorSegmentationTest.java | 29 +++--- .../cassandra/index/sai/cql/VectorTester.java | 10 .../index/sai/cql/VectorUpdateDeleteTest.java | 1 - .../apache/cassandra/index/sai/utils/Glove.java| 5 6 files changed, 35 insertions(+), 50 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19486) Add method for sizes of hints and hints per node in HintsServiceMBean
Stefan Miklosovic created CASSANDRA-19486: - Summary: Add method for sizes of hints and hints per node in HintsServiceMBean Key: CASSANDRA-19486 URL: https://issues.apache.org/jira/browse/CASSANDRA-19486 Project: Cassandra Issue Type: Improvement Components: Consistency/Hints Reporter: Stefan Miklosovic Assignee: Stefan Miklosovic I am happy to close this if somebody shows me how to get total size of all hints or all hints per particular node via JMX. I could find is StorageMetrics.totalHints but that is how many hints there are, not their sizes. We also have org.apache.cassandra.metrics:type=HintedHandOffManager name= org.apache.cassandra.metrics:type=HintsService name= But that is again showing other metrics not sizes. I would add two methods into HintsServiceMBeans returning this. Seems to be very easy to do once we do CASSANDRA-19477. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19486) Add method for sizes of hints and hints per node in HintsServiceMBean
[ https://issues.apache.org/jira/browse/CASSANDRA-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-19486: -- Description: I am happy to close this if somebody shows me how to get total size of all hints or all hints per particular node via JMX. I could find StorageMetrics.totalHints but that is how many hints there are, not their sizes. We also have org.apache.cassandra.metrics:type=HintedHandOffManager name= org.apache.cassandra.metrics:type=HintsService name= But that is again showing other metrics not sizes. I would add two methods into HintsServiceMBeans returning this. Seems to be very easy to do once we do CASSANDRA-19477. was: I am happy to close this if somebody shows me how to get total size of all hints or all hints per particular node via JMX. I could find is StorageMetrics.totalHints but that is how many hints there are, not their sizes. We also have org.apache.cassandra.metrics:type=HintedHandOffManager name= org.apache.cassandra.metrics:type=HintsService name= But that is again showing other metrics not sizes. I would add two methods into HintsServiceMBeans returning this. Seems to be very easy to do once we do CASSANDRA-19477. > Add method for sizes of hints and hints per node in HintsServiceMBean > - > > Key: CASSANDRA-19486 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19486 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Hints >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 5.x > > > I am happy to close this if somebody shows me how to get total size of all > hints or all hints per particular node via JMX. > I could find StorageMetrics.totalHints but that is how many hints there are, > not their sizes. We also have > org.apache.cassandra.metrics:type=HintedHandOffManager name= > org.apache.cassandra.metrics:type=HintsService name= > But that is again showing other metrics not sizes. > I would add two methods into HintsServiceMBeans returning this. Seems to be > very easy to do once we do CASSANDRA-19477. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19486) Add method for sizes of hints and hints per node in HintsServiceMBean
[ https://issues.apache.org/jira/browse/CASSANDRA-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-19486: -- Description: I am happy to close this if somebody shows me how to get total size of all hints or all hints per particular node via JMX. I could find StorageMetrics.totalHints but that is how many hints there are, not their sizes. We also have org.apache.cassandra.metrics:type=HintedHandOffManager name= org.apache.cassandra.metrics:type=HintsService name= But that is again showing other metrics not sizes. I would add two methods into HintsServiceMBean returning this. Seems to be very easy to do once we do CASSANDRA-19477. was: I am happy to close this if somebody shows me how to get total size of all hints or all hints per particular node via JMX. I could find StorageMetrics.totalHints but that is how many hints there are, not their sizes. We also have org.apache.cassandra.metrics:type=HintedHandOffManager name= org.apache.cassandra.metrics:type=HintsService name= But that is again showing other metrics not sizes. I would add two methods into HintsServiceMBeans returning this. Seems to be very easy to do once we do CASSANDRA-19477. > Add method for sizes of hints and hints per node in HintsServiceMBean > - > > Key: CASSANDRA-19486 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19486 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Hints >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 5.x > > > I am happy to close this if somebody shows me how to get total size of all > hints or all hints per particular node via JMX. > I could find StorageMetrics.totalHints but that is how many hints there are, > not their sizes. We also have > org.apache.cassandra.metrics:type=HintedHandOffManager name= > org.apache.cassandra.metrics:type=HintsService name= > But that is again showing other metrics not sizes. > I would add two methods into HintsServiceMBean returning this. Seems to be > very easy to do once we do CASSANDRA-19477. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19486) Add method for sizes of hints and hints per node in HintsServiceMBean
[ https://issues.apache.org/jira/browse/CASSANDRA-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-19486: -- Fix Version/s: 5.x > Add method for sizes of hints and hints per node in HintsServiceMBean > - > > Key: CASSANDRA-19486 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19486 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Hints >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 5.x > > > I am happy to close this if somebody shows me how to get total size of all > hints or all hints per particular node via JMX. > I could find is StorageMetrics.totalHints but that is how many hints there > are, not their sizes. We also have > org.apache.cassandra.metrics:type=HintedHandOffManager name= > org.apache.cassandra.metrics:type=HintsService name= > But that is again showing other metrics not sizes. > I would add two methods into HintsServiceMBeans returning this. Seems to be > very easy to do once we do CASSANDRA-19477. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19477) Do not go to disk to get HintsStore.getTotalFileSize
[
https://issues.apache.org/jira/browse/CASSANDRA-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-19477:
--
Fix Version/s: (was: 4.0.x)
> Do not go to disk to get HintsStore.getTotalFileSize
>
>
> Key: CASSANDRA-19477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19477
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
>Reporter: Jon Haddad
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.1.x, 5.0-rc, 5.x
>
> Attachments: flamegraph.cpu.html
>
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> When testing a cluster with more requests than it could handle, I noticed
> significant CPU time (25%) spent in HintsStore.getTotalFileSize. Here's what
> I'm seeing from profiling:
> 10% of CPU time spent in HintsDescriptor.fileName which only does this:
>
> {noformat}
> return String.format("%s-%s-%s.hints", hostId, timestamp, version);{noformat}
> At a bare minimum here we should create this string up front with the host
> and version and eliminate 2 of the 3 substitutions, but I think it's probably
> faster to use a StringBuilder and avoid the underlying regular expression
> altogether.
> 12% of the time is spent in org.apache.cassandra.io.util.File.length. It
> looks like this is called once for each hint file on disk for each host we're
> hinting to. In the case of an overloaded cluster, this is significant. It
> would be better if we were to track the file size in memory for each hint
> file and reference that rather than go to the filesystem.
> These fairly small changes should make Cassandra more reliable when under
> load spikes.
> CPU Flame graph attached.
> I only tested this in 4.1 but it looks like this is present up to trunk.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19457: Object reference in Micrometer metrics prevent GC from reclaiming Session instances [cassandra-java-driver]
adutra commented on code in PR #1916:
URL:
https://github.com/apache/cassandra-java-driver/pull/1916#discussion_r1532813593
##
metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerMetricUpdater.java:
##
@@ -100,7 +100,10 @@ protected void initializeGauge(
m -> {
MetricId id = getMetricId(m);
Iterable tags = MicrometerTags.toMicrometerTags(id.getTags());
-return Gauge.builder(id.getName(),
supplier).tags(tags).register(registry);
+return Gauge.builder(id.getName(), supplier)
+.strongReference(false)
Review Comment:
Well I think we can keep both approaches, the weak ref trick can still be
useful.
Also, what would be really awesome is to have a test for this. Maybe
adapting the code that was used to detect the leak in the first place? FYI we
have a `SessionLeakIT` class that runs in isolation, so we can get crazy there
and create 100 sessions if it's necessary. (Although, it's probably enough to
create just one session, generate some activity, assert that the registry isn't
empty, close the session, then assert that the registry becomes empty again
within some configured timeframe.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19485) if max_hints_size_per_host < max_hints_file_size then it will write hints after max_hints_size_per_host is reached
[
https://issues.apache.org/jira/browse/CASSANDRA-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829504#comment-17829504
]
Brandon Williams commented on CASSANDRA-19485:
--
Seems like we should just disallow that configuration since it doesn't make any
sense.
> if max_hints_size_per_host < max_hints_file_size then it will write hints
> after max_hints_size_per_host is reached
> --
>
> Key: CASSANDRA-19485
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19485
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
>Reporter: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
>
> there is one problem in the current solution being that if we have this config
> {noformat}
> max_hints_size_per_host: 2MiB
> max_hints_file_size: 128MiB
> {noformat}
> basically, max size > size per host, then it will not stop it from writing
> hints after 2MiB for a particular node, because HintsDescriptor is added
> among dispatching ones after writer is closed, which happens after there is
> 128MiB written (all logic in HintsStore), so it will not be included into
> total sizes, it will be 0 until at least one hints file for that node is
> written to disk.
> I consider this to be the flaw of CASSANDRA-17142, however it is questionable
> if this is serious enough to deal with in the first place, I don't think
> somebody would set it up in practice to these values, normally one puts there
> like max per host is few gigs so this problem is not so visible but it shows
> in tests almost instantly and it is technically just wrong, regardless of the
> probability this would happen in real ...
> cc [~yifanc]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-19484:
Reviewers: Berenguer Blasi, Joshua McKenzie (was: Joshua McKenzie)
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829491#comment-17829491
]
Berenguer Blasi commented on CASSANDRA-19484:
-
Yep. I was referring to the attribute in the task invocation. If an empty
string is passed it will be ignored. At least that's the theory.
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-19191) Optimisations to PlacementForRange, improve lookup on r/w path
[
https://issues.apache.org/jira/browse/CASSANDRA-19191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson reassigned CASSANDRA-19191:
---
Assignee: Marcus Eriksson
> Optimisations to PlacementForRange, improve lookup on r/w path
> --
>
> Key: CASSANDRA-19191
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19191
> Project: Cassandra
> Issue Type: Improvement
> Components: Transactional Cluster Metadata
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 5.1-alpha1
>
>
> The lookup used when selecting the appropriate replica group for a range or
> token while peforming reads and writes is extremely simplistic and
> inefficient. There is plenty of scope to improve {{PlacementsForRange}} to by
> replacing the current naive iteration with use a more efficient lookup.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19485) if max_hints_size_per_host < max_hints_file_size then it will write hints after max_hints_size_per_host is reached
[
https://issues.apache.org/jira/browse/CASSANDRA-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-19485:
--
Fix Version/s: 4.1.x
5.0.x
5.x
> if max_hints_size_per_host < max_hints_file_size then it will write hints
> after max_hints_size_per_host is reached
> --
>
> Key: CASSANDRA-19485
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19485
> Project: Cassandra
> Issue Type: Bug
>Reporter: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
>
> there is one problem in the current solution being that if we have this config
> {noformat}
> max_hints_size_per_host: 2MiB
> max_hints_file_size: 128MiB
> {noformat}
> basically, max size > size per host, then it will not stop it from writing
> hints after 2MiB for a particular node, because HintsDescriptor is added
> among dispatching ones after writer is closed, which happens after there is
> 128MiB written (all logic in HintsStore), so it will not be included into
> total sizes, it will be 0 until at least one hints file for that node is
> written to disk.
> I consider this to be the flaw of CASSANDRA-17142, however it is questionable
> if this is serious enough to deal with in the first place, I don't think
> somebody would set it up in practice to these values, normally one puts there
> like max per host is few gigs so this problem is not so visible but it shows
> in tests almost instantly and it is technically just wrong, regardless of the
> probability this would happen in real ...
> cc [~yifanc]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19485) if max_hints_size_per_host < max_hints_file_size then it will write hints after max_hints_size_per_host is reached
[
https://issues.apache.org/jira/browse/CASSANDRA-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-19485:
--
Component/s: Consistency/Hints
> if max_hints_size_per_host < max_hints_file_size then it will write hints
> after max_hints_size_per_host is reached
> --
>
> Key: CASSANDRA-19485
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19485
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
>Reporter: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
>
> there is one problem in the current solution being that if we have this config
> {noformat}
> max_hints_size_per_host: 2MiB
> max_hints_file_size: 128MiB
> {noformat}
> basically, max size > size per host, then it will not stop it from writing
> hints after 2MiB for a particular node, because HintsDescriptor is added
> among dispatching ones after writer is closed, which happens after there is
> 128MiB written (all logic in HintsStore), so it will not be included into
> total sizes, it will be 0 until at least one hints file for that node is
> written to disk.
> I consider this to be the flaw of CASSANDRA-17142, however it is questionable
> if this is serious enough to deal with in the first place, I don't think
> somebody would set it up in practice to these values, normally one puts there
> like max per host is few gigs so this problem is not so visible but it shows
> in tests almost instantly and it is technically just wrong, regardless of the
> probability this would happen in real ...
> cc [~yifanc]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19485) if max_hints_size_per_host < max_hints_file_size then it will write hints after max_hints_size_per_host is reached
[
https://issues.apache.org/jira/browse/CASSANDRA-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-19485:
--
Description:
there is one problem in the current solution being that if we have this config
{noformat}
max_hints_size_per_host: 2MiB
max_hints_file_size: 128MiB
{noformat}
basically, max size > size per host, then it will not stop it from writing
hints after 2MiB for a particular node, because HintsDescriptor is added among
dispatching ones after writer is closed, which happens after there is 128MiB
written (all logic in HintsStore), so it will not be included into total sizes,
it will be 0 until at least one hints file for that node is written to disk.
I consider this to be the flaw of CASSANDRA-17142, however it is questionable
if this is serious enough to deal with in the first place, I don't think
somebody would set it up in practice to these values, normally one puts there
like max per host is few gigs so this problem is not so visible but it shows in
tests almost instantly and it is technically just wrong, regardless of the
probability this would happen in real ...
cc [~yifanc]
> if max_hints_size_per_host < max_hints_file_size then it will write hints
> after max_hints_size_per_host is reached
> --
>
> Key: CASSANDRA-19485
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19485
> Project: Cassandra
> Issue Type: Bug
>Reporter: Stefan Miklosovic
>Priority: Normal
>
> there is one problem in the current solution being that if we have this config
> {noformat}
> max_hints_size_per_host: 2MiB
> max_hints_file_size: 128MiB
> {noformat}
> basically, max size > size per host, then it will not stop it from writing
> hints after 2MiB for a particular node, because HintsDescriptor is added
> among dispatching ones after writer is closed, which happens after there is
> 128MiB written (all logic in HintsStore), so it will not be included into
> total sizes, it will be 0 until at least one hints file for that node is
> written to disk.
> I consider this to be the flaw of CASSANDRA-17142, however it is questionable
> if this is serious enough to deal with in the first place, I don't think
> somebody would set it up in practice to these values, normally one puts there
> like max per host is few gigs so this problem is not so visible but it shows
> in tests almost instantly and it is technically just wrong, regardless of the
> probability this would happen in real ...
> cc [~yifanc]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19485) if max_hints_size_per_host < max_hints_file_size then it will write hints after max_hints_size_per_host is reached
[ https://issues.apache.org/jira/browse/CASSANDRA-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-19485: -- Summary: if max_hints_size_per_host < max_hints_file_size then it will write hints after max_hints_size_per_host is reached (was: if max_hints_size_per_host < max_hints_file_size then it will write hints after max_hints_file_size is reached) > if max_hints_size_per_host < max_hints_file_size then it will write hints > after max_hints_size_per_host is reached > -- > > Key: CASSANDRA-19485 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19485 > Project: Cassandra > Issue Type: Bug >Reporter: Stefan Miklosovic >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19485) if max_hints_size_per_host < max_hints_file_size then it will write hints after max_hints_file_size is reached
Stefan Miklosovic created CASSANDRA-19485: - Summary: if max_hints_size_per_host < max_hints_file_size then it will write hints after max_hints_file_size is reached Key: CASSANDRA-19485 URL: https://issues.apache.org/jira/browse/CASSANDRA-19485 Project: Cassandra Issue Type: Bug Reporter: Stefan Miklosovic -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829460#comment-17829460
]
Ariel Weisberg commented on CASSANDRA-19484:
Setting a property has no effect. At least none I can find. It has to be an
attribute on the task invocation.
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19418 - Changes to report additional bulk analytics job stats for instrumentation [cassandra-analytics]
arjunashok commented on code in PR #41:
URL:
https://github.com/apache/cassandra-analytics/pull/41#discussion_r1533473809
##
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/CassandraBulkSourceRelation.java:
##
@@ -129,6 +143,64 @@ private void persist(@NotNull JavaPairRDD sortedRDD, Str
}
}
+private void recordSuccessfulJobStats(List writeResults)
+{
+List streamResults = writeResults.stream()
+
.map(WriteResult::streamResults)
+
.flatMap(Collection::stream)
+
.collect(Collectors.toList());
+
+long rowCount = streamResults.stream().mapToLong(res ->
res.rowCount).sum();
+long totalBytesWritten = streamResults.stream().mapToLong(res ->
res.bytesWritten).sum();
+boolean hasClusterTopologyChanged = writeResults.stream()
+
.map(WriteResult::isClusterResizeDetected)
+.anyMatch(b -> b);
+LOGGER.info("Bulk writer has written {} rows and {} bytes with
cluster-resize status: {}",
+rowCount,
+totalBytesWritten,
+hasClusterTopologyChanged);
+writerContext.recordJobStats(new HashMap<>()
+{
+{
+put("rowsWritten", Long.toString(rowCount));
+put("bytesWritten", Long.toString(totalBytesWritten));
+put("jobStatus", "Succeeded");
+put("clusterResizeDetected",
String.valueOf(hasClusterTopologyChanged));
+put("jobElapsedTimeMillis",
Long.toString(getElapsedTimeMillis()));
+}
+});
+}
+
+private void recordFailureStats(String reason)
+{
+writerContext.recordJobStats(new HashMap<>()
+{
+{
+put("jobStatus", "Failed");
+put("failureReason", reason);
+put("jobElapsedTimeMillis",
Long.toString(getElapsedTimeMillis()));
+}
+});
+}
+
+private long getElapsedTimeMillis()
+{
+long now = System.nanoTime();
+return TimeUnit.NANOSECONDS.toMillis(now - this.startTimeNanos);
+}
+
+/**
+ * Get a ref copy of BulkWriterContext broadcast variable and compose a
function to transform a partition into StreamResult
+ *
+ * @param ctx BulkWriterContext broadcast variable
+ * @return FlatMapFunction
+ */
+private static FlatMapFunction>,
WriteResult>
+partitionsFlatMapFunc(Broadcast ctx, String[]
columnNames)
Review Comment:
Makes sense. Reverted
##
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/RingInstance.java:
##
@@ -125,40 +126,28 @@ private void writeObject(ObjectOutputStream out) throws
IOException
out.writeUTF(ringEntry.address());
out.writeInt(ringEntry.port());
out.writeUTF(ringEntry.datacenter());
-out.writeUTF(ringEntry.load());
Review Comment:
Unfortunately, the fields are members of the `RingEntry` instance (part of
the sidecar client library) so they cannot be removed here.
I can add checks for the ones not in `ReplicaMetadata` for completeness, but
the intention here was to not write the fields that we know are not used anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829421#comment-17829421
]
Berenguer Blasi commented on CASSANDRA-19484:
-
Looking at the code that sets the properties it looks like nvdDatafeedUrl will
be ignored if empty. You could test that and spare the extra task if it works
and add a comment. Otherwise I don't see a way out of it. +1 once tested even
locally/manually.
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
