[jira] [Commented] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out of that
[
https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17806214#comment-17806214
]
shylaja kokoori commented on CASSANDRA-18688:
-
Thanks for fixing it Ekaterina:)
Ekaterina asked me to capture here the steps I followed to test the packages
manually.
I followed instructions in [this
script|[https://github.com/apache/cassandra-builds/blob/f5a33a85dbcfbce92d37bb0682d2cb1adc805e2e/cassandra-release/cassandra-check-release.sh#L131-L162]]
shared by mck
+Build commands:+
{code:java}
/.build/docker/build-debian.sh
/.build/docker/build-redhat.sh{code}
+Creating the docker container & running the test:+
Redhat
{code:java}
docker run -it -v :/redhat almalinux bash
yum install -y procps-ng python3-pip
yum install -y
rpm -i /redhat/build/cassandra*.rpm
cassandra -R -f{code}
Debian
{code:java}
docker run -it -v :/debian debian:bullseye-slim bash
apt -qq update
apt -qq install -y python
apt -qq install -y python3 procps
apt -qq install -y
dpkg --ignore-depends=java7-runtime --ignore-depends=java8-runtime -i
/debian/build/*.deb
Run the test{code}
> Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out
> of that
>
>
> Key: CASSANDRA-18688
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18688
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Ekaterina Dimitrova
>Assignee: shylaja kokoori
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Currently, we limit our users from building with non-default Java versions in
> build.xml.
> They can easily hack build.xml for test purposes with different versions.
> Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit
> people to those two, but only to everything >= 11. We should also put an
> upper limit of 17 in our Cassandra startup scripts. We can also add a flag to
> opt-out if someone wants to test with newer versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out of that
[ https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17804545#comment-17804545 ] shylaja kokoori edited comment on CASSANDRA-18688 at 1/9/24 1:48 AM: - Thank you, updated the pull request with suggested changes [https://github.com/apache/cassandra/pull/2563/commits/0a77b30a326ea0e3c77ebd369a4b504c4e6ad354] was (Author: shylaja.koko...@intel.com): Thank you, updated the changes [here |[https://github.com/apache/cassandra/pull/2563/commits/0a77b30a326ea0e3c77ebd369a4b504c4e6ad354]] > Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out > of that > > > Key: CASSANDRA-18688 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18688 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.0.x, 5.x > > Time Spent: 3.5h > Remaining Estimate: 0h > > Currently, we limit our users from building with non-default Java versions in > build.xml. > They can easily hack build.xml for test purposes with different versions. > Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit > people to those two, but only to everything >= 11. We should also put an > upper limit of 17 in our Cassandra startup scripts. We can also add a flag to > opt-out if someone wants to test with newer versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out of that
[ https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17804545#comment-17804545 ] shylaja kokoori commented on CASSANDRA-18688: - Thank you, updated the changes [here |[https://github.com/apache/cassandra/pull/2563/commits/0a77b30a326ea0e3c77ebd369a4b504c4e6ad354]] > Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out > of that > > > Key: CASSANDRA-18688 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18688 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.0.x, 5.x > > Time Spent: 3.5h > Remaining Estimate: 0h > > Currently, we limit our users from building with non-default Java versions in > build.xml. > They can easily hack build.xml for test purposes with different versions. > Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit > people to those two, but only to everything >= 11. We should also put an > upper limit of 17 in our Cassandra startup scripts. We can also add a flag to > opt-out if someone wants to test with newer versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out of that
[ https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17801925#comment-17801925 ] shylaja kokoori commented on CASSANDRA-18688: - Thanks [~mck] , I will wait to hear from Ekaterina also & push the changes > Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out > of that > > > Key: CASSANDRA-18688 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18688 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.0.x, 5.x > > Time Spent: 3.5h > Remaining Estimate: 0h > > Currently, we limit our users from building with non-default Java versions in > build.xml. > They can easily hack build.xml for test purposes with different versions. > Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit > people to those two, but only to everything >= 11. We should also put an > upper limit of 17 in our Cassandra startup scripts. We can also add a flag to > opt-out if someone wants to test with newer versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out of that
[
https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17801866#comment-17801866
]
shylaja kokoori commented on CASSANDRA-18688:
-
Thanks everyone for all the feedback you have given me, helped me understand
the process:)
I was on vacation last week, saw this just now. I do agree with what was
discussed.
I could change the messaging to say,
{code:java}
echo "If you would like to test with newer Java versions set
CASSANDRA_JDK_UNSUPPORTED to any value (for example,
CASSANDRA_JDK_UNSUPPORTED=true), unset the parameter for default behavior"{code}
Please let me know if everyone agrees & I will change it & remove the check for
'false' value
> Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out
> of that
>
>
> Key: CASSANDRA-18688
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18688
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Ekaterina Dimitrova
>Assignee: shylaja kokoori
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Currently, we limit our users from building with non-default Java versions in
> build.xml.
> They can easily hack build.xml for test purposes with different versions.
> Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit
> people to those two, but only to everything >= 11. We should also put an
> upper limit of 17 in our Cassandra startup scripts. We can also add a flag to
> opt-out if someone wants to test with newer versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out of that
[ https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787446#comment-17787446 ] shylaja kokoori commented on CASSANDRA-18688: - I have updated the debian & redhat scripts and tested using docker containers. Tested with JDKs 15, 17, 21. Behavior as we decided 15 - cannot execute 17- does not depend on CASSANDRA_JDK_UNSUPPORTED 21 - works only when CASSANDRA_JDK_UNSUPPORTED is set Please let me know if I have missed anything. > Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out > of that > > > Key: CASSANDRA-18688 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18688 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.0.x > > Time Spent: 2h > Remaining Estimate: 0h > > Currently, we limit our users from building with non-default Java versions in > build.xml. > They can easily hack build.xml for test purposes with different versions. > Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit > people to those two, but only to everything >= 11. We should also put an > upper limit of 17 in our Cassandra startup scripts. We can also add a flag to > opt-out if someone wants to test with newer versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out of that
[ https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17783408#comment-17783408 ] shylaja kokoori commented on CASSANDRA-18688: - Are there any other changes needed for this ticket? > Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out > of that > > > Key: CASSANDRA-18688 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18688 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.0.x > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently, we limit our users from building with non-default Java versions in > build.xml. > They can easily hack build.xml for test purposes with different versions. > Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit > people to those two, but only to everything >= 11. We should also put an > upper limit of 17 in our Cassandra startup scripts. We can also add a flag to > opt-out if someone wants to test with newer versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out of that
[ https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772408#comment-17772408 ] shylaja kokoori commented on CASSANDRA-18688: - Thank you everyone for helping with the name for the env var. [~e.dimitrova] I removed (or newer) to make it consistent with other messages. I missed that earlier:( > Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out > of that > > > Key: CASSANDRA-18688 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18688 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.0.x > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently, we limit our users from building with non-default Java versions in > build.xml. > They can easily hack build.xml for test purposes with different versions. > Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit > people to those two, but only to everything >= 11. We should also put an > upper limit of 17 in our Cassandra startup scripts. We can also add a flag to > opt-out if someone wants to test with newer versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out of that
[ https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770627#comment-17770627 ] shylaja kokoori commented on CASSANDRA-18688: - Thank you very much for the input. I think I have addressed all the suggestions in the new commit. I have also tested the code with JDK versions 11, 12, 17, 20, 21 with & without the env variable set. I have a band around the warning since it was not visible otherwise. Is that ok? [~bereng] does GT mean greater than? > Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out > of that > > > Key: CASSANDRA-18688 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18688 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.0.x > > Time Spent: 1h 40m > Remaining Estimate: 0h > > Currently, we limit our users from building with non-default Java versions in > build.xml. > They can easily hack build.xml for test purposes with different versions. > Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit > people to those two, but only to everything >= 11. We should also put an > upper limit of 17 in our Cassandra startup scripts. We can also add a flag to > opt-out if someone wants to test with newer versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out of that
[ https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765878#comment-17765878 ] shylaja kokoori commented on CASSANDRA-18688: - Hi All, I have updated the pull request with this commit [https://github.com/apache/cassandra/pull/2563/commits/e1d189f86909d8307798cd2f39966d94c0742226] Please let me know if this looks better I have added an env var CASSANDRA_USE_ALL_JDK to enable non-LTS Java use. Does that name work? > Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out > of that > > > Key: CASSANDRA-18688 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18688 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.0.x > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Currently, we limit our users from building with non-default Java versions in > build.xml. > They can easily hack build.xml for test purposes with different versions. > Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit > people to those two, but only to everything >= 11. We should also put an > upper limit of 17 in our Cassandra startup scripts. We can also add a flag to > opt-out if someone wants to test with newer versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out of that
[ https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17763285#comment-17763285 ] shylaja kokoori commented on CASSANDRA-18688: - I have pinged everybody offline & you were ok with either choice. I have decided to go with the env var option like Mick suggested. That seems like the simplest solution which would work for both bin/cassandra & scripts in tools/bin. Regarding which versions of Java should this apply, will apply to versions > 17, since we are limiting it to 11 & 17 only in 5.0 like Ekaterina originally said. Please let me know if this works for everyone & I will update the patch > Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out > of that > > > Key: CASSANDRA-18688 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18688 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.0.x > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Currently, we limit our users from building with non-default Java versions in > build.xml. > They can easily hack build.xml for test purposes with different versions. > Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit > people to those two, but only to everything >= 11. We should also put an > upper limit of 17 in our Cassandra startup scripts. We can also add a flag to > opt-out if someone wants to test with newer versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out of that
[
https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755652#comment-17755652
]
shylaja kokoori commented on CASSANDRA-18688:
-
Mick, to answer your questions
{quote}How difficult is it to edit the bin/cassandra script (or
cassandra.in.sh) for them to get what they want ?
{quote}
Like, Ekaterina said, currently we are allowing versions 11, 17 and anything
above. With current patch, server bring up fails if not a supported version.
bin/cassandra.in.sh changes are not difficult, but there is a bit of learning
curve in my opinion, if one is new to the code.
{quote}What signal (advertently or not) are we giving by providing the `-J`
option ? Would a env var be a better signal ? (I think the use of `-J`, if we
accept it, should still print a loud warning.)
{quote}
-J option is inline with -R, f etc. we already provide for bin/cassandra. By
env var, I assume you mean something like what we had to enable JDK11 in 4.x,
with a -D. This maybe a better option perhaps, if we want to do something
similar with scripts in tools folder. Regardless, I feel providing an option
from command line will be convenient. I agree, with everyone the message needs
to be louder than what I have.
> Limit Java runtime in 5.0 to JDK 11 and 17 in scripts; add a flag to opt out
> of that
>
>
> Key: CASSANDRA-18688
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18688
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Ekaterina Dimitrova
>Assignee: shylaja kokoori
>Priority: Normal
> Fix For: 5.0.x
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently, we limit our users from building with non-default Java versions in
> build.xml.
> They can easily hack build.xml for test purposes with different versions.
> Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit
> people to those two, but only to everything >= 11. We should also put an
> upper limit of 17 in our Cassandra startup scripts. We can also add a flag to
> opt-out if someone wants to test with newer versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK17 in scripts; add a flag to opt out of that
[ https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shylaja kokoori updated CASSANDRA-18688: Mentor: Ekaterina Dimitrova Test and Documentation Plan: Cassandra startup was tested with JDK 8,11, 12, 17 & 19 with and without the -J option Status: Patch Available (was: Open) The patch limits startup to JDK11 & 17. I have added an option J to override this behavior > Limit Java runtime in 5.0 to JDK17 in scripts; add a flag to opt out of that > > > Key: CASSANDRA-18688 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18688 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > Currently, we limit our users from building with non-default Java versions in > build.xml. > They can easily hack build.xml for test purposes with different versions. > Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit > people to those two, but only to everything >= 11. We should also put an > upper limit of 17 in our Cassandra startup scripts. We can also add a flag to > opt-out if someone wants to test with newer versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17951) Update AssertJ
[ https://issues.apache.org/jira/browse/CASSANDRA-17951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748681#comment-17748681 ] shylaja kokoori commented on CASSANDRA-17951: - Thank you for updating Junit, Stefan. How should we document the dependency, in the pom file? > Update AssertJ > -- > > Key: CASSANDRA-17951 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17951 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.x > > Attachments: 17951.patch, 17951_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > From the changelog of AssertJ > [here,|https://assertj.github.io/doc/#assertj-core-3-23-1-release-notes] it > seems JDK17 is tested and relevant issues fixed as per 3.23.1, and we are on > 3.15.0 > This ticket should accommodate an upgrade to the latest version of AssertJ > for trunk. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18688) Limit Java runtime in 5.0 to JDK17 in scripts; add a flag to opt out of that
[ https://issues.apache.org/jira/browse/CASSANDRA-18688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shylaja kokoori reassigned CASSANDRA-18688: --- Assignee: shylaja kokoori > Limit Java runtime in 5.0 to JDK17 in scripts; add a flag to opt out of that > > > Key: CASSANDRA-18688 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18688 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > > Currently, we limit our users from building with non-default Java versions in > build.xml. > They can easily hack build.xml for test purposes with different versions. > Cassandra–5.0 will be run on JDK11 and JDK17, but on startup, we do not limit > people to those two, but only to everything >= 11. We should also put an > upper limit of 17 in our Cassandra startup scripts. We can also add a flag to > opt-out if someone wants to test with newer versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17951) Update AssertJ
[ https://issues.apache.org/jira/browse/CASSANDRA-17951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747097#comment-17747097 ] shylaja kokoori commented on CASSANDRA-17951: - [~smiklosovic] I have attached a patch with changes to SimplePerfTest & LongOpOrderTest removed. Will a pull request be better? > Update AssertJ > -- > > Key: CASSANDRA-17951 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17951 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.x > > Attachments: 17951.patch, 17951_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > From the changelog of AssertJ > [here,|https://assertj.github.io/doc/#assertj-core-3-23-1-release-notes] it > seems JDK17 is tested and relevant issues fixed as per 3.23.1, and we are on > 3.15.0 > This ticket should accommodate an upgrade to the latest version of AssertJ > for trunk. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17951) Update AssertJ
[ https://issues.apache.org/jira/browse/CASSANDRA-17951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shylaja kokoori updated CASSANDRA-17951: Attachment: 17951_1.patch > Update AssertJ > -- > > Key: CASSANDRA-17951 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17951 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.x > > Attachments: 17951.patch, 17951_1.patch > > Time Spent: 10m > Remaining Estimate: 0h > > From the changelog of AssertJ > [here,|https://assertj.github.io/doc/#assertj-core-3-23-1-release-notes] it > seems JDK17 is tested and relevant issues fixed as per 3.23.1, and we are on > 3.15.0 > This ticket should accommodate an upgrade to the latest version of AssertJ > for trunk. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17951) Update AssertJ
[ https://issues.apache.org/jira/browse/CASSANDRA-17951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747028#comment-17747028 ] shylaja kokoori commented on CASSANDRA-17951: - Hi Stefan, Thank you for reviewing the patch. You are correct, rest of the files have modifications that fixes the build. I had run the Cajon plugin on the code, modification to the 2 files you mentioned were based on the scan. I read that was the preferred way with Assertj. I can revert them back and submit another patch > Update AssertJ > -- > > Key: CASSANDRA-17951 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17951 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.x > > Attachments: 17951.patch > > Time Spent: 10m > Remaining Estimate: 0h > > From the changelog of AssertJ > [here,|https://assertj.github.io/doc/#assertj-core-3-23-1-release-notes] it > seems JDK17 is tested and relevant issues fixed as per 3.23.1, and we are on > 3.15.0 > This ticket should accommodate an upgrade to the latest version of AssertJ > for trunk. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17951) Update AssertJ
[ https://issues.apache.org/jira/browse/CASSANDRA-17951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shylaja kokoori updated CASSANDRA-17951: Mentor: Ekaterina Dimitrova Test and Documentation Plan: Tested with JDK11 & JDK17 Status: Patch Available (was: Open) > Update AssertJ > -- > > Key: CASSANDRA-17951 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17951 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.x > > Attachments: 17951.patch > > > From the changelog of AssertJ > [here,|https://assertj.github.io/doc/#assertj-core-3-23-1-release-notes] it > seems JDK17 is tested and relevant issues fixed as per 3.23.1, and we are on > 3.15.0 > This ticket should accommodate an upgrade to the latest version of AssertJ > for trunk. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-17951) Update AssertJ
[ https://issues.apache.org/jira/browse/CASSANDRA-17951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shylaja kokoori reassigned CASSANDRA-17951: --- Assignee: shylaja kokoori (was: Ekaterina Dimitrova) > Update AssertJ > -- > > Key: CASSANDRA-17951 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17951 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Ekaterina Dimitrova >Assignee: shylaja kokoori >Priority: Normal > Fix For: 5.x > > Attachments: 17951.patch > > > From the changelog of AssertJ > [here,|https://assertj.github.io/doc/#assertj-core-3-23-1-release-notes] it > seems JDK17 is tested and relevant issues fixed as per 3.23.1, and we are on > 3.15.0 > This ticket should accommodate an upgrade to the latest version of AssertJ > for trunk. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17951) Update AssertJ
[ https://issues.apache.org/jira/browse/CASSANDRA-17951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745310#comment-17745310 ] shylaja kokoori commented on CASSANDRA-17951: - In the patch I have upgraded AssertJ version to 3.24.2 and fixed the build issues. I have run the unit tests with JDK11 & JDK17. I also ran the Cajon IntelliJ plugin and added some of the modifications. > Update AssertJ > -- > > Key: CASSANDRA-17951 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17951 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 5.x > > Attachments: 17951.patch > > > From the changelog of AssertJ > [here,|https://assertj.github.io/doc/#assertj-core-3-23-1-release-notes] it > seems JDK17 is tested and relevant issues fixed as per 3.23.1, and we are on > 3.15.0 > This ticket should accommodate an upgrade to the latest version of AssertJ > for trunk. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17951) Update AssertJ
[ https://issues.apache.org/jira/browse/CASSANDRA-17951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shylaja kokoori updated CASSANDRA-17951: Attachment: 17951.patch > Update AssertJ > -- > > Key: CASSANDRA-17951 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17951 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 5.x > > Attachments: 17951.patch > > > From the changelog of AssertJ > [here,|https://assertj.github.io/doc/#assertj-core-3-23-1-release-notes] it > seems JDK17 is tested and relevant issues fixed as per 3.23.1, and we are on > 3.15.0 > This ticket should accommodate an upgrade to the latest version of AssertJ > for trunk. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17850) Find a way to get FileDescriptor.fd and sun.nio.ch.FileChannelImpl.fd without opening internals
[
https://issues.apache.org/jira/browse/CASSANDRA-17850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17743318#comment-17743318
]
shylaja kokoori commented on CASSANDRA-17850:
-
Hi,
I have been doing some research for this ticket with Benjamin, these are my
observations,
1) NativeLibrary class, uses the file descriptor to work with file system cache
(via JNA->Libc)
2) Filechannel.force will flush all the data from the channel to storage. But
we also want to make sure the OS does not unnecessarily cache the data. This is
achieved currently by using functions in the NativeLibrary
My suggestion, FileChannel now has ExtendedOpenOption.DIRECT option to do
DirectIO which will bypass filesystem cache. This I believe will eliminate the
need to use file descriptors. Will this approach work?
> Find a way to get FileDescriptor.fd and sun.nio.ch.FileChannelImpl.fd without
> opening internals
> ---
>
> Key: CASSANDRA-17850
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17850
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ekaterina Dimitrova
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 5.x
>
>
> With Java 17 if we do not add below to the jvm17 server options:
> {code:java}
> --add-opens java.base/sun.nio.ch=ALL-UNNAMED
> --add-opens java.base/java.io=ALL-UNNAMED{code}
> we get on startup (considering I comment out the scripted UDFs and apply a
> few changes to the startup scripts):
> {code:java}
> ERROR [ScheduledTasks:1] 2022-08-23 12:29:25,652
> JVMStabilityInspector.java:68 - Exception in thread
> Thread[ScheduledTasks:1,5,ScheduledTasks]
> java.lang.AssertionError: java.lang.reflect.InaccessibleObjectException:
> Unable to make field private int java.io.FileDescriptor.fd accessible: module
> java.base does not "opens java.io" to unnamed module @11d8ae8b
> at
> org.apache.cassandra.utils.FBUtilities.getProtectedField(FBUtilities.java:801)
> at org.apache.cassandra.utils.NativeLibrary.(NativeLibrary.java:84)
> at org.apache.cassandra.utils.TimeUUID$Generator.hash(TimeUUID.java:496)
> at org.apache.cassandra.utils.TimeUUID$Generator.makeNode(TimeUUID.java:474)
> at
> org.apache.cassandra.utils.TimeUUID$Generator.makeClockSeqAndNode(TimeUUID.java:452)
> at org.apache.cassandra.utils.TimeUUID$Generator.(TimeUUID.java:368)
> at
> org.apache.cassandra.streaming.StreamingState.(StreamingState.java:50)
> at org.apache.cassandra.streaming.StreamManager.(StreamManager.java:257)
> at
> org.apache.cassandra.streaming.StreamManager.(StreamManager.java:58)
> at org.apache.cassandra.service.StorageService.(StorageService.java:376)
> at
> org.apache.cassandra.service.StorageService.(StorageService.java:226)
> at
> org.apache.cassandra.locator.DynamicEndpointSnitch.updateScores(DynamicEndpointSnitch.java:274)
> at
> org.apache.cassandra.locator.DynamicEndpointSnitch$1.run(DynamicEndpointSnitch.java:91)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$1.run(ExecutionFailure.java:133)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
> at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make
> field private int java.io.FileDescriptor.fd accessible: module java.base does
> not "opens java.io" to unnamed module @11d8ae8b
> at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
> at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
> at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
> at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
> at
> org.apache.cassandra.utils.FBUtilities.getProtectedField(FBUtilities.java:796)
> ... 20 common frames omitted
> {code}
> and
> {code:java}
> ERROR [ScheduledTasks:1] 2022-08-23 12:31:18,443
> JVMStabilityInspector.java:68 - Exception in thread
> Thread[ScheduledTasks:1,5,ScheduledTasks]
> java.lang.AssertionError: java.lang.reflect.InaccessibleObjectException:
> Unable to make field private final java.io.FileDescriptor
> sun.nio.ch.FileChannelImpl.fd accessible: module java.base does not "opens
> sun.nio.ch"
[jira] [Assigned] (CASSANDRA-17884) [jamm] Test failure: o.a.c.repair.RepairJobTest.testNoTreesRetainedAfterDifference
[
https://issues.apache.org/jira/browse/CASSANDRA-17884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shylaja kokoori reassigned CASSANDRA-17884:
---
Assignee: (was: shylaja kokoori)
> [jamm] Test failure:
> o.a.c.repair.RepairJobTest.testNoTreesRetainedAfterDifference
> --
>
> Key: CASSANDRA-17884
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17884
> Project: Cassandra
> Issue Type: Bug
> Components: Jamm, Test/unit
>Reporter: Andres de la Peña
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
> Attachments: RepairSessionTree.txt
>
>
> The unit test
> {{org.apache.cassandra.repair.RepairJobTest#testNoTreesRetainedAfterDifference}}
> seems to be slightly flaky at least in 4.0. We haven't seen it failing on
> Butler, but after [a failure on a patch
> branch|https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/281/workflows/e6094c00-b611-4772-9772-205f4c76ecba/jobs/2126/tests],
> this repeated run shows a 0.28% flakiness:
> *
> https://app.circleci.com/pipelines/github/adelapena/cassandra/2076/workflows/8adbfe99-afb5-43af-84ad-43df2a2a86e2/jobs/20816/tests
> *
> https://app.circleci.com/pipelines/github/adelapena/cassandra/2076/workflows/e550d8c2-7c35-4326-bd95-6909978d344b/jobs/20817/tests
> {code}
> java.lang.reflect.InaccessibleObjectException: Unable to make field private
> jdk.internal.platform.cgroupv1.SubSystem$MemorySubSystem
> jdk.internal.platform.cgroupv1.Metrics.memory accessible: module java.base
> does not "opens jdk.internal.platform.cgroupv1" to unnamed module @157853da
> at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:340)
> at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:280)
> at
> java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:176)
> at java.base/java.lang.reflect.Field.setAccessible(Field.java:170)
> at org.github.jamm.MemoryMeter.addFieldChildren(MemoryMeter.java:330)
> at org.github.jamm.MemoryMeter.measureDeep(MemoryMeter.java:269)
> at
> org.apache.cassandra.utils.ObjectSizes.measureDeep(ObjectSizes.java:216)
> at
> org.apache.cassandra.repair.RepairJobTest.testNoTreesRetainedAfterDifference(RepairJobTest.java:271)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> Flakiness is really low but the failure is clearly reproducible in j11.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17884) Test failure: o.a.c.repair.RepairJobTest.testNoTreesRetainedAfterDifference
[
https://issues.apache.org/jira/browse/CASSANDRA-17884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684019#comment-17684019
]
shylaja kokoori commented on CASSANDRA-17884:
-
There are couple of tasks being worked on to resolve this issue
1) We suspect there are objects which should not be considered in the object
size calculation and those classes need to be marked Unmetered.
[^RepairSessionTree.txt] contains a printout of the tree traversed by Jamm for
a MeasurableRepairSession object, DebuggableThreadPoolExecutor is an example.
2) Jamm library needs to handle modules/hidden classes/Java internal classes
appropriately, Benjamin is looking into it.
> Test failure: o.a.c.repair.RepairJobTest.testNoTreesRetainedAfterDifference
> ---
>
> Key: CASSANDRA-17884
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17884
> Project: Cassandra
> Issue Type: Bug
> Components: Jamm, Test/unit
>Reporter: Andres de la Peña
>Assignee: shylaja kokoori
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
> Attachments: RepairSessionTree.txt
>
>
> The unit test
> {{org.apache.cassandra.repair.RepairJobTest#testNoTreesRetainedAfterDifference}}
> seems to be slightly flaky at least in 4.0. We haven't seen it failing on
> Butler, but after [a failure on a patch
> branch|https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/281/workflows/e6094c00-b611-4772-9772-205f4c76ecba/jobs/2126/tests],
> this repeated run shows a 0.28% flakiness:
> *
> https://app.circleci.com/pipelines/github/adelapena/cassandra/2076/workflows/8adbfe99-afb5-43af-84ad-43df2a2a86e2/jobs/20816/tests
> *
> https://app.circleci.com/pipelines/github/adelapena/cassandra/2076/workflows/e550d8c2-7c35-4326-bd95-6909978d344b/jobs/20817/tests
> {code}
> java.lang.reflect.InaccessibleObjectException: Unable to make field private
> jdk.internal.platform.cgroupv1.SubSystem$MemorySubSystem
> jdk.internal.platform.cgroupv1.Metrics.memory accessible: module java.base
> does not "opens jdk.internal.platform.cgroupv1" to unnamed module @157853da
> at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:340)
> at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:280)
> at
> java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:176)
> at java.base/java.lang.reflect.Field.setAccessible(Field.java:170)
> at org.github.jamm.MemoryMeter.addFieldChildren(MemoryMeter.java:330)
> at org.github.jamm.MemoryMeter.measureDeep(MemoryMeter.java:269)
> at
> org.apache.cassandra.utils.ObjectSizes.measureDeep(ObjectSizes.java:216)
> at
> org.apache.cassandra.repair.RepairJobTest.testNoTreesRetainedAfterDifference(RepairJobTest.java:271)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> Flakiness is really low but the failure is clearly reproducible in j11.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17884) Test failure: o.a.c.repair.RepairJobTest.testNoTreesRetainedAfterDifference
[
https://issues.apache.org/jira/browse/CASSANDRA-17884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shylaja kokoori updated CASSANDRA-17884:
Attachment: RepairSessionTree.txt
> Test failure: o.a.c.repair.RepairJobTest.testNoTreesRetainedAfterDifference
> ---
>
> Key: CASSANDRA-17884
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17884
> Project: Cassandra
> Issue Type: Bug
> Components: Jamm, Test/unit
>Reporter: Andres de la Peña
>Assignee: shylaja kokoori
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
> Attachments: RepairSessionTree.txt
>
>
> The unit test
> {{org.apache.cassandra.repair.RepairJobTest#testNoTreesRetainedAfterDifference}}
> seems to be slightly flaky at least in 4.0. We haven't seen it failing on
> Butler, but after [a failure on a patch
> branch|https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/281/workflows/e6094c00-b611-4772-9772-205f4c76ecba/jobs/2126/tests],
> this repeated run shows a 0.28% flakiness:
> *
> https://app.circleci.com/pipelines/github/adelapena/cassandra/2076/workflows/8adbfe99-afb5-43af-84ad-43df2a2a86e2/jobs/20816/tests
> *
> https://app.circleci.com/pipelines/github/adelapena/cassandra/2076/workflows/e550d8c2-7c35-4326-bd95-6909978d344b/jobs/20817/tests
> {code}
> java.lang.reflect.InaccessibleObjectException: Unable to make field private
> jdk.internal.platform.cgroupv1.SubSystem$MemorySubSystem
> jdk.internal.platform.cgroupv1.Metrics.memory accessible: module java.base
> does not "opens jdk.internal.platform.cgroupv1" to unnamed module @157853da
> at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:340)
> at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:280)
> at
> java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:176)
> at java.base/java.lang.reflect.Field.setAccessible(Field.java:170)
> at org.github.jamm.MemoryMeter.addFieldChildren(MemoryMeter.java:330)
> at org.github.jamm.MemoryMeter.measureDeep(MemoryMeter.java:269)
> at
> org.apache.cassandra.utils.ObjectSizes.measureDeep(ObjectSizes.java:216)
> at
> org.apache.cassandra.repair.RepairJobTest.testNoTreesRetainedAfterDifference(RepairJobTest.java:271)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> Flakiness is really low but the failure is clearly reproducible in j11.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17884) Test failure: o.a.c.repair.RepairJobTest.testNoTreesRetainedAfterDifference
[
https://issues.apache.org/jira/browse/CASSANDRA-17884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17629134#comment-17629134
]
shylaja kokoori commented on CASSANDRA-17884:
-
Thanks for your help Ekaterina, posting summary of conversation we had here
# Issue I have seen so far with this test is when MemoryMeter tries to access
fields from JDK internal classes like com.sun.*, jdk.internal.* etc. There is a
commit in jamm
[https://github.com/jbellis/jamm/commit/33d78c1035211e5f9e69ad9247f3a0f6ef75c923]
(Ekaterina pointed me to) addressing this specific issue, but it does not seem
to filter out all the JDK internal classes.
# Also there might be a timing issue involved here. When I step into the code
using IntelliJ and look at the values of the variables (adding latency),
occurrence of the failure is delayed.
Still trying to figure out when the iteration in MemoryMeter.addFieldChildren
should terminate.
Avoiding access to these internal functions will help JDK11. Foreign function
interface available as part of project Panama maybe another option, in later
JDK versions, to get the object size.
> Test failure: o.a.c.repair.RepairJobTest.testNoTreesRetainedAfterDifference
> ---
>
> Key: CASSANDRA-17884
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17884
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Andres de la Peña
>Assignee: shylaja kokoori
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
>
> The unit test
> {{org.apache.cassandra.repair.RepairJobTest#testNoTreesRetainedAfterDifference}}
> seems to be slightly flaky at least in 4.0. We haven't seen it failing on
> Butler, but after [a failure on a patch
> branch|https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/281/workflows/e6094c00-b611-4772-9772-205f4c76ecba/jobs/2126/tests],
> this repeated run shows a 0.28% flakiness:
> *
> https://app.circleci.com/pipelines/github/adelapena/cassandra/2076/workflows/8adbfe99-afb5-43af-84ad-43df2a2a86e2/jobs/20816/tests
> *
> https://app.circleci.com/pipelines/github/adelapena/cassandra/2076/workflows/e550d8c2-7c35-4326-bd95-6909978d344b/jobs/20817/tests
> {code}
> java.lang.reflect.InaccessibleObjectException: Unable to make field private
> jdk.internal.platform.cgroupv1.SubSystem$MemorySubSystem
> jdk.internal.platform.cgroupv1.Metrics.memory accessible: module java.base
> does not "opens jdk.internal.platform.cgroupv1" to unnamed module @157853da
> at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:340)
> at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:280)
> at
> java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:176)
> at java.base/java.lang.reflect.Field.setAccessible(Field.java:170)
> at org.github.jamm.MemoryMeter.addFieldChildren(MemoryMeter.java:330)
> at org.github.jamm.MemoryMeter.measureDeep(MemoryMeter.java:269)
> at
> org.apache.cassandra.utils.ObjectSizes.measureDeep(ObjectSizes.java:216)
> at
> org.apache.cassandra.repair.RepairJobTest.testNoTreesRetainedAfterDifference(RepairJobTest.java:271)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> Flakiness is really low but the failure is clearly reproducible in j11.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-17884) Test failure: o.a.c.repair.RepairJobTest.testNoTreesRetainedAfterDifference
[
https://issues.apache.org/jira/browse/CASSANDRA-17884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shylaja kokoori reassigned CASSANDRA-17884:
---
Assignee: shylaja kokoori
> Test failure: o.a.c.repair.RepairJobTest.testNoTreesRetainedAfterDifference
> ---
>
> Key: CASSANDRA-17884
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17884
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Andres de la Peña
>Assignee: shylaja kokoori
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
>
> The unit test
> {{org.apache.cassandra.repair.RepairJobTest#testNoTreesRetainedAfterDifference}}
> seems to be slightly flaky at least in 4.0. We haven't seen it failing on
> Butler, but after [a failure on a patch
> branch|https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/281/workflows/e6094c00-b611-4772-9772-205f4c76ecba/jobs/2126/tests],
> this repeated run shows a 0.28% flakiness:
> *
> https://app.circleci.com/pipelines/github/adelapena/cassandra/2076/workflows/8adbfe99-afb5-43af-84ad-43df2a2a86e2/jobs/20816/tests
> *
> https://app.circleci.com/pipelines/github/adelapena/cassandra/2076/workflows/e550d8c2-7c35-4326-bd95-6909978d344b/jobs/20817/tests
> {code}
> java.lang.reflect.InaccessibleObjectException: Unable to make field private
> jdk.internal.platform.cgroupv1.SubSystem$MemorySubSystem
> jdk.internal.platform.cgroupv1.Metrics.memory accessible: module java.base
> does not "opens jdk.internal.platform.cgroupv1" to unnamed module @157853da
> at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:340)
> at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:280)
> at
> java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:176)
> at java.base/java.lang.reflect.Field.setAccessible(Field.java:170)
> at org.github.jamm.MemoryMeter.addFieldChildren(MemoryMeter.java:330)
> at org.github.jamm.MemoryMeter.measureDeep(MemoryMeter.java:269)
> at
> org.apache.cassandra.utils.ObjectSizes.measureDeep(ObjectSizes.java:216)
> at
> org.apache.cassandra.repair.RepairJobTest.testNoTreesRetainedAfterDifference(RepairJobTest.java:271)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> Flakiness is really low but the failure is clearly reproducible in j11.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17422) Test Failure: org.apache.cassandra.net.OutboundMessageQueueTest.testRemove-cdc
[
https://issues.apache.org/jira/browse/CASSANDRA-17422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600989#comment-17600989
]
shylaja kokoori commented on CASSANDRA-17422:
-
[~bereng] , I was on vacation and did not see your previous comment. Thank you
for resolving the issue
> Test Failure: org.apache.cassandra.net.OutboundMessageQueueTest.testRemove-cdc
> --
>
> Key: CASSANDRA-17422
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17422
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Josh McKenzie
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0.6, 4.1-beta, 4.x
>
> Attachments: CASSANDRA-17422.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Branch: 4.0
> https://ci-cassandra.apache.org/job/Cassandra-4.0/350/testReport/org.apache.cassandra.net/OutboundMessageQueueTest/testRemove_cdc/
> {code}
> java.lang.NullPointerException
> at
> org.apache.cassandra.net.OutboundMessageQueueTest.testRemove(OutboundMessageQueueTest.java:91)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> Failure: 1 of 3
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17422) Test Failure: org.apache.cassandra.net.OutboundMessageQueueTest.testRemove-cdc
[
https://issues.apache.org/jira/browse/CASSANDRA-17422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17558184#comment-17558184
]
shylaja kokoori commented on CASSANDRA-17422:
-
I have created a pull request with the changes
> Test Failure: org.apache.cassandra.net.OutboundMessageQueueTest.testRemove-cdc
> --
>
> Key: CASSANDRA-17422
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17422
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Josh McKenzie
>Priority: Normal
> Fix For: 4.0.x
>
> Attachments: CASSANDRA-17422.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Branch: 4.0
> https://ci-cassandra.apache.org/job/Cassandra-4.0/350/testReport/org.apache.cassandra.net/OutboundMessageQueueTest/testRemove_cdc/
> {code}
> java.lang.NullPointerException
> at
> org.apache.cassandra.net.OutboundMessageQueueTest.testRemove(OutboundMessageQueueTest.java:91)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> Failure: 1 of 3
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17422) Test Failure: org.apache.cassandra.net.OutboundMessageQueueTest.testRemove-cdc
[
https://issues.apache.org/jira/browse/CASSANDRA-17422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557745#comment-17557745
]
shylaja kokoori commented on CASSANDRA-17422:
-
I have attached a patch for review. Working on pull request
> Test Failure: org.apache.cassandra.net.OutboundMessageQueueTest.testRemove-cdc
> --
>
> Key: CASSANDRA-17422
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17422
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Josh McKenzie
>Priority: Normal
> Fix For: 4.0.x
>
> Attachments: CASSANDRA-17422.patch
>
>
> Branch: 4.0
> https://ci-cassandra.apache.org/job/Cassandra-4.0/350/testReport/org.apache.cassandra.net/OutboundMessageQueueTest/testRemove_cdc/
> {code}
> java.lang.NullPointerException
> at
> org.apache.cassandra.net.OutboundMessageQueueTest.testRemove(OutboundMessageQueueTest.java:91)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> Failure: 1 of 3
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17422) Test Failure: org.apache.cassandra.net.OutboundMessageQueueTest.testRemove-cdc
[
https://issues.apache.org/jira/browse/CASSANDRA-17422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shylaja kokoori updated CASSANDRA-17422:
Attachment: CASSANDRA-17422.patch
> Test Failure: org.apache.cassandra.net.OutboundMessageQueueTest.testRemove-cdc
> --
>
> Key: CASSANDRA-17422
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17422
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Josh McKenzie
>Priority: Normal
> Fix For: 4.0.x
>
> Attachments: CASSANDRA-17422.patch
>
>
> Branch: 4.0
> https://ci-cassandra.apache.org/job/Cassandra-4.0/350/testReport/org.apache.cassandra.net/OutboundMessageQueueTest/testRemove_cdc/
> {code}
> java.lang.NullPointerException
> at
> org.apache.cassandra.net.OutboundMessageQueueTest.testRemove(OutboundMessageQueueTest.java:91)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> Failure: 1 of 3
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17422) Test Failure: org.apache.cassandra.net.OutboundMessageQueueTest.testRemove-cdc
[
https://issues.apache.org/jira/browse/CASSANDRA-17422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shylaja kokoori updated CASSANDRA-17422:
Mentor: Brandon Williams
Test and Documentation Plan: Executed the test
_OutboundMessageQueueTest.testRemove_ more than 100K iterations on my box
multiple times
Status: Patch Available (was: Open)
> Test Failure: org.apache.cassandra.net.OutboundMessageQueueTest.testRemove-cdc
> --
>
> Key: CASSANDRA-17422
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17422
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Josh McKenzie
>Priority: Normal
> Fix For: 4.0.x
>
>
> Branch: 4.0
> https://ci-cassandra.apache.org/job/Cassandra-4.0/350/testReport/org.apache.cassandra.net/OutboundMessageQueueTest/testRemove_cdc/
> {code}
> java.lang.NullPointerException
> at
> org.apache.cassandra.net.OutboundMessageQueueTest.testRemove(OutboundMessageQueueTest.java:91)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> Failure: 1 of 3
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17422) Test Failure: org.apache.cassandra.net.OutboundMessageQueueTest.testRemove-cdc
[
https://issues.apache.org/jira/browse/CASSANDRA-17422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17555776#comment-17555776
]
shylaja kokoori commented on CASSANDRA-17422:
-
>From my analysis, it looks like a timing issue.
Occasionally in this test, by the time code flow reaches the line
_try (OutboundMessageQueue.WithLock lock = queue.lockOrCallback(0, () -> {}))_
lock acquired on the queue prior to it has not been released. Therefore, lock
has a null value resulting in the {_}NullPointerException{_}.
When I added _Uninterruptibles.awaitUninterruptibly(lockUntil)_ before the line
of code mentioned above, test ran for a long time without failing.
Another option to fix this test, perhaps is to add a callback and check for the
content of the queue in the callback.
> Test Failure: org.apache.cassandra.net.OutboundMessageQueueTest.testRemove-cdc
> --
>
> Key: CASSANDRA-17422
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17422
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Josh McKenzie
>Priority: Normal
> Fix For: 4.0.x
>
>
> Branch: 4.0
> https://ci-cassandra.apache.org/job/Cassandra-4.0/350/testReport/org.apache.cassandra.net/OutboundMessageQueueTest/testRemove_cdc/
> {code}
> java.lang.NullPointerException
> at
> org.apache.cassandra.net.OutboundMessageQueueTest.testRemove(OutboundMessageQueueTest.java:91)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> Failure: 1 of 3
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17436124#comment-17436124 ] shylaja kokoori commented on CASSANDRA-13981: - The code is under active development. The implementation has been modified to make use of the pluggable memtable interface introduced by CEP11. We are in the process of making the code feature complete while we wait for resolution of following JIRAs. CASSANDRA-17034 CEP-11: Memtable API implementation CASSANDRA-6936 Make all byte representations of types comparable by their unsigned byte representation only > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Normal > Fix For: 4.x > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > in-mem-cassandra-2.1.patch, readme.txt, readme2.1.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17253777#comment-17253777 ] shylaja kokoori commented on CASSANDRA-13981: - [~vongosling] We are waiting on version 4.0 release and an interface such as the one described in CASSANDRA-13474, to plugin the persistent memory code. > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Normal > Fix For: 4.x > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > in-mem-cassandra-2.1.patch, readme.txt, readme2.1.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-9633) Add ability to encrypt sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250665#comment-17250665 ] shylaja kokoori commented on CASSANDRA-9633: Hi All, I have ported Jason Brown's implementation to 4.0. Few related questions, 1) In cassandra.yaml file, for transparent_data_encryption_options, +keystore_password+ and +key_password+ are provided in plain text. Can we rely on Linux file system permission to protect the information? 2) During a read, when data is transferred between a node containing the data and the coordinator node, can we rely on encryption over wire to provide confidentiality? 3) I am not too familiar with streaming. When zero copy streaming is enabled and SSTables are transferred in its entirety, can I assume both the nodes have same keys or should key exchange happen? 4) Our security expert says, no more than 2^56 16 byte blocks (~1 EB) should be encrypted with a single key. Depending on the amount of writes that happen, that is probably several years of data encryption. Should that be a concern? > Add ability to encrypt sstables > --- > > Key: CASSANDRA-9633 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9633 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Core >Reporter: Jason Brown >Assignee: shylaja kokoori >Priority: Normal > Labels: encryption, security, sstable > Fix For: 4.x > > > Add option to allow encrypting of sstables. > I have a version of this functionality built on cassandra 2.0 that > piggy-backs on the existing sstable compression functionality and ICompressor > interface (similar in nature to what DataStax Enterprise does). However, if > we're adding the feature to the main OSS product, I'm not sure if we want to > use the pluggable compression framework or if it's worth investigating a > different path. I think there's a lot of upside in reusing the sstable > compression scheme, but perhaps add a new component in cqlsh for table > encryption and a corresponding field in CFMD. > Encryption configuration in the yaml can use the same mechanism as > CASSANDRA-6018 (which is currently pending internal review). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15809) ASF CI builds for JDK11
[ https://issues.apache.org/jira/browse/CASSANDRA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157153#comment-17157153 ] shylaja kokoori commented on CASSANDRA-15809: - [~mck] Your changes look good > ASF CI builds for JDK11 > --- > > Key: CASSANDRA-15809 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15809 > Project: Cassandra > Issue Type: Task > Components: Build, CI >Reporter: Michael Semb Wever >Assignee: shylaja kokoori >Priority: Normal > Fix For: 4.0-beta > > Attachments: Screenshot 2020-05-13 at 09.39.56.png, > build_with_multiple_jdk.patch > > > ASF CI builds today only run on JDK1.8 > On the Jenkins cluster JDKs from 1.4 through to 15 are available. See > attached screenshot for naming specifics. > This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for > parity to CircleCI's workflows. (There is also the question about > testing/running on Cassandra-2.2 which needs to support JDK1.7, though > Cassandra-2.2 is nearing EOL.) > > The JDK is specified in the groovy DSL: > [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11] > > > Some examples: > * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with > both JDK1.8 and JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8] > * CircleCI JDK11 workflow example. This builds and tests only with JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306] > * Jenkins Cqlshlib tests showing matrix builds: > ** > [https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/] > > Background > thread:[https://lists.apache.org/thread.html/rbaeb960901fa53b50227b37d64e5a456b68f749f4229c6e3e086ff85%40%3Cdev.cassandra.apache.org%3E] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15809) ASF CI builds for JDK11
[ https://issues.apache.org/jira/browse/CASSANDRA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shylaja kokoori updated CASSANDRA-15809: Test and Documentation Plan: With this patch, Jenkins will display 2 JDK configurations jdk=JDK 1.8 (latest) & jdk=JDK 11 (latest) for cassandra-trunk-artifacts & cassandra-devbranch-artifacts. A build issued will (depending on the availability of executors) start both the builds parallelly. I have tested the build artifacts on trunk. [^build_with_multiple_jdk.patch] Status: Patch Available (was: In Progress) Currently only JDK8 & 11 have been enabled based on Mick's suggestion. I can add others as needed. Like David mentioned JDK14 will need GC change. Removal of Nashorn Javascript Engine might cause a build problem JDK15 onwards > ASF CI builds for JDK11 > --- > > Key: CASSANDRA-15809 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15809 > Project: Cassandra > Issue Type: Task > Components: Build, CI >Reporter: Michael Semb Wever >Assignee: shylaja kokoori >Priority: Normal > Fix For: 4.0-beta > > Attachments: Screenshot 2020-05-13 at 09.39.56.png, > build_with_multiple_jdk.patch > > > ASF CI builds today only run on JDK1.8 > On the Jenkins cluster JDKs from 1.4 through to 15 are available. See > attached screenshot for naming specifics. > This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for > parity to CircleCI's workflows. (There is also the question about > testing/running on Cassandra-2.2 which needs to support JDK1.7, though > Cassandra-2.2 is nearing EOL.) > > The JDK is specified in the groovy DSL: > [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11] > > > Some examples: > * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with > both JDK1.8 and JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8] > * CircleCI JDK11 workflow example. This builds and tests only with JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306] > * Jenkins Cqlshlib tests showing matrix builds: > ** > [https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/] > > Background > thread:[https://lists.apache.org/thread.html/rbaeb960901fa53b50227b37d64e5a456b68f749f4229c6e3e086ff85%40%3Cdev.cassandra.apache.org%3E] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15809) ASF CI builds for JDK11
[ https://issues.apache.org/jira/browse/CASSANDRA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shylaja kokoori updated CASSANDRA-15809: Attachment: build_with_multiple_jdk.patch > ASF CI builds for JDK11 > --- > > Key: CASSANDRA-15809 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15809 > Project: Cassandra > Issue Type: Task > Components: Build, CI >Reporter: Michael Semb Wever >Assignee: shylaja kokoori >Priority: Normal > Fix For: 4.0-beta > > Attachments: Screenshot 2020-05-13 at 09.39.56.png, > build_with_multiple_jdk.patch > > > ASF CI builds today only run on JDK1.8 > On the Jenkins cluster JDKs from 1.4 through to 15 are available. See > attached screenshot for naming specifics. > This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for > parity to CircleCI's workflows. (There is also the question about > testing/running on Cassandra-2.2 which needs to support JDK1.7, though > Cassandra-2.2 is nearing EOL.) > > The JDK is specified in the groovy DSL: > [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11] > > > Some examples: > * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with > both JDK1.8 and JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8] > * CircleCI JDK11 workflow example. This builds and tests only with JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306] > * Jenkins Cqlshlib tests showing matrix builds: > ** > [https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/] > > Background > thread:[https://lists.apache.org/thread.html/rbaeb960901fa53b50227b37d64e5a456b68f749f4229c6e3e086ff85%40%3Cdev.cassandra.apache.org%3E] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15809) ASF CI builds for JDK11
[ https://issues.apache.org/jira/browse/CASSANDRA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139900#comment-17139900 ] shylaja kokoori commented on CASSANDRA-15809: - I have a patch ready. Will upload soon > ASF CI builds for JDK11 > --- > > Key: CASSANDRA-15809 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15809 > Project: Cassandra > Issue Type: Task > Components: Build, CI >Reporter: Michael Semb Wever >Assignee: shylaja kokoori >Priority: Normal > Fix For: 4.0-beta > > Attachments: Screenshot 2020-05-13 at 09.39.56.png > > > ASF CI builds today only run on JDK1.8 > On the Jenkins cluster JDKs from 1.4 through to 15 are available. See > attached screenshot for naming specifics. > This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for > parity to CircleCI's workflows. (There is also the question about > testing/running on Cassandra-2.2 which needs to support JDK1.7, though > Cassandra-2.2 is nearing EOL.) > > The JDK is specified in the groovy DSL: > [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11] > > > Some examples: > * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with > both JDK1.8 and JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8] > * CircleCI JDK11 workflow example. This builds and tests only with JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306] > * Jenkins Cqlshlib tests showing matrix builds: > ** > [https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/] > > Background > thread:[https://lists.apache.org/thread.html/rbaeb960901fa53b50227b37d64e5a456b68f749f4229c6e3e086ff85%40%3Cdev.cassandra.apache.org%3E] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742600#comment-16742600 ] shylaja kokoori commented on CASSANDRA-13981: - We have a new implementation of the [persistent memory storage engine|https://github.com/shyla226/cassandra/tree/13981_llpl_engine] based on the design and sketch that [~jasobrown] proposed. This implementation uses an improved version of [LLPL|https://github.com/pmem/llpl]. As suggested in Jason's design, this engine uses the Adaptive Radix Tree as the core data structure. Also, as suggested, it uses the default Cassandra serialization / deserialization code. The engine implements creates, reads, updates, deletes, and table schema modification. We have tested these features on a single node using the insanity schema with > 2 billion partitions. We would like to get feedback on the storage engine and guidance from the Cassandra community on next steps. In particular, we have implemented placeholder read path code because the pluggable storage engine read path API is not yet resolved (https://issues.apache.org/jira/browse/CASSANDRA-14117). > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.x > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > in-mem-cassandra-2.1.patch, readme.txt, readme2.1.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1650#comment-1650 ] shylaja kokoori commented on CASSANDRA-13981: - Uploading patch version 2.1. A branch with this patch applied is at https://github.com/shyla226/cassandra/tree/13981. This patch ties up work that was in progress to improve performance and persistent memory footprint of the PCJ-based design. We are now exploring the design & sketch that Jason Brown has proposed. [^in-mem-cassandra-2.1.patch] [^readme2.1.txt] > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.x > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > in-mem-cassandra-2.1.patch, readme.txt, readme2.1.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shylaja kokoori updated CASSANDRA-13981: Attachment: readme2.1.txt > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.x > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > in-mem-cassandra-2.1.patch, readme.txt, readme2.1.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shylaja kokoori updated CASSANDRA-13981: Attachment: in-mem-cassandra-2.1.patch > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.x > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > in-mem-cassandra-2.1.patch, readme.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479709#comment-16479709 ] shylaja kokoori commented on CASSANDRA-13981: - Uploading patch version 2.0. This patch bypasses the use of memtable in the read/write path. Please refer to attached readme2_0.txt for more details. > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.0 > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > readme.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shylaja kokoori updated CASSANDRA-13981: Attachment: readme2_0.txt > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.0 > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > readme.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shylaja kokoori updated CASSANDRA-13981: Attachment: in-mem-cassandra-2.0.patch > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.0 > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > readme.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12787) Reduce instanceOf() type checking to improve performance
[ https://issues.apache.org/jira/browse/CASSANDRA-12787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15589148#comment-15589148 ] shylaja kokoori commented on CASSANDRA-12787: - Thank you for the suggestion and comments. I am in the process of collecting some data on what change can produce a result similar to the change we did in the JVM. I have some ideas will post an update soon. On the question about tools used, we use an internal tool to measure CPU cycles. Linux perf and VTune should be able to provide similar information. > Reduce instanceOf() type checking to improve performance > > > Key: CASSANDRA-12787 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12787 > Project: Cassandra > Issue Type: Improvement > Environment: The tests and examples stated were run on: > Intel (R) Xeon (R) CPU E5-2699 v3 @ 2.30GHz, HT Enabled > Oracle JDK 1.8 > Cassandra 3.10-SNAPSHOT > Linux kernel 4.7 >Reporter: shylaja kokoori > Fix For: 3.x > > Attachments: instanceof.png, reduce_instanceof_typechecking.pdf > > > While performance profiling Cassandra with cassandra-stress test on a pure > write workload, we noticed that one of the hot methods for Cassandra server > is the compareTo( PartitionPosition pos) function in > org.apache.cassandra.db.DecoratedKey. The actual root cause of the problem is > a slowdown in Java's instanceof operator. > Our initial performance testing using a hacked JVM shows about 61% increase > in throughput and about 15% reduction in 99 percentile latency. Data shows > that improvements are from the removal of thread contention caused by > Instanceof operator. > Currently, we are working on a JDK fix to solve this issue. In the meantime, > we think it will be beneficial to address this issue at Java application > level as well. We are in the process of creating a patch to Cassandra > replacing instanceof check with virtual method calls. The patch will be > available in this thread for review soon. Please let us know your feedback > and comments. > For more details please refer to the attached pdf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-12787) Reduce instanceOf() type checking to improve performance
shylaja kokoori created CASSANDRA-12787: --- Summary: Reduce instanceOf() type checking to improve performance Key: CASSANDRA-12787 URL: https://issues.apache.org/jira/browse/CASSANDRA-12787 Project: Cassandra Issue Type: Improvement Environment: The tests and examples stated were run on: Intel (R) Xeon (R) CPU E5-2699 v3 @ 2.30GHz, HT Enabled Oracle JDK 1.8 Cassandra 3.10-SNAPSHOT Linux kernel 4.7 Reporter: shylaja kokoori Fix For: 3.x Attachments: reduce_instanceof_typechecking.pdf While performance profiling Cassandra with cassandra-stress test on a pure write workload, we noticed that one of the hot methods for Cassandra server is the compareTo( PartitionPosition pos) function in org.apache.cassandra.db.DecoratedKey. The actual root cause of the problem is a slowdown in Java's instanceof operator. Our initial performance testing using a hacked JVM shows about 61% increase in throughput and about 15% reduction in 99 percentile latency. Data shows that improvements are from the removal of thread contention caused by Instanceof operator. Currently, we are working on a JDK fix to solve this issue. In the meantime, we think it will be beneficial to address this issue at Java application level as well. We are in the process of creating a patch to Cassandra replacing instanceof check with virtual method calls. The patch will be available in this thread for review soon. Please let us know your feedback and comments. For more details please refer to the attached pdf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11987) Cassandra support for JDK 9
shylaja kokoori created CASSANDRA-11987: --- Summary: Cassandra support for JDK 9 Key: CASSANDRA-11987 URL: https://issues.apache.org/jira/browse/CASSANDRA-11987 Project: Cassandra Issue Type: Bug Components: Core Environment: JDK9 Reporter: shylaja kokoori Hi, I tried to compile Cassandra with JDK 9 and ran into compilation issues because monitorEnter/monitorExit functions have been removed from sun.misc.Unsafe in JDK9 (http://mail.openjdk.java.net/pipermail/jdk9-hs-rt-changes/2015-January/000773.html). Is there a specific reason why Cassandra uses these Unsafe APIs instead of say java.util.concurrent.locks? Thanks, shylaja -- This message was sent by Atlassian JIRA (v6.3.4#6332)
