[jira] [Updated] (CASSANDRA-16365) Cannot run tests on Java 11 with coverage analysis
[
https://issues.apache.org/jira/browse/CASSANDRA-16365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacek Lewandowski updated CASSANDRA-16365:
--
Source Control Link: https://github.com/apache/cassandra/pull/861
> Cannot run tests on Java 11 with coverage analysis
> --
>
> Key: CASSANDRA-16365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16365
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 4.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When running tests with coverage analysis on Java 11, we get the problem as
> follows:
> {noformat}
> [junit-timeout] FATAL ERROR in native method: processing of -javaagent
> failed, processJavaStart failed
> [junit-timeout] Exception in thread "main"
> java.lang.reflect.InvocationTargetException
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout] at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)
> [junit-timeout] at
> java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
> [junit-timeout] at
> java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:525)
> [junit-timeout] Caused by: java.lang.RuntimeException: Class java/util/UUID
> could not be instrumented.
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:139)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:100)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.PreMain.createRuntime(PreMain.java:55)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.PreMain.premain(PreMain.java:47)
> [junit-timeout] ... 6 more
> [junit-timeout] Caused by: java.lang.NoSuchFieldException: $jacocoAccess
> [junit-timeout] at java.base/java.lang.Class.getField(Class.java:1999)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:137)
> [junit-timeout] ... 9 more
> [junit-timeout] *** java.lang.instrument ASSERTION FAILED ***: "result" with
> message agent load/premain call failed at
> src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 422
> {noformat}
> It is caused by too old Jacoco which does seem to work well with Java 11.
> Upgrading Jacoco to the newest version 0.8.6 fixes the problem
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16365) Cannot run tests on Java 11 with coverage analysis
[
https://issues.apache.org/jira/browse/CASSANDRA-16365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacek Lewandowski updated CASSANDRA-16365:
--
Test and Documentation Plan: Run unit tests with code coverage analysis
with Java 8 and Java 11
Status: Patch Available (was: In Progress)
> Cannot run tests on Java 11 with coverage analysis
> --
>
> Key: CASSANDRA-16365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16365
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 4.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When running tests with coverage analysis on Java 11, we get the problem as
> follows:
> {noformat}
> [junit-timeout] FATAL ERROR in native method: processing of -javaagent
> failed, processJavaStart failed
> [junit-timeout] Exception in thread "main"
> java.lang.reflect.InvocationTargetException
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout] at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)
> [junit-timeout] at
> java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
> [junit-timeout] at
> java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:525)
> [junit-timeout] Caused by: java.lang.RuntimeException: Class java/util/UUID
> could not be instrumented.
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:139)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:100)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.PreMain.createRuntime(PreMain.java:55)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.PreMain.premain(PreMain.java:47)
> [junit-timeout] ... 6 more
> [junit-timeout] Caused by: java.lang.NoSuchFieldException: $jacocoAccess
> [junit-timeout] at java.base/java.lang.Class.getField(Class.java:1999)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:137)
> [junit-timeout] ... 9 more
> [junit-timeout] *** java.lang.instrument ASSERTION FAILED ***: "result" with
> message agent load/premain call failed at
> src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 422
> {noformat}
> It is caused by too old Jacoco which does seem to work well with Java 11.
> Upgrading Jacoco to the newest version 0.8.6 fixes the problem
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16365) Cannot run tests on Java 11 with coverage analysis
[
https://issues.apache.org/jira/browse/CASSANDRA-16365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacek Lewandowski updated CASSANDRA-16365:
--
Bug Category: Parent values: Code(13163)Level 1 values: Bug - Unclear
Impact(13164)
Complexity: Low Hanging Fruit
Discovered By: User Report
Severity: Low
Status: Open (was: Triage Needed)
> Cannot run tests on Java 11 with coverage analysis
> --
>
> Key: CASSANDRA-16365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16365
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 4.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When running tests with coverage analysis on Java 11, we get the problem as
> follows:
> {noformat}
> [junit-timeout] FATAL ERROR in native method: processing of -javaagent
> failed, processJavaStart failed
> [junit-timeout] Exception in thread "main"
> java.lang.reflect.InvocationTargetException
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout] at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)
> [junit-timeout] at
> java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
> [junit-timeout] at
> java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:525)
> [junit-timeout] Caused by: java.lang.RuntimeException: Class java/util/UUID
> could not be instrumented.
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:139)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:100)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.PreMain.createRuntime(PreMain.java:55)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.PreMain.premain(PreMain.java:47)
> [junit-timeout] ... 6 more
> [junit-timeout] Caused by: java.lang.NoSuchFieldException: $jacocoAccess
> [junit-timeout] at java.base/java.lang.Class.getField(Class.java:1999)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:137)
> [junit-timeout] ... 9 more
> [junit-timeout] *** java.lang.instrument ASSERTION FAILED ***: "result" with
> message agent load/premain call failed at
> src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 422
> {noformat}
> It is caused by too old Jacoco which does seem to work well with Java 11.
> Upgrading Jacoco to the newest version 0.8.6 fixes the problem
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16365) Cannot run tests on Java 11 with coverage analysis
Jacek Lewandowski created CASSANDRA-16365:
-
Summary: Cannot run tests on Java 11 with coverage analysis
Key: CASSANDRA-16365
URL: https://issues.apache.org/jira/browse/CASSANDRA-16365
Project: Cassandra
Issue Type: Bug
Components: Build
Reporter: Jacek Lewandowski
When running tests with coverage analysis on Java 11, we get the problem as
follows:
{noformat}
[junit-timeout] FATAL ERROR in native method: processing of -javaagent failed,
processJavaStart failed
[junit-timeout] Exception in thread "main"
java.lang.reflect.InvocationTargetException
[junit-timeout] at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit-timeout] at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit-timeout] at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit-timeout] at
java.base/java.lang.reflect.Method.invoke(Method.java:566)
[junit-timeout] at
java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
[junit-timeout] at
java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:525)
[junit-timeout] Caused by: java.lang.RuntimeException: Class java/util/UUID
could not be instrumented.
[junit-timeout] at
org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:139)
[junit-timeout] at
org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:100)
[junit-timeout] at
org.jacoco.agent.rt.internal_b0d6a23.PreMain.createRuntime(PreMain.java:55)
[junit-timeout] at
org.jacoco.agent.rt.internal_b0d6a23.PreMain.premain(PreMain.java:47)
[junit-timeout] ... 6 more
[junit-timeout] Caused by: java.lang.NoSuchFieldException: $jacocoAccess
[junit-timeout] at java.base/java.lang.Class.getField(Class.java:1999)
[junit-timeout] at
org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:137)
[junit-timeout] ... 9 more
[junit-timeout] *** java.lang.instrument ASSERTION FAILED ***: "result" with
message agent load/premain call failed at
src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 422
{noformat}
It is caused by too old Jacoco which does seem to work well with Java 11.
Upgrading Jacoco to the newest version 0.8.6 fixes the problem
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16365) Cannot run tests on Java 11 with coverage analysis
[
https://issues.apache.org/jira/browse/CASSANDRA-16365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacek Lewandowski updated CASSANDRA-16365:
--
Fix Version/s: 4.0
> Cannot run tests on Java 11 with coverage analysis
> --
>
> Key: CASSANDRA-16365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16365
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 4.0
>
>
> When running tests with coverage analysis on Java 11, we get the problem as
> follows:
> {noformat}
> [junit-timeout] FATAL ERROR in native method: processing of -javaagent
> failed, processJavaStart failed
> [junit-timeout] Exception in thread "main"
> java.lang.reflect.InvocationTargetException
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout] at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)
> [junit-timeout] at
> java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
> [junit-timeout] at
> java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:525)
> [junit-timeout] Caused by: java.lang.RuntimeException: Class java/util/UUID
> could not be instrumented.
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:139)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:100)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.PreMain.createRuntime(PreMain.java:55)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.PreMain.premain(PreMain.java:47)
> [junit-timeout] ... 6 more
> [junit-timeout] Caused by: java.lang.NoSuchFieldException: $jacocoAccess
> [junit-timeout] at java.base/java.lang.Class.getField(Class.java:1999)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:137)
> [junit-timeout] ... 9 more
> [junit-timeout] *** java.lang.instrument ASSERTION FAILED ***: "result" with
> message agent load/premain call failed at
> src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 422
> {noformat}
> It is caused by too old Jacoco which does seem to work well with Java 11.
> Upgrading Jacoco to the newest version 0.8.6 fixes the problem
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16365) Cannot run tests on Java 11 with coverage analysis
[
https://issues.apache.org/jira/browse/CASSANDRA-16365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacek Lewandowski reassigned CASSANDRA-16365:
-
Assignee: Jacek Lewandowski
> Cannot run tests on Java 11 with coverage analysis
> --
>
> Key: CASSANDRA-16365
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16365
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>
> When running tests with coverage analysis on Java 11, we get the problem as
> follows:
> {noformat}
> [junit-timeout] FATAL ERROR in native method: processing of -javaagent
> failed, processJavaStart failed
> [junit-timeout] Exception in thread "main"
> java.lang.reflect.InvocationTargetException
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit-timeout] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit-timeout] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit-timeout] at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)
> [junit-timeout] at
> java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
> [junit-timeout] at
> java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:525)
> [junit-timeout] Caused by: java.lang.RuntimeException: Class java/util/UUID
> could not be instrumented.
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:139)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:100)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.PreMain.createRuntime(PreMain.java:55)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.PreMain.premain(PreMain.java:47)
> [junit-timeout] ... 6 more
> [junit-timeout] Caused by: java.lang.NoSuchFieldException: $jacocoAccess
> [junit-timeout] at java.base/java.lang.Class.getField(Class.java:1999)
> [junit-timeout] at
> org.jacoco.agent.rt.internal_b0d6a23.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:137)
> [junit-timeout] ... 9 more
> [junit-timeout] *** java.lang.instrument ASSERTION FAILED ***: "result" with
> message agent load/premain call failed at
> src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 422
> {noformat}
> It is caused by too old Jacoco which does seem to work well with Java 11.
> Upgrading Jacoco to the newest version 0.8.6 fixes the problem
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16333) Document provide_overlapping_tombstones compaction option
[ https://issues.apache.org/jira/browse/CASSANDRA-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumanth Pasupuleti reassigned CASSANDRA-16333: -- Assignee: Sumanth Pasupuleti > Document provide_overlapping_tombstones compaction option > - > > Key: CASSANDRA-16333 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16333 > Project: Cassandra > Issue Type: Improvement > Components: Documentation/Website >Reporter: Paulo Motta >Assignee: Sumanth Pasupuleti >Priority: Normal > > This option was added on CASSANDRA-7019 but it's not documented. We should > add it to > https://cassandra.apache.org/doc/latest/operating/compaction/index.html. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17253235#comment-17253235 ] Von Gosling commented on CASSANDRA-13981: - Any update for this? > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Normal > Fix For: 4.x > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > in-mem-cassandra-2.1.patch, readme.txt, readme2.1.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16318) Memtable heap size is severely underestimated
[
https://issues.apache.org/jira/browse/CASSANDRA-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17253188#comment-17253188
]
Ekaterina Dimitrova edited comment on CASSANDRA-16318 at 12/22/20, 12:53 AM:
-
Patch [here |https://github.com/ekaterinadimitrova2/cassandra/pull/80]
CI run: [Java 8
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/545/workflows/f797bf58-b572-4d8c-831e-a61936d23624]
and [Java 11
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/545/workflows/6dbe4bca-b51d-473e-bfc4-1bbf6c4ff26d]
>From the preliminary test I pushed looks like the issue was solved. I will
>check/publish full report of the microbench tests when I am back to the office
>on the 4th January. Pushed the patch here in case Branimir or anyone else
>wants to look at it in the meanwhile.
was (Author: e.dimitrova):
Patch [here |https://github.com/ekaterinadimitrova2/cassandra/pull/80]
CI run: [Java 8
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/545/workflows/f797bf58-b572-4d8c-831e-a61936d23624]
and [Java 11
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/545/workflows/6dbe4bca-b51d-473e-bfc4-1bbf6c4ff26d]
>From the preliminary test I pushed looks like the issue was solved. I will
>check/publish full report of the microbench tests when I am back to the office
>on the 4th January. Pushed the patch here in case Branimir or anyone else is
>willing to look at it in the meanwhile.
> Memtable heap size is severely underestimated
> -
>
> Key: CASSANDRA-16318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16318
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Memtable
>Reporter: Branimir Lambov
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: image-2020-12-09-10-57-21-994.png,
> image-2020-12-09-11-01-31-273.png
>
>
> We seem to be estimating the size of the on-heap memtable metadata to be
> around half of what it actually is. For example, during a [read benchmark
> which writes 1 million single-long
> rows|https://github.com/blambov/cassandra/blob/memtable-heap/test/microbench/org/apache/cassandra/test/microbench/instance/ReadTestSmallPartitions.java]
> the memtable reports
> {code}
> 100 ops, 58.174MiB serialized bytes, 385.284MiB (19%) on heap, 0.000KiB
> (0%) off-heap
> {code}
> while a heap dump taken at this point:
> !image-2020-12-09-10-57-21-994.png!
> lists an usage of about 666MB altogether.
> Switching to {{offheap_objects}}, the reported numbers are
> {code}
> 100 ops, 58.174MiB serialized bytes, 233.650MiB (12%) on heap, 53.406MiB
> (3%) off-heap
> {code}
> while actual heap usage:
> !image-2020-12-09-11-01-31-273.png!
> is about 442MB.
> Looking at the code we definitely are not counting the
> {{AtomicBTreePartition.Holder}}, {{EncodingStats}}, liveness and deletion
> info objects associated with each partition, and most probably others.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16318) Memtable heap size is severely underestimated
[
https://issues.apache.org/jira/browse/CASSANDRA-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17253188#comment-17253188
]
Ekaterina Dimitrova edited comment on CASSANDRA-16318 at 12/22/20, 12:53 AM:
-
Patch [here |https://github.com/ekaterinadimitrova2/cassandra/pull/80]
CI run: [Java 8
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/545/workflows/f797bf58-b572-4d8c-831e-a61936d23624]
and [Java 11
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/545/workflows/6dbe4bca-b51d-473e-bfc4-1bbf6c4ff26d]
>From the preliminary test I pushed looks like the issue was solved. I will
>check/publish full report of the microbench tests when I am back to the office
>on the 4th January. Pushed the patch here in case Branimir or anyone else is
>willing to look at it in the meanwhile.
was (Author: e.dimitrova):
Patch [here |https://github.com/ekaterinadimitrova2/cassandra/pull/80]
CI run: [Java 8
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/545/workflows/f797bf58-b572-4d8c-831e-a61936d23624]
and [Java 11
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/545/workflows/6dbe4bca-b51d-473e-bfc4-1bbf6c4ff26d]
>From the preliminary test I pushed looks like the issue was solved. I will
>check/publish full report of the microbench tests when I am back to the office
>on the 4th January. Pushed the patch here in case anyone is willing to look at
>it in the meanwhile.
> Memtable heap size is severely underestimated
> -
>
> Key: CASSANDRA-16318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16318
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Memtable
>Reporter: Branimir Lambov
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: image-2020-12-09-10-57-21-994.png,
> image-2020-12-09-11-01-31-273.png
>
>
> We seem to be estimating the size of the on-heap memtable metadata to be
> around half of what it actually is. For example, during a [read benchmark
> which writes 1 million single-long
> rows|https://github.com/blambov/cassandra/blob/memtable-heap/test/microbench/org/apache/cassandra/test/microbench/instance/ReadTestSmallPartitions.java]
> the memtable reports
> {code}
> 100 ops, 58.174MiB serialized bytes, 385.284MiB (19%) on heap, 0.000KiB
> (0%) off-heap
> {code}
> while a heap dump taken at this point:
> !image-2020-12-09-10-57-21-994.png!
> lists an usage of about 666MB altogether.
> Switching to {{offheap_objects}}, the reported numbers are
> {code}
> 100 ops, 58.174MiB serialized bytes, 233.650MiB (12%) on heap, 53.406MiB
> (3%) off-heap
> {code}
> while actual heap usage:
> !image-2020-12-09-11-01-31-273.png!
> is about 442MB.
> Looking at the code we definitely are not counting the
> {{AtomicBTreePartition.Holder}}, {{EncodingStats}}, liveness and deletion
> info objects associated with each partition, and most probably others.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16318) Memtable heap size is severely underestimated
[
https://issues.apache.org/jira/browse/CASSANDRA-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17253188#comment-17253188
]
Ekaterina Dimitrova edited comment on CASSANDRA-16318 at 12/22/20, 12:53 AM:
-
Patch [here |https://github.com/ekaterinadimitrova2/cassandra/pull/80]
CI run: [Java 8
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/545/workflows/f797bf58-b572-4d8c-831e-a61936d23624]
and [Java 11
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/545/workflows/6dbe4bca-b51d-473e-bfc4-1bbf6c4ff26d]
>From the preliminary test I pushed looks like the issue was solved. I will
>check/publish full report of the microbench tests when I am back to the office
>on the 4th January. Pushed the patch here in case anyone is willing to look at
>it in the meanwhile.
was (Author: e.dimitrova):
Patch [here |https://github.com/ekaterinadimitrova2/cassandra/pull/80]
CI run: [Java 8
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/545/workflows/f797bf58-b572-4d8c-831e-a61936d23624]
and [Java 11
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/545/workflows/6dbe4bca-b51d-473e-bfc4-1bbf6c4ff26d]
>From the preliminary test I pushed looks like the issue was solved. I will
>check/publish full report of the microbench tests when I am back to the office
>on the 4th January.
> Memtable heap size is severely underestimated
> -
>
> Key: CASSANDRA-16318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16318
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Memtable
>Reporter: Branimir Lambov
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: image-2020-12-09-10-57-21-994.png,
> image-2020-12-09-11-01-31-273.png
>
>
> We seem to be estimating the size of the on-heap memtable metadata to be
> around half of what it actually is. For example, during a [read benchmark
> which writes 1 million single-long
> rows|https://github.com/blambov/cassandra/blob/memtable-heap/test/microbench/org/apache/cassandra/test/microbench/instance/ReadTestSmallPartitions.java]
> the memtable reports
> {code}
> 100 ops, 58.174MiB serialized bytes, 385.284MiB (19%) on heap, 0.000KiB
> (0%) off-heap
> {code}
> while a heap dump taken at this point:
> !image-2020-12-09-10-57-21-994.png!
> lists an usage of about 666MB altogether.
> Switching to {{offheap_objects}}, the reported numbers are
> {code}
> 100 ops, 58.174MiB serialized bytes, 233.650MiB (12%) on heap, 53.406MiB
> (3%) off-heap
> {code}
> while actual heap usage:
> !image-2020-12-09-11-01-31-273.png!
> is about 442MB.
> Looking at the code we definitely are not counting the
> {{AtomicBTreePartition.Holder}}, {{EncodingStats}}, liveness and deletion
> info objects associated with each partition, and most probably others.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16318) Memtable heap size is severely underestimated
[
https://issues.apache.org/jira/browse/CASSANDRA-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-16318:
Test and Documentation Plan:
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/545/workflows/f797bf58-b572-4d8c-831e-a61936d23624
Status: Patch Available (was: In Progress)
> Memtable heap size is severely underestimated
> -
>
> Key: CASSANDRA-16318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16318
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Memtable
>Reporter: Branimir Lambov
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: image-2020-12-09-10-57-21-994.png,
> image-2020-12-09-11-01-31-273.png
>
>
> We seem to be estimating the size of the on-heap memtable metadata to be
> around half of what it actually is. For example, during a [read benchmark
> which writes 1 million single-long
> rows|https://github.com/blambov/cassandra/blob/memtable-heap/test/microbench/org/apache/cassandra/test/microbench/instance/ReadTestSmallPartitions.java]
> the memtable reports
> {code}
> 100 ops, 58.174MiB serialized bytes, 385.284MiB (19%) on heap, 0.000KiB
> (0%) off-heap
> {code}
> while a heap dump taken at this point:
> !image-2020-12-09-10-57-21-994.png!
> lists an usage of about 666MB altogether.
> Switching to {{offheap_objects}}, the reported numbers are
> {code}
> 100 ops, 58.174MiB serialized bytes, 233.650MiB (12%) on heap, 53.406MiB
> (3%) off-heap
> {code}
> while actual heap usage:
> !image-2020-12-09-11-01-31-273.png!
> is about 442MB.
> Looking at the code we definitely are not counting the
> {{AtomicBTreePartition.Holder}}, {{EncodingStats}}, liveness and deletion
> info objects associated with each partition, and most probably others.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16318) Memtable heap size is severely underestimated
[
https://issues.apache.org/jira/browse/CASSANDRA-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17253188#comment-17253188
]
Ekaterina Dimitrova commented on CASSANDRA-16318:
-
Patch [here |https://github.com/ekaterinadimitrova2/cassandra/pull/80]
CI run: [Java 8
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/545/workflows/f797bf58-b572-4d8c-831e-a61936d23624]
and [Java 11
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/545/workflows/6dbe4bca-b51d-473e-bfc4-1bbf6c4ff26d]
>From the preliminary test I pushed looks like the issue was solved. I will
>check/publish full report of the microbench tests when I am back to the office
>on the 4th January.
> Memtable heap size is severely underestimated
> -
>
> Key: CASSANDRA-16318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16318
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Memtable
>Reporter: Branimir Lambov
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: image-2020-12-09-10-57-21-994.png,
> image-2020-12-09-11-01-31-273.png
>
>
> We seem to be estimating the size of the on-heap memtable metadata to be
> around half of what it actually is. For example, during a [read benchmark
> which writes 1 million single-long
> rows|https://github.com/blambov/cassandra/blob/memtable-heap/test/microbench/org/apache/cassandra/test/microbench/instance/ReadTestSmallPartitions.java]
> the memtable reports
> {code}
> 100 ops, 58.174MiB serialized bytes, 385.284MiB (19%) on heap, 0.000KiB
> (0%) off-heap
> {code}
> while a heap dump taken at this point:
> !image-2020-12-09-10-57-21-994.png!
> lists an usage of about 666MB altogether.
> Switching to {{offheap_objects}}, the reported numbers are
> {code}
> 100 ops, 58.174MiB serialized bytes, 233.650MiB (12%) on heap, 53.406MiB
> (3%) off-heap
> {code}
> while actual heap usage:
> !image-2020-12-09-11-01-31-273.png!
> is about 442MB.
> Looking at the code we definitely are not counting the
> {{AtomicBTreePartition.Holder}}, {{EncodingStats}}, liveness and deletion
> info objects associated with each partition, and most probably others.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16337) LCS steady state load vs. steady state load with garbagecollect running performance test
[ https://issues.apache.org/jira/browse/CASSANDRA-16337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17253063#comment-17253063 ] Yifan Cai edited comment on CASSANDRA-16337 at 12/21/20, 11:14 PM: --- The workload used in the test was generated from tlp-stress. Steady state load Read : Write : Delete == 5 : 4 : 1, and the QPS was kept at 3K/s. was (Author: yifanc): The workload used in the test was generated from tlp-stress. Steady state load Read : Write : Delete == 5 : 4 : 1, and the QPS was kept at 1.5K/s. > LCS steady state load vs. steady state load with garbagecollect running > performance test > > > Key: CASSANDRA-16337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16337 > Project: Cassandra > Issue Type: Sub-task > Components: Test/benchmark >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > > The testing cluster should be pre-populated with ~200GB data in each node. > Run the steady state workload to compare the read, write and compaction > performance between before and during garbagecollect is running. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16338) STCS steady state load vs. steady state load with garbagecollect running performance test
[
https://issues.apache.org/jira/browse/CASSANDRA-16338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifan Cai updated CASSANDRA-16338:
--
Test and Documentation Plan:
Perf. Test Plan
# Run data prepopulation
# Run steady state load for X seconds as Phase 1.
# Make one test case dependent change, e.g. triggering garbagecollect, altering
table, etc., and continue the steady state load for Y seconds as Phase 2.
# Compare the performance metrics beween Phase 1 and Phase 2.
The workload used in the test was generated from tlp-stress.
Steady state load Read : Write : Delete == 5 : 4 : 1, and the QPS was kept at
3K/s.
{{garbagecollect}} ran with 2 threads/jobs.
was:
Perf. Test Plan
# Run data prepopulation
# Run steady state load for X seconds as Phase 1.
# Make one test case dependent change, e.g. triggering garbagecollect, altering
table, etc., and continue the steady state load for Y seconds as Phase 2.
# Compare the performance metrics beween Phase 1 and Phase 2.
The workload used in the test was generated from tlp-stress.
Steady state load Read : Write : Delete == 5 : 4 : 1, and the QPS was kept at
1.5K/s.
{{garbagecollect}} ran with 2 threads/jobs.
> STCS steady state load vs. steady state load with garbagecollect running
> performance test
> -
>
> Key: CASSANDRA-16338
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16338
> Project: Cassandra
> Issue Type: Sub-task
> Components: Test/benchmark
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>
> The testing cluster should be pre-populated with ~200GB data in each node.
> Run the steady state workload to compare the read, write and compaction
> performance between before and during garbagecollect is running.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16340) STCS steady state load of table with vs. w/o GC performance test
[
https://issues.apache.org/jira/browse/CASSANDRA-16340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifan Cai updated CASSANDRA-16340:
--
Test and Documentation Plan:
Perf. Test Plan
# Run data prepopulation
# Run steady state load for X seconds as Phase 1.
# Make one test case dependent change, e.g. triggering garbagecollect, altering
table, etc., and continue the steady state load for Y seconds as Phase 2.
# Compare the performance metrics beween Phase 1 and Phase 2.
The workload used in the test was generated from tlp-stress.
Steady state load Read : Write : Delete == 5 : 4 : 1, and the QPS was kept at
3K/s.
Status: Patch Available (was: Open)
Result report link:
https://github.com/yifan-c/CASSANDRA-15581-COMPACTION-TEST/blob/main/CASSANDRA-16340/7019-Test:%20Perf%20Comparison%20%5BSTCS%20-%20provide_overlapping_tombstones%5D.pdf
In this test, with 'provided_overlapping_tombstones' == 'row', each background
compaction amortizes the garbage data collection. Each compaction task now
takes a longer time, but results into more compacted SSTables.
* The read latency rose up roughly 20% after altering the table with
'provided_overlapping_tombstones' to 'row'.
* Meanwhile, no noticeable change was observed from the write latency.
* No timeout was observed during the test in both phases.
* ParNew GC happened slightly more frequent, but the time taken was about the
same.
* The number of the pending compaction tasks rose slightly after the schema
change.
* The number of the SSTables was stable during the entire test.
> STCS steady state load of table with vs. w/o GC performance test
>
>
> Key: CASSANDRA-16340
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16340
> Project: Cassandra
> Issue Type: Sub-task
> Components: Test/benchmark
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>
> The testing cluster should be pre-populated with ~200GB data in each node.
> The baseline cluster has the table created with
> {{provide_overlapping_tombstones}} disabled. The other cluster has the table
> with {{provide_overlapping_tombstones == row}}. Compare the read, write and
> compaction performance between those 2 clusters.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16340) STCS steady state load of table with vs. w/o GC performance test
[
https://issues.apache.org/jira/browse/CASSANDRA-16340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifan Cai updated CASSANDRA-16340:
--
Change Category: Quality Assurance
Complexity: Normal
Status: Open (was: Triage Needed)
> STCS steady state load of table with vs. w/o GC performance test
>
>
> Key: CASSANDRA-16340
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16340
> Project: Cassandra
> Issue Type: Sub-task
> Components: Test/benchmark
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>
> The testing cluster should be pre-populated with ~200GB data in each node.
> The baseline cluster has the table created with
> {{provide_overlapping_tombstones}} disabled. The other cluster has the table
> with {{provide_overlapping_tombstones == row}}. Compare the read, write and
> compaction performance between those 2 clusters.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16338) STCS steady state load vs. steady state load with garbagecollect running performance test
[
https://issues.apache.org/jira/browse/CASSANDRA-16338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifan Cai updated CASSANDRA-16338:
--
Test and Documentation Plan:
Perf. Test Plan
# Run data prepopulation
# Run steady state load for X seconds as Phase 1.
# Make one test case dependent change, e.g. triggering garbagecollect, altering
table, etc., and continue the steady state load for Y seconds as Phase 2.
# Compare the performance metrics beween Phase 1 and Phase 2.
The workload used in the test was generated from tlp-stress.
Steady state load Read : Write : Delete == 5 : 4 : 1, and the QPS was kept at
1.5K/s.
{{garbagecollect}} ran with 2 threads/jobs.
Status: Patch Available (was: Open)
Result report link (saved as a pdf):
https://github.com/yifan-c/CASSANDRA-15581-COMPACTION-TEST/blob/main/CASSANDRA-16338/7019-Test:%20Perf%20Comparison%20%5BSTCS%20-%20garbagecollect%5D.pdf
Similar setup to CASSANDRA-16337, except the compaction strategy used was STCS.
Looking at the result, during {{garbagecollect}}
* Both read & write latencies (avg., p95, p99) increased significantly.
* The avg. read latency increased ~4.5X and p99 increased ~5X.
* The avg. write latency increased 11X and p99 increased ~36X.
* We observed several write timeouts.
* ParNew happened more frequent and took longer.
* The number of pending compaction tasks accumulated.
The conclusion is that for STCS, {{garbagecollect}} has a significant impact on
the system performance. We could observe latency increment and throughput
decrement when running the command in the production.
> STCS steady state load vs. steady state load with garbagecollect running
> performance test
> -
>
> Key: CASSANDRA-16338
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16338
> Project: Cassandra
> Issue Type: Sub-task
> Components: Test/benchmark
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>
> The testing cluster should be pre-populated with ~200GB data in each node.
> Run the steady state workload to compare the read, write and compaction
> performance between before and during garbagecollect is running.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16337) LCS steady state load vs. steady state load with garbagecollect running performance test
[ https://issues.apache.org/jira/browse/CASSANDRA-16337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17253063#comment-17253063 ] Yifan Cai commented on CASSANDRA-16337: --- The workload used in the test was generated from tlp-stress. Steady state load Read : Write : Delete == 5 : 4 : 1, and the QPS was kept at 1.5K/s. > LCS steady state load vs. steady state load with garbagecollect running > performance test > > > Key: CASSANDRA-16337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16337 > Project: Cassandra > Issue Type: Sub-task > Components: Test/benchmark >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > > The testing cluster should be pre-populated with ~200GB data in each node. > Run the steady state workload to compare the read, write and compaction > performance between before and during garbagecollect is running. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16338) STCS steady state load vs. steady state load with garbagecollect running performance test
[ https://issues.apache.org/jira/browse/CASSANDRA-16338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-16338: -- Change Category: Quality Assurance Complexity: Normal Status: Open (was: Triage Needed) > STCS steady state load vs. steady state load with garbagecollect running > performance test > - > > Key: CASSANDRA-16338 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16338 > Project: Cassandra > Issue Type: Sub-task > Components: Test/benchmark >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > > The testing cluster should be pre-populated with ~200GB data in each node. > Run the steady state workload to compare the read, write and compaction > performance between before and during garbagecollect is running. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16337) LCS steady state load vs. steady state load with garbagecollect running performance test
[ https://issues.apache.org/jira/browse/CASSANDRA-16337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-16337: -- Test and Documentation Plan: Perf. Test Plan # Run data prepopulation # Run steady state load for X seconds as Phase 1. # Make one test case dependent change, e.g. triggering garbagecollect, altering table, etc., and continue the steady state load for Y seconds as Phase 2. # Compare the performance metrics beween Phase 1 and Phase 2. Status: Patch Available (was: In Progress) > LCS steady state load vs. steady state load with garbagecollect running > performance test > > > Key: CASSANDRA-16337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16337 > Project: Cassandra > Issue Type: Sub-task > Components: Test/benchmark >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > > The testing cluster should be pre-populated with ~200GB data in each node. > Run the steady state workload to compare the read, write and compaction > performance between before and during garbagecollect is running. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16337) LCS steady state load vs. steady state load with garbagecollect running performance test
[ https://issues.apache.org/jira/browse/CASSANDRA-16337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-16337: -- Change Category: Quality Assurance Complexity: Normal Status: Open (was: Triage Needed) > LCS steady state load vs. steady state load with garbagecollect running > performance test > > > Key: CASSANDRA-16337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16337 > Project: Cassandra > Issue Type: Sub-task > Components: Test/benchmark >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > > The testing cluster should be pre-populated with ~200GB data in each node. > Run the steady state workload to compare the read, write and compaction > performance between before and during garbagecollect is running. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16337) LCS steady state load vs. steady state load with garbagecollect running performance test
[
https://issues.apache.org/jira/browse/CASSANDRA-16337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17253060#comment-17253060
]
Yifan Cai commented on CASSANDRA-16337:
---
I have run the test that compares the system performance with and w/o running
{{garbagecollect}} during the steady state load.
h3. Test Plan
# Run data prepopulation
# Run steady state load for X seconds as Phase 1.
# Make one test case dependent change, e.g. triggering garbagecollect, altering
table, etc., and continue the steady state load for Y seconds as Phase 2.
# Compare the performance metrics beween Phase 1 and Phase 2.
Result report link (saved as a pdf):
https://github.com/yifan-c/CASSANDRA-15581-COMPACTION-TEST/blob/main/CASSANDRA-16337/7019-Test:%20Perf%20Comparison%20%5BLCS%20-%20garbagecollect%5D.pdf
We can observe that both read & write latencies (avg., p95, p99) increased
significantly when {{garbagecollect}} is running. Read avg. was doubled, p99
was 3x. For the write latency, its avg. was ~6x and p99 was over 100x.
We can also observe several write timeouts during {{garbagecollect}}.
The number of L0 sstables increased very fast after triggering
{{garbagecollect}}. Meanwhile, the number was kept at a low level during just
the steady state load.
The conclusion is that for LCS, {{garbagecollect}} has a significant impact on
the system performance. We could observe latency increment and throughput
decrement when running the command in the production.
> LCS steady state load vs. steady state load with garbagecollect running
> performance test
>
>
> Key: CASSANDRA-16337
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16337
> Project: Cassandra
> Issue Type: Sub-task
> Components: Test/benchmark
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>
> The testing cluster should be pre-populated with ~200GB data in each node.
> Run the steady state workload to compare the read, write and compaction
> performance between before and during garbagecollect is running.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16279) Incorrect check for -Xlog in cassandra-env.sh
[
https://issues.apache.org/jira/browse/CASSANDRA-16279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-16279:
--
Test and Documentation Plan: The change is verified by uncommenting the gc
log option in the respective server.options file
([8|https://github.com/apache/cassandra/blob/ea52f1d2fa032dd28bbe69a1914497e48a6eb99a/conf/jvm8-server.options#L71],
[11|https://github.com/apache/cassandra/blob/ea52f1d2fa032dd28bbe69a1914497e48a6eb99a/conf/jvm11-server.options#L83])
and checking that the options from {{cassandra-ev.sh}} are not added if those
are already present.
Status: Patch Available (was: In Progress)
[patch|https://github.com/aholmberg/cassandra/pull/27]
[ci|https://app.circleci.com/pipelines/github/aholmberg/cassandra?branch=CASSANDRA-16279]
> Incorrect check for -Xlog in cassandra-env.sh
> --
>
> Key: CASSANDRA-16279
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16279
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Yakir Gibraltar
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
> [https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
> The following command incorrect:
> {code}
> echo "$JVM_OPTS" | grep -q "^-[X]log:gc"
> {code}
> Should be:
> {code}
> echo "$JVM_OPTS" | grep -qe "-[X]log:gc"
> {code}
> The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always
> return 1, remove {{^}} and add {{-qe}} will solve the problem.
> It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
> {{jvm11-server.options}} and {{jvm8-server.options}}
>
> Right now, jvm11-server.options with:
> {code:java}
> -Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M{code}
> Will generate process of:
> {code:java}
> -Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M
>
> -Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=10485760
> {code}
> With fix it will generate correct gc option of :
> {code}
> -Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16279) Incorrect check for -Xlog in cassandra-env.sh
[
https://issues.apache.org/jira/browse/CASSANDRA-16279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg reassigned CASSANDRA-16279:
-
Assignee: Adam Holmberg
> Incorrect check for -Xlog in cassandra-env.sh
> --
>
> Key: CASSANDRA-16279
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16279
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Yakir Gibraltar
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
> [https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
> The following command incorrect:
> {code}
> echo "$JVM_OPTS" | grep -q "^-[X]log:gc"
> {code}
> Should be:
> {code}
> echo "$JVM_OPTS" | grep -qe "-[X]log:gc"
> {code}
> The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always
> return 1, remove {{^}} and add {{-qe}} will solve the problem.
> It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
> {{jvm11-server.options}} and {{jvm8-server.options}}
>
> Right now, jvm11-server.options with:
> {code:java}
> -Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M{code}
> Will generate process of:
> {code:java}
> -Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M
>
> -Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=10485760
> {code}
> With fix it will generate correct gc option of :
> {code}
> -Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15386) Use multiple data directories in the in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15386: --- Source Control Link: https://github.com/apache/cassandra/commit/4d173e0a3f97b68b2ce0fb72befe2912efd31102 (was: https://github.com/apache/cassandra/commit/b3013a4ac5ee816cafe7492775126d1fa72ced75) > Use multiple data directories in the in-jvm dtests > -- > > Key: CASSANDRA-15386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15386 > Project: Cassandra > Issue Type: Improvement > Components: Local/Compaction, Test/dtest/java >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Labels: pull-request-available > Fix For: 4.0, 2.2.19, 3.0.23, 3.11.9, 4.0-beta3 > > > We should default to using 3 data directories when running the in-jvm dtests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15386) Use multiple data directories in the in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15386: --- Fix Version/s: (was: 4.0-beta) 4.0-beta3 4.0 > Use multiple data directories in the in-jvm dtests > -- > > Key: CASSANDRA-15386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15386 > Project: Cassandra > Issue Type: Improvement > Components: Local/Compaction, Test/dtest/java >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Labels: pull-request-available > Fix For: 4.0, 2.2.19, 3.0.23, 3.11.9, 4.0-beta3 > > > We should default to using 3 data directories when running the in-jvm dtests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16109) Don't adjust nodeCount when setting node id topology in in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-16109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16109: --- Fix Version/s: (was: NA) 2.2.19 3.0.23 3.11.9 4.0-beta3 4.0 > Don't adjust nodeCount when setting node id topology in in-jvm dtests > - > > Key: CASSANDRA-16109 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16109 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Low > Labels: pull-request-available > Fix For: 4.0, 2.2.19, 3.0.23, 3.11.9, 4.0-beta3 > > > We update the node count when setting the node id topology in in-jvm dtests, > this should only happen if node count is smaller than the node id topology, > otherwise bootstrap tests error out. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16109) Don't adjust nodeCount when setting node id topology in in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-16109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16109: --- Source Control Link: https://github.com/apache/cassandra/commit/4d173e0a3f97b68b2ce0fb72befe2912efd31102 (was: https://github.com/apache/cassandra/commit/b3013a4ac5ee816cafe7492775126d1fa72ced75) > Don't adjust nodeCount when setting node id topology in in-jvm dtests > - > > Key: CASSANDRA-16109 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16109 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Low > Labels: pull-request-available > Fix For: NA > > > We update the node count when setting the node id topology in in-jvm dtests, > this should only happen if node count is smaller than the node id topology, > otherwise bootstrap tests error out. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15879) Flaky unit test: CorruptedSSTablesCompactionsTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15879:
---
Fix Version/s: (was: 4.0-beta)
(was: 3.11.x)
(was: 3.0.x)
3.0.21
3.11.7
4.0-beta1
4.0
> Flaky unit test: CorruptedSSTablesCompactionsTest
> -
>
> Key: CASSANDRA-15879
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15879
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0, 4.0-beta1
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> CASSANDRA-14238 addressed the failure in
> {{BlacklistingCompactionsTest.testBlacklistingWithSizeTieredCompactionStrategy}},
> but only on 2.2. While working on CASSANDRA-14888, we’ve reproduced [the
> failure|https://app.circleci.com/pipelines/github/dineshjoshi/cassandra/47/workflows/de5f7cdb-06b6-4869-9d19-81a145e79f3f/jobs/2516/tests]
> on trunk.
> It looks like this should be a clean merge forward.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15816) Transports are stopped in the wrong order
[ https://issues.apache.org/jira/browse/CASSANDRA-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15816: --- Fix Version/s: (was: 4.0-beta) 4.0-beta2 4.0 > Transports are stopped in the wrong order > - > > Key: CASSANDRA-15816 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15816 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa >Priority: Normal > Fix For: 4.0, 3.0.22, 3.11.8, 4.0-beta2 > > Time Spent: 10m > Remaining Estimate: 0h > > Stopping gossip while native is running is almost always wrong, change the > order of shutdown and log a warning when done manually -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15853) @since missing/wrong for upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33
[ https://issues.apache.org/jira/browse/CASSANDRA-15853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15853: --- Fix Version/s: (was: 4.0-beta) 4.0-beta1 4.0 > @since missing/wrong for > upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33 > - > > Key: CASSANDRA-15853 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15853 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Normal > Fix For: 3.0.21, 3.11.7, 4.0, 4.0-beta1 > > > @since missing/wrong for > upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_22/33 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15858) TestRepairDataSystemTable test_repair_parent_table and test_repair_table fail
[ https://issues.apache.org/jira/browse/CASSANDRA-15858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15858: --- Fix Version/s: (was: 4.0-beta) 4.0-beta1 4.0 > TestRepairDataSystemTable test_repair_parent_table and test_repair_table fail > - > > Key: CASSANDRA-15858 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15858 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 3.0.21, 3.11.7, 4.0, 4.0-beta1 > > Time Spent: 20m > Remaining Estimate: 0h > > They fail > [consistently|https://ci-cassandra.apache.org/job/Cassandra-trunk/159/testReport/] > and locally as well -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15855) Mark test_populate_mv_after_insert_wide_rows as flaky
[ https://issues.apache.org/jira/browse/CASSANDRA-15855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15855: --- Fix Version/s: (was: 4.0-beta) 4.0-beta1 4.0 > Mark test_populate_mv_after_insert_wide_rows as flaky > - > > Key: CASSANDRA-15855 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15855 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 3.0.21, 3.11.7, 4.0, 4.0-beta1 > > Time Spent: 20m > Remaining Estimate: 0h > > See CASSANDRA-15845. This test can still fail in a flaky way so we better > mark it as such to avoid confusion and dup investigation efforts on failing > tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15938) Fix support for adding UDT fields to clustering keys
[ https://issues.apache.org/jira/browse/CASSANDRA-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15938: --- Fix Version/s: (was: 4.0-beta) 4.0-beta3 4.0 > Fix support for adding UDT fields to clustering keys > > > Key: CASSANDRA-15938 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15938 > Project: Cassandra > Issue Type: Bug > Components: Feature/UDT >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Fix For: 4.0, 2.2.19, 3.0.23, 3.11.9, 4.0-beta3 > > > Adding UDT fields to clustering keys is broken in all versions, however > slightly differently. > In 4.0, there will be a brief moment while schema changes are propagated > during which we won’t be able to decode and compare byte sequences. > Unfortunately, it is unclear what we should do in such cases, since we can’t > just come up with a comparator, and we can’t ignore non-null trailing values, > since this will lead to cases where compare for tuples `a;1` and `a;2` would > return 0, effectively making them equal, and we don’t know how to compare > unknown trailing values. Probably we should reject such query since we can’t > sort correctly, but we should make the error message more descriptive than > just "Index 1 out of bounds for length 1”. The only problem is that we get > this exception only on flush right now, so data already propagates to the > node by that time. > In 3.0, the problem is a bit worse than that, since in 3.0 we do not ignore > trailing nulls, so some of the values, written before `ALTER TYPE .. ADD` > become inaccessible. Both old values, and the new ones should always be > accessible. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16220) python dtest pending_range_test.py::TestPendingRangeMovements::test_pending_range (@pytest.mark.resource_intensive) fails on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-16220:
---
Fix Version/s: (was: 4.0-beta)
(was: 3.11.x)
(was: 3.0.x)
(was: 2.2.x)
2.2.19
3.0.23
3.11.9
4.0-beta3
4.0
> python dtest
> pending_range_test.py::TestPendingRangeMovements::test_pending_range
> (@pytest.mark.resource_intensive) fails on trunk
> --
>
> Key: CASSANDRA-16220
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16220
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: David Capwell
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 2.2.19, 3.0.23, 3.11.9, 4.0-beta3
>
>
> The test has the following section
> {code}
> if cluster.version() >= '2.2':
> node2.watch_log_for('127.0.0.1 state moving', timeout=10,
> filename='debug.log’)
> {code}
> The issue is that in trunk we have the port attached to the log, so the log
> is now
> {code}
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,104 StorageService.java:2452 - Node
> /127.0.0.1:7000 state MOVING, tokens [-9223372036854775808]
> DEBUG [GossipStage:1] 2020-10-21 00:48:20,105 StorageService.java:2670 - Node
> /127.0.0.1:7000 state moving, new token -634023222112864484
> {code}
> Since the log now contains the port, this test always times out on trunk when
> it hits this
> {code}
> self =
> @pytest.mark.resource_intensive
> def test_pending_range(self):
> """
> @jira_ticket CASSANDRA-10887
> """
> cluster = self.cluster
> # If we are on 2.1, we need to set the log level to debug or higher,
> as debug.log does not exist.
> if cluster.version() < '2.2':
> cluster.set_log_level('DEBUG')
>
> # Create 5 node cluster
> cluster.populate(5).start()
> node1, node2 = cluster.nodelist()[0:2]
>
> # Set up RF=3 keyspace
> session = self.patient_cql_connection(node1)
> create_ks(session, 'ks', 3)
>
> session.execute("CREATE TABLE users (login text PRIMARY KEY, email
> text, name text, login_count int)")
>
> # We use the partition key 'jdoe3' because it belongs to node1.
> # The key MUST belong to node1 to repro the bug.
> session.execute("INSERT INTO users (login, email, name, login_count)
> VALUES ('jdoe3', 'j...@abc.com', 'Jane Doe', 1) IF NOT EXISTS;")
>
> lwt_query = SimpleStatement("UPDATE users SET email =
> 'jane...@abc.com' WHERE login = 'jdoe3' IF email = 'j...@abc.com'")
>
> # Show we can execute LWT no problem
> for i in range(1000):
> session.execute(lwt_query)
>
> token = '-634023222112864484'
>
> mark = node1.mark_log()
>
> # Move a node
> node1.nodetool('move {}'.format(token))
>
> # Watch the log so we know when the node is moving
> node1.watch_log_for('Moving .* to {}'.format(token), timeout=10,
> from_mark=mark)
> node1.watch_log_for('Sleeping 3 ms before start
> streaming/fetching ranges', timeout=10, from_mark=mark)
>
> if cluster.version() >= '2.2':
> > node2.watch_log_for('127.0.0.1 state moving', timeout=10,
> > filename='debug.log')
> pending_range_test.py:57:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> self =
> exprs = '127.0.0.1 state moving', from_mark = None, timeout = 10, process =
> None
> verbose = False, filename = 'debug.log'
> def watch_log_for(self, exprs, from_mark=None, timeout=600,
> process=None, verbose=False, filename='system.log'):
> """
> Watch the log until one or more (regular) expressions are found
> or timeouts (a
> TimeoutError is then raised). On successful completion, a list of
> pair (line matched,
> match object) is returned.
> """
> start = time.time()
> tofind = [exprs] if isinstance(exprs, string_types) else exprs
> tofind = [re.compile(e) for e in tofind]
> matchings = []
> reads = ""
> if len(tofind) == 0:
> return None
>
> log_file = os.path.join(self.log_directory(), filename)
> output_read = False
> while not os.path.exists(log_file):
> time.sleep(.5)
> if start + timeout < time.time():
> raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S",
> time.gmtime()) + " [" + self.name
[jira] [Updated] (CASSANDRA-16114) Fix tests CQLTester.assertLastSchemaChange causes ClassCastException
[
https://issues.apache.org/jira/browse/CASSANDRA-16114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-16114:
---
Fix Version/s: (was: 4.0-beta)
4.0-beta4
4.0
> Fix tests CQLTester.assertLastSchemaChange causes ClassCastException
>
>
> Key: CASSANDRA-16114
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16114
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0, 2.2.19, 3.0.23, 3.11.9, 4.0-beta4
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> Build:
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/494/workflows/b3765545-7b09-48dd-85ff-830c4f348329/jobs/2681
> {code}
> java.lang.ClassCastException:
> org.apache.cassandra.transport.messages.ResultMessage$Void cannot be cast to
> org.apache.cassandra.transport.messages.ResultMessage$SchemaChange
> at
> org.apache.cassandra.cql3.CQLTester.assertLastSchemaChange(CQLTester.java:916)
> at
> org.apache.cassandra.cql3.validation.entities.UFTest.testSchemaChange(UFTest.java:94)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14925) DecimalSerializer.toString() can be used as OOM attack
[
https://issues.apache.org/jira/browse/CASSANDRA-14925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-14925:
---
Fix Version/s: (was: 4.0-beta)
4.0-beta4
4.0
> DecimalSerializer.toString() can be used as OOM attack
> ---
>
> Key: CASSANDRA-14925
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14925
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: Zhao Yang
>Assignee: Zhao Yang
>Priority: Urgent
> Fix For: 4.0, 4.0-beta4, 3.0.24, 3.11.10
>
>
> Currently, in {{DecimalSerializer.toString(value)}}, it uses
> {{BigDecimal.toPlainString()}} which generates huge string for large scale
> values.
>
> {code:java}
> BigDecimal d = new BigDecimal("1e-" + (Integer.MAX_VALUE - 6));
> d.toPlainString(); // oom{code}
>
> Propose to use {{BigDecimal.toString()}} when scale is larger than 100 which
> is configurable via {{-Dcassandra.decimal.maxscaleforstring}}
>
> | patch | circle-ci |
> |[trunk|https://github.com/jasonstack/cassandra/commits/decimal-tostring-trunk]|[unit|https://circleci.com/gh/jasonstack/cassandra/751?utm_campaign=vcs-integration-link_medium=referral_source=github-build-link]|
> The code should apply cleanly to 3.0+.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15158) Wait for schema agreement rather than in flight schema requests when bootstrapping
[ https://issues.apache.org/jira/browse/CASSANDRA-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15158: --- Fix Version/s: (was: 4.0-beta) (was: 3.11.x) (was: 3.0.x) 3.11.10 3.0.24 4.0-beta4 4.0 > Wait for schema agreement rather than in flight schema requests when > bootstrapping > -- > > Key: CASSANDRA-15158 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15158 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip, Cluster/Schema >Reporter: Vincent White >Assignee: Blake Eggleston >Priority: Normal > Fix For: 4.0, 4.0-beta4, 3.0.24, 3.11.10 > > Time Spent: 20m > Remaining Estimate: 0h > > Currently when a node is bootstrapping we use a set of latches > (org.apache.cassandra.service.MigrationTask#inflightTasks) to keep track of > in-flight schema pull requests, and we don't proceed with > bootstrapping/stream until all the latches are released (or we timeout > waiting for each one). One issue with this is that if we have a large schema, > or the retrieval of the schema from the other nodes was unexpectedly slow > then we have no explicit check in place to ensure we have actually received a > schema before we proceed. > While it's possible to increase "migration_task_wait_in_seconds" to force the > node to wait on each latche longer, there are cases where this doesn't help > because the callbacks for the schema pull requests have expired off the > messaging service's callback map > (org.apache.cassandra.net.MessagingService#callbacks) after > request_timeout_in_ms (default 10 seconds) before the other nodes were able > to respond to the new node. > This patch checks for schema agreement between the bootstrapping node and the > rest of the live nodes before proceeding with bootstrapping. It also adds a > check to prevent the new node from flooding existing nodes with simultaneous > schema pull requests as can happen in large clusters. > Removing the latch system should also prevent new nodes in large clusters > getting stuck for extended amounts of time as they wait > `migration_task_wait_in_seconds` on each of the latches left orphaned by the > timed out callbacks. > > ||3.11|| > |[PoC|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:check_for_schema]| > |[dtest|https://github.com/apache/cassandra-dtest/compare/master...vincewhite:wait_for_schema_agreement]| > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16247) Fix flaky test testTPStats - org.apache.cassandra.tools.NodeToolGossipInfoTest
[
https://issues.apache.org/jira/browse/CASSANDRA-16247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-16247:
---
Fix Version/s: (was: 4.0-beta)
4.0
> Fix flaky test testTPStats - org.apache.cassandra.tools.NodeToolGossipInfoTest
> --
>
> Key: CASSANDRA-16247
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16247
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0, 4.0-beta4
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/764/workflows/6d7a6adc-59d1-4f3c-baae-1f8329dca9b7/jobs/4363
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.tools.NodeToolGossipInfoTest.testTPStats(NodeToolGossipInfoTest.java:128)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16311) Extend the exclusion of replica filtering protection to other indices instead of just SASI
[ https://issues.apache.org/jira/browse/CASSANDRA-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16311: --- Fix Version/s: (was: 4.0-beta) (was: 3.11.x) 3.11.10 4.0-beta4 4.0 > Extend the exclusion of replica filtering protection to other indices instead > of just SASI > -- > > Key: CASSANDRA-16311 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16311 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index, Feature/SASI >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 4.0, 4.0-beta4, 3.0.24, 3.11.10 > > > There was a check introduced in CASSANDRA-8272 telling if an index is a SASI > index and if it is, replica filtering protection was not triggered. > There might be other custom index implementations which also do not support > filtering protection and they do not have to be SASI indices neccessarily, > however it is not possible to exclude them. > > PR trunk > [https://github.com/apache/cassandra/pull/844] > PR 3.11 > [https://github.com/apache/cassandra/pull/847] > PR 3.0 > https://github.com/apache/cassandra/pull/848 > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16311) Extend the exclusion of replica filtering protection to other indices instead of just SASI
[ https://issues.apache.org/jira/browse/CASSANDRA-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16311: --- Fix Version/s: (was: 3.0.x) 3.0.24 > Extend the exclusion of replica filtering protection to other indices instead > of just SASI > -- > > Key: CASSANDRA-16311 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16311 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index, Feature/SASI >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.11.x, 4.0-beta, 3.0.24 > > > There was a check introduced in CASSANDRA-8272 telling if an index is a SASI > index and if it is, replica filtering protection was not triggered. > There might be other custom index implementations which also do not support > filtering protection and they do not have to be SASI indices neccessarily, > however it is not possible to exclude them. > > PR trunk > [https://github.com/apache/cassandra/pull/844] > PR 3.11 > [https://github.com/apache/cassandra/pull/847] > PR 3.0 > https://github.com/apache/cassandra/pull/848 > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16319) Bump native protocol framing from v5 to v6
[ https://issues.apache.org/jira/browse/CASSANDRA-16319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16319: --- Fix Version/s: (was: 4.0-beta) 4.0-beta4 4.0 > Bump native protocol framing from v5 to v6 > -- > > Key: CASSANDRA-16319 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16319 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: 4.0, 4.0-beta4 > > > CASSANDRA-15299 revised the wire format of CQL native protocol to add a > framing layer around the existing CQL messages. This was originally targetted > at protocol v5, which is (still) currently in beta. There's a complication > with this though; while v5-beta is supported in Cassandra 3.11.x and 4.0.0, > the new wire format is only committed to trunk (4.0.0). This means that if > clients upgrade to a version of their driver which supports the new wire > format (3.10.0 for the java driver, python driver is not yet offically > released), connections to Cassandra 3.11.x nodes will break if the client > specifies v5-beta as the preferred protocol version. > Without the wire format changes the v5 changelog is pretty minimal, but > pulling v5 support from 3.11.x would make for a pretty bad user experience, > despite it being "only" a beta. > I propose finalizing CASSANDRA-14973 to promote v5 from beta to a final > version and bumping the new wire format changes to v6-beta, making both of > these changes in 4.0 and 3.11. Because v5 requires the beta flag to be set in > the frame header if using a beta version, this requires that users upgrade > clusters to a version with v5-final before upgrading to a driver with > v5-final. We should include this in NEWS.txt > The changes required to Cassandra itself are minor, as are the corresponding > patches to the java and python drivers. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16328) python upgrade tests include tests which are not impacted by the version under test
[ https://issues.apache.org/jira/browse/CASSANDRA-16328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16328: --- Fix Version/s: (was: NA) 4.0-beta4 4.0 > python upgrade tests include tests which are not impacted by the version > under test > --- > > Key: CASSANDRA-16328 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16328 > Project: Cassandra > Issue Type: Bug > Components: CI, Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0, 4.0-beta4 > > Time Spent: 40m > Remaining Estimate: 0h > > When we try to run the upgrade tests for trunk, we get tests such as > upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_indev_2_2_x_To_indev_3_0_x > to run, which do not include the trunk source. Since these tests are costly > and can be flaky, running the unneeded tests for a release increases the cost > and can make it harder to see if any new failures. > See > https://app.circleci.com/pipelines/github/dcapwell/cassandra/841/workflows/bcadf6e6-8d04-4010-8a47-99f7f9b5ac1d/jobs/4949/tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16329) python dtest upgrade tests do not generate 3.0 to trunk tests
[ https://issues.apache.org/jira/browse/CASSANDRA-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16329: --- Fix Version/s: (was: 4.0-beta) 4.0-beta4 4.0 > python dtest upgrade tests do not generate 3.0 to trunk tests > - > > Key: CASSANDRA-16329 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16329 > Project: Cassandra > Issue Type: Improvement > Components: CI, Test/dtest/python >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0, 4.0-beta4 > > > It looks like 3.11 is the only pair tested against trunk, we should update > this to include 3.0 -> trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13047) Point cqlsh help to the new doc
[ https://issues.apache.org/jira/browse/CASSANDRA-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-13047: --- Fix Version/s: (was: 4.0) 4.0.x > Point cqlsh help to the new doc > --- > > Key: CASSANDRA-13047 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13047 > Project: Cassandra > Issue Type: Bug > Components: Legacy/CQL >Reporter: Sylvain Lebresne >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 3.11.x, 4.0.x > > Time Spent: 0.5h > Remaining Estimate: 0h > > Cqlsh still points to the "old" textile CQL doc, but that's not really > maintain anymore now that we have the new doc (which include everything the > old doc had and more). We should update cqlsh to point to the new doc so we > can remove the old one completely. > Any takers? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14834) Avoid keeping StreamingTombstoneHistogramBuilder.Spool in memory during the whole compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-14834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-14834:
---
Fix Version/s: (was: 4.0)
> Avoid keeping StreamingTombstoneHistogramBuilder.Spool in memory during the
> whole compaction
>
>
> Key: CASSANDRA-14834
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14834
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Marcus Eriksson
>Assignee: Adam Holmberg
>Priority: Low
> Fix For: 4.0-beta
>
>
> Since CASSANDRA-13444 {{StreamingTombstoneHistogramBuilder.Spool}} is
> allocated to keep around an array with 131072 * 2 * 2 integers *per written
> sstable* during the whole compaction. With LCS at times creating 1000s of
> sstables during a compaction it kills the node.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14825) Expose table schema for drivers
[ https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-14825: --- Fix Version/s: (was: 4.0-alpha) 4.0-beta1 4.0 > Expose table schema for drivers > --- > > Key: CASSANDRA-14825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14825 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/CQL >Reporter: Chris Lohfink >Assignee: Robert Stupp >Priority: Normal > Labels: pull-request-available > Fix For: 4.0, 4.0-beta1 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently the drivers recreate the CQL for the tables by putting together the > system table values. This is very difficult to keep up to date and buggy > enough that its only even supported in Java and Python drivers. Cassandra > already has some limited output available for snapshots that we could provide > in a virtual table or new query that the drivers can fetch. This can greatly > reduce the complexity of drivers while also reducing bugs like > CASSANDRA-14822 as the underlying schema and properties change. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15146) Transitional TLS server configuration options are overly complex
[
https://issues.apache.org/jira/browse/CASSANDRA-15146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15146:
---
Fix Version/s: (was: 4.0-alpha)
> Transitional TLS server configuration options are overly complex
>
>
> Key: CASSANDRA-15146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15146
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Encryption, Local/Config
>Reporter: Joey Lynch
>Assignee: Ekaterina Dimitrova
>Priority: Normal
>
> It appears as part of the port from transitional client TLS to transitional
> server TLS in CASSANDRA-10404 (the ability to switch a cluster to using
> {{internode_encryption}} without listening on two ports and without downtime)
> we carried the {{enabled}} setting over from the client implementation. I
> believe that the {{enabled}} option is redundant to {{internode_encryption}}
> and {{optional}} and it should therefore be removed prior to the 4.0 release
> where we will have to start respecting that interface.
> Current trunk yaml:
> {noformat}
> server_encryption_options:
>
> # set to true for allowing secure incoming connections
>
> enabled: false
>
> # If enabled and optional are both set to true, encrypted and unencrypted
> connections are handled on the storage_port
> optional: false
>
>
>
>
> # if enabled, will open up an encrypted listening socket on
> ssl_storage_port. Should be used
> # during upgrade to 4.0; otherwise, set to false.
>
> enable_legacy_ssl_storage_port: false
>
> # on outbound connections, determine which type of peers to securely
> connect to. 'enabled' must be set to true.
> internode_encryption: none
>
> keystore: conf/.keystore
>
> keystore_password: cassandra
>
> truststore: conf/.truststore
>
> truststore_password: cassandra
> {noformat}
> I propose we eliminate {{enabled}} and just use {{optional}} and
> {{internode_encryption}} to determine the listener setup. I also propose we
> change the default of {{optional}} to true. We could also re-name
> {{optional}} since it's a new option but I think it's good to stay consistent
> with the client and use {{optional}}.
> ||optional||internode_encryption||description||
> |true|none|(default) No encryption is used but if a server reaches out with
> it we'll use it|
> |false|dc|Encryption is required for inter-dc communication, but not intra-dc|
> |false|all|Encryption is required for all communication|
> |false|none|We only listen for unencrypted connections|
> |true|dc|Encryption is used for inter-dc communication but is not required|
> |true|all|Encryption is used for all communication but is not required|
> From these states it is clear when we should be accepting TLS connections
> (all except for false and none) as well as when we must enforce it.
> To transition without downtime from an un-encrypted cluster to an encrypted
> cluster the user would do the following:
> 1. After adding valid truststores, change {{internode_encryption}} to the
> desired level of encryption (recommended {{all}}) and restart Cassandra
> 2. Change {{optional=false}} and restart Cassandra to enforce #1
> If {{optional}} defaulted to {{false}} as it does right now we'd need a third
> restart to first change {{optional}} to {{true}}, which given my
> understanding of the OptionalSslHandler isn't really relevant.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15667) StreamResultFuture check for completeness is inconsistent, leading to races
[
https://issues.apache.org/jira/browse/CASSANDRA-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15667:
---
Fix Version/s: (was: 4.0-alpha)
4.0-beta1
4.0
> StreamResultFuture check for completeness is inconsistent, leading to races
> ---
>
> Key: CASSANDRA-15667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15667
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: Massimiliano Tomassi
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0, 4.0-beta1
>
> Attachments: log_bootstrap_resumable
>
>
> {{StreamResultFuture#maybeComplete()}} uses
> {{StreamCoordinator#hasActiveSessions()}} to determine if all sessions are
> completed, but then accesses each session state via
> {{StreamCoordinator#getAllSessionInfo()}}: this is inconsistent, as the
> former relies on the actual {{StreamSession}} state, while the latter on the
> {{SessionInfo}} state, and the two are concurrently updated with no
> coordination whatsoever.
> This leads to races, i.e. apparent in some dtest spurious failures, such as
> {{TestBootstrap.resumable_bootstrap_test}} in CASSANDRA-15614 cc
> [~e.dimitrova].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15676) flaky test testWriteUnknownResult- org.apache.cassandra.distributed.test.CasWriteTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15676:
---
Fix Version/s: (was: 4.0-alpha)
4.0-beta1
4.0
> flaky test testWriteUnknownResult-
> org.apache.cassandra.distributed.test.CasWriteTest
> -
>
> Key: CASSANDRA-15676
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15676
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/java
>Reporter: Kevin Gallardo
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0, 4.0-beta1
>
> Attachments: Screen Shot 2020-05-07 at 7.25.19 PM.png
>
>
> Failure observed in:
> https://app.circleci.com/pipelines/github/newkek/cassandra/33/workflows/54007cf7-4424-4ec1-9655-665f6044e6d1/jobs/187/tests
> {noformat}
> testWriteUnknownResult - org.apache.cassandra.distributed.test.CasWriteTest
> junit.framework.AssertionFailedError: Expecting cause to be
> CasWriteUncertainException
> at
> org.apache.cassandra.distributed.test.CasWriteTest.testWriteUnknownResult(CasWriteTest.java:257)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15790) EmptyType doesn't override writeValue so could attempt to write bytes when expected not to
[
https://issues.apache.org/jira/browse/CASSANDRA-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15790:
---
Fix Version/s: (was: 4.0-alpha)
(was: 3.11.x)
3.11.7
4.0-beta1
4.0
> EmptyType doesn't override writeValue so could attempt to write bytes when
> expected not to
> --
>
> Key: CASSANDRA-15790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15790
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0, 4.0-beta1
>
>
> EmptyType.writeValues is defined here
> https://github.com/apache/cassandra/blob/e394dc0bb32f612a476269010930c617dd1ed3cb/src/java/org/apache/cassandra/db/marshal/AbstractType.java#L407-L414
> {code}
> public void writeValue(ByteBuffer value, DataOutputPlus out) throws
> IOException
> {
> assert value.hasRemaining();
> if (valueLengthIfFixed() >= 0)
> out.write(value);
> else
> ByteBufferUtil.writeWithVIntLength(value, out);
> }
> {code}
> This is fine when the value is empty as the write of empty no-ops (the
> readValue also noops since the length is 0), but if the value is not empty
> (possible during upgrades or random bugs) then this could silently cause
> corruption; ideally this should throw a exception if the ByteBuffer has data.
> This was called from
> org.apache.cassandra.db.rows.BufferCell.Serializer#serialize, here we check
> to see if data is present or not and update the flags. If data is present
> then and only then do we call type.writeValue (which requires bytes is not
> empty). The problem is that EmptyType never expects writes to happen, but it
> still writes them; and does not read them (since it says it is fixed length
> of 0, so does read(buffer, 0)).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15725) Add support for adding custom Verbs
[ https://issues.apache.org/jira/browse/CASSANDRA-15725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15725: --- Fix Version/s: (was: 4.0-alpha) 4.0-beta1 4.0 > Add support for adding custom Verbs > --- > > Key: CASSANDRA-15725 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15725 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Internode >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0, 4.0-beta1 > > Attachments: feedback_15725.patch > > > It should be possible to safely add custom/internal Verbs - without risking > conflicts when new ones are added. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15790) EmptyType doesn't override writeValue so could attempt to write bytes when expected not to
[
https://issues.apache.org/jira/browse/CASSANDRA-15790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15790:
---
Fix Version/s: (was: 3.0.x)
3.0.21
> EmptyType doesn't override writeValue so could attempt to write bytes when
> expected not to
> --
>
> Key: CASSANDRA-15790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15790
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.0.21, 3.11.x, 4.0-alpha
>
>
> EmptyType.writeValues is defined here
> https://github.com/apache/cassandra/blob/e394dc0bb32f612a476269010930c617dd1ed3cb/src/java/org/apache/cassandra/db/marshal/AbstractType.java#L407-L414
> {code}
> public void writeValue(ByteBuffer value, DataOutputPlus out) throws
> IOException
> {
> assert value.hasRemaining();
> if (valueLengthIfFixed() >= 0)
> out.write(value);
> else
> ByteBufferUtil.writeWithVIntLength(value, out);
> }
> {code}
> This is fine when the value is empty as the write of empty no-ops (the
> readValue also noops since the length is 0), but if the value is not empty
> (possible during upgrades or random bugs) then this could silently cause
> corruption; ideally this should throw a exception if the ByteBuffer has data.
> This was called from
> org.apache.cassandra.db.rows.BufferCell.Serializer#serialize, here we check
> to see if data is present or not and update the flags. If data is present
> then and only then do we call type.writeValue (which requires bytes is not
> empty). The problem is that EmptyType never expects writes to happen, but it
> still writes them; and does not read them (since it says it is fixed length
> of 0, so does read(buffer, 0)).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15806) C* 4.0 is missing a way to identify transient/non-transient SSTables on disk
[ https://issues.apache.org/jira/browse/CASSANDRA-15806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15806: --- Fix Version/s: (was: 4.0-alpha) 4.0-beta1 4.0 > C* 4.0 is missing a way to identify transient/non-transient SSTables on disk > > > Key: CASSANDRA-15806 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15806 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: sequoyha pelletier >Assignee: sequoyha pelletier >Priority: Normal > Fix For: 4.0, 4.0-beta1 > > Attachments: 15806-4.0.txt > > > Currently, there is no way to identify SSTables that were created as > transient replicated data. Even thought the feature is experimental we should > open that up for those that want to experiment. This seems to be an > oversight. I have added the missing line of code to the SStableMetadataViewer > and will attach a patch shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15807) Support multiple keyspaces in Cassandra-Diff
[ https://issues.apache.org/jira/browse/CASSANDRA-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15807: --- Fix Version/s: (was: 4.0-alpha) 4.0-beta1 4.0 > Support multiple keyspaces in Cassandra-Diff > > > Key: CASSANDRA-15807 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15807 > Project: Cassandra > Issue Type: Improvement > Components: Tool/diff >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Labels: pull-request-available > Fix For: 4.0, 4.0-beta1 > > Attachments: local_e2e_log.txt > > > Adding the support to run diff comparison of tables across multiple keyspaces > to avoid bringing up multiple spark jobs and give a better view of data diffs > in the whole cluster. > Cassandra-diff currently only support compare multiple tables under one > single keyspace. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
