[jira] [Commented] (CASSANDRA-15991) 15583 - Add UX tests to intree LHF tooling
[ https://issues.apache.org/jira/browse/CASSANDRA-15991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205275#comment-17205275 ] Berenguer Blasi commented on CASSANDRA-15991: - [~dcapwell] thanks thanks a million for reviewing this patch I really appreciate that :-) > 15583 - Add UX tests to intree LHF tooling > -- > > Key: CASSANDRA-15991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15991 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta3 > > > As per CASSANDRA-15583 many in tree tools lack proper UX tooling: mandatory > params are indeed mandatory, 'help' produces an actual help, return codes etc > This ticket is an attempt to add it to those tools that classify as LHF. > Other tools such as nodetool, with many sub-commands, deserve a separate > ticket of their own -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15993) Fix flaky python dtest test_view_metadata_cleanup - materialized_views_test.TestMaterializedViews
[
https://issues.apache.org/jira/browse/CASSANDRA-15993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205273#comment-17205273
]
Berenguer Blasi commented on CASSANDRA-15993:
-
I can't repro but the reasoning sounds solid to me. +1 to the current patch and
watch the job a few runs for that elusive 3.11 occasional timeout.
> Fix flaky python dtest test_view_metadata_cleanup -
> materialized_views_test.TestMaterializedViews
> -
>
> Key: CASSANDRA-15993
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15993
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: David Capwell
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/355/workflows/7b8df61d-706f-4094-a206-7cdc6b4e0451/jobs/1818
> {code}
> E cassandra.OperationTimedOut: errors={'127.0.0.2': 'Client request
> timeout. See Session.execute[_async](timeout)'}, last_host=127.0.0.2
> cassandra/cluster.py:4026: OperationTimedOut
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15993) Fix flaky python dtest test_view_metadata_cleanup - materialized_views_test.TestMaterializedViews
[
https://issues.apache.org/jira/browse/CASSANDRA-15993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-15993:
Reviewers: Berenguer Blasi, Berenguer Blasi (was: Berenguer Blasi)
Berenguer Blasi, Berenguer Blasi (was: Berenguer Blasi)
Status: Review In Progress (was: Patch Available)
> Fix flaky python dtest test_view_metadata_cleanup -
> materialized_views_test.TestMaterializedViews
> -
>
> Key: CASSANDRA-15993
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15993
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: David Capwell
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/355/workflows/7b8df61d-706f-4094-a206-7cdc6b4e0451/jobs/1818
> {code}
> E cassandra.OperationTimedOut: errors={'127.0.0.2': 'Client request
> timeout. See Session.execute[_async](timeout)'}, last_host=127.0.0.2
> cassandra/cluster.py:4026: OperationTimedOut
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15993) Fix flaky python dtest test_view_metadata_cleanup - materialized_views_test.TestMaterializedViews
[
https://issues.apache.org/jira/browse/CASSANDRA-15993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-15993:
Reviewers: Berenguer Blasi
> Fix flaky python dtest test_view_metadata_cleanup -
> materialized_views_test.TestMaterializedViews
> -
>
> Key: CASSANDRA-15993
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15993
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: David Capwell
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/355/workflows/7b8df61d-706f-4094-a206-7cdc6b4e0451/jobs/1818
> {code}
> E cassandra.OperationTimedOut: errors={'127.0.0.2': 'Client request
> timeout. See Session.execute[_async](timeout)'}, last_host=127.0.0.2
> cassandra/cluster.py:4026: OperationTimedOut
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16153) Cassandra 4b2 - JVM options from *.options not read/set
Thomas Steinmaurer created CASSANDRA-16153:
--
Summary: Cassandra 4b2 - JVM options from *.options not read/set
Key: CASSANDRA-16153
URL: https://issues.apache.org/jira/browse/CASSANDRA-16153
Project: Cassandra
Issue Type: Bug
Components: Local/Scripts
Reporter: Thomas Steinmaurer
Trying out Cassandra 4 beta 2 with Java 8 (AdoptOpenJDK) on Ubuntu 18.04 LTS.
It seems the Cassandra JVM results in using Parallel GC. In Cassandra 3,
default has been CMS.
Digging a bit further, it seems like the {{jvm8-server.options}} resp.
{{jvm11-server.options}} files aren't used/processed in e.g.
{{cassandra-env.sh}}.
E.g. in Cassandra 3.11, here we something like that in {{cassandra-env.sh}}.
{noformat}
# Read user-defined JVM options from jvm.options file
JVM_OPTS_FILE=$CASSANDRA_CONF/jvm.options
for opt in `grep "^-" $JVM_OPTS_FILE`
do
JVM_OPTS="$JVM_OPTS $opt"
done
{noformat}
Can't find something similar in {{cassandra-env.sh}} for Cassandra 4 beta2.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205246#comment-17205246 ] Rahul Nandi commented on CASSANDRA-16150: - Here is the pull request: https://github.com/apache/cassandra/pull/736#pullrequestreview-499670496 > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.x > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16115) New Cassandra website design, content and layout to work with Antora
[ https://issues.apache.org/jira/browse/CASSANDRA-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205147#comment-17205147 ] Melissa Logan commented on CASSANDRA-16115: --- As shared on the ML and dev Slack, homepage design concepts are now available here: [https://projects.invisionapp.com/freehand/document/7CLfsYoVb.|https://projects.invisionapp.com/freehand/document/7CLfsYoVb] We'd love community input in which comp they like best, as it will inform the rest of the pages. Elements from each design can be mixed-and-matched. Please share which one you like via this poll: [https://doodle.com/poll/4yc369baaug5dzw7.] > New Cassandra website design, content and layout to work with Antora > > > Key: CASSANDRA-16115 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16115 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Melissa Logan >Assignee: Melissa Logan >Priority: Normal > Fix For: 4.0-rc > > Attachments: Screen Shot 2020-09-03 at 09.48.53.png > > > This task is related to CASSANDRA-16066 (Update and rework the > cassandra-website material to work with Antora). The goal is to update the > front-end of the C* website (design, IA and content) to work with Antora to > help modernize the website as discussed on the [mailing > list|https://www.mail-archive.com/dev@cassandra.apache.org/msg15537.html]. > *Design Concepts:* A minimum of two homepage design concepts will be created > and shared for input, which will help standardize a brand palette for C* and > a design language for the site. This may include custom iconography and > graphics. The chosen design language will be used to develop the remaining > templates. > *Template Design*: It's estimated that 7 template designs will be needed > including the creation of several new pages: > * Homepage template > * Toplevel template - e.g. Community. > * General template - Mostly textual with some images, e.g. Intro, Quickstart > * “Library” template - A library of assets (links, downloads, logos etc) > that are sortable by metadata, e.g Resources, or Kafka's Powered By page). > * Blog landing template > * Blog single template > * Docs template > *Website Content:* Along with new design will be a need for new or updated > content to fit the new page layouts. The intention is to use as much as > possible from existing content, and augment with new content where needed. > *Template Development:* This includes the frontend development, such as any > HTML markup to achieve designs. HTML would be crafted so as to preserve any > backend/API calls, such that content is pulled in as designed. The majority > of the frontend work would come in the form of crafting CSS to bring the > designs to life, plus any minor Javascript to add subtle delights to key > pages. > *Style Guide*: Once all is complete, a Style Guide be added to GitHub for > contributors. > The [cassandra-website|https://github.com/apache/cassandra-website] > repository would need to be modified. Specific changes to be determined. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205131#comment-17205131 ] David Capwell commented on CASSANDRA-16150: --- your review is more than welcome =D > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.x > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16148) Test failures caused by merging CASSANDRA-15833
[
https://issues.apache.org/jira/browse/CASSANDRA-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205126#comment-17205126
]
David Capwell commented on CASSANDRA-16148:
---
bq. Can you say more about what you mean by "avoid caching when the version
isn't known".
Guava memoize doesn't support it so would require a custom implementation but
could indicate not to cache via a special exception that returns a value or
have the return type be a ADT that has Success, and NoCache.
bq. Otherwise, I think we need to opt for correctness over performance here.
I agree correctness is important here, but just concerned about mixed mode with
larger clusters. It would be great if we could get a baseline with and without
memoize with a 50-100 node cluster with at least 1 node in 3.x.
Also, my thinking was mostly that we could always fork the guava class, it is
15 lines of code (depending on implementation of NoCache, would be around 20~.
> Test failures caused by merging CASSANDRA-15833
> ---
>
> Key: CASSANDRA-16148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16148
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> Three issues were caused by merging CASSANDRA-15833:
> 1. `GossiperTest#testHaveAnyVersion3Nodes` was failing on trunk:
> https://app.circleci.com/pipelines/github/jrwest/cassandra/53/workflows/95f9f401-1ef8-4b8d-9c64-3703d9669d95/jobs/771
> 2. python dtest ReadRepairTest#test_atomic_writes[blocking] was failing
> 3. In-jvm dtests being worked on as part of CASSANDRA-15977 uncovered an
> issue with how CASSANDRA-15833 changes interacted with in-jvm dtests running
> without {{Feature.GOSSIP}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16152) Minor in-jvm dtest improvements
Jon Meredith created CASSANDRA-16152: Summary: Minor in-jvm dtest improvements Key: CASSANDRA-16152 URL: https://issues.apache.org/jira/browse/CASSANDRA-16152 Project: Cassandra Issue Type: Improvement Reporter: Jon Meredith Boring. Details to follow. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16148) Test failures caused by merging CASSANDRA-15833
[
https://issues.apache.org/jira/browse/CASSANDRA-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205090#comment-17205090
]
Jordan West commented on CASSANDRA-16148:
-
[~dcapwell] good catch re: volatile. I had that locally but forgot to commit.
Thank you. Regarding the hot path, it certainly adds more work during mixed
mode. Can you say more about what you mean by "avoid caching when the version
isn't known". If memoization is involved won't we be "caching" the value? Or is
there a short circuit I am not aware of? Otherwise, I think we need to opt for
correctness over performance here.
Will remove the extraneous comment
> Test failures caused by merging CASSANDRA-15833
> ---
>
> Key: CASSANDRA-16148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16148
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> Three issues were caused by merging CASSANDRA-15833:
> 1. `GossiperTest#testHaveAnyVersion3Nodes` was failing on trunk:
> https://app.circleci.com/pipelines/github/jrwest/cassandra/53/workflows/95f9f401-1ef8-4b8d-9c64-3703d9669d95/jobs/771
> 2. python dtest ReadRepairTest#test_atomic_writes[blocking] was failing
> 3. In-jvm dtests being worked on as part of CASSANDRA-15977 uncovered an
> issue with how CASSANDRA-15833 changes interacted with in-jvm dtests running
> without {{Feature.GOSSIP}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205080#comment-17205080 ] Ekaterina Dimitrova edited comment on CASSANDRA-16150 at 9/30/20, 10:38 PM: Ok, bad example. I meant to say that sometimes vendors (we do it in cassandra) make changes which will require some additional work during upgrade and if we read the notes in advance that my save some time and efforts, sometimes even headache. :) Tried to help not to point out someone for something... Hope I was understood correctly. Moving away, there is a committer already doing the review :) was (Author: e.dimitrova): Ok, bad example. I meant to say that sometimes vendors (we do it in cassandra) make changes which will require some additional work and if we read the notes in advance that my save some time and efforts, sometimes even headache. :) Tried to help not to point out someone for something... Hope I was understood correctly. Moving away, there is a committer already doing the review :) > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.x > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205080#comment-17205080 ] Ekaterina Dimitrova commented on CASSANDRA-16150: - Ok, bad example. I meant to say that sometimes vendors (we do it in cassandra) make changes which will require some additional work and if we read the notes in advance that my save some time and efforts, sometimes even headache. :) Tried to help not to point out someone for something... Hope I was understood correctly. Moving away, there is a committer already doing the review :) > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.x > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15229) BufferPool Regression
[
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205072#comment-17205072
]
Caleb Rackliffe commented on CASSANDRA-15229:
-
I'm okay with another issue as well, but the issue title here is pretty broad ;)
(Also looking for feedback on the last paragraph of my last comment...)
> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Assignee: Zhao Yang
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: 15229-count.png, 15229-direct.png, 15229-hit-rate.png,
> 15229-recirculate-count.png, 15229-recirculate-hit-rate.png,
> 15229-recirculate-size.png, 15229-recirculate.png, 15229-size.png,
> 15229-unsafe.png
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we
> need to either change our behaviour to handle uncorrelated lifetimes or use
> something else. This is particularly important with the default chunk size
> for compressed sstables being reduced. If we address the problem, we should
> also utilise the BufferPool for native transport connections like we do for
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used
> for things with uncorrelated lifetimes, which essentially boils down to
> tracking those chunks that have not been freed and re-circulating them when
> we run out of completely free blocks. We should probably also permit
> instantiating separate {{BufferPool}}, so that we can insulate internode
> messaging from the {{ChunkCache}}, or at least have separate memory bounds
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce
> the amount of global coordination and per-allocation overhead. We don’t need
> 1KiB granularity for allocations, nor 16 byte granularity for tiny
> allocations.
> -
> Since CASSANDRA-5863, chunk cache is implemented to use buffer pool. When
> local pool is full, one of its chunks will be evicted and only put back to
> global pool when all buffers in the evicted chunk are released. But due to
> chunk cache, buffers can be held for long period of time, preventing evicted
> chunk to be recycled even though most of space in the evicted chunk are free.
> There two things need to be improved:
> 1. Evicted chunk with free space should be recycled to global pool, even if
> it's not fully free. It's doable in 4.0.
> 2. Reduce fragmentation caused by different buffer size. With #1, partially
> freed chunk will be available for allocation, but "holes" in the partially
> freed chunk are with different sizes. We should consider allocating fixed
> buffer size which is unlikely to fit in 4.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16148) Test failures caused by merging CASSANDRA-15833
[
https://issues.apache.org/jira/browse/CASSANDRA-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-16148:
--
Status: Changes Suggested (was: Review In Progress)
> Test failures caused by merging CASSANDRA-15833
> ---
>
> Key: CASSANDRA-16148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16148
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> Three issues were caused by merging CASSANDRA-15833:
> 1. `GossiperTest#testHaveAnyVersion3Nodes` was failing on trunk:
> https://app.circleci.com/pipelines/github/jrwest/cassandra/53/workflows/95f9f401-1ef8-4b8d-9c64-3703d9669d95/jobs/771
> 2. python dtest ReadRepairTest#test_atomic_writes[blocking] was failing
> 3. In-jvm dtests being worked on as part of CASSANDRA-15977 uncovered an
> issue with how CASSANDRA-15833 changes interacted with in-jvm dtests running
> without {{Feature.GOSSIP}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16148) Test failures caused by merging CASSANDRA-15833
[
https://issues.apache.org/jira/browse/CASSANDRA-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205069#comment-17205069
]
David Capwell commented on CASSANDRA-16148:
---
The change to Gossiper isn't thread safe. Previous we used memoize which
guarded the state with a sync on the memoizer, now that we have removed this
there are no locks/fences in place to make sure the field access is safe (tests
would directly access, but read/write was the same thread so not a issue
there); it should be enough to add volatile to
org.apache.cassandra.gms.Gossiper#haveMajorVersion3Nodes.
Also, I am concerned that there may be perf implications with larger clusters
during mixed-mode case, specifically this call site
org.apache.cassandra.db.filter.ColumnFilter.Builder#build; would need to look
closer but this looks like the query hot path so would want to confirm. A
simple way to avoid this would be to still memoize with the 1m expires, but to
avoid caching when version isn't known (the assumption the race window is
"small" so performance hit during this window).
{code}
Gossiper.instance.injectApplicationState(addressAndPort,
ApplicationState.RELEASE_VERSION,
new
VersionedValue.VersionedValueFactory(partitioner).releaseVersion());
{code}
This logic isn't correct as you need the release version of the node you are
adding, this method takes the release version of the current node (example:
node1 is 3.0, node2 is 4.0. node2 would update gossip to say node1 is 4.0).
https://github.com/apache/cassandra/compare/trunk...jrwest:jwest/16148#diff-9d7e7c7ee33324ba61d6240be9e88f7eR51
Can you remove the comment which switches to Feature.GOSSIP?
> Test failures caused by merging CASSANDRA-15833
> ---
>
> Key: CASSANDRA-16148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16148
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> Three issues were caused by merging CASSANDRA-15833:
> 1. `GossiperTest#testHaveAnyVersion3Nodes` was failing on trunk:
> https://app.circleci.com/pipelines/github/jrwest/cassandra/53/workflows/95f9f401-1ef8-4b8d-9c64-3703d9669d95/jobs/771
> 2. python dtest ReadRepairTest#test_atomic_writes[blocking] was failing
> 3. In-jvm dtests being worked on as part of CASSANDRA-15977 uncovered an
> issue with how CASSANDRA-15833 changes interacted with in-jvm dtests running
> without {{Feature.GOSSIP}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16148) Test failures caused by merging CASSANDRA-15833
[
https://issues.apache.org/jira/browse/CASSANDRA-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-16148:
Reviewers: Caleb Rackliffe, David Capwell (was: David Capwell)
> Test failures caused by merging CASSANDRA-15833
> ---
>
> Key: CASSANDRA-16148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16148
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> Three issues were caused by merging CASSANDRA-15833:
> 1. `GossiperTest#testHaveAnyVersion3Nodes` was failing on trunk:
> https://app.circleci.com/pipelines/github/jrwest/cassandra/53/workflows/95f9f401-1ef8-4b8d-9c64-3703d9669d95/jobs/771
> 2. python dtest ReadRepairTest#test_atomic_writes[blocking] was failing
> 3. In-jvm dtests being worked on as part of CASSANDRA-15977 uncovered an
> issue with how CASSANDRA-15833 changes interacted with in-jvm dtests running
> without {{Feature.GOSSIP}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15993) Fix flaky python dtest test_view_metadata_cleanup - materialized_views_test.TestMaterializedViews
[
https://issues.apache.org/jira/browse/CASSANDRA-15993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205065#comment-17205065
]
Adam Holmberg commented on CASSANDRA-15993:
---
There are a couple things going on here.
The main source of timeouts was running all the create views at once:
The view builders run asynchronously in the background. As each new CREATE VIEW
is issued, the schema migration causes any in-progress build to stop and
restart anew. This causes a snowball of stopping and restarting builds that
then makes one of the later DDL statements timeout (they would complete with
longer timeouts).
Changing the dtest to simply wait for each build synchronously removes that
contention. Although they are built serially, due to avoiding contention there
is also no increase in runtime. My test setup that previously failed 1/12 times
runs hundreds of times with just this change, and no increased timeout.
However, there's another mechanism here that makes me think we should raise the
timeouts as well:
It just so happens that [this
task|https://github.com/aholmberg/cassandra/blob/c6ef476278ec783b77faa367e82d9b1ffabc/src/java/org/apache/cassandra/service/CassandraDaemon.java#L406-L414]
is scheduled to run at roughly the same as the
[drop_keyspace|https://github.com/aholmberg/cassandra-dtest/blob/efc64a670955eaf91533911c7cbbb792fd5add19/materialized_views_test.py#L236]
DML is usually running, making another chance for contention as the build
tasks are stopped for schema migration. I haven't been able to reproduce, but
my theory is that's what's causing the [occasional
timeout|https://ci-cassandra.apache.org/job/Cassandra-3.11/lastCompletedBuild/testReport/dtest-novnode.materialized_views_test/TestMaterializedViews/test_view_metadata_cleanup/]
observed in 3.11. Rather than try to find timing around that, my suggestion is
simply to raise the request timeout for any DDL that happens while views are
present in the keyspace.
[patch|https://github.com/aholmberg/cassandra-dtest/commit/efc64a670955eaf91533911c7cbbb792fd5add19]
[ci|https://app.circleci.com/pipelines/github/aholmberg/cassandra?branch=CASSANDRA-15993]
> Fix flaky python dtest test_view_metadata_cleanup -
> materialized_views_test.TestMaterializedViews
> -
>
> Key: CASSANDRA-15993
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15993
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: David Capwell
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/355/workflows/7b8df61d-706f-4094-a206-7cdc6b4e0451/jobs/1818
> {code}
> E cassandra.OperationTimedOut: errors={'127.0.0.2': 'Client request
> timeout. See Session.execute[_async](timeout)'}, last_host=127.0.0.2
> cassandra/cluster.py:4026: OperationTimedOut
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15993) Fix flaky python dtest test_view_metadata_cleanup - materialized_views_test.TestMaterializedViews
[
https://issues.apache.org/jira/browse/CASSANDRA-15993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg updated CASSANDRA-15993:
--
Test and Documentation Plan:
looped a ton in my test setup
Look for stability on ci-cassandra
Status: Patch Available (was: In Progress)
> Fix flaky python dtest test_view_metadata_cleanup -
> materialized_views_test.TestMaterializedViews
> -
>
> Key: CASSANDRA-15993
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15993
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: David Capwell
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/355/workflows/7b8df61d-706f-4094-a206-7cdc6b4e0451/jobs/1818
> {code}
> E cassandra.OperationTimedOut: errors={'127.0.0.2': 'Client request
> timeout. See Session.execute[_async](timeout)'}, last_host=127.0.0.2
> cassandra/cluster.py:4026: OperationTimedOut
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16148) Test failures caused by merging CASSANDRA-15833
[
https://issues.apache.org/jira/browse/CASSANDRA-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-16148:
Description:
Three issues were caused by merging CASSANDRA-15833:
1. `GossiperTest#testHaveAnyVersion3Nodes` was failing on trunk:
https://app.circleci.com/pipelines/github/jrwest/cassandra/53/workflows/95f9f401-1ef8-4b8d-9c64-3703d9669d95/jobs/771
2. python dtest ReadRepairTest#test_atomic_writes[blocking] was failing
3. In-jvm dtests being worked on as part of CASSANDRA-15977 uncovered an issue
with how CASSANDRA-15833 changes interacted with in-jvm dtests running without
{{Feature.GOSSIP}}
was:https://app.circleci.com/pipelines/github/jrwest/cassandra/53/workflows/95f9f401-1ef8-4b8d-9c64-3703d9669d95/jobs/771
> Test failures caused by merging CASSANDRA-15833
> ---
>
> Key: CASSANDRA-16148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16148
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> Three issues were caused by merging CASSANDRA-15833:
> 1. `GossiperTest#testHaveAnyVersion3Nodes` was failing on trunk:
> https://app.circleci.com/pipelines/github/jrwest/cassandra/53/workflows/95f9f401-1ef8-4b8d-9c64-3703d9669d95/jobs/771
> 2. python dtest ReadRepairTest#test_atomic_writes[blocking] was failing
> 3. In-jvm dtests being worked on as part of CASSANDRA-15977 uncovered an
> issue with how CASSANDRA-15833 changes interacted with in-jvm dtests running
> without {{Feature.GOSSIP}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16148) Test failures caused by merging CASSANDRA-15833
[
https://issues.apache.org/jira/browse/CASSANDRA-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204356#comment-17204356
]
Jordan West edited comment on CASSANDRA-16148 at 9/30/20, 10:10 PM:
Branch updated to address all 3 issues:
[branch | https://github.com/jrwest/cassandra/tree/jwest/16148] [tests |
https://app.circleci.com/pipelines/github/jrwest/cassandra?branch=jwest%2F16148]
was (Author: jrwest):
[branch | https://github.com/jrwest/cassandra/tree/jwest/16148] [tests |
https://app.circleci.com/pipelines/github/jrwest/cassandra?branch=jwest%2F16148]
> Test failures caused by merging CASSANDRA-15833
> ---
>
> Key: CASSANDRA-16148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16148
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> Three issues were caused by merging CASSANDRA-15833:
> 1. `GossiperTest#testHaveAnyVersion3Nodes` was failing on trunk:
> https://app.circleci.com/pipelines/github/jrwest/cassandra/53/workflows/95f9f401-1ef8-4b8d-9c64-3703d9669d95/jobs/771
> 2. python dtest ReadRepairTest#test_atomic_writes[blocking] was failing
> 3. In-jvm dtests being worked on as part of CASSANDRA-15977 uncovered an
> issue with how CASSANDRA-15833 changes interacted with in-jvm dtests running
> without {{Feature.GOSSIP}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16148) Test failures caused by merging CASSANDRA-15833
[ https://issues.apache.org/jira/browse/CASSANDRA-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-16148: Summary: Test failures caused by merging CASSANDRA-15833 (was: GossiperTest#testHaveVersion3Nodes is failing on trunk) > Test failures caused by merging CASSANDRA-15833 > --- > > Key: CASSANDRA-16148 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16148 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > https://app.circleci.com/pipelines/github/jrwest/cassandra/53/workflows/95f9f401-1ef8-4b8d-9c64-3703d9669d95/jobs/771 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15991) 15583 - Add UX tests to intree LHF tooling
[ https://issues.apache.org/jira/browse/CASSANDRA-15991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204969#comment-17204969 ] David Capwell edited comment on CASSANDRA-15991 at 9/30/20, 10:05 PM: -- CI results Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-15991-trunk-BE343BE9-1595-4210-867C-65946BBC011A Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/53/ Results only failed where expected (other issues being fixed atm). There was a conflict commit between merge and testing; I choose not to rerun the tests as the changes looked localized to Jenkins and shouldn't impact the build results of this patch. was (Author: dcapwell): CI results (pending) Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-15991-trunk-BE343BE9-1595-4210-867C-65946BBC011A Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/53/ > 15583 - Add UX tests to intree LHF tooling > -- > > Key: CASSANDRA-15991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15991 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta3 > > > As per CASSANDRA-15583 many in tree tools lack proper UX tooling: mandatory > params are indeed mandatory, 'help' produces an actual help, return codes etc > This ticket is an attempt to add it to those tools that classify as LHF. > Other tools such as nodetool, with many sub-commands, deserve a separate > ticket of their own -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15991) 15583 - Add UX tests to intree LHF tooling
[ https://issues.apache.org/jira/browse/CASSANDRA-15991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205060#comment-17205060 ] David Capwell commented on CASSANDRA-15991: --- Thanks [~Bereng] for all the hard work! > 15583 - Add UX tests to intree LHF tooling > -- > > Key: CASSANDRA-15991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15991 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta3 > > > As per CASSANDRA-15583 many in tree tools lack proper UX tooling: mandatory > params are indeed mandatory, 'help' produces an actual help, return codes etc > This ticket is an attempt to add it to those tools that classify as LHF. > Other tools such as nodetool, with many sub-commands, deserve a separate > ticket of their own -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15991) 15583 - Add UX tests to intree LHF tooling
[ https://issues.apache.org/jira/browse/CASSANDRA-15991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15991: -- Fix Version/s: (was: 4.0-beta) 4.0-beta3 Source Control Link: https://github.com/apache/cassandra/commit/9802a70f68cdcd239a9128f90f5d8f9a941168de Resolution: Fixed Status: Resolved (was: Ready to Commit) > 15583 - Add UX tests to intree LHF tooling > -- > > Key: CASSANDRA-15991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15991 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta3 > > > As per CASSANDRA-15583 many in tree tools lack proper UX tooling: mandatory > params are indeed mandatory, 'help' produces an actual help, return codes etc > This ticket is an attempt to add it to those tools that classify as LHF. > Other tools such as nodetool, with many sub-commands, deserve a separate > ticket of their own -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Add UX tests to intree LHF tooling
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 9802a70 Add UX tests to intree LHF tooling
9802a70 is described below
commit 9802a70f68cdcd239a9128f90f5d8f9a941168de
Author: Berenguer Blasi
AuthorDate: Wed Sep 30 12:13:15 2020 -0700
Add UX tests to intree LHF tooling
patch by Berenguer Blasi; reviewed by Brandon Williams, David Capwell for
CASSANDRA-15991
---
.../org/apache/cassandra/tools/SSTableExport.java | 2 -
.../cassandra/tools/SSTableMetadataViewer.java | 1 -
.../cassandra/tools/StandaloneSSTableUtil.java | 4 +-
.../test/FqlReplayDDLExclusionTest.java| 27 +-
test/unit/org/apache/cassandra/cql3/CQLTester.java | 30 +-
.../apache/cassandra/tools/AuditLogViewerTest.java | 93 +++
.../org/apache/cassandra/tools/BulkLoaderTest.java | 120 ++--
.../apache/cassandra/tools/ClearSnapshotTest.java | 41 +-
.../cassandra/tools/CompactionStressTest.java | 47 +-
.../cassandra/tools/GetFullQueryLogTest.java | 36 +-
.../org/apache/cassandra/tools/GetVersionTest.java | 6 +-
.../cassandra/tools/JMXCompatabilityTest.java | 9 +-
.../org/apache/cassandra/tools/JMXToolTest.java| 17 +-
.../apache/cassandra/tools/OfflineToolUtils.java | 9 +-
.../tools/SSTableExpiredBlockersTest.java | 45 +-
.../apache/cassandra/tools/SSTableExportTest.java | 151 -
.../cassandra/tools/SSTableLevelResetterTest.java | 55 +-
.../cassandra/tools/SSTableMetadataViewerTest.java | 139 -
.../cassandra/tools/SSTableOfflineRelevelTest.java | 45 +-
.../tools/SSTableRepairedAtSetterTest.java | 84 ++-
.../cassandra/tools/StandaloneSSTableUtilTest.java | 136 +++-
.../cassandra/tools/StandaloneScrubberTest.java| 152 -
.../cassandra/tools/StandaloneSplitterTest.java| 100 ++-
.../cassandra/tools/StandaloneUpgraderTest.java| 86 ++-
.../cassandra/tools/StandaloneVerifierTest.java| 141 -
.../org/apache/cassandra/tools/ToolRunner.java | 693 -
...tVersionTest.java => ToolsEnvsConfigsTest.java} | 30 +-
.../tools/cassandrastress/CassandrastressTest.java | 50 ++
.../{GetVersionTest.java => cqlsh/CqlshTest.java} | 35 +-
29 files changed, 1801 insertions(+), 583 deletions(-)
diff --git a/src/java/org/apache/cassandra/tools/SSTableExport.java
b/src/java/org/apache/cassandra/tools/SSTableExport.java
index 394f4b6..ca01cc3 100644
--- a/src/java/org/apache/cassandra/tools/SSTableExport.java
+++ b/src/java/org/apache/cassandra/tools/SSTableExport.java
@@ -46,8 +46,6 @@ import org.apache.commons.cli.Option;
import org.apache.commons.cli.Options;
import org.apache.commons.cli.ParseException;
import org.apache.commons.cli.PosixParser;
-import org.apache.cassandra.io.sstable.metadata.MetadataComponent;
-import org.apache.cassandra.io.sstable.metadata.MetadataType;
import org.apache.cassandra.schema.TableMetadataRef;
import org.apache.cassandra.utils.FBUtilities;
diff --git a/src/java/org/apache/cassandra/tools/SSTableMetadataViewer.java
b/src/java/org/apache/cassandra/tools/SSTableMetadataViewer.java
index e99f454..a4da97c 100755
--- a/src/java/org/apache/cassandra/tools/SSTableMetadataViewer.java
+++ b/src/java/org/apache/cassandra/tools/SSTableMetadataViewer.java
@@ -73,7 +73,6 @@ import org.apache.commons.cli.ParseException;
import org.apache.commons.cli.PosixParser;
import com.google.common.collect.MinMaxPriorityQueue;
-import org.apache.commons.lang3.time.DurationFormatUtils;
/**
* Shows the contents of sstable metadata
diff --git a/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java
b/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java
index 9a7847a..cca48fc 100644
--- a/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java
+++ b/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java
@@ -20,7 +20,6 @@ package org.apache.cassandra.tools;
import org.apache.cassandra.schema.TableMetadata;
import org.apache.cassandra.schema.Schema;
-import org.apache.cassandra.db.ColumnFamilyStore;
import org.apache.cassandra.db.Directories;
import org.apache.cassandra.db.lifecycle.LifecycleTransaction;
import org.apache.cassandra.utils.OutputHandler;
@@ -28,7 +27,6 @@ import org.apache.commons.cli.*;
import java.io.File;
import java.io.IOException;
-import java.util.function.BiFunction;
import java.util.function.BiPredicate;
import static org.apache.cassandra.tools.BulkLoader.CmdLineOptions;
@@ -62,7 +60,7 @@ public class StandaloneSSTableUtil
if (options.cleanup)
{
-handler.output("Cleanuping up...");
+handler.output("Cleaning up...");
LifecycleTransaction.removeUnfinishedLeftovers(metadata);
}
else
diff --git
[jira] [Commented] (CASSANDRA-15229) BufferPool Regression
[
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205054#comment-17205054
]
David Capwell commented on CASSANDRA-15229:
---
bq. The question is whether we should work through that here or as part of
another issue.
I am 100% fine tackling that in a different issue. Looking at the first line
in the description "The BufferPool was never intended to be used for a
ChunkCache, and we need to either change our behaviour to handle uncorrelated
lifetimes or use something else", the separate pools does isolate the issue so
networking isn't impacted by the chunk cache; this gives breathing room to
figure scope of chunk cache work.
> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Assignee: Zhao Yang
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: 15229-count.png, 15229-direct.png, 15229-hit-rate.png,
> 15229-recirculate-count.png, 15229-recirculate-hit-rate.png,
> 15229-recirculate-size.png, 15229-recirculate.png, 15229-size.png,
> 15229-unsafe.png
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we
> need to either change our behaviour to handle uncorrelated lifetimes or use
> something else. This is particularly important with the default chunk size
> for compressed sstables being reduced. If we address the problem, we should
> also utilise the BufferPool for native transport connections like we do for
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used
> for things with uncorrelated lifetimes, which essentially boils down to
> tracking those chunks that have not been freed and re-circulating them when
> we run out of completely free blocks. We should probably also permit
> instantiating separate {{BufferPool}}, so that we can insulate internode
> messaging from the {{ChunkCache}}, or at least have separate memory bounds
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce
> the amount of global coordination and per-allocation overhead. We don’t need
> 1KiB granularity for allocations, nor 16 byte granularity for tiny
> allocations.
> -
> Since CASSANDRA-5863, chunk cache is implemented to use buffer pool. When
> local pool is full, one of its chunks will be evicted and only put back to
> global pool when all buffers in the evicted chunk are released. But due to
> chunk cache, buffers can be held for long period of time, preventing evicted
> chunk to be recycled even though most of space in the evicted chunk are free.
> There two things need to be improved:
> 1. Evicted chunk with free space should be recycled to global pool, even if
> it's not fully free. It's doable in 4.0.
> 2. Reduce fragmentation caused by different buffer size. With #1, partially
> freed chunk will be available for allocation, but "holes" in the partially
> freed chunk are with different sizes. We should consider allocating fixed
> buffer size which is unlikely to fit in 4.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15229) BufferPool Regression
[
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205042#comment-17205042
]
Caleb Rackliffe commented on CASSANDRA-15229:
-
Finished my review, and dropped my comment inline in the PR.
Looking at the larger picture of this issue, CASSANDRA-16036, and our general
goal of making sure 4.0 does not have any egregious performance issues around
the chunk cache, it seems like the most pressing thing in front of us is making
sure compaction doesn't absolutely trash the chunk cache. [~dcapwell] seems to
have [documented
this|https://issues.apache.org/jira/browse/CASSANDRA-16036?focusedCommentId=17173291=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17173291]
pretty clearly, and both tests done in that issue and Fallout tests done by
[~jasonstack] would be capable of verifying the effectiveness of any changes
around this. The question is whether we should work through that here or as
part of another issue.
I don't know the compaction code super well, but it seems like we could avoid
most of the cache churn mess by having the {{ISSTableScanner}} implementations
returned by {{SSTableReader#getScanner()}} use file handles that don't use
{{CachingRebufferer}}. {{FileHandle.Builder#complete()}} already seems to
roughly have the logic we would need to produce the correct (uncached)
{{RebuffererFactory}}. If it's that simple, and we've already got a performance
testing scaffolding set up here, perhaps it would make sense to roll into this
Jira...
> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Assignee: Zhao Yang
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: 15229-count.png, 15229-direct.png, 15229-hit-rate.png,
> 15229-recirculate-count.png, 15229-recirculate-hit-rate.png,
> 15229-recirculate-size.png, 15229-recirculate.png, 15229-size.png,
> 15229-unsafe.png
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we
> need to either change our behaviour to handle uncorrelated lifetimes or use
> something else. This is particularly important with the default chunk size
> for compressed sstables being reduced. If we address the problem, we should
> also utilise the BufferPool for native transport connections like we do for
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used
> for things with uncorrelated lifetimes, which essentially boils down to
> tracking those chunks that have not been freed and re-circulating them when
> we run out of completely free blocks. We should probably also permit
> instantiating separate {{BufferPool}}, so that we can insulate internode
> messaging from the {{ChunkCache}}, or at least have separate memory bounds
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce
> the amount of global coordination and per-allocation overhead. We don’t need
> 1KiB granularity for allocations, nor 16 byte granularity for tiny
> allocations.
> -
> Since CASSANDRA-5863, chunk cache is implemented to use buffer pool. When
> local pool is full, one of its chunks will be evicted and only put back to
> global pool when all buffers in the evicted chunk are released. But due to
> chunk cache, buffers can be held for long period of time, preventing evicted
> chunk to be recycled even though most of space in the evicted chunk are free.
> There two things need to be improved:
> 1. Evicted chunk with free space should be recycled to global pool, even if
> it's not fully free. It's doable in 4.0.
> 2. Reduce fragmentation caused by different buffer size. With #1, partially
> freed chunk will be available for allocation, but "holes" in the partially
> freed chunk are with different sizes. We should consider allocating fixed
> buffer size which is unlikely to fit in 4.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16128) Jenkins: dsl for website build, logging repo SHAs, and using nightlies.a.o instead of archiving
[ https://issues.apache.org/jira/browse/CASSANDRA-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16128: --- Fix Version/s: (was: 4.0-beta) (was: 3.11.x) (was: 3.0.x) (was: 2.2.x) 4.0-beta3 3.11.9 3.0.23 2.2.19 > Jenkins: dsl for website build, logging repo SHAs, and using nightlies.a.o > instead of archiving > --- > > Key: CASSANDRA-16128 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16128 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 2.2.19, 3.0.23, 3.11.9, 4.0-beta3 > > > Jenkins improvements > 1. Add the cassandra-website job into cassandra_job_dsl.seed.groovy (so we > don't lose it next time the Jenkins master is corrupted) > 2. Print the SHAs of the different git repos used during the build process. > Also store them in the .head files (so the pipeline can print them out too). > 3. Instead of archiving artefacts, ssh them to > https://nightlies.apache.org/cassandra/ > (Disk usage on agents is largely under control, but disk usage on master was > the new problem. The suspicion here is the Cassandra-*-artifact's artefacts > was the disk usage culprit, though we have to evidence to support it.) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16128) Jenkins: dsl for website build, logging repo SHAs, and using nightlies.a.o instead of archiving
[ https://issues.apache.org/jira/browse/CASSANDRA-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16128: --- Source Control Link: https://github.com/apache/cassandra/commit/aaef142f48c2bff43eb358f3901e5959ce8e6ebf Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as [aaef142f48c2bff43eb358f3901e5959ce8e6ebf|https://github.com/apache/cassandra/commit/aaef142f48c2bff43eb358f3901e5959ce8e6ebf]. > Jenkins: dsl for website build, logging repo SHAs, and using nightlies.a.o > instead of archiving > --- > > Key: CASSANDRA-16128 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16128 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta > > > Jenkins improvements > 1. Add the cassandra-website job into cassandra_job_dsl.seed.groovy (so we > don't lose it next time the Jenkins master is corrupted) > 2. Print the SHAs of the different git repos used during the build process. > Also store them in the .head files (so the pipeline can print them out too). > 3. Instead of archiving artefacts, ssh them to > https://nightlies.apache.org/cassandra/ > (Disk usage on agents is largely under control, but disk usage on master was > the new problem. The suspicion here is the Cassandra-*-artifact's artefacts > was the disk usage culprit, though we have to evidence to support it.) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (7b190b8 -> 936a81b)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 7b190b8 ninja: Revert circleci changes new aaef142 In Jenkins propagate FAILURE from stages to pipeline, ssh artefacts to nightlies, and move ant's junitreport generation to a separate target "generate-test-report" new 9ee74c9 Merge branch 'cassandra-2.2' into cassandra-3.0 new f5d5e72 Merge branch 'cassandra-3.0' into cassandra-3.11 new 936a81b Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .jenkins/Jenkinsfile | 20 +++- build.xml| 21 + 2 files changed, 28 insertions(+), 13 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 936a81bf340b6072991d763ccfbb73446820ce0f
Merge: 7b190b8 f5d5e72
Author: Mick Semb Wever
AuthorDate: Wed Sep 30 21:28:44 2020 +0200
Merge branch 'cassandra-3.11' into trunk
.jenkins/Jenkinsfile | 20 +++-
build.xml| 21 +
2 files changed, 28 insertions(+), 13 deletions(-)
diff --cc .jenkins/Jenkinsfile
index 274891f,257ba1f..670b68c
--- a/.jenkins/Jenkinsfile
+++ b/.jenkins/Jenkinsfile
@@@ -55,23 -56,6 +56,24 @@@ pipeline
}
}
}
+ stage('fqltool') {
+steps {
+script {
+ fqltool = build job: "${env.JOB_NAME}-fqltool-test",
propagate: false
+ if (fqltool.result != 'SUCCESS') unstable('fqltool test
failures')
++ if (fqltool.result == 'FAILURE')
currentBuild.result='FAILURE'
+}
+}
+post {
+ always {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('fqltool-test', fqltool.getNumber())
+ }
+ }
+ }
+}
+ }
stage('jvm-dtest') {
steps {
script {
@@@ -196,17 -187,18 +205,18 @@@
script {
cqlsh = build job: "${env.JOB_NAME}-cqlsh-tests", propagate:
false
if (cqlsh.result != 'SUCCESS') unstable('cqlsh failures')
+ if (cqlsh.result == 'FAILURE') currentBuild.result='FAILURE'
}
-}
-post {
- always {
- warnError('missing test xml files') {
- script {
- copyTestResults('cqlsh-tests', cqlsh.getNumber())
- }
- }
+ }
+ post {
+always {
+warnError('missing test xml files') {
+script {
+copyTestResults('cqlsh-tests', cqlsh.getNumber())
+}
+}
}
-}
+ }
}
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: In Jenkins propagate FAILURE from stages to pipeline, ssh artefacts to nightlies, and move ant's junitreport generation to a separate target "generate-test-re
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-2.2 by this push:
new aaef142 In Jenkins propagate FAILURE from stages to pipeline, ssh
artefacts to nightlies, and move ant's junitreport generation to a separate
target "generate-test-report"
aaef142 is described below
commit aaef142f48c2bff43eb358f3901e5959ce8e6ebf
Author: Mick Semb Wever
AuthorDate: Fri Aug 28 08:47:06 2020 +0200
In Jenkins propagate FAILURE from stages to pipeline, ssh artefacts to
nightlies, and move ant's junitreport generation to a separate target
"generate-test-report"
patch by Mick Semb Wever; reviewed by Berenguer Blasi for CASSANDRA-16128
---
.jenkins/Jenkinsfile | 15 ++-
build.xml| 21 +
2 files changed, 23 insertions(+), 13 deletions(-)
diff --git a/.jenkins/Jenkinsfile b/.jenkins/Jenkinsfile
index dc5982a..09930ee 100644
--- a/.jenkins/Jenkinsfile
+++ b/.jenkins/Jenkinsfile
@@ -43,6 +43,7 @@ pipeline {
script {
jvm_dtest = build job: "${env.JOB_NAME}-jvm-dtest", propagate:
false
if (jvm_dtest.result != 'SUCCESS') unstable('jvm-dtest
failures')
+if (jvm_dtest.result == 'FAILURE')
currentBuild.result='FAILURE'
}
}
post {
@@ -60,6 +61,7 @@ pipeline {
script {
test = build job: "${env.JOB_NAME}-test", propagate: false
if (test.result != 'SUCCESS') unstable('unit test failures')
+if (test.result == 'FAILURE') currentBuild.result='FAILURE'
}
}
post {
@@ -77,6 +79,7 @@ pipeline {
script {
long_test = build job: "${env.JOB_NAME}-long-test", propagate:
false
if (long_test.result != 'SUCCESS') unstable('long unit test
failures')
+if (long_test.result == 'FAILURE')
currentBuild.result='FAILURE'
}
}
post {
@@ -94,6 +97,7 @@ pipeline {
script {
burn = build job: "${env.JOB_NAME}-test-burn", propagate: false
if (burn.result != 'SUCCESS') unstable('burn test failures')
+if (burn.result == 'FAILURE') currentBuild.result='FAILURE'
}
}
post {
@@ -111,6 +115,7 @@ pipeline {
script {
compression = build job: "${env.JOB_NAME}-test-compression",
propagate: false
if (compression.result != 'SUCCESS') unstable('compression
failures')
+if (compression.result == 'FAILURE')
currentBuild.result='FAILURE'
}
}
post {
@@ -132,6 +137,7 @@ pipeline {
script {
dtest = build job: "${env.JOB_NAME}-dtest", propagate: false
if (dtest.result != 'SUCCESS') unstable('dtest failures')
+if (dtest.result == 'FAILURE') currentBuild.result='FAILURE'
}
}
post {
@@ -149,6 +155,7 @@ pipeline {
script {
dtest_large = build job: "${env.JOB_NAME}-dtest-large",
propagate: false
if (dtest_large.result != 'SUCCESS') unstable('dtest-large
failures')
+if (dtest_large.result == 'FAILURE')
currentBuild.result='FAILURE'
}
}
post {
@@ -166,6 +173,7 @@ pipeline {
script {
dtest_novnode = build job: "${env.JOB_NAME}-dtest-novnode",
propagate: false
if (dtest_novnode.result != 'SUCCESS') unstable('dtest-novnode
failures')
+if (dtest_novnode.result == 'FAILURE')
currentBuild.result='FAILURE'
}
}
post {
@@ -183,6 +191,7 @@ pipeline {
script {
dtest_offheap = build job: "${env.JOB_NAME}-dtest-offheap",
propagate: false
if (dtest_offheap.result != 'SUCCESS') unstable('dtest-offheap
failures')
+if (dtest_offheap.result == 'FAILURE')
currentBuild.result='FAILURE'
}
}
post {
@@ -226,10 +235,14 @@
${FAILED_TESTS,maxTests=500,showMessage=false,showStack=false}
${FILE,path="cassandra-test-report.txt"}
'''
}
+ sh "echo \"cassandra-builds at: `git -C cassandra-builds log -1
--pretty=format:'%h %an %ad %s'`\" > builds.head"
+ sh "find . -type f -name \\*.head -exec cat {} \\;"
+ sh "xz cassandra-test-report.txt TESTS-TestSuites.xml"
}
post {
always {
- archiveArtifacts artifacts: 'cassandra-test-report.txt',
fingerprint: true
+ archiveArtifacts artifacts: 'cassandra-test-report.txt.xz',
[cassandra] branch cassandra-3.0 updated (8a4c1d7 -> 9ee74c9)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 8a4c1d7 Add flag to ignore unreplicated keyspaces during repair new aaef142 In Jenkins propagate FAILURE from stages to pipeline, ssh artefacts to nightlies, and move ant's junitreport generation to a separate target "generate-test-report" new 9ee74c9 Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .jenkins/Jenkinsfile | 16 +++- build.xml| 21 + 2 files changed, 24 insertions(+), 13 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit f5d5e72f61cffa16b591e4dfb527aac6c728e638
Merge: cf27558 9ee74c9
Author: Mick Semb Wever
AuthorDate: Wed Sep 30 21:10:55 2020 +0200
Merge branch 'cassandra-3.0' into cassandra-3.11
.jenkins/Jenkinsfile | 19 ++-
build.xml| 21 +
2 files changed, 27 insertions(+), 13 deletions(-)
diff --cc .jenkins/Jenkinsfile
index 404ce60,eceab9f..257ba1f
--- a/.jenkins/Jenkinsfile
+++ b/.jenkins/Jenkinsfile
@@@ -35,26 -35,9 +35,27 @@@ pipeline
steps {
build job: "${env.JOB_NAME}-artifacts"
}
- }
- stage('Test') {
+}
+stage('Test') {
parallel {
+ stage('stress') {
+steps {
+script {
+ stress = build job: "${env.JOB_NAME}-stress-test",
propagate: false
+ if (stress.result != 'SUCCESS') unstable('stress test
failures')
++ if (stress.result == 'FAILURE')
currentBuild.result='FAILURE'
+}
+}
+post {
+ always {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('stress-test', stress.getNumber())
+ }
+ }
+ }
+}
+ }
stage('jvm-dtest') {
steps {
script {
@@@ -140,23 -128,6 +146,24 @@@
}
}
}
+ stage('cdc') {
+steps {
+ script {
+cdc = build job: "${env.JOB_NAME}-test-cdc", propagate: false
+if (cdc.result != 'SUCCESS') unstable('cdc failures')
++if (cdc.result == 'FAILURE') currentBuild.result='FAILURE'
+ }
+}
+post {
+ always {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('test-cdc', cdc.getNumber())
+ }
+ }
+ }
+}
+ }
stage('compression') {
steps {
script {
@@@ -179,21 -151,22 +187,22 @@@
script {
cqlsh = build job: "${env.JOB_NAME}-cqlsh-tests", propagate:
false
if (cqlsh.result != 'SUCCESS') unstable('cqlsh failures')
+ if (cqlsh.result == 'FAILURE') currentBuild.result='FAILURE'
}
- }
- post {
-always {
-warnError('missing test xml files') {
-script {
-copyTestResults('cqlsh-test', cqlsh.getNumber())
-}
-}
+}
+post {
+ always {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('cqlsh-tests', cqlsh.getNumber())
+ }
+ }
}
- }
}
}
- }
- stage('Distributed Test') {
+}
+}
+stage('Distributed Test') {
parallel {
stage('dtest') {
steps {
@@@ -246,23 -222,6 +258,24 @@@
}
}
}
+ stage('dtest-offheap') {
+steps {
+ script {
+dtest_offheap = build job: "${env.JOB_NAME}-dtest-offheap",
propagate: false
+if (dtest_offheap.result != 'SUCCESS')
unstable('dtest-offheap failures')
++if (dtest_offheap.result == 'FAILURE')
currentBuild.result='FAILURE'
+ }
+}
+post {
+ always {
+warnError('missing test xml files') {
+script {
+copyTestResults('dtest-offheap',
dtest_offheap.getNumber())
+}
+}
+ }
+}
+ }
}
}
stage('Summary') {
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (cf27558 -> f5d5e72)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from cf27558 Don't attempt value skipping with mixed cluster new aaef142 In Jenkins propagate FAILURE from stages to pipeline, ssh artefacts to nightlies, and move ant's junitreport generation to a separate target "generate-test-report" new 9ee74c9 Merge branch 'cassandra-2.2' into cassandra-3.0 new f5d5e72 Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .jenkins/Jenkinsfile | 19 ++- build.xml| 21 + 2 files changed, 27 insertions(+), 13 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 9ee74c9dc003fa5d8a0bde3971763723ad163f09
Merge: 8a4c1d7 aaef142
Author: Mick Semb Wever
AuthorDate: Wed Sep 30 21:07:21 2020 +0200
Merge branch 'cassandra-2.2' into cassandra-3.0
.jenkins/Jenkinsfile | 16 +++-
build.xml| 21 +
2 files changed, 24 insertions(+), 13 deletions(-)
diff --cc .jenkins/Jenkinsfile
index de9761a,09930ee..eceab9f
--- a/.jenkins/Jenkinsfile
+++ b/.jenkins/Jenkinsfile
@@@ -55,23 -56,6 +56,24 @@@ pipeline
}
}
}
+ stage('jvm-dtest-upgrade') {
+steps {
+ script {
+jvm_dtest_upgrade = build job:
"${env.JOB_NAME}-jvm-dtest-upgrade", propagate: false
+if (jvm_dtest_upgrade.result != 'SUCCESS')
unstable('jvm-dtest-upgrade failures')
++if (jvm_dtest_upgrade.result == 'FAILURE')
currentBuild.result='FAILURE'
+ }
+}
+post {
+ always {
+ warnError('missing test xml files') {
+ script {
+ copyTestResults('jvm-dtest-upgrade',
jvm_dtest_upgrade.getNumber())
+ }
+ }
+ }
+}
+ }
stage('units') {
steps {
script {
@@@ -140,26 -128,9 +146,27 @@@
}
}
}
+ stage('cqlsh') {
+steps {
+ script {
+cqlsh = build job: "${env.JOB_NAME}-cqlsh-tests", propagate:
false
+ if (cqlsh.result != 'SUCCESS') unstable('cqlsh failures')
++ if (cqlsh.result == 'FAILURE') currentBuild.result='FAILURE'
+}
+ }
+ post {
+always {
+warnError('missing test xml files') {
+script {
+copyTestResults('cqlsh-test', cqlsh.getNumber())
+}
+}
+}
+ }
+}
+ }
}
-}
-stage('Distributed Test') {
+ stage('Distributed Test') {
parallel {
stage('dtest') {
steps {
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14746) Ensure Netty Internode Messaging Refactor is Solid
[ https://issues.apache.org/jira/browse/CASSANDRA-14746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204985#comment-17204985 ] Paulo Motta edited comment on CASSANDRA-14746 at 9/30/20, 7:50 PM: --- This sounds like a reasonable plan, thanks for the update [~vinaykumarcse]! If you don't mind I will set the state to "In Progress" so we can differentiate from tasks that haven't started. was (Author: pauloricardomg): This sounds like a reasonable plan, thanks for the update [~vinaykumarcse]! > Ensure Netty Internode Messaging Refactor is Solid > -- > > Key: CASSANDRA-14746 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14746 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Streaming and Messaging >Reporter: Joey Lynch >Assignee: Joey Lynch >Priority: Normal > Labels: 4.0-QA > Fix For: 4.0-beta > > > Before we release 4.0 let's ensure that the internode messaging refactor is > 100% solid. As internode messaging is naturally used in many code paths and > widely configurable we have a large number of cluster configurations and test > configurations that must be vetted. > We plan to vary the following: > * Version of Cassandra 3.0.17 vs 4.0-alpha > * Cluster sizes with *multi-dc* deployments ranging from 6 - 100 nodes > * Client request rates varying between 1k QPS and 100k QPS of varying sizes > and shapes (BATCH, INSERT, SELECT point, SELECT range, etc ...) > * Internode compression > * Internode SSL (as well as openssl vs jdk) > * Internode Coalescing options > We are looking to measure the following as appropriate: > * Latency distributions of reads and writes (lower is better) > * Scaling limit, aka maximum throughput before violating p99 latency > deadline of 10ms @ LOCAL_QUORUM, on a fixed hardware deployment for 100% > writes, 100% reads and 50-50 writes+reads (higher is better) > * Thread counts (lower is better) > * Context switches (lower is better) > * On-CPU time of tasks (higher periods without context switch is better) > * GC allocation rates / throughput for a fixed size heap (lower allocation > better) > * Streaming recovery time for a single node failure, i.e. can Cassandra > saturate the NIC > > The goal is that 4.0 should have better latency, more throughput, fewer > threads, fewer context switches, less GC allocation, and faster recovery > time. I'm putting Jason Brown as the reviewer since he implemented most of > the internode refactor. > Current collaborators driving this QA task: Dinesh Joshi, Jordan West, Joey > Lynch (Netflix), Vinay Chella (Netflix) > Owning committer(s): Jason Brown -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14746) Ensure Netty Internode Messaging Refactor is Solid
[ https://issues.apache.org/jira/browse/CASSANDRA-14746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204985#comment-17204985 ] Paulo Motta commented on CASSANDRA-14746: - This sounds like a reasonable plan, thanks for the update [~vinaykumarcse]! > Ensure Netty Internode Messaging Refactor is Solid > -- > > Key: CASSANDRA-14746 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14746 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Streaming and Messaging >Reporter: Joey Lynch >Assignee: Joey Lynch >Priority: Normal > Labels: 4.0-QA > Fix For: 4.0-beta > > > Before we release 4.0 let's ensure that the internode messaging refactor is > 100% solid. As internode messaging is naturally used in many code paths and > widely configurable we have a large number of cluster configurations and test > configurations that must be vetted. > We plan to vary the following: > * Version of Cassandra 3.0.17 vs 4.0-alpha > * Cluster sizes with *multi-dc* deployments ranging from 6 - 100 nodes > * Client request rates varying between 1k QPS and 100k QPS of varying sizes > and shapes (BATCH, INSERT, SELECT point, SELECT range, etc ...) > * Internode compression > * Internode SSL (as well as openssl vs jdk) > * Internode Coalescing options > We are looking to measure the following as appropriate: > * Latency distributions of reads and writes (lower is better) > * Scaling limit, aka maximum throughput before violating p99 latency > deadline of 10ms @ LOCAL_QUORUM, on a fixed hardware deployment for 100% > writes, 100% reads and 50-50 writes+reads (higher is better) > * Thread counts (lower is better) > * Context switches (lower is better) > * On-CPU time of tasks (higher periods without context switch is better) > * GC allocation rates / throughput for a fixed size heap (lower allocation > better) > * Streaming recovery time for a single node failure, i.e. can Cassandra > saturate the NIC > > The goal is that 4.0 should have better latency, more throughput, fewer > threads, fewer context switches, less GC allocation, and faster recovery > time. I'm putting Jason Brown as the reviewer since he implemented most of > the internode refactor. > Current collaborators driving this QA task: Dinesh Joshi, Jordan West, Joey > Lynch (Netflix), Vinay Chella (Netflix) > Owning committer(s): Jason Brown -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16128) Jenkins: dsl for website build, logging repo SHAs, and using nightlies.a.o instead of archiving
[ https://issues.apache.org/jira/browse/CASSANDRA-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16128: --- Reviewers: Berenguer Blasi, Michael Semb Wever (was: Berenguer Blasi) Status: Review In Progress (was: Patch Available) > Jenkins: dsl for website build, logging repo SHAs, and using nightlies.a.o > instead of archiving > --- > > Key: CASSANDRA-16128 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16128 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta > > > Jenkins improvements > 1. Add the cassandra-website job into cassandra_job_dsl.seed.groovy (so we > don't lose it next time the Jenkins master is corrupted) > 2. Print the SHAs of the different git repos used during the build process. > Also store them in the .head files (so the pipeline can print them out too). > 3. Instead of archiving artefacts, ssh them to > https://nightlies.apache.org/cassandra/ > (Disk usage on agents is largely under control, but disk usage on master was > the new problem. The suspicion here is the Cassandra-*-artifact's artefacts > was the disk usage culprit, though we have to evidence to support it.) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16128) Jenkins: dsl for website build, logging repo SHAs, and using nightlies.a.o instead of archiving
[ https://issues.apache.org/jira/browse/CASSANDRA-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16128: --- Status: Ready to Commit (was: Review In Progress) > Jenkins: dsl for website build, logging repo SHAs, and using nightlies.a.o > instead of archiving > --- > > Key: CASSANDRA-16128 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16128 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta > > > Jenkins improvements > 1. Add the cassandra-website job into cassandra_job_dsl.seed.groovy (so we > don't lose it next time the Jenkins master is corrupted) > 2. Print the SHAs of the different git repos used during the build process. > Also store them in the .head files (so the pipeline can print them out too). > 3. Instead of archiving artefacts, ssh them to > https://nightlies.apache.org/cassandra/ > (Disk usage on agents is largely under control, but disk usage on master was > the new problem. The suspicion here is the Cassandra-*-artifact's artefacts > was the disk usage culprit, though we have to evidence to support it.) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: ninja: Revert circleci changes
This is an automated email from the ASF dual-hosted git repository. bdeggleston pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 7b190b8 ninja: Revert circleci changes 7b190b8 is described below commit 7b190b8d837a721a35c3f083ae003cc04e45324b Author: Blake Eggleston AuthorDate: Wed Sep 30 12:15:54 2020 -0700 ninja: Revert circleci changes --- .circleci/config.yml | 98 ++-- 1 file changed, 49 insertions(+), 49 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index 45c8820..8ba8949 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -3,10 +3,10 @@ jobs: j8_jvm_upgrade_dtests: docker: - image: nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200603 -resource_class: xlarge +resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 2 +parallelism: 1 steps: - attach_workspace: at: /home/cassandra @@ -94,10 +94,10 @@ jobs: j8_cqlsh-dtests-py2-with-vnodes: docker: - image: nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200603 -resource_class: xlarge +resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 100 +parallelism: 4 steps: - attach_workspace: at: /home/cassandra @@ -171,10 +171,10 @@ jobs: j11_unit_tests: docker: - image: nastra/cassandra-testing-ubuntu1910-java11:20200603 -resource_class: xlarge +resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 100 +parallelism: 4 steps: - attach_workspace: at: /home/cassandra @@ -263,10 +263,10 @@ jobs: j8_cqlsh-dtests-py38-no-vnodes: docker: - image: nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200603 -resource_class: xlarge +resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 100 +parallelism: 4 steps: - attach_workspace: at: /home/cassandra @@ -340,10 +340,10 @@ jobs: j11_cqlsh-dtests-py3-with-vnodes: docker: - image: nastra/cassandra-testing-ubuntu1910-java11:20200603 -resource_class: xlarge +resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 100 +parallelism: 4 steps: - attach_workspace: at: /home/cassandra @@ -418,10 +418,10 @@ jobs: j11_cqlsh-dtests-py3-no-vnodes: docker: - image: nastra/cassandra-testing-ubuntu1910-java11:20200603 -resource_class: xlarge +resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 100 +parallelism: 4 steps: - attach_workspace: at: /home/cassandra @@ -496,10 +496,10 @@ jobs: j11_cqlsh-dtests-py38-with-vnodes: docker: - image: nastra/cassandra-testing-ubuntu1910-java11:20200603 -resource_class: xlarge +resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 100 +parallelism: 4 steps: - attach_workspace: at: /home/cassandra @@ -574,10 +574,10 @@ jobs: j8_cqlsh-dtests-py3-with-vnodes: docker: - image: nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200603 -resource_class: xlarge +resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 100 +parallelism: 4 steps: - attach_workspace: at: /home/cassandra @@ -651,10 +651,10 @@ jobs: j8_cqlsh-dtests-py2-no-vnodes: docker: - image: nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200603 -resource_class: xlarge +resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 100 +parallelism: 4 steps: - attach_workspace: at: /home/cassandra @@ -728,10 +728,10 @@ jobs: j11_cqlsh-dtests-py2-with-vnodes: docker: - image: nastra/cassandra-testing-ubuntu1910-java11:20200603 -resource_class: xlarge +resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 100 +parallelism: 4 steps: - attach_workspace: at: /home/cassandra @@ -806,10 +806,10 @@ jobs: j11_dtests-with-vnodes: docker: - image: nastra/cassandra-testing-ubuntu1910-java11:20200603 -resource_class: xlarge +resource_class: medium working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 100 +parallelism: 4 steps: - attach_workspace: at: /home/cassandra @@ -887,10 +887,10 @@ jobs: j8_dtests-no-vnodes: docker: - image:
[jira] [Commented] (CASSANDRA-15991) 15583 - Add UX tests to intree LHF tooling
[ https://issues.apache.org/jira/browse/CASSANDRA-15991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204969#comment-17204969 ] David Capwell commented on CASSANDRA-15991: --- CI results (pending) Circle: https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-15991-trunk-BE343BE9-1595-4210-867C-65946BBC011A Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/53/ > 15583 - Add UX tests to intree LHF tooling > -- > > Key: CASSANDRA-15991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15991 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > As per CASSANDRA-15583 many in tree tools lack proper UX tooling: mandatory > params are indeed mandatory, 'help' produces an actual help, return codes etc > This ticket is an attempt to add it to those tools that classify as LHF. > Other tools such as nodetool, with many sub-commands, deserve a separate > ticket of their own -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[
https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204967#comment-17204967
]
David Capwell commented on CASSANDRA-16150:
---
bq. I think you reverted a snakeyaml upgrade patch from another ticket
recently?
What? [~ifesdjeen] just upgraded to 1.23 for a harry patch and he fixed a bug
caused after the merge; trunk is currently 1.23.
bq. Just decided to mention it as, as far as I remember, the update there led
to ClassCastExceptions.
Fixed in commit
{code}
commit fb49ab2b12bf813697971b41fe47ac11f4a240c0
Author: Alex Petrov
Date: Sun Sep 20 13:24:22 2020 +0300
Fix test failure caused by CASSANDRA-16102
Patch by Alex Petrov; reviewed by David Capwell for CASSANDRA-16102
{code}
bq. So further to the Cassandra specifics, probably worth to check on snakeyaml
side what breaking changes were done in the new version that might require
additional work on our end (if it wasn't checked already)
Yep, a good place to start is unit + dtest. I can run the dtests, was hoping
[~crazylab] would take the unit tests.
> Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
> ---
>
> Key: CASSANDRA-16150
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16150
> Project: Cassandra
> Issue Type: Bug
> Components: Dependencies
>Reporter: Rahul Nandi
>Assignee: Rahul Nandi
>Priority: Normal
> Fix For: 4.x
>
>
> There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml
> version earlier to 1.26. This has been patched into snakeyaml version 1.26.
> Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640]
> This card is expected to upgrade the snakeyaml version to 1.26.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16147) ValueAccessor is using signed shorts in sliceWithShortLength
[ https://issues.apache.org/jira/browse/CASSANDRA-16147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-16147: Since Version: 4.0-beta Source Control Link: https://github.com/apache/cassandra/commit/8be1cbe9ab14155773bfab765a3567df9ff9833f Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed to trunk, thanks! > ValueAccessor is using signed shorts in sliceWithShortLength > > > Key: CASSANDRA-16147 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16147 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 4.0-beta > > > ValueAccessor is using a signed short when interpreting byte lengths, causing > exceptions when reading blobs over 32767 bytes in length -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Use unsigned short in ValueAccessor.sliceWithShortLength
This is an automated email from the ASF dual-hosted git repository. bdeggleston pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 8be1cbe Use unsigned short in ValueAccessor.sliceWithShortLength 8be1cbe is described below commit 8be1cbe9ab14155773bfab765a3567df9ff9833f Author: Blake Eggleston AuthorDate: Tue Sep 29 14:20:05 2020 -0700 Use unsigned short in ValueAccessor.sliceWithShortLength Patch by Blake Eggleston; Reviewed by Caleb Rackliffe and David Capwell for CASSANDRA-16147 --- .circleci/config.yml | 98 +++--- CHANGES.txt| 1 + .../cassandra/db/marshal/ByteArrayAccessor.java| 53 .../cassandra/db/marshal/ByteBufferAccessor.java | 53 .../apache/cassandra/db/marshal/CompositeType.java | 2 +- .../apache/cassandra/db/marshal/ValueAccessor.java | 14 +--- .../org/apache/cassandra/utils/ByteArrayUtil.java | 4 + .../org/apache/cassandra/utils/ByteBufferUtil.java | 8 +- .../cassandra/db/marshal/CompositeTypeTest.java| 22 + .../cassandra/db/marshal/ValueAccessorTest.java| 54 10 files changed, 247 insertions(+), 62 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index 8ba8949..45c8820 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -3,10 +3,10 @@ jobs: j8_jvm_upgrade_dtests: docker: - image: nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200603 -resource_class: medium +resource_class: xlarge working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 1 +parallelism: 2 steps: - attach_workspace: at: /home/cassandra @@ -94,10 +94,10 @@ jobs: j8_cqlsh-dtests-py2-with-vnodes: docker: - image: nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200603 -resource_class: medium +resource_class: xlarge working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 4 +parallelism: 100 steps: - attach_workspace: at: /home/cassandra @@ -171,10 +171,10 @@ jobs: j11_unit_tests: docker: - image: nastra/cassandra-testing-ubuntu1910-java11:20200603 -resource_class: medium +resource_class: xlarge working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 4 +parallelism: 100 steps: - attach_workspace: at: /home/cassandra @@ -263,10 +263,10 @@ jobs: j8_cqlsh-dtests-py38-no-vnodes: docker: - image: nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200603 -resource_class: medium +resource_class: xlarge working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 4 +parallelism: 100 steps: - attach_workspace: at: /home/cassandra @@ -340,10 +340,10 @@ jobs: j11_cqlsh-dtests-py3-with-vnodes: docker: - image: nastra/cassandra-testing-ubuntu1910-java11:20200603 -resource_class: medium +resource_class: xlarge working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 4 +parallelism: 100 steps: - attach_workspace: at: /home/cassandra @@ -418,10 +418,10 @@ jobs: j11_cqlsh-dtests-py3-no-vnodes: docker: - image: nastra/cassandra-testing-ubuntu1910-java11:20200603 -resource_class: medium +resource_class: xlarge working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 4 +parallelism: 100 steps: - attach_workspace: at: /home/cassandra @@ -496,10 +496,10 @@ jobs: j11_cqlsh-dtests-py38-with-vnodes: docker: - image: nastra/cassandra-testing-ubuntu1910-java11:20200603 -resource_class: medium +resource_class: xlarge working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 4 +parallelism: 100 steps: - attach_workspace: at: /home/cassandra @@ -574,10 +574,10 @@ jobs: j8_cqlsh-dtests-py3-with-vnodes: docker: - image: nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200603 -resource_class: medium +resource_class: xlarge working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 4 +parallelism: 100 steps: - attach_workspace: at: /home/cassandra @@ -651,10 +651,10 @@ jobs: j8_cqlsh-dtests-py2-no-vnodes: docker: - image: nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200603 -resource_class: medium +resource_class: xlarge working_directory: ~/ shell: /bin/bash -eo pipefail -l -parallelism: 4 +parallelism: 100 steps: - attach_workspace: at: /home/cassandra @@ -728,10 +728,10 @@ jobs: j11_cqlsh-dtests-py2-with-vnodes: docker: - image:
[jira] [Commented] (CASSANDRA-16074) Add metric for client concurrent byte throttle
[ https://issues.apache.org/jira/browse/CASSANDRA-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204956#comment-17204956 ] Michael Semb Wever commented on CASSANDRA-16074: +1 > Add metric for client concurrent byte throttle > -- > > Key: CASSANDRA-16074 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16074 > Project: Cassandra > Issue Type: New Feature > Components: Messaging/Client, Observability/Metrics >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Normal > Fix For: 4.0-beta > > > Add a metric to expose the current bytes and bytes per ip used that is used > in the existing throttle so its possible to determine what to set it to. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16074) Add metric for client concurrent byte throttle
[ https://issues.apache.org/jira/browse/CASSANDRA-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16074: --- Status: Ready to Commit (was: Review In Progress) > Add metric for client concurrent byte throttle > -- > > Key: CASSANDRA-16074 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16074 > Project: Cassandra > Issue Type: New Feature > Components: Messaging/Client, Observability/Metrics >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Normal > Fix For: 4.0-beta > > > Add a metric to expose the current bytes and bytes per ip used that is used > in the existing throttle so its possible to determine what to set it to. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16115) New Cassandra website design, content and layout to work with Antora
[ https://issues.apache.org/jira/browse/CASSANDRA-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16115: --- Change Category: Semantic Complexity: Normal Fix Version/s: 4.0-rc Status: Open (was: Triage Needed) > New Cassandra website design, content and layout to work with Antora > > > Key: CASSANDRA-16115 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16115 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Melissa Logan >Assignee: Melissa Logan >Priority: Normal > Fix For: 4.0-rc > > Attachments: Screen Shot 2020-09-03 at 09.48.53.png > > > This task is related to CASSANDRA-16066 (Update and rework the > cassandra-website material to work with Antora). The goal is to update the > front-end of the C* website (design, IA and content) to work with Antora to > help modernize the website as discussed on the [mailing > list|https://www.mail-archive.com/dev@cassandra.apache.org/msg15537.html]. > *Design Concepts:* A minimum of two homepage design concepts will be created > and shared for input, which will help standardize a brand palette for C* and > a design language for the site. This may include custom iconography and > graphics. The chosen design language will be used to develop the remaining > templates. > *Template Design*: It's estimated that 7 template designs will be needed > including the creation of several new pages: > * Homepage template > * Toplevel template - e.g. Community. > * General template - Mostly textual with some images, e.g. Intro, Quickstart > * “Library” template - A library of assets (links, downloads, logos etc) > that are sortable by metadata, e.g Resources, or Kafka's Powered By page). > * Blog landing template > * Blog single template > * Docs template > *Website Content:* Along with new design will be a need for new or updated > content to fit the new page layouts. The intention is to use as much as > possible from existing content, and augment with new content where needed. > *Template Development:* This includes the frontend development, such as any > HTML markup to achieve designs. HTML would be crafted so as to preserve any > backend/API calls, such that content is pulled in as designed. The majority > of the frontend work would come in the form of crafting CSS to bring the > designs to life, plus any minor Javascript to add subtle delights to key > pages. > *Style Guide*: Once all is complete, a Style Guide be added to GitHub for > contributors. > The [cassandra-website|https://github.com/apache/cassandra-website] > repository would need to be modified. Specific changes to be determined. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204932#comment-17204932 ] Ekaterina Dimitrova edited comment on CASSANDRA-16150 at 9/30/20, 5:57 PM: --- [~dcapwell] I think you reverted a snakeyaml upgrade patch from another ticket recently? Just decided to mention it as, as far as I remember, the update there led to ClassCastExceptions. So further to the Cassandra specifics, probably worth to check on snakeyaml side what breaking changes were done in the new version that might require additional work on our end (if it wasn't checked already) was (Author: e.dimitrova): [~dcapwell] I think you reverted a snakeyaml patch from another ticket recently? Just decided to mention it as, as far as I remember, the update there led to ClassCastExceptions. So further to the Cassandra specifics, probably worth to check on snakeyaml side what breaking changes were done in the new version that might require additional work on our end (if it wasn't checked already) > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.x > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204932#comment-17204932 ] Ekaterina Dimitrova commented on CASSANDRA-16150: - [~dcapwell] I think you reverted a snakeyaml patch from another ticket recently? Just decided to mention it as, as far as I remember, the update there led to ClassCastExceptions. So further to the Cassandra specifics, probably worth to check on snakeyaml side what breaking changes were done in the new version that might require additional work on our end (if it wasn't checked already) > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.x > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15991) 15583 - Add UX tests to intree LHF tooling
[ https://issues.apache.org/jira/browse/CASSANDRA-15991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204924#comment-17204924 ] David Capwell commented on CASSANDRA-15991: --- thanks. I need to work on something due today, after that is done ill restart the commit process. > 15583 - Add UX tests to intree LHF tooling > -- > > Key: CASSANDRA-15991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15991 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > As per CASSANDRA-15583 many in tree tools lack proper UX tooling: mandatory > params are indeed mandatory, 'help' produces an actual help, return codes etc > This ticket is an attempt to add it to those tools that classify as LHF. > Other tools such as nodetool, with many sub-commands, deserve a separate > ticket of their own -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16150: -- Reviewers: David Capwell, David Capwell (was: David Capwell) David Capwell, David Capwell Status: Review In Progress (was: Patch Available) left comment in PR but its related to the document [~e.dimitrova] shared. After making the change, it would be good to run the config tests to make sure things work properly; I can run the whole suite once this is done. > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.x > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16150: -- Test and Documentation Plan: unit tests Status: Patch Available (was: Open) > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.x > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16147) ValueAccessor is using signed shorts in sliceWithShortLength
[ https://issues.apache.org/jira/browse/CASSANDRA-16147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16147: -- Status: Ready to Commit (was: Review In Progress) > ValueAccessor is using signed shorts in sliceWithShortLength > > > Key: CASSANDRA-16147 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16147 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 4.0-beta > > > ValueAccessor is using a signed short when interpreting byte lengths, causing > exceptions when reading blobs over 32767 bytes in length -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16148) GossiperTest#testHaveVersion3Nodes is failing on trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16148: -- Reviewers: David Capwell, David Capwell (was: David Capwell) David Capwell, David Capwell Status: Review In Progress (was: Patch Available) > GossiperTest#testHaveVersion3Nodes is failing on trunk > -- > > Key: CASSANDRA-16148 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16148 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > https://app.circleci.com/pipelines/github/jrwest/cassandra/53/workflows/95f9f401-1ef8-4b8d-9c64-3703d9669d95/jobs/771 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15229) BufferPool Regression
[
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204853#comment-17204853
]
Benedict Elliott Smith commented on CASSANDRA-15229:
Ditto
> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Assignee: Zhao Yang
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: 15229-count.png, 15229-direct.png, 15229-hit-rate.png,
> 15229-recirculate-count.png, 15229-recirculate-hit-rate.png,
> 15229-recirculate-size.png, 15229-recirculate.png, 15229-size.png,
> 15229-unsafe.png
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we
> need to either change our behaviour to handle uncorrelated lifetimes or use
> something else. This is particularly important with the default chunk size
> for compressed sstables being reduced. If we address the problem, we should
> also utilise the BufferPool for native transport connections like we do for
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used
> for things with uncorrelated lifetimes, which essentially boils down to
> tracking those chunks that have not been freed and re-circulating them when
> we run out of completely free blocks. We should probably also permit
> instantiating separate {{BufferPool}}, so that we can insulate internode
> messaging from the {{ChunkCache}}, or at least have separate memory bounds
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce
> the amount of global coordination and per-allocation overhead. We don’t need
> 1KiB granularity for allocations, nor 16 byte granularity for tiny
> allocations.
> -
> Since CASSANDRA-5863, chunk cache is implemented to use buffer pool. When
> local pool is full, one of its chunks will be evicted and only put back to
> global pool when all buffers in the evicted chunk are released. But due to
> chunk cache, buffers can be held for long period of time, preventing evicted
> chunk to be recycled even though most of space in the evicted chunk are free.
> There two things need to be improved:
> 1. Evicted chunk with free space should be recycled to global pool, even if
> it's not fully free. It's doable in 4.0.
> 2. Reduce fragmentation caused by different buffer size. With #1, partially
> freed chunk will be available for allocation, but "holes" in the partially
> freed chunk are with different sizes. We should consider allocating fixed
> buffer size which is unlikely to fit in 4.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14788) Add test coverage workflows to CircleCI config
[ https://issues.apache.org/jira/browse/CASSANDRA-14788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204847#comment-17204847 ] Ekaterina Dimitrova commented on CASSANDRA-14788: - I think the coverage is a good topic but I am also not sure when I'm gonna be able to take this ticket so I will refrain from assigning it now. It was more of a ping our of curiosity related to some discussion I had lately about code coverage in Cassandra. I was just wondering where do we stand here :) Thanks [~jmeredithco] and [~pauloricardomg] > Add test coverage workflows to CircleCI config > -- > > Key: CASSANDRA-14788 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14788 > Project: Cassandra > Issue Type: Improvement > Components: Build, CI >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Low > Labels: CI, pull-request-available > Fix For: 4.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > To support 4.0 testing efforts it's helpful to know how much of the code is > being exercised by unit tests and dtests. > Add support for running the unit tests and dtests instrumented for test > coverage on CircleCI and then combine the results of all tests (unit, dtest > with vnodes, dtest without vnodes) into a single coverage report. > All of the hard work of getting JaCoCo to work with unit tests and dtests has > already been done, it just needs wiring up. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16147) ValueAccessor is using signed shorts in sliceWithShortLength
[
https://issues.apache.org/jira/browse/CASSANDRA-16147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204841#comment-17204841
]
Caleb Rackliffe commented on CASSANDRA-16147:
-
+1 (with the minor nit about changing {{ByteBufferUtil.getShortLength()}} to
{{ByteBufferUtil.getUnsignedShort()}})
> ValueAccessor is using signed shorts in sliceWithShortLength
>
>
> Key: CASSANDRA-16147
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16147
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Other
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0-beta
>
>
> ValueAccessor is using a signed short when interpreting byte lengths, causing
> exceptions when reading blobs over 32767 bytes in length
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204839#comment-17204839 ] Ekaterina Dimitrova edited comment on CASSANDRA-16150 at 9/30/20, 4:05 PM: --- Hi [~crazylab], please, let us know if you need any help with this update (I guess you will be working on it as you assigned it?) Some reference about updating dependencies in Cassandra(sorry if you are already aware of this doc, just wanted to save you some time if you haven't seen it yet :)): [https://cassandra.apache.org/doc/latest/development/dependencies.html] was (Author: e.dimitrova): Hi [~crazylab], please, let us know if you need any help with this update (I guess you will be working on it as you assigned it?) Some reference about updating dependencies in Cassandra(sorry if you are already aware of this doc, just wanted to save you some time if you haven't :)): https://cassandra.apache.org/doc/latest/development/dependencies.html > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.x > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204839#comment-17204839 ] Ekaterina Dimitrova commented on CASSANDRA-16150: - Hi [~crazylab], please, let us know if you need any help with this update (I guess you will be working on it as you assigned it?) Some reference about updating dependencies in Cassandra(sorry if you are already aware of this doc, just wanted to save you some time if you haven't :)): https://cassandra.apache.org/doc/latest/development/dependencies.html > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.x > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16147) ValueAccessor is using signed shorts in sliceWithShortLength
[ https://issues.apache.org/jira/browse/CASSANDRA-16147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-16147: Reviewers: Caleb Rackliffe, David Capwell (was: David Capwell) > ValueAccessor is using signed shorts in sliceWithShortLength > > > Key: CASSANDRA-16147 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16147 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 4.0-beta > > > ValueAccessor is using a signed short when interpreting byte lengths, causing > exceptions when reading blobs over 32767 bytes in length -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15229) BufferPool Regression
[
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204807#comment-17204807
]
Caleb Rackliffe commented on CASSANDRA-15229:
-
I'm still in the middle of my review, but just want to +1 something like
{{chunkCache}} and {{networking}} for the names of the pools.
> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Assignee: Zhao Yang
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: 15229-count.png, 15229-direct.png, 15229-hit-rate.png,
> 15229-recirculate-count.png, 15229-recirculate-hit-rate.png,
> 15229-recirculate-size.png, 15229-recirculate.png, 15229-size.png,
> 15229-unsafe.png
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we
> need to either change our behaviour to handle uncorrelated lifetimes or use
> something else. This is particularly important with the default chunk size
> for compressed sstables being reduced. If we address the problem, we should
> also utilise the BufferPool for native transport connections like we do for
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used
> for things with uncorrelated lifetimes, which essentially boils down to
> tracking those chunks that have not been freed and re-circulating them when
> we run out of completely free blocks. We should probably also permit
> instantiating separate {{BufferPool}}, so that we can insulate internode
> messaging from the {{ChunkCache}}, or at least have separate memory bounds
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce
> the amount of global coordination and per-allocation overhead. We don’t need
> 1KiB granularity for allocations, nor 16 byte granularity for tiny
> allocations.
> -
> Since CASSANDRA-5863, chunk cache is implemented to use buffer pool. When
> local pool is full, one of its chunks will be evicted and only put back to
> global pool when all buffers in the evicted chunk are released. But due to
> chunk cache, buffers can be held for long period of time, preventing evicted
> chunk to be recycled even though most of space in the evicted chunk are free.
> There two things need to be improved:
> 1. Evicted chunk with free space should be recycled to global pool, even if
> it's not fully free. It's doable in 4.0.
> 2. Reduce fragmentation caused by different buffer size. With #1, partially
> freed chunk will be available for allocation, but "holes" in the partially
> freed chunk are with different sizes. We should consider allocating fixed
> buffer size which is unlikely to fit in 4.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
[ https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-16150: - Bug Category: Parent values: Correctness(12982) Complexity: Low Hanging Fruit Discovered By: User Report Fix Version/s: 4.x Severity: Low Status: Open (was: Triage Needed) > Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix > --- > > Key: CASSANDRA-16150 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16150 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Rahul Nandi >Assignee: Rahul Nandi >Priority: Normal > Fix For: 4.x > > > There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml > version earlier to 1.26. This has been patched into snakeyaml version 1.26. > Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640] > This card is expected to upgrade the snakeyaml version to 1.26. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15229) BufferPool Regression
[
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204787#comment-17204787
]
Aleksey Yeschenko commented on CASSANDRA-15229:
---
Thanks for fixing the test issues in the past couple commits (and sorry for the
delay in review).
One thing I'm not a fan of is names of the two pools - permanent and temporary
- as neither describe their respective pools. Something along the lines of
'long lived' and 'short-lived' would work better. Or, perhaps, name them after
their use cases - 'chunk-cache' and 'networking' pools.
Other than that:
1. {{PermanentBufferPool}} - unused class
2. {{Chunk#fullyRecycled}} is never read, only written to
3. {{putUnusedPortion()}} probably shouldn’t update overflow metric, as this
will double-count some of the size when it’s {{put()}} back
4. nit: {{else if}} on L807 doesn’t need a pair of braces for the first two
conditions
> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Assignee: Zhao Yang
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: 15229-count.png, 15229-direct.png, 15229-hit-rate.png,
> 15229-recirculate-count.png, 15229-recirculate-hit-rate.png,
> 15229-recirculate-size.png, 15229-recirculate.png, 15229-size.png,
> 15229-unsafe.png
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we
> need to either change our behaviour to handle uncorrelated lifetimes or use
> something else. This is particularly important with the default chunk size
> for compressed sstables being reduced. If we address the problem, we should
> also utilise the BufferPool for native transport connections like we do for
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used
> for things with uncorrelated lifetimes, which essentially boils down to
> tracking those chunks that have not been freed and re-circulating them when
> we run out of completely free blocks. We should probably also permit
> instantiating separate {{BufferPool}}, so that we can insulate internode
> messaging from the {{ChunkCache}}, or at least have separate memory bounds
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce
> the amount of global coordination and per-allocation overhead. We don’t need
> 1KiB granularity for allocations, nor 16 byte granularity for tiny
> allocations.
> -
> Since CASSANDRA-5863, chunk cache is implemented to use buffer pool. When
> local pool is full, one of its chunks will be evicted and only put back to
> global pool when all buffers in the evicted chunk are released. But due to
> chunk cache, buffers can be held for long period of time, preventing evicted
> chunk to be recycled even though most of space in the evicted chunk are free.
> There two things need to be improved:
> 1. Evicted chunk with free space should be recycled to global pool, even if
> it's not fully free. It's doable in 4.0.
> 2. Reduce fragmentation caused by different buffer size. With #1, partially
> freed chunk will be available for allocation, but "holes" in the partially
> freed chunk are with different sizes. We should consider allocating fixed
> buffer size which is unlikely to fit in 4.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14788) Add test coverage workflows to CircleCI config
[ https://issues.apache.org/jira/browse/CASSANDRA-14788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204771#comment-17204771 ] Paulo Motta commented on CASSANDRA-14788: - I'm not sure if this is already being covered by this ticket but something useful I saw in another project is to add a notification on the PR on how much a changeset increases or decreases overall code coverage. For example, on this auto-generated ASF Github bot comment from LANG-1252: Coverage increased (+0.009%) to 93.463% when pulling **0a0a35f54f5e7ab2d10022d3ee244cbc876bdde2 on chtompki:LANG-1252** into **d53d0419f1c948d3cc7454254ab9a3cb18ca9d3a on apache:master**. > Add test coverage workflows to CircleCI config > -- > > Key: CASSANDRA-14788 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14788 > Project: Cassandra > Issue Type: Improvement > Components: Build, CI >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Low > Labels: CI, pull-request-available > Fix For: 4.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > To support 4.0 testing efforts it's helpful to know how much of the code is > being exercised by unit tests and dtests. > Add support for running the unit tests and dtests instrumented for test > coverage on CircleCI and then combine the results of all tests (unit, dtest > with vnodes, dtest without vnodes) into a single coverage report. > All of the hard work of getting JaCoCo to work with unit tests and dtests has > already been done, it just needs wiring up. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16151) Package tools/bin scripts as executable
[
https://issues.apache.org/jira/browse/CASSANDRA-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-16151:
Test and Documentation Plan: No documentation needed.
Status: Patch Available (was: Open)
> Package tools/bin scripts as executable
> ---
>
> Key: CASSANDRA-16151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16151
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 4.0-beta, 3.11.9
>
> Attachments: 3.11-Package-tools-bin-scripts-as-executable.patch,
> trunk-Package-tools-bin-scripts-as-executable.patch
>
>
> The tools/bin scripts aren't packaged as executable in the source
> distributions, though in the repository the scripts have the right bits.
> This causes, on 3.11.8 for example, the tests in
> org.apache.cassandra.cql3.EmptyValuesTest to fail:
> {{java.io.IOException: Cannot run program "tools/bin/sstabledump": error=13,
> Permission denied}}
> {{[junit-timeout] junit.framework.AssertionFailedError: java.io.IOException}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:85)}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)}}
> See attached patch of build.xml for the trunk and cassandra-3.11 branches.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16151) Package tools/bin scripts as executable
[
https://issues.apache.org/jira/browse/CASSANDRA-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-16151:
Bug Category: Parent values: Code(13163)Level 1 values: Bug - Unclear
Impact(13164)
Complexity: Low Hanging Fruit
Discovered By: User Report
Fix Version/s: 3.11.9
4.0-beta
Severity: Normal
Status: Open (was: Triage Needed)
> Package tools/bin scripts as executable
> ---
>
> Key: CASSANDRA-16151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16151
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 4.0-beta, 3.11.9
>
> Attachments: 3.11-Package-tools-bin-scripts-as-executable.patch,
> trunk-Package-tools-bin-scripts-as-executable.patch
>
>
> The tools/bin scripts aren't packaged as executable in the source
> distributions, though in the repository the scripts have the right bits.
> This causes, on 3.11.8 for example, the tests in
> org.apache.cassandra.cql3.EmptyValuesTest to fail:
> {{java.io.IOException: Cannot run program "tools/bin/sstabledump": error=13,
> Permission denied}}
> {{[junit-timeout] junit.framework.AssertionFailedError: java.io.IOException}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:85)}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)}}
> See attached patch of build.xml for the trunk and cassandra-3.11 branches.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16151) Package tools/bin scripts as executable
[
https://issues.apache.org/jira/browse/CASSANDRA-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta reassigned CASSANDRA-16151:
---
Assignee: Angelo Polo
> Package tools/bin scripts as executable
> ---
>
> Key: CASSANDRA-16151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16151
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Attachments: 3.11-Package-tools-bin-scripts-as-executable.patch,
> trunk-Package-tools-bin-scripts-as-executable.patch
>
>
> The tools/bin scripts aren't packaged as executable in the source
> distributions, though in the repository the scripts have the right bits.
> This causes, on 3.11.8 for example, the tests in
> org.apache.cassandra.cql3.EmptyValuesTest to fail:
> {{java.io.IOException: Cannot run program "tools/bin/sstabledump": error=13,
> Permission denied}}
> {{[junit-timeout] junit.framework.AssertionFailedError: java.io.IOException}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:85)}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)}}
> See attached patch of build.xml for the trunk and cassandra-3.11 branches.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14788) Add test coverage workflows to CircleCI config
[ https://issues.apache.org/jira/browse/CASSANDRA-14788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204729#comment-17204729 ] Jon Meredith commented on CASSANDRA-14788: -- Well, I was right it wasn't going to be that week. I would really like to have it working again, but I don't see getting to it in the next two weeks. You're very welcome to take it, otherwise I'll try and get to it once I've closed out some of the other 4.0 things I'm working on. > Add test coverage workflows to CircleCI config > -- > > Key: CASSANDRA-14788 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14788 > Project: Cassandra > Issue Type: Improvement > Components: Build, CI >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Low > Labels: CI, pull-request-available > Fix For: 4.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > To support 4.0 testing efforts it's helpful to know how much of the code is > being exercised by unit tests and dtests. > Add support for running the unit tests and dtests instrumented for test > coverage on CircleCI and then combine the results of all tests (unit, dtest > with vnodes, dtest without vnodes) into a single coverage report. > All of the hard work of getting JaCoCo to work with unit tests and dtests has > already been done, it just needs wiring up. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14788) Add test coverage workflows to CircleCI config
[ https://issues.apache.org/jira/browse/CASSANDRA-14788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204714#comment-17204714 ] Ekaterina Dimitrova commented on CASSANDRA-14788: - I was wondering whether this is still on the table? :) > Add test coverage workflows to CircleCI config > -- > > Key: CASSANDRA-14788 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14788 > Project: Cassandra > Issue Type: Improvement > Components: Build, CI >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Low > Labels: CI, pull-request-available > Fix For: 4.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > To support 4.0 testing efforts it's helpful to know how much of the code is > being exercised by unit tests and dtests. > Add support for running the unit tests and dtests instrumented for test > coverage on CircleCI and then combine the results of all tests (unit, dtest > with vnodes, dtest without vnodes) into a single coverage report. > All of the hard work of getting JaCoCo to work with unit tests and dtests has > already been done, it just needs wiring up. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14788) Add test coverage workflows to CircleCI config
[ https://issues.apache.org/jira/browse/CASSANDRA-14788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204714#comment-17204714 ] Ekaterina Dimitrova edited comment on CASSANDRA-14788 at 9/30/20, 1:06 PM: --- [~jmeredithco] I was wondering whether this is still on the table? :) was (Author: e.dimitrova): I was wondering whether this is still on the table? :) > Add test coverage workflows to CircleCI config > -- > > Key: CASSANDRA-14788 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14788 > Project: Cassandra > Issue Type: Improvement > Components: Build, CI >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Low > Labels: CI, pull-request-available > Fix For: 4.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > To support 4.0 testing efforts it's helpful to know how much of the code is > being exercised by unit tests and dtests. > Add support for running the unit tests and dtests instrumented for test > coverage on CircleCI and then combine the results of all tests (unit, dtest > with vnodes, dtest without vnodes) into a single coverage report. > All of the hard work of getting JaCoCo to work with unit tests and dtests has > already been done, it just needs wiring up. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15769) cqllib testing not running in CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-15769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15769: Resolution: Duplicate Status: Resolved (was: Open) > cqllib testing not running in CircleCI > -- > > Key: CASSANDRA-15769 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15769 > Project: Cassandra > Issue Type: Improvement > Components: Build, CI >Reporter: David Capwell >Priority: Normal > Labels: CI > Time Spent: 10m > Remaining Estimate: 0h > > CASSANDRA-14951 added the ability to run the cqllib tests but didn’t add them > to CircleCI and are missing currently. It would be best to add the matrix of > supported cases to CircleCI. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16151) Package tools/bin scripts as executable
[
https://issues.apache.org/jira/browse/CASSANDRA-16151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Angelo Polo updated CASSANDRA-16151:
Labels: patch (was: )
> Package tools/bin scripts as executable
> ---
>
> Key: CASSANDRA-16151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16151
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Angelo Polo
>Priority: Normal
> Labels: patch
> Attachments: 3.11-Package-tools-bin-scripts-as-executable.patch,
> trunk-Package-tools-bin-scripts-as-executable.patch
>
>
> The tools/bin scripts aren't packaged as executable in the source
> distributions, though in the repository the scripts have the right bits.
> This causes, on 3.11.8 for example, the tests in
> org.apache.cassandra.cql3.EmptyValuesTest to fail:
> {{java.io.IOException: Cannot run program "tools/bin/sstabledump": error=13,
> Permission denied}}
> {{[junit-timeout] junit.framework.AssertionFailedError: java.io.IOException}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:85)}}
> {{[junit-timeout] at
> org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)}}
> See attached patch of build.xml for the trunk and cassandra-3.11 branches.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16151) Package tools/bin scripts as executable
Angelo Polo created CASSANDRA-16151:
---
Summary: Package tools/bin scripts as executable
Key: CASSANDRA-16151
URL: https://issues.apache.org/jira/browse/CASSANDRA-16151
Project: Cassandra
Issue Type: Bug
Components: Packaging
Reporter: Angelo Polo
Attachments: 3.11-Package-tools-bin-scripts-as-executable.patch,
trunk-Package-tools-bin-scripts-as-executable.patch
The tools/bin scripts aren't packaged as executable in the source
distributions, though in the repository the scripts have the right bits.
This causes, on 3.11.8 for example, the tests in
org.apache.cassandra.cql3.EmptyValuesTest to fail:
{{java.io.IOException: Cannot run program "tools/bin/sstabledump": error=13,
Permission denied}}
{{[junit-timeout] junit.framework.AssertionFailedError: java.io.IOException}}
{{[junit-timeout] at
org.apache.cassandra.cql3.EmptyValuesTest.verify(EmptyValuesTest.java:85)}}
{{[junit-timeout] at
org.apache.cassandra.cql3.EmptyValuesTest.verifyJsonInsert(EmptyValuesTest.java:112)}}
See attached patch of build.xml for the trunk and cassandra-3.11 branches.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15943) Tests fail with a space in the path
[
https://issues.apache.org/jira/browse/CASSANDRA-15943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204569#comment-17204569
]
Michael Semb Wever commented on CASSANDRA-15943:
ASF Infra is fixing the underlying issue (that multi-jdk matrix builds leads to
spaces in the build directory), rolling it out on the Sunday 4th October at
around 1300 UTC.
https://lists.apache.org/thread.html/rb4c2834b9874b9f4a74c528de9055958483d2bc6e62c3464bc5c053f%40%3Cbuilds.apache.org%3E
> Tests fail with a space in the path
> ---
>
> Key: CASSANDRA-15943
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15943
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0, 2.2.x, 3.0.x, 3.11.x
>
>
> Tests fail if there's a space in {{$\{basedir}}}
> For example, when the absolute path is used when specifying {{-javaagent}}
> arguments.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15581) 4.0 quality testing: Compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer reassigned CASSANDRA-15581: -- Assignee: (was: Benjamin Lerer) > 4.0 quality testing: Compaction > --- > > Key: CASSANDRA-15581 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15581 > Project: Cassandra > Issue Type: Task > Components: Test/dtest/python >Reporter: Josh McKenzie >Priority: Normal > Fix For: 4.0-beta > > > Reference [doc from > NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#] > for context. > *Shepherd: Marcus Eriksson* > Alongside the local and distributed read/write paths, we'll also want to > validate compaction. CASSANDRA-6696 introduced substantial > changes/improvements that require testing (esp. JBOD). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15581) 4.0 quality testing: Compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204562#comment-17204562 ] Benjamin Lerer commented on CASSANDRA-15581: @pauloricard...@gmail.com No, I have not had the time to start working on this ticket yet . I will remove myself as assignee so if somebody has more bandwidth than me he will be able to jump in. Le mer. 30 sept. 2020 à 00:18, Paulo Motta (Jira) a > 4.0 quality testing: Compaction > --- > > Key: CASSANDRA-15581 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15581 > Project: Cassandra > Issue Type: Task > Components: Test/dtest/python >Reporter: Josh McKenzie >Assignee: Benjamin Lerer >Priority: Normal > Fix For: 4.0-beta > > > Reference [doc from > NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#] > for context. > *Shepherd: Marcus Eriksson* > Alongside the local and distributed read/write paths, we'll also want to > validate compaction. CASSANDRA-6696 introduced substantial > changes/improvements that require testing (esp. JBOD). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15581) 4.0 quality testing: Compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204563#comment-17204563 ] Benjamin Lerer commented on CASSANDRA-15581: [~pauloricardomg] No, I have not had the time to start working on this ticket yet . I will remove myself as assignee so if somebody has more bandwidth than me he will be able to jump in. > 4.0 quality testing: Compaction > --- > > Key: CASSANDRA-15581 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15581 > Project: Cassandra > Issue Type: Task > Components: Test/dtest/python >Reporter: Josh McKenzie >Assignee: Benjamin Lerer >Priority: Normal > Fix For: 4.0-beta > > > Reference [doc from > NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#] > for context. > *Shepherd: Marcus Eriksson* > Alongside the local and distributed read/write paths, we'll also want to > validate compaction. CASSANDRA-6696 introduced substantial > changes/improvements that require testing (esp. JBOD). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16048) Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and Value Columns
[
https://issues.apache.org/jira/browse/CASSANDRA-16048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204558#comment-17204558
]
Marcus Eriksson commented on CASSANDRA-16048:
-
We should probably make sure that flags in {{system_schema.tables}} matches
what we have in-memory, to make sure that clients have a correct view of the
table (they probably don't do anything different in this case, but anyway).
Currently it only gets saved on the next ALTER to that table.
Would it also make sense to hide this behind a flag (start with
{{-Dcassandra.allow_ignore_compact_storage}} or something)?
> Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and
> Value Columns
> --
>
> Key: CASSANDRA-16048
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16048
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Some compact storage tables, specifically those where the user has defined
> both at least one clustering and the value column, can be safely handled in
> 4.0 because besides the DENSE flag they are not materially different post 3.0
> and there is no visible change to the user facing schema after dropping
> compact storage. We can detect this case and allow these tables to silently
> drop the DENSE flag while still throwing a start-up error for COMPACT STORAGE
> tables that don’t meet the criteria.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
