I would love to see Ekaterina's work from CASSANDRA-15234 to standardize configuration be resumed in the next releases.
Thought it would be worth mentioning since we had quite a productive discussion before putting it on hold to focus on 4.0, so it would be great to have that conversation resumed. Em seg., 26 de abr. de 2021 às 14:16, Benedict Elliott Smith < bened...@apache.org> escreveu: > I think my earlier response vanished into the moderator queue. Just a few > comments: > > 1) The Paxos latency (and correctness) improvements I think should land in > 4.0.x, as we have introduced a fairly significant regression and this work > mostly resolves outstanding issues with LWTs today. > 2) If we aim to deliver multi-partition LWTs in 4.x/5.0, we may likely > want to pair this with work to further reduce latency beyond the above > work, as contention will become a more significant problem. Should I be > involved in delivering multi-partition LWTs I will also be aiming to > deliver even lower latencies for the release they land in. > 3) To support all of the above work, I also aim to deliver a Simulator > facility for deterministically executing cluster workloads under > adversarial scheduling (i.e. that intercepts all message and thread events > and evaluates them sequentially, in pseudorandom order), alongside > linearizability verification built upon this. This work will include (or > have as a prerequisite) significant clean-ups to internal functionality > like executors, use of futures and other concurrency primitives, and > mocking out of time and the filesystem. > > > On 23/04/2021, 14:46, "Benjamin Lerer" <b.le...@gmail.com> wrote: > > Hi everybody, > > Thanks for all the responses. I went through the emails and aggregated > the > proposals to give us an idea on where we stand at this point. > > I only included the improvements in the list and left on the side the > bug > fixes. > Regarding bug fixes, I wonder if we should not have discussions every > month > to discuss what are the important issues that should be fixed in > priority. > I feel that we sometimes tend to forget old issues even if they are > more > important than some new ones. > > Do not hesitate to tell me if I missed something or misinterpreted some > proposal. > > *Query side improvements:* > > * Storage Attached Index or SAI. The CEP can be found at > > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-7%3A+Storage+Attached+Index > * Add support for OR predicates in the CQL where clause > * Allow to aggregate by time intervals (CASSANDRA-11871) and allow > UDFs > in GROUP BY clause > * Ability to read the TTL and WRITE TIME of an element in a > collection > (CASSANDRA-8877) > * Multi-Partition LWTs > * Materialized views hardening: Addressing the different Materialized > Views issues (see CASSANDRA-15921 and [1] for some of the work > involved) > > *Security improvements:* > > * SSTables encryption (CASSANDRA-9633) > * Add support for Dynamic Data Masking (CEP pending) > * Allow the creation of roles that have the ability to assign > arbitrary > privileges, or scoped privileges without also granting those roles > access > to database objects. > * Filter rows from system and system_schema based on users > permissions > (CASSANDRA-15871) > > *Performance improvements:* > > * Trie-based index format (CEP pending) > * Trie-based memtables (CEP pending) > * Paxos improvements: Paxos / LWT implementation that would enable > the > database to serve serial writes with two round-trips and serial reads > with > one round-trip in the uncontended case > > *Safety/Usability improvements:* > > * Guardrails. The CEP can be found at > > https://cwiki.apache.org/confluence/display/CASSANDRA/%28DRAFT%29+-+CEP-3%3A+Guardrails > * Add ability to track state in repair (CASSANDRA-15399) > * Repair coordinator improvements (CASSANDRA-15399) > * Make incremental backup configurable per keyspace and table > (CASSANDRA-15402) > * Add ability to blacklist a CQL partition so all requests are > ignored > (CASSANDRA-12106) > * Add default and required keyspace replication options > (CASSANDRA-14557) > * Transactional Cluster Metadata: Use of transactions to propagate > cluster metadata > * Downgrade-ability: Ability to downgrade to downgrade in the event > that > a serious issue has been identified > > *Pluggability improvements:* > > * Pluggable schema manager (CEP pending) > * Pluggable filesystem (CEP pending) > * Pluggable authenticator for CQLSH (CASSANDRA-16456). A CEP draft > can be > found at > > https://docs.google.com/document/d/1_G-OZCAEmDyuQuAN2wQUYUtZBEJpMkHWnkYELLhqvKc/edit > * Memtable API (CEP pending). The goal being to allow improvements > such > as CASSANDRA-13981 to be easily plugged into Cassandra > > *Memtable pluggable implementation:* > > * Enable Cassandra for Persistent Memory (CASSANDRA-13981) > > *Other tools:* > > * CQL compatibility test suite (CEP pending) > > Le jeu. 22 avr. 2021 à 16:11, Benjamin Lerer <b.le...@gmail.com> a > écrit : > > > Finally, I think it's important we work to maintain trunk in a > shippable > >> state. > > > > > > I am +100 on this. Bringing Cassandra to such a state was a huge > effort > > and keeping it that way will help us to ensure the quality of the > > releases. > > > > Le jeu. 15 avr. 2021 à 17:30, Scott Andreas <sc...@paradoxica.net> a > > écrit : > > > >> Thanks for starting this discussion, Benjamin! > >> > >> I share others’ enthusiasm on this thread for improvements to > secondary > >> indexes, trie-based partition indexes, guardrails, and encryption > at rest. > >> > >> Here are some other post-4.0 areas for investment that have been on > my > >> mind: > >> > >> – Transactional Cluster Metadata > >> Migrating from optimistic modification and propagation of cluster > >> metadata via gossip to a transactional implementation opens a lot of > >> possibilities. Token movements and instance replacements get safer > and > >> faster. Schema changes can be made atomic, enabling users to > execute DDL > >> rapidly without waiting for convergence. Operations like expansions > and > >> shrinks become easier to automate with less care and feeding. > >> > >> – Paxos improvements > >> During discussion on C-12126, Benedict expressed interest in > post-4.0 > >> improvements that can be made to Cassandra’s Paxos / LWT > implementation > >> that would enable the database to serve serial writes with two > round-trips > >> and serial reads with one round-trip in the uncontended case. For > many > >> cross-WAN serial use cases, this may halve the latency of CAS > queries. > >> > >> – Multi-Partition LWTs > >> LWT is a great primitive, but modeling applications with the > constraint > >> of single-key CAS can be a game of Twister. Extending the paxos > >> improvements discussed above to enable multi-partition CAS would > enable > >> users of Apache Cassandra to perform serial operations across > partition > >> boundaries. > >> > >> – Downgrade-ability > >> I also see “downgradeability” as important to future new release > >> adoption. Taking file format changes as an example, it’s currently > not > >> possible to downgrade in the event that a serious issue has been > identified > >> – unless you’re able to host-replace yourself out after upgrading > one > >> replica, or revert to a pre-upgrade snapshot and accept data loss. > It would > >> be excellent if it were possible for v.next to continue writing the > >> previous SSTable/commitlog/hint/etc. format until a switch is > flipped to > >> opt into new file formats. Apache HDFS takes a similar approach, > enabling > >> downgrade until NameNode metadata is finalized [1]. This would be an > >> excellent capability to have in Apache Cassandra, and dramatically > lower > >> the stakes for new release adoption. > >> > >> On pluggability / disaggregation: > >> I agree that these are important themes. We’ll want to bring a lot > of > >> care and attention to this work. Disaggregation can open a lot of > >> possibilities - with the drawback of future changes being > restricted to the > >> defined interface and an inability to optimize across interface > boundaries. > >> We can probably hit a sweet spot, though. > >> > >> Toolchains to validate implementations of pluggable components will > >> become very important. It would be bad for the project’s users if > bundled > >> implementations were of uneven quality or supported subsets of > >> functionality. Converging on a common validation toolchain for > pluggable > >> subsystems can help us ensure that quality while minimizing the > effort > >> required to test new implementations. > >> > >> Finally, I think it's important we work to maintain trunk in a > shippable > >> state. This might look like major changes and new features hiding > behind > >> feature flags that enable users to selectively enable them as > development > >> and validation proceeds, with new code executed regardless of the > flag held > >> to a higher standard. > >> > >> Cheers, > >> > >> – Scott > >> > >> [1] > >> > https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html > >> > >> > >> ________________________________________ > >> From: guo Maxwell <cclive1...@gmail.com> > >> Sent: Wednesday, April 14, 2021 10:25 PM > >> To: dev@cassandra.apache.org > >> Subject: Re: [DISCUSSION] Next release roadmap > >> > >> +1 > >> > >> Brandon Williams <dri...@gmail.com> 于2021年4月15日周四 上午4:48写道： > >> > >> > Agreed. Everyone just please keep in mind this thread is for > roadmap > >> > contributions you plan to make, not contributions you would like > to > >> > see. > >> > > >> > On Wed, Apr 14, 2021 at 3:45 PM Nate McCall <zznat...@gmail.com> > wrote: > >> > > > >> > > Agree with Stefan 100% on this. We need to move towards > pluggability. > >> Our > >> > > users are asking for it, it makes sense architecturally, and > people > >> are > >> > > doing it anyway. > >> > > > >> > > > >> > > ... > >> > > > for me definitely > >> https://issues.apache.org/jira/browse/CASSANDRA-9633 > >> > > > > >> > > > I am surprised nobody mentioned this in the previous answers, > there > >> is > >> > > > ~50 people waiting for it to happen and multiple people > working on > >> it > >> > > > seriously and wanting that feature to be there for so so long. > >> > > > ... > >> > > >> > > --------------------------------------------------------------------- > >> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > >> > For additional commands, e-mail: dev-h...@cassandra.apache.org > >> > > >> > > >> > >> -- > >> you are the apple of my eye ! > >> > >> > --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > >> For additional commands, e-mail: dev-h...@cassandra.apache.org > >> > >> > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
