[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17718713#comment-17718713 ] Jonathan Ellis commented on CASSANDRA-13981: I'm not aware of any mainstream persistent memory now that Optane is dead. > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Normal > Fix For: 5.x > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > in-mem-cassandra-2.1.patch, readme.txt, readme2.1.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17717634#comment-17717634 ] Brandon Williams commented on CASSANDRA-13981: -- Should this ticket be closed? > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Normal > Fix For: 5.x > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > in-mem-cassandra-2.1.patch, readme.txt, readme2.1.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17717633#comment-17717633 ] Jyothilekshmi commented on CASSANDRA-13981: --- The final version of the PMem code can be found at [https://github.com/apache/cassandra/pull/2274]. It is only intended for design reference and should not be merged. > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Normal > Fix For: 5.x > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > in-mem-cassandra-2.1.patch, readme.txt, readme2.1.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17436124#comment-17436124 ] shylaja kokoori commented on CASSANDRA-13981: - The code is under active development. The implementation has been modified to make use of the pluggable memtable interface introduced by CEP11. We are in the process of making the code feature complete while we wait for resolution of following JIRAs. CASSANDRA-17034 CEP-11: Memtable API implementation CASSANDRA-6936 Make all byte representations of types comparable by their unsigned byte representation only > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Normal > Fix For: 4.x > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > in-mem-cassandra-2.1.patch, readme.txt, readme2.1.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17253981#comment-17253981 ] Von Gosling commented on CASSANDRA-13981: - It seems many tasks still not resolved in CASSANDRA-13474. Do we have any release plan for version 4? > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Normal > Fix For: 4.x > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > in-mem-cassandra-2.1.patch, readme.txt, readme2.1.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17253777#comment-17253777 ] shylaja kokoori commented on CASSANDRA-13981: - [~vongosling] We are waiting on version 4.0 release and an interface such as the one described in CASSANDRA-13474, to plugin the persistent memory code. > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Normal > Fix For: 4.x > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > in-mem-cassandra-2.1.patch, readme.txt, readme2.1.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17253235#comment-17253235 ] Von Gosling commented on CASSANDRA-13981: - Any update for this? > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Normal > Fix For: 4.x > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > in-mem-cassandra-2.1.patch, readme.txt, readme2.1.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16742600#comment-16742600 ] shylaja kokoori commented on CASSANDRA-13981: - We have a new implementation of the [persistent memory storage engine|https://github.com/shyla226/cassandra/tree/13981_llpl_engine] based on the design and sketch that [~jasobrown] proposed. This implementation uses an improved version of [LLPL|https://github.com/pmem/llpl]. As suggested in Jason's design, this engine uses the Adaptive Radix Tree as the core data structure. Also, as suggested, it uses the default Cassandra serialization / deserialization code. The engine implements creates, reads, updates, deletes, and table schema modification. We have tested these features on a single node using the insanity schema with > 2 billion partitions. We would like to get feedback on the storage engine and guidance from the Cassandra community on next steps. In particular, we have implemented placeholder read path code because the pluggable storage engine read path API is not yet resolved (https://issues.apache.org/jira/browse/CASSANDRA-14117). > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.x > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > in-mem-cassandra-2.1.patch, readme.txt, readme2.1.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1650#comment-1650 ] shylaja kokoori commented on CASSANDRA-13981: - Uploading patch version 2.1. A branch with this patch applied is at https://github.com/shyla226/cassandra/tree/13981. This patch ties up work that was in progress to improve performance and persistent memory footprint of the PCJ-based design. We are now exploring the design & sketch that Jason Brown has proposed. [^in-mem-cassandra-2.1.patch] [^readme2.1.txt] > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.x > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > in-mem-cassandra-2.1.patch, readme.txt, readme2.1.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482490#comment-16482490 ] Tony Ruiz commented on CASSANDRA-13981: --- I am out of the office returning returning Monday May 18. For urgent matters please contact my manager: eric.kaczma...@intel.com Thanks, Tony > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.0 > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > readme.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[
https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482488#comment-16482488
]
Jason Brown commented on CASSANDRA-13981:
-
Thanks, [~pree] and [~shylaja.koko...@intel.com], for the patches. I've been
reading them, understanding the scope of the technology, and see the direction
you are going. However, I'd like to propose a slightly different direction.
Stepping back, the pcj library is divided into two parts: the higher-level pcj
components (as used in the version of this patch as previously posted), and the
lower-level API, called LLPL in the library. LLPL is much smaller than the pcj
parts, and offers a direct and simple way to just write bytes into a backing
array from the persistent memory. In my option this will be far more natural
for the cassandra community and developers, and provides a more direct access
to the storage bytes. We already have lots of serialization code, and we
understand that quite well; thus I'd like to keep leveraging that lower-level
thinking. We will need to write custom, non-generic data structures (like we
already have for our LSM-based engine), but I only see this as complete win. We
need to optimize, in every way we reasonably can, our data structures as we are
a database, after all. LLPL has some rough edges wrt code optimization and we
will want to modify the transaction model a bit, but I suspect the pcj authors
will work with us toward that end.
With this as background, I've started sketching out a direction I think we
should pursue. This sketch primarily shows the direction for thinking about
serialization and memory allocation using LLPL. DISCLAIMER: this code doesn't
compile, is not syntactically correct, and is wholly incomplete. It should be
thought of a loose blueprint (sketch!) for discussion.
The sketch compromises of the following concepts:
- thread per sub-range (to reduce lock contention in the data structures).
This is kinda inspired by the thread-per-core notion, but on a smaller scale.
({{TreeManager}} in this patch is a rudimentary dispatch class.)
- how partitions should be stored - allocate a {{MemoryRegion}} from the LLPL
allocator, wrap it with a {{DataOutputPlus}}, and write as we normally would.
- rough implementations of the data structures for the primary index and
storing rows. A longer treatment of this topic will be in the deisgn doc (see
below), but using a tree for the primary index (for partition look up) and then
a map for the cql rows is the basic idea. I mostly want to show the ideas
around serialization so I didn't actually implement the index nor the map -
except for the leaf/entry nodes which show how the serailization/data layout
fits into the data structure.
- explicitly pass the transaction around on writes (instead of looking for it
in a {{ThreadLocal}}, as the pcj transactions does).
||13981-sketch-1||
|[branch|https://github.com/jasobrown/cassandra/tree/13981-sketch-1]|
I am proposing this sketch as a starting for for discussion, along with a
forthcoming design doc to help us work out more high-level details of how
cassandra as a main memory database should look. I'm working on design doc now.
It will explore how we can have a pluggable storage engine implementation that
allows cassandra to run as a main memory database using persistent memory,
while supporting the existing behaviors of cassandra in that kind of system.
> Enable Cassandra for Persistent Memory
> ---
>
> Key: CASSANDRA-13981
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13981
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
>Reporter: Preetika Tyagi
>Assignee: Preetika Tyagi
>Priority: Major
> Fix For: 4.0
>
> Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch,
> readme.txt, readme2_0.txt
>
>
> Currently, Cassandra relies on disks for data storage and hence it needs data
> serialization, compaction, bloom filters and partition summary/index for
> speedy access of the data. However, with persistent memory, data can be
> stored directly in the form of Java objects and collections, which can
> greatly simplify the retrieval mechanism of the data. What we are proposing
> is to make use of faster and scalable B+ tree-based data collections built
> for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a
> complete in-memory version of Cassandra, while still keeping the data
> persistent.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16479709#comment-16479709 ] shylaja kokoori commented on CASSANDRA-13981: - Uploading patch version 2.0. This patch bypasses the use of memtable in the read/write path. Please refer to attached readme2_0.txt for more details. > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.0 > > Attachments: in-mem-cassandra-1.0.patch, in-mem-cassandra-2.0.patch, > readme.txt, readme2_0.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16463929#comment-16463929 ] Michael Burman commented on CASSANDRA-13981: Just my 2 cents, but the approach seems slightly odd if one wants to take advantage of the persistent memory. This stuff should override the whole CommitLog path also. So instead of writing first to WAL and then to the memtable, this should skip both the memtable as well as commitlog in my opinion and basically rewrite the whole storage path. And same applies to reads, compactions, etc as it should never create SSTables. Same way, when starting Cassandra the recovery process should be changed also. I would actually redesign also the query parsing, query execution and other parts as well as these do not fit the model of persistent memory too well either. I have no idea how this would affect the streaming processing as suddenly the tables are not necessarily anymore immutable (as LSM isn't designed for persistent memory). I'd say this is quite difficult approach to integrate to Cassandra if it's anything more than "half-baked exercise". Just replacing the data model isn't enough as the whole approach of having a single type of memory is so fundamentally different to what Cassandra was designed for. > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.0 > > Attachments: in-mem-cassandra-1.0.patch, readme.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16461647#comment-16461647 ] Steve Dohrmann commented on CASSANDRA-13981: We intend to change the license for the PCJ library (referenced in the description above) to A BSD-style license. We'll post a comment here when that is done. > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.0 > > Attachments: in-mem-cassandra-1.0.patch, readme.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16421955#comment-16421955 ] Preetika Tyagi commented on CASSANDRA-13981: [~jjirsa] Yeah, not sure. Thanks! > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.0 > > Attachments: in-mem-cassandra-1.0.patch, readme.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16421954#comment-16421954 ] Jeff Jirsa commented on CASSANDRA-13981: Not sure who set it Ready to commit, but changing it back to in progress. > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.0 > > Attachments: in-mem-cassandra-1.0.patch, readme.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349334#comment-16349334 ] Jason Brown commented on CASSANDRA-13981: - [~pree] good. I'll try to start looking at this next week. > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.0 > > Attachments: in-mem-cassandra-1.0.patch, readme.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348028#comment-16348028 ] Preetika Tyagi commented on CASSANDRA-13981: [~jasobrown] Thank you for clarifying! I have pushed the branch here: https://github.com/preetyagi/cassandra/tree/CASSANDRA-13981 > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.0 > > Attachments: in-mem-cassandra-1.0.patch, readme.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16345604#comment-16345604 ] Jason Brown commented on CASSANDRA-13981: - [~pree] you do not have push privileges to the apache repos; that's for Apache committers. If you can just create a branch on your own github account, and create a PR (perhaps against the sha you based the work on), that should be enough to get started here. > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.0 > > Attachments: in-mem-cassandra-1.0.patch, readme.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16345399#comment-16345399 ] Preetika Tyagi commented on CASSANDRA-13981: [~jasobrown] Shall I create a branch and push it to the existing Cassandra GitHub? I believe it will show up in the list of branches on GitHub: [https://github.com/apache/cassandra]. Just wanted to confirm. > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.0 > > Attachments: in-mem-cassandra-1.0.patch, readme.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13981) Enable Cassandra for Persistent Memory
[ https://issues.apache.org/jira/browse/CASSANDRA-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16344411#comment-16344411 ] Jason Brown commented on CASSANDRA-13981: - [~pree] I understand this is a PoC, but is it possible to post a branch to github for easier review and commenting? There's no way anybody can comment intelligibly on a 387K patch. > Enable Cassandra for Persistent Memory > --- > > Key: CASSANDRA-13981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13981 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Preetika Tyagi >Assignee: Preetika Tyagi >Priority: Major > Fix For: 4.0 > > Attachments: in-mem-cassandra-1.0.patch, readme.txt > > > Currently, Cassandra relies on disks for data storage and hence it needs data > serialization, compaction, bloom filters and partition summary/index for > speedy access of the data. However, with persistent memory, data can be > stored directly in the form of Java objects and collections, which can > greatly simplify the retrieval mechanism of the data. What we are proposing > is to make use of faster and scalable B+ tree-based data collections built > for persistent memory in Java (PCJ: https://github.com/pmem/pcj) and enable a > complete in-memory version of Cassandra, while still keeping the data > persistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
