Chendi.Xue created SPARK-27412:
----------------------------------
Summary: Add a new shuffle manager to use Persistent Memory as
shuffle and spilling storage
Key: SPARK-27412
URL: https://issues.apache.org/jira/browse/SPARK-27412
Project: Spark
Issue Type: New Feature
Components: Shuffle, Spark Core
Affects Versions: 3.0.0
Reporter: Chendi.Xue
Add a new shuffle manager called "PmemShuffleManager", by using which, we can
use Persistent Memory Device as storage for shuffle and external sorter
spilling.
In this implementation, we leveraged Persistent Memory Development Kit(PMDK) to
support transaction write with high performance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
