[
https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773040#comment-16773040
]
Sammi Chen commented on HDFS-13762:
-----------------------------------
Thanks [~PhiloHe] for continue working on this. Here are some feebacks for
008.patch.
1. NativeIO.java, suggest to define the different PMDK support code and it's
meaning using a enum, so it will be easy to map the code to description.
2. NativeIO.c, it would be nice to refactor the error message, to avoid the
potential array overflow if pmem_errormsg() returns too long content. There are
several piece of similar code
{code:java}
char msg[1000];
snprintf(msg, sizeof(msg), "Fail to unmap region. address: %x, length: %x,
error msg: %s", address, length, pmem_errormsg());
{code}
3. Support persistent memeory cache on Windows can be a follow-up task since
Windows is not very common used in user environment. For build patch on
Windows, if it's a goal of this JIRA, please update the "Building on Windows"
in BUILDING.txt for any specific steps user need to take. If it's not a goal of
this JIRA, make sure build on Windows will have clear message for this case.
4. Cmakelists.txt "#Require ISA-L" is for ISA-L, better keep it.
5. make PmemVolumeManager pmemManager final
6.
This piece of code has the chance to delete user files accidentally. Better to
provide user a choice to decide auto delete or not.
{code:java}
// Remove all files under the volume. Files may been left after a
// unexpected data node restart.
FileUtils.cleanDirectory(locFile);
{code}
7. make count in PmemVolumeManager final
8. fillBuffer in MemoryMappedBlock and PmemMappedBlock are the same. refactor
to keep only one piece of code
> Support non-volatile storage class memory(SCM) in HDFS cache directives
> -----------------------------------------------------------------------
>
> Key: HDFS-13762
> URL: https://issues.apache.org/jira/browse/HDFS-13762
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: caching, datanode
> Reporter: Sammi Chen
> Assignee: Feilong He
> Priority: Major
> Attachments: HDFS-13762.000.patch, HDFS-13762.001.patch,
> HDFS-13762.002.patch, HDFS-13762.003.patch, HDFS-13762.004.patch,
> HDFS-13762.005.patch, HDFS-13762.006.patch, HDFS-13762.007.patch,
> HDFS-13762.008.patch, SCMCacheDesign-2018-11-08.pdf, SCMCacheTestPlan.pdf
>
>
> No-volatile storage class memory is a type of memory that can keep the data
> content after power failure or between the power cycle. Non-volatile storage
> class memory device usually has near access speed as memory DIMM while has
> lower cost than memory. So today It is usually used as a supplement to
> memory to hold long tern persistent data, such as data in cache.
> Currently in HDFS, we have OS page cache backed read only cache and RAMDISK
> based lazy write cache. Non-volatile memory suits for both these functions.
> This Jira aims to enable storage class memory first in read cache. Although
> storage class memory has non-volatile characteristics, to keep the same
> behavior as current read only cache, we don't use its persistent
> characteristics currently.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
