[
https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16760449#comment-16760449
]
Rakesh R commented on HDFS-13762:
---------------------------------
Thanks [~PhiloHe] for the continuous efforts!
{quote}Yes, it's a limitation for all the volumes together.
{quote}
OK, it would be concern when there are different sizes volumes. Agreed to take
up in a follow up jira task.
{quote}We maintains the map from block file to cache-file on pmem storage, so
no need to search for it among the volumes
{quote}
Great!
{quote}We don't have a evict logic here or throw a VolumeFullException, it just
works like the memory-cache for compatible now.
{quote}
Please add this case to the system test plan. It would be good to test it and
document/know the behavior.
{quote}(iii), we will update the patch accordingly
{quote}
Please change below one also.
{code:java}
+ Pmem.unmapBlock(region.getAddress(), region.getLength());
+ boolean deled = false;
{code}
Adding few more comments, please take care:
# MappableBlock interface looks good, please add javadocs for the below
functions. Also, please reflect specific class MappableBlock, PmemMappedBlock,
MemoryMappedBlock responsibility in javadoc instead of using the same
"Represents an HDFS block that is mmapped by the DataNode."
{code:java}
long getLength();
void afterCache();
{code}
# Change the comment to PMDK instead of ISA-L!
{code:java}
+ // Load Intel ISA-L
+ #ifdef UNIX
{code}
# Could you please brief the difference between {{pmemDrain}} and
{{pmemSync}}. Also, I would appreciate if you could add javadoc so that the
functionality will be visible to the readers.
{code:java}
private static native boolean isPmemCheck(long address, long length);
private static native PmemMappedRegion pmemCreateMapFile(String path,
long length);
private static native boolean pmemUnMap(long address, long length);
private static native void pmemCopy(byte[] src, long dest, boolean isPmem,
long length);
private static native void pmemDrain();
private static native void pmemSync(long address, long length);
{code}
> Support non-volatile storage class memory(SCM) in HDFS cache directives
> -----------------------------------------------------------------------
>
> Key: HDFS-13762
> URL: https://issues.apache.org/jira/browse/HDFS-13762
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: caching, datanode
> Reporter: Sammi Chen
> Assignee: Feilong He
> Priority: Major
> Attachments: HDFS-13762.000.patch, HDFS-13762.001.patch,
> HDFS-13762.002.patch, HDFS-13762.003.patch, HDFS-13762.004.patch,
> HDFS-13762.005.patch, HDFS-13762.006.patch, HDFS-13762.007.patch,
> HDFS-13762.008.patch, SCMCacheDesign-2018-11-08.pdf, SCMCacheTestPlan.pdf
>
>
> No-volatile storage class memory is a type of memory that can keep the data
> content after power failure or between the power cycle. Non-volatile storage
> class memory device usually has near access speed as memory DIMM while has
> lower cost than memory. So today It is usually used as a supplement to
> memory to hold long tern persistent data, such as data in cache.
> Currently in HDFS, we have OS page cache backed read only cache and RAMDISK
> based lazy write cache. Non-volatile memory suits for both these functions.
> This Jira aims to enable storage class memory first in read cache. Although
> storage class memory has non-volatile characteristics, to keep the same
> behavior as current read only cache, we don't use its persistent
> characteristics currently.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
