[ https://issues.apache.org/jira/browse/HDFS-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14005285#comment-14005285 ]
Arpit Agarwal commented on HDFS-5851: ------------------------------------- Hi [~eric14], I apologize for the long delayed response. bq. The case of a local short circuit read having access to the open file is interesting... does this pin the memory until the possibly misbehaved client process closes the socket / FD? Yes this is correct. This should be the existing behavior with short-circuit reads. bq. Single replicas? Why would one want to triple replicate discardable memory? One should at least have the option to only keep a single local copy in HDFS. We will have a single replica to begin with. There are use cases for triple replicas to share memory across sessions/applications. Multiple replicas would have to be optional as you suggest. bq. If we can not prevent random access writes to DDM (we could presumably limit this in client API), then I don't think we can checksum or replicate until a file is closed. My gut is delaying such until close is the right call... Direct writes to DN memory are appealing for performance reasons but there are some open questions including the ones you raised. Hence phase 1 will avoid short-circuit writes. I will let Sanjay respond to the remaining questions since he is more familiar with those aspects. > Support memory as a storage medium > ---------------------------------- > > Key: HDFS-5851 > URL: https://issues.apache.org/jira/browse/HDFS-5851 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode > Affects Versions: 3.0.0 > Reporter: Arpit Agarwal > Assignee: Arpit Agarwal > Attachments: > SupportingMemoryStorageinHDFSPersistentandDiscardableMemory.pdf, > SupportingMemoryStorageinHDFSPersistentandDiscardableMemory.pdf > > > Memory can be used as a storage medium for smaller/transient files for fast > write throughput. > More information/design will be added later. -- This message was sent by Atlassian JIRA (v6.2#6252)