[
https://issues.apache.org/jira/browse/HIVE-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15236103#comment-15236103
]
Gopal V commented on HIVE-13029:
--------------------------------
bq. The patch basically looks good to me.
I haven't tested this properly.
bq. should this be addressed, or followed up on?
The follow up for fallocate is ideally done in tje JNI layer in hadoop-common's
libhadoop.so (HADOOP-7834).
Right now, the truncate mode allows for overcommit - so, running out of space
on storage kills the daemon with a fatal SIGBUS.
bq. The SSD cache would require the 2-tiered cache, wouldn't it?
The NVDIMM case doesn't need a 2-tiered model, because the read latencies with
DAX are too small to need the extra complexity. In fact, a 2-tier model would
be slower because it would eat up bandwidth moving data between tiers - a
zero-copy paging approach directly by the OS is much better.
The SSD mode is just left there for systems which are restricted in RAM
(<64Gb), but still want to utilize a large cache.
> NVDIMM support for LLAP Cache
> -----------------------------
>
> Key: HIVE-13029
> URL: https://issues.apache.org/jira/browse/HIVE-13029
> Project: Hive
> Issue Type: New Feature
> Components: llap
> Affects Versions: 2.1.0
> Reporter: Gopal V
> Assignee: Gopal V
> Priority: Critical
> Attachments: HIVE-13029.1.patch
>
>
> LLAP cache has been designed so that the cache can be offloaded easily to a
> pmem API without restart coherence.
> The tricky part about NVDIMMs are restart coherence, while most of the cache
> gains can be obtained without keeping state across refreshes, since LLAP is
> not the system of record, HDFS is.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
