Thanks again for the great presentation given by Feilong. Feilong also graciously agreed to share the presentation slides. To not spam your inbox, I uploaded the slides in my personal Google Drive folder here: https://drive.google.com/file/d/1OxtviDL6N7jwhQDAe2r1oM-MbioeSgpb/view?usp=sharing
Here's the notes I took for your reference: 11/20/2019 Feilong from Intel will discuss the Storage Class Memory support (HDFS-13762 <https://issues.apache.org/jira/browse/HDFS-13762>) Attendee: weichiu, feilong, haiyang, Hui, rakesh, Yisheng - HDFS-13762 extends HDFS centralized cache management to use persistent memory as the cache. It offers larger memory at a lower price. - The cache (NativePmemMappableBlockLoader) bypasses kernel space, directly to the pmem (via PMDK library). There’s another implementation (PmemMappableBlockLoader) that uses Java NIO/Common File API to access pmem. The former is about 20% better in performance. - A configuration dfs.datanode.cache.pmem.dirs determines whether to use DRAM or Pmem to cache. - Can support more than one pmem devices. - Will ship with Hadoop 3.3.0, 3.2.2 and 3.1.4. - Benchmark: DFSIO. If input data is fully cached in Pmem, Pmem is 6x than DRAM. Otherwise if a small input data set is cached fully in DRAM, Pmem is about 50% throughput - HDFS-14740: Pmem read cache recovery pending community review. Upon DN restart, scan pmem and find existing cached block files. Hierarchical cache storage. - Alibaba evaluated HBase BucketCache with Pmem. There’s a benchmark report somewhere that Rakesh On Wed, Nov 20, 2019 at 9:13 PM Wei-Chiu Chuang <weic...@cloudera.com> wrote: > Friendly reminder: this event is happening in an hour! > > On Mon, Nov 18, 2019 at 11:38 PM Wei-Chiu Chuang <weic...@apache.org> > wrote: > >> I'm really bad at converting time across different time zones. >> >> Here's the correction: >> Pacific Time: Wed Nov 20 10pm >> Beijing Time: Thu Nov 21 2pm >> India Time: Thu Nov 21 11:30am >> >> https://www.timeanddate.com/worldclock/converter.html?iso=20191121T060000&p1=tz_pt&p2=33&p3=tz_ist >> [image: Screen Shot 2019-11-18 at 11.37.02 PM.png] >> >> On Mon, Nov 18, 2019 at 8:23 PM Wei-Chiu Chuang <weic...@cloudera.com> >> wrote: >> >>> Hi! >>> >>> I am happy to have Feilong from Intel to join us and talk about the >>> Storage Class Memory support for HDFS ( HDFS-13762 >>> <https://issues.apache.org/jira/browse/HDFS-13762>). This is a new >>> feature that will land in the next Hadoop minor release (Hadoop 3.3.0) >>> >>> Date/Time: Wednesday Nov 20 10pm Pacific Time / Thursday Nov 21 1pm >>> Beijing Time / Thursday Nov 21 9:30am Bangalore Time >>> >>> Join Zoom Meeting >>> >>> https://cloudera.zoom.us/j/880548968 >>> >>> Past sessions: >>> >>> https://docs.google.com/document/d/1jXM5Ujvf-zhcyw_5kiQVx6g-HeKe-YGnFS_1-qFXomI/edit >>> >>>
