Hi Haeyoon,

I like the overall design. I have minor questions about some details.

In (5. a. iv.), what do you mean by freeing the MemoryChunks to be
garbage-collected? My understanding was that a MemoryChunk is outside of
the JVM heap, and thus is never accessed by the JVM garbage collector.
Also, why does the garbage collection happen "when the executor shuts down"?

I like the interfaces in (6. b.), and am curious to know whether they can
be also used in performance-critical places other than
DirectByteBufferOutputstream/SerializedMemoryStore. For example, can we use
the interfaces to sort data off-heap as in Apache Crail and Apache Spark
[1.2]?

[1] Sorting on a 100Gbit/s Cluster using Spark/Crail
https://crail.incubator.apache.org/blog/2017/01/sorting.html
[2] Project Tungsten: Bringing Apache Spark Closer to Bare Metal
https://databricks.com/blog/2015/04/28/project-tungsten-bringing-spark-closer-to-bare-metal.html

Thanks,
John


On Thu, May 23, 2019 at 11:33 AM Gyewon Lee <strayyy...@gmail.com> wrote:

> Hi, Haeyoon.
>
> Thanks a lot for your working. The initial design seems reasonable.
>
> Folks, could you give some feedbacks on the design document, so that
> Haeyoon can quickly move onto implementation? She will welcome any
> feedbacks, as she told us :)
>
> Best,
> Gyewon
>
> 2019년 5월 22일 (수) 오후 7:45, Haeyoon Cho <chohy0...@gmail.com>님이 작성:
>
> > Hi all,
> >
> > Just as I mentioned in the previous email, we are about to implement
> > components for off-heap memory management. Before that, we wrote an
> initial
> > design document and reported a JIRA issue for this(NEMO-388
> > <https://issues.apache.org/jira/browse/NEMO-388>). We welcome any
> > feedbacks
> > regarding the design.
> >
> >
> > Best,
> > Haeyoon
> >
>

Reply via email to