Thanks for starting the vote Xintong Also +1 for the proposed FLIP-49.
@Stephan regarding namings: network vs shuffle. My understanding so far was that the network memory is what we basically give to Shuffle implementations and default netty implementation uses it in particular mostly for networking. Are the network pools used for something else outside of the shuffling scope? best, Andrey On Tue, Sep 3, 2019 at 1:01 PM Stephan Ewen <se...@apache.org> wrote: > +1 to the proposal in general > > A few things seems to be a bit put of sync with the latest discussions > though. > > The section about JVM Parameters states that the > -XX:MaxDirectMemorySize value is set to Task Off-heap Memory, Shuffle > Memory and JVM Overhead. > The way I understand the last discussion conclusion is that it is only the > sum of shuffle memory and user-defined direct memory. > > I am someone neutral but unsure about is the separation between > "taskmanager.memory.framework.heap" and "taskmanager.memory.task.heap". > Could that be simply combined under "taskmanager.memory.javaheap"? > > It might be good to also expose these values somehow in the web UI so that > users see immediately what amount of memory TMs assume to use for what. > > I assume config key names and default values might be adjusted over time as > we get feedback. > - I would keep the network memory under the name > "taskmanager.memory.network". Because network memory is actually used for > more than shuffling. Also, the old config key seems good, so why change it? > > One thing to be aware of is that often, the Java Heap is understood as > "managed memory" as a whole, because it is managed by the GC not explicitly > by the user. > So we need to make sure that we don't confuse users by speaking of managed > heap and unmanaged heap. All heap is managed in Java. Some memory is > explicitly managed by Flink. > > Best, > Stephan > > > On Mon, Sep 2, 2019 at 3:08 PM Xintong Song <tonysong...@gmail.com> wrote: > > > Hi everyone, > > > > I'm here to re-start the voting process for FLIP-49 [1], with respect to > > consensus reached in this thread [2] regarding some new comments and > > concerns. > > > > This voting will be open for at least 72 hours. I'll try to close it Sep. > > 5, 14:00 UTC, unless there is an objection or not enough votes. > > > > Thank you~ > > > > Xintong Song > > > > > > [1] > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-49%3A+Unified+Memory+Configuration+for+TaskExecutors > > [2] > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-49-Unified-Memory-Configuration-for-TaskExecutors-td31436.html > > > > On Tue, Aug 27, 2019 at 9:29 PM Xintong Song <tonysong...@gmail.com> > > wrote: > > > > > Alright, then let's keep the discussion in the DISCUSS mailing thread, > > and > > > see whether we need to restart the vote. > > > > > > Thank you~ > > > > > > Xintong Song > > > > > > > > > > > > On Tue, Aug 27, 2019 at 8:12 PM Till Rohrmann <trohrm...@apache.org> > > > wrote: > > > > > >> I had a couple of comments concerning the implementation plan. I've > > posted > > >> them to the original discussion thread. Depending on the outcome of > this > > >> discussion we might need to restart the vote. > > >> > > >> Cheers, > > >> Till > > >> > > >> On Tue, Aug 27, 2019 at 11:14 AM Xintong Song <tonysong...@gmail.com> > > >> wrote: > > >> > > >> > Hi all, > > >> > > > >> > I would like to start the voting process for FLIP-49 [1], which is > > >> > discussed and reached consensus in this thread [2]. > > >> > > > >> > This voting will be open for at least 72 hours. I'll try to close it > > >> Aug. > > >> > 30 10:00 UTC, unless there is an objection or not enough votes. > > >> > > > >> > Thank you~ > > >> > > > >> > Xintong Song > > >> > > > >> > > > >> > [1] > > >> > > > >> > > > >> > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-49%3A+Unified+Memory+Configuration+for+TaskExecutors > > >> > [2] > > >> > > > >> > > > >> > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-49-Unified-Memory-Configuration-for-TaskExecutors-td31436.html > > >> > > > >> > > > > > >
