On Wed, Sep 23, 2020 at 3:57 AM Shakeel Butt <shake...@google.com> wrote: > > On Tue, Sep 22, 2020 at 5:37 AM Chunxin Zang <zangchun...@bytedance.com> > wrote: > > > > On Tue, Sep 22, 2020 at 6:42 PM Chris Down <ch...@chrisdown.name> wrote: > > > > > > Chunxin Zang writes: > > > >On Tue, Sep 22, 2020 at 5:51 PM Chris Down <ch...@chrisdown.name> wrote: > > > >> > > > >> Chunxin Zang writes: > > > >> >My usecase is that there are two types of services in one server. They > > > >> >have difference > > > >> >priorities. Type_A has the highest priority, we need to ensure it's > > > >> >schedule latency、I/O > > > >> >latency、memory enough. Type_B has the lowest priority, we expect it > > > >> >will not affect > > > >> >Type_A when executed. > > > >> >So Type_A could use memory without any limit. Type_B could use memory > > > >> >only when the > > > >> >memory is absolutely sufficient. But we cannot estimate how much > > > >> >memory Type_B should > > > >> >use. Because everything is dynamic. So we can't set Type_B's > > > >> >memory.high. > > > >> > > > > >> >So we want to release the memory of Type_B when global memory is > > > >> >insufficient in order > > > >> >to ensure the quality of service of Type_A . In the past, we used the > > > >> >'force_empty' interface > > > >> >of cgroup v1. > > > >> > > > >> This sounds like a perfect use case for memory.low on Type_A, and it's > > > >> pretty > > > >> much exactly what we invented it for. What's the problem with that? > > > > > > > >But we cannot estimate how much memory Type_A uses at least. > > > > > > memory.low allows ballparking, you don't have to know exactly how much it > > > uses. > > > Any amount of protection biases reclaim away from that cgroup. > > > > > > >For example: > > > >total memory: 100G > > > >At the beginning, Type_A was in an idle state, and it only used 10G of > > > >memory. > > > >The load is very low. We want to run Type_B to avoid wasting machine > > > >resources. > > > >When Type_B runs for a while, it used 80G of memory. > > > >At this time Type_A is busy, it needs more memory. > > > > > > Ok, so set memory.low for Type_A close to your maximum expected value. > > > > Please forgive me for not being able to understand why setting > > memory.low for Type_A can solve the problem. > > In my scene, Type_A is the most important, so I will set 100G to memory.low. > > But 'memory.low' only takes effect passively when the kernel is > > reclaiming memory. It means that reclaim Type_B's memory only when > > Type_A in alloc memory slow path. This will affect Type_A's > > performance. > > We want to reclaim Type_B's memory in advance when A is expected to be busy. > > > > How will you know when to reclaim from B? Are you polling /proc/meminfo? >
Monitor global memory usage through the daemon. If the memory is used 80% or 90%, it will reclaim B's memory. > From what I understand, you want to proactively reclaim from B, so > that A does not go into global reclaim and in the worst case kill B, > right? Yes, it is. > > BTW you can use memory.high to reclaim from B by setting it lower than > memory.current of B and reset it to 'max' once the reclaim is done. > Since 'B' is not high priority (I am assuming not a latency sensitive > workload), B hitting temporary memory.high should not be an issue. > Also I am assuming you don't much care about the amount of memory to > be reclaimed from B, so I think memory.high can fulfil your use-case. > However if in future you decide to proactively reclaim from all the > jobs based on their priority i.e. more aggressive reclaim from B and a > little bit reclaim from A then memory.high is not a good interface. > > Shakeel Thanks for these suggestions, I will give it a try. Best wishes Chunxin
