OpenVZ Forum - RDF feed
type: channel
items: Re: [PATCH v2 02/13] memcg: Kernel memory accounting infrastructure.

Re: [PATCH v2 02/13] memcg: Kernel memory accounting infrastructure.
type: item
description: On 03/10/2012 12:39 AM, Suleiman Souhlal wrote: > Enabled with CONFIG_CGROUP_MEM_RES_CTLR_KMEM. > > Adds the following files: > - memory.kmem.independent_kmem_limit > - memory.kmem.usage_in_bytes > - memory.kmem.limit_in_bytes > > Signed-off-by: Suleiman Souhlal<suleiman@google.com> > --- > mm/memcontrol.c | 136 ++++++++++++++++++++++++++++++++++++++++++++++++++++++- > 1 files changed, 135 insertions(+), 1 deletions(-) > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > index 37ad2cb..e6fd558 100644 > --- a/mm/memcontrol.c > +++ b/mm/memcontrol.c > @@ -220,6 +220,10 @@ enum memcg_flags { > */ > MEMCG_MEMSW_IS_MINIMUM, /* Set when res.limit == memsw.limit */ > MEMCG_OOM_KILL_DISABLE, /* OOM-Killer disable */ > + MEMCG_INDEPENDENT_KMEM_LIMIT, /* > + * kernel memory is not counted in > + * memory.usage_in_bytes > + */ > }; > > /* > @@ -244,6 +248,10 @@ struct mem_cgroup { > */ > struct res_counter memsw; > /* > + * the counter to account for kernel memory usage. > + */ > + struct res_counter kmem; > + /* > * Per cgroup active and inactive list, similar to the > * per zone LRU lists. > */ > @@ -355,6 +363,7 @@ enum charge_type { > #define _MEM (0) > #define _MEMSWAP (1) > #define _OOM_TYPE (2) > +#define _KMEM (3) > #define MEMFILE_PRIVATE(x, val) (((x)<< 16) | (val)) > #define MEMFILE_TYPE(val) (((val)>> 16)& 0xffff) > #define MEMFILE_ATTR(val) ((val)& 0xffff) > @@ -371,6 +380,8 @@ enum charge_type { > > static void mem_cgroup_get(struct mem_cgroup *memcg); > static void mem_cgroup_put(struct mem_cgroup *memcg); > +static void memcg_kmem_init(struct mem_cgroup *memcg, > + struct mem_cgroup *parent); > > static inline bool > mem_cgroup_test_flag(const struct mem_cgroup *memcg, enum memcg_flags flag) > @@ -1435,6 +1446,10 @@ done: > res_counter_read_u64(&memcg->memsw, RES_USAGE)>> 10, > res_counter_read_u64(&memcg->memsw, RES_LIMIT)>> 10, > res_counter_read_u64(&memcg->memsw, RES_FAILCNT)); > + printk(KERN_INFO "kmem: usage %llukB, limit %llukB, failcnt %llu\n", > + res…
creator: Glauber Costa
date: 2012-03-11T08:12:04-00:00

Re: [PATCH v2 02/13] memcg: Kernel memory accounting infrastructure.
type: item
description: On Sun, 11 Mar 2012 12:12:04 +0400 Glauber Costa <glommer@parallels.com> wrote: > On 03/10/2012 12:39 AM, Suleiman Souhlal wrote: > > Enabled with CONFIG_CGROUP_MEM_RES_CTLR_KMEM. > > > > Adds the following files: > > - memory.kmem.independent_kmem_limit > > - memory.kmem.usage_in_bytes > > - memory.kmem.limit_in_bytes > > > > Signed-off-by: Suleiman Souhlal<suleiman@google.com> > > --- > > mm/memcontrol.c | 136 ++++++++++++++++++++++++++++++++++++++++++++++++++++++- > > 1 files changed, 135 insertions(+), 1 deletions(-) > > > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > > index 37ad2cb..e6fd558 100644 > > --- a/mm/memcontrol.c > > +++ b/mm/memcontrol.c > > @@ -220,6 +220,10 @@ enum memcg_flags { > > */ > > MEMCG_MEMSW_IS_MINIMUM, /* Set when res.limit == memsw.limit */ > > MEMCG_OOM_KILL_DISABLE, /* OOM-Killer disable */ > > + MEMCG_INDEPENDENT_KMEM_LIMIT, /* > > + * kernel memory is not counted in > > + * memory.usage_in_bytes > > + */ > > }; After looking codes, I think we need to think whether independent_kmem_limit is good or not.... How about adding MEMCG_KMEM_ACCOUNT flag instead of this and use only memcg->res/memcg->memsw rather than adding a new counter, memcg->kmem ? if MEMCG_KMEM_ACCOUNT is set -> slab is accoutned to mem->res/memsw. if MEMCG_KMEM_ACCOUNT is not set -> slab is never accounted. (I think On/Off switch is required..) Thanks, -Kame
creator: KAMEZAWA Hiroyuki
date: 2012-03-13T06:24:46-00:00

Re: [PATCH v2 02/13] memcg: Kernel memory accounting infrastructure.
type: item
description: > After looking codes, I think we need to think > whether independent_kmem_limit is good or not.... > > How about adding MEMCG_KMEM_ACCOUNT flag instead of this and use only > memcg->res/memcg->memsw rather than adding a new counter, memcg->kmem ? > > if MEMCG_KMEM_ACCOUNT is set -> slab is accoutned to mem->res/memsw. > if MEMCG_KMEM_ACCOUNT is not set -> slab is never accounted. > > (I think On/Off switch is required..) > > Thanks, > -Kame > This has been discussed before, I can probably find it in the archives if you want to go back and see it. But in a nutshell: 1) Supposing independent knob disappear (I will explain in item 2 why I don't want it to), I don't thing a flag makes sense either. *If* we are planning to enable/disable this, it might make more sense to put some work on it, and allow particular slabs to be enabled/disabled by writing to memory.kmem.slabinfo (-* would disable all, +* enable all, +kmalloc* enable all kmalloc, etc). Alternatively, what we could do instead, is something similar to what ended up being done for tcp, by request of the network people: if you never touch the limit file, don't bother with it at all, and simply does not account. With Suleiman's lazy allocation infrastructure, that should actually be trivial. And then again, a flag is not necessary, because writing to the limit file does the job, and also convey the meaning well enough. 2) For the kernel itself, we are mostly concerned that a malicious container may pin into memory big amounts of kernel memory which is, ultimately, unreclaimable. In particular, with overcommit allowed scenarios, you can fill the whole physical memory (or at least a significant part) with those objects, well beyond your softlimit allowance, making the creation of further containers impossible. With user memory, you can reclaim the cgroup back to its place. With kernel memory, you can't. In the particular example of 32-bit boxes, you can easily fill up a large part of the available 1gb kernel memory…
creator: Glauber Costa
date: 2012-03-13T10:37:30-00:00

Re: [PATCH v2 02/13] memcg: Kernel memory accounting infrastructure.
type: item
description: Glauber Costa <glommer@parallels.com> writes: > 2) For the kernel itself, we are mostly concerned that a malicious container may > pin into memory big amounts of kernel memory which is, ultimately, > unreclaimable. In particular, with overcommit allowed scenarios, you can fill > the whole physical memory (or at least a significant part) with those objects, > well beyond your softlimit allowance, making the creation of further containers > impossible. > With user memory, you can reclaim the cgroup back to its place. With kernel > memory, you can't. In overcommit situations the page allocator starts failing even though memcg page can charge pages. When page allocations fail the oom killer plays a role. Page allocations can fail even without malicious usage of kernel memory (e.g. lots of mlock or anon without swap can fill a machine). I assume that the kernel memory pinned the malicious containers will be freed or at least become reclaimable once the processes in malicious containers are killed (oom or otherwise). We have been making use of the oom killer to save a system from irreconcilable overcommit situations.
creator: Greg Thelen
date: 2012-03-13T17:00:58-00:00

Re: [PATCH v2 02/13] memcg: Kernel memory accounting infrastructure.
type: item
description: On 03/13/2012 09:00 PM, Greg Thelen wrote: > Glauber Costa<glommer@parallels.com> writes: >> 2) For the kernel itself, we are mostly concerned that a malicious container may >> pin into memory big amounts of kernel memory which is, ultimately, >> unreclaimable. In particular, with overcommit allowed scenarios, you can fill >> the whole physical memory (or at least a significant part) with those objects, >> well beyond your softlimit allowance, making the creation of further containers >> impossible. >> With user memory, you can reclaim the cgroup back to its place. With kernel >> memory, you can't. > > In overcommit situations the page allocator starts failing even though > memcg page can charge pages. If you overcommit mem+swap, yes. If you overcommit mem, no: reclaim happens first. And we don't have that option with pinned kernel memory. Of course you *can* run your system without swap, but the whole thing exists exactly because there is a large enough # of ppl who wants to be able to overcommit their physical memory, without failing allocations.
creator: Glauber Costa
date: 2012-03-13T17:31:40-00:00

Re: [PATCH v2 02/13] memcg: Kernel memory accounting infrastructure.
type: item
description: On Tue, 13 Mar 2012 14:37:30 +0400 Glauber Costa <glommer@parallels.com> wrote: > > After looking codes, I think we need to think > > whether independent_kmem_limit is good or not.... > > > > How about adding MEMCG_KMEM_ACCOUNT flag instead of this and use only > > memcg->res/memcg->memsw rather than adding a new counter, memcg->kmem ? > > > > if MEMCG_KMEM_ACCOUNT is set -> slab is accoutned to mem->res/memsw. > > if MEMCG_KMEM_ACCOUNT is not set -> slab is never accounted. > > > > (I think On/Off switch is required..) > > > > Thanks, > > -Kame > > > > This has been discussed before, I can probably find it in the archives > if you want to go back and see it. > Yes. IIUC, we agreed to have independet kmem limit. I just want to think it again because there are too many proposals and it seems I'm in confusion. As far as I see, there are ongoing works as - kmem limit by 2 guys. - hugetlb limit - per lru locking (by 2 guys) - page cgroup diet (by me, but stops now.) - drity-ratio and writeback - Tejun's proposal to remove pre_destroy() - moving shared resource I'm thinking what is a simple plan and implementation. Most of series consists of 10+ patches... Thank you for your help of clarification. > But in a nutshell: > > 1) Supposing independent knob disappear (I will explain in item 2 why I > don't want it to), I don't thing a flag makes sense either. *If* we are > planning to enable/disable this, it might make more sense to put some > work on it, and allow particular slabs to be enabled/disabled by writing > to memory.kmem.slabinfo (-* would disable all, +* enable all, +kmalloc* > enable all kmalloc, etc). > seems interesting. > Alternatively, what we could do instead, is something similar to what > ended up being done for tcp, by request of the network people: if you > never touch the limit file, don't bother with it at all, and simply does > not account. With Suleiman's lazy allocation infrastructure, that should > actually be trivial. And then again, a flag is not …
creator: KAMEZAWA Hiroyuki
date: 2012-03-14T00:15:26-00:00

Re: [PATCH v2 02/13] memcg: Kernel memory accounting infrastructure.
type: item
description: >> This has been discussed before, I can probably find it in the archives >> if you want to go back and see it. >> > > Yes. IIUC, we agreed to have independet kmem limit. I just want to think it > again because there are too many proposals and it seems I'm in confusion. > Sure thing. The discussion turned out good, so I'm glad you asked =) > >> But in a nutshell: >> >> 1) Supposing independent knob disappear (I will explain in item 2 why I >> don't want it to), I don't thing a flag makes sense either. *If* we are >> planning to enable/disable this, it might make more sense to put some >> work on it, and allow particular slabs to be enabled/disabled by writing >> to memory.kmem.slabinfo (-* would disable all, +* enable all, +kmalloc* >> enable all kmalloc, etc). >> > seems interesting. I'll try to cook a PoC. >> All that said, while reading your message, thinking a bit, the following >> crossed my mind: >> >> - We can account the slabs to memcg->res normally, and just store the >> information that this is kernel memory into a percpu counter, as >> I proposed recently. > > Ok, then user can see the amount of kernel memory. > > >> - The knob goes away, and becomes implicit: if you ever write anything >> to memory.kmem.limit_in_bytes, we transfer that memory to a separate >> kmem res_counter, and proceed from there. We can keep accounting to >> memcg->res anyway, just that kernel memory will now have a separate >> limit. > > Okay, then, > > kmem_limit< memory.limit< memsw.limit > > ...seems reasonable to me. > This means, user can specify 'ratio' of kmem in memory.limit. Yes, I believe so. It is a big improvement over the current interface we have today, IMHO. > > More consideration will be interesting. > > - We can show the amount of reclaimable kmem by some means ? That's hard to do. The users of the cache have this information, the underlying slab/slub/slut code do not. We need to rely on the cache owner to provide this, and provide correctly. So the chances we'll ha…
creator: Glauber Costa
date: 2012-03-14T12:29:25-00:00

Re: [PATCH v2 02/13] memcg: Kernel memory accounting infrastructure.
type: item
description: (2012/03/14 21:29), Glauber Costa wrote: >> - What happens when a new cgroup created ? > > mem_cgroup_create() is called =) > Heh, jokes apart, I don't really follow here. What exactly do you mean? > There shouldn't be anything extremely out of the ordinary. > Sorry, too short words. Assume a cgroup with cgroup.memory.limit_in_bytes=1G cgroup.memory.kmem.limit_in_bytes=400M When a child cgroup is created, what should be the default values. 'unlimited' as current implementation ? Hmm..maybe yes. Thanks, -Kame
creator: KAMEZAWA Hiroyuki
date: 2012-03-15T00:48:27-00:00

Re: [PATCH v2 02/13] memcg: Kernel memory accounting infrastructure.
type: item
description: On 03/15/2012 04:48 AM, KAMEZAWA Hiroyuki wrote: >>> - What happens when a new cgroup created ? >> > >> > mem_cgroup_create() is called =) >> > Heh, jokes apart, I don't really follow here. What exactly do you mean? >> > There shouldn't be anything extremely out of the ordinary. >> > > > Sorry, too short words. > > Assume a cgroup with > cgroup.memory.limit_in_bytes=1G > cgroup.memory.kmem.limit_in_bytes=400M > > When a child cgroup is created, what should be the default values. > 'unlimited' as current implementation ? > Hmm..maybe yes. I think so, yes. I see no reason to come up with any default values in memcg. Yes, your allocations can fail due to your parent limits. But since I never heard of any machine with 9223372036854775807 bytes of memory, that is true even for the root memcg =)
creator: Glauber Costa
date: 2012-03-15T11:07:15-00:00

Re: [PATCH v2 02/13] memcg: Kernel memory accounting infrastructure.
type: item
description: On Thu, 2012-03-15 at 15:07 +0400, Glauber Costa wrote: > But since I never heard of any machine with > 9223372036854775807 bytes of memory, that is true even for the root memcg What, you don't have more than 8 exabyte of memory in your laptop !? Surely you're due for an upgrade then.
creator: Peter Zijlstra
date: 2012-03-15T11:13:34-00:00

Re: [PATCH v2 02/13] memcg: Kernel memory accounting infrastructure.
type: item
description: On 03/15/2012 03:13 PM, Peter Zijlstra wrote: > On Thu, 2012-03-15 at 15:07 +0400, Glauber Costa wrote: >> But since I never heard of any machine with >> 9223372036854775807 bytes of memory, that is true even for the root memcg > > What, you don't have more than 8 exabyte of memory in your laptop !? > Surely you're due for an upgrade then. Yeah, I requested it already, but I was told it could take a while
creator: Glauber Costa
date: 2012-03-15T11:21:23-00:00