OpenVZ Forum - RDF feed
type: channel
items: [RFC 0/4] memcg: background reclaim (v1)
items: [RFC 1/4] memcg: drop pages at rmdir (v1)
items: [RFC 2/4] memcg: high-low watermark
items: [RFC 4/4] memcg: NUMA background reclaim
items: [RFC 3/4] memcg: background reclaim
items: Re: [RFC 2/4] memcg: high-low watermark
items: Re: [RFC 1/4] memcg: drop pages at rmdir (v1)
items: Re: [RFC 3/4] memcg: background reclaim
items: Re: [RFC 4/4] memcg: NUMA background reclaim

[RFC 0/4] memcg: background reclaim (v1)
type: item
description: This is my current set of add-on patches for memory resource controller. This works well but not well tested and not for usual people. i.e..request for comments at early stage before being more complicated. This set inculdes an implementation of background reclaim to memory resource controller. I expect this helps I/O under memory resource controller very much. (some good result with "dd") pathces are based on 2.6.26-rc2-mm1 + remove_refcnt patch set (in mm queue) So, I don't ask you "pleaset test" ;) plz tell me if you don't like the concept or you have better idea. [1/4] freeing all at force_empty. [2/4] high-low watermark to resource counter. [3/4] background reclaim for memcg. [4/4] background reclaim for memcg, NUMA extension. Consideration: One problem of background reclaim is that it uses CPU. I think it's necessary to make them more moderate. But what can I do against kthread rather than nice() ? Thanks, -Kame _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: KAMEZAWA Hiroyuki
date: 2008-05-27T05:01:16-00:00

[RFC 1/4] memcg: drop pages at rmdir (v1)
type: item
description: Now, when we remove memcg, we call force_empty(). This call drops all page_cgroup accounting in this mem_cgroup but doesn't drop pages. So, some page caches can be remaind as "not accounted" memory while they are alive. (because it's accounted only when add_to_page_cache()) If they are not used by other memcg, global LRU will drop them. This patch tries to drop pages at removing memcg. Other memcg will reload and re-account page caches. (but this will increase page-in after rmdir().) Consideration: should we recharge all pages to the parent at last ? But it's not precise logic. Changelog v1->v2 - renamed res_counter_empty(). Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> --- include/linux/res_counter.h | 11 +++++++++++ mm/memcontrol.c | 19 +++++++++++++++++++ 2 files changed, 30 insertions(+) Index: mm-2.6.26-rc2-mm1/mm/memcontrol.c =================================================================== --- mm-2.6.26-rc2-mm1.orig/mm/memcontrol.c +++ mm-2.6.26-rc2-mm1/mm/memcontrol.c @@ -791,6 +791,20 @@ int mem_cgroup_shrink_usage(struct mm_st return 0; } + +static void mem_cgroup_drop_all_pages(struct mem_cgroup *mem) +{ + int progress; + while (!res_counter_empty(&mem->res)) { + progress = try_to_free_mem_cgroup_pages(mem, + GFP_HIGHUSER_MOVABLE); + if (!progress) /* we did as much as possible */ + break; + cond_resched(); + } + return; +} + /* * This routine traverse page_cgroup in given list and drop them all. * *And* this routine doesn't reclaim page itself, just removes page_cgroup. @@ -848,7 +862,12 @@ static int mem_cgroup_force_empty(struct if (mem_cgroup_subsys.disabled) return 0; + if (atomic_read(&mem->css.cgroup->count) > 0) + goto out; + css_get(&mem->css); + /* drop pages as much as possible */ + mem_cgroup_drop_all_pages(mem); /* * page reclaim code (kswapd etc..) will move pages between * active_list <-> inactive_list while we don't take a lock. Index: mm-2.6.26-rc2-mm1/include/linux/res_counter.h =====================================…
creator: KAMEZAWA Hiroyuki
date: 2008-05-27T05:04:30-00:00

[RFC 2/4] memcg: high-low watermark
type: item
description: Add high/low watermarks to res_counter. *This patch itself has no behavior changes to memory resource controller. Changelog: very old one -> this one (v1) - watarmark_state is removed and all state check is done under lock. - changed res_counter_charge() interface. The only user is memory resource controller. Anyway, returning -ENOMEM here is a bit starnge. - Added watermark enable/disable flag for someone don't want watermarks. - Restarted against 2.6.25-mm1. - some subsystem which doesn't want high-low watermark can work withou it. Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> From: YAMAMOTO Takashi <yamamoto@valinux.co.jp> --- include/linux/res_counter.h | 41 ++++++++++++++++++++++++--- kernel/res_counter.c | 66 ++++++++++++++++++++++++++++++++++++++++---- mm/memcontrol.c | 2 - 3 files changed, 99 insertions(+), 10 deletions(-) Index: mm-2.6.26-rc2-mm1/include/linux/res_counter.h =================================================================== --- mm-2.6.26-rc2-mm1.orig/include/linux/res_counter.h +++ mm-2.6.26-rc2-mm1/include/linux/res_counter.h @@ -16,6 +16,16 @@ #include <linux/cgroup.h> /* + * status of resource coutner's usage. + */ +enum res_state { + RES_BELOW_LOW, /* usage < lwmark */ + RES_BELOW_HIGH, /* lwmark < usage < hwmark */ + RES_BELOW_LIMIT, /* hwmark < usage < limit. */ + RES_OVER_LIMIT, /* only used at chage. */ +}; + +/* * The core object. the cgroup that wishes to account for some * resource may include this counter into its structures and use * the helpers described beyond @@ -39,6 +49,12 @@ struct res_counter { */ unsigned long long failcnt; /* + * watermarks. needs to keep lwmark <= hwmark <= limit. + */ + unsigned long long hwmark; + unsigned long long lwmark; + int use_watermark; + /* * the lock to protect all of the above. * the routines below consider this to be IRQ-safe */ @@ -76,13 +92,18 @@ enum { RES_MAX_USAGE, RES_LIMIT, RES_FAILCNT, + RES_HWMARK, + RES_LWMARK, }; /* * helpers for accounting + * res_counter_…
creator: KAMEZAWA Hiroyuki
date: 2008-05-27T05:06:39-00:00

[RFC 4/4] memcg: NUMA background reclaim
type: item
description: One aspect of difference in reclaim logic between global lru and memcg is * global LRU triggers memory reclaim at memory shortage. * memcg LRU triggers memory reclaim at excess of usage. Then, global LRU _know_ which node we should start reclaim from. * start from a node at memory shortage or * start from a node where memory allocation is waiting WRT memcg, it's difficult to find where we should start because there is no memory shortage and LRU is splitted. (But per-zone-LRU is definitely necessary for scalability.) This patch tries to deteremine a node for starting recalim by checking ratio of inactive pages/active pages in a node. And trying to avoid starting from a node with relatively small usage. Better algorithm is welcome. Singed-off-by: KAMEZAWA Hiruyuki <kamezawa.hiroyu@jp.fujitsu.com> Index: mm-2.6.26-rc2-mm1/mm/memcontrol.c =================================================================== --- mm-2.6.26-rc2-mm1.orig/mm/memcontrol.c +++ mm-2.6.26-rc2-mm1/mm/memcontrol.c @@ -578,7 +578,7 @@ retry: if (state == RES_OVER_LIMIT) { if (!(gfp_mask & __GFP_WAIT)) goto out; - if (try_to_free_mem_cgroup_pages(mem, gfp_mask)) + if (try_to_free_mem_cgroup_pages(mem, -1, gfp_mask)) goto retry; /* * try_to_free_mem_cgroup_pages() might not give us a @@ -801,7 +801,7 @@ int mem_cgroup_shrink_usage(struct mm_st rcu_read_unlock(); do { - progress = try_to_free_mem_cgroup_pages(mem, gfp_mask); + progress = try_to_free_mem_cgroup_pages(mem, -1, gfp_mask); } while (!progress && --retry); if (!retry) @@ -814,7 +814,7 @@ static void mem_cgroup_drop_all_pages(st { int progress; while (!res_counter_empty(&mem->res)) { - progress = try_to_free_mem_cgroup_pages(mem, + progress = try_to_free_mem_cgroup_pages(mem, -1, GFP_HIGHUSER_MOVABLE); if (!progress) /* we did as much as possible */ break; @@ -912,6 +912,62 @@ out: /* * background reclaim daemon. */ + +#ifdef CONFIG_NUMA +/* + * Because memory controller's memory reclaim doesn't come from memory shortage, + * we cannot know wh…
creator: KAMEZAWA Hiroyuki
date: 2008-05-27T05:06:52-00:00

[RFC 3/4] memcg: background reclaim
type: item
description: Do background reclaim based on high-low watermarks. This feature helps smooth work of processes under memcg by reclaiming memory in the kernel thread. # of limitation failure at mem_cgroup_charge() will dramatically reduced. But this also means a CPU is continuously used for reclaiming memory. This one is very simple. Anyway, we need to update this when we add new complexity to memcg. Major logic: - add high-low watermark support to memory resource controller. - create a kernel thread for cgroup when hwmark is changed. (once) - stop a kernel thread at rmdir(). - start background reclaim if res_counter is over high-watermark. - stop background reclaim if res_coutner is below low-watermark. - for reclaiiming, just calls try_to_free_mem_cgroup_pages(). - kthread for reclaim 's priority is nice(0). default is (-5). (weaker is better ?) - kthread for reclaim calls yield() on each loop. TODO: - add an interface to start/stop daemon ? - wise numa support - too small "low watermark" targe just consumes CPU. Should we warn ? and what is the best value for hwmark/lwmark in general....? Chagelog: old one -> this (v1) - start a thread at write of hwmark. Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> --- mm/memcontrol.c | 147 +++++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 130 insertions(+), 17 deletions(-) Index: mm-2.6.26-rc2-mm1/mm/memcontrol.c =================================================================== --- mm-2.6.26-rc2-mm1.orig/mm/memcontrol.c +++ mm-2.6.26-rc2-mm1/mm/memcontrol.c @@ -32,7 +32,8 @@ #include <linux/fs.h> #include <linux/seq_file.h> #include <linux/vmalloc.h> - +#include <linux/freezer.h> +#include <linux/kthread.h> #include <asm/uaccess.h> struct cgroup_subsys mem_cgroup_subsys __read_mostly; @@ -119,10 +120,6 @@ struct mem_cgroup_lru_info { * statistics based on the statistics developed by Rik Van Riel for clock-pro, * to help the administrator determine what knobs to tune. * - * TODO: Add a water mark fo…
creator: KAMEZAWA Hiroyuki
date: 2008-05-27T05:08:46-00:00

Re: [RFC 2/4] memcg: high-low watermark
type: item
description: > +enum res_state res_counter_state(struct res_counter *counter) > +{ > + unsigned long flags; > + enum res_state ret = RES_BELOW_LIMIT; > + > + spin_lock_irqsave(&counter->lock, flags); > + if (counter->use_watermark) { > + if (counter->usage <= counter->lwmark) > + ret = RES_BELOW_LOW; > + else if (counter->usage <= counter->hwmark) > + ret = RES_BELOW_HIGH; > + } > + spin_unlock_irqrestore(&counter->lock, flags); > + return ret; > +} can't it be RES_OVER_LIMIT? eg. when you lower the limit. YAMAMOTO Takashi _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: yamamoto
date: 2008-05-27T05:30:27-00:00

Re: [RFC 2/4] memcg: high-low watermark
type: item
description: On Tue, 27 May 2008 14:30:27 +0900 (JST) yamamoto@valinux.co.jp (YAMAMOTO Takashi) wrote: > > +enum res_state res_counter_state(struct res_counter *counter) > > +{ > > + unsigned long flags; > > + enum res_state ret = RES_BELOW_LIMIT; > > + > > + spin_lock_irqsave(&counter->lock, flags); > > + if (counter->use_watermark) { > > + if (counter->usage <= counter->lwmark) > > + ret = RES_BELOW_LOW; > > + else if (counter->usage <= counter->hwmark) > > + ret = RES_BELOW_HIGH; > > + } > > + spin_unlock_irqrestore(&counter->lock, flags); > > + return ret; > > +} > > can't it be RES_OVER_LIMIT? > eg. when you lower the limit. Ah, ok. I missed it. I'll add checks. Thank you ! -Kame _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: KAMEZAWA Hiroyuki
date: 2008-05-27T07:14:30-00:00

Re: [RFC 2/4] memcg: high-low watermark
type: item
description: KAMEZAWA Hiroyuki wrote: > Add high/low watermarks to res_counter. > *This patch itself has no behavior changes to memory resource controller. > > Changelog: very old one -> this one (v1) > - watarmark_state is removed and all state check is done under lock. > - changed res_counter_charge() interface. The only user is memory > resource controller. Anyway, returning -ENOMEM here is a bit starnge. > - Added watermark enable/disable flag for someone don't want watermarks. > - Restarted against 2.6.25-mm1. > - some subsystem which doesn't want high-low watermark can work withou it. > > Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> > From: YAMAMOTO Takashi <yamamoto@valinux.co.jp> > > --- > include/linux/res_counter.h | 41 ++++++++++++++++++++++++--- > kernel/res_counter.c | 66 ++++++++++++++++++++++++++++++++++++++++---- > mm/memcontrol.c | 2 - > 3 files changed, 99 insertions(+), 10 deletions(-) > > Index: mm-2.6.26-rc2-mm1/include/linux/res_counter.h > =================================================================== > --- mm-2.6.26-rc2-mm1.orig/include/linux/res_counter.h > +++ mm-2.6.26-rc2-mm1/include/linux/res_counter.h > @@ -16,6 +16,16 @@ > #include <linux/cgroup.h> > > /* > + * status of resource coutner's usage. > + */ > +enum res_state { > + RES_BELOW_LOW, /* usage < lwmark */ It seems it's 'usage <= lwmark' > + RES_BELOW_HIGH, /* lwmark < usage < hwmark */ and 'lwmark < usage <= hwmark' > + RES_BELOW_LIMIT, /* hwmark < usage < limit. */ and 'hwmark < usage <= limit' _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Li Zefan
date: 2008-05-27T07:51:40-00:00

Re: [RFC 2/4] memcg: high-low watermark
type: item
description: On Tue, 27 May 2008 15:51:40 +0800 Li Zefan <lizf@cn.fujitsu.com> wrote: > KAMEZAWA Hiroyuki wrote: > > Add high/low watermarks to res_counter. > > *This patch itself has no behavior changes to memory resource controller. > > > > Changelog: very old one -> this one (v1) > > - watarmark_state is removed and all state check is done under lock. > > - changed res_counter_charge() interface. The only user is memory > > resource controller. Anyway, returning -ENOMEM here is a bit starnge. > > - Added watermark enable/disable flag for someone don't want watermarks. > > - Restarted against 2.6.25-mm1. > > - some subsystem which doesn't want high-low watermark can work withou it. > > > > Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> > > From: YAMAMOTO Takashi <yamamoto@valinux.co.jp> > > > > --- > > include/linux/res_counter.h | 41 ++++++++++++++++++++++++--- > > kernel/res_counter.c | 66 ++++++++++++++++++++++++++++++++++++++++---- > > mm/memcontrol.c | 2 - > > 3 files changed, 99 insertions(+), 10 deletions(-) > > > > Index: mm-2.6.26-rc2-mm1/include/linux/res_counter.h > > =================================================================== > > --- mm-2.6.26-rc2-mm1.orig/include/linux/res_counter.h > > +++ mm-2.6.26-rc2-mm1/include/linux/res_counter.h > > @@ -16,6 +16,16 @@ > > #include <linux/cgroup.h> > > > > /* > > + * status of resource coutner's usage. > > + */ > > +enum res_state { > > + RES_BELOW_LOW, /* usage < lwmark */ > > It seems it's 'usage <= lwmark' > > > + RES_BELOW_HIGH, /* lwmark < usage < hwmark */ > > and 'lwmark < usage <= hwmark' > > > + RES_BELOW_LIMIT, /* hwmark < usage < limit. */ > > and 'hwmark < usage <= limit' > Thank you. I'll fix. -Kame _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: KAMEZAWA Hiroyuki
date: 2008-05-27T09:42:15-00:00

Re: [RFC 1/4] memcg: drop pages at rmdir (v1)
type: item
description: KAMEZAWA Hiroyuki wrote: > Now, when we remove memcg, we call force_empty(). > This call drops all page_cgroup accounting in this mem_cgroup but doesn't > drop pages. So, some page caches can be remaind as "not accounted" memory > while they are alive. (because it's accounted only when add_to_page_cache()) > If they are not used by other memcg, global LRU will drop them. > > This patch tries to drop pages at removing memcg. Other memcg will > reload and re-account page caches. (but this will increase page-in > after rmdir().) > The approach seems fair, but I am not sure about the overhead of flushing out cached pages. Might well be worth it. > Consideration: should we recharge all pages to the parent at last ? > But it's not precise logic. > We should look into this - I should send out the multi-hierarchy patches soon. We should discuss this after that. > Changelog v1->v2 > - renamed res_counter_empty(). > > Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> > > --- > include/linux/res_counter.h | 11 +++++++++++ > mm/memcontrol.c | 19 +++++++++++++++++++ > 2 files changed, 30 insertions(+) > > Index: mm-2.6.26-rc2-mm1/mm/memcontrol.c > =================================================================== > --- mm-2.6.26-rc2-mm1.orig/mm/memcontrol.c > +++ mm-2.6.26-rc2-mm1/mm/memcontrol.c > @@ -791,6 +791,20 @@ int mem_cgroup_shrink_usage(struct mm_st > return 0; > } > > + > +static void mem_cgroup_drop_all_pages(struct mem_cgroup *mem) > +{ > + int progress; > + while (!res_counter_empty(&mem->res)) { > + progress = try_to_free_mem_cgroup_pages(mem, > + GFP_HIGHUSER_MOVABLE); > + if (!progress) /* we did as much as possible */ > + break; > + cond_resched(); > + } > + return; > +} > + > /* > * This routine traverse page_cgroup in given list and drop them all. > * *And* this routine doesn't reclaim page itself, just removes page_cgroup. > @@ -848,7 +862,12 @@ static int mem_cgroup_force_empty(struct > if (mem_cgroup_subsys.disabled) > return 0; > > + if (…
creator: Balbir Singh
date: 2008-05-27T16:11:26-00:00

Re: [RFC 2/4] memcg: high-low watermark
type: item
description: KAMEZAWA Hiroyuki wrote: > Add high/low watermarks to res_counter. > *This patch itself has no behavior changes to memory resource controller. > > Changelog: very old one -> this one (v1) > - watarmark_state is removed and all state check is done under lock. > - changed res_counter_charge() interface. The only user is memory > resource controller. Anyway, returning -ENOMEM here is a bit starnge. > - Added watermark enable/disable flag for someone don't want watermarks. > - Restarted against 2.6.25-mm1. > - some subsystem which doesn't want high-low watermark can work withou it. > > Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> > From: YAMAMOTO Takashi <yamamoto@valinux.co.jp> > The From: line should be the first line IIRC. > --- > include/linux/res_counter.h | 41 ++++++++++++++++++++++++--- > kernel/res_counter.c | 66 ++++++++++++++++++++++++++++++++++++++++---- > mm/memcontrol.c | 2 - > 3 files changed, 99 insertions(+), 10 deletions(-) > > Index: mm-2.6.26-rc2-mm1/include/linux/res_counter.h > =================================================================== > --- mm-2.6.26-rc2-mm1.orig/include/linux/res_counter.h > +++ mm-2.6.26-rc2-mm1/include/linux/res_counter.h > @@ -16,6 +16,16 @@ > #include <linux/cgroup.h> > > /* > + * status of resource coutner's usage. > + */ > +enum res_state { > + RES_BELOW_LOW, /* usage < lwmark */ > + RES_BELOW_HIGH, /* lwmark < usage < hwmark */ > + RES_BELOW_LIMIT, /* hwmark < usage < limit. */ > + RES_OVER_LIMIT, /* only used at chage. */ > +}; > + > +/* > * The core object. the cgroup that wishes to account for some > * resource may include this counter into its structures and use > * the helpers described beyond > @@ -39,6 +49,12 @@ struct res_counter { > */ > unsigned long long failcnt; > /* > + * watermarks. needs to keep lwmark <= hwmark <= limit. > + */ > + unsigned long long hwmark; > + unsigned long long lwmark; > + int use_watermark; Is it routine to comment this way? I prefer not to have spaces in th…
creator: Balbir Singh
date: 2008-05-27T16:26:17-00:00

Re: [RFC 3/4] memcg: background reclaim
type: item
description: KAMEZAWA Hiroyuki wrote: > Do background reclaim based on high-low watermarks. > > This feature helps smooth work of processes under memcg by reclaiming memory > in the kernel thread. # of limitation failure at mem_cgroup_charge() will > dramatically reduced. But this also means a CPU is continuously used for > reclaiming memory. > > This one is very simple. Anyway, we need to update this when we add new > complexity to memcg. > > Major logic: > - add high-low watermark support to memory resource controller. > - create a kernel thread for cgroup when hwmark is changed. (once) > - stop a kernel thread at rmdir(). > - start background reclaim if res_counter is over high-watermark. > - stop background reclaim if res_coutner is below low-watermark. > - for reclaiiming, just calls try_to_free_mem_cgroup_pages(). > - kthread for reclaim 's priority is nice(0). default is (-5). > (weaker is better ?) > - kthread for reclaim calls yield() on each loop. > > TODO: > - add an interface to start/stop daemon ? > - wise numa support > - too small "low watermark" targe just consumes CPU. Should we warn ? > and what is the best value for hwmark/lwmark in general....? > > Chagelog: old one -> this (v1) > - start a thread at write of hwmark. > > > Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> > > --- > mm/memcontrol.c | 147 +++++++++++++++++++++++++++++++++++++++++++++++++------- > 1 file changed, 130 insertions(+), 17 deletions(-) > > Index: mm-2.6.26-rc2-mm1/mm/memcontrol.c > =================================================================== > --- mm-2.6.26-rc2-mm1.orig/mm/memcontrol.c > +++ mm-2.6.26-rc2-mm1/mm/memcontrol.c > @@ -32,7 +32,8 @@ > #include <linux/fs.h> > #include <linux/seq_file.h> > #include <linux/vmalloc.h> > - > +#include <linux/freezer.h> > +#include <linux/kthread.h> > #include <asm/uaccess.h> > > struct cgroup_subsys mem_cgroup_subsys __read_mostly; > @@ -119,10 +120,6 @@ struct mem_cgroup_lru_info { > * statistics based on the statistics…
creator: Balbir Singh
date: 2008-05-27T17:08:09-00:00

Re: [RFC 4/4] memcg: NUMA background reclaim
type: item
description: KAMEZAWA Hiroyuki wrote: > One aspect of difference in reclaim logic between global lru and memcg is > * global LRU triggers memory reclaim at memory shortage. > * memcg LRU triggers memory reclaim at excess of usage. > > Then, global LRU _know_ which node we should start reclaim from. > * start from a node at memory shortage or > * start from a node where memory allocation is waiting > > WRT memcg, it's difficult to find where we should start because > there is no memory shortage and LRU is splitted. > (But per-zone-LRU is definitely necessary for scalability.) > > This patch tries to deteremine a node for starting recalim by checking > ratio of inactive pages/active pages in a node. And trying to avoid starting > from a node with relatively small usage. > Better algorithm is welcome. > > Singed-off-by: KAMEZAWA Hiruyuki <kamezawa.hiroyu@jp.fujitsu.com> > > Index: mm-2.6.26-rc2-mm1/mm/memcontrol.c > =================================================================== > --- mm-2.6.26-rc2-mm1.orig/mm/memcontrol.c > +++ mm-2.6.26-rc2-mm1/mm/memcontrol.c > @@ -578,7 +578,7 @@ retry: > if (state == RES_OVER_LIMIT) { > if (!(gfp_mask & __GFP_WAIT)) > goto out; > - if (try_to_free_mem_cgroup_pages(mem, gfp_mask)) > + if (try_to_free_mem_cgroup_pages(mem, -1, gfp_mask)) > goto retry; > /* > * try_to_free_mem_cgroup_pages() might not give us a > @@ -801,7 +801,7 @@ int mem_cgroup_shrink_usage(struct mm_st > rcu_read_unlock(); > > do { > - progress = try_to_free_mem_cgroup_pages(mem, gfp_mask); > + progress = try_to_free_mem_cgroup_pages(mem, -1, gfp_mask); > } while (!progress && --retry); > > if (!retry) > @@ -814,7 +814,7 @@ static void mem_cgroup_drop_all_pages(st > { > int progress; > while (!res_counter_empty(&mem->res)) { > - progress = try_to_free_mem_cgroup_pages(mem, > + progress = try_to_free_mem_cgroup_pages(mem, -1, > GFP_HIGHUSER_MOVABLE); > if (!progress) /* we did as much as possible */ > break; > @@ -912,6 +912,62 @@ out: > /* > * background reclaim daemon. >…
creator: Balbir Singh
date: 2008-05-27T17:26:43-00:00

Re: [RFC 1/4] memcg: drop pages at rmdir (v1)
type: item
description: On Tue, 27 May 2008 21:41:26 +0530 Balbir Singh <balbir@linux.vnet.ibm.com> wrote: > KAMEZAWA Hiroyuki wrote: > > Now, when we remove memcg, we call force_empty(). > > This call drops all page_cgroup accounting in this mem_cgroup but doesn't > > drop pages. So, some page caches can be remaind as "not accounted" memory > > while they are alive. (because it's accounted only when add_to_page_cache()) > > If they are not used by other memcg, global LRU will drop them. > > > > This patch tries to drop pages at removing memcg. Other memcg will > > reload and re-account page caches. (but this will increase page-in > > after rmdir().) > > > > The approach seems fair, but I am not sure about the overhead of flushing out > cached pages. Might well be worth it. > > > Consideration: should we recharge all pages to the parent at last ? > > But it's not precise logic. > > > > We should look into this - I should send out the multi-hierarchy patches soon. Yes. I'll write my version (if I can). please pick it up if you like it. > We should discuss this after that. > ok. I'd like to forget this patch for a while. Thanks, -Kame _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: KAMEZAWA Hiroyuki
date: 2008-05-28T00:12:37-00:00

Re: [RFC 2/4] memcg: high-low watermark
type: item
description: On Tue, 27 May 2008 21:56:17 +0530 Balbir Singh <balbir@linux.vnet.ibm.com> wrote: > KAMEZAWA Hiroyuki wrote: > > Add high/low watermarks to res_counter. > > *This patch itself has no behavior changes to memory resource controller. > > > > Changelog: very old one -> this one (v1) > > - watarmark_state is removed and all state check is done under lock. > > - changed res_counter_charge() interface. The only user is memory > > resource controller. Anyway, returning -ENOMEM here is a bit starnge. > > - Added watermark enable/disable flag for someone don't want watermarks. > > - Restarted against 2.6.25-mm1. > > - some subsystem which doesn't want high-low watermark can work withou it. > > > > Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> > > From: YAMAMOTO Takashi <yamamoto@valinux.co.jp> > > > > The From: line should be the first line IIRC. > ok. > > > /* > > + * watermarks. needs to keep lwmark <= hwmark <= limit. > > + */ > > + unsigned long long hwmark; > > + unsigned long long lwmark; > > + int use_watermark; > > Is it routine to comment this way? I prefer not to have spaces in the type and > the member, makes it easier for my eyes. > Hmm. will fix. > > + * RES_BELOW_LIMIT -- usage is smaller than limt, success. > > ^^^^ typo > sure, will fix. > > + spin_unlock_irqrestore(&counter->lock, flags); > > + return ret; > > +} > > + > > When do we return RES_OVER_LIMIT? Are we missing that here? > It's Bug. Yamamoto-san pointed out and I'm now fixing. > > spin_lock_irqsave(&counter->lock, flags); > > + switch (member) { > > + case RES_LIMIT: > > + if (counter->use_watermark && counter->hwmark > tmp) > > + goto unlock_free; > > We need to document such API changes in the Documentation/controllers/memory.txt > file. > ok, I'll add patch for documentation. to memory.txt and res_counter.txt. > > + break; > > + case RES_HWMARK: > > + if (tmp < counter->lwmark || tmp > counter->limit) > > + goto unlock_free; > > + break; > > + case RES_LWMARK: > > + if (tmp …
creator: KAMEZAWA Hiroyuki
date: 2008-05-28T00:13:30-00:00

Re: [RFC 4/4] memcg: NUMA background reclaim
type: item
description: On Tue, 27 May 2008 22:56:43 +0530 Balbir Singh <balbir@linux.vnet.ibm.com> wrote: > KAMEZAWA Hiroyuki wrote: > > One aspect of difference in reclaim logic between global lru and memcg is > > * global LRU triggers memory reclaim at memory shortage. > > * memcg LRU triggers memory reclaim at excess of usage. > > > > Then, global LRU _know_ which node we should start reclaim from. > > * start from a node at memory shortage or > > * start from a node where memory allocation is waiting > > > > WRT memcg, it's difficult to find where we should start because > > there is no memory shortage and LRU is splitted. > > (But per-zone-LRU is definitely necessary for scalability.) > > > > This patch tries to deteremine a node for starting recalim by checking > > ratio of inactive pages/active pages in a node. And trying to avoid starting > > from a node with relatively small usage. > > Better algorithm is welcome. > > > > Singed-off-by: KAMEZAWA Hiruyuki <kamezawa.hiroyu@jp.fujitsu.com> > > > > Index: mm-2.6.26-rc2-mm1/mm/memcontrol.c > > =================================================================== > > --- mm-2.6.26-rc2-mm1.orig/mm/memcontrol.c > > +++ mm-2.6.26-rc2-mm1/mm/memcontrol.c > > @@ -578,7 +578,7 @@ retry: > > if (state == RES_OVER_LIMIT) { > > if (!(gfp_mask & __GFP_WAIT)) > > goto out; > > - if (try_to_free_mem_cgroup_pages(mem, gfp_mask)) > > + if (try_to_free_mem_cgroup_pages(mem, -1, gfp_mask)) > > goto retry; > > /* > > * try_to_free_mem_cgroup_pages() might not give us a > > @@ -801,7 +801,7 @@ int mem_cgroup_shrink_usage(struct mm_st > > rcu_read_unlock(); > > > > do { > > - progress = try_to_free_mem_cgroup_pages(mem, gfp_mask); > > + progress = try_to_free_mem_cgroup_pages(mem, -1, gfp_mask); > > } while (!progress && --retry); > > > > if (!retry) > > @@ -814,7 +814,7 @@ static void mem_cgroup_drop_all_pages(st > > { > > int progress; > > while (!res_counter_empty(&mem->res)) { > > - progress = try_to_free_mem_cgroup_pages(mem, > > + progress = try_to_f…
creator: KAMEZAWA Hiroyuki
date: 2008-05-28T00:44:23-00:00

Re: [RFC 3/4] memcg: background reclaim
type: item
description: On Tue, 27 May 2008 22:38:09 +0530 Balbir Singh <balbir@linux.vnet.ibm.com> wrote: > > struct cgroup_subsys mem_cgroup_subsys __read_mostly; > > @@ -119,10 +120,6 @@ struct mem_cgroup_lru_info { > > * statistics based on the statistics developed by Rik Van Riel for clock-pro, > > * to help the administrator determine what knobs to tune. > > * > > - * TODO: Add a water mark for the memory controller. Reclaim will begin when > > - * we hit the water mark. May be even add a low water mark, such that > > - * no reclaim occurs from a cgroup at it's low water mark, this is > > - * a feature that will be implemented much later in the future. > > */ > > struct mem_cgroup { > > struct cgroup_subsys_state css; > > @@ -131,6 +128,13 @@ struct mem_cgroup { > > */ > > struct res_counter res; > > /* > > + * background reclaim. > > + */ > > + struct { > > + wait_queue_head_t waitq; > > + struct task_struct *thread; > > + } daemon; > > Comments on each of the members would be nice. > ok, I'll add. > > + /* > > * Per cgroup active and inactive list, similar to the > > * per zone LRU lists. > > */ > > @@ -143,6 +147,7 @@ struct mem_cgroup { > > struct mem_cgroup_stat stat; > > }; > > static struct mem_cgroup init_mem_cgroup; > > +static DEFINE_MUTEX(memcont_daemon_lock); > > > > /* > > * We use the lower bit of the page->page_cgroup pointer as a bit spin > > @@ -374,6 +379,15 @@ void mem_cgroup_move_lists(struct page * > > unlock_page_cgroup(page); > > } > > > > +static void mem_cgroup_schedule_reclaim(struct mem_cgroup *mem) > > +{ > > + if (!mem->daemon.thread) > > + return; > > > I suspect we are using threads. Aren't workqueues better than threads? > Hmm, I don't like to use generic workqueue for something which can take long time and can _sleep_. How about using a thread for the whole system for making service to all memcg ? > > + if (!waitqueue_active(&mem->daemon.waitq)) > > + return; > > + wake_up_interruptible(&mem->daemon.waitq); > > +} > > + > > /* > > * Calculate mapped_r…
creator: KAMEZAWA Hiroyuki
date: 2008-05-28T00:45:30-00:00