OpenVZ Forum - RDF feed
type: channel
items: [BUGFIX][RFC][PATCH][only -mm] FIX memory leak in memory cgroup vs. page migration [0/1]
items: Re: [BUGFIX][RFC][PATCH][only -mm] FIX memory leak in memory cgroup vs. page migration [1/1] fix pag
items: [BUGFIX][RFC][PATCH][only -mm] FIX memory leak in memory cgroup vs. page migration [2/1] additional
items: Re: [BUGFIX][RFC][PATCH][only -mm] FIX memory leak in memory cgroup vs. page migration [0/1]
items: Re: [BUGFIX][RFC][PATCH][only -mm] FIX memory leak in memory cgroup vs. page migration [2/1] additio

[BUGFIX][RFC][PATCH][only -mm] FIX memory leak in memory cgroup vs. page migration [0/1]
type: item
description: Current implementation of memory cgroup controller does following in migration. 1. uncharge when unmapped. 2. charge again when remapped. Consider migrate a page from OLD to NEW. In following case, memory (for page_cgroup) will leak. 1. charge OLD page as page-cache. (charge = 1 2. A process mmap OLD page. (chage + 1 = 2) 3. A process migrates it. try_to_unmap(OLD) (charge - 1 = 1) replace OLD with NEW remove_migration_pte(NEW) (New is newly charged.) discard OLD page. (page_cgroup for OLD page is not reclaimed.) patch is in the next mail. Test Log on 2.6.18-rc8-mm2. == # mount cgroup and create group_A group_B [root@drpq kamezawa]# mount -t cgroup none /opt/mem_control/ -o memory [root@drpq kamezawa]# mkdir /opt/mem_control/group_A/ [root@drpq kamezawa]# mkdir /opt/mem_control/group_B/ [root@drpq kamezawa]# bash [root@drpq kamezawa]# echo $$ > /opt/mem_control/group_A/tasks [root@drpq kamezawa]# cat /opt/mem_control/group_A/memory.usage_in_bytes 475136 [root@drpq kamezawa]# grep size-64 /proc/slabinfo size-64(DMA) 0 0 64 240 1 : tunables 120 60 8 : slabdata 0 0 0 size-64 30425 30960 64 240 1 : tunables 120 60 8 : slabdata 129 129 12 # charge file cache 512Mfile to groupA [root@drpq kamezawa]# cat 512Mfile > /dev/null [root@drpq kamezawa]# cat /opt/mem_control/group_A/memory.usage_in_bytes 539525120 # for test, try drop_caches. drop_cache works well and chage decreased. [root@drpq kamezawa]# echo 3 > /proc/sys/vm/drop_caches [root@drpq kamezawa]# cat /opt/mem_control/group_A/memory.usage_in_bytes 983040 # chage file cache 512Mfile again. [root@drpq kamezawa]# taskset 01 cat 512Mfile > /dev/null [root@drpq kamezawa]# exit exit [root@drpq kamezawa]# cat /opt/mem_control/group_?/memory.usage_in_bytes 539738112 0 [root@drpq kamezawa]# bash #enter group B [root@drpq kamezawa]# echo $$ > /opt/mem_control/group_B/tasks [root@drpq kamezawa]# cat /opt/mem_control/group_?/memory.usage_in_bytes 539738112 557056 [root@drpq kamezawa]# grep size-64 /proc/slabinfo size-64(DMA) 0 0…
creator: KAMEZAWA Hiroyuki
date: 2007-10-02T09:30:31-00:00

Re: [BUGFIX][RFC][PATCH][only -mm] FIX memory leak in memory cgroup vs. page migration [1/1] fix pag
type: item
description: While using memory control cgroup, page-migration under it works as following. == 1. uncharge all refs at try to unmap. 2. charge regs again remove_migration_ptes() == This is simple but has following problems. == The page is uncharged and chaged back again if *mapped*. - This means that cgroup before migraion can be different from one after migraion - If page is not mapped but charged as page cache, charge is just ignored (because not mapped, it will not be uncharged before migration) This is memory leak. == This is bad. And migration can migrate *not mapped* pages in future by migration-by-kernel driven by memory-unplug and defragment-by-migration at el. This patch tries to keep memory cgroup at page migration by increasing one refcnt during it. 3 functions are added. mem_cgroup_prepare_migration() --- increase refcnt of page->page_cgroup mem_cgroup_end_migration() --- decrease refcnt of page->page_cgroup mem_cgroup_page_migration() --- copy page->page_cgroup from old page to new page. Obviously, mem_cgroup_isolate_pages() and this page migration, which copies page_cgroup from old page to new page, has race. There seem to be 3 ways for avoiding this race. A. take mem_group->lock while mem_cgroup_page_migration(). B. isolate pc from mem_cgroup's LRU when we isolate page from zone's LRU. C. ignore non-LRU page at mem_cgroup_isolate_pages(). This patch uses method (C.) and modifes mem_cgroup_isolate_pages() igonres !PageLRU pages. Tested and worked well in ia64/NUMA box. Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> --- include/linux/memcontrol.h | 22 +++++++++++++++ mm/memcontrol.c | 62 ++++++++++++++++++++++++++++++++++++++++++--- mm/migrate.c | 13 +++++++-- 3 files changed, 90 insertions(+), 7 deletions(-) Index: linux-2.6.23-rc8-mm2/include/linux/memcontrol.h =================================================================== --- linux-2.6.23-rc8-mm2.orig/include/linux/memcontrol.h +++ linux-2.6.23-rc8-mm2/include/linux/memcontrol.h @@ -43,8 +…
creator: KAMEZAWA Hiroyuki
date: 2007-10-02T09:33:06-00:00

[BUGFIX][RFC][PATCH][only -mm] FIX memory leak in memory cgroup vs. page migration [2/1] additional
type: item
description: The patch I sent needs following fix, sorry. Anyway, I'll repost good-version with reflected comments again. Thanks, -Kame = fix-page-migration-under-memory-controller patch needs this fix..sorry. - We should uncharge page by mem_cgroup_end_migration() only if page is not charged by mem_cgroup_prepare_migration(). - move mem_cgroup_prepare_migration() after goto: Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> Index: linux-2.6.23-rc8-mm2/mm/memcontrol.c =================================================================== --- linux-2.6.23-rc8-mm2.orig/mm/memcontrol.c +++ linux-2.6.23-rc8-mm2/mm/memcontrol.c @@ -445,7 +445,7 @@ retry: } } -void mem_cgroup_prepare_migration(struct page *page) +int mem_cgroup_prepare_migration(struct page *page) { struct page_cgroup *pc; lock_page_cgroup(page); @@ -453,7 +453,7 @@ void mem_cgroup_prepare_migration(struct if (pc) atomic_inc(&pc->ref_cnt); unlock_page_cgroup(page); - return; + return (pc != NULL); } void mem_cgroup_end_migration(struct page *page) Index: linux-2.6.23-rc8-mm2/include/linux/memcontrol.h =================================================================== --- linux-2.6.23-rc8-mm2.orig/include/linux/memcontrol.h +++ linux-2.6.23-rc8-mm2/include/linux/memcontrol.h @@ -47,7 +47,7 @@ extern void mem_cgroup_out_of_memory(str extern int mem_cgroup_cache_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask); /* For handling page migration in proper way */ -extern void mem_cgroup_prepare_migration(struct page *page); +extern int mem_cgroup_prepare_migration(struct page *page); extern void mem_cgroup_end_migration(struct page *page); extern void mem_cgroup_page_migration(struct page *newpage, struct page *page); @@ -114,9 +114,9 @@ static inline struct mem_cgroup *mm_cgro } /* For page migration */ -static inline void mem_cgroup_prepare_migration(struct page *page) +static inline int mem_cgroup_prepare_migration(struct page *page) { - return; + return 0; } static inline void mem_cgroup_end_m…
creator: KAMEZAWA Hiroyuki
date: 2007-10-02T10:12:17-00:00

Re: [BUGFIX][RFC][PATCH][only -mm] FIX memory leak in memory cgroup vs. page migration [0/1]
type: item
description: KAMEZAWA Hiroyuki wrote: > Current implementation of memory cgroup controller does following in migration. > > 1. uncharge when unmapped. > 2. charge again when remapped. > > Consider migrate a page from OLD to NEW. > > In following case, memory (for page_cgroup) will leak. > > 1. charge OLD page as page-cache. (charge = 1 > 2. A process mmap OLD page. (chage + 1 = 2) > 3. A process migrates it. > try_to_unmap(OLD) (charge - 1 = 1) > replace OLD with NEW > remove_migration_pte(NEW) (New is newly charged.) > discard OLD page. (page_cgroup for OLD page is not reclaimed.) > Interesting test scenario, I'll try and reproduce the problem here. Why does discard OLD page not reclaim page_cgroup? > patch is in the next mail. > Thanks > Test Log on 2.6.18-rc8-mm2. > == > # mount cgroup and create group_A group_B > [root@drpq kamezawa]# mount -t cgroup none /opt/mem_control/ -o memory > [root@drpq kamezawa]# mkdir /opt/mem_control/group_A/ > [root@drpq kamezawa]# mkdir /opt/mem_control/group_B/ > [root@drpq kamezawa]# bash > [root@drpq kamezawa]# echo $$ > /opt/mem_control/group_A/tasks > [root@drpq kamezawa]# cat /opt/mem_control/group_A/memory.usage_in_bytes > 475136 > [root@drpq kamezawa]# grep size-64 /proc/slabinfo > size-64(DMA) 0 0 64 240 1 : tunables 120 60 8 : slabdata 0 0 0 > size-64 30425 30960 64 240 1 : tunables 120 60 8 : slabdata 129 129 12 > > # charge file cache 512Mfile to groupA > [root@drpq kamezawa]# cat 512Mfile > /dev/null > [root@drpq kamezawa]# cat /opt/mem_control/group_A/memory.usage_in_bytes > 539525120 > > # for test, try drop_caches. drop_cache works well and chage decreased. > [root@drpq kamezawa]# echo 3 > /proc/sys/vm/drop_caches > [root@drpq kamezawa]# cat /opt/mem_control/group_A/memory.usage_in_bytes > 983040 > > # chage file cache 512Mfile again. > [root@drpq kamezawa]# taskset 01 cat 512Mfile > /dev/null > [root@drpq kamezawa]# exit > exit > [root@drpq kamezawa]# cat /opt/mem_control/group_?/memory.usage_in_bytes > 539738112 > 0 > [root@dr…
creator: Balbir Singh
date: 2007-10-02T13:34:15-00:00

Re: [BUGFIX][RFC][PATCH][only -mm] FIX memory leak in memory cgroup vs. page migration [1/1] fix pag
type: item
description: KAMEZAWA Hiroyuki wrote: > While using memory control cgroup, page-migration under it works as following. > == > 1. uncharge all refs at try to unmap. > 2. charge regs again remove_migration_ptes() > == > This is simple but has following problems. > == > The page is uncharged and chaged back again if *mapped*. > - This means that cgroup before migraion can be different from one after > migraion >From the test case mentioned earlier, this happens because the task has moved from one cgroup to another, right? > - If page is not mapped but charged as page cache, charge is just ignored > (because not mapped, it will not be uncharged before migration) OK. This is an interesting situation, we uncharge page cache only in __remove_from_page_cache(). This is a combination of task migration followed by page migration, which the memory controller does not handle very well at the moment. > This is memory leak. Yes, it is. > == > This is bad. Absolutely! > And migration can migrate *not mapped* pages in future by migration-by-kernel > driven by memory-unplug and defragment-by-migration at el. > > This patch tries to keep memory cgroup at page migration by increasing > one refcnt during it. 3 functions are added. > mem_cgroup_prepare_migration() --- increase refcnt of page->page_cgroup > mem_cgroup_end_migration() --- decrease refcnt of page->page_cgroup > mem_cgroup_page_migration() --- copy page->page_cgroup from old page to > new page. > > Obviously, mem_cgroup_isolate_pages() and this page migration, which > copies page_cgroup from old page to new page, has race. > > There seem to be 3 ways for avoiding this race. > A. take mem_group->lock while mem_cgroup_page_migration(). > B. isolate pc from mem_cgroup's LRU when we isolate page from zone's LRU. > C. ignore non-LRU page at mem_cgroup_isolate_pages(). > > This patch uses method (C.) and modifes mem_cgroup_isolate_pages() igonres > !PageLRU pages. > The page(s) is(are) !PageLRU only during page migration right? > Tested and w…
creator: Balbir Singh
date: 2007-10-02T15:34:40-00:00

Re: [BUGFIX][RFC][PATCH][only -mm] FIX memory leak in memory cgroup vs. page migration [2/1] additio
type: item
description: KAMEZAWA Hiroyuki wrote: > The patch I sent needs following fix, sorry. > Anyway, I'll repost good-version with reflected comments again. > > Thanks, > -Kame Just saw this now, I'll apply both the fixes, but it would be helpful if you could post, one combined patch. -- Warm Regards, Balbir Singh Linux Technology Center IBM, ISTL _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Balbir Singh
date: 2007-10-02T15:36:27-00:00

Re: [BUGFIX][RFC][PATCH][only -mm] FIX memory leak in memory cgroup vs. page migration [2/1] additio
type: item
description: On Tue, 02 Oct 2007 21:06:27 +0530 Balbir Singh <balbir@linux.vnet.ibm.com> wrote: > KAMEZAWA Hiroyuki wrote: > > The patch I sent needs following fix, sorry. > > Anyway, I'll repost good-version with reflected comments again. > > > > Thanks, > > -Kame > > Just saw this now, I'll apply both the fixes, but it would be helpful > if you could post, one combined patch. > Yes, I'll post refleshed easy-to-review version again. I'm now planing to post a patch against next -mm. Thanks, -Kame _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: KAMEZAWA Hiroyuki
date: 2007-10-03T00:29:09-00:00

Re: [BUGFIX][RFC][PATCH][only -mm] FIX memory leak in memory cgroup vs. page migration [0/1]
type: item
description: On Tue, 02 Oct 2007 19:04:15 +0530 Balbir Singh <balbir@linux.vnet.ibm.com> wrote: > KAMEZAWA Hiroyuki wrote: > > Current implementation of memory cgroup controller does following in migration. > > > > 1. uncharge when unmapped. > > 2. charge again when remapped. > > > > Consider migrate a page from OLD to NEW. > > > > In following case, memory (for page_cgroup) will leak. > > > > 1. charge OLD page as page-cache. (charge = 1 > > 2. A process mmap OLD page. (chage + 1 = 2) > > 3. A process migrates it. > > try_to_unmap(OLD) (charge - 1 = 1) > > replace OLD with NEW > > remove_migration_pte(NEW) (New is newly charged.) > > discard OLD page. (page_cgroup for OLD page is not reclaimed.) > > > > Interesting test scenario, I'll try and reproduce the problem here. > Why does discard OLD page not reclaim page_cgroup? Just because OLD page is not page-cache at discarding. (it is replaced with NEW page) > > > [root@drpq kamezawa]# ./migrate_test 512Mfile 1 & > > [1] 4108 > > #At the end of migration, > > Where can I find migrate_test? > here, (just I wrote for this test. works on my RHEL5/2.6.18-rc8-mm2/ia64 NUMA box) This program doesn't check 'where is file cache ?' before migration. So please check it by yourself before run. == #include <stdio.h> #include <stdlib.h> #include <syscall.h> #include <sys/mman.h> #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <errno.h> #include <numaif.h> #include <unistd.h> #define PAGESIZE (16384) static inline int move_pages(pid_t pid, unsigned long nr_pages, const void **address, const int *nodes, int *status, int flags) { return syscall(SYS_move_pages, pid, nr_pages, address, nodes, status, flags); } /* * migrate_test.c -- mmap file and migrate it to specified node. * %migrate_task file nodeid * */ int main(int argc, char *argv[]) { int ret, fd, val, node[1], status[1]; char *addr, *c; unsigned long size, nr_pages, pos; struct stat statbuf; void *address[1]; int target; if (argc != 3) { fprintf(stderr,"usage: mig…
creator: KAMEZAWA Hiroyuki
date: 2007-10-03T00:39:32-00:00

Re: [BUGFIX][RFC][PATCH][only -mm] FIX memory leak in memory cgroup vs. page migration [1/1] fix pag
type: item
description: On Tue, 02 Oct 2007 21:04:40 +0530 Balbir Singh <balbir@linux.vnet.ibm.com> wrote: > KAMEZAWA Hiroyuki wrote: > > While using memory control cgroup, page-migration under it works as following. > > == > > 1. uncharge all refs at try to unmap. > > 2. charge regs again remove_migration_ptes() > > == > > This is simple but has following problems. > > == > > The page is uncharged and chaged back again if *mapped*. > > - This means that cgroup before migraion can be different from one after > > migraion > > >From the test case mentioned earlier, this happens because the task has > moved from one cgroup to another, right? Ah, yes. > > And migration can migrate *not mapped* pages in future by migration-by-kernel > > driven by memory-unplug and defragment-by-migration at el. > > > > This patch tries to keep memory cgroup at page migration by increasing > > one refcnt during it. 3 functions are added. > > mem_cgroup_prepare_migration() --- increase refcnt of page->page_cgroup > > mem_cgroup_end_migration() --- decrease refcnt of page->page_cgroup > > mem_cgroup_page_migration() --- copy page->page_cgroup from old page to > > new page. > > > > Obviously, mem_cgroup_isolate_pages() and this page migration, which > > copies page_cgroup from old page to new page, has race. > > > > There seem to be 3 ways for avoiding this race. > > A. take mem_group->lock while mem_cgroup_page_migration(). > > B. isolate pc from mem_cgroup's LRU when we isolate page from zone's LRU. > > C. ignore non-LRU page at mem_cgroup_isolate_pages(). > > > > This patch uses method (C.) and modifes mem_cgroup_isolate_pages() igonres > > !PageLRU pages. > > > > The page(s) is(are) !PageLRU only during page migration right? > Hmm...!PageLRU() means that page is not on LRU. Then, kswapd can remove a page from LRU. > > - if (page_zone(page) != z) > > + if (page_zone(page) != z || !PageLRU(page)) { > > I would prefer to do unlikely(!PageLRU(page)), since most of the > times the page is not under migration > I see…
creator: KAMEZAWA Hiroyuki
date: 2007-10-03T00:53:09-00:00

Re: [BUGFIX][RFC][PATCH][only -mm] FIX memory leak in memory cgroup vs. page migration [0/1]
type: item
description: KAMEZAWA Hiroyuki wrote: [snip] > # migrate_test mmaps 512Mfile and call system call move_pages(). and sleep. > [root@drpq kamezawa]# ./migrate_test 512Mfile 1 & > [1] 4108 This step fails for me. I get an -ENOENT error from the utility you sent me. As I look through the migration code more (It's too late for me to double check), but it seems that only pages mapped in the process are migrated. cat(1) won't really map anything. Am I missing some of the reproduction steps? > #At the end of migration, > [root@drpq kamezawa]# cat /opt/mem_control/group_?/memory.usage_in_bytes > 539738112 > 537706496 > > #Wow, charge is twice ;) -- Warm Regards, Balbir Singh Linux Technology Center IBM, ISTL _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Balbir Singh
date: 2007-10-16T21:03:28-00:00