OpenVZ Forum - RDF feed
type: channel
items: [PATCH O/4] Block I/O tracking
items: [PATCH 1/4] Block I/O tracking
items: [PATCH 2/4] Block I/O tracking
items: [PATCH 3/4] Block I/O tracking
items: [PATCH 4/4] Block I/O tracking
items: Re: [PATCH O/4] Block I/O tracking
items: Re: [PATCH 2/4] Block I/O tracking
items: Re: [PATCH 1/4] Block I/O tracking

[PATCH O/4] Block I/O tracking
type: item
description: Hi, When you want to implement some kind of Block I/O controllers, you have to determine who issued each I/O. I just implemented this feature, with which you can track down the I/Os. When you have to find the owner which issued the I/O, it is the one which owns the page where the IO is going to start. The cgroup memory subsystem already has this feature, so I realized that it would make easy to implemented Block I/O tracking mechanism on the memory subsystem. I named it "bio cgroup." I made dm-ioband -- I/O bandwidth controller -- work with the bio cgroup, whose implementation is just experimental though. I have a plan on making the bio cgroup support io_context. Each bio cgroup will have one or more io_contexts so the I/O bandwidth controller can use it to control the bandwidths. I also have another plan on move the implementation of dm-ioband from the device mapper layer to somewhere before the I/O schedulers in the block layer. The following patches are against linux-2.6.25-rc5-mm1 and you have to apply the patch of dm-ioband v0.0.3, which you can download from http://people.valinux.co.jp/~ryov/dm-ioband/patches/dm-ioband-0.0.3.patch before applying the following patches. Let's say you want make two bio cgroups and assign them to ioband device "ioband1". First, you have to mount the bio cgroup filesystem. # mount -t cgroup -o bio none /cgroup/bio Then, you make new bio cgroups and put some processes in them. # mkdir /cgroup/bio/bgroup1 # mkdir /cgroup/bio/bgroup2 # echo 1234 /cgroup/bio/bgroup1/tasks # echo 5678 /cgroup/bio/bgroup1/tasks Now you check the ids of the bio cgroups which you just created. # cat /cgroup/bio/bgroup1/bio.id 1 # cat /cgroup/bio/bgroup2/bio.id 2 Finally, you can attach the cgroups to "ioband1" and assign them weights. # dmsetup message ioband1 0 type cgroup # dmsetup message ioband1 0 attach 1 # dmsetup message ioband1 0 attach 2 # dmsetup message ioband1 0 weight 1:30 # dmsetup message ioband1 0 weight 2:60 You can find the manual of dm-…
creator: Hirokazu Takahashi
date: 2008-03-18T09:22:51-00:00

[PATCH 1/4] Block I/O tracking
type: item
description: Hi, This patch splits the cgroup memory subsystem into two parts. One is for tracking which cgroup which pages and the other is for controlling how much amount of memory should be assigned to each cgroup. With this patch, you can use the page tracking mechanism even if the memory subsystem is off. Signed-off-by: Hirokazu Takahashi <taka@valinux.co.jp> --- linux-2.6.25-rc5.pagecgroup/include/linux/memcontrol.h 2008-03-17 21:45:16.000000000 +0900 +++ linux-2.6.25-rc5-mm1/include/linux/memcontrol.h 2008-03-18 11:58:13.000000000 +0900 @@ -20,12 +20,61 @@ #ifndef _LINUX_MEMCONTROL_H #define _LINUX_MEMCONTROL_H +#include <linux/rcupdate.h> +#include <linux/mm.h> +#include <linux/smp.h> +#include <linux/bit_spinlock.h> + struct mem_cgroup; struct page_cgroup; struct page; struct mm_struct; +#ifdef CONFIG_CGROUP_PAGE +/* + * We use the lower bit of the page->page_cgroup pointer as a bit spin + * lock. We need to ensure that page->page_cgroup is at least two + * byte aligned (based on comments from Nick Piggin). But since + * bit_spin_lock doesn't actually set that lock bit in a non-debug + * uniprocessor kernel, we should avoid setting it here too. + */ +#define PAGE_CGROUP_LOCK_BIT 0x0 +#if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK) +#define PAGE_CGROUP_LOCK (1 << PAGE_CGROUP_LOCK_BIT) +#else +#define PAGE_CGROUP_LOCK 0x0 +#endif + +/* + * A page_cgroup page is associated with every page descriptor. The + * page_cgroup helps us identify information about the cgroup + */ +struct page_cgroup { #ifdef CONFIG_CGROUP_MEM_RES_CTLR + struct list_head lru; /* per cgroup LRU list */ + struct mem_cgroup *mem_cgroup; +#endif /* CONFIG_CGROUP_MEM_RES_CTLR */ + struct page *page; + int ref_cnt; /* cached, mapped, migrating */ + int flags; +}; +#define PAGE_CGROUP_FLAG_CACHE (0x1) /* charged as cache */ +#define PAGE_CGROUP_FLAG_ACTIVE (0x2) /* page is active in this cgroup */ + +static inline void lock_page_cgroup(struct page *page) +{ + bit_spin_lock(PAGE_CGROUP_LOCK_BIT, &…
creator: Hirokazu Takahashi
date: 2008-03-18T09:25:08-00:00

[PATCH 2/4] Block I/O tracking
type: item
description: Hi, This patch implements the bio cgroup on the memory cgroup. Signed-off-by: Hirokazu Takahashi <taka@valinux.co.jp> --- linux-2.6.25-rc5.pagecgroup2/include/linux/memcontrol.h 2008-03-18 12:45:14.000000000 +0900 +++ linux-2.6.25-rc5-mm1/include/linux/memcontrol.h 2008-03-18 12:55:59.000000000 +0900 @@ -54,6 +54,10 @@ struct page_cgroup { struct list_head lru; /* per cgroup LRU list */ struct mem_cgroup *mem_cgroup; #endif /* CONFIG_CGROUP_MEM_RES_CTLR */ +#ifdef CONFIG_CGROUP_BIO + struct list_head blist; /* for bio_cgroup page list */ + struct bio_cgroup *bio_cgroup; +#endif struct page *page; int ref_cnt; /* cached, mapped, migrating */ int flags; --- linux-2.6.25-rc5.pagecgroup2/include/linux/biocontrol.h 2008-03-17 22:06:49.000000000 +0900 +++ linux-2.6.25-rc5-mm1/include/linux/biocontrol.h 2008-03-18 14:19:53.000000000 +0900 @@ -0,0 +1,148 @@ +#include <linux/cgroup.h> +#include <linux/mm.h> +#include <linux/memcontrol.h> + +#ifndef _LINUX_BIOCONTROL_H +#define _LINUX_BIOCONTROL_H + +#ifdef CONFIG_CGROUP_BIO + +struct io_context; + +struct bio_cgroup { + struct cgroup_subsys_state css; + int id; + struct io_context *io_context; /* default io_context */ +/* struct radix_tree_root io_context_root; per device io_context */ + spinlock_t page_list_lock; + struct list_head page_list; +}; + +static inline int bio_cgroup_disabled(void) +{ + return bio_cgroup_subsys.disabled; +} + +extern void mm_init_bio_cgroup(struct mm_struct *, struct task_struct *); + +static inline void __bio_cgroup_add_page(struct page_cgroup *pc) +{ + struct bio_cgroup *biog = pc->bio_cgroup; + list_add(&pc->blist, &biog->page_list); +} + +static inline void bio_cgroup_add_page(struct page_cgroup *pc) +{ + struct bio_cgroup *biog = pc->bio_cgroup; + unsigned long flags; + spin_lock_irqsave(&biog->page_list_lock, flags); + __bio_cgroup_add_page(pc); + spin_unlock_irqrestore(&biog->page_list_lock, flags); +} + +static inline void __bio_cgroup_remove_page(struct page_cgroup *pc) +{ + list_del_ini…
creator: Hirokazu Takahashi
date: 2008-03-18T09:29:06-00:00

[PATCH 3/4] Block I/O tracking
type: item
description: Hi, This patch is for cleaning up the code of the cgroup memory subsystem to remove some "#ifdef"s. Signed-off-by: Hirokazu Takahashi <taka@valinux.co.jp> --- linux-2.6.25-rc5.pagecgroup3/mm/memcontrol.c 2008-03-18 13:01:57.000000000 +0900 +++ linux-2.6.25-rc5-mm1/mm/memcontrol.c 2008-03-18 14:00:23.000000000 +0900 @@ -216,6 +216,51 @@ static struct mem_cgroup *mem_cgroup_fro struct mem_cgroup, css); } +static inline void get_mem_cgroup(struct mem_cgroup *mem) +{ + css_get(&mem->css); +} + +static inline void put_mem_cgroup(struct mem_cgroup *mem) +{ + css_put(&mem->css); +} + +static inline void set_mem_cgroup(struct page_cgroup *pc, + struct mem_cgroup *mem) +{ + pc->mem_cgroup = mem; +} + +static inline void clear_mem_cgroup(struct page_cgroup *pc) +{ + struct mem_cgroup *mem = pc->mem_cgroup; + res_counter_uncharge(&mem->res, PAGE_SIZE); + pc->mem_cgroup = NULL; + put_mem_cgroup(mem); +} + +/* This sould be called in an RCU-protected section. */ +static inline struct mem_cgroup *mm_get_mem_cgroup(struct mm_struct *mm) +{ + struct mem_cgroup *mem = rcu_dereference(mm->mem_cgroup); + get_mem_cgroup(mem); + return mem; +} + +static inline void mm_init_mem_cgroup(struct mm_struct *mm, + struct task_struct *p) +{ + struct mem_cgroup *mem = mem_cgroup_from_task(p); + get_mem_cgroup(mem); + mm->mem_cgroup = mem; +} + +static inline void mm_free_mem_cgroup(struct mm_struct *mm) +{ + put_mem_cgroup(mm->mem_cgroup); +} + static void __mem_cgroup_remove_list(struct page_cgroup *pc) { int from = pc->flags & PAGE_CGROUP_FLAG_ACTIVE; @@ -266,6 +311,26 @@ static void __mem_cgroup_move_lists(stru } } +static inline void mem_cgroup_add_page(struct page_cgroup *pc) +{ + struct mem_cgroup_per_zone *mz = page_cgroup_zoneinfo(pc); + unsigned long flags; + + spin_lock_irqsave(&mz->lru_lock, flags); + __mem_cgroup_add_list(pc); + spin_unlock_irqrestore(&mz->lru_lock, flags); +} + +static inline void mem_cgroup_remove_page(struct page_cgroup *pc) +{ + struct mem_cgroup_per_zone *mz = p…
creator: Hirokazu Takahashi
date: 2008-03-18T09:30:33-00:00

[PATCH 4/4] Block I/O tracking
type: item
description: Hi, With this patch, dm-ioband can work with the bio cgroup. Signed-off-by: Hirokazu Takahashi <taka@valinux.co.jp> --- linux-2.6.25-rc3.ioload/drivers/md/dm-ioband-type.c 2008-03-13 22:27:45.000000000 +0900 +++ linux-2.6.25-rc3-mm1/drivers/md/dm-ioband-type.c 2008-03-17 15:57:32.000000000 +0900 @@ -6,6 +6,7 @@ * This file is released under the GPL. */ #include <linux/bio.h> +#include <linux/biocontrol.h> #include "dm.h" #include "dm-bio-list.h" #include "dm-ioband.h" @@ -57,13 +58,7 @@ static int ioband_node(struct bio *bio) static int ioband_cgroup(struct bio *bio) { - /* - * This function should return the ID of the cgroup which issued "bio". - * The ID of the cgroup which the current process belongs to won't be - * suitable ID for this purpose, since some BIOs will be handled by kernel - * threads like aio or pdflush on behalf of the process requesting the BIOs. - */ - return 0; /* not implemented yet */ + return get_bio_cgroup_id(bio->bi_io_vec[0].bv_page); } struct group_type dm_ioband_group_type[] = { _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Hirokazu Takahashi
date: 2008-03-18T09:31:48-00:00

Re: [PATCH O/4] Block I/O tracking
type: item
description: On Tue, 18 Mar 2008 18:22:51 +0900 (JST) Hirokazu Takahashi <taka@valinux.co.jp> wrote: > # mount -t cgroup -o bio none /cgroup/bio > > Then, you make new bio cgroups and put some processes in them. > > # mkdir /cgroup/bio/bgroup1 > # mkdir /cgroup/bio/bgroup2 > # echo 1234 /cgroup/bio/bgroup1/tasks > # echo 5678 /cgroup/bio/bgroup1/tasks > > Now you check the ids of the bio cgroups which you just created. > > # cat /cgroup/bio/bgroup1/bio.id > 1 > # cat /cgroup/bio/bgroup2/bio.id > 2 > > Finally, you can attach the cgroups to "ioband1" and assign them weights. > > # dmsetup message ioband1 0 type cgroup > # dmsetup message ioband1 0 attach 1 > # dmsetup message ioband1 0 attach 2 > # dmsetup message ioband1 0 weight 1:30 > # dmsetup message ioband1 0 weight 2:60 > > You can find the manual of dm-ioband at > http://people.valinux.co.jp/~ryov/dm-ioband/manual/index.html. > But the user interface for the bio cgroup is temporal and it will be > changed after the io_context support. > I'm grad if these some kinds of params rather than 'id' are also shown under cgroup. Thanks, -Kame _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: KAMEZAWA Hiroyuki
date: 2008-03-18T10:15:11-00:00

Re: [PATCH 2/4] Block I/O tracking
type: item
description: On Tue, 18 Mar 2008 18:29:06 +0900 (JST) Hirokazu Takahashi <taka@valinux.co.jp> wrote: > Hi, > > This patch implements the bio cgroup on the memory cgroup. > > Signed-off-by: Hirokazu Takahashi <taka@valinux.co.jp> > > > --- linux-2.6.25-rc5.pagecgroup2/include/linux/memcontrol.h 2008-03-18 12:45:14.000000000 +0900 > +++ linux-2.6.25-rc5-mm1/include/linux/memcontrol.h 2008-03-18 12:55:59.000000000 +0900 > @@ -54,6 +54,10 @@ struct page_cgroup { > struct list_head lru; /* per cgroup LRU list */ > struct mem_cgroup *mem_cgroup; > #endif /* CONFIG_CGROUP_MEM_RES_CTLR */ > +#ifdef CONFIG_CGROUP_BIO > + struct list_head blist; /* for bio_cgroup page list */ > + struct bio_cgroup *bio_cgroup; > +#endif Hmm, definition like this == enum { #ifdef CONFIG_CGROUP_MEM_RES_CTLR MEM_RES_CTLR, #endif #ifdef CONFIG_CGROURP_BIO BIO_CTLR, #endif NR_VM_CTRL, }; void *cgroups[NR_VM_CGROUP]; == Can save another #ifdefs ? And, blist seems to be just used for force_empty. Do you really need this ? no alternative ? Thanks, -Kame _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: KAMEZAWA Hiroyuki
date: 2008-03-18T10:20:14-00:00

Re: [PATCH 1/4] Block I/O tracking
type: item
description: On Tue, 18 Mar 2008 18:25:08 +0900 (JST) Hirokazu Takahashi <taka@valinux.co.jp> wrote: > Hi, > > This patch splits the cgroup memory subsystem into two parts. > One is for tracking which cgroup which pages and the other is > for controlling how much amount of memory should be assigned to > each cgroup. > > With this patch, you can use the page tracking mechanism even if > the memory subsystem is off. > > Signed-off-by: Hirokazu Takahashi <taka@valinux.co.jp> > I think current my work, radix-tree page cgroup will help this work. It creates page_cgroup.h and page_cgroup.c. Patches are posted last week to the mm-list.(And now rewriting..) Please let me know if you have requests. Thanks, -Kame _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: KAMEZAWA Hiroyuki
date: 2008-03-18T10:24:02-00:00

Re: [PATCH 2/4] Block I/O tracking
type: item
description: Hi, > > Hi, > > > > This patch implements the bio cgroup on the memory cgroup. > > > > Signed-off-by: Hirokazu Takahashi <taka@valinux.co.jp> > > > > > > --- linux-2.6.25-rc5.pagecgroup2/include/linux/memcontrol.h 2008-03-18 12:45:14.000000000 +0900 > > +++ linux-2.6.25-rc5-mm1/include/linux/memcontrol.h 2008-03-18 12:55:59.000000000 +0900 > > @@ -54,6 +54,10 @@ struct page_cgroup { > > struct list_head lru; /* per cgroup LRU list */ > > struct mem_cgroup *mem_cgroup; > > #endif /* CONFIG_CGROUP_MEM_RES_CTLR */ > > +#ifdef CONFIG_CGROUP_BIO > > + struct list_head blist; /* for bio_cgroup page list */ > > + struct bio_cgroup *bio_cgroup; > > +#endif > > Hmm, definition like this > == > enum { > #ifdef CONFIG_CGROUP_MEM_RES_CTLR > MEM_RES_CTLR, > #endif > #ifdef CONFIG_CGROURP_BIO > BIO_CTLR, > #endif > NR_VM_CTRL, > }; > > void *cgroups[NR_VM_CGROUP]; > == > Can save another #ifdefs ? I guess the following code can be possible if you really want to remove the #ifdefs here. But the both of them look overkill. #ifdef CONFIG_CGROUP_MEM_RES_CTLR #define MEM_CONTROL \ struct list_head lru; \ struct mem_cgroup *mem_cgroup \ #else #define MEM_CONTROL #endif #define BIO_CONTROL \ struct list_head blist; \ struct bio_cgroup *bio_cgroup \ #else #define BIO_CONTROL #endif struct page_cgroup { MEM_CONTROL; BIO_CONTROL; struct page *page; int ref_cnt; int flags; }; > And, blist seems to be just used for force_empty. > Do you really need this ? no alternative ? I selected this approach because it was the simplest way for the first implementation. I've been also thinking about what you pointed. If you don't mind taking a long time to remove a bio cgroup, it will be the easiest way that you can scan all pages to find the pages which belong to the cgroup and delete them. It may be enough since you may say it will rarely happen. But it might cause some trouble on machines with huge memory. So I'm actually looking for anther way if we could release them lazily. But I don't think I have…
creator: Hirokazu Takahashi
date: 2008-03-18T11:34:22-00:00

Re: [PATCH 2/4] Block I/O tracking
type: item
description: On Tue, 18 Mar 2008 20:34:22 +0900 (JST) Hirokazu Takahashi <taka@valinux.co.jp> wrote: > > > And, blist seems to be just used for force_empty. > > Do you really need this ? no alternative ? > > I selected this approach because it was the simplest way for the > first implementation. > > I've been also thinking about what you pointed. > If you don't mind taking a long time to remove a bio cgroup, it will be > the easiest way that you can scan all pages to find the pages which > belong to the cgroup and delete them. It may be enough since you may > say it will rarely happen. But it might cause some trouble on machines > with huge memory. > Hmm, force_empty itself is necessary ? Thanks, -Kame _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: KAMEZAWA Hiroyuki
date: 2008-03-18T11:52:18-00:00

Re: [PATCH 2/4] Block I/O tracking
type: item
description: Hi, > > > And, blist seems to be just used for force_empty. > > > Do you really need this ? no alternative ? > > > > I selected this approach because it was the simplest way for the > > first implementation. > > > > I've been also thinking about what you pointed. > > If you don't mind taking a long time to remove a bio cgroup, it will be > > the easiest way that you can scan all pages to find the pages which > > belong to the cgroup and delete them. It may be enough since you may > > say it will rarely happen. But it might cause some trouble on machines > > with huge memory. > > > Hmm, force_empty itself is necessary ? It is called when bio cgroups are removed. With the current implementation, when you delete a bio cgroup, the bio_cgroup members of page_cgroups which point the cgroup have to be cleared. So I'm looking for another way like: - Use some kind of id instead of a pointer to a bio cgroup, so you can check whether the id is valid before you use it. - Don't free the bio cgroup until all the pages referring to the cgroup. I also want to implement that if you find a page whose cgroup is already removed, the page should be assigned to a new cgroup. Thank you, Hirokazu Takahashi. _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Hirokazu Takahashi
date: 2008-03-18T12:15:05-00:00

Re: [PATCH O/4] Block I/O tracking
type: item
description: Hi, > > # mount -t cgroup -o bio none /cgroup/bio > > > > Then, you make new bio cgroups and put some processes in them. > > > > # mkdir /cgroup/bio/bgroup1 > > # mkdir /cgroup/bio/bgroup2 > > # echo 1234 /cgroup/bio/bgroup1/tasks > > # echo 5678 /cgroup/bio/bgroup1/tasks > > > > Now you check the ids of the bio cgroups which you just created. > > > > # cat /cgroup/bio/bgroup1/bio.id > > 1 > > # cat /cgroup/bio/bgroup2/bio.id > > 2 > > > > Finally, you can attach the cgroups to "ioband1" and assign them weights. > > > > # dmsetup message ioband1 0 type cgroup > > # dmsetup message ioband1 0 attach 1 > > # dmsetup message ioband1 0 attach 2 > > # dmsetup message ioband1 0 weight 1:30 > > # dmsetup message ioband1 0 weight 2:60 > > > > You can find the manual of dm-ioband at > > http://people.valinux.co.jp/~ryov/dm-ioband/manual/index.html. > > But the user interface for the bio cgroup is temporal and it will be > > changed after the io_context support. > > > I'm grad if these some kinds of params rather than 'id' are also shown > under cgroup. You mean each bio cgroup has to have a lot of files which shows the status of the cgroup or allows you to control the cgroup. I think this should be done after the cgroup bio subsystem supports io_context since the interface will be changed to support it. > Thanks, > -Kame Thank you, Hirokazu Takahashi. _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Hirokazu Takahashi
date: 2008-03-19T03:59:08-00:00