OpenVZ Forum - RDF feed
type: channel
items: [PATCH 0/2] dm-band: The I/O bandwidth controller: Overview
items: [PATCH 1/2] dm-band: The I/O bandwidth controller: Source code patch
items: [PATCH 2/2] dm-band: The I/O bandwidth controller: Document
items: Re: [PATCH 0/2] dm-band: The I/O bandwidth controller: Overview
items: Re: [PATCH 2/2] dm-band: The I/O bandwidth controller: Document
items: dm-band: The I/O bandwidth controller: Performance Report
items: Re: [Xen-devel] dm-band: The I/O bandwidth controller: Performance Report

[PATCH 0/2] dm-band: The I/O bandwidth controller: Overview
type: item
description: Hi everyone, I'm happy to announce that I've implemented a Block I/O bandwidth controller. The controller is designed to be of use in a cgroup or virtual machine environment. The current approach is that the controller is implemented as a device-mapper driver. What's dm-band all about? ======================== Dm-band is an I/O bandwidth controller implemented as a device-mapper driver. Several jobs using the same physical device have to share the bandwidth of the device. Dm-band gives bandwidth to each job according to its weight, which each job can set its own value to. At this time, a job is a group of processes with the same pid or pgrp or uid. There is also a plan to make it support cgroup. A job can also be a virtual machine such as KVM or Xen. +------+ +------+ +------+ +------+ +------+ +------+ |cgroup| |cgroup| | the | | pid | | pid | | the | jobs | A | | B | |others| | X | | Y | |others| +--|---+ +--|---+ +--|---+ +--|---+ +--|---+ +--|---+ +--V----+---V---+----V---+ +--V----+---V---+----V---+ | group | group | default| | group | group | default| band groups | | | group | | | | group | +-------+-------+--------+ +-------+-------+--------+ | band1 | | band2 | band devices +-----------|------------+ +-----------|------------+ +-----------V--------------+-------------V------------+ | | | | sdb1 | sdb2 | physical devices +--------------------------+--------------------------+ How dm-band works. ======================== Every band device has one band group, which by default is called the default group. Band devices can also have extra band groups in them. Each band group has a job to support and a weight. Proportional to the weight, dm-band gives tokens to the group. A group passes on I/O requests that its job issues to the underlying layer so long as it has tokens left, while requests are blocked if there aren't any tokens left in the group. One token is consumed each time the group passes on a request. Dm-band will refill groups with tokens once all of group…
creator: Ryo Tsuruta
date: 2008-01-23T12:53:50-00:00

[PATCH 1/2] dm-band: The I/O bandwidth controller: Source code patch
type: item
description: Here is the patch of dm-band. Based on 2.6.23.14 Signed-off-by: Ryo Tsuruta <ryov@valinux.co.jp> Signed-off-by: Hirokazu Takahashi <taka@valinux.co.jp> diff -uprN linux-2.6.23.14.orig/drivers/md/Kconfig linux-2.6.23.14/drivers/md/Kconfig --- linux-2.6.23.14.orig/drivers/md/Kconfig 2008-01-15 05:49:56.000000000 +0900 +++ linux-2.6.23.14/drivers/md/Kconfig 2008-01-21 16:09:41.000000000 +0900 @@ -276,4 +276,13 @@ config DM_DELAY If unsure, say N. +config DM_BAND + tristate "I/O band width control " + depends on BLK_DEV_DM + ---help--- + Any processes or cgroups can use the same storage + with its band-width fairly shared. + + If unsure, say N. + endif # MD diff -uprN linux-2.6.23.14.orig/drivers/md/Makefile linux-2.6.23.14/drivers/md/Makefile --- linux-2.6.23.14.orig/drivers/md/Makefile 2008-01-15 05:49:56.000000000 +0900 +++ linux-2.6.23.14/drivers/md/Makefile 2008-01-21 20:45:03.000000000 +0900 @@ -8,6 +8,7 @@ dm-multipath-objs := dm-hw-handler.o dm- dm-snapshot-objs := dm-snap.o dm-exception-store.o dm-mirror-objs := dm-log.o dm-raid1.o dm-rdac-objs := dm-mpath-rdac.o +dm-band-objs := dm-bandctl.o dm-band-policy.o dm-band-type.o md-mod-objs := md.o bitmap.o raid456-objs := raid5.o raid6algos.o raid6recov.o raid6tables.o \ raid6int1.o raid6int2.o raid6int4.o \ @@ -39,6 +40,7 @@ obj-$(CONFIG_DM_MULTIPATH_RDAC) += dm-rd obj-$(CONFIG_DM_SNAPSHOT) += dm-snapshot.o obj-$(CONFIG_DM_MIRROR) += dm-mirror.o obj-$(CONFIG_DM_ZERO) += dm-zero.o +obj-$(CONFIG_DM_BAND) += dm-band.o quiet_cmd_unroll = UNROLL $@ cmd_unroll = $(PERL) $(srctree)/$(src)/unroll.pl $(UNROLL) \ diff -uprN linux-2.6.23.14.orig/drivers/md/dm-band-policy.c linux-2.6.23.14/drivers/md/dm-band-policy.c --- linux-2.6.23.14.orig/drivers/md/dm-band-policy.c 1970-01-01 09:00:00.000000000 +0900 +++ linux-2.6.23.14/drivers/md/dm-band-policy.c 2008-01-21 20:31:14.000000000 +0900 @@ -0,0 +1,185 @@ +/* + * Copyright (C) 2008 VA Linux Systems Japan K.K. + * + * I/O bandwidth control + * + * This file is released under th…
creator: Ryo Tsuruta
date: 2008-01-23T12:56:32-00:00

[PATCH 2/2] dm-band: The I/O bandwidth controller: Document
type: item
description: Here is the document of dm-band. Based on 2.6.23.14 Signed-off-by: Ryo Tsuruta <ryov@valinux.co.jp> Signed-off-by: Hirokazu Takahashi <taka@valinux.co.jp> diff -uprN linux-2.6.23.14.orig/Documentation/device-mapper/band.txt linux-2.6.23.14/Documentation/device-mapper/band.txt --- linux-2.6.23.14.orig/Documentation/device-mapper/band.txt 1970-01-01 09:00:00.000000000 +0900 +++ linux-2.6.23.14/Documentation/device-mapper/band.txt 2008-01-23 21:48:46.000000000 +0900 @@ -0,0 +1,431 @@ +==================== +Document for dm-band +==================== + +Contents: + What's dm-band all about? + How dm-band works + Setup and Installation + Command Reference + TODO + + +What's dm-band all about? +======================== +Dm-band is an I/O bandwidth controller implemented as a device-mapper driver. +Several jobs using the same physical device have to share the bandwidth of +the device. Dm-band gives bandwidth to each job according to its weight, +which each job can set its own value to. + +At this time, a job is a group of processes with the same pid or pgrp or uid. +There is also a plan to make it support cgroup. A job can also be a virtual +machine such as KVM or Xen. + + +------+ +------+ +------+ +------+ +------+ +------+ + |cgroup| |cgroup| | the | | pid | | pid | | the | jobs + | A | | B | |others| | X | | Y | |others| + +--|---+ +--|---+ +--|---+ +--|---+ +--|---+ +--|---+ + +--V----+---V---+----V---+ +--V----+---V---+----V---+ + | group | group | default| | group | group | default| band groups + | | | group | | | | group | + +-------+-------+--------+ +-------+-------+--------+ + | band1 | | band2 | band devices + +-----------|------------+ +-----------|------------+ + +-----------V--------------+-------------V------------+ + | | | + | sdb1 | sdb2 | physical devices + +--------------------------+--------------------------+ + + +How dm-band works. +======================== +Every band device has one band group, which by default is called the default +group. + +Band d…
creator: Ryo Tsuruta
date: 2008-01-23T12:58:44-00:00

Re: [PATCH 0/2] dm-band: The I/O bandwidth controller: Overview
type: item
description: Hi, I believe this work is very important especially in the context of virtual machines. I think it would be more useful though implemented in the context of the IO scheduler. Since we already support a notion of IO priority, it seems reasonable to add a notion of an IO cap. Regards, Anthony Liguori Ryo Tsuruta wrote: > Hi everyone, > > I'm happy to announce that I've implemented a Block I/O bandwidth controller. > The controller is designed to be of use in a cgroup or virtual machine > environment. The current approach is that the controller is implemented as > a device-mapper driver. > > What's dm-band all about? > ======================== > Dm-band is an I/O bandwidth controller implemented as a device-mapper driver. > Several jobs using the same physical device have to share the bandwidth of > the device. Dm-band gives bandwidth to each job according to its weight, > which each job can set its own value to. > > At this time, a job is a group of processes with the same pid or pgrp or uid. > There is also a plan to make it support cgroup. A job can also be a virtual > machine such as KVM or Xen. > > +------+ +------+ +------+ +------+ +------+ +------+ > |cgroup| |cgroup| | the | | pid | | pid | | the | jobs > | A | | B | |others| | X | | Y | |others| > +--|---+ +--|---+ +--|---+ +--|---+ +--|---+ +--|---+ > +--V----+---V---+----V---+ +--V----+---V---+----V---+ > | group | group | default| | group | group | default| band groups > | | | group | | | | group | > +-------+-------+--------+ +-------+-------+--------+ > | band1 | | band2 | band devices > +-----------|------------+ +-----------|------------+ > +-----------V--------------+-------------V------------+ > | | | > | sdb1 | sdb2 | physical devices > +--------------------------+--------------------------+ > > > How dm-band works. > ======================== > Every band device has one band group, which by default is called the default > group. > > Band devices can also have extra band groups in them. Each band gr…
creator: Anthony Liguori
date: 2008-01-23T19:22:36-00:00

Re: [PATCH 2/2] dm-band: The I/O bandwidth controller: Document
type: item
description: Ryo Tsuruta <ryov@valinux.co.jp> writes: > Here is the document of dm-band. Could you please address in the document how the intended use cases/feature set etc. differs from CFQ2 io priorities? Thanks, -Andi _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Andi Kleen
date: 2008-01-23T19:57:57-00:00

Re: [PATCH 0/2] dm-band: The I/O bandwidth controller: Overview
type: item
description: Hi, > Hi, > > I believe this work is very important especially in the context of > virtual machines. I think it would be more useful though implemented in > the context of the IO scheduler. Since we already support a notion of > IO priority, it seems reasonable to add a notion of an IO cap. I agree that what you proposed is the most straightforward approach. Ryo and I have also investigated the CFQ scheduler that It will be possible to enhance it to support bandwidth control with quite a few modification. I think both approach have pros and cons. At this time, we have chosen the device-mapper approach because: - it can work with any I/O scheduler. Some people will want use the NOOP scheduler against high-end storages. - only people who need I/O bandwidth control should use it. - it is independent to the I/O schedulers so that it will be easy to maintain. - it can keep the CFQ implementation simple. The current the CFQ scheduler has some limitations if you want to control the bandwidths. The scheduler only has seven priority levels, which also means it has only seven classes. If you assign the same io-priority A to several VMs --- virtual machines ---, these machines have to share the I/O bandwidth which is assign to the io-priority A class. If other VM with io-priority B which is lower than io-priority A and there is no other VM in the same io-priority B class, VM in the io-priority B class may be able to use large bandwidth than VMs in io-priority the A class. I guess two level scheduling should be introduced in the CFQ scheduler if needed. The one is to choose the best cgroup or job, and the other is to choose the highest io-priority class. There is another limitation that io-priority is global so that it affects all the disks to access. It isn't allowed to have a job use several io-priorities to access several disks respectively. I think "per disk io-priority" will be required. But the device-mapper approach also has a bad points. It is hard to get the capabiliti…
creator: Hirokazu Takahashi
date: 2008-01-24T08:11:45-00:00

Re: [PATCH 2/2] dm-band: The I/O bandwidth controller: Document
type: item
description: Hi, > > Here is the document of dm-band. > > Could you please address in the document how the intended use > cases/feature set etc. differs from CFQ2 io priorities? Thank you for your suggestion, I'll do that step by step. Thanks, Ryo Tsuruta _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Ryo Tsuruta
date: 2008-01-24T10:32:02-00:00

dm-band: The I/O bandwidth controller: Performance Report
type: item
description: Hi, Now I report the result of dm-band bandwidth control test I did yesterday. I've got really good results that dm-band works as I expected. I made several band-groups on several disk partitions and gave them heavy I/O loads. Hardware Spec. ============== DELL Dimention E521: Linux kappa.local.valinux.co.jp 2.6.23.14 #1 SMP Thu Jan 24 17:24:59 JST 2008 i686 athlon i386 GNU/Linux Detected 2004.217 MHz processor. CPU0: AMD Athlon(tm) 64 X2 Dual Core Processor 3800+ stepping 02 Memory: 966240k/981888k available (2102k kernel code, 14932k reserved, 890k data, 216k init, 64384k highmem) scsi 2:0:0:0: Direct-Access ATA ST3250620AS 3.AA PQ: 0 ANSI: 5 sd 2:0:0:0: [sdb] 488397168 512-byte hardware sectors (250059 MB) sd 2:0:0:0: [sdb] Write Protect is off sd 2:0:0:0: [sdb] Mode Sense: 00 3a 00 00 sd 2:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA sdb: sdb1 sdb2 < sdb5 sdb6 sdb7 sdb8 sdb9 sdb10 sdb11 sdb12 sdb13 sdb14 sdb15 > The results of bandwidth control test on partitions =================================================== The configurations of the test #1: o Prepare three partitions sdb5, sdb6 and sdb7. o Give weights of 40, 20 and 10 to sdb5, sdb6 and sdb7 respectively. o Run 128 processes issuing random read/write direct I/O with 4KB data on each device at the same time. o Count up the number of I/Os and sectors which have done in 60 seconds. The result of the test #1 --------------------------------------------------------------------------- | device | sdb5 | sdb6 | sdb7 | | weight | 40 (57.0%) | 20 (29.0%) | 10 (14.0%) | |-----------------+-------------------+-------------------+-----------------| | I/Os (r/w) | 6640( 3272/ 3368)| 3434( 1719/ 1715)| 1689( 857/ 832)| | sectors (r/w) | 53120(26176/26944)| 27472(13752/13720)| 13512(6856/6656)| | ratio to total | 56.4% | 29.2% | 14.4% | --------------------------------------------------------------------------- The configurations of the test #2: o The configurations are the same as …
creator: Ryo Tsuruta
date: 2008-01-25T07:07:20-00:00

Re: [Xen-devel] dm-band: The I/O bandwidth controller: Performance Report
type: item
description: Hi, Ryo Tsuruta wrote: > The results of bandwidth control test on band-groups. > ===================================================== > The configurations of the test #3: > o Prepare three partitions sdb5 and sdb6. > o Create two extra band-groups on sdb5, the first is of user1 and the > second is of user2. > o Give weights of 40, 20, 10 and 10 to the user1 band-group, the user2 > band-group, the default group of sdb5 and sdb6 respectively. > o Run 128 processes issuing random read/write direct I/O with 4KB data > on each device at the same time. you mean that you run 128 processes on each user-device pairs? Namely, I guess that user1: 128 processes on sdb5, user2: 128 processes on sdb5, another: 128 processes on sdb5, user2: 128 processes on sdb6. > Conclusions and future works > ============================ > Dm-band works well with random I/Os. I have a plan on running some tests > using various real applications such as databases or file servers. > If you have any other good idea to test dm-band, please let me know. The second preliminary studies might be: - What if you use a different I/O size on each device (or device-user pair)? - What if you use a different number of processes on each device (or device-user pair)? And my impression is that it's natural dm-band is in device-mapper, separated from I/O scheduler. Because bandwidth control and I/O scheduling are two different things, it may be simpler that they are implemented in different layers. Regards, Hiroya. > > Thank you, > Ryo Tsuruta. > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > > _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: INAKOSHI Hiroya
date: 2008-01-29T06:42:17-00:00