OpenVZ Forum - RDF feed
type: channel
items: [RFC][PATCH] Per container statistics
items: Re: [RFC][PATCH] Per container statistics

[RFC][PATCH] Per container statistics
type: item
description: This patch is inspired by the discussion at http://lkml.org/lkml/2007/4/11/187 and implements per container statistics as suggested by Andrew Morton in http://lkml.org/lkml/2007/4/11/263. The patch is on top of 2.6.21-mm1 with Paul's containers v9 patches (forward ported) This patch implements per container statistics infrastructure and re-uses code from the taskstats interface. A new set of container operations are registered with commands and attributes. It should be very easy to extend per container statistics, by adding members to the containerstats structure. The current model for containerstats is a pull, a push model (to post statistics on interesting events), should be very easy to add. Currently user space requests for statistics by passing the container path, if a container is found to belong to the specified hierarchy, then statistics about the state of all the tasks in the container is returned to user space. TODO's 1. All data is shared as filesystem paths, it involves a lot of string and path name handling. Simplify and make the mechanism more elegant 2. Cache the entire container path instead of just the mount point path Feedback, comments, test results are always welcome! Signed-off-by: Balbir Singh <balbir@linux.vnet.ibm.com> --- Documentation/accounting/containerstats.txt | 27 ++++ include/linux/Kbuild | 1 include/linux/container.h | 10 + include/linux/containerstats.h | 72 ++++++++++++ include/linux/delayacct.h | 11 + kernel/container.c | 158 +++++++++++++++++++++++++++- kernel/cpuset.c | 2 kernel/sched.c | 4 kernel/taskstats.c | 70 ++++++++++++ 9 files changed, 349 insertions(+), 6 deletions(-) diff -puN kernel/container.c~containers-taskstats kernel/container.c --- linux-2.6.21-mm1/kernel/container.c~containers-taskstats 2007-05-11 08:50:29.000000000 +0530 +++ linux-2.6.21-mm1-balbir/kernel/container.c 2007-05-14 22:32:05.000000000 +0530 @@ -59,8 +59,11 @@ #include <asm/uaccess.h> #include <asm/atomic.h> #include <linux/mutex.h> +#include <linux…
creator: Balbir Singh
date: 2007-05-14T17:44:54-00:00

Re: [RFC][PATCH] Per container statistics
type: item
description: Balbir Singh wrote: > This patch is inspired by the discussion at http://lkml.org/lkml/2007/4/11/187 > and implements per container statistics as suggested by Andrew Morton > in http://lkml.org/lkml/2007/4/11/263. The patch is on top of 2.6.21-mm1 > with Paul's containers v9 patches (forward ported) > > This patch implements per container statistics infrastructure and re-uses > code from the taskstats interface. A new set of container operations are > registered with commands and attributes. It should be very easy to > extend per container statistics, by adding members to the containerstats > structure. > > The current model for containerstats is a pull, a push model (to post > statistics on interesting events), should be very easy to add. Currently > user space requests for statistics by passing the container path, if > a container is found to belong to the specified hierarchy, then statistics > about the state of all the tasks in the container is returned to user space. > > TODO's > > 1. All data is shared as filesystem paths, it involves a lot of string > and path name handling. Simplify and make the mechanism more elegant > 2. Cache the entire container path instead of just the mount point path > > Feedback, comments, test results are always welcome! > > Signed-off-by: Balbir Singh <balbir@linux.vnet.ibm.com> > --- > > Documentation/accounting/containerstats.txt | 27 ++++ > include/linux/Kbuild | 1 > include/linux/container.h | 10 + > include/linux/containerstats.h | 72 ++++++++++++ > include/linux/delayacct.h | 11 + > kernel/container.c | 158 +++++++++++++++++++++++++++- > kernel/cpuset.c | 2 > kernel/sched.c | 4 > kernel/taskstats.c | 70 ++++++++++++ > 9 files changed, 349 insertions(+), 6 deletions(-) > > diff -puN kernel/container.c~containers-taskstats kernel/container.c > --- linux-2.6.21-mm1/kernel/container.c~containers-taskstats 2007-05-11 08:50:29.000000000 +0530 > +++ linux-2.6.21-mm1-balbir/kernel/container.c 2007-05-14 22:32:05.000000000 +0530 > @@ …
creator: dev
date: 2007-05-15T07:12:12-00:00

Re: [RFC][PATCH] Per container statistics
type: item
description: Kirill Korotaev wrote: > oh, please no... Andrew, this loop can be very very long when having > 10,000 tasks on the machine... > we have had enough such issues in OpenVZ and just don't want to come through this again. > Also sum of RUNNING + UNINTERRUPTIBLE is required for per-container loadavg calculations. > Hi, Kirill, I've asked Paul Menage to help us with this scenario by adding a per-container task list. Would you be ok, if we walked the per-container task list. We could further optimize the routine by adding 1. A container id (avoid parsing file paths to uniquely identify a container) 2. If we cannot do (1), then cache container paths The other option is of-course what Pavel posted, but I think the amortized cost of tracking task state at the time of task state change or looping over tasks on demand should be the same. Do you agree in general with containerstats? How we build (walk the task list or at the time of task state change) is flexible with containerstats. Thanks for your comments! -- Warm Regards, Balbir Singh Linux Technology Center IBM, ISTL
creator: Balbir Singh
date: 2007-05-15T07:23:00-00:00

Re: [RFC][PATCH] Per container statistics
type: item
description: Hi Balbir, On 5/14/07, Balbir Singh <balbir@linux.vnet.ibm.com> wrote: > > This patch implements per container statistics infrastructure and re-uses > code from the taskstats interface. A new set of container operations are > registered with commands and attributes. It should be very easy to > extend per container statistics, by adding members to the containerstats > structure. Sorry for the delay in looking at this. (I've been travelling a bit). The basic idea of being able to get stats on a per-container basis seems good, but I've got some suggestions on the API/implementation: - saving a mount pointer in the containerfsroot structure won't work because a hierarchy (superblock) can be mounted in more than one place, or even in zero places (if you unmount a hierarchy with active containers, the superblock and the containers stay active). Also it might be possible to move a mounted container hierarchy via mount --move, although I've not actually tried that. - a cleaner way to pass in a container id would be to pass a file descriptor on a container directory. The dentry associated with this fd would unambiguously identify the hierarchy and the container, so then even if we didn't maintain a per-container task likst, the for_each_thread() loop would involve just a single comparison per task to see if the task was in the desired container. - if we're trying to integrate this with taskstats, then it would be nice to support retrieving all the other taskstats values (where they make sense) on a per-container basis (in the same way that we can currently request them on a per-thread or a per-process basis). i.e. don't create a new container_taskstats structure, but instead augment the current taskstats structure, and allow the user to retrieve the aggregate of that for the entire container. Similarly, being able to get taskstats notifications based on the container memberships of a task might be nice. Paul
creator: Paul Menage
date: 2007-05-21T20:56:31-00:00

Re: [RFC][PATCH] Per container statistics
type: item
description: Paul Menage wrote: > Hi Balbir, > > On 5/14/07, Balbir Singh <balbir@linux.vnet.ibm.com> wrote: >> >> This patch implements per container statistics infrastructure and re-uses >> code from the taskstats interface. A new set of container operations are >> registered with commands and attributes. It should be very easy to >> extend per container statistics, by adding members to the containerstats >> structure. > > Sorry for the delay in looking at this. (I've been travelling a bit). > > The basic idea of being able to get stats on a per-container basis > seems good, but I've got some suggestions on the API/implementation: > > - saving a mount pointer in the containerfsroot structure won't work > because a hierarchy (superblock) can be mounted in more than one > place, or even in zero places (if you unmount a hierarchy with active > containers, the superblock and the containers stay active). Also it > might be possible to move a mounted container hierarchy via mount > --move, although I've not actually tried that. Yes, Good catch! > > - a cleaner way to pass in a container id would be to pass a file > descriptor on a container directory. The dentry associated with this > fd would unambiguously identify the hierarchy and the container, so > then even if we didn't maintain a per-container task likst, the > for_each_thread() loop would involve just a single comparison per task > to see if the task was in the desired container. I thought about this approach, but did not implement the code this way because a system could have thousands of containers and expecting a statistics application to open a file descriptor each time for each container will turn out to be an expensive operation overhead = 2 syscalls (open + close) * number of containers * frequency of stats collections I guess for now this is a good choice > > - if we're trying to integrate this with taskstats, then it would be > nice to support retrieving all the other taskstats values (where they > make sense) on a …
creator: Balbir Singh
date: 2007-05-24T15:59:36-00:00

Re: [RFC][PATCH] Per container statistics
type: item
description: On 5/24/07, Balbir Singh <balbir@linux.vnet.ibm.com> wrote: > > I thought about this approach, but did not implement the code this way > because a system could have thousands of containers and expecting a > statistics application to open a file descriptor each time for each > container will turn out to be an expensive operation > > overhead = 2 syscalls (open + close) * number of containers * frequency > of stats collections But you can cache the file handles open between stats calls, assuming you're doing them repeatedly. Paul
creator: Paul Menage
date: 2007-05-24T16:12:41-00:00