OpenVZ Forum - RDF feed
type: channel
items: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
items: [PATCH 1/2] rcfs core patch
items: [PATCH 2/2] cpu_accounting controller
items: Re: [ckrm-tech] [PATCH 1/2] rcfs core patch
items: Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
items: Re: [ckrm-tech] [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
items: Re: [PATCH 1/2] rcfs core patch

[PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: Paul, Based on some of the feedback to container patches, I have respun them to avoid the "container" structure abstraction and instead use nsproxy structure in the kernel. User interface (which I felt was neat in your patches) has been retained to be same. What follows is the core (big) patch and the cpu_acct subsystem to serve as an example of how to use it. I suspect we can make cpusets also work on top of this very easily. Oh and since most of the code is serving the purpose of being a filesystem, I have renamed the patch to be a resource control file system - rcfs! -- Regards, vatsa _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Srivatsa Vaddagiri
date: 2007-03-01T13:35:43-00:00

[PATCH 1/2] rcfs core patch
type: item
description: Heavily based on Paul Menage's (inturn cpuset) work. The big difference is that the patch uses task->nsproxy to group tasks for resource control purpose (instead of task->containers). The patch retains the same user interface as Paul Menage's patches. In particular, you can have multiple hierarchies, each hierarchy giving a different composition/view of task-groups. (Ideally this patch should have been split into 2 or 3 sub-patches, but will do that on a subsequent version post) Signed-off-by : Srivatsa Vaddagiri <vatsa@in.ibm.com> Signed-off-by : Paul Menage <menage@google.com> --- linux-2.6.20-vatsa/include/linux/init_task.h | 4 linux-2.6.20-vatsa/include/linux/nsproxy.h | 5 linux-2.6.20-vatsa/init/Kconfig | 22 linux-2.6.20-vatsa/init/main.c | 1 linux-2.6.20-vatsa/kernel/Makefile | 1 --- diff -puN include/linux/init_task.h~rcfs include/linux/init_task.h --- linux-2.6.20/include/linux/init_task.h~rcfs 2007-03-01 14:20:47.000000000 +0530 +++ linux-2.6.20-vatsa/include/linux/init_task.h 2007-03-01 14:20:47.000000000 +0530 @@ -71,6 +71,16 @@ } extern struct nsproxy init_nsproxy; + +#ifdef CONFIG_RCFS +#define INIT_RCFS(nsproxy) \ + .list = LIST_HEAD_INIT(nsproxy.list), \ + .ctlr_data = {[ 0 ... CONFIG_MAX_RC_SUBSYS-1 ] = NULL }, +#else +#define INIT_RCFS(nsproxy) +#endif + + #define INIT_NSPROXY(nsproxy) { \ .pid_ns = &init_pid_ns, \ .count = ATOMIC_INIT(1), \ @@ -78,6 +88,7 @@ extern struct nsproxy init_nsproxy; .uts_ns = &init_uts_ns, \ .mnt_ns = NULL, \ INIT_IPC_NS(ipc_ns) \ + INIT_RCFS(nsproxy) \ } #define INIT_SIGHAND(sighand) { \ diff -puN include/linux/nsproxy.h~rcfs include/linux/nsproxy.h --- linux-2.6.20/include/linux/nsproxy.h~rcfs 2007-03-01 14:20:47.000000000 +0530 +++ linux-2.6.20-vatsa/include/linux/nsproxy.h 2007-03-01 14:20:47.000000000 +0530 @@ -28,6 +28,10 @@ struct nsproxy { struct ipc_namespace *ipc_ns; struct mnt_namespace *mnt_ns; struct pid_namespace *pid_ns; +#ifdef CONFIG_RCFS + struct list_head list; + void *ctlr_data[CONFIG_MAX_RC_SUBSYS]; …
creator: Srivatsa Vaddagiri
date: 2007-03-01T13:45:28-00:00

[PATCH 2/2] cpu_accounting controller
type: item
description: This patch demonstrates how a resource controller can work with rcfs. The controller counts the total CPU time used by all processes in a resource container, during the time that they're members of the container. Written by Paul Menage. Adapted to work with rcfs by Srivatsa. Signed-off-by : Paul Menage <menage@google.com> Signed-off-by : Srivatsa Vaddagiri <vatsa@in.ibm.com> diff -puN /dev/null include/linux/cpu_acct.h --- /dev/null 2006-02-25 03:06:56.000000000 +0530 +++ linux-2.6.20-vatsa/include/linux/cpu_acct.h 2007-03-01 16:53:39.000000000 +0530 @@ -0,0 +1,14 @@ + +#ifndef _LINUX_CPU_ACCT_H +#define _LINUX_CPU_ACCT_H + +#include <linux/rcfs.h> +#include <asm/cputime.h> + +#ifdef CONFIG_RC_CPUACCT +extern void cpuacct_charge(struct task_struct *, cputime_t cputime); +#else +static void inline cpuacct_charge(struct task_struct *p, cputime_t cputime) {} +#endif + +#endif diff -puN init/Kconfig~cpu_acct init/Kconfig --- linux-2.6.20/init/Kconfig~cpu_acct 2007-03-01 16:53:39.000000000 +0530 +++ linux-2.6.20-vatsa/init/Kconfig 2007-03-01 16:53:39.000000000 +0530 @@ -291,6 +291,13 @@ config SYSFS_DEPRECATED If you are using a distro that was released in 2006 or later, it should be safe to say N here. +config RC_CPUACCT + bool "Simple CPU accounting container subsystem" + select RCFS + help + Provides a simple Resource Controller for monitoring the + total CPU consumed by the tasks in a container + config RELAY bool "Kernel->user space relay support (formerly relayfs)" help diff -puN /dev/null kernel/cpu_acct.c --- /dev/null 2006-02-25 03:06:56.000000000 +0530 +++ linux-2.6.20-vatsa/kernel/cpu_acct.c 2007-03-01 16:53:39.000000000 +0530 @@ -0,0 +1,221 @@ +/* + * kernel/cpu_acct.c - CPU accounting container subsystem + * + * Copyright (C) Google Inc, 2006 + * + * Developed by Paul Menage (menage@google.com) and Balbir Singh + * (balbir@in.ibm.com) + * + */ + +/* + * Container subsystem for reporting total CPU usage of tasks in a + * container, along with percentage load …
creator: Srivatsa Vaddagiri
date: 2007-03-01T13:50:08-00:00

Re: [ckrm-tech] [PATCH 1/2] rcfs core patch
type: item
description: Srivatsa Vaddagiri wrote: > Heavily based on Paul Menage's (inturn cpuset) work. The big difference > is that the patch uses task->nsproxy to group tasks for resource control > purpose (instead of task->containers). > > The patch retains the same user interface as Paul Menage's patches. In > particular, you can have multiple hierarchies, each hierarchy giving a > different composition/view of task-groups. > > (Ideally this patch should have been split into 2 or 3 sub-patches, but > will do that on a subsequent version post) > With this don't we end up with a lot of duplicate between cpusets and rcfs. > Signed-off-by : Srivatsa Vaddagiri <vatsa@in.ibm.com> > Signed-off-by : Paul Menage <menage@google.com> > > > --- > > linux-2.6.20-vatsa/include/linux/init_task.h | 4 > linux-2.6.20-vatsa/include/linux/nsproxy.h | 5 > linux-2.6.20-vatsa/init/Kconfig | 22 > linux-2.6.20-vatsa/init/main.c | 1 > linux-2.6.20-vatsa/kernel/Makefile | 1 > > > --- The diffstat does not look quite right. -- Warm Regards, Balbir Singh Linux Technology Center IBM, ISTL _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Balbir Singh
date: 2007-03-02T05:06:49-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Fri, 02 Mar 2007 18:45:06 +0300 Kirill Korotaev <dev@openvz.org> wrote: > > I'm wagering you'll break either the semantics, and/or the > > performance, of cpusets doing this. > I like Paul's containers patch. It looks good and pretty well. > After some of the context issues are resolved it's fine. > Maybe it is even the best way of doing things. Have you thought about the relationship between it and UBC? _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: akpm
date: 2007-03-02T16:52:24-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: Andrew, >>>I'm wagering you'll break either the semantics, and/or the >>>performance, of cpusets doing this. >> >>I like Paul's containers patch. It looks good and pretty well. >>After some of the context issues are resolved it's fine. >>Maybe it is even the best way of doing things. > > > Have you thought about the relationship between it and UBC? Sure. Mostly containers patch does 2 things: 1. user space interfaces (if people don't like system calls used in UBC we are fine with filesystems approach. why not?) 2. context handling So (1) is ok with us. (2) requires some more work to be done before we are fine. Actually all we want is lockless context handling all over the code and looks like it is clear how to do it. Good. UBC on the other hand can provide containers subsystems, i.e. memory and other resources accounting and limiting etc. This is what we are working on right now. Thanks, Kirill _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: dev
date: 2007-03-02T17:25:36-00:00

Re: [ckrm-tech] [PATCH 1/2] rcfs core patch
type: item
description: On Fri, Mar 02, 2007 at 10:36:49AM +0530, Balbir Singh wrote: > With this don't we end up with a lot of duplicate between cpusets and rcfs. Unless we remove the duplication in cpusets and make it work with rcfs/containers! I wonder if we can avoid so much of filesystem code and use something like configfs to configure the resource objects. In particular I dont know if, using configfs, it is possible to provide a multi-hierarchy feature (each hierarchy bound to separate set of controllers) > > linux-2.6.20-vatsa/include/linux/init_task.h | 4 > > linux-2.6.20-vatsa/include/linux/nsproxy.h | 5 > > linux-2.6.20-vatsa/init/Kconfig | 22 > > linux-2.6.20-vatsa/init/main.c | 1 > > linux-2.6.20-vatsa/kernel/Makefile | 1 > > > > > >--- > > The diffstat does not look quite right. Hmm that was generated using refpatch. Will find out what went wrong. -- Regards, vatsa _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Srivatsa Vaddagiri
date: 2007-03-03T09:38:24-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: > Regarding semantics, can you be more specific? Unfortunately not - sorry. I've been off in other areas, and not found the time to read through this current PATCH or think about it carefully enough to be really useful. Your reply seemed reasonable enough. > It should have the same perf overhead as the original container patches > (basically a double dereference - task->containers/nsproxy->cpuset - > required to get to the cpuset from a task). There is just one spot that this might matter to cpusets. Except for one hook, cpusets uses the mems_allowed and cpus_allowed masks in the task struct to avoid having to look at the cpuset on hot code paths. There is one RCU guarded reference per memory allocation to current->cpuset->mems_generation in the call to cpuset_update_task_memory_state(), for tasks that are in some cpuset -other- than the default top cpuset, on systems that have explicitly created additional (other than the top cpuset) cpusets after boot. If that RCU guarded reference turned into taking a global lock, or pulling in a cache line that was frequently off dirty in some other node, that would be unfortunate. But that's the key hook so far as cpuset performance impact is concerned. Perhaps you could summarize what becomes of this hook, in this brave new world of rcfs ... -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson <pj@sgi.com> 1.925.600.0401 _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Paul Jackson
date: 2007-03-03T10:21:00-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: Herbert wrote: > I agree here, there is not much difference for the > following aspects: Whether two somewhat similar needs should be met by one shared mechanism, or two distinct mechanisms, cannot really be decided by listing the similarities. One has to determine if there are any significant differences in needs that are too difficult for a shared mechanism to provide. A couple of things you wrote in your second message might touch on such possible significant differences: - resources must be hierarchically suballocated, and - key resource management code hooks can't cause hot cache lines. In a later message, Herbert wrote: > well, the thing is, as nsproxy is working now, you > will get a new one (with a changed subset of entries) > every time a task does a clone() with one of the > space flags set, which means, that you will end up > with quite a lot of them, but resource limits have > to address a group of them, not a single nsproxy > (or act in a deeply hierarchical way which is not > there atm, and probably will never be, as it simply > adds too much overhead) I still can't claim to have my head around this, but what you write here, Herbert, writes here touches on what I suspect is a key difference between namespaces and resources that would make it impractical to accomplish both with a shared mechanism for aggregating tasks. It is a natural and desirable capability when managing resources, which are relatively scarce (that's why they're worth all this trouble) commodities of which we have some limited amount, to subdivide allowances of them. Some group of tasks gets the right to use certain memory pages or cpu time slices, and in turn suballocates that allotment to some subgroup of itself. This naturally leads to a hierarchy of allocated resources. There is no such necessary, hierarchy for name spaces. One name space might be derived from another at setup, by some arbitrary conventions, but once initialized, this way or that, they are separate name spaces, or…
creator: Paul Jackson
date: 2007-03-03T21:22:44-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Sat, Mar 03, 2007 at 02:21:00AM -0800, Paul Jackson wrote: > Perhaps you could summarize what becomes of this hook, in this > brave new world of rcfs ... attach_task() still uses a synchronize_rcu before doing a put_nsproxy in the rcfs patches. This means cpuset_update_task_memory_state() can read a task's cpuset->mems_generation under just a rcu_read_lock() (as it is doing currently). void cpuset_update_task_memory_state(void) { tsk = current; ... } else { rcu_read_lock(); ns = rcu_dereference(tsk->nsproxy); my_cpusets_mem_gen = ns->ctlr_data[cpuset_ctlr.subsys_id]->mems_generation; rcu_read_unlock(); } ... } -- Regards, vatsa _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Srivatsa Vaddagiri
date: 2007-03-05T17:02:44-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Sat, Mar 03, 2007 at 01:22:44PM -0800, Paul Jackson wrote: > I still can't claim to have my head around this, but what you write > here, Herbert, writes here touches on what I suspect is a key > difference between namespaces and resources that would make it > impractical to accomplish both with a shared mechanism for aggregating > tasks. The way nsproxy is structured, its all pointers to actual namespace (or in case of rcfs patch) resource objects. This lets namespaces objects be in a flat hierarchy while resource objects are in tree-like hierarchy. nsproxy itself doesnt decide any hierarchy. Its those objects pointed to by nsproxy which can form different hierarchies. In fact the rcfs patches allows such a combination afaics. > > on every limit accounting or check? I think that > > is quite a lot of overhead ... > > Do either of these dereferences require locks? A rcu_read_lock() should be required, which is not that expensive. -- Regards, vatsa _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Srivatsa Vaddagiri
date: 2007-03-05T17:47:50-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: Hi Vatsa, Sorry for the delayed reply - the last week has been very busy ... On 3/1/07, Srivatsa Vaddagiri <vatsa@in.ibm.com> wrote: > Paul, > Based on some of the feedback to container patches, I have > respun them to avoid the "container" structure abstraction and instead use > nsproxy structure in the kernel. User interface (which I felt was neat > in your patches) has been retained to be same. I'm not really sure that I see the value of having this be part of nsproxy rather than the previous independent container (and container_group) structure. As far as I can see, you're putting the container subsystem state pointers and the various task namespace pointers into the same structure (nsproxy) but then they're remaining pretty much independent in terms of code. The impression that I'm getting (correct me if I'm wrong) is: - when you do a mkdir within an rcfs directory, the nsproxy associated with the parent is duplicated, and then each rcfs subsystem gets to set a subsystem-state pointer in that nsproxy - when you move a task into an rcfs container, you create a new nsproxy consisting of the task's old namespaces and its new subsystem pointers. Then you look through the current list of nsproxy objects to see if you find one that matches. If you do, you reuse it, else you create a new nsproxy and link it into the list - when you do sys_unshare() or a clone that creates new namespaces, then the task (or its child) will get a new nsproxy that has the rcfs subsystem state associated with the old nsproxy, and one or more namespace pointers cloned to point to new namespaces. So this means that the nsproxy for the task is no longer the nsproxy associated with any directory in rcfs. (So the task will disappear from any "tasks" file in rcfs?) You seem to have lost some features, including fork/exit subsystem callbacks > > What follows is the core (big) patch and the cpu_acct subsystem to serve > as an example of how to use it. I suspect we can make cpusets also work > on t…
creator: Paul Menage
date: 2007-03-07T02:32:07-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On 3/7/07, Srivatsa Vaddagiri <vatsa@in.ibm.com> wrote: > > > - when you do sys_unshare() or a clone that creates new namespaces, > > then the task (or its child) will get a new nsproxy that has the rcfs > > subsystem state associated with the old nsproxy, and one or more > > namespace pointers cloned to point to new namespaces. So this means > > that the nsproxy for the task is no longer the nsproxy associated with > > any directory in rcfs. (So the task will disappear from any "tasks" > > file in rcfs?) > > it "should" disappear yes, although I haven't carefully studied the > unshare requirements yet. That seems bad. With the current way you're doing it, if I mount hierarchies A and B on /mnt/A and /mnt/B, then initially all tasks are in /mnt/A/tasks and /mnt/B/tasks. If I then create /mnt/A/foo and move a process into it, that process disappears from /mnt/B/tasks, since its nsproxy no longer matches the nsproxy of B's root container. Or am I missing something? Paul _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Paul Menage
date: 2007-03-07T17:29:12-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Tue, Mar 06, 2007 at 06:32:07PM -0800, Paul Menage wrote: > I'm not really sure that I see the value of having this be part of > nsproxy rather than the previous independent container (and > container_group) structure. *shrug* I wrote the patch mainly to see whether the stuff container folks (Sam Vilain et al) were complaining abt (that container structure abstraction inside the kernel is redundant/unnecessary) made sense or not. The rcfs patches demonstrate that it is possible to implement resource control on top of just nsproxy -and- give the same interface that you now have. In essense, I would say that the rcfs patches are about 70% same as your original V7 container patches. However as I am converting over cpusets to work on top of nsproxy, I have learnt few things: container structure in your patches provides for these things: a. A way to group tasks b. A way to maintain several hierarchies of such groups If you consider just a. then I agree that container abstraction is redundant, esp for vserver resource control (nsproxy can already be used to group tasks). What nsproxy doesn't provide is b - a way to represent hierarchies of groups. So we got several choices here. 1. Introduce the container abstraction as is in your patches 2. Extend nsproxy somehow to represent hierarchies 3. Let individual resource controllers that -actually- support hierarchical resource management maintain hierarchy in their code. In the last option, nsproxy still is unaware of any hierarchy. Some of the resource objects it points to (for ex: cpuset) may maintain a hierarchy. For ex: nsproxy->ctlr_data[cpuset_subsys.subsys_id] points to a 'struct cpuset' structure which could maintains the hierarchical relationship among cpuset objects. If we consider that most resource controllers may not implement hierarchical resource management, then 3 may not be a bad compromise. OTOH if we expect *most* resource controllers to support hierarchical resource management, then we could be better of…
creator: Srivatsa Vaddagiri
date: 2007-03-07T17:30:31-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Wed, Mar 07, 2007 at 11:00:31PM +0530, Srivatsa Vaddagiri wrote: > So we got several choices here. > > 1. Introduce the container abstraction as is in your patches > 2. Extend nsproxy somehow to represent hierarchies > 3. Let individual resource controllers that -actually- support > hierarchical resource management maintain hierarchy in their code. 3 is what I am following for the cpuset conversion (currently under test). -- Regards, vatsa _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Srivatsa Vaddagiri
date: 2007-03-07T17:32:26-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: Quoting Srivatsa Vaddagiri (vatsa@in.ibm.com): > On Tue, Mar 06, 2007 at 06:32:07PM -0800, Paul Menage wrote: > > I'm not really sure that I see the value of having this be part of > > nsproxy rather than the previous independent container (and > > container_group) structure. > > *shrug* > > I wrote the patch mainly to see whether the stuff container folks (Sam Vilain > et al) were complaining abt (that container structure abstraction > inside the kernel is redundant/unnecessary) made sense or not. I still think the complaint was about terminology, not implementation. They just didn't want you calling them containers. > The rcfs patches demonstrate that it is possible to implement resource control > on top of just nsproxy -and- give the same interface that you now > have. In essense, I would say that the rcfs patches are about 70% same as your > original V7 container patches. > > However as I am converting over cpusets to work on top of nsproxy, I > have learnt few things: > > container structure in your patches provides for these things: > > a. A way to group tasks > b. A way to maintain several hierarchies of such groups > > If you consider just a. then I agree that container abstraction is > redundant, esp for vserver resource control (nsproxy can already be used > to group tasks). > > What nsproxy doesn't provide is b - a way to represent hierarchies of > groups. > > So we got several choices here. > > 1. Introduce the container abstraction as is in your patches > 2. Extend nsproxy somehow to represent hierarchies > 3. Let individual resource controllers that -actually- support > hierarchical resource management maintain hierarchy in their code. > > In the last option, nsproxy still is unaware of any hierarchy. Some of > the resource objects it points to (for ex: cpuset) may maintain a > hierarchy. For ex: nsproxy->ctlr_data[cpuset_subsys.subsys_id] points to > a 'struct cpuset' structure which could maintains the hierarchical > relationship among cpuset objects…
creator: serue
date: 2007-03-07T17:43:46-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On 3/7/07, Serge E. Hallyn <serue@us.ibm.com> wrote: > Quoting Srivatsa Vaddagiri (vatsa@in.ibm.com): > > On Tue, Mar 06, 2007 at 06:32:07PM -0800, Paul Menage wrote: > > > I'm not really sure that I see the value of having this be part of > > > nsproxy rather than the previous independent container (and > > > container_group) structure. > > > > *shrug* > > > > I wrote the patch mainly to see whether the stuff container folks (Sam Vilain > > et al) were complaining abt (that container structure abstraction > > inside the kernel is redundant/unnecessary) made sense or not. > > I still think the complaint was about terminology, not implementation. No, Sam was saying that nsproxy should be the object that all resource controllers hook off. Paul _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Paul Menage
date: 2007-03-07T17:46:35-00:00

Re: [ckrm-tech] [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Wed, Mar 07, 2007 at 09:29:12AM -0800, Paul Menage wrote: > That seems bad. With the current way you're doing it, if I mount > hierarchies A and B on /mnt/A and /mnt/B, then initially all tasks are > in /mnt/A/tasks and /mnt/B/tasks. If I then create /mnt/A/foo and move > a process into it, that process disappears from /mnt/B/tasks, since > its nsproxy no longer matches the nsproxy of B's root container. Or am > I missing something? I realized that bug as I was doing cpuset conversion. Basically, we can't use just tsk->nsproxy to find what tasks are in a directory (/mnt/B for ex). Here's what I was think we should be doing instead: struct nsproxy *ns; void *data; ns = dentry_of(/mnt/B/tasks)->d_parent->d_fsdata; data = ns->ctlr_data[some subsystem id which is bound in /mnt/B hierarchy] we now scan tasklist and find a match if: tsk->nsproxy->ctlr_data[the above id] == data (maybe we need to match on all data from all subsystems bound to B) There is a similar bug in rcfs_rmdir also. We can't just use the nsproxy pointed to by dentry to know whether the resource objects are free or not. I am thinking (if at all resource control has to be provided on top of nsproxy) that we should have a get_res_ns, similar to get_mnt_ns or get_uts_ns, which will track number of nsproxies pointing to the same resource object. If we do that, then rmdir() needs to go and check those resource object's refcounts to see if a dir is in use or not. -- Regards, vatsa _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Srivatsa Vaddagiri
date: 2007-03-07T17:52:57-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Wed, Mar 07, 2007 at 11:43:46AM -0600, Serge E. Hallyn wrote: > I still think the complaint was about terminology, not implementation. I don't think that is what http://lkml.org/lkml/2007/2/12/426 conveyed! > They just didn't want you calling them containers. Yes that too. > > Anyway, summarizing on "why nsproxy", the main point (I think) is about > > using existing abstraction in the kernel. s/abstraction/"implementation detail" then :) > But nsproxy is not an abstraction, it's an implementation > detail/optimization. -- Regards, vatsa _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Srivatsa Vaddagiri
date: 2007-03-07T18:00:55-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: Quoting Srivatsa Vaddagiri (vatsa@in.ibm.com): > On Wed, Mar 07, 2007 at 11:43:46AM -0600, Serge E. Hallyn wrote: > > I still think the complaint was about terminology, not implementation. > > I don't think that is what http://lkml.org/lkml/2007/2/12/426 conveyed! I don't have that in my inbox anymore so won't reply to it itself unfortunately, but what it conveyed is also not that nsproxy should be the 'resource control' object. If anything it seems to argue that all of Paul's patchset should be done in userspace. Sam writes > That's a great idea for a set of > tightly integrated userland utilities to simplify the presentation to > the admin, but I don't see why you need to enshrine this in the kernel. > Certainly not for any of the other patches in your set as far as I can > see. I disagree. Sam, there are two very separate concepts here. Actually three. What you had originally presented in a patchset was resource virtualization: so when process A asks for some resource X, rather than get resource_table[X] he gets resource_table[hash(x)]. The concensus you mention at the start, and which you say Eric was arguing for, was to not do such translation, but just get rid of the global 'resource_table'. By allowing processes to have and manipulate their own private resource_table, you implement namespaces. And as I've said, the nsproxy is just an implementation detail to keep namespace pointers out of the task struct. Using nsproxy or not has nothing to do with the namespace approach versus global resource tables with id translation at all userspace->kernel boundaries. So virtualization via explicit translation through global resource tables is one concept, and private namespaces could be said to be a second. The third is resource controls, which Paul's container patchset implements. It has nothing to do with the previous two, and it is what Paul's patches are addressing. Sam asks: > Ask yourself this - what do you need the container structure for so > badly, that virtual…
creator: serue
date: 2007-03-07T20:58:46-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On 3/7/07, Serge E. Hallyn <serue@us.ibm.com> wrote: > > All that being said, if it were going to save space without overly > complicating things I'm actually not opposed to using nsproxy, but it If space-saving is the main issue, then the latest version of my containers patches uses just a single pointer in the task_struct, and all tasks in the same set of containers (across all hierarchies) will share a single container_group object, which holds the actual pointers to container state. Effectively, container_group is to container as nsproxy is to namespace. Paul _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Paul Menage
date: 2007-03-07T21:20:18-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: Quoting Paul Menage (menage@google.com): > On 3/7/07, Serge E. Hallyn <serue@us.ibm.com> wrote: > > > >All that being said, if it were going to save space without overly > >complicating things I'm actually not opposed to using nsproxy, but it > > If space-saving is the main issue, then the latest version of my Space saving was the only reason for nsproxy to exist. Now of course it also provides the teensiest reduction in # instructions since every clone results in just one reference count inc for the nsproxy rather than one for each namespace. > containers patches uses just a single pointer in the task_struct, and > all tasks in the same set of containers (across all hierarchies) will > share a single container_group object, which holds the actual pointers > to container state. Yes, that's why this consolidation doesn't make sense to me. Especially considering again that we will now have nsproxies pointing to containers pointing to... nsproxies. > Effectively, container_group is to container as nsproxy is to namespace. > > Paul _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: serue
date: 2007-03-07T21:59:19-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Wed, 2007-03-07 at 15:59 -0600, Serge E. Hallyn wrote: > Space saving was the only reason for nsproxy to exist. > > Now of course it also provides the teensiest reduction in # instructions > since every clone results in just one reference count inc for the > nsproxy rather than one for each namespace. If we have 7 or 8 namespaces, then it can save us a significant number of atomic instructions on _each_ of the refcounts, plus touching all of the cachelines, etc... -- Dave _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Dave Hansen
date: 2007-03-07T22:13:46-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: "Paul Menage" <menage@google.com> writes: > On 3/7/07, Serge E. Hallyn <serue@us.ibm.com> wrote: >> >> All that being said, if it were going to save space without overly >> complicating things I'm actually not opposed to using nsproxy, but it > > If space-saving is the main issue, then the latest version of my > containers patches uses just a single pointer in the task_struct, and > all tasks in the same set of containers (across all hierarchies) will > share a single container_group object, which holds the actual pointers > to container state. Yes. However: > Effectively, container_group is to container as nsproxy is to namespace. The statement above nicely summarizes the confusion in terminology. In the namespace world when we say container we mean roughly at the level of nsproxy and container_group. Although it is expected to be a user space concept like an application, not a concept implemented directly in the kernel. i.e. User space is expected to combine separate resource controls and namespaces and run processes inside that combination. You are calling something that is on par with a namespace a container. Which seriously muddies the waters. About as much as calling as referring to your shoe as your whole outfit. Without fixing the terminology it is going to be very hard to successfully communicate. Eric _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: ebiederm
date: 2007-03-07T22:32:48-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: Dave Hansen <hansendc@us.ibm.com> writes: > On Wed, 2007-03-07 at 15:59 -0600, Serge E. Hallyn wrote: >> Space saving was the only reason for nsproxy to exist. >> >> Now of course it also provides the teensiest reduction in # instructions >> since every clone results in just one reference count inc for the >> nsproxy rather than one for each namespace. > > If we have 7 or 8 namespaces, then it can save us a significant number > of atomic instructions on _each_ of the refcounts, plus touching all of > the cachelines, etc... Well we still have a global lock on the fork path so there is only so much we can do to improve things. The global process list, and there are some interesting posix signal handling rules that limit how much we can relax that restriction. However with namespaces we have a natural limit on how many we will have. There aren't that many spaces for global names. I don't know the situation well enough for resource controllers but I suspect we might not have any kind of natural limit (except what a single person can comprehend) to the kind of resource we will ultimately want to control which tends to imply we will have more of those. Eric _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: ebiederm
date: 2007-03-07T23:13:15-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: "Paul Menage" <menage@google.com> writes: > No, Sam was saying that nsproxy should be the object that all resource > controllers hook off. I think implementation wise this tends to make sense. However it should have nothing to do with semantics. If we have a lot of independent resource controllers. Placing the pointer to their data structures directly in nsproxy instead of in task_struct sounds like a reasonable idea but it should not be user visible. Eric _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: ebiederm
date: 2007-03-07T23:16:00-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On 3/7/07, Eric W. Biederman <ebiederm@xmission.com> wrote: > > Effectively, container_group is to container as nsproxy is to namespace. > > The statement above nicely summarizes the confusion in terminology. > > In the namespace world when we say container we mean roughly at the level > of nsproxy and container_group. So you're saying that a task can only be in a single system-wide container. My patch provides multiple potentially-independent ways of dividing up the tasks on the system - if the "container" is the set of all divisions that the process is in, what's an appropriate term for the sub-units? > > You are calling something that is on par with a namespace a container. Yes. > Which > seriously muddies the waters. About as much as calling as referring to your > shoe as your whole outfit. > Without fixing the terminology it is going to be very hard to > successfully communicate. That assumes the viewpoint that your terminology is "correct" and other people's needs "fixing". :-) But as I've said I'm not particularly wedded to the term "container" if that really turned out to be what's blocking acceptance from people like Andrew or Linus. Do you have a suggestion for a better name? To me, "process container" seems like the ideal name, since it's an abstraction that "contains" processes and associates them with some (subsystem-provided) state. Paul _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Paul Menage
date: 2007-03-07T23:18:25-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: Paul Menage wrote: >> In the namespace world when we say container we mean roughly at the level >> of nsproxy and container_group. >> > So you're saying that a task can only be in a single system-wide container. > Nope, we didn't make the mistake of nailing down what a "container" was too far before it is implemented. We talked before about containers-within-containers because, inevitably if you provide a feature you'll end up having to deal with virtualising systems that in turn use that feature. > My patch provides multiple potentially-independent ways of dividing up > the tasks on the system - if the "container" is the set of all > divisions that the process is in, what's an appropriate term for the > sub-units? > namespace, since 2.4.x > That assumes the viewpoint that your terminology is "correct" and > other people's needs "fixing". :-) > Absolutely. Please respect the semantics established so far; changing them adds nothing at the cost of much confusion. > But as I've said I'm not particularly wedded to the term "container" > if that really turned out to be what's blocking acceptance from people > like Andrew or Linus. Do you have a suggestion for a better name? To > me, "process container" seems like the ideal name, since it's an > abstraction that "contains" processes and associates them with some > (subsystem-provided) state. > It's not even really the term, it's the semantics. Sam. _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Sam Vilain
date: 2007-03-08T00:35:46-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On 3/7/07, Sam Vilain <sam@vilain.net> wrote: > Paul Menage wrote: > >> In the namespace world when we say container we mean roughly at the level > >> of nsproxy and container_group. > >> > > So you're saying that a task can only be in a single system-wide container. > > > > Nope, we didn't make the mistake of nailing down what a "container" was > too far before it is implemented. We talked before about > containers-within-containers because, inevitably if you provide a > feature you'll end up having to deal with virtualising systems that in > turn use that feature. Sure, my aproach allows containers hierarchically as children of other containers too. > > > My patch provides multiple potentially-independent ways of dividing up > > the tasks on the system - if the "container" is the set of all > > divisions that the process is in, what's an appropriate term for the > > sub-units? > > > > namespace, since 2.4.x > > > That assumes the viewpoint that your terminology is "correct" and > > other people's needs "fixing". :-) > > > > Absolutely. Please respect the semantics established so far; changing > them adds nothing at the cost of much confusion. But "namespace" has well-established historical semantics too - a way of changing the mappings of local names to global objects. This doesn't describe things liek resource controllers, cpusets, resource monitoring, etc. Trying to extend the well-known term namespace to refer to things that aren't namespaces isn't a useful approach, IMO. Paul _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Paul Menage
date: 2007-03-08T00:42:05-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: Srivatsa Vaddagiri wrote: > container structure in your patches provides for these things: > > a. A way to group tasks > b. A way to maintain several hierarchies of such groups > > If you consider just a. then I agree that container abstraction is > redundant, esp for vserver resource control (nsproxy can already be used > to group tasks). > > What nsproxy doesn't provide is b - a way to represent hierarchies of > groups. > Well, that's like saying you can't put hierarchical data in a relational database. The hierarchy question is an interesting one, though. However I believe it first needs to be broken down into subsystems and considered on a subsystem-by-subsystem basis again, and if general patterns are observed, then a common solution should stand out. Let's go back to the namespaces we know about and discuss how hierarchies apply to them. Please those able to brainstorm, do so - I call green hat time. 1. UTS namespaces Can a UTS namespace set any value it likes? Can you inspect or set the UTS namespace values of a subservient UTS namespace? 2. IPC namespaces Can a process in an IPC namespace send a signal to those in a subservient namespace? 3. PID namespaces Can a process in a PID namespace see the processes in a subservient namespace? Do the processes in a subservient namespace appear in a higher level namespace mapped to a different set of PIDs? 4. Filesystem namespaces Can we see all of the mounts in a subservient namespace? Does our namespace receive updates when their namespace mounts change? (perhaps under a sub-directory) 5. L2 network namespaces Can we see or alter the subservient network namespace's interfaces/iptables/routing? Are any of the subservient network namespace's interfaces visible in our namespace, and by which mapping? 6. L3 network namespaces Can we bind to a subservient network namespace's addresses? Can we give or remove addresses to and from the subservient network namespace's namespace? Can we allow the namespace access to modify par…
creator: Sam Vilain
date: 2007-03-08T00:50:01-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: Paul Menage wrote: > But "namespace" has well-established historical semantics too - a way > of changing the mappings of local names to global objects. This > doesn't describe things liek resource controllers, cpusets, resource > monitoring, etc. > > Trying to extend the well-known term namespace to refer to things that > aren't namespaces isn't a useful approach, IMO. > > Paul > But "namespace" has well-established historical semantics too - a way of changing the mappings of local * to global objects. This accurately describes things liek resource controllers, cpusets, resource monitoring, etc. Trying to extend the well-known term namespace to refer to things that are semantically equivalent namespaces is a useful approach, IMHO. Sam. _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Sam Vilain
date: 2007-03-08T00:53:05-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: Srivatsa Vaddagiri <vatsa@in.ibm.com> writes: > Heavily based on Paul Menage's (inturn cpuset) work. The big difference > is that the patch uses task->nsproxy to group tasks for resource control > purpose (instead of task->containers). > > The patch retains the same user interface as Paul Menage's patches. In > particular, you can have multiple hierarchies, each hierarchy giving a > different composition/view of task-groups. > > (Ideally this patch should have been split into 2 or 3 sub-patches, but > will do that on a subsequent version post) After looking at the discussion that happened immediately after this was posted this feels like the right general direction to get the different parties talking to each other. I'm not convinced about the whole idea yet but this looks like a step in a useful direction. I have a big request. Please next time this kind of patch is posted add a description of what is happening and why. I have yet to see people explain why this is a good idea. Why the current semantics were chosen. The review is still largely happening at the why level but no one is addressing that yet. So please can we have a why. I have a question? What does rcfs look like if we start with the code that is in the kernel? That is start with namespaces and nsproxy and just build a filesystem to display/manipulate them? With the code built so it will support adding resource controllers when they are ready? > Signed-off-by : Srivatsa Vaddagiri <vatsa@in.ibm.com> > Signed-off-by : Paul Menage <menage@google.com> > > > --- > > linux-2.6.20-vatsa/include/linux/init_task.h | 4 > linux-2.6.20-vatsa/include/linux/nsproxy.h | 5 > linux-2.6.20-vatsa/init/Kconfig | 22 > linux-2.6.20-vatsa/init/main.c | 1 > linux-2.6.20-vatsa/kernel/Makefile | 1 > > > --- > > diff -puN include/linux/nsproxy.h~rcfs include/linux/nsproxy.h > --- linux-2.6.20/include/linux/nsproxy.h~rcfs 2007-03-01 14:20:47.000000000 > +0530 > +++ linux-2.6.20-vatsa/include/linux/nsproxy.h 2007-03-01 14:20:47.000…
creator: ebiederm
date: 2007-03-08T03:12:00-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: On 3/7/07, Eric W. Biederman <ebiederm@xmission.com> wrote: > > Please next time this kind of patch is posted add a description of > what is happening and why. I have yet to see people explain why > this is a good idea. Why the current semantics were chosen. OK. I thought that the descriptions in my last patch 0/7 and Documentation/containers.txt gave a reasonable amount of "why", but I can look at adding more details. > > I have a question? What does rcfs look like if we start with > the code that is in the kernel? That is start with namespaces > and nsproxy and just build a filesystem to display/manipulate them? > With the code built so it will support adding resource controllers > when they are ready? There's at least one resource controller that's already in the kernel - cpusets. > We probably want to rename this struct task_proxy.... > And then we can rename most of the users things like: > dup_task_proxy, clone_task_proxy, get_task_proxy, free_task_proxy, > put_task_proxy, exit_task_proxy, init_task_proxy.... That could be a good start. > > This extra list of nsproxy's is unneeded and a performance problem the > way it is used. In general we want to talk about the individual resource > controllers not the nsproxy. There's one important reason why it's needed, and highlights one of the ways that "resource controllers" are different from the way that "namespaces" have currently been used. Currently with a namespace, you can only unshare, either by sys_unshare() or clone() - you can't "reshare" a namespace with some other task. But resource controllers tend to have the concept a lot more of being able to move between resource classes. If you're going to have an ns_proxy/container_group object that gathers together a group of pointers to namespaces/subsystem-states, then either: 1) you only allow a task to reshare *all* namespaces/subsystems with another task, i.e. you can update current->task_proxy to point to other->task_proxy. But that restricts flexibility of …
creator: Paul Menage
date: 2007-03-08T09:10:24-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: On Wed, Mar 07, 2007 at 08:12:00PM -0700, Eric W. Biederman wrote: > The review is still largely happening at the why level but no > one is addressing that yet. So please can we have a why. Here's a brief summary of what's happening and why. If its not clear, pls get back to us with specific questions. There have been various projects attempting to provide resource management support in Linux, including CKRM/Resource Groups and UBC. Each had its own task-grouping mechanism. Paul Menage observed [1] that cpusets in the kernel already has a grouping mechanism which was working well for cpusets. He went ahead and generalized the grouping code in cpusets so that it could be used for overall resource management purpose. With his patches, it is possible to even create multiple hierarchies of groups (see [2] on why multiple hierarchies) as follows: mount -t container -o cpuset none /dev/cpuset <- cpuset hierarchy mount -t container -o mem,cpu none /dev/mem <- memory/cpu hierarchy mount -t container -o disk none /dev/disk <- disk hierarchy In each hierarchy, you can create task groups and manipulate the resource parameters of each group. You can also move tasks between groups at run-time (see [3] on why this is required). Each hierarchy is also manipulated independent of the other. Paul's patches also introduced a 'struct container' in the kernel, which serves these key purposes: - Task-grouping 'struct container' represents a task-group created in each hierarchy. So every directory created under /dev/cpuset or /dev/mem above will have a corresponding 'struct container' inside the kernel. All tasks pointing to the same 'struct container' are considered to be part of a group The 'struct container' in turn has pointers to resource objects which store actual resource parameters for that group. In above example, 'struct container' created under /dev/cpuset will have a pointer to 'struct cpuset' while 'struct container' created under /dev/disk will have pointer to 'struct disk_q…
creator: Srivatsa Vaddagiri
date: 2007-03-08T10:13:47-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Thu, Mar 08, 2007 at 01:50:01PM +1300, Sam Vilain wrote: > 7. resource namespaces It should be. Imagine giving 20% bandwidth to a user X. X wants to divide this bandwidth further between multi-media (10%), kernel compilation (5%) and rest (5%). So, > Is the subservient namespace's resource usage counting against ours too? Yes, the resource usage of children should be accounted when capping parent resource usage. > Can we dynamically alter the subservient namespace's resource allocations? Should be possible yes. That lets user X completely manage his allocation among whatever sub-groups he creates. > So let's bring this back to your patches. If they are providing > visibility of ns_proxy, then it should be called namesfs or some such. The patches should give visibility to both nsproxy objects (by showing what tasks share the same nsproxy objects and letting tasks move across nsproxy objects if allowed) and the resource control objects pointed to by nsproxy (struct cpuset, struct cpu_limit, struct rss_limit etc). > It doesn't really matter if processes disappear from namespace > aggregates, because that's what's really happening anyway. The only > problem is that if you try to freeze a namespace that has visibility of > things at this level, you might not be able to reconstruct the > filesystem in the same way. This may or may not be considered a problem, > but open filehandles and directory handles etc surviving a freeze/thaw > is part of what we're trying to achieve. Then again, perhaps some > visibility is better than none for the time being. > > If they are restricted entirely to resource control, then don't use the > nsproxy directly - use the structure or structures which hang off the > nsproxy (or even task_struct) related to resource control. -- Regards, vatsa _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Srivatsa Vaddagiri
date: 2007-03-08T11:30:54-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Wed, Mar 07, 2007 at 04:16:00PM -0700, Eric W. Biederman wrote: > I think implementation wise this tends to make sense. > However it should have nothing to do with semantics. > > If we have a lot of independent resource controllers. Placing the > pointer to their data structures directly in nsproxy instead of in > task_struct sounds like a reasonable idea Thats what the rcfs patches do. > but it should not be user visible. What do you mean by this? We do want the user to be able to manipulate the resource parameters (which are normally present in the data structures/resource objects pointed to by nsproxy - nsproxy->ctlr_data[]) -- Regards, vatsa _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Srivatsa Vaddagiri
date: 2007-03-08T11:39:41-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Wed, Mar 07, 2007 at 11:00:31PM +0530, Srivatsa Vaddagiri wrote: > > I'd like to see that. I suspect it will be a bit more fiddly than the > > simple cpu_acct subsystem. > > I am almost done with the conversion. And yes cpuset is a beast to > convert over! Will test and send the patches out tomorrow. Ok ..I am not in a state yet where I can post the patches to lkml in the usual conventions (breaking down neatly/good documentation etc). But I do have something which seems to work! I could mount cpuset as: mount -t rcfs -ocpuset none cpuset cd cpuset mkdir a cd a cat tasks # shows nothing echo 7 > cpus echo 0 > mems echo 1 > cpu_exclusive echo some_pid > tasks cat tasks # shows some_pid top now shows some_pid running on CPU7, as expected :) Instead of the usual convention of inlining patches and sending them in separate mails, I am sending all of them as attachments (beware, bugs around!). But this gives you an idea on which direction this is proceeding .. Todo: - Introduce refcounting of resource objects (get/put_res_ns) - rmdir needs to check resource object refcount rather than nsproxy's - Trace couple of other lockdep warnings I have hit Patches attached. -- Regards, vatsa --- linux-2.6.20-vatsa/include/linux/init_task.h | 11 linux-2.6.20-vatsa/include/linux/nsproxy.h | 11 linux-2.6.20-vatsa/init/Kconfig | 22 linux-2.6.20-vatsa/init/main.c | 3 linux-2.6.20-vatsa/kernel/Makefile | 1 --- linux-2.6.20.1-vatsa/include/linux/init_task.h | 11 linux-2.6.20.1-vatsa/include/linux/nsproxy.h | 11 linux-2.6.20.1-vatsa/include/linux/rcfs.h | 76 + linux-2.6.20.1-vatsa/init/Kconfig | 22 linux-2.6.20.1-vatsa/init/main.c | 3 linux-2.6.20.1-vatsa/kernel/Makefile | 1 linux-2.6.20.1-vatsa/kernel/nsproxy.c | 65 + linux-2.6.20.1-vatsa/kernel/rcfs.c | 1202 +++++++++++++++++++++++++ 8 files changed, 1391 insertions(+) diff -puN include/linux/init_task.h~rcfs include/linux/init_task.h --- linux-2.6.20.1/include/linux/init_task.h~rcfs 2007-03-08 21:21:33.000000000 +0530 +++ linux-2.6.20…
creator: Srivatsa Vaddagiri
date: 2007-03-08T18:13:53-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: On Thu, Mar 08, 2007 at 01:10:24AM -0800, Paul Menage wrote: > On 3/7/07, Eric W. Biederman <ebiederm@xmission.com> wrote: > > > > Please next time this kind of patch is posted add a description of > > what is happening and why. I have yet to see people explain why > > this is a good idea. Why the current semantics were chosen. > > OK. I thought that the descriptions in my last patch 0/7 and > Documentation/containers.txt gave a reasonable amount of "why", but I > can look at adding more details. > > > > > I have a question? What does rcfs look like if we start with > > the code that is in the kernel? That is start with namespaces > > and nsproxy and just build a filesystem to display/manipulate them? > > With the code built so it will support adding resource controllers > > when they are ready? > > There's at least one resource controller that's already in the kernel - cpusets. > > > We probably want to rename this struct task_proxy.... > > And then we can rename most of the users things like: > > dup_task_proxy, clone_task_proxy, get_task_proxy, free_task_proxy, > > put_task_proxy, exit_task_proxy, init_task_proxy.... > > That could be a good start. > > > > > This extra list of nsproxy's is unneeded and a performance problem the > > way it is used. In general we want to talk about the individual resource > > controllers not the nsproxy. > > There's one important reason why it's needed, and highlights one of > the ways that "resource controllers" are different from the way that > "namespaces" have currently been used. > > Currently with a namespace, you can only unshare, either by > sys_unshare() or clone() - you can't "reshare" a namespace with some > other task. But resource controllers tend to have the concept a lot > more of being able to move between resource classes. If you're going > to have an ns_proxy/container_group object that gathers together a > group of pointers to namespaces/subsystem-states, then either: > > 1) you only allow a task to reshare *all*…
creator: Herbert Poetzl
date: 2007-03-09T00:38:19-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: On Thu, Mar 08, 2007 at 03:43:47PM +0530, Srivatsa Vaddagiri wrote: > On Wed, Mar 07, 2007 at 08:12:00PM -0700, Eric W. Biederman wrote: > > The review is still largely happening at the why level but no > > one is addressing that yet. So please can we have a why. > > Here's a brief summary of what's happening and why. If its not clear, > pls get back to us with specific questions. > > There have been various projects attempting to provide resource > management support in Linux, including CKRM/Resource Groups and UBC. let me note here, once again, that you forgot Linux-VServer which does quite non-intrusive resource management ... > Each had its own task-grouping mechanism. the basic 'context' (pid space) is the grouping mechanism we use for resource management too > Paul Menage observed [1] that cpusets in the kernel already has a > grouping mechanism which was working well for cpusets. He went ahead > and generalized the grouping code in cpusets so that it could be used > for overall resource management purpose. > With his patches, it is possible to even create multiple hierarchies > of groups (see [2] on why multiple hierarchies) as follows: do we need or even want that? IMHO the hierarchical concept CKRM was designed with, was also the reason for it being slow, unuseable and complicated > mount -t container -o cpuset none /dev/cpuset <- cpuset hierarchy > mount -t container -o mem,cpu none /dev/mem <- memory/cpu hierarchy > mount -t container -o disk none /dev/disk <- disk hierarchy > > In each hierarchy, you can create task groups and manipulate the > resource parameters of each group. You can also move tasks between > groups at run-time (see [3] on why this is required). > Each hierarchy is also manipulated independent of the other. > Paul's patches also introduced a 'struct container' in the kernel, > which serves these key purposes: > > - Task-grouping > 'struct container' represents a task-group created in each hierarchy. > So every directory created under /…
creator: Herbert Poetzl
date: 2007-03-09T00:48:16-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Thu, Mar 08, 2007 at 05:00:54PM +0530, Srivatsa Vaddagiri wrote: > On Thu, Mar 08, 2007 at 01:50:01PM +1300, Sam Vilain wrote: > > 7. resource namespaces > > It should be. Imagine giving 20% bandwidth to a user X. X wants to > divide this bandwidth further between multi-media (10%), kernel > compilation (5%) and rest (5%). So, sounds quite nice, but ... > > Is the subservient namespace's resource usage counting against ours too? > > Yes, the resource usage of children should be accounted when capping > parent resource usage. it will require to do accounting many times (and limit checks of course), which in itself might be a way to DoS the kernel by creating more and more resource groups > > > Can we dynamically alter the subservient namespace's resource > > allocations? > > Should be possible yes. That lets user X completely manage his > allocation among whatever sub-groups he creates. what happens if the parent changes, how is the resource change (if it was a reduction) propagated to the children? e.g. your guest has 1024 file handles, now you reduce it to 512, but the guest had two children, both with 256 file handles each ... > > So let's bring this back to your patches. If they are providing > > visibility of ns_proxy, then it should be called namesfs or some > > such. > > The patches should give visibility to both nsproxy objects (by showing > what tasks share the same nsproxy objects and letting tasks move across > nsproxy objects if allowed) and the resource control objects pointed to > by nsproxy (struct cpuset, struct cpu_limit, struct rss_limit etc). the nsproxy is not really relevant, as it is some kind of strange indirection, which does not necessarily depict the real relations, regardless wether you do the re-sharing of those nsproies or not .. let me know if you need examples to verify that ... best, Herbert > > It doesn't really matter if processes disappear from namespace > > aggregates, because that's what's really happening anyway. The only > > …
creator: Herbert Poetzl
date: 2007-03-09T01:16:08-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: Herbert wrote: > why is the filesystem approach so favored for this > kind of manipulations? I don't have any clear sense of whether the additional uses of file systems being considered here are a good idea or not, but the use of a file system for cpusets has turned out quite well, in my (vain and biased ;) view. Cpusets are subsets of the CPUs and memory nodes on a system. These subsets naturally form a partial ordering, where one cpuset is below another if its CPUs and nodes are a subset of the other ones. This forms a natural hierarchical space. It is quite convenient to be able to add names and file system like attributes, so that one can do things like -name- the set of CPUs to which you are attaching a job, as in "this job is to run on the CPUs in cpuset /foo/bar", and to further have file system like permissions on these subsets, to control who can access or modify them. For such hierarchical data structures, especially ones where names and permissions are useful, file systems are a more natural interface than traditional system call usage patterns. The key, in my view, is the 'shape' of the data. If the data schema is basically a single table, with uniform rows having a few fields each, where each field is a simple integer or string (not a fancy formatted string encoding some more elaborate shape) then classic system call patterns work well. If the schema is tree shaped, and especially if the usual file system attributes such as a hierarchical name space and permissions are useful, then a file system based API is probably best. -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson <pj@sgi.com> 1.925.600.0401 _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Paul Jackson
date: 2007-03-09T02:35:07-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: > But "namespace" has well-established historical semantics too - a way > of changing the mappings of local * to global objects. This > accurately describes things liek resource controllers, cpusets, resource > monitoring, etc. No! Cpusets don't rename or change the mapping of objects. I suspect you seriously misunderstand cpusets and are trying to cram them into a 'namespace' remapping role into which they don't fit. So far as cpusets are concerned, CPU #17 is CPU #17, for all tasks, regardless of what cpuset they are in. They just might not happen to be allowed to execute on CPU #17 at the moment, because that CPU is not allowed by the cpuset they are in. But they still call it CPU #17. Similary the namespace of cpusets and of tasks (pid's) are single system-wide namespaces, so far as cpusets are concerned. Cpusets are not about alternative or multiple or variant name spaces. They are about (considering just CPUs for the moment): 1) creating a set of maps M0, M1, ... from the set of CPUs to a Boolean, 2) creating a mapping Q from the set of tasks to these M0, ... maps, and 3) imposing constraints on where tasks can run, as follows: For any task t, that task is allowed to run on CPU x iff Q(t)(x) is True. Here, Q(t) will be one of the maps M0, ... aka a cpuset. So far as cpusets are concerned, there is only one each of: A] a namespace numbering CPUs, B] a namespace numbering tasks (the process id), C] a namespace naming cpusets (the hierarchical name space normally mounted at /dev/cpuset, and corresponding to the Mn maps above) and D] a mapping of tasks to cpusets, system wide (just a map, not a namespace.) All tasks (of sufficient authority) can see each of these, using a single system wide name space for each of [A], [B], and [C]. Unless, that is, you call any mapping a "way of changing mappings". To do so would be a senseless abuse of the phrase, in my view. More generally, these resource managers all tend to divide some external limited physical resource into m…
creator: Paul Jackson
date: 2007-03-09T04:27:13-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: > nobody actually cares about a precise accounting and > calculating shares or partitions of whatever resource, > all that matters is that you have a way to prevent a > potential hostile environment from sucking up all your > resources (or even a single one) resulting in a DoS This is not true. People care. Reasons: - resource planning - fairness - guarantees What you talk is about security only. Not the above issues. So good precision is required. If there is no precision at all, security sucks as well and can be exploited, e.g. for CPU schedulers doing an accounting based on jiffies accounting in scheduler_tick() it is easy to build an application consuming 90% of CPU, but ~0% from scheduler POV. Kirill _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: dev
date: 2007-03-09T09:07:27-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: >>There have been various projects attempting to provide resource >>management support in Linux, including CKRM/Resource Groups and UBC. > > > let me note here, once again, that you forgot Linux-VServer > which does quite non-intrusive resource management ... Herbert, do you care to send patches except for ask others to do something that works for you? Looks like your main argument is non-intrusive... "working", "secure", "flexible" are not required to people any more? :/ >> Each had its own task-grouping mechanism. > > > the basic 'context' (pid space) is the grouping mechanism > we use for resource management too > > >>Paul Menage observed [1] that cpusets in the kernel already has a >>grouping mechanism which was working well for cpusets. He went ahead >>and generalized the grouping code in cpusets so that it could be used >>for overall resource management purpose. > > >>With his patches, it is possible to even create multiple hierarchies >>of groups (see [2] on why multiple hierarchies) as follows: > > > do we need or even want that? IMHO the hierarchical > concept CKRM was designed with, was also the reason > for it being slow, unuseable and complicated 1. cpusets are hierarchical already. So hierarchy is required. 2. As it was discussed on the call controllers which are flat can just prohibit creation of hierarchy on the filesystem. i.e. allow only 1 depth and continue being fast. >>mount -t container -o cpuset none /dev/cpuset <- cpuset hierarchy >>mount -t container -o mem,cpu none /dev/mem <- memory/cpu hierarchy >>mount -t container -o disk none /dev/disk <- disk hierarchy >> >>In each hierarchy, you can create task groups and manipulate the >>resource parameters of each group. You can also move tasks between >>groups at run-time (see [3] on why this is required). > > >>Each hierarchy is also manipulated independent of the other. > > >>Paul's patches also introduced a 'struct container' in the kernel, >>which serves these key purposes: >> >>- Task-grouping…
creator: dev
date: 2007-03-09T09:23:55-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: Kirill, responding to Herbert: > > do we need or even want that? IMHO the hierarchical > > concept CKRM was designed with, was also the reason > > for it being slow, unuseable and complicated > 1. cpusets are hierarchical already. So hierarchy is required. I think that CKRM has a harder time doing a hierarchy than cpusets. CKRM is trying to account for and control how much of an amorphous resource is used, whereas cpusets is trying to control whether a specifically identifiable resource is used, or not used, not how much of it is used. A child cpuset gets configured to allow certain CPUs and Nodes, and then does not need to dynamically pass back any information about what is actually used - it's a one-way control with no feedback. That's a relatively easier problem. CKRM (as I recall it, from long ago ...) has to track the amount of usage dynamically, across parent and child groups (whatever they were called.) That's a harder problem. So, yes, as Kirill observes, we need the hierarchy because cpusets has it, cpuset users make good use of the hierarchy, and the hierarchy works fine in that case, even if a hierarchy is more difficult for CKRM. -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson <pj@sgi.com> 1.925.600.0401 _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Paul Jackson
date: 2007-03-09T09:38:17-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: On Fri, Mar 09, 2007 at 12:23:55PM +0300, Kirill Korotaev wrote: >>> There have been various projects attempting to provide >>> resource management support in Linux, including >>> CKRM/Resource Groups and UBC. >> >> let me note here, once again, that you forgot Linux-VServer >> which does quite non-intrusive resource management ... > Herbert, do you care to send patches except for ask > others to do something that works for you? sorry, I'm not in the lucky position that I get payed for sending patches to LKML, so I have to think twice before I invest time in coding up extra patches ... i.e. you will have to live with my comments for now > Looks like your main argument is non-intrusive... > "working", "secure", "flexible" are not required to > people any more? :/ well, Linux-VServer is "working", "secure", "flexible" _and_ non-intrusive ... it is quite natural that less won't work for me ... and regarding patches, there will be a 2.2 release soon, with all the patches ... >>> Each had its own task-grouping mechanism. >> the basic 'context' (pid space) is the grouping mechanism >> we use for resource management too >>> Paul Menage observed [1] that cpusets in the kernel already has a >>> grouping mechanism which was working well for cpusets. He went ahead >>> and generalized the grouping code in cpusets so that it could be >>> used for overall resource management purpose. >>> With his patches, it is possible to even create multiple hierarchies >>> of groups (see [2] on why multiple hierarchies) as follows: >> do we need or even want that? IMHO the hierarchical >> concept CKRM was designed with, was also the reason >> for it being slow, unuseable and complicated > 1. cpusets are hierarchical already. So hierarchy is required. > 2. As it was discussed on the call controllers which are flat > can just prohibit creation of hierarchy on the filesystem. > i.e. allow only 1 depth and continue being fast. > >>> mount -t container -o cpuset none /dev/cpuset <- cpuset hierarchy…
creator: Herbert Poetzl
date: 2007-03-09T13:21:37-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: On Fri, Mar 09, 2007 at 12:07:27PM +0300, Kirill Korotaev wrote: >> nobody actually cares about a precise accounting and >> calculating shares or partitions of whatever resource, >> all that matters is that you have a way to prevent a >> potential hostile environment from sucking up all your >> resources (or even a single one) resulting in a DoS > This is not true. People care. Reasons: > - resource planning > - fairness > - guarantees let me make that a little more clear ... _nobody_ cares wether a shared memory page is accounted as full page or as fraction of a page (depending on the number of guests sharing it) as long as the accounted amount is substracted correctly when the page is disposed so there _is_ a difference between _false_ accounting (which seems what you are referring to in the next paragraph) and imprecise, but consistant accounting (which is what I was talking about) best, Herbert > What you talk is about security only. Not the above issues. > So good precision is required. If there is no precision at all, > security sucks as well and can be exploited, e.g. for CPU > schedulers doing an accounting based on jiffies accounting in > scheduler_tick() it is easy to build an application consuming > 90% of CPU, but ~0% from scheduler POV. > Kirill _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Herbert Poetzl
date: 2007-03-09T13:29:22-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: Quoting Srivatsa Vaddagiri (vatsa@in.ibm.com): > On Wed, Mar 07, 2007 at 08:12:00PM -0700, Eric W. Biederman wrote: > > I have a question? What does rcfs look like if we start with > > the code that is in the kernel? That is start with namespaces > > and nsproxy and just build a filesystem to display/manipulate them? > > With the code built so it will support adding resource controllers > > when they are ready? > > If I am not mistaken, Serge did attempt something in that direction, > only that it was based on Paul's container patches. rcfs can no doubt > support the same feature. My first nsproxy control fs was not based on Paul's set, but was very nsproxy-specific. After that I based it on Paul's set. The nsproxy subsystem is in Paul's latest patchset. On top of that set I've posted a set to implement namespace entering, but that is on hold (or dropped) because the only people so far interested in namespace entering don't like an fs interface. If you wanted to take my original non-container fs nsproxy interface and try to rebuild resource containers on top of them I don't think you'll get anything better than containers, but then maybe you would... -serge _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: serue
date: 2007-03-09T16:16:18-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Wed, Mar 07, 2007 at 01:20:18PM -0800, Paul Menage wrote: > On 3/7/07, Serge E. Hallyn <serue@us.ibm.com> wrote: > > > >All that being said, if it were going to save space without overly > >complicating things I'm actually not opposed to using nsproxy, but it > > If space-saving is the main issue, then the latest version of my > containers patches uses just a single pointer in the task_struct, and > all tasks in the same set of containers (across all hierarchies) will > share a single container_group object, which holds the actual pointers > to container state. Paul, Some more thoughts, mostly coming from the point of view of vservers/containers/"whaever is the set of tasks sharing a nsproxy is called". 1. What is the fundamental unit over which resource-management is applied? Individual tasks or individual containers? /me thinks latter. In which case, it makes sense to stick resource control information in the container somewhere. Just like when controlling a user's resource consumption, 'struct user_struct' may be a natural place to put these resource limits. 2. Regarding space savings, if 100 tasks are in a container (I dont know what is a typical number) -and- lets say that all tasks are to share the same resource allocation (which seems to be natural), then having a 'struct container_group *' pointer in each task_struct seems to be not very efficient (simply because we dont need that task-level granularity of managing resource allocation). 3. This next leads me to think that 'tasks' file in each directory doesnt make sense for containers. In fact it can lend itself to error situations (by administrator/script mistake) when some tasks of a container are in one resource class while others are in a different class. Instead, from a containers pov, it may be usefull to write a 'container id' (if such a thing exists) into the tasks file which will move all the tasks of the container into the new resource class. This is the same requirement we discussed long back o…
creator: Srivatsa Vaddagiri
date: 2007-03-09T16:34:30-00:00

Re: [ckrm-tech] [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Fri, Mar 09, 2007 at 10:04:30PM +0530, Srivatsa Vaddagiri wrote: > 2. Regarding space savings, if 100 tasks are in a container (I dont know > what is a typical number) -and- lets say that all tasks are to share > the same resource allocation (which seems to be natural), then having > a 'struct container_group *' pointer in each task_struct seems to be not > very efficient (simply because we dont need that task-level granularity of > managing resource allocation). Note that this 'struct container_group *' pointer is in addition to the 'struct nsproxy *' pointer already in task_struct. If the set of tasks over which resorce control is applied is typically the same set of tasks which share the same 'struct nsproxy *' pointer, then IMHO 'struct container_group *' in each task_struct is not very optimal. -- Regards, vatsa _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Srivatsa Vaddagiri
date: 2007-03-09T16:41:18-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: On Fri, Mar 09, 2007 at 01:38:19AM +0100, Herbert Poetzl wrote: > > 2) you allow a task to selectively reshare namespaces/subsystems with > > another task, i.e. you can update current->task_proxy to point to > > a proxy that matches your existing task_proxy in some ways and the > > task_proxy of your destination in others. In that case a trivial > > implementation would be to allocate a new task_proxy and copy some > > pointers from the old task_proxy and some from the new. But then > > whenever a task moves between different groupings it acquires a > > new unique task_proxy. So moving a bunch of tasks between two > > groupings, they'd all end up with unique task_proxy objects with > > identical contents. > > this is exactly what Linux-VServer does right now, and I'm > still not convinced that the nsproxy really buys us anything > compared to a number of different pointers to various spaces > (located in the task struct) Are you saying that the current scheme of storing pointers to different spaces (uts_ns, ipc_ns etc) in nsproxy doesn't buy anything? Or are you referring to storage of pointers to resource (name)spaces in nsproxy doesn't buy anything? In either case, doesn't it buy speed and storage space? > I'd prefer to do accounting (and limits) in a very simple > and especially performant way, and the reason for doing > so is quite simple: Can you elaborate on the relationship between data structures used to store those limits to the task_struct? Does task_struct store pointers to those objects directly? -- Regards, vatsa _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Srivatsa Vaddagiri
date: 2007-03-09T17:57:07-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: On Fri, Mar 09, 2007 at 01:48:16AM +0100, Herbert Poetzl wrote: > > There have been various projects attempting to provide resource > > management support in Linux, including CKRM/Resource Groups and UBC. > > let me note here, once again, that you forgot Linux-VServer > which does quite non-intrusive resource management ... Sorry, not intentionally. Maybe it slipped because I haven't seen much res mgmt related patches from Linux Vserver on lkml recently. Note that I -did- talk about VServer at one point in past (http://lkml.org/lkml/2006/06/15/112)! > the basic 'context' (pid space) is the grouping mechanism > we use for resource management too so tasks sharing the same nsproxy->pid_ns is the fundamental unit of resource management (as far as vserver/container goes)? > > As you know, the introduction of 'struct container' was objected > > to and was felt redundant as a means to group tasks. Thats where I > > took a shot at converting over Paul Menage's patch to avoid 'struct > > container' abstraction and insead work with 'struct nsproxy'. > > which IMHO isn't a step in the right direction, as > you will need to handle different nsproxies within > the same 'resource container' (see previous email) Isn't that made simple because of the fact that we have pointers to namespace objects (and not actual objects themselves) in nsproxy? I mean, all that is required to manage multiple nsproxy's is to have the pointer to the same resource object in all of them. In system call terms, if someone does a unshare of uts namespace, he will get into a new nsproxy object sure (which has a pointer to the new uts namespace) but the new nsproxy object will still be pointing to the old resource controlling objects. > > When we support task movement across resource classes, we need to find a > > nsproxy which has the right combination of resource classes that the > > task's nsproxy can be hooked to. > > no, not necessarily, we can simply create a new one > and give it the proper resource …
creator: Srivatsa Vaddagiri
date: 2007-03-09T18:14:22-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Fri, Mar 09, 2007 at 02:16:08AM +0100, Herbert Poetzl wrote: > On Thu, Mar 08, 2007 at 05:00:54PM +0530, Srivatsa Vaddagiri wrote: > > On Thu, Mar 08, 2007 at 01:50:01PM +1300, Sam Vilain wrote: > > > 7. resource namespaces > > > > It should be. Imagine giving 20% bandwidth to a user X. X wants to > > divide this bandwidth further between multi-media (10%), kernel > > compilation (5%) and rest (5%). So, > > sounds quite nice, but ... > > > > Is the subservient namespace's resource usage counting against ours too? > > > > Yes, the resource usage of children should be accounted when capping > > parent resource usage. > > it will require to do accounting many times > (and limit checks of course), which in itself > might be a way to DoS the kernel by creating > more and more resource groups I was only pointing out the usefullness of the feature and not necessarily saying it -should- be implemented! Ofcourse I understand it will make the controller complicated and thats why probably none of the recontrollers we are seeing posted on lkml don't support hierarchical res mgmt. > > > Can we dynamically alter the subservient namespace's resource > > > allocations? > > > > Should be possible yes. That lets user X completely manage his > > allocation among whatever sub-groups he creates. > > what happens if the parent changes, how is > the resource change (if it was a reduction) > propagated to the children? > > e.g. your guest has 1024 file handles, now > you reduce it to 512, but the guest had two > children, both with 256 file handles each ... I believe CKRM handled this quite neatly (by defining child shares to be relative to parent shares). In your example, 256+256 add up to 512 which is within the parent's new limit, so nothing happens :) You also picked an example of exhaustible/non-reclaimable resource, which makes it hard to define what should happen if parent's limit goes below 512. Either nothing happens or perhaps a task is killed, don't know. In case of memory, I…
creator: Srivatsa Vaddagiri
date: 2007-03-09T18:41:05-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: > Ease of use maybe. Scripts can be more readily used with a fs-based > interface. And, as I might have already stated, file system API's are a natural fit for hierarchically shaped data, especially if the nodes in the hierarchy would benefit from file system like permission attributes. -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson <pj@sgi.com> 1.925.600.0401 _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Paul Jackson
date: 2007-03-09T19:25:47-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On 3/9/07, Srivatsa Vaddagiri <vatsa@in.ibm.com> wrote: > > 1. What is the fundamental unit over which resource-management is > applied? Individual tasks or individual containers? > > /me thinks latter. Yes > In which case, it makes sense to stick > resource control information in the container somewhere. Yes, that's what all my patches have been doing. > 2. Regarding space savings, if 100 tasks are in a container (I dont know > what is a typical number) -and- lets say that all tasks are to share > the same resource allocation (which seems to be natural), then having > a 'struct container_group *' pointer in each task_struct seems to be not > very efficient (simply because we dont need that task-level granularity of > managing resource allocation). I think you should re-read my patches. Previously, each task had N pointers, one for its container in each potential hierarchy. The container_group concept means that each task has 1 pointer, to a set of container pointers (one per hierarchy) shared by all tasks that have exactly the same set of containers (in the various different hierarchies). It doesn't give task-level granularity of resource management (unless you create a separate container for each task), it just gives a space saving. > > 3. This next leads me to think that 'tasks' file in each directory doesnt make > sense for containers. In fact it can lend itself to error situations (by > administrator/script mistake) when some tasks of a container are in one > resource class while others are in a different class. > > Instead, from a containers pov, it may be usefull to write > a 'container id' (if such a thing exists) into the tasks file > which will move all the tasks of the container into > the new resource class. This is the same requirement we > discussed long back of moving all threads of a process into new > resource class. I think you need to give a more concrete example and use case of what you're trying to propose here. I don't really see what advantage…
creator: Paul Menage
date: 2007-03-09T22:09:35-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: On Fri, Mar 09, 2007 at 11:44:22PM +0530, Srivatsa Vaddagiri wrote: > On Fri, Mar 09, 2007 at 01:48:16AM +0100, Herbert Poetzl wrote: > > > There have been various projects attempting to provide resource > > > management support in Linux, including CKRM/Resource Groups and UBC. > > let me note here, once again, that you forgot Linux-VServer > > which does quite non-intrusive resource management ... > Sorry, not intentionally. Maybe it slipped because I haven't > seen much res mgmt related patches from Linux Vserver on > lkml recently. mainly because I got the impression that we planned to work on the various spaces first, and handle things like resource management later .. but it seems that resource management is now in focus, while the spaces got somewhat delayed ... > Note that I -did- talk about VServer at one point in past > (http://lkml.org/lkml/2006/06/15/112)! noted and appreciated (although this was about CPU resources, which IMHO is a special resource like the networking, as you are mostly interested in 'bandwidth' limitations there, not in resource limits per se (and of course, it wasn't even cited correctly, as it is Linux-VServer not vserver ...) > > the basic 'context' (pid space) is the grouping mechanism > > we use for resource management too > so tasks sharing the same nsproxy->pid_ns is the fundamental > unit of resource management (as far as vserver/container goes)? we currently have a 'process' context, which holds the administrative data (capabilities and flags) and the resource accounting and limits, which basically contains the pid namespace, so yes and no it contains a reference to the 'main' nsproxy, which is used to copy spaces from when you enter the guest (or some set of spaces), and it defines the unit we consider a process container > > > As you know, the introduction of 'struct container' was objected > > > to and was felt redundant as a means to group tasks. Thats where > > > I took a shot at converting over Paul Menage's patch to avoi…
creator: Herbert Poetzl
date: 2007-03-10T00:56:43-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: On Fri, Mar 09, 2007 at 11:25:47AM -0800, Paul Jackson wrote: > > Ease of use maybe. Scripts can be more readily used with a fs-based > > interface. > > And, as I might have already stated, file system API's are a natural > fit for hierarchically shaped data, especially if the nodes in the > hierarchy would benefit from file system like permission attributes. personally, I'd prefer to avoid hierarchical structures wherever possible, because they tend to make processing and checks a lot more complicated than necessary, and if we really want hierarchical structures, it might be more than sufficient to keep the hierarchy in userspace, and use a flat representation inside the kernel ... but hey, I'm all for running a hypervisor under a hypervisor running inside a hypervisor :) best, Herbert > -- > I won't rest till it's the best ... > Programmer, Linux Scalability > Paul Jackson <pj@sgi.com> 1.925.600.0401 > _______________________________________________ > Containers mailing list > Containers@lists.osdl.org > https://lists.osdl.org/mailman/listinfo/containers _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Herbert Poetzl
date: 2007-03-10T01:00:41-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: On Fri, Mar 09, 2007 at 11:27:07PM +0530, Srivatsa Vaddagiri wrote: > On Fri, Mar 09, 2007 at 01:38:19AM +0100, Herbert Poetzl wrote: > > > 2) you allow a task to selectively reshare namespaces/subsystems with > > > another task, i.e. you can update current->task_proxy to point to > > > a proxy that matches your existing task_proxy in some ways and the > > > task_proxy of your destination in others. In that case a trivial > > > implementation would be to allocate a new task_proxy and copy some > > > pointers from the old task_proxy and some from the new. But then > > > whenever a task moves between different groupings it acquires a > > > new unique task_proxy. So moving a bunch of tasks between two > > > groupings, they'd all end up with unique task_proxy objects with > > > identical contents. > > this is exactly what Linux-VServer does right now, and I'm > > still not convinced that the nsproxy really buys us anything > > compared to a number of different pointers to various spaces > > (located in the task struct) > Are you saying that the current scheme of storing pointers to > different spaces (uts_ns, ipc_ns etc) in nsproxy doesn't buy > anything? > Or are you referring to storage of pointers to resource > (name)spaces in nsproxy doesn't buy anything? > In either case, doesn't it buy speed and storage space? let's do a few examples here, just to illustrate the advantages and disadvantages of nsproxy as separate structure over nsproxy as part of the task_struct 1) typical setup, 100 guests as shell servers, 5 tasks each when unused, 10 tasks when used 10% used in average a) separate nsproxy, we need at least 100 structs to handle that (saves some space) we might end up with ~500 nsproxies, if the shell clones a new namespace (so might not save that much space) we do a single inc/dec when the nsproxy is reused, but do the full N inc/dec when we have to copy an nsproxy (might save some refcounting) we need to do the indirection step, from task to nsproxy to space (…
creator: Herbert Poetzl
date: 2007-03-10T01:19:53-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: Herbert wrote: > personally, I'd prefer to avoid hierarchical > structures wherever possible, Sure - avoid them if you like. But sometimes they work out rather well. And file system API's are sometimes the best fit for them. I'm all for choosing the simplest API topology that makes sense. But encoding higher dimension topologies into lower dimension API's, just because they seem "simpler" results in obfuscation, convolution and obscurity, which ends up costing everyone more than getting the natural fit. "Make everything as simple as possible, but not simpler." -- Albert Einstein -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson <pj@sgi.com> 1.925.600.0401 _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Paul Jackson
date: 2007-03-10T01:31:15-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: I think maybe I didnt communicate what I mean by a container here (although I thought I did). I am referring to a container in a vserver context (set of tasks which share the same namespace). On Fri, Mar 09, 2007 at 02:09:35PM -0800, Paul Menage wrote: > >2. Regarding space savings, if 100 tasks are in a container (I dont know > > what is a typical number) -and- lets say that all tasks are to share > > the same resource allocation (which seems to be natural), then having > > a 'struct container_group *' pointer in each task_struct seems to be not > > very efficient (simply because we dont need that task-level granularity > > of > > managing resource allocation). > > I think you should re-read my patches. > > Previously, each task had N pointers, one for its container in each > potential hierarchy. The container_group concept means that each task > has 1 pointer, to a set of container pointers (one per hierarchy) > shared by all tasks that have exactly the same set of containers (in > the various different hierarchies). Ok, let me see if I can convey what I had in mind better: uts_ns pid_ns ipc_ns \ | / --------------- | nsproxy | ---------------- / | \ \ <-- 'nsproxy' pointer T1 T2 T3 ...T1000 | | | | <-- 'containers' pointer (4/8 KB for 1000 task) ------------------- | container_group | ------------------ / ---------- | container | ---------- | ---------- | cpu_limit | ---------- (T1, T2, T3 ..T1000) are part of a vserver lets say sharing the same uts/pid/ipc_ns. Now where do we store the resource control information for this unit/set-of-tasks in your patches? (tsk->containers->container[cpu_ctlr.hierarchy] + X)->cpu_limit (The X is to account for the fact that cotainer structure points to a 'struct container_subsys_state' embedded in some other structure. Its usually zero if the structure is embedded at the top) I understand that container_group also points directly to 'struct container_subsys_state', in which case, the above is optimized to: (tsk->containers->sub…
creator: Srivatsa Vaddagiri
date: 2007-03-10T02:02:20-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Sat, Mar 10, 2007 at 12:11:05AM +0530, Srivatsa Vaddagiri wrote: > On Fri, Mar 09, 2007 at 02:16:08AM +0100, Herbert Poetzl wrote: > > On Thu, Mar 08, 2007 at 05:00:54PM +0530, Srivatsa Vaddagiri wrote: > > > On Thu, Mar 08, 2007 at 01:50:01PM +1300, Sam Vilain wrote: > > > > 7. resource namespaces > > > > > > It should be. Imagine giving 20% bandwidth to a user X. X wants to > > > divide this bandwidth further between multi-media (10%), kernel > > > compilation (5%) and rest (5%). So, > > > > sounds quite nice, but ... > > > > > > Is the subservient namespace's resource usage counting against > > > > ours too? > > > > > > Yes, the resource usage of children should be accounted when capping > > > parent resource usage. > > > > it will require to do accounting many times > > (and limit checks of course), which in itself > > might be a way to DoS the kernel by creating > > more and more resource groups > > I was only pointing out the usefullness of the feature and not > necessarily saying it -should- be implemented! Ofcourse I understand it > will make the controller complicated and thats why probably none of the > recontrollers we are seeing posted on lkml don't support hierarchical > res mgmt. > > > > > Can we dynamically alter the subservient namespace's resource > > > > allocations? > > > > > > Should be possible yes. That lets user X completely manage his > > > allocation among whatever sub-groups he creates. > > > > what happens if the parent changes, how is > > the resource change (if it was a reduction) > > propagated to the children? > > > > e.g. your guest has 1024 file handles, now > > you reduce it to 512, but the guest had two > > children, both with 256 file handles each ... > > I believe CKRM handled this quite neatly (by defining child shares to be > relative to parent shares). > > In your example, 256+256 add up to 512 which is within the parent's > new limit, so nothing happens :) yes, but that might as well be fatal, because now the children can …
creator: Herbert Poetzl
date: 2007-03-10T02:03:48-00:00

Re: [ckrm-tech] [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Sat, Mar 10, 2007 at 07:32:20AM +0530, Srivatsa Vaddagiri wrote: > Ok, let me see if I can convey what I had in mind better: > > uts_ns pid_ns ipc_ns > \ | / > --------------- > | nsproxy | > ---------------- > / | \ \ <-- 'nsproxy' pointer > T1 T2 T3 ...T1000 > | | | | <-- 'containers' pointer (4/8 KB for 1000 task) > ------------------- > | container_group | > ------------------ > / > ---------- > | container | > ---------- > | > ---------- > | cpu_limit | > ---------- [snip] > We save on 4/8 KB (for 1000 tasks) by avoiding the 'containers' pointer > in each task_struct (just to get to the resource limit information). Having the 'containers' pointer in each task-struct is great from a non-container res mgmt perspective. It lets you dynamically decide what is the fundamental unit of res mgmt. It could be {T1, T5} tasks/threads of a process, or {T1, T3, T8, T10} tasks of a session (for limiting login time per session), or {T1, T2 ..T10, T18, T27} tasks of a user etc. But from a vserver/container pov, this level flexibility (at a -task- level) of deciding the unit of res mgmt is IMHO not needed. The vserver/container/namespace (tsk->nsproxy->some_ns) to which a task belongs automatically defines that unit of res mgmt. -- Regards, vatsa _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Srivatsa Vaddagiri
date: 2007-03-10T03:19:09-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: Paul Jackson wrote: >> But "namespace" has well-established historical semantics too - a way >> of changing the mappings of local * to global objects. This >> accurately describes things liek resource controllers, cpusets, resource >> monitoring, etc. >> > > No! > > Cpusets don't rename or change the mapping of objects. > > I suspect you seriously misunderstand cpusets and are trying to cram them > into a 'namespace' remapping role into which they don't fit. > Look, you're absolutely right, I'm stretching the terms much too far. namespaces implies some kind of domain, which is the namespace, and entities within the domain, which are the names, and there is a (task, domain) mapping. I was thinking that this implies all similar (task, domain) mappings could be treated in the same way. But when you apply this to something like cpusets, it gets a little abstract. Like the entities are (task,cpu) pairs and the domains the set of cpus that a process can run on. Sam. _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Sam Vilain
date: 2007-03-10T08:52:35-00:00

Re: [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: Sam wrote: > But when you apply this to something like cpusets, it gets a little abstract. Just a tad abstract <grin>. Thanks. -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson <pj@sgi.com> 1.925.600.0401 _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Paul Jackson
date: 2007-03-10T09:11:15-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: Quoting Herbert Poetzl (herbert@13thfloor.at): > On Fri, Mar 09, 2007 at 11:27:07PM +0530, Srivatsa Vaddagiri wrote: > > On Fri, Mar 09, 2007 at 01:38:19AM +0100, Herbert Poetzl wrote: > > > > 2) you allow a task to selectively reshare namespaces/subsystems with > > > > another task, i.e. you can update current->task_proxy to point to > > > > a proxy that matches your existing task_proxy in some ways and the > > > > task_proxy of your destination in others. In that case a trivial > > > > implementation would be to allocate a new task_proxy and copy some > > > > pointers from the old task_proxy and some from the new. But then > > > > whenever a task moves between different groupings it acquires a > > > > new unique task_proxy. So moving a bunch of tasks between two > > > > groupings, they'd all end up with unique task_proxy objects with > > > > identical contents. > > > > this is exactly what Linux-VServer does right now, and I'm > > > still not convinced that the nsproxy really buys us anything > > > compared to a number of different pointers to various spaces > > > (located in the task struct) > > > Are you saying that the current scheme of storing pointers to > > different spaces (uts_ns, ipc_ns etc) in nsproxy doesn't buy > > anything? > > > Or are you referring to storage of pointers to resource > > (name)spaces in nsproxy doesn't buy anything? > > > In either case, doesn't it buy speed and storage space? > > let's do a few examples here, just to illustrate the > advantages and disadvantages of nsproxy as separate > structure over nsproxy as part of the task_struct But you're forgetting the *common* case, which is hundreds or thousands of tasks with just one nsproxy. That's case for which we have to optimize. When that case is no longer the common case, we can yank the nsproxy. As I keep saying, it *is* just an optimization. -serge > 1) typical setup, 100 guests as shell servers, 5 > tasks each when unused, 10 tasks when used 10% > used in average > > a) separat…
creator: serue
date: 2007-03-11T16:36:04-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: Herbert, > sorry, I'm not in the lucky position that I get payed > for sending patches to LKML, so I have to think twice > before I invest time in coding up extra patches ... > > i.e. you will have to live with my comments for now looks like you have no better argurments then that... >>Looks like your main argument is non-intrusive... >>"working", "secure", "flexible" are not required to >>people any more? :/ > > > well, Linux-VServer is "working", "secure", "flexible" > _and_ non-intrusive ... it is quite natural that less > won't work for me ... and regarding patches, there > will be a 2.2 release soon, with all the patches ... ok. please check your dcache and slab accounting then (analyzed according to patch-2.6.20.1-vs2.3.0.11.diff): Both are full of races and problems. Some of them: 1. Slabs allocated from interrupt context are charged to current context. So charged values contain arbitrary mess, since during interrupts context can be arbitrary. 2. Due to (1) I guess you do not make any limiting of slabs. So there are number of ways how to consume a lot of kernel memory from inside container and OOM killer will kill arbitrary tasks in case of memory-shortage after that. Don't think it is secure... real DoS. 3. Dcache accounting simply doesn't work, since charges/uncharges are done on current context (sic!!!), which is arbitrary. i.e. lookup can be done in VE context, while dcache shrink can be done from another context. So the whole problem with dcache DoS is not solved at all, it is just hard to trigger. 4. Dcache accounting is racy, since your checks look like: if (atomic_read(de->d_count)) charge(); which obviously races with other dput()'s/lookups. 5. Dcache accounting can be hit if someone does `find /` inside container. After that it is impossible to open something new, since all the dentries for directories in dcache will have d_count > 0 (due it's children). It is a BUG. 6. Counters can be non-zero on container stop due to all of the above. There are mo…
creator: dev
date: 2007-03-11T17:09:29-00:00

Re: [ckrm-tech] [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Wed, Mar 07, 2007 at 03:59:19PM -0600, Serge E. Hallyn wrote: > > containers patches uses just a single pointer in the task_struct, and > > all tasks in the same set of containers (across all hierarchies) will > > share a single container_group object, which holds the actual pointers > > to container state. > > Yes, that's why this consolidation doesn't make sense to me. > > Especially considering again that we will now have nsproxies pointing to > containers pointing to... nsproxies. nsproxies needn't point to containers. It (or as Herbert pointed - nsproxy->pid_ns) can have direct pointers to resource objects (whatever struct container->subsys[] points to). -- Regards, vatsa _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Srivatsa Vaddagiri
date: 2007-03-12T14:11:44-00:00

Re: [ckrm-tech] [PATCH 0/2] resource control file system - aka containers on top of nsproxy!
type: item
description: On Fri, Mar 09, 2007 at 02:09:35PM -0800, Paul Menage wrote: > > 3. This next leads me to think that 'tasks' file in each directory doesnt make > > sense for containers. In fact it can lend itself to error situations (by > > administrator/script mistake) when some tasks of a container are in one > > resource class while others are in a different class. > > > > Instead, from a containers pov, it may be usefull to write > > a 'container id' (if such a thing exists) into the tasks file > > which will move all the tasks of the container into > > the new resource class. This is the same requirement we > > discussed long back of moving all threads of a process into new > > resource class. > > I think you need to give a more concrete example and use case of what > you're trying to propose here. I don't really see what advantage > you're getting. Ok, this is what I had in mind: mount -t container -o ns /dev/namespace mount -t container -o cpu /dev/cpu Lets we have the namespaces/resource-groups created as under: /dev/namespace |-- prof | |- tasks <- (T1, T2) | |- container_id <- 1 (doesnt exist today perhaps) | |-- student | |- tasks <- (T3, T4) | |- container_id <- 2 (doesnt exist today perhaps) /dev/cpu |-- prof | |-- tasks | |-- cpu_limit (40%) | |-- student | |-- tasks | |-- cpu_limit (20%) | | Is it possible to create the above structure in container patches? /me thinks so. If so, then accidentally someone can do this: echo T1 > /dev/cpu/prof/tasks echo T2 > /dev/cpu/student/tasks with the result that tasks of the same container are now in different resource classes. Thats why in case of containers I felt we shldnt allow individual tasks to be cat'ed to tasks file. Or rather, it may be nice to say : echo "cid 2" > /dev/cpu/prof/tasks and have all tasks belonging to container id 2 move to the new resource group. -- Regards, vatsa _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/cont…
creator: Srivatsa Vaddagiri
date: 2007-03-12T15:07:56-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: On Sun, Mar 11, 2007 at 08:09:29PM +0300, Kirill Korotaev wrote: > Herbert, > > > sorry, I'm not in the lucky position that I get payed > > for sending patches to LKML, so I have to think twice > > before I invest time in coding up extra patches ... > > > > i.e. you will have to live with my comments for now > looks like you have no better argurments then that... pardon? if you want to make that personal, please do it offline ... I'm sick of (lkml) folks wasting time for (political) hick hack instead of trying to improve the kernel ... >>> Looks like your main argument is non-intrusive... >>> "working", "secure", "flexible" are not required to >>> people any more? :/ >> well, Linux-VServer is "working", "secure", "flexible" >> _and_ non-intrusive ... it is quite natural that less >> won't work for me ... and regarding patches, there >> will be a 2.2 release soon, with all the patches ... > ok. please check your dcache and slab accounting then > (analyzed according to patch-2.6.20.1-vs2.3.0.11.diff): development branch, good choice for new features and code which is currently tested ... > Both are full of races and problems. Some of them: > 1. Slabs allocated from interrupt context are charged to > current context. > So charged values contain arbitrary mess, since during > interrupts context can be arbitrary. > 2. Due to (1) I guess you do not make any limiting of slabs. > So there are number of ways how to consume a lot of kernel > memory from inside container and > OOM killer will kill arbitrary tasks in case of > memory-shortage after that. > Don't think it is secure... real DoS. > 3. Dcache accounting simply doesn't work, since > charges/uncharges are done on current context (sic!!!), > which is arbitrary. i.e. lookup can be done in VE context, > while dcache shrink can be done from another context. > So the whole problem with dcache DoS is not solved at > all, it is just hard to trigger. > 4. Dcache accounting is racy, since your checks look like: > if (atomic_r…
creator: Herbert Poetzl
date: 2007-03-12T23:00:53-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: On Sun, Mar 11, 2007 at 11:36:04AM -0500, Serge E. Hallyn wrote: > Quoting Herbert Poetzl (herbert@13thfloor.at): > > On Fri, Mar 09, 2007 at 11:27:07PM +0530, Srivatsa Vaddagiri wrote: > > > On Fri, Mar 09, 2007 at 01:38:19AM +0100, Herbert Poetzl wrote: > > > > > 2) you allow a task to selectively reshare namespaces/subsystems with > > > > > another task, i.e. you can update current->task_proxy to point to > > > > > a proxy that matches your existing task_proxy in some ways and the > > > > > task_proxy of your destination in others. In that case a trivial > > > > > implementation would be to allocate a new task_proxy and copy some > > > > > pointers from the old task_proxy and some from the new. But then > > > > > whenever a task moves between different groupings it acquires a > > > > > new unique task_proxy. So moving a bunch of tasks between two > > > > > groupings, they'd all end up with unique task_proxy objects with > > > > > identical contents. > > > > > > this is exactly what Linux-VServer does right now, and I'm > > > > still not convinced that the nsproxy really buys us anything > > > > compared to a number of different pointers to various spaces > > > > (located in the task struct) > > > > > Are you saying that the current scheme of storing pointers to > > > different spaces (uts_ns, ipc_ns etc) in nsproxy doesn't buy > > > anything? > > > > > Or are you referring to storage of pointers to resource > > > (name)spaces in nsproxy doesn't buy anything? > > > > > In either case, doesn't it buy speed and storage space? > > > > let's do a few examples here, just to illustrate the > > advantages and disadvantages of nsproxy as separate > > structure over nsproxy as part of the task_struct > > But you're forgetting the *common* case, which is hundreds or > thousands of tasks with just one nsproxy. That's case for > which we have to optimize. yes, I agree here, maybe we should do something I suggested (and submitted a patch for some time ago) and add some kind of…
creator: Herbert Poetzl
date: 2007-03-12T23:16:18-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: >>>well, Linux-VServer is "working", "secure", "flexible" >>>_and_ non-intrusive ... it is quite natural that less >>>won't work for me ... and regarding patches, there >>>will be a 2.2 release soon, with all the patches ... > > >>ok. please check your dcache and slab accounting then >>(analyzed according to patch-2.6.20.1-vs2.3.0.11.diff): > > > development branch, good choice for new features > and code which is currently tested ... you know better than I that stable branch doesn't differ much, especially in securiy (because it lacks these controls at all). BTW, killing arbitrary task in case of RSS limit hit doesn't look acceptable resource management approach, does it? >>Both are full of races and problems. Some of them: >>1. Slabs allocated from interrupt context are charged to >> current context. >> So charged values contain arbitrary mess, since during >> interrupts context can be arbitrary. > > >>2. Due to (1) I guess you do not make any limiting of slabs. >> So there are number of ways how to consume a lot of kernel >> memory from inside container and >> OOM killer will kill arbitrary tasks in case of >> memory-shortage after that. >> Don't think it is secure... real DoS. > > >>3. Dcache accounting simply doesn't work, since >> charges/uncharges are done on current context (sic!!!), >> which is arbitrary. i.e. lookup can be done in VE context, >> while dcache shrink can be done from another context. >> So the whole problem with dcache DoS is not solved at >> all, it is just hard to trigger. > > >>4. Dcache accounting is racy, since your checks look like: >> if (atomic_read(de->d_count)) >> charge(); >> which obviously races with other dput()'s/lookups. > > >>5. Dcache accounting can be hit if someone does `find /` >> inside container. >> After that it is impossible to open something new, >> since all the dentries for directories in dcache will >> have d_count > 0 (due it's children). >> It is a BUG. > > >>6. Counters can be non-zero on container stop due to…
creator: dev
date: 2007-03-13T08:28:06-00:00

Re: [PATCH 1/2] rcfs core patch
type: item
description: On Tue, Mar 13, 2007 at 11:28:06AM +0300, Kirill Korotaev wrote: > >>>well, Linux-VServer is "working", "secure", "flexible" > >>>_and_ non-intrusive ... it is quite natural that less > >>>won't work for me ... and regarding patches, there > >>>will be a 2.2 release soon, with all the patches ... first, fix your mail client to get the quoting right, it is quite unreadable the way it is (not the first time I tell you that) > >>ok. please check your dcache and slab accounting then > >>(analyzed according to patch-2.6.20.1-vs2.3.0.11.diff): > > > > > > development branch, good choice for new features > > and code which is currently tested ... > you know better than I that stable branch doesn't differ much, > especially in securiy (because it lacks these controls at all). > > BTW, killing arbitrary task in case of RSS limit hit > doesn't look acceptable resource management approach, does it? > > >>Both are full of races and problems. Some of them: > >>1. Slabs allocated from interrupt context are charged to > >> current context. > >> So charged values contain arbitrary mess, since during > >> interrupts context can be arbitrary. > > > > > >>2. Due to (1) I guess you do not make any limiting of slabs. > >> So there are number of ways how to consume a lot of kernel > >> memory from inside container and > >> OOM killer will kill arbitrary tasks in case of > >> memory-shortage after that. > >> Don't think it is secure... real DoS. > > > > > >>3. Dcache accounting simply doesn't work, since > >> charges/uncharges are done on current context (sic!!!), > >> which is arbitrary. i.e. lookup can be done in VE context, > >> while dcache shrink can be done from another context. > >> So the whole problem with dcache DoS is not solved at > >> all, it is just hard to trigger. > > > > > >>4. Dcache accounting is racy, since your checks look like: > >> if (atomic_read(de->d_count)) > >> charge(); > >> which obviously races with other dput()'s/lookups. > > > > > >>5. Dcache accounting ca…
creator: Herbert Poetzl
date: 2007-03-13T13:55:05-00:00

Re: [ckrm-tech] [PATCH 1/2] rcfs core patch
type: item
description: On Tue, Mar 13, 2007 at 02:55:05PM +0100, Herbert Poetzl wrote: > yes, tons of locking, complicated indirections and > a lot of (partially hard to understand) code ... Are you referring to these issues in the general Paul Menage's container code or in the RSS-control code posted by Pavel? -- Regards, vatsa _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Srivatsa Vaddagiri
date: 2007-03-13T14:11:37-00:00