OpenVZ Forum - RDF feed
type: channel
items: [PATCH v8 0/3] cgroups: implement moving a threadgroup's threads atomically with cgroup.procs
items: [PATCH v8 1/3] cgroups: read-write lock CLONE_THREAD forking per threadgroup
items: [PATCH v8 2/3] cgroups: add per-thread subsystem callbacks
items: [PATCH v8 3/3] cgroups: make procs file writable
items: Re: [PATCH v8 0/3] cgroups: implement moving a threadgroup's threads atomically with cgroup.procs
items: [PATCH v8 4/3] cgroups: use flex_array in attach_proc

[PATCH v8 0/3] cgroups: implement moving a threadgroup's threads atomically with cgroup.procs
type: item
description: On Sun, Dec 26, 2010 at 07:09:19AM -0500, Ben Blum wrote: > On Fri, Dec 24, 2010 at 03:22:26AM -0500, Ben Blum wrote: > > On Wed, Aug 11, 2010 at 01:46:04AM -0400, Ben Blum wrote: > > > On Fri, Jul 30, 2010 at 07:56:49PM -0400, Ben Blum wrote: > > > > This patch series is a revision of http://lkml.org/lkml/2010/6/25/11 . > > > > > > > > This patch series implements a write function for the 'cgroup.procs' > > > > per-cgroup file, which enables atomic movement of multithreaded > > > > applications between cgroups. Writing the thread-ID of any thread in a > > > > threadgroup to a cgroup's procs file causes all threads in the group to > > > > be moved to that cgroup safely with respect to threads forking/exiting. > > > > (Possible usage scenario: If running a multithreaded build system that > > > > sucks up system resources, this lets you restrict it all at once into a > > > > new cgroup to keep it under control.) > > > > > > > > Example: Suppose pid 31337 clones new threads 31338 and 31339. > > > > > > > > # cat /dev/cgroup/tasks > > > > ... > > > > 31337 > > > > 31338 > > > > 31339 > > > > # mkdir /dev/cgroup/foo > > > > # echo 31337 > /dev/cgroup/foo/cgroup.procs > > > > # cat /dev/cgroup/foo/tasks > > > > 31337 > > > > 31338 > > > > 31339 > > > > > > > > A new lock, called threadgroup_fork_lock and living in signal_struct, is > > > > introduced to ensure atomicity when moving threads between cgroups. It's > > > > taken for writing during the operation, and taking for reading in fork() > > > > around the calls to cgroup_fork() and cgroup_post_fork(). > > Well this time everything here is actually safe and correct, as far as > my best efforts and keen eyes can tell. I dropped the per_thread call > from the last series in favour of revising the subsystem callback > interface. It now looks like this: > > ss->can_attach() > - Thread-independent, possibly expensive/sleeping. > > ss->can_attach_task() > - Called per-thread, run with rcu_read so must not sleep. > > ss->pre_…
creator: Ben Blum
date: 2011-02-08T01:35:42-00:00

[PATCH v8 1/3] cgroups: read-write lock CLONE_THREAD forking per threadgroup
type: item
description: Adds functionality to read/write lock CLONE_THREAD fork()ing per-threadgroup From: Ben Blum <bblum@andrew.cmu.edu> This patch adds an rwsem that lives in a threadgroup's signal_struct that's taken for reading in the fork path, under CONFIG_CGROUPS. If another part of the kernel later wants to use such a locking mechanism, the CONFIG_CGROUPS ifdefs should be changed to a higher-up flag that CGROUPS and the other system would both depend on. This is a pre-patch for cgroup-procs-write.patch. Signed-off-by: Ben Blum <bblum@andrew.cmu.edu> --- include/linux/init_task.h | 9 +++++++++ include/linux/sched.h | 37 +++++++++++++++++++++++++++++++++++++ kernel/fork.c | 10 ++++++++++ 3 files changed, 56 insertions(+), 0 deletions(-) diff --git a/include/linux/init_task.h b/include/linux/init_task.h index 6b281fa..b560381 100644 --- a/include/linux/init_task.h +++ b/include/linux/init_task.h @@ -15,6 +15,14 @@ extern struct files_struct init_files; extern struct fs_struct init_fs; +#ifdef CONFIG_CGROUPS +#define INIT_THREADGROUP_FORK_LOCK(sig) \ + .threadgroup_fork_lock = \ + __RWSEM_INITIALIZER(sig.threadgroup_fork_lock), +#else +#define INIT_THREADGROUP_FORK_LOCK(sig) +#endif + #define INIT_SIGNALS(sig) { \ .nr_threads = 1, \ .wait_chldexit = __WAIT_QUEUE_HEAD_INITIALIZER(sig.wait_chldexit),\ @@ -31,6 +39,7 @@ extern struct fs_struct init_fs; }, \ .cred_guard_mutex = \ __MUTEX_INITIALIZER(sig.cred_guard_mutex), \ + INIT_THREADGROUP_FORK_LOCK(sig) \ } extern struct nsproxy init_nsproxy; diff --git a/include/linux/sched.h b/include/linux/sched.h index 8580dc6..2fdbeb1 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -509,6 +509,8 @@ struct thread_group_cputimer { spinlock_t lock; }; +#include <linux/rwsem.h> + /* * NOTE! "signal_struct" does not have it's own * locking, because a shared signal_struct always @@ -623,6 +625,16 @@ struct signal_struct { unsigned audit_tty; struct tty_audit_buf *tty_audit_buf; #endif +#ifdef CONFIG_CGROUPS + /* + * The threadgroup_fo…
creator: Ben Blum
date: 2011-02-08T01:37:41-00:00

[PATCH v8 2/3] cgroups: add per-thread subsystem callbacks
type: item
description: Add cgroup subsystem callbacks for per-thread attachment From: Ben Blum <bblum@andrew.cmu.edu> This patch adds can_attach_task, pre_attach, and attach_task as new callbacks for cgroups's subsystem interface. Unlike can_attach and attach, these are for per-thread operations, to be called potentially many times when attaching an entire threadgroup. Also, the old "bool threadgroup" interface is removed, as replaced by this. All subsystems are modified for the new interface - of note is cpuset, which requires from/to nodemasks for attach to be globally scoped (though per-cpuset would work too) to persist from its pre_attach to attach_task and attach. This is a pre-patch for cgroup-procs-writable.patch. Signed-off-by: Ben Blum <bblum@andrew.cmu.edu> --- Documentation/cgroups/cgroups.txt | 30 ++++++++--- block/blk-cgroup.c | 18 ++---- include/linux/cgroup.h | 10 ++-- kernel/cgroup.c | 17 +++++- kernel/cgroup_freezer.c | 26 ++++----- kernel/cpuset.c | 105 ++++++++++++++++--------------------- kernel/ns_cgroup.c | 23 +++----- kernel/sched.c | 38 +------------ mm/memcontrol.c | 18 ++---- security/device_cgroup.c | 3 - 10 files changed, 122 insertions(+), 166 deletions(-) diff --git a/Documentation/cgroups/cgroups.txt b/Documentation/cgroups/cgroups.txt index 190018b..d3c9a24 100644 --- a/Documentation/cgroups/cgroups.txt +++ b/Documentation/cgroups/cgroups.txt @@ -563,7 +563,7 @@ rmdir() will fail with it. From this behavior, pre_destroy() can be called multiple times against a cgroup. int can_attach(struct cgroup_subsys *ss, struct cgroup *cgrp, - struct task_struct *task, bool threadgroup) + struct task_struct *task) (cgroup_mutex held by caller) Called prior to moving a task into a cgroup; if the subsystem @@ -572,9 +572,14 @@ task is passed, then a successful result indicates that *any* unspecified task can be moved into the cgroup. Note that this isn't called on a fork. If this method returns 0 (success) then this should remain valid while the caller holds cgroup_mutex …
creator: Ben Blum
date: 2011-02-08T01:39:15-00:00

[PATCH v8 3/3] cgroups: make procs file writable
type: item
description: Makes procs file writable to move all threads by tgid at once From: Ben Blum <bblum@andrew.cmu.edu> This patch adds functionality that enables users to move all threads in a threadgroup at once to a cgroup by writing the tgid to the 'cgroup.procs' file. This current implementation makes use of a per-threadgroup rwsem that's taken for reading in the fork() path to prevent newly forking threads within the threadgroup from "escaping" while the move is in progress. Signed-off-by: Ben Blum <bblum@andrew.cmu.edu> --- Documentation/cgroups/cgroups.txt | 9 + kernel/cgroup.c | 437 +++++++++++++++++++++++++++++++++---- 2 files changed, 397 insertions(+), 49 deletions(-) diff --git a/Documentation/cgroups/cgroups.txt b/Documentation/cgroups/cgroups.txt index d3c9a24..92d93d6 100644 --- a/Documentation/cgroups/cgroups.txt +++ b/Documentation/cgroups/cgroups.txt @@ -236,7 +236,8 @@ containing the following files describing that cgroup: - cgroup.procs: list of tgids in the cgroup. This list is not guaranteed to be sorted or free of duplicate tgids, and userspace should sort/uniquify the list if this property is required. - This is a read-only file, for now. + Writing a thread group id into this file moves all threads in that + group into this cgroup. - notify_on_release flag: run the release agent on exit? - release_agent: the path to use for release notifications (this file exists in the top cgroup only) @@ -426,6 +427,12 @@ You can attach the current shell task by echoing 0: # echo 0 > tasks +You can use the cgroup.procs file instead of the tasks file to move all +threads in a threadgroup at once. Echoing the pid of any task in a +threadgroup to cgroup.procs causes all tasks in that threadgroup to be +be attached to the cgroup. Writing 0 to cgroup.procs moves all tasks +in the writing task's threadgroup. + 2.3 Mounting hierarchies by name -------------------------------- diff --git a/kernel/cgroup.c b/kernel/cgroup.c index 616f27a..58b364a 100644 --- a/kernel/cgroup.c +++ b/ker…
creator: Ben Blum
date: 2011-02-08T01:39:50-00:00

Re: [PATCH v8 0/3] cgroups: implement moving a threadgroup's threads atomically with cgroup.procs
type: item
description: On Mon, 7 Feb 2011 20:35:42 -0500 Ben Blum <bblum@andrew.cmu.edu> wrote: > On Sun, Dec 26, 2010 at 07:09:19AM -0500, Ben Blum wrote: > > On Fri, Dec 24, 2010 at 03:22:26AM -0500, Ben Blum wrote: > > > On Wed, Aug 11, 2010 at 01:46:04AM -0400, Ben Blum wrote: > > > > On Fri, Jul 30, 2010 at 07:56:49PM -0400, Ben Blum wrote: > > > > > This patch series is a revision of http://lkml.org/lkml/2010/6/25/11 . > > > > > > > > > > This patch series implements a write function for the 'cgroup.procs' > > > > > per-cgroup file, which enables atomic movement of multithreaded > > > > > applications between cgroups. Writing the thread-ID of any thread in a > > > > > threadgroup to a cgroup's procs file causes all threads in the group to > > > > > be moved to that cgroup safely with respect to threads forking/exiting. > > > > > (Possible usage scenario: If running a multithreaded build system that > > > > > sucks up system resources, this lets you restrict it all at once into a > > > > > new cgroup to keep it under control.) > > > > > > > > > > Example: Suppose pid 31337 clones new threads 31338 and 31339. > > > > > > > > > > # cat /dev/cgroup/tasks > > > > > ... > > > > > 31337 > > > > > 31338 > > > > > 31339 > > > > > # mkdir /dev/cgroup/foo > > > > > # echo 31337 > /dev/cgroup/foo/cgroup.procs > > > > > # cat /dev/cgroup/foo/tasks > > > > > 31337 > > > > > 31338 > > > > > 31339 > > > > > > > > > > A new lock, called threadgroup_fork_lock and living in signal_struct, is > > > > > introduced to ensure atomicity when moving threads between cgroups. It's > > > > > taken for writing during the operation, and taking for reading in fork() > > > > > around the calls to cgroup_fork() and cgroup_post_fork(). The above six month old text is the best (and almost the only) explanation of the rationale for the entire patch series. Is it still correct and complete? Assuming "yes", then... how do we determine whether the feature is sufficiently useful to justify merging and maintaining it? Will…
creator: akpm
date: 2011-02-09T23:10:46-00:00

Re: [PATCH v8 0/3] cgroups: implement moving a threadgroup's threads atomically with cgroup.procs
type: item
description: On Wed, 9 Feb 2011 15:10:46 -0800 Andrew Morton <akpm@linux-foundation.org> wrote: > On Mon, 7 Feb 2011 20:35:42 -0500 > Ben Blum <bblum@andrew.cmu.edu> wrote: > > > On Sun, Dec 26, 2010 at 07:09:19AM -0500, Ben Blum wrote: > > > On Fri, Dec 24, 2010 at 03:22:26AM -0500, Ben Blum wrote: > > > > On Wed, Aug 11, 2010 at 01:46:04AM -0400, Ben Blum wrote: > > > > > On Fri, Jul 30, 2010 at 07:56:49PM -0400, Ben Blum wrote: > > > > > > This patch series is a revision of http://lkml.org/lkml/2010/6/25/11 . > > > > > > > > > > > > This patch series implements a write function for the 'cgroup.procs' > > > > > > per-cgroup file, which enables atomic movement of multithreaded > > > > > > applications between cgroups. Writing the thread-ID of any thread in a > > > > > > threadgroup to a cgroup's procs file causes all threads in the group to > > > > > > be moved to that cgroup safely with respect to threads forking/exiting. > > > > > > (Possible usage scenario: If running a multithreaded build system that > > > > > > sucks up system resources, this lets you restrict it all at once into a > > > > > > new cgroup to keep it under control.) > > > > > > > > > > > > Example: Suppose pid 31337 clones new threads 31338 and 31339. > > > > > > > > > > > > # cat /dev/cgroup/tasks > > > > > > ... > > > > > > 31337 > > > > > > 31338 > > > > > > 31339 > > > > > > # mkdir /dev/cgroup/foo > > > > > > # echo 31337 > /dev/cgroup/foo/cgroup.procs > > > > > > # cat /dev/cgroup/foo/tasks > > > > > > 31337 > > > > > > 31338 > > > > > > 31339 > > > > > > > > > > > > A new lock, called threadgroup_fork_lock and living in signal_struct, is > > > > > > introduced to ensure atomicity when moving threads between cgroups. It's > > > > > > taken for writing during the operation, and taking for reading in fork() > > > > > > around the calls to cgroup_fork() and cgroup_post_fork(). > > The above six month old text is the best (and almost the only) > explanation of the rationale for the entire patch series. Is …
creator: KAMEZAWA Hiroyuki
date: 2011-02-10T01:02:10-00:00

Re: [PATCH v8 0/3] cgroups: implement moving a threadgroup's threads atomically with cgroup.procs
type: item
description: On Thu, Feb 10, 2011 at 10:02:10AM +0900, KAMEZAWA Hiroyuki wrote: > On Wed, 9 Feb 2011 15:10:46 -0800 > Andrew Morton <akpm@linux-foundation.org> wrote: > > > On Mon, 7 Feb 2011 20:35:42 -0500 > > Ben Blum <bblum@andrew.cmu.edu> wrote: > > > > > On Sun, Dec 26, 2010 at 07:09:19AM -0500, Ben Blum wrote: > > > > On Fri, Dec 24, 2010 at 03:22:26AM -0500, Ben Blum wrote: > > > > > On Wed, Aug 11, 2010 at 01:46:04AM -0400, Ben Blum wrote: > > > > > > On Fri, Jul 30, 2010 at 07:56:49PM -0400, Ben Blum wrote: > > > > > > > This patch series is a revision of http://lkml.org/lkml/2010/6/25/11 . > > > > > > > > > > > > > > This patch series implements a write function for the 'cgroup.procs' > > > > > > > per-cgroup file, which enables atomic movement of multithreaded > > > > > > > applications between cgroups. Writing the thread-ID of any thread in a > > > > > > > threadgroup to a cgroup's procs file causes all threads in the group to > > > > > > > be moved to that cgroup safely with respect to threads forking/exiting. > > > > > > > (Possible usage scenario: If running a multithreaded build system that > > > > > > > sucks up system resources, this lets you restrict it all at once into a > > > > > > > new cgroup to keep it under control.) > > > > > > > > > > > > > > Example: Suppose pid 31337 clones new threads 31338 and 31339. > > > > > > > > > > > > > > # cat /dev/cgroup/tasks > > > > > > > ... > > > > > > > 31337 > > > > > > > 31338 > > > > > > > 31339 > > > > > > > # mkdir /dev/cgroup/foo > > > > > > > # echo 31337 > /dev/cgroup/foo/cgroup.procs > > > > > > > # cat /dev/cgroup/foo/tasks > > > > > > > 31337 > > > > > > > 31338 > > > > > > > 31339 > > > > > > > > > > > > > > A new lock, called threadgroup_fork_lock and living in signal_struct, is > > > > > > > introduced to ensure atomicity when moving threads between cgroups. It's > > > > > > > taken for writing during the operation, and taking for reading in fork() > > > > > > > around the calls to cgroup_fork() and cgrou…
creator: Ben Blum
date: 2011-02-10T01:36:06-00:00

Re: [PATCH v8 0/3] cgroups: implement moving a threadgroup's threads atomically with cgroup.procs
type: item
description: On Wed, Feb 9, 2011 at 5:02 PM, KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> wrote: > > So, I think it's ok to have 'procs' interface for cgroup if > overhead/impact of patch is not heavy. > Agreed - it's definitely an operation that comes up as either confusing or annoying for users, depending on whether or not they understand how threads and cgroups interact. (We've been getting people wanting to do this internally at Google, and I'm guessing that we're one of the bigger users of cgroups.) In theory it's something that could be handled in userspace, in one of two ways: - repeatedly scan the old cgroup's tasks file and sweep any threads from the given process into the destination cgroup, until you complete a clean sweep finding none. (Possibly even this is racy if a thread is being slow to fork) - use a process event notifier to catch thread fork events and keep track of any newly created threads that appear after your first sweep of threads, and be prepared to handle them for some reasonable length of time (tens of milliseconds?) after the last thread has been apparently moved. (The alternative approach, of course, is to give up and never try to move a process into a cgroup except right when you're in the middle of forking it, before the exec(), when you know that it has only a single thread and you're in control of it.) These are both painful procedures, compared to the very simple approach of letting the kernel move the entire process atomically. It's true that it's a pretty heavyweight operation, but that weight is only paid when you actually use it on a very large process (and which would be even more expensive to do in userspace). For the rest of the kernel, it's just an extra read lock in the fork path on a semaphore in a structure that's pretty much guaranteed to be in cache. Paul _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containe rs
creator: Paul Menage
date: 2011-02-14T06:12:19-00:00

Re: [PATCH v8 0/3] cgroups: implement moving a threadgroup's threads atomically with cgroup.procs
type: item
description: On Wed, Feb 9, 2011 at 3:10 PM, Andrew Morton <akpm@linux-foundation.org> wrote: > On Mon, 7 Feb 2011 20:35:42 -0500 > Ben Blum <bblum@andrew.cmu.edu> wrote: > >> On Sun, Dec 26, 2010 at 07:09:19AM -0500, Ben Blum wrote: >> > On Fri, Dec 24, 2010 at 03:22:26AM -0500, Ben Blum wrote: >> > > On Wed, Aug 11, 2010 at 01:46:04AM -0400, Ben Blum wrote: >> > > > On Fri, Jul 30, 2010 at 07:56:49PM -0400, Ben Blum wrote: >> > > > > This patch series is a revision of http://lkml.org/lkml/2010/6/25/11 . >> > > > > >> > > > > This patch series implements a write function for the 'cgroup.procs' >> > > > > per-cgroup file, which enables atomic movement of multithreaded >> > > > > applications between cgroups. Writing the thread-ID of any thread in a >> > > > > threadgroup to a cgroup's procs file causes all threads in the group to >> > > > > be moved to that cgroup safely with respect to threads forking/exiting. >> > > > > (Possible usage scenario: If running a multithreaded build system that >> > > > > sucks up system resources, this lets you restrict it all at once into a >> > > > > new cgroup to keep it under control.) >> > > > > > > The above six month old text is the best (and almost the only) > explanation of the rationale for the entire patch series. Is > it still correct and complete? > It's still correct, but I'm sure we could come up with a more detailed justification if necessary. Paul _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containe rs
creator: Paul Menage
date: 2011-02-14T06:12:51-00:00

[PATCH v8 4/3] cgroups: use flex_array in attach_proc
type: item
description: Convert cgroup_attach_proc to use flex_array. From: Ben Blum <bblum@andrew.cmu.edu> The cgroup_attach_proc implementation requires a pre-allocated array to store task pointers to atomically move a thread-group, but asking for a monolithic array with kmalloc() may be unreliable for very large groups. Using flex_array provides the same functionality with less risk of failure. This is a post-patch for cgroup-procs-write.patch. Signed-off-by: Ben Blum <bblum@andrew.cmu.edu> --- kernel/cgroup.c | 37 ++++++++++++++++++++++++++++--------- 1 files changed, 28 insertions(+), 9 deletions(-) diff --git a/kernel/cgroup.c b/kernel/cgroup.c index 58b364a..feba784 100644 --- a/kernel/cgroup.c +++ b/kernel/cgroup.c @@ -57,6 +57,7 @@ #include <linux/vmalloc.h> /* TODO: replace with more sophisticated array */ #include <linux/eventfd.h> #include <linux/poll.h> +#include <linux/flex_array.h> /* used in cgroup_attach_proc */ #include <asm/atomic.h> @@ -1985,7 +1986,7 @@ int cgroup_attach_proc(struct cgroup *cgrp, struct task_struct *leader) struct cgroupfs_root *root = cgrp->root; /* threadgroup list cursor and array */ struct task_struct *tsk; - struct task_struct **group; + struct flex_array *group; /* * we need to make sure we have css_sets for all the tasks we're * going to move -before- we actually start moving them, so that in @@ -2002,9 +2003,15 @@ int cgroup_attach_proc(struct cgroup *cgrp, struct task_struct *leader) * and if threads exit, this will just be an over-estimate. */ group_size = get_nr_threads(leader); - group = kmalloc(group_size * sizeof(*group), GFP_KERNEL); + /* flex_array supports very large thread-groups better than kmalloc. */ + group = flex_array_alloc(sizeof(struct task_struct *), group_size, + GFP_KERNEL); if (!group) return -ENOMEM; + /* pre-allocate to guarantee space while iterating in rcu read-side. */ + retval = flex_array_prealloc(group, 0, group_size - 1, GFP_KERNEL); + if (retval) + goto out_free_group_list; /* prevent changes to the threadgroup l…
creator: Ben Blum
date: 2011-02-16T19:22:00-00:00