OpenVZ Forum - RDF feed
type: channel
items: [RFC PATCH 0/5] Resend -v2 - Use procfs to change a syscall behavior
items: [RFC PATCH 1/5] adds the procfs facilities
items: [RFC PATCH 2/5] use next syscall data to predefine ipc objects ids
items: [RFC PATCH 4/5] use next syscall data to change the behavior of IPC_SET
items: Re: [RFC PATCH 1/5] adds the procfs facilities
items: Re: [RFC PATCH 2/5] use next syscall data to predefine ipc objects ids
items: Re: [RFC PATCH 4/5] use next syscall data to change the behavior of IPC_SET
items: Re: [RFC PATCH 0/5] Resend -v2 - Use procfs to change a syscall behavior

[RFC PATCH 0/5] Resend -v2 - Use procfs to change a syscall behavior
type: item
description: Resending after fixing the issues pointed out by Serge. Also ported to 2.6.26-rc8-mm1. Regards, Nadia -------------- This patchset is a part of an effort to change some syscalls behavior for checkpoint restart. When restarting an object that has previously been checkpointed, its state should be unchanged compared to the checkpointed image. For example, a restarted process should have the same upid nr as the one it used to have when being checkpointed; an ipc object should have the same id as the one it had when the checkpoint occured. Also, talking about system V ipcs, they should be restored with the same state (e.g. in terms of pid of last operation). This means that several syscalls should not behave in a default mode when they are called during a restart phase. One solution consists in defining a new syscall for each syscall that is called during restart: . sys_fork_with_id() would fork a process with a predefined id. . sys_msgget_with_id() would create a msg queue with a predefined id . sys_semget_with_id() would create a semaphore set with a predefined id . etc, This solution requires defining a new syscall each time we need an existing syscall to behave in a non-default way. An alternative to this solution consists in defining a new field in the task structure (let's call it next_syscall_data) that, if set, would change the behavior of next syscall to be called. The sys_fork_with_id() previously cited can be replaced by 1) set next_syscall_data to a target upid nr 2) call fork(). This patch series implements the 2nd solution. Actually I've already sent it some times ago, and things ended up with Pavel complaining about the "ugly interface" (see https://lists.linux-foundation.org/pipermail/containers/2008-April/010909.html). Now, I'm resending the series because this 2nd solution has the advantage of being easily reusable for many subsystems: the only thing needed is just to set a field in the task structure and rewrite the code portion that is sensitive to th…
creator: Nadia Derbey
date: 2008-07-08T11:24:22-00:00

[RFC PATCH 1/5] adds the procfs facilities
type: item
description: [PATCH 01/05] This patch adds the procfs facility needed to feed some data for the next syscall to be called. The effect of issuing echo "LONG<Y> <XX>" > /proc/self/task/<tid>/next_syscall_data is that <XX> will be stored in a new field of the task structure (next_syscall_data). This field, in turn will be taken as the data to feed next syscall that supports the feature. <Y> is the number of values provided on the line. For the sake of simplicity it is now fixed to 1, but this can be extended as needed, in the future. This is particularly useful when restarting an application, as we need sometimes the syscalls to have a non-default behavior. Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> --- fs/exec.c | 6 + fs/proc/base.c | 75 ++++++++++++++++++ include/linux/next_syscall_data.h | 32 ++++++++ include/linux/sched.h | 6 + kernel/Makefile | 3 kernel/exit.c | 4 + kernel/fork.c | 2 kernel/next_syscall_data.c | 151 ++++++++++++++++++++++++++++++++++++++ 8 files changed, 278 insertions(+), 1 deletion(-) Index: linux-2.6.26-rc8-mm1/include/linux/sched.h =================================================================== --- linux-2.6.26-rc8-mm1.orig/include/linux/sched.h 2008-07-08 09:04:21.000000000 +0200 +++ linux-2.6.26-rc8-mm1/include/linux/sched.h 2008-07-08 09:13:43.000000000 +0200 @@ -87,6 +87,7 @@ struct sched_param { #include <linux/task_io_accounting.h> #include <linux/kobject.h> #include <linux/latencytop.h> +#include <linux/next_syscall_data.h> #include <asm/processor.h> @@ -1296,6 +1297,11 @@ struct task_struct { int latency_record_count; struct latency_record latency_record[LT_SAVECOUNT]; #endif + /* + * If non-NULL indicates that next operation will be forced, e.g. + * that next object to be created will have a predefined id. + */ + struct next_syscall_data *nsd; }; /* Index: linux-2.6.26-rc8-mm1/include/linux/next_syscall_data.h =================================================================== --- /dev/null 1970-01-01 00:00:00.000000000 +0000 +++ linu…
creator: Nadia Derbey
date: 2008-07-08T11:24:23-00:00

[RFC PATCH 2/5] use next syscall data to predefine ipc objects ids
type: item
description: [PATCH 02/05] This patch uses the value written into the next_syscall_data proc file as a target id for the next IPC object to be created. The following syscalls have a new behavior if next_syscall_data is set: . mssget() . semget() . shmget() Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> --- include/linux/next_syscall_data.h | 17 ++++++++++++++-- ipc/util.c | 39 ++++++++++++++++++++++++++++++-------- 2 files changed, 46 insertions(+), 10 deletions(-) Index: linux-2.6.26-rc8-mm1/include/linux/next_syscall_data.h =================================================================== --- linux-2.6.26-rc8-mm1.orig/include/linux/next_syscall_data.h 2008-07-08 09:24:38.000000000 +0200 +++ linux-2.6.26-rc8-mm1/include/linux/next_syscall_data.h 2008-07-08 12:12:39.000000000 +0200 @@ -1,7 +1,10 @@ /* * include/linux/next_syscall_data.h * - * Definitions to support fixed data for next syscall to be called. + * Definitions to support fixed data for next syscall to be called. The + * following is supported today: + * . object creation with a predefined id + * . for a sysv ipc object */ #ifndef _LINUX_NEXT_SYSCALL_DATA_H @@ -13,13 +16,23 @@ * If this structure is pointed to by a task_struct, next syscall to be called * by the task will have a non-default behavior. * For example, it can be used to pre-set the id of the object to be created - * by next syscall. + * by next syscall. The following syscalls support this feature: + * . msgget(), semget(), shmget() */ struct next_syscall_data { int ndata; long data[NDATA]; }; +/* + * Returns true if tsk has some data set in its next_syscall_data, 0 else + */ +#define next_data_set(tsk) ((tsk)->nsd \ + ? ((tsk)->nsd->ndata ? 1 : 0) \ + : 0) + +#define get_next_data(tsk) ((tsk)->nsd->data[0]) + extern ssize_t get_next_syscall_data(struct task_struct *, char *, size_t); extern int set_next_syscall_data(struct task_struct *, char *); extern void reset_next_syscall_data(struct task_struct *); Index: linux-2.6.26-rc8-mm1/ipc/util.c =====…
creator: Nadia Derbey
date: 2008-07-08T11:24:24-00:00

[RFC PATCH 4/5] use next syscall data to change the behavior of IPC_SET
type: item
description: [PATCH 04/05] This patch uses the value written into the next_syscall_data proc file as a flag to change the way msgctl(IPC_SET), semctl(IPC_SET) and shmctl(IPC_SET) behave. When "LONG1 1" is echoed to this file, xxxctl(IPC_SET) will set the time fields and the pid fields according to what is specified in the input parameter (while currently only the permission fields are allowed to be set). The following syscalls are impacted: . msgctl(IPC_SET) . semctl(IPC_SET) . shmctl(IPC_SET) This makes it easy to restart an ipc object exactly is it was during the checkpoint phase. Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> --- include/linux/next_syscall_data.h | 12 ++++++++++++ ipc/msg.c | 19 ++++++++++++++++++- ipc/sem.c | 16 +++++++++++++++- ipc/shm.c | 19 ++++++++++++++++++- 4 files changed, 63 insertions(+), 3 deletions(-) Index: linux-2.6.26-rc8-mm1/include/linux/next_syscall_data.h =================================================================== --- linux-2.6.26-rc8-mm1.orig/include/linux/next_syscall_data.h 2008-07-08 12:22:47.000000000 +0200 +++ linux-2.6.26-rc8-mm1/include/linux/next_syscall_data.h 2008-07-08 12:24:29.000000000 +0200 @@ -6,6 +6,7 @@ * . object creation with a predefined id * . for a sysv ipc object * . for a process + * . set more than the usual ipc_perm fields during and IPC_SET operation. */ #ifndef _LINUX_NEXT_SYSCALL_DATA_H @@ -20,6 +21,10 @@ * by next syscall. The following syscalls support this feature: * . msgget(), semget(), shmget() * . fork(), vfork(), clone() + * + * If it is set to a non null value before a call to: + * . msgctl(IPC_SET), semctl(IPC_SET), shmctl(IPC_SET), + * this means that we are going to set more than the usual ipc_perms fields. */ struct next_syscall_data { int ndata; @@ -35,6 +40,13 @@ struct next_syscall_data { #define get_next_data(tsk) ((tsk)->nsd->data[0]) +/* + * Returns true if next call to xxxctl(IPC_SET) should have a non-default + * behavior. + */ +#define ipc_set_all(tsk) (next_data_set(tsk) ? ge…
creator: Nadia Derbey
date: 2008-07-08T11:24:26-00:00

Re: [RFC PATCH 1/5] adds the procfs facilities
type: item
description: Quoting Nadia.Derbey@bull.net (Nadia.Derbey@bull.net): > [PATCH 01/05] > > This patch adds the procfs facility needed to feed some data for the > next syscall to be called. > > The effect of issuing > echo "LONG<Y> <XX>" > /proc/self/task/<tid>/next_syscall_data > is that <XX> will be stored in a new field of the task structure > (next_syscall_data). This field, in turn will be taken as the data to feed > next syscall that supports the feature. > > <Y> is the number of values provided on the line. > For the sake of simplicity it is now fixed to 1, but this can be extended as > needed, in the future. > > This is particularly useful when restarting an application, as we need > sometimes the syscalls to have a non-default behavior. > > Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> Acked-by: Serge Hallyn <serue@us.ibm.com> thanks, -serge > > --- > fs/exec.c | 6 + > fs/proc/base.c | 75 ++++++++++++++++++ > include/linux/next_syscall_data.h | 32 ++++++++ > include/linux/sched.h | 6 + > kernel/Makefile | 3 > kernel/exit.c | 4 + > kernel/fork.c | 2 > kernel/next_syscall_data.c | 151 ++++++++++++++++++++++++++++++++++++++ > 8 files changed, 278 insertions(+), 1 deletion(-) > > Index: linux-2.6.26-rc8-mm1/include/linux/sched.h > =================================================================== > --- linux-2.6.26-rc8-mm1.orig/include/linux/sched.h 2008-07-08 09:04:21.000000000 +0200 > +++ linux-2.6.26-rc8-mm1/include/linux/sched.h 2008-07-08 09:13:43.000000000 +0200 > @@ -87,6 +87,7 @@ struct sched_param { > #include <linux/task_io_accounting.h> > #include <linux/kobject.h> > #include <linux/latencytop.h> > +#include <linux/next_syscall_data.h> > > #include <asm/processor.h> > > @@ -1296,6 +1297,11 @@ struct task_struct { > int latency_record_count; > struct latency_record latency_record[LT_SAVECOUNT]; > #endif > + /* > + * If non-NULL indicates that next operation will be forced, e.g. > + * that next object to be created will have a predefined id. > + */ > + struct ne…
creator: serue
date: 2008-07-08T19:32:54-00:00

Re: [RFC PATCH 2/5] use next syscall data to predefine ipc objects ids
type: item
description: Quoting Nadia.Derbey@bull.net (Nadia.Derbey@bull.net): > [PATCH 02/05] > > This patch uses the value written into the next_syscall_data proc file > as a target id for the next IPC object to be created. > The following syscalls have a new behavior if next_syscall_data is set: > . mssget() > . semget() > . shmget() > > Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> Acked-by: Serge Hallyn <serue@us.ibm.com> thanks, -serge > > --- > include/linux/next_syscall_data.h | 17 ++++++++++++++-- > ipc/util.c | 39 ++++++++++++++++++++++++++++++-------- > 2 files changed, 46 insertions(+), 10 deletions(-) > > Index: linux-2.6.26-rc8-mm1/include/linux/next_syscall_data.h > =================================================================== > --- linux-2.6.26-rc8-mm1.orig/include/linux/next_syscall_data.h 2008-07-08 09:24:38.000000000 +0200 > +++ linux-2.6.26-rc8-mm1/include/linux/next_syscall_data.h 2008-07-08 12:12:39.000000000 +0200 > @@ -1,7 +1,10 @@ > /* > * include/linux/next_syscall_data.h > * > - * Definitions to support fixed data for next syscall to be called. > + * Definitions to support fixed data for next syscall to be called. The > + * following is supported today: > + * . object creation with a predefined id > + * . for a sysv ipc object > */ > > #ifndef _LINUX_NEXT_SYSCALL_DATA_H > @@ -13,13 +16,23 @@ > * If this structure is pointed to by a task_struct, next syscall to be called > * by the task will have a non-default behavior. > * For example, it can be used to pre-set the id of the object to be created > - * by next syscall. > + * by next syscall. The following syscalls support this feature: > + * . msgget(), semget(), shmget() > */ > struct next_syscall_data { > int ndata; > long data[NDATA]; > }; > > +/* > + * Returns true if tsk has some data set in its next_syscall_data, 0 else > + */ > +#define next_data_set(tsk) ((tsk)->nsd \ > + ? ((tsk)->nsd->ndata ? 1 : 0) \ > + : 0) > + > +#define get_next_data(tsk) ((tsk)->nsd->data[0]) > + > extern ssize_t get_ne…
creator: serue
date: 2008-07-08T19:38:08-00:00

Re: [RFC PATCH 4/5] use next syscall data to change the behavior of IPC_SET
type: item
description: Quoting Nadia.Derbey@bull.net (Nadia.Derbey@bull.net): > [PATCH 04/05] > > This patch uses the value written into the next_syscall_data proc file > as a flag to change the way msgctl(IPC_SET), semctl(IPC_SET) and > shmctl(IPC_SET) behave. > > When "LONG1 1" is echoed to this file, xxxctl(IPC_SET) will set the time > fields and the pid fields according to what is specified in the input > parameter (while currently only the permission fields are allowed to be set). > The following syscalls are impacted: > . msgctl(IPC_SET) > . semctl(IPC_SET) > . shmctl(IPC_SET) > > This makes it easy to restart an ipc object exactly is it was during the > checkpoint phase. > > Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> Acked-by: Serge Hallyn <serue@us.ibm.com> thanks, -serge > --- > include/linux/next_syscall_data.h | 12 ++++++++++++ > ipc/msg.c | 19 ++++++++++++++++++- > ipc/sem.c | 16 +++++++++++++++- > ipc/shm.c | 19 ++++++++++++++++++- > 4 files changed, 63 insertions(+), 3 deletions(-) > > Index: linux-2.6.26-rc8-mm1/include/linux/next_syscall_data.h > =================================================================== > --- linux-2.6.26-rc8-mm1.orig/include/linux/next_syscall_data.h 2008-07-08 12:22:47.000000000 +0200 > +++ linux-2.6.26-rc8-mm1/include/linux/next_syscall_data.h 2008-07-08 12:24:29.000000000 +0200 > @@ -6,6 +6,7 @@ > * . object creation with a predefined id > * . for a sysv ipc object > * . for a process > + * . set more than the usual ipc_perm fields during and IPC_SET operation. > */ > > #ifndef _LINUX_NEXT_SYSCALL_DATA_H > @@ -20,6 +21,10 @@ > * by next syscall. The following syscalls support this feature: > * . msgget(), semget(), shmget() > * . fork(), vfork(), clone() > + * > + * If it is set to a non null value before a call to: > + * . msgctl(IPC_SET), semctl(IPC_SET), shmctl(IPC_SET), > + * this means that we are going to set more than the usual ipc_perms fields. > */ > struct next_syscall_data { > int ndata; > @@ -35,6 +40,13 @@ struct next_sy…
creator: serue
date: 2008-07-08T19:56:18-00:00

Re: [RFC PATCH 0/5] Resend -v2 - Use procfs to change a syscall behavior
type: item
description: On Tue, Jul 08, 2008 at 01:24:22PM +0200, Nadia.Derbey@bull.net wrote: > # echo "LONG1 XX" > /proc/self/task/<my_tid>/next_syscall_data Same stuff. There is struct task_struct::did_exec , what about it? Also, patches are about de-serializing, how serializing from userspace looks like? You freezed group of processes, then what? How, for example, dump all VMAs correctly? [prepares counter-example] _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Alexey Dobriyan
date: 2008-07-09T22:10:28-00:00

Re: [RFC PATCH 0/5] Resend -v2 - Use procfs to change a syscall behavior
type: item
description: Nadia.Derbey@bull.net writes: > This patchset is a part of an effort to change some syscalls behavior for > checkpoint restart. Thanks for doing this. Unfortunately this makes a very good case of why we don't want to go down this route. Adding magic parameters to syscalls that are only useful in one very specific restart case. We need good clean interfaces with well defined semantics. Something as narrow focused on this is not really useful and it takes a lot of code to do something very few people will want to actively do. > The syntax is: > # echo "LONG1 XX" > /proc/self/task/<my_tid>/next_syscall_data > next object to be created will have an id set to XX Which his horrible in another way because it is hugely race prone. Eric _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: ebiederm
date: 2008-07-10T00:36:46-00:00

Re: [RFC PATCH 0/5] Resend -v2 - Use procfs to change a syscall behavior
type: item
description: Alexey Dobriyan <adobriyan@gmail.com> writes: > On Tue, Jul 08, 2008 at 01:24:22PM +0200, Nadia.Derbey@bull.net wrote: >> # echo "LONG1 XX" > /proc/self/task/<my_tid>/next_syscall_data > > Same stuff. > > There is struct task_struct::did_exec , what about it? > > Also, patches are about de-serializing, how serializing from userspace looks > like? > You freezed group of processes, then what? > > How, for example, dump all VMAs correctly? > [prepares counter-example] Alexey userspace vs a kernel space implementation is the wrong argument. It is clearly established that the current user space interfaces are insufficient to do the job. So we need to implement something in the kernel. Further I have heard of no one suggesting running a single kernel on multiple machines. Therefore there no one seems to be doing this entirely in the kernel and so we need a user space component. So the question should not be user space vs. kernel space but can we build clean interfaces for checkpoint/restart? What will those interfaces be? Although I think it is good that we are seeing more people play with this as that should mean that our pool of people for doing code review on the implementation should be reasonable. Eric _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: ebiederm
date: 2008-07-10T00:43:04-00:00

Re: [RFC PATCH 0/5] Resend -v2 - Use procfs to change a syscall behavior
type: item
description: On Wed, Jul 09, 2008 at 05:43:04PM -0700, Eric W. Biederman wrote: > Alexey Dobriyan <adobriyan@gmail.com> writes: > > > On Tue, Jul 08, 2008 at 01:24:22PM +0200, Nadia.Derbey@bull.net wrote: > >> # echo "LONG1 XX" > /proc/self/task/<my_tid>/next_syscall_data > > > > Same stuff. > > > > There is struct task_struct::did_exec , what about it? > > > > Also, patches are about de-serializing, how serializing from userspace looks > > like? > > You freezed group of processes, then what? > > > > How, for example, dump all VMAs correctly? > > [prepares counter-example] > > Alexey userspace vs a kernel space implementation is the wrong argument. > > It is clearly established that the current user space interfaces are > insufficient to do the job. So we need to implement something in the kernel. > > Further I have heard of no one suggesting running a single kernel on multiple > machines. Therefore there no one seems to be doing this entirely in the kernel > and so we need a user space component. > > So the question should not be user space vs. kernel space but can we build clean > interfaces for checkpoint/restart? > What will those interfaces be? In case of ->did_exec the only clean interface I see is: tsk->did_exec = !!tsk_img->did_exec; It would be pretty silly to wrap this one line in a system call (two actually -- one in, one out), since you're going to restore some more fields of such variety anyway (like ->pdeath_signal). Given the diversity of kernel internal data structures and all sorts of links between them, the only system call suitable is ioctl(2), not all this zoo of system calls proposed. They are all extendable and without rules, but ioctl(2) is also without rules. This is all said in assumption that serializing kernel-internal data for checkpoint/restart to userspace is acceptable for mainline. I don't think it is. _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/ma…
creator: Alexey Dobriyan
date: 2008-07-10T01:39:15-00:00

Re: [RFC PATCH 0/5] Resend -v2 - Use procfs to change a syscall behavior
type: item
description: Alexey Dobriyan <adobriyan@gmail.com> writes: > In case of ->did_exec the only clean interface I see is: > > tsk->did_exec = !!tsk_img->did_exec; > > It would be pretty silly to wrap this one line in a system call (two > actually -- one in, one out), since you're going to restore some more > fields of such variety anyway (like ->pdeath_signal). There I agree the granularity seems small enough to be a major pain for the implementation. > Given the diversity of kernel internal data structures and all sorts of > links between them, the only system call suitable is ioctl(2), not all > this zoo of system calls proposed. They are all extendable and without > rules, but ioctl(2) is also without rules. At least for processes my gut reaction is to look at binary formats and coredumps. Something with at least that large of a granularity seems to make most sense. > This is all said in assumption that serializing kernel-internal data for > checkpoint/restart to userspace is acceptable for mainline. > I don't think it is. I don't believe that serializing kernel-internal data is acceptable for mainline. I believe that serializing user-visible data is acceptable. Note: user-visible data does not mean user-manipulatable data. On a socket you may not save the skbs but you can save the pending packets for example. Assuming the transition cost is not too high. Eric _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: ebiederm
date: 2008-07-10T02:14:37-00:00

Re: [RFC PATCH 0/5] Resend -v2 - Use procfs to change a syscall behavior
type: item
description: Eric W. Biederman wrote: > Nadia.Derbey@bull.net writes: > > >>This patchset is a part of an effort to change some syscalls behavior for >>checkpoint restart. > > > Thanks for doing this. > > Unfortunately this makes a very good case of why we don't want to go down > this route. Adding magic parameters to syscalls that are only useful > in one very specific restart case. > > We need good clean interfaces with well defined semantics. > > Something as narrow focused on this is not really useful and it takes > a lot of code to do something very few people will want to actively > do. All this seems reasonable. Ok, so since we are taking the "new syscalls" direction, I'll try to make a list of the potentially duplicated syscalls. Regards, Nadia > > >>The syntax is: >># echo "LONG1 XX" > /proc/self/task/<my_tid>/next_syscall_data >> next object to be created will have an id set to XX > > > Which his horrible in another way because it is hugely race prone. > > Eric > > > _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Nadia Derbey
date: 2008-07-10T09:54:07-00:00

Re: [RFC PATCH 0/5] Resend -v2 - Use procfs to change a syscall behavior
type: item
description: On Thu, 2008-07-10 at 02:10 +0400, Alexey Dobriyan wrote: > How, for example, dump all VMAs correctly? > [prepares counter-example] Are there some particular pitfalls that you'd like to share? I'd love to hear some of the issues the you've run into with Virtuozzo as its implementation was created. -- Dave _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Dave Hansen
date: 2008-07-10T16:01:42-00:00

Re: [RFC PATCH 0/5] Resend -v2 - Use procfs to change a syscall behavior
type: item
description: Alexey Dobriyan <adobriyan@gmail.com> writes: > This is all said in assumption that serializing kernel-internal data for > checkpoint/restart to userspace is acceptable for mainline. > I don't think it is. Just a quick comment here. We mentioned checkpoint/restart is where we were going last kernel summit, and no one was opposed. So while I expect technical objects if we are not careful, I believe a well chosen checkpoint/restart framework has every chance of being merged into mainline. Eric _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: ebiederm
date: 2008-07-15T18:18:29-00:00

Re: [RFC PATCH 0/5] Resend -v2 - Use procfs to change a syscall behavior
type: item
description: Eric W. Biederman wrote: > Alexey Dobriyan <adobriyan@gmail.com> writes: I seem to not have received any of Alexey's emails... ? > >> On Tue, Jul 08, 2008 at 01:24:22PM +0200, Nadia.Derbey@bull.net wrote: >>> # echo "LONG1 XX" > /proc/self/task/<my_tid>/next_syscall_data >> Same stuff. >> >> There is struct task_struct::did_exec , what about it? >> >> Also, patches are about de-serializing, how serializing from userspace looks >> like? >> You freezed group of processes, then what? >> >> How, for example, dump all VMAs correctly? >> [prepares counter-example] > > Alexey userspace vs a kernel space implementation is the wrong argument. > > It is clearly established that the current user space interfaces are > insufficient to do the job. So we need to implement something in the kernel. > > Further I have heard of no one suggesting running a single kernel on multiple > machines. Therefore there no one seems to be doing this entirely in the kernel > and so we need a user space component. I'm not sure I understand this argument ? In a kernel implementation, the component will merely open a file descriptor (to which the data will be streamed), freeze the container and invoke a system call. In a userland implementation, the component will do most of the work by continuously probing the kernel for information about the processes that are being checkpointed. So, of course we need a "component" - but what does that component do ? > So the question should not be user space vs. kernel space but can we build clean > interfaces for checkpoint/restart? What will those interfaces be? My question is why build a set of interfaces to export this and that from the kernel to user space ? if a kernel implementation (with minimal user space support) is chosen, then information extraction (and restoration) is straightforward and we don't get ourselves tied until the end of times to API exported to userland. The output of the module will be a binary (like a core dump) that can be used by the…
creator: Oren Laadan
date: 2008-07-17T22:42:56-00:00

Re: [RFC PATCH 0/5] Resend -v2 - Use procfs to change a syscall behavior
type: item
description: On Thu, 2008-07-17 at 18:42 -0400, Oren Laadan wrote: > > Eric W. Biederman wrote: > > Alexey Dobriyan <adobriyan@gmail.com> writes: > > I seem to not have received any of Alexey's emails... ? > > > > >> On Tue, Jul 08, 2008 at 01:24:22PM +0200, Nadia.Derbey@bull.net wrote: > >>> # echo "LONG1 XX" > /proc/self/task/<my_tid>/next_syscall_data > >> Same stuff. > >> > >> There is struct task_struct::did_exec , what about it? > >> > >> Also, patches are about de-serializing, how serializing from userspace looks > >> like? > >> You freezed group of processes, then what? > >> > >> How, for example, dump all VMAs correctly? > >> [prepares counter-example] > > > > Alexey userspace vs a kernel space implementation is the wrong argument. > > > > It is clearly established that the current user space interfaces are > > insufficient to do the job. So we need to implement something in the kernel. > > > > Further I have heard of no one suggesting running a single kernel on multiple > > machines. Therefore there no one seems to be doing this entirely in the kernel > > and so we need a user space component. > > I'm not sure I understand this argument ? > > In a kernel implementation, the component will merely open a file descriptor > (to which the data will be streamed), freeze the container and invoke a > system call. In a userland implementation, the component will do most of > the work by continuously probing the kernel for information about the > processes that are being checkpointed. > > So, of course we need a "component" - but what does that component do ? > > > So the question should not be user space vs. kernel space but can we build clean > > interfaces for checkpoint/restart? What will those interfaces be? > > My question is why build a set of interfaces to export this and that from > the kernel to user space ? if a kernel implementation (with minimal user > space support) is chosen, then information extraction (and restoration) is > straightforward and we don't get ourse…
creator: Matt Helsley
date: 2008-07-18T01:09:25-00:00

Re: [RFC PATCH 0/5] Resend -v2 - Use procfs to change a syscall behavior
type: item
description: Oren Laadan <orenl@cs.columbia.edu> writes: > I think the kernel space vs. user space must be the first issue on our > table to solve, as it has a wide impact on the rest of the work. We first need to talk about what kinds of problems we are trying to solve. If we don't agree what the problem is I expect we will have a hard time agreeing on a solution. For example we are using namespaces now instead of the potentially simpler isolation mechanism of Vserver because checkpoint/restart could not be done with the Vserver approach. The use case that I expect we all have in common is migrating an isolated container from one machine to another transparent to applications. Except those that directly access the hardware at which point we can treat it as a hotplug event from the perspective of userspace. There are several other interesting use cases that I think we should solve if possible. - Live/Incremental migration. - Remote fork. Which can be seen as an extreme case of migrating only a partial container. - A checkpoint that can be restarted multiple times and work properly. Which means you need to include the state of the filesystem. - A distributed checkpoint of multiple containers at the same time. Given how brutally hard and inefficient it is to restore a checkpoint using the existing system calls even with namespaces in the kernel. We can pretty much rule that implementation out as it does not match our efficiency criteria, and likely isn't especially maintainble either. On the maintenance side we can generally rule out an out of tree module. As that does not afford visible to people changing a subsystem that the checkpoint/restart code needs to change as well. I believe the live migration will have the most stringent performance requirements and at the same time be one of the most useful features, as it immediately improve maintenance of clusters. In the extreme case of a distributed checkpoint the kernel simply does not have enough state so we need user space code …
creator: ebiederm
date: 2008-07-18T02:40:45-00:00

Re: [RFC PATCH 0/5] Resend -v2 - Use procfs to change a syscall behavior
type: item
description: Matt Helsley <matthltc@us.ibm.com> writes: > On Thu, 2008-07-17 at 18:42 -0400, Oren Laadan wrote: >> >> My question is why build a set of interfaces to export this and that from >> the kernel to user space ? if a kernel implementation (with minimal user >> space support) is chosen, then information extraction (and restoration) is >> straightforward and we don't get ourselves tied until the end of times to >> API exported to userland. > > That still seems like an API exported to userland. It just combines the > data into one block rather than distributing it amongst a bunch of > pseudo-filesystems. Does this form of API really free us from always > supporting it in the future? A larger granularity reduces the support burden. You don't wind up introducing a bunch of little system calls that you only use for restore. You introduce one that does exactly what you need it to do. Because you know it is only used in checkpoint/restart conditions you can make assumptions about the users and have more freedom. Yes it would still be a user/kernel interface. If we abstract it something like binformats are abstracted we may eventually be able to stop including an old format that no one uses anymore. > > Userspace is expected to inspect or convert the binary data. How does > that truly avoid many of the API issues mentioned above? If it's really > supposed to be a minimal API then the binary should be considered opaque > and userspace tools which inspect or convert these binaries should be > considered unreliable hacks at best. Otherwise it seems to me that it > has most of the familiar problems associated with a kernel/userspace API > -- including an obligation to support it. The best precedent we have for something like this today is the core dump. That is a single process and does not do well at tying multiple processes together. Even though you can inspect a core dump there is still a lot of freedom in the implementation that we would not have in a more general API. As for u…
creator: ebiederm
date: 2008-07-18T02:49:05-00:00