OpenVZ Forum - RDF feed
type: channel
items: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
items: [RFC PATCH 1/5] adds the procfs facilities
items: [RFC PATCH 2/5] use next syscall data to predefine ipc objects ids
items: [RFC PATCH 3/5] use next syscall data to predefine process ids
items: [RFC PATCH 4/5] use next syscall data to change the behavior of IPC_SET
items: [RFC PATCH 5/5] use next syscall data to predefine the file descriptor value
items: Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
items: Re: [RFC PATCH 1/5] adds the procfs facilities
items: Re: [RFC PATCH 2/5] use next syscall data to predefine ipc objects ids
items: Re: [RFC PATCH 3/5] use next syscall data to predefine process ids
items: Re: Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior

[RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: This patchset is a part of an effort to change some syscalls behavior for checkpoint restart. When restarting an object that has previously been checkpointed, its state should be unchanged compared to the checkpointed image. For example, a restarted process should have the same upid nr as the one it used to have when being checkpointed; an ipc object should have the same id as the one it had when the checkpoint occured. Also, talking about system V ipcs, they should be restored with the same state (e.g. in terms of pid of last operation). This means that several syscalls should not behave in a default mode when they are called during a restart phase. One solution consists in defining a new syscall for each syscall that is called during restart: . sys_fork_with_id() would fork a process with a predefined id. . sys_msgget_with_id() would create a msg queue with a predefined id . sys_semget_with_id() would create a semaphore set with a predefined id . etc, This solution requires defining a new syscall each time we need an existing syscall to behave in a non-default way. An alternative to this solution consists in defining a new field in the task structure (let's call it next_syscall_data) that, if set, would change the behavior of next syscall to be called. The sys_fork_with_id() previously cited can be replaced by 1) set next_syscall_data to a target upid nr 2) call fork(). This patch series implements the 2nd solution. Actually I've already sent it some times ago, and things ended up with Pavel complaining about the "ugly interface" (see https://lists.linux-foundation.org/pipermail/containers/2008-April/010909.html). Now, I'm resending the series because this 2nd solution has the advantage of being easily reusable for many subsystems: the only thing needed is just to set a field in the task structure and rewrite the code portion that is sensitive to this field being set (it's successfully being used in cryo code - git tree at git://git.sr71.net/~hallyn/cryodev.git). …
creator: Nadia Derbey
date: 2008-07-03T14:40:13-00:00

[RFC PATCH 1/5] adds the procfs facilities
type: item
description: [PATCH 01/05] This patch adds the procfs facility needed to feed some data for the next syscall to be called. The effect of issuing echo "LONG<Y> <XX>" > /proc/self/task/<tid>/next_syscall_data is that <XX> will be stored in a new field of the task structure (next_syscall_data). This field, in turn will be taken as the data to feed next syscall that supports the feature. <Y> is the number of values provided on the line. For the sake of simplicity it is now fixed to 1, but this can be extended as needed, in the future. This is particularly useful when restarting an application, as we need sometimes the syscalls to have a non-default behavior. Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> --- fs/exec.c | 6 + fs/proc/base.c | 75 ++++++++++++++++++ include/linux/next_syscall_data.h | 35 ++++++++ include/linux/sched.h | 6 + kernel/Makefile | 3 kernel/exit.c | 4 + kernel/fork.c | 2 kernel/next_syscall_data.c | 151 ++++++++++++++++++++++++++++++++++++++ 8 files changed, 281 insertions(+), 1 deletion(-) Index: linux-2.6.26-rc5-mm3/include/linux/sched.h =================================================================== --- linux-2.6.26-rc5-mm3.orig/include/linux/sched.h 2008-06-25 17:10:38.000000000 +0200 +++ linux-2.6.26-rc5-mm3/include/linux/sched.h 2008-06-27 14:18:56.000000000 +0200 @@ -87,6 +87,7 @@ struct sched_param { #include <linux/task_io_accounting.h> #include <linux/kobject.h> #include <linux/latencytop.h> +#include <linux/next_syscall_data.h> #include <asm/processor.h> @@ -1312,6 +1313,11 @@ struct task_struct { int latency_record_count; struct latency_record latency_record[LT_SAVECOUNT]; #endif + /* + * If non-NULL indicates that next operation will be forced, e.g. + * that next object to be created will have a predefined id. + */ + struct next_syscall_data *nsd; }; /* Index: linux-2.6.26-rc5-mm3/include/linux/next_syscall_data.h =================================================================== --- /dev/null 1970-01-01 00:00:00.000000000 +0000 +++ linu…
creator: Nadia Derbey
date: 2008-07-03T14:40:14-00:00

[RFC PATCH 2/5] use next syscall data to predefine ipc objects ids
type: item
description: [PATCH 02/05] This patch uses the value written into the next_syscall_data proc file as a target id for the next IPC object to be created. The following syscalls have a new behavior if next_syscall_data is set: . mssget() . semget() . shmget() Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> --- include/linux/next_syscall_data.h | 17 +++++++++++++++-- ipc/util.c | 38 ++++++++++++++++++++++++++++++-------- 2 files changed, 45 insertions(+), 10 deletions(-) Index: linux-2.6.26-rc5-mm3/include/linux/next_syscall_data.h =================================================================== --- linux-2.6.26-rc5-mm3.orig/include/linux/next_syscall_data.h 2008-07-01 10:25:48.000000000 +0200 +++ linux-2.6.26-rc5-mm3/include/linux/next_syscall_data.h 2008-07-01 11:35:11.000000000 +0200 @@ -3,7 +3,8 @@ * * Definitions to support fixed data for next syscall to be called. The * following is supported today: - * . object creation with a predefined id. + * . object creation with a predefined id + * . for a sysv ipc object * */ @@ -16,13 +17,25 @@ * If this structure is pointed to by a task_struct, next syscall to be called * by the task will have a non-default behavior. * For example, it can be used to pre-set the id of the object to be created - * by next syscall. + * by next syscall. The following syscalls support this feature: + * . msgget(), semget(), shmget() */ struct next_syscall_data { int ndata; long data[NDATA]; }; +/* + * Returns true if tsk has some data set in its next_syscall_data, 0 else + */ +#define next_data_set(tsk) ((tsk)->nsd \ + ? ((tsk)->nsd->ndata ? 1 : 0) \ + : 0) + +#define get_next_data(tsk) ((tsk)->nsd->data[0]) + + + extern ssize_t get_next_syscall_data(struct task_struct *, char *, size_t); extern int set_next_syscall_data(struct task_struct *, char *); extern int reset_next_syscall_data(struct task_struct *); Index: linux-2.6.26-rc5-mm3/ipc/util.c =================================================================== --- linux-2.6.26-rc5-mm3.orig/ipc/u…
creator: Nadia Derbey
date: 2008-07-03T14:40:15-00:00

[RFC PATCH 3/5] use next syscall data to predefine process ids
type: item
description: [PATCH 03/05] This patch uses the value written into the next_syscall_data proc file as a target upid nr for the next process to be created. The following syscalls have a new behavior if next_syscall_data is set: . fork() . vfork() . clone() In the current version, if the process belongs to nested namespaces, only the upper namespace level upid nr is allowed to be predefined, since there is not yet a way to take a snapshot of upid nrs at all namespaces levels. But this can easily be extended in the future. Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> --- include/linux/next_syscall_data.h | 2 include/linux/pid.h | 2 kernel/fork.c | 3 - kernel/pid.c | 111 ++++++++++++++++++++++++++++++++------ 4 files changed, 98 insertions(+), 20 deletions(-) Index: linux-2.6.26-rc5-mm3/kernel/pid.c =================================================================== --- linux-2.6.26-rc5-mm3.orig/kernel/pid.c 2008-07-01 10:25:46.000000000 +0200 +++ linux-2.6.26-rc5-mm3/kernel/pid.c 2008-07-01 11:25:38.000000000 +0200 @@ -122,6 +122,26 @@ static void free_pidmap(struct upid *upi atomic_inc(&map->nr_free); } +static inline int alloc_pidmap_page(struct pidmap *map) +{ + if (unlikely(!map->page)) { + void *page = kzalloc(PAGE_SIZE, GFP_KERNEL); + /* + * Free the page if someone raced with us + * installing it: + */ + spin_lock_irq(&pidmap_lock); + if (map->page) + kfree(page); + else + map->page = page; + spin_unlock_irq(&pidmap_lock); + if (unlikely(!map->page)) + return -1; + } + return 0; +} + static int alloc_pidmap(struct pid_namespace *pid_ns) { int i, offset, max_scan, pid, last = pid_ns->last_pid; @@ -134,21 +154,8 @@ static int alloc_pidmap(struct pid_names map = &pid_ns->pidmap[pid/BITS_PER_PAGE]; max_scan = (pid_max + BITS_PER_PAGE - 1)/BITS_PER_PAGE - !offset; for (i = 0; i <= max_scan; ++i) { - if (unlikely(!map->page)) { - void *page = kzalloc(PAGE_SIZE, GFP_KERNEL); - /* - * Free the page if someone raced with us - * installing it: - */ - spin_lock_irq(&pidmap_lock);…
creator: Nadia Derbey
date: 2008-07-03T14:40:16-00:00

[RFC PATCH 4/5] use next syscall data to change the behavior of IPC_SET
type: item
description: [PATCH 04/05] This patch uses the value written into the next_syscall_data proc file as a flag to change the way msgctl(IPC_SET), semctl(IPC_SET) and shmctl(IPC_SET) behave. When "LONG1 1" is echoed to this file, xxxctl(IPC_SET) will set the time fields and the pid fields according to what is specified in the input parameter (while currently only the permission fields are allowed to be set). The following syscalls are impacted: . msgctl(IPC_SET) . semctl(IPC_SET) . shmctl(IPC_SET) This makes it easy to restart an ipc object exactly is it was during the checkpoint phase. Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> --- include/linux/next_syscall_data.h | 12 +++++++++++- ipc/msg.c | 19 ++++++++++++++++++- ipc/sem.c | 16 +++++++++++++++- ipc/shm.c | 19 ++++++++++++++++++- 4 files changed, 62 insertions(+), 4 deletions(-) Index: linux-2.6.26-rc5-mm3/include/linux/next_syscall_data.h =================================================================== --- linux-2.6.26-rc5-mm3.orig/include/linux/next_syscall_data.h 2008-07-01 12:07:46.000000000 +0200 +++ linux-2.6.26-rc5-mm3/include/linux/next_syscall_data.h 2008-07-01 12:07:50.000000000 +0200 @@ -6,7 +6,7 @@ * . object creation with a predefined id * . for a sysv ipc object * . for a process - * + * . set more than the usual ipc_perm fields during and IPC_SET operation. */ #ifndef _LINUX_NEXT_SYSCALL_DATA_H @@ -21,6 +21,10 @@ * by next syscall. The following syscalls support this feature: * . msgget(), semget(), shmget() * . fork(), vfork(), clone() + * + * If it is set to a non null value before a call to: + * . msgctl(IPC_SET), semctl(IPC_SET), shmctl(IPC_SET), + * this means that we are going to set more than the usual ipc_perms fields. */ struct next_syscall_data { int ndata; @@ -36,6 +40,12 @@ struct next_syscall_data { #define get_next_data(tsk) ((tsk)->nsd->data[0]) +/* + * Returns true if next call to xxxctl(IPC_SET) should have a non-default + * behavior. + */ +#define ipc_set_all(tsk) (next_data_set(tsk) …
creator: Nadia Derbey
date: 2008-07-03T14:40:17-00:00

[RFC PATCH 5/5] use next syscall data to predefine the file descriptor value
type: item
description: [PATCH 05/05] This patch uses the value written into the next_syscall_data proc file as a target file descriptor for the next file to be opened. This makes it easy to restart a process with the same fds as the ones it was using during the checkpoint phase, instead of 1. opening the file, 2. dup2'ing the open file descriptor. The following syscalls are impacted if next_syscall_data is set: . open() . openat() Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> --- fs/open.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 57 insertions(+), 1 deletion(-) Index: linux-2.6.26-rc5-mm3/fs/open.c =================================================================== --- linux-2.6.26-rc5-mm3.orig/fs/open.c 2008-06-25 17:11:06.000000000 +0200 +++ linux-2.6.26-rc5-mm3/fs/open.c 2008-07-01 17:51:53.000000000 +0200 @@ -967,6 +967,55 @@ struct file *dentry_open(struct dentry * EXPORT_SYMBOL(dentry_open); /* + * Marks a given file descriptor entry as busy (should not be busy when this + * routine is called. + * + * files->next_fd is not updated: this lets the potentially created hole be + * filled up on next calls to get_unused_fd_flags. + * + * Returns the specified fd if successful, -errno else. + */ +static int get_predefined_fd_flags(int fd, int flags) +{ + struct files_struct *files = current->files; + int error; + struct fdtable *fdt; + + error = -EINVAL; + if (fd < 0) + goto out; + + error = -EMFILE; + if (fd >= current->signal->rlim[RLIMIT_NOFILE].rlim_cur) + goto out; + + spin_lock(&files->file_lock); + fdt = files_fdtable(files); + + error = expand_files(files, fd); + if (error < 0) + goto out_unlock; + + error = -EBUSY; + if (FD_ISSET(fd, fdt->open_fds)) + goto out_unlock; + + FD_SET(fd, fdt->open_fds); + if (flags & O_CLOEXEC) + FD_SET(fd, fdt->close_on_exec); + else + FD_CLR(fd, fdt->close_on_exec); + + BUG_ON(fdt->fd[fd] != NULL); + + error = fd; +out_unlock: + spin_unlock(&files->file_lock); +out: + return error; +} + +/* * Find an empty…
creator: Nadia Derbey
date: 2008-07-03T14:40:18-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: Hi! > This patchset is a part of an effort to change some syscalls behavior for > checkpoint restart. > > When restarting an object that has previously been checkpointed, its state > should be unchanged compared to the checkpointed image. > For example, a restarted process should have the same upid nr as the one it > used to have when being checkpointed; an ipc object should have the same id > as the one it had when the checkpoint occured. > Also, talking about system V ipcs, they should be restored with the same > state (e.g. in terms of pid of last operation). > > This means that several syscalls should not behave in a default mode when > they are called during a restart phase. > > One solution consists in defining a new syscall for each syscall that is > called during restart: > . sys_fork_with_id() would fork a process with a predefined id. > . sys_msgget_with_id() would create a msg queue with a predefined id > . sys_semget_with_id() would create a semaphore set with a predefined id > . etc, > > This solution requires defining a new syscall each time we need an existing > syscall to behave in a non-default way. Yes, and I believe that's better than... > An alternative to this solution consists in defining a new field in the > task structure (let's call it next_syscall_data) that, if set, would change > the behavior of next syscall to be called. The sys_fork_with_id() previously > cited can be replaced by > 1) set next_syscall_data to a target upid nr > 2) call fork(). ...bloat task struct and > A new file is created in procfs: /proc/self/task/<my_tid>/next_syscall_data. > This makes it possible to avoid races between several threads belonging to > the same process. ...introducing this kind of uglyness. Actually, there were proposals for sys_indirect(), which is slightly less ugly, but IIRC we ended up with adding syscalls, too. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog…
creator: Pavel Machek
date: 2008-07-04T10:27:02-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: Pavel Machek wrote: > Hi! > > >>This patchset is a part of an effort to change some syscalls behavior for >>checkpoint restart. >> >>When restarting an object that has previously been checkpointed, its state >>should be unchanged compared to the checkpointed image. >>For example, a restarted process should have the same upid nr as the one it >>used to have when being checkpointed; an ipc object should have the same id >>as the one it had when the checkpoint occured. >>Also, talking about system V ipcs, they should be restored with the same >>state (e.g. in terms of pid of last operation). >> >>This means that several syscalls should not behave in a default mode when >>they are called during a restart phase. >> >>One solution consists in defining a new syscall for each syscall that is >>called during restart: >> . sys_fork_with_id() would fork a process with a predefined id. >> . sys_msgget_with_id() would create a msg queue with a predefined id >> . sys_semget_with_id() would create a semaphore set with a predefined id >> . etc, >> >>This solution requires defining a new syscall each time we need an existing >>syscall to behave in a non-default way. > > > Yes, and I believe that's better than... > > >>An alternative to this solution consists in defining a new field in the >>task structure (let's call it next_syscall_data) that, if set, would change >>the behavior of next syscall to be called. The sys_fork_with_id() previously >>cited can be replaced by >> 1) set next_syscall_data to a target upid nr >> 2) call fork(). > > > ...bloat task struct and > > >>A new file is created in procfs: /proc/self/task/<my_tid>/next_syscall_data. >>This makes it possible to avoid races between several threads belonging to >>the same process. > > > ...introducing this kind of uglyness. > > Actually, there were proposals for sys_indirect(), which is slightly > less ugly, but IIRC we ended up with adding syscalls, too. > Pavel Pavel, I had a look at the lwn.net article that describes t…
creator: Nadia Derbey
date: 2008-07-04T12:07:18-00:00

Re: [RFC PATCH 1/5] adds the procfs facilities
type: item
description: Quoting Nadia.Derbey@bull.net (Nadia.Derbey@bull.net): > [PATCH 01/05] > > This patch adds the procfs facility needed to feed some data for the > next syscall to be called. > > The effect of issuing > echo "LONG<Y> <XX>" > /proc/self/task/<tid>/next_syscall_data > is that <XX> will be stored in a new field of the task structure > (next_syscall_data). This field, in turn will be taken as the data to feed > next syscall that supports the feature. > > <Y> is the number of values provided on the line. > For the sake of simplicity it is now fixed to 1, but this can be extended as > needed, in the future. > > This is particularly useful when restarting an application, as we need > sometimes the syscalls to have a non-default behavior. > > Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> > > --- > fs/exec.c | 6 + > fs/proc/base.c | 75 ++++++++++++++++++ > include/linux/next_syscall_data.h | 35 ++++++++ > include/linux/sched.h | 6 + > kernel/Makefile | 3 > kernel/exit.c | 4 + > kernel/fork.c | 2 > kernel/next_syscall_data.c | 151 ++++++++++++++++++++++++++++++++++++++ > 8 files changed, 281 insertions(+), 1 deletion(-) > > Index: linux-2.6.26-rc5-mm3/include/linux/sched.h > =================================================================== > --- linux-2.6.26-rc5-mm3.orig/include/linux/sched.h 2008-06-25 17:10:38.000000000 +0200 > +++ linux-2.6.26-rc5-mm3/include/linux/sched.h 2008-06-27 14:18:56.000000000 +0200 > @@ -87,6 +87,7 @@ struct sched_param { > #include <linux/task_io_accounting.h> > #include <linux/kobject.h> > #include <linux/latencytop.h> > +#include <linux/next_syscall_data.h> > > #include <asm/processor.h> > > @@ -1312,6 +1313,11 @@ struct task_struct { > int latency_record_count; > struct latency_record latency_record[LT_SAVECOUNT]; > #endif > + /* > + * If non-NULL indicates that next operation will be forced, e.g. > + * that next object to be created will have a predefined id. > + */ > + struct next_syscall_data *nsd; > }; > > /* > Index: linux-2.6.26-r…
creator: serue
date: 2008-07-07T18:30:30-00:00

Re: [RFC PATCH 2/5] use next syscall data to predefine ipc objects ids
type: item
description: Quoting Nadia.Derbey@bull.net (Nadia.Derbey@bull.net): > [PATCH 02/05] > > This patch uses the value written into the next_syscall_data proc file > as a target id for the next IPC object to be created. > The following syscalls have a new behavior if next_syscall_data is set: > . mssget() > . semget() > . shmget() > > Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> > > --- > include/linux/next_syscall_data.h | 17 +++++++++++++++-- > ipc/util.c | 38 ++++++++++++++++++++++++++++++-------- > 2 files changed, 45 insertions(+), 10 deletions(-) > > Index: linux-2.6.26-rc5-mm3/include/linux/next_syscall_data.h > =================================================================== > --- linux-2.6.26-rc5-mm3.orig/include/linux/next_syscall_data.h 2008-07-01 10:25:48.000000000 +0200 > +++ linux-2.6.26-rc5-mm3/include/linux/next_syscall_data.h 2008-07-01 11:35:11.000000000 +0200 > @@ -3,7 +3,8 @@ > * > * Definitions to support fixed data for next syscall to be called. The > * following is supported today: > - * . object creation with a predefined id. > + * . object creation with a predefined id > + * . for a sysv ipc object > * > */ > > @@ -16,13 +17,25 @@ > * If this structure is pointed to by a task_struct, next syscall to be called > * by the task will have a non-default behavior. > * For example, it can be used to pre-set the id of the object to be created > - * by next syscall. > + * by next syscall. The following syscalls support this feature: > + * . msgget(), semget(), shmget() > */ > struct next_syscall_data { > int ndata; > long data[NDATA]; > }; > > +/* > + * Returns true if tsk has some data set in its next_syscall_data, 0 else > + */ > +#define next_data_set(tsk) ((tsk)->nsd \ > + ? ((tsk)->nsd->ndata ? 1 : 0) \ > + : 0) > + > +#define get_next_data(tsk) ((tsk)->nsd->data[0]) > + > + > + > extern ssize_t get_next_syscall_data(struct task_struct *, char *, size_t); > extern int set_next_syscall_data(struct task_struct *, char *); > extern int reset_next_syscall_d…
creator: serue
date: 2008-07-07T18:35:12-00:00

Re: [RFC PATCH 3/5] use next syscall data to predefine process ids
type: item
description: Quoting Nadia.Derbey@bull.net (Nadia.Derbey@bull.net): > [PATCH 03/05] > > This patch uses the value written into the next_syscall_data proc file > as a target upid nr for the next process to be created. > The following syscalls have a new behavior if next_syscall_data is set: > . fork() > . vfork() > . clone() > > In the current version, if the process belongs to nested namespaces, only > the upper namespace level upid nr is allowed to be predefined, since there > is not yet a way to take a snapshot of upid nrs at all namespaces levels. > > But this can easily be extended in the future. Good point, we will want to discuss the right way to dump that data. Do we add a new file under /proc/<pid>, use /proc/pid/status, or find some other way? > Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> > > --- > include/linux/next_syscall_data.h | 2 > include/linux/pid.h | 2 > kernel/fork.c | 3 - > kernel/pid.c | 111 ++++++++++++++++++++++++++++++++------ > 4 files changed, 98 insertions(+), 20 deletions(-) > > Index: linux-2.6.26-rc5-mm3/kernel/pid.c > =================================================================== > --- linux-2.6.26-rc5-mm3.orig/kernel/pid.c 2008-07-01 10:25:46.000000000 +0200 > +++ linux-2.6.26-rc5-mm3/kernel/pid.c 2008-07-01 11:25:38.000000000 +0200 > @@ -122,6 +122,26 @@ static void free_pidmap(struct upid *upi > atomic_inc(&map->nr_free); > } > > +static inline int alloc_pidmap_page(struct pidmap *map) > +{ > + if (unlikely(!map->page)) { > + void *page = kzalloc(PAGE_SIZE, GFP_KERNEL); > + /* > + * Free the page if someone raced with us > + * installing it: > + */ > + spin_lock_irq(&pidmap_lock); > + if (map->page) > + kfree(page); > + else > + map->page = page; > + spin_unlock_irq(&pidmap_lock); > + if (unlikely(!map->page)) > + return -1; > + } > + return 0; > +} > + > static int alloc_pidmap(struct pid_namespace *pid_ns) > { > int i, offset, max_scan, pid, last = pid_ns->last_pid; > @@ -134,21 +154,8 @@ static int alloc_pidmap(struct pid_names >…
creator: serue
date: 2008-07-07T18:54:24-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: Quoting Pavel Machek (pavel@ucw.cz): > Hi! > > > This patchset is a part of an effort to change some syscalls behavior for > > checkpoint restart. > > > > When restarting an object that has previously been checkpointed, its state > > should be unchanged compared to the checkpointed image. > > For example, a restarted process should have the same upid nr as the one it > > used to have when being checkpointed; an ipc object should have the same id > > as the one it had when the checkpoint occured. > > Also, talking about system V ipcs, they should be restored with the same > > state (e.g. in terms of pid of last operation). > > > > This means that several syscalls should not behave in a default mode when > > they are called during a restart phase. > > > > One solution consists in defining a new syscall for each syscall that is > > called during restart: > > . sys_fork_with_id() would fork a process with a predefined id. > > . sys_msgget_with_id() would create a msg queue with a predefined id > > . sys_semget_with_id() would create a semaphore set with a predefined id > > . etc, > > > > This solution requires defining a new syscall each time we need an existing > > syscall to behave in a non-default way. > > Yes, and I believe that's better than... > > > An alternative to this solution consists in defining a new field in the > > task structure (let's call it next_syscall_data) that, if set, would change > > the behavior of next syscall to be called. The sys_fork_with_id() previously > > cited can be replaced by > > 1) set next_syscall_data to a target upid nr > > 2) call fork(). > > ...bloat task struct and > > > A new file is created in procfs: /proc/self/task/<my_tid>/next_syscall_data. > > This makes it possible to avoid races between several threads belonging to > > the same process. > > ...introducing this kind of uglyness. > > Actually, there were proposals for sys_indirect(), which is slightly > less ugly, but IIRC we ended up with adding syscalls, too. > Pavel …
creator: serue
date: 2008-07-07T19:01:19-00:00

Re: [RFC PATCH 1/5] adds the procfs facilities
type: item
description: Serge E. Hallyn wrote: > Quoting Nadia.Derbey@bull.net (Nadia.Derbey@bull.net): > >>[PATCH 01/05] >> >>This patch adds the procfs facility needed to feed some data for the >>next syscall to be called. >> >>The effect of issuing >>echo "LONG<Y> <XX>" > /proc/self/task/<tid>/next_syscall_data >>is that <XX> will be stored in a new field of the task structure >>(next_syscall_data). This field, in turn will be taken as the data to feed >>next syscall that supports the feature. >> >><Y> is the number of values provided on the line. >>For the sake of simplicity it is now fixed to 1, but this can be extended as >>needed, in the future. >> >>This is particularly useful when restarting an application, as we need >>sometimes the syscalls to have a non-default behavior. >> >>Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> >> >>--- >> fs/exec.c | 6 + >> fs/proc/base.c | 75 ++++++++++++++++++ >> include/linux/next_syscall_data.h | 35 ++++++++ >> include/linux/sched.h | 6 + >> kernel/Makefile | 3 >> kernel/exit.c | 4 + >> kernel/fork.c | 2 >> kernel/next_syscall_data.c | 151 ++++++++++++++++++++++++++++++++++++++ >> 8 files changed, 281 insertions(+), 1 deletion(-) >> >>Index: linux-2.6.26-rc5-mm3/include/linux/sched.h >>=================================================================== >>--- linux-2.6.26-rc5-mm3.orig/include/linux/sched.h 2008-06-25 17:10:38.000000000 +0200 >>+++ linux-2.6.26-rc5-mm3/include/linux/sched.h 2008-06-27 14:18:56.000000000 +0200 >>@@ -87,6 +87,7 @@ struct sched_param { >> #include <linux/task_io_accounting.h> >> #include <linux/kobject.h> >> #include <linux/latencytop.h> >>+#include <linux/next_syscall_data.h> >> >> #include <asm/processor.h> >> >>@@ -1312,6 +1313,11 @@ struct task_struct { >> int latency_record_count; >> struct latency_record latency_record[LT_SAVECOUNT]; >> #endif >>+ /* >>+ * If non-NULL indicates that next operation will be forced, e.g. >>+ * that next object to be created will have a predefined id. >>+ */ >>+ struct next_sy…
creator: Nadia Derbey
date: 2008-07-08T05:25:18-00:00

Re: [RFC PATCH 2/5] use next syscall data to predefine ipc objects ids
type: item
description: Serge E. Hallyn wrote: > Quoting Nadia.Derbey@bull.net (Nadia.Derbey@bull.net): > >>[PATCH 02/05] >> >>This patch uses the value written into the next_syscall_data proc file >>as a target id for the next IPC object to be created. >>The following syscalls have a new behavior if next_syscall_data is set: >>. mssget() >>. semget() >>. shmget() >> >>Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> >> >>--- >> include/linux/next_syscall_data.h | 17 +++++++++++++++-- >> ipc/util.c | 38 ++++++++++++++++++++++++++++++-------- >> 2 files changed, 45 insertions(+), 10 deletions(-) >> >>Index: linux-2.6.26-rc5-mm3/include/linux/next_syscall_data.h >>=================================================================== >>--- linux-2.6.26-rc5-mm3.orig/include/linux/next_syscall_data.h 2008-07-01 10:25:48.000000000 +0200 >>+++ linux-2.6.26-rc5-mm3/include/linux/next_syscall_data.h 2008-07-01 11:35:11.000000000 +0200 >>@@ -3,7 +3,8 @@ >> * >> * Definitions to support fixed data for next syscall to be called. The >> * following is supported today: >>- * . object creation with a predefined id. >>+ * . object creation with a predefined id >>+ * . for a sysv ipc object >> * >> */ >> >>@@ -16,13 +17,25 @@ >> * If this structure is pointed to by a task_struct, next syscall to be called >> * by the task will have a non-default behavior. >> * For example, it can be used to pre-set the id of the object to be created >>- * by next syscall. >>+ * by next syscall. The following syscalls support this feature: >>+ * . msgget(), semget(), shmget() >> */ >> struct next_syscall_data { >> int ndata; >> long data[NDATA]; >> }; >> >>+/* >>+ * Returns true if tsk has some data set in its next_syscall_data, 0 else >>+ */ >>+#define next_data_set(tsk) ((tsk)->nsd \ >>+ ? ((tsk)->nsd->ndata ? 1 : 0) \ >>+ : 0) >>+ >>+#define get_next_data(tsk) ((tsk)->nsd->data[0]) >>+ >>+ >>+ >> extern ssize_t get_next_syscall_data(struct task_struct *, char *, size_t); >> extern int set_next_syscall_data(struct task_s…
creator: Nadia Derbey
date: 2008-07-08T05:30:37-00:00

Re: [RFC PATCH 3/5] use next syscall data to predefine process ids
type: item
description: Serge E. Hallyn wrote: > Quoting Nadia.Derbey@bull.net (Nadia.Derbey@bull.net): > >>[PATCH 03/05] >> >>This patch uses the value written into the next_syscall_data proc file >>as a target upid nr for the next process to be created. >>The following syscalls have a new behavior if next_syscall_data is set: >>. fork() >>. vfork() >>. clone() >> >>In the current version, if the process belongs to nested namespaces, only >>the upper namespace level upid nr is allowed to be predefined, since there >>is not yet a way to take a snapshot of upid nrs at all namespaces levels. >> >>But this can easily be extended in the future. > > > Good point, we will want to discuss the right way to dump that data. Do > we add a new file under /proc/<pid>, use /proc/pid/status, or find some > other way? > > >>Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net> >> >>--- >> include/linux/next_syscall_data.h | 2 >> include/linux/pid.h | 2 >> kernel/fork.c | 3 - >> kernel/pid.c | 111 ++++++++++++++++++++++++++++++++------ >> 4 files changed, 98 insertions(+), 20 deletions(-) >> >>Index: linux-2.6.26-rc5-mm3/kernel/pid.c >>=================================================================== >>--- linux-2.6.26-rc5-mm3.orig/kernel/pid.c 2008-07-01 10:25:46.000000000 +0200 >>+++ linux-2.6.26-rc5-mm3/kernel/pid.c 2008-07-01 11:25:38.000000000 +0200 >>@@ -122,6 +122,26 @@ static void free_pidmap(struct upid *upi >> atomic_inc(&map->nr_free); >> } >> >>+static inline int alloc_pidmap_page(struct pidmap *map) >>+{ >>+ if (unlikely(!map->page)) { >>+ void *page = kzalloc(PAGE_SIZE, GFP_KERNEL); >>+ /* >>+ * Free the page if someone raced with us >>+ * installing it: >>+ */ >>+ spin_lock_irq(&pidmap_lock); >>+ if (map->page) >>+ kfree(page); >>+ else >>+ map->page = page; >>+ spin_unlock_irq(&pidmap_lock); >>+ if (unlikely(!map->page)) >>+ return -1; >>+ } >>+ return 0; >>+} >>+ >> static int alloc_pidmap(struct pid_namespace *pid_ns) >> { >> int i, offset, max_scan, pid, last = pid_ns->last_pid; >>@@ -13…
creator: Nadia Derbey
date: 2008-07-08T05:44:22-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: Hi! >>> An alternative to this solution consists in defining a new field in the >>> task structure (let's call it next_syscall_data) that, if set, would change >>> the behavior of next syscall to be called. The sys_fork_with_id() previously >>> cited can be replaced by >>> 1) set next_syscall_data to a target upid nr >>> 2) call fork(). >> >> >> ...bloat task struct and >> >> >>> A new file is created in procfs: /proc/self/task/<my_tid>/next_syscall_data. >>> This makes it possible to avoid races between several threads belonging to >>> the same process. >> >> >> ...introducing this kind of uglyness. >> >> Actually, there were proposals for sys_indirect(), which is slightly >> less ugly, but IIRC we ended up with adding syscalls, too. > I had a look at the lwn.net article that describes the sys_indirect() > interface. > It does exactly what we need here, so I do like it, but it has the same > drawbacks as the one you're complaining about: > . a new field is needed in the task structure > . looks like many people found it ugly... > Now, coming back to what I'm proposing: what we need is actually to change > the behavior of *existing* syscalls, since we are in a very particular > context (restarting an application). Changing existing syscalls is _bad_: for backwards compatibility reasons. strace will be very confusing to read, etc... > Defining brand new syscalls is very touchy: needs to be careful about the > interface + I can't imagine the number of syscalls that would be > needed. Of course new syscalls is touchy... modifying _existing_ should be even more touchy. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Pavel Machek
date: 2008-07-08T10:51:43-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: On Mon 2008-07-07 14:01:19, Serge E. Hallyn wrote: > Quoting Pavel Machek (pavel@ucw.cz): > > Hi! > > > > > This patchset is a part of an effort to change some syscalls behavior for > > > checkpoint restart. > > > > > > When restarting an object that has previously been checkpointed, its state > > > should be unchanged compared to the checkpointed image. > > > For example, a restarted process should have the same upid nr as the one it > > > used to have when being checkpointed; an ipc object should have the same id > > > as the one it had when the checkpoint occured. > > > Also, talking about system V ipcs, they should be restored with the same > > > state (e.g. in terms of pid of last operation). > > > > > > This means that several syscalls should not behave in a default mode when > > > they are called during a restart phase. > > > > > > One solution consists in defining a new syscall for each syscall that is > > > called during restart: > > > . sys_fork_with_id() would fork a process with a predefined id. > > > . sys_msgget_with_id() would create a msg queue with a predefined id > > > . sys_semget_with_id() would create a semaphore set with a predefined id > > > . etc, > > > > > > This solution requires defining a new syscall each time we need an existing > > > syscall to behave in a non-default way. > > > > Yes, and I believe that's better than... > > > > > An alternative to this solution consists in defining a new field in the > > > task structure (let's call it next_syscall_data) that, if set, would change > > > the behavior of next syscall to be called. The sys_fork_with_id() previously > > > cited can be replaced by > > > 1) set next_syscall_data to a target upid nr > > > 2) call fork(). > > > > ...bloat task struct and > > > > > A new file is created in procfs: /proc/self/task/<my_tid>/next_syscall_data. > > > This makes it possible to avoid races between several threads belonging to > > > the same process. > > > > ...introducing this kind of uglyness. > > >…
creator: Pavel Machek
date: 2008-07-08T10:52:28-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: Quoting Pavel Machek (pavel@ucw.cz): > Hi! > > >>> An alternative to this solution consists in defining a new field in the > >>> task structure (let's call it next_syscall_data) that, if set, would change > >>> the behavior of next syscall to be called. The sys_fork_with_id() previously > >>> cited can be replaced by > >>> 1) set next_syscall_data to a target upid nr > >>> 2) call fork(). > >> > >> > >> ...bloat task struct and > >> > >> > >>> A new file is created in procfs: /proc/self/task/<my_tid>/next_syscall_data. > >>> This makes it possible to avoid races between several threads belonging to > >>> the same process. > >> > >> > >> ...introducing this kind of uglyness. > >> > >> Actually, there were proposals for sys_indirect(), which is slightly > >> less ugly, but IIRC we ended up with adding syscalls, too. > > > I had a look at the lwn.net article that describes the sys_indirect() > > interface. > > It does exactly what we need here, so I do like it, but it has the same > > drawbacks as the one you're complaining about: > > . a new field is needed in the task structure > > . looks like many people found it ugly... > > > Now, coming back to what I'm proposing: what we need is actually to change > > the behavior of *existing* syscalls, since we are in a very particular > > context (restarting an application). > > Changing existing syscalls is _bad_: for backwards compatibility > reasons. strace will be very confusing to read, etc... I dunno... if you normally open(), you get back a random fd. If you do it having set the next_id inadvertently, then as far as you know you get back a random fd, no? > > Defining brand new syscalls is very touchy: needs to be careful about the > > interface + I can't imagine the number of syscalls that would be > > needed. > > Of course new syscalls is touchy... modifying _existing_ should be > even more touchy. -serge _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org http…
creator: serue
date: 2008-07-08T21:47:21-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: Quoting Pavel Machek (pavel@ucw.cz): > On Mon 2008-07-07 14:01:19, Serge E. Hallyn wrote: > > Quoting Pavel Machek (pavel@ucw.cz): > > > Hi! > > > > > > > This patchset is a part of an effort to change some syscalls behavior for > > > > checkpoint restart. > > > > > > > > When restarting an object that has previously been checkpointed, its state > > > > should be unchanged compared to the checkpointed image. > > > > For example, a restarted process should have the same upid nr as the one it > > > > used to have when being checkpointed; an ipc object should have the same id > > > > as the one it had when the checkpoint occured. > > > > Also, talking about system V ipcs, they should be restored with the same > > > > state (e.g. in terms of pid of last operation). > > > > > > > > This means that several syscalls should not behave in a default mode when > > > > they are called during a restart phase. > > > > > > > > One solution consists in defining a new syscall for each syscall that is > > > > called during restart: > > > > . sys_fork_with_id() would fork a process with a predefined id. > > > > . sys_msgget_with_id() would create a msg queue with a predefined id > > > > . sys_semget_with_id() would create a semaphore set with a predefined id > > > > . etc, > > > > > > > > This solution requires defining a new syscall each time we need an existing > > > > syscall to behave in a non-default way. > > > > > > Yes, and I believe that's better than... > > > > > > > An alternative to this solution consists in defining a new field in the > > > > task structure (let's call it next_syscall_data) that, if set, would change > > > > the behavior of next syscall to be called. The sys_fork_with_id() previously > > > > cited can be replaced by > > > > 1) set next_syscall_data to a target upid nr > > > > 2) call fork(). > > > > > > ...bloat task struct and > > > > > > > A new file is created in procfs: /proc/self/task/<my_tid>/next_syscall_data. > > > > This makes it possible to avoid…
creator: serue
date: 2008-07-08T21:50:34-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: On Tue 2008-07-08 16:47:21, Serge E. Hallyn wrote: > Quoting Pavel Machek (pavel@ucw.cz): > > Hi! > > > > >>> An alternative to this solution consists in defining a new field in the > > >>> task structure (let's call it next_syscall_data) that, if set, would change > > >>> the behavior of next syscall to be called. The sys_fork_with_id() previously > > >>> cited can be replaced by > > >>> 1) set next_syscall_data to a target upid nr > > >>> 2) call fork(). > > >> > > >> > > >> ...bloat task struct and > > >> > > >> > > >>> A new file is created in procfs: /proc/self/task/<my_tid>/next_syscall_data. > > >>> This makes it possible to avoid races between several threads belonging to > > >>> the same process. > > >> > > >> > > >> ...introducing this kind of uglyness. > > >> > > >> Actually, there were proposals for sys_indirect(), which is slightly > > >> less ugly, but IIRC we ended up with adding syscalls, too. > > > > > I had a look at the lwn.net article that describes the sys_indirect() > > > interface. > > > It does exactly what we need here, so I do like it, but it has the same > > > drawbacks as the one you're complaining about: > > > . a new field is needed in the task structure > > > . looks like many people found it ugly... > > > > > Now, coming back to what I'm proposing: what we need is actually to change > > > the behavior of *existing* syscalls, since we are in a very particular > > > context (restarting an application). > > > > Changing existing syscalls is _bad_: for backwards compatibility > > reasons. strace will be very confusing to read, etc... > > I dunno... if you normally open(), you get back a random fd. If you do > it having set the next_id inadvertently, then as far as you know you get > back a random fd, no? Sorry?! No, open does not return random fds. It allocates them bottom-up. So you do not need any changes in open case. (If you want to open "/foo/bar" as fd #50, open /dev/zero 49 times, then open "/foo/bar"; bash already uses that trick.…
creator: Pavel Machek
date: 2008-07-08T21:53:15-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: > > > > > An alternative to this solution consists in defining a new field in the > > > > > task structure (let's call it next_syscall_data) that, if set, would change > > > > > the behavior of next syscall to be called. The sys_fork_with_id() previously > > > > > cited can be replaced by > > > > > 1) set next_syscall_data to a target upid nr > > > > > 2) call fork(). > > > > > > > > ...bloat task struct and > > > > > > > > > A new file is created in procfs: /proc/self/task/<my_tid>/next_syscall_data. > > > > > This makes it possible to avoid races between several threads belonging to > > > > > the same process. > > > > > > > > ...introducing this kind of uglyness. > > > > > > > > Actually, there were proposals for sys_indirect(), which is slightly > > > > less ugly, but IIRC we ended up with adding syscalls, too. > > > > > > Silly question... > > > > > > Oren, would you object to defining sys_fork_with_id(), > > > sys_msgget_with_id(), and sys_semget_with_id()? > > > > > > Eric, Pavel (Emelyanov), Dave, do you have preferences? > > > > > > For the cases Nadia has implemented here I'd be tempted to side with > > > Pavel Machek, but once we get to things like open() and socket(), (a) > > > the # new syscalls starts to jump, and (b) the per-syscall api starts to > > > seem a lot more cumbersome. > > > > You should not need to modify open/socket. You can already select fd > > by creatively using open/dup/close... > > That's what we do right now in cryo. And if we end up patching up every > API with separate syscalls, then we wouldn't create open_with_id(). But > so long as the next_id were to exist, exploiting it in open is nigh on > trivial and much nicer. Ok, so ignore previous email. You know how unix works. I believe you should just introduce syscalls you need. Yes, introducing new syscalls is hard/expensive, but changing existing syscalls is simply bad idea. So what new syscalls do you _really_ need? Not open_this_fd, nor socket_this_fd. Pavel -- (english) http://…
creator: Pavel Machek
date: 2008-07-08T21:58:21-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: Quoting Pavel Machek (pavel@ucw.cz): > > > > > > > An alternative to this solution consists in defining a new field in the > > > > > > task structure (let's call it next_syscall_data) that, if set, would change > > > > > > the behavior of next syscall to be called. The sys_fork_with_id() previously > > > > > > cited can be replaced by > > > > > > 1) set next_syscall_data to a target upid nr > > > > > > 2) call fork(). > > > > > > > > > > ...bloat task struct and > > > > > > > > > > > A new file is created in procfs: /proc/self/task/<my_tid>/next_syscall_data. > > > > > > This makes it possible to avoid races between several threads belonging to > > > > > > the same process. > > > > > > > > > > ...introducing this kind of uglyness. > > > > > > > > > > Actually, there were proposals for sys_indirect(), which is slightly > > > > > less ugly, but IIRC we ended up with adding syscalls, too. > > > > > > > > Silly question... > > > > > > > > Oren, would you object to defining sys_fork_with_id(), > > > > sys_msgget_with_id(), and sys_semget_with_id()? > > > > > > > > Eric, Pavel (Emelyanov), Dave, do you have preferences? > > > > > > > > For the cases Nadia has implemented here I'd be tempted to side with > > > > Pavel Machek, but once we get to things like open() and socket(), (a) > > > > the # new syscalls starts to jump, and (b) the per-syscall api starts to > > > > seem a lot more cumbersome. > > > > > > You should not need to modify open/socket. You can already select fd > > > by creatively using open/dup/close... > > > > That's what we do right now in cryo. And if we end up patching up every > > API with separate syscalls, then we wouldn't create open_with_id(). But > > so long as the next_id were to exist, exploiting it in open is nigh on > > trivial and much nicer. > > Ok, so ignore previous email. You know how unix works. > > I believe you should just introduce syscalls you need. Yes, > introducing new syscalls is hard/expensive, but changing existing > syscalls is…
creator: serue
date: 2008-07-09T02:20:35-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: Pavel Machek wrote: > On Tue 2008-07-08 16:47:21, Serge E. Hallyn wrote: > >>Quoting Pavel Machek (pavel@ucw.cz): >> >>>Hi! >>> >>> >>>>>>An alternative to this solution consists in defining a new field in the >>>>>>task structure (let's call it next_syscall_data) that, if set, would change >>>>>>the behavior of next syscall to be called. The sys_fork_with_id() previously >>>>>>cited can be replaced by >>>>>>1) set next_syscall_data to a target upid nr >>>>>>2) call fork(). >>>>> >>>>> >>>>>...bloat task struct and >>>>> >>>>> >>>>> >>>>>>A new file is created in procfs: /proc/self/task/<my_tid>/next_syscall_data. >>>>>>This makes it possible to avoid races between several threads belonging to >>>>>>the same process. >>>>> >>>>> >>>>>...introducing this kind of uglyness. >>>>> >>>>>Actually, there were proposals for sys_indirect(), which is slightly >>>>>less ugly, but IIRC we ended up with adding syscalls, too. >>> >>>>I had a look at the lwn.net article that describes the sys_indirect() >>>>interface. >>>>It does exactly what we need here, so I do like it, but it has the same >>>>drawbacks as the one you're complaining about: >>>>. a new field is needed in the task structure >>>>. looks like many people found it ugly... >>> >>>>Now, coming back to what I'm proposing: what we need is actually to change >>>>the behavior of *existing* syscalls, since we are in a very particular >>>>context (restarting an application). >>> >>>Changing existing syscalls is _bad_: for backwards compatibility >>>reasons. strace will be very confusing to read, etc... >> >>I dunno... if you normally open(), you get back a random fd. If you do >>it having set the next_id inadvertently, then as far as you know you get >>back a random fd, no? > > > Sorry?! > > No, open does not return random fds. It allocates them bottom-up. So > you do not need any changes in open case. > > (If you want to open "/foo/bar" as fd #50, open /dev/zero 49 times, 49 times - <# of already busy fds> Don't you think …
creator: Nadia Derbey
date: 2008-07-10T06:54:10-00:00

Re: Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: On Wed, Jul 9, 2008 at 11:54 PM, Nadia Derbey <Nadia.Derbey@bull.net> wrote: > > Don't you think it's simpler to specify the target fd, and then open the > file. Maybe. But: - this can already be done without extra kernel support, via open() followed by dup2() - if you were going to add it to the kernel, the precedent set by openat() is that you create a new system call that supports the extended semantics. Paul _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Paul Menage
date: 2008-07-10T07:01:55-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: Pavel Machek wrote: > Hi! > > >>>>An alternative to this solution consists in defining a new field in the >>>>task structure (let's call it next_syscall_data) that, if set, would change >>>>the behavior of next syscall to be called. The sys_fork_with_id() previously >>>>cited can be replaced by >>>>1) set next_syscall_data to a target upid nr >>>>2) call fork(). >>> >>> >>>...bloat task struct and >>> >>> >>> >>>>A new file is created in procfs: /proc/self/task/<my_tid>/next_syscall_data. >>>>This makes it possible to avoid races between several threads belonging to >>>>the same process. >>> >>> >>>...introducing this kind of uglyness. >>> >>>Actually, there were proposals for sys_indirect(), which is slightly >>>less ugly, but IIRC we ended up with adding syscalls, too. > > >>I had a look at the lwn.net article that describes the sys_indirect() >>interface. >>It does exactly what we need here, so I do like it, but it has the same >>drawbacks as the one you're complaining about: >>. a new field is needed in the task structure >>. looks like many people found it ugly... > > >>Now, coming back to what I'm proposing: what we need is actually to change >>the behavior of *existing* syscalls, since we are in a very particular >>context (restarting an application). > > > Changing existing syscalls is _bad_: for backwards compatibility > reasons. I'm sorry but I don't see a backward compatibility problem: same interface, same functionality provided. The only change is in the way ids are assigned. Actually, one drawback I'm seeing is that we are adding a test to the classical syscall path (the test on the current->next_syscall_data being set or not). > strace will be very confusing to read, etc... We'll have the 3 following lines added to an strace output each time we fill the proc file: open("/proc/15084/task/15084/next_syscall_data", O_RDWR) = 4 write(4, "LONG1 100", 9) = 9 close(4) = 0 I don't see anthing confusing here ;-) Regards, Nadia > > >>Defining brand new syscalls…
creator: Nadia Derbey
date: 2008-07-10T07:42:03-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: Serge E. Hallyn wrote: > Quoting Pavel Machek (pavel@ucw.cz): > >>>>>>>An alternative to this solution consists in defining a new field in the >>>>>>>task structure (let's call it next_syscall_data) that, if set, would change >>>>>>>the behavior of next syscall to be called. The sys_fork_with_id() previously >>>>>>>cited can be replaced by >>>>>>> 1) set next_syscall_data to a target upid nr >>>>>>> 2) call fork(). >>>>>> >>>>>>...bloat task struct and >>>>>> >>>>>> >>>>>>>A new file is created in procfs: /proc/self/task/<my_tid>/next_syscall_data. >>>>>>>This makes it possible to avoid races between several threads belonging to >>>>>>>the same process. >>>>>> >>>>>>...introducing this kind of uglyness. >>>>>> >>>>>>Actually, there were proposals for sys_indirect(), which is slightly >>>>>>less ugly, but IIRC we ended up with adding syscalls, too. >>>>> >>>>>Silly question... >>>>> >>>>>Oren, would you object to defining sys_fork_with_id(), >>>>>sys_msgget_with_id(), and sys_semget_with_id()? >>>>> >>>>>Eric, Pavel (Emelyanov), Dave, do you have preferences? >>>>> >>>>>For the cases Nadia has implemented here I'd be tempted to side with >>>>>Pavel Machek, but once we get to things like open() and socket(), (a) >>>>>the # new syscalls starts to jump, and (b) the per-syscall api starts to >>>>>seem a lot more cumbersome. >>>> >>>>You should not need to modify open/socket. You can already select fd >>>>by creatively using open/dup/close... >>> >>>That's what we do right now in cryo. And if we end up patching up every >>>API with separate syscalls, then we wouldn't create open_with_id(). But >>>so long as the next_id were to exist, exploiting it in open is nigh on >>>trivial and much nicer. >> >>Ok, so ignore previous email. You know how unix works. >> >>I believe you should just introduce syscalls you need. Yes, >>introducing new syscalls is hard/expensive, but changing existing >>syscalls is simply bad idea. > > > Ok, thanks, Pavel. I'm really far more inclined to agr…
creator: Nadia Derbey
date: 2008-07-10T07:58:48-00:00

Re: Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: On Thu, Jul 10, 2008 at 12:58 AM, Nadia Derbey <Nadia.Derbey@bull.net> wrote: > > Actually, what I've started working on these days is replace the proc > interface by a syscall to set the next_syscall_data field: I think this > might help us avoid defining a precise list of the new syscalls we need? Isn't that just sys_indirect(), but split into two syscall invocations rather than one? Paul _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Paul Menage
date: 2008-07-10T08:34:32-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: On Thu 2008-07-10 09:42:03, Nadia Derbey wrote: > Pavel Machek wrote: >> Hi! >> >> >>>>> An alternative to this solution consists in defining a new field in the >>>>> task structure (let's call it next_syscall_data) that, if set, would change >>>>> the behavior of next syscall to be called. The sys_fork_with_id() previously >>>>> cited can be replaced by >>>>> 1) set next_syscall_data to a target upid nr >>>>> 2) call fork(). >>>> >>>> >>>> ...bloat task struct and >>>> >>>> >>>> >>>>> A new file is created in procfs: /proc/self/task/<my_tid>/next_syscall_data. >>>>> This makes it possible to avoid races between several threads belonging to >>>>> the same process. >>>> >>>> >>>> ...introducing this kind of uglyness. >>>> >>>> Actually, there were proposals for sys_indirect(), which is slightly >>>> less ugly, but IIRC we ended up with adding syscalls, too. >> >> >>> I had a look at the lwn.net article that describes the sys_indirect() >>> interface. >>> It does exactly what we need here, so I do like it, but it has the same >>> drawbacks as the one you're complaining about: >>> . a new field is needed in the task structure >>> . looks like many people found it ugly... >> >> >>> Now, coming back to what I'm proposing: what we need is actually to >>> change the behavior of *existing* syscalls, since we are in a very >>> particular context (restarting an application). >> >> >> Changing existing syscalls is _bad_: for backwards compatibility >> reasons. > > I'm sorry but I don't see a backward compatibility problem: same interface, > same functionality provided. The only change is in the way ids are > assigned. If you don't see a backward compatibility problem here, perhaps you should not be hacking kernel...? The way ids are assigned is certainly part of syscall semantics (applications rely on), at least for open. If you want to claim that your solution is better than adding milion of syscalls, I guess you need to list the milion of syscalls, so we can compare. > Actua…
creator: Pavel Machek
date: 2008-07-10T08:54:06-00:00

Re: Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: Paul Menage wrote: > On Wed, Jul 9, 2008 at 11:54 PM, Nadia Derbey <Nadia.Derbey@bull.net> wrote: > >>Don't you think it's simpler to specify the target fd, and then open the >>file. > > > Maybe. But: > > - this can already be done without extra kernel support, via open() > followed by dup2() Sure, I completely agree with you. Actually, that's the way it is handled in cryo code. But I think that both ways of doing are not exactly the same in case of failure: open + dup2 will close newfd if it is already busy. while next-syscall_data + open will fail if the target fd is already busy. And that's the functionality we need during restart, isn't it? Regards, Nadia > > - if you were going to add it to the kernel, the precedent set by > openat() is that you create a new system call that supports the > extended semantics. > > _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Nadia Derbey
date: 2008-07-10T09:14:18-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: Pavel Machek wrote: > On Thu 2008-07-10 09:42:03, Nadia Derbey wrote: > >>Pavel Machek wrote: >> >>>Hi! >>> >>> >>> >>>>>>An alternative to this solution consists in defining a new field in the >>>>>>task structure (let's call it next_syscall_data) that, if set, would change >>>>>>the behavior of next syscall to be called. The sys_fork_with_id() previously >>>>>>cited can be replaced by >>>>>>1) set next_syscall_data to a target upid nr >>>>>>2) call fork(). >>>>> >>>>> >>>>>...bloat task struct and >>>>> >>>>> >>>>> >>>>> >>>>>>A new file is created in procfs: /proc/self/task/<my_tid>/next_syscall_data. >>>>>>This makes it possible to avoid races between several threads belonging to >>>>>>the same process. >>>>> >>>>> >>>>>...introducing this kind of uglyness. >>>>> >>>>>Actually, there were proposals for sys_indirect(), which is slightly >>>>>less ugly, but IIRC we ended up with adding syscalls, too. >>> >>> >>>>I had a look at the lwn.net article that describes the sys_indirect() >>>>interface. >>>>It does exactly what we need here, so I do like it, but it has the same >>>>drawbacks as the one you're complaining about: >>>>. a new field is needed in the task structure >>>>. looks like many people found it ugly... >>> >>> >>>>Now, coming back to what I'm proposing: what we need is actually to >>>>change the behavior of *existing* syscalls, since we are in a very >>>>particular context (restarting an application). >>> >>> >>>Changing existing syscalls is _bad_: for backwards compatibility >>>reasons. >> >>I'm sorry but I don't see a backward compatibility problem: same interface, >>same functionality provided. The only change is in the way ids are >>assigned. > > > If you don't see a backward compatibility problem here, perhaps you > should not be hacking kernel...? Thx for the advice, will try think about it... > The way ids are assigned is certainly > part of syscall semantics (applications rely on), at least for open. > > If you want to claim that your solution …
creator: Nadia Derbey
date: 2008-07-10T09:29:45-00:00

Re: Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: On Thu, Jul 10, 2008 at 2:14 AM, Nadia Derbey <Nadia.Derbey@bull.net> wrote: > But I think that both ways of doing are not exactly the same in case of > failure: > open + dup2 will close newfd if it is already busy. > while > next-syscall_data + open will fail if the target fd is already busy. And > that's the functionality we need during restart, isn't it? No, I don't think so. The cryo restart code should be aware of exactly which fds it has open and which it needs to open, and can shuffle them around as necessary via dup2() to get them into the right places. Paul _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Paul Menage
date: 2008-07-10T09:30:26-00:00

Re: Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: Paul Menage wrote: > On Thu, Jul 10, 2008 at 12:58 AM, Nadia Derbey <Nadia.Derbey@bull.net> wrote: > >>Actually, what I've started working on these days is replace the proc >>interface by a syscall to set the next_syscall_data field: I think this >>might help us avoid defining a precise list of the new syscalls we need? > > > Isn't that just sys_indirect(), but split into two syscall invocations > rather than one? > Yes, from what I've read about the sys_indirect(), it is. Unfortunalty, I hadn't followed the thread, so except because of its "ugliness" (again ;-) ) I don't exactly know why the idea has been given up. Regards, Nadia _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Nadia Derbey
date: 2008-07-10T09:38:45-00:00

Re: Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: Paul Menage wrote: > On Thu, Jul 10, 2008 at 2:14 AM, Nadia Derbey <Nadia.Derbey@bull.net> wrote: > >>But I think that both ways of doing are not exactly the same in case of >>failure: >>open + dup2 will close newfd if it is already busy. >>while >>next-syscall_data + open will fail if the target fd is already busy. And >>that's the functionality we need during restart, isn't it? > > > No, I don't think so. The cryo restart code should be aware of exactly > which fds it has open and which it needs to open, and can shuffle them > around as necessary via dup2() to get them into the right places. > Yes sure, that's exactly what it does. what I just wanted to say here is that the concept of opening + dup2'ing is not exactly the same as the open_with_id concept. But I agree with you: a restart code knows exactly what it does and can safely play with the open and dup syscalls. Regards, Nadia _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Nadia Derbey
date: 2008-07-10T10:11:20-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: On Thu, 2008-07-10 at 10:54 +0200, Pavel Machek wrote: > > If you don't see a backward compatibility problem here, perhaps you > should not be hacking kernel...? The way ids are assigned is certainly > part of syscall semantics (applications rely on), at least for open. We also used to have a pretty defined ordering for handing out address space with mmap(). That all changed with address space randomization. Are file descriptors different somehow? Anyway, it's not like we're actually changing existing behavior. An application has to do something special and new to trigger this new behavior. Nobody is going to stumble over it, and it will *not* break backward compatibility. -- Dave _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/containers
creator: Dave Hansen
date: 2008-07-10T17:53:35-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: On Thu 2008-07-10 10:53:35, Dave Hansen wrote: > On Thu, 2008-07-10 at 10:54 +0200, Pavel Machek wrote: > > > > If you don't see a backward compatibility problem here, perhaps you > > should not be hacking kernel...? The way ids are assigned is certainly > > part of syscall semantics (applications rely on), at least for open. > > We also used to have a pretty defined ordering for handing out address > space with mmap(). That all changed with address space randomization. > Are file descriptors different somehow? > > Anyway, it's not like we're actually changing existing behavior. An > application has to do something special and new to trigger this new > behavior. Nobody is going to stumble over it, and it will *not* break > backward compatibility. It will break compatibility, but not in a way you expect. There's application called "subterfugue" that monitors other applications using ptrace and enforces security policy (or does other stuff). Such hacks depend on existing syscalls behaving in a way they are specified... Then you'll have to update open.2 man page: DESCRIPTION Given a pathname for a file, open() returns a file descriptor, a small, non- negative integer for use in subsequent system calls (read(2), write(2), lseek(2), fcntl(2), etc.). The file descriptor returned by a successful call will be the lowest-numbered file descriptor not currently open for the process. ...you'll need to add "unless someone write some number in file in /proc somewhere"... hmm... is new behaviour even POSIX compliant? open() is specified in POSIX... Ok, so it will not break too many apps... but echo "123 > /proc/something" breaking bash (etc) is not nice. (Plus proposed interface is so ugly that this discussion is moot.) Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html _______________________________________________ Containers mailing list Containers@lists.linux-foundation.org https://lists.…
creator: Pavel Machek
date: 2008-07-10T18:45:12-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: On Thu, 2008-07-10 at 20:45 +0200, Pavel Machek wrote: > On Thu 2008-07-10 10:53:35, Dave Hansen wrote: > > On Thu, 2008-07-10 at 10:54 +0200, Pavel Machek wrote: > > > > > > If you don't see a backward compatibility problem here, perhaps you > > > should not be hacking kernel...? The way ids are assigned is certainly > > > part of syscall semantics (applications rely on), at least for open. > > > > We also used to have a pretty defined ordering for handing out address > > space with mmap(). That all changed with address space randomization. > > Are file descriptors different somehow? > > > > Anyway, it's not like we're actually changing existing behavior. An > > application has to do something special and new to trigger this new > > behavior. Nobody is going to stumble over it, and it will *not* break > > backward compatibility. > > It will break compatibility, but not in a way you expect. There's > application called "subterfugue" that monitors other applications > using ptrace and enforces security policy (or does other stuff). Such > hacks depend on existing syscalls behaving in a way they are > specified... > > Then you'll have to update open.2 man page: > > DESCRIPTION > Given a pathname for a file, open() returns a file descriptor, > a small, non- > negative integer for use in subsequent system calls > (read(2), write(2), > lseek(2), fcntl(2), etc.). The file descriptor returned by > a successful > call will be the lowest-numbered file descriptor not currently > open for the > process. > > ...you'll need to add "unless someone write some number in file in > /proc somewhere"... hmm... is new behaviour even POSIX compliant? > open() is specified in POSIX... Yup, that's true. Good point. > Ok, so it will not break too many apps... but echo "123 > > /proc/something" breaking bash (etc) is not nice. > > (Plus proposed interface is so ugly that this discussion is moot.) Yes, I agree that the current proposed interface is too ugly to live. :) -- Dave _______________…
creator: Dave Hansen
date: 2008-07-10T19:04:03-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: Quoting Dave Hansen (dave@linux.vnet.ibm.com): > On Thu, 2008-07-10 at 20:45 +0200, Pavel Machek wrote: > > On Thu 2008-07-10 10:53:35, Dave Hansen wrote: > > > On Thu, 2008-07-10 at 10:54 +0200, Pavel Machek wrote: > > > > > > > > If you don't see a backward compatibility problem here, perhaps you > > > > should not be hacking kernel...? The way ids are assigned is certainly > > > > part of syscall semantics (applications rely on), at least for open. > > > > > > We also used to have a pretty defined ordering for handing out address > > > space with mmap(). That all changed with address space randomization. > > > Are file descriptors different somehow? > > > > > > Anyway, it's not like we're actually changing existing behavior. An > > > application has to do something special and new to trigger this new > > > behavior. Nobody is going to stumble over it, and it will *not* break > > > backward compatibility. > > > > It will break compatibility, but not in a way you expect. There's > > application called "subterfugue" that monitors other applications > > using ptrace and enforces security policy (or does other stuff). Such > > hacks depend on existing syscalls behaving in a way they are > > specified... > > > > Then you'll have to update open.2 man page: > > > > DESCRIPTION > > Given a pathname for a file, open() returns a file descriptor, > > a small, non- > > negative integer for use in subsequent system calls > > (read(2), write(2), > > lseek(2), fcntl(2), etc.). The file descriptor returned by > > a successful > > call will be the lowest-numbered file descriptor not currently > > open for the > > process. > > > > ...you'll need to add "unless someone write some number in file in > > /proc somewhere"... hmm... is new behaviour even POSIX compliant? > > open() is specified in POSIX... > > Yup, that's true. Good point. I didn't think it was, as I thought it was current behavior but not mandated by the spec. But I was wrong. So this patch must be dropped, at any rate.…
creator: serue
date: 2008-07-10T19:27:20-00:00

Re: [RFC PATCH 0/5] Resend - Use procfs to change a syscall behavior
type: item
description: Wow ... the volume of messages in this thread is overwhelming. I guess it had to happen when I was off a month long vacation ! Ok .. lemme see if I can catch up: Serge E. Hallyn wrote: > Quoting Pavel Machek (pavel@ucw.cz): >>>>>>> An alternative to this solution consists in defining a new field in the >>>>>>> task structure (let's call it next_syscall_data) that, if set, would change >>>>>>> the behavior of next syscall to be called. The sys_fork_with_id() previously >>>>>>> cited can be replaced by >>>>>>> 1) set next_syscall_data to a target upid nr >>>>>>> 2) call fork(). >>>>>> ...bloat task struct and >>>>>> >>>>>>> A new file is created in procfs: /proc/self/task/<my_tid>/next_syscall_data. >>>>>>> This makes it possible to avoid races between several threads belonging to >>>>>>> the same process. >>>>>> ...introducing this kind of uglyness. >>>>>> >>>>>> Actually, there were proposals for sys_indirect(), which is slightly >>>>>> less ugly, but IIRC we ended up with adding syscalls, too. >>>>> Silly question... >>>>> >>>>> Oren, would you object to defining sys_fork_with_id(), >>>>> sys_msgget_with_id(), and sys_semget_with_id()? I don't object, in particular given the backward-compatibility issue that was discussed later in this thread. However, see more below. >>>>> >>>>> Eric, Pavel (Emelyanov), Dave, do you have preferences? >>>>> >>>>> For the cases Nadia has implemented here I'd be tempted to side with >>>>> Pavel Machek, but once we get to things like open() and socket(), (a) >>>>> the # new syscalls starts to jump, and (b) the per-syscall api starts to >>>>> seem a lot more cumbersome. >>>> You should not need to modify open/socket. You can already select fd >>>> by creatively using open/dup/close... >>> That's what we do right now in cryo. And if we end up patching up every >>> API with separate syscalls, then we wouldn't create open_with_id(). But >>> so long as the next_id were to exist, exploiting it in open is nigh on >>> trivial and much nicer. …
creator: Oren Laadan
date: 2008-07-17T22:26:43-00:00