OpenVZ Forum - RDF feed
type: channel
items: [RFC] network namespaces
items: [PATCH 1/9] network namespaces: core and device list
items: [PATCH 2/9] network namespaces: IPv4 routing
items: [PATCH 4/9] network namespaces: socket hashes
items: [PATCH 5/9] network namespaces: async socket operations
items: [PATCH 6/9] allow proc_dir_entries to have destructor
items: [PATCH 7/9] net_device seq_file
items: [PATCH 8/9] network namespaces: device to pass packets between namespaces
items: [PATCH 9/9] network namespaces: playing with pass-through device
items: Re: [RFC] network namespaces
items: Re: [PATCH 1/9] network namespaces: core and device list
items: Re: Re: [RFC] network namespaces
items: RE: [RFC] network namespaces
items: Re: [PATCH 4/9] network namespaces: socket hashes
items: Re: [PATCH 5/9] network namespaces: async socket operations

[RFC] network namespaces
type: item
description: Hi All, I'd like to resurrect our discussion about network namespaces. In our previous discussions it appeared that we have rather polar concepts which seemed hard to reconcile. Now I have an idea how to look at all discussed concepts to enable everyone's usage scenario. 1. The most straightforward concept is complete separation of namespaces, covering device list, routing tables, netfilter tables, socket hashes, and everything else. On input path, each packet is tagged with namespace right from the place where it appears from a device, and is processed by each layer in the context of this namespace. Non-root namespaces communicate with the outside world in two ways: by owning hardware devices, or receiving packets forwarded them by their parent namespace via pass-through device. This complete separation of namespaces is very useful for at least two purposes: - allowing users to create and manage by their own various tunnels and VPNs, and - enabling easier and more straightforward live migration of groups of processes with their environment. 2. People expressed concerns that complete separation of namespaces may introduce an undesired overhead in certain usage scenarios. The overhead comes from packets traversing input path, then output path, then input path again in the destination namespace if root namespace acts as a router. So, we may introduce short-cuts, when input packet starts to be processes in one namespace, but changes it at some upper layer. The places where packet can change namespace are, for example: routing, post-routing netfilter hook, or even lookup in socket hash. The cleanest example among them is post-routing netfilter hook. Tagging of input packets there means that the packets is checked against root namespace's routing table, found to be local, and go directly to the socket hash lookup in the destination namespace. In this scheme the ability to change routing tables or netfilter rules on a per-namespace basis is traded for lower overhead. All …
creator: Andrey Savochkin
date: 2006-08-15T14:20:29-00:00

[PATCH 1/9] network namespaces: core and device list
type: item
description: CONFIG_NET_NS and net_namespace structure are introduced. List of network devices is made per-namespace. Each namespace gets its own loopback device. Signed-off-by: Andrey Savochkin <saw@swsoft.com> --- drivers/net/loopback.c | 69 ++++++++++++--------- include/linux/init_task.h | 9 ++ include/linux/net_ns.h | 82 +++++++++++++++++++++++++ include/linux/netdevice.h | 13 +++ include/linux/nsproxy.h | 3 include/linux/sched.h | 3 kernel/nsproxy.c | 14 ++++ net/Kconfig | 7 ++ net/core/dev.c | 150 ++++++++++++++++++++++++++++++++++++++++++++-- net/core/net-sysfs.c | 24 +++++++ net/ipv4/devinet.c | 2 net/ipv6/addrconf.c | 2 net/ipv6/route.c | 9 +- 13 files changed, 349 insertions(+), 38 deletions(-) --- ./drivers/net/loopback.c.vensdev Mon Aug 14 17:02:18 2006 +++ ./drivers/net/loopback.c Mon Aug 14 17:18:20 2006 @@ -196,42 +196,55 @@ static struct ethtool_ops loopback_ethto .set_tso = ethtool_op_set_tso, }; -struct net_device loopback_dev = { - .name = "lo", - .mtu = (16 * 1024) + 20 + 20 + 12, - .hard_start_xmit = loopback_xmit, - .hard_header = eth_header, - .hard_header_cache = eth_header_cache, - .header_cache_update = eth_header_cache_update, - .hard_header_len = ETH_HLEN, /* 14 */ - .addr_len = ETH_ALEN, /* 6 */ - .tx_queue_len = 0, - .type = ARPHRD_LOOPBACK, /* 0x0001*/ - .rebuild_header = eth_rebuild_header, - .flags = IFF_LOOPBACK, - .features = NETIF_F_SG | NETIF_F_FRAGLIST +struct net_device loopback_dev_static; +EXPORT_SYMBOL(loopback_dev_static); + +void loopback_dev_dtor(struct net_device *dev) +{ + if (dev->priv) { + kfree(dev->priv); + dev->priv = NULL; + } + free_netdev(dev); +} + +void loopback_dev_ctor(struct net_device *dev) +{ + struct net_device_stats *stats; + + memset(dev, 0, sizeof(*dev)); + strcpy(dev->name, "lo"); + dev->mtu = (16 * 1024) + 20 + 20 + 12; + dev->hard_start_xmit = loopback_xmit; + dev->hard_header = eth_header; + dev->hard_header_cache = eth_header_cache; + dev->header_cache_update = eth_header_cache_update; + dev->hard_header_len …
creator: Andrey Savochkin
date: 2006-08-15T14:48:36-00:00

[PATCH 2/9] network namespaces: IPv4 routing
type: item
description: Structures related to IPv4 rounting (FIB and routing cache) are made per-namespace. Signed-off-by: Andrey Savochkin <saw@swsoft.com> --- include/linux/net_ns.h | 10 +++ include/net/flow.h | 3 + include/net/ip_fib.h | 46 ++++++++++++---- net/core/dev.c | 8 ++ net/core/fib_rules.c | 43 ++++++++++++--- net/ipv4/Kconfig | 4 - net/ipv4/fib_frontend.c | 132 +++++++++++++++++++++++++++++++++++++---------- net/ipv4/fib_hash.c | 13 +++- net/ipv4/fib_rules.c | 86 +++++++++++++++++++++++++----- net/ipv4/fib_semantics.c | 99 +++++++++++++++++++++++------------ net/ipv4/route.c | 26 ++++++++- 11 files changed, 375 insertions(+), 95 deletions(-) --- ./include/linux/net_ns.h.vensroute Mon Aug 14 17:18:59 2006 +++ ./include/linux/net_ns.h Mon Aug 14 19:19:14 2006 @@ -14,7 +14,17 @@ struct net_namespace { atomic_t active_ref, use_ref; struct net_device *dev_base_p, **dev_tail_p; struct net_device *loopback; +#ifndef CONFIG_IP_MULTIPLE_TABLES + struct fib_table *fib4_local_table, *fib4_main_table; +#else + struct list_head fib_rules_ops_list; + struct fib_rules_ops *fib4_rules_ops; + struct hlist_head *fib4_tables; +#endif + struct hlist_head *fib4_hash, *fib4_laddrhash; + unsigned fib4_hash_size, fib4_info_cnt; unsigned int hash; + char destroying; struct work_struct destroy_work; }; --- ./include/net/flow.h.vensroute Mon Aug 14 17:04:04 2006 +++ ./include/net/flow.h Mon Aug 14 17:18:59 2006 @@ -79,6 +79,9 @@ struct flowi { #define fl_icmp_code uli_u.icmpt.code #define fl_ipsec_spi uli_u.spi __u32 secid; /* used by xfrm; see secid.txt */ +#ifdef CONFIG_NET_NS + struct net_namespace *net_ns; +#endif } __attribute__((__aligned__(BITS_PER_LONG/8))); #define FLOW_DIR_IN 0 --- ./include/net/ip_fib.h.vensroute Mon Aug 14 17:04:04 2006 +++ ./include/net/ip_fib.h Tue Aug 15 11:53:22 2006 @@ -18,6 +18,7 @@ #include <net/flow.h> #include <linux/seq_file.h> +#include <linux/net_ns.h> #include <net/fib_rules.h> /* WARNING: The ordering of these elements must match ordering @@ -171,14 +172,21 @@…
creator: Andrey Savochkin
date: 2006-08-15T14:48:42-00:00

[PATCH 4/9] network namespaces: socket hashes
type: item
description: Socket hash lookups are made within namespace. Hash tables are common for all namespaces, with additional permutation of indexes. Signed-off-by: Andrey Savochkin <saw@swsoft.com> --- include/linux/ipv6.h | 3 ++- include/net/inet6_hashtables.h | 6 ++++-- include/net/inet_hashtables.h | 38 +++++++++++++++++++++++++------------- include/net/inet_sock.h | 6 ++++-- include/net/inet_timewait_sock.h | 2 ++ include/net/sock.h | 4 ++++ include/net/udp.h | 12 +++++++++--- net/core/sock.c | 5 +++++ net/ipv4/inet_connection_sock.c | 19 +++++++++++++++---- net/ipv4/inet_hashtables.c | 29 ++++++++++++++++++++++------- net/ipv4/inet_timewait_sock.c | 8 ++++++-- net/ipv4/raw.c | 2 ++ net/ipv4/udp.c | 20 +++++++++++++------- net/ipv6/inet6_connection_sock.c | 2 ++ net/ipv6/inet6_hashtables.c | 25 ++++++++++++++++++------- net/ipv6/raw.c | 4 ++++ net/ipv6/udp.c | 21 ++++++++++++++------- 17 files changed, 151 insertions(+), 55 deletions(-) --- ./include/linux/ipv6.h.venssock Mon Aug 14 17:02:45 2006 +++ ./include/linux/ipv6.h Tue Aug 15 13:38:47 2006 @@ -428,10 +428,11 @@ static inline struct raw6_sock *raw6_sk( #define inet_v6_ipv6only(__sk) 0 #endif /* defined(CONFIG_IPV6) || defined(CONFIG_IPV6_MODULE) */ -#define INET6_MATCH(__sk, __hash, __saddr, __daddr, __ports, __dif)\ +#define INET6_MATCH(__sk, __hash, __saddr, __daddr, __ports, __dif, __ns)\ (((__sk)->sk_hash == (__hash)) && \ ((*((__u32 *)&(inet_sk(__sk)->dport))) == (__ports)) && \ ((__sk)->sk_family == AF_INET6) && \ + net_ns_match((__sk)->sk_net_ns, __ns) && \ ipv6_addr_equal(&inet6_sk(__sk)->daddr, (__saddr)) && \ ipv6_addr_equal(&inet6_sk(__sk)->rcv_saddr, (__daddr)) && \ (!((__sk)->sk_bound_dev_if) || ((__sk)->sk_bound_dev_if == (__dif)))) --- ./include/net/inet6_hashtables.h.venssock Mon Aug 14 17:02:47 2006 +++ ./include/net/inet6_hashtables.h Tue Aug 15 13:38:47 2006 @@ -26,11 +26,13 @@ struct inet_hashinfo; /* I have no idea if this is a good hash for v6 or not. -DaveM */ static inline unsigned int inet6_ehashfn(…
creator: Andrey Savochkin
date: 2006-08-15T14:48:44-00:00

[PATCH 5/9] network namespaces: async socket operations
type: item
description: Non-trivial part of socket namespaces: asynchronous events should be run in proper context. Signed-off-by: Andrey Savochkin <saw@swsoft.com> --- af_inet.c | 10 ++++++++++ inet_timewait_sock.c | 8 ++++++++ tcp_timer.c | 9 +++++++++ 3 files changed, 27 insertions(+) --- ./net/ipv4/af_inet.c.venssock-asyn Mon Aug 14 17:04:07 2006 +++ ./net/ipv4/af_inet.c Tue Aug 15 13:45:44 2006 @@ -366,10 +366,17 @@ out_rcu_unlock: int inet_release(struct socket *sock) { struct sock *sk = sock->sk; + struct net_namespace *ns, *orig_net_ns; if (sk) { long timeout; + /* Need to change context here since protocol ->close + * operation may send packets. + */ + ns = get_net_ns(sk->sk_net_ns); + push_net_ns(ns, orig_net_ns); + /* Applications forget to leave groups before exiting */ ip_mc_drop_socket(sk); @@ -386,6 +393,9 @@ int inet_release(struct socket *sock) timeout = sk->sk_lingertime; sock->sk = NULL; sk->sk_prot->close(sk, timeout); + + pop_net_ns(orig_net_ns); + put_net_ns(ns); } return 0; } --- ./net/ipv4/inet_timewait_sock.c.venssock-asyn Tue Aug 15 13:45:44 2006 +++ ./net/ipv4/inet_timewait_sock.c Tue Aug 15 13:45:44 2006 @@ -129,6 +129,7 @@ static int inet_twdr_do_twkill_work(stru { struct inet_timewait_sock *tw; struct hlist_node *node; + struct net_namespace *orig_net_ns; unsigned int killed; int ret; @@ -140,8 +141,10 @@ static int inet_twdr_do_twkill_work(stru */ killed = 0; ret = 0; + push_net_ns(current_net_ns, orig_net_ns); rescan: inet_twsk_for_each_inmate(tw, node, &twdr->cells[slot]) { + switch_net_ns(tw->tw_net_ns); __inet_twsk_del_dead_node(tw); spin_unlock(&twdr->death_lock); __inet_twsk_kill(tw, twdr->hashinfo); @@ -164,6 +167,7 @@ rescan: twdr->tw_count -= killed; NET_ADD_STATS_BH(LINUX_MIB_TIMEWAITED, killed); + pop_net_ns(orig_net_ns); return ret; } @@ -338,10 +342,12 @@ void inet_twdr_twcal_tick(unsigned long int n, slot; unsigned long j; unsigned long now = jiffies; + struct net_namespace *orig_net_ns; int killed = 0; int adv = 0; twdr = (struct inet_timewait_…
creator: Andrey Savochkin
date: 2006-08-15T14:48:45-00:00

[PATCH 6/9] allow proc_dir_entries to have destructor
type: item
description: Destructor field added proc_dir_entries, standard destructor kfree'ing data introduced. Signed-off-by: Andrey Savochkin <saw@swsoft.com> --- fs/proc/generic.c | 10 ++++++++-- fs/proc/root.c | 1 + include/linux/proc_fs.h | 4 ++++ 3 files changed, 13 insertions(+), 2 deletions(-) --- ./fs/proc/generic.c.veprocdtor Mon Aug 14 16:43:41 2006 +++ ./fs/proc/generic.c Tue Aug 15 13:45:51 2006 @@ -608,6 +608,11 @@ static struct proc_dir_entry *proc_creat return ent; } +void proc_data_destructor(struct proc_dir_entry *ent) +{ + kfree(ent->data); +} + struct proc_dir_entry *proc_symlink(const char *name, struct proc_dir_entry *parent, const char *dest) { @@ -620,6 +625,7 @@ struct proc_dir_entry *proc_symlink(cons ent->data = kmalloc((ent->size=strlen(dest))+1, GFP_KERNEL); if (ent->data) { strcpy((char*)ent->data,dest); + ent->destructor = proc_data_destructor; if (proc_register(parent, ent) < 0) { kfree(ent->data); kfree(ent); @@ -698,8 +704,8 @@ void free_proc_entry(struct proc_dir_ent release_inode_number(ino); - if (S_ISLNK(de->mode) && de->data) - kfree(de->data); + if (de->destructor) + de->destructor(de); kfree(de); } --- ./fs/proc/root.c.veprocdtor Mon Aug 14 17:02:38 2006 +++ ./fs/proc/root.c Tue Aug 15 13:45:51 2006 @@ -154,6 +154,7 @@ EXPORT_SYMBOL(proc_symlink); EXPORT_SYMBOL(proc_mkdir); EXPORT_SYMBOL(create_proc_entry); EXPORT_SYMBOL(remove_proc_entry); +EXPORT_SYMBOL(proc_data_destructor); EXPORT_SYMBOL(proc_root); EXPORT_SYMBOL(proc_root_fs); EXPORT_SYMBOL(proc_net); --- ./include/linux/proc_fs.h.veprocdtor Mon Aug 14 17:02:47 2006 +++ ./include/linux/proc_fs.h Tue Aug 15 13:45:51 2006 @@ -46,6 +46,8 @@ typedef int (read_proc_t)(char *page, ch typedef int (write_proc_t)(struct file *file, const char __user *buffer, unsigned long count, void *data); typedef int (get_info_t)(char *, char **, off_t, int); +struct proc_dir_entry; +typedef void (destroy_proc_t)(struct proc_dir_entry *); struct proc_dir_entry { unsigned int low_ino; @@ -65,6 +67,7 @@ struct proc_dir…
creator: Andrey Savochkin
date: 2006-08-15T14:48:46-00:00

[PATCH 7/9] net_device seq_file
type: item
description: Library function to create a seq_file in proc filesystem, showing some information for each netdevice. This code is present in the kernel in about 10 instances, and all of them can be converted to using introduced library function. Signed-off-by: Andrey Savochkin <saw@swsoft.com> --- include/linux/netdevice.h | 7 +++ net/core/dev.c | 96 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 103 insertions(+) --- ./include/linux/netdevice.h.venetproc Tue Aug 15 13:46:08 2006 +++ ./include/linux/netdevice.h Tue Aug 15 13:46:08 2006 @@ -592,6 +592,13 @@ extern int register_netdevice(struct ne extern int unregister_netdevice(struct net_device *dev); extern void free_netdev(struct net_device *dev); extern void synchronize_net(void); +#ifdef CONFIG_PROC_FS +extern int netdev_proc_create(char *name, + int (*show)(struct seq_file *, + struct net_device *, void *), + void *data, struct module *mod); +void netdev_proc_remove(char *name); +#endif extern int register_netdevice_notifier(struct notifier_block *nb); extern int unregister_netdevice_notifier(struct notifier_block *nb); extern int call_netdevice_notifiers(unsigned long val, void *v); --- ./net/core/dev.c.venetproc Tue Aug 15 13:46:08 2006 +++ ./net/core/dev.c Tue Aug 15 13:46:08 2006 @@ -2100,6 +2100,102 @@ static int dev_ifconf(char __user *arg) } #ifdef CONFIG_PROC_FS + +struct netdev_proc_data { + struct file_operations fops; + int (*show)(struct seq_file *, struct net_device *, void *); + void *data; +}; + +static void *netdev_proc_seq_start(struct seq_file *seq, loff_t *pos) +{ + struct net_device *dev; + loff_t off; + + read_lock(&dev_base_lock); + if (*pos == 0) + return SEQ_START_TOKEN; + for (dev = dev_base, off = 1; dev; dev = dev->next, off++) { + if (*pos == off) + return dev; + } + return NULL; +} + +static void *netdev_proc_seq_next(struct seq_file *seq, void *v, loff_t *pos) +{ + ++*pos; + return (v == SEQ_START_TOKEN) ? dev_base + : ((struct net_device *)v)->next; +} + +static void netdev_pro…
creator: Andrey Savochkin
date: 2006-08-15T14:48:47-00:00

[PATCH 8/9] network namespaces: device to pass packets between namespaces
type: item
description: A simple device to pass packets between a namespace and its child. Signed-off-by: Andrey Savochkin <saw@swsoft.com> --- Makefile | 3 veth.c | 327 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++ 2 files changed, 330 insertions(+) --- ./drivers/net/Makefile.veveth Mon Aug 14 17:03:45 2006 +++ ./drivers/net/Makefile Tue Aug 15 13:46:15 2006 @@ -124,6 +124,9 @@ obj-$(CONFIG_SLIP) += slip.o obj-$(CONFIG_SLHC) += slhc.o obj-$(CONFIG_DUMMY) += dummy.o +ifeq ($(CONFIG_NET_NS),y) +obj-m += veth.o +endif obj-$(CONFIG_IFB) += ifb.o obj-$(CONFIG_DE600) += de600.o obj-$(CONFIG_DE620) += de620.o --- ./drivers/net/veth.c.veveth Tue Aug 15 13:44:46 2006 +++ ./drivers/net/veth.c Tue Aug 15 13:46:15 2006 @@ -0,0 +1,327 @@ +/* + * Copyright (C) 2006 SWsoft + * + * Written by Andrey Savochkin <saw@sw.ru>, + * reusing code by Andrey Mirkin <amirkin@sw.ru>. + */ +#include <linux/list.h> +#include <linux/spinlock.h> +#include <linux/ctype.h> +#include <asm/semaphore.h> +#include <linux/netdevice.h> +#include <linux/etherdevice.h> +#include <linux/proc_fs.h> +#include <linux/seq_file.h> +#include <net/dst.h> +#include <net/xfrm.h> + +struct veth_struct +{ + struct net_device *pair; + struct net_device_stats stats; +}; + +#define veth_from_netdev(dev) ((struct veth_struct *)(netdev_priv(dev))) + +/* ------------------------------------------------------------ ------- * + * + * Device functions + * + * ------------------------------------------------------------ ------- */ + +static struct net_device_stats *get_stats(struct net_device *dev); +static int veth_xmit(struct sk_buff *skb, struct net_device *dev) +{ + struct net_device_stats *stats; + struct veth_struct *entry; + struct net_device *rcv; + struct net_namespace *orig_net_ns; + int length; + + stats = get_stats(dev); + entry = veth_from_netdev(dev); + rcv = entry->pair; + + if (!(rcv->flags & IFF_UP)) + /* Target namespace does not want to receive packets */ + goto outf; + + dst_release(skb->dst); + skb->dst = NULL; …
creator: Andrey Savochkin
date: 2006-08-15T14:48:48-00:00

[PATCH 9/9] network namespaces: playing with pass-through device
type: item
description: Temporary code to debug and play with pass-through device. Create device pair by modprobe veth echo 'add veth1 0:1:2:3:4:1 eth0 0:1:2:3:4:2' >/proc/net/veth_ctl and your shell will appear into a new namespace with `eth0' device. Configure device in this namespace ip l s eth0 up ip a a 1.2.3.4/24 dev eth0 and in the root namespace ip l s veth1 up ip a a 1.2.3.1/24 dev veth1 to establish a communication channel between root namespace and the newly created one. Signed-off-by: Andrey Savochkin <saw@swsoft.com> --- veth.c | 113 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++ 1 files changed, 113 insertions(+) --- ./drivers/net/veth.c.veveth-dbg Tue Aug 15 13:47:48 2006 +++ ./drivers/net/veth.c Tue Aug 15 14:08:04 2006 @@ -251,6 +251,116 @@ void veth_entry_del_all(void) /* ------------------------------------------------------------ ------- * * + * Temporary interface to create veth devices + * + * ------------------------------------------------------------ ------- */ + +#ifdef CONFIG_PROC_FS + +static int veth_debug_open(struct inode *inode, struct file *file) +{ + return 0; +} + +static char *parse_addr(char *s, char *addr) +{ + int i, v; + + for (i = 0; i < ETH_ALEN; i++) { + if (!isxdigit(*s)) + return NULL; + *addr = 0; + v = isdigit(*s) ? *s - '0' : toupper(*s) - 'A' + 10; + s++; + if (isxdigit(*s)) { + *addr += v << 16; + v = isdigit(*s) ? *s - '0' : toupper(*s) - 'A' + 10; + s++; + } + *addr++ += v; + if (i < ETH_ALEN - 1 && ispunct(*s)) + s++; + } + return s; +} + +extern int net_ns_start(void); +static ssize_t veth_debug_write(struct file *file, const char __user *user_buf, + size_t size, loff_t *ppos) +{ + char buf[128], *s, *parent_name, *child_name; + char parent_addr[ETH_ALEN], child_addr[ETH_ALEN]; + struct net_namespace *parent_ns, *child_ns; + int err; + + s = buf; + err = -EINVAL; + if (size >= sizeof(buf)) + goto out; + err = -EFAULT; + if (copy_from_user(buf, user_buf, size)) + goto out; + buf[size] = 0; + + err = -EBADRQC; + if (!s…
creator: Andrey Savochkin
date: 2006-08-15T14:48:49-00:00

Re: [RFC] network namespaces
type: item
description: Quoting Andrey Savochkin (saw@sw.ru): > Hi All, > > I'd like to resurrect our discussion about network namespaces. > In our previous discussions it appeared that we have rather polar concepts > which seemed hard to reconcile. > Now I have an idea how to look at all discussed concepts to enable everyone's > usage scenario. > > 1. The most straightforward concept is complete separation of namespaces, > covering device list, routing tables, netfilter tables, socket hashes, and > everything else. > > On input path, each packet is tagged with namespace right from the > place where it appears from a device, and is processed by each layer > in the context of this namespace. > Non-root namespaces communicate with the outside world in two ways: by > owning hardware devices, or receiving packets forwarded them by their parent > namespace via pass-through device. > > This complete separation of namespaces is very useful for at least two > purposes: > - allowing users to create and manage by their own various tunnels and > VPNs, and > - enabling easier and more straightforward live migration of groups of > processes with their environment. I conceptually prefer this approach, but I seem to recall there were actual problems in using this for checkpoint/restart of lightweight (application) containers. Performance aside, are there any reasons why this approach would be problematic for c/r? I'm afraid Daniel may be on vacation, and don't know who else other than Eric might have thoughts on this. > 2. People expressed concerns that complete separation of namespaces > may introduce an undesired overhead in certain usage scenarios. > The overhead comes from packets traversing input path, then output path, > then input path again in the destination namespace if root namespace > acts as a router. > > So, we may introduce short-cuts, when input packet starts to be processes > in one namespace, but changes it at some upper layer. > The places where packet can change namespace are, for exa…
creator: serue
date: 2006-08-16T11:53:13-00:00

Re: [PATCH 1/9] network namespaces: core and device list
type: item
description: On Tue, 2006-08-15 at 18:48 +0400, Andrey Savochkin wrote: > > /* Can survive without statistics */ > stats = kmalloc(sizeof(struct net_device_stats), GFP_KERNEL); > if (stats) { > memset(stats, 0, sizeof(struct net_device_stats)); > - loopback_dev.priv = stats; > - loopback_dev.get_stats = &get_stats; > + dev->priv = stats; > + dev->get_stats = &get_stats; > } With this much surgery it might be best to start using things that have come along since this code was touched last, like kzalloc(). -- Dave
creator: Dave Hansen
date: 2006-08-16T14:46:43-00:00

Re: [RFC] network namespaces
type: item
description: Hello! > (application) containers. Performance aside, are there any reasons why > this approach would be problematic for c/r? This approach is just perfect for c/r. Probably, this is the only approach when migration can be done in a clean and self-consistent way. Alexey
creator: Alexey Kuznetsov
date: 2006-08-16T15:12:26-00:00

Re: [PATCH 1/9] network namespaces: core and device list
type: item
description: On Wed, 16 Aug 2006 07:46:43 -0700 Dave Hansen <haveblue@us.ibm.com> wrote: > On Tue, 2006-08-15 at 18:48 +0400, Andrey Savochkin wrote: > > > > /* Can survive without statistics */ > > stats = kmalloc(sizeof(struct net_device_stats), GFP_KERNEL); > > if (stats) { > > memset(stats, 0, sizeof(struct net_device_stats)); > > - loopback_dev.priv = stats; > > - loopback_dev.get_stats = &get_stats; > > + dev->priv = stats; > > + dev->get_stats = &get_stats; > > } > > With this much surgery it might be best to start using things that have > come along since this code was touched last, like kzalloc(). > > If you are going to make the loopback device dynamic, it MUST use alloc_netdev().
creator: Stephen Hemminger
date: 2006-08-16T16:45:28-00:00

Re: [RFC] network namespaces
type: item
description: Alexey Kuznetsov <kuznet@ms2.inr.ac.ru> writes: > Hello! > >> (application) containers. Performance aside, are there any reasons why >> this approach would be problematic for c/r? > > This approach is just perfect for c/r. Yes. For c/r you need to take your state with you. > Probably, this is the only approach when migration can be done > in a clean and self-consistent way. Basically there are currently 3 approaches that have been proposed. The trivial bsdjail style as implemented by Serge and in a slightly more sophisticated version in vserver. This approach as it does not touch the packets has little to no packet level overhead. Basically this is what I have called the Level 3 approach. The more in depth approach where we modify the packet processing based upon which network interface the packet comes in on, and it looks like each namespace has it's own instance of the network stack. Roughly what was proposed earlier in this thread the Level 2 approach. This potentially has per packet overhead so we need to watch the implementation very carefully. Some weird hybrid as proposed by Daniel, that I was never clear on the semantics. >From the previous conversations my impression was that as long as we could get a Layer 2 approach that did not slow down the networking stack and was clean, everyone would be happy. I'm buried in the process id namespace at the moment, and except to be so for the rest of the month, so I'm not going to be very helpful except for a few stray comments. Eric
creator: ebiederm
date: 2006-08-16T17:35:41-00:00

Re: [RFC] network namespaces
type: item
description: > Basically there are currently 3 approaches that have been proposed. > > The trivial bsdjail style as implemented by Serge and in a slightly > more sophisticated version in vserver. This approach as it does not > touch the packets has little to no packet level overhead. Basically > this is what I have called the Level 3 approach. > > The more in depth approach where we modify the packet processing based > upon which network interface the packet comes in on, and it looks like > each namespace has it's own instance of the network stack. Roughly > what was proposed earlier in this thread the Level 2 approach. This > potentially has per packet overhead so we need to watch the implementation > very carefully. > > Some weird hybrid as proposed by Daniel, that I was never clear on the > semantics. The good thing is that these approaches do not contradict each other. We discussed it with Daniel during the summit and as Andrey proposed some shortcuts can be created to avoid double stack traversing. >>From the previous conversations my impression was that as long as > we could get a Layer 2 approach that did not slow down the networking > stack and was clean, everyone would be happy. agree. > I'm buried in the process id namespace at the moment, and except > to be so for the rest of the month, so I'm not going to be > very helpful except for a few stray comments. I will be very much obliged if you find some time to review these new patches so that we could make some progress here. Thanks, Kirill
creator: dev
date: 2006-08-17T08:28:03-00:00

Re: [RFC] network namespaces
type: item
description: Hi all, >> This complete separation of namespaces is very useful for at least two >> purposes: >> - allowing users to create and manage by their own various tunnels and >> VPNs, and >> - enabling easier and more straightforward live migration of groups of >> processes with their environment. > > > I conceptually prefer this approach, but I seem to recall there were > actual problems in using this for checkpoint/restart of lightweight > (application) containers. Performance aside, are there any reasons why > this approach would be problematic for c/r? I agree with this approach too, separated namespaces is the best way to identify the network ressources for a specific container. > I'm afraid Daniel may be on vacation, and don't know who else other than > Eric might have thoughts on this. Yes, I was in "vacation", but I am back :) >>2. People expressed concerns that complete separation of namespaces >> may introduce an undesired overhead in certain usage scenarios. >> The overhead comes from packets traversing input path, then output path, >> then input path again in the destination namespace if root namespace >> acts as a router. Yes, performance is probably one issue. My concerns was for layer 2 / layer 3 virtualization. I agree a layer 2 isolation/virtualization is the best for the "system container". But there is another family of container called "application container", it is not a system which is run inside a container but only the application. If you want to run a oracle database inside a container, you can run it inside an application container without launching <init> and all the services. This family of containers are used too for HPC (high performance computing) and for distributed checkpoint/restart. The cluster runs hundred of jobs, spawning them on different hosts inside an application container. Usually the jobs communicates with broadcast and multicast. Application containers does not care of having different MAC address and rely on a layer 3 approach…
creator: Daniel Lezcano
date: 2006-09-05T13:34:40-00:00

Re: [RFC] network namespaces
type: item
description: Daniel Lezcano <dlezcano@fr.ibm.com> writes: >>>2. People expressed concerns that complete separation of namespaces >>> may introduce an undesired overhead in certain usage scenarios. >>> The overhead comes from packets traversing input path, then output path, >>> then input path again in the destination namespace if root namespace >>> acts as a router. > > Yes, performance is probably one issue. > > My concerns was for layer 2 / layer 3 virtualization. I agree a layer 2 > isolation/virtualization is the best for the "system container". > But there is another family of container called "application container", it is > not a system which is run inside a container but only the application. If you > want to run a oracle database inside a container, you can run it inside an > application container without launching <init> and all the services. > > This family of containers are used too for HPC (high performance computing) and > for distributed checkpoint/restart. The cluster runs hundred of jobs, spawning > them on different hosts inside an application container. Usually the jobs > communicates with broadcast and multicast. > Application containers does not care of having different MAC address and rely on > a layer 3 approach. > > Are application containers comfortable with a layer 2 virtualization ? I don't > think so, because several jobs running inside the same host communicate via > broadcast/multicast between them and between other jobs running on different > hosts. The IP consumption is a problem too: 1 container == 2 IP (one for the > root namespace/ one for the container), multiplicated with the number of > jobs. Furthermore, lot of jobs == lot of virtual devices. > > However, after a discussion with Kirill at the OLS, it appears we can merge the > layer 2 and 3 approaches if the level of network virtualization is tunable and > we can choose layer 2 or layer 3 when doing the "unshare". The determination of > the namespace for the incoming traffic can be done wit…
creator: ebiederm
date: 2006-09-05T14:45:39-00:00

Re: [RFC] network namespaces
type: item
description: > For HPC if you are interested in migration you need a separate IP per > container. If you can take you IP address with you migration of > networking state is simple. If you can't take your IP address with > you a network container is nearly pointless from a migration > perspective. Eric, please, I know... I showed you a migration demo at OLS ;) > Beyond that from everything I have seen layer 2 is just much cleaner > than any layer 3 approach short of Serge's bind filtering. > Beyond that I have yet to see a clean semantics for anything > resembling your layer 2 layer 3 hybrid approach. If we can't have > clear semantics it is by definition impossible to implement correctly > because no one understands what it is supposed to do. > Note. A true layer 3 approach has no impact on TCP/UDP filtering > because it filters at bind time not at packet reception time. Once > you start inspecting packets I don't see what the gain is from not > going all of the way to layer 2. The bsdjail was just for information ... - Daniel
creator: Daniel Lezcano
date: 2006-09-05T15:32:01-00:00

Re: [RFC] network namespaces
type: item
description: > Yes, performance is probably one issue. > > My concerns was for layer 2 / layer 3 virtualization. I agree a layer 2 > isolation/virtualization is the best for the "system container". > But there is another family of container called "application container", > it is not a system which is run inside a container but only the > application. If you want to run a oracle database inside a container, > you can run it inside an application container without launching <init> > and all the services. > > This family of containers are used too for HPC (high performance > computing) and for distributed checkpoint/restart. The cluster runs > hundred of jobs, spawning them on different hosts inside an application > container. Usually the jobs communicates with broadcast and multicast. > Application containers does not care of having different MAC address and > rely on a layer 3 approach. > > Are application containers comfortable with a layer 2 virtualization ? I > don't think so, because several jobs running inside the same host > communicate via broadcast/multicast between them and between other jobs > running on different hosts. The IP consumption is a problem too: 1 > container == 2 IP (one for the root namespace/ one for the container), > multiplicated with the number of jobs. Furthermore, lot of jobs == lot > of virtual devices. > > However, after a discussion with Kirill at the OLS, it appears we can > merge the layer 2 and 3 approaches if the level of network > virtualization is tunable and we can choose layer 2 or layer 3 when > doing the "unshare". The determination of the namespace for the incoming > traffic can be done with an specific iptable module as a first step. > While looking at the network namespace patches, it appears that the > TCP/UDP part is **very** similar at what is needed for a layer 3 approach. > > Any thoughts ? My humble opinion is that your approach doesn't intersect with this one. So we can freely go with both *if needed*. And hear the comments fr…
creator: dev
date: 2006-09-05T15:44:19-00:00

Re: [RFC] network namespaces
type: item
description: On Tue, Sep 05, 2006 at 08:45:39AM -0600, Eric W. Biederman wrote: > Daniel Lezcano <dlezcano@fr.ibm.com> writes: > > >>>2. People expressed concerns that complete separation of namespaces > >>> may introduce an undesired overhead in certain usage scenarios. > >>> The overhead comes from packets traversing input path, then output path, > >>> then input path again in the destination namespace if root namespace > >>> acts as a router. > > > > Yes, performance is probably one issue. > > > > My concerns was for layer 2 / layer 3 virtualization. I agree > > a layer 2 isolation/virtualization is the best for the "system > > container". But there is another family of container called > > "application container", it is not a system which is run inside a > > container but only the application. If you want to run a oracle > > database inside a container, you can run it inside an application > > container without launching <init> and all the services. > > > > This family of containers are used too for HPC (high performance > > computing) and for distributed checkpoint/restart. The cluster > > runs hundred of jobs, spawning them on different hosts inside an > > application container. Usually the jobs communicates with broadcast > > and multicast. Application containers does not care of having > > different MAC address and rely on a layer 3 approach. > > > > Are application containers comfortable with a layer 2 virtualization > > ? I don't think so, because several jobs running inside the same > > host communicate via broadcast/multicast between them and between > > other jobs running on different hosts. The IP consumption is a > > problem too: 1 container == 2 IP (one for the root namespace/ > > one for the container), multiplicated with the number of jobs. > > Furthermore, lot of jobs == lot of virtual devices. > > > > However, after a discussion with Kirill at the OLS, it appears we > > can merge the layer 2 and 3 approaches if the level of network > > virtualization is tunab…
creator: Herbert Poetzl
date: 2006-09-05T16:53:28-00:00

Re: [RFC] network namespaces
type: item
description: > This family of containers are used too for HPC (high performance computing) and > for distributed checkpoint/restart. The cluster runs hundred of jobs, spawning > them on different hosts inside an application container. Usually the jobs > communicates with broadcast and multicast. > Application containers does not care of having different MAC address and rely on > a layer 3 approach. Ok I think to understand this we need some precise definitions. In the normal case it is an error for a job to communication with a different job. The basic advantage with a different MAC is that you can found out who the intended recipient is sooner in the networking stack and you have truly separate network devices. Allowing for a cleaner implementation. Changing the MAC after migration is likely to be fine. > Are application containers comfortable with a layer 2 virtualization ? I don't > think so, because several jobs running inside the same host communicate via > broadcast/multicast between them and between other jobs running on different > hosts. The IP consumption is a problem too: 1 container == 2 IP (one for the > root namespace/ one for the container), multiplicated with the number of > jobs. Furthermore, lot of jobs == lot of virtual devices. First if you hook you network namespaces with ethernet bridging you don't need any extra IPs. Second don't see the conflict you perceive between application containers and layer 2 containment. The bottom line is that you need at least one loopback interface per non-trivial network namespace. One you get that having a virtual is no big deal. In addition network devices don't consume less memory than a process. So lots of network devices should not be a problem. Eric
creator: ebiederm
date: 2006-09-05T17:09:21-00:00

Re: [RFC] network namespaces
type: item
description: Herbert Poetzl <herbert@13thfloor.at> writes: > On Tue, Sep 05, 2006 at 08:45:39AM -0600, Eric W. Biederman wrote: >> Daniel Lezcano <dlezcano@fr.ibm.com> writes: >> >> For HPC if you are interested in migration you need a separate IP >> per container. If you can take you IP address with you migration of >> networking state is simple. If you can't take your IP address with you >> a network container is nearly pointless from a migration perspective. >> >> Beyond that from everything I have seen layer 2 is just much cleaner >> than any layer 3 approach short of Serge's bind filtering. > > well, the 'ip subset' approach Linux-VServer and > other Jail solutions use is very clean, it just does > not match your expectations of a virtual interface > (as there is none) and it does not cope well with > all kinds of per context 'requirements', which IMHO > do not really exist on the application layer (only > on the whole system layer) I probably expressed that wrong. There are currently three basic approaches under discussion. Layer 3 (Basically bind filtering) nothing at the packet level. The approach taken by Serge's version of bsdjails and Vserver. Layer 2.5 What Daniel proposed. Layer 2. (Trivially mapping each packet to a different interface) And then treating everything as multiple instances of the network stack. Roughly what OpenVZ and I have implemented. You can get into some weird complications at layer 3 but because it doesn't touch each packet the proof it is fast is trivial. >> Beyond that I have yet to see a clean semantics for anything >> resembling your layer 2 layer 3 hybrid approach. If we can't have >> clear semantics it is by definition impossible to implement correctly >> because no one understands what it is supposed to do. > > IMHO that would be quite simple, have a 'namespace' > for limiting port binds to a subset of the available > ips and another one which does complete network > virtualization with all the whistles and bells, IMHO > most of them are …
creator: ebiederm
date: 2006-09-05T18:27:20-00:00

Re: [RFC] network namespaces
type: item
description: Hi Herbert, > well, the 'ip subset' approach Linux-VServer and > other Jail solutions use is very clean, it just does > not match your expectations of a virtual interface > (as there is none) and it does not cope well with > all kinds of per context 'requirements', which IMHO > do not really exist on the application layer (only > on the whole system layer) > > IMHO that would be quite simple, have a 'namespace' > for limiting port binds to a subset of the available > ips and another one which does complete network > virtualization with all the whistles and bells, IMHO > most of them are orthogonal and can easily be combined > > - full network virtualization > - lightweight ip subset > - both > > IMHO this requirement only arises from the full system > virtualization approach, just look at the other jail > solutions (solaris, bsd, ...) some of them do not even > allow for more than a single ip but they work quite > well when used properly ... As far as I see, vserver use a layer 3 solution but, when needed, the veth "component", made by Nestor Pena, is used to provide a layer 2 virtualization. Right ? Having the two solutions, you have certainly a lot if information about use cases. From the point of view of vserver, can you give some examples of when a layer 3 solution is better/worst than a layer 2 solution ? Who wants a layer 2/3 virtualization and why ? These informations will be very useful. Regards -- Daniel
creator: Daniel Lezcano
date: 2006-09-06T09:10:23-00:00

Re: [RFC] network namespaces
type: item
description: >>On Tue, Sep 05, 2006 at 08:45:39AM -0600, Eric W. Biederman wrote: >> >>>Daniel Lezcano <dlezcano@fr.ibm.com> writes: >>> >>>For HPC if you are interested in migration you need a separate IP >>>per container. If you can take you IP address with you migration of >>>networking state is simple. If you can't take your IP address with you >>>a network container is nearly pointless from a migration perspective. >>> >>>Beyond that from everything I have seen layer 2 is just much cleaner >>>than any layer 3 approach short of Serge's bind filtering. >> >>well, the 'ip subset' approach Linux-VServer and >>other Jail solutions use is very clean, it just does >>not match your expectations of a virtual interface >>(as there is none) and it does not cope well with >>all kinds of per context 'requirements', which IMHO >>do not really exist on the application layer (only >>on the whole system layer) > > > I probably expressed that wrong. There are currently three > basic approaches under discussion. > Layer 3 (Basically bind filtering) nothing at the packet level. > The approach taken by Serge's version of bsdjails and Vserver. > > Layer 2.5 What Daniel proposed. > > Layer 2. (Trivially mapping each packet to a different interface) > And then treating everything as multiple instances of the > network stack. > Roughly what OpenVZ and I have implemented. I think classifying network virtualization by Layer X is not good enough. OpenVZ has Layer 3 (venet) and Layer 2 (veth) implementations, but in both cases networking stack inside VE remains fully virtualized. Thanks, Kirill _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: dev
date: 2006-09-06T14:52:50-00:00

Re: Re: [RFC] network namespaces
type: item
description: Kirill Korotaev wrote: > I think classifying network virtualization by Layer X is not good enough. > OpenVZ has Layer 3 (venet) and Layer 2 (veth) implementations, but > in both cases networking stack inside VE remains fully virtualized. > Let's describe all those (three?) approaches at http://wiki.openvz.org/Containers/Networking Everyone is able to read and contribute to that, and (I hope) we will come to the common understanding. I have started the article, please enlarge.
creator: kir
date: 2006-09-06T15:09:28-00:00

Re: [RFC] network namespaces
type: item
description: On Wed, Sep 06, 2006 at 11:10:23AM +0200, Daniel Lezcano wrote: > Hi Herbert, > > >well, the 'ip subset' approach Linux-VServer and > >other Jail solutions use is very clean, it just does > >not match your expectations of a virtual interface > >(as there is none) and it does not cope well with > >all kinds of per context 'requirements', which IMHO > >do not really exist on the application layer (only > >on the whole system layer) > > > >IMHO that would be quite simple, have a 'namespace' > >for limiting port binds to a subset of the available > >ips and another one which does complete network > >virtualization with all the whistles and bells, IMHO > >most of them are orthogonal and can easily be combined > > > > - full network virtualization > > - lightweight ip subset > > - both > > > >IMHO this requirement only arises from the full system > >virtualization approach, just look at the other jail > >solutions (solaris, bsd, ...) some of them do not even > >allow for more than a single ip but they work quite > >well when used properly ... > > As far as I see, vserver use a layer 3 solution but, when needed, the > veth "component", made by Nestor Pena, is used to provide a layer 2 > virtualization. Right ? well, no, we do not explicitely use the VETH daemon for networking, although some folks probably make use of it, mainly because if you realize that this kind of isolation is something different and partially complementary to network virtualization, you can do live without the layer 2 virtualization in almost all cases, nevertheless, for certain purposes layer 2/3 virtualization is required and/or makes perfect sense > Having the two solutions, you have certainly a lot if information > about use cases. > From the point of view of vserver, can you give some > examples of when a layer 3 solution is better/worst than > a layer 2 solution ? my point (until we have an implementation which clearly shows that performance is equal/better to isolation) is simply this: of cours…
creator: Herbert Poetzl
date: 2006-09-06T16:56:43-00:00

Re: Re: [RFC] network namespaces
type: item
description: Herbert Poetzl wrote: > my point (until we have an implementation which clearly > shows that performance is equal/better to isolation) > is simply this: > > of course, you can 'simulate' or 'construct' all the > isolation scenarios with kernel bridging and routing > and tricky injection/marking of packets, but, this > usually comes with an overhead ... > Well, TANSTAAFL*, and pretty much everything comes with an overhead. Multitasking comes with the (scheduler, context switch, CPU cache, etc.) overhead -- is that the reason to abandon it? OpenVZ and Linux-VServer resource management also adds some overhead -- do we want to throw it away? The question is not just "equal or better performance", the question is "what do we get and how much we pay for it". Finally, as I understand both network isolation and network virtualization (both level2 and level3) can happily co-exist. We do have several filesystems in kernel. Let's have several network virtualization approaches, and let a user choose. Is that makes sense? * -- http://en.wikipedia.org/wiki/TANSTAAFL
creator: kir
date: 2006-09-06T17:36:17-00:00

Re: [RFC] network namespaces
type: item
description: Herbert Poetzl <herbert@13thfloor.at> writes: > On Wed, Sep 06, 2006 at 11:10:23AM +0200, Daniel Lezcano wrote: >> >> As far as I see, vserver use a layer 3 solution but, when needed, the >> veth "component", made by Nestor Pena, is used to provide a layer 2 >> virtualization. Right ? > > well, no, we do not explicitely use the VETH daemon > for networking, although some folks probably make use > of it, mainly because if you realize that this kind > of isolation is something different and partially > complementary to network virtualization, you can do > live without the layer 2 virtualization in almost > all cases, nevertheless, for certain purposes layer > 2/3 virtualization is required and/or makes perfect > sense > >> Having the two solutions, you have certainly a lot if information >> about use cases. > >> From the point of view of vserver, can you give some >> examples of when a layer 3 solution is better/worst than >> a layer 2 solution ? > > my point (until we have an implementation which clearly > shows that performance is equal/better to isolation) > is simply this: > > of course, you can 'simulate' or 'construct' all the > isolation scenarios with kernel bridging and routing > and tricky injection/marking of packets, but, this > usually comes with an overhead ... > >> Who wants a layer 2/3 virtualization and why ? > > there are some reasons for virtualization instead of > pure isolation (as Linux-VServer does it for now) > > - context migration/snapshot (probably reason #1) > - creating network devices inside a guest > (can help with vpn and similar) > - allowing non IP protocols (like DHCP, ICMP, etc) > > the problem which arises with this kind of network > virtualization is that you need some additional policy > for example to avoid sending 'evil' packets and/or > (D)DoSing one guest from another, which again adds > further overhead, so basically if you 'just' want > to have network isolation, you have to do this: > > - create a 'copy' of your hosts netw…
creator: ebiederm
date: 2006-09-06T17:58:39-00:00

Re: [RFC] network namespaces
type: item
description: Kir Kolyshkin <kir@openvz.org> writes: > Herbert Poetzl wrote: >> my point (until we have an implementation which clearly >> shows that performance is equal/better to isolation) >> is simply this: >> >> of course, you can 'simulate' or 'construct' all the >> isolation scenarios with kernel bridging and routing >> and tricky injection/marking of packets, but, this >> usually comes with an overhead ... >> > Well, TANSTAAFL*, and pretty much everything comes with an overhead. > Multitasking comes with the (scheduler, context switch, CPU cache, etc.) > overhead -- is that the reason to abandon it? OpenVZ and Linux-VServer > resource management also adds some overhead -- do we want to throw it away? > > The question is not just "equal or better performance", the question is > "what do we get and how much we pay for it". Equal or better performance is certainly required when we have the code compiled in but aren't using it. We must not penalize the current code. > Finally, as I understand both network isolation and network > virtualization (both level2 and level3) can happily co-exist. We do have > several filesystems in kernel. Let's have several network virtualization > approaches, and let a user choose. Is that makes sense? If there are not compelling arguments for using both ways of doing it is silly to merge both, as it is more maintenance overhead. That said I think there is a real chance if we can look at the bind filtering and find a way to express that in the networking stack through iptables. Using the security hooks conflicts with things like selinux. Although it would be interesting to see if selinux can already implement general purpose layer 3 filtering. The more I look the gut feel I have is that the way to proceed would be to add a new table that filters binds, and connects. Plus a new module that would look at a process creating a socket and tell us if it is the appropriate group of processes. With a little care that would be a general solution to the lay…
creator: ebiederm
date: 2006-09-06T18:34:59-00:00

Re: [RFC] network namespaces
type: item
description: Eric W. Biederman wrote: > Kir Kolyshkin <kir@openvz.org> writes: > > >> Herbert Poetzl wrote: >> >>> my point (until we have an implementation which clearly >>> shows that performance is equal/better to isolation) >>> is simply this: >>> >>> of course, you can 'simulate' or 'construct' all the >>> isolation scenarios with kernel bridging and routing >>> and tricky injection/marking of packets, but, this >>> usually comes with an overhead ... >>> >>> >> Well, TANSTAAFL*, and pretty much everything comes with an overhead. >> Multitasking comes with the (scheduler, context switch, CPU cache, etc.) >> overhead -- is that the reason to abandon it? OpenVZ and Linux-VServer >> resource management also adds some overhead -- do we want to throw it away? >> >> The question is not just "equal or better performance", the question is >> "what do we get and how much we pay for it". >> > > Equal or better performance is certainly required when we have the code > compiled in but aren't using it. We must not penalize the current code. > That's a valid argument. Although it's not applicable here (at least for both network virtualization types which OpenVZ offers). Kirill/Andrey, please correct me if I'm wrong here. >> Finally, as I understand both network isolation and network >> virtualization (both level2 and level3) can happily co-exist. We do have >> several filesystems in kernel. Let's have several network virtualization >> approaches, and let a user choose. Is that makes sense? >> > o > If there are not compelling arguments for using both ways of doing > it is silly to merge both, as it is more maintenance overhead. > Definitely a valid argument as well. I am not sure about "network isolation" (used by Linux-VServer), but as it comes for level2 vs. level3 virtualization, I see a need for both. Here is the easy-to-understand comparison which can shed some light: http://wiki.openvz.org/Differences_between_venet_and_veth Here are a couple of examples * Do we want to let container…
creator: kir
date: 2006-09-06T18:56:43-00:00

Re: [RFC] network namespaces
type: item
description: Eric W. Biederman wrote: >> This family of containers are used too for HPC (high performance computing) and >> for distributed checkpoint/restart. The cluster runs hundred of jobs, spawning >> them on different hosts inside an application container. Usually the jobs >> communicates with broadcast and multicast. >> Application containers does not care of having different MAC address and rely on >> a layer 3 approach. > > Ok I think to understand this we need some precise definitions. > In the normal case it is an error for a job to communication with a different > job. hmm ? What about an MPI application ? I would expect each MPI task to be run in its container on different nodes or on the same node. These individual tasks _communicate_ between each other through the MPI layer (not only TCP btw) to complete a large calculation. > The basic advantage with a different MAC is that you can found out who the > intended recipient is sooner in the networking stack and you have truly > separate network devices. Allowing for a cleaner implementation. > > Changing the MAC after migration is likely to be fine. indeed. C.
creator: Cedric Le Goater
date: 2006-09-06T20:25:10-00:00

Re: [RFC] network namespaces
type: item
description: Cedric Le Goater <clg@fr.ibm.com> writes: > Eric W. Biederman wrote: > > hmm ? What about an MPI application ? > > I would expect each MPI task to be run in its container on different nodes > or on the same node. These individual tasks _communicate_ between each > other through the MPI layer (not only TCP btw) to complete a large calculation. All parts of the MPI application are part of the same job. Communication between processes on multiple machines that are part of the job is fine. At least that is how I used job in HPC computer context. Eric
creator: ebiederm
date: 2006-09-06T20:40:13-00:00

Re: [RFC] network namespaces
type: item
description: Kir Kolyshkin wrote: <snip> > I am not sure about "network isolation" (used by Linux-VServer), but as > it comes for level2 vs. level3 virtualization, I see a need for both. > Here is the easy-to-understand comparison which can shed some light: > http://wiki.openvz.org/Differences_between_venet_and_veth thanks kir, > Here are a couple of examples > * Do we want to let container's owner (i.e. root) to add/remove IP > addresses? Most probably not, but in some cases we want that. > * Do we want to be able to run DHCP server and/or DHCP client inside a > container? Sometimes...but not always. > * Do we want to let container's owner to create/manage his own set of > iptables? In half of the cases we do. > > The problem here is single solution will not cover all those scenarios. some would argue that there is one single solution : Xen or similar. IMO, I think containers should try to leverage their difference, performance, and not try to simulate a real hardware environment. Restricting the network environment of a container should be considered acceptable if this is for the sake of performance. The network interface(s) could be pre-configured and provided to the container. Protocol(s) could be forbidden. Now, if you need more network power in a container, you will need a real or a virtualized interface. But let's consider both alternatives. thanks, C. _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Cedric Le Goater
date: 2006-09-06T20:53:08-00:00

Re: Re: [RFC] network namespaces
type: item
description: Kir Kolyshkin wrote: > Herbert Poetzl wrote: > >> my point (until we have an implementation which clearly >> shows that performance is equal/better to isolation) >> is simply this: >> >> of course, you can 'simulate' or 'construct' all the >> isolation scenarios with kernel bridging and routing >> and tricky injection/marking of packets, but, this >> usually comes with an overhead ... >> > > Well, TANSTAAFL*, and pretty much everything comes with an overhead. > Multitasking comes with the (scheduler, context switch, CPU cache, etc.) > overhead -- is that the reason to abandon it? OpenVZ and Linux-VServer > resource management also adds some overhead -- do we want to throw it away? > > The question is not just "equal or better performance", the question is > "what do we get and how much we pay for it". > > Finally, as I understand both network isolation and network > virtualization (both level2 and level3) can happily co-exist. We do have > several filesystems in kernel. Let's have several network virtualization > approaches, and let a user choose. Is that makes sense? Definitly yes, I agree.
creator: Daniel Lezcano
date: 2006-09-06T21:44:35-00:00

RE: [RFC] network namespaces
type: item
description: ebiederm@xmission.com wrote: > >> Finally, as I understand both network isolation and network >> virtualization (both level2 and level3) can happily co-exist. We do >> have several filesystems in kernel. Let's have several network >> virtualization approaches, and let a user choose. Is that makes >> sense? > > If there are not compelling arguments for using both ways of > doing it is silly to merge both, as it is more maintenance overhead. > My reading is that full virtualization (Xen, etc.) calls for implementing L2 switching between the partitions and the physical NIC(s). The tradeoffs between L2 and L3 switching are indeed complex, but there are two implications of doing L2 switching between partitions: 1) Do we really want to ask device drivers to support L2 switching for partitions and something *different* for containers? 2) Do we really want any single packet to traverse an L2 switch (for the partition-style virtualization layer) and then an L3 switch (for the container-style layer)? The full virtualization solution calls for virtual NICs with distinct MAC addresses. Is there any reason why this same solution cannot work for containers (just creating more than one VNIC for the partition, and then assigning each VNIC to a container?)
creator: Caitlin Bestler
date: 2006-09-06T23:06:16-00:00

Re: [RFC] network namespaces
type: item
description: "Caitlin Bestler" <caitlinb@broadcom.com> writes: > ebiederm@xmission.com wrote: > >> >>> Finally, as I understand both network isolation and network >>> virtualization (both level2 and level3) can happily co-exist. We do >>> have several filesystems in kernel. Let's have several network >>> virtualization approaches, and let a user choose. Is that makes >>> sense? >> >> If there are not compelling arguments for using both ways of >> doing it is silly to merge both, as it is more maintenance overhead. >> > > My reading is that full virtualization (Xen, etc.) calls for > implementing > L2 switching between the partitions and the physical NIC(s). > > The tradeoffs between L2 and L3 switching are indeed complex, but > there are two implications of doing L2 switching between partitions: > > 1) Do we really want to ask device drivers to support L2 switching for > partitions and something *different* for containers? No. > 2) Do we really want any single packet to traverse an L2 switch (for > the partition-style virtualization layer) and then an L3 switch > (for the container-style layer)? In general what has been done with layer 3 is to simply filter which processes can use which IP addresses and it all happens at socket creation time. So it is very cheap, and it can be done purely in the network layer without any driver intervention. Basically think of what is happening at layer 3 as an extremely light-weight version of traffic filtering. > The full virtualization solution calls for virtual NICs with distinct > MAC addresses. Is there any reason why this same solution cannot work > for containers (just creating more than one VNIC for the partition, > and then assigning each VNIC to a container?) The VNIC approach is the fundamental idea with the layer two networking and if we can push the work down into the device driver it so different destination macs show up a in different packet queues it should be as fast as a normal networking stack. Implementing VNICs so far is th…
creator: ebiederm
date: 2006-09-06T23:25:50-00:00

Re: [RFC] network namespaces
type: item
description: On Wed, 06 Sep 2006 17:25:50 -0600 ebiederm@xmission.com (Eric W. Biederman) wrote: > "Caitlin Bestler" <caitlinb@broadcom.com> writes: > > > ebiederm@xmission.com wrote: > > > >> > >>> Finally, as I understand both network isolation and network > >>> virtualization (both level2 and level3) can happily co-exist. We do > >>> have several filesystems in kernel. Let's have several network > >>> virtualization approaches, and let a user choose. Is that makes > >>> sense? > >> > >> If there are not compelling arguments for using both ways of > >> doing it is silly to merge both, as it is more maintenance overhead. > >> > > > > My reading is that full virtualization (Xen, etc.) calls for > > implementing > > L2 switching between the partitions and the physical NIC(s). > > > > The tradeoffs between L2 and L3 switching are indeed complex, but > > there are two implications of doing L2 switching between partitions: > > > > 1) Do we really want to ask device drivers to support L2 switching for > > partitions and something *different* for containers? > > No. > > > 2) Do we really want any single packet to traverse an L2 switch (for > > the partition-style virtualization layer) and then an L3 switch > > (for the container-style layer)? > > In general what has been done with layer 3 is to simply filter which > processes can use which IP addresses and it all happens at socket > creation time. So it is very cheap, and it can be done purely > in the network layer without any driver intervention. > > Basically think of what is happening at layer 3 as an extremely light-weight > version of traffic filtering. > > > The full virtualization solution calls for virtual NICs with distinct > > MAC addresses. Is there any reason why this same solution cannot work > > for containers (just creating more than one VNIC for the partition, > > and then assigning each VNIC to a container?) > > The VNIC approach is the fundamental idea with the layer two networking > and if we can push the work down…
creator: Stephen Hemminger
date: 2006-09-07T00:53:12-00:00

Re: [RFC] network namespaces
type: item
description: Stephen Hemminger <shemminger@osdl.org> writes: > The problem with VNIC's is it won't work for all devices (without lots of > work), and for many device's it requires putting the device in promiscuous > mode. It also plays havoc with network access control devices. Which is fine. If it works it is a cool performance optimization. But it doesn't stop anything from my side if it doesn't. Eric
creator: ebiederm
date: 2006-09-07T05:11:27-00:00

Re: [RFC] network namespaces
type: item
description: Caitlin Bestler wrote: > ebiederm@xmission.com wrote: > > >>>Finally, as I understand both network isolation and network >>>virtualization (both level2 and level3) can happily co-exist. We do >>>have several filesystems in kernel. Let's have several network >>>virtualization approaches, and let a user choose. Is that makes >>>sense? >> >>If there are not compelling arguments for using both ways of >>doing it is silly to merge both, as it is more maintenance overhead. >> > > > My reading is that full virtualization (Xen, etc.) calls for > implementing > L2 switching between the partitions and the physical NIC(s). > > The tradeoffs between L2 and L3 switching are indeed complex, but > there are two implications of doing L2 switching between partitions: > > 1) Do we really want to ask device drivers to support L2 switching for > partitions and something *different* for containers? > > 2) Do we really want any single packet to traverse an L2 switch (for > the partition-style virtualization layer) and then an L3 switch > (for the container-style layer)? > > The full virtualization solution calls for virtual NICs with distinct > MAC addresses. Is there any reason why this same solution cannot work > for containers (just creating more than one VNIC for the partition, > and then assigning each VNIC to a container?) IHMO, I think there is one reason. The unsharing mechanism is not only for containers, its aim other kind of isolation like a "bsdjail" for example. The unshare syscall is flexible, shall the network unsharing be one-block solution ? For example, we want to launch an application using TCP/IP and we want to have an IP address only used by the application, nothing more. With a layer 2, we must after unsharing: 1) create a virtual device into the application namespace 2) assign an IP address 3) create a virtual device pass-through in the root namespace 4) set the virtual device IP All this stuff, need a lot of administration (check mac addresses conflicts, check int…
creator: Daniel Lezcano
date: 2006-09-07T08:25:56-00:00

Re: Re: [RFC] network namespaces
type: item
description: >>Herbert Poetzl wrote: >> >>>my point (until we have an implementation which clearly >>>shows that performance is equal/better to isolation) >>>is simply this: >>> >>> of course, you can 'simulate' or 'construct' all the >>> isolation scenarios with kernel bridging and routing >>> and tricky injection/marking of packets, but, this >>> usually comes with an overhead ... >>> >> >>Well, TANSTAAFL*, and pretty much everything comes with an overhead. >>Multitasking comes with the (scheduler, context switch, CPU cache, etc.) >>overhead -- is that the reason to abandon it? OpenVZ and Linux-VServer >>resource management also adds some overhead -- do we want to throw it away? >> >>The question is not just "equal or better performance", the question is >>"what do we get and how much we pay for it". > > > Equal or better performance is certainly required when we have the code > compiled in but aren't using it. We must not penalize the current code. you talk about host system performance. Both approaches do not introduce overhead to host networking. >>Finally, as I understand both network isolation and network >>virtualization (both level2 and level3) can happily co-exist. We do have >>several filesystems in kernel. Let's have several network virtualization >>approaches, and let a user choose. Is that makes sense? > > > If there are not compelling arguments for using both ways of doing > it is silly to merge both, as it is more maintenance overhead. > > That said I think there is a real chance if we can look at the bind > filtering and find a way to express that in the networking stack > through iptables. Using the security hooks conflicts with things > like selinux. Although it would be interesting to see if selinux > can already implement general purpose layer 3 filtering. > > The more I look the gut feel I have is that the way to proceed would > be to add a new table that filters binds, and connects. Plus a new > module that would look at a process creating a socket and tel…
creator: dev
date: 2006-09-07T16:20:47-00:00

Re: Re: [RFC] network namespaces
type: item
description: On Thu, Sep 07, 2006 at 08:23:53PM +0400, Kirill Korotaev wrote: > >>Herbert Poetzl wrote: > >> > >>>my point (until we have an implementation which clearly > >>>shows that performance is equal/better to isolation) > >>>is simply this: > >>> > >>> of course, you can 'simulate' or 'construct' all the > >>> isolation scenarios with kernel bridging and routing > >>> and tricky injection/marking of packets, but, this > >>> usually comes with an overhead ... > >>> > >> > >>Well, TANSTAAFL*, and pretty much everything comes with an overhead. > >>Multitasking comes with the (scheduler, context switch, CPU cache, etc.) > >>overhead -- is that the reason to abandon it? OpenVZ and Linux-VServer > >>resource management also adds some overhead -- do we want to throw it away? > >> > >>The question is not just "equal or better performance", the question is > >>"what do we get and how much we pay for it". > > > > > > Equal or better performance is certainly required when we have the code > > compiled in but aren't using it. We must not penalize the current code. > you talk about host system performance. > Both approaches do not introduce overhead to host networking. > > >>Finally, as I understand both network isolation and network > >>virtualization (both level2 and level3) can happily co-exist. We do have > >>several filesystems in kernel. Let's have several network virtualization > >>approaches, and let a user choose. Is that makes sense? > > > > > > If there are not compelling arguments for using both ways of doing > > it is silly to merge both, as it is more maintenance overhead. > > > > That said I think there is a real chance if we can look at the bind > > filtering and find a way to express that in the networking stack > > through iptables. Using the security hooks conflicts with things > > like selinux. Although it would be interesting to see if selinux > > can already implement general purpose layer 3 filtering. > > > > The more I look the gut feel I have is that the way …
creator: Herbert Poetzl
date: 2006-09-07T17:27:59-00:00

Re: [RFC] network namespaces
type: item
description: Daniel Lezcano <dlezcano@fr.ibm.com> writes: > > IHMO, I think there is one reason. The unsharing mechanism is not only for > containers, its aim other kind of isolation like a "bsdjail" for example. The > unshare syscall is flexible, shall the network unsharing be one-block solution ? > For example, we want to launch an application using TCP/IP and we want to have > an IP address only used by the application, nothing more. > With a layer 2, we must after unsharing: > 1) create a virtual device into the application namespace > 2) assign an IP address > 3) create a virtual device pass-through in the root namespace > 4) set the virtual device IP > > All this stuff, need a lot of administration (check mac addresses conflicts, > check interface names collision in root namespace, ...) for a simple network > isolation. Yes, and even more it is hard to show that it will perform as well. Although by dropping CAP_NET_ADMIN the actual runtime administration is about the same. > With a layer 3: > 1) assign an IP address > > In the other hand, a layer 3 isolation is not sufficient to reach the level of > isolation/virtualization needed for the system containers. Agreed. > Very soon, I will commit more info at: > > http://wiki.openvz.org/Containers/Networking > > So the consensus is based on the fact that there is a lot of common code for the > layer 2 and layer 3 isolation/virtualization and we can find a way to merge the > 2 implementation in order to have a flexible network virtualization/isolation. NACK In a real level 3 implementation there is very little common code with a layer 2 implementation. You don't need to muck with the socket handling code as you are not allowed to dup addresses between containers. Look at what Serge did that is layer 3. A layer 3 isolation implementation should either be a new security module or a new form of iptables. The problem with using the lsm is that it seems to be an all or nothing mechanism so is a very coarse grained tool for this job. …
creator: ebiederm
date: 2006-09-07T18:29:21-00:00

Re: Re: [RFC] network namespaces
type: item
description: Herbert Poetzl <herbert@13thfloor.at> writes: > On Thu, Sep 07, 2006 at 08:23:53PM +0400, Kirill Korotaev wrote: > > well, who said that you need to have things like RAW sockets > or other protocols except IP, not to speak of iptable and > routing entries ... > > folks who _want_ full network virtualization can use the > more complete virtual setup and be happy ... Exactly this was a proposal for isolation for containers that don't get CAP_NET_ADMIN, with a facility that could easily be general purpose. Eric _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: ebiederm
date: 2006-09-07T19:50:55-00:00

Re: [RFC] network namespaces
type: item
description: On Thu, Sep 07, 2006 at 12:29:21PM -0600, Eric W. Biederman wrote: > Daniel Lezcano <dlezcano@fr.ibm.com> writes: > > > > IHMO, I think there is one reason. The unsharing mechanism is > > not only for containers, its aim other kind of isolation like a > > "bsdjail" for example. The unshare syscall is flexible, shall the > > network unsharing be one-block solution ? For example, we want to > > launch an application using TCP/IP and we want to have > > an IP address only used by the application, nothing more. > > With a layer 2, we must after unsharing: > > 1) create a virtual device into the application namespace > > 2) assign an IP address > > 3) create a virtual device pass-through in the root namespace > > 4) set the virtual device IP > > > > All this stuff, need a lot of administration (check mac addresses > > conflicts, check interface names collision in root namespace, ...) > > for a simple network isolation. > > Yes, and even more it is hard to show that it will perform as well. > Although by dropping CAP_NET_ADMIN the actual runtime administration > is about the same. > > > With a layer 3: > > 1) assign an IP address > > > > In the other hand, a layer 3 isolation is not sufficient to reach > > the level of isolation/virtualization needed for the system > > containers. > > Agreed. > > > Very soon, I will commit more info at: > > > > http://wiki.openvz.org/Containers/Networking > > > > So the consensus is based on the fact that there is a lot of common > > code for the layer 2 and layer 3 isolation/virtualization and we can > > find a way to merge the 2 implementation in order to have a flexible > > network virtualization/isolation. > > NACK In a real level 3 implementation there is very little common > code with a layer 2 implementation. You don't need to muck with the > socket handling code as you are not allowed to dup addresses between > containers. Look at what Serge did that is layer 3. > > A layer 3 isolation implementation should either be a new securit…
creator: Herbert Poetzl
date: 2006-09-08T06:02:00-00:00

Re: Re: [RFC] network namespaces
type: item
description: On Thursday 07 September 2006 21:27, Herbert Poetzl wrote: > well, who said that you need to have things like RAW sockets > or other protocols except IP, not to speak of iptable and > routing entries ... > > folks who _want_ full network virtualization can use the > more complete virtual setup and be happy ... Let's think about how to implement this. As I understood VServer's design, your proposal is to split CAP_NET_ADMIN to multiple capabilities and use them if required. So, for your light-weight container it is enough to implement context isolation for protected by CAP_NET_IP capability (for example) code and put 'if (!capable(CAP_NET_*))' checks to all other places. But this could be easily implemented over OpenVZ code by CAP_VE_NET_ADMIN split. So, the question is: Could you point out the places in Andrey's implementation of network namespaces, which prevents you to add CAP_NET_ADMIN separation later? -- Thanks, Dmitry.
creator: Mishin Dmitry
date: 2006-09-08T13:10:08-00:00

Re: Re: [RFC] network namespaces
type: item
description: On Fri, Sep 08, 2006 at 05:10:08PM +0400, Dmitry Mishin wrote: > On Thursday 07 September 2006 21:27, Herbert Poetzl wrote: > > well, who said that you need to have things like RAW sockets > > or other protocols except IP, not to speak of iptable and > > routing entries ... > > > > folks who _want_ full network virtualization can use the > > more complete virtual setup and be happy ... > Let's think about how to implement this. > As I understood VServer's design, your proposal is to split > CAP_NET_ADMIN to multiple capabilities and use them if required. So, > for your light-weight container it is enough to implement context > isolation for protected by CAP_NET_IP capability (for example) code > and put 'if (!capable(CAP_NET_*))' checks to all other places. actually the light-weight ip isolation runs perfectly fine _without_ CAP_NET_ADMIN, as you do not want the guest to be able to mess with the 'configured' ips at all (not to speak of interfaces here) best, Herbert > But this could be easily implemented over OpenVZ code by > CAP_VE_NET_ADMIN split. > > So, the question is: > Could you point out the places in Andrey's implementation of network > namespaces, which prevents you to add CAP_NET_ADMIN separation later? > > -- > Thanks, > Dmitry.
creator: Herbert Poetzl
date: 2006-09-08T18:11:54-00:00

Re: Re: [RFC] network namespaces
type: item
description: On Friday 08 September 2006 22:11, Herbert Poetzl wrote: > actually the light-weight ip isolation runs perfectly > fine _without_ CAP_NET_ADMIN, as you do not want the > guest to be able to mess with the 'configured' ips at > all (not to speak of interfaces here) It was only an example. I'm thinking about how to implement flexible solution, which permits light-weight ip isolation as well as full-fledged netwrok virtualization. Another solution is to split CONFIG_NET_NAMESPACE. Is it good for you? -- Thanks, Dmitry.
creator: Mishin Dmitry
date: 2006-09-09T07:57:24-00:00

Re: Re: [RFC] network namespaces
type: item
description: On Sat, Sep 09, 2006 at 11:57:24AM +0400, Dmitry Mishin wrote: > On Friday 08 September 2006 22:11, Herbert Poetzl wrote: > > actually the light-weight ip isolation runs perfectly > > fine _without_ CAP_NET_ADMIN, as you do not want the > > guest to be able to mess with the 'configured' ips at > > all (not to speak of interfaces here) > It was only an example. I'm thinking about how to implement flexible > solution, which permits light-weight ip isolation as well as > full-fledged netwrok virtualization. Another solution is to split > CONFIG_NET_NAMESPACE. Is it good for you? well, I think it would be best to have both, as they are complementary to some degree, and IMHO both, the full virtualization _and_ the isolation will require a separate namespace to work, I also think that limiting the isolation to something very simple (like one IP + network or so) would be acceptable for a start, because especially multi IP or network range checks require a little more efford to get them right ... I do not think that folks would want to recompile their kernel just to get a light-weight guest or a fully virtualized one best, Herbert > -- > Thanks, > Dmitry.
creator: Herbert Poetzl
date: 2006-09-10T02:47:09-00:00

Re: Re: [RFC] network namespaces
type: item
description: Herbert Poetzl <herbert@13thfloor.at> writes: > On Sat, Sep 09, 2006 at 11:57:24AM +0400, Dmitry Mishin wrote: >> On Friday 08 September 2006 22:11, Herbert Poetzl wrote: >> > actually the light-weight ip isolation runs perfectly >> > fine _without_ CAP_NET_ADMIN, as you do not want the >> > guest to be able to mess with the 'configured' ips at >> > all (not to speak of interfaces here) > >> It was only an example. I'm thinking about how to implement flexible >> solution, which permits light-weight ip isolation as well as >> full-fledged netwrok virtualization. Another solution is to split >> CONFIG_NET_NAMESPACE. Is it good for you? > > well, I think it would be best to have both, as > they are complementary to some degree, and IMHO > both, the full virtualization _and_ the isolation > will require a separate namespace to work, I also > think that limiting the isolation to something > very simple (like one IP + network or so) would > be acceptable for a start, because especially > multi IP or network range checks require a little > more efford to get them right ... > > I do not think that folks would want to recompile > their kernel just to get a light-weight guest or > a fully virtualized one I certainly agree that we are not at a point where a final decision can be made. A major piece of that is that a layer 2 approach has not shown to be without a performance penalty. A practical question. Do the IPs assigned to guests ever get used by anything besides the guest? Eric _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: ebiederm
date: 2006-09-10T03:41:35-00:00

Re: Re: [RFC] network namespaces
type: item
description: On Sunday 10 September 2006 06:47, Herbert Poetzl wrote: > well, I think it would be best to have both, as > they are complementary to some degree, and IMHO > both, the full virtualization _and_ the isolation > will require a separate namespace to work, [snip] > I do not think that folks would want to recompile > their kernel just to get a light-weight guest or > a fully virtualized one In this case light-weight guest will have unnecessary overhead. For example, instead of using static pointer, we have to find the required common namespace before. And there will be no advantages for such guest over full-featured. > > best, > Herbert > > > -- > > Thanks, > > Dmitry. -- Thanks, Dmitry.
creator: Mishin Dmitry
date: 2006-09-10T07:45:35-00:00

Re: Re: [RFC] network namespaces
type: item
description: On Sunday 10 September 2006 07:41, Eric W. Biederman wrote: > I certainly agree that we are not at a point where a final decision > can be made. A major piece of that is that a layer 2 approach has > not shown to be without a performance penalty. But it is required. Why to limit possible usages? > A practical question. Do the IPs assigned to guests ever get used > by anything besides the guest? In case of level2 virtualization - no. -- Thanks, Dmitry. _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Mishin Dmitry
date: 2006-09-10T08:11:35-00:00

Re: [RFC] network namespaces
type: item
description: Dmitry Mishin <dim@openvz.org> writes: > On Sunday 10 September 2006 07:41, Eric W. Biederman wrote: >> I certainly agree that we are not at a point where a final decision >> can be made. A major piece of that is that a layer 2 approach has >> not shown to be without a performance penalty. > But it is required. Why to limit possible usages? Wrong perspective. The point is that we need to dig in and show that there is no measurable penalty for the current cases. Showing that there is little penalty for the advanced configurations is a plus. The practical question is, do we need to implement the grand unified lookup before we can do this cheaply, or can we implement this without needing that optimization? To get a perspective, to get a good implementation of the pid namespace I am having to refactor significant parts of the kernel so it uses abstractions that can cleanly express what we are doing. The networking stack is in better shape but there is a lot of it. >> A practical question. Do the IPs assigned to guests ever get used >> by anything besides the guest? > In case of level2 virtualization - no. Actually that is one of the benefits of a layer 2 implementation you can set up weird things like shared IPs, that various types of fail over scenarios want. My question was really about the layer 3 bind filtering techniques, and how people are using them. The basic attraction with layer 3 is that you can do a simple implementation, and it will run very fast, and it doesn't need to conflict with the layer 2 work at all. If you can make that layer 3 implementation clean and generally mergeable as well it is worth pursuing. Eric _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: ebiederm
date: 2006-09-10T11:48:02-00:00

Re: Re: [RFC] network namespaces
type: item
description: On Sat, Sep 09, 2006 at 09:41:35PM -0600, Eric W. Biederman wrote: > Herbert Poetzl <herbert@13thfloor.at> writes: > > > On Sat, Sep 09, 2006 at 11:57:24AM +0400, Dmitry Mishin wrote: > >> On Friday 08 September 2006 22:11, Herbert Poetzl wrote: > >> > actually the light-weight ip isolation runs perfectly > >> > fine _without_ CAP_NET_ADMIN, as you do not want the > >> > guest to be able to mess with the 'configured' ips at > >> > all (not to speak of interfaces here) > > > >> It was only an example. I'm thinking about how to implement flexible > >> solution, which permits light-weight ip isolation as well as > >> full-fledged netwrok virtualization. Another solution is to split > >> CONFIG_NET_NAMESPACE. Is it good for you? > > > > well, I think it would be best to have both, as > > they are complementary to some degree, and IMHO > > both, the full virtualization _and_ the isolation > > will require a separate namespace to work, I also > > think that limiting the isolation to something > > very simple (like one IP + network or so) would > > be acceptable for a start, because especially > > multi IP or network range checks require a little > > more efford to get them right ... > > > > I do not think that folks would want to recompile > > their kernel just to get a light-weight guest or > > a fully virtualized one > > I certainly agree that we are not at a point where a final decision > can be made. A major piece of that is that a layer 2 approach has > not shown to be without a performance penalty. > > A practical question. Do the IPs assigned to guests ever get used > by anything besides the guest? only in special setups and for testing routing and general operation of course, i.e. one typical failure scenario is this: - 'provider' has a bunch of ips assigned - 'host' ip works perfectly - 'guest' ip is not routed (by the external router) in this case, for example, I always suggest to test on the host with a guest ip, simplest example: ping -I <guest-ip> google.com …
creator: Herbert Poetzl
date: 2006-09-10T19:19:00-00:00

Re: Re: [RFC] network namespaces
type: item
description: On Sun, Sep 10, 2006 at 11:45:35AM +0400, Dmitry Mishin wrote: > On Sunday 10 September 2006 06:47, Herbert Poetzl wrote: > > well, I think it would be best to have both, as > > they are complementary to some degree, and IMHO > > both, the full virtualization _and_ the isolation > > will require a separate namespace to work, > [snip] > > I do not think that folks would want to recompile > > their kernel just to get a light-weight guest or > > a fully virtualized one > In this case light-weight guest will have unnecessary overhead. For > example, instead of using static pointer, we have to find the required > common namespace before. this is only required at 'bind' time, which makes a non measurable fraction of the actual connection usage (unless you keep binding ports over and over without ever using them) > And there will be no advantages for such guest over full-featured. the advantage is in the flexibility, simplicity of setup and the basically non-existant overhead on the hot (conenction/transfer) part ... > > best, > > Herbert > > > > > -- > > > Thanks, > > > Dmitry. > > -- > Thanks, > Dmitry.
creator: Herbert Poetzl
date: 2006-09-10T19:22:44-00:00

Re: Re: [RFC] network namespaces
type: item
description: Dmitry Mishin wrote: > On Friday 08 September 2006 22:11, Herbert Poetzl wrote: > >>actually the light-weight ip isolation runs perfectly >>fine _without_ CAP_NET_ADMIN, as you do not want the >>guest to be able to mess with the 'configured' ips at >>all (not to speak of interfaces here) > > It was only an example. I'm thinking about how to implement flexible solution, > which permits light-weight ip isolation as well as full-fledged netwrok > virtualization. Another solution is to split CONFIG_NET_NAMESPACE. Is it good > for you? Hi Dmitry, I am currently working on this and I am finishing a prototype bringing isolation at the ip layer. The prototype code is very closed to Andrey's patches at TCP/UDP level. So the next step is to merge the prototype code with the existing network namespace layer 2 isolation. IHMO, the solution of spliting CONFIG_NET_NS into CONFIG_L2_NET_NS and CONFIG_L3_NET_NS is for me not acceptable because you will need to recompile the kernel. The proper way is certainly to have a specific flag for the unshare, something like CLONE_NEW_L2_NET and CLONE_NEW_L3_NET for example. -- Daniel _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Daniel Lezcano
date: 2006-09-11T14:40:59-00:00

Re: Re: [RFC] network namespaces
type: item
description: On Mon, Sep 11, 2006 at 04:40:59PM +0200, Daniel Lezcano wrote: > Dmitry Mishin wrote: > >On Friday 08 September 2006 22:11, Herbert Poetzl wrote: > > > >>actually the light-weight ip isolation runs perfectly > >>fine _without_ CAP_NET_ADMIN, as you do not want the > >>guest to be able to mess with the 'configured' ips at > >>all (not to speak of interfaces here) > > > >It was only an example. I'm thinking about how to implement flexible > >solution, which permits light-weight ip isolation as well as full-fledged > >netwrok virtualization. Another solution is to split CONFIG_NET_NAMESPACE. > >Is it good for you? > > Hi Dmitry, > > I am currently working on this and I am finishing a prototype bringing > isolation at the ip layer. The prototype code is very closed to > Andrey's patches at TCP/UDP level. So the next step is to merge the > prototype code with the existing network namespace layer 2 isolation. you might want to take a look at the current Linux-VServer implementation for the network isolation too, should be quite similar to Andrey's approach, but maybe you can gather some additional information from there > IHMO, the solution of spliting CONFIG_NET_NS into CONFIG_L2_NET_NS > and CONFIG_L3_NET_NS is for me not acceptable because you will need > to recompile the kernel. The proper way is certainly to have a > specific flag for the unshare, something like CLONE_NEW_L2_NET and > CLONE_NEW_L3_NET for example. I completely agree here, we need a separate namespace for that, so that we can combine isolation and virtualization as needed, unless the bind restrictions can be completely expressed with an additional mangle or filter table (as was suggested) best, Herbert > -- Daniel _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Herbert Poetzl
date: 2006-09-11T14:57:24-00:00

Re: Re: [RFC] network namespaces
type: item
description: Herbert Poetzl wrote: > On Mon, Sep 11, 2006 at 04:40:59PM +0200, Daniel Lezcano wrote: > >>I am currently working on this and I am finishing a prototype bringing >>isolation at the ip layer. The prototype code is very closed to >>Andrey's patches at TCP/UDP level. So the next step is to merge the >>prototype code with the existing network namespace layer 2 isolation. > > > you might want to take a look at the current Linux-VServer > implementation for the network isolation too, should be > quite similar to Andrey's approach, but maybe you can > gather some additional information from there ok, thanks. I will. >>IHMO, the solution of spliting CONFIG_NET_NS into CONFIG_L2_NET_NS >>and CONFIG_L3_NET_NS is for me not acceptable because you will need >>to recompile the kernel. The proper way is certainly to have a >>specific flag for the unshare, something like CLONE_NEW_L2_NET and >>CLONE_NEW_L3_NET for example. > > > I completely agree here, we need a separate namespace > for that, so that we can combine isolation and virtualization > as needed, unless the bind restrictions can be completely > expressed with an additional mangle or filter table (as > was suggested) What is the bind restriction ? Do you want to force binding to a specific source address ? -- Daniel _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Daniel Lezcano
date: 2006-09-11T15:04:38-00:00

Re: Re: [RFC] network namespaces
type: item
description: On Monday 11 September 2006 18:57, Herbert Poetzl wrote: > I completely agree here, we need a separate namespace > for that, so that we can combine isolation and virtualization > as needed, unless the bind restrictions can be completely > expressed with an additional mangle or filter table (as > was suggested) iptables are designed for packet flow decisions and filtering, it has nothing common with bind restrictions. So, it may be only packet flow scheduling/filtering, but it will not help to resolve bind-time IP conflicts. -- Thanks, Dmitry. _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Mishin Dmitry
date: 2006-09-11T15:10:31-00:00

Re: [RFC] network namespaces
type: item
description: Dmitry Mishin <dim@openvz.org> writes: > On Sunday 10 September 2006 06:47, Herbert Poetzl wrote: >> well, I think it would be best to have both, as >> they are complementary to some degree, and IMHO >> both, the full virtualization _and_ the isolation >> will require a separate namespace to work, > [snip] >> I do not think that folks would want to recompile >> their kernel just to get a light-weight guest or >> a fully virtualized one > In this case light-weight guest will have unnecessary overhead. > For example, instead of using static pointer, we have to find the required > common namespace before. And there will be no advantages for such guest over > full-featured. Dmitry that just isn't true if implemented properly. Eric _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: ebiederm
date: 2006-09-12T03:26:52-00:00

Re: [RFC] network namespaces
type: item
description: Dmitry Mishin <dim@openvz.org> writes: > On Monday 11 September 2006 18:57, Herbert Poetzl wrote: >> I completely agree here, we need a separate namespace >> for that, so that we can combine isolation and virtualization >> as needed, unless the bind restrictions can be completely >> expressed with an additional mangle or filter table (as >> was suggested) > > iptables are designed for packet flow decisions and filtering, it has nothing > common with bind restrictions. So, it may be only packet flow > scheduling/filtering, but it will not help to resolve bind-time IP conflicts. Please read the archive, where the suggestion was made. What was suggested was a new table, with it's own set of chains. So we could make filtering decisions on where sockets could be bound. That is not a far stretch from where iptables is today. Do you have some concrete arguments against the proposal? Eric _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: ebiederm
date: 2006-09-12T03:28:47-00:00

Re: [RFC] network namespaces
type: item
description: Sorry, dont' understand your proposal correctly from the previous talk. :) But... On Tuesday 12 September 2006 07:28, Eric W. Biederman wrote: > Do you have some concrete arguments against the proposal? Yes, I have. I think it is unnecessary complication. This complication will followed in additional bugs. Especially if we'll accept rules creation in userspace. Why we need complex solution, if there are only two approaches to socket bound - isolation and virtualization? These approaches could co-exist without hooks. Or you probably have thoughts about other ways? -- Thanks, Dmitry. _______________________________________________ Containers mailing list Containers@lists.osdl.org https://lists.osdl.org/mailman/listinfo/containers
creator: Mishin Dmitry
date: 2006-09-12T07:38:16-00:00

Re: [PATCH 4/9] network namespaces: socket hashes
type: item
description: Andrey Savochkin wrote: > Socket hash lookups are made within namespace. > Hash tables are common for all namespaces, with > additional permutation of indexes. Hi Andrey, why is the hash table common and not instanciated multiple times for each namespace like the routes ?
creator: Daniel Lezcano
date: 2006-09-18T15:12:49-00:00

Re: [PATCH 4/9] network namespaces: socket hashes
type: item
description: Hi, On Mon, Sep 18, 2006 at 05:12:49PM +0200, Daniel Lezcano wrote: > Andrey Savochkin wrote: > > Socket hash lookups are made within namespace. > > Hash tables are common for all namespaces, with > > additional permutation of indexes. > > Hi Andrey, > > why is the hash table common and not instanciated multiple times for > each namespace like the routes ? The main reason is that socket hash tables should be large enough to work efficiently, but it isn't good to waste a lot of memory for each namespace. Namespaces should be cheap enough, to allow to have hundreds of them. This reason of memory efficiency, of course, has a priority unless/until socket hash tables start to resize automatically. Another point is that routing lookup is much more complicated than the socket's one to add another search key. Routing also have additional routines for deleting entries matching some patterns, and so on. In short, routing is much more complicated, and it already quite efficient for various sizes of routing tables. Andrey
creator: Andrey Savochkin
date: 2006-09-20T16:32:22-00:00

Re: [PATCH 4/9] network namespaces: socket hashes
type: item
description: Andrey Savochkin wrote: > The main reason is that socket hash tables should be large enough to work > efficiently, but it isn't good to waste a lot of memory for each namespace. > Namespaces should be cheap enough, to allow to have hundreds of them. > This reason of memory efficiency, of course, has a priority unless/until > socket hash tables start to resize automatically. > > Another point is that routing lookup is much more complicated than the > socket's one to add another search key. > Routing also have additional routines for deleting entries matching some > patterns, and so on. > In short, routing is much more complicated, and it already quite efficient > for various sizes of routing tables. That makes sense, thx for the explanation. Cheers. -- Daniel.
creator: Daniel Lezcano
date: 2006-09-21T12:34:53-00:00

Re: [PATCH 5/9] network namespaces: async socket operations
type: item
description: Andrey Savochkin wrote: > Non-trivial part of socket namespaces: asynchronous events > should be run in proper context. > > Signed-off-by: Andrey Savochkin <saw@swsoft.com> > --- > af_inet.c | 10 ++++++++++ > inet_timewait_sock.c | 8 ++++++++ > tcp_timer.c | 9 +++++++++ > 3 files changed, 27 insertions(+) > > --- ./net/ipv4/af_inet.c.venssock-asyn Mon Aug 14 17:04:07 2006 > +++ ./net/ipv4/af_inet.c Tue Aug 15 13:45:44 2006 > @@ -366,10 +366,17 @@ out_rcu_unlock: > int inet_release(struct socket *sock) > { > struct sock *sk = sock->sk; > + struct net_namespace *ns, *orig_net_ns; > > if (sk) { > long timeout; > > + /* Need to change context here since protocol ->close > + * operation may send packets. > + */ > + ns = get_net_ns(sk->sk_net_ns); > + push_net_ns(ns, orig_net_ns); > + Is it not a race condition here ? What happens if you have a packet incoming during the namespace context switching ? IHMO doing namespace switching is something dangerous, you can probably handle that with locks but it will be difficult and will decrease all network performance. In an other hand, I don't see how you can handle the "sk->sk_prot->close" after ... -- Cheers
creator: Daniel Lezcano
date: 2006-09-22T15:33:56-00:00

Re: [PATCH 5/9] network namespaces: async socket operations
type: item
description: On Fri, Sep 22, 2006 at 05:33:56PM +0200, Daniel Lezcano wrote: > Andrey Savochkin wrote: > > Non-trivial part of socket namespaces: asynchronous events > > should be run in proper context. > > > > Signed-off-by: Andrey Savochkin <saw@swsoft.com> > > --- > > af_inet.c | 10 ++++++++++ > > inet_timewait_sock.c | 8 ++++++++ > > tcp_timer.c | 9 +++++++++ > > 3 files changed, 27 insertions(+) > > > > --- ./net/ipv4/af_inet.c.venssock-asyn Mon Aug 14 17:04:07 2006 > > +++ ./net/ipv4/af_inet.c Tue Aug 15 13:45:44 2006 > > @@ -366,10 +366,17 @@ out_rcu_unlock: > > int inet_release(struct socket *sock) > > { > > struct sock *sk = sock->sk; > > + struct net_namespace *ns, *orig_net_ns; > > > > if (sk) { > > long timeout; > > > > + /* Need to change context here since protocol ->close > > + * operation may send packets. > > + */ > > + ns = get_net_ns(sk->sk_net_ns); > > + push_net_ns(ns, orig_net_ns); > > + > > Is it not a race condition here ? What happens if you have a packet > incoming during the namespace context switching ? All asynchronous operations (RX softirq, timers) should set their context explicitly, and can't rely on the current context being the right one (or a valid pointer at all). Andrey
creator: Andrey Savochkin
date: 2006-09-23T13:16:43-00:00

Re: [RFC] network namespaces
type: item
description: Andrey Savochkin wrote: > Hi All, > > I'd like to resurrect our discussion about network namespaces. > In our previous discussions it appeared that we have rather polar concepts > which seemed hard to reconcile. > Now I have an idea how to look at all discussed concepts to enable everyone's > usage scenario. Hi Andrey, I have a few questions ... sorry for asking so late ;) > > 1. The most straightforward concept is complete separation of namespaces, > covering device list, routing tables, netfilter tables, socket hashes, and > everything else. > > On input path, each packet is tagged with namespace right from the > place where it appears from a device, and is processed by each layer > in the context of this namespace. If you have the namespace where is coming the packet, why do you tag the packet instead of switching to the right namespace ? > Non-root namespaces communicate with the outside world in two ways: by > owning hardware devices, or receiving packets forwarded them by their parent > namespace via pass-through device. Do you will do proxy arp and ip forwarding into the root namespace in order to make non-root namespace visible from the outside world ? Regards. -- Daniel
creator: Daniel Lezcano
date: 2006-10-04T09:40:32-00:00