OpenVZ Forum - RDF feed
type: channel
items: [PATCH v5 0/8] per-cgroup tcp buffer pressure settings
items: [PATCH v5 2/8] socket: initial cgroup code.
items: [PATCH v5 3/8] foundations of per-cgroup memory pressure controlling.
items: [PATCH v5 4/8] per-cgroup tcp buffers control
items: [PATCH v5 5/8] per-netns ipv4 sysctl_tcp_mem
items: [PATCH v5 6/8] tcp buffer limitation: per-cgroup limit
items: [PATCH v5 7/8] Display current tcp memory allocation in kmem cgroup
items: [PATCH v5 8/8] Disable task moving when using kernel memory accounting
items: Re: [PATCH v5 6/8] tcp buffer limitation: per-cgroup limit
items: Re: [PATCH v5 0/8] per-cgroup tcp buffer pressure settings

[PATCH v5 0/8] per-cgroup tcp buffer pressure settings
type: item
description: v3: merge Kirill's suggestions, + a destroy-related bugfix v4: Fix a bug with non-mounted cgroups + disallow task movement v5: Compile bug with modular ipv6 + tcp files in bytes Kame, Kirill, I am submitting this again merging most of your comments. I've decided to leave some of them out: * I am not using res_counters for allocated_memory. Besides being more expensive than what we need, to make it work in a nice way, we'd have to change the !cgroup code, including other protocols than tcp. Also, * I am not using failcnt and max_usage_in_bytes for it. I believe the value of those lies more in the allocation than in the pressure control. Besides, fail conditions lie mostly outside of the memory cgroup's control. (Actually, a soft_limit makes a lot of sense, and I do plan to introduce it in a follow up series) If you agree with the above, and there are any other pressing issues, let me know and I will address them ASAP. Otherwise, let's discuss it. I'm always open. All: This patch introduces per-cgroup tcp buffers limitation. This allows sysadmins to specify a maximum amount of kernel memory that tcp connections can use at any point in time. TCP is the main interest in this work, but extending it to other protocols would be easy. For this to work, I am hooking it into memcg, after the introdution of an extension for tracking and controlling objects in kernel memory. Since they are usually not found in page granularity, and are fundamentally different from userspace memory (not swappable, can't overcommit), they need their special place inside the Memory Controller. Right now, the kmem extension is quite basic, and just lays down the basic infrastucture for the ongoing work. Although it does not account kernel memory allocated - I preferred to keep this series simple and leave accounting to the slab allocations when they arrive. What it does is to piggyback in the memory control mechanism already present in /proc/sys/net/ipv4/tcp_mem. There is a soft l…
creator: Glauber Costa
date: 2011-10-04T12:17:52-00:00

[PATCH v5 2/8] socket: initial cgroup code.
type: item
description: We aim to control the amount of kernel memory pinned at any time by tcp sockets. To lay the foundations for this work, this patch adds a pointer to the kmem_cgroup to the socket structure. Signed-off-by: Glauber Costa <glommer@parallels.com> Acked-by: Kirill A. Shutemov<kirill@shutemov.name> Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujtsu.com> CC: David S. Miller <davem@davemloft.net> CC: Eric W. Biederman <ebiederm@xmission.com> --- include/linux/memcontrol.h | 15 +++++++++++++++ include/net/sock.h | 2 ++ mm/memcontrol.c | 36 ++++++++++++++++++++++++++++++++++++ net/core/sock.c | 3 +++ 4 files changed, 56 insertions(+), 0 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 343bd76..88aea1b 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -376,5 +376,20 @@ mem_cgroup_print_bad_page(struct page *page) } #endif +#ifdef CONFIG_INET +struct sock; +#ifdef CONFIG_CGROUP_MEM_RES_CTLR_KMEM +void sock_update_memcg(struct sock *sk); +void sock_release_memcg(struct sock *sk); + +#else +static inline void sock_update_memcg(struct sock *sk) +{ +} +static inline void sock_release_memcg(struct sock *sk) +{ +} +#endif /* CONFIG_CGROUP_MEM_RES_CTLR_KMEM */ +#endif /* CONFIG_INET */ #endif /* _LINUX_MEMCONTROL_H */ diff --git a/include/net/sock.h b/include/net/sock.h index 8e4062f..afe1467 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -228,6 +228,7 @@ struct sock_common { * @sk_security: used by security modules * @sk_mark: generic packet mark * @sk_classid: this socket's cgroup classid + * @sk_cgrp: this socket's kernel memory (kmem) cgroup * @sk_write_pending: a write to stream socket waits to start * @sk_state_change: callback to indicate change in the state of the sock * @sk_data_ready: callback to indicate there is data to be processed @@ -339,6 +340,7 @@ struct sock { #endif __u32 sk_mark; u32 sk_classid; + struct mem_cgroup *sk_cgrp; void (*sk_state_change)(struct sock *sk); void (*sk_data_r…
creator: Glauber Costa
date: 2011-10-04T12:17:54-00:00

[PATCH v5 3/8] foundations of per-cgroup memory pressure controlling.
type: item
description: This patch converts struct sock fields memory_pressure, memory_allocated, sockets_allocated, and sysctl_mem (now prot_mem) to function pointers, receiving a struct mem_cgroup parameter. enter_memory_pressure is kept the same, since all its callers have socket a context, and the kmem_cgroup can be derived from the socket itself. To keep things working, the patch convert all users of those fields to use acessor functions. In my benchmarks I didn't see a significant performance difference with this patch applied compared to a baseline (around 1 % diff, thus inside error margin). Signed-off-by: Glauber Costa <glommer@parallels.com> CC: David S. Miller <davem@davemloft.net> CC: Hiroyouki Kamezawa <kamezawa.hiroyu@jp.fujitsu.com> CC: Eric W. Biederman <ebiederm@xmission.com> --- crypto/af_alg.c | 7 ++- include/linux/memcontrol.h | 29 +++++++++++- include/net/sock.h | 112 +++++++++++++++++++++++++++++++++++++++++-- include/net/tcp.h | 11 +++-- include/net/udp.h | 3 +- include/trace/events/sock.h | 10 ++-- mm/memcontrol.c | 45 +++++++++++++++++- net/core/sock.c | 62 ++++++++++++++---------- net/decnet/af_decnet.c | 21 +++++++- net/ipv4/proc.c | 7 ++- net/ipv4/tcp.c | 27 +++++++++- net/ipv4/tcp_input.c | 12 ++-- net/ipv4/tcp_ipv4.c | 12 ++-- net/ipv4/tcp_output.c | 2 +- net/ipv4/tcp_timer.c | 2 +- net/ipv4/udp.c | 20 ++++++-- net/ipv6/tcp_ipv6.c | 10 ++-- net/ipv6/udp.c | 4 +- net/sctp/socket.c | 35 ++++++++++--- 19 files changed, 345 insertions(+), 86 deletions(-) diff --git a/crypto/af_alg.c b/crypto/af_alg.c index ac33d5f..9f41324 100644 --- a/crypto/af_alg.c +++ b/crypto/af_alg.c @@ -29,10 +29,15 @@ struct alg_type_list { static atomic_long_t alg_memory_allocated; +static atomic_long_t *memory_allocated_alg(struct mem_cgroup *memcg) +{ + return &alg_memory_allocated; +} + static struct proto alg_proto = { .name = "ALG", .owner = THIS_MODULE, - .memory_allocated = &alg_memory_allocated, + .memory_allocated = memory_allocated_alg, .obj_size = sizeof(struct alg_sock), }; di…
creator: Glauber Costa
date: 2011-10-04T12:17:55-00:00

[PATCH v5 4/8] per-cgroup tcp buffers control
type: item
description: With all the infrastructure in place, this patch implements per-cgroup control for tcp memory pressure handling. Signed-off-by: Glauber Costa <glommer@parallels.com> Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujtisu.com> CC: David S. Miller <davem@davemloft.net> CC: Eric W. Biederman <ebiederm@xmission.com> --- include/linux/memcontrol.h | 4 ++ include/net/sock.h | 15 ++++++- include/net/tcp.h | 15 +++++++ mm/memcontrol.c | 97 ++++++++++++++++++++++++++++++++++++++++++++ net/core/sock.c | 39 ++++++++++++++++- net/ipv4/tcp.c | 44 +++++++++---------- net/ipv4/tcp_ipv4.c | 12 +++++- net/ipv6/tcp_ipv6.c | 10 ++++- 8 files changed, 207 insertions(+), 29 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 2012060..50af61d 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -391,6 +391,10 @@ void memcg_sock_mem_alloc(struct mem_cgroup *memcg, struct proto *prot, void memcg_sock_mem_free(struct mem_cgroup *memcg, struct proto *prot, int amt); void memcg_sockets_allocated_dec(struct mem_cgroup *memcg, struct proto *prot); void memcg_sockets_allocated_inc(struct mem_cgroup *memcg, struct proto *prot); +int tcp_init_cgroup(struct proto *prot, struct cgroup *cgrp, + struct cgroup_subsys *ss); +void tcp_destroy_cgroup(struct proto *prot, struct cgroup *cgrp, + struct cgroup_subsys *ss); #else /* memcontrol includes sockets.h, that includes memcontrol.h ... */ static inline void memcg_sock_mem_alloc(struct mem_cgroup *memcg, diff --git a/include/net/sock.h b/include/net/sock.h index c6983cf..0625d79 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -64,6 +64,8 @@ #include <net/dst.h> #include <net/checksum.h> +int sockets_populate(struct cgroup *cgrp, struct cgroup_subsys *ss); +void sockets_destroy(struct cgroup *cgrp, struct cgroup_subsys *ss); /* * This structure really needs to be cleaned up. * Most of it is for TCP, and not used by any of @@ -814,7 +816,18 @@ struct proto { int *(*memory_pressure)…
creator: Glauber Costa
date: 2011-10-04T12:17:56-00:00

[PATCH v5 5/8] per-netns ipv4 sysctl_tcp_mem
type: item
description: This patch allows each namespace to independently set up its levels for tcp memory pressure thresholds. This patch alone does not buy much: we need to make this values per group of process somehow. This is achieved in the patches that follows in this patchset. Signed-off-by: Glauber Costa <glommer@parallels.com> Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> CC: David S. Miller <davem@davemloft.net> CC: Eric W. Biederman <ebiederm@xmission.com> --- include/net/netns/ipv4.h | 1 + include/net/tcp.h | 1 - mm/memcontrol.c | 8 ++++-- net/ipv4/sysctl_net_ipv4.c | 51 +++++++++++++++++++++++++++++++++++++------ net/ipv4/tcp.c | 13 ++-------- 5 files changed, 53 insertions(+), 21 deletions(-) diff --git a/include/net/netns/ipv4.h b/include/net/netns/ipv4.h index d786b4f..bbd023a 100644 --- a/include/net/netns/ipv4.h +++ b/include/net/netns/ipv4.h @@ -55,6 +55,7 @@ struct netns_ipv4 { int current_rt_cache_rebuild_count; unsigned int sysctl_ping_group_range[2]; + long sysctl_tcp_mem[3]; atomic_t rt_genid; atomic_t dev_addr_genid; diff --git a/include/net/tcp.h b/include/net/tcp.h index 7ae7b4b..04fedf7 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -231,7 +231,6 @@ extern int sysctl_tcp_fack; extern int sysctl_tcp_reordering; extern int sysctl_tcp_ecn; extern int sysctl_tcp_dsack; -extern long sysctl_tcp_mem[3]; extern int sysctl_tcp_wmem[3]; extern int sysctl_tcp_rmem[3]; extern int sysctl_tcp_app_win; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 9598e3e..39e575e 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -309,6 +309,7 @@ static struct mem_cgroup *parent_mem_cgroup(struct mem_cgroup *memcg); #ifdef CONFIG_INET #include <net/sock.h> #include <net/ip.h> +#include <linux/nsproxy.h> void sock_update_memcg(struct sock *sk) { @@ -434,14 +435,15 @@ int tcp_init_cgroup(struct proto *prot, struct cgroup *cgrp, struct cgroup_subsys *ss) { struct mem_cgroup *cg = mem_cgroup_from_cont(cgrp); + struct net *net = current->nsproxy->net_ns; /* *…
creator: Glauber Costa
date: 2011-10-04T12:17:57-00:00

[PATCH v5 6/8] tcp buffer limitation: per-cgroup limit
type: item
description: This patch uses the "tcp_max_mem" field of the kmem_cgroup to effectively control the amount of kernel memory pinned by a cgroup. We have to make sure that none of the memory pressure thresholds specified in the namespace are bigger than the current cgroup. Signed-off-by: Glauber Costa <glommer@parallels.com> CC: David S. Miller <davem@davemloft.net> CC: Hiroyouki Kamezawa <kamezawa.hiroyu@jp.fujitsu.com> CC: Eric W. Biederman <ebiederm@xmission.com> --- Documentation/cgroups/memory.txt | 1 + include/linux/memcontrol.h | 10 +++++ include/net/tcp.h | 1 + mm/memcontrol.c | 80 +++++++++++++++++++++++++++++++++++--- net/ipv4/sysctl_net_ipv4.c | 20 +++++++++ 5 files changed, 106 insertions(+), 6 deletions(-) diff --git a/Documentation/cgroups/memory.txt b/Documentation/cgroups/memory.txt index bf00cd2..c1db134 100644 --- a/Documentation/cgroups/memory.txt +++ b/Documentation/cgroups/memory.txt @@ -78,6 +78,7 @@ Brief summary of control files. memory.independent_kmem_limit # select whether or not kernel memory limits are independent of user limits + memory.kmem.tcp.limit_in_bytes # set/show hard limit for tcp buf memory 1. History diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 50af61d..6191ba1 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -395,6 +395,9 @@ int tcp_init_cgroup(struct proto *prot, struct cgroup *cgrp, struct cgroup_subsys *ss); void tcp_destroy_cgroup(struct proto *prot, struct cgroup *cgrp, struct cgroup_subsys *ss); + +unsigned long tcp_max_memory(struct mem_cgroup *cg); +void tcp_prot_mem(struct mem_cgroup *cg, long val, int idx); #else /* memcontrol includes sockets.h, that includes memcontrol.h ... */ static inline void memcg_sock_mem_alloc(struct mem_cgroup *memcg, @@ -420,6 +423,13 @@ static inline void sock_update_memcg(struct sock *sk) static inline void sock_release_memcg(struct sock *sk) { } +static inline unsigned long tcp_max_memory(struct mem_cgroup *cg) +{ + return 0; +} +static inline …
creator: Glauber Costa
date: 2011-10-04T12:17:58-00:00

[PATCH v5 7/8] Display current tcp memory allocation in kmem cgroup
type: item
description: This patch introduces kmem.tcp_current_memory file, living in the kmem_cgroup filesystem. It is a simple read-only file that displays the amount of kernel memory currently consumed by the cgroup. Signed-off-by: Glauber Costa <glommer@parallels.com> CC: David S. Miller <davem@davemloft.net> CC: Hiroyouki Kamezawa <kamezawa.hiroyu@jp.fujitsu.com> CC: Eric W. Biederman <ebiederm@xmission.com> --- Documentation/cgroups/memory.txt | 1 + mm/memcontrol.c | 11 +++++++++++ 2 files changed, 12 insertions(+), 0 deletions(-) diff --git a/Documentation/cgroups/memory.txt b/Documentation/cgroups/memory.txt index c1db134..00f1a88 100644 --- a/Documentation/cgroups/memory.txt +++ b/Documentation/cgroups/memory.txt @@ -79,6 +79,7 @@ Brief summary of control files. memory.independent_kmem_limit # select whether or not kernel memory limits are independent of user limits memory.kmem.tcp.limit_in_bytes # set/show hard limit for tcp buf memory + memory.kmem.tcp.usage_in_bytes # show current tcp buf memory allocation 1. History diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 6fb14bb..f178a64 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -465,12 +465,23 @@ static u64 tcp_read_limit(struct cgroup *cgrp, struct cftype *cft) return memcg->tcp.tcp_max_memory << PAGE_SHIFT; } +static u64 tcp_usage_in_bytes(struct cgroup *cgrp, struct cftype *cft) +{ + struct mem_cgroup *memcg = mem_cgroup_from_cont(cgrp); + + return atomic_long_read(&memcg->tcp.tcp_memory_allocated) << PAGE_SHIFT; +} + static struct cftype tcp_files[] = { { .name = "kmem.tcp.limit_in_bytes", .write_u64 = tcp_write_limit, .read_u64 = tcp_read_limit, }, + { + .name = "kmem.tcp.usage_in_bytes", + .read_u64 = tcp_usage_in_bytes, + }, }; static void tcp_create_cgroup(struct mem_cgroup *cg, struct cgroup_subsys *ss) -- 1.7.6
creator: Glauber Costa
date: 2011-10-04T12:17:59-00:00

[PATCH v5 8/8] Disable task moving when using kernel memory accounting
type: item
description: Since this code is still experimental, we are leaving the exact details of how to move tasks between cgroups when kernel memory accounting is used as future work. For now, we simply disallow movement if there are any pending accounted memory. Signed-off-by: Glauber Costa <glommer@parallels.com> CC: Hiroyouki Kamezawa <kamezawa.hiroyu@jp.fujitsu.com> --- include/net/tcp.h | 1 + mm/memcontrol.c | 37 ++++++++++++++++++++++--------------- 2 files changed, 23 insertions(+), 15 deletions(-) diff --git a/include/net/tcp.h b/include/net/tcp.h index 716a42e..62ad282 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -257,6 +257,7 @@ struct mem_cgroup; struct tcp_memcontrol { /* per-cgroup tcp memory pressure knobs */ int tcp_max_memory; + atomic_t refcnt; atomic_long_t tcp_memory_allocated; struct percpu_counter tcp_sockets_allocated; /* those two are read-mostly, leave them at the end */ diff --git a/mm/memcontrol.c b/mm/memcontrol.c index f178a64..db1f511 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -328,28 +328,17 @@ void sock_update_memcg(struct sock *sk) rcu_read_lock(); sk->sk_cgrp = mem_cgroup_from_task(current); - - /* - * We don't need to protect against anything task-related, because - * we are basically stuck with the sock pointer that won't change, - * even if the task that originated the socket changes cgroups. - * - * What we do have to guarantee, is that the chain leading us to - * the top level won't change under our noses. Incrementing the - * reference count via cgroup_exclude_rmdir guarantees that. - */ - cgroup_exclude_rmdir(mem_cgroup_css(sk->sk_cgrp)); rcu_read_unlock(); } void sock_release_memcg(struct sock *sk) { - cgroup_release_and_wakeup_rmdir(mem_cgroup_css(sk->sk_cgrp)); } void memcg_sock_mem_alloc(struct mem_cgroup *memcg, struct proto *prot, int amt, int *parent_failure) { + atomic_inc(&memcg->tcp.refcnt); memcg = parent_mem_cgroup(memcg); for (; memcg != NULL; memcg = parent_mem_cgroup(memcg)) { long alloc; @@ -368,9 +357,12 @@…
creator: Glauber Costa
date: 2011-10-04T12:18:00-00:00

Re: [PATCH v5 6/8] tcp buffer limitation: per-cgroup limit
type: item
description: Le mardi 04 octobre 2011 à 16:17 +0400, Glauber Costa a écrit : > This patch uses the "tcp_max_mem" field of the kmem_cgroup to > effectively control the amount of kernel memory pinned by a cgroup. > > We have to make sure that none of the memory pressure thresholds > specified in the namespace are bigger than the current cgroup. > > Signed-off-by: Glauber Costa <glommer@parallels.com> > CC: David S. Miller <davem@davemloft.net> > CC: Hiroyouki Kamezawa <kamezawa.hiroyu@jp.fujitsu.com> > CC: Eric W. Biederman <ebiederm@xmission.com> > --- > --- a/include/net/tcp.h > +++ b/include/net/tcp.h > @@ -256,6 +256,7 @@ extern int sysctl_tcp_thin_dupack; > struct mem_cgroup; > struct tcp_memcontrol { > /* per-cgroup tcp memory pressure knobs */ > + int tcp_max_memory; > atomic_long_t tcp_memory_allocated; > struct percpu_counter tcp_sockets_allocated; > /* those two are read-mostly, leave them at the end */ > diff --git a/mm/memcontrol.c b/mm/memcontrol.c So tcp_max_memory is an "int". > +static u64 tcp_read_limit(struct cgroup *cgrp, struct cftype *cft) > +{ > + struct mem_cgroup *memcg = mem_cgroup_from_cont(cgrp); > + return memcg->tcp.tcp_max_memory << PAGE_SHIFT; > +} 1) Typical integer overflow here. You need : return ((u64)memcg->tcp.tcp_max_memory) << PAGE_SHIFT; 2) Could you add const qualifiers when possible to your pointers ?
creator: Eric Dumazet
date: 2011-10-04T12:48:55-00:00

Re: [PATCH v5 0/8] per-cgroup tcp buffer pressure settings
type: item
description: On Tue, 4 Oct 2011 16:17:52 +0400 Glauber Costa <glommer@parallels.com> wrote: > v3: merge Kirill's suggestions, + a destroy-related bugfix > v4: Fix a bug with non-mounted cgroups + disallow task movement > v5: Compile bug with modular ipv6 + tcp files in bytes > > Kame, Kirill, > > I am submitting this again merging most of your comments. I've decided to > leave some of them out: > * I am not using res_counters for allocated_memory. Besides being more > expensive than what we need, to make it work in a nice way, we'd have > to change the !cgroup code, including other protocols than tcp. Also, > > * I am not using failcnt and max_usage_in_bytes for it. I believe the value > of those lies more in the allocation than in the pressure control. Besides, > fail conditions lie mostly outside of the memory cgroup's control. (Actually, > a soft_limit makes a lot of sense, and I do plan to introduce it in a follow > up series) > > If you agree with the above, and there are any other pressing issues, let me > know and I will address them ASAP. Otherwise, let's discuss it. I'm always open. > I'm not familar with reuqirements of users. So, I appreciate your choices. What I adivse you here is taking a deep breath. Making new version every day is not good for reviewing process ;) (It's now -rc8 and merge will not be so quick, anyway.) At this stage, my concern is view of interfaces and documenation, and future plans. Let me give a try explanation by myself. (Correct me ;) I added some questions but I'm sorry you've already answered. New interfaces are 5 files. All files exists only for non-root memory cgroup. 1. memory.independent_kmem_limit 2. memory.kmem.usage_in_bytes 3. memory.kmem.limit_in_bytes 4. memory.kmem.tcp.limit_in_bytes 5. memory.kmem.tcp.usage_in_bytes * memory.independent_kmem_limit If 1, kmem_limit_in_bytes/kmem_usage_in_bytes works. If 0, kmem_limit_in_bytes/kmem_usage_in_bytes doesn't work and all kmem usages are controlled under memory.limit_…
creator: KAMEZAWA Hiroyuki
date: 2011-10-05T00:29:54-00:00

Re: [PATCH v5 0/8] per-cgroup tcp buffer pressure settings
type: item
description: On 10/05/2011 04:29 AM, KAMEZAWA Hiroyuki wrote: > On Tue, 4 Oct 2011 16:17:52 +0400 > Glauber Costa<glommer@parallels.com> wrote: > >> v3: merge Kirill's suggestions, + a destroy-related bugfix >> v4: Fix a bug with non-mounted cgroups + disallow task movement >> v5: Compile bug with modular ipv6 + tcp files in bytes >> >> Kame, Kirill, >> >> I am submitting this again merging most of your comments. I've decided to >> leave some of them out: >> * I am not using res_counters for allocated_memory. Besides being more >> expensive than what we need, to make it work in a nice way, we'd have >> to change the !cgroup code, including other protocols than tcp. Also, >> >> * I am not using failcnt and max_usage_in_bytes for it. I believe the value >> of those lies more in the allocation than in the pressure control. Besides, >> fail conditions lie mostly outside of the memory cgroup's control. (Actually, >> a soft_limit makes a lot of sense, and I do plan to introduce it in a follow >> up series) >> >> If you agree with the above, and there are any other pressing issues, let me >> know and I will address them ASAP. Otherwise, let's discuss it. I'm always open. >> > > I'm not familar with reuqirements of users. So, I appreciate your choices. > What I adivse you here is taking a deep breath. Making new version every day > is not good for reviewing process ;) > (It's now -rc8 and merge will not be so quick, anyway.) Kame, Absolutely. I only did it this time because the difference between them were really, really small. > At this stage, my concern is view of interfaces and documenation, and future plans. Okay. I will try to address them as well as I can. > Let me give a try explanation by myself. (Correct me ;) > I added some questions but I'm sorry you've already answered. > > New interfaces are 5 files. All files exists only for non-root memory cgroup. > > 1. memory.independent_kmem_limit > 2. memory.kmem.usage_in_bytes > 3. memory.kmem.limit_in_bytes > 4. me…
creator: Glauber Costa
date: 2011-10-05T07:25:50-00:00

Re: [PATCH v5 6/8] tcp buffer limitation: per-cgroup limit
type: item
description: On 10/04/2011 04:48 PM, Eric Dumazet wrote: > Le mardi 04 octobre 2011 à 16:17 +0400, Glauber Costa a écrit : >> This patch uses the "tcp_max_mem" field of the kmem_cgroup to >> effectively control the amount of kernel memory pinned by a cgroup. >> >> We have to make sure that none of the memory pressure thresholds >> specified in the namespace are bigger than the current cgroup. >> >> Signed-off-by: Glauber Costa<glommer@parallels.com> >> CC: David S. Miller<davem@davemloft.net> >> CC: Hiroyouki Kamezawa<kamezawa.hiroyu@jp.fujitsu.com> >> CC: Eric W. Biederman<ebiederm@xmission.com> >> --- > > >> --- a/include/net/tcp.h >> +++ b/include/net/tcp.h >> @@ -256,6 +256,7 @@ extern int sysctl_tcp_thin_dupack; >> struct mem_cgroup; >> struct tcp_memcontrol { >> /* per-cgroup tcp memory pressure knobs */ >> + int tcp_max_memory; >> atomic_long_t tcp_memory_allocated; >> struct percpu_counter tcp_sockets_allocated; >> /* those two are read-mostly, leave them at the end */ >> diff --git a/mm/memcontrol.c b/mm/memcontrol.c > > So tcp_max_memory is an "int". > > >> +static u64 tcp_read_limit(struct cgroup *cgrp, struct cftype *cft) >> +{ >> + struct mem_cgroup *memcg = mem_cgroup_from_cont(cgrp); >> + return memcg->tcp.tcp_max_memory<< PAGE_SHIFT; >> +} > > 1) Typical integer overflow here. > > You need : > > return ((u64)memcg->tcp.tcp_max_memory)<< PAGE_SHIFT; Thanks for spotting this, Eric. > > 2) Could you add const qualifiers when possible to your pointers ? Well, I'll go over the patches again and see where I can add them. Any specific place site you're concerned about?
creator: Glauber Costa
date: 2011-10-05T08:08:04-00:00

Re: [PATCH v5 6/8] tcp buffer limitation: per-cgroup limit
type: item
description: Le mercredi 05 octobre 2011 à 12:08 +0400, Glauber Costa a écrit : > On 10/04/2011 04:48 PM, Eric Dumazet wrote: > > 2) Could you add const qualifiers when possible to your pointers ? > > Well, I'll go over the patches again and see where I can add them. > Any specific place site you're concerned about? Everywhere its possible : It helps reader to instantly knows if a function is about to change some part of the object or only read it, without reading function body.
creator: Eric Dumazet
date: 2011-10-05T08:58:10-00:00

Re: [PATCH v5 6/8] tcp buffer limitation: per-cgroup limit
type: item
description: On 10/05/2011 12:58 PM, Eric Dumazet wrote: > Le mercredi 05 octobre 2011 à 12:08 +0400, Glauber Costa a écrit : >> On 10/04/2011 04:48 PM, Eric Dumazet wrote: > >>> 2) Could you add const qualifiers when possible to your pointers ? >> >> Well, I'll go over the patches again and see where I can add them. >> Any specific place site you're concerned about? > > Everywhere its possible : > > It helps reader to instantly knows if a function is about to change some > part of the object or only read it, without reading function body. Sure it does. So, give me your opinion on this: most of the acessors inside struct sock do not modify the pointers, but return an address of an element inside it (that can later on be modified by the caller. I think it is fine for the purpose of clarity, but to avoid warnings we end up having to do stuff like this: +#define CONSTCG(m) ((struct mem_cgroup *)(m)) +long *tcp_sysctl_mem(const struct mem_cgroup *memcg) +{ + return CONSTCG(memcg)->tcp.tcp_prot_mem; +} Is it acceptable?
creator: Glauber Costa
date: 2011-10-06T08:38:59-00:00

Re: [PATCH v5 0/8] per-cgroup tcp buffer pressure settings
type: item
description: Sorry for lazy answer. On Wed, 5 Oct 2011 11:25:50 +0400 Glauber Costa <glommer@parallels.com> wrote: > On 10/05/2011 04:29 AM, KAMEZAWA Hiroyuki wrote: > > On Tue, 4 Oct 2011 16:17:52 +0400 > > Glauber Costa<glommer@parallels.com> wrote: > > > > At this stage, my concern is view of interfaces and documenation, and future plans. > > Okay. I will try to address them as well as I can. > > > * memory.independent_kmem_limit > > If 1, kmem_limit_in_bytes/kmem_usage_in_bytes works. > > If 0, kmem_limit_in_bytes/kmem_usage_in_bytes doesn't work and all kmem > > usages are controlled under memory.limit_in_bytes. > > Correct. For the questions below, I won't even look at the code not to > get misguided. Let's settle on the desired behavior, and everything that > deviates from it, is a bug. > > > Question: > > - What happens when parent/chidlren cgroup has different indepedent_kmem_limit ? > I think it should be forbidden. It was raised by Kirill before, and > IIRC, he specifically requested it to be. (Okay: Saying it now, makes me > realizes that the child can have set it to 1 while parent was 1. But > then parent sets it to 0... I don't think I am handling this case). > ok, please put it into TODO list ;) > > In future plan, kmem.usage_in_bytes should includes tcp.kmem_usage_in_bytes. > > And kmem.limit_in_bytes should be the limiation of sum of all kmem.xxxx.limit_in_bytes. > > I am not sure there will be others xxx.limit_in_bytes. (see below) > ok. > > > > Question: > > - Why this integration is difficult ? > It is not that it is difficult. > What happens is that there are two things taking place here: > One of them is allocation. > The other, is tcp-specific pressure thresholds. Bear with me with the > following example code: (from sk_stream_alloc_skb, net/ipv4/tcp.c) > > 1: skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp); > if (skb) { > 3: if (sk_wmem_schedule(sk, skb->truesize)) { > /* > * Make sure that we have exactly size bytes > * available to the calle…
creator: KAMEZAWA Hiroyuki
date: 2011-10-07T08:05:22-00:00

Re: [PATCH v5 0/8] per-cgroup tcp buffer pressure settings
type: item
description: On 10/07/2011 12:05 PM, KAMEZAWA Hiroyuki wrote: > > > Sorry for lazy answer. Hi Kame, Now matter how hard you try, you'll never be as lazy as I am. So that's okay. > > On Wed, 5 Oct 2011 11:25:50 +0400 > Glauber Costa<glommer@parallels.com> wrote: > >> On 10/05/2011 04:29 AM, KAMEZAWA Hiroyuki wrote: >>> On Tue, 4 Oct 2011 16:17:52 +0400 >>> Glauber Costa<glommer@parallels.com> wrote: >>> > >>> At this stage, my concern is view of interfaces and documenation, and future plans. >> >> Okay. I will try to address them as well as I can. >> >>> * memory.independent_kmem_limit >>> If 1, kmem_limit_in_bytes/kmem_usage_in_bytes works. >>> If 0, kmem_limit_in_bytes/kmem_usage_in_bytes doesn't work and all kmem >>> usages are controlled under memory.limit_in_bytes. >> >> Correct. For the questions below, I won't even look at the code not to >> get misguided. Let's settle on the desired behavior, and everything that >> deviates from it, is a bug. >> >>> Question: >>> - What happens when parent/chidlren cgroup has different indepedent_kmem_limit ? >> I think it should be forbidden. It was raised by Kirill before, and >> IIRC, he specifically requested it to be. (Okay: Saying it now, makes me >> realizes that the child can have set it to 1 while parent was 1. But >> then parent sets it to 0... I don't think I am handling this case). >> > > ok, please put it into TODO list ;) Done. > > >>> In future plan, kmem.usage_in_bytes should includes tcp.kmem_usage_in_bytes. >>> And kmem.limit_in_bytes should be the limiation of sum of all kmem.xxxx.limit_in_bytes. >> >> I am not sure there will be others xxx.limit_in_bytes. (see below) >> > > ok. > > >>> >>> Question: >>> - Why this integration is difficult ? >> It is not that it is difficult. >> What happens is that there are two things taking place here: >> One of them is allocation. >> The other, is tcp-specific pressure thresholds. Bear with me with the >> following example code: (from sk_stream_alloc_skb, net/ipv4/tcp.c) >> >> 1: sk…
creator: Glauber Costa
date: 2011-10-07T08:20:04-00:00

Re: [PATCH v5 0/8] per-cgroup tcp buffer pressure settings
type: item
description: On Fri, 7 Oct 2011 12:20:04 +0400 Glauber Costa <glommer@parallels.com> wrote: > > > >> So what I really mean here with "will integrate later", is that I think > >> that we'd be better off tracking the allocations themselves at the slab > >> level. > >> > >>> Can't tcp-limit-code borrows some amount of charges in batch from kmem_limit > >>> and use it ? > >> Sorry, I don't know what exactly do you mean. Can you clarify? > >> > > Now, tcp-usage is independent from kmem-usage. > > > > My idea is > > > > 1. when you account tcp usage, charge kmem, too. > > Absolutely. > > Now, your work is > > a) tcp use new xxxx bytes. > > b) account it to tcp.uage and check tcp limit > > > > To ingegrate kmem, > > a) tcp use new xxxx bytes. > > b) account it to tcp.usage and check tcp limit > > c) account it to kmem.usage > > > > ? 2 counters may be slow ? > > Well, the way I see it, 1 counter is slow already =) > I honestly think we need some optimizations here. But > that is a side issue. > > To begin with: The new patchset that I intend to spin > today or Monday, depending on my progress, uses res_counters, > as you and Kirill requested. > > So what makes res_counters slow IMHO, is two things: > > 1) interrupts are always disabled. > 2) All is done under a lock. > > Now, we are starting to have resources that are billed to multiple > counters. One simple way to work around it, is to have child counters > that has to be accounted for as well everytime a resource is counted. > > Like this: > > 1) tcp has kmem as child. When we bill to tcp, we bill to kmem as well. > For protocols that do memory pressure, we then don't bill kmem from > the slab. > 2) When kmem_independent_account is set to 0, kmem has mem as child. > Seems reasonable. > > > > > >>> - Don't you need a stat file to indicate "tcp memory pressure works!" ? > >>> It can be obtained already ? > >> > >> Not 100 % clear as well. We can query the amount of buffer used, and the > >> amount of buffer allowed. What else do we ne…
creator: KAMEZAWA Hiroyuki
date: 2011-10-07T08:55:00-00:00