OpenVZ Forum: Devel » [PATCH] BC: resource beancounters (v2)

Home » Mailing lists » Devel » [PATCH] BC: resource beancounters (v2)

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: [PATCH] BC: resource beancounters (v2) [message #5654 is a reply to message #5570]

Fri, 25 August 2006 11:47

dev
Messages: 1693
Registered: September 2005
Location: Moscow

Senior Member

Andrew Morton wrote:
>>As the first step we want to propose for discussion
>>the most complicated parts of resource management:
>>kernel memory and virtual memory.
>
>
> The patches look reasonable to me - mergeable after updating them for
> today's batch of review commentlets.
sure. will do updates as long as there are reasonable comments.

> I have two high-level problems though.
>
> a) I don't yet have a sense of whether this implementation
> is appropriate/sufficient for the various other
> applications which people are working on.
>
> If the general shape is OK and we think this
> implementation can be grown into one which everyone can
> use then fine.
>
> And...
>
>
>>The patch set to be sent provides core for BC and
>>management of kernel memory only. Virtual memory
>>management will be sent in a couple of days.
>
>
> We need to go over this work before we can commit to the BC
> core. Last time I looked at the VM accounting patch it
> seemed rather unpleasing from a maintainability POV.
hmmm... in which regard?

> And, if I understand it correctly, the only response to a job
> going over its VM limits is to kill it, rather than trimming
> it. Which sounds like a big problem?
No, UBC virtual memory management refuses occur on mmap()'s.
Andrey Savochkin wrote already a brief summary on vm resource management:

------------- cut ----------------
The task of limiting a container to 4.5GB of memory bottles down to the
question: what to do when the container starts to use more than assigned
4.5GB of memory?

At this moment there are only 3 viable alternatives.

A) Have separate memory management for each container,
with separate buddy allocator, lru lists, page replacement mechanism.
That implies a considerable overhead, and the main challenge there
is sharing of pages between these separate memory managers.

B) Return errors on extension of mappings, but not on page faults, where
memory is actually consumed.
In this case it makes sense to take into account not only the size of used
memory, but the size of created mappings as well.
This is approximately what "privvmpages" accounting/limiting provides in
UBC.

C) Rely on OOM killer.
This is a fall-back method in UBC, for the case "privvmpages" limits
still leave the possibility to overload the system.

It would be nice, indeed, to invent something new.
The ideal mechanism would
- slow down the container over-using memory, to signal the user that
he is over his limits,
- at the same time this slowdown shouldn't lead to the increase of memory
usage: for example, a simple slowdown of apache web server would lead
to the growth of the number of serving children and consumption of more
memory while showing worse performance,
- and, at the same time, it shouldn't penalize the rest of the system from
the performance point of view...
May be this can be achieved via carefully tuned swapout mechanism together
with disk bandwidth management capable of tracking asynchronous write
requests, may be something else is required.
It's really a big challenge.

Meanwhile, I guess we can only make small steps in improving Linux resource
management features for this moment.
------------- cut ----------------

Thanks,
Kirill

Report message to a moderator

[Message index]

		[PATCH] BC: resource beancounters (v2) By: dev on Wed, 23 August 2006 10:44
		[PATCH 1/6] BC: kconfig By: dev on Wed, 23 August 2006 10:59
		Re: [PATCH 1/6] BC: kconfig By: Alexey Dobriyan on Wed, 23 August 2006 13:04
		Re: [PATCH 1/6] BC: kconfig By: Dave Hansen on Wed, 23 August 2006 22:04
		Re: [PATCH 1/6] BC: kconfig By: Matt Helsley on Wed, 23 August 2006 22:13
		Re: [PATCH 1/6] BC: kconfig By: dev on Fri, 25 August 2006 11:27
		Re: [PATCH 1/6] BC: kconfig By: dev on Fri, 25 August 2006 11:31
		Re: [PATCH 1/6] BC: kconfig By: rdunlap on Wed, 23 August 2006 22:29
		Re: [PATCH 1/6] BC: kconfig By: Matt Helsley on Wed, 23 August 2006 22:27
		Re: [PATCH 1/6] BC: kconfig By: Chandra Seetharaman on Thu, 24 August 2006 00:23
		Re: [ckrm-tech] [PATCH 1/6] BC: kconfig By: dev on Thu, 24 August 2006 11:47
		Re: [ckrm-tech] [PATCH 1/6] BC: kconfig By: Matt Helsley on Thu, 24 August 2006 22:23
		[PATCH 2/6] BC: beancounters core (API) By: dev on Wed, 23 August 2006 11:00
		Re: [PATCH 2/6] BC: beancounters core (API) By: Andi Kleen on Wed, 23 August 2006 11:37
		Re: [PATCH 2/6] BC: beancounters core (API) By: dev on Wed, 23 August 2006 13:25
		Re: [PATCH 2/6] BC: beancounters core (API) By: Andi Kleen on Wed, 23 August 2006 13:48
		Re: [PATCH 2/6] BC: beancounters core (API) By: dev on Wed, 23 August 2006 13:46
		Re: [PATCH 2/6] BC: beancounters core (API) By: Alexey Dobriyan on Wed, 23 August 2006 13:53
		Re: [PATCH 2/6] BC: beancounters core (API) By: Matt Helsley on Wed, 23 August 2006 22:05
		Re: [PATCH 2/6] BC: beancounters core (API) By: Alexey Dobriyan on Wed, 23 August 2006 13:30
		Re: [PATCH 2/6] BC: beancounters core (API) By: Andrew Morton on Wed, 23 August 2006 16:42
		Re: [PATCH 2/6] BC: beancounters core (API) By: Oleg Nesterov on Thu, 24 August 2006 11:09
		Re: [PATCH 2/6] BC: beancounters core (API) By: dev on Thu, 24 August 2006 12:03
		Re: [PATCH 2/6] BC: beancounters core (API) By: Andrew Morton on Thu, 24 August 2006 15:00
		Re: [PATCH 2/6] BC: beancounters core (API) By: dev on Fri, 25 August 2006 10:51
		Re: [PATCH 2/6] BC: beancounters core (API) By: Oleg Nesterov on Thu, 24 August 2006 17:09
		[PATCH 3/6] BC: context inheriting and changing By: dev on Wed, 23 August 2006 11:02
		[PATCH 4/6] BC: user interface (syscalls) By: dev on Wed, 23 August 2006 11:03
		Re: [PATCH 4/6] BC: user interface (syscalls) By: dev on Wed, 23 August 2006 13:40
		Re: [PATCH 4/6] BC: user interface (syscalls) By: Andrew Morton on Wed, 23 August 2006 16:50
		Re: [PATCH 4/6] BC: user interface (syscalls) By: Alan Cox on Wed, 23 August 2006 17:08
		Re: [PATCH 4/6] BC: user interface (syscalls) By: Andrew Morton on Thu, 24 August 2006 04:35
		Re: [PATCH 4/6] BC: user interface (syscalls) By: Alan Cox on Thu, 24 August 2006 10:42
		Re: [PATCH 4/6] BC: user interface (syscalls) By: Alexey Dobriyan on Thu, 24 August 2006 13:08
		Re: [PATCH 4/6] BC: user interface (syscalls) By: dev on Fri, 25 August 2006 10:54
		Re: [PATCH 4/6] BC: user interface (syscalls) By: Alexey Dobriyan on Wed, 23 August 2006 13:41
		Re: [PATCH 4/6] BC: user interface (syscalls) By: Chandra Seetharaman on Thu, 24 August 2006 00:30
		[PATCH 5/6] BC: kernel memory accounting (core) By: dev on Wed, 23 August 2006 11:04
		Re: [PATCH 5/6] BC: kernel memory accounting (core) By: Chandra Seetharaman on Thu, 24 August 2006 00:36
		Re: [PATCH 5/6] BC: kernel memory accounting (core) By: Oleg Nesterov on Thu, 24 August 2006 16:59
		Re: [PATCH 5/6] BC: kernel memory accounting (core) By: dev on Fri, 25 August 2006 10:06
		[PATCH 6/6] BC: kernel memory accounting (marks) By: dev on Wed, 23 August 2006 11:05
		Re: [PATCH 6/6] BC: kernel memory accounting (marks) By: Dave Hansen on Wed, 23 August 2006 18:30
		Re: [PATCH 6/6] BC: kernel memory accounting (marks) By: dev on Tue, 29 August 2006 09:52
		Re: [PATCH 6/6] BC: kernel memory accounting (marks) By: Dave Hansen on Tue, 29 August 2006 15:48
		Re: [PATCH 6/6] BC: kernel memory accounting (marks) By: Dave Hansen on Wed, 23 August 2006 23:03
		Re: [PATCH 6/6] BC: kernel memory accounting (marks) By: Geert Uytterhoeven on Thu, 24 August 2006 09:30
		Re: [PATCH 6/6] BC: kernel memory accounting (marks) By: Dave Hansen on Thu, 24 August 2006 15:52
		Re: [PATCH 6/6] BC: kernel memory accounting (marks) By: dev on Tue, 29 August 2006 14:34
		Re: [PATCH 6/6] BC: kernel memory accounting (marks) By: dev on Tue, 29 August 2006 15:53
		Re: [PATCH] BC: resource beancounters (v2) By: Andrew Morton on Wed, 23 August 2006 17:05
		Re: [PATCH] BC: resource beancounters (v2) By: Chandra Seetharaman on Thu, 24 August 2006 00:17
		Re: [PATCH] BC: resource beancounters (v2) By: dev on Fri, 25 August 2006 11:47
		Re: [PATCH] BC: resource beancounters (v2) By: Andrew Morton on Fri, 25 August 2006 14:30
		Re: [PATCH] BC: resource beancounters (v2) By: Andi Kleen on Fri, 25 August 2006 14:48
		Re: [PATCH] BC: resource beancounters (v2) By: dev on Mon, 28 August 2006 08:27
		Re: [PATCH] BC: resource beancounters (v2) By: Nick Piggin on Fri, 25 August 2006 15:14
		Re: BC: resource beancounters (v2) By: Andrey Savochkin on Fri, 25 August 2006 16:30
		Re: BC: resource beancounters (v2) By: Andrew Morton on Fri, 25 August 2006 17:50
		Re: BC: resource beancounters (v2) By: Chandra Seetharaman on Fri, 25 August 2006 19:00
		Re: BC: resource beancounters (v2) By: Rohit Seth on Sat, 26 August 2006 02:15
		Re: BC: resource beancounters (v2) By: Alan Cox on Sat, 26 August 2006 16:16
		Re: BC: resource beancounters (v2) By: Rohit Seth on Mon, 28 August 2006 16:48
		Re: Re: BC: resource beancounters (v2) By: kir on Mon, 28 August 2006 17:40
		Re: Re: BC: resource beancounters (v2) By: Rohit Seth on Mon, 28 August 2006 22:28
		Re: Re: BC: resource beancounters (v2) By: Alan Cox on Tue, 29 August 2006 10:15
		Re: Re: BC: resource beancounters (v2) By: Rohit Seth on Tue, 29 August 2006 17:30
		Re: Re: BC: resource beancounters (v2) By: Alan Cox on Tue, 29 August 2006 18:46
		Re: Re: BC: resource beancounters (v2) By: Rohit Seth on Tue, 29 August 2006 19:15
		Re: [PATCH] BC: resource beancounters (v2) By: dev on Tue, 29 August 2006 15:33
		Re: [PATCH] BC: resource beancounters (v2) By: Balbir Singh on Tue, 29 August 2006 17:08
		Re: [PATCH] BC: resource beancounters (v2) By: Cedric Le Goater on Wed, 23 August 2006 21:00
		Re: [PATCH] BC: resource beancounters (v2) By: Jan Engelhardt on Thu, 24 August 2006 05:52
		Re: [PATCH] BC: resource beancounters (v2) By: Alan Cox on Thu, 24 August 2006 10:38
		Re: [PATCH] BC: resource beancounters (v2) By: Alan Cox on Fri, 25 August 2006 15:36
		Re: [PATCH] BC: resource beancounters (v2) By: Nick Piggin on Sat, 26 August 2006 03:55

Previous Topic:	Re: pspace child_reaper
Next Topic:	Re: pspace child_reaper

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Jul 05 08:59:22 GMT 2025

Total time taken to generate the page: 0.02510 seconds