Home » Mailing lists » Devel » [RFC][PATCH] UBC: user resource beancounters
|
|
| Re: [ckrm-tech] [PATCH 4/7] UBC: syscalls (user interface) [message #5530 is a reply to message #5504] |
Tue, 22 August 2006 18:34   |
Chandra Seetharaman
Messages: 88
Registered: August 2006
|
Member |
|
|
On Tue, 2006-08-22 at 12:58 +0900, Magnus Damm wrote:
> On Mon, 2006-08-21 at 18:16 -0700, Rohit Seth wrote:
> > On Mon, 2006-08-21 at 11:47 +0900, Magnus Damm wrote:
> > > On Fri, 2006-08-18 at 07:45 -0700, Dave Hansen wrote:
> > > > On Fri, 2006-08-18 at 12:08 +0400, Andrey Savochkin wrote:
> > > > >
> > > > > A) Have separate memory management for each container,
> > > > > with separate buddy allocator, lru lists, page replacement mechanism.
> > > > > That implies a considerable overhead, and the main challenge there
> > > > > is sharing of pages between these separate memory managers.
> > > >
> > > > Hold on here for just a sec...
> > > >
> > > > It is quite possible to do memory management aimed at one container
> > > > while that container's memory still participates in the main VM.
> > > >
> > > > There is overhead here, as the LRU scanning mechanisms get less
> > > > efficient, but I'd rather pay a penalty at LRU scanning time than divide
> > > > up the VM, or coarsely start failing allocations.
> > >
> > > This could of course be solved with one LRU per container, which is how
> > > the CKRM memory controller implemented things about a year ago.
> >
> > Effectively Andrew's idea of faking up nodes is also giving per
> > container LRUs.
>
> Yes, but the NUMA emulation approach is using fixed size containers
> where the size is selectable at the kernel command line, while the CKRM
> (and pzone) approach provides a more dynamic (and complex) solution.
NUMA emulation does not allow guarantee, only limits. It also doesn't
allow over commit (ove commit issue is present in pzone based approach
also).
>
> / magnus
>
>
> ------------------------------------------------------------ -------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&b id=263057&dat=121642
> _______________________________________________
> ckrm-tech mailing list
> https://lists.sourceforge.net/lists/listinfo/ckrm-tech
--
------------------------------------------------------------ ----------
Chandra Seetharaman | Be careful what you choose....
- sekharan@us.ibm.com | .......you may get it.
------------------------------------------------------------ ----------
|
|
|
|
| Re: [ckrm-tech] [RFC][PATCH] UBC: user resource beancounters [message #5531 is a reply to message #5502] |
Tue, 22 August 2006 18:55 |
Chandra Seetharaman
Messages: 88
Registered: August 2006
|
Member |
|
|
On Mon, 2006-08-21 at 18:45 -0700, Rohit Seth wrote:
> On Mon, 2006-08-21 at 14:45 -0700, Chandra Seetharaman wrote:
> > On Mon, 2006-08-21 at 17:24 +0400, Kirill Korotaev wrote:
> > > Chandra Seetharaman wrote:
> > > > Kirill,
> > > >
> > > > Here are some concerns I have (as of now) w.r.t using UBC for resource
> > > > management (in the context of resource groups).
> > > >
> > > > - guarantee support is missing. I do not see any code to provide the
> > > > minimum amount of resource a group can get. It is important for
> > > > providing QoS. (In a different email you did mention guarantee, i am
> > > > referring it here for completeness).
> > > I mentioned a couple of times that this is a limited core functionality
> > > in this patch set.
> > > guarantees are implementable as a separate UBC parameters.
> >
> > I will wait for oomguarpages patches :)
> >
> > >
> > > > - Creation of a UBC and assignment of task to a UBC always happen in
> > > > the context of the task that is affected. I can understand it works in
> > > > OpenVZ environment, but IMO has issues if one wants it to be used for
> > > > basic resource management
> > > > - application needs to be changed to use this feature.
> > > > - System administrator does not have the control to assign tasks to a
> > > > UBC. Application does by itself.
> > > > - Assignment of task to a UBC need to be transparent to the
> > > > application.
>
> I agree with the above points. Just want to add that assignment of a
> task to a container may not be transparent to the application. For
> example it may hit some limits and some reclaim may happen...
By transparent I meant that the task _need_ not have to know that there
is a resource manager sitting and managing its resources. Task will
still see the effects of resource crunch etc., (but it will handle the
situation the same way as it would handle today).
So, it is transparent. A task don't have to know that the reclamation is
happening due to its affiliation to a resource group. Task will be
handling it as if there is a pressure for that particular resource.
>
> > > this is not 100% true.
> > > UBC itself doesn't prevent from changing context on the fly.
> > > But since this leads to part of resources to be charged to
> > > one UBC and another part to another UBC and so long and so
> >
> > Let the controllers and the users worry about that part.
> >
>
> I think as the tasks move around, it becomes very heavy to move all the
> pages belonging to previous container to a new container.
Not for all resources, CPU could handle it very nicely, whereas memory
would be heavy. My point is that the infrastructure should be open, and
controller is the one that decides whether it wants to handle it or not.
>
> > As I mentioned UBC might be perfect for container resource management,
> > but what I am talking for is resource management _without_ a container.
> >
>
> Can you explain that part a bit more?
Basically I was saying that even though resource management in container
and non-container have mostly same requirements, there are few
requirements that are critical in non-container scenario which are non-
issue in container scenario (for example, moving tasks from one resource
group to another).
In effect, Design of the infrastructure should not limit non-container
usages.
IMO, non-container requirements are a superset of container requirements
(resource management purposes only :).
>
> > >
> > > > - No ability to maintain resource specific data in the controller.
> > > it's false. fields can be added to user_beancounter struct easily.
> > > and that's what our controllers do.
> >
> > With the model of static array for resources (struct ubparm ub_parms
> > [UB_RESOURCES] in struct user_beancounter), it is not be possible to
> > attach _different_ "controller specific" information to each of the
> > entries.
> >
> > I do not think it is good idea to add controller specific information of
> > _different_ controllers to the user_beancounter. Think of all the fields
> > it will have when all the numproc controller needs is just the basic 3-4
> > fields.
> >
>
> IMO it is okay to add the fields whenever necessary as Kirill
> suggested. I think once the container subject gets baked a little more,
> the controllers will also get tightly coupled.
I think my point is not understood. I do not think it is right to add
_controller specific_ fields to the generic data structure (struct
user_beancounter), as we will end up with a generic data structure which
will have so many fields that are not used in so many controllers.
>
> > >
> > > > - No ability to get the list of tasks belonging to a UBC.
> > > it is not true. it can be read from /proc or system calls interface,
> > > just like the way one finds all tasks belonging to one user :)
> > >
> > > BTW, what is so valueable in this feature?
> >
> > Again, it may not be useful for container type usages (you can probably
> > get the list from somewhere else, but for resource management it is
> > useful for sysadmins).
> >
>
> I'm also debating about whether printing task information is really any
> useful. If a sysadm wants to get information about any particular task
> then that can come from /proc/<pid>/container Though container list
> will be one place where one can easily get the list of all the contained
> tasks (and other resources like files).
In non-container environment, there is _no_ /proc/pid/container, as
there is no concept of container :). This will be useful for non-
container scenario.
<snip>
--
------------------------------------------------------------ ----------
Chandra Seetharaman | Be careful what you choose....
- sekharan@us.ibm.com | .......you may get it.
------------------------------------------------------------ ----------
|
|
|
|
| Re: [ckrm-tech] [PATCH 4/7] UBC: syscalls (user interface) [message #5590 is a reply to message #5504] |
Thu, 24 August 2006 01:20 |
Rohit Seth
Messages: 101
Registered: August 2006
|
Senior Member |
|
|
On Tue, 2006-08-22 at 12:58 +0900, Magnus Damm wrote:
> On Mon, 2006-08-21 at 18:16 -0700, Rohit Seth wrote:
> > On Mon, 2006-08-21 at 11:47 +0900, Magnus Damm wrote:
> > > On Fri, 2006-08-18 at 07:45 -0700, Dave Hansen wrote:
> > > > On Fri, 2006-08-18 at 12:08 +0400, Andrey Savochkin wrote:
> > > > >
> > > > > A) Have separate memory management for each container,
> > > > > with separate buddy allocator, lru lists, page replacement mechanism.
> > > > > That implies a considerable overhead, and the main challenge there
> > > > > is sharing of pages between these separate memory managers.
> > > >
> > > > Hold on here for just a sec...
> > > >
> > > > It is quite possible to do memory management aimed at one container
> > > > while that container's memory still participates in the main VM.
> > > >
> > > > There is overhead here, as the LRU scanning mechanisms get less
> > > > efficient, but I'd rather pay a penalty at LRU scanning time than divide
> > > > up the VM, or coarsely start failing allocations.
> > >
> > > This could of course be solved with one LRU per container, which is how
> > > the CKRM memory controller implemented things about a year ago.
> >
> > Effectively Andrew's idea of faking up nodes is also giving per
> > container LRUs.
>
> Yes, but the NUMA emulation approach is using fixed size containers
> where the size is selectable at the kernel command line,
[Apologies for late reply..]
Yup, if we run with fake NUMA support for providing container
functionality then dynamic resizing will be important (and that is why I
made the initial comment of possibly using memory hot-plug)
> while the CKRM
> (and pzone) approach provides a more dynamic (and complex) solution.
...this complexity is not always a positive thing ;-) (I do like core
of CKRM stuff FWIW).
-rohit
|
|
|
|
|
|
| Re: [ckrm-tech] [RFC][PATCH] UBC: user resource beancounters [message #5592 is a reply to message #5531] |
Thu, 24 August 2006 01:44 |
Rohit Seth
Messages: 101
Registered: August 2006
|
Senior Member |
|
|
On Tue, 2006-08-22 at 11:55 -0700, Chandra Seetharaman wrote:
> On Mon, 2006-08-21 at 18:45 -0700, Rohit Seth wrote:
> >
> > > > this is not 100% true.
> > > > UBC itself doesn't prevent from changing context on the fly.
> > > > But since this leads to part of resources to be charged to
> > > > one UBC and another part to another UBC and so long and so
> > >
> > > Let the controllers and the users worry about that part.
> > >
> >
> > I think as the tasks move around, it becomes very heavy to move all the
> > pages belonging to previous container to a new container.
>
> Not for all resources, CPU could handle it very nicely, whereas memory
> would be heavy. My point is that the infrastructure should be open, and
> controller is the one that decides whether it wants to handle it or not.
With open you are implying being able to use different ones. It would
be nice to get one in and make sure it is stable and optimized...
>
> >
> > > As I mentioned UBC might be perfect for container resource management,
> > > but what I am talking for is resource management _without_ a container.
> > >
> >
> > Can you explain that part a bit more?
>
> Basically I was saying that even though resource management in container
> and non-container have mostly same requirements, there are few
> requirements that are critical in non-container scenario which are non-
> issue in container scenario (for example, moving tasks from one resource
> group to another).
>
> In effect, Design of the infrastructure should not limit non-container
> usages.
>
> IMO, non-container requirements are a superset of container requirements
> (resource management purposes only :).
>
hmm, non-container world (and its resource management part) already
exist. And sure those requirements are superset of this discussion.
And hopefully container support will not break/modify that much.
> >
> > > >
> > > > > - No ability to maintain resource specific data in the controller.
> > > > it's false. fields can be added to user_beancounter struct easily.
> > > > and that's what our controllers do.
> > >
> > > With the model of static array for resources (struct ubparm ub_parms
> > > [UB_RESOURCES] in struct user_beancounter), it is not be possible to
> > > attach _different_ "controller specific" information to each of the
> > > entries.
> > >
> > > I do not think it is good idea to add controller specific information of
> > > _different_ controllers to the user_beancounter. Think of all the fields
> > > it will have when all the numproc controller needs is just the basic 3-4
> > > fields.
> > >
> >
> > IMO it is okay to add the fields whenever necessary as Kirill
> > suggested. I think once the container subject gets baked a little more,
> > the controllers will also get tightly coupled.
>
> I think my point is not understood. I do not think it is right to add
> _controller specific_ fields to the generic data structure (struct
> user_beancounter), as we will end up with a generic data structure which
> will have so many fields that are not used in so many controllers.
>
A single centralized structure that has fields that are mostly used by
every one should be okay I think.
> >
> > > >
> > > > > - No ability to get the list of tasks belonging to a UBC.
> > > > it is not true. it can be read from /proc or system calls interface,
> > > > just like the way one finds all tasks belonging to one user :)
> > > >
> > > > BTW, what is so valueable in this feature?
> > >
> > > Again, it may not be useful for container type usages (you can probably
> > > get the list from somewhere else, but for resource management it is
> > > useful for sysadmins).
> > >
> >
> > I'm also debating about whether printing task information is really any
> > useful. If a sysadm wants to get information about any particular task
> > then that can come from /proc/<pid>/container Though container list
> > will be one place where one can easily get the list of all the contained
> > tasks (and other resources like files).
>
> In non-container environment, there is _no_ /proc/pid/container, as
> there is no concept of container :). This will be useful for non-
> container scenario.
>
I'm sure when container support gets in then for the above scenario it
will read -1 ...
-rohit
|
|
|
|
| Re: [ckrm-tech] [RFC][PATCH] UBC: user resource beancounters [message #5593 is a reply to message #5592] |
Thu, 24 August 2006 02:04 |
Chandra Seetharaman
Messages: 88
Registered: August 2006
|
Member |
|
|
On Wed, 2006-08-23 at 18:44 -0700, Rohit Seth wrote:
> On Tue, 2006-08-22 at 11:55 -0700, Chandra Seetharaman wrote:
> > On Mon, 2006-08-21 at 18:45 -0700, Rohit Seth wrote:
> > >
> > > > > this is not 100% true.
> > > > > UBC itself doesn't prevent from changing context on the fly.
> > > > > But since this leads to part of resources to be charged to
> > > > > one UBC and another part to another UBC and so long and so
> > > >
> > > > Let the controllers and the users worry about that part.
> > > >
> > >
> > > I think as the tasks move around, it becomes very heavy to move all the
> > > pages belonging to previous container to a new container.
> >
> > Not for all resources, CPU could handle it very nicely, whereas memory
> > would be heavy. My point is that the infrastructure should be open, and
> > controller is the one that decides whether it wants to handle it or not.
>
> With open you are implying being able to use different ones. It would
> be nice to get one in and make sure it is stable and optimized...
No, what I mean is that the infrastructure should allow the task moving
from one group to another, it should also notify the controller about
that movement and let the controller decide if it wants to take any
action. (instead of not having the capability stating that it is not
useful for all type of controllers).
>
> >
> > >
> > > > As I mentioned UBC might be perfect for container resource management,
> > > > but what I am talking for is resource management _without_ a container.
> > > >
> > >
> > > Can you explain that part a bit more?
> >
> > Basically I was saying that even though resource management in container
> > and non-container have mostly same requirements, there are few
> > requirements that are critical in non-container scenario which are non-
> > issue in container scenario (for example, moving tasks from one resource
> > group to another).
> >
> > In effect, Design of the infrastructure should not limit non-container
> > usages.
> >
> > IMO, non-container requirements are a superset of container requirements
> > (resource management purposes only :).
> >
>
> hmm, non-container world (and its resource management part) already
> exist. And sure those requirements are superset of this discussion.
What do you mean by "resource management part for non-container world
already exist ?
It does not. CKRM/Resource Groups is trying to do that, but is not in
Linus's tree.
> And hopefully container support will not break/modify that much.
>
> > >
> > > > >
> > > > > > - No ability to maintain resource specific data in the controller.
> > > > > it's false. fields can be added to user_beancounter struct easily.
> > > > > and that's what our controllers do.
> > > >
> > > > With the model of static array for resources (struct ubparm ub_parms
> > > > [UB_RESOURCES] in struct user_beancounter), it is not be possible to
> > > > attach _different_ "controller specific" information to each of the
> > > > entries.
> > > >
> > > > I do not think it is good idea to add controller specific information of
> > > > _different_ controllers to the user_beancounter. Think of all the fields
> > > > it will have when all the numproc controller needs is just the basic 3-4
> > > > fields.
> > > >
> > >
> > > IMO it is okay to add the fields whenever necessary as Kirill
> > > suggested. I think once the container subject gets baked a little more,
> > > the controllers will also get tightly coupled.
> >
> > I think my point is not understood. I do not think it is right to add
> > _controller specific_ fields to the generic data structure (struct
> > user_beancounter), as we will end up with a generic data structure which
> > will have so many fields that are not used in so many controllers.
> >
>
> A single centralized structure that has fields that are mostly used by
> every one should be okay I think.
You mean to say definition like
struct user_beancounter {
fields;/* fields that exists now */
int kmemsize_ctlr_info1;
char *kmemsize_ctlr_info2;
char *oomguar_ctlr_info1;
char *oomguar_ctlr_info2;
/* and so on */
}
is the right thing to do ? even though oomguar controller doesn't care
about kmemsize_ctlr_info* etc.,
>
> > >
> > > > >
> > > > > > - No ability to get the list of tasks belonging to a UBC.
> > > > > it is not true. it can be read from /proc or system calls interface,
> > > > > just like the way one finds all tasks belonging to one user :)
> > > > >
> > > > > BTW, what is so valueable in this feature?
> > > >
> > > > Again, it may not be useful for container type usages (you can probably
> > > > get the list from somewhere else, but for resource management it is
> > > > useful for sysadmins).
> > > >
> > >
> > > I'm also debating about whether printing task information is really any
> > > useful. If a sysadm wants to get information about any particular task
> > > then that can come from /proc/<pid>/container Though container list
> > > will be one place where one can easily get the list of all the contained
> > > tasks (and other resources like files).
> >
> > In non-container environment, there is _no_ /proc/pid/container, as
> > there is no concept of container :). This will be useful for non-
> > container scenario.
> >
>
> I'm sure when container support gets in then for the above scenario it
> will read -1 ...
So, how can one get the list of tasks belonging to a resource group in
that case ?
>
> -rohit
>
>
> ------------------------------------------------------------ -------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&b id=263057&dat=121642
> _______________________________________________
> ckrm-tech mailing list
> https://lists.sourceforge.net/lists/listinfo/ckrm-tech
--
------------------------------------------------------------ ----------
Chandra Seetharaman | Be careful what you choose....
- sekharan@us.ibm.com | .......you may get it.
------------------------------------------------------------ ----------
|
|
|
|
|
|
| Re: [ckrm-tech] [RFC][PATCH] UBC: user resource beancounters [message #5627 is a reply to message #5593] |
Thu, 24 August 2006 17:27 |
Rohit Seth
Messages: 101
Registered: August 2006
|
Senior Member |
|
|
On Wed, 2006-08-23 at 19:04 -0700, Chandra Seetharaman wrote:
> On Wed, 2006-08-23 at 18:44 -0700, Rohit Seth wrote:
> No, what I mean is that the infrastructure should allow the task moving
> from one group to another, it should also notify the controller about
> that movement and let the controller decide if it wants to take any
> action. (instead of not having the capability stating that it is not
> useful for all type of controllers).
>
> >
Okay.
> > >
> > > >
> > > > > As I mentioned UBC might be perfect for container resource management,
> > > > > but what I am talking for is resource management _without_ a container.
> > > > >
> > > >
> > > > Can you explain that part a bit more?
> > >
> > > Basically I was saying that even though resource management in container
> > > and non-container have mostly same requirements, there are few
> > > requirements that are critical in non-container scenario which are non-
> > > issue in container scenario (for example, moving tasks from one resource
> > > group to another).
> > >
> > > In effect, Design of the infrastructure should not limit non-container
> > > usages.
> > >
> > > IMO, non-container requirements are a superset of container requirements
> > > (resource management purposes only :).
> > >
> >
> > hmm, non-container world (and its resource management part) already
> > exist. And sure those requirements are superset of this discussion.
>
> What do you mean by "resource management part for non-container world
> already exist ?
>
> It does not. CKRM/Resource Groups is trying to do that, but is not in
> Linus's tree.
>
Please, non-container is the environment that exist today in Linux.
Actually cpuset does provide some part of it. But beyond that no.
But then we are all using different terminology like beancounters,
containers, resource groups and now non-containers...
> > And hopefully container support will not break/modify that much.
> >
> > > >
> > > > > >
> > > > > > > - No ability to maintain resource specific data in the controller.
> > > > > > it's false. fields can be added to user_beancounter struct easily.
> > > > > > and that's what our controllers do.
> > > > >
> > > > > With the model of static array for resources (struct ubparm ub_parms
> > > > > [UB_RESOURCES] in struct user_beancounter), it is not be possible to
> > > > > attach _different_ "controller specific" information to each of the
> > > > > entries.
> > > > >
> > > > > I do not think it is good idea to add controller specific information of
> > > > > _different_ controllers to the user_beancounter. Think of all the fields
> > > > > it will have when all the numproc controller needs is just the basic 3-4
> > > > > fields.
> > > > >
> > > >
> > > > IMO it is okay to add the fields whenever necessary as Kirill
> > > > suggested. I think once the container subject gets baked a little more,
> > > > the controllers will also get tightly coupled.
> > >
> > > I think my point is not understood. I do not think it is right to add
> > > _controller specific_ fields to the generic data structure (struct
> > > user_beancounter), as we will end up with a generic data structure which
> > > will have so many fields that are not used in so many controllers.
> > >
> >
> > A single centralized structure that has fields that are mostly used by
> > every one should be okay I think.
>
> You mean to say definition like
>
> struct user_beancounter {
> fields;/* fields that exists now */
>
> int kmemsize_ctlr_info1;
> char *kmemsize_ctlr_info2;
>
> char *oomguar_ctlr_info1;
> char *oomguar_ctlr_info2;
>
> /* and so on */
> }
>
> is the right thing to do ? even though oomguar controller doesn't care
> about kmemsize_ctlr_info* etc.,
>
No. I think it is appropriate to add all the accounting related fields
and object fields in the core container definition. Controllers only
make decisions based on the information contained in container. And it
should maintain its own data structures if needed(like what Alan said in
one of the later mails).
> >
> > > >
> > > > > >
> > > > > > > - No ability to get the list of tasks belonging to a UBC.
> > > > > > it is not true. it can be read from /proc or system calls interface,
> > > > > > just like the way one finds all tasks belonging to one user :)
> > > > > >
> > > > > > BTW, what is so valueable in this feature?
> > > > >
> > > > > Again, it may not be useful for container type usages (you can probably
> > > > > get the list from somewhere else, but for resource management it is
> > > > > useful for sysadmins).
> > > > >
> > > >
> > > > I'm also debating about whether printing task information is really any
> > > > useful. If a sysadm wants to get information about any particular task
> > > > then that can come from /proc/<pid>/container Though container list
> > > > will be one place where one can easily get the list of all the contained
> > > > tasks (and other resources like files).
> > >
> > > In non-container environment, there is _no_ /proc/pid/container, as
> > > there is no concept of container :). This will be useful for non-
> > > container scenario.
> > >
> >
> > I'm sure when container support gets in then for the above scenario it
> > will read -1 ...
>
> So, how can one get the list of tasks belonging to a resource group in
> that case ?
> >
...and that brings to the starting question...why do you need it?
Commands like ps and top will show appropriate container number for each
task.
-rohit
|
|
|
|
| Re: [ckrm-tech] [RFC][PATCH] UBC: user resource beancounters [message #5638 is a reply to message #5627] |
Thu, 24 August 2006 23:52 |
Chandra Seetharaman
Messages: 88
Registered: August 2006
|
Member |
|
|
On Thu, 2006-08-24 at 10:27 -0700, Rohit Seth wrote:
<snip>
> > What do you mean by "resource management part for non-container world
> > already exist ?
> >
> > It does not. CKRM/Resource Groups is trying to do that, but is not in
> > Linus's tree.
> >
>
> Please, non-container is the environment that exist today in Linux.
> Actually cpuset does provide some part of it. But beyond that no.
cpuset provides resource _isolation_, not necessarily resource
management.
>
> But then we are all using different terminology like beancounters,
> containers, resource groups and now non-containers...
>
<snip>
> > > I'm sure when container support gets in then for the above scenario it
> > > will read -1 ...
> >
> > So, how can one get the list of tasks belonging to a resource group in
> > that case ?
> > >
>
> ...and that brings to the starting question...why do you need it?
Like I said earlier, there is _no_ other way to get the list of tasks
belonging to a resource group.
> Commands like ps and top will show appropriate container number for each
> task.
There is _no_ container number in the non-container environment (or it
will be same for _all_ tasks).
>
> -rohit
>
--
------------------------------------------------------------ ----------
Chandra Seetharaman | Be careful what you choose....
- sekharan@us.ibm.com | .......you may get it.
------------------------------------------------------------ ----------
|
|
|
|
|
|
| Re: [ckrm-tech] [RFC][PATCH] UBC: user resource beancounters [message #5640 is a reply to message #5639] |
Thu, 24 August 2006 23:55 |
Kyle Moffett
Messages: 4
Registered: February 2006
|
Junior Member |
|
|
On Aug 24, 2006, at 19:48:28, Chandra Seetharaman wrote:
> On Thu, 2006-08-24 at 12:10 +0100, Alan Cox wrote:
>> All you need is
>>
>> struct wombat_controller
>> {
>> struct user_beancounter counter;
>> void (*wombat_pest_control)(struct wombat *w);
>> atomic_t wombat_population;
>> int (*wombat_destructor)(struct wombat *w);
>> };
>
> This may not solve the problem, as
> - we won't be able get the controller data structure given the
> beancounter data structure.
Of course you can! This is what we do for linked lists too. Here's
an example of how to get a pointer to your wombat_controller given
the user_beancounter pointer:
struct wombat_controller *wombat = containerof
(ptr_to_user_beancounter, struct wombat_controller, counter);
The containerof(PTR, TYPE, MEMBER) returns a pointer to the parent
object of type "TYPE" whose member "MEMBER" has address "PTR".
Cheers,
Kyle Moffett
|
|
|
|
| Re: [ckrm-tech] [RFC][PATCH] UBC: user resource beancounters [message #5651 is a reply to message #5638] |
Fri, 25 August 2006 11:10 |
dev
Messages: 1693
Registered: September 2005
Location: Moscow
|
Senior Member |
|
|
Chandra Seetharaman wrote:
> On Thu, 2006-08-24 at 10:27 -0700, Rohit Seth wrote:
>
> <snip>
>
>>>What do you mean by "resource management part for non-container world
>>>already exist ?
>>>
>>>It does not. CKRM/Resource Groups is trying to do that, but is not in
>>>Linus's tree.
>>>
>>
>>Please, non-container is the environment that exist today in Linux.
>>Actually cpuset does provide some part of it. But beyond that no.
>
>
> cpuset provides resource _isolation_, not necessarily resource
> management.
>
>
>>But then we are all using different terminology like beancounters,
>>containers, resource groups and now non-containers...
>>
>
>
> <snip>
>
>>>>I'm sure when container support gets in then for the above scenario it
>>>>will read -1 ...
>>>
>>>So, how can one get the list of tasks belonging to a resource group in
>>>that case ?
>>>
>>...and that brings to the starting question...why do you need it?
>
>
> Like I said earlier, there is _no_ other way to get the list of tasks
> belonging to a resource group.
>
>
>>Commands like ps and top will show appropriate container number for each
>>task.
>
>
> There is _no_ container number in the non-container environment (or it
> will be same for _all_ tasks).
Chandra, virtual container number is essentially the same as user id
in non-container environment. UBC were desgined for _users_ first.
Containers were just the first environment which started to use it widely.
And I really disagree when you say that non-container usecase is
a superset of container usecase. I believe it is vice versa, since
in container usecase you have a _full_ environment with root user and need
more resources to be taken into account.
Thanks,
Kirill
|
|
|
|
| Re: [ckrm-tech] [RFC][PATCH] UBC: user resource beancounters [message #5664 is a reply to message #5640] |
Fri, 25 August 2006 18:21 |
Chandra Seetharaman
Messages: 88
Registered: August 2006
|
Member |
|
|
On Thu, 2006-08-24 at 19:55 -0400, Kyle Moffett wrote:
> On Aug 24, 2006, at 19:48:28, Chandra Seetharaman wrote:
> > On Thu, 2006-08-24 at 12:10 +0100, Alan Cox wrote:
> >> All you need is
> >>
> >> struct wombat_controller
> >> {
> >> struct user_beancounter counter;
> >> void (*wombat_pest_control)(struct wombat *w);
> >> atomic_t wombat_population;
> >> int (*wombat_destructor)(struct wombat *w);
> >> };
> >
> > This may not solve the problem, as
> > - we won't be able get the controller data structure given the
> > beancounter data structure.
>
> Of course you can! This is what we do for linked lists too. Here's
> an example of how to get a pointer to your wombat_controller given
> the user_beancounter pointer:
> struct wombat_controller *wombat = containerof
> (ptr_to_user_beancounter, struct wombat_controller, counter);
>
> The containerof(PTR, TYPE, MEMBER) returns a pointer to the parent
> object of type "TYPE" whose member "MEMBER" has address "PTR".
Yes, it would work nicely.
But, the problem is that the struct user_beancounter (part of
wombat_controller above) is a _copy_ of the original, not the original
itself. We cannot keep the original (in _each_ controller), as there may
be more than one controller in the system and user_beancounter structure
is created/owned/destroyed by the beancounter infrastructure and not the
controller.
> Cheers,
> Kyle Moffett
>
>
>
>
> ------------------------------------------------------------ -------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&b id=263057&dat=121642
> _______________________________________________
> ckrm-tech mailing list
> https://lists.sourceforge.net/lists/listinfo/ckrm-tech
--
------------------------------------------------------------ ----------
Chandra Seetharaman | Be careful what you choose....
- sekharan@us.ibm.com | .......you may get it.
------------------------------------------------------------ ----------
|
|
|
|
| Re: [ckrm-tech] [RFC][PATCH] UBC: user resource beancounters [message #5665 is a reply to message #5651] |
Fri, 25 August 2006 18:47 |
Chandra Seetharaman
Messages: 88
Registered: August 2006
|
Member |
|
|
On Fri, 2006-08-25 at 15:12 +0400, Kirill Korotaev wrote:
<snip>
> >
> >
> > Like I said earlier, there is _no_ other way to get the list of tasks
> > belonging to a resource group.
> >
> >
> >>Commands like ps and top will show appropriate container number for each
> >>task.
> >
> >
> > There is _no_ container number in the non-container environment (or it
> > will be same for _all_ tasks).
>
> Chandra, virtual container number is essentially the same as user id
> in non-container environment. UBC were desgined for _users_ first.
> Containers were just the first environment which started to use it widely.
I am not denying any of the above :)
I think my original point is getting lost in the discussion, which is,
there should be way (for the sysadmin) to get a list of tasks belonging
to a resource group (in a non-container environment).
>
> And I really disagree when you say that non-container usecase is
> a superset of container usecase. I believe it is vice versa, since
I meant _only_ w.r.t resource management. My earlier replies were
pointing quite a few of those. here are a few:
- ability for the sysadmin to move a task to a resource group.
- assignment of task to a resource group should be transparent to the
app.
- a resource group could exist with no tasks associated.
Containers can work without these features (and as OpenVZ proves it does
work). But, for a QoS type of resource management framework these are
mandatory.
> in container usecase you have a _full_ environment with root user and need
> more resources to be taken into account.
Support for different resources is a different topic. Users (of the two
models) can decide to control as many (or as few) resources as they
want. What I am talking here is about the ability of the framework.
>
> Thanks,
> Kirill
>
>
> ------------------------------------------------------------ -------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&b id=263057&dat=121642
> _______________________________________________
> ckrm-tech mailing list
> https://lists.sourceforge.net/lists/listinfo/ckrm-tech
--
------------------------------------------------------------ ----------
Chandra Seetharaman | Be careful what you choose....
- sekharan@us.ibm.com | .......you may get it.
------------------------------------------------------------ ----------
|
|
|
|
|
|
|
|
| Re: [ckrm-tech] [RFC][PATCH] UBC: user resource beancounters [message #5669 is a reply to message #5667] |
Fri, 25 August 2006 21:37 |
Chandra Seetharaman
Messages: 88
Registered: August 2006
|
Member |
|
|
On Fri, 2006-08-25 at 21:46 +0100, Alan Cox wrote:
> Ar Gwe, 2006-08-25 am 11:21 -0700, ysgrifennodd Chandra Seetharaman:
> > But, the problem is that the struct user_beancounter (part of
> > wombat_controller above) is a _copy_ of the original, not the original
> > itself. We cannot keep the original (in _each_ controller), as there may
> > be more than one controller in the system
>
> Why would you want more than one controller for a given beancounter (and
> thus a single measured resource). Can you give an example ?
>
Hmm... from what I see, data structure user_beancounter is _not_ defined
for a single resource, it is a beancounter for _all_ resources.
struct user_beancounter
{
atomic_t ub_refcount;
spinlock_t ub_lock;
uid_t ub_uid;
struct hlist_node hash;
struct user_beancounter *parent;
void *private_data;
/* resources statistics and settings */
struct ubparm ub_parms[UB_RESOURCES];
};
ub_parms of _all_ controllers are held in this data structure.
So, keeping the beancounter data structure inside _a_ controller
specific data structure doesn't sound right to me, as other controllers
might also have the same need ?!
Controller _owns_ only ub_parms[controller_id], not the whole
user_beancounter, right ?
--
------------------------------------------------------------ ----------
Chandra Seetharaman | Be careful what you choose....
- sekharan@us.ibm.com | .......you may get it.
------------------------------------------------------------ ----------
|
|
|
|
| Re: [ckrm-tech] [RFC][PATCH] UBC: user resource beancounters [message #5671 is a reply to message #5668] |
Fri, 25 August 2006 22:23 |
Chandra Seetharaman
Messages: 88
Registered: August 2006
|
Member |
|
|
On Fri, 2006-08-25 at 21:52 +0100, Alan Cox wrote:
> Ar Gwe, 2006-08-25 am 11:47 -0700, ysgrifennodd Chandra Seetharaman:
> > I think my original point is getting lost in the discussion, which is,
> > there should be way (for the sysadmin) to get a list of tasks belonging
> > to a resource group (in a non-container environment).
>
> Ok that much is easy to deal with. You print the luid in /proc.
>
> > - ability for the sysadmin to move a task to a resource group.
>
> So you want a setpluid(pid, luid) ? Trivial to add although you might
> want to refuse it in many secure environments but thats an SELinux rule
> again.
yes.
>
> > - assignment of task to a resource group should be transparent to the
> > app.
>
> In those cases its akin to and matches security domain transitions which
> says to me SELinux (or AppArmour) should do it.
If setpluid(pid, luid) exists, then this is more easy to handle.
>
> > - a resource group could exist with no tasks associated.
>
> Bean counters can exist with no tasks, and the CKRM people have been
> corrected repeatedly on this point.
Hmm... from what I understand from the code, when the last resource in
the beancounter is dropped, the beancounter is destroyed. Which to me
means that when there are no tasks in a beancounter it will be
destroyed. (I just tested the code and verified that the beancounter is
destroyed when the task dies).
Please correct me if my understanding is incorrect.
Let me reword the requirement: beancounter/resource group should _not_
be destroyed implicitly. It should be destroyed only when requested by
the user/sysadmin. In other words, we need a create_luid() and
destroy_luid().
>
>
>
> ------------------------------------------------------------ -------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&b id=263057&dat=121642
> _______________________________________________
> ckrm-tech mailing list
> https://lists.sourceforge.net/lists/listinfo/ckrm-tech
--
------------------------------------------------------------ ----------
Chandra Seetharaman | Be careful what you choose....
- sekharan@us.ibm.com | .......you may get it.
------------------------------------------------------------ ----------
|
|
|
|
|
|
|
|
| Re: [ckrm-tech] [RFC][PATCH] UBC: user resource beancounters [message #5674 is a reply to message #5672] |
Fri, 25 August 2006 22:59 |
Chandra Seetharaman
Messages: 88
Registered: August 2006
|
Member |
|
|
On Fri, 2006-08-25 at 23:51 +0100, Alan Cox wrote:
> Ar Gwe, 2006-08-25 am 14:37 -0700, ysgrifennodd Chandra Seetharaman:
> > /* resources statistics and settings */
> > struct ubparm ub_parms[UB_RESOURCES];
> > };
> >
> > ub_parms of _all_ controllers are held in this data structure.
> >
> > So, keeping the beancounter data structure inside _a_ controller
> > specific data structure doesn't sound right to me, as other controllers
> > might also have the same need ?!
> >
> > Controller _owns_ only ub_parms[controller_id], not the whole
> > user_beancounter, right ?
>
> Right now I understand you
>
> So you need
>
> struct controller *ub_controller[UB_RESOURCES];
>
> ?
exactly :)
>
> Alan
>
>
> ------------------------------------------------------------ -------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&b id=263057&dat=121642
> _______________________________________________
> ckrm-tech mailing list
> https://lists.sourceforge.net/lists/listinfo/ckrm-tech
--
------------------------------------------------------------ ----------
Chandra Seetharaman | Be careful what you choose....
- sekharan@us.ibm.com | .......you may get it.
------------------------------------------------------------ ----------
|
|
|
|
| Re: [ckrm-tech] [RFC][PATCH] UBC: user resource beancounters [message #5675 is a reply to message #5673] |
Fri, 25 August 2006 23:00 |
Chandra Seetharaman
Messages: 88
Registered: August 2006
|
Member |
|
|
On Sat, 2006-08-26 at 00:12 +0100, Alan Cox wrote:
> Ar Gwe, 2006-08-25 am 15:23 -0700, ysgrifennodd Chandra Seetharaman:
> > > Bean counters can exist with no tasks, and the CKRM people have been
> > > corrected repeatedly on this point.
> >
> > Hmm... from what I understand from the code, when the last resource in
> > the beancounter is dropped, the beancounter is destroyed. Which to me
> > means that when there are no tasks in a beancounter it will be
> > destroyed. (I just tested the code and verified that the beancounter is
> > destroyed when the task dies).
>
> If a task created resource remains then the beancounter remains until
> the resources are destroyed, so it may exit well after the last task (eg
> an object handed to another process with a different luid is stil
> charged to us)
>
It is the _implicit destruction_ that is a problem.
> > Let me reword the requirement: beancounter/resource group should _not_
> > be destroyed implicitly. It should be destroyed only when requested by
> > the user/sysadmin. In other words, we need a create_luid() and
> > destroy_luid().
>
> So that you can preserve the limits on the resource group ? That also
> makes sense if you are trying to do long term resource management.
Yup.
>
> Alan
--
------------------------------------------------------------ ----------
Chandra Seetharaman | Be careful what you choose....
- sekharan@us.ibm.com | .......you may get it.
------------------------------------------------------------ ----------
|
|
|
|
| Re: [RFC][PATCH 1/7] UBC: kconfig [message #5677 is a reply to message #5195] |
Fri, 25 August 2006 15:12 |
Pavel Machek
Messages: 34
Registered: February 2006
|
Member |
|
|
Hi!
> --- ./kernel/ub/Kconfig.ubkm 2006-07-28
> 13:07:38.000000000 +0400
> +++ ./kernel/ub/Kconfig 2006-07-28
> 13:09:51.000000000 +0400
> @@ -0,0 +1,25 @@
> +#
> +# User resources part (UBC)
> +#
> +# Copyright (C) 2006 OpenVZ. SWsoft Inc
If you add copyright, add GPL, too.
> +config USER_RESOURCE
> + bool "Enable user resource accounting"
> + default y
New features should be disabled by default.
Pavel
--
Thanks for all the (sleeping) penguins.
|
|
|
|
Goto Forum:
Current Time: Sat Nov 09 00:47:08 GMT 2024
Total time taken to generate the page: 0.03407 seconds
|
OpenVZ Forum
Members
Search
Help
Register
Login
Home
