Home » Mailing lists » Devel » containers development plans
containers development plans [message #14580] |
Mon, 02 July 2007 16:55 |
serge
Messages: 72 Registered: January 2007
|
Member |
|
|
We are trying to create a roadmap for the next year of
'container' development, to be reported to the upcoming kernel
summit. Containers here is a bit of an ambiguous term, so we are
taking it to mean all of:
1. namespaces
2. process containers
3. checkpoint/restart
Naturally we can't actually predict what will and won't be worked on,
let alone what will be going upstream. But the following is a list
of features which it seems reasonable to think might be worked on
next year:
1. completion of ongoing namespaces
pid namespace
net namespace
ro bind mounts
2. continuation with new namespaces
devpts, console, and ttydrivers
user
time
namespace management tools
namespace entering
3. any additional work needed for virtual servers?
i.e. in-kernel keyring usage for cross-usernamespace permissions, etc
4. task containers functionality
base features
specific containers
poll to see who has plans
5. checkpoint/restart
memory c/r
(there are a few designs and prototypes)
(though this may be ironed out by then)
per-container swapfile?
overall checkpoint strategy
in-kernel vs userspace-driven
overall restart strategy
What more needs to be added to this list?
A list of the people we are currently aware of who are showing interest
in these features follows. What I'd like to know is, from this list, do
some people know what general or specific areas they plan to or want to
work on over the next year?
Stakeholders:
Eric Biederman
google
ibm
kerlabs
openvz
osdl (Masahiko Takahashi?)
vserver
Who is missing from the list?
thanks,
-serge
|
|
|
Re: containers development plans [message #14760 is a reply to message #14580] |
Tue, 10 July 2007 05:56 |
Paul Menage
Messages: 642 Registered: September 2006
|
Senior Member |
|
|
On 7/2/07, Serge E. Hallyn <serge@hallyn.com> wrote:
> 4. task containers functionality
How about if we adopt "process containers" or "task containers" as the
term for the generic container framework, to distinguish from more
general user-space containers? In the same way that "task_struct" in
the kernel is understood to be separate from the concept of a "task"
in a job scheduling system in userspace.
> base features
Features that I'd like to see in the short and medum term:
- support for virtualized containerfs mounts, so that virtual servers
can mount their own containerfs and manage sub-containers
- automatically prefixing control file names with the subsystem name,
unless changed or disabled by the user at mount time
- removing unnecessary locking where possible.
- simplifying the control file API
- a userspace RBCE along with simple configuration so that you can
easily use generic containers to apply subsystem controls on a
per-user, per-group, per-pgrp, per-executable, etc, basis. (E.g. to
easily apply CFS to be fair between pgrps rather than fair between
processes)
> specific containers
> poll to see who has plans
Some possible subsystems that I'm thinking of include:
- splitting the memory and cpu isolation parts of cpusets into two
separate subsystems (still backwards-compatible)
- some kind of network connect/bind/accept controller. Eric came up
with a nice way of doing this by adding iptables hooks for
connect/bind/accept, and then adding an iptables match module that
could match based on container id. This would give us all the
flexibility of iptables and the existing iptables tools. The drawback
is that it could be rather tricky to virtualize. A less flexible
solution that just allowed you to specify permitted
local-port-range/remote-port-range/remote-netmask tuples would be more
virtualizable, even if it doesn't make as much reuse of existing
iptables support.
- some way of controlling which network flow ids (used as inputs into
standard Linux queueing) processes in a container can use.
- userspace per-container OOM handler, maybe as part of cpusets or
some other memory controller.
Paul
|
|
|
|
|
|
|
|
Re: containers development plans [message #14871 is a reply to message #14760] |
Thu, 12 July 2007 10:32 |
dev
Messages: 1693 Registered: September 2005 Location: Moscow
|
Senior Member |
|
|
Paul Menage wrote:
> On 7/2/07, Serge E. Hallyn <serge@hallyn.com> wrote:
>
>> 4. task containers functionality
>
>
> How about if we adopt "process containers" or "task containers" as the
> term for the generic container framework, to distinguish from more
> general user-space containers? In the same way that "task_struct" in
> the kernel is understood to be separate from the concept of a "task"
> in a job scheduling system in userspace.
>
>
>> base features
>
>
> Features that I'd like to see in the short and medum term:
>
> - support for virtualized containerfs mounts, so that virtual servers
> can mount their own containerfs and manage sub-containers
>
> - automatically prefixing control file names with the subsystem name,
> unless changed or disabled by the user at mount time
>
> - removing unnecessary locking where possible.
>
> - simplifying the control file API
>
> - a userspace RBCE along with simple configuration so that you can
> easily use generic containers to apply subsystem controls on a
> per-user, per-group, per-pgrp, per-executable, etc, basis. (E.g. to
> easily apply CFS to be fair between pgrps rather than fair between
> processes)
>
>
>> specific containers
>> poll to see who has plans
>
>
> Some possible subsystems that I'm thinking of include:
>
> - splitting the memory and cpu isolation parts of cpusets into two
> separate subsystems (still backwards-compatible)
>
> - some kind of network connect/bind/accept controller. Eric came up
> with a nice way of doing this by adding iptables hooks for
> connect/bind/accept, and then adding an iptables match module that
> could match based on container id. This would give us all the
> flexibility of iptables and the existing iptables tools. The drawback
> is that it could be rather tricky to virtualize. A less flexible
> solution that just allowed you to specify permitted
> local-port-range/remote-port-range/remote-netmask tuples would be more
> virtualizable, even if it doesn't make as much reuse of existing
> iptables support.
Not sure why it requires some additional controller, but surely
it is possible to create a match for iptables matching container ID.
Is it what you are thinkinh about or I got something wrong?
Thanks,
Kirill
|
|
|
|
|
Re: containers development plans [message #14952 is a reply to message #14580] |
Sat, 14 July 2007 03:21 |
|
I just got an idea -- what about organizing a one-day "containers
mini-summit" just before the Kernel Summit? We can all meet face to
face, discuss all the issues and come out with a good plan for KS talk.
That does not mean we shouldn't discuss it here, of course -- it's just
to make it "polished".
I guess organizers of LinuxConf Europe (which takes place just before
the KS) will have a facility (i.e. a room) for us.
What do you think?
Serge E. Hallyn wrote:
> We are trying to create a roadmap for the next year of
> 'container' development, to be reported to the upcoming kernel
> summit.
|
|
|
|
Re: containers development plans [message #14959 is a reply to message #14958] |
Sat, 14 July 2007 23:28 |
|
Serge E. Hallyn wrote:
> Quoting Kir Kolyshkin (kir@openvz.org):
>
>> I just got an idea -- what about organizing a one-day "containers
>> mini-summit" just before the Kernel Summit? We can all meet face to
>> face, discuss all the issues and come out with a good plan for KS talk.
>> That does not mean we shouldn't discuss it here, of course -- it's just
>> to make it "polished".
>>
>> I guess organizers of LinuxConf Europe (which takes place just before
>> the KS) will have a facility (i.e. a room) for us.
>>
>> What do you think?
>>
>
> I like the idea, but I'm not sure it's practical since I suspect most
> people who aren't going to kernel summit won't be able to get funding to
> go to the mini-summit. I could be wrong about that...
>
> How many people will be at LinuxConf Europe - can we see a show of
> hands?
>
Me and Pavel Emelyanov -- since we both have talks on the LinuxConf.
|
|
|
|
|
|
|
Re: containers development plans [message #19143 is a reply to message #14580] |
Mon, 02 July 2007 21:53 |
serue
Messages: 750 Registered: February 2006
|
Senior Member |
|
|
Quoting Erich Focht (efocht@hpce.nec.com):
> Hi,
>
> On Monday 02 July 2007 18:55, Serge E. Hallyn wrote:
> > A list of the people we are currently aware of who are showing interest
> > in these features follows. What I'd like to know is, from this list, do
> > some people know what general or specific areas they plan to or want to
> > work on over the next year?
> >
> > Stakeholders:
> > Eric Biederman
> > google
> > ibm
> > kerlabs
> > openvz
> > osdl (Masahiko Takahashi?)
> > vserver
> >
> > Who is missing from the list?
>
> you could add the XtreemOS project to the list. It's grid project and we
> need to use containers for grid job isolation. We're currently using BLCR
> for checkpointing, and also kerrighed, but aim at switching to containers
> and will work on container checkpointing. XtreemOS is related to
> kerlabs & kerrighed, but has broader interests in containers.
That's cool. I'll go read up on XtreemOS before I ask stupid
questions...
> > What more needs to be added to this list?
>
> By the way, there are potential synergies between container checkpointing
> work and the need of the VM to have something like per cpuset of per
> container swap (as explained in the VM BoFS at OLS, there are unnecessary
> OOMs with a common swap).
Ah, cool, I hadn't heard that. At the risk of really exploding the # of
container subsystems needed to be composed in order to do
checkpoint/restart, this does sound like a separate container subsystem
which just attaches a swapfile to it's tasks might then be a useful
thing. It can then be composed with a cpuset or a freezer subsystem, or
both (at the same time - separately might be weird :).
I guess then for checkpoint we just freeze the subsystem, force a write
of all dirty pages using something like Dave's patch, somehow create a
copy-on-write clone as the checkpoint, then unfreeze. (and oh yeah,
checkpoint all the other crap too :)
> Regards,
> Erich
thanks,
-serge
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Re: containers development plans [message #19144 is a reply to message #14580] |
Mon, 02 July 2007 19:15 |
Erich Focht
Messages: 1 Registered: July 2007
|
Junior Member |
|
|
Hi,
On Monday 02 July 2007 18:55, Serge E. Hallyn wrote:
> A list of the people we are currently aware of who are showing interest
> in these features follows. What I'd like to know is, from this list, do
> some people know what general or specific areas they plan to or want to
> work on over the next year?
>
> Stakeholders:
> Eric Biederman
> google
> ibm
> kerlabs
> openvz
> osdl (Masahiko Takahashi?)
> vserver
>
> Who is missing from the list?
you could add the XtreemOS project to the list. It's grid project and we
need to use containers for grid job isolation. We're currently using BLCR
for checkpointing, and also kerrighed, but aim at switching to containers
and will work on container checkpointing. XtreemOS is related to
kerlabs & kerrighed, but has broader interests in containers.
> What more needs to be added to this list?
By the way, there are potential synergies between container checkpointing
work and the need of the VM to have something like per cpuset of per
container swap (as explained in the VM BoFS at OLS, there are unnecessary
OOMs with a common swap).
Regards,
Erich
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
|
|
Re: containers development plans [message #19156 is a reply to message #14580] |
Thu, 05 July 2007 17:41 |
Paul Jackson
Messages: 157 Registered: February 2006
|
Senior Member |
|
|
Thanks for responding, Eric.
For those like me (I suspect I'm not alone) who drift in and out of
this discussion, perhaps this would be a good time to describe again
what are the containers you're discussing here - their key purpose(s)
and their essential architecture.
For extra credit, putting aside the conflicting use of the word
'container', how would, in your view, PMC's (Paul Menage's Containers)
relate to TTC's (This Thread's Containers)? Would either use the other?
Are they complimentary, conflicting, ...?
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <pj@sgi.com> 1.925.600.0401
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
|
netns summary (was Re: containers development plans) [message #19161 is a reply to message #14580] |
Thu, 05 July 2007 14:19 |
Cedric Le Goater
Messages: 443 Registered: February 2006
|
Senior Member |
|
|
Serge E. Hallyn wrote:
> We are trying to create a roadmap for the next year of
> 'container' development, to be reported to the upcoming kernel
> summit. Containers here is a bit of an ambiguous term, so we are
> taking it to mean all of:
>
> 1. namespaces
> 2. process containers
> 3. checkpoint/restart
>
> Naturally we can't actually predict what will and won't be worked on,
> let alone what will be going upstream. But the following is a list
> of features which it seems rvseasonable to think might be worked on
> next year:
>
> 1. completion of ongoing namespaces
> pid namespace
> net namespace
I'm not sure if this has been said already :
At OLS, we had a talk with denis, pavel, eric, daniel, benjamin and others,
and we agreed to cooperate on eric's netns patchset which seems to satisfy
most of the stake holders :
* openvz
* ibm
* eric (hopefully :)
* planetlab (heavy users of linux-vserver)
* hp (also interested)
* google (?)
the planetlab team successfully included eric's netns patchset in the
linux-vserver patch and had good results with it.
the following tasks were discussed :
* improve the patchset to make it acceptable by the netdev community
* share a netns git tree
* share some tests framework all parties have been developing
independently.
Are we in sync ? I'm sure there are more interesting stuff to be said :)
We're looking for a place to host the netns git tree while we are working on
it, which means we will need first a federator for the pachset. Eric declined
as he is too busy. Any proposals ?
Thanks,
C.
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Re: containers development plans [message #19162 is a reply to message #14580] |
Thu, 05 July 2007 15:53 |
Cedric Le Goater
Messages: 443 Registered: February 2006
|
Senior Member |
|
|
Hello,
some more comments on what we talked about at OLS and what we are
working on.
Serge E. Hallyn wrote:
> We are trying to create a roadmap for the next year of
> 'container' development, to be reported to the upcoming kernel
> summit. Containers here is a bit of an ambiguous term, so we are
> taking it to mean all of:
>
> 1. namespaces
> 2. process containers
> 3. checkpoint/restart
>
> Naturally we can't actually predict what will and won't be worked on,
> let alone what will be going upstream. But the following is a list
> of features which it seems reasonable to think might be worked on
> next year:
>
> 1. completion of ongoing namespace
the ipc namespace would need a "set identifier" feature if we were
to use it for C/R. this is not available right now. a patchset was
sent introducing a new IPC_SETID but it didn't get much attention.
> pid namespace
At OLS, we agreed that suka's hierarchical pidns patchset should be
fine if we can make sure perfs are OK when the namespace is not
used. right ?
I get < 1% today, so it should be okay :)
There are still some issues around /proc that we are working on.
Hopefully, we should be able to merge most of the helpers patch
real soon.
we need a clone_with_pid() kind of syscall for C/R. I had planned
to work on a :
clone64(struct clone64_arg_struct *arg)
to extend the clone flags which will soon overflow. we could
easily add a pid attribute to implement the clone_with_pid()
the kthread cleanup is not completed yet. some patch are pending
but i would say that the most important ones are around NFS and
i'm not sure anyone worked on these.
af unix credentials still hold some pid_t's. they need a clean up.
> net namespace
see previous email
> ro bind mounts
work in progress. dave ?
what about mounting /proc and /sys multiple times ?
> 2. continuation with new namespaces
> devpts, console, and ttydrivers
> user
merged experimental. we still need to work on the (user,userns) checks.
however, openvz and linux-vserver should already be able to use it.
> time
> namespace management tools
> namespace entering
there are a few patchsets on the topic :
* bind_ns() syscall
* container subsystem identifying a nsproxy object
but they didn't get much review :(
> 3. any additional work needed for virtual servers?
> i.e. in-kernel keyring usage for cross-usernamespace permissions, etc
> 4. task containers functionality
> base features
> specific containers
> poll to see who has plans
> 5. checkpoint/restart
we really need to leverage the freezer and suspend to disk for that.
there are some talks about it right now but it seems a bit early to
have clear directions yet.
generalizing the refrigerator to all arch seems a good idea to freeze
a container. then how do we initiate checkpoint ? syscall ? signal ?
etc.
These topics were addressed at the BOF and people are now aware of
different solutions. we hope that the email storm on what directions
to take for mainline will start soon.
C.
> memory c/r
> (there are a few designs and prototypes)
> (though this may be ironed out by then)
> per-container swapfile?
> overall checkpoint strategy
> in-kernel vs userspace-driven
hybrid ?
> overall restart strategy
>
> What more needs to be added to this list?
>
> A list of the people we are currently aware of who are showing interest
> in these features follows. What I'd like to know is, from this list, do
> some people know what general or specific areas they plan to or want to
> work on over the next year?
>
> Stakeholders:
> Eric Biederman
> google
> ibm
> kerlabs
> openvz
> osdl (Masahiko Takahashi?)
> vserver
>
> Who is missing from the list?
>
> thanks,
> -serge
> _______________________________________________
> Containers mailing list
> Containers@lists.linux-foundation.org
> https://lists.linux-foundation.org/mailman/listinfo/containers
>
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
|
|
Re: containers development plans [message #19219 is a reply to message #19155] |
Mon, 09 July 2007 15:17 |
serge
Messages: 72 Registered: January 2007
|
Member |
|
|
Quoting Paul Jackson (pj@sgi.com):
> > they are indirectly related. Sorry for the noise.
>
> Well ... if this is noise, it should be me apologizing, not you <grin>.
> I'm the one who asked.
>
> > Paul Menage's container patches provide a process aggregation mechanism,
> > like PAGG did. This is a common requirement for resource management and
> > other container features like resource isolation and checkpoint/restart.
>
> So ... will you (those using the netns patches) be using Paul Menage's
> containers, or competing with them for the same ends, or what?
>
> To repeat my original question, what is the relation between the work
> described on this thread (which some OLS discussions have decided to
Hi Paul,
containers does mean two things in this thread, but note that we
want to discuss both meanings.
So if any development is planned over the next year on top of
Pauls Menage's containers and/or cpusets, we'd like that listed
here as well.
thanks,
-serge
> base on the netns patches) and Paul Menage's containers and my (now the
> community's) cpusets.
>
> --
> I won't rest till it's the best ...
> Programmer, Linux Scalability
> Paul Jackson <pj@sgi.com> 1.925.600.0401
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Re: containers development plans [message #19220 is a reply to message #19156] |
Mon, 09 July 2007 15:25 |
serge
Messages: 72 Registered: January 2007
|
Member |
|
|
Quoting Paul Jackson (pj@sgi.com):
> Thanks for responding, Eric.
>
> For those like me (I suspect I'm not alone) who drift in and out of
> this discussion, perhaps this would be a good time to describe again
> what are the containers you're discussing here - their key purpose(s)
> and their essential architecture.
>
> For extra credit, putting aside the conflicting use of the word
> 'container', how would, in your view, PMC's (Paul Menage's Containers)
> relate to TTC's (This Thread's Containers)? Would either use the other?
Again, we want to discuss both in this roadmap.
Let's stick to talking about PMCs and namespaces (TTC's).
The namespace work just aims to take any resources which userspace might
wish to identify by a global id, and allow a process to unshare it's
view of that namespace so that
1. it can isolate its resources from those of other processes
(virtual servers)
2. it can guarantee that certain ids are available, to make
process restart possible.
PMCs are very useful to both process restart and virtual servers:
1. for resource management
2. for providing APIs for namespace operations
a. for example, the freezer subsystem Cedric recently
sent out, which can be used to freeze a checkpoint/restart
job or virtual server (or any type of container) to allow
safe checkpoint.
b. to restrict a virtual server's cpu and memory usage
> Are they complimentary, conflicting, ...?
Very much complimentary.
thanks,
-serge
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Re: containers development plans [message #19221 is a reply to message #19216] |
Mon, 09 July 2007 15:27 |
serge
Messages: 72 Registered: January 2007
|
Member |
|
|
Quoting Paul Jackson (pj@sgi.com):
> Serge wrote:
> > So if any development is planned over the next year on top of
> > Pauls Menage's containers and/or cpusets, we'd like that listed
> > here as well.
>
> Well, apparenntly you guys are planning some "container" stuff
> that is on top of or relates to in some way unclear to me with
> Menage's containers and/or cpusets.
>
> I'll repeat my question a third time ... not knowing any clearer
> way to state it than before, I'll just quote myself:
Please see the next response I sent, and let me know what I've
still failed to clarify there.
(trying to avoid more repeating :)
thanks,
-serge
> > So ... will you (those using the netns patches) be using Paul Menage's
> > containers, or competing with them for the same ends, or what?
> >
> > To repeat my original question, what is the relation between the work
> > described on this thread (which some OLS discussions have decided to
> > base on the netns patches) and Paul Menage's containers and my (now the
> > community's) cpusets.
>
> --
> I won't rest till it's the best ...
> Programmer, Linux Scalability
> Paul Jackson <pj@sgi.com> 1.925.600.0401
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Re: containers development plans [message #19229 is a reply to message #14580] |
Mon, 09 July 2007 12:12 |
Herbert Poetzl
Messages: 239 Registered: February 2006
|
Senior Member |
|
|
On Mon, Jul 02, 2007 at 11:55:04AM -0500, Serge E. Hallyn wrote:
> We are trying to create a roadmap for the next year of
> 'container' development, to be reported to the upcoming kernel
> summit. Containers here is a bit of an ambiguous term, so we are
> taking it to mean all of:
>
> 1. namespaces
> 2. process containers
> 3. checkpoint/restart
>
> Naturally we can't actually predict what will and won't be worked on,
> let alone what will be going upstream. But the following is a list
> of features which it seems reasonable to think might be worked on
> next year:
>
> 1. completion of ongoing namespaces
> pid namespace
> net namespace
> ro bind mounts
> 2. continuation with new namespaces
> devpts, console, and ttydrivers
> user
> time
various accounting and limits including disk space
and memory
> namespace management tools
> namespace entering
well, that is currently covered by the Linux-VServer
interfaces (and I assume by the OpenVZ API too), but
I guess you are more interested in a mainline tool
and interface
> 3. any additional work needed for virtual servers?
> i.e. in-kernel keyring usage for cross-usernamespace permissions, etc
nfs and rpc, as well as general security/auth
frameworks have to be adjusted ...
> 4. task containers functionality
> base features
> specific containers
> poll to see who has plans
> 5. checkpoint/restart
> memory c/r
> (there are a few designs and prototypes)
> (though this may be ironed out by then)
> per-container swapfile?
> overall checkpoint strategy
> in-kernel vs userspace-driven
> overall restart strategy
>
> What more needs to be added to this list?
>
> A list of the people we are currently aware of who are showing interest
> in these features follows. What I'd like to know is, from this list, do
> some people know what general or specific areas they plan to or want to
> work on over the next year?
>
> Stakeholders:
> Eric Biederman
> google
> ibm
> kerlabs
> openvz
> osdl (Masahiko Takahashi?)
> vserver
>
> Who is missing from the list?
Linux-VServer as usual :)
best,
Herbert
> thanks,
> -serge
> _______________________________________________
> Containers mailing list
> Containers@lists.linux-foundation.org
> https://lists.linux-foundation.org/mailman/listinfo/containers
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Re: netns summary (was Re: containers development plans) [message #19230 is a reply to message #19161] |
Mon, 09 July 2007 12:16 |
Herbert Poetzl
Messages: 239 Registered: February 2006
|
Senior Member |
|
|
On Thu, Jul 05, 2007 at 04:19:08PM +0200, Cedric Le Goater wrote:
> Serge E. Hallyn wrote:
> > We are trying to create a roadmap for the next year of
> > 'container' development, to be reported to the upcoming kernel
> > summit. Containers here is a bit of an ambiguous term, so we are
> > taking it to mean all of:
> >
> > 1. namespaces
> > 2. process containers
> > 3. checkpoint/restart
> >
> > Naturally we can't actually predict what will and won't be worked on,
> > let alone what will be going upstream. But the following is a list
> > of features which it seems rvseasonable to think might be worked on
> > next year:
> >
> > 1. completion of ongoing namespaces
> > pid namespace
> > net namespace
>
> I'm not sure if this has been said already :
>
> At OLS, we had a talk with denis, pavel, eric, daniel, benjamin and
> others, and we agreed to cooperate on eric's netns patchset which
> seems to satisfy most of the stake holders :
> * openvz
> * ibm
> * eric (hopefully :)
> * planetlab (heavy users of linux-vserver)
> * hp (also interested)
> * google (?)
>
> the planetlab team successfully included eric's netns patchset in the
> linux-vserver patch and had good results with it.
except for the increased overhead (just for the record)
(so functionality wise, it seemed fine)
> the following tasks were discussed :
>
> * improve the patchset to make it acceptable by the netdev community
> * share a netns git tree
> * share some tests framework all parties have been developing
> independently.
>
> Are we in sync ? I'm sure there are more interesting stuff to be said :)
>
> We're looking for a place to host the netns git tree while we are
> working on it, which means we will need first a federator for the
> pachset. Eric declined as he is too busy. Any proposals ?
interesting ...
best,
Herbert
> Thanks,
>
> C.
> _______________________________________________
> Containers mailing list
> Containers@lists.linux-foundation.org
> https://lists.linux-foundation.org/mailman/listinfo/containers
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Re: [ckrm-tech] containers development plans [message #19235 is a reply to message #19216] |
Mon, 09 July 2007 15:32 |
dev
Messages: 1693 Registered: September 2005 Location: Moscow
|
Senior Member |
|
|
Paul Jackson wrote:
> Serge wrote:
>
>>So if any development is planned over the next year on top of
>>Pauls Menage's containers and/or cpusets, we'd like that listed
>>here as well.
>
>
> Well, apparenntly you guys are planning some "container" stuff
> that is on top of or relates to in some way unclear to me with
> Menage's containers and/or cpusets.
>
> I'll repeat my question a third time ... not knowing any clearer
> way to state it than before, I'll just quote myself:
>
>
>>So ... will you (those using the netns patches) be using Paul Menage's
>>containers, or competing with them for the same ends, or what?
>>
>>To repeat my original question, what is the relation between the work
>>described on this thread (which some OLS discussions have decided to
>>base on the netns patches) and Paul Menage's containers and my (now the
>>community's) cpusets.
Paul,
Paul Menage's containers is a good interfaces for gouping tasks
which is planned to be used for resource controls (including cpusets which actually
control cpu resources as well). So it is planned to be used for:
cpusets, RSS, fair CPU and other resource controls.
Network namespaces, ipc and other namespaces use unshare() mechanisms
and can have a simple view in containers fs.
Thanks,
Kirill
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Re: [ckrm-tech] containers development plans [message #19248 is a reply to message #19235] |
Tue, 10 July 2007 07:25 |
Paul Jackson
Messages: 157 Registered: February 2006
|
Senior Member |
|
|
Kirill, Serge, et al,
Is it fair to say then that Paul Menage's containers are primarily
for the purposes of managing resources, while namespaces are for the
purposes of managing identifiers?
We've got some resources, like cpu cycles, memory bytes, network
bandwidth, that we want to allocate and account for differentially
by groups of tasks -- that's Menage's containers.
We've got some system wide namespaces, like process id's, that we
want to virtualize, for more flexible uses -- these are the name-
space containers.
In Serge's opening post to this thread, he wrote:
1. namespaces
2. process containers
3. checkpoint/restart
Are the 'process containers' of item (2) the containers of Paul Menage?
If so, then I propose that this thread is misnamed. It should not be
"containers development plans", but rather "namespace, container and
c/r development plans." And if so, there is really no conflict over
the use of the word 'container' -- that applies just to the resource
virtualization efforts, of which my cpusets is the granddaddy example,
being generalized by Paul Menage with his container patches. The other
work is, as Serge actually termed it in the body of his post, better
called 'namespaces'.
Perhaps the confusion arose from looking for a single word to encompass
all three parts, listed above, of this work. The efforts have some strong
dependencies, but taking the name of one of the efforts, containers, and
trying to make it serve double duty as the umbrella term, might be an
unnecessary confusion.
Perhaps also, on a separate point, the word 'process' in item (2) is
not the right focus. I think that the essential purpose of (2) is
resource management. While most of those resource management is done
per-process, it might also be per-file or per-virtual-address-range.
For example, disk i/o backing pages in a particular range of an
applications address space might have certain bandwidth limits, and the
memory backing the pages in that range might have certain memory node
placement restrictions, or the i/o to and from a particular disk file
might have certain bandwidth or placement constraints. We see a bit of
this in present day kernels, with the mbind(2) system call.
So, I suggest, we have three efforts:
1. namespaces,
2. resource containers, and
3. checkpoint/restart.
And they are very much in this order. Names, such as pathnames, task
pids, user uids, and other system wide identifiers, are essential to
the implementation of all else. Resource containers depend on some
naming scheme, and reach out to manage the use of resources outside
the operating system, such as disk, network, memory and processor.
Checkpoint/restart is a particular feature of interest, that requires
that both names and resources be virtualized to some degree.
Are there any Python programmers in the namespace work? The use of
namespaces in Python might serve as a informative example for the work
we need in Linux namespaces. See further page 418, section "A.2
Namespaces and Binds" of David Mertz's "Text Processing in Python" for
a clear and concise exposition of the central role of namespaces in
Python.
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <pj@sgi.com> 1.925.600.0401
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Re: [ckrm-tech] containers development plans [message #19250 is a reply to message #19248] |
Tue, 10 July 2007 07:53 |
Paul Menage
Messages: 642 Registered: September 2006
|
Senior Member |
|
|
On 7/10/07, Paul Jackson <pj@sgi.com> wrote:
> Kirill, Serge, et al,
>
> Is it fair to say then that Paul Menage's containers are primarily
> for the purposes of managing resources, while namespaces are for the
> purposes of managing identifiers?
Sort of - but one thing that we're trying to figure out how to do
nicely is integrate namespaces into the container filesystem (this was
the purpose of the post_clone() container API callback) so that we can
both get a filesystem view of task namespaces, and combine namespaces
with other process container subsystems.
>
> We've got some resources, like cpu cycles, memory bytes, network
> bandwidth, that we want to allocate and account for differentially
> by groups of tasks -- that's Menage's containers.
Plus things that aren't necessarily resource controllers, such as the
container freezer, or permissions on network ports, or userspace OOM
handlers. I don't think that lumping all of these in together as
"resource containers" is the right thing to do.
> virtualization efforts, of which my cpusets is the granddaddy example,
> being generalized by Paul Menage with his container patches. The other
> work is, as Serge actually termed it in the body of his post, better
> called 'namespaces'.
Purely within the kernel, yes. The more general encompassing effort to
have a combined kernel/userspace solution for virtual servers is also
referred to as "containers". (And to be fair that term was already in
use when I started using the term "process containers" to refer to the
specific framework in the kernel that handles process tracking).
Paul
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Re: [ckrm-tech] containers development plans [message #19251 is a reply to message #19250] |
Tue, 10 July 2007 08:30 |
Paul Jackson
Messages: 157 Registered: February 2006
|
Senior Member |
|
|
Paul M wrote:
> Sort of - but ...
Thanks for your reply.
I guess I'll just have to wander off, still a tad confused.
I suspect my instincts are running in a contrary direction to those,
including yourself, doing the work here. My preference is to pick
a subset of the task before me that I can clearly "brand", with a
name and description that others can quickly understand. Better
to clearly express 80% of the goal in a well branded product, than
to confuse the 'customers' (those you want to buy into to the proposal)
with a conceptually more elaborate, but less well defined, attempt to
cover all 100% of the goal.
Often times, the quickest way to convey a complex idea is to convey
an overly simplified approximation, and then allow the listener to
tell you why what you said is too simple minded. This is because the
bandwidth within a brain vastly exceeds that between two brains.
But since I'm not volunteering to do the work, I should not waste
too much more time of those who are.
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <pj@sgi.com> 1.925.600.0401
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Re: [ckrm-tech] containers development plans [message #19262 is a reply to message #19248] |
Tue, 10 July 2007 16:32 |
serge
Messages: 72 Registered: January 2007
|
Member |
|
|
Quoting Paul Jackson (pj@sgi.com):
> Kirill, Serge, et al,
>
> Is it fair to say then that Paul Menage's containers are primarily
> for the purposes of managing resources, while namespaces are for the
> purposes of managing identifiers?
>
> We've got some resources, like cpu cycles, memory bytes, network
> bandwidth, that we want to allocate and account for differentially
> by groups of tasks -- that's Menage's containers.
>
> We've got some system wide namespaces, like process id's, that we
> want to virtualize, for more flexible uses -- these are the name-
> space containers.
>
> In Serge's opening post to this thread, he wrote:
> 1. namespaces
> 2. process containers
> 3. checkpoint/restart
>
> Are the 'process containers' of item (2) the containers of Paul Menage?
Yup.
> If so, then I propose that this thread is misnamed. It should not be
> "containers development plans", but rather "namespace, container and
> c/r development plans." And if so, there is really no conflict over
> the use of the word 'container' -- that applies just to the resource
> virtualization efforts, of which my cpusets is the granddaddy example,
> being generalized by Paul Menage with his container patches. The other
> work is, as Serge actually termed it in the body of his post, better
> called 'namespaces'.
>
> Perhaps the confusion arose from looking for a single word to encompass
> all three parts, listed above, of this work. The efforts have some strong
Not exactly - the "confusion" arose because the ksummit committee wanted
to hear about "containers", and agreed that by that term they mean each
of those three. So I kept the term 'containers' in the roadmap title,
but we can change that if it's preferred.
thanks,
-serge
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Re: containers development plans [message #19274 is a reply to message #19229] |
Tue, 10 July 2007 20:33 |
serge
Messages: 72 Registered: January 2007
|
Member |
|
|
Quoting Herbert Poetzl (herbert@13thfloor.at):
> On Mon, Jul 02, 2007 at 11:55:04AM -0500, Serge E. Hallyn wrote:
> > We are trying to create a roadmap for the next year of
> > 'container' development, to be reported to the upcoming kernel
> > summit. Containers here is a bit of an ambiguous term, so we are
> > taking it to mean all of:
> >
> > 1. namespaces
> > 2. process containers
> > 3. checkpoint/restart
> >
> > Naturally we can't actually predict what will and won't be worked on,
> > let alone what will be going upstream. But the following is a list
> > of features which it seems reasonable to think might be worked on
> > next year:
> >
> > 1. completion of ongoing namespaces
> > pid namespace
> > net namespace
> > ro bind mounts
>
> > 2. continuation with new namespaces
> > devpts, console, and ttydrivers
> > user
> > time
>
> various accounting and limits including disk space
> and memory
Is it fair to leave that under task container subsystems?
> > namespace management tools
> > namespace entering
>
> well, that is currently covered by the Linux-VServer
> interfaces (and I assume by the OpenVZ API too), but
> I guess you are more interested in a mainline tool
> and interface
I'm interested in anything someone is willing to try to push
upstream :)
> > 3. any additional work needed for virtual servers?
> > i.e. in-kernel keyring usage for cross-usernamespace permissions, etc
>
> nfs and rpc, as well as general security/auth
> frameworks have to be adjusted ...
>
> > 4. task containers functionality
> > base features
> > specific containers
> > poll to see who has plans
> > 5. checkpoint/restart
> > memory c/r
> > (there are a few designs and prototypes)
> > (though this may be ironed out by then)
> > per-container swapfile?
> > overall checkpoint strategy
> > in-kernel vs userspace-driven
> > overall restart strategy
> >
> > What more needs to be added to this list?
> >
> > A list of the people we are currently aware of who are showing interest
> > in these features follows. What I'd like to know is, from this list, do
> > some people know what general or specific areas they plan to or want to
> > work on over the next year?
> >
> > Stakeholders:
> > Eric Biederman
> > google
> > ibm
> > kerlabs
> > openvz
> > osdl (Masahiko Takahashi?)
> > vserver
> >
> > Who is missing from the list?
>
> Linux-VServer as usual :)
I stand corrected as usual.
-serge
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Re: [ckrm-tech] containers development plans [message #19277 is a reply to message #19262] |
Tue, 10 July 2007 21:30 |
serge
Messages: 72 Registered: January 2007
|
Member |
|
|
Quoting Serge E. Hallyn (serge@hallyn.com):
> Quoting Paul Jackson (pj@sgi.com):
> > Kirill, Serge, et al,
> >
> > Is it fair to say then that Paul Menage's containers are primarily
> > for the purposes of managing resources, while namespaces are for the
> > purposes of managing identifiers?
> >
> > We've got some resources, like cpu cycles, memory bytes, network
> > bandwidth, that we want to allocate and account for differentially
> > by groups of tasks -- that's Menage's containers.
> >
> > We've got some system wide namespaces, like process id's, that we
> > want to virtualize, for more flexible uses -- these are the name-
> > space containers.
> >
> > In Serge's opening post to this thread, he wrote:
> > 1. namespaces
> > 2. process containers
> > 3. checkpoint/restart
> >
> > Are the 'process containers' of item (2) the containers of Paul Menage?
>
> Yup.
>
> > If so, then I propose that this thread is misnamed. It should not be
> > "containers development plans", but rather "namespace, container and
> > c/r development plans." And if so, there is really no conflict over
> > the use of the word 'container' -- that applies just to the resource
> > virtualization efforts, of which my cpusets is the granddaddy example,
> > being generalized by Paul Menage with his container patches. The other
> > work is, as Serge actually termed it in the body of his post, better
> > called 'namespaces'.
> >
> > Perhaps the confusion arose from looking for a single word to encompass
> > all three parts, listed above, of this work. The efforts have some strong
>
> Not exactly - the "confusion" arose because the ksummit committee wanted
> to hear about "containers", and agreed that by that term they mean each
> of those three. So I kept the term 'containers' in the roadmap title,
> but we can change that if it's preferred.
I plan to keep the thread titled 'containers' precisely because *I*
don't care whose work gets renamed, while several other people on both
sides care so strongly, so it would seem rude for me to make that
decision de-facto in this way.
Maybe renaming one or both projects should be listed in the roadmap as a
todo :)
thanks,
-serge
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
|
Goto Forum:
Current Time: Thu Dec 05 16:13:00 GMT 2024
Total time taken to generate the page: 0.10762 seconds
|