Home » Mailing lists » Devel » L2 network namespace benchmarking
L2 network namespace benchmarking [message #18035] |
Tue, 27 March 2007 22:16 |
Daniel Lezcano
Messages: 417 Registered: June 2006
|
Senior Member |
|
|
Hi,
I did some benchmarking on the existing L2 network namespaces.
These patches are included in the lxc patchset at:
http://lxc.sourceforge.net/patches/2.6.20
The lxc7 patchset series contains Dmitry's patchset
The lxc8 patchset series contains Eric's patchset
Here are the following scenarii I made in order to do some simple
benchmarking on the network namespace. I tested three kernels:
* Vanilla kernel 2.6.20
* lxc7 with Dmitry's patchset based on 2.6.20
* L3 network namespace has been removed to do testing
* lxc8 with Eric's patchset based on 2.6.20
I didn't do any tests on Linux-Vserver because it is L3 namespace and
it is not comparable with the L2 namespace implementation. If anyone
is interessted by Linux-Vserver performances, that can be found at
http://lxc.sf.net. Roughly, we know there is no performance
degradation.
For each kernel, several configurations were tested:
* vanilla, obviously, only one configuration was tested for reference
values.
* lxc7, network namespace
- compiled out
- compiled in
- without container
- inside a container with ip_forward, route and veth
- inside a container with a bridge and veth
* lxc8, network namespace
- compiled out
- compiled in
- without container
- inside a container with a real network device (eth1 was moved
in the container instead of using an etun device)
- inside a container with ip_forward, route and etun
- inside a container with a bridge and etun
Each benchmarking has been done with 2 machines running netperf and
tbench. A dedicated machine with a RH4 kernel run the bench servers.
For each bench, netperf and tbench, the tests are ran on:
* Intel Xeon EM64T, Bi-processor 2,8GHz with hyperthreading
activated, 4GB of RAM and Gigabyte NIC (tg3)
* AMD Athlon MP 1800+, Bi-processor 1,5GHz, 1GB of RAM and Gigabyte
NIC (dl2000)
Each tests are run on these machines in order to have a CPU relative
overhead.
# bench on vanilla
===================
----------- --------------------------------------
| Netperf | CPU usage (%) | Throughput (Mbits/s) |
----------- --------------------------------------
| on xeon | 5.99 | 941.38 |
--------------------------------------------------
| on athlon | 28.17 | 844.82 |
--------------------------------------------------
----------- -----------------------
| Tbench | Throughput (MBytes/s) |
----------- -----------------------
| on xeon | 66.35 |
-----------------------------------
| on athlon | 65.31 |
-----------------------------------
# bench from Dmitry's patchset
==============================
1 - with net_ns compiled out
----------------------------
----------- -----------------------------------------------------------
| Netperf | CPU usage (%) / overhead | Throughput (Mbits/s) / changed |
----------- -----------------------------------------------------------
| on xeon | 5.93 / -1 % | 941.32 / 0 % |
-----------------------------------------------------------------------
| on athlon | 28.89 / +2.5 % | 842.78 / -0.2 % |
-----------------------------------------------------------------------
----------- ---------------------------------
| Tbench | Throughput (MBytes/s) / changed |
----------- ---------------------------------
| on xeon | 67.00 / +0.9 % |
---------------------------------------------
| on athlon | 65.45 / 0 % |
---------------------------------------------
Observation : no noticeable overhead
2 - with net_ns compiled in
---------------------------
2.1 - without container
-----------------------
----------- -----------------------------------------------------------
| Netperf | CPU usage (%) / overhead | Throughput (Mbits/s) / changed |
----------- -----------------------------------------------------------
| on xeon | 6.23 / +4 % | 941.35 / 0 % |
-----------------------------------------------------------------------
| on athlon | 28.83 / +2.3 % | 850.76 / +0.7 % |
-----------------------------------------------------------------------
----------- ---------------------------------
| Tbench | Throughput (MBytes/s) / changed |
----------- ---------------------------------
| on xeon | 67.00 / 0 % |
---------------------------------------------
| on athlon | 65.45 / 0 % |
---------------------------------------------
Observation : no noticeable overhead
2.2 - inside the container with veth and routes
-----------------------------------------------
----------- -----------------------------------------------------------
| Netperf | CPU usage (%) / overhead | Throughput (Mbits/s) / changed |
----------- -----------------------------------------------------------
| on xeon | 17.14 / +186.1 % | 941.34 / 0 % |
-----------------------------------------------------------------------
| on athlon | 49.99 / +77.45 % | 838.85 / +0.7 % |
-----------------------------------------------------------------------
----------- ---------------------------------
| Tbench | Throughput (MBytes/s) / changed |
----------- ---------------------------------
| on xeon | 66.00 / -0.5 % |
---------------------------------------------
| on athlon | 61.00 / -6.65 % |
---------------------------------------------
Observation : CPU overhead is very big, throughput is impacted on
the less powerful machine
2.3 - inside the container with veth and bridge
-----------------------------------------------
----------- -----------------------------------------------------------
| Netperf | CPU usage (%) / overhead | Throughput (Mbits/s) / changed |
----------- -----------------------------------------------------------
| on xeon | 19.14 / +299 % | 941.18 / 0 % |
-----------------------------------------------------------------------
| on athlon | 49.98 / +77.42 % | 831.65 / -1.5 % |
-----------------------------------------------------------------------
----------- ---------------------------------
| Tbench | Throughput (MBytes/s) / changed |
----------- ---------------------------------
| on xeon | 64.00 / -3.5 % |
---------------------------------------------
| on athlon | 60.07 / -8.3 % |
---------------------------------------------
Observation : CPU overhead is very big, throughput is impacted on
the less powerful machine
# bench from Eric's patchset
============================
1 - with net_ns compiled out
----------------------------
----------- -----------------------------------------------------------
| Netperf | CPU usage (%) / overhead | Throughput (Mbits/s) / changed |
----------- -----------------------------------------------------------
| on xeon | 6.04 / +0.8 % | 941.33 / 0 % |
-----------------------------------------------------------------------
| on athlon | 28.45 / +1 % | 840.76 / -0.5 % |
-----------------------------------------------------------------------
----------- ---------------------------------
| Tbench | Throughput (MBytes/s) / changed |
----------- ---------------------------------
| on xeon | 65.69 / -1 % |
---------------------------------------------
| on athlon | 65.35 / -0.2 % |
---------------------------------------------
Observation : no noticeable overhead
2 - with net_ns compiled in
---------------------------
2.1 - without container
-----------------------
----------- -----------------------------------------------------------
| Netperf | CPU usage (%) / overhead | Throughput (Mbits/s) / changed |
----------- -----------------------------------------------------------
| on xeon | 6.02 / +0.5 % | 941.34 / 0 % |
-----------------------------------------------------------------------
| on athlon | 27.93 / -0.8 % | 833.53 / -1.3 % |
-----------------------------------------------------------------------
----------- ---------------------------------
| Tbench | Throughput (MBytes/s) / changed |
----------- ---------------------------------
| on xeon | 66.00 / -0.5 % |
---------------------------------------------
| on athlon | 64.94 / -0.9 % |
---------------------------------------------
Observation : no noticeable overhead
2.2 - inside the container with real device
-------------------------------------------
----------- -----------------------------------------------------------
| Netperf | CPU usage (%) / overhead | Throughput (Mbits/s) / changed |
----------- -----------------------------------------------------------
| on xeon | 5.60 / -6.5 % | 941.42 / 0 % |
-----------------------------------------------------------------------
| on athlon | 27.73 / -1.5 % | 835.11 / +1.5 % |
-----------------------------------------------------------------------
----------- ---------------------------------
| Tbench | Throughput (MBytes/s) / changed |
----------- ---------------------------------
| on xeon | 74.36 / +12 % |
---------------------------------------------
| on athlon | 70.87 / +8.2 % |
---------------------------------------------
Observation : no noticeable overhead. The network interface is only
used by the container, so I guess it does not interact with another
network traffic and that explains the performances are better.
2.3 - inside the container with etun and routes
----
...
|
|
|
Re: L2 network namespace benchmarking [message #18036 is a reply to message #18035] |
Tue, 27 March 2007 23:08 |
Herbert Poetzl
Messages: 239 Registered: February 2006
|
Senior Member |
|
|
On Wed, Mar 28, 2007 at 12:16:34AM +0200, Daniel Lezcano wrote:
>
> Hi,
>
> I did some benchmarking on the existing L2 network namespaces.
>
> These patches are included in the lxc patchset at:
> http://lxc.sourceforge.net/patches/2.6.20
> The lxc7 patchset series contains Dmitry's patchset
> The lxc8 patchset series contains Eric's patchset
>
> Here are the following scenarii I made in order to do some simple
> benchmarking on the network namespace. I tested three kernels:
>
> * Vanilla kernel 2.6.20
>
> * lxc7 with Dmitry's patchset based on 2.6.20
> * L3 network namespace has been removed to do testing
>
> * lxc8 with Eric's patchset based on 2.6.20
>
> I didn't do any tests on Linux-Vserver because it is L3 namespace and
> it is not comparable with the L2 namespace implementation. If anyone
> is interessted by Linux-Vserver performances, that can be found at
> http://lxc.sf.net. Roughly, we know there is no performance
> degradation.
>
> For each kernel, several configurations were tested:
>
> * vanilla, obviously, only one configuration was tested for reference
> values.
>
> * lxc7, network namespace
> - compiled out
> - compiled in
> - without container
> - inside a container with ip_forward, route and veth
> - inside a container with a bridge and veth
>
> * lxc8, network namespace
> - compiled out
> - compiled in
> - without container
> - inside a container with a real network device (eth1 was moved
> in the container instead of using an etun device)
> - inside a container with ip_forward, route and etun
> - inside a container with a bridge and etun
>
> Each benchmarking has been done with 2 machines running netperf and
> tbench. A dedicated machine with a RH4 kernel run the bench servers.
>
> For each bench, netperf and tbench, the tests are ran on:
>
> * Intel Xeon EM64T, Bi-processor 2,8GHz with hyperthreading
> activated, 4GB of RAM and Gigabyte NIC (tg3)
>
> * AMD Athlon MP 1800+, Bi-processor 1,5GHz, 1GB of RAM and Gigabyte
> NIC (dl2000)
>
> Each tests are run on these machines in order to have a CPU relative
> overhead.
>
>
> # bench on vanilla
> ===================
>
>
> ----------- --------------------------------------
> | Netperf | CPU usage (%) | Throughput (Mbits/s) |
> ----------- --------------------------------------
> | on xeon | 5.99 | 941.38 |
> --------------------------------------------------
> | on athlon | 28.17 | 844.82 |
> --------------------------------------------------
>
> ----------- -----------------------
> | Tbench | Throughput (MBytes/s) |
> ----------- -----------------------
> | on xeon | 66.35 |
> -----------------------------------
> | on athlon | 65.31 |
> -----------------------------------
>
>
> # bench from Dmitry's patchset
> ==============================
>
>
> 1 - with net_ns compiled out
> ----------------------------
>
> ----------- -----------------------------------------------------------
> | Netperf | CPU usage (%) / overhead | Throughput (Mbits/s) / changed |
> ----------- -----------------------------------------------------------
> | on xeon | 5.93 / -1 % | 941.32 / 0 % |
> -----------------------------------------------------------------------
> | on athlon | 28.89 / +2.5 % | 842.78 / -0.2 % |
> -----------------------------------------------------------------------
>
> ----------- ---------------------------------
> | Tbench | Throughput (MBytes/s) / changed |
> ----------- ---------------------------------
> | on xeon | 67.00 / +0.9 % |
> ---------------------------------------------
> | on athlon | 65.45 / 0 % |
> ---------------------------------------------
>
> Observation : no noticeable overhead
>
>
> 2 - with net_ns compiled in
> ---------------------------
>
>
> 2.1 - without container
> -----------------------
>
> ----------- -----------------------------------------------------------
> | Netperf | CPU usage (%) / overhead | Throughput (Mbits/s) / changed |
> ----------- -----------------------------------------------------------
> | on xeon | 6.23 / +4 % | 941.35 / 0 % |
> -----------------------------------------------------------------------
> | on athlon | 28.83 / +2.3 % | 850.76 / +0.7 % |
> -----------------------------------------------------------------------
>
> ----------- ---------------------------------
> | Tbench | Throughput (MBytes/s) / changed |
> ----------- ---------------------------------
> | on xeon | 67.00 / 0 % |
> ---------------------------------------------
> | on athlon | 65.45 / 0 % |
> ---------------------------------------------
>
> Observation : no noticeable overhead
>
>
> 2.2 - inside the container with veth and routes
> -----------------------------------------------
>
> ----------- -----------------------------------------------------------
> | Netperf | CPU usage (%) / overhead | Throughput (Mbits/s) / changed |
> ----------- -----------------------------------------------------------
> | on xeon | 17.14 / +186.1 % | 941.34 / 0 % |
> -----------------------------------------------------------------------
> | on athlon | 49.99 / +77.45 % | 838.85 / +0.7 % |
> -----------------------------------------------------------------------
>
> ----------- ---------------------------------
> | Tbench | Throughput (MBytes/s) / changed |
> ----------- ---------------------------------
> | on xeon | 66.00 / -0.5 % |
> ---------------------------------------------
> | on athlon | 61.00 / -6.65 % |
> ---------------------------------------------
>
> Observation : CPU overhead is very big, throughput is impacted on
> the less powerful machine
>
>
> 2.3 - inside the container with veth and bridge
> -----------------------------------------------
>
> ----------- -----------------------------------------------------------
> | Netperf | CPU usage (%) / overhead | Throughput (Mbits/s) / changed |
> ----------- -----------------------------------------------------------
> | on xeon | 19.14 / +299 % | 941.18 / 0 % |
> -----------------------------------------------------------------------
> | on athlon | 49.98 / +77.42 % | 831.65 / -1.5 % |
> -----------------------------------------------------------------------
>
> ----------- ---------------------------------
> | Tbench | Throughput (MBytes/s) / changed |
> ----------- ---------------------------------
> | on xeon | 64.00 / -3.5 % |
> ---------------------------------------------
> | on athlon | 60.07 / -8.3 % |
> ---------------------------------------------
>
> Observation : CPU overhead is very big, throughput is impacted on
> the less powerful machine
>
>
> # bench from Eric's patchset
> ============================
>
>
> 1 - with net_ns compiled out
> ----------------------------
>
> ----------- -----------------------------------------------------------
> | Netperf | CPU usage (%) / overhead | Throughput (Mbits/s) / changed |
> ----------- -----------------------------------------------------------
> | on xeon | 6.04 / +0.8 % | 941.33 / 0 % |
> -----------------------------------------------------------------------
> | on athlon | 28.45 / +1 % | 840.76 / -0.5 % |
> -----------------------------------------------------------------------
>
> ----------- ---------------------------------
> | Tbench | Throughput (MBytes/s) / changed |
> ----------- ---------------------------------
> | on xeon | 65.69 / -1 % |
> ---------------------------------------------
> | on athlon | 65.35 / -0.2 % |
> ---------------------------------------------
>
> Observation : no noticeable overhead
>
>
> 2 - with net_ns compiled in
> ---------------------------
>
>
> 2.1 - without container
> -----------------------
>
> ----------- -----------------------------------------------------------
> | Netperf | CPU usage (%) / overhead | Throughput (Mbits/s) / changed |
> ----------- -----------------------------------------------------------
> | on xeon | 6.02 / +0.5 % | 941.34 / 0 % |
> -----------------------------------------------------------------------
> | on athlon | 27.93 / -0.8 % | 833.53 / -1.3 % |
> -----------------------------------------------------------------------
>
> ----------- ---------------------------------
> | Tbench | Throughput (MBytes/s) / changed |
> ----------- ---------------------------------
> | on xeon | 66.00 / -0.5 % |
> ---------------------------------------------
> | on athlon | 64.94 / -0.9 % |
> ---------------------------------------------
>
> Observation : no noticeable ove
...
|
|
|
Re: L2 network namespace benchmarking [message #18037 is a reply to message #18035] |
Wed, 28 March 2007 11:52 |
ebiederm
Messages: 1354 Registered: February 2006
|
Senior Member |
|
|
Daniel Lezcano <daniel.lezcano@free.fr> writes:
> Eric W. Biederman wrote:
>> Daniel Lezcano <dlezcano@fr.ibm.com> writes:
>>
>>> 3. General observations
>>> -----------------------
>>>
>>> The objective to have no performances degrations, when the network
>>> namespace is off in the kernel, is reached in both solutions.
>>>
>>> When the network is used outside the container and the network
>>> namespace are compiled in, there is no performance degradations.
>>>
>>> Eric's patchset allows to move network devices between namespaces and
>>> this is clearly a good feature, missing in the Dmitry's patchset. This
>>> feature helps us to see that the network namespace code does not add
>>> overhead when using directly the physical network device into the
>>> container.
>>
>> Assuming these results are not contradicted this says that the extra
>> dereference where we need it does not add measurable to the overhead
>> in the Linus network stack. Performance wise this should be good
>> enough to allow merging the code into the linux kernel, as it does
>> not measurably affect networking when we do not have multiple
>> containers in use.
>
> I have a few questions about merging code into the linux kernel.
>
> * How do you plan to do that ?
One small comprehensible piece at a time.
Basically some variant of etun should not be a problem to merge
then I have to get some part of the network namespace code merged,
and the concept accepted.
Once the basic acceptance occurs it just becomes a long slog of
merging more and more patches.
> * When do you expect to have the network namespace into mainline ?
My current goal is to finish my rebase against 2.6.linus_lastest in
the next couple of days after having figured out how to deal with sysfs.
I have been doing reviewing in more code then I know what to do with,
and fighting some very strange bugs during the stabilization window.
Which has kept me from doing additional development. Plus I have
had a cold.
> * Are Dave Miller and Alexey Kuznetov aware of the network namespace ?
Aware yes, reviewed not yet. I believe Alexey is a little more
familiar with the OpenVZ work. The high level concepts still apply.
> * Did they saw your patchset or ever know it exists ?
Yes.
> * Do you have any feedbacks from netdev about the network namespace ?
Not really. Except that Dave Miller wanted to review what I posted
last time but the timing was bad and he failed to get around to it.
>> To be fully satisfactory how we get the packets to the namespace
>> still appears to need work.
>>
>> We have overhead in routing. That may simply be the cost of
>> performing routing or there may be some optimizations opportunities
>> there.
>> We have about the same overhead when performing bridging which I
>> actually find more surprising, as the bridging code should involve
>> less packet handling.
>
> Yep. I will try to figure out what is happening.
Thanks.
>> Ideally we can optimize the bridge code or something equivalent to
>> it so that we can take one look at the destination mac address and
>> know which network namespace we should be in. Potentially moving this
>> work to hardware when the hardware supports multiple queues.
>>
>> If we can get the overhead out of the routing code that would be
>> tremendous. However I think it may be more realistic to get the
>> overhead out of the ethernet bridging code where we know we don't need
>> to modify the packet.
>
> The routing was optimized for the loopback, no ? Why can't we do the same for
> the etun device ?
I have no problem with it if we can use valid optimizations. Avoiding a
packet copy when the packet is marked as having a second copy somewhere
else does not sound like a valid optimization to me.
Routing through both network namespaces so that we can set up a dst
cache entry that takes you to the final destination I am will to
working with. Perhaps something that hits this piece of the etun driver,
so we don't have to make a second set of routing decisions.
if (skb->dst)
skb->dst = dst_pop(skb->dst); /* Allow for smart routing */
tcpdump at any phase of the process should be able to do the right thing.
Mostly I care right now in that it is interesting to know where the
performance overhead is coming from. Unless it is something of a
merge stopper I don't much care about how we are going to fix it yet,
especially if it is only cross network namespace traffic.
If I read the results right it took a 32bit machine from AMD with
a gigabit interface before you could measure a throughput difference.
That isn't shabby for a non-optimized code path.
Eric
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
|
Re: L2 network namespace benchmarking [message #18041 is a reply to message #18036] |
Wed, 28 March 2007 07:07 |
Daniel Lezcano
Messages: 417 Registered: June 2006
|
Senior Member |
|
|
Herbert Poetzl wrote:
> On Wed, Mar 28, 2007 at 12:16:34AM +0200, Daniel Lezcano wrote:
>> Hi,
[ cut ]
>> 3. General observations
>> -----------------------
>>
>> The objective to have no performances degrations, when the network
>> namespace is off in the kernel, is reached in both solutions.
>>
>> When the network is used outside the container and the network
>> namespace are compiled in, there is no performance degradations.
>>
>> Eric's patchset allows to move network devices between namespaces and
>> this is clearly a good feature, missing in the Dmitry's patchset. This
>> feature helps us to see that the network namespace code does not add
>> overhead when using directly the physical network device into the
>> container.
>>
>> The loss of performances is very noticeable inside the container and
>> seems to be directly related to the usage of the pair device and the
>> specific network configuration needed for the container. When the
>> packets are sent by the container, the mac address is for the pair
>> device but the IP address is not owned by the host. That directly
>> implies to have the host to act as a router and the packets to be
>> forwarded. That adds a lot of overhead.
>>
>> A hack has been made in the ip_forward function to avoid useless
>> skb_cow when using the pair device/tunnel device and the overhead
>> is reduced by the half.
>
> would it be possible to do some tests regarding scalability?
>
> i.e. I would be interested how the following would look like:
>
> 10 connections on a single host (in parallel, overall performance)
> 10 connections from the same net space
> 10 connections from 10 different net spaces
> (i.e. one connection from each space)
>
> we can assume that L3 isolation will give similar results to
> the first case, but if needed, we can provide a patch to
> test this too ...
>
Ok. Assuming, Eric's and Dmitry's patchset are very similar, I will
focus on the Eric's patchset because it is more mature and more easy to
setup. I will have a look on the bridge optimization before doing that.
>
> PS: great work! tx!
>
Thanks.
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Re: L2 network namespace benchmarking [message #18044 is a reply to message #18037] |
Wed, 28 March 2007 18:08 |
Rick Jones
Messages: 4 Registered: March 2007
|
Junior Member |
|
|
> If I read the results right it took a 32bit machine from AMD with
> a gigabit interface before you could measure a throughput difference.
> That isn't shabby for a non-optimized code path.
Just some paranoid ramblings - one needs to look beyond just whether or
not the performance of a bulk transfer test (eg TCP_STREAM) remains able
to hit link-rate. One has to also consider the change in service demand
(the normalization of CPU util and throughput). Also, with
functionality like TSO in place, the ability to pass very large things
down the stack can help cover for a multitude of path-length sins. And
with either multiple 1G or 10G NICs becoming more and more prevalent, we
have another one of those "NIC speed vs CPU speed" switch-overs, so
maintaining single-NIC 1 gigabit throughput, while necessary, isn't
(IMO) sufficient.
Soooo, it becomes very important to go beyond just TCP_STREAM tests when
evaluating these sorts of things. Another test to run would be the
TCP_RR test. TCP_RR with single-byte request/response sizes will
"bypass" the TSO stuff, and the transaction rate will be more directly
affected by the change in path length than a TCP_STREAM test. It will
also show-up quite clearly in the service demand. Now, with NICs doing
interrupt coalescing, if the NIC is strapped "poorly" (IMO) then you may
not see a change in transaction rate - it may be getting limited
artifically by the NIC's interrupt coalescing. So, one has to fall-back
on service demand, or better yet, disable the interrupt coalescing.
Otherwise, measuring peak aggregate request/response becomes necessary.
rick jones
don't be blinded by bit-rate
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
|
Re: L2 network namespace benchmarking [message #18054 is a reply to message #18044] |
Wed, 28 March 2007 19:47 |
Daniel Lezcano
Messages: 417 Registered: June 2006
|
Senior Member |
|
|
Rick Jones wrote:
>> If I read the results right it took a 32bit machine from AMD with
>> a gigabit interface before you could measure a throughput difference.
>> That isn't shabby for a non-optimized code path.
>
> Just some paranoid ramblings - one needs to look beyond just whether
> or not the performance of a bulk transfer test (eg TCP_STREAM) remains
> able to hit link-rate. One has to also consider the change in service
> demand (the normalization of CPU util and throughput). Also, with
> functionality like TSO in place, the ability to pass very large things
> down the stack can help cover for a multitude of path-length sins.
> And with either multiple 1G or 10G NICs becoming more and more
> prevalent, we have another one of those "NIC speed vs CPU speed"
> switch-overs, so maintaining single-NIC 1 gigabit throughput, while
> necessary, isn't (IMO) sufficient.
>
> Soooo, it becomes very important to go beyond just TCP_STREAM tests
> when evaluating these sorts of things. Another test to run would be
> the TCP_RR test. TCP_RR with single-byte request/response sizes will
> "bypass" the TSO stuff, and the transaction rate will be more directly
> affected by the change in path length than a TCP_STREAM test. It will
> also show-up quite clearly in the service demand. Now, with NICs
> doing interrupt coalescing, if the NIC is strapped "poorly" (IMO) then
> you may not see a change in transaction rate - it may be getting
> limited artifically by the NIC's interrupt coalescing. So, one has to
> fall-back on service demand, or better yet, disable the interrupt
> coalescing.
>
> Otherwise, measuring peak aggregate request/response becomes necessary.
>
>
> rick jones
> don't be blinded by bit-rate
Thanks Rick,
Do you have any pointer to help on benchmarking the network, perhaps a
checklist or some scripts for netperf ?
Regards.
-- Daniel
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Re: L2 network namespace benchmarking [message #18055 is a reply to message #18035] |
Thu, 29 March 2007 13:01 |
ebiederm
Messages: 1354 Registered: February 2006
|
Senior Member |
|
|
Benjamin Thery <benjamin.thery@bull.net> writes:
> Eric W. Biederman wrote:
>> Daniel Lezcano <daniel.lezcano@free.fr> writes:
>
> [...]
>
>>> * When do you expect to have the network namespace into mainline ?
>> My current goal is to finish my rebase against 2.6.linus_lastest in
>> the next couple of days after having figured out how to deal with sysfs.
>
> Great news!
> I also have some questions about this updated version:
>
> - Have you integrated the bug fixes and cleanups(*) Daniel wrote for
> your previous netns patchset (and the few glitches I reported too)?
About half of them so far. It is my intention to incorporate all of them.
They weren't all trivial to include. A couple of Daniel's patches
address a real issue in the wrong way so I have to give them some more
thought.
> (*) available in LXC8 patchset
>
> - Do you already have a public git repository set up for the rebase?
> - If it is private, any plan to make it public soon? (That would be great)
Yes. Where the current one is now.
>> I have been doing reviewing in more code then I know what to do with,
>> and fighting some very strange bugs during the stabilization window.
>> Which has kept me from doing additional development. Plus I have
>> had a cold.
>
> I hope you're getting better... and you'll be able to provide us the
> updated patchset very soon :)
Hopefully. I think I have fixed my last non network regression I know
about for 2.6.21-rcX. Which means I can begin to focus again.
> [...]
>
>> If I read the results right it took a 32bit machine from AMD with
>> a gigabit interface before you could measure a throughput difference.
>> That isn't shabby for a non-optimized code path.
>
> Indeed the throughput difference is not significant.
> This is very good to see that it stays constant when using the container.
> What I'm more worried about is the CPU load increase. But it seems
> we've identified some of the culprits.
Yes, and the good news is that they all seem to be in getting the
packets to the network namespace.
> This afternoon I had a look at why the bridge setup isn't better than
> the route setup (section 2.3 and 2.4 of Daniel's report).
>
> In the bridge case, we encounter the same problems as the routes case.
> The oprofile profile is the same: the most demanding routines are
> pskb_expand_head and csum_partial_copy_generic.
> pskb_expand_head() is also called by skb_cow(), but this time
> skb_cow() is called by netfilter's nf_bridge_copy_header().
>
> We can avoid this copy by removing option CONFIG_BRIDGE_NETFILTER.
> This copy is made even if netfilter is not used on the host.
> Maybe some optimizations can be made in netfilter's code to prevent this.
Sounds reasonable. I guess the next step is to get some numbers with
CONFIG_BRIDGE_NETFILTER disabled. (So we don't hit that case and just
in case there are more). I suspect the bridging code has a small
enough user base right now it just hasn't been optimized much.
Eric
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Re: L2 network namespace benchmarking [message #18062 is a reply to message #18037] |
Thu, 29 March 2007 07:37 |
Benjamin Thery
Messages: 79 Registered: March 2007
|
Member |
|
|
Eric W. Biederman wrote:
> Daniel Lezcano <daniel.lezcano@free.fr> writes:
[...]
>> * When do you expect to have the network namespace into mainline ?
> My current goal is to finish my rebase against 2.6.linus_lastest in
> the next couple of days after having figured out how to deal with sysfs.
Great news!
I also have some questions about this updated version:
- Have you integrated the bug fixes and cleanups(*) Daniel wrote for
your previous netns patchset (and the few glitches I reported too)?
(*) available in LXC8 patchset
- Do you already have a public git repository set up for the rebase?
- If it is private, any plan to make it public soon? (That would be great)
> I have been doing reviewing in more code then I know what to do with,
> and fighting some very strange bugs during the stabilization window.
> Which has kept me from doing additional development. Plus I have
> had a cold.
I hope you're getting better... and you'll be able to provide us the
updated patchset very soon :)
[...]
> If I read the results right it took a 32bit machine from AMD with
> a gigabit interface before you could measure a throughput difference.
> That isn't shabby for a non-optimized code path.
Indeed the throughput difference is not significant.
This is very good to see that it stays constant when using the container.
What I'm more worried about is the CPU load increase. But it seems
we've identified some of the culprits.
This afternoon I had a look at why the bridge setup isn't better than
the route setup (section 2.3 and 2.4 of Daniel's report).
In the bridge case, we encounter the same problems as the routes case.
The oprofile profile is the same: the most demanding routines are
pskb_expand_head and csum_partial_copy_generic.
pskb_expand_head() is also called by skb_cow(), but this time
skb_cow() is called by netfilter's nf_bridge_copy_header().
We can avoid this copy by removing option CONFIG_BRIDGE_NETFILTER.
This copy is made even if netfilter is not used on the host.
Maybe some optimizations can be made in netfilter's code to prevent this.
Regards,
Benjamin
--
B e n j a m i n T h e r y - BULL/DT/Open Software R&D
http://www.bull.com
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/containers
|
|
|
Goto Forum:
Current Time: Sun Oct 13 04:12:32 GMT 2024
Total time taken to generate the page: 0.06258 seconds
|