| VDS with Debian 5 performs much worse than with Debian 3. [message #38711] |
Mon, 18 January 2010 20:51  |
tryl
Messages: 4
Registered: January 2010
Location: Denmark
|
Junior Member |
|
|
I'm transfering my sites to a VDS running Debian 5 from a Debian 3 VDS. When the site with the most load was transfered, the new server started to response very slow, where it performed very well on the old server.
The wird thing is, that in this period the CPU (acording to top) was running at 90%-95% idle, with peaks of 85% idle. Disc-wise I could very fast grep through a folder with a lot of files and dd an about 200 MiB file to /dev/null with about 40 MiB/s. At the same time, there were 120 MiB of memory free. Apache would take about respond 10 sec to respond to any request to a dynamic generated webpage (PHP or ssi) and the same for the first serve of small static files (e.g. robots.txt). On subsequent serves, these small static files would be lightning fast. There is no significant swap and load average is never above 0.20. On the old server, the apache processes would acoring to top (the RES column) rarely consume more than 10 MB of memory, where the new ones easily takes 20 MB and some times more. That is with the same memory settings in apache2.conf and php.ini.
From this I conclude:
The problem is not CPU overload.
The problem is not lack of memory.
The problem is not slow disk access.
The problem is not lack of free httpd processes.
I.e. the problem does not exist and I have absolutely no clue why the new server starts to slow down.
The script which runs the most loaded site, is from zipstat.org and can be browsed at http://zipstat.cvs.sourceforge.net/viewvc/zipstat/zipstat/ (start looking at zipstat.php in the root) and is developed by myself.
The old server runs PHP Version 4.3.10-19 with MySQL 4.0.24 and Apache 2.0 on a Debian 3.1 VDS with kernel 2.6.18-028stab066.11.
The new server runs PHP Version 5.2.6-1+lenny3 with the Suhosin Patch 0.9.6.2, MySQL 5.0.51a and Apache 2.0 on a Debian 5 VDS with kernel 2.6.18-028stab066.11 (yep, the same).
I hope somebody can help or give me a pointer (!= 0) in a right direction. And please, just ask for more info or for me to run tests.
Best regards,
Simon Mikkelsen, Denmark
|
|
|
|
|
|
| Re: VDS with Debian 5 performs much worse than with Debian 3. [message #38749 is a reply to message #38735] |
Sun, 24 January 2010 09:51  |
tryl
Messages: 4
Registered: January 2010
Location: Denmark
|
Junior Member |
|
|
Hello Maratrus
failcnt are all 0.
Iptables does not seem to have any special setup:
zip:~# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
zip:~# iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
The performance issue also exists when I just try to wget a static file (like /) on the server.
I have looked at the configuration files of Apache, MySQL and PHP and with regard to memory (limits, number of processes etc.) there are no differences. However, in top, each apache process uses 20-30 MB on the new server, where they are all on 10 MB on the old server.
The obvious difference between the two are the Debian 3/5 versions and PHP 4/5 versions. Unfortunaly PHP 4 is not supported under Debian 5, so I cannot try that. What I plan to try is to run the service from one server and use the database of the other server - and switch that around.
I run the server at a payed provider (yourshelter.net) so I can easily put another configuration on a server...there is just the catch that I rent a server for a whole month and I on production sites the 2 servers I got. I spend a weekend creating accounts and reinstalling apps on the new server, so I really do not want to trash it.
When the load is not at the worst on the new server, after having waited 3-4 sec for e.g. the static file /index.html to load, then it is lightening fast for subsequent requests. The same is also true with regard to access to webapps that uses the database. It is just like that file or that part of the database file is cached in memory. Letting it sit idle for a while it will be slow one time again.
Running on full load, there does not seem to be such caching.
However, top shows at least 250 MB free memory all the time and that is enough to fit half of my databases in memory - the web apps in total takes about 1 MB.
I've thought if that 250 MB is not enough to cache enough data to keep the system fast. Does the 10-20 MB extra for each apache process really make the difference?
I have the indices on the database required to not do full table scans and a read test during the worst load, yielded 40 MB/s when dd'ing an old 250 MB file into /dev/null.
Setting the maximum number of apache processes to either 5 or 20 does not make a difference.
I'm really stuck here. Some things points to memory issues, but others tell otherwise. The same is true for disk speed.
|
|
|
|
| Solved: Re: VDS with Debian 5 performs much worse than with Debian 3. [message #38939 is a reply to message #38711] |
Sun, 21 February 2010 18:01 |
tryl
Messages: 4
Registered: January 2010
Location: Denmark
|
Junior Member |
|
|
I found the problem and the solution:
I the PHP scripts with the most hits I used gethostbyaddr() to get the host name of the web client. It is an advanced web counter, so I have to do this.
The problem doing it with that function is that it is relatively heavy on the system, but that had not been a problem on several PHP 4 installations over several years.
With PHP 5 that changed. Usually there were no problems, but some times a DNS request would take a long time or just hang. Maybe that hanging request blocked for others or maybe Apache had to launch more processes and maybe it hit the ceiling so it could not create more.
It probably only hit 1 of 100 requests, but with the amount of traffic I have, that quickly built up within minutes.
Solution?
<?php
$ip = long2ip(ip2long($this->ipAddr)*1);
$ip = escapeshellarg($ip);
$out = shell_exec("host --quick --timeout=2 $ip");
// Parse the host name.
$out = split("\n", $out);
if (count($out) >= 2) {
$out = split(" ", $out[0]);
if (count($out) >= 2) {
$name = $out[1];
}
}
?>
where the IP address is in $this->ipAddr and the host name ends up in $name if it can be set. The script runs a process calling the version of the host command that exists on my system. Other systems may have other versions which would require the parse block to be different.
The implementation is a little paranoid: I convert the IP address to a number, multiply by 1 (so it will always be a integer) and converts it back again. Even with that, I do what you always must: Escape anything from the user which goes into a a shell, file name, SQL etc.
|
|
|
|
|
|
OpenVZ Forum
Members
Search
Help
Register
Login
Home
