Hi,

I have a VE in production for a couple of months. HN and VE are running 
Ubuntu Dapper 6.06.2 LTS

> vz01:~$ uname -a
> Linux vz01 2.6.18-028stab053 #1 SMP Tue May 27 15:40:07 CEST 2008 x86_64 GNU/Linux
> vz01:~$ vzctl --version
> vzctl version 3.0.22

This VE is only used for http(s) services. Everything have worked well 
until this week end where we got apache process stalled on CLOSE_WAIT 
state and no more process are available to deliver pages

We get this in logs :

> server reached MaxClients setting, consider raising the MaxClients setting

If I set MaxClients to a higher value, that is just delaying the moment 
where apache fall in a stalled state.

I gave a look into user beancounter but no hard limit or barrier limit 
was reached when it occurred.

since it's a production server I can't "play" too much with. I don't 
know if it's a fully openvz related but I restarted the server on a 
physical blade and the problem has been fixed.

For now, the server is running on a physical blade, but I can reboot it 
as a VE server to do some test.

The server is behind a reverse proxy cluster (lighthttp) but nothing has 
been change on those servers for a couple of weeks.

Note : In our architecture, each physical or virtual server is format 
after each reboot and populated with a clear boot image. After that, we 
are using cfengine to apply the right configuration function of the name 
of the server. I can boot a server on a physical or a virtual server 
with exactly the same configuration even if specific openvz 
configuration (such as not start ntp server for example).

> ii  apache2                2.0.55-4ubuntu2.3            next generation, scalable, extendable web se
> ii  apache2-common         2.0.55-4ubuntu2.3            next generation, scalable, extendable web se
> ii  apache2-mpm-prefork    2.0.55-4ubuntu2.3            traditional model for Apache2
> ii  apache2-utils          2.0.55-4ubuntu2.3            utility programs for webservers
> ii  libapache2-mod-php5    5.1.2-1ubuntu3.10            server-side, HTML-embedded scripting languag
> ii  libc6                  2.3.6-0ubuntu20.5            GNU C Library: Shared libraries and Timezone
> ii  php5-cgi               5.1.2-1ubuntu3.10            server-side, HTML-embedded scripting languag
> ii  php5-common            5.1.2-1ubuntu3.10            Common files for packages built from the php
> ii  php5-mcrypt            5.1.2-1                      MCrypt module for php5
> ii  php5-mysql             5.1.2-1ubuntu3.10            MySQL module for php5
> ii  php5-mysqli            5.1.2-1ubuntu3.10            MySQL Improved module for php5

Thanks for you help.

Regards,

Yoann

On 23 June 2008 14:13:04 Yoann Moulin wrote:
> Hi,
>
> I have a VE in production for a couple of months. HN and VE are running
> Ubuntu Dapper 6.06.2 LTS
>
> > vz01:~$ uname -a
> > Linux vz01 2.6.18-028stab053 #1 SMP Tue May 27 15:40:07 CEST 2008 x86_64
> > GNU/Linux vz01:~$ vzctl --version
> > vzctl version 3.0.22
>
> This VE is only used for http(s) services. Everything have worked well
> until this week end where we got apache process stalled on CLOSE_WAIT
> state and no more process are available to deliver pages
>
> We get this in logs :
> > server reached MaxClients setting, consider raising the MaxClients
> > setting

Any suspicious messages in dmesg? Please show output of the commands (in 
apache's CT):

  cat /proc/net/netstat
  cat /proc/user_beancounters
  netstat --tcp -nop
 
>
> If I set MaxClients to a higher value, that is just delaying the moment
> where apache fall in a stalled state.

Maybe these articles can help you:

http://www.usenet-forums.com/linux-web-servers/5368-need-help-server-reached-maxclients-setting.html
https://issues.apache.org/bugzilla/show_bug.cgi?id=41252

>
> I gave a look into user beancounter but no hard limit or barrier limit
> was reached when it occurred.
>
> since it's a production server I can't "play" too much with. I don't
> know if it's a fully openvz related but I restarted the server on a
> physical blade and the problem has been fixed.
>
> For now, the server is running on a physical blade, but I can reboot it
> as a VE server to do some test.
>
> The server is behind a reverse proxy cluster (lighthttp) but nothing has
> been change on those servers for a couple of weeks.
>
> Note : In our architecture, each physical or virtual server is format
> after each reboot and populated with a clear boot image. After that, we
> are using cfengine to apply the right configuration function of the name
> of the server. I can boot a server on a physical or a virtual server
> with exactly the same configuration even if specific openvz
> configuration (such as not start ntp server for example).
>
> > ii  apache2                2.0.55-4ubuntu2.3            next generation,
> > scalable, extendable web se ii  apache2-common         2.0.55-4ubuntu2.3 
> >           next generation, scalable, extendable web se ii 
> > apache2-mpm-prefork    2.0.55-4ubuntu2.3            traditional model for
> > Apache2 ii  apache2-utils          2.0.55-4ubuntu2.3            utility
> > programs for webservers ii  libapache2-mod-php5    5.1.2-1ubuntu3.10     
> >       server-side, HTML-embedded scripting languag ii  libc6             
> >     2.3.6-0ubuntu20.5            GNU C Library: Shared libraries and
> > Timezone ii  php5-cgi               5.1.2-1ubuntu3.10           
> > server-side, HTML-embedded scripting languag ii  php5-common           
> > 5.1.2-1ubuntu3.10            Common files for packages built from the php
> > ii  php5-mcrypt            5.1.2-1                      MCrypt module for
> > php5 ii  php5-mysql             5.1.2-1ubuntu3.10            MySQL module
> > for php5 ii  php5-mysqli            5.1.2-1ubuntu3.10            MySQL
> > Improved module for php5
>
> Thanks for you help.
>
> Regards,
>
> Yoann
-- 
Thank,
Vitaliy Gusev

Hi,

>> I have a VE in production for a couple of months. HN and VE are running
>> Ubuntu Dapper 6.06.2 LTS
>>
>>> vz01:~$ uname -a
>>> Linux vz01 2.6.18-028stab053 #1 SMP Tue May 27 15:40:07 CEST 2008 x86_64
>>> GNU/Linux vz01:~$ vzctl --version
>>> vzctl version 3.0.22
>> This VE is only used for http(s) services. Everything have worked well
>> until this week end where we got apache process stalled on CLOSE_WAIT
>> state and no more process are available to deliver pages
>>
>> We get this in logs :
>>> server reached MaxClients setting, consider raising the MaxClients
>>> setting
> 
> Any suspicious messages in dmesg? Please show output of the commands (in 
> apache's CT):
> 
>   cat /proc/net/netstat
>   cat /proc/user_beancounters
>   netstat --tcp -nop

I have restarted the server as a VE, but I'm not able to reproduce that 
issue... if it occures again, I will get those informations

>> If I set MaxClients to a higher value, that is just delaying the moment
>> where apache fall in a stalled state.
> 
> Maybe these articles can help you:
> 
> http://www.usenet-forums.com/linux-web-servers/5368-need-help-server-reached-maxclients-setting.html
> https://issues.apache.org/bugzilla/show_bug.cgi?id=41252

thanks

Regards,

Yoann Moulin