OpenVZ Forum: Support » OpenVZ HWNODE becomes unresponsive

Home » General » Support » OpenVZ HWNODE becomes unresponsive

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

OpenVZ HWNODE becomes unresponsive [message #38272]

Thu, 03 December 2009 09:52

defiancenl
Messages: 15
Registered: June 2009

Junior Member

I have a couple of HNodes running openvz.
One of these nodes has a weird issue, wich i cannot seem to debug/resolve.
Let me first list the hardware and relevant info

Kernel : 2.6.24-23-openvz

CPU : Intel C2Q Q6600

MEM : 4GB DDR2

HDD : 160GB

All in all this node runs fine, however one time per 3 weeks/month the node becomes unresponsive. And cannot be reached with ssh.(also the vz's are unresponsive and cannot be reached)
All that is left is to boot it using the APC.
After reboot the server is ok, and runs fine for another 3weeks/month.

After a reboot i start digging trough the logs but nowhere, and i mean nowhere is there a hint why it became unresponsive

/var/log/messages

Dec  2 21:25:35 hn01 -- MARK --
Dec  2 21:45:35 hn01 -- MARK --
Dec  2 21:57:40 hn01 syslogd 1.5.0#1ubuntu1: restart.
Dec  2 21:57:40 hn01 kernel: Inspecting /boot/System.map-2.6.24-23-openvz
Dec  2 21:57:40 hn01 kernel: Loaded 29198 symbols from /boot/System.map-2.6.24-23-openvz.

21:57 i booted the server via apc

in no single log file i can see why the server became unrepsonsive.

Can someone help me out and tell me how to best debug this server, or does anyone know this behaviour?

I am not oversetting the limits on the hwnode

root@hn01:~# vzmemcheck -v 
Output values in %
veid        LowMem  LowMem     RAM MemSwap MemSwap   Alloc   Alloc   Alloc
              util  commit    util    util  commit    util  commit   limit
101           1.04  105.00    1.76    0.90   35.41    1.00   35.41   38.50
104           2.13   35.00    4.93    2.54   32.39    4.66   32.39   35.47
102           0.82   52.50    1.07    0.55   33.14    2.24   33.14   36.23
-------------------------------------------------------------------------
Summary:      4.00  192.50    7.75    3.99  100.94    7.90  100.94  110.20

I see another topic on this forum with the exact same problem.
Can this be a issue in the kernel?

[Updated on: Thu, 03 December 2009 10:00]

Report message to a moderator

Re: OpenVZ HWNODE becomes unresponsive [message #38294 is a reply to message #38272]

Fri, 04 December 2009 14:15

maratrus
Messages: 1495
Registered: August 2007
Location: Moscow

Senior Member

Hi,

- the kernel that you have in use is quite old. Is it possible to use RHEL5 based kernel instead?
- nevertheless, it can be the kernel problem. Is it possible to install serial console to be able to gather all logs
http://wiki.openvz.org/Remote_console_setup
When the system is locked try Alt-Sysrq-*
- p (twice the number of CPUs)
- w (several times)
- t (for all processes' calltraces. Please, note this is a time consuming operation.)

Please, don't hesitate to file a new bug report.
http://bugzilla.openvz.org/
It is the best way to have your problem fixed.

Report message to a moderator

Re: OpenVZ HWNODE becomes unresponsive [message #38296 is a reply to message #38294]

Fri, 04 December 2009 14:51

defiancenl
Messages: 15
Registered: June 2009

Junior Member

You say the kernel is quite old.....
However if i look at the stable RHEL 5 branche the newest kernel is : 2.6.18-164
This looks older to me ?

Or are you suggesting that i use 2.6.27-briullov.1 kernel?

Furthermore, i use the kernel on hn01 on all nodes, and the problems on hn01 are not there on the other nodes.
I will look into remote logging to debug this issue.

Report message to a moderator

Re: OpenVZ HWNODE becomes unresponsive [message #38298 is a reply to message #38296]

Fri, 04 December 2009 15:06

maratrus
Messages: 1495
Registered: August 2007
Location: Moscow

Senior Member

Hi,

Quote:

You say the kernel is quite old.....
However if i look at the stable RHEL 5 branche the newest kernel is : 2.6.18-164
This looks older to me ?

let me explain what I had in mind.
RHEL5 kernel is based on 2.6.18 kernel which is of course older than 2.6.24 but that kernel is constantly being improved by Red Hat and OpenVZ teams. Many security fixes are applied, different drivers are included in that kernel. 2.6.18 is a stable and maintained branch. That is why I suggested using it instead of 2.6.24.

Report message to a moderator