OpenVZ Forum


Home » General » Support » Kernel Panic with 028stab064.4 ? (Anyone seeing this?)
Kernel Panic with 028stab064.4 ? [message #37138] Fri, 21 August 2009 06:38 Go to next message
subigo is currently offline  subigo
Messages: 7
Registered: August 2009
Junior Member
Hey guys,

Here's my problem...

Basically, we have a node running on CentOS 5.3 64-bit and it has been solid forever. And then 2-3 days ago we upgraded the kernel from 028stab062.3 to 028stab064.4.

About 24-48 hours after the upgrade, the node crashed. The datacenter rebooted it and we checked the logs... nothing. So exactly 24 hours after the first crash, it happened again. I asked the datacenter to give me all of the information they had and I was told:

"Looks like a kernel panic related to a crond process, something about __posix_lock_file_conf".

A few hours later, it crashed again.

The problem is I just can't pinpoint where this is coming from. There are no oops messages, no conflicts, nothing starting right before the crash, nothing. There's simply nothing being logged when this happens.

Anyway, I know this is a long-shot, but has anyone ever experienced a problem related to "posix_lock_file_conf"? I've done the obvious and reverted to the old kernel for now, which so far has remained stable.

The node is a quad-core with 8GB ram and normally runs at about 0.10 to 0.40 with around 6GB of ram actually used.

So yeah, has anyone had any issues with this latest kernel?
Re: Kernel Panic with 028stab064.4 ? [message #37140 is a reply to message #37138] Fri, 21 August 2009 07:56 Go to previous messageGo to next message
khorenko is currently offline  khorenko
Messages: 533
Registered: January 2006
Location: Moscow, Russia
Senior Member
Hi.

i'm afraid you faced some new issue, so we are very interested in details on it.

Could you please configure a serial console on the node (or at least - network console)
http://wiki.openvz.org/Remote_console_setup

Unfortunately without logs we can do nothing so we have to wait next crash - please, collect the logs, take picture of the oops on the local console and file a bug in bugzilla for this issue.

Good luck!

--
Konstantin


If your problem is solved - please, report it!
It's even more important than reporting the problem itself...
Re: Kernel Panic with 028stab064.4 ? [message #37141 is a reply to message #37138] Fri, 21 August 2009 08:44 Go to previous messageGo to next message
kir is currently offline  kir
Messages: 1645
Registered: August 2005
Location: Moscow, Russia
Senior Member

You need to gather console logs (full kernel oops message etc.), see http://wiki.openvz.org/Remote_console_setup. This is very important to get this info.

When done, file a bug report directly to bugzilla.openvz.org.

And no, I haven't seen any complaints like this so far.


Kir Kolyshkin
http://static.openvz.org/userbars/openvz-developer.png
Re: Kernel Panic with 028stab064.4 ? [message #37147 is a reply to message #37138] Fri, 21 August 2009 22:05 Go to previous messageGo to next message
subigo is currently offline  subigo
Messages: 7
Registered: August 2009
Junior Member
Thanks for the suggestions everyone... Unfortunately I don't have the ability to setup something like serial console. The only thing I can really do is ask the datacenter to take a picture next time if they can.

However, I'm in no hurry to submit a bug, because since rolling back to the old kernel everything has been stable (knock on wood). I'll try to recreate this on a test server if I can.
Re: Kernel Panic with 028stab064.4 ? [message #37168 is a reply to message #37138] Mon, 24 August 2009 14:45 Go to previous messageGo to next message
Jimbo is currently offline  Jimbo
Messages: 9
Registered: November 2008
Junior Member
Hello,

We saw what looks like this exact same crash this morning. We're on RHEL 5.3, kernel 2.6.18-128.2.1.el5.028stab064.4. Unfortunately I do not have a picture or the complete text of the panic, however I wrote down the stack trace on a notepad.

the process was sshd. We had some rdiff-backups running at the time over ssh on the physical machine, however it is also possible that the sshd process in question was running in a VE.

The stack trace was:
__posix_lock_file_conf
fnctl_setlk
do_sigaction
sys_fnctl
system_call

and the actual function that crashed it:
locks_alloc_lock

I'll try to setup remote console so that I can give you the complete stack trace if this happens again.
Re: Kernel Panic with 028stab064.4 ? [message #37432 is a reply to message #37138] Thu, 10 September 2009 13:38 Go to previous messageGo to next message
andre is currently offline  andre
Messages: 36
Registered: January 2008
Member
Hello,

We are using same kernel and it crashed on the same hour three consecutive days. Since we were not sure if it was hardware related we moved all VE's to another hardware (with a different kernel).

I believe we have the same issue here. Couldn't get kernel message because it is a colocated box. Is this issue fixed on newer kernel?

Now I have netconsole enabled on this server which had problems but could not reproduce the crash (after we moved all VE's). Even running stress tests for 10+ hours everything seems fine (hardware seems to be 100%).
Re: Kernel Panic with 028stab064.4 ? [message #37433 is a reply to message #37432] Thu, 10 September 2009 13:42 Go to previous messageGo to next message
kir is currently offline  kir
Messages: 1645
Registered: August 2005
Location: Moscow, Russia
Senior Member

Again, without kernel console logs we can't say or do much. Please reproduce the crash, get the logs and file a bug to http://bugzilla.openvz.org/

Thanks!


Kir Kolyshkin
http://static.openvz.org/userbars/openvz-developer.png
Re: Kernel Panic with 028stab064.4 ? [message #37434 is a reply to message #37138] Thu, 10 September 2009 14:01 Go to previous message
Jimbo is currently offline  Jimbo
Messages: 9
Registered: November 2008
Junior Member
kernel 028stab064.7 fixes a crash bug:
http://wiki.openvz.org/Download/kernel/rhel5/028stab064.7

I'm hoping that it resolves the issue that I had. I haven't had any problems since upgrading to it.
Previous Topic: howto compile kernel with some special features ? [SOLVED]
Next Topic: VLAN IP routing issue in containers
Goto Forum:
  


Current Time: Thu Mar 28 19:28:08 GMT 2024

Total time taken to generate the page: 0.01759 seconds