OpenVZ Forum: Support » After upgrade, host reboots when trying to start container

Home » General » Support » After upgrade, host reboots when trying to start container

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

After upgrade, host reboots when trying to start container [message #51277]

Sun, 30 March 2014 18:10

joepal1976
Messages: 3
Registered: March 2014

Junior Member

I just did a yum update to get:

vzctl version 4.6.1

Linux vz.home.jwp.se 2.6.32-042stab085.20 #1 SMP Fri Mar 21 19:07:05 MSK 2014 x86_64 x86_64 x86_64 GNU/Linux

Up until this upgrade everything was working smoothly. However, now I can't start containers. The host reboots as soon as I do a vzctl start.

Since the host is remote, I don't see if there are any console messages, but there is nothing in the kern log, which makes me believe it's an instant freeze+reboot.

Please advise.

Report message to a moderator

Re: After upgrade, host reboots when trying to start container [message #51279 is a reply to message #51277]

Sun, 30 March 2014 23:03

Ales
Messages: 330
Registered: May 2009

Senior Member

You could:

- try your current vzctl with an older vzkernel (2.6.32-042stab084.26 for example, last of the previous stable ..084.xx series)
- downgrade vzctl (did you have 4.5.1 or 4.6 before?), you can find older versions here https://openvz.org/Download/vzctl/4.5.1 (links to all versions are listed at the bottom of the page)
- check if there are any relevant bugs reported for your vzctl/vzkernel combination and if not, file one once you have more information.

The odd thing is, vzctl 4.6.1 has been out since November 2013, that's why I'm suspecting it might be a vzkernel/vzctl combination (or even plain vzkernel) that gives you trouble.

[Updated on: Sun, 30 March 2014 23:04]

Report message to a moderator

Re: After upgrade, host reboots when trying to start container [message #51281 is a reply to message #51277]

Mon, 31 March 2014 18:12

Paparaciz
Messages: 302
Registered: August 2009

Senior Member

Hi,

can it be related to this one:

https://lists.openvz.org/pipermail/users/2014-March/005537.h tml

and related bug report:

https://bugzilla.openvz.org/show_bug.cgi?id=2926

Report message to a moderator

Re: After upgrade, host reboots when trying to start container [message #51282 is a reply to message #51279]

Mon, 31 March 2014 18:20

joepal1976
Messages: 3
Registered: March 2014

Junior Member

Thank you, I will try that kernel.

For now I booted up on the kernel I had previously. This was rather old (2.6.32-042stab076.5) since I have been reluctant do do the kernel upgrades in the first place.

This way the containers started, but shared memory stopped working within them. Any application wanting to allocate shared memory crashed with an "out of space" error.

I'm assuming this comes down to mixing new userland libraries with an old kernel, but that's just my guess.

Anyway, I'm in disaster recovery mode. Once I've migrated away the containers so I know I can start them somewhere else if more stuff hits the fan, I'll try the kernel you suggest.

Report message to a moderator

Re: After upgrade, host reboots when trying to start container [message #51283 is a reply to message #51281]

Mon, 31 March 2014 18:23

joepal1976
Messages: 3
Registered: March 2014

Junior Member

Possibly. It's the best guess I've seen so far.

It would have been a lot easier if I could see the kernel output, but nothing is flushed to the kernel log when the event happens.

(and since I get the host machine back a bit later, I'm just assuming it's an insta-reboot)

Report message to a moderator

Re: After upgrade, host reboots when trying to start container [message #51284 is a reply to message #51277]

Mon, 31 March 2014 18:35

Paparaciz
Messages: 302
Registered: August 2009

Senior Member

to see kernel crash logs you can:

https://wiki.openvz.org/Remote_console_setup

of course if this possible.
I remember did few times netconsole setup and if I remember everything it was only modprobe and testing part in sending side and netcat (nc) in receiving side.

Report message to a moderator

Re: After upgrade, host reboots when trying to start container [message #51339 is a reply to message #51277]

Tue, 22 April 2014 05:30

iansison
Messages: 17
Registered: March 2008

Junior Member

HI all. I've been bitten by this bug as well - I cannot vzctl start a container without the entire server rebooting on me.

This happens with kernels

042stab085.20ovz,
042stab088.4ovz, using vzctl 4.7

Strangely enough

042stab076.8ovz using the same vzctl 4.7 correctly starts containers (no panic, no reboot).

However 042stab076.8ovz has a USB 3.x bug which panics with newer motherboards, so i cannot downgrade Sad

I went on to enable the netconsole so that i can get an idea what is printed on the screen before it reboots, but after all the prep work for netconsole to work, to my surprise, no logs are sent to the netconsole server before the forced reboot. So i got nothing.

I hope someone out there has a solution to offer! Is there anything else i can do to isolate out the problem? HELP!!!

System - CentOS 5.10
Hardware: MSI Z87-G45, Intel(R) Core(TM) i7-4770, 16GB RAM
I also transplated the disks to another system (intel Core-i7-3770, ASUS board) with the same results, so it's not my hardware.

Report message to a moderator

Re: After upgrade, host reboots when trying to start container [message #51340 is a reply to message #51277]

Tue, 22 April 2014 13:08

Ales
Messages: 330
Registered: May 2009

Senior Member

Did you try any of the other kernels in between? Or older vzctl / libs?

For example, 2.6.32-042stab084.26 works nicely for us (stuck there because of an unrelated libvirt bug).

From the available information, I can't see any common configuration points between all of you that have this problem...

What could it be, NFS, RHEL/CentOS 5.x, a part of HW?

I'm not sure what to suggest here, apart from going to the last known working configuration and then testing cautiously.

Report message to a moderator

Re: After upgrade, host reboots when trying to start container [message #51341 is a reply to message #51340]

Wed, 23 April 2014 01:39

iansison
Messages: 17
Registered: March 2008

Junior Member

I've tried 042stab084.17ovz and this one works. Luckily for me this has some fixes for USB3 so i can now use this on the mobo i have.

As for common configuration points, i'd say that the most common intersection is the kernel version, and after 042stab084.17ovz for me, things stopped working already.

Report message to a moderator

Re: After upgrade, host reboots when trying to start container [message #51383 is a reply to message #51277]

Sat, 03 May 2014 10:41

non7top
Messages: 5
Registered: October 2013

Junior Member

I'm seeing the same behavior and I'm stuck at 2.6.32-042stab084.26. This appears to happen on all kernels older then 2.6.32-042stab084.x. And I don't use ct snapshots/restore.
The symptom is simple: boot system, vzctl start xxx and server instantly restarts.
Mostly strange that on another server I have 2.6.32-042stab085.20 running without issues.

Report message to a moderator

Re: After upgrade, host reboots when trying to start container [message #51390 is a reply to message #51277]

Mon, 05 May 2014 10:49

dmc_dtc
Messages: 15
Registered: May 2014
Location: Serbia

Junior Member

I am having the same issue on all 5 of my servers running OpenVZ, after upgrading to vzctl 4.6.x/4.7 and 85+ kernel and rebooting into 88.4 kernel system just reboots ...

I tried a bit of experimenting and if i disable containers i boot into the kernel and the dom0 machine works fine, but as soon as i try to start VZ container it reboots ..

I tried to add VE_LAYOUT=simfs to /etc/vz/vz.conf thinking that it could be related since it was known to be big change in vzctl ... but it didnt help

Any solution to this bug?

>> dmc / dtc <<

[Updated on: Mon, 05 May 2014 14:00]

Report message to a moderator

Re: After upgrade, host reboots when trying to start container [message #51394 is a reply to message #51277]

Tue, 06 May 2014 06:06

Paparaciz
Messages: 302
Registered: August 2009

Senior Member

you have to know how to reproduce this, than submit a bug report.

All who have this problem should report exact kernel, os, tools versions. both HN and CT.
maybe something specific in CT?
maybe something in all setup?

currently I don't have such problem with
1.
2.6.32-042stab088.4
vzctl version 4.7
HN: Centos6
CTs: Centos6

2.
2.6.32-042stab088.4
vzctl version 4.7.2
HN: Centos6
CTs: Centos5

Report message to a moderator

Re: After upgrade, host reboots when trying to start container [message #51505 is a reply to message #51339]

Wed, 02 July 2014 01:42

iansison
Messages: 17
Registered: March 2008

Junior Member

Answering my own post, i've finally found out the cause of this problem (at least on my servers). I've filed a bug report on the openvz bugzilla

https://bugzilla.openvz.org/show_bug.cgi?id=3010

In a nutshell, the sysctl setting

kernel.pid_ns_hide_child = 1

which i used to hide the pids of containers from the HN ps list causes the kernel panic/reboot.

Setting this to 0 will solve the problem, at least for me.

I have replicated this problem on the stab090.5 kernel, vzctl 4.7.2 on both CentOS 5.10 and 6.5

Report message to a moderator

Previous Topic:	OpenVZ on OVH vRack (Multiple subnet on same IF)
Next Topic:	OVZ does not delete root and private directores in some cases

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Mar 31 13:17:51 GMT 2023

Total time taken to generate the page: 0.01107 seconds