OpenVZ Forum


Home » General » Support » BUG: "prlctl destroy <UUID>" autostarts containers
BUG: "prlctl destroy <UUID>" autostarts containers [message #53710] Wed, 18 November 2020 21:13 Go to next message
mstauber is currently offline  mstauber
Messages: 6
Registered: April 2009
Junior Member
From: *tigo.com.co
Hello,

I just ran into a nasty "split-brain" situation because of something that appears a bug. \o/

Situation:

Two nodes, "master" and "slave". Both have the same CTs. "master" actively runs them, while "slave" has a copy of each CT for backup purposes.

Relevant versions involved:

[root@slave ~]# cat /etc/redhat-release 
Virtuozzo Linux release 7.8

[root@slave ~]# prlctl --version
prlctl version 7.0.201

[root@slave ~]# uname -a
Linux master.xxx.xxx 3.10.0-957.12.2.vz7.96.21 #1 SMP Thu Jun 27 15:10:55 MSK 2019 x86_64 x86_64 x86_64 GNU/Linux

Yes, both nodes haven't been rebooted in a few months, so the running kernel is a bit older. Mea culpa. Uptime is king.

On "slave" I needed to remove an older CT. Which was stopped. In fact all CTs on "slave" were stopped AND had "autostart" off:

[root@slave ~]# prlctl list --info|grep Autostart
Autostart: off
Autostart: off
Autostart: off
Autostart: off
Autostart: off
Autostart: off
Autostart: off
Autostart: off
Autostart: off
Autostart: off

So I ran "prlctl destroy ff717715-916c-497d-b9b5-bfab7cf8e3b5" and the CT in question was gone.

All good? Hell no! Take a look at this:

[root@slave ~]# cat /var/log/vzctl.log|grep "2020-11-18"|egrep "Start|destroyed"
2020-11-18T12:13:54-0600 vzctl : CT ff717715-916c-497d-b9b5-bfab7cf8e3b5 : Container private area was destroyed
2020-11-18T12:14:40-0600 vzctl : CT e8f386c2-08b5-45b9-b3ec-74e825ab082a : Starting Container ...
2020-11-18T12:14:40-0600 vzctl : CT f539028b-89a3-4f48-ad3a-6ea123e84d68 : Starting Container ...
2020-11-18T12:14:40-0600 vzctl : CT 48073f02-aeb7-4be8-911a-94daf2ce410d : Starting Container ...
2020-11-18T12:14:40-0600 vzctl : CT 3e2b0ace-604f-4297-9136-66f0cd0b48dc : Starting Container ...
2020-11-18T12:14:40-0600 vzctl : CT 4d615464-229e-4bfd-9b99-638563d116de : Starting Container ...
[...]

So *right* after destroying ff717715 as desired, without being asked to do so OpenVZ started up all *other* containers. Despite them all having 'autostart' off.

And of course the CTs on "slave" won the ARP-roulette and started serving traffic, which caused a hell of a mess.

I'm sorry, but why does deletion of a CT cause OpenVZ to fire up all stopped CTs even if they have "autostart" off? That's HIGHLY unexpected behavior.


With best regards,

Michael Stauber
Re: BUG: "prlctl destroy <UUID>" autostarts containers [message #53711 is a reply to message #53710] Thu, 19 November 2020 07:09 Go to previous message
khorenko is currently offline  khorenko
Messages: 527
Registered: January 2006
Location: Moscow, Russia
Senior Member
From: *qwerty.ru
Hi Michael,

i failed to reproduce the issue in an easy way:
[root@sbf7 ~]# prlctl list --info|grep Autostart
Autostart: off
Autostart: off
Autostart: off
Autostart: off
Autostart: off
Autostart: off
Autostart: off
Autostart: off
Autostart: off

[root@sbf7 ~]# prlctl list -a
UUID                                    STATUS       IP_ADDR         T  NAME
{e184beed-db92-4b07-971d-e8a4c9bff9f2}  stopped      -               CT 100
{49bd6422-d6e7-4777-baaa-baa908e93f91}  stopped      -               CT 101
{99b797a2-9b50-4a60-a9d7-30d3268c8ed9}  stopped      -               CT 102
{863f365c-bcf0-45b0-84e8-229fac815129}  stopped      -               CT 1024
{e0860e37-19f1-4c97-9374-343244f476fa}  stopped      -               CT 200
{24f9c50f-f9a4-4784-b9da-4975363665fd}  stopped      192.169.9.89    CT 300
{a6fb4d5a-2adb-443c-a01b-c12b12becb29}  stopped      192.169.9.90    CT 301
{bd7900ba-a98d-47ae-b232-4ffd0e12e3b0}  stopped      -               CT 302
{fbb33119-82a4-40a7-9d4e-dd28c3a3ee3e}  stopped      -               CT 500

[root@sbf7 ~]# prlctl destroy fbb33119-82a4-40a7-9d4e-dd28c3a3ee3e
Removing the CT...
The CT has been successfully removed.

[root@sbf7 ~]# prlctl list -a
UUID                                    STATUS       IP_ADDR         T  NAME
{e184beed-db92-4b07-971d-e8a4c9bff9f2}  stopped      -               CT 100
{49bd6422-d6e7-4777-baaa-baa908e93f91}  stopped      -               CT 101
{99b797a2-9b50-4a60-a9d7-30d3268c8ed9}  stopped      -               CT 102
{863f365c-bcf0-45b0-84e8-229fac815129}  stopped      -               CT 1024
{e0860e37-19f1-4c97-9374-343244f476fa}  stopped      -               CT 200
{24f9c50f-f9a4-4784-b9da-4975363665fd}  stopped      192.169.9.89    CT 300
{a6fb4d5a-2adb-443c-a01b-c12b12becb29}  stopped      192.169.9.90    CT 301
{bd7900ba-a98d-47ae-b232-4ffd0e12e3b0}  stopped      -               CT 302


So please file a bug at bugs.openvz.org and provide full logs: vzctl.log and prl* logs.


If you problem is solved - please, report it!
It's even more important than reporting the problem itself...
Previous Topic: OpenVZ 7 OOM killing systemd in containers
Next Topic: Bug reports should go to bugs.openvz.org
Goto Forum:
  


Current Time: Wed Dec 02 05:56:14 GMT 2020