OpenVZ Forum: Support » Unable to stop a container from blocking the host

Home » General » Support » Unable to stop a container from blocking the host (A MySQL index creation in a container is blocking everything else.)

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Unable to stop a container from blocking the host [message #40259]

Fri, 06 August 2010 08:46

rsauvat
Messages: 5
Registered: May 2010

Junior Member

Hi,

On a production server I have setup 8 virtual machine. They share the 16Go of RAM on the host but only using 5Go when running. My problem is when I start an index creation in MySQL on a particular VM after 2-3 minutes the whole host stop responding correctly. I can see in top a lot of processes waiting for I/O and the load on the host goes up to 300. I have the ioprio parameter to 0 for this vm and 7 for other but the problem is still the same. The process take a really long time to respond to a kill -9, more than 10 minutes.

The mysql table is really big (29Go) so the index creation must be I/O consuming. But a iostat on the host show no more than a 1Mo/s on I/O.

I am running Proxmox 1.5 with debian 5 on the host.

root@HWnode:~# pveversion -v
pve-manager: 1.5-10 (pve-manager/1.5/4822)
running kernel: 2.6.24-11-pve
proxmox-ve-2.6.24: 1.5-23
pve-kernel-2.6.18-2-pve: 2.6.18-5
pve-kernel-2.6.24-11-pve: 2.6.24-23
pve-kernel-2.6.24-10-pve: 2.6.24-21
qemu-server: 1.1-16
pve-firmware: 1.0-5
libpve-storage-perl: 1.0-13
vncterm: 0.9-2
vzctl: 3.0.23-1pve11
vzdump: 1.2-5
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1dso1
pve-qemu-kvm: 0.12.4-1

This problem is very annoying because I can't create any indexes in MySQL. And this defies the purpose of containers. How a single container can make the others unstable. I can understand a container slowing down the others but in this case even postfix daemon on an other container can't write mail to the user mailbox.

If anyone has an idea on what is causing this problem any help will be appreciated.

Best.

Report message to a moderator

Re: Unable to stop a container from blocking the host [message #40277 is a reply to message #40259]

Mon, 09 August 2010 08:04

rsauvat
Messages: 5
Registered: May 2010

Junior Member

I may have found the source of the problem. Due to the software RAID implementation by my reseller it seams that the number of fsync/second is very low compared to a standard disk. So this seams to be the problem since the mysql database might be using it very often so the other nodes don't have enough fsync for themselves.

So this leads me to a question. Why the number of fsync is not limited per VM? Or at least there should be a priority to let other containers do their fsync maybe based on ioprio.

Report message to a moderator

Re: Unable to stop a container from blocking the host [message #40346 is a reply to message #40277]

Fri, 13 August 2010 13:25

maratrus
Messages: 1495
Registered: August 2007
Location: Moscow

Senior Member

Hello,

I do understand your problem. So I would like to recommend you the following way: if your server is located in a reliable place i.e. there is no electricity problem you can disable fsyncs inside the VE by writing 0 to /proc/sys/fs/fsync-enable on the HN. This means that all fsyncs inside VE would be very fast as fsync actually doesn't happen. Again, it wouldn't be a problem if your server never go down because of the power issues.

Report message to a moderator

Previous Topic:	DDOS in VM
Next Topic:	Missing virtio drivers in 2.6.27-kuindzi.1

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Thu Nov 06 01:11:47 GMT 2025

Total time taken to generate the page: 0.09454 seconds