OpenVZ Forum


Home » General » Support » openvz6 ploop backups and mysql innodb corruption
openvz6 ploop backups and mysql innodb corruption [message #53204] Mon, 12 March 2018 19:06 Go to next message
Jcats is currently offline  Jcats
Messages: 15
Registered: May 2011
Location: FL
Junior Member
From: *cmdnnj.fios.verizon.net
Hello,

We are suffering from a critical issue with ploop backups.

Basically, in some instances, when restoring a ploop backup, MySQL fails to start because of Innodb corruption, example:

2018-03-12 18:04:35 7fd3d4864900 InnoDB: Error: page 818 log sequence number 71622558711
InnoDB: is in the future! Current system log sequence number 71620133300.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recove ry.html
InnoDB: for more information.

We use the following method to perform the backups:

https://openvz.org/Ploop/Backup#Image-based_backup

What would be the proper way to perform the backups to ensure we do not run into this problem? We fear if we ever had fatal issue with the node where all containers needed to be restored we would be stuck with many containers with innodb corruption.

[Updated on: Mon, 12 March 2018 19:06]

Report message to a moderator

Re: openvz6 ploop backups and mysql innodb corruption [message #53206 is a reply to message #53204] Tue, 13 March 2018 03:35 Go to previous messageGo to next message
Jcats is currently offline  Jcats
Messages: 15
Registered: May 2011
Location: FL
Junior Member
From: *cmdnnj.fios.verizon.net
Follow up question on this and sorry if this is a dumb question but..

In https://openvz.org/Ploop/Backup#Image-based_backup

vzctl snapshot $CTID --id $ID --skip-suspend --skip-config
cp $VE_PRIVATE/root.hdd/* $BACKUPPATH/

Should we be including the snapshot in the backup itself? Wouldn't copying the snapshot be copying where any new data is taking place which is possibly why we are seeing the corruption? Maybe I'm not understand the snapshot but if we instead did..


vzctl snapshot $CTID --id $ID --skip-suspend --skip-config
tar -cvf backup.tar --exclude='./dump' --exclude='./root.hdd/root.hdd.*' $VE_PRIVATE/root.hdd/

then deleted the snapshot.

Wouldn't this method 'freeze' root.hdd basically ensuring no data will change within it while we backup the image to ensure the db integrity?

Re: openvz6 ploop backups and mysql innodb corruption [message #53254 is a reply to message #53206] Sun, 29 April 2018 20:06 Go to previous messageGo to next message
websavers is currently offline  websavers
Messages: 15
Registered: March 2018
Location: Halifax, NS
Junior Member
From: *eastlink.ca
Yes, this is how I understand it to work as well.

Presumably you would either need to 1) exclude the snapshot differentials like you've shown, OR 2) backup and restore all the files as they show, but then upon restore, also snapshot-switch it to the pre-snapshot-taken state.

I'm afraid I haven't tested restoring like this yet, but will be quite soon.
Re: openvz6 ploop backups and mysql innodb corruption [message #53289 is a reply to message #53254] Sat, 19 May 2018 01:33 Go to previous message
websavers is currently offline  websavers
Messages: 15
Registered: March 2018
Location: Halifax, NS
Junior Member
From: *eastlink.ca
This has now been tested and I can confirm that:

1. If you copy the ploop drive without first snapshotting, it *will* be marked dirty and refuse to mount
2. If you snapshot prior to copying, the backup/dupe *will* be mountable and data is fully intact and restorable
3. You can simply ignore the snapshot files (granted skipping them during backup will mean bandwidth and storage savings). You can simply mount the ploop disk file directly (ignoring the snapshots) or to recover the whole container, edit the XML file and remove the snapshot references.
Previous Topic: OpenVZ7 - Migrate options: vzmigrate, ovztransfer.sh or prlctl migrate?
Next Topic: OpenVZ Maintenance Partnership program
Goto Forum:
  


Current Time: Thu Dec 13 09:41:55 GMT 2018