Hi During testing and debugging of a problem on Debian systems (which I later found out was corrected in 3.0.13 version) I found that the rsync commands in vzmigrate can give quite unexpected results. The command looks basically like this: rsync -aH ${from} ${to} This means that files are copied from ${from} to ${to}, but if files already exist on ${to} from some older copy the old files that do not exist on ${from} will not be deleted. This is a problem if --keep-dst or --online option is used. There are two places where the rsync command is used for transfer of filesystem data. The first one is right after 'log 1 "Syncing private"'. If --keep-dst is used this is a problem. The solution is to add the --delete option to rsync. The second one is right after 'log 2 "Syncing private (2nd pass)"' and it is a problem if --online option is used and a file has been deleted between this run and the first rsync run. The solution is to add the --delete option to rsync. The following patch has been applied to the Debian (experimental version) packaged version of vzctl. Patched against 3.0.13: --- vzctl-3.0.13.orig/vzmigrate +++ vzctl-3.0.13/vzmigrate @@ -391,7 +391,7 @@ fi log 1 "Syncing private" -if ! rsync -aH --progress "$VE_PRIVATE" "root@$host:${VE_PRIVATE%/*}" | \ +if ! rsync -aH --delete --progress "$VE_PRIVATE" "root@$host:${VE_PRIVATE%/*}" | \ grep "% of" | awk -v ORS="\r" '{print $10}'; then log 1 "Failed to sync VE private areas" undo_quota_on @@ -444,7 +444,7 @@ if [ "$state" = "running" ]; then log 2 "Syncing private (2nd pass)" time_rsync2=$(date +%s.%N) - if ! rsync -aH "$VE_PRIVATE" "root@$host:${VE_PRIVATE%/*}"; then + if ! rsync -aH --delete "$VE_PRIVATE" "root@$host:${VE_PRIVATE%/*}"; then log 1 "Failed to sync VE private areas" undo_source_stage exit $MIG_ERR_COPY Will attach as well if the cut and paste made it non-applieable. The patch is licensed under GNU/GPL. Regards, // Ola -- --------------------- Ola Lundqvist --------------------------- / opal@debian.org Annebergsslingan 37 \ | ola@opalsys.net 654 65 KARLSTAD | | +46 (0)54-10 14 30 +46 (0)70-332 1551 | | http://opalsys.net/ UIN/icq: 4912500 | \ gpg/f.p.: 7090 A92B 18FE 7994 0C36 4FE4 18A1 B1CF 0FE5 3DD9 / ------------------------------------------------------------ ---
Re: vzmigrate filesystem rsync correction
Hi Sorry forgot to attach. Here it is. I have also thinked again and there are more cases when this is a problem. On "-r no" with failed sync and in any case when there are files on the destination already. In any case the patch solves this problem. Regards, // Ola On Tue, Jan 02, 2007 at 09:11:17PM +0100, Ola Lundqvist wrote: > Hi > > During testing and debugging of a problem on Debian systems > (which I later found out was corrected in 3.0.13 version) I found > that the rsync commands in vzmigrate can give quite unexpected > results. > > The command looks basically like this: > rsync -aH ${from} ${to} > > This means that files are copied from ${from} to ${to}, but if > files already exist on ${to} from some older copy the old files > that do not exist on ${from} will not be deleted. > > This is a problem if --keep-dst or --online option is used. > > There are two places where the rsync command is used for transfer > of filesystem data. > > The first one is right after 'log 1 "Syncing private"'. If --keep-dst > is used this is a problem. The solution is to add the --delete option > to rsync. > > The second one is right after 'log 2 "Syncing private (2nd pass)"' > and it is a problem if --online option is used and a file has been > deleted between this run and the first rsync run. > The solution is to add the --delete option to rsync. > > The following patch has been applied to the Debian (experimental version) > packaged version of vzctl. Patched against 3.0.13: > > --- vzctl-3.0.13.orig/vzmigrate > +++ vzctl-3.0.13/vzmigrate > @@ -391,7 +391,7 @@ > fi > > log 1 "Syncing private" > -if ! rsync -aH --progress "$VE_PRIVATE" "root@$host:${VE_PRIVATE%/*}" | \ > +if ! rsync -aH --delete --progress "$VE_PRIVATE" "root@$host:${VE_PRIVATE%/*}" | \ > grep "% of" | awk -v ORS="\r" '{print $10}'; then > log 1 "Failed to sync VE private areas" > undo_quota_on > @@ -444,7 +444,7 @@ > if [ "$state" = "running" ]; then > log 2 "Syncing private (2nd pass)" > time_rsync2=$(date +%s.%N) > - if ! rsync -aH "$VE_PRIVATE" "root@$host:${VE_PRIVATE%/*}"; then > + if ! rsync -aH --delete "$VE_PRIVATE" "root@$host:${VE_PRIVATE%/*}"; then > log 1 "Failed to sync VE private areas" > undo_source_stage > exit $MIG_ERR_COPY > > Will attach as well if the cut and paste made it non-applieable. > > The patch is licensed under GNU/GPL. > > Regards, > > // Ola > > -- > --------------------- Ola Lundqvist --------------------------- > / opal@debian.org Annebergsslingan 37 \ > | ola@opalsys.net 654 65 KARLSTAD | > | +46 (0)54-10 14 30 +46 (0)70-332 1551 | > | http://opalsys.net/ UIN/icq: 4912500 | > \ gpg/f.p.: 7090 A92B 18FE 7994 0C36 4FE4 18A1 B1CF 0FE5 3DD9 / > ------------------------------------------------------------ --- > -- --------------------- Ola Lundqvist --------------------------- / opal@debian.org Annebergsslingan 37 \ | ola@opalsys.net 654 65 KARLSTAD | | +46 (0)54-10 14 30 +46 (0)70-332 1551 | | http://opalsys.net/ UIN/icq: 4912500 | \ gpg/f.p.: 7090 A92B 18FE 7994 0C36 4FE4 18A1 B1CF 0FE5 3DD9 / ------------------------------------------------------------ --- --- vzctl-3.0.13.orig/vzmigrate +++ vzctl-3.0.13/vzmigrate @@ -391,7 +391,7 @@ fi log 1 "Syncing private" -if ! rsync -aH --progress "$VE_PRIVATE" "root@$host:${VE_PRIVATE%/*}" | \ +if ! rsync -aH --delete --progress "$VE_PRIVATE" "root@$host:${VE_PRIVATE%/*}" | \ grep "% of" | awk -v ORS="\r" '{print $10}'; then log 1 "Failed to sync VE private areas" undo_quota_on @@ -444,7 +444,7 @@ if [ "$state" = "running" ]; then log 2 "Syncing private (2nd pass)" time_rsync2=$(date +%s.%N) - if ! rsync -aH "$VE_PRIVATE" "root@$host:${VE_PRIVATE%/*}"; then + if ! rsync -aH --delete "$VE_PRIVATE" "root@$host:${VE_PRIVATE%/*}"; then log 1 "Failed to sync VE private areas" undo_source_stage exit $MIG_ERR_COPY