OpenVZ Forum: Support » Online migration "Error: undump failed: Invalid argument"

Home » General » Support » Online migration "Error: undump failed: Invalid argument"

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Online migration "Error: undump failed: Invalid argument" [message #15099]

Thu, 19 July 2007 07:45

whity
Messages: 5
Registered: July 2007
Location: Switzerland

Junior Member

I did some tests with OpenVZ this week. Now im testing the online migration and run into a problem i could not solve yet.

Both systems have the same OS:
- System: CentOS 5 (x86_64)
- Kernel: 2.6.18-8.1.4.el5.028stab035.1 (from yum repo)

- Source: 172.30.101.24 (00:1B:24:1D:EA:F4)
- Target: 172.30.101.96 (00:16:36:D2:F0:FE)

Output of vzmigration:

[root@ovz02 ~]# vzmigrate -r yes --online -v 172.30.101.96 1001
OPT:-r
OPT:--online
OPT:-v
OPT:172.30.101.96
Starting online migration of VE 1001 on 172.30.101.96
OpenVZ is running...
    Loading /etc/vz/vz.conf and /etc/vz/conf/1001.conf files
    Check IPs on destination node: 172.30.101.26
Preparing remote node
    Copying config file
1001.conf                                                                                                                          100% 1809     1.8KB/s   00:00
Saved parameters for VE 1001
    Creating remote VE root dir
    Creating remote VE private dir
Initializing remote quota
    Quota init
    Turning remote quota on
Syncing private
Live migrating VE
    Suspending VE
Setting up checkpoint...
        suspend...
        get context...
Checkpointing completed succesfully
    Dumping VE
Setting up checkpoint...
        join context..
        dump...
Checkpointing completed succesfully
    Copying dumpfile
dump.1001                                                                                                                          100%  127MB  11.5MB/s   00:11
    Syncing private (2nd pass)
Syncing 2nd level quota
    Dumping 2nd level quota
    Copying 2nd level quota
quotadump.1001                                                                                                                     100%   21     0.0KB/s   00:00
    Load 2nd level quota
    Undumping VE
Restoring VE ...
Starting VE ...
vzquota : (warning) Quota is running for id 1001 already
VE is mounted
        undump...
Adding IP address(es): 172.30.101.26
Setting CPU units: 36185
Configure meminfo: 262144
Error: undump failed: Invalid argument
Restoring failed:
do_rst_mm 64331744
rst_mm: -22
make_baby: -22
rst_clone_children
Stopping VE ...
VE start failed
VE was stopped
VE is unmounted
Error:  Failed to undump VE
Resuming...
arpsend: 172.30.101.26 is detected on another computer : 00:16:36:d2:f0:fe
vps-net_add WARNING: arpsend -c 1 -w 1 -D  -e 172.30.101.26 eth0 FAILED
vzquota : (error) Quota is not running for id 1001

Kernel log:

Jul 19 09:20:21 ovz02 kernel: CPT DBG: ffff81010618f000,1001: cpt_dump_vm: 1750 shared private anon pages
Jul 19 09:20:21 ovz02 kernel: CPT DBG: ffff81010618f000,1001: cpt_dump_vm: 12857 anon pages are cloned
Jul 19 09:20:21 ovz02 kernel: CPT DBG: ffff81010618f000,1001: cpt_dump_vm: 6 silly pages canceled
Jul 19 09:20:23 ovz02 kernel: CPT DBG: cpt itimer 12667,13691(syslogd) 10002626000
Jul 19 09:20:23 ovz02 kernel: CPT DBG: ffff81010618f000,1001: 12800,13824(pcscd) 104919000
Jul 19 09:20:23 ovz02 kernel: CPT DBG: cpt itimer 13056,14080(mysqld) 27997593863000
Jul 19 09:20:23 ovz02 kernel: CPT DBG: cpt itimer 13151,14175(httpsd) 315268800184000
Jul 19 09:20:23 ovz02 kernel: CPT DBG: ffff81010618f000,1001: 13174,14198(crond) 41725421000
Jul 19 09:20:23 ovz02 kernel: CPT DBG: ffff81010618f000,1001: 13193,14217(atd) 230325730000

vzctl.log (Source):

2007-07-19T09:34:34+0200 vzctl : VE 1001 : Setting up checkpoint...
2007-07-19T09:34:34+0200 vzctl : VE 1001 :      suspend...
2007-07-19T09:34:34+0200 vzctl : VE 1001 :      get context...
2007-07-19T09:34:34+0200 vzctl : VE 1001 : Checkpointing completed succesfully
2007-07-19T09:34:34+0200 vzctl : VE 1001 : Setting up checkpoint...
2007-07-19T09:34:34+0200 vzctl : VE 1001 :      join context..
2007-07-19T09:34:34+0200 vzctl : VE 1001 :      dump...
2007-07-19T09:34:35+0200 vzctl : VE 1001 : Running: /usr/lib64/vzctl/scripts/vps-net_del
2007-07-19T09:34:35+0200 vzctl : VE 1001 : Checkpointing completed succesfully
2007-07-19T09:34:47+0200 vzctl : VE 1001 : Resuming...
2007-07-19T09:34:47+0200 vzctl : VE 1001 : Running: /usr/lib64/vzctl/scripts/vps-net_add
2007-07-19T09:34:48+0200 vzctl : VE 1001 :      put context

vzctl.log (Target):

2007-07-19T09:33:59+0200 vzctl : VE 1001 : Saved parameters for VE 1001
2007-07-19T09:34:00+0200 vzctl : VE 1001 : Running: /usr/sbin/vzquota init 1001 -b 10485760 -B 12582912 -i 200000 -I 220000 -p /vz/private/1001 -e 0 -n 0 -s 0
2007-07-19T09:34:00+0200 vzctl : VE 1001 : Running: /usr/sbin/vzquota show 1001
2007-07-19T09:34:00+0200 vzctl : VE 1001 : Running: /usr/sbin/vzquota on 1001 -r 0 -b 10485760 -B 12582912 -i 200000 -I 220000 -e 0 -n 0 -s 0
2007-07-19T09:37:12+0200 vzctl : VE 1001 : Restoring VE ...
2007-07-19T09:37:12+0200 vzctl : VE 1001 : Starting VE ...
2007-07-19T09:37:12+0200 vzctl : VE 1001 : Running: /usr/sbin/vzquota show 1001
2007-07-19T09:37:12+0200 vzctl : VE 1001 : Running: /usr/sbin/vzquota on 1001 -r 0 -b 10485860 -B 12583012 -i 200100 -I 220100 -e 0 -n 0 -s 0
2007-07-19T09:37:12+0200 vzctl : VE 1001 : Running: vzquota setlimit 1001 -p /vz/private/1001 -b 10485860 -B 12583012 -i 200100 -I 220100 -e 0 -n 0
2007-07-19T09:37:12+0200 vzctl : VE 1001 : Mounting root: /vz/root/1001 /vz/private/1001
2007-07-19T09:37:12+0200 vzctl : VE 1001 : VE is mounted
2007-07-19T09:37:13+0200 vzctl : VE 1001 : Adding IP address(es): 172.30.101.26
2007-07-19T09:37:13+0200 vzctl : VE 1001 : Running: /usr/lib64/vzctl/scripts/vps-net_add
2007-07-19T09:37:13+0200 vzctl : VE 1001 : Setting CPU units: 36185
2007-07-19T09:37:13+0200 vzctl : VE 1001 : Configure meminfo: 262144
2007-07-19T09:37:13+0200 vzctl : VE 1001 : Running: /usr/sbin/vzquota stat 1001 -f
2007-07-19T09:37:13+0200 vzctl : VE 1001 : Running: vzquota setlimit 1001 -b 10485760 -B 12582912 -i 200000 -I 220000 -e 0 -n 0
2007-07-19T09:37:13+0200 vzctl : VE 1001 : VE start failed
2007-07-19T09:37:13+0200 vzctl : VE 1001 : Stopping VE ...
2007-07-19T09:37:13+0200 vzctl : VE 1001 : VE was stopped
2007-07-19T09:37:13+0200 vzctl : VE 1001 : Running: /usr/sbin/vzquota stat 1001 -f
2007-07-19T09:37:13+0200 vzctl : VE 1001 : Running: vzquota setlimit 1001 -b 10485760 -B 12582912 -i 200000 -I 220000 -e 0 -n 0
2007-07-19T09:37:13+0200 vzctl : VE 1001 : Running: /usr/sbin/vzquota stat 1001 -f
2007-07-19T09:37:13+0200 vzctl : VE 1001 : Running: /usr/sbin/vzquota off 1001
2007-07-19T09:37:14+0200 vzctl : VE 1001 : VE is unmounted

Any ideas??

Thank you very much
Urs

Report message to a moderator

Re: Online migration "Error: undump failed: Invalid argument" [message #15101 is a reply to message #15099]

Thu, 19 July 2007 08:11

Vasily Tarasov
Messages: 1345
Registered: January 2006

Senior Member

Hello,

As far as I understand you have atached the kernel log from source node. Can you post here the kernel logs at destination also?

Thank you,
Vasily.

Report message to a moderator

Re: Online migration "Error: undump failed: Invalid argument" [message #15106 is a reply to message #15101]

Thu, 19 July 2007 08:42

whity
Messages: 5
Registered: July 2007
Location: Switzerland

Junior Member

Hi Vasily

Yes, was the kernel log of the source. Forgot the one from the target:

Jul 19 10:34:35 ovz01 kernel: VE: 1001: started
Jul 19 10:34:36 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: JFixup 139010 168949078000
Jul 19 10:34:37 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: 12734,1(init)open RDONLY fifo ino 392664 ffff810109b11bc0 1180
Jul 19 10:34:43 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:48 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:48 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:48 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:48 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:48 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:48 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:48 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:48 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:48 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:48 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:48 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:48 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:49 ovz01 last message repeated 4 times
Jul 19 10:34:49 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:49 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:49 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:49 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:49 ovz01 last message repeated 2 times
Jul 19 10:34:49 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:49 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:49 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:49 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:49 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:49 ovz01 last message repeated 2 times
Jul 19 10:34:49 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:49 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:50 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:50 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:50 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:50 ovz01 last message repeated 2 times
Jul 19 10:34:50 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:50 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:50 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:50 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:50 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:50 ovz01 last message repeated 2 times
Jul 19 10:34:50 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:50 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: bsd lock restored
Jul 19 10:34:50 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: 12778,9832(qmail-send)open RDONLY fifo ino 396584 ffff8100be07c5c0 1192
Jul 19 10:34:50 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:50 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:50 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:50 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:51 ovz01 last message repeated 6 times
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:51 ovz01 last message repeated 6 times
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: deleted file created as /tmp/ibK1GLR3, ffff8100bd816380, 8180
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: deleted file created as /tmp/ibtmNmrP, ffff8100bd816280, 8180
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: deleted file created as /tmp/ibjM9X0A, ffff8100bd816180, 8180
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: deleted file created as /tmp/ibgci0Am, ffff8100bd816080, 8180
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: deleted file created as /tmp/ibhHdhm8, ffff8100bd433cc0, 8180
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: restoring file data block: 00000000-00001000
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: restoring file data block: 00000000-00055000
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: restoring file data block: 00000000-00001000
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: restoring file data block: 00000000-00002000
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: restoring file data block: 00011000-00012000
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:51 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:51 ovz01 last message repeated 3 times
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: rst_file: file obtained by dentry_open
Jul 19 10:34:52 ovz01 last message repeated 2 times
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: file is attached to a socket
Jul 19 10:34:52 ovz01 last message repeated 35 times
Jul 19 10:34:52 ovz01 kernel: CPT ERR: ffff8101194b1800,1001 :do_rst_mm 22770776
Jul 19 10:34:52 ovz01 kernel: CPT ERR: ffff8101194b1800,1001 :rst_mm: -22
Jul 19 10:34:52 ovz01 kernel: CPT ERR: ffff8101194b1800,1001 :make_baby: -22
Jul 19 10:34:52 ovz01 kernel: CPT ERR: ffff8101194b1800,1001 :rst_clone_children
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12734/1 ffff81007f44b180
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12755/8150 ffff81011e101780
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12757/9563 ffff81011e774c40
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12756/9560 ffff81011f5a6800
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12760/9766 ffff81011f13eb00
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12758/9661 ffff81011e3a6c40
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12759/9739 ffff81011e3a6c40
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12792/9964 ffff8100beeebb80
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12790/9898 ffff8100be7340c0
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12789/9897 ffff8100be7340c0
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12788/9896 ffff8100be7340c0
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12787/9895 ffff8100be7340c0
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12786/9894 ffff8100be7340c0
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12784/9837 ffff8100be734440
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12783/9836 ffff8100be7347c0
Jul 19 10:34:52 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12782/9835 ffff8100be734b40
Jul 19 10:34:53 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12805/10037 ffff8100beeeb100
Jul 19 10:34:53 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12781/9834 ffff81010ef4d780
Jul 19 10:34:53 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12806/10041 ffff8100bd399bc0
Jul 19 10:34:53 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12785/9893 ffff8100be7340c0
Jul 19 10:34:53 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12778/9832 ffff81011df46c40
Jul 19 10:34:53 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12776/9822 ffff81011e78b780
Jul 19 10:34:53 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12812/10047 ffff8100bcf8d500
Jul 19 10:34:53 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12761/9777 ffff81011f688b40
Jul 19 10:34:53 ovz01 kernel: CPT DBG: ffff8101194b1800,1001: leaked through 12814/10058 ffff8100bcacfc40

...

[ Show the rest of the message ]

Report message to a moderator

Re: Online migration "Error: undump failed: Invalid argument" [message #15117 is a reply to message #15106]

Thu, 19 July 2007 09:59

Vasily Tarasov
Messages: 1345
Registered: January 2006

Senior Member

Hmmm... It seems to be some checkpointing problem. Can you, please, create an archive of your VE (/vz/private/<veid>/) and put this archive somethere, where I'll be able to download it?

Then I'll try to reproduce your problem locally and give it to the creators of checkpointing.

Thank you.

Report message to a moderator

Previous Topic:	SOLVED NFS again: ls: reading directory .: Input/output error
Next Topic:	DISCUSSED Does OpenVZ have support for GrSecurity?

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Tue Dec 03 07:36:09 GMT 2024

Total time taken to generate the page: 0.11337 seconds