OpenVZ Forum


Home » General » Support » undump failed: Invalid argument
undump failed: Invalid argument [message #30701] Wed, 04 June 2008 09:32 Go to next message
ebasley is currently offline  ebasley
Messages: 14
Registered: February 2008
Junior Member
Hello,

I compiled a kernel to add fuse module (see howto for debian, using .config from openvz) on HN bleuet.

Everythings sounds good, I can start, stop , suspend, dump VEs, many VEs are running for many days, BUT I can't vzmigrate online from other HN.

I receive 'Error: undump failed: Invalid argument' and no message on target HN !

How can I debug, I can guess this is due to new kernel ?

I saw older topics, checked vz.conf : all my HN config are similars.

Thanks

Eric


using debian, vzctl vers 3.0.22-1dso1
bleuet:/etc/vz/conf# cat /proc/version
Linux version 2.6.18-1-openvz (Version:) (root@bleuet) (gcc version 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)) #1 Tue Jun 3 17:07:57 CEST 2008


iris:/var/lib/vzquota# vzmigrate -v --online bleuet 115
OPT:-v
OPT:--online
OPT:bleuet
Starting online migration of VE 115 on bleuet
==> ssh -o BatchMode=yes root@bleuet /bin/true
==> ssh -o BatchMode=yes root@bleuet /etc/init.d/vz status
OpenVZ is running...
==> ssh -o BatchMode=yes root@bleuet test -f /proc/rst
Loading /etc/vz/vz.conf and /etc/vz/conf/115.conf files
Check IPs on destination node: 10.1.60.115
Preparing remote node
Copying config file
==> scp /etc/vz/conf/115.conf root@bleuet:/etc/vz/conf/115.conf
115.conf 100% 1782 1.7KB/s 00:00
==> ssh root@bleuet vzctl set 115 --applyconfig_map name --save
Name emcf assigned
Saved parameters for VE 115
Creating remote VE root dir
Creating remote VE private dir
Initializing remote quota
Quota init
Turning remote quota on
Syncing private
Live migrating VE
Suspending VE
==> vzctl chkpnt 115 --suspend
Setting up checkpoint...
suspend...
get context...
Checkpointing completed succesfully
Dumping VE
==> vzctl chkpnt 115 --dump --dumpfile /var/tmp/dump.115
Setting up checkpoint...
join context..
dump...
Running: /usr/lib/vzctl/scripts/vps-net_del
Checkpointing completed succesfully
Copying dumpfile
==> scp /var/tmp/dump.115 root@bleuet:/var/tmp/dump.115
dump.115 100% 3321KB 3.2MB/s 00:00
Syncing private (2nd pass)
Syncing 2nd level quota
Dumping 2nd level quota
Copying 2nd level quota
==> scp /var/tmp/quotadump.115 root@bleuet:/var/tmp/quotadump.115
quotadump.115 100% 21 0.0KB/s 00:00
Load 2nd level quota
Undumping VE
==> ssh root@bleuet vzctl restore 115 --undump --dumpfile /var/tmp/dump.115 --skip_arpdetect
Restoring VE ...
Starting VE ...
VE is mounted
undump...
VE is unmounted
vzquota : (warning) Quota is running for id 115 already
Error: undump failed: Invalid argument
Restoring failed:
VE start failed
Error: Failed to undump VE
==> vzctl chkpnt 115 --resume
Resuming...
Running: /usr/lib/vzctl/scripts/vps-net_add
put context
vzquota : (error) Quota is not running for id 115
is:/var/lib/vzquota# vzmigrate -v --online bleuet 115
OPT:-v
OPT:--online
OPT:bleuet
Starting online migration of VE 115 on bleuet
==> ssh -o BatchMode=yes root@bleuet /bin/true
==> ssh -o BatchMode=yes root@bleuet /etc/init.d/vz status
OpenVZ is running...
==> ssh -o BatchMode=yes root@bleuet test -f /proc/rst
Loading /etc/vz/vz.conf and /etc/vz/conf/115.conf files
Check IPs on destination node: 10.1.60.115
Preparing remote node
Copying config file
==> scp /etc/vz/conf/115.conf root@bleuet:/etc/vz/conf/115.conf
115.conf 100% 1782 1.7KB/s 00:00
==> ssh root@bleuet vzctl set 115 --applyconfig_map name --save
Name emcf assigned
Saved parameters for VE 115
Creating remote VE root dir
Creating remote VE private dir
Initializing remote quota
Quota init
Turning remote quota on
Syncing private
Live migrating VE
Suspending VE
==> vzctl chkpnt 115 --suspend
Setting up checkpoint...
suspend...
get context...
Checkpointing completed succesfully
Dumping VE
==> vzctl chkpnt 115 --dump --dumpfile /var/tmp/dump.115
Setting up checkpoint...
join context..
dump...
Running: /usr/lib/vzctl/scripts/vps-net_del
Checkpointing completed succesfully
Copying dumpfile
==> scp /var/tmp/dump.115 root@bleuet:/var/tmp/dump.115
dump.115 100% 3321KB 3.2MB/s 00:00
Syncing private (2nd pass)
Syncing 2nd level quota
Dumping 2nd level quota
Copying 2nd level quota
==> scp /var/tmp/quotadump.115 root@bleuet:/var/tmp/quotadump.115
quotadump.115 100% 21 0.0KB/s 00:00
Load 2nd level quota
Undumping VE
==> ssh root@bleuet vzctl restore 115 --undump --dumpfile /var/tmp/dump.115 --skip_arpdetect
Restoring VE ...
Starting VE ...
VE is mounted
undump...
VE is unmounted
vzquota : (warning) Quota is running for id 115 already
Error: undump failed: Invalid argument
Restoring failed:
VE start failed
Error: Failed to undump VE
==> vzctl chkpnt 115 --resume
Resuming...
Running: /usr/lib/vzctl/scripts/vps-net_add
put context
vzquota : (error) Quota is not running for id 115
Re: undump failed: Invalid argument [message #30705 is a reply to message #30701] Wed, 04 June 2008 11:20 Go to previous messageGo to next message
maratrus is currently offline  maratrus
Messages: 1495
Registered: August 2007
Location: Moscow
Senior Member
Hi,

anything in /var/log/messages? (on source HN and on the destination HN) "grep CPT /var/log/messages"
anything in dmesg or in vzctl.log? (on both source and destination nodes)?
Re: undump failed: Invalid argument [message #30766 is a reply to message #30705] Fri, 06 June 2008 10:21 Go to previous messageGo to next message
ebasley is currently offline  ebasley
Messages: 14
Registered: February 2008
Junior Member
Hi,

nothing in all logs.
after : # vzmigrate -v --online bleuet 113

from source HN in vzctl.log :
2008-06-06T12:14:15+0200 vzctl : VE 113 : Setting up checkpoint...
2008-06-06T12:14:15+0200 vzctl : VE 113 : suspend...
2008-06-06T12:14:15+0200 vzctl : VE 113 : get context...
2008-06-06T12:14:15+0200 vzctl : VE 113 : Checkpointing completed succesfully
2008-06-06T12:14:15+0200 vzctl : VE 113 : Setting up checkpoint...
2008-06-06T12:14:15+0200 vzctl : VE 113 : join context..
2008-06-06T12:14:15+0200 vzctl : VE 113 : dump...
2008-06-06T12:14:15+0200 vzctl : VE 113 : Running: /usr/lib/vzctl/scripts/vps-net_del
2008-06-06T12:14:15+0200 vzctl : VE 113 : Checkpointing completed succesfully
2008-06-06T12:14:20+0200 vzctl : VE 113 : Resuming...
2008-06-06T12:14:20+0200 vzctl : VE 113 : Running: /usr/lib/vzctl/scripts/vps-net_add
2008-06-06T12:14:21+0200 vzctl : VE 113 : put context

on target HN, in vzctl.log :

2008-06-06T12:14:16+0200 vzctl : VE 113 : Restoring VE ...
2008-06-06T12:14:16+0200 vzctl : VE 113 : Starting VE ...
2008-06-06T12:14:16+0200 vzctl : VE 113 : Running: /usr/sbin/vzquota show 113
2008-06-06T12:14:16+0200 vzctl : VE 113 : Running: /usr/sbin/vzquota on 113 -r 0 -b 3145828 -B 3459172 -i 200100 -I 220100 -e 0 -n 0 -s 0
2008-06-06T12:14:16+0200 vzctl : VE 113 : Running: vzquota setlimit 113 -p /var/lib/vz/private/113 -b 3145828 -B 3459172 -i 200100 -I 220100 -e 0 -n 0
2008-06-06T12:14:16+0200 vzctl : VE 113 : Mounting root: /var/lib/vz/root/113 /var/lib/vz/private/113
2008-06-06T12:14:16+0200 vzctl : VE 113 : VE is mounted
2008-06-06T12:14:16+0200 vzctl : VE 113 : VE start failed
2008-06-06T12:14:16+0200 vzctl : VE 113 : Running: /usr/sbin/vzquota stat 113 -f
2008-06-06T12:14:16+0200 vzctl : VE 113 : Running: vzquota setlimit 113 -b 3145728 -B 3459072 -i 200000 -I 220000 -e 0 -n 0
2008-06-06T12:14:16+0200 vzctl : VE 113 : Running: /usr/sbin/vzquota stat 113 -f
2008-06-06T12:14:16+0200 vzctl : VE 113 : Running: /usr/sbin/vzquota off 113
2008-06-06T12:14:19+0200 vzctl : VE 113 : VE is unmounted

Can I increase debug level ?

on both target and source HNs, I've : LOG_LEVEL=9 VERBOSE=1

Regards.

Eric
Re: undump failed: Invalid argument [message #30768 is a reply to message #30766] Fri, 06 June 2008 11:34 Go to previous messageGo to next message
maratrus is currently offline  maratrus
Messages: 1495
Registered: August 2007
Location: Moscow
Senior Member
Hello,

please show the output

1. dmesg
2. grep CPT /var/log/messages*

P.S. And if its possible please attach /var/log/messages* too. (I mean /var/log/messages1,/var/log/messages2 etc)

[Updated on: Fri, 06 June 2008 11:35]

Report message to a moderator

Re: undump failed: Invalid argument [message #30771 is a reply to message #30768] Fri, 06 June 2008 12:36 Go to previous messageGo to next message
ebasley is currently offline  ebasley
Messages: 14
Registered: February 2008
Junior Member
Here are the files.
iris is the source HN, bleuet the target.

Regards.

Eric
  • Attachment: bleuet.dmesg
    (Size: 20.46KB, Downloaded 307 times)
  • Attachment: iris.messages
    (Size: 15.07KB, Downloaded 215 times)
  • Attachment: iris.dmesg
    (Size: 29.75KB, Downloaded 347 times)
  • Attachment: bleuet.messages
    (Size: 42.22KB, Downloaded 268 times)
Re: undump failed: Invalid argument [message #30779 is a reply to message #30771] Fri, 06 June 2008 15:17 Go to previous messageGo to next message
maratrus is currently offline  maratrus
Messages: 1495
Registered: August 2007
Location: Moscow
Senior Member
Hello,

here is strange message:
Quote:


Jun 5 01:02:18 bleuet kernel: CPT ERR: e2fc0800,114 :unsupported fs type fuse



Did you try to migrate this VE? Do you use fuse inside VE?
Re: undump failed: Invalid argument [message #30780 is a reply to message #30779] Fri, 06 June 2008 15:32 Go to previous messageGo to next message
ebasley is currently offline  ebasley
Messages: 14
Registered: February 2008
Junior Member
This message was generated during a vzdump using '--suspend' option for a VE using fuse.

VEs I'm trying to migrate (online) are not using Fuse, I can migrate them on other HN but not on bleuet (HN), however all of them are running correctly on bleuet (vzmigrate whithout --online).
Re: undump failed: Invalid argument [message #30781 is a reply to message #30780] Fri, 06 June 2008 15:49 Go to previous messageGo to next message
maratrus is currently offline  maratrus
Messages: 1495
Registered: August 2007
Location: Moscow
Senior Member
Hi,

then please show the output of
- uname -a (from source and from target)
- vzctl --version (from source and from target)
- kernel configs file from source and from target

Thank You!
Re: undump failed: Invalid argument [message #30846 is a reply to message #30781] Mon, 09 June 2008 08:50 Go to previous messageGo to next message
ebasley is currently offline  ebasley
Messages: 14
Registered: February 2008
Junior Member
source :

iris:/var/log# uname -a
Linux iris 2.6.18-fza-028stab053.5-686 #1 SMP Sat Mar 1 10:53:11 CET 2008 i686 GNU/Linux

iris:/var/log# vzctl --version
vzctl version 3.0.22-1dso1

NO .config file : it's a stock kernel from debian repo.


target :

# uname -a
Linux bleuet 2.6.18-1-openvz #1 Tue Jun 3 17:07:57 CEST 2008 i686 GNU/Linux


bleuet:/var/log# vzctl --version
vzctl version 3.0.22-1dso1


see attached .config files
  • Attachment: bleuet.config
    (Size: 38.25KB, Downloaded 250 times)
Re: undump failed: Invalid argument [message #31038 is a reply to message #30846] Mon, 16 June 2008 05:50 Go to previous messageGo to next message
Andrey Mirkin is currently offline  Andrey Mirkin
Messages: 193
Registered: May 2006
Senior Member
Hello,

Can you please increase CPT log level on source and destination servers with help of following commands:
echo 3 > /proc/sys/debug/cpt
echo 3 > /proc/sys/debug/rst


And try migration again.

Also please post here output of following command on destination server:
cat /proc/rst


Andrey Mirkin
http://static.openvz.org/userbars/openvz-developer.png
Re: undump failed: Invalid argument [message #31048 is a reply to message #31038] Mon, 16 June 2008 09:51 Go to previous messageGo to next message
ebasley is currently offline  ebasley
Messages: 14
Registered: February 2008
Junior Member
# cat /proc/rst
Ctx Id VE State


Nothing returned neither on target not on source.
Re: undump failed: Invalid argument [message #31051 is a reply to message #30846] Mon, 16 June 2008 11:13 Go to previous messageGo to next message
curx
Messages: 739
Registered: February 2006
Location: N├╝rnberg, Germany
Senior Member

@ebasley:

.config file at debian kernels located at /boot/config-<kernel_version>

Please show on source and target:

# cat /proc/version /proc/vz/version

and which kind of OpenVZ patch is used on Server bleuet ...

Re: undump failed: Invalid argument [message #31093 is a reply to message #31048] Wed, 18 June 2008 07:26 Go to previous message
Andrey Mirkin is currently offline  Andrey Mirkin
Messages: 193
Registered: May 2006
Senior Member
BTW, can you please give an access to your nodes? It will be much easier to investigate this problem then.
If you can, then please send me credentials via private message, if not, then please increase log level (echo 3 > /proc/sys/debug/cpt, echo 3 > /proc/sys/debug/rst) on both servers (source and destination), try migration and attach here /var/log/messages files from both servers.


Andrey Mirkin
http://static.openvz.org/userbars/openvz-developer.png
Previous Topic: Do yo firewall your HN and VE
Next Topic: OpenVZ and multiple physical interfaces
Goto Forum:
  


Current Time: Thu Feb 02 18:19:03 GMT 2023

Total time taken to generate the page: 0.00826 seconds