OpenVZ Forum


Home » General » Support » vzyum database recovery -- was vzpkgcache fail
vzyum database recovery -- was vzpkgcache fail [message #22087] Fri, 19 October 2007 18:28 Go to next message
dranch is currently offline  dranch
Messages: 33
Registered: August 2007
Member
Hello Everyone,

Per the Russian forum, another user had an issue with "vzyum" where it would fail. I've noticed the following and I was curious if there is a fix for this issue.


What was the solution to this problem (the answer isn't in English)? I've noticed that vzyum will always stop working but if I go into each VE and locally run "yum update", things get cleaned up for vzyum to run again without issue for a while more. But, a day later, the issue comes back.


Below I show that:

1. "vzyum update" fails
2. I enter the VE, locally run "yum update"
3. I exit the VE, rerun "vzyum 100 update" and it works (for
some period of time


1. "vzyum 100 update" fails
---------------------------------------------------------
# vzyum 100 install am-utils
exec /usr/share/vzyum/bin/yum -c /vz/template/centos/5/i386/config/yum.conf --installroot /vz/root/100 --vps=100 install am-utils
rpmdb: unable to initialize mutex: Invalid argument
rpmdb: PANIC: Invalid argument
rpmdb: PANIC: DB_RUNRECOVERY: Fatal error, run database recovery
error: db4 error(-30977) from dbenv->open: DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db3 - (-30977)
error: cannot open Packages database in /vz/root/100/var/lib/rpm
Traceback (most recent call last):
File "/usr/share/vzyum/bin/yum", line 28, in ?
yummain.main(sys.argv[1:])
File "/usr/share/vzyum/yum-cli/yummain.py", line 75, in main
base.getOptionsConfig(args)
File "/usr/share/vzyum/yum-cli/cli.py", line 172, in getOptionsConfig
self.doConfigSetup(fn=opts.conffile, root=root, vps=opts.vps)
File "/usr/share/vzyum/lib/yum/__init__.py", line 82, in doConfigSetup
self.conf = config.yumconf(configfile=fn, root=root, vps=vps)
File "/usr/share/vzyum/lib/yum/config.py", line 271, in __init__
self.yumvar['releasever'] = self._getsysver()
File "/usr/share/vzyum/lib/yum/config.py", line 384, in _getsysver
idx = ts.dbMatch('provides', self.getConfigOption('distroverpkg'))
TypeError: rpmdb open failed


2. Enter VE and run "yum update"
--------------------------------------------------------
# /usr/sbin/vzctl enter 100
entered into VE 100
dolly-centos5-A:/# yum update
Loading "installonlyn" plugin
Setting up Update Process
Setting up repositories
base 100% |=========================| 1.1 kB 00:00
updates 100% |=========================| 951 B 00:00
addons 100% |=========================| 951 B 00:00
extras 100% |=========================| 1.1 kB 00:00
Reading repository metadata in from local files
No Packages marked for Update/Obsoletion


3. Exit the VE and again run "vzyum 100 update" and it works
for some period of time
------------------------------------------------------------
# exit
logout

exited from VE 100
dolly-centos5-openvz:/etc# vzyum 100 install am-utils
exec /usr/share/vzyum/bin/yum -c /vz/template/centos/5/i386/config/yum.conf --installroot /vz/root/100 --vps=100 install am-utils
Setting up Install Process
Setting up repositories
centos5-vz-addons 100% |=========================| 951 B 00:00
centos5-updates-released 100% |=========================| 951 B 00:00
centos5-base 100% |=========================| 1.1 kB 00:00
Reading repository metadata in from local files
Parsing package install arguments
Nothing to do"
Re: vzyum database recovery -- was vzpkgcache fail [message #22159 is a reply to message #22087] Mon, 22 October 2007 16:08 Go to previous messageGo to next message
dranch is currently offline  dranch
Messages: 33
Registered: August 2007
Member
As an update, it seems this issue comes up when the VPS is rebooted. As a work around, I've added the following to each VPS:

/etc/rc.d/rc.local
--
yum check-update > /dev/null
--

This cleans the issue up but it's only a hack that works around the a root problem.
Re: vzyum database recovery -- was vzpkgcache fail [message #28110 is a reply to message #22087] Sun, 09 March 2008 06:54 Go to previous messageGo to next message
steve is currently offline  steve
Messages: 25
Registered: June 2007
Location: Orange County, California
Junior Member
I'm getting this problem too after just having upgraded some packages on the host node. The fact that this can be corrected by entering the VPS and running any rpm command makes it look suspiciously like a bug. Any rpm database inside a VPS is somehow left in an inconsistent state. Of course, this makes vzyum unusable. Does vzyum need an upgrade in order to handle this?

Run: vzyum 139 install wget

Yields:
exec /usr/share/vzyum/bin/yum -c /vz/template/centos/5/i386/config/yum.conf --installroot /vz/root/139 --vps=139 install wget
rpmdb: unable to initialize mutex: Invalid argument
rpmdb: PANIC: Invalid argument
rpmdb: PANIC: DB_RUNRECOVERY: Fatal error, run database recovery
error: db4 error(-30977) from dbenv->open: DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db3 - (-30977)
error: cannot open Packages database in /vz/root/139/var/lib/rpm
Traceback (most recent call last):
File "/usr/share/vzyum/bin/yum", line 28, in ?
yummain.main(sys.argv[1:])
File "/usr/share/vzyum/yum-cli/yummain.py", line 75, in main
base.getOptionsConfig(args)
File "/usr/share/vzyum/yum-cli/cli.py", line 172, in getOptionsConfig
self.doConfigSetup(fn=opts.conffile, root=root, vps=opts.vps)
File "/usr/share/vzyum/lib/yum/__init__.py", line 82, in doConfigSetup
self.conf = config.yumconf(configfile=fn, root=root, vps=vps)
File "/usr/share/vzyum/lib/yum/config.py", line 271, in __init__
self.yumvar['releasever'] = self._getsysver()
File "/usr/share/vzyum/lib/yum/config.py", line 384, in _getsysver
idx = ts.dbMatch('provides', self.getConfigOption('distroverpkg'))
TypeError: rpmdb open failed
Re: vzyum database recovery -- was vzpkgcache fail [message #32295 is a reply to message #22087] Wed, 30 July 2008 12:53 Go to previous messageGo to next message
tranky is currently offline  tranky
Messages: 1
Registered: July 2008
Junior Member
I've experienced the same behavior and found that a simple "rpm -qa" ran inside the VE (either via 'vzctl exec' or entering the VE and running it inside) solves the issue

HTH

edit: unfortunately, this fix isn't permanent and the issue reappears as soon as the VE is restarted as mentioned above
At least this workaround doesn't need the installation of yum inside the VE

[Updated on: Wed, 30 July 2008 13:03]

Report message to a moderator

Re: vzyum database recovery -- was vzpkgcache fail [message #32358 is a reply to message #32295] Fri, 01 August 2008 19:42 Go to previous messageGo to next message
bchapman is currently offline  bchapman
Messages: 2
Registered: August 2008
Junior Member
This works perfectly. Thanks for the workaround.
Re: vzyum database recovery -- was vzpkgcache fail [message #32359 is a reply to message #32358] Fri, 01 August 2008 20:05 Go to previous messageGo to next message
bchapman is currently offline  bchapman
Messages: 2
Registered: August 2008
Junior Member
Here is some further information on this issue.

When booted, the virtual machine does not have the /var/lib/rpm/__db.00{1,2,3} files. This causes the vzyum issue regarding the database.

Running "rpm -qa" inside the virtual machine recreates the /var/lib/rpm/__db.00{1,2,3} files. It looks like the files are deleted on start up; doing a "shutdown now -h" from inside the machine does not delete them. This leaves the machine "mounted". Running "vzctl umount VE_ID" doesn't remove them either. However, running "vzctl start VE_ID" deletes the db files.

Not sure where to go next with this...

[EDIT]

Here is a small enhancement to the previous workaround - edit /usr/bin/vzyum so that it runs the 'rpm -qa' command prior to running yum.

Add the following to the next to last line of /usr/bin/vzyum:

$VZCTL exec $VEID rpm --quiet -qa

After the change the last four lines of vzyum should look like this:

log4 PYTHONPATH=$PYTHONPATH
log3 exec $YUM $YUM_ARGS $USER_ARGS
$VZCTL exec $VEID rpm --quiet -qa
exec $YUM $YUM_ARGS $USER_ARGS

This will rebuild the db files as in the earlier post.

Ben

[Updated on: Fri, 01 August 2008 21:07]

Report message to a moderator

Re: vzyum database recovery -- was vzpkgcache fail [message #33779 is a reply to message #22087] Sun, 09 November 2008 22:50 Go to previous messageGo to next message
n00b_admin is currently offline  n00b_admin
Messages: 77
Registered: July 2006
Location: Romania
Member
So, this is a bug ?

It will be fixed, it is reported somewhere ?

I don't think using workarounds as permanent solutions is THE solution.

Some admins please ?

For example, i'm using the contrib template of centos 5 minimal and the issue is not fixed by running "rpm -qa" inside VE or by "vzctl <ve_id> rpm -qa"

[Updated on: Sun, 09 November 2008 23:10]

Report message to a moderator

Re: vzyum database recovery -- was vzpkgcache fail [message #33780 is a reply to message #22087] Sun, 09 November 2008 23:51 Go to previous message
n00b_admin is currently offline  n00b_admin
Messages: 77
Registered: July 2006
Location: Romania
Member
I apologize for my frustration in my previous post.

It seems i discovered the problem.

In my setup i have two vz storage locations on two separate drives.

It appears that the fix in this post (http://forum.openvz.org/index.php?t=msg&goto=30830&) does not work in my case since the ve_root dir for my particular ve is in another location than the one in vz.conf

In my setup a require two locations for the private and root dirs and the vz.conf file allows only one location.

As a quick fix i have made a copy of the vzyum wrapper in vzyum2 and hardcoded the second location inside vzyum2.

In this way i can run vzyum for VE's in the default location and vzyum2 for VE's in the second location as the paths differ.

Maybe an official solution would be to make the functions file read the private and root paths from the VE config file since vzctl supports creating VE's in different locations than the default one.
Previous Topic: kernel Problem
Next Topic: [solved] How to determine which HN a VE is running on
Goto Forum:
  


Current Time: Mon Feb 26 17:55:31 GMT 2024

Total time taken to generate the page: 0.02853 seconds