Hello, I noticed that some of ips are sometimes disappearing from my host, on HN it looks like this: devamd:# vzlist VPSID NPROC STATUS IP_ADDR HOSTNAME 1002 4 running 192.168.89.106 - 1003 4 running 192.168.89.107 etchdev386 1004 4 running 192.168.89.108 etchdevamd64 but those ips are unavailable, ping doesn't work, so I check the routing: devamd:# ip r 192.168.89.0/24 dev eth3 proto kernel scope link src 192.168.89.105 default via 192.168.89.1 dev eth3 I think it shouldn't look like this, so I try such sequence: devamd:~# vzctl set 1004 --ipadd 192.168.89.108 Adding IP address(es): 192.168.89.108 Unable to add IP 192.168.89.108: Address already in use strange, if it's in use then where can I see that?.. so: devamd:# vzctl set 1003 --ipdel 192.168.89.107 Deleting IP address(es): 192.168.89.107 vps-net_del WARNING: Function proxy_arp for eth3 is set to 0. Enable with 'sysctl -w net.ipv4.conf.eth3.proxy_arp=1'. See /usr/share/doc/vzctl/README.Debian. WARNING: Settings were not saved and will be resetted to original values on next start (use --save flag) devamd:# vzctl set 1003 --ipadd 192.168.89.107 Adding IP address(es): 192.168.89.107 vps-net_add WARNING: Function proxy_arp for eth3 is set to 0. Enable with 'sysctl -w net.ipv4.conf.eth3.proxy_arp=1'. See /usr/share/doc/vzctl/README.Debian. WARNING: Settings were not saved and will be resetted to original values on next start (use --save flag) and then it looks like this: devamd:# ip r 192.168.89.107 dev venet0 scope link src 192.168.89.105 192.168.89.0/24 dev eth3 proto kernel scope link src 192.168.89.105 default via 192.168.89.1 dev eth3 is there a reason why such disappearing act should happen? (I'm running fairly minimal debian etch on HN, there's only vzctl and few things like mtr, screen installed, nothing should be touching networking) this is my only amd64 machine using venet, on i386s with almost identical setup I haven't noticed anything similiar -- Key fingerprint = 40D0 9FFB 9939 7320 8294 05E0 BCC7 02C4 75CC 50D9 Total Existance Failure
this can happen if some hotplug/udev event has happened and removed the routes in host node. check /var/log/messages for any kind of events like eth link DOWN/UP, DHCP lease reacquiring etc. You can also replace ip and route utilities with some wrapper which logs who and when removes the routes and find the one to blame. Thanks, Kirill Dariush Pietrzak wrote: > Hello, > I noticed that some of ips are sometimes disappearing from my host, > on HN it looks like this: > > devamd:# vzlist > VPSID NPROC STATUS IP_ADDR HOSTNAME > 1002 4 running 192.168.89.106 - > 1003 4 running 192.168.89.107 etchdev386 > 1004 4 running 192.168.89.108 etchdevamd64 > > but those ips are unavailable, ping doesn't work, so I check the routing: > > devamd:# ip r > 192.168.89.0/24 dev eth3 proto kernel scope link src 192.168.89.105 > default via 192.168.89.1 dev eth3 > > I think it shouldn't look like this, so I try such sequence: > > devamd:~# vzctl set 1004 --ipadd 192.168.89.108 > Adding IP address(es): 192.168.89.108 > Unable to add IP 192.168.89.108: Address already in use > > strange, if it's in use then where can I see that?.. so: > > > devamd:# vzctl set 1003 --ipdel 192.168.89.107 > Deleting IP address(es): 192.168.89.107 > vps-net_del WARNING: Function proxy_arp for eth3 is set to 0. Enable with 'sysctl -w net.ipv4.conf.eth3.proxy_arp=1'. See /usr/share/doc/vzctl/README.Debian. > WARNING: Settings were not saved and will be resetted to original values on next start (use --save flag) > > devamd:# vzctl set 1003 --ipadd 192.168.89.107 > Adding IP address(es): 192.168.89.107 > vps-net_add WARNING: Function proxy_arp for eth3 is set to 0. Enable with 'sysctl -w net.ipv4.conf.eth3.proxy_arp=1'. See /usr/share/doc/vzctl/README.Debian. > WARNING: Settings were not saved and will be resetted to original values on next start (use --save flag) > > and then it looks like this: > > devamd:# ip r > 192.168.89.107 dev venet0 scope link src 192.168.89.105 > 192.168.89.0/24 dev eth3 proto kernel scope link src 192.168.89.105 > default via 192.168.89.1 dev eth3 > > > is there a reason why such disappearing act should happen? (I'm running > fairly minimal debian etch on HN, there's only vzctl and few things like > mtr, screen installed, nothing should be touching networking) > > this is my only amd64 machine using venet, on i386s with almost identical > setup I haven't noticed anything similiar
> this can happen if some hotplug/udev event has happened and removed the routes > in host node. check /var/log/messages for any kind of events like eth link DOWN/UP, > DHCP lease reacquiring etc. I can't find anything like that, BUT I can correllate those events ( ie direct route entries in route table disappearing ) with network problems, but this shouldn't be visible to the host ( things like upstream provider loosing link, snow severely slowing down radio uplink etc ). This seems strange, and identical routes created by hand, and not by vz stay in place. This looks very much like vz is doing this, so this mail is a broadcast - did anyone else see something like this or something similiar? The funny part is that it happens only on freshly installed dev machine and not on machines I'm actually using;) -- Key fingerprint = 40D0 9FFB 9939 7320 8294 05E0 BCC7 02C4 75CC 50D9 Total Existance Failure
Dariush Pietrzak wrote: >>this can happen if some hotplug/udev event has happened and removed the routes >>in host node. check /var/log/messages for any kind of events like eth link DOWN/UP, >>DHCP lease reacquiring etc. > > I can't find anything like that, BUT I can correllate those events ( ie > direct route entries in route table disappearing ) with network problems, > but this shouldn't be visible to the host ( things like upstream provider > loosing link, snow severely slowing down radio uplink etc ). > This seems strange, and identical routes created by hand, and not by vz > stay in place. This looks very much like vz is doing this, so this mail is > a broadcast - did anyone else see something like this or something > similiar? > > The funny part is that it happens only on freshly installed dev machine > and not on machines I'm actually using;) Hm... this is quite interesting... Does it often happens? I'm aware only of 2 places where routing can be removed: 1. hotplug, which I mentioned already. maybe some other "clever" daemon running on your system? RIP/OCFS? 2. OVZ crontab sript which should delete dead routes. can you check this one as well? All I can suggest for debugging is to: 1. replace ip and route commands with a wrapper which logs the exact commands and when/who called them. 2. add similar debug to kernel. BTW... does your host system uses DHCP or static IP assigned? Thanks, Kirill
> BTW... does your host system uses DHCP or static IP assigned? aaah, now that you mention it, this is the only dhcp-configured machine with openvz I've got around... and today I disabled DHCP server and few hours later noticed the problem with openvz. This might be it, and it would explaing non-immediate connection with networking problems. Thanks. -- Key fingerprint = 40D0 9FFB 9939 7320 8294 05E0 BCC7 02C4 75CC 50D9 Total Existance Failure
Dariush Pietrzak wrote: >>BTW... does your host system uses DHCP or static IP assigned? > > aaah, now that you mention it, this is the only dhcp-configured machine > with openvz I've got around... and today I disabled DHCP server and few > hours later noticed the problem with openvz. > This might be it, and it would explaing non-immediate connection with > networking problems. Thanks. Yep. DHCP client tries to be too smart :/ I've added this KB: http://wiki.openvz.org/Networking:_disappering_routes_in_HN Thanks, Kirill
Kirill Korotaev wrote: > Dariush Pietrzak wrote: > >>> BTW... does your host system uses DHCP or static IP assigned? >>> >> aaah, now that you mention it, this is the only dhcp-configured machine >> with openvz I've got around... and today I disabled DHCP server and few >> hours later noticed the problem with openvz. >> This might be it, and it would explaing non-immediate connection with >> networking problems. Thanks. >> > > Yep. DHCP client tries to be too smart :/ > I've added this KB: > http://wiki.openvz.org/Networking:_disappering_routes_in_HN > The proper URL now is http://wiki.openvz.org/Disappearing_routes_in_HN