Hello, 
 I noticed that some of ips are sometimes disappearing from my host,
on HN it looks like this:

devamd:# vzlist
     VPSID      NPROC STATUS  IP_ADDR         HOSTNAME
      1002          4 running 192.168.89.106  -
      1003          4 running 192.168.89.107  etchdev386
      1004          4 running 192.168.89.108  etchdevamd64

but those ips are unavailable, ping doesn't work, so I check the routing:

devamd:# ip r
192.168.89.0/24 dev eth3  proto kernel  scope link  src 192.168.89.105
default via 192.168.89.1 dev eth3

I think it shouldn't look like this, so I try such sequence:

	devamd:~# vzctl set 1004 --ipadd 192.168.89.108
	Adding IP address(es): 192.168.89.108
	Unable to add IP 192.168.89.108: Address already in use

strange, if it's in use then where can I see that?.. so:


	devamd:# vzctl set 1003 --ipdel 192.168.89.107
	Deleting IP address(es): 192.168.89.107
	vps-net_del WARNING: Function proxy_arp for eth3 is set to 0. Enable with 'sysctl -w net.ipv4.conf.eth3.proxy_arp=1'. See /usr/share/doc/vzctl/README.Debian.
WARNING: Settings were not saved and will be resetted to original values on next start (use --save flag)

	devamd:# vzctl set 1003 --ipadd 192.168.89.107
	Adding IP address(es): 192.168.89.107
	vps-net_add WARNING: Function proxy_arp for eth3 is set to 0. Enable with 'sysctl -w net.ipv4.conf.eth3.proxy_arp=1'. See /usr/share/doc/vzctl/README.Debian.
WARNING: Settings were not saved and will be resetted to original values on next start (use --save flag)

and then it looks like this:

devamd:# ip r
192.168.89.107 dev venet0  scope link  src 192.168.89.105
192.168.89.0/24 dev eth3  proto kernel  scope link  src 192.168.89.105
default via 192.168.89.1 dev eth3


is there a reason why such disappearing act should happen? (I'm running 
fairly minimal debian etch on HN, there's only vzctl and few things like
mtr, screen installed, nothing should be touching networking)

 this is my only amd64 machine using venet, on i386s with almost identical
setup I haven't noticed anything similiar
-- 
Key fingerprint = 40D0 9FFB 9939 7320 8294  05E0 BCC7 02C4 75CC 50D9
 Total Existance Failure

this can happen if some hotplug/udev event has happened and removed the routes
in host node. check /var/log/messages for any kind of events like eth link DOWN/UP,
DHCP lease reacquiring etc.

You can also replace ip and route utilities with some wrapper which logs who
and when removes the routes and find the one to blame.

Thanks,
Kirill


Dariush Pietrzak wrote:
> Hello, 
>  I noticed that some of ips are sometimes disappearing from my host,
> on HN it looks like this:
> 
> devamd:# vzlist
>      VPSID      NPROC STATUS  IP_ADDR         HOSTNAME
>       1002          4 running 192.168.89.106  -
>       1003          4 running 192.168.89.107  etchdev386
>       1004          4 running 192.168.89.108  etchdevamd64
> 
> but those ips are unavailable, ping doesn't work, so I check the routing:
> 
> devamd:# ip r
> 192.168.89.0/24 dev eth3  proto kernel  scope link  src 192.168.89.105
> default via 192.168.89.1 dev eth3
> 
> I think it shouldn't look like this, so I try such sequence:
> 
> 	devamd:~# vzctl set 1004 --ipadd 192.168.89.108
> 	Adding IP address(es): 192.168.89.108
> 	Unable to add IP 192.168.89.108: Address already in use
> 
> strange, if it's in use then where can I see that?.. so:
> 
> 
> 	devamd:# vzctl set 1003 --ipdel 192.168.89.107
> 	Deleting IP address(es): 192.168.89.107
> 	vps-net_del WARNING: Function proxy_arp for eth3 is set to 0. Enable with 'sysctl -w net.ipv4.conf.eth3.proxy_arp=1'. See /usr/share/doc/vzctl/README.Debian.
> WARNING: Settings were not saved and will be resetted to original values on next start (use --save flag)
> 
> 	devamd:# vzctl set 1003 --ipadd 192.168.89.107
> 	Adding IP address(es): 192.168.89.107
> 	vps-net_add WARNING: Function proxy_arp for eth3 is set to 0. Enable with 'sysctl -w net.ipv4.conf.eth3.proxy_arp=1'. See /usr/share/doc/vzctl/README.Debian.
> WARNING: Settings were not saved and will be resetted to original values on next start (use --save flag)
> 
> and then it looks like this:
> 
> devamd:# ip r
> 192.168.89.107 dev venet0  scope link  src 192.168.89.105
> 192.168.89.0/24 dev eth3  proto kernel  scope link  src 192.168.89.105
> default via 192.168.89.1 dev eth3
> 
> 
> is there a reason why such disappearing act should happen? (I'm running 
> fairly minimal debian etch on HN, there's only vzctl and few things like
> mtr, screen installed, nothing should be touching networking)
> 
>  this is my only amd64 machine using venet, on i386s with almost identical
> setup I haven't noticed anything similiar

> this can happen if some hotplug/udev event has happened and removed the routes
> in host node. check /var/log/messages for any kind of events like eth link DOWN/UP,
> DHCP lease reacquiring etc.
 I can't find anything like that, BUT I can correllate those events ( ie
direct route entries in route table disappearing ) with network problems,
but this shouldn't be visible to the host ( things like upstream provider
loosing link, snow severely slowing down radio uplink etc ).
 This seems strange, and identical routes created by hand, and not by vz
stay in place. This looks very much like vz is doing this, so this mail is
a broadcast - did anyone else see something like this or something
similiar?

 The funny part is that it happens only on freshly installed dev machine
and not on machines I'm actually using;)

-- 
Key fingerprint = 40D0 9FFB 9939 7320 8294  05E0 BCC7 02C4 75CC 50D9
 Total Existance Failure

Dariush Pietrzak wrote:
>>this can happen if some hotplug/udev event has happened and removed the routes
>>in host node. check /var/log/messages for any kind of events like eth link DOWN/UP,
>>DHCP lease reacquiring etc.
> 
>  I can't find anything like that, BUT I can correllate those events ( ie
> direct route entries in route table disappearing ) with network problems,
> but this shouldn't be visible to the host ( things like upstream provider
> loosing link, snow severely slowing down radio uplink etc ).
>  This seems strange, and identical routes created by hand, and not by vz
> stay in place. This looks very much like vz is doing this, so this mail is
> a broadcast - did anyone else see something like this or something
> similiar?
> 
>  The funny part is that it happens only on freshly installed dev machine
> and not on machines I'm actually using;)

Hm... this is quite interesting...

Does it often happens?
I'm aware only of 2 places where routing can be removed:
1. hotplug, which I mentioned already.
   maybe some other "clever" daemon running on your system?
   RIP/OCFS?
2. OVZ crontab sript which should delete dead routes.
   can you check this one as well?

All I can suggest for debugging is to:
1. replace ip and route commands with a wrapper which logs the exact
   commands and when/who called them.
2. add similar debug to kernel.

BTW... does your host system uses DHCP or static IP assigned?

Thanks,
Kirill

> BTW... does your host system uses DHCP or static IP assigned?
 aaah, now that you mention it, this is the only dhcp-configured machine
with openvz I've got around... and today I disabled DHCP server and few
hours later noticed the problem with openvz. 
 This might be it, and it would explaing non-immediate connection with
networking problems. Thanks.
-- 
Key fingerprint = 40D0 9FFB 9939 7320 8294  05E0 BCC7 02C4 75CC 50D9
 Total Existance Failure

Dariush Pietrzak wrote:
>>BTW... does your host system uses DHCP or static IP assigned?
> 
>  aaah, now that you mention it, this is the only dhcp-configured machine
> with openvz I've got around... and today I disabled DHCP server and few
> hours later noticed the problem with openvz. 
>  This might be it, and it would explaing non-immediate connection with
> networking problems. Thanks.

Yep. DHCP client tries to be too smart :/
I've added this KB:
http://wiki.openvz.org/Networking:_disappering_routes_in_HN

Thanks,
Kirill

Kirill Korotaev wrote:
> Dariush Pietrzak wrote:
>   
>>> BTW... does your host system uses DHCP or static IP assigned?
>>>       
>>  aaah, now that you mention it, this is the only dhcp-configured machine
>> with openvz I've got around... and today I disabled DHCP server and few
>> hours later noticed the problem with openvz. 
>>  This might be it, and it would explaing non-immediate connection with
>> networking problems. Thanks.
>>     
>
> Yep. DHCP client tries to be too smart :/
> I've added this KB:
> http://wiki.openvz.org/Networking:_disappering_routes_in_HN
>   
The proper URL now is http://wiki.openvz.org/Disappearing_routes_in_HN