Hi, I got 2 interfaces on HE and wanted one for management and the second for VE's (I need several VLANs there). So I setup HE interfaces like this (fat trimmed) eth0 Link encap:Ethernet HWaddr 00:21:5e:26:b3:cc inet addr:126.66.64.151 Bcast:126.66.65.255 Mask:255.255.254.0 eth1 Link encap:Ethernet HWaddr 00:21:5e:26:b3:ce inet6 addr: fe80::221:5eff:fe26:b3ce/64 Scope:Link eth1.20 Link encap:Ethernet HWaddr 00:21:5e:26:b3:ce inet6 addr: fe80::221:5eff:fe26:b3ce/64 Scope:Link eth1.980 Link encap:Ethernet HWaddr 00:21:5e:26:b3:ce inet6 addr: fe80::221:5eff:fe26:b3ce/64 Scope:Link venet0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 UP BROADCAST POINTOPOINT RUNNING NOARP MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) (lo, sit0, other VLANs ommitted). Then I added the vlan to VE like this: vzctl set 110 --netif_add vlan980 --save Then I setup the VE settings like this: (snippet from 110.conf) IP_ADDRESS="10.0.0.2" NETIF=" ifname=vlan980,mac=00:18:51:F6:CD:87,host_ifname=veth110.980 ,host_mac=00:18:51:70:E9:7F " And the problem is I still cannot access the other ip on the network (PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data. From 10.0.0.2 icmp_seq=1 Destination Host Unreachable)! How to debug this?? BTW. when I assign an address from the VLAN to eth1.980 on HN it works, but VE - its not. Thanks!
Hello, you should configure eth0 interface inside VE the same way you've done it on your HN. # vzctl set 110 --netif_add eth0 --save (So, eth0 interface must appear inside VE). Then enter your VE and configure appropriate vlan just like on the HN.
Yeah that sounds cool, but why the f%$!k I need to add eth0?? And it still doesn't work. I added eth1 to VE like you said, I got eth1 inside VE. I configured the VLAN 980, set the ip and still nothing: eth1 Link encap:Ethernet HWaddr 00:18:51:44:65:A8 inet6 addr: fe80::218:51ff:fe44:65a8/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:24 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:1176 (1.1 Kb) eth1.980 Link encap:Ethernet HWaddr 00:18:51:44:65:A8 inet addr:10.0.0.3 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::218:51ff:fe44:65a8/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:18 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:1044 (1.0 Kb) ping 10.0.0.1 PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data. From 10.0.0.3 icmp_seq=2 Destination Host Unreachable I just think this is impossible, after spending a whole day on this I get use to that this is just fake that OpenVZ ever supported VLANs? please admit that this is not true...
OK, I.m posting all relevant info: 2.6.18-028stab056 110.conf: NETIF=" ifname=eth1,mac=00:18:51:44:65:A8,host_ifname=veth110.1,host _mac=00:18:51:42:20:9A " Networking in HE: eth0 Link encap:Ethernet HWaddr 00:21:5e:26:b3:cc inet addr:126.66.64.151 Bcast:126.66.65.255 Mask:255.255.254.0 inet6 addr: fe80::221:5eff:fe26:b3cc/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:3880 errors:0 dropped:0 overruns:0 frame:0 TX packets:1188 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:319773 (312.2 KiB) TX bytes:297744 (290.7 KiB) Interrupt:90 Memory:ce000000-ce011100 eth1 Link encap:Ethernet HWaddr 00:21:5e:26:b3:ce inet6 addr: fe80::221:5eff:fe26:b3ce/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2257 errors:0 dropped:0 overruns:0 frame:0 TX packets:24 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:182530 (178.2 KiB) TX bytes:2040 (1.9 KiB) Interrupt:169 Memory:ca000000-ca011100 eth1.20 Link encap:Ethernet HWaddr 00:21:5e:26:b3:ce inet6 addr: fe80::221:5eff:fe26:b3ce/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2258 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:132922 (129.8 KiB) TX bytes:468 (468.0 B) eth1.980 Link encap:Ethernet HWaddr 00:21:5e:26:b3:ce inet6 addr: fe80::221:5eff:fe26:b3ce/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:46 (46.0 B) TX bytes:468 (468.0 B) eth1.990 Link encap:Ethernet HWaddr 00:21:5e:26:b3:ce inet6 addr: fe80::221:5eff:fe26:b3ce/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:46 (46.0 B) TX bytes:468 (468.0 B) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:52 errors:0 dropped:0 overruns:0 frame:0 TX packets:52 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:3792 (3.7 KiB) TX bytes:3792 (3.7 KiB) sit0 Link encap:IPv6-in-IPv4 NOARP MTU:1480 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) venet0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 UP BROADCAST POINTOPOINT RUNNING NOARP MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) veth110.1 Link encap:Ethernet HWaddr 00:18:51:42:20:9a inet6 addr: fe80::218:51ff:fe42:209a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:12 errors:0 dropped:0 overruns:0 frame:0 TX packets:4 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:792 (792.0 B) TX bytes:244 (244.0 B) Networking in VE: eth1 Link encap:Ethernet HWaddr 00:18:51:44:65:A8 inet6 addr: fe80::218:51ff:fe44:65a8/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:4 errors:0 dropped:0 overruns:0 frame:0 TX packets:12 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:244 (244.0 b) TX bytes:792 (792.0 b) eth1.980 Link encap:Ethernet HWaddr 00:18:51:44:65:A8 inet addr:10.0.0.3 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::218:51ff:fe44:65a8/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:492 (492.0 b) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) venet0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 UP BROADCAST POINTOPOINT RUNNING NOARP MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) Physical eth1 is connected to trunk port... and it works if I configure it in HE. What else I can do?
- First of all, make sure that you've configured eth0 interface inside VE properly http://wiki.openvz.org/Veth#Simple_configuration_with_virtua l_Ethernet_device - Then check your routing records inside VE. (make sure that network packets don't leave your VE from venet0 interface.) - Use tcpdump utility to find out: * do the packets leave your VE propely * do they reach HN * do they leave HN * does the external node respond * does the response reach veth interface on the HN * does the response reach eth* interface inside VE (try to ping something and at the same moment run tcpdump on eth* interface inside VE, veth* interface on the HN and eth* interface on the HN) http://wiki.openvz.org/VLAN Make sure that firewall in the HN/VE allow your VE to communicate with external nodes.
- First of all, make sure that you've configured eth0 interface inside VE properly Now its like in wiki, previously I've used vconfig to add vlan. - Then check your routing records inside VE. (make sure that network packets don't leave your VE from venet0 interface.) Its OK: ip r s 127.0.0.0/8 via 127.0.0.1 dev lo default dev eth0 scope link - Use tcpdump utility to find out: * do the packets leave your VE propely they leave VE, * do they reach HN they reach HE, * do they leave HN I don't know, as I don't have a second node and netadmin is on holiday http://wiki.openvz.org/VLAN Excuse me, I try to stay positive, but there is no doc in here, those three lines simply doesn't work, either when I try to move venet interface or create new veth in VE. Maybe because I do have two physical interfaces? Make sure that firewall in the HN/VE allow your VE to communicate with external nodes. There is no netfilter setup. I attach screenlog.
Quote: Now its like in wiki, previously I've used vconfig to add vlan. As far as I can understand from your screenlog you've configured eth0 interface inside VE. Ok, after that you should create vlan interface inside VE via vconfig utility. But you haven't done that.
OK, lets try that: Do I have to configure vlan on physical interface in HE? Do I have to add ip to that interface? I guess answer is yes and no? So on HE I should have (Gentoo net config): config_eth0=("126.66.64.151 netmask 255.255.254.0") routes_eth0=("default via 126.66.65.249") vlans_eth1="20 980 990" config_eth1=( "null" ) vconfig_eth1=( "set_name_type DEV_PLUS_VID_NO_PAD" ) config_eth1_980=( "null" ) config_eth1_990=( "null" )
Quote: Do I have to configure vlan on physical interface in HE? No. --------------- I'd like emphasize that we should consider VE as a common machine (at least try to do that). So, if you have a standard machine and eth0 interface on it, what should you do to enable vlan functionality? You should create vlan interface and then do appropriate settings. The way enabling VLAN inside container can be divided into two steps: - enable eth0 inside container - create vlan interface inside container and issue some settings You've already created eth0 interface inside your VE. So, you have to pass through the second step.
ok, I configured vlan in VE, still nothing. Host unreachable this time. I'm switching to one NIC only (trunked; although I doubt it would help in this maze).
- did you change default route inside VE? (sreenlog shows that it was eth0) - could you please show tcpdump output? (from inside the VE and from inside the HN)
OK.... maybe this will be easier - I have one interface now, with 3 vlans, all interfaces are configured with ip, all works fine. I got this Gentoo VE and want to set up vlans inside. Have anyone step-by step example of VLAN configuration? Sorry, I'm too dumb to crack it by myself.... (concerning 4 years of running openvz thats strange, but this is how no documentation works...)
I'm posting tcpdump (its the same on HN as in VE): 11:35:59.979932 IP 10.0.0.3 > 10.0.0.1: ICMP echo request, id 23060, seq 179, length 64 11:36:00.979954 arp who-has 10.0.0.1 tell 10.0.0.3 11:36:00.979959 arp reply 10.0.0.1 is-at 00:18:51:3a:d4:5d 11:36:00.979984 IP 10.0.0.3 > 10.0.0.1: ICMP echo request, id 23060, seq 180, length 64 etc. There is no echo reply. If I configure devices without vlan in VE I can ping HE and VE each other, I suspect that I could ping also remote interface, but there is no vlan so I cant. Its all about those vlans. Maybe a step-by-step howto for one interface, multiple vlans?
- enable eth0 inside container - create vlan interface inside container and issue some settings well, I have setup eth0.980 5 minutes since installation of the kernel, but it doesn't work. Kernel IP routing table (HE): Destination Gateway Genmask Flags Metric Ref Use Iface 10.0.0.2 0.0.0.0 255.255.255.255 UH 0 0 0 veth110.0 126.66.64.0 0.0.0.0 255.255.254.0 U 0 0 0 eth0.20 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo 0.0.0.0 126.66.65.249 0.0.0.0 UG 0 0 0 eth0.20 I do all the stuff: echo 1 > /proc/sys/net/ipv4/conf/veth110.0/forwarding echo 1 > /proc/sys/net/ipv4/conf/veth110.0/proxy_arp echo 1 > /proc/sys/net/ipv4/conf/eth0/forwarding echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp And I setup vlan the Gentoo way in VE (so its standard Linux way): vlans_eth0="980" config_eth0=( "null" ) vconfig_eth0=( "set_name_type DEV_PLUS_VID_NO_PAD" ) config_eth0_980=( "10.0.0.2 netmask 255.255.255.0" ) i get these routes: 10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0.980 127.0.0.0 127.0.0.1 255.0.0.0 UG 0 0 0 lo this doesn't work, (Destination Host Unreachable) so packets do not reach HE. If I do in VE: ifconfig eth0 0 ip r a default dev eth0 it doesn't work either (sorry! this doesn't work, but ping goes to HE, I forgot to remove previous route!). I do in VE: cat /proc/net/vlan/config VLAN Dev name | VLAN ID Name-Type: VLAN_NAME_TYPE_RAW_PLUS_VID_NO_PAD eth0.980 | 980 | eth0 I do in HE: cat /proc/net/vlan/config VLAN Dev name | VLAN ID Name-Type: VLAN_NAME_TYPE_RAW_PLUS_VID_NO_PAD eth0.20 | 20 | eth0 Its ok, because I'm using vlan 20 for accessing the HE. I'm loosing the second day on this....
Quote: Do I have to configure vlan on physical interface in HE? Quote: No. Oops, it seems that was a wrong answer. a packet leaves VE from eth0.980 interface then it reaches veth110.0 interface and then according your routing table on the HN goes to eth0.20 but it's wrong way. I'm terribly sorry. You should direct the packet through .980 vlan on the HN.
OK, partly works (first vlan i VE works ok). Now I'm trying to add second VLAN to veth device and it goes ok, but packets are not going back to VE. Why? In VE 130 I've got two vlans added: eth0.20 Link encap:Ethernet HWaddr 00:18:51:DB:4E:F2 inet addr:126.66.64.158 eth1.980 Link encap:Ethernet HWaddr 00:18:51:05:AF:37 inet addr:10.0.0.5 Bcast:10.0.0.255 Mask:255.255.255.0 Routes are set OK iv VE: Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 126.66.64.0 0.0.0.0 255.255.254.0 U 0 0 0 eth0 127.0.0.0 127.0.0.1 255.0.0.0 UG 0 0 0 lo And routes in HN aore set OK: Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 10.0.0.5 0.0.0.0 255.255.255.255 UH 0 0 0 veth130.0 126.66.64.158 0.0.0.0 255.255.255.255 UH 0 0 0 veth130.0 10.0.0.2 0.0.0.0 255.255.255.255 UH 0 0 0 veth110.0 10.0.1.2 0.0.0.0 255.255.255.255 UH 0 0 0 veth130.0 10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0.980 10.0.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0.990 126.66.64.0 0.0.0.0 255.255.254.0 U 0 0 0 eth0.20 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo 0.0.0.0 126.66.65.249 0.0.0.0 UG 0 0 0 eth0.20 Packets goes fine to HN, but not going back (tcpdump outpup from HN): 15:08:32.501126 IP 126.66.64.158 > 10.0.0.1: ICMP echo request, id 45655, seq 14, length 64 Any ideas how to define source ip on VE? BTW: I can ping interface in vlan on HN (ping 10.0.0.254 issued in VE is ok, but to 10.0.0.1 - which is gateway - its not working). Ping to 10.0.0.1 from HN - it works.
I think the routing table inside VE is wrong. It should be "" 10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1.980 126.66.64.0 0.0.0.0 255.255.254.0 U 0 0 0 eth0.20 "" Please, try to change it. Does this help?
no, it doesn't help: why I would want to set up route through vlan interface now; I was said that I shouldn't and it really works through eth0 not eth0.<vlanid> (of course - only one works then...) Now I get immortal "destination host unreacheable", on two vlans I thought its worth new thread so I marked this solved and started new one (there are different ips and vlans but thats the same, I got several tens of vlans to test on )
and what about the kernel itself? maybe there is some newer than mine which is ready to production? I took 2.6.18-028stab056.
Quote: why I would want to set up route through vlan interface now because you should mark a packet as belonging to vlan interface. If a packet pass eth0 interface how it could be marked? Quote: 15:08:32.501126 IP 126.66.64.158 > 10.0.0.1: ICMP echo request, id 45655, seq 14, length 64 why do you expect 10.0.0.1 answer to the ip from different that 10.0.0.0/24 network?
well, I understand that, that is why I obssesically did set routes through vlan interfaces (and it didn't work). If I set like you say now I got: route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1.980 126.66.64.0 0.0.0.0 255.255.254.0 U 0 0 0 eth0.20 127.0.0.0 127.0.0.1 255.0.0.0 UG 0 0 0 lo 130 / # ping 10.0.0.1 PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data. From 10.0.0.5 icmp_seq=2 Destination Host Unreachable From 10.0.0.5 icmp_seq=3 Destination Host Unreachable From 10.0.0.5 icmp_seq=4 Destination Host Unreachable --- 10.0.0.1 ping statistics --- 4 packets transmitted, 0 received, +3 errors, 100% packet loss, time 3005ms , pipe 3 130 / # ping 126.66.64.1 PING 126.66.64.1 (126.66.64.1) 56(84) bytes of data. From 126.66.64.158 icmp_seq=2 Destination Host Unreachable From 126.66.64.158 icmp_seq=3 Destination Host Unreachable From 126.66.64.158 icmp_seq=4 Destination Host Unreachable --- 126.66.64.1 ping statistics --- 4 packets transmitted, 0 received, +3 errors, 100% packet loss, time 3009ms , pipe 3 130 / #
Re: VLAN not working in VE? <SOLVED>
Ok, I got it. There are several ways how to do this, but in my opinion openvz wiki does bad job and explains nothing. Moreover, there is nothing about setting vlans permananently. I do explain later on another wiki how to set it up and post a link here. Thanks a lot maratrus for your offsite help! EDIT: http://en.gentoo-wiki.com/wiki/OpenVZ_VLANs