OpenVZ Forum


Home » General » Support » Container networking broken after upgrade to 2.6.18-308.8.2.el5.028stab101.1, stange kernel errors (After upgrading my CentOS 5.9 server to 2.6.18-308.8.2.el5.028stab101.1 I'm having issues with networking in containers working and kernel CPT ERR's)
Container networking broken after upgrade to 2.6.18-308.8.2.el5.028stab101.1, stange kernel errors [message #48963] Mon, 28 January 2013 18:01 Go to next message
PJF5000 is currently offline  PJF5000
Messages: 5
Registered: January 2013
Junior Member
Hi there, not sure if this should be a bug report, or if there is a strange configuraiton issue on my server.

Running a CentOS 5.9 x86_64 server, recelntly rebooted into the 2.6.18-308.8.2.el5.028stab101.1 kernel last weekend.

Upon booting into the new kernel I saw many kernel CPT ERR's while starting the containers:

"
Jan 20:46:08 vz02 last message repeated 8 times
Jan 26 20:46:12 vz02 kernel: CT: 104: started
Jan 26 20:46:14 vz02 kernel: CPT ERR: ffff81031e46a000,104 :open_listening_socket: sock_create_kern: -97
Jan 26 20:46:14 vz02 kernel: CPT ERR: ffff81031e46a000,104 :rst_sockets: open_listening_socket: -97
Jan 26 20:46:14 vz02 kernel: CPT ERR: ffff81031e46a000,104 :rst_sockets: -97
Jan 26 20:46:14 vz02 kernel: CT: 104: stopped
Jan 26 20:46:15 vz02 kernel: CT: 104: started
Jan 26 20:46:17 vz02 kernel: CT: 105: started
Jan 26 20:46:19 vz02 kernel: CPT ERR: ffff81032156a000,105 :NLMERR: -22
Jan 26 20:46:19 vz02 last message repeated 8 times
Jan 26 20:46:19 vz02 kernel: CPT ERR: ffff81032156a000,105 :open_listening_socket: sock_create_kern: -97
Jan 26 20:46:19 vz02 kernel: CPT ERR: ffff81032156a000,105 :rst_sockets: open_listening_socket: -97
Jan 26 20:46:19 vz02 kernel: CPT ERR: ffff81032156a000,105 :rst_sockets: -97
Jan 26 20:46:19 vz02 kernel: CT: 105: stopped
Jan 26 20:46:21 vz02 kernel: CT: 105: started
Jan 26 20:46:24 vz02 gconfd (root-8825): GConf server is not in use, shutting down.
Jan 26 20:46:24 vz02 gconfd (root-8825): Exiting
Jan 26 20:46:24 vz02 kernel: CT: 106: started
Jan 26 20:46:26 vz02 kernel: CPT ERR: ffff8103185f3000,106 :NLMERR: -22
Jan 26 20:46:26 vz02 last message repeated 8 times
Jan 26 20:46:26 vz02 kernel: CPT ERR: ffff8103185f3000,106 :open_listening_socket: sock_create_kern: -97
Jan 26 20:46:26 vz02 kernel: CPT ERR: ffff8103185f3000,106 :rst_sockets: open_listening_socket: -97
Jan 26 20:46:26 vz02 kernel: CPT ERR: ffff8103185f3000,106 :rst_sockets: -97
Jan 26 20:46:27 vz02 kernel: CT: 106: stopped
Jan 26 20:46:28 vz02 kernel: CT: 106: started
Jan 26 20:46:31 vz02 kernel: CT: 107: started
Jan 26 20:46:35 vz02 kernel: CPT ERR: ffff810310d5b000,107 :open_listening_socket: sock_create_kern: -97
Jan 26 20:46:35 vz02 kernel: CPT ERR: ffff810310d5b000,107 :rst_sockets: open_listening_socket: -97
Jan 26 20:46:35 vz02 kernel: CPT ERR: ffff810310d5b000,107 :rst_sockets: -97
Jan 26 20:46:35 vz02 kernel: CT: 107: stopped
Jan 26 20:46:36 vz02 kernel: CT: 107: started
Jan 26 20:46:40 vz02 kernel: Can't load ipv6 module due to service vz is started
Jan 26 20:46:40 vz02 modprobe: FATAL: Error inserting ipv6 (/lib/modules/2.6.18-308.8.2.el5.028stab101.1/kernel
/net/ipv6/ipv6.ko): Device or resource busy
Jan 26 20:46:42 vz02 kernel: CT: 108: started
Jan 26 20:46:44 vz02 kernel: CPT ERR: ffff81030be0f000,108 :add ifaddr err -96 for 5 eth0
Jan 26 20:46:44 vz02 kernel: CPT ERR: ffff81030be0f000,108 :rst_restore_net: -96
Jan 26 20:46:46 vz02 kernel: CT: 108: stopped
Jan 26 20:46:47 vz02 kernel: CT: 108: started
Jan 26 20:46:51 vz02 kernel: CT: 111: started
Jan 26 20:46:54 vz02 kernel: CPT ERR: ffff8103108ba000,111 :unknown interface 2 as0t0
Jan 26 20:46:54 vz02 kernel: CPT ERR: ffff8103108ba000,111 :unknown ifaddr 2 for 25
Jan 26 20:46:54 vz02 kernel: CPT ERR: ffff8103108ba000,111 :rst_restore_net: -22
Jan 26 20:46:54 vz02 kernel: CT: 111: stopped
Jan 26 20:46:55 vz02 kernel: CT: 111: started
Jan 26 20:46:57 vz02 kernel: CT: 201: started
Jan 26 20:47:08 vz02 kernel: CPT ERR: ffff8103108ba000,201 :open_listening_socket: sock_create_kern: -97
Jan 26 20:47:08 vz02 kernel: CPT ERR: ffff8103108ba000,201 :rst_sockets: open_listening_socket: -97
Jan 26 20:47:08 vz02 kernel: CPT ERR: ffff8103108ba000,201 :rst_sockets: -97
Jan 26 20:47:08 vz02 kernel: CT: 201: stopped
Jan 26 20:47:10 vz02 kernel: CT: 201: started
Jan 26 20:47:21 vz02 kernel: CT: 203: started
Jan 26 20:47:26 vz02 kernel: CT: 204: started
Jan 26 20:47:28 vz02 kernel: CPT ERR: ffff8102fd325000,204 :open_listening_socket: sock_create_kern: -97
Jan 26 20:47:28 vz02 kernel: CPT ERR: ffff8102fd325000,204 :rst_sockets: open_listening_socket: -97
Jan 26 20:47:28 vz02 kernel: CPT ERR: ffff8102fd325000,204 :rst_sockets: -97
Jan 26 20:47:28 vz02 kernel: CT: 204: stopped
Jan 26 20:47:30 vz02 kernel: CT: 204: started
Jan 26 20:47:35 vz02 kernel: CT: 205: started
Jan 26 20:47:53 vz02 kernel: CT: 206: started
Jan 26 20:47:59 vz02 kernel: CPT ERR: ffff8102fd216000,206 :open_listening_socket: sock_create_kern: -97
Jan 26 20:47:59 vz02 kernel: CPT ERR: ffff8102fd216000,206 :rst_sockets: open_listening_socket: -97
Jan 26 20:47:59 vz02 kernel: CPT ERR: ffff8102fd216000,206 :rst_sockets: -97
Jan 26 20:47:59 vz02 kernel: CT: 206: stopped
Jan 26 20:48:00 vz02 kernel: CT: 206: started
"

The containers started, but could not access the internet and I could not access them.

After manually doing a vzctl restart on each container, networking started working again. Very strange!
Saw different errors after vzctl restart:

"
Jan 26 21:06:33 vz02 kernel: proc: unrecognized mount option "relatime" or missing value
Jan 26 21:06:36 vz02 kernel: CT: 103: stopped
Jan 26 21:06:37 vz02 kernel: CT: 103: started
Jan 26 21:06:51 vz02 kernel: proc: unrecognized mount option "relatime" or missing value
Jan 26 21:06:53 vz02 kernel: CT: 104: stopped
Jan 26 21:06:54 vz02 kernel: CT: 104: started
Jan 26 21:07:16 vz02 kernel: proc: unrecognized mount option "relatime" or missing value
Jan 26 21:07:18 vz02 kernel: CT: 107: stopped
Jan 26 21:07:19 vz02 kernel: CT: 107: started
Jan 26 21:07:59 vz02 kernel: proc: unrecognized mount option "relatime" or missing value
Jan 26 21:08:01 vz02 kernel: CT: 111: stopped
Jan 26 21:08:02 vz02 kernel: CT: 111: started
"

But networking started working...

I've spent so many hours trying to figure this out, I have no idea where to look at this point.

Running:

vzctl-core-4.1.2-1
vzctl-4.1.2-1

Example ctid.conf, the x.x.x.x's are of course not in there:

"
ONBOOT="yes"

NUMPROC="5102:5102"
AVNUMPROC="2551:2551"
NUMTCPSOCK="5102:5102"
NUMOTHERSOCK="5102:5102"
VMGUARPAGES="262144:9223372036854775807"

# Secondary parameters
KMEMSIZE="209012940:229914234"
TCPSNDBUF="48773188:69670980"
TCPRCVBUF="48773188:69670980"
OTHERSOCKBUF="24386594:45284386"
DGRAMRCVBUF="24386594:24386594"
OOMGUARPAGES="151485:9223372036854775807"
PRIVVMPAGES="256000:262140"

# Auxiliary parameters
LOCKEDPAGES="10205:10205"
SHMPAGES="90891:90891"
PHYSPAGES="0:9223372036854775807"
NUMFILE="81632:81632"
NUMFLOCK="1000:1100"
NUMPTY="510:510"
NUMSIGINFO="1024:1024"
DCACHESIZE="45650516:47020032"

NUMIPTENT="3072:3072"
# Disk Resource Limits
DISKINODES="4560000:4800000"
DISKSPACE="39845888:41943040"

# Quota Resource Limits
QUOTATIME="0"
QUOTAUGIDLIMIT="3000"

# CPU Resource Limits
CPUUNITS="1000"
#RATE="eth0:1:6000"

# IPTables config
IPTABLES="ipt_REJECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length ip_conntrack ip_conntrack_ftp ipt_LOG ipt_conntrack ipt_helper ipt_state iptable_nat ip_nat_ftp ipt_TOS ipt_REDIRECT"


IP_ADDRESS="x.x.x.x"
HOSTNAME="x.x.x.x"
VE_ROOT="/vz/root/$VEID"
VE_PRIVATE="/vz/private/$VEID"
OSTEMPLATE="centos-5-x86_64"
ORIGIN_SAMPLE="512mb"
NAMESERVER="x.x.x.x"
NAME="x.x.x.x"

Any help would be greatly appreciated!

Thanks in advance.

Report message to a moderator

Re: Container networking broken after upgrade to 2.6.18-308.8.2.el5.028stab101.1, stange kernel erro [message #51239 is a reply to message #48963] Sun, 16 March 2014 17:52 Go to previous messageGo to next message
bithead2 is currently offline  bithead2
Messages: 5
Registered: July 2009
Junior Member
Did you ever get a response to this?

I'm seeing this today:

Mar 16 10:47:34 MyVMName kernel: CPT ERR: ffff8101fe2c5000,252224 :NLMERR: -17

[root@MyVMName1 ~]# uname -a
Linux MyVMName.someplace.com 2.6.18-371.3.1.el5.028stab110.1 #1 SMP Fri Dec 13 15:01:51 MSK 2013 x86_64 x86_64 x86_64 GNU/Linux

My biggest complaint with OpenVZ is just the forums don't get a lot of traffic so when you find problems it's like a ghost town.

CJ

Report message to a moderator

Re: Container networking broken after upgrade to 2.6.18-308.8.2.el5.028stab101.1, stange kernel erro [message #51241 is a reply to message #51239] Mon, 17 March 2014 16:44 Go to previous message
PJF5000 is currently offline  PJF5000
Messages: 5
Registered: January 2013
Junior Member
Hi,

Good to know I'm not the only person who experiences this.

I ended up posting on OpenVZ's bugzilla and did indeed get a reply.

I can't seem to find the bug now, it's been closed or removed maybe?

It's not resolved for me, I have to restart the containers after a reboot by hand to get them online.

Out of curiosity, how many network cards do you have on this server?

I have many OpenVZ servers and this one has 3 physical network cards, all the others have 2 network cards and do not experience this issue. Could be totally unrelated...

Report message to a moderator

Previous Topic: Public IP via VETH seen as the bridge interface IP
Next Topic: Time inside container
Goto Forum:
  

[ Syndicate this forum (XML) ] [ RSS ] [ PDF ]

Current Time: Mon Jul 29 01:14:52 GMT 2024

Total time taken to generate the page: 0.02805 seconds
Powered by FUDforum Powered by Virtuozzo Containers