Home » General » Support » Container networking broken after upgrade to 2.6.18-308.8.2.el5.028stab101.1, stange kernel errors (After upgrading my CentOS 5.9 server to 2.6.18-308.8.2.el5.028stab101.1 I'm having issues with networking in containers working and kernel CPT ERR's)
| Container networking broken after upgrade to 2.6.18-308.8.2.el5.028stab101.1, stange kernel errors [message #48963] |
Mon, 28 January 2013 18:01  |
PJF5000
Messages: 5
Registered: January 2013
|
Junior Member |
|
|
Hi there, not sure if this should be a bug report, or if there is a strange configuraiton issue on my server.
Running a CentOS 5.9 x86_64 server, recelntly rebooted into the 2.6.18-308.8.2.el5.028stab101.1 kernel last weekend.
Upon booting into the new kernel I saw many kernel CPT ERR's while starting the containers:
"
Jan 20:46:08 vz02 last message repeated 8 times
Jan 26 20:46:12 vz02 kernel: CT: 104: started
Jan 26 20:46:14 vz02 kernel: CPT ERR: ffff81031e46a000,104 :open_listening_socket: sock_create_kern: -97
Jan 26 20:46:14 vz02 kernel: CPT ERR: ffff81031e46a000,104 :rst_sockets: open_listening_socket: -97
Jan 26 20:46:14 vz02 kernel: CPT ERR: ffff81031e46a000,104 :rst_sockets: -97
Jan 26 20:46:14 vz02 kernel: CT: 104: stopped
Jan 26 20:46:15 vz02 kernel: CT: 104: started
Jan 26 20:46:17 vz02 kernel: CT: 105: started
Jan 26 20:46:19 vz02 kernel: CPT ERR: ffff81032156a000,105 :NLMERR: -22
Jan 26 20:46:19 vz02 last message repeated 8 times
Jan 26 20:46:19 vz02 kernel: CPT ERR: ffff81032156a000,105 :open_listening_socket: sock_create_kern: -97
Jan 26 20:46:19 vz02 kernel: CPT ERR: ffff81032156a000,105 :rst_sockets: open_listening_socket: -97
Jan 26 20:46:19 vz02 kernel: CPT ERR: ffff81032156a000,105 :rst_sockets: -97
Jan 26 20:46:19 vz02 kernel: CT: 105: stopped
Jan 26 20:46:21 vz02 kernel: CT: 105: started
Jan 26 20:46:24 vz02 gconfd (root-8825): GConf server is not in use, shutting down.
Jan 26 20:46:24 vz02 gconfd (root-8825): Exiting
Jan 26 20:46:24 vz02 kernel: CT: 106: started
Jan 26 20:46:26 vz02 kernel: CPT ERR: ffff8103185f3000,106 :NLMERR: -22
Jan 26 20:46:26 vz02 last message repeated 8 times
Jan 26 20:46:26 vz02 kernel: CPT ERR: ffff8103185f3000,106 :open_listening_socket: sock_create_kern: -97
Jan 26 20:46:26 vz02 kernel: CPT ERR: ffff8103185f3000,106 :rst_sockets: open_listening_socket: -97
Jan 26 20:46:26 vz02 kernel: CPT ERR: ffff8103185f3000,106 :rst_sockets: -97
Jan 26 20:46:27 vz02 kernel: CT: 106: stopped
Jan 26 20:46:28 vz02 kernel: CT: 106: started
Jan 26 20:46:31 vz02 kernel: CT: 107: started
Jan 26 20:46:35 vz02 kernel: CPT ERR: ffff810310d5b000,107 :open_listening_socket: sock_create_kern: -97
Jan 26 20:46:35 vz02 kernel: CPT ERR: ffff810310d5b000,107 :rst_sockets: open_listening_socket: -97
Jan 26 20:46:35 vz02 kernel: CPT ERR: ffff810310d5b000,107 :rst_sockets: -97
Jan 26 20:46:35 vz02 kernel: CT: 107: stopped
Jan 26 20:46:36 vz02 kernel: CT: 107: started
Jan 26 20:46:40 vz02 kernel: Can't load ipv6 module due to service vz is started
Jan 26 20:46:40 vz02 modprobe: FATAL: Error inserting ipv6 (/lib/modules/2.6.18-308.8.2.el5.028stab101.1/kernel
/net/ipv6/ipv6.ko): Device or resource busy
Jan 26 20:46:42 vz02 kernel: CT: 108: started
Jan 26 20:46:44 vz02 kernel: CPT ERR: ffff81030be0f000,108 :add ifaddr err -96 for 5 eth0
Jan 26 20:46:44 vz02 kernel: CPT ERR: ffff81030be0f000,108 :rst_restore_net: -96
Jan 26 20:46:46 vz02 kernel: CT: 108: stopped
Jan 26 20:46:47 vz02 kernel: CT: 108: started
Jan 26 20:46:51 vz02 kernel: CT: 111: started
Jan 26 20:46:54 vz02 kernel: CPT ERR: ffff8103108ba000,111 :unknown interface 2 as0t0
Jan 26 20:46:54 vz02 kernel: CPT ERR: ffff8103108ba000,111 :unknown ifaddr 2 for 25
Jan 26 20:46:54 vz02 kernel: CPT ERR: ffff8103108ba000,111 :rst_restore_net: -22
Jan 26 20:46:54 vz02 kernel: CT: 111: stopped
Jan 26 20:46:55 vz02 kernel: CT: 111: started
Jan 26 20:46:57 vz02 kernel: CT: 201: started
Jan 26 20:47:08 vz02 kernel: CPT ERR: ffff8103108ba000,201 :open_listening_socket: sock_create_kern: -97
Jan 26 20:47:08 vz02 kernel: CPT ERR: ffff8103108ba000,201 :rst_sockets: open_listening_socket: -97
Jan 26 20:47:08 vz02 kernel: CPT ERR: ffff8103108ba000,201 :rst_sockets: -97
Jan 26 20:47:08 vz02 kernel: CT: 201: stopped
Jan 26 20:47:10 vz02 kernel: CT: 201: started
Jan 26 20:47:21 vz02 kernel: CT: 203: started
Jan 26 20:47:26 vz02 kernel: CT: 204: started
Jan 26 20:47:28 vz02 kernel: CPT ERR: ffff8102fd325000,204 :open_listening_socket: sock_create_kern: -97
Jan 26 20:47:28 vz02 kernel: CPT ERR: ffff8102fd325000,204 :rst_sockets: open_listening_socket: -97
Jan 26 20:47:28 vz02 kernel: CPT ERR: ffff8102fd325000,204 :rst_sockets: -97
Jan 26 20:47:28 vz02 kernel: CT: 204: stopped
Jan 26 20:47:30 vz02 kernel: CT: 204: started
Jan 26 20:47:35 vz02 kernel: CT: 205: started
Jan 26 20:47:53 vz02 kernel: CT: 206: started
Jan 26 20:47:59 vz02 kernel: CPT ERR: ffff8102fd216000,206 :open_listening_socket: sock_create_kern: -97
Jan 26 20:47:59 vz02 kernel: CPT ERR: ffff8102fd216000,206 :rst_sockets: open_listening_socket: -97
Jan 26 20:47:59 vz02 kernel: CPT ERR: ffff8102fd216000,206 :rst_sockets: -97
Jan 26 20:47:59 vz02 kernel: CT: 206: stopped
Jan 26 20:48:00 vz02 kernel: CT: 206: started
"
The containers started, but could not access the internet and I could not access them.
After manually doing a vzctl restart on each container, networking started working again. Very strange!
Saw different errors after vzctl restart:
"
Jan 26 21:06:33 vz02 kernel: proc: unrecognized mount option "relatime" or missing value
Jan 26 21:06:36 vz02 kernel: CT: 103: stopped
Jan 26 21:06:37 vz02 kernel: CT: 103: started
Jan 26 21:06:51 vz02 kernel: proc: unrecognized mount option "relatime" or missing value
Jan 26 21:06:53 vz02 kernel: CT: 104: stopped
Jan 26 21:06:54 vz02 kernel: CT: 104: started
Jan 26 21:07:16 vz02 kernel: proc: unrecognized mount option "relatime" or missing value
Jan 26 21:07:18 vz02 kernel: CT: 107: stopped
Jan 26 21:07:19 vz02 kernel: CT: 107: started
Jan 26 21:07:59 vz02 kernel: proc: unrecognized mount option "relatime" or missing value
Jan 26 21:08:01 vz02 kernel: CT: 111: stopped
Jan 26 21:08:02 vz02 kernel: CT: 111: started
"
But networking started working...
I've spent so many hours trying to figure this out, I have no idea where to look at this point.
Running:
vzctl-core-4.1.2-1
vzctl-4.1.2-1
Example ctid.conf, the x.x.x.x's are of course not in there:
"
ONBOOT="yes"
NUMPROC="5102:5102"
AVNUMPROC="2551:2551"
NUMTCPSOCK="5102:5102"
NUMOTHERSOCK="5102:5102"
VMGUARPAGES="262144:9223372036854775807"
# Secondary parameters
KMEMSIZE="209012940:229914234"
TCPSNDBUF="48773188:69670980"
TCPRCVBUF="48773188:69670980"
OTHERSOCKBUF="24386594:45284386"
DGRAMRCVBUF="24386594:24386594"
OOMGUARPAGES="151485:9223372036854775807"
PRIVVMPAGES="256000:262140"
# Auxiliary parameters
LOCKEDPAGES="10205:10205"
SHMPAGES="90891:90891"
PHYSPAGES="0:9223372036854775807"
NUMFILE="81632:81632"
NUMFLOCK="1000:1100"
NUMPTY="510:510"
NUMSIGINFO="1024:1024"
DCACHESIZE="45650516:47020032"
NUMIPTENT="3072:3072"
# Disk Resource Limits
DISKINODES="4560000:4800000"
DISKSPACE="39845888:41943040"
# Quota Resource Limits
QUOTATIME="0"
QUOTAUGIDLIMIT="3000"
# CPU Resource Limits
CPUUNITS="1000"
#RATE="eth0:1:6000"
# IPTables config
IPTABLES="ipt_REJECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length ip_conntrack ip_conntrack_ftp ipt_LOG ipt_conntrack ipt_helper ipt_state iptable_nat ip_nat_ftp ipt_TOS ipt_REDIRECT"
IP_ADDRESS="x.x.x.x"
HOSTNAME="x.x.x.x"
VE_ROOT="/vz/root/$VEID"
VE_PRIVATE="/vz/private/$VEID"
OSTEMPLATE="centos-5-x86_64"
ORIGIN_SAMPLE="512mb"
NAMESERVER="x.x.x.x"
NAME="x.x.x.x"
Any help would be greatly appreciated!
Thanks in advance.
|
|
|
|
Current Time: Sun Nov 10 14:33:31 GMT 2024
Total time taken to generate the page: 0.04569 seconds
|
OpenVZ Forum
Members
Search
Help
Register
Login
Home
