*HARDWARE* Node crashed with Machine Check Exception error
Hi, Our openvz server crashed three times in a day with these errors. Is it something to with openvz kernel or can it be because of bad cpus (we have dual core dual amd opteron 270)? we use: 2.6.8-022stab078.10-enterprise Jul 9 12:19:41 vz01 kernel: CPU 0: Machine Check Exception: 0000000000000004 Jul 9 12:19:41 vz01 kernel: Bank 4: b604a00100000813 at 000000007e4f2f68 Jul 9 12:19:41 vz01 kernel: Kernel panic: CPU context corrupt Jul 9 05:44:28 vz01 kernel: CPU 0: Machine Check Exception: 0000000000000007 Jul 9 05:44:28 vz01 kernel: Unable to handle kernel paging request at virtual address 00f00094 Jul 9 05:44:28 vz01 kernel: printing eip: Jul 9 05:44:28 vz01 kernel: 023d7d02 Jul 9 05:44:28 vz01 kernel: *pde = 00000000 Jul 9 05:44:28 vz01 kernel: Oops: 0000 [#1] Jul 9 05:44:28 vz01 kernel: Bank 4: b404a00000000a13 at 0000000055c8df68 Jul 9 05:44:28 vz01 kernel: Kernel panic: Unable to continue Jul 9 05:44:28 vz01 kernel: SMP Jul 9 05:44:28 vz01 kernel: Modules linked in: simfs vzdquota af_packet ip_nat_ftp ip_conntrack_ftp iptable_nat ipt_state ipt_length ipt_ttl ipt_tcpmss ipt_$ Jul 9 05:44:28 vz01 kernel: CPU: 3, VCPU: 101:3 Jul 9 05:44:28 vz01 kernel: EIP: 0060:[<023d7d02>] Not tainted Jul 9 05:44:28 vz01 kernel: EFLAGS: 00010282 (2.6.8-022stab078.10-enterprise) Jul 9 05:44:30 vz01 kernel: EIP is at skb_drop_fraglist+0x22/0x50 Jul 9 05:44:30 vz01 kernel: eax: 57c8df60 ebx: 00f00000 ecx: 57c8df60 edx: 00f00000 Jul 9 05:44:30 vz01 kernel: esi: c0c0b0c0 edi: 0000003a ebp: 09724354 esp: 875cddc8 Jul 9 05:44:30 vz01 kernel: ds: 007b es: 007b ss: 0068 Jul 9 05:44:30 vz01 kernel: Process spamd (pid: 2662, veid=101, threadinfo=875cc000 task=b7170d40) Jul 9 05:44:31 vz01 kernel: Stack: 001b1b1f c0c0b0c0 023d7dea c0c0b0c0 c0c0b0c0 c0c0b0c0 023d7e20 c0c0b0c0 Jul 9 05:44:31 vz01 kernel: 0000003a 00000000 023d7ee1 c0c0b0c0 00000000 c0c0b0c0 0000003a 00000000 Jul 9 05:44:31 vz01 kernel: 02402e35 c0c0b0c0 00000000 875cdedc 0000003a 7690e600 c059f200 875cc000 Jul 9 05:44:31 vz01 kernel: Call Trace: Jul 9 05:44:31 vz01 kernel: [<023d7dea>] skb_release_data+0x9a/0xc0 Jul 9 05:44:31 vz01 kernel: [<023d7e20>] kfree_skbmem+0x10/0x30 Jul 9 05:44:32 vz01 kernel: [<023d7ee1>] __kfree_skb+0xa1/0x140 Jul 9 05:44:32 vz01 kernel: [<02402e35>] tcp_recvmsg+0x715/0x890 Jul 9 05:44:32 vz01 kernel: [<023d7942>] sock_common_recvmsg+0x52/0x70 Jul 9 05:44:32 vz01 kernel: [<023d3cf0>] sock_aio_read+0x100/0x120 Jul 9 05:44:32 vz01 kernel: [<02171dc0>] do_sync_read+0x80/0xc0 Jul 9 05:44:32 vz01 kernel: [<0211bfbb>] vcpu_put+0x8b/0x110 Jul 9 05:44:32 vz01 kernel: [<0211d61e>] finish_task_switch+0x3e/0x90 Jul 9 05:44:32 vz01 kernel: [<02171e9a>] vfs_read+0x9a/0x160 Jul 9 05:44:32 vz01 kernel: [<021721d1>] sys_read+0x51/0x80 Jul 9 05:44:32 vz01 kernel: Code: 8b 82 94 00 00 00 8b 1b 48 74 0e f0 ff 8a 94 00 00 00 0f 94 Jul 9 22:35:59 vz01 kernel: CPU 2: Machine Check Exception: 0000000000000004 Jul 9 22:35:59 vz01 kernel: Bank 4: b603200100000813 at 0000000104b27ef8 Jul 9 22:35:59 vz01 kernel: Kernel panic: CPU context corrupt
Re: Node crashed with Machine Check Exception error
Such messages are printed when CPU reports about detected hardware problems, e.g. memory corruptions or some other problems. i.e. it is 100% hardware problem with your server. http://www.answers.com/topic/machine-check-exception You can try to check your memory/CPU as decribed here: http://wiki.openvz.org/Hardware_testing
AFAICS it is the same machine, yeah? it can be also your hardware, though I will check for obvious errors.
Thank you, Yes thats the same machine, I also compiled the kernel myself but got this error, DC also replaced all the RAM modules. Now they replaced them again, I think they will swap the CPUs next time.
I really suggest you to run some hardware tests as described in http://wiki.openvz.org/Hardware_testing