Home » General » Support » Server Crash, seemingly related to VZQUOTA
| Re: Server Crash, seemingly related to VZQUOTA [message #44939 is a reply to message #43721] |
Sat, 14 January 2012 20:30   |
epineda04
Messages: 2
Registered: September 2011
Location: Panama
|
Junior Member |
|
|
Hi all,
This seems to be a common problem.
We are a hosting provide and we had worked with Centos 5 before, and everything used to run smoothly, now that we have a new server with Centos 6, its different.
Despite of having scripts to kill abusers, certain processes as well, and a script to kill processes that use more than 20% of CPU for more than 1 minute, the server keeps on crashing.
Last time i had top running, and the load was 0.78 and all of a sudden the server becomes completely dead, ssh was dead and tty access was dead as well.
Then i started looking at the logs, and the amount of information is simply overwhelming.
Here is the info i can provide to maybe try to come up with a solution to this problem.
Kernel: 2.6.32-042stab039.11
vzctl version 3.0.29.3
Vzquota version 2.5.0
Centos 6 86_64 Bits
Hardware Node: Dual Intel(R) Xeon(R) CPU E5620 @ 2.40GHz Processors
RAM: 48GB
The logs are way to long, but this is kind of what it looks like before it goes completely dead:
Jan 14 13:13:28 S04001011820 kernel: [ 2.499590] usb usb1: Product: EHCI Host Controller
Jan 14 13:13:28 S04001011820 kernel: [ 2.499593] usb usb1: Manufacturer: Linux 2.6.32-042stab039.11 ehci_hcd
Jan 14 13:13:28 S04001011820 kernel: [ 2.499597] usb usb1: SerialNumber: 0000:00:1a.7
Jan 14 13:13:28 S04001011820 kernel: [ 2.499645] usb usb1: configuration #1 chosen from 1 choice
Jan 14 13:13:28 S04001011820 kernel: [ 2.499668] hub 1-0:1.0: USB hub found
Jan 14 13:13:28 S04001011820 kernel: [ 2.499672] hub 1-0:1.0: 6 ports detected
Jan 14 13:13:28 S04001011820 kernel: [ 2.499769] ehci_hcd 0000:00:1d.7: PCI INT A -> GSI 23 (level, low) -> IRQ 23
Jan 14 13:13:28 S04001011820 kernel: [ 2.499785] ehci_hcd 0000:00:1d.7: EHCI Host Controller
Jan 14 13:13:28 S04001011820 kernel: [ 2.499817] ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 2
Jan 14 13:13:28 S04001011820 kernel: [ 2.499841] ehci_hcd 0000:00:1d.7: debug port 1
Jan 14 13:13:28 S04001011820 kernel: [ 2.503722] ehci_hcd 0000:00:1d.7: irq 23, io mem 0xfbed8000
Jan 14 13:13:28 S04001011820 kernel: [ 2.513517] ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00
Jan 14 13:13:28 S04001011820 kernel: [ 2.513535] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
Jan 14 13:13:28 S04001011820 kernel: [ 2.513539] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Jan 14 13:13:28 S04001011820 kernel: [ 2.513543] usb usb2: Product: EHCI Host Controller
Jan 14 13:13:28 S04001011820 kernel: [ 2.513546] usb usb2: Manufacturer: Linux 2.6.32-042stab039.11 ehci_hcd
Jan 14 13:13:28 S04001011820 kernel: [ 2.513549] usb usb2: SerialNumber: 0000:00:1d.7
Jan 14 13:13:28 S04001011820 kernel: [ 2.513602] usb usb2: configuration #1 chosen from 1 choice
Jan 14 13:13:28 S04001011820 kernel: [ 2.513621] hub 2-0:1.0: USB hub found
Jan 14 13:13:28 S04001011820 kernel: [ 2.513625] hub 2-0:1.0: 6 ports detected
Jan 14 13:13:28 S04001011820 kernel: [ 2.513692] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
Jan 14 13:13:28 S04001011820 kernel: [ 2.513705] uhci_hcd: USB Universal Host Controller Interface driver
Jan 14 13:13:28 S04001011820 kernel: [ 2.513773] uhci_hcd 0000:00:1a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Jan 14 13:13:28 S04001011820 kernel: [ 2.513789] uhci_hcd 0000:00:1a.0: UHCI Host Controller
Jan 14 13:13:28 S04001011820 kernel: [ 2.513823] uhci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 3
Jan 14 13:13:28 S04001011820 kernel: [ 2.513855] uhci_hcd 0000:00:1a.0: irq 16, io base 0x0000bc00
Jan 14 13:13:28 S04001011820 kernel: [ 2.513882] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
Jan 14 13:13:28 S04001011820 kernel: [ 2.513884] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Jan 14 13:13:28 S04001011820 kernel: [ 2.513887] usb usb3: Product: UHCI Host Controller
Jan 14 13:13:28 S04001011820 kernel: [ 2.513889] usb usb3: Manufacturer: Linux 2.6.32-042stab039.11 uhci_hcd
Jan 14 13:13:28 S04001011820 kernel: [ 2.513891] usb usb3: SerialNumber: 0000:00:1a.0
Jan 14 13:13:28 S04001011820 kernel: [ 2.513924] usb usb3: configuration #1 chosen from 1 choice
Jan 14 13:13:28 S04001011820 kernel: [ 2.513944] hub 3-0:1.0: USB hub found
Jan 14 13:13:28 S04001011820 kernel: [ 2.513948] hub 3-0:1.0: 2 ports detected
Jan 14 13:13:28 S04001011820 kernel: [ 2.514045] uhci_hcd 0000:00:1a.1: PCI INT B -> GSI 21 (level, low) -> IRQ 21
Jan 14 13:13:28 S04001011820 kernel: [ 2.514061] uhci_hcd 0000:00:1a.1: UHCI Host Controller
Jan 14 13:13:28 S04001011820 kernel: [ 2.514089] uhci_hcd 0000:00:1a.1: new USB bus registered, assigned bus number 4
Jan 14 13:13:28 S04001011820 kernel: [ 2.514122] uhci_hcd 0000:00:1a.1: irq 21, io base 0x0000b880
Jan 14 13:13:28 S04001011820 kernel: [ 2.514148] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001
Jan 14 13:13:28 S04001011820 kernel: [ 2.514150] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Jan 14 13:13:28 S04001011820 kernel: [ 2.514153] usb usb4: Product: UHCI Host Controller
Jan 14 13:13:28 S04001011820 kernel: [ 2.514155] usb usb4: Manufacturer: Linux 2.6.32-042stab039.11 uhci_hcd
Jan 14 13:13:28 S04001011820 kernel: [ 2.514157] usb usb4: SerialNumber: 0000:00:1a.1
Jan 14 13:13:28 S04001011820 kernel: [ 2.514191] usb usb4: configuration #1 chosen from 1 choice
Jan 14 13:13:28 S04001011820 kernel: [ 2.514210] hub 4-0:1.0: USB hub found
Jan 14 13:13:28 S04001011820 kernel: [ 2.514214] hub 4-0:1.0: 2 ports detected
Jan 14 13:13:28 S04001011820 kernel: [ 2.514295] uhci_hcd 0000:00:1a.2: PCI INT D -> GSI 19 (level, low) -> IRQ 19
Jan 14 13:13:28 S04001011820 kernel: [ 2.514305] uhci_hcd 0000:00:1a.2: UHCI Host Controller
Jan 14 13:13:28 S04001011820 kernel: [ 2.514334] uhci_hcd 0000:00:1a.2: new USB bus registered, assigned bus number 5
Jan 14 13:13:28 S04001011820 kernel: [ 2.514365] uhci_hcd 0000:00:1a.2: irq 19, io base 0x0000b800
Jan 14 13:13:28 S04001011820 kernel: [ 2.514392] usb usb5: New USB device found, idVendor=1d6b, idProduct=0001
Jan 14 13:13:28 S04001011820 kernel: [ 2.514395] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Jan 14 13:13:28 S04001011820 kernel: [ 2.514397] usb usb5: Product: UHCI Host Controller
Jan 14 13:13:28 S04001011820 kernel: [ 2.514399] usb usb5: Manufacturer: Linux 2.6.32-042stab039.11 uhci_hcd
Jan 14 13:13:28 S04001011820 kernel: [ 2.514401] usb usb5: SerialNumber: 0000:00:1a.2
Jan 14 13:13:28 S04001011820 kernel: [ 2.514435] usb usb5: configuration #1 chosen from 1 choice
Jan 14 13:13:28 S04001011820 kernel: [ 2.514455] hub 5-0:1.0: USB hub found
Jan 14 13:13:28 S04001011820 kernel: [ 2.514458] hub 5-0:1.0: 2 ports detected
Jan 14 13:13:28 S04001011820 kernel: [ 2.514540] uhci_hcd 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
Jan 14 13:13:28 S04001011820 kernel: [ 2.514549] uhci_hcd 0000:00:1d.0: UHCI Host Controller
Jan 14 13:13:28 S04001011820 kernel: [ 3.531204] scsi 0:0:1:0: Direct-Access ATA ST32000542AS CC34 PQ: 0 ANSI: 5
Jan 14 13:13:28 S04001011820 kernel: [ 3.536701] ata2.01: configured for UDMA/133
Jan 14 13:13:28 S04001011820 kernel: [ 3.536942] scsi 1:0:0:0: Direct-Access ATA ST32000542AS CC34 PQ: 0 ANSI: 5
Jan 14 13:13:28 S04001011820 kernel: [ 3.537071] scsi 1:0:1:0: Direct-Access ATA ST31000524AS JC45 PQ: 0 ANSI: 5
Jan 14 13:13:28 S04001011820 kernel: [ 3.584547] sd 1:0:0:0: [sdc] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
Jan 14 13:13:28 S04001011820 kernel: [ 3.584563] sd 0:0:0:0: [sda] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
Jan 14 13:13:28 S04001011820 kernel: [ 3.584594] sd 0:0:1:0: [sdb] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
Jan 14 13:13:28 S04001011820 kernel: [ 3.584600] sd 1:0:0:0: [sdc] Write Protect is off
Jan 14 13:13:28 S04001011820 kernel: [ 3.584621] sd 1:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Jan 14 13:13:28 S04001011820 kernel: [ 3.584687] sd 0:0:0:0: [sda] Write Protect is off
Jan 14 13:13:28 S04001011820 kernel: [ 3.584692] sd 1:0:1:0: [sdd] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
Jan 14 13:13:28 S04001011820 kernel: [ 3.584701] sd 0:0:1:0: [sdb] Write Protect is off
Jan 14 13:13:28 S04001011820 kernel: [ 3.584728] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Jan 14 13:13:28 S04001011820 kernel: [ 3.584734] sd 0:0:1:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Jan 14 13:13:28 S04001011820 kernel: [ 3.584845] sd 1:0:1:0: [sdd] Write Protect is off
Jan 14 13:13:28 S04001011820 kernel: [ 3.584885] sd 1:0:1:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Jan 14 13:13:28 S04001011820 kernel: [ 3.584975] sdb:
Jan 14 13:13:28 S04001011820 kernel: [ 3.585067] sda:
Jan 14 13:13:28 S04001011820 kernel: [ 3.585103] sdc: sdb1
Jan 14 13:13:28 S04001011820 kernel: [ 3.591953] sda1 sda2
Jan 14 13:13:28 S04001011820 kernel: [ 3.592688] sd 0:0:0:0: [sda] Attached SCSI disk
Jan 14 13:13:28 S04001011820 kernel: [ 3.592746] sd 0:0:1:0: [sdb] Attached SCSI disk
Jan 14 13:13:28 S04001011820 kernel: [ 3.628574] sdc1
Jan 14 13:13:28 S04001011820 kernel: [ 3.629237] sdd: sdd1
Jan 14 13:13:28 S04001011820 kernel: [ 3.647975] sd 1:0:0:0: [sdc] Attached SCSI disk
Jan 14 13:13:28 S04001011820 kernel: [ 3.648718] sd 1:0:1:0: [sdd] Attached SCSI disk
Jan 14 13:13:28 S04001011820 kernel: [ 3.893018] dracut: Scanning for dmraid devices ddf1_4c5349202020202010000055000000004711471100001450
Jan 14 13:13:28 S0400101182
...
|
|
|
|
Current Time: Fri Aug 22 21:27:11 GMT 2025
Total time taken to generate the page: 0.14722 seconds
|
OpenVZ Forum
Members
Search
Help
Register
Login
Home
