I'm had a similar crash using the latest 686 kernel from
download.openvz.org:
linux-image-2.6.18-openvz-13-39.1d2-686_028.39.1d2_i386.deb
vzctl --version
vzctl version 3.0.11
Two boxes are using it, one is fine, the other hung once.
All the logs had was this:
Oct 5 19:15:01 penguin /USR/SBIN/CRON[31518]: (root) CMD (/usr/share/vzctl/scripts/vpsnetclean)
Oct 8 10:34:35 penguin syslogd 1.4.1#18: restart.
There was some spew on the console, but I didn't know how to capture that.
E Frank Ball efball@efball.com
On Wed, Oct 10, 2007 at 07:54:11AM +0200, Martin Trtusek wrote:
> I installed kernel 2.6.18-openvz-13-39.1d1-amd64 from
> http://download.openvz.org/debian on Debian Etch one week ago and
> experienced kernel oops (complete freezing, off/on necessary) after 2-3
> days of running (3 times). Oops is always after cron.daily scripts (in
> my case 06:25) but not everyday. Yesterday I configured netconsole for
> capturing useful info, enclosed.
>
> Hardware was tested very strong on installation. With stock Debian
> kernel (initrd.img-2.6.18-5-amd64) server does not have any problem (3
> months of operation). There are 3 VPS running, without really using.
>
> Enclosed last entry in syslog (before crash). Looks like problem is
> invoking by /usr/share/vzctl/scripts/vpsnetclean
> or /usr/share/vzctl/scripts/vpsreboot. Booth scripts are from vzctl
> package, I installed it from http://debian.systs.org/
>
> # vzctl --version
> vzctl version 3.0.18-1dso1
>
> I am leaving office now, additional info (if necessary) I can send
> tomorrow.
>
> Martin Trtusek
> netconsole: network logging started
> Warning: /proc/ide/hd?/settings interface is obsolete, and will be removed soon!
> st: Version 20050830, fixed bufsize 32768, s/g segs 256
> sd 0:0:0:0: Attached scsi generic sg0 type 0
> sd 0:0:1:0: Attached scsi generic sg1 type 0
> sd 1:0:0:0: Attached scsi generic sg2 type 0
> sd 1:0:1:0: Attached scsi generic sg3 type 0
> BIOS EDD facility v0.16 2004-Jun-25, 4 devices found
> ----------- [cut here ] --------- [please bite here ] ---------
> Kernel BUG at kernel/sched.c:3798
> invalid opcode: 0000 [1] SMP
> CPU: 0
> Modules linked in: edd joydev sg st sr_mod netconsole vzethdev vznetdev simfs vzrst vzcpt vzdquota vzmon vzdev button ac battery ip6table_filter ip6_tables iptable_raw xt_policy xt_multiport ipt_ULOG ipt_TTL ipt_ttl ipt_TOS ipt_tos ipt_TCPMSS ipt_SAME ipt_REJECT ipt_REDIRECT ipt_recent ipt_owner ipt_NETMAP ipt_MASQUERADE ipt_LOG ipt_iprange ipt_hashlimit ipt_ECN ipt_ecn ipt_DSCP ipt_dscp ipt_CLUSTERIP ipt_ah ipt_addrtype ip_nat_tftp ip_nat_snmp_basic ip_nat_pptp ip_nat_irc ip_nat_ftp ip_nat_amanda ip_conntrack_tftp ip_conntrack_pptp ip_conntrack_netbios_ns ip_conntrack_irc ip_conntrack_ftp ts_kmp ip_conntrack_amanda xt_tcpmss xt_pkttype xt_physdev bridge xt_NFQUEUE xt_MARK xt_mark xt_mac xt_limit xt_length xt_helper xt_dccp xt_conntrack xt_CONNMARK xt_connmark xt_CLASSIFY xt_tcpudp xt_state iptable_nat ip_nat ip_conntrack iptable_mangle nfnetlink iptable_filter ip_tables x_tables ipv6 dummy aes_x86_64 sha512 sha256 loop evdev psmouse i2c_i801 shpchp pci_hotplug serio_raw !
i2c_core pcspkr floppy ext3 jbd mbcache raid10 ide_generic sd_mod ide_cd cdrom ata_piix libata generic piix ide_core ehci_hcd e1000 uhci_hcd thermal processor fan cciss scsi_mod dm_snapshot dm_mirror dm_crypt dm_mod raid456 xor raid1 md_mod
> Pid: 0, comm: swapper Not tainted 2.6.18-openvz-13-39.1d1-amd64 #1
> RIP: 0060:[<ffffffff8027fd35>] [<ffffffff8027fd35>] rebalance_tick+0x391/0x57a
> RSP: 0068:ffffffff804c4b18 EFLAGS: 00010046
> RAX: ffffffff804e1980 RBX: ffffffff804e1980 RCX: 0000000000000020
> RDX: 0000000000000020 RSI: ffffffff804e17d0 RDI: 0000000000000001
> RBP: ffffffff804c4bb8 R08: ffffffff804c4b68 R09: ffffffff804c4b68
> R10: 0000000000000000 R11: 0000000000000002 R12: ffff810001020340
> R13: ffff81011b0c2000 R14: ffff81011b0c2000 R15: ffffffff804e2c80
> FS: 0000000000000000(0000) GS:ffffffff80526000(0000) knlGS:0000000000000000
> CS: 0060 DS: 0068 ES: 0068 CR0: 000000008005003b
> CR2: 0000000000a10048 CR3: 000000010196d000 CR4: 00000000000006e0
> Process swapper (pid: 0, veid=0, threadinfo ffffffff80534000, task ffffffff80449be0)
> Stack: 0000000000000002 ffff810101161240 0000000000000000 ffffffff804e2c80
> 0000000202555ed8 ffff81011b0c2000 ffffffff8027b532 0000000000000001
> 0000000000000003 0000000000000082 00000000ffffffff 000047783a3fde8a
> Call Trace:
> <IRQ> [<ffffffff8027b532>] vcpu_attach+0x7e/0xc3
> [<ffffffff8028a68f>] update_process_times+0x5c/0x68
> [<ffffffff8026c8e1>] smp_local_timer_interrupt+0x23/0x47
> [<ffffffff8026ce7f>] smp_apic_timer_interrupt+0x99/0x9f
> [<ffffffff8025bdda>] apic_timer_interrupt+0x66/0x6c
> [<ffffffff8024e78d>] bio_fs_destructor+0x0/0xc
> [<ffffffff88124830>] :libata:ata_scsi_rw_xlat+0x0/0x37e
> [<ffffffff8022d780>] mempool_free+0x10/0x74
> [<ffffffff8023f729>] bio_free+0x33/0x43
> [<ffffffff8023f176>] end_bio_bh_io_sync+0x37/0x3b
> [<ffffffff88042255>] :dm_mod:dec_pending+0xab/0xce
> [<ffffffff88042392>] :dm_mod:clone_endio+0x7f/0x9b
> [<ffffffff88162aa6>] :raid10:raid_end_bio_io+0x2c/0x80
> [<ffffffff881645b2>] :raid10:raid10_end_read_request+0x66/0xe9
> [<ffffffff803010b1>] elv_next_request+0x141/0x151
> [<ffffffff8022b938>] __end_that_request_first+0x153/0x49e
> [<ffffffff803117ce>] swiotlb_unmap_sg+0x9c/0xed
> [<ffffffff8806b597>] :scsi_mod:scsi_delete_timer+0x12/0x59
> [<ffffffff8806cbf9>] :scsi_mod:scsi_end_request+0x27/0xcb
> [<ffffffff8806cdf3>] :scsi_mod:scsi_io_completion+0x156/0x334
> [<ffffffff881203fd>] :libata:ata_hsm_move+0x642/0x661
> [<ffffffff881584a3>] :sd_mod:sd_rw_intr+0x217/0x244
> [<ffffffff8806d091>] :scsi_mod:scsi_device_unbusy+0x67/0x81
> [<ffffffff80236488>] blk_done_softirq+0x5f/0x6d
> [<ffffffff8021030f>] __do_softirq+0x98/0x138
> [<ffffffff8025c43c>] call_softirq+0x1c/0x28
> [<ffffffff802661c3>] do_softirq+0x2c/0x7d
> [<ffffffff802662d6>] do_IRQ+0xc2/0xcb
> [<ffffffff80255128>] mwait_idle+0x0/0x4a
> [<ffffffff8025b761>] ret_from_intr+0x0/0xa
> <EOI> [<ffffffff8025515e>] mwait_idle+0x36/0x4a
> [<ffffffff8024703e>] cpu_idle+0x60/0x7f
> [<ffffffff8053e7be>] start_kernel+0x23b/0x240
> [<ffffffff8053e288>] _sinittext+0x288/0x28c
>
>
> Code: 0f 0b 68 df 6e 40 80 c2 d6 0e 4c 39 eb 48 89 5d 88 0f 84 57
> RIP [<ffffffff8027fd35>] rebalance_tick+0x391/0x57a
> RSP <ffffffff804c4b18>
> Kernel panic - not syncing: Aiee, killing interrupt handler!
> Oct 10 06:15:01 vochomurka /USR/SBIN/CRON[14743]: (root) CMD (if [ -x /etc/munin/plugins/apt_all ]; then /etc/munin/plugins/apt_all update 7200 12 >/dev/null; elif [ -x /etc/munin/plugins/apt ]; then /etc/munin/plugins/apt update 7200 12 >/dev/null;
> fi)
> Oct 10 06:15:01 vochomurka /USR/SBIN/CRON[14745]: (munin) CMD (if [ -x /usr/bin/munin-cron ]; then /usr/bin/munin-cron; chmod -R o+r /var/www/munin; fi)
> Oct 10 06:15:01 vochomurka /USR/SBIN/CRON[14747]: (root) CMD (/usr/share/vzctl/scripts/vpsreboot)
> Oct 10 06:15:01 vochomurka /USR/SBIN/CRON[14749]: (root) CMD (/usr/share/vzctl/scripts/vpsnetclean)
> Oct 10 06:17:01 vochomurka /USR/SBIN/CRON[16783]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
> Oct 10 06:17:10 vochomurka ntpdate[16786]: adjust time server 195.113.144.238 offset -0.007761 sec
> Oct 10 06:20:01 vochomurka /USR/SBIN/CRON[16788]: (root) CMD (if [ -x /etc/munin/plugins/apt_all ]; then /etc/munin/plugins/apt_all update 7200 12 >/dev/null; elif [ -x /etc/munin/plugins/apt ]; then /etc/munin/plugins/apt update 7200 12 >/dev/null;
> fi)
> Oct 10 06:20:01 vochomurka /USR/SBIN/CRON[16790]: (munin) CMD (if [ -x /usr/bin/munin-cron ]; then /usr/bin/munin-cron; chmod -R o+r /var/www/munin; fi)
> Oct 10 06:20:01 vochomurka /USR/SBIN/CRON[16792]: (root) CMD (/usr/share/vzctl/scripts/vpsreboot)
> Oct 10 06:20:01 vochomurka /USR/SBIN/CRON[16794]: (root) CMD (/usr/share/vzctl/scripts/vpsnetclean)
> Oct 10 06:25:01 vochomurka /USR/SBIN/CRON[18831]: (root) CMD (test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ))
> Oct 10 06:25:01 vochomurka /USR/SBIN/CRON[18833]: (root) CMD (if [ -x /etc/munin/plugins/apt_all ]; then /etc/munin/plugins/apt_all update 7200 12 >/dev/null; elif [ -x /etc/munin/plugins/apt ]; then /etc/munin/plugins/apt update 7200 12 >/dev/null;
> fi)
> Oct 10 06:25:01 vochomurka /USR/SBIN/CRON[18839]: (munin) CMD (if [ -x /usr/bin/munin-cron ]; then /usr/bin/munin-cron; chmod -R o+r /var/www/munin; fi)
> Oct 10 06:25:01 vochomurka /USR/SBIN/CRON[18840]: (root) CMD (/usr/share/vzctl/scripts/vpsreboot)
> Oct 10 06:25:01 vochomurka /USR/SBIN/CRON[18842]: (root) CMD (/usr/share/vzctl/scripts/vpsnetclean)
> Oct 10 07:19:13 vochomurka syslogd 1.4.1#18: restart.
--
E Frank Ball efball@efball.com