Fsck needed after out of memory [message #53580] |
Mon, 30 September 2019 15:13 ![Go to next message Go to previous message](/theme/ovz3/images/down.png) |
andre
Messages: 36 Registered: January 2008
|
Member |
|
|
Hello,
We are seeing lot of cases where fsck is needed after out of memory situations (different hardwares/ves/etc)
============================================================ ============================================
Sep 30 10:18:34 kernel: EXT4-fs error (device ploop43739p1) in ext4_free_blocks:4879: Out of memory
Sep 30 10:18:34 kernel: Aborting journal on device ploop43739p1-8.
Sep 30 10:18:34 kernel: EXT4-fs (ploop43739p1): Remounting filesystem read-only
Sep 30 10:18:34 kernel: EXT4-fs error (device ploop43739p1) in ext4_reserve_inode_write:5354: Journal has aborted
Sep 30 10:18:34 kernel: EXT4-fs error (device ploop43739p1) in ext4_reserve_inode_write:5354: Journal has aborted
Sep 30 10:18:34 kernel: EXT4-fs error (device ploop43739p1) in ext4_ext_remove_space:3060: Journal has aborted
Sep 30 10:18:35 kernel: EXT4-fs error (device ploop43739p1) in ext4_ext_truncate:4698: Journal has aborted
Sep 30 10:18:35 kernel: EXT4-fs error (device ploop43739p1) in ext4_reserve_inode_write:5354: Journal has aborted
Sep 30 10:18:35 kernel: EXT4-fs error (device ploop43739p1) in ext4_truncate:4139: Journal has aborted
Sep 30 10:18:35 kernel: EXT4-fs error (device ploop43739p1) in ext4_reserve_inode_write:5354: Journal has aborted
Sep 30 10:18:35 kernel: httpd invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
Sep 30 10:18:35 kernel: httpd cpuset=3657 mems_allowed=0
Sep 30 10:18:35 kernel: CPU: 1 PID: 828588 Comm: httpd ve: 3657 Kdump: loaded Not tainted 3.10.0-957.12.2.vz7.96.21 #1 96.21
Sep 30 10:18:35 kernel: Hardware name: Supermicro X9SCL/X9SCM/X9SCL/X9SCM, BIOS 2.0b 09/17/2012
Sep 30 10:18:35 kernel: Call Trace:
Sep 30 10:18:35 kernel: [<ffffffff8b594677>] dump_stack+0x19/0x1b
Sep 30 10:18:35 kernel: [<ffffffff8b58ef52>] dump_header+0x90/0x229
Sep 30 10:18:35 kernel: [<ffffffff8afcdcf8>] oom_kill_process+0x5e8/0x640
Sep 30 10:18:35 kernel: [<ffffffff8aff82ce>] ? get_task_oom_score_adj+0xee/0x100
Sep 30 10:18:35 kernel: [<ffffffff8b0514e9>] mem_cgroup_oom_synchronize+0x4a9/0x500
Sep 30 10:18:35 kernel: [<ffffffff8afce2a3>] pagefault_out_of_memory+0x13/0x50
Sep 30 10:18:35 kernel: [<ffffffff8b58d027>] mm_fault_error+0x6a/0x157
Sep 30 10:18:35 kernel: [<ffffffff8b5a28a8>] __do_page_fault+0x3c8/0x4f0
Sep 30 10:18:35 kernel: [<ffffffff8b5a2a05>] do_page_fault+0x35/0x90
Sep 30 10:18:35 kernel: [<ffffffff8b59e768>] page_fault+0x28/0x30
Sep 30 10:18:35 kernel: Task in /machine.slice/3657 killed as a result of limit of /machine.slice/3657
Sep 30 10:18:35 kernel: memory: usage 2097056kB, limit 2097152kB, failcnt 45598816
Sep 30 10:18:35 kernel: memory+swap: usage 2621444kB, limit 2621440kB, failcnt 13806662
Sep 30 10:18:35 kernel: kmem: usage 407384kB, limit 9007199254740988kB, failcnt 0
Sep 30 10:18:35 kernel: Memory cgroup stats for /machine.slice/3657: rss_huge:0KB mapped_file:338136KB shmem:406204KB slab_unreclaimable:13824KB swap:524388KB cache:406476KB rss:1283196KB slab_reclaimable:355864KB inactive_anon:432588KB active_anon:1256812KB inactive_file:40KB active_file:12KB unevictable:0KB
Sep 30 10:18:35 kernel: Memory cgroup out of memory: Kill process 828656 (clamd) score 283 or sacrifice child
Sep 30 10:18:35 kernel: Killed process 794108 (clamd) in VE "3657" total-vm:1032160kB, anon-rss:628748kB, file-rss:0kB, shmem-rss:0kB
Sep 30 10:18:35 kernel: rsyslogd invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
Sep 30 10:18:35 kernel: rsyslogd cpuset=3657 mems_allowed=0
Sep 30 10:18:35 kernel: CPU: 1 PID: 72938 Comm: rsyslogd ve: 3657 Kdump: loaded Not tainted 3.10.0-957.12.2.vz7.96.21 #1 96.21
Sep 30 10:18:35 kernel: Hardware name: Supermicro X9SCL/X9SCM/X9SCL/X9SCM, BIOS 2.0b 09/17/2012
Sep 30 10:18:35 kernel: Call Trace:
Sep 30 10:18:35 kernel: [<ffffffff8b594677>] dump_stack+0x19/0x1b
Sep 30 10:18:35 kernel: [<ffffffff8b58ef52>] dump_header+0x90/0x229
Sep 30 10:18:35 kernel: [<ffffffff8afcdcf8>] oom_kill_process+0x5e8/0x640
Sep 30 10:18:35 kernel: [<ffffffff8aff82ce>] ? get_task_oom_score_adj+0xee/0x100
Sep 30 10:18:35 kernel: [<ffffffff8b0514e9>] mem_cgroup_oom_synchronize+0x4a9/0x500
Sep 30 10:18:35 kernel: [<ffffffff8afce2a3>] pagefault_out_of_memory+0x13/0x50
Sep 30 10:18:35 kernel: [<ffffffff8b58d027>] mm_fault_error+0x6a/0x157
Sep 30 10:18:35 kernel: [<ffffffff8b5a28a8>] __do_page_fault+0x3c8/0x4f0
Sep 30 10:18:35 kernel: [<ffffffff8b5a2a05>] do_page_fault+0x35/0x90
Sep 30 10:18:35 kernel: [<ffffffff8b59e768>] page_fault+0x28/0x30
Sep 30 10:18:35 kernel: Task in /machine.slice/3657 killed as a result of limit of /machine.slice/3657
Sep 30 10:18:35 kernel: memory: usage 2097056kB, limit 2097152kB, failcnt 45598831
Sep 30 10:18:35 kernel: memory+swap: usage 2621444kB, limit 2621440kB, failcnt 13806683
Sep 30 10:18:35 kernel: kmem: usage 407384kB, limit 9007199254740988kB, failcnt 0
Sep 30 10:18:35 kernel: Memory cgroup stats for /machine.slice/3657: rss_huge:0KB mapped_file:338136KB shmem:406204KB slab_unreclaimable:13824KB swap:524388KB cache:406476KB rss:1283196KB slab_reclaimable:355864KB inactive_anon:432588KB active_anon:1256812KB inactive_file:40KB active_file:12KB unevictable:0KB
Sep 30 10:18:35 kernel: Memory cgroup out of memory: Kill process 828656 (clamd) score 283 or sacrifice child
Sep 30 10:18:35 kernel: Killed process 828656 (clamd) in VE "3657" total-vm:1032160kB, anon-rss:628756kB, file-rss:120kB, shmem-rss:0kB
Sep 30 10:18:35 kernel: EXT4-fs error (device ploop43739p1) in ext4_orphan_del:2723: Journal has aborted
Sep 30 10:18:35 kernel: EXT4-fs error (device ploop43739p1) in ext4_reserve_inode_write:5354: Journal has aborted
============================================================ ============================================
After that, VE will only mount with an fsck.
Looks like a bug.
|
|
|