OpenVZ Forum


Home » General » Support » [kernel: Out of socket memory] Is there a solution?
icon5.gif  [kernel: Out of socket memory] Is there a solution? [message #24675] Fri, 07 December 2007 16:34 Go to next message
arghbis is currently offline  arghbis
Messages: 35
Registered: November 2007
Member
hi folks,

I'm running openVZ on a server (6Go RAM, dual gigabit net devices, bixeon quadcore). The server supports a high load and i've a lot of "kernel: Out of socket memory" in syslog (at least 10/minute).

I've googled aroud to find solutions to this issue and, despite the modifications i made, the errors are still there.

here is my tcp config from sysctl
net.ipv4.tcp_slow_start_after_idle = 1
net.ipv4.tcp_workaround_signed_windows = 0
net.ipv4.tcp_base_mss = 512
net.ipv4.tcp_mtu_probing = 0
net.ipv4.tcp_abc = 0
net.ipv4.tcp_congestion_control = bic
net.ipv4.tcp_tso_win_divisor = 3
net.ipv4.tcp_moderate_rcvbuf = 1
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_max_tw_buckets_ub = 66144
net.ipv4.tcp_max_tw_kmem_fraction = 384
net.ipv4.tcp_low_latency = 0
net.ipv4.tcp_frto = 0
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_adv_win_scale = 2
net.ipv4.tcp_app_win = 31
net.ipv4.tcp_rmem = 4096        87380   16777216
net.ipv4.tcp_wmem = 4096        65536   16777216
net.ipv4.tcp_mem = 16777216     16777216        16777216
net.ipv4.tcp_dsack = 0
net.ipv4.tcp_ecn = 0
net.ipv4.tcp_reordering = 5
net.ipv4.tcp_fack = 1
net.ipv4.tcp_orphan_retries = 0
net.ipv4.tcp_max_syn_backlog = 1024
net.ipv4.tcp_rfc1337 = 0
net.ipv4.tcp_stdurg = 0
net.ipv4.tcp_abort_on_overflow = 0
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_fin_timeout = 20
net.ipv4.tcp_retries2 = 15
net.ipv4.tcp_retries1 = 3
net.ipv4.tcp_keepalive_intvl = 75
net.ipv4.tcp_keepalive_probes = 9
net.ipv4.tcp_keepalive_time = 1000
net.ipv4.tcp_max_tw_buckets = 360000
net.ipv4.tcp_max_orphans = 131072
net.ipv4.tcp_synack_retries = 5
net.ipv4.tcp_syn_retries = 5
net.ipv4.tcp_retrans_collapse = 1
net.ipv4.tcp_sack = 0
net.ipv4.tcp_window_scaling = 0
net.ipv4.tcp_timestamps = 0


uname -a :
Linux ns6812.ovh.net 2.6.18-ovz-028stab048.1-enterprise #1 SMP Thu Nov 1 19:34:25 CET 2007 i686 GNU/Linu


If anybody has a clue to solve this problem, thanks to him! Wink

Re: [kernel: Out of socket memory] Is there a solution? [message #24694 is a reply to message #24675] Fri, 07 December 2007 21:14 Go to previous messageGo to next message
ugob is currently offline  ugob
Messages: 271
Registered: March 2007
Senior Member
I'm having a similar problem...

Please read the manual before asking questions:
http://download.openvz.org/doc/OpenVZ-Users-Guide.pdf

Please have a look at the wiki before asking questions:
http://wiki.openvz.org/Main_Page
Re: [kernel: Out of socket memory] Is there a solution? [message #24821 is a reply to message #24675] Tue, 11 December 2007 07:13 Go to previous messageGo to next message
arghbis is currently offline  arghbis
Messages: 35
Registered: November 2007
Member
up!

no one has an idea about this problem?
Re: [kernel: Out of socket memory] Is there a solution? [message #24847 is a reply to message #24821] Tue, 11 December 2007 10:55 Go to previous messageGo to next message
gblond is currently offline  gblond
Messages: 64
Registered: May 2007
Member
What about failcnt for numtcpsock in /proc/user_beancounters ?

Also check "sysctl net.ipv4.tcp_mem"
Re: [kernel: Out of socket memory] Is there a solution? [message #24849 is a reply to message #24847] Tue, 11 December 2007 11:04 Go to previous messageGo to next message
arghbis is currently offline  arghbis
Messages: 35
Registered: November 2007
Member
no failcount (i have a script that check beancounters and upgrade them dynamically).

net.ipv4.tcp_mem = 2388608 2388608 16777216


do you see something wrong?
Re: [kernel: Out of socket memory] Is there a solution? [message #24932 is a reply to message #24849] Wed, 12 December 2007 10:19 Go to previous messageGo to next message
gblond is currently offline  gblond
Messages: 64
Registered: May 2007
Member
No, I think it is enough.

Try to increase twice

sysctl net.ipv4.tcp_max_orphans
Re: [kernel: Out of socket memory] Is there a solution? [message #25076 is a reply to message #24932] Fri, 14 December 2007 07:14 Go to previous messageGo to next message
arghbis is currently offline  arghbis
Messages: 35
Registered: November 2007
Member
i have increased max_orphans and wait a day to see the changes. As a results, i might have fewer errors, but still get plenty of them.

I also notice an other error message :kernel: TCP: Treason uncloaked! Peer xxx.xxx.xxx.xxx:29434/80 shrinks window 3922039675:3922045515. Repaired

i've ggogled around to check this message, but the answers are a bit confusing for me (some people say that it's not a problem, some that it is a huge one)

thanks for your help
Re: [kernel: Out of socket memory] Is there a solution? [message #25080 is a reply to message #25076] Fri, 14 December 2007 08:46 Go to previous messageGo to next message
gblond is currently offline  gblond
Messages: 64
Registered: May 2007
Member
Problem probably is in next:

Number of orphaned socket is limited to numtcpsock/4. If number is more then warning message printed (but there isn't failcnt!)

So you can try to increase numtcpsock.
Re: [kernel: Out of socket memory] Is there a solution? [message #25081 is a reply to message #25080] Fri, 14 December 2007 08:56 Go to previous messageGo to next message
arghbis is currently offline  arghbis
Messages: 35
Registered: November 2007
Member
but the problem occurs in the HN syslog? i can't increase its numtcpsock, can i?

Your explanation is very interesting since it semms logical, but for a VE.

Thanks for all
Re: [kernel: Out of socket memory] Is there a solution? [message #25083 is a reply to message #25081] Fri, 14 December 2007 09:31 Go to previous messageGo to next message
gblond is currently offline  gblond
Messages: 64
Registered: May 2007
Member
I told about message "kernel: Out of socket memory"

you can increase numtcpsock by:

vzctl set $VENUM --numtcpsock $WANTED_NUM_TCPSOCK

Re: [kernel: Out of socket memory] Is there a solution? [message #25085 is a reply to message #25083] Fri, 14 December 2007 09:39 Go to previous messageGo to next message
arghbis is currently offline  arghbis
Messages: 35
Registered: November 2007
Member
sure, i know how to increase numtcpsock.

My point was : How numtcpsock (and so, orphans) on vps can induce "kernel: Out of socket memory" messages on the hardware node, despite this message does not appear on any VPS?

thanks
Re: [kernel: Out of socket memory] Is there a solution? [message #25089 is a reply to message #25085] Fri, 14 December 2007 10:02 Go to previous messageGo to next message
vaverin is currently offline  vaverin
Messages: 708
Registered: September 2005
Senior Member
Ok,
we will change the message and add VE number and number of orphans.

http://bugzilla.openvz.org/show_bug.cgi?id=760

thank You,
Vasily Averin
Re: [kernel: Out of socket memory] Is there a solution? [message #25090 is a reply to message #25089] Fri, 14 December 2007 10:07 Go to previous messageGo to next message
arghbis is currently offline  arghbis
Messages: 35
Registered: November 2007
Member
ok! thanks

so if i understand correctly, the HN is monitoring the orphans for each VE and reports the errors in its own syslog. Is there a way to know which VE is reporting these errors?

Maybe i will wait for the feature you mention Wink

thanks
Re: [kernel: Out of socket memory] Is there a solution? [message #25091 is a reply to message #25090] Fri, 14 December 2007 10:17 Go to previous messageGo to next message
vaverin is currently offline  vaverin
Messages: 708
Registered: September 2005
Senior Member
net/ipv4/tcp_timer.c::tcp_out_of_resources()
....
        if (ub_too_many_orphans(sk, orphans)) {
                if (net_ratelimit())
                        printk(KERN_INFO "Out of socket memory\n");
We can extract VEID from struct sock *sk.

Thank you,
Vasily Averin
Re: [kernel: Out of socket memory] Is there a solution? [message #29972 is a reply to message #24675] Sat, 03 May 2008 10:12 Go to previous messageGo to next message
ittec is currently offline  ittec
Messages: 41
Registered: February 2008
Member
Just a little question, when you speak about sysctl setup is about sysctl.conf on Hardware Node? When you edit sysctl.conf on HN are you editing general values to all server?

Im a confuse. I don't know if I must change the values from HN or each VE.

Thanks
Re: [kernel: Out of socket memory] Is there a solution? [message #29977 is a reply to message #29972] Sun, 04 May 2008 07:28 Go to previous messageGo to next message
gblond is currently offline  gblond
Messages: 64
Registered: May 2007
Member
You have to set sysctl net.ipv4.tcp_max_orphans in VE0. This sysctl will effect to all VEs.

Re: [kernel: Out of socket memory] Is there a solution? [message #29999 is a reply to message #24675] Mon, 05 May 2008 07:57 Go to previous message
ittec is currently offline  ittec
Messages: 41
Registered: February 2008
Member
thanks gblond

I did some changes in systl.conf setup to avoid orphan problems and it seems that works.

Bye.
Previous Topic: crash + dmesg shows: unregister_netdevice: waiting for lo to become free
Next Topic: Error running vzdump
Goto Forum:
  


Current Time: Fri Mar 29 10:50:33 GMT 2024

Total time taken to generate the page: 0.02670 seconds