Iperf hangs
I have been having problems with 'too many orphaned sockets', and after watching the count for several days I never seen
A lot (mostly 0 and < 5).
However while watching the machine I also notice that at times there are several (usually 8) connections from iepmbw.bnl.org to port 5000 on iepm-bw.slac.stanford.edu.
I just ran a multistream iperf from the command line on the bnl node to the slac node and it appears to hang when done and not return to the command line. I waited over a minute. Here is the output:
iepm --at-- iepmbw ~/v3src]$ /home/iepm/v3src/bin/iperf -c 134.79.240.38 -p 5000 -i 5 -t 20 -w 8M -P 8
------------------------------------------------------------
Client connecting to 134.79.240.38, TCP port 5000
TCP window size: 16.0 MByte (WARNING: requested 8.00 MByte)
------------------------------------------------------------
[ 10] local 192.203.218.61 port 33094 connected with 134.79.240.38 port 5000
[ 4] local 192.203.218.61 port 33088 connected with 134.79.240.38 port 5000
[ 3] local 192.203.218.61 port 33087 connected with 134.79.240.38 port 5000
[ 5] local 192.203.218.61 port 33089 connected with 134.79.240.38 port 5000
[ 6] local 192.203.218.61 port 33090 connected with 134.79.240.38 port 5000
[ 7] local 192.203.218.61 port 33091 connected with 134.79.240.38 port 5000
[ 8] local 192.203.218.61 port 33092 connected with 134.79.240.38 port 5000
[ 9] local 192.203.218.61 port 33093 connected with 134.79.240.38 port 5000
[ 5] 0.0- 5.0 sec 34.0 MBytes 57.0 Mbits/sec
[ 7] 0.0- 5.0 sec 19.8 MBytes 33.2 Mbits/sec
[ 9] 0.0- 5.0 sec 32.3 MBytes 54.1 Mbits/sec
[ 8] 0.0- 5.0 sec 23.8 MBytes 39.9 Mbits/sec
[ 4] 0.0- 5.0 sec 38.4 MBytes 64.4 Mbits/sec
[ 8] 5.0-10.0 sec 16.4 MBytes 27.5 Mbits/sec
[ 7] 5.0-10.0 sec 76.8 MBytes 129 Mbits/sec
[ 5] 5.0-10.0 sec 47.9 MBytes 80.3 Mbits/sec
[ 4] 5.0-10.0 sec 53.9 MBytes 90.4 Mbits/sec
[ 9] 5.0-10.0 sec 20.2 MBytes 33.9 Mbits/sec
[ 7] 10.0-15.0 sec 68.2 MBytes 114 Mbits/sec
[ 4] 10.0-15.0 sec 43.6 MBytes 73.1 Mbits/sec
[ 5] 10.0-15.0 sec 42.2 MBytes 70.9 Mbits/sec
[ 9] 10.0-15.0 sec 20.6 MBytes 34.6 Mbits/sec
[ 8] 10.0-15.0 sec 36.9 MBytes 62.0 Mbits/sec
[ 4] 15.0-20.0 sec 46.1 MBytes 77.3 Mbits/sec
[ 4] 0.0-20.3 sec 182 MBytes 75.3 Mbits/sec
[ 8] 15.0-20.0 sec 37.5 MBytes 62.9 Mbits/sec
[ 8] 0.0-20.3 sec 115 MBytes 47.4 Mbits/sec
[ 5] 15.0-20.0 sec 38.3 MBytes 64.2 Mbits/sec
[ 5] 0.0-20.4 sec 162 MBytes 66.8 Mbits/sec
[SUM] 0.0-20.4 sec 607 MBytes 250 Mbits/sec
[ 9] 15.0-20.0 sec 28.3 MBytes 47.5 Mbits/sec
[ 9] 0.0-20.7 sec 101 MBytes 41.0 Mbits/sec
[ 7] 15.0-20.0 sec 13.6 MBytes 22.8 Mbits/sec
[ 7] 0.0-21.2 sec 178 MBytes 70.4 Mbits/sec
[iepm --at-- iepmbw ~/v3src]$ I did a <ctrl c> to get out of it.
Does anyone have any ideas what may be causing this?
I am running the iperf again, and it is hung before it finishes. I get the following output:
[iepm --at-- iepmbw bin]$ /home/iepm/v3src/bin/iperf -c 134.79.240.38 -p 5000 -i 5 -t 20 -w 8M -P 8
------------------------------------------------------------
Client connecting to 134.79.240.38, TCP port 5000
TCP window size: 16.0 MByte (WARNING: requested 8.00 MByte)
------------------------------------------------------------
[ 10] local 192.203.218.61 port 34526 connected with 134.79.240.38 port 5000
[ 3] local 192.203.218.61 port 34519 connected with 134.79.240.38 port 5000
[ 4] local 192.203.218.61 port 34520 connected with 134.79.240.38 port 5000
[ 5] local 192.203.218.61 port 34521 connected with 134.79.240.38 port 5000
[ 8] local 192.203.218.61 port 34524 connected with 134.79.240.38 port 5000
[ 7] local 192.203.218.61 port 34523 connected with 134.79.240.38 port 5000
[ 9] local 192.203.218.61 port 34525 connected with 134.79.240.38 port 5000
[ 6] local 192.203.218.61 port 34522 connected with 134.79.240.38 port 5000
[ 8] 0.0- 5.0 sec 24.3 MBytes 40.7 Mbits/sec
[ 7] 0.0- 5.0 sec 24.5 MBytes 41.1 Mbits/sec
[ 7] 5.0-10.0 sec 20.4 MBytes 34.2 Mbits/sec
[ 8] 5.0-10.0 sec 15.8 MBytes 26.5 Mbits/sec
[ 7] 10.0-15.0 sec 33.0 MBytes 55.3 Mbits/sec
[ 8] 10.0-15.0 sec 24.4 MBytes 40.9 Mbits/sec
[ 8] 15.0-20.0 sec 32.5 MBytes 54.5 Mbits/sec
[ 8] 0.0-20.3 sec 96.9 MBytes 40.1 Mbits/sec
[ 7] 15.0-20.0 sec 37.4 MBytes 62.8 Mbits/sec
[ 7] 0.0-20.4 sec 115 MBytes 47.5 Mbits/sec
And it just hangs there while I see on the SLAC end
48cal --at-- iepm-bw: netstat -a | grep bnl
tcp 0 0 iepm-bw.slac.stanford:33962 iepmbw.bnl.org:ssh ESTABLISHED
tcp 0 0 iepm-bw.slac.stanford.:5000 iepmbw.bnl.org:34519 ESTABLISHED
tcp 0 0 iepm-bw.slac.stanford.:5000 iepmbw.bnl.org:34526 ESTABLISHED
tcp 0 0 iepm-bw.slac.stanford.:5000 iepmbw.bnl.org:34525 ESTABLISHED
tcp 0 0 iepm-bw.slac.stanford.:5000 iepmbw.bnl.org:34522 ESTABLISHED
tcp 0 0 iepm-bw.slac.stanford.:5000 iepmbw.bnl.org:34520 ESTABLISHED
tcp 0 0 iepm-bw.slac.stanford.:5000 iepmbw.bnl.org:34521 ESTABLISHED
tcp 0 0 iepm-bw.slac.stanford.:5000 iepmbw.bnl.org:33087 ESTABLISHED
tcp 0 0 iepm-bw.slac.stanford.:5000 iepmbw.bnl.org:33094 ESTABLISHED
tcp 0 0 iepm-bw.slac.stanford.:5000 iepmbw.bnl.org:33090 ESTABLISHED
I eventually killed the iperf command runnning from BNL, and the sockets are still allocated
Several minutes later on the slac node.
Any ideas?
Thanks Connie
Connie Logg, Network Analyst
Stanford Linear Accelerator Center
ph: 650-926-2879
"Happiness is found along the way, not at the end of the road, and 'IF' is the middle word in life."