Strange iperf results
My name is Marc Garcia and I am doing my Diploma-Thesis in the Forschungszentrum Karlsruhe in the field of Networking in the Gridka project (www.gridka.de). We are using iperf as a basis to messure the bandwidth and quality of our 1G - 10G network but we are experimenting strange results with it.
In a back to back UDP test, iperf gives around 500Mbps with a high packet loss but under the same conditions, the behaviour of an TCP test is stable and close to 1Gbps. We tried the same test in other equipment (IBM) and both test succeeded with preety and stable results; no errors, both close to 1Gbps. We thought it was a hardware problem and after an exhaustive analisys of the device driver of the first group of equipments the problems still persisted; far from the good IBM results.
I would not be sending you an email just to explain this, but the point is that when -l parameter is enabled in both ends with a value bigger than the default one (1700 and upper), it works perfectly; no errors, very good throughput. It doesn't make sense because the MTU of the GEthernet is set to 1500 so the TCP/IP must fragment the packet before sending it. On the other hand, we've been testing the same scenario with pathload and pathrate and they say that the available bandwith is close to 1Gbps. Here must be something strange. Does your program count on the Interruption Calescence? I attach you some results just to clarify.
Thanks a lot in advance for your time,
Best Regards,
Marc Garcia Marti
IWR - Forschungszentrum Karlsruhe
[root --at-- 10gtk109 tmp]# iperf -s -u -i 1 -b M -l 3000
WARNING: option -b is not valid for server mode
------------------------------------------------------------
Server listening on UDP port 5001
Receiving 3000 byte datagrams
UDP buffer size: 64.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.108.46.109 port 5001 connected with 192.108.46.108 port 32769
[ 3] 0.0- 1.0 sec 112 MBytes 943 Mbits/sec 0.021 ms 7/39297 (0.018%)
[ 3] 1.0- 2.0 sec 112 MBytes 943 Mbits/sec 0.025 ms 0/39278 (0%)
[ 3] 2.0- 3.0 sec 112 MBytes 943 Mbits/sec 0.023 ms 0/39286 (0%)
[ 3] 3.0- 4.0 sec 112 MBytes 943 Mbits/sec 0.024 ms 5/39282 (0.013%)
[ 3] 4.0- 5.0 sec 112 MBytes 943 Mbits/sec 0.034 ms 0/39282 (0%)
[ 3] 5.0- 6.0 sec 112 MBytes 943 Mbits/sec 0.025 ms 0/39284 (0%)
[ 3] 6.0- 7.0 sec 112 MBytes 943 Mbits/sec 0.033 ms 0/39281 (0%)
[ 3] 7.0- 8.0 sec 112 MBytes 943 Mbits/sec 0.034 ms 0/39283 (0%)
[ 3] 8.0- 9.0 sec 112 MBytes 943 Mbits/sec 0.025 ms 0/39281 (0%)
[ 3] 9.0-10.0 sec 112 MBytes 943 Mbits/sec 0.027 ms 0/39283 (0%)
[ 3] 0.0-10.0 sec 1.10 GBytes 943 Mbits/sec 0.025 ms 11/392838 (0.0028%)
[ 3] 0.0-10.0 sec 1 datagrams received out-of-order
------------------------------------------------------------
Server listening on UDP port 5001
Receiving 1700 byte datagrams
UDP buffer size: 64.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.108.46.109 port 5001 connected with 192.108.46.108 port 32769
[ 3] 0.0- 1.0 sec 110 MBytes 923 Mbits/sec 0.010 ms 691/68538 (1%)
[ 3] 1.0- 2.0 sec 110 MBytes 923 Mbits/sec 0.013 ms 30/67880 (0.044%)
[ 3] 2.0- 3.0 sec 111 MBytes 932 Mbits/sec 0.012 ms 27/68532 (0.039%)
[ 3] 3.0- 4.0 sec 111 MBytes 932 Mbits/sec 0.015 ms 26/68524 (0.038%)
[ 3] 4.0- 5.0 sec 111 MBytes 931 Mbits/sec 0.014 ms 36/68528 (0.053%)
[ 3] 5.0- 6.0 sec 110 MBytes 922 Mbits/sec 0.016 ms 22/67851 (0.032%)
[ 3] 6.0- 7.0 sec 111 MBytes 932 Mbits/sec 0.015 ms 0/68531 (0%)
[ 3] 7.0- 8.0 sec 111 MBytes 932 Mbits/sec 0.013 ms 0/68529 (0%)
[ 3] 8.0- 9.0 sec 111 MBytes 932 Mbits/sec 0.011 ms 0/68531 (0%)
[ 3] 9.0-10.0 sec 111 MBytes 932 Mbits/sec 0.011 ms 2/68528 (0.0029%)
[ 3] 0.0-10.0 sec 1.08 GBytes 929 Mbits/sec 0.011 ms 833/683987 (0.12%)
[ 3] 0.0-10.0 sec 1 datagrams received out-of-order
Waiting for server threads to complete. Interrupt again to force quit.
[root --at-- 10gtk109 tmp]# iperf -s -u -i 1 -b M
WARNING: option -b is not valid for server mode
------------------------------------------------------------
Server listening on UDP port 5001
Receiving 1470 byte datagrams
UDP buffer size: 64.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.108.46.109 port 5001 connected with 192.108.46.108 port 32769
[ 3] 0.0- 1.0 sec 97.3 MBytes 816 Mbits/sec 0.014 ms 751/70147 (1.1%)
[ 3] 1.0- 2.0 sec 96.9 MBytes 813 Mbits/sec 0.021 ms 323/69457 (0.47%)
[ 3] 2.0- 3.0 sec 80.3 MBytes 673 Mbits/sec 0.011 ms 14/57260 (0.024%)
[ 3] 3.0- 4.0 sec 66.6 MBytes 559 Mbits/sec 0.014 ms 0/47495 (0%)
[ 3] 4.0- 5.0 sec 80.3 MBytes 673 Mbits/sec 0.011 ms 13/57261 (0.023%)
[ 3] 5.0- 6.0 sec 65.6 MBytes 551 Mbits/sec 0.014 ms 682/47495 (1.4%)
[ 3] 6.0- 7.0 sec 80.3 MBytes 673 Mbits/sec 0.017 ms 13/57268 (0.023%)
[ 3] 7.0- 8.0 sec 66.6 MBytes 559 Mbits/sec 0.015 ms 0/47495 (0%)
[ 3] 8.0- 9.0 sec 79.3 MBytes 665 Mbits/sec 0.017 ms 705/57261 (1.2%)
[ 3] 9.0-10.0 sec 65.6 MBytes 550 Mbits/sec 0.018 ms 0/46800 (0%)
[ 3] 0.0-10.0 sec 779 MBytes 653 Mbits/sec 0.043 ms 2500/557940 (0.45%)
[ 3] 0.0-10.0 sec 1 datagrams received out-of-order
[root --at-- 10gtk109 tmp]# iperf -s -u -i 1 -b M -l 1650
WARNING: option -b is not valid for server mode
------------------------------------------------------------
Server listening on UDP port 5001
Receiving 1650 byte datagrams
UDP buffer size: 64.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.108.46.109 port 5001 connected with 192.108.46.108 port 32769
[ 3] 0.0- 1.0 sec 82.4 MBytes 692 Mbits/sec 0.013 ms 15823/68216 (23%)
[ 3] 1.0- 2.0 sec 61.2 MBytes 514 Mbits/sec 0.011 ms 16970/55881 (30%)
[ 3] 2.0- 3.0 sec 77.3 MBytes 648 Mbits/sec 0.019 ms 859/49981 (1.7%)
[ 3] 3.0- 4.0 sec 56.8 MBytes 477 Mbits/sec 0.011 ms 18669/54772 (34%)
[ 3] 4.0- 5.0 sec 58.4 MBytes 490 Mbits/sec 0.012 ms 12851/49981 (26%)
[ 3] 5.0- 6.0 sec 79.0 MBytes 663 Mbits/sec 0.015 ms 4974/55197 (9%)
[ 3] 6.0- 7.0 sec 56.7 MBytes 476 Mbits/sec 0.011 ms 13549/49572 (27%)
[ 3] 7.0- 8.0 sec 55.7 MBytes 467 Mbits/sec 0.012 ms 19363/54772 (35%)
[ 3] 8.0- 9.0 sec 61.7 MBytes 518 Mbits/sec 0.011 ms 10750/49981 (22%)
[ 3] 0.0- 9.4 sec 609 MBytes 545 Mbits/sec 0.037 ms 113808/500935 (23%)
[ 4] local 192.108.46.109 port 5001 connected with 192.108.46.108 port 32769
[ 4] 0.0- 1.0 sec 39.5 MBytes 331 Mbits/sec 0.006 ms 28801/70185 (41%)
[ 4] 1.0- 2.0 sec 52.6 MBytes 441 Mbits/sec 0.010 ms 92/55213 (0.17%)
[ 4] 2.0- 3.0 sec 65.8 MBytes 552 Mbits/sec 0.009 ms 442/69489 (0.64%)
[ 4] 3.0- 4.0 sec 40.8 MBytes 342 Mbits/sec 0.013 ms 20735/63534 (33%)
[ 4] 4.0- 5.0 sec 39.3 MBytes 330 Mbits/sec 0.009 ms 3/41222 (0.0073%)
[ 4] 5.0- 6.0 sec 40.1 MBytes 337 Mbits/sec 0.005 ms 21466/63535 (34%)
[ 4] 6.0- 7.0 sec 58.2 MBytes 488 Mbits/sec 0.009 ms 2210/63204 (3.5%)
[ 4] 7.0- 8.0 sec 60.6 MBytes 508 Mbits/sec 0.005 ms 30/63534 (0.047%)
[ 4] 8.0- 9.0 sec 39.3 MBytes 330 Mbits/sec 0.005 ms 0/41229 (0%)
[ 4] 9.0-10.0 sec 60.6 MBytes 508 Mbits/sec 0.013 ms 14/63533 (0.022%)
[ 4] 0.0-10.0 sec 497 MBytes 417 Mbits/sec 0.033 ms 73792/594679 (12%)
[ 4] 0.0-10.0 sec 1 datagrams received out-of-order
The bigger the -l parameter is, the better results you have.