RE: How iperf puts delay between the packets??
Hello Kevin,
I have tried with other network topologies to test the iperf. The latest
toplogy that I have tested is
1000Mbps 100Mbps
A-------Gigabit-------------Machine-------------Gigabit---------C
B-------Switch Switch----------D
With this topology I have kept a machine (Intel Xeon, Linux 8) between the 2
switches. One of the interface cards is fixed at 1000Mbps and the other card
is fixed at 100Mbps. The intermediate machine acts as the router which
switches traffic that comes on one card to the other card. Thus this
intermediate machine acts as the bottleneck on which traffic can come at
1000Mbps but it looses packets from the traffic to reduce it to 100 Mbps. With
this configuration I am able to create a perfect bottleneck of 100 Mbps and
Iperf UDP stream does not reduce its sending rate on the introduction of the
TCP cross traffic. As up till 1000Mbps data could reach on one interface card
of the intermediate machine and only 100Mbs data could pass on.
With the toplogy I stated in my last e-mail (without intermediate machine),
what I coud think of is as I introduce the cross traffic with the UDP stream
on the bottleneck, the bottleneck gets filled and the UDP sender does not put
the data on the bottleneck and stsrts feeling its send buffer and blocks until
some space is made for it on the bottleneck. I confirmed this by converting
the UDP socket as non-blocking (earlier it was blocking socket), and as I
introduce cross traffc, the send system call returns an error as
"send data: Resource Temporarily Unavailable"
which makes it sure that UDP send buffer gets full and the machine does not
puts the UDP send buffer data on the bottleneck to empty the send buffer and
hence it gives an error as Resource temporarily unavailable.
I dont know if this is the bug in network stack as they do not put the data
on the bottleneck until some space is made for it. But I want some perfect
toplogy to create an bottleneck of 100 Mbps. I tried to create the bottleneck
of 100Mbps with Nistnet Emulator, but Nistnet is not creating the perfect
bottleneck. That is the reason I am using a router instead of any perfect
bottleneck.
I would really appreciate if somebody could suggest me some topology to
create a perfect 100Mbps bottleneck between 4 machines (2 switches).
Thanks in advance for your help..
Regards,
Aditya Maroo
>===== Original Message From Kevin Gibbs <kgibbs --at-- ncsa.uiuc.edu> =====
>This seems to be the ideal setup for the test you are performing. The
>reason being the busy waiting that you noticed in the UDP case. Obviously
>if you ran both the TCP and UDP client on the same machine the timing
>would be off as the busy waiting would be unpredictable. However that is
>not a reason for the backing off of the sending rate. The high utilization
>is normal and pretty much a neccessary evil since any of the sleep
>functions do not have a tight enough granularity to achieve the packet/s
>rates most people use Iperf to generate. (As a side note the current
>implementation does not have a very good granularity on Win32 machines,
>but since you are on Linux that is not a problem.) I am not sure what the
>issue with this test is. I am wondering if there may be some generation of
>ICMP traffic or something that is telling the linux stack to slow down.
>Many people have been posting lately that Iperf is slowing down to reduce
>loses overtime. This is not a feature of Iperf and I am wondering if newer
>kernels are doing some trickery here (there has been many changes lately
>to the networking stack along these lines. I am more familar with changes
>to TCP with dynamic buffers and window sizes much like the Web100 project.
>Perhaps they have added some stuff to UDP as well which has a negative
>impact on your tests but would greatly help real applications from losing
>lots of packets). I am sorry I could not be more help.
>
>Kevin
>
>On Sun, 25 Jul 2004, maroo wrote:
>
>> Hello Kevin,
>> Thanks a lot for replying my mail. For the questions you have asked,
>> A) UDP Stream is generated between 2 different machines (Red Hat8 Linux,
>> Pentium III machines) and a parallel TCPstream is generated between 2
>> different machines (same configuration). So in total there are 4 machines.
>>
>> A-----Gigabit ----100Mbps Bottleneck----Gigabit---------B
>> C-----Switch1 Switch2---------D
>>
>> >From A to B, I am running an Iperf UDP Stream and from C to D I am running
a
>> parallel TCP Stream. In the above configuratiion A is UDP client and C is
TCP
>> client. I am running the test for 300 seconds. First I start UDP stream
>> between A and B and then after 30 seconds I start TCP stream from C to D.
As
>> the TCP stream is started the sending rate at the UDP client starts
decreasing
>> (if it was giving 94 Mbps (max), it decreases to 65 Mbps and the parallel
TCP
>> stream gets around 30Mbps).
>> Aniother problem I noticed when I had gone through the source code of
Iperf,
>> it has implemented busy wait (delay_loop instead of any timers) to
introduce
>> the delay between the consecutive UDP packets and it does not releases the
CPU
>> until the program is over. It takes 99.8 % CPU utilization until the
program
>> is over. I am just wondering could this be related to the problem I am
facing
>> on the introduction of cross traffic.
>>
>> I would really appreciate if somebody could answer this problem.
>>
>> With best regards and thanks for help.
>>
>> Aditya Maroo
>>
>> I have gone through the source code of iperf and I saw busy_wait
implemented
>> to put delay between consecutive packets due to which CPU is not released
any
>> time by the iperf program. So, as I originated 2 or more than 2 iperf UDP
>> streams from the same machine I saw a signnificant reduction in the sending
>> rate of both the iperf streams and both didnt achieved the desired sending
>> rate.
>>
>> >= ==== Original Message From Kevin Gibbs <kgibbs --at-- ncsa.uiuc.edu> =====
>> >More details are needed for properly diagnosing what you are seeing here.
>> >It would be helpful to know these things:
>> >
>> >A) Are the tests terminating or orginating from a single machine or
>> >multiple?
>> >
>> >B) What does the client say the UDP send rate is? Does that differ from
>> >what the server reported?
>> >
>> >C) Do you completely understand the method your router is using for
>> >queuing? So like is it possible that the router is queuing these streams
>> >separately and thus always giving the TCP stream 35M?
>> >
>> >However to answer your questions:
>> >Iperf attempts to generate a CBR (Constant Bit Rate) UDP stream. To
>> >accomplish this it sits in a tight loop between sends. It makes the
>> >assumption that once the send() call returns the packet has been sent. (Or
>> >atleast a extremely close approximation) It then calculates the time
>> >needed to wait and loops till it is over. Repeat for specified timeout or
>> >amount to send. The amount actually sent is reported on the client side.
>> >If this is less than was specified then there are some issues with your
>> >setup that prevent higher speeds. (i.e. Windows has a horrid time
>> >resolution with the function we currently use (we want to fix it in the
>> >future), Iperf is not holding the CPU but got swapped out for another
>> >process thus making our timing way off (this would be _one_ reason why
>> >having both streams generated from the system would not work well)....)
>> >
>> >Iperf does not control TCP throughput. It simply sends the defined chunk
>> >size (default 8K) and repeats for specified timeout or amount to send.
>> >
>> >
>> >Kevin
>> >
>> >On Fri, 16 Jul 2004, maroo wrote:
>> >
>> >> Hello,
>> >> I am doing an experiment to test the TCP-friendliness of my protocol.
To
>> >> test it I am passing a stream using my protocol and an iperf TCP stream
in
>> >> parallel through a botlleneck link that supports 100Mbps.
>> >>
>> >> To compare TCP-friendliness of my protocol with that of UDP (which is
>> >> non-TCP-friendly), I am running an iperf UDP stream in parallel with an
>> iperf
>> >> TCP stream from the same botllneck link. But to my surprise, whatever I
set
>> >> the sending rate of UDP stream (anything above 70Mbps), it always gives
35
>> >> Mbps to the parallel TCP stream. That is UDP stream lowers down its rate
to
>> >> accomodate the parallel TCP stream on the bottleneck link.
>> >>
>> >> So, this is the reason, I want to know how does iperf controls the
>> sending
>> >> rate of UDP stream (or even for a TCP stream). How does iperf puts the
>> delay
>> >> between the packets to control the rate.
>> >>
>> >> Could anybody please help me.
>> >>
>> >> With best regards and thanks for the help.
>> >>
>> >> Aditya Maroo
>> >>
>> >>
>>