A Iperf Performance Question

["Sahadevan, Samson joseph dev" <samson.joseph.dev.sahadevan --at-- intel.com> (by way of Jim Ferguson <ferguson --at-- ncsa.uiuc.edu>)]

Thanks in advance for your time. It would be very helpful if you could
give me hint about the performance characteristics I am facing running
Iperf.
 
I have connected two 3GZ Xeon 5100 series (Bensley platform) systems
using a 1GB Ethernet cable.
I use Iperf 2.0.2 (I think that?s the latest version).
I use Linux 2.6.15.4 vanilla version
 
For an IO buffer size of 64 bytes ?-l 64? the CPU utilization varies
drastically as I move from 1 ? 4 TCP streams.
 
For 1 stream there is almost no CPU utilization and I get a BW of about
90Mbytes/sec
For 2 there is about 25% CPU utilization per stream counting to about 50%
busy (with each stream giving about 60 Mbytes/sec)
 
Thus it almost goes to 0 running 4 TCP streams. 
 
My question is why is that 1 stream taking practically 0% CPU and giving
a better BW / stream while 2 streams take 25% and gives almost half the
BW. Why does the CPU utilization increase so drastically? Does it use the
whole BW available, which I think it doesn?t?
 
I can?t understand the way it works.
 
I know 64 bytes is very small. But we are doing some performance study
and I am caught up with this Iperf anomaly of CPU usage for smaller size
buffers. I would just like to know what exactly happens in these
scenarios
 
 
 
I appreciate your time and help,
 
 
 
Thanks in advance,
 
Samson