Re: question about the set tcp window size in iperf 1.6.3
On Fri, 29 Nov 2002 Marc.Herbert --at-- ens-lyon.fr wrote:
> On 29 Nov, Chong Zhang wrote:
>
> > I have another question and hope you could help me a bit:
> > I have two linux boxes connected via internet, and both
> > have default 16k buffer size on client mode and 85.3k on server mode.
>
> ... and you use these defaults ? (i.e., no iperf -w option?)
>
>
> > But I got different Bandwidth result on the two direction:
> >
> > A--> B ( 400K)
> > B--> A(1.4M)
>
> Moctets/s (or Mbits/s ?)
Using the same paramaters the K and M will be either both in bytes or
both in bits, so which it is does not really affect much (also if there
was a obvious factor of 8 like 200K and 1.6M then I would give more
thought along this line, but there isn't). You say that
you are using the "internet" to run the tests. What kind of "internet"
connection do you have at both nodes? Often DSL and other internet
connections do not have symmetric bandwidth ratings, eg. 1.5M down 512K
up. Are both of your connections identical and is the loading on both
about the same. Typical loading patterns on shared links is large amount
incoming and small outgoing. This would make outgoing connections run much
faster than incoming. Iperf uses straight TCP so it will not use more
bandwidth than other TCP flows are recieving on the network.
> > They are not identical boxes but I don't see too much difference (>1G,
> >>=256M, low cpu load)
>
> What is the linux kernel version ?
Looking from the choice of TCP ports in your tcpdump (32787 vs 1052) I
would guess that you are using different kernels or even different
distribuitions.
> > Why they have different win advertisements?
Well to begin with they _are_ the SAME. The only advertised window that
matters is the one advertised by the server. The client's advertised
window is meaningless since no data is being accepted by the client (as
evidenced with the continual acking of sequence number 1). Both
servers in your example are advertising about 64K.
Also a semi-confusing thing is that the kernel decides what the window
will be, not the application. The SO_SNDBUF/SO_RCVBUF socket settings
change the send/recieve buffer sizes not advertised window sizes.
Typically there is a one to one correspondance between the two, but not
always. Also newer linux kernels do autotuning if and only if window sizes
(socket buffers) are not explicitly given via the SO_SNDBUF/SO_RCVBUF. Any
or all of these things could be causing very different readings in one
direction verses the other.
> Assuming the latency between your boxes is in the 10ms range (true?),
> yes it seems the receiver window is your bottleneck indeed, as opposed
> to some other asymetry in the network...
64K window will support up to around 6.4Mbit/s at 10ms so that is not the
problem, I would look into asymetries in the network. Only if the latency
is above 150-200ms would I look again at the recieve windows.
Kevin
--
Mitch Kutzko | mitch --at-- dast.nlanr.net | mitch --at-- ncsa.uiuc.edu | 217-333-1199
http://hobbes.ncsa.uiuc.edu/