Re: Clarification on behaviour of Iperf's -w option
Hi Marc,
See comments inline.
Marc Herbert wrote:
On Thu, 12 Jul 2007, Lawrence Stewart wrote:
It appears the "-w" switch is in fact modifying the size of the send
buffer on the client, and is doing so successfully at that.
Well, and this is a Good Thing, since this is the only parameter that
matters on the sender. If it's too small, the sender may be blocked
waiting for acknowledgements. On the other hand, setting the *receive*
buffer size on the *sender* is absolutely useless for performance.
Agreed that setting the *receive* buffer size on the *sender* for a
unidirectional test is absolutely useless for performance.
If the documentation could drop the misleading "window size"
terminology in profit for the correct "socket buffer size" name which
it is actually setting (SO_SNDBUF / SO_RCVBUF), this would make such
recurrent confusions go away.
Agreed. This would save quite a bit of confusion. Although dropping the
"window size" term completely is perhaps not quite the right way to
handle it. As you note below, the receive buffer size is related to the
TCP window. Perhaps changing the terminology to refer to "maximum
advertised window" would make it clearer.
Receive socket buffer size and TCP window are definitely related, and
definitely different (Hint: buffers are not always empty. Else why
allocate them?)
Agreed. The receive socket buffer size sets an upper bound limit on the
advertised window, but is not necessarily equal in size to the actual
window in use by TCP at any given moment. FreeBSD for example defaults
to advertising the receive socket buffer size as the initial window.
Recent Linux kernels, which have buffer tuning enabled by default,
actually advertise an initial window that is less than the receive
buffer size (most of the time).
Send socket buffer size and TCP window are very very indirectly
related => bad terminology.
Agreed.
The -w option as it is currently performs The Only Perfect Thing That
Matters For Performance (for one-way tests). It is only confusingly
misnamed. It sets the send buffer size on the sender and the receive
buffer on the receiver: I like it, and I suspect that most people that
got used to it for ages like it too.
I feel that having a switch behave differently when used in 2 different
contexts is very confusing. You could change the documentation to
explain the current behaviour, but that would involve describing 2
different behaviours depending on running in client or server mode, and
also makes it more difficult to implement an additional switch to
provide the fine grained control over either send or receive buffer
sizes depending on whether you run in client or server mode.
I took the approach with the patch I created (see
http://caia.swin.edu.au/urp/newtcp/tools.html for a copy) of setting a
fixed behaviour for the "-w" switch and the new "-e" switch regardless
of the client/server context. I also changed the documentation in the
Iperf help output to refer to send and receive buffer sizes (granted the
patch probably needs to more carefully explain that the "-w" option sets
the size of the receive buffer, which relates to the TCP window by
setting an *upper bound* on the advertised TCP window, and doesn't
actually fix the TCP window for the duration of the test).
Maybe it is interesting to add new options for people wanting to
control more finely and more explicitely receive versus send buffers,
but... please do not change the existing and useful -w option?
As the patch is currently implemented, achieving the same behaviour as
specifying the "-w" switch on both client and server would change the
commands executed on the client and server to "iperf -c <ip> -e 100K"
and "iperf -s -w 100K" respectively, where -e explicitly sets the send
buffer size to 100K and -w explicitly sets the receive buffer size to
100K. To me as a user, this new usage model makes it much clearer what
we are explicitly overriding, as opposed to a single switch doing
different things when run in client or server mode and incorrectly
reporting information relating to the TCP window.
You do make a fair point regarding the entrenched use of the "-w" switch
in its current form. Perhaps the "-w" switch could be left intact and a
pair of new switches added to specifically set the send/receive socket
buffer sizes. Use of these switches and the "-w" switch could be made
mutually exclusive so they don't step on eachother's toes. To me,
leaving the "-w" switch intact still feels a bit hacky... but I can see
why it would be useful.
Any thoughts on this Marc (and indeed other Iperf users out there)?
Bidirectional tests are yet another story... *sigh*
These definitely require the ability to tune the individual buffers, and
the "-w" switch is of little use here. This is perhaps another argument
in favour of changing the behaviour of the "-w" switch so that it
doesn't confuse people running bidirectional tests.
Cheers,
Lawrence