does it make sense that single thread iperf3 gives me much worse results than a web test…
- User
does it make sense that single thread
iperf3gives me much worse results than a web test speed, but say 8 parallel threads come close to it? - Assistant
Yep — that behaviour is totally normal and there’s nothing “wrong” with your link.
Here’s why a single-stream iperf3 test often looks slow while 8 parallel streams (and most web-based speed tests) get much closer to the line rate.
- What a web speed test actually does
Speedtest.net, fast.com, Google’s speed test, etc. don’t pull the file down over one single TCP connection.
They usually spin up 4–8 (sometimes 16) parallel HTTP(S) downloads and then add the bytes together. That hides a lot of TCP-level limitations that show up when you test with only one flow.
- The TCP math behind it
Throughput of one TCP flow is limited by
throughput ≈ window_size / RTTWindow size = min(congestion_window, receiver_window)
Examples: RTT 20 ms 100 Mb/s ⇒ need a 250 kB window
RTT 20 ms 1000 Mb/s ⇒ need a 2.5 MB windowWhat trips people up:
• iperf3’s default socket buffer is only 85 kB. Unless the OS aggressively auto-grows it, the single stream tops out at ~34 Mb/s on a 20 ms path (85 kB / 0.02 s = 34 Mb/s).
• Slow-start & congestion-control (CUBIC, BBR, …) take a little time to ramp up anyway.
• Some routers and policers do “per-flow” fairness; one flow may get carved down while eight flows share the bucket.Open 8 flows ⇒ you’ve got 8 separate congestion windows; together they can fill the pipe even if each one is smaller than ideal.
- Proving it to yourself
Try a single stream but crank the TCP window:
# sender / client side iperf3 -c <server> -t 30 -w 4Mor both sides (if you control the server):
iperf3 -s -w 4M # server iperf3 -c <server> -w 4M # clientYou’ll notice the single-stream speed climbs dramatically, often matching the multi-stream result.
- Other factors that can widen the gap
• High latency (satellite, 5G mid-band, trans-continental links)→ bigger BDP
• Buffer-bloat / AQM along the path
• CPU off-load disabled or running inside a VM (iperf3 is single-threaded per stream)
• ISP speed tests hosted inside their own network → lower RTT & zero congestion
- TL;DR / What to remember
• Yes, the discrepancy is expected.
• Web speed tests purposely use multiple parallel TCP connections.
• A single TCP flow can saturate a fast line only if its send/receive buffers are big enough for the RTT (or if you use a congestion control like BBR that keeps cwnd large).
• You can replicate “speed-test-style” numbers with iperf3 using-P 8, or tweak the window size (-w).So your link is fine; your iperf3 just needed either more streams or a larger window to show its true colors. 👍
- User
to get closer to raw speed, should I test in UDP?
- Assistant
“Should I switch to UDP if I want to know the absolute capacity of the pipe?”
Short answer
• UDP is the quickest way to remove the TCP-window / congestion-control ceiling, so yes, it can show you the raw “can I push X Gbit/s through this link?” number.
• But you have to read the loss and jitter columns, not just the “sender Mbps” figure, and you need to dial the bitrate in carefully. Otherwise UDP can lie to you just as badly (sometimes worse) than TCP.────────────────────────────────────────────
- What UDP testing actually measures
────────────────────────────────────────────
• iperf3 in UDP mode (
-u) just fires datagrams at the rate you request (-b <rate>).
• There is no built-in flow control or recovery.
– If the network can’t keep up, packets are dropped.
– The sender has no idea; only the receiver reports loss.
So a UDP test tells you two things simultaneously:
throughput_attempted – what you asked iperf to send
throughput_received – what actually arrived intact (= attempted – loss)If
loss ≈ 0%, the attempted rate is within the real capacity of the path.
Ifloss > 0%, you’re overdriving it.──────────────────────────────────────────── 2. A quick “find the ceiling” recipe ──────────────────────────────────────────── (Assume you think the link is ≈1 Gbit/s)
Step 1 – warm-up
iperf3 -c -u -b 500M -t 10Step 2 – binary search upward
- Double the rate each run until you see >0.1 % loss.
e.g. 500 → 750 → 900 → 980 → 1000 → … - Drop back to the highest rate that still shows “lost = 0 (0.00%)”.
Step 3 – sanity check
Run 30–60 s at that chosen rate and verify loss and jitter stay low.Example output (server side):
[ 5] 0.00-30.00 sec 3.42 GBytes 979 Mbits/sec 0.016 ms 0/37091 (0%)
That is basically the raw payload rate. Add L2 overhead (~5–6 %), and you’re at the physical 1 Gb/s line rate.
──────────────────────────────────────────── 3. What can trip you up ──────────────────────────────────────────── • Policing / shaping – Some ISPs or firewalls apply stricter rules to UDP than TCP. The test may under-represent what TCP would actually do.
• Asymmetric paths – You must test in both directions; one side might choke sooner.
• Hidden bottlenecks – Switch fabric, NIC off-load, or a VPN tunnel may drop packets well before the theoretical line rate. UDP exposes this bluntly.
• Too-large rate jump – If you immediately slam 3 Gbit/s into a 1 Gbit/s link, you’ll see “sender 3 Gbit/s, receiver 1 Gbit/s, 66 % loss”. Doesn’t help you pinpoint the ceiling. Go gradually.
• Packet size – iperf3 defaults to 1470-byte UDP payloads on IPv4 (1458 on v6). If the network uses jumbo frames or has per-packet processing costs, adjust with-l.──────────────────────────────────────────── 4. How is this different from TCP tests? ────────────────────────────────────────────
TCP UDP Paced by congestion & receive window User-defined pacing ( -b)Automatically retransmits No retransmission Gives “real-world” goodput Gives “raw pipe” minus loss/jitter Needs multiple streams or large cwnd Needs careful rate selection Shows capacity after 3-way handshake Measures link even if one-way (e.g. multicast) So if your goal is “How fast will my Steam download go?” stick with TCP.
If your goal is “Does this new 10 Gb fiber link actually forward 9.5 Gb/s without dropping a packet?” use UDP.──────────────────────────────────────────── 5. Practical tips ──────────────────────────────────────────── • Want to throttle precisely to interface line rate?
iperf3 -u -b 9.4G # for a 10 Gb/s link (accounts for headers)• Need RFC 2544-style automation (throughput, latency, frame loss)? Look at
trex,pktgen, or commercial gear; iperf is a quick-and-dirty tool, not a full benchmarking suite.• Capture with
tcpdumpor use SNMP counters on the switch to cross-check that the bits are really on the wire.──────────────────────────────────────────── TL;DR ──────────────────────────────────────────── Yes—running iperf3 in UDP mode with an intelligently chosen
-bvalue is the most straightforward way to probe the raw capacity of a link, unencumbered by TCP’s congestion window. Just remember: the moment packets start dropping, you’ve gone past that capacity, so read the loss column, not just the Mbps number. - What UDP testing actually measures
────────────────────────────────────────────
• iperf3 in UDP mode (
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.