Thanks Dmitry,
I added measurements to curl's scorecard and ran this with some resource/rate
limits. This uses the statistic counter "speed_download" as reported by libcurl
itself and runs a number of transfers *over the same* connection. As you can
see, the limits are obeyed very closely.
You and me might measure slightly different things. For the libcurl statistic
counter, the time the download actually starts is relevant, e.g. when the
response arrives from the server. This does not account for the time to
schedule the transfer or sending the request.
I think the implementation in #20228 looks rather fine.
Cheers,
Stefan
==================================
> python3 tests/http/scorecard.py -d --download-count=5
> --download-sizes=128kb,512kb,1mb,10mb --download-parallel=1
> --limit-rate=$speed h1
Download Speed, limit=256.000 KB/s, from httpd/2.4.66
size serial(5) [cpu/rss]
128KB 257.401 KB/s, +0.5% [0.6%/12MB]
512KB 257.676 KB/s, +0.7% [0.1%/12MB]
1MB 256.890 KB/s, +0.3% [0.1%/12MB]
10MB 256.147 KB/s, +0.1% [0.0%/12MB]
Download Speed, limit=500.000 KB/s, from httpd/2.4.66
size serial(5) [cpu/rss]
128KB 501.302 KB/s, +0.3% [1.5%/12MB]
512KB 502.791 KB/s, +0.6% [0.3%/12MB]
1MB 501.455 KB/s, +0.3% [0.2%/12MB]
10MB 500.431 KB/s, +0.1% [0.1%/12MB]
Download Speed, limit=800.000 KB/s, from httpd/2.4.66
size serial(5) [cpu/rss]
128KB 801.391 KB/s, +0.2% [2.6%/12MB]
512KB 802.930 KB/s, +0.4% [0.5%/12MB]
1MB 801.564 KB/s, +0.2% [0.3%/12MB]
10MB 800.357 KB/s, +0.0% [0.1%/12MB]
Download Speed, limit=1024.000 KB/s, from httpd/2.4.66
size serial(5) [cpu/rss]
128KB 1025.588 KB/s, +0.2% [3.7%/12MB]
512KB 1027.610 KB/s, +0.4% [0.6%/12MB]
1MB 1024.773 KB/s, +0.1% [0.4%/12MB]
10MB 1024.813 KB/s, +0.1% [0.1%/12MB]
Download Speed, limit=5120.000 KB/s, from httpd/2.4.66
size serial(5) [cpu/rss]
128KB 4959.631 KB/s, -3.1% [25.9%/11MB]
512KB 5048.880 KB/s, -1.4% [5.3%/12MB]
1MB 5086.098 KB/s, -0.7% [2.5%/12MB]
10MB 5114.312 KB/s, -0.1% [0.5%/12MB]
> Am 12.01.2026 um 21:24 schrieb Dmitry Karpov <[email protected]>:
>
> Hi Stefan,
> I tried your updated PR, and it seems that it "overthrottles" significantly
> on all rate limits.
> Hopefully, you can fine tune it a little bit more and get a bit less
> "overthrottling", and I will be happy to test it again.
>
> Thanks,
> Dmitry
>
> Here are the stats for 1MB test:
>
> 1 MB downloads
> ###################################################################################
>
> Speed limit test [LAN (~700 mbps)], iterations=5
>
> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=8000000 bps
> time=1162 ms, dnld=1048576 B, speed=7217515 bps, spd_diff=-782485 bps,
> pct=-9.8 %
> time=1121 ms, dnld=1048576 B, speed=7480851 bps, spd_diff=-519149 bps,
> pct=-6.5 %
> time=1134 ms, dnld=1048576 B, speed=7396800 bps, spd_diff=-603200 bps,
> pct=-7.5 %
> time=1114 ms, dnld=1048576 B, speed=7528708 bps, spd_diff=-471292 bps,
> pct=-5.9 %
> time=1120 ms, dnld=1048576 B, speed=7488892 bps, spd_diff=-511108 bps,
> pct=-6.4 %
> --------------------------------------------------------------------
> avg_deviation=-7.2 %, max_deviation=-9.8 %
>
> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=16000000 bps
> time=606 ms, dnld=1048576 B, speed=13820670 bps, spd_diff=-2179330 bps,
> pct=-13.6 %
> time=636 ms, dnld=1048576 B, speed=13186898 bps, spd_diff=-2813102 bps,
> pct=-17.6 %
> time=618 ms, dnld=1048576 B, speed=13573140 bps, spd_diff=-2426860 bps,
> pct=-15.2 %
> time=592 ms, dnld=1048576 B, speed=14154834 bps, spd_diff=-1845166 bps,
> pct=-11.5 %
> time=638 ms, dnld=1048576 B, speed=13129890 bps, spd_diff=-2870110 bps,
> pct=-17.9 %
> --------------------------------------------------------------------
> avg_deviation=-15.2 %, max_deviation=-17.9 %
>
> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=20000000 bps
> time=496 ms, dnld=1048576 B, speed=16895212 bps, spd_diff=-3104788 bps,
> pct=-15.5 %
> time=512 ms, dnld=1048576 B, speed=16383808 bps, spd_diff=-3616192 bps,
> pct=-18.1 %
> time=506 ms, dnld=1048576 B, speed=16556386 bps, spd_diff=-3443614 bps,
> pct=-17.2 %
> time=510 ms, dnld=1048576 B, speed=16421429 bps, spd_diff=-3578571 bps,
> pct=-17.9 %
> time=494 ms, dnld=1048576 B, speed=16952436 bps, spd_diff=-3047564 bps,
> pct=-15.2 %
> --------------------------------------------------------------------
> avg_deviation=-16.8 %, max_deviation=-18.1 %
>
> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=32000000 bps
> time=366 ms, dnld=1048576 B, speed=22919130 bps, spd_diff=-9080870 bps,
> pct=-28.4 %
> time=303 ms, dnld=1048576 B, speed=27633100 bps, spd_diff=-4366900 bps,
> pct=-13.6 %
> time=304 ms, dnld=1048576 B, speed=27572518 bps, spd_diff=-4427482 bps,
> pct=-13.8 %
> time=318 ms, dnld=1048576 B, speed=26365921 bps, spd_diff=-5634079 bps,
> pct=-17.6 %
> time=301 ms, dnld=1048576 B, speed=27827342 bps, spd_diff=-4172658 bps,
> pct=-13.0 %
> --------------------------------------------------------------------
> avg_deviation=-17.3 %, max_deviation=-28.4 %
>
> ###################################################################################
> Speed limit test [50 mbps], iterations=5
>
> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=8000000 bps
> time=1147 ms, dnld=1048576 B, speed=7310760 bps, spd_diff=-689240 bps,
> pct=-8.6 %
> time=1132 ms, dnld=1048576 B, speed=7405884 bps, spd_diff=-594116 bps,
> pct=-7.4 %
> time=1124 ms, dnld=1048576 B, speed=7461461 bps, spd_diff=-538539 bps,
> pct=-6.7 %
> time=1118 ms, dnld=1048576 B, speed=7497768 bps, spd_diff=-502232 bps,
> pct=-6.3 %
> time=1141 ms, dnld=1048576 B, speed=7349718 bps, spd_diff=-650282 bps,
> pct=-8.1 %
> --------------------------------------------------------------------
> avg_deviation=-7.4 %, max_deviation=-8.6 %
>
> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=16000000 bps
> time=589 ms, dnld=1048576 B, speed=14222511 bps, spd_diff=-1777489 bps,
> pct=-11.1 %
> time=607 ms, dnld=1048576 B, speed=13816641 bps, spd_diff=-2183359 bps,
> pct=-13.6 %
> time=621 ms, dnld=1048576 B, speed=13496076 bps, spd_diff=-2503924 bps,
> pct=-15.6 %
> time=604 ms, dnld=1048576 B, speed=13870511 bps, spd_diff=-2129489 bps,
> pct=-13.3 %
> time=628 ms, dnld=1048576 B, speed=13336716 bps, spd_diff=-2663284 bps,
> pct=-16.6 %
> --------------------------------------------------------------------
> avg_deviation=-14.1 %, max_deviation=-16.6 %
>
> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=20000000 bps
> time=509 ms, dnld=1048576 B, speed=16468335 bps, spd_diff=-3531665 bps,
> pct=-17.7 %
> time=510 ms, dnld=1048576 B, speed=16430596 bps, spd_diff=-3569404 bps,
> pct=-17.8 %
> time=492 ms, dnld=1048576 B, speed=17026794 bps, spd_diff=-2973206 bps,
> pct=-14.9 %
> time=510 ms, dnld=1048576 B, speed=16429502 bps, spd_diff=-3570498 bps,
> pct=-17.9 %
> time=524 ms, dnld=1048576 B, speed=15988807 bps, spd_diff=-4011193 bps,
> pct=-20.1 %
> --------------------------------------------------------------------
> avg_deviation=-17.7 %, max_deviation=-20.1 %
>
> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=32000000 bps
> time=358 ms, dnld=1048576 B, speed=23423818 bps, spd_diff=-8576182 bps,
> pct=-26.8 %
> time=283 ms, dnld=1048576 B, speed=29537976 bps, spd_diff=-2462024 bps,
> pct=-7.7 %
> time=269 ms, dnld=1048576 B, speed=31078472 bps, spd_diff=-921528 bps,
> pct=-2.9 %
> time=286 ms, dnld=1048576 B, speed=29328541 bps, spd_diff=-2671459 bps,
> pct=-8.3 %
> time=285 ms, dnld=1048576 B, speed=29375373 bps, spd_diff=-2624627 bps,
> pct=-8.2 %
> --------------------------------------------------------------------
> avg_deviation=-10.8 %, max_deviation=-26.8 %
>
> ###################################################################################
> Speed limit test [20 mbps], iterations=5
>
> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=8000000 bps
> time=1162 ms, dnld=1048576 B, speed=7214871 bps, spd_diff=-785129 bps,
> pct=-9.8 %
> time=1129 ms, dnld=1048576 B, speed=7423612 bps, spd_diff=-576388 bps,
> pct=-7.2 %
> time=1143 ms, dnld=1048576 B, speed=7338729 bps, spd_diff=-661271 bps,
> pct=-8.3 %
> time=1100 ms, dnld=1048576 B, speed=7619530 bps, spd_diff=-380470 bps,
> pct=-4.8 %
> time=1133 ms, dnld=1048576 B, speed=7403106 bps, spd_diff=-596894 bps,
> pct=-7.5 %
> --------------------------------------------------------------------
> avg_deviation=-7.5 %, max_deviation=-9.8 %
>
> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=16000000 bps
> time=616 ms, dnld=1048576 B, speed=13605302 bps, spd_diff=-2394698 bps,
> pct=-15.0 %
> time=555 ms, dnld=1048576 B, speed=15091142 bps, spd_diff=-908858 bps,
> pct=-5.7 %
> time=572 ms, dnld=1048576 B, speed=14647524 bps, spd_diff=-1352476 bps,
> pct=-8.5 %
> time=557 ms, dnld=1048576 B, speed=15049421 bps, spd_diff=-950579 bps,
> pct=-5.9 %
> time=557 ms, dnld=1048576 B, speed=15044509 bps, spd_diff=-955491 bps,
> pct=-6.0 %
> --------------------------------------------------------------------
> avg_deviation=-8.2 %, max_deviation=-15.0 %
>
> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=20000000 bps
> time=453 ms, dnld=1048576 B, speed=18511360 bps, spd_diff=-1488640 bps,
> pct=-7.4 %
> time=457 ms, dnld=1048576 B, speed=18342771 bps, spd_diff=-1657229 bps,
> pct=-8.3 %
> time=446 ms, dnld=1048576 B, speed=18783479 bps, spd_diff=-1216521 bps,
> pct=-6.1 %
> time=462 ms, dnld=1048576 B, speed=18141374 bps, spd_diff=-1858626 bps,
> pct=-9.3 %
> time=447 ms, dnld=1048576 B, speed=18758781 bps, spd_diff=-1241219 bps,
> pct=-6.2 %
> --------------------------------------------------------------------
> avg_deviation=-7.5 %, max_deviation=-9.3 %
>
> -----Original Message-----
> From: Stefan Eissing <[email protected]>
> Sent: Monday, January 12, 2026 8:28 AM
> To: Dmitry Karpov <[email protected]>
> Cc: libcurl development <[email protected]>
> Subject: [EXTERNAL] Re: Rate limit regressions in libcurl 8.18.0 vs 8.17.0
>
>
>
>> Am 09.01.2026 um 21:24 schrieb Dmitry Karpov <[email protected]>:
>>
>> Hi Stefan,
>> Thanks a lot for looking into it!
>>
>> I tried your PR, and it seems that it has improved the precision in the most
>> problematic cases (I tried it on 1MB and 2MB downloads), and now I don't see
>> any cases that look like the rate limit is not applied.
>> But it looks like it tends to "overthrottle", and sometimes the measured
>> transfer speed is below the rate limit more than ~30-50% (please see my
>> stats below).
>>
>> In my opinion, a small "overthrottling" (< 5%) is better than
>> "underthrottling", but transfer speeds that drop too low below the rate
>> limit may be create issues for some applications.
>> But hopefully, you can fine tune it, and I will be happy to help with
>> testing.
>
> Ah, right. That does not work.
>
> I changed strategy in https://github.com/curl/curl/pull/20228 and hope that
> this addresses all cases better now. Would love to get your feedback.
>
> Cheers,
> Stefan
>
>>
>> Thanks!
>> Dmitry
>>
>> Here are the stats for 1MB and 2MB download tests for ~700 mbps, 50 mbps, 20
>> mbps network speeds and 8 mbps, 16 mbps, 20 mbps and 32 mbps rate limits:
>>
>> ######################################################################
>> #############
>> 1 MB downloads
>> ######################################################################
>> #############
>>
>> Speed limit test [LAN (~700 mbps)], iterations=5
>>
>> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=8000000 bps
>> time=1072 ms, dnld=1048576 B, speed=7820634 bps, spd_diff=-179366 bps,
>> pct=-2.2 %
>> time=1071 ms, dnld=1048576 B, speed=7830197 bps, spd_diff=-169803 bps,
>> pct=-2.1 %
>> time=1063 ms, dnld=1048576 B, speed=7890697 bps, spd_diff=-109303 bps,
>> pct=-1.4 %
>> time=1055 ms, dnld=1048576 B, speed=7950382 bps, spd_diff=-49618 bps,
>> pct=-0.6 %
>> time=1071 ms, dnld=1048576 B, speed=7831827 bps, spd_diff=-168173 bps,
>> pct=-2.1 %
>> --------------------------------------------------------------------
>> avg_deviation=-1.7 %, max_deviation=-2.2 %
>>
>> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=16000000 bps
>> time=542 ms, dnld=1048576 B, speed=15463841 bps, spd_diff=-536159 bps,
>> pct=-3.4 %
>> time=536 ms, dnld=1048576 B, speed=15639592 bps, spd_diff=-360408 bps,
>> pct=-2.3 %
>> time=542 ms, dnld=1048576 B, speed=15464154 bps, spd_diff=-535846 bps,
>> pct=-3.3 %
>> time=542 ms, dnld=1048576 B, speed=15469744 bps, spd_diff=-530256 bps,
>> pct=-3.3 %
>> time=527 ms, dnld=1048576 B, speed=15909541 bps, spd_diff=-90459 bps,
>> pct=-0.6 %
>> --------------------------------------------------------------------
>> avg_deviation=-2.6 %, max_deviation=-3.4 %
>>
>> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=20000000 bps
>> time=439 ms, dnld=1048576 B, speed=19103703 bps, spd_diff=-896297 bps,
>> pct=-4.5 %
>> time=443 ms, dnld=1048576 B, speed=18897772 bps, spd_diff=-1102228
>> bps, pct=-5.5 %
>> time=431 ms, dnld=1048576 B, speed=19429137 bps, spd_diff=-570863 bps,
>> pct=-2.9 %
>> time=441 ms, dnld=1048576 B, speed=19018250 bps, spd_diff=-981750 bps,
>> pct=-4.9 %
>> time=431 ms, dnld=1048576 B, speed=19433368 bps, spd_diff=-566632 bps,
>> pct=-2.8 %
>> --------------------------------------------------------------------
>> avg_deviation=-4.1 %, max_deviation=-5.5 %
>>
>> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=32000000 bps
>> time=283 ms, dnld=1048576 B, speed=29608454 bps, spd_diff=-2391546
>> bps, pct=-7.5 %
>> time=286 ms, dnld=1048576 B, speed=29281244 bps, spd_diff=-2718756
>> bps, pct=-8.5 %
>> time=290 ms, dnld=1048576 B, speed=28913572 bps, spd_diff=-3086428
>> bps, pct=-9.6 %
>> time=285 ms, dnld=1048576 B, speed=29405957 bps, spd_diff=-2594043
>> bps, pct=-8.1 %
>> time=287 ms, dnld=1048576 B, speed=29219232 bps, spd_diff=-2780768
>> bps, pct=-8.7 %
>> --------------------------------------------------------------------
>> avg_deviation=-8.5 %, max_deviation=-9.6 %
>>
>> ######################################################################
>> ############# Speed limit test [50 mbps], iterations=5
>>
>> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=8000000 bps
>> time=1057 ms, dnld=1048576 B, speed=7929438 bps, spd_diff=-70562 bps,
>> pct=-0.9 %
>> time=1064 ms, dnld=1048576 B, speed=7882452 bps, spd_diff=-117548 bps,
>> pct=-1.5 %
>> time=1068 ms, dnld=1048576 B, speed=7850084 bps, spd_diff=-149916 bps,
>> pct=-1.9 %
>> time=1074 ms, dnld=1048576 B, speed=7806653 bps, spd_diff=-193347 bps,
>> pct=-2.4 %
>> time=1066 ms, dnld=1048576 B, speed=7866227 bps, spd_diff=-133773 bps,
>> pct=-1.7 %
>> --------------------------------------------------------------------
>> avg_deviation=-1.7 %, max_deviation=-2.4 %
>>
>> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=16000000 bps
>> time=522 ms, dnld=1048576 B, speed=16059423 bps, spd_diff=59423 bps,
>> pct=0.4 %
>> time=540 ms, dnld=1048576 B, speed=15523564 bps, spd_diff=-476436 bps,
>> pct=-3.0 %
>> time=549 ms, dnld=1048576 B, speed=15274231 bps, spd_diff=-725769 bps,
>> pct=-4.5 %
>> time=539 ms, dnld=1048576 B, speed=15544361 bps, spd_diff=-455639 bps,
>> pct=-2.8 %
>> time=539 ms, dnld=1048576 B, speed=15552027 bps, spd_diff=-447973 bps,
>> pct=-2.8 %
>> --------------------------------------------------------------------
>> avg_deviation=-2.6 %, max_deviation=-4.5 %
>>
>> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=20000000 bps
>> time=577 ms, dnld=1048576 B, speed=14525426 bps, spd_diff=-5474574
>> bps, pct=-27.4 %
>> time=602 ms, dnld=1048576 B, speed=13927046 bps, spd_diff=-6072954
>> bps, pct=-30.4 %
>> time=587 ms, dnld=1048576 B, speed=14268378 bps, spd_diff=-5731622
>> bps, pct=-28.7 %
>> time=600 ms, dnld=1048576 B, speed=13957796 bps, spd_diff=-6042204
>> bps, pct=-30.2 %
>> time=577 ms, dnld=1048576 B, speed=14527941 bps, spd_diff=-5472059
>> bps, pct=-27.4 %
>> --------------------------------------------------------------------
>> avg_deviation=-28.8 %, max_deviation=-30.4 %
>>
>> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=32000000 bps
>> time=431 ms, dnld=1048576 B, speed=19442602 bps, spd_diff=-12557398
>> bps, pct=-39.2 %
>> time=409 ms, dnld=1048576 B, speed=20489454 bps, spd_diff=-11510546
>> bps, pct=-36.0 %
>> time=427 ms, dnld=1048576 B, speed=19600100 bps, spd_diff=-12399900
>> bps, pct=-38.7 %
>> time=426 ms, dnld=1048576 B, speed=19689534 bps, spd_diff=-12310466
>> bps, pct=-38.5 %
>> time=419 ms, dnld=1048576 B, speed=19977490 bps, spd_diff=-12022510
>> bps, pct=-37.6 %
>> --------------------------------------------------------------------
>> avg_deviation=-38.0 %, max_deviation=-39.2 %
>>
>> ######################################################################
>> ############# Speed limit test [20 mbps], iterations=5
>>
>> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=8000000 bps
>> time=1066 ms, dnld=1048576 B, speed=7863558 bps, spd_diff=-136442 bps,
>> pct=-1.7 %
>> time=1068 ms, dnld=1048576 B, speed=7848108 bps, spd_diff=-151892 bps,
>> pct=-1.9 %
>> time=1062 ms, dnld=1048576 B, speed=7892040 bps, spd_diff=-107960 bps,
>> pct=-1.3 %
>> time=1057 ms, dnld=1048576 B, speed=7934613 bps, spd_diff=-65387 bps,
>> pct=-0.8 %
>> time=1069 ms, dnld=1048576 B, speed=7844182 bps, spd_diff=-155818 bps,
>> pct=-1.9 %
>> --------------------------------------------------------------------
>> avg_deviation=-1.5 %, max_deviation=-1.9 %
>>
>> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=16000000 bps
>> time=551 ms, dnld=1048576 B, speed=15206422 bps, spd_diff=-793578 bps,
>> pct=-5.0 %
>> time=528 ms, dnld=1048576 B, speed=15882191 bps, spd_diff=-117809 bps,
>> pct=-0.7 %
>> time=526 ms, dnld=1048576 B, speed=15942559 bps, spd_diff=-57441 bps,
>> pct=-0.4 %
>> time=542 ms, dnld=1048576 B, speed=15468688 bps, spd_diff=-531312 bps,
>> pct=-3.3 %
>> time=525 ms, dnld=1048576 B, speed=15951199 bps, spd_diff=-48801 bps,
>> pct=-0.3 %
>> --------------------------------------------------------------------
>> avg_deviation=-1.9 %, max_deviation=-5.0 %
>>
>> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=20000000 bps
>> time=776 ms, dnld=1048576 B, speed=10796719 bps, spd_diff=-9203281
>> bps, pct=-46.0 %
>> time=792 ms, dnld=1048576 B, speed=10591462 bps, spd_diff=-9408538
>> bps, pct=-47.0 %
>> time=773 ms, dnld=1048576 B, speed=10841258 bps, spd_diff=-9158742
>> bps, pct=-45.8 %
>> time=793 ms, dnld=1048576 B, speed=10566860 bps, spd_diff=-9433140
>> bps, pct=-47.2 %
>> time=790 ms, dnld=1048576 B, speed=10612754 bps, spd_diff=-9387246
>> bps, pct=-46.9 %
>> --------------------------------------------------------------------
>> avg_deviation=-46.6 %, max_deviation=-47.2 %
>>
>> Url=http://192.168.1.218/Data/file_1M.bin, max_speed=32000000 bps
>> time=655 ms, dnld=1048576 B, speed=12793733 bps, spd_diff=-19206267
>> bps, pct=-60.0 %
>> time=653 ms, dnld=1048576 B, speed=12843644 bps, spd_diff=-19156356
>> bps, pct=-59.9 %
>> time=629 ms, dnld=1048576 B, speed=13334787 bps, spd_diff=-18665213
>> bps, pct=-58.3 %
>> time=617 ms, dnld=1048576 B, speed=13579578 bps, spd_diff=-18420422
>> bps, pct=-57.6 %
>> time=648 ms, dnld=1048576 B, speed=12931094 bps, spd_diff=-19068906
>> bps, pct=-59.6 %
>> --------------------------------------------------------------------
>> avg_deviation=-59.1 %, max_deviation=-60.0 %
>>
>>
>> ######################################################################
>> #############
>> 2 MB downloads
>> ######################################################################
>> #############
>>
>> Speed limit test [LAN (~700 mbps)], iterations=5
>>
>> Url=http://192.168.1.218/Data/file_2M.bin, max_speed=8000000 bps
>> time=2099 ms, dnld=2097152 B, speed=7991289 bps, spd_diff=-8711 bps,
>> pct=-0.1 %
>> time=2117 ms, dnld=2097152 B, speed=7923378 bps, spd_diff=-76622 bps,
>> pct=-1.0 %
>> time=2114 ms, dnld=2097152 B, speed=7933832 bps, spd_diff=-66168 bps,
>> pct=-0.8 %
>> time=2127 ms, dnld=2097152 B, speed=7887428 bps, spd_diff=-112572 bps,
>> pct=-1.4 %
>> time=2125 ms, dnld=2097152 B, speed=7892716 bps, spd_diff=-107284 bps,
>> pct=-1.3 %
>> --------------------------------------------------------------------
>> avg_deviation=-0.9 %, max_deviation=-1.4 %
>>
>> Url=http://192.168.1.218/Data/file_2M.bin, max_speed=16000000 bps
>> time=1062 ms, dnld=2097152 B, speed=15793888 bps, spd_diff=-206112
>> bps, pct=-1.3 %
>> time=1069 ms, dnld=2097152 B, speed=15680930 bps, spd_diff=-319070
>> bps, pct=-2.0 %
>> time=1070 ms, dnld=2097152 B, speed=15674103 bps, spd_diff=-325897
>> bps, pct=-2.0 %
>> time=1078 ms, dnld=2097152 B, speed=15550571 bps, spd_diff=-449429
>> bps, pct=-2.8 %
>> time=1068 ms, dnld=2097152 B, speed=15694690 bps, spd_diff=-305310
>> bps, pct=-1.9 %
>> --------------------------------------------------------------------
>> avg_deviation=-2.0 %, max_deviation=-2.8 %
>>
>> Url=http://192.168.1.218/Data/file_2M.bin, max_speed=20000000 bps
>> time=860 ms, dnld=2097152 B, speed=19486411 bps, spd_diff=-513589 bps,
>> pct=-2.6 %
>> time=845 ms, dnld=2097152 B, speed=19837391 bps, spd_diff=-162609 bps,
>> pct=-0.8 %
>> time=839 ms, dnld=2097152 B, speed=19990582 bps, spd_diff=-9418 bps,
>> pct=-0.0 %
>> time=845 ms, dnld=2097152 B, speed=19849525 bps, spd_diff=-150475 bps,
>> pct=-0.8 %
>> time=869 ms, dnld=2097152 B, speed=19289411 bps, spd_diff=-710589 bps,
>> pct=-3.6 %
>> --------------------------------------------------------------------
>> avg_deviation=-1.5 %, max_deviation=-3.6 %
>>
>> Url=http://192.168.1.218/Data/file_2M.bin, max_speed=32000000 bps
>> time=541 ms, dnld=2097152 B, speed=30958042 bps, spd_diff=-1041958
>> bps, pct=-3.3 %
>> time=541 ms, dnld=2097152 B, speed=30967700 bps, spd_diff=-1032300
>> bps, pct=-3.2 %
>> time=541 ms, dnld=2097152 B, speed=31003867 bps, spd_diff=-996133 bps,
>> pct=-3.1 %
>> time=542 ms, dnld=2097152 B, speed=30947307 bps, spd_diff=-1052693
>> bps, pct=-3.3 %
>> time=545 ms, dnld=2097152 B, speed=30776880 bps, spd_diff=-1223120
>> bps, pct=-3.8 %
>> --------------------------------------------------------------------
>> avg_deviation=-3.3 %, max_deviation=-3.8 %
>>
>> ######################################################################
>> ############# Speed limit test [50 mbps], iterations=5
>>
>> Url=http://192.168.1.218/Data/file_2M.bin, max_speed=8000000 bps
>> time=2112 ms, dnld=2097152 B, speed=7942723 bps, spd_diff=-57277 bps,
>> pct=-0.7 %
>> time=2118 ms, dnld=2097152 B, speed=7918513 bps, spd_diff=-81487 bps,
>> pct=-1.0 %
>> time=2114 ms, dnld=2097152 B, speed=7935585 bps, spd_diff=-64415 bps,
>> pct=-0.8 %
>> time=2119 ms, dnld=2097152 B, speed=7915999 bps, spd_diff=-84001 bps,
>> pct=-1.1 %
>> time=2106 ms, dnld=2097152 B, speed=7963194 bps, spd_diff=-36806 bps,
>> pct=-0.5 %
>> --------------------------------------------------------------------
>> avg_deviation=-0.8 %, max_deviation=-1.1 %
>>
>> Url=http://192.168.1.218/Data/file_2M.bin, max_speed=16000000 bps
>> time=1065 ms, dnld=2097152 B, speed=15742286 bps, spd_diff=-257714
>> bps, pct=-1.6 %
>> time=1072 ms, dnld=2097152 B, speed=15641866 bps, spd_diff=-358134
>> bps, pct=-2.2 %
>> time=1075 ms, dnld=2097152 B, speed=15594077 bps, spd_diff=-405923
>> bps, pct=-2.5 %
>> time=1067 ms, dnld=2097152 B, speed=15721751 bps, spd_diff=-278249
>> bps, pct=-1.7 %
>> time=1067 ms, dnld=2097152 B, speed=15714653 bps, spd_diff=-285347
>> bps, pct=-1.8 %
>> --------------------------------------------------------------------
>> avg_deviation=-2.0 %, max_deviation=-2.5 %
>>
>> Url=http://192.168.1.218/Data/file_2M.bin, max_speed=20000000 bps
>> time=935 ms, dnld=2097152 B, speed=17931176 bps, spd_diff=-2068824
>> bps, pct=-10.3 %
>> time=939 ms, dnld=2097152 B, speed=17851539 bps, spd_diff=-2148461
>> bps, pct=-10.7 %
>> time=927 ms, dnld=2097152 B, speed=18079130 bps, spd_diff=-1920870
>> bps, pct=-9.6 %
>> time=922 ms, dnld=2097152 B, speed=18195737 bps, spd_diff=-1804263
>> bps, pct=-9.0 %
>> time=935 ms, dnld=2097152 B, speed=17942471 bps, spd_diff=-2057529
>> bps, pct=-10.3 %
>> --------------------------------------------------------------------
>> avg_deviation=-10.0 %, max_deviation=-10.7 %
>>
>> Url=http://192.168.1.218/Data/file_2M.bin, max_speed=32000000 bps
>> time=537 ms, dnld=2097152 B, speed=31188242 bps, spd_diff=-811758 bps,
>> pct=-2.5 %
>> time=535 ms, dnld=2097152 B, speed=31324971 bps, spd_diff=-675029 bps,
>> pct=-2.1 %
>> time=524 ms, dnld=2097152 B, speed=31993531 bps, spd_diff=-6469 bps,
>> pct=-0.0 %
>> time=549 ms, dnld=2097152 B, speed=30523564 bps, spd_diff=-1476436
>> bps, pct=-4.6 %
>> time=539 ms, dnld=2097152 B, speed=31108610 bps, spd_diff=-891390 bps,
>> pct=-2.8 %
>> --------------------------------------------------------------------
>> avg_deviation=-2.4 %, max_deviation=-4.6 %
>>
>> ######################################################################
>> ############# Speed limit test [20 mbps], iterations=5
>>
>> Url=http://192.168.1.218/Data/file_2M.bin, max_speed=8000000 bps
>> time=2109 ms, dnld=2097152 B, speed=7951826 bps, spd_diff=-48174 bps,
>> pct=-0.6 %
>> time=2118 ms, dnld=2097152 B, speed=7920782 bps, spd_diff=-79218 bps,
>> pct=-1.0 %
>> time=2117 ms, dnld=2097152 B, speed=7923865 bps, spd_diff=-76135 bps,
>> pct=-1.0 %
>> time=2101 ms, dnld=2097152 B, speed=7983159 bps, spd_diff=-16841 bps,
>> pct=-0.2 %
>> time=2120 ms, dnld=2097152 B, speed=7912620 bps, spd_diff=-87380 bps,
>> pct=-1.1 %
>> --------------------------------------------------------------------
>> avg_deviation=-0.8 %, max_deviation=-1.1 %
>>
>> Url=http://192.168.1.218/Data/file_2M.bin, max_speed=16000000 bps
>> time=1062 ms, dnld=2097152 B, speed=15792758 bps, spd_diff=-207242
>> bps, pct=-1.3 %
>> time=1051 ms, dnld=2097152 B, speed=15954445 bps, spd_diff=-45555 bps,
>> pct=-0.3 %
>> time=1075 ms, dnld=2097152 B, speed=15595788 bps, spd_diff=-404212
>> bps, pct=-2.5 %
>> time=1071 ms, dnld=2097152 B, speed=15661125 bps, spd_diff=-338875
>> bps, pct=-2.1 %
>> time=1076 ms, dnld=2097152 B, speed=15588282 bps, spd_diff=-411718
>> bps, pct=-2.6 %
>> --------------------------------------------------------------------
>> avg_deviation=-1.8 %, max_deviation=-2.6 %
>>
>> Url=http://192.168.1.218/Data/file_2M.bin, max_speed=20000000 bps
>> time=1170 ms, dnld=2097152 B, speed=14327818 bps, spd_diff=-5672182
>> bps, pct=-28.4 %
>> time=1157 ms, dnld=2097152 B, speed=14492201 bps, spd_diff=-5507799
>> bps, pct=-27.5 %
>> time=1158 ms, dnld=2097152 B, speed=14477419 bps, spd_diff=-5522581
>> bps, pct=-27.6 %
>> time=1146 ms, dnld=2097152 B, speed=14630854 bps, spd_diff=-5369146
>> bps, pct=-26.8 %
>> time=1175 ms, dnld=2097152 B, speed=14269531 bps, spd_diff=-5730469
>> bps, pct=-28.7 %
>> --------------------------------------------------------------------
>> avg_deviation=-27.8 %, max_deviation=-28.7 %
>>
>> Url=http://192.168.1.218/Data/file_2M.bin, max_speed=32000000 bps
>> time=873 ms, dnld=2097152 B, speed=19210450 bps, spd_diff=-12789550
>> bps, pct=-40.0 %
>> time=873 ms, dnld=2097152 B, speed=19201589 bps, spd_diff=-12798411
>> bps, pct=-40.0 %
>> time=886 ms, dnld=2097152 B, speed=18935311 bps, spd_diff=-13064689
>> bps, pct=-40.8 %
>> time=873 ms, dnld=2097152 B, speed=19209900 bps, spd_diff=-12790100
>> bps, pct=-40.0 %
>> time=875 ms, dnld=2097152 B, speed=19170017 bps, spd_diff=-12829983
>> bps, pct=-40.1 %
>> --------------------------------------------------------------------
>> avg_deviation=-40.2 %, max_deviation=-40.8 %
>>
>>
>> -----Original Message-----
>> From: Stefan Eissing <[email protected]>
>> Sent: Friday, January 9, 2026 4:31 AM
>> To: libcurl development <[email protected]>
>> Cc: Dmitry Karpov <[email protected]>
>> Subject: [EXTERNAL] Re: Rate limit regressions in libcurl 8.18.0 vs
>> 8.17.0
>>
>> Dimitry,
>>
>> having let this stew around in my mind the last couple of days, I came up
>> with a tweak that hopefully addresses your issue.
>>
>> Could you throw https://github.com/curl/curl/pull/20228 into your statistics
>> grinder? I would be interested to hear how it turns out.
>>
>> Thanks,
>> Stefan
>>
>>> Am 07.01.2026 um 21:16 schrieb Dmitry Karpov via curl-library
>>> <[email protected]>:
>>>
>>>> My point is that just the same statement: "there are cases that look like
>>>> as the rate limit hasn’t applied at all" - is also true for 8.6.0. The
>>>> only difference is that with 8.6.0 you have to go to smaller transfer
>>>> sizes to see the effect.
>>>
>>> No, this is not true. As I mentioned in my message, in 8.6.0, the smaller
>>> transfer sizes decreased accuracy, but it was always applied unless the
>>> transfers were extremely small.
>>> But in 8.18.0 we have cases when no rate limit is applied to the transfers
>>> depending on run-time conditions.
>>>
>>>> So yes, at a given transfer size the behavior may be different, because
>>>> the rate-limiting "resolution" has changed, but claiming that this is a
>>>> radical change in behavior seems wrong.
>>>
>>> It is a radical change, because we now have cases in 8.18.0, when the rate
>>> limit is either not applied at all or there is a huge gap between the rate
>>> limit and the measured transfer speed.
>>> And it may increase the operational costs significantly for libcurl users
>>> in such cases.
>>>
>>> Let’s consider an example from my test where the rate limit wasn’t applied:
>>> Url=… file_1M.bin, max_speed=20000000 bps
>>> time=18 ms, dnld=1048576 B, speed=457518843 bps, spd_diff=437518843
>>> bps, pct=2187.6 %
>>>
>>> If I do multiple downloads like this back-to-back (i.e. 100+ times) then
>>> CPU utilization for these transfers will be as high as without the rate
>>> limit both for client and the server.
>>> But in 8.6.0 – where the rate limit for these conditions was applied
>>> correctly, the total CPU utilization (both for client and server) was a few
>>> times lower.
>>>
>>> So, while 8.18.0 rate limiting mechanism decreased CPU utilization
>>> compared to 8.6.0 for the cases where it works, it increased CPU
>>> utilization on order of magnitude both for client and server for run-time
>>> conditions where it doesn’t work. And this may have a large cost effect for
>>> applications with large population using cloud services.
>>>
>>> And what makes things worse is that it is now very difficult to predict
>>> where the new mechanism will “not work”.
>>>
>>> In the same test example, the rate limit of 16 Mbps was applied correctly:
>>> Url=… file_1M.bin, max_speed=16000000 bps
>>> time=510 ms, dnld=1048576 B, speed=16423165 bps, spd_diff=423165 bps,
>>> pct=2.6 %
>>>
>>> This means that in 8.18.0 some client app doing multiple back-to-back
>>> transfers, may suddenly step on cases where the rate limiting doesn’t work
>>> as before and thus incur much higher operational costs.
>>> And this kind of defeats the purpose of optimizing the rate limiting
>>> mechanism to use less CPU cycles.
>>>
>>> I am trying to avoid calling it “regression”, but it is a radical behavior
>>> change, which potentially can cost much more to libcurl users than previous
>>> implementation.
>>>
>>> Thanks,
>>> Dmitry
>>> From: David Pfitzner <[email protected]>
>>> Sent: Tuesday, January 6, 2026 7:36 PM
>>> To: Dmitry Karpov <[email protected]>
>>> Cc: libcurl development <[email protected]>
>>> Subject: Re: [EXTERNAL] Re: Rate limit regressions in libcurl 8.18.0 vs
>>> 8.17.0
>>> On Wed, Jan 7, 2026 at 1:02 PM Dmitry Karpov <[email protected]> wrote:
>>> In the 8.6.0, the rate limiter mechanism was kind of embedded into the run
>>> loop, so it was always engaged and showed very good precision (< 5%
>>> deviation of the measured speed from the rate limit) even for relatively
>>> small transfer sizes (~300 KB) and super-fast network speeds.
>>>
>>> It worked properly even when client and server were on the same host
>>> machine where there were no network delays.
>>> Here is the test example for 390 KB download throttling test on 8.6.0 when
>>> server and client are on the same machine:
>>> (8.6.0) Speed limit test [server on the same host as client],
>>> iterations=5 Url=http://192.168.1.3/Data/DSC_391kB.jpg,
>>> max_speed=8000000 bps
>>> time=399 ms, dnld=400096 B, speed=8020969 bps, spd_diff=20969 bps,
>>> pct=0.3 %
>>> time=405 ms, dnld=400096 B, speed=7884597 bps, spd_diff=-115403 bps,
>>> pct=-1.4 %
>>> time=406 ms, dnld=400096 B, speed=7882480 bps, spd_diff=-117520 bps,
>>> pct=-1.5 %
>>> time=404 ms, dnld=400096 B, speed=7910005 bps, spd_diff=-89995 bps,
>>> pct=-1.1 %
>>> time=397 ms, dnld=400096 B, speed=8056036 bps, spd_diff=56036 bps,
>>> pct=0.7 %
>>> --------------------------------------------------------------------
>>> avg_deviation=-0.6 %, max_deviation=-1.5 %
>>> Url=http://192.168.1.3/Data/DSC_391kB.jpg, max_speed=16000000 bps
>>> time=209 ms, dnld=400096 B, speed=15286666 bps, spd_diff=-713334 bps,
>>> pct=-4.5 %
>>> time=197 ms, dnld=400096 B, speed=16231980 bps, spd_diff=231980 bps,
>>> pct=1.4 %
>>> time=197 ms, dnld=400096 B, speed=16202236 bps, spd_diff=202236 bps,
>>> pct=1.3 %
>>> time=198 ms, dnld=400096 B, speed=16110410 bps, spd_diff=110410 bps,
>>> pct=0.7 %
>>> time=195 ms, dnld=400096 B, speed=16394185 bps, spd_diff=394185 bps,
>>> pct=2.5 %
>>> --------------------------------------------------------------------
>>> avg_deviation=0.3 %, max_deviation=-4.5 %
>>> Url=http://192.168.1.3/Data/DSC_391kB.jpg, max_speed=20000000 bps
>>> time=162 ms, dnld=400096 B, speed=19697882 bps, spd_diff=-302118 bps,
>>> pct=-1.5 %
>>> time=161 ms, dnld=400096 B, speed=19856004 bps, spd_diff=-143996 bps,
>>> pct=-0.7 %
>>> time=166 ms, dnld=400096 B, speed=19249034 bps, spd_diff=-750966 bps,
>>> pct=-3.8 %
>>> time=162 ms, dnld=400096 B, speed=19721549 bps, spd_diff=-278451 bps,
>>> pct=-1.4 %
>>> time=159 ms, dnld=400096 B, speed=20035479 bps, spd_diff=35479 bps,
>>> pct=0.2 %
>>> --------------------------------------------------------------------
>>> avg_deviation=-1.4 %, max_deviation=-3.8 %
>>> Url=http://192.168.1.3/Data/DSC_391kB.jpg, max_speed=32000000 bps
>>> time=97 ms, dnld=400096 B, speed=32785684 bps, spd_diff=785684 bps,
>>> pct=2.5 %
>>> time=99 ms, dnld=400096 B, speed=32143324 bps, spd_diff=143324 bps,
>>> pct=0.4 %
>>> time=99 ms, dnld=400096 B, speed=32290545 bps, spd_diff=290545 bps,
>>> pct=0.9 %
>>> time=98 ms, dnld=400096 B, speed=32509298 bps, spd_diff=509298 bps,
>>> pct=1.6 %
>>> time=98 ms, dnld=400096 B, speed=32478290 bps, spd_diff=478290 bps,
>>> pct=1.5 %
>>> --------------------------------------------------------------------
>>> avg_deviation=1.4 %, max_deviation=2.5 %
>>>
>>> The precision started deteriorating (> 15%) only for the downloads <
>>> 250 KB, but the measured transfer speeds were always less than the rate
>>> limit, so, the back-to-back transfers never created high load issues on the
>>> server side.
>>>
>>> So, to be on the safe side, my statement about high precision of rate
>>> limiting in 8.6.0 should be corrected as "for any transfer sizes > 1MB and
>>> any network conditions".
>>>
>>> Unfortunately, the price for such a precision in 8.6.0 was higher CPU
>>> utilization as more cycles were needed to calculate and apply delays.
>>>
>>> So, the intent to minimize CPU usage while losing some precision in 8.18 is
>>> totally understandable, but the loss of precision due to lesser points of
>>> delay calculations should be somehow compensated at the end of the transfer
>>> to avoid regressions for applications that expect more or less reliable
>>> correlation between applied rate limit and measured transfer speeds.
>>>
>>> Unfortunately, in 8.18.0 there are cases that look like as the rate
>>> limit hasn’t applied at all, and it creates issues for logic making
>>> decisions based on transfer speed measurements, and unexpected higher load
>>> on the server side.
>>> My point is that just the same statement: "there are cases that look like
>>> as the rate limit hasn’t applied at all" - is also true for 8.6.0. The only
>>> difference is that with 8.6.0 you have to go to smaller transfer sizes to
>>> see the effect. So it depends on what transfer sizes you are interested in
>>> (and other parameters). So yes, at a given transfer size the behavior may
>>> be different, because the rate-limiting "resolution" has changed, but
>>> claiming that this is a radical change in behavior seems wrong.
>>> -- David
>>>
>>>
>>> From: curl-library <[email protected]> On Behalf Of
>>> David Pfitzner via curl-library
>>> Sent: Tuesday, January 6, 2026 5:17 PM
>>> To: libcurl development <[email protected]>
>>> Cc: David Pfitzner <[email protected]>
>>> Subject: [EXTERNAL] Re: Rate limit regressions in libcurl 8.18.0 vs 8.17.0
>>> On Wed, Jan 7, 2026 at 10:05 AM Dmitry Karpov via curl-library
>>> <[email protected]> wrote:
>>>
>>> The 8.6.0 worked very well for rate limiting as it provided stable and
>>> predictable measured transfer download speeds for any transfer sizes and
>>> network conditions.
>>> That "for any transfer sizes and network conditions" is not true. The rate
>>> limiting may have done what you wanted for the parameters you care about,
>>> or the parameters that you happened to test, but even for 8.6.0 there are
>>> possible cases where rate limiting is specified but has no effect. (That
>>> is, because the transfer finishes before any rate limiting delay is
>>> imposed.).
>>> -- David
>>> --
>>> Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
>>> Etiquette: https://curl.se/mail/etiquette.html
>>
>>
>
--
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
