On Thu, Jun 18, 2009 at 8:50 PM, Evan Daniel<evanbd at gmail.com> wrote:
> On Thu, Jun 18, 2009 at 8:00 PM, Matthew
> Toseland<toad at amphibian.dyndns.org> wrote:
>> Are you doing more testing?
>>
>> On Saturday 13 June 2009 19:05:36 Evan Daniel wrote:
>>> On Sat, Jun 13, 2009 at 1:08 PM, Matthew
>>> Toseland<toad at amphibian.dyndns.org> wrote:
>>> > Now that 0.7.5 has shipped, we can start making disruptive changes again
>>> > in a few days. The number one item on freenet.uservoice.com has been for
>>> > some time to allow more opennet peers for fast nodes. We have discussed
>>> > this in the past, and the conclusions which I agree with and some others
>>> > do:
>>> > - This is feasible.
>>> > - It will not seriously break routing.
>>> > - Reducing the number of connections on slow nodes may actually be a gain
>>> > in security, by increasing opportunities for coalescing. It will improve
>>> > payload percentages, improve average transfer rates, let slow nodes
>>> > accept more requests from each connection, and should improve overall
>>> > performance.
>>> > - The network should be less impacted by the speed of the slower nodes.
>>> > - But we have tested using fewer connections on slow nodes in the past
>>> > and had anecdotal evidence that it is slower. We need to evaluate it more
>>> > rigorously somehow.
>>> > - Increasing the number of peers allowed for fast opennet nodes, within
>>> > reason, should not have a severe security impact. It should improve
>>> > routing (by a smaller network diameter). It will of course allow fast
>>> > nodes to contribute more to the network. We do need to be careful to
>>> > avoid overreliance on ubernodes (hence an upper limit of maybe 50 peers).
>>> > - Routing security: FOAF routing allows you to capture most of the
>>> > traffic from a node already, the only thing stopping this is the
>>> > 30%-to-one-peer limit.
>>> > - Coalescing security: Increasing the number of peers without increasing
>>> > the bandwidth usage does increase vulnerability to traffic analysis by
>>> > doing less coalescing. On the other hand, this is not a problem if the
>>> > bandwidth usage scales with the number of nodes.
>>> >
>>> > How can we move forward? We need some reliable test results on whether a
>>> > 10KB/sec node is better off with 10 peers or with 20 peers. I think it's
>>> > a fair assumption for faster nodes. Suggestions?
>>>
>>> I haven't tested at numbers that low. ?At 15KiB/s, the stats page
>>> suggests your slightly better off with 12-15 peers than 20. ?I saw no
>>> subjective difference in browsing speed either way.
>>>
>>> I'm happy to do some testing here, if you tell me what data you want
>>> me to collect. ?More testers would obviously be good.
>>>
>>> >
>>> > We also need to set some arbitrary parameters. There is an argument for
>>> > linearity, to avoid penalising nodes with different bandwidth levels, but
>>> > nodes with more peers and the same amount of bandwidth per peer are
>>> > likely to be favoured by opennet anyway... Non-linearity, in the sense of
>>> > having a lower threshold and an upper threshold and linearly add peers
>>> > between them but not necessarily consistently with the lower threshold,
>>> > would mean fewer nodes with lots of peers, and might achieve better
>>> > results? E.g.
>>> >
>>> > 10 peers at 10KB/sec ... 20 peers at 20KB/sec (1 per KB/sec)
>>> > 20 peers at 20KB/sec ... 50 peers at 80KB/sec (1 per 3KB/sec)
>>>
>>> I wouldn't go as low as 10 peers, simply because I haven't tested it.
>>> Other than that, those seem perfectly sensible to me.
>>>
>>> We should also watch for excessive cpu usage. ?If there's lots of bw
>>> available, we'd want to have just enough connections to not quite
>>> limit on available cpu power. ?Of course, I don't really know how many
>>> connections / how much bw it is before that becomes a concern.
>>>
>>> Evan Daniel
>>
>
> I'd been running the Spider, and trying to get a complete run out of
> it in order to provide a full set of bug reports. ?Unfortunately,
> after spidering over 100k keys (representing over a week of runtime),
> the .dbs file became unrecoverably corrupted, and it won't write index
> files. ?I had started rerunning it; I've since paused that and started
> taking data on connections.
>
> I've got a little data so far at 12KiB/s limit, 10 and 12 peers.
> Basically, I don't see a difference between 10 and 12 peers. ?Both
> produce reasonable performance numbers. ?My node has 2 darknet peers,
> remainder opennet. ?I'm not using the node much during these tests; it
> has a few MiB of downloads queued that aren't making progress (old
> files that have probably dropped off).
>
> Evan Daniel
>
>
> 12 peers, 12 KiB/s limit
>
> # bwlimitDelayTime: 91ms
> # nodeAveragePingTime: 408ms
> # darknetSizeEstimateSession: 0 nodes
> # opennetSizeEstimateSession: 63 nodes
> # nodeUptime: 1h37m
>
> # Connected: 10
> # Backed off: 2
>
> # Input Rate: 2.54 KiB/s (of 60.0 KiB/s)
> # Output Rate: 12.9 KiB/s (of 12.0 KiB/s)
> # Total Input: 31.3 MiB (5.5 KiB/s average)
> # Total Output: 47.5 MiB (8.34 KiB/s average)
> # Payload Output: 32.6 MiB (5.73 KiB/sec)(68%)
>
> 1469 ? ?Output bandwidth liability
> 18 ? ? ?>SUB_MAX_PING_TIME
>
> Success rates
> Group ? P(success) ? ? ?Count
> All requests ? ?3.329% ?10,633
> CHKs ? ?9.654% ?3,377
> SSKs ? ?0.386% ?7,256
> Local requests ?2.022% ?2,176
> Remote requests ? ? ? ? 3.666% ?8,457
> Block transfers ? ? ? ? 95.646% ? ? ? ? 666
> Turtled downstream ? ? ?87.500% ? ? ? ? 8
> Transfers timed out ? ? 0.000% ?8
> Turtle requests ? ? ? ? 100.000% ? ? ? ?6
>
> Detailed timings (local CHK fetches)
> Successful ? ? ?9.503s
> Unsuccessful ? ?6.656s
> Average ? ? ? ? 6.700s
>
>
>
> 12 peers, 12 KiB/s limit
>
> # bwlimitDelayTime: 108ms
> # nodeAveragePingTime: 380ms
> # darknetSizeEstimateSession: 20 nodes
> # opennetSizeEstimateSession: 107 nodes
> # nodeUptime: 4h4m
>
> ? ?* Connected: 10
> ? ?* Backed off: 2
>
> # Input Rate: 4.51 KiB/s (of 60.0 KiB/s)
> # Output Rate: 12.9 KiB/s (of 12.0 KiB/s)
> # Total Input: 95.3 MiB (6.64 KiB/s average)
> # Total Output: 144 MiB (10.0 KiB/s average)
> # Payload Output: 102 MiB (7.13 KiB/sec)(70%)
>
> 8661 ? ?Output bandwidth liability
> 26 ? ? ?>SUB_MAX_PING_TIME
>
> Success rates
> Group ? P(success) ? ? ?Count
> All requests ? ?3.497% ?30,738
> CHKs ? ?9.048% ?11,119
> SSKs ? ?0.352% ?19,619
> Local requests ?4.497% ?4,025
> Remote requests ? ? ? ? 3.347% ?26,713
> Block transfers ? ? ? ? 96.388% ? ? ? ? 2,021
> Turtled downstream ? ? ?79.545% ? ? ? ? 44
> Transfers timed out ? ? 0.000% ?44
> Turtle requests ? ? ? ? 50.000% ? ? ? ? 16
>
> Detailed timings (local CHK fetches)
> Successful ? ? ?12.386s
> Unsuccessful ? ?7.091s
> Average ? ? ? ? 7.128s
>
>
>
> 10 peers, 12 KiB/s limit
>
> # bwlimitDelayTime: 78ms
> # nodeAveragePingTime: 385ms
> # darknetSizeEstimateSession: 0 nodes
> # opennetSizeEstimateSession: 79 nodes
> # nodeUptime: 3h31m
>
> ? ?* Connected: 9
> ? ?* Backed off: 1
>
> # Input Rate: 8.25 KiB/s (of 60.0 KiB/s)
> # Output Rate: 13.1 KiB/s (of 12.0 KiB/s)
> # Total Input: 91.2 MiB (7.37 KiB/s average)
> # Total Output: 127 MiB (10.3 KiB/s average)
> # Payload Output: 92.9 MiB (7.50 KiB/sec)(72%)
>
> 4208 ? ?Output bandwidth liability
> 8 ? ? ? >SUB_MAX_PING_TIME
> 1 ? ? ? Insufficient output bandwidth
>
> Success rates
> Group ? P(success) ? ? ?Count
> All requests ? ?4.811% ?25,400
> CHKs ? ?11.310% ? ? ? ? 10,115
> SSKs ? ?0.510% ?15,285
> Local requests ?8.772% ?2,679
> Remote requests ? ? ? ? 4.344% ?22,721
> Block transfers ? ? ? ? 95.744% ? ? ? ? 2,091
> Turtled downstream ? ? ?81.481% ? ? ? ? 54
> Transfers timed out ? ? 0.000% ?54
> Turtle requests ? ? ? ? 33.333% ? ? ? ? 9
>
> Detailed timings (local CHK fetches)
> Successful ? ? ?7.040s
> Unsuccessful ? ?13.240s
> Average ? ? ? ? 13.220s
>
Update with additional data:
12 KiB/s, 20 peers
# bwlimitDelayTime: 87ms
# nodeAveragePingTime: 456ms
# darknetSizeEstimateSession: 0 nodes
# opennetSizeEstimateSession: 913 nodes
# nodeUptime: 3h53s
# Connected: 12
# Backed off: 6
# Input Rate: 2.68 KiB/s (of 60.0 KiB/s)
# Output Rate: 10.9 KiB/s (of 12.0 KiB/s)
# Total Input: 90.9 MiB (8.58 KiB/s average)
# Total Output: 126 MiB (11.9 KiB/s average)
# Payload Output: 78.7 MiB (7.42 KiB/sec)(62%)
11077 Output bandwidth liability
Success rates
Group P(success) Count
All requests 4.310% 25,872
CHKs 12.214% 7,950
SSKs 0.803% 17,922
Local requests 6.115% 2,633
Remote requests 4.105% 23,239
Block transfers 97.431% 1,518
Turtled downstream 86.667% 30
Transfers timed out 0.000% 30
Turtle requests 50.000% 4
Detailed timings (local CHK fetches)
Successful 12.723s
Unsuccessful 12.790s
Average 12.789s
12 KiB/s, 20 peers
# bwlimitDelayTime: 62ms
# nodeAveragePingTime: 359ms
# darknetSizeEstimateSession: 0 nodes
# opennetSizeEstimateSession: 1661 nodes
# nodeUptime: 8h31m
* Connected: 14
* Backed off: 3
# Input Rate: 14.3 KiB/s (of 60.0 KiB/s)
# Output Rate: 13.2 KiB/s (of 12.0 KiB/s)
# Total Input: 250 MiB (8.35 KiB/s average)
# Total Output: 349 MiB (11.6 KiB/s average)
# Payload Output: 222 MiB (7.41 KiB/sec)(63%)
31310 Output bandwidth liability
Success rates
Group P(success) Count
All requests 5.131% 65,481
CHKs 14.236% 21,038
SSKs 0.821% 44,443
Local requests 6.523% 6,485
Remote requests 4.978% 58,996
Block transfers 96.367% 4,459
Turtled downstream 86.290% 124
Transfers timed out 0.806% 124
Turtle requests 36.364% 11
Detailed timings (local CHK fetches)
Successful 15.514s
Unsuccessful 12.646s
Average 12.673s
20 peers, 20 KiB/s
# bwlimitDelayTime: 50ms
# nodeAveragePingTime: 433ms
# darknetSizeEstimateSession: 0 nodes
# opennetSizeEstimateSession: 1758 nodes
# nodeUptime: 8h27m
# Connected: 13
# Backed off: 6
# Input Rate: 7.99 KiB/s (of 60.0 KiB/s)
# Output Rate: 21.1 KiB/s (of 20.0 KiB/s)
# Total Input: 431 MiB (14.5 KiB/s average)
# Total Output: 577 MiB (19.4 KiB/s average)
# Payload Output: 384 MiB (12.9 KiB/sec)(66%)
Success rates
Group P(success) Count
All requests 5.234% 119,096
CHKs 13.031% 43,373
SSKs 0.769% 75,723
Local requests 7.479% 10,295
Remote requests 5.022% 108,801
Block transfers 95.967% 8,628
Turtled downstream 82.773% 238
Transfers timed out 1.681% 238
Turtle requests 49.020% 51
Detailed timings (local CHK fetches)
Successful 18.917s
Unsuccessful 12.768s
Average 12.803s
20 peers, 20 KiB/s
# bwlimitDelayTime: 56ms
# nodeAveragePingTime: 371ms
# darknetSizeEstimateSession: 0 nodes
# opennetSizeEstimateSession: 1285 nodes
# nodeUptime: 6h43m
# Connected: 16
# Backed off: 4
# Input Rate: 7.02 KiB/s (of 60.0 KiB/s)
# Output Rate: 20.9 KiB/s (of 20.0 KiB/s)
# Total Input: 308 MiB (13.0 KiB/s average)
# Total Output: 452 MiB (19.1 KiB/s average)
# Payload Output: 310 MiB (13.1 KiB/sec)(68%)
Success rates
Group P(success) Count
All requests 4.048% 94,150
CHKs 8.544% 41,524
SSKs 0.500% 52,626
Local requests 7.341% 8,541
Remote requests 3.719% 85,609
Block transfers 95.561% 6,128
Turtled downstream 81.176% 170
Transfers timed out 2.353% 170
Turtle requests 72.727% 55
Detailed timings (local CHK fetches)
Successful 20.007s
Unsuccessful 6.894s
Average 6.942s
15 peers, 20 KiB/s
# bwlimitDelayTime: 48ms
# nodeAveragePingTime: 402ms
# darknetSizeEstimateSession: 0 nodes
# opennetSizeEstimateSession: 1988 nodes
# nodeUptime: 5h48m
# Connected: 12
# Backed off: 3
# Input Rate: 15.2 KiB/s (of 60.0 KiB/s)
# Output Rate: 19.0 KiB/s (of 20.0 KiB/s)
# Total Input: 255 MiB (12.4 KiB/s average)
# Total Output: 372 MiB (18.2 KiB/s average)
# Payload Output: 246 MiB (12.0 KiB/sec)(66%)
Success rates
Group P(success) Count
All requests 4.594% 69,998
CHKs 9.933% 30,274
SSKs 0.526% 39,724
Local requests 6.951% 6,848
Remote requests 4.339% 63,150
Block transfers 96.622% 4,826
Turtled downstream 87.619% 105
Transfers timed out 0.000% 105
Turtle requests 50.000% 6
Detailed timings (local CHK fetches)
Successful 8.490s
Unsuccessful 6.634s
Average 6.638s
