[freenet-dev] Wonder why bootstrapping is slow

[Matthew Toseland]

On Tuesday 08 Feb 2011 04:16:25 Juiceman wrote:
> Bootstrapping from seednodes is really slow right now.  I run one of
> those seednodes.  Here are my stats, they have been horrible for about
> 2 weeks now.  I don't suppose my node is serving many announcements,
> huh?  Are other seednodes in similarly bad shape?

I have some theories.

It looks like most of the announcement load is caused by old nodes trying and 
repeatedly failing to update using Update Over Mandatory.

Recent builds should help with this, 1349 should help even more (and might help 
improve announcement performance generally).
> 
> Node status overview
> 
> bwlimitDelayTime: 6706ms
> bwlimitDelayTimeBulk: 6706ms
> bwlimitDelayTimeRT: 0ms
> nodeAveragePingTime: 2452ms
> darknetSizeEstimateSession: 0 nodes
> opennetSizeEstimateSession: 762 nodes
> nodeUptimeSession: 44m57s
> nodeUptimeTotal: 8w5d
> routingMissDistanceLocal: 0.1065
> routingMissDistanceRemote: 0.1270
> routingMissDistanceOverall: 0.1234
> backedOffPercent: 34.4%
> pInstantReject: 92.5%
> unclaimedFIFOSize: 4155
> RAMBucketPoolSize: 12.5 MiB / 150 MiB
> uptimeAverage: 97.2%
> 
> Peer statistics
> 
> Connected: 17
> Backed off: 4
> Too old: 65
> Disconnected: 9
> Never connected: 6
> Disconnecting: 1
> Seeding for: 199
> Max peers: 36
> Max strangers: 36
> 
> Bandwidth
> 
> Input Rate: 42.2 KiB/s (of 1.0 MiB/s)
> Output Rate: 31.2 KiB/s (of 105 KiB/s)
> Session Total Input: 115 MiB (43.8 KiB/s average)
> Session Total Output: 86.0 MiB (32.6 KiB/s average)
> Payload Output: 253 KiB (96 B/sec)(0%)
> Global Total Input: 357 GiB
> Global Total Output: 422 GiB
> Request output (excluding payload): CHK 39.7 KiB SSK 111 KiB.
> Insert output (excluding payload): CHK 7.53 KiB SSK 1.70 KiB.
> Offered keys: sending keys 100 B, sending offers 1016 B
> Swapping Output: 1.99 MiB.
> Connection setup: 30.3 MiB output
> Ack-only packets: 31.6 MiB
> Resent bytes: 5.83 MiB (6%)
> Updater Output: 104 KiB
> Announcement output: 6.29 MiB (transferring node refs payload 6.48 MiB)
> Admin bytes: 1.08 MiB initial messages, 1.37 KiB IP change messages,
> 2.10 KiB disconnection notifications, 0 B routing status
> Debugging bytes: 0 B network coloring, 0 B ping, 6.12 KiB probe
> requests, 0 B routed test messages.
> Node to node messages: 1.45 MiB
> Load allocation notices: 288 KiB
> Other output: 6.59 MiB (7%)
> Total non-request overhead: 31.4 KiB/sec (96%).
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: 

[freenet-dev] Wonder why bootstrapping is slow

[Juiceman]

On Thu, Feb 10, 2011 at 8:47 PM, Matthew Toseland
 wrote:
>
> On Tuesday 08 Feb 2011 04:16:25 Juiceman wrote:
> > Bootstrapping from seednodes is really slow right now. ?I run one of
> > those seednodes. ?Here are my stats, they have been horrible for about
> > 2 weeks now. ?I don't suppose my node is serving many announcements,
> > huh? ?Are other seednodes in similarly bad shape?
>
> I have some theories.
>
> It looks like most of the announcement load is caused by old nodes trying and 
> repeatedly failing to update using Update Over Mandatory.
>
> Recent builds should help with this, 1349 should help even more (and might 
> help improve announcement performance generally).
>

Much better!? 1349 stats:

Node status overview

bwlimitDelayTime:?1152ms
bwlimitDelayTimeBulk:?1152ms
bwlimitDelayTimeRT:?444ms
nodeAveragePingTime:?356ms
darknetSizeEstimateSession:?0?nodes
opennetSizeEstimateSession:?1804?nodes
nodeUptimeSession:?1h28m
nodeUptimeTotal:?9w21h
routingMissDistanceLocal:?0.1760
routingMissDistanceRemote:?0.1475
routingMissDistanceOverall:?0.1481
backedOffPercent:?36.7%
pInstantReject:?0.0%
unclaimedFIFOSize:?1398
RAMBucketPoolSize:?66.2 MiB / 150 MiB
uptimeAverage:?90.1%

Peer statistics

Connected:?26
Backed off:?9
Disconnected:?1
Never connected:?1
Seeding for:?32
Max peers: 36
Max strangers: 36

Bandwidth

Input Rate: 31.7?KiB/s (of 1.0?MiB/s)
Output Rate: 33.4?KiB/s (of 105?KiB/s)
Session Total Input: 267?MiB (51.4?KiB/s average)
Session Total Output: 311?MiB (60.0?KiB/s average)
Payload Output: 236?MiB (45.4?KiB/sec)(75%)
Global Total Input: 369?GiB
Global Total Output: 433?GiB
Request output (excluding payload): CHK -6.61?MiB SSK 23.6?MiB.
Insert output (excluding payload): CHK -2.26?MiB SSK 492?KiB.
Offered keys: sending keys -79.8?KiB, sending offers 307?KiB
Swapping Output: 2.75?MiB.
Connection setup: 4.78?MiB output
Ack-only packets: 6.74?MiB
Resent bytes: 6.70?MiB (2%)
Updater Output: 113?KiB
Announcement output: 9.14?MiB (transferring node refs payload 8.26?MiB)
Admin bytes: 313?KiB initial messages, 0?B IP change messages, 917?B
disconnection notifications, 0?B routing status
Debugging bytes: 0?B network coloring, 0?B ping, 0?B probe requests, 0
B routed test messages.
Node to node messages: 564?KiB
Load allocation notices: 1.08?MiB
Other output: 28.0?MiB (8%)
Total non-request overhead: 6.19 KiB/sec (10%).

Re: [freenet-dev] Wonder why bootstrapping is slow

[Juiceman]

On Thu, Feb 10, 2011 at 8:47 PM, Matthew Toseland
 wrote:
>
> On Tuesday 08 Feb 2011 04:16:25 Juiceman wrote:
> > Bootstrapping from seednodes is really slow right now.  I run one of
> > those seednodes.  Here are my stats, they have been horrible for about
> > 2 weeks now.  I don't suppose my node is serving many announcements,
> > huh?  Are other seednodes in similarly bad shape?
>
> I have some theories.
>
> It looks like most of the announcement load is caused by old nodes trying and 
> repeatedly failing to update using Update Over Mandatory.
>
> Recent builds should help with this, 1349 should help even more (and might 
> help improve announcement performance generally).
>

Much better!  1349 stats:

Node status overview

bwlimitDelayTime: 1152ms
bwlimitDelayTimeBulk: 1152ms
bwlimitDelayTimeRT: 444ms
nodeAveragePingTime: 356ms
darknetSizeEstimateSession: 0 nodes
opennetSizeEstimateSession: 1804 nodes
nodeUptimeSession: 1h28m
nodeUptimeTotal: 9w21h
routingMissDistanceLocal: 0.1760
routingMissDistanceRemote: 0.1475
routingMissDistanceOverall: 0.1481
backedOffPercent: 36.7%
pInstantReject: 0.0%
unclaimedFIFOSize: 1398
RAMBucketPoolSize: 66.2 MiB / 150 MiB
uptimeAverage: 90.1%

Peer statistics

Connected: 26
Backed off: 9
Disconnected: 1
Never connected: 1
Seeding for: 32
Max peers: 36
Max strangers: 36

Bandwidth

Input Rate: 31.7 KiB/s (of 1.0 MiB/s)
Output Rate: 33.4 KiB/s (of 105 KiB/s)
Session Total Input: 267 MiB (51.4 KiB/s average)
Session Total Output: 311 MiB (60.0 KiB/s average)
Payload Output: 236 MiB (45.4 KiB/sec)(75%)
Global Total Input: 369 GiB
Global Total Output: 433 GiB
Request output (excluding payload): CHK -6.61 MiB SSK 23.6 MiB.
Insert output (excluding payload): CHK -2.26 MiB SSK 492 KiB.
Offered keys: sending keys -79.8 KiB, sending offers 307 KiB
Swapping Output: 2.75 MiB.
Connection setup: 4.78 MiB output
Ack-only packets: 6.74 MiB
Resent bytes: 6.70 MiB (2%)
Updater Output: 113 KiB
Announcement output: 9.14 MiB (transferring node refs payload 8.26 MiB)
Admin bytes: 313 KiB initial messages, 0 B IP change messages, 917 B
disconnection notifications, 0 B routing status
Debugging bytes: 0 B network coloring, 0 B ping, 0 B probe requests, 0
B routed test messages.
Node to node messages: 564 KiB
Load allocation notices: 1.08 MiB
Other output: 28.0 MiB (8%)
Total non-request overhead: 6.19 KiB/sec (10%).
___
Devl mailing list
Devl@freenetproject.org
http://freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Wonder why bootstrapping is slow

[Matthew Toseland]

On Tuesday 08 Feb 2011 04:16:25 Juiceman wrote:
> Bootstrapping from seednodes is really slow right now.  I run one of
> those seednodes.  Here are my stats, they have been horrible for about
> 2 weeks now.  I don't suppose my node is serving many announcements,
> huh?  Are other seednodes in similarly bad shape?

I have some theories.

It looks like most of the announcement load is caused by old nodes trying and 
repeatedly failing to update using Update Over Mandatory.

Recent builds should help with this, 1349 should help even more (and might help 
improve announcement performance generally).
> 
> Node status overview
> 
> bwlimitDelayTime: 6706ms
> bwlimitDelayTimeBulk: 6706ms
> bwlimitDelayTimeRT: 0ms
> nodeAveragePingTime: 2452ms
> darknetSizeEstimateSession: 0 nodes
> opennetSizeEstimateSession: 762 nodes
> nodeUptimeSession: 44m57s
> nodeUptimeTotal: 8w5d
> routingMissDistanceLocal: 0.1065
> routingMissDistanceRemote: 0.1270
> routingMissDistanceOverall: 0.1234
> backedOffPercent: 34.4%
> pInstantReject: 92.5%
> unclaimedFIFOSize: 4155
> RAMBucketPoolSize: 12.5 MiB / 150 MiB
> uptimeAverage: 97.2%
> 
> Peer statistics
> 
> Connected: 17
> Backed off: 4
> Too old: 65
> Disconnected: 9
> Never connected: 6
> Disconnecting: 1
> Seeding for: 199
> Max peers: 36
> Max strangers: 36
> 
> Bandwidth
> 
> Input Rate: 42.2 KiB/s (of 1.0 MiB/s)
> Output Rate: 31.2 KiB/s (of 105 KiB/s)
> Session Total Input: 115 MiB (43.8 KiB/s average)
> Session Total Output: 86.0 MiB (32.6 KiB/s average)
> Payload Output: 253 KiB (96 B/sec)(0%)
> Global Total Input: 357 GiB
> Global Total Output: 422 GiB
> Request output (excluding payload): CHK 39.7 KiB SSK 111 KiB.
> Insert output (excluding payload): CHK 7.53 KiB SSK 1.70 KiB.
> Offered keys: sending keys 100 B, sending offers 1016 B
> Swapping Output: 1.99 MiB.
> Connection setup: 30.3 MiB output
> Ack-only packets: 31.6 MiB
> Resent bytes: 5.83 MiB (6%)
> Updater Output: 104 KiB
> Announcement output: 6.29 MiB (transferring node refs payload 6.48 MiB)
> Admin bytes: 1.08 MiB initial messages, 1.37 KiB IP change messages,
> 2.10 KiB disconnection notifications, 0 B routing status
> Debugging bytes: 0 B network coloring, 0 B ping, 6.12 KiB probe
> requests, 0 B routed test messages.
> Node to node messages: 1.45 MiB
> Load allocation notices: 288 KiB
> Other output: 6.59 MiB (7%)
> Total non-request overhead: 31.4 KiB/sec (96%).


signature.asc
Description: This is a digitally signed message part.
___
Devl mailing list
Devl@freenetproject.org
http://freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Wonder why bootstrapping is slow

[Juiceman]

Bootstrapping from seednodes is really slow right now.? I run one of
those seednodes.? Here are my stats, they have been horrible for about
2 weeks now.? I don't suppose my node is serving many announcements,
huh?  Are other seednodes in similarly bad shape?

Node status overview

bwlimitDelayTime:?6706ms
bwlimitDelayTimeBulk:?6706ms
bwlimitDelayTimeRT:?0ms
nodeAveragePingTime:?2452ms
darknetSizeEstimateSession:?0?nodes
opennetSizeEstimateSession:?762?nodes
nodeUptimeSession:?44m57s
nodeUptimeTotal:?8w5d
routingMissDistanceLocal:?0.1065
routingMissDistanceRemote:?0.1270
routingMissDistanceOverall:?0.1234
backedOffPercent:?34.4%
pInstantReject:?92.5%
unclaimedFIFOSize:?4155
RAMBucketPoolSize:?12.5 MiB / 150 MiB
uptimeAverage:?97.2%

Peer statistics

Connected:?17
Backed off:?4
Too old:?65
Disconnected:?9
Never connected:?6
Disconnecting:?1
Seeding for:?199
Max peers: 36
Max strangers: 36

Bandwidth

Input Rate: 42.2?KiB/s (of 1.0?MiB/s)
Output Rate: 31.2?KiB/s (of 105?KiB/s)
Session Total Input: 115?MiB (43.8?KiB/s average)
Session Total Output: 86.0?MiB (32.6?KiB/s average)
Payload Output: 253?KiB (96?B/sec)(0%)
Global Total Input: 357?GiB
Global Total Output: 422?GiB
Request output (excluding payload): CHK 39.7?KiB SSK 111?KiB.
Insert output (excluding payload): CHK 7.53?KiB SSK 1.70?KiB.
Offered keys: sending keys 100?B, sending offers 1016?B
Swapping Output: 1.99?MiB.
Connection setup: 30.3?MiB output
Ack-only packets: 31.6?MiB
Resent bytes: 5.83?MiB (6%)
Updater Output: 104?KiB
Announcement output: 6.29?MiB (transferring node refs payload 6.48?MiB)
Admin bytes: 1.08?MiB initial messages, 1.37?KiB IP change messages,
2.10?KiB disconnection notifications, 0?B routing status
Debugging bytes: 0?B network coloring, 0?B ping, 6.12?KiB probe
requests, 0?B routed test messages.
Node to node messages: 1.45?MiB
Load allocation notices: 288?KiB
Other output: 6.59?MiB (7%)
Total non-request overhead: 31.4 KiB/sec (96%).

[freenet-dev] Wonder why bootstrapping is slow

[Juiceman]

Bootstrapping from seednodes is really slow right now.  I run one of
those seednodes.  Here are my stats, they have been horrible for about
2 weeks now.  I don't suppose my node is serving many announcements,
huh?  Are other seednodes in similarly bad shape?

Node status overview

bwlimitDelayTime: 6706ms
bwlimitDelayTimeBulk: 6706ms
bwlimitDelayTimeRT: 0ms
nodeAveragePingTime: 2452ms
darknetSizeEstimateSession: 0 nodes
opennetSizeEstimateSession: 762 nodes
nodeUptimeSession: 44m57s
nodeUptimeTotal: 8w5d
routingMissDistanceLocal: 0.1065
routingMissDistanceRemote: 0.1270
routingMissDistanceOverall: 0.1234
backedOffPercent: 34.4%
pInstantReject: 92.5%
unclaimedFIFOSize: 4155
RAMBucketPoolSize: 12.5 MiB / 150 MiB
uptimeAverage: 97.2%

Peer statistics

Connected: 17
Backed off: 4
Too old: 65
Disconnected: 9
Never connected: 6
Disconnecting: 1
Seeding for: 199
Max peers: 36
Max strangers: 36

Bandwidth

Input Rate: 42.2 KiB/s (of 1.0 MiB/s)
Output Rate: 31.2 KiB/s (of 105 KiB/s)
Session Total Input: 115 MiB (43.8 KiB/s average)
Session Total Output: 86.0 MiB (32.6 KiB/s average)
Payload Output: 253 KiB (96 B/sec)(0%)
Global Total Input: 357 GiB
Global Total Output: 422 GiB
Request output (excluding payload): CHK 39.7 KiB SSK 111 KiB.
Insert output (excluding payload): CHK 7.53 KiB SSK 1.70 KiB.
Offered keys: sending keys 100 B, sending offers 1016 B
Swapping Output: 1.99 MiB.
Connection setup: 30.3 MiB output
Ack-only packets: 31.6 MiB
Resent bytes: 5.83 MiB (6%)
Updater Output: 104 KiB
Announcement output: 6.29 MiB (transferring node refs payload 6.48 MiB)
Admin bytes: 1.08 MiB initial messages, 1.37 KiB IP change messages,
2.10 KiB disconnection notifications, 0 B routing status
Debugging bytes: 0 B network coloring, 0 B ping, 6.12 KiB probe
requests, 0 B routed test messages.
Node to node messages: 1.45 MiB
Load allocation notices: 288 KiB
Other output: 6.59 MiB (7%)
Total non-request overhead: 31.4 KiB/sec (96%).
___
Devl mailing list
Devl@freenetproject.org
http://freenetproject.org/cgi-bin/mailman/listinfo/devl