[freenet-dev] Post-0.7.0 priorities

[Victor Denisov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

|> My store had filled to 100% in less than 72 hours (that would be
|> slightly less than 1 Gb of data), so I doubt that it really stores
|> anything past the last week at best. And one week isn't exactly what we
|> should aim for when talking about data retention, even for unpopular
data.
|
| I guess you're referring to your *cache*? My store took ages to fill,
and it
| was 4GB, whereas the cache filled pretty quickly.

No, I'm talking about *the store*. My cache filled in a matter of hours,
~ while I was experimenting with different third-party clients and
checking out all the indices and freesites linked from the front page
(I'm downloading about 2 Gb/day, with Freenet responsible for roughly
3/4 of it, according to its stats page).

I remember that when I tried out Freenet about 2 months ago, it took
about 3 weeks to fill the 5 Gb store. It's only to be expected for it to
be filled quicker - I have more bandwidth today, and Freenet (probably)
has more users :-).

Regards,
Victor Denisov.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIKuDQS81Mh9/iCDgRAvt8AKDcLNJZjBHuoxJEpSKKWgVnrtKmMgCguqAt
QUmwAfYeJGqZgLpwX8nvEFk=
=xAkH
-END PGP SIGNATURE-

[freenet-dev] Post-0.7.0 priorities

[Jano]

Florent Daigni?re wrote:

> * Jano  [2008-05-14
> 12:55:49]:
> 
>> Florent Daigni?re wrote:
>> 
>> > * Jano  [2008-05-14
>> > 11:21:05]:
>> > 
>> >> > Personally I'm pretty skeptical of anything requiring more than 100MB.
>> >> 
>> >> However, current implementation (IINM) uses the cache to resume
>> >> downloads. Thus, downloading anything bigger than that in more than one
>> >> go has the potential of a lot of waste in retries (hence BW & time).
>> >> 
>> >> I know, it's a spurious reason since downloads in progress could be saved
>> >> somewhere else until completion... but still is a reason for now.
>> >> 
>> > 
>> > They are good reasons why we shouldn't implement download-resuming.
>> 
>> Could you please elaborate?
>> 
> 
> For the n-th. time : not having that "feature" gives users a good
> incentive to keep their nodes up.

OK. I completely disagree, but I don't want to re-discuss this old topic right
now. I thought you had some security-wise reasons in mind.

[freenet-dev] Post-0.7.0 priorities

[Matthew Toseland]

On Wednesday 14 May 2008 10:21, Jano wrote:
> Ian Clarke wrote:
> 
> > On Tue, May 13, 2008 at 12:53 PM, Matthew Toseland
> >  wrote:
> > 
> >>  > It could be related to the fact that I've only been able to dedicate
> >>  > about 2 Gb for my store, but I doubt it.
> >>
> >>  That certainly won't help.
> > 
> > What evidence is there that people need to have multi-gigabyte
> > datastores?  We aren't necessarily helping ourselves by telling people
> > they need to devote anywhere from 1-5% of their total hard disks to
> > Freenet, unless it *really is* necessary.  Freenet enthusiasts may be
> > ok with this, but casual users probably won't be, and we *need* casual
> > users.
> 
> Totally in agreement with this.
> 
> > 
> > Personally I'm pretty skeptical of anything requiring more than 100MB.

I disagree. If there isn't enough space for there to be a useful amount of 
content, then Freenet won't work well. And I don't see why 1GB is such a big 
deal anyway.
> 
> However, current implementation (IINM) uses the cache to resume downloads.
> Thus, downloading anything bigger than that in more than one go has the
> potential of a lot of waste in retries (hence BW & time).

That will be fixed soon.
> 
> I know, it's a spurious reason since downloads in progress could be saved
> somewhere else until completion... but still is a reason for now.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 

[freenet-dev] Post-0.7.0 priorities

[Florent Daignière]

* Jano  [2008-05-14 12:55:49]:

> Florent Daigni?re wrote:
> 
> > * Jano  [2008-05-14
> > 11:21:05]:
> > 
> >> > Personally I'm pretty skeptical of anything requiring more than 100MB.
> >> 
> >> However, current implementation (IINM) uses the cache to resume downloads.
> >> Thus, downloading anything bigger than that in more than one go has the
> >> potential of a lot of waste in retries (hence BW & time).
> >> 
> >> I know, it's a spurious reason since downloads in progress could be saved
> >> somewhere else until completion... but still is a reason for now.
> >> 
> > 
> > They are good reasons why we shouldn't implement download-resuming.
> 
> Could you please elaborate?
> 

For the n-th. time : not having that "feature" gives users a good
incentive to keep their nodes up.
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: 

[freenet-dev] Post-0.7.0 priorities

[Jano]

Florent Daigni?re wrote:

> * Jano  [2008-05-14
> 11:21:05]:
> 
>> > Personally I'm pretty skeptical of anything requiring more than 100MB.
>> 
>> However, current implementation (IINM) uses the cache to resume downloads.
>> Thus, downloading anything bigger than that in more than one go has the
>> potential of a lot of waste in retries (hence BW & time).
>> 
>> I know, it's a spurious reason since downloads in progress could be saved
>> somewhere else until completion... but still is a reason for now.
>> 
> 
> They are good reasons why we shouldn't implement download-resuming.

Could you please elaborate?

[freenet-dev] Post-0.7.0 priorities

[Florent Daignière]

* Jano  [2008-05-14 11:21:05]:

> > Personally I'm pretty skeptical of anything requiring more than 100MB.
> 
> However, current implementation (IINM) uses the cache to resume downloads.
> Thus, downloading anything bigger than that in more than one go has the
> potential of a lot of waste in retries (hence BW & time).
> 
> I know, it's a spurious reason since downloads in progress could be saved
> somewhere else until completion... but still is a reason for now.
> 

They are good reasons why we shouldn't implement download-resuming.
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: 

[freenet-dev] Post-0.7.0 priorities

[Jano]

Victor Denisov wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> | What evidence is there that people need to have multi-gigabyte
> | datastores?  We aren't necessarily helping ourselves by telling people
> | they need to devote anywhere from 1-5% of their total hard disks to
> | Freenet, unless it *really is* necessary.  Freenet enthusiasts may be
> | ok with this, but casual users probably won't be, and we *need* casual
> | users.
> |
> | Personally I'm pretty skeptical of anything requiring more than 100MB.
> 
> My store had filled to 100% in less than 72 hours (that would be
> slightly less than 1 Gb of data), so I doubt that it really stores
> anything past the last week at best. And one week isn't exactly what we
> should aim for when talking about data retention, even for unpopular data.

I guess you're referring to your *cache*? My store took ages to fill, and it
was 4GB, whereas the cache filled pretty quickly.

[freenet-dev] Post-0.7.0 priorities

[Jano]

Ian Clarke wrote:

> On Tue, May 13, 2008 at 12:53 PM, Matthew Toseland
>  wrote:
> 
>>  > It could be related to the fact that I've only been able to dedicate
>>  > about 2 Gb for my store, but I doubt it.
>>
>>  That certainly won't help.
> 
> What evidence is there that people need to have multi-gigabyte
> datastores?  We aren't necessarily helping ourselves by telling people
> they need to devote anywhere from 1-5% of their total hard disks to
> Freenet, unless it *really is* necessary.  Freenet enthusiasts may be
> ok with this, but casual users probably won't be, and we *need* casual
> users.

Totally in agreement with this.

> 
> Personally I'm pretty skeptical of anything requiring more than 100MB.

However, current implementation (IINM) uses the cache to resume downloads.
Thus, downloading anything bigger than that in more than one go has the
potential of a lot of waste in retries (hence BW & time).

I know, it's a spurious reason since downloads in progress could be saved
somewhere else until completion... but still is a reason for now.

[freenet-dev] Post-0.7.0 priorities

[Victor Denisov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

| I disagree. Your actual bandwidth usage is determined by how many
requests the
| other nodes send you. This is largely determined by the *average
bandwidth
| limit* across the whole network. If we increase the average bandwidth
limit,
| we increase the traffic on your node.

Ah, ok, that's probably true. I was thinking you've been talking about
that helping *my* node to better utilize the bandwidth available, hence
my reply.

|> Great that we agree on this one. I've been unsuccessful in bringing at
|> least two of my friends to Freenet because they were running into
|> memory-related problems, one of them going as far as calling Freenet
|> "that damn bloatware" (well, actual wording also included a couple of
|> pretty strong Russian expletives :-).
|
| Hehe. Consensus is good. They are specifically talking about
memory/CPU here?
| Or bandwidth usage, total transfer in a month, etc etc?

Well, memory usage was what finally turned them off, as far as I can
see. One of the people I've mentioned (a pretty experienced computer and
p2p user) had immediately and repeatedly crashed his node upon
discovering Thaw and Frost file sharing tools - he put a couple of
hundreds of large files there for both insertion and download, not
really understanding how Freenet handles this (on a side note, this is
something which is *really* unclear to most casual P2P users I've talked
to, especially those with Gnutella/eMule/Kazaa background). Naturally,
Freenet quickly ran out of memory, and simply wasn't recovering on its
own. We've finally been able to solve this problem by giving Java enough
memory to load the queue, then quickly removing files from it (there's
probably a better way, but I wasn't aware of it).

In the second case, the machine on which we tried to install Freenet was
a little bit old (but nothing ancient - P4 2.4 GHz/1 Gb RAM), used as a
dedicated P2P machine by my other friend. Shortly after installing
Freenet, it became unresponsive - almost constant disk access, high CPU
usage, etc. After stopping Freenet, the load went down immediately. We
tried actually giving Freenet less than default memory (96 Mb, IIRC),
but it didn't really help.

|> Well, it seems more or less straightforward from the outside: handle
|> additional update URLs and a set of (revocable?) public keys + expose
|> the API over FCP. Am I missing something important?
|
| Yeah but we'd have to unpack, support post-unpack scripts, etc etc ...
really
| we'd want the apps to provide their own unpacker and just feed them a
single
| file for them to do what they want with?

I think that the latter approach is preferable - they could be using
full-blown native installers, etc. No need for Freenet to open that can
of worms. Just allow the app to check for updates over FCP and feed it
the (verified) update file when one becomes available. What to do with
it is up to the application.

Regards,
Victor Denisov.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIKekCS81Mh9/iCDgRAirPAKDMp5pzomB5CCZgDSJmS9iHv0I0RgCg02+v
eEJQYj+Qj0VTyGW2I1uXJDE=
=6bPm
-END PGP SIGNATURE-

Re: [freenet-dev] Post-0.7.0 priorities

[Florent Daignière]

* Jano [EMAIL PROTECTED] [2008-05-14 11:21:05]:

  Personally I'm pretty skeptical of anything requiring more than 100MB.
 
 However, current implementation (IINM) uses the cache to resume downloads.
 Thus, downloading anything bigger than that in more than one go has the
 potential of a lot of waste in retries (hence BW  time).
 
 I know, it's a spurious reason since downloads in progress could be saved
 somewhere else until completion... but still is a reason for now.
 

They are good reasons why we shouldn't implement download-resuming.


signature.asc
Description: Digital signature
___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Post-0.7.0 priorities

[Jano]

Victor Denisov wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 | What evidence is there that people need to have multi-gigabyte
 | datastores?  We aren't necessarily helping ourselves by telling people
 | they need to devote anywhere from 1-5% of their total hard disks to
 | Freenet, unless it *really is* necessary.  Freenet enthusiasts may be
 | ok with this, but casual users probably won't be, and we *need* casual
 | users.
 |
 | Personally I'm pretty skeptical of anything requiring more than 100MB.
 
 My store had filled to 100% in less than 72 hours (that would be
 slightly less than 1 Gb of data), so I doubt that it really stores
 anything past the last week at best. And one week isn't exactly what we
 should aim for when talking about data retention, even for unpopular data.

I guess you're referring to your *cache*? My store took ages to fill, and it
was 4GB, whereas the cache filled pretty quickly.

___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Post-0.7.0 priorities

[Jano]

Ian Clarke wrote:

 On Tue, May 13, 2008 at 12:53 PM, Matthew Toseland
 [EMAIL PROTECTED] wrote:
 
   It could be related to the fact that I've only been able to dedicate
   about 2 Gb for my store, but I doubt it.

  That certainly won't help.
 
 What evidence is there that people need to have multi-gigabyte
 datastores?  We aren't necessarily helping ourselves by telling people
 they need to devote anywhere from 1-5% of their total hard disks to
 Freenet, unless it *really is* necessary.  Freenet enthusiasts may be
 ok with this, but casual users probably won't be, and we *need* casual
 users.

Totally in agreement with this.

 
 Personally I'm pretty skeptical of anything requiring more than 100MB.

However, current implementation (IINM) uses the cache to resume downloads.
Thus, downloading anything bigger than that in more than one go has the
potential of a lot of waste in retries (hence BW  time).

I know, it's a spurious reason since downloads in progress could be saved
somewhere else until completion... but still is a reason for now.

___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Post-0.7.0 priorities

[Victor Denisov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

| My store had filled to 100% in less than 72 hours (that would be
| slightly less than 1 Gb of data), so I doubt that it really stores
| anything past the last week at best. And one week isn't exactly what we
| should aim for when talking about data retention, even for unpopular
data.
|
| I guess you're referring to your *cache*? My store took ages to fill,
and it
| was 4GB, whereas the cache filled pretty quickly.

No, I'm talking about *the store*. My cache filled in a matter of hours,
~ while I was experimenting with different third-party clients and
checking out all the indices and freesites linked from the front page
(I'm downloading about 2 Gb/day, with Freenet responsible for roughly
3/4 of it, according to its stats page).

I remember that when I tried out Freenet about 2 months ago, it took
about 3 weeks to fill the 5 Gb store. It's only to be expected for it to
be filled quicker - I have more bandwidth today, and Freenet (probably)
has more users :-).

Regards,
Victor Denisov.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIKuDQS81Mh9/iCDgRAvt8AKDcLNJZjBHuoxJEpSKKWgVnrtKmMgCguqAt
QUmwAfYeJGqZgLpwX8nvEFk=
=xAkH
-END PGP SIGNATURE-
___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Post-0.7.0 priorities

[Florent Daignière]

* Jano [EMAIL PROTECTED] [2008-05-14 12:55:49]:

 Florent Daignière wrote:
 
  * Jano [EMAIL PROTECTED] [2008-05-14
  11:21:05]:
  
   Personally I'm pretty skeptical of anything requiring more than 100MB.
  
  However, current implementation (IINM) uses the cache to resume downloads.
  Thus, downloading anything bigger than that in more than one go has the
  potential of a lot of waste in retries (hence BW  time).
  
  I know, it's a spurious reason since downloads in progress could be saved
  somewhere else until completion... but still is a reason for now.
  
  
  They are good reasons why we shouldn't implement download-resuming.
 
 Could you please elaborate?
 

For the n-th. time : not having that feature gives users a good
incentive to keep their nodes up.


signature.asc
Description: Digital signature
___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Post-0.7.0 priorities

[Jano]

Florent Daignière wrote:

 * Jano [EMAIL PROTECTED] [2008-05-14
 12:55:49]:
 
 Florent Daignière wrote:
 
  * Jano [EMAIL PROTECTED] [2008-05-14
  11:21:05]:
  
   Personally I'm pretty skeptical of anything requiring more than 100MB.
  
  However, current implementation (IINM) uses the cache to resume
  downloads. Thus, downloading anything bigger than that in more than one
  go has the potential of a lot of waste in retries (hence BW  time).
  
  I know, it's a spurious reason since downloads in progress could be saved
  somewhere else until completion... but still is a reason for now.
  
  
  They are good reasons why we shouldn't implement download-resuming.
 
 Could you please elaborate?
 
 
 For the n-th. time : not having that feature gives users a good
 incentive to keep their nodes up.

OK. I completely disagree, but I don't want to re-discuss this old topic right
now. I thought you had some security-wise reasons in mind.

___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Post-0.7.0 priorities

[Matthew Toseland]

On Wednesday 14 May 2008 10:21, Jano wrote:
 Ian Clarke wrote:
 
  On Tue, May 13, 2008 at 12:53 PM, Matthew Toseland
  [EMAIL PROTECTED] wrote:
  
It could be related to the fact that I've only been able to dedicate
about 2 Gb for my store, but I doubt it.
 
   That certainly won't help.
  
  What evidence is there that people need to have multi-gigabyte
  datastores?  We aren't necessarily helping ourselves by telling people
  they need to devote anywhere from 1-5% of their total hard disks to
  Freenet, unless it *really is* necessary.  Freenet enthusiasts may be
  ok with this, but casual users probably won't be, and we *need* casual
  users.
 
 Totally in agreement with this.
 
  
  Personally I'm pretty skeptical of anything requiring more than 100MB.

I disagree. If there isn't enough space for there to be a useful amount of 
content, then Freenet won't work well. And I don't see why 1GB is such a big 
deal anyway.
 
 However, current implementation (IINM) uses the cache to resume downloads.
 Thus, downloading anything bigger than that in more than one go has the
 potential of a lot of waste in retries (hence BW  time).

That will be fixed soon.
 
 I know, it's a spurious reason since downloads in progress could be saved
 somewhere else until completion... but still is a reason for now.


pgpBFv339aB7p.pgp
Description: PGP signature
___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Post-0.7.0 priorities

[Victor Denisov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

| What evidence is there that people need to have multi-gigabyte
| datastores?  We aren't necessarily helping ourselves by telling people
| they need to devote anywhere from 1-5% of their total hard disks to
| Freenet, unless it *really is* necessary.  Freenet enthusiasts may be
| ok with this, but casual users probably won't be, and we *need* casual
| users.
|
| Personally I'm pretty skeptical of anything requiring more than 100MB.

My store had filled to 100% in less than 72 hours (that would be
slightly less than 1 Gb of data), so I doubt that it really stores
anything past the last week at best. And one week isn't exactly what we
should aim for when talking about data retention, even for unpopular data.

As a side note, I *really* miss a stat for the LRU timestamp for the
store (or at least I wasn't able to find it) - so I coule be wrong with
my "one week" estimate. It would be really interesting to see for how
long does your store actually store things.

Regards,
Victor Denisov.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIKeMhS81Mh9/iCDgRAkD8AJ0WoWa2n7W5t/W6CAgtFKEOugdNzACgvr2d
ybupUuIQ/AcE7mCFQh0wQkY=
=BGZ/
-END PGP SIGNATURE-

[freenet-dev] Post-0.7.0 priorities

[Matthew Toseland]

On Tuesday 13 May 2008 19:38, Ian Clarke wrote:
> On Tue, May 13, 2008 at 12:53 PM, Matthew Toseland
>  wrote:
> 
> >  > It could be related to the fact that I've only been able to dedicate
> >  > about 2 Gb for my store, but I doubt it.
> >
> >  That certainly won't help.
> 
> What evidence is there that people need to have multi-gigabyte
> datastores?  We aren't necessarily helping ourselves by telling people
> they need to devote anywhere from 1-5% of their total hard disks to
> Freenet, unless it *really is* necessary.  Freenet enthusiasts may be
> ok with this, but casual users probably won't be, and we *need* casual
> users.
> 
> Personally I'm pretty skeptical of anything requiring more than 100MB.

How big is your current hard disk?

Do you have any ideas for some empirical or scientific way to determine how 
much storage is needed? AFAICS "as much as possible" has to be a good thing 
for data retention, no?

Assuming a node uses 16K/sec bandwidth, which is the default, uses the same 
output as input (normally input is a bit lower than output, but not a lot 
lower), and uses 50% of that for receiving data it didn't have before, and 
has a 100MB cache, a new block which isn't subsequently requested should 
reach the bottom of the LRU in 3.55 hours.

Data will stay in the store for much longer however. But that relies on us 
being able to find it, and the nodes it is stored to being online.

Being able to find it is probably largely a matter at the moment of location 
churn. We need to deal with this. There are some ideas, we need to talk to 
oskar and vive about them.

The nodes we stored the data to being online is probably intractable. :(
> 
> Ian.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 

[freenet-dev] Post-0.7.0 priorities

[Matthew Toseland]

On Monday 12 May 2008 23:10, Michael Rogers wrote:
> Victor Denisov wrote:
> > Input Rate: 17.6 KiB/sec (of 300 KiB)
> > Output Rate: 15.9 KiB/sec (of 200 KiB)
> > Total Input: 4.83 GiB (28.3 KiB/sec)
> > Total Output: 5.66 GiB (33.2 KiB/sec)
> > 
> > Used Java memory: 122 MiB
> > Allocated Java memory: 127 MiB
> > Maximum Java memory: 284 MiB
> > Running threads: 152/700
> > 
> > So, basically, network had grown about 3x after 0.7 release. My node has
> > been up for 2 days, and is pretty well established in the network. It's
> > not overloaded (CPU usage is ~ 5-10%). Yet it doesn't use more than
> > about 15% of the allowed bandwidth, on average.
> 
> If the average speed of a node is, say, 20 KB/s then your node's 
> unlikely to be able to use 300 KB/s because its peers won't be 
> sending/accepting enough traffic (due to their own bandwidth limits, not 
> yours). This could be solved by allowing fast nodes to have more peers, 
> but of course that creates the possibility of ubernodes.

It could also be solved to a degree by a different load management scheme that 
allowed us more fine grained control over the degree to which ubernodes can 
take more requests... But a) that's probably too big a job for 0.7.1, and b) 
the current system seems to more or less work.
> 
> Cheers,
> Michael
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 

[freenet-dev] Post-0.7.0 priorities

[Matthew Toseland]

On Monday 12 May 2008 20:19, Victor Denisov wrote:
> | On Friday 09 May 2008 07:27, Victor Denisov wrote:
> |> | Automatic bandwidth calibration. Other p2p apps have this, we should
> |> have it.
> |>
> |> Good idea. Also, we should definitely look into better utilizing
> |> available bandwidth. Freenet's the only p2p app which consistently
> |> underutilizes my upload limit (~ 2 Mbit/s out of 8 Mbit/s of link
> |> capacity). I understand that we don't want to create supernodes, but
> |> come on, 2 Mbit/s is *nothing* these days.
> |
> | IMHO automatic bandwidth calibration will help a lot with this. Beyond
> that
> | we're looking at token passing, which may be too big for 0.7.1.
> 
> Excerpt from my current node stats:
> 
> networkSizeEstimateSession: 6039 nodes
> nodeUptime: 2d1h
> pInstantReject: 0,0%
> uptimeAverage: 100,0%
> Peer statistics
> ~* Connected: 17
> ~* Backed off: 3
> ~* Seeding for: 111
> Input Rate: 17.6 KiB/sec (of 300 KiB)
> Output Rate: 15.9 KiB/sec (of 200 KiB)
> Total Input: 4.83 GiB (28.3 KiB/sec)
> Total Output: 5.66 GiB (33.2 KiB/sec)
> 
> Used Java memory: 122 MiB
> Allocated Java memory: 127 MiB
> Maximum Java memory: 284 MiB
> Running threads: 152/700
> 
> So, basically, network had grown about 3x after 0.7 release. 

Great!

> My node has 
> been up for 2 days, and is pretty well established in the network. It's
> not overloaded (CPU usage is ~ 5-10%). Yet it doesn't use more than
> about 15% of the allowed bandwidth, on average. Automatic bandwidth
> calibration won't help - I've allowed Freenet to use much less that my
> uplink allows to.

I disagree. Your actual bandwidth usage is determined by how many requests the 
other nodes send you. This is largely determined by the *average bandwidth 
limit* across the whole network. If we increase the average bandwidth limit, 
we increase the traffic on your node.
> 
> It could be related to the fact that I've only been able to dedicate
> about 2 Gb for my store, but I doubt it.

That certainly won't help.
> 
> | Agreed, memory usage is a usability issue: the user shouldn't have to
> care
> | about it.
> 
> Great that we agree on this one. I've been unsuccessful in bringing at
> least two of my friends to Freenet because they were running into
> memory-related problems, one of them going as far as calling Freenet
> "that damn bloatware" (well, actual wording also included a couple of
> pretty strong Russian expletives :-).

Hehe. Consensus is good. They are specifically talking about memory/CPU here? 
Or bandwidth usage, total transfer in a month, etc etc?
> 
> |> Shouldn't we consider auto-updating bundled applications as well? Or
> |> perhaps providing an auto-update API for use by third-party apps? Just a
> |> thought.
> |
> | Maybe, that would be harder though. I would be happy to discuss it
> with their
> | authors.
> 
> Well, it seems more or less straightforward from the outside: handle
> additional update URLs and a set of (revocable?) public keys + expose
> the API over FCP. Am I missing something important?

Yeah but we'd have to unpack, support post-unpack scripts, etc etc ... really 
we'd want the apps to provide their own unpacker and just feed them a single 
file for them to do what they want with?
> 
> Regards,
> Victor Denisov.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 

[freenet-dev] Post-0.7.0 priorities

[Ian Clarke]

On Tue, May 13, 2008 at 12:53 PM, Matthew Toseland
 wrote:

>  > It could be related to the fact that I've only been able to dedicate
>  > about 2 Gb for my store, but I doubt it.
>
>  That certainly won't help.

What evidence is there that people need to have multi-gigabyte
datastores?  We aren't necessarily helping ourselves by telling people
they need to devote anywhere from 1-5% of their total hard disks to
Freenet, unless it *really is* necessary.  Freenet enthusiasts may be
ok with this, but casual users probably won't be, and we *need* casual
users.

Personally I'm pretty skeptical of anything requiring more than 100MB.

Ian.

-- 
Email: ian at uprizer.com
Cell: +1 512 422 3588
Skype: sanity

[freenet-dev] Post-0.7.0 priorities

[Victor Denisov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

| On Friday 09 May 2008 07:27, Victor Denisov wrote:
|> | Automatic bandwidth calibration. Other p2p apps have this, we should
|> have it.
|>
|> Good idea. Also, we should definitely look into better utilizing
|> available bandwidth. Freenet's the only p2p app which consistently
|> underutilizes my upload limit (~ 2 Mbit/s out of 8 Mbit/s of link
|> capacity). I understand that we don't want to create supernodes, but
|> come on, 2 Mbit/s is *nothing* these days.
|
| IMHO automatic bandwidth calibration will help a lot with this. Beyond
that
| we're looking at token passing, which may be too big for 0.7.1.

Excerpt from my current node stats:

networkSizeEstimateSession: 6039 nodes
nodeUptime: 2d1h
pInstantReject: 0,0%
uptimeAverage: 100,0%
Peer statistics
~* Connected: 17
~* Backed off: 3
~* Seeding for: 111
Input Rate: 17.6 KiB/sec (of 300 KiB)
Output Rate: 15.9 KiB/sec (of 200 KiB)
Total Input: 4.83 GiB (28.3 KiB/sec)
Total Output: 5.66 GiB (33.2 KiB/sec)

Used Java memory: 122 MiB
Allocated Java memory: 127 MiB
Maximum Java memory: 284 MiB
Running threads: 152/700

So, basically, network had grown about 3x after 0.7 release. My node has
been up for 2 days, and is pretty well established in the network. It's
not overloaded (CPU usage is ~ 5-10%). Yet it doesn't use more than
about 15% of the allowed bandwidth, on average. Automatic bandwidth
calibration won't help - I've allowed Freenet to use much less that my
uplink allows to.

It could be related to the fact that I've only been able to dedicate
about 2 Gb for my store, but I doubt it.

| Agreed, memory usage is a usability issue: the user shouldn't have to
care
| about it.

Great that we agree on this one. I've been unsuccessful in bringing at
least two of my friends to Freenet because they were running into
memory-related problems, one of them going as far as calling Freenet
"that damn bloatware" (well, actual wording also included a couple of
pretty strong Russian expletives :-).

|> Shouldn't we consider auto-updating bundled applications as well? Or
|> perhaps providing an auto-update API for use by third-party apps? Just a
|> thought.
|
| Maybe, that would be harder though. I would be happy to discuss it
with their
| authors.

Well, it seems more or less straightforward from the outside: handle
additional update URLs and a set of (revocable?) public keys + expose
the API over FCP. Am I missing something important?

Regards,
Victor Denisov.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIKJhIS81Mh9/iCDgRAkPFAJ96CPlcNouiHiVjavq/xtY6y8XR6QCglIpB
IwLrElLZxQZyo9WKTTWdbyo=
=FMsn
-END PGP SIGNATURE-

[freenet-dev] Post-0.7.0 priorities

[Michael Rogers]

Victor Denisov wrote:
> Input Rate: 17.6 KiB/sec (of 300 KiB)
> Output Rate: 15.9 KiB/sec (of 200 KiB)
> Total Input: 4.83 GiB (28.3 KiB/sec)
> Total Output: 5.66 GiB (33.2 KiB/sec)
> 
> Used Java memory: 122 MiB
> Allocated Java memory: 127 MiB
> Maximum Java memory: 284 MiB
> Running threads: 152/700
> 
> So, basically, network had grown about 3x after 0.7 release. My node has
> been up for 2 days, and is pretty well established in the network. It's
> not overloaded (CPU usage is ~ 5-10%). Yet it doesn't use more than
> about 15% of the allowed bandwidth, on average.

If the average speed of a node is, say, 20 KB/s then your node's 
unlikely to be able to use 300 KB/s because its peers won't be 
sending/accepting enough traffic (due to their own bandwidth limits, not 
yours). This could be solved by allowing fast nodes to have more peers, 
but of course that creates the possibility of ubernodes.

Cheers,
Michael

Re: [freenet-dev] Post-0.7.0 priorities

[Matthew Toseland]

On Monday 12 May 2008 20:19, Victor Denisov wrote:
 | On Friday 09 May 2008 07:27, Victor Denisov wrote:
 | | Automatic bandwidth calibration. Other p2p apps have this, we should
 | have it.
 |
 | Good idea. Also, we should definitely look into better utilizing
 | available bandwidth. Freenet's the only p2p app which consistently
 | underutilizes my upload limit (~ 2 Mbit/s out of 8 Mbit/s of link
 | capacity). I understand that we don't want to create supernodes, but
 | come on, 2 Mbit/s is *nothing* these days.
 |
 | IMHO automatic bandwidth calibration will help a lot with this. Beyond
 that
 | we're looking at token passing, which may be too big for 0.7.1.
 
 Excerpt from my current node stats:
 
 networkSizeEstimateSession: 6039 nodes
 nodeUptime: 2d1h
 pInstantReject: 0,0%
 uptimeAverage: 100,0%
 Peer statistics
 ~* Connected: 17
 ~* Backed off: 3
 ~* Seeding for: 111
 Input Rate: 17.6 KiB/sec (of 300 KiB)
 Output Rate: 15.9 KiB/sec (of 200 KiB)
 Total Input: 4.83 GiB (28.3 KiB/sec)
 Total Output: 5.66 GiB (33.2 KiB/sec)
 
 Used Java memory: 122 MiB
 Allocated Java memory: 127 MiB
 Maximum Java memory: 284 MiB
 Running threads: 152/700
 
 So, basically, network had grown about 3x after 0.7 release. 

Great!

 My node has 
 been up for 2 days, and is pretty well established in the network. It's
 not overloaded (CPU usage is ~ 5-10%). Yet it doesn't use more than
 about 15% of the allowed bandwidth, on average. Automatic bandwidth
 calibration won't help - I've allowed Freenet to use much less that my
 uplink allows to.

I disagree. Your actual bandwidth usage is determined by how many requests the 
other nodes send you. This is largely determined by the *average bandwidth 
limit* across the whole network. If we increase the average bandwidth limit, 
we increase the traffic on your node.
 
 It could be related to the fact that I've only been able to dedicate
 about 2 Gb for my store, but I doubt it.

That certainly won't help.
 
 | Agreed, memory usage is a usability issue: the user shouldn't have to
 care
 | about it.
 
 Great that we agree on this one. I've been unsuccessful in bringing at
 least two of my friends to Freenet because they were running into
 memory-related problems, one of them going as far as calling Freenet
 that damn bloatware (well, actual wording also included a couple of
 pretty strong Russian expletives :-).

Hehe. Consensus is good. They are specifically talking about memory/CPU here? 
Or bandwidth usage, total transfer in a month, etc etc?
 
 | Shouldn't we consider auto-updating bundled applications as well? Or
 | perhaps providing an auto-update API for use by third-party apps? Just a
 | thought.
 |
 | Maybe, that would be harder though. I would be happy to discuss it
 with their
 | authors.
 
 Well, it seems more or less straightforward from the outside: handle
 additional update URLs and a set of (revocable?) public keys + expose
 the API over FCP. Am I missing something important?

Yeah but we'd have to unpack, support post-unpack scripts, etc etc ... really 
we'd want the apps to provide their own unpacker and just feed them a single 
file for them to do what they want with?
 
 Regards,
 Victor Denisov.


pgpNEFjXe9HnW.pgp
Description: PGP signature
___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Post-0.7.0 priorities

[Matthew Toseland]

On Monday 12 May 2008 23:10, Michael Rogers wrote:
 Victor Denisov wrote:
  Input Rate: 17.6 KiB/sec (of 300 KiB)
  Output Rate: 15.9 KiB/sec (of 200 KiB)
  Total Input: 4.83 GiB (28.3 KiB/sec)
  Total Output: 5.66 GiB (33.2 KiB/sec)
  
  Used Java memory: 122 MiB
  Allocated Java memory: 127 MiB
  Maximum Java memory: 284 MiB
  Running threads: 152/700
  
  So, basically, network had grown about 3x after 0.7 release. My node has
  been up for 2 days, and is pretty well established in the network. It's
  not overloaded (CPU usage is ~ 5-10%). Yet it doesn't use more than
  about 15% of the allowed bandwidth, on average.
 
 If the average speed of a node is, say, 20 KB/s then your node's 
 unlikely to be able to use 300 KB/s because its peers won't be 
 sending/accepting enough traffic (due to their own bandwidth limits, not 
 yours). This could be solved by allowing fast nodes to have more peers, 
 but of course that creates the possibility of ubernodes.

It could also be solved to a degree by a different load management scheme that 
allowed us more fine grained control over the degree to which ubernodes can 
take more requests... But a) that's probably too big a job for 0.7.1, and b) 
the current system seems to more or less work.
 
 Cheers,
 Michael


pgpGcHdLATgRD.pgp
Description: PGP signature
___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Post-0.7.0 priorities

[Ian Clarke]

On Tue, May 13, 2008 at 12:53 PM, Matthew Toseland
[EMAIL PROTECTED] wrote:

   It could be related to the fact that I've only been able to dedicate
   about 2 Gb for my store, but I doubt it.

  That certainly won't help.

What evidence is there that people need to have multi-gigabyte
datastores?  We aren't necessarily helping ourselves by telling people
they need to devote anywhere from 1-5% of their total hard disks to
Freenet, unless it *really is* necessary.  Freenet enthusiasts may be
ok with this, but casual users probably won't be, and we *need* casual
users.

Personally I'm pretty skeptical of anything requiring more than 100MB.

Ian.

-- 
Email: [EMAIL PROTECTED]
Cell: +1 512 422 3588
Skype: sanity
___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Post-0.7.0 priorities

[Matthew Toseland]

On Tuesday 13 May 2008 19:38, Ian Clarke wrote:
 On Tue, May 13, 2008 at 12:53 PM, Matthew Toseland
 [EMAIL PROTECTED] wrote:
 
It could be related to the fact that I've only been able to dedicate
about 2 Gb for my store, but I doubt it.
 
   That certainly won't help.
 
 What evidence is there that people need to have multi-gigabyte
 datastores?  We aren't necessarily helping ourselves by telling people
 they need to devote anywhere from 1-5% of their total hard disks to
 Freenet, unless it *really is* necessary.  Freenet enthusiasts may be
 ok with this, but casual users probably won't be, and we *need* casual
 users.
 
 Personally I'm pretty skeptical of anything requiring more than 100MB.

How big is your current hard disk?

Do you have any ideas for some empirical or scientific way to determine how 
much storage is needed? AFAICS as much as possible has to be a good thing 
for data retention, no?

Assuming a node uses 16K/sec bandwidth, which is the default, uses the same 
output as input (normally input is a bit lower than output, but not a lot 
lower), and uses 50% of that for receiving data it didn't have before, and 
has a 100MB cache, a new block which isn't subsequently requested should 
reach the bottom of the LRU in 3.55 hours.

Data will stay in the store for much longer however. But that relies on us 
being able to find it, and the nodes it is stored to being online.

Being able to find it is probably largely a matter at the moment of location 
churn. We need to deal with this. There are some ideas, we need to talk to 
oskar and vive about them.

The nodes we stored the data to being online is probably intractable. :(
 
 Ian.


pgpt56KXwLvWB.pgp
Description: PGP signature
___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Post-0.7.0 priorities

[Victor Denisov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

| What evidence is there that people need to have multi-gigabyte
| datastores?  We aren't necessarily helping ourselves by telling people
| they need to devote anywhere from 1-5% of their total hard disks to
| Freenet, unless it *really is* necessary.  Freenet enthusiasts may be
| ok with this, but casual users probably won't be, and we *need* casual
| users.
|
| Personally I'm pretty skeptical of anything requiring more than 100MB.

My store had filled to 100% in less than 72 hours (that would be
slightly less than 1 Gb of data), so I doubt that it really stores
anything past the last week at best. And one week isn't exactly what we
should aim for when talking about data retention, even for unpopular data.

As a side note, I *really* miss a stat for the LRU timestamp for the
store (or at least I wasn't able to find it) - so I coule be wrong with
my one week estimate. It would be really interesting to see for how
long does your store actually store things.

Regards,
Victor Denisov.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIKeMhS81Mh9/iCDgRAkD8AJ0WoWa2n7W5t/W6CAgtFKEOugdNzACgvr2d
ybupUuIQ/AcE7mCFQh0wQkY=
=BGZ/
-END PGP SIGNATURE-
___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Post-0.7.0 priorities

[Victor Denisov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

| I disagree. Your actual bandwidth usage is determined by how many
requests the
| other nodes send you. This is largely determined by the *average
bandwidth
| limit* across the whole network. If we increase the average bandwidth
limit,
| we increase the traffic on your node.

Ah, ok, that's probably true. I was thinking you've been talking about
that helping *my* node to better utilize the bandwidth available, hence
my reply.

| Great that we agree on this one. I've been unsuccessful in bringing at
| least two of my friends to Freenet because they were running into
| memory-related problems, one of them going as far as calling Freenet
| that damn bloatware (well, actual wording also included a couple of
| pretty strong Russian expletives :-).
|
| Hehe. Consensus is good. They are specifically talking about
memory/CPU here?
| Or bandwidth usage, total transfer in a month, etc etc?

Well, memory usage was what finally turned them off, as far as I can
see. One of the people I've mentioned (a pretty experienced computer and
p2p user) had immediately and repeatedly crashed his node upon
discovering Thaw and Frost file sharing tools - he put a couple of
hundreds of large files there for both insertion and download, not
really understanding how Freenet handles this (on a side note, this is
something which is *really* unclear to most casual P2P users I've talked
to, especially those with Gnutella/eMule/Kazaa background). Naturally,
Freenet quickly ran out of memory, and simply wasn't recovering on its
own. We've finally been able to solve this problem by giving Java enough
memory to load the queue, then quickly removing files from it (there's
probably a better way, but I wasn't aware of it).

In the second case, the machine on which we tried to install Freenet was
a little bit old (but nothing ancient - P4 2.4 GHz/1 Gb RAM), used as a
dedicated P2P machine by my other friend. Shortly after installing
Freenet, it became unresponsive - almost constant disk access, high CPU
usage, etc. After stopping Freenet, the load went down immediately. We
tried actually giving Freenet less than default memory (96 Mb, IIRC),
but it didn't really help.

| Well, it seems more or less straightforward from the outside: handle
| additional update URLs and a set of (revocable?) public keys + expose
| the API over FCP. Am I missing something important?
|
| Yeah but we'd have to unpack, support post-unpack scripts, etc etc ...
really
| we'd want the apps to provide their own unpacker and just feed them a
single
| file for them to do what they want with?

I think that the latter approach is preferable - they could be using
full-blown native installers, etc. No need for Freenet to open that can
of worms. Just allow the app to check for updates over FCP and feed it
the (verified) update file when one becomes available. What to do with
it is up to the application.

Regards,
Victor Denisov.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIKekCS81Mh9/iCDgRAirPAKDMp5pzomB5CCZgDSJmS9iHv0I0RgCg02+v
eEJQYj+Qj0VTyGW2I1uXJDE=
=6bPm
-END PGP SIGNATURE-
___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Post-0.7.0 priorities

[Victor Denisov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

| On Friday 09 May 2008 07:27, Victor Denisov wrote:
| | Automatic bandwidth calibration. Other p2p apps have this, we should
| have it.
|
| Good idea. Also, we should definitely look into better utilizing
| available bandwidth. Freenet's the only p2p app which consistently
| underutilizes my upload limit (~ 2 Mbit/s out of 8 Mbit/s of link
| capacity). I understand that we don't want to create supernodes, but
| come on, 2 Mbit/s is *nothing* these days.
|
| IMHO automatic bandwidth calibration will help a lot with this. Beyond
that
| we're looking at token passing, which may be too big for 0.7.1.

Excerpt from my current node stats:

networkSizeEstimateSession: 6039 nodes
nodeUptime: 2d1h
pInstantReject: 0,0%
uptimeAverage: 100,0%
Peer statistics
~* Connected: 17
~* Backed off: 3
~* Seeding for: 111
Input Rate: 17.6 KiB/sec (of 300 KiB)
Output Rate: 15.9 KiB/sec (of 200 KiB)
Total Input: 4.83 GiB (28.3 KiB/sec)
Total Output: 5.66 GiB (33.2 KiB/sec)

Used Java memory: 122 MiB
Allocated Java memory: 127 MiB
Maximum Java memory: 284 MiB
Running threads: 152/700

So, basically, network had grown about 3x after 0.7 release. My node has
been up for 2 days, and is pretty well established in the network. It's
not overloaded (CPU usage is ~ 5-10%). Yet it doesn't use more than
about 15% of the allowed bandwidth, on average. Automatic bandwidth
calibration won't help - I've allowed Freenet to use much less that my
uplink allows to.

It could be related to the fact that I've only been able to dedicate
about 2 Gb for my store, but I doubt it.

| Agreed, memory usage is a usability issue: the user shouldn't have to
care
| about it.

Great that we agree on this one. I've been unsuccessful in bringing at
least two of my friends to Freenet because they were running into
memory-related problems, one of them going as far as calling Freenet
that damn bloatware (well, actual wording also included a couple of
pretty strong Russian expletives :-).

| Shouldn't we consider auto-updating bundled applications as well? Or
| perhaps providing an auto-update API for use by third-party apps? Just a
| thought.
|
| Maybe, that would be harder though. I would be happy to discuss it
with their
| authors.

Well, it seems more or less straightforward from the outside: handle
additional update URLs and a set of (revocable?) public keys + expose
the API over FCP. Am I missing something important?

Regards,
Victor Denisov.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIKJhIS81Mh9/iCDgRAkPFAJ96CPlcNouiHiVjavq/xtY6y8XR6QCglIpB
IwLrElLZxQZyo9WKTTWdbyo=
=FMsn
-END PGP SIGNATURE-
___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Post-0.7.0 priorities

[Michael Rogers]

Victor Denisov wrote:
 Input Rate: 17.6 KiB/sec (of 300 KiB)
 Output Rate: 15.9 KiB/sec (of 200 KiB)
 Total Input: 4.83 GiB (28.3 KiB/sec)
 Total Output: 5.66 GiB (33.2 KiB/sec)
 
 Used Java memory: 122 MiB
 Allocated Java memory: 127 MiB
 Maximum Java memory: 284 MiB
 Running threads: 152/700
 
 So, basically, network had grown about 3x after 0.7 release. My node has
 been up for 2 days, and is pretty well established in the network. It's
 not overloaded (CPU usage is ~ 5-10%). Yet it doesn't use more than
 about 15% of the allowed bandwidth, on average.

If the average speed of a node is, say, 20 KB/s then your node's 
unlikely to be able to use 300 KB/s because its peers won't be 
sending/accepting enough traffic (due to their own bandwidth limits, not 
yours). This could be solved by allowing fast nodes to have more peers, 
but of course that creates the possibility of ubernodes.

Cheers,
Michael
___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Post-0.7.0 priorities

[Matthew Toseland]

On Thursday 08 May 2008 23:22, Matthew Toseland wrote:
> I've made a bug on the bug tracker to which I've linked all the things that 
I 
> think *might* be important for 0.7.1. Please contribute to this bug by 
> setting it related to anything that you think it should be related to, or 
> reply to this thread.
> 
> Stuff I think is important for the next release:
> 
> Automatic bandwidth calibration. Other p2p apps have this, we should have 
it. 
> It should significantly improve the average output bandwidth available, 
since 
> most users presumably don't set the output limit. Further it would improve 
> usability by making the connection more responsive.
> Timescale: Unclear at this point.
> Priority: High.
> 
> Datastore changes: It looks very much like we can have both better network 
> performance and much less memory overhead by using a non-associative salted 
> hash table on disk. Daniel Cheng (sdiz) is currently implementing this on a 
> branch. I will be reviewing the code soon. Nextgens has asked about its 
> suitability for the cache as opposed to the store; mrogers' simulations were 
> based on it being for the store.
> Timescale: A lot of this is done already, most work will be sdiz's.
> Priority: High.
> 
> More work on ULPRs: Various changes related to coalescing and temporarily 
> suppressing requests if there are too many for a single key over a period. 
> Should help with FMS and to boost payload %.
> Timescale: 1 week
> Priority: High.
> 
> Use peers' peers' locations for routing: According to oskar and vive, this 
> should significantly cut average path lengths. There is minimal security 
> impact as this data is already exposed by swapping.
> Timescale: 1 week
> Priority: High.
> 
> Faster swapping: Vive has some ideas to improve swapping performance 
> significantly. Hopefully he can turn these into implementible proposals. 
This 
> would significantly improve performance (especially given the level of churn 
> we see), and also help with security (because we could reset more to prevent 
> swapping attacks).
> Timescale: Depends on Vive. Implementation probably relatively easy.
> Priority: High if possible. Would justify calling it 0.8.0 IMHO.
> 
> Streams and MTU: We can improve our payload proportion significantly, allow 
> for much smaller packets, support smaller MTUs, avoid padding with random 
> data, and support simple stego, by implementing transport layer streams. 
> There are also a number of other mostly minor changes which we should 
> implement to make Freenet work well on low MTU connections.
> Timescale: 2-4 weeks??
> Priority: High.
> 
> Better Librarian integration: We should have a search box on the homepage, 
it 
> should be embeddable in freesites. And XMLSpider probably needs some more 
> debugging.
> Timescale: 1 week or less.
> Priority: High.
> 
> Client layer changes: I propose to move the entire client layer onto disk. 
We 
> would continue to store enough data to restart requests from scratch in 
> downloads.dat.gz, but we would have an on-disk queue structure instead of an 
> in-memory queue structure. This combined with the datastore changes would 
> make our memory usage fixed, regardless of the size of your store or the 
> number of requests you queue. It would solve many bug reports, it would 
> incorporate the long-awaited true request resuming, would make Freenet run 
> well on home servers with 128M or maybe even 64M of RAM, and generally is a 
> good idea.
> Timescale: 2 weeks??
> Priority: Medium.
> 
> Auto-update for plugins: We should have had this ages ago. Several people 
have 
> been complaining about it, it is a security issue if you want them kept up 
to 
> date. It shouldn't be difficult.
> Timescale: 3 days
> Priority: Medium.
> 
> Better content filter: Filter on insert by default (for performance on 
fetch), 
> support more HTML etc, support RSS, support some other formats (e.g. mp3), 
> etc.
> Timescale: 1 week, more for more formats.
> Priority: Medium.
> 
> Better USKs: Hierarchical DBRs would help USK lookups to find something 
close 
> to the latest version quickly, then plod through the editions from there to 
> find the latest version.
> Timescale: 2 days.
> Priority: Medium.
> 
> System tray icon: IMHO this would be a good usability feature. It would make 
> it obvious that Freenet is running in the background and make it easy to 
> throttle it or disable it when you want to play an MMORPG etc.
> Timescale: 1 week???
> Priority: Medium.
> 
> Bandwidth: mister_wavey is working on a bandwidth scheduler. It would be a 
big 
> help for a lot of users who have off-peak quotas etc. There will be a user 
> friendly config sub-page for it. Average bandwidth limits, maybe actually 
> telling the node a monthly quota, would also be useful for many users. More 
> accurate bandwidth limiting (limiting all packets not just transfers) would 
> also be a big help for users on slower upstream connections.
> Timescale: Unknown.
> Priority: 

[freenet-dev] Post-0.7.0 priorities

[Matthew Toseland]

On Friday 09 May 2008 07:27, Victor Denisov wrote:
> | Automatic bandwidth calibration. Other p2p apps have this, we should
> have it.
> 
> Good idea. Also, we should definitely look into better utilizing
> available bandwidth. Freenet's the only p2p app which consistently
> underutilizes my upload limit (~ 2 Mbit/s out of 8 Mbit/s of link
> capacity). I understand that we don't want to create supernodes, but
> come on, 2 Mbit/s is *nothing* these days.

IMHO automatic bandwidth calibration will help a lot with this. Beyond that 
we're looking at token passing, which may be too big for 0.7.1.
> 
> | Client layer changes: I propose to move the entire client layer onto
> disk. We
> 
> I say this deserves to be moved to High or Very High priority. The main
> problem is not memory usage as is (most people have 1 Gb+ of RAM now),
> but rather inability of Java to properly grow and shrink its memory
> usage on the "as needed" basis. Not a single native Windows application
> behaves this way, and I doubt many users are prepared to understand and
> adjust memory limits manually. Heck, even I, being a Java developer,
> spent several days trying to understand what heap limit to set for
> Freenet so that it won't run out of memory (and that it uses ~ 2x memory
> on 64-bit JVM doesn't help any). Also, perhaps we can detect OOM errors
> and offer the user to increase the memory limit next time he tries to
> run Freenet?

Agreed, memory usage is a usability issue: the user shouldn't have to care 
about it.
> 
> | Auto-update for plugins: We should have had this ages ago. Several
> people have
> 
> Shouldn't we consider auto-updating bundled applications as well? Or
> perhaps providing an auto-update API for use by third-party apps? Just a
> thought.

Maybe, that would be harder though. I would be happy to discuss it with their 
authors.
> 
> Hope the above was helpful,
> 
> Regards,
> Victor Denisov.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 

[freenet-dev] Post-0.7.0 priorities

[Victor Denisov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

| Automatic bandwidth calibration. Other p2p apps have this, we should
have it.

Good idea. Also, we should definitely look into better utilizing
available bandwidth. Freenet's the only p2p app which consistently
underutilizes my upload limit (~ 2 Mbit/s out of 8 Mbit/s of link
capacity). I understand that we don't want to create supernodes, but
come on, 2 Mbit/s is *nothing* these days.

| Client layer changes: I propose to move the entire client layer onto
disk. We

I say this deserves to be moved to High or Very High priority. The main
problem is not memory usage as is (most people have 1 Gb+ of RAM now),
but rather inability of Java to properly grow and shrink its memory
usage on the "as needed" basis. Not a single native Windows application
behaves this way, and I doubt many users are prepared to understand and
adjust memory limits manually. Heck, even I, being a Java developer,
spent several days trying to understand what heap limit to set for
Freenet so that it won't run out of memory (and that it uses ~ 2x memory
on 64-bit JVM doesn't help any). Also, perhaps we can detect OOM errors
and offer the user to increase the memory limit next time he tries to
run Freenet?

| Auto-update for plugins: We should have had this ages ago. Several
people have

Shouldn't we consider auto-updating bundled applications as well? Or
perhaps providing an auto-update API for use by third-party apps? Just a
thought.

Hope the above was helpful,

Regards,
Victor Denisov.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFII+7pS81Mh9/iCDgRAhN7AJ93mwVa2Ogqo+6JLiY2322AgPm2WwCgwYmw
DQ2jkkYDeZ07pXaoE/qH/Uo=
=WSBl
-END PGP SIGNATURE-

[freenet-dev] Post-0.7.0 priorities

[Matthew Toseland]

I've made a bug on the bug tracker to which I've linked all the things that I 
think *might* be important for 0.7.1. Please contribute to this bug by 
setting it related to anything that you think it should be related to, or 
reply to this thread.

Stuff I think is important for the next release:

Automatic bandwidth calibration. Other p2p apps have this, we should have it. 
It should significantly improve the average output bandwidth available, since 
most users presumably don't set the output limit. Further it would improve 
usability by making the connection more responsive.
Timescale: Unclear at this point.
Priority: High.

Datastore changes: It looks very much like we can have both better network 
performance and much less memory overhead by using a non-associative salted 
hash table on disk. Daniel Cheng (sdiz) is currently implementing this on a 
branch. I will be reviewing the code soon. Nextgens has asked about its 
suitability for the cache as opposed to the store; mrogers' simulations were 
based on it being for the store.
Timescale: A lot of this is done already, most work will be sdiz's.
Priority: High.

More work on ULPRs: Various changes related to coalescing and temporarily 
suppressing requests if there are too many for a single key over a period. 
Should help with FMS and to boost payload %.
Timescale: 1 week
Priority: High.

Use peers' peers' locations for routing: According to oskar and vive, this 
should significantly cut average path lengths. There is minimal security 
impact as this data is already exposed by swapping.
Timescale: 1 week
Priority: High.

Faster swapping: Vive has some ideas to improve swapping performance 
significantly. Hopefully he can turn these into implementible proposals. This 
would significantly improve performance (especially given the level of churn 
we see), and also help with security (because we could reset more to prevent 
swapping attacks).
Timescale: Depends on Vive. Implementation probably relatively easy.
Priority: High if possible. Would justify calling it 0.8.0 IMHO.

Streams and MTU: We can improve our payload proportion significantly, allow 
for much smaller packets, support smaller MTUs, avoid padding with random 
data, and support simple stego, by implementing transport layer streams. 
There are also a number of other mostly minor changes which we should 
implement to make Freenet work well on low MTU connections.
Timescale: 2-4 weeks??
Priority: High.

Better Librarian integration: We should have a search box on the homepage, it 
should be embeddable in freesites. And XMLSpider probably needs some more 
debugging.
Timescale: 1 week or less.
Priority: High.

Client layer changes: I propose to move the entire client layer onto disk. We 
would continue to store enough data to restart requests from scratch in 
downloads.dat.gz, but we would have an on-disk queue structure instead of an 
in-memory queue structure. This combined with the datastore changes would 
make our memory usage fixed, regardless of the size of your store or the 
number of requests you queue. It would solve many bug reports, it would 
incorporate the long-awaited true request resuming, would make Freenet run 
well on home servers with 128M or maybe even 64M of RAM, and generally is a 
good idea.
Timescale: 2 weeks??
Priority: Medium.

Auto-update for plugins: We should have had this ages ago. Several people have 
been complaining about it, it is a security issue if you want them kept up to 
date. It shouldn't be difficult.
Timescale: 3 days
Priority: Medium.

Better content filter: Filter on insert by default (for performance on fetch), 
support more HTML etc, support RSS, support some other formats (e.g. mp3), 
etc.
Timescale: 1 week, more for more formats.
Priority: Medium.

Better USKs: Hierarchical DBRs would help USK lookups to find something close 
to the latest version quickly, then plod through the editions from there to 
find the latest version.
Timescale: 2 days.
Priority: Medium.

System tray icon: IMHO this would be a good usability feature. It would make 
it obvious that Freenet is running in the background and make it easy to 
throttle it or disable it when you want to play an MMORPG etc.
Timescale: 1 week???
Priority: Medium.

Bandwidth: mister_wavey is working on a bandwidth scheduler. It would be a big 
help for a lot of users who have off-peak quotas etc. There will be a user 
friendly config sub-page for it. Average bandwidth limits, maybe actually 
telling the node a monthly quota, would also be useful for many users. More 
accurate bandwidth limiting (limiting all packets not just transfers) would 
also be a big help for users on slower upstream connections.
Timescale: Unknown.
Priority: Medium.

Allocate temporary files out of a small number of blob files: Further 
reduction in memory usage, some other benefits, cuts the number of fd's we 
use (thus allowing more FEC threads on mutli-core systems), easy.
Timescale: 3 days
Priority: Low.
-- 

Re: [freenet-dev] Post-0.7.0 priorities

[Victor Denisov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

| Automatic bandwidth calibration. Other p2p apps have this, we should
have it.

Good idea. Also, we should definitely look into better utilizing
available bandwidth. Freenet's the only p2p app which consistently
underutilizes my upload limit (~ 2 Mbit/s out of 8 Mbit/s of link
capacity). I understand that we don't want to create supernodes, but
come on, 2 Mbit/s is *nothing* these days.

| Client layer changes: I propose to move the entire client layer onto
disk. We

I say this deserves to be moved to High or Very High priority. The main
problem is not memory usage as is (most people have 1 Gb+ of RAM now),
but rather inability of Java to properly grow and shrink its memory
usage on the as needed basis. Not a single native Windows application
behaves this way, and I doubt many users are prepared to understand and
adjust memory limits manually. Heck, even I, being a Java developer,
spent several days trying to understand what heap limit to set for
Freenet so that it won't run out of memory (and that it uses ~ 2x memory
on 64-bit JVM doesn't help any). Also, perhaps we can detect OOM errors
and offer the user to increase the memory limit next time he tries to
run Freenet?

| Auto-update for plugins: We should have had this ages ago. Several
people have

Shouldn't we consider auto-updating bundled applications as well? Or
perhaps providing an auto-update API for use by third-party apps? Just a
thought.

Hope the above was helpful,

Regards,
Victor Denisov.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFII+7pS81Mh9/iCDgRAhN7AJ93mwVa2Ogqo+6JLiY2322AgPm2WwCgwYmw
DQ2jkkYDeZ07pXaoE/qH/Uo=
=WSBl
-END PGP SIGNATURE-
___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Post-0.7.0 priorities

[Matthew Toseland]

On Friday 09 May 2008 07:27, Victor Denisov wrote:
 | Automatic bandwidth calibration. Other p2p apps have this, we should
 have it.
 
 Good idea. Also, we should definitely look into better utilizing
 available bandwidth. Freenet's the only p2p app which consistently
 underutilizes my upload limit (~ 2 Mbit/s out of 8 Mbit/s of link
 capacity). I understand that we don't want to create supernodes, but
 come on, 2 Mbit/s is *nothing* these days.

IMHO automatic bandwidth calibration will help a lot with this. Beyond that 
we're looking at token passing, which may be too big for 0.7.1.
 
 | Client layer changes: I propose to move the entire client layer onto
 disk. We
 
 I say this deserves to be moved to High or Very High priority. The main
 problem is not memory usage as is (most people have 1 Gb+ of RAM now),
 but rather inability of Java to properly grow and shrink its memory
 usage on the as needed basis. Not a single native Windows application
 behaves this way, and I doubt many users are prepared to understand and
 adjust memory limits manually. Heck, even I, being a Java developer,
 spent several days trying to understand what heap limit to set for
 Freenet so that it won't run out of memory (and that it uses ~ 2x memory
 on 64-bit JVM doesn't help any). Also, perhaps we can detect OOM errors
 and offer the user to increase the memory limit next time he tries to
 run Freenet?

Agreed, memory usage is a usability issue: the user shouldn't have to care 
about it.
 
 | Auto-update for plugins: We should have had this ages ago. Several
 people have
 
 Shouldn't we consider auto-updating bundled applications as well? Or
 perhaps providing an auto-update API for use by third-party apps? Just a
 thought.

Maybe, that would be harder though. I would be happy to discuss it with their 
authors.
 
 Hope the above was helpful,
 
 Regards,
 Victor Denisov.


pgpcz1OB58GLH.pgp
Description: PGP signature
___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Post-0.7.0 priorities

[Matthew Toseland]

On Thursday 08 May 2008 23:22, Matthew Toseland wrote:
 I've made a bug on the bug tracker to which I've linked all the things that 
I 
 think *might* be important for 0.7.1. Please contribute to this bug by 
 setting it related to anything that you think it should be related to, or 
 reply to this thread.
 
 Stuff I think is important for the next release:
 
 Automatic bandwidth calibration. Other p2p apps have this, we should have 
it. 
 It should significantly improve the average output bandwidth available, 
since 
 most users presumably don't set the output limit. Further it would improve 
 usability by making the connection more responsive.
 Timescale: Unclear at this point.
 Priority: High.
 
 Datastore changes: It looks very much like we can have both better network 
 performance and much less memory overhead by using a non-associative salted 
 hash table on disk. Daniel Cheng (sdiz) is currently implementing this on a 
 branch. I will be reviewing the code soon. Nextgens has asked about its 
 suitability for the cache as opposed to the store; mrogers' simulations were 
 based on it being for the store.
 Timescale: A lot of this is done already, most work will be sdiz's.
 Priority: High.
 
 More work on ULPRs: Various changes related to coalescing and temporarily 
 suppressing requests if there are too many for a single key over a period. 
 Should help with FMS and to boost payload %.
 Timescale: 1 week
 Priority: High.
 
 Use peers' peers' locations for routing: According to oskar and vive, this 
 should significantly cut average path lengths. There is minimal security 
 impact as this data is already exposed by swapping.
 Timescale: 1 week
 Priority: High.
 
 Faster swapping: Vive has some ideas to improve swapping performance 
 significantly. Hopefully he can turn these into implementible proposals. 
This 
 would significantly improve performance (especially given the level of churn 
 we see), and also help with security (because we could reset more to prevent 
 swapping attacks).
 Timescale: Depends on Vive. Implementation probably relatively easy.
 Priority: High if possible. Would justify calling it 0.8.0 IMHO.
 
 Streams and MTU: We can improve our payload proportion significantly, allow 
 for much smaller packets, support smaller MTUs, avoid padding with random 
 data, and support simple stego, by implementing transport layer streams. 
 There are also a number of other mostly minor changes which we should 
 implement to make Freenet work well on low MTU connections.
 Timescale: 2-4 weeks??
 Priority: High.
 
 Better Librarian integration: We should have a search box on the homepage, 
it 
 should be embeddable in freesites. And XMLSpider probably needs some more 
 debugging.
 Timescale: 1 week or less.
 Priority: High.
 
 Client layer changes: I propose to move the entire client layer onto disk. 
We 
 would continue to store enough data to restart requests from scratch in 
 downloads.dat.gz, but we would have an on-disk queue structure instead of an 
 in-memory queue structure. This combined with the datastore changes would 
 make our memory usage fixed, regardless of the size of your store or the 
 number of requests you queue. It would solve many bug reports, it would 
 incorporate the long-awaited true request resuming, would make Freenet run 
 well on home servers with 128M or maybe even 64M of RAM, and generally is a 
 good idea.
 Timescale: 2 weeks??
 Priority: Medium.
 
 Auto-update for plugins: We should have had this ages ago. Several people 
have 
 been complaining about it, it is a security issue if you want them kept up 
to 
 date. It shouldn't be difficult.
 Timescale: 3 days
 Priority: Medium.
 
 Better content filter: Filter on insert by default (for performance on 
fetch), 
 support more HTML etc, support RSS, support some other formats (e.g. mp3), 
 etc.
 Timescale: 1 week, more for more formats.
 Priority: Medium.
 
 Better USKs: Hierarchical DBRs would help USK lookups to find something 
close 
 to the latest version quickly, then plod through the editions from there to 
 find the latest version.
 Timescale: 2 days.
 Priority: Medium.
 
 System tray icon: IMHO this would be a good usability feature. It would make 
 it obvious that Freenet is running in the background and make it easy to 
 throttle it or disable it when you want to play an MMORPG etc.
 Timescale: 1 week???
 Priority: Medium.
 
 Bandwidth: mister_wavey is working on a bandwidth scheduler. It would be a 
big 
 help for a lot of users who have off-peak quotas etc. There will be a user 
 friendly config sub-page for it. Average bandwidth limits, maybe actually 
 telling the node a monthly quota, would also be useful for many users. More 
 accurate bandwidth limiting (limiting all packets not just transfers) would 
 also be a big help for users on slower upstream connections.
 Timescale: Unknown.
 Priority: Medium.
 
 Allocate temporary files out of a small number of blob files: Further 
 reduction in memory 

[freenet-dev] Post-0.7.0 priorities

[Matthew Toseland]

I've made a bug on the bug tracker to which I've linked all the things that I 
think *might* be important for 0.7.1. Please contribute to this bug by 
setting it related to anything that you think it should be related to, or 
reply to this thread.

Stuff I think is important for the next release:

Automatic bandwidth calibration. Other p2p apps have this, we should have it. 
It should significantly improve the average output bandwidth available, since 
most users presumably don't set the output limit. Further it would improve 
usability by making the connection more responsive.
Timescale: Unclear at this point.
Priority: High.

Datastore changes: It looks very much like we can have both better network 
performance and much less memory overhead by using a non-associative salted 
hash table on disk. Daniel Cheng (sdiz) is currently implementing this on a 
branch. I will be reviewing the code soon. Nextgens has asked about its 
suitability for the cache as opposed to the store; mrogers' simulations were 
based on it being for the store.
Timescale: A lot of this is done already, most work will be sdiz's.
Priority: High.

More work on ULPRs: Various changes related to coalescing and temporarily 
suppressing requests if there are too many for a single key over a period. 
Should help with FMS and to boost payload %.
Timescale: 1 week
Priority: High.

Use peers' peers' locations for routing: According to oskar and vive, this 
should significantly cut average path lengths. There is minimal security 
impact as this data is already exposed by swapping.
Timescale: 1 week
Priority: High.

Faster swapping: Vive has some ideas to improve swapping performance 
significantly. Hopefully he can turn these into implementible proposals. This 
would significantly improve performance (especially given the level of churn 
we see), and also help with security (because we could reset more to prevent 
swapping attacks).
Timescale: Depends on Vive. Implementation probably relatively easy.
Priority: High if possible. Would justify calling it 0.8.0 IMHO.

Streams and MTU: We can improve our payload proportion significantly, allow 
for much smaller packets, support smaller MTUs, avoid padding with random 
data, and support simple stego, by implementing transport layer streams. 
There are also a number of other mostly minor changes which we should 
implement to make Freenet work well on low MTU connections.
Timescale: 2-4 weeks??
Priority: High.

Better Librarian integration: We should have a search box on the homepage, it 
should be embeddable in freesites. And XMLSpider probably needs some more 
debugging.
Timescale: 1 week or less.
Priority: High.

Client layer changes: I propose to move the entire client layer onto disk. We 
would continue to store enough data to restart requests from scratch in 
downloads.dat.gz, but we would have an on-disk queue structure instead of an 
in-memory queue structure. This combined with the datastore changes would 
make our memory usage fixed, regardless of the size of your store or the 
number of requests you queue. It would solve many bug reports, it would 
incorporate the long-awaited true request resuming, would make Freenet run 
well on home servers with 128M or maybe even 64M of RAM, and generally is a 
good idea.
Timescale: 2 weeks??
Priority: Medium.

Auto-update for plugins: We should have had this ages ago. Several people have 
been complaining about it, it is a security issue if you want them kept up to 
date. It shouldn't be difficult.
Timescale: 3 days
Priority: Medium.

Better content filter: Filter on insert by default (for performance on fetch), 
support more HTML etc, support RSS, support some other formats (e.g. mp3), 
etc.
Timescale: 1 week, more for more formats.
Priority: Medium.

Better USKs: Hierarchical DBRs would help USK lookups to find something close 
to the latest version quickly, then plod through the editions from there to 
find the latest version.
Timescale: 2 days.
Priority: Medium.

System tray icon: IMHO this would be a good usability feature. It would make 
it obvious that Freenet is running in the background and make it easy to 
throttle it or disable it when you want to play an MMORPG etc.
Timescale: 1 week???
Priority: Medium.

Bandwidth: mister_wavey is working on a bandwidth scheduler. It would be a big 
help for a lot of users who have off-peak quotas etc. There will be a user 
friendly config sub-page for it. Average bandwidth limits, maybe actually 
telling the node a monthly quota, would also be useful for many users. More 
accurate bandwidth limiting (limiting all packets not just transfers) would 
also be a big help for users on slower upstream connections.
Timescale: Unknown.
Priority: Medium.

Allocate temporary files out of a small number of blob files: Further 
reduction in memory usage, some other benefits, cuts the number of fd's we 
use (thus allowing more FEC threads on mutli-core systems), easy.
Timescale: 3 days
Priority: Low.