Re: [freenet-chat] FCPtools now with AutoSplit support
From: "Ian Clarke" <[EMAIL PROTECTED]> > Any progress on the new FreeWeb built around "fcpputsite"? That would be a > real boost to the number of Freesites on 0.4. That's less of a feature issue and more of an internal maintenance issue. I say this because FreeWeb automatically inherits the new auto-split functionality, since it's using the 'fcpPutKeyFromFile()' primitive (see my previous email). So FreeWebbers can just dive in now and insert their huge sites, safe in the knowledge that big files will automatically split up. Yes - it'll be more ideal for FreeWeb to be using fcpputsite, but I say again, it's an internal maintenance issue. FreeWeb's site management has always been vastly superior to that of fcpputsite. If anything, the real requirement is adding FreeWeb's (non-gui) functionality to fcpputsite. David ___ Chat mailing list [EMAIL PROTECTED] http://lists.freenetproject.org/mailman/listinfo/chat
Re: [freenet-chat] FCPtools now with AutoSplit support
> > I've checked into CVS a new version of the fcptools (freenet CVS, in > Contrib/fcptools), which now supports auto-splitting of all inserted > files. > Cool! It is all coming together. I have just updated the Linux snapshot builder to check out FCPtools from its new location. Any progress on the new FreeWeb built around "fcpputsite"? That would be a real boost to the number of Freesites on 0.4. Question: Since all messages are quantized to the next largest power of 2, is the default splitsize of 262144 optimal? We need to think about this. Ian. ___ Chat mailing list [EMAIL PROTECTED] http://lists.freenetproject.org/mailman/listinfo/chat
Re: [freenet-chat] FCPtools now with AutoSplit support
Oh, I forgot to mention - once all this autosplit
stuff fully stabilises, fcpputsplit will be deleted, since it will no longer
been needed.
- Original Message -
From:
David
McNab
To: Freenet
Chat ; Freenet Development
Sent: Friday, October 05, 2001 1:11
AM
Subject: [freenet-chat] FCPtools now with
AutoSplit support
Hi all,
I've checked into CVS a new version of the
fcptools (freenet CVS, in Contrib/fcptools), which now supports auto-splitting
of all inserted files.
Since auto-split is implemented completely in the
ezFCPlib library layer, all client software which uses fcpPutKeyFromFile()
will inherit the autosplit functionality with no changes needed (except a new
parameter to fcpStartup()).
Summary of changes
1) fcpStartup() now launches a 'splitfile
insert manager' thread, which manages and dispatches all splitfile insertions.
This manager thread maintains a queue of all file insert jobs, and spawns
further insert threads as needed.
2) All files are split if they exceed the
default chunksize (256k). This can not be overridden.
3) Default thread limit for chunk insertions
is 8 for the whole process
4) New command line arguments recognised by
fcpput and fcpputsite:
-ss size of
splitfile chunks in bytes ('k','m','g' at end recognised, eg
'256K')
-st maximum
number of splitfile threads, default 8
5) Modules fcpSetHost.c and fcpSetHtl.c
removed, consolidated into fcpSetParam.c
6) Splitting is *not* done with
fcpPutKeyFromMem()
7) Splitting is *not* done from streamed key
insert (fcpOpenKey()/fcpWriteKey()/fcpCloseKey())
8) Splitting *is* done when key is
inserted via fcpPutKeyFromFile()
9) The global variable 'int fcpSplitChunkSize',
if set before calling fcpStartup(), governs the splitfile chunk size (default
256k).
10) fcpStartup() parameters change:
int fcpStartup(char *host, int port, int
defaultHtl, int raw, int maxSplitThreads)
11) Obviously, fcpput and fcpputsite now insert
splitfiles, automatically and transparently. There is no way to override
filesplitting for large files.
12) Lots of debug messages available if tools are
run with '-v 4'.
Status - I've spent the last day and a half
weeding out bugs and stress-testing. The whole thing seems to be holding up
well on Linux and Windows, even with big jobs.
Bugs?
Probably infinite. If you find bugs, please find
the simplest possible way to consistently reproduce them, run the
applicable tool at full verbosity ('-v 4'), capture the output to a file, and
send the file to me, along with your OS information etc. Feel free to 'censor'
out any SSK private keys if necessary.
Cheers
David
[freenet-chat] FCPtools now with AutoSplit support
Hi all,
I've checked into CVS a new version of the fcptools
(freenet CVS, in Contrib/fcptools), which now supports auto-splitting of all
inserted files.
Since auto-split is implemented completely in the
ezFCPlib library layer, all client software which uses fcpPutKeyFromFile() will
inherit the autosplit functionality with no changes needed (except a new
parameter to fcpStartup()).
Summary of changes
1) fcpStartup() now launches a 'splitfile
insert manager' thread, which manages and dispatches all splitfile insertions.
This manager thread maintains a queue of all file insert jobs, and spawns
further insert threads as needed.
2) All files are split if they exceed the
default chunksize (256k). This can not be overridden.
3) Default thread limit for chunk insertions
is 8 for the whole process
4) New command line arguments recognised by
fcpput and fcpputsite:
-ss size of
splitfile chunks in bytes ('k','m','g' at end recognised, eg
'256K')
-st maximum
number of splitfile threads, default 8
5) Modules fcpSetHost.c and fcpSetHtl.c
removed, consolidated into fcpSetParam.c
6) Splitting is *not* done with
fcpPutKeyFromMem()
7) Splitting is *not* done from streamed key
insert (fcpOpenKey()/fcpWriteKey()/fcpCloseKey())
8) Splitting *is* done when key is
inserted via fcpPutKeyFromFile()
9) The global variable 'int fcpSplitChunkSize', if
set before calling fcpStartup(), governs the splitfile chunk size (default
256k).
10) fcpStartup() parameters change:
int fcpStartup(char *host, int port, int
defaultHtl, int raw, int maxSplitThreads)
11) Obviously, fcpput and fcpputsite now insert
splitfiles, automatically and transparently. There is no way to override
filesplitting for large files.
12) Lots of debug messages available if tools are
run with '-v 4'.
Status - I've spent the last day and a half weeding
out bugs and stress-testing. The whole thing seems to be holding up well on
Linux and Windows, even with big jobs.
Bugs?
Probably infinite. If you find bugs, please find
the simplest possible way to consistently reproduce them, run the
applicable tool at full verbosity ('-v 4'), capture the output to a file, and
send the file to me, along with your OS information etc. Feel free to 'censor'
out any SSK private keys if necessary.
Cheers
David
