Send Devl mailing list submissions to
devl at freenetproject.org
To subscribe or unsubscribe via the World Wide Web, visit
http://www.uprizer.com/mailman/listinfo/devl
or, via email, send a message with subject or body 'help' to
devl-request at freenetproject.org
You can reach the person managing the list at
devl-admin at freenetproject.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Devl digest..."
Today's Topics:
1. Re: IMPORTANT: Where is the Freenet Windows release? (Sebastian Spaeth)
2. Re: Compression of Jars (Mr.Bad)
3. Re: Aardvark (Chris Anderson)
4. Re: Compression of Jars (Theodore Hong)
5. Re: Aardvark (Scott G. Miller)
6. Re: Compression of Jars (Scott G. Miller)
7. Re: Espra - what's that? (Adam Langley)
8. progress on 0.4 data store rewrite (Tavin Cole)
9. Re: Compression of Jars (Steven Hazel)
--__--__--
Message: 1
Date: Tue, 06 Feb 2001 08:06:32 +0100
From: Sebastian Spaeth <[email protected]>
Organization: University of =?iso-8859-1?Q?Link=F6ping?=
To: devl at freenetproject.org
Subject: Re: [freenet-devl] IMPORTANT: Where is the Freenet Windows release?
Reply-To: devl at freenetproject.org
Ian Clarke wrote:
> The Freenet Windows release has gone missing (it was there this-morning)
> - can whoever has a copy please replace it?!
Done
--__--__--
Message: 2
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Compression of Jars
From: Mr.Bad <[email protected]>
Organization: Pigdog Journal
Date: 05 Feb 2001 23:12:55 -0800
Reply-To: devl at freenetproject.org
>>>>> "DM" == Don Marti <dmarti at zgp.org> writes:
DM> My guess is that it uses compress-then-archive (like BRU)
DM> instead of archive-then-compress (like .tar.gz)
DM> Compress-then-archive makes it more likely that you can
DM> recover all but one file from a corrupted archive.
Hmm. That doesn't explain the .zip discrepancies, though.
~Mr. Bad
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/\____/\ Mr. Bad <mr.bad at pigdog.org>
\ / Pigdog Journal | http://pigdog.org/ | *Stay*Real*Bad*
| (X \x)
( ((**) "If it's not bad, don't do it.
\ <vvv> If it's not crazy, don't say it." - Ben Franklin
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--__--__--
Message: 3
Date: Tue, 6 Feb 2001 09:37:51 -0500 (EST)
From: Chris Anderson <[email protected]>
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Aardvark
Reply-To: devl at freenetproject.org
I got the time down to 6.3 secs by processing blocks in the cipher
streams. I think somewhere around 5.5 secs is probably the speed limit
imposed by the twofish & rijndael crypto gods, though.
On Mon, 5 Feb 2001, Chris Anderson wrote:
> On Mon, 5 Feb 2001, I wrote:
>
> > With a 4.2 mb mp3 file it takes 11.96 secs for RequestClient to transfer
> > it by CHK.
> >
> > Without rijndael cipher streams it takes 6.65 secs
> >
> > Without rijndael and without twofish, but with twofish's cipherstream
> > (disable twofish.encipher), it takes 5.29 secs
> >
> > Without any cipherstreams, it take 4.16 secs
> >
> > Without the sha1 verify stream it takes 1.99 secs
> >
> > I'm still digesting...
> >
>
> Looks like it may be possible to knock the time down to 8.86 secs without
> too much work, but there's a catch.
>
> It turns out that the synchronized flag is fairly expensive on the
> PCFBMode.[en|de]cipher methods and on the SHA1.update method. The
> CipherStream's have it as a private member, so the only way PCFBMode
> can be clobbered is if two threads write to the same stream at the
> same time, and my guess is that this may happen if a message needs to
> break a Conduit. The SHA1 update sync flag is another story, they
> are used on so many places...
>
> Here's a summary of the time in modules from the above run:
>
> rijndael: 1.52 * 2
> cipher-stream: 1.13 * 3
> twofish: 1.36 * 1
> hash-verify: 2.17 * 1
> other: 1.99
>
> Just removing the sync flags cuts the cipher-stream and hash-verify times
> down by about .3 secs. I ended up doing some monster inlining on the
> SHA1 transform to cut another .55 secs off of the verify stream. So, now
> we have this:
>
> rijndael: 1.52 * 2
> cipher-stream: .71 * 3
> twofish: 1.36 * 1
> hash-verify: 1.30 * 1
> other: 1.99
>
> I don't see much that can be done with the two ciphers, but the 1.99 other
> category which is mostly at the client, looks mighty suspicious...
>
>
>
--__--__--
Message: 4
From: Theodore Hong <[email protected]>
Subject: Re: [freenet-devl] Compression of Jars
To: devl at freenetproject.org
Date: Tue, 6 Feb 2001 15:32:24 +0000 (GMT)
Reply-To: devl at freenetproject.org
Steven Hazel <sah at thalassocracy.org> wrote:
> Don Marti <dmarti at zgp.org> writes:
> > My guess is that it uses compress-then-archive (like BRU) instead of
> > archive-then-compress (like .tar.gz) Compress-then-archive makes it
> > more likely that you can recover all but one file from a corrupted
> > archive. Even Sun isn't retarded enough to pick a compression
> > algorithm that makes files twice as big as zip or gzip.
>
> Compress-then-archive will compress considerably less well than
> archive-then-compress, since compress works better when applied to
> more data. And unless pkzip, infozip, and winzip can all decode
> compress-then-archive files without any special indications that they
> need to do so, that's not how jars work.
The zip format -is- compress-then-archive. Notice that when you list the
contents of a zip archive, the compression % of each file is listed
separately.
I suspect that if you did: "find . -name '*.class' | xargs zip freenet.zip",
the size would be virtually the same as the jar.
theo
--__--__--
Message: 5
Date: Tue, 6 Feb 2001 10:41:44 -0500
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Aardvark
From: "Scott G. Miller" <[email protected]>
Reply-To: devl at freenetproject.org
--5mCyUwZo2JvN/JJP
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Tue, Feb 06, 2001 at 09:37:51AM -0500, Chris Anderson wrote:
>=20
> I got the time down to 6.3 secs by processing blocks in the cipher
> streams. I think somewhere around 5.5 secs is probably the speed limit
> imposed by the twofish & rijndael crypto gods, though.
What do you mean by processing blocks? =20
>=20
> On Mon, 5 Feb 2001, Chris Anderson wrote:
>=20
> > On Mon, 5 Feb 2001, I wrote:
> >=20
> > > With a 4.2 mb mp3 file it takes 11.96 secs for RequestClient to trans=
fer
> > > it by CHK.
> > >
> > > Without rijndael cipher streams it takes 6.65 secs
> > >
> > > Without rijndael and without twofish, but with twofish's cipherstream
> > > (disable twofish.encipher), it takes 5.29 secs
> > >
> > > Without any cipherstreams, it take 4.16 secs
> > >
> > > Without the sha1 verify stream it takes 1.99 secs
> > >
> > > I'm still digesting...
> > >
> >=20
> > Looks like it may be possible to knock the time down to 8.86 secs witho=
ut
> > too much work, but there's a catch.
> >=20
> > It turns out that the synchronized flag is fairly expensive on the
> > PCFBMode.[en|de]cipher methods and on the SHA1.update method. The
> > CipherStream's have it as a private member, so the only way PCFBMode
> > can be clobbered is if two threads write to the same stream at the
> > same time, and my guess is that this may happen if a message needs to
> > break a Conduit. The SHA1 update sync flag is another story, they
> > are used on so many places...
> >=20
> > Here's a summary of the time in modules from the above run:
> >=20
> > rijndael: 1.52 * 2
> > cipher-stream: 1.13 * 3
> > twofish: 1.36 * 1
> > hash-verify: 2.17 * 1
> > other: 1.99
> >=20
> > Just removing the sync flags cuts the cipher-stream and hash-verify tim=
es
> > down by about .3 secs. I ended up doing some monster inlining on the
> > SHA1 transform to cut another .55 secs off of the verify stream. So, n=
ow
> > we have this:
> >=20
> > rijndael: 1.52 * 2
> > cipher-stream: .71 * 3
> > twofish: 1.36 * 1
> > hash-verify: 1.30 * 1
> > other: 1.99
> >=20
> > I don't see much that can be done with the two ciphers, but the 1.99 ot=
her
> > category which is mostly at the client, looks mighty suspicious...
> >=20
> >=20
> >=20
>=20
>=20
> _______________________________________________
> Devl mailing list
> Devl at freenetproject.org
> http://www.uprizer.com/mailman/listinfo/devl
>=20
--=20
greg lyric rot motels
--5mCyUwZo2JvN/JJP
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE6gBs4r9IW4v3mHtQRAsLbAJ4xqEwxpoanUywrG+/VtBf3xnPsugCfcntP
3wmxJuoxnUJmkzXZVf7oK4c=
=BCYA
-----END PGP SIGNATURE-----
--5mCyUwZo2JvN/JJP--
--__--__--
Message: 6
Date: Tue, 6 Feb 2001 11:08:19 -0500
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Compression of Jars
From: "Scott G. Miller" <[email protected]>
Reply-To: devl at freenetproject.org
--8P1HSweYDcXXzwPJ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Tue, Feb 06, 2001 at 03:32:24PM +0000, Theodore Hong wrote:
> Steven Hazel <sah at thalassocracy.org> wrote:
> > Don Marti <dmarti at zgp.org> writes:
> > > My guess is that it uses compress-then-archive (like BRU) instead of
> > > archive-then-compress (like .tar.gz) Compress-then-archive makes it
> > > more likely that you can recover all but one file from a corrupted
> > > archive. Even Sun isn't retarded enough to pick a compression
> > > algorithm that makes files twice as big as zip or gzip.
> >=20
> > Compress-then-archive will compress considerably less well than
> > archive-then-compress, since compress works better when applied to
> > more data. And unless pkzip, infozip, and winzip can all decode
> > compress-then-archive files without any special indications that they
> > need to do so, that's not how jars work.
>=20
> The zip format -is- compress-then-archive. Notice that when you list the
> contents of a zip archive, the compression % of each file is listed
> separately.
>=20
> I suspect that if you did: "find . -name '*.class' | xargs zip freenet.zi=
p",
> the size would be virtually the same as the jar.
This is correct. Thats the main reason .tar.gzs are smaller than zip or
jar archives.
--8P1HSweYDcXXzwPJ
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE6gCFzr9IW4v3mHtQRAoVMAJ90jfNCgNCYJt33xlo7f2aes18nQwCfdEYl
ZayCbMZPb6SNnMCc9E6Fa+w=
=NN4P
-----END PGP SIGNATURE-----
--8P1HSweYDcXXzwPJ--
--__--__--
Message: 7
Date: Tue, 6 Feb 2001 18:49:10 +0000
From: Adam Langley <[email protected]>
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Espra - what's that?
Reply-To: devl at freenetproject.org
--7AUc2qLy4jB3hD7Z
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Mon, Feb 05, 2001 at 05:33:28PM -0500, Mark J. Roberts wrote:
> Whiterose gives me "Error in crypto handshake" and Fred is weirder. The
> first time I try, it gives me something like:
Easy - single step the process (from Crypto::handshake) and see where
it borks.
AGL
--=20
When will people realise that we don't care for their damm stupid laws? We =
can handle ourselves, thank you very much.
--7AUc2qLy4jB3hD7Z
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iEYEARECAAYFAjqARyYACgkQzaVS3yy2PWDRUACfQb/XsahvHWm2+pXDFUP2CUb6
HicAoKq6hb4ToLQAQVnyhc6qHfYfTKJ6
=hIdj
-----END PGP SIGNATURE-----
--7AUc2qLy4jB3hD7Z--
--__--__--
Message: 8
Date: Tue, 6 Feb 2001 13:02:02 -0500
From: Tavin Cole <[email protected]>
To: devl at freenetproject.org
Subject: [freenet-devl] progress on 0.4 data store rewrite
Reply-To: devl at freenetproject.org
To anyone who is interested, I have committed the first stage of my
datastore rewrite for 0.4 to experimental cvs. The files are in
Freenet/node/store so cvs update -d please.
The new store has 3 interfaces:
EntityStore
EntityFactory
RoutingTable
The EntityStore manages the document cache, while the EntityFactory
allocates new storage in the document cache. There is a stub
implementation of both in DFSEntityStore.java which will use Scott's
DatastoreFS. It will require three new classes, yet to be written,
which mimick the current FileEntity, FileData, and FileProperties:
they will be DFSEntity, DFSData, and DFSProperties.
RoutingTable is implemented by TreeRoutingTable.java which is
almost completely written (just have to put in the serialization
routines). It uses a red-black binary tree to look up keys and
traverse them in order of closeness.
This code is thoroughly untested, but doesn't give me any errors
on compilation at least.. :)
DISCLAIMER:
No matter how it looks, I have NOT CHANGED ANY ALGORITHMS THAT
WOULD ALTER HOW FREENET WORKS. All I'm doing is making stuff
faster.
TODO:
Write serialization routines for TreeRoutingTable
Finish DFSEntityStore.java
Test, test, test.
Critiques would be appreciated.. I have done some pretty weird things,
but not without justification. I humbly await being torn to shreds by
the pack of wild dogs who read this list.. ;^)
cheers
--
// Tavin Cole
--__--__--
Message: 9
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Compression of Jars
From: Steven Hazel <[email protected]>
Date: 06 Feb 2001 13:35:46 -0600
Reply-To: devl at freenetproject.org
Theodore Hong <twh1 at doc.ic.ac.uk> writes:
> Steven Hazel <sah at thalassocracy.org> wrote:
> > Compress-then-archive will compress considerably less well than
> > archive-then-compress, since compress works better when applied to
> > more data. And unless pkzip, infozip, and winzip can all decode
> > compress-then-archive files without any special indications that they
> > need to do so, that's not how jars work.
>
> The zip format -is- compress-then-archive. Notice that when you list the
> contents of a zip archive, the compression % of each file is listed
> separately.
Oh, duh. I should have realized that.
-S
--__--__--
_______________________________________________
Devl mailing list
Devl at freenetproject.org
http://www.uprizer.com/mailman/listinfo/devl
End of Devl Digest
