[freenet-dev] [freenet-support] Freenet 0.7.5 build 1253: UPGRADE NOW!!!
On Mon, Jun 14, 2010 at 4:56 PM, Matthew Toseland wrote: > - Even segment splitting. Freenet divides large files up into "splitfiles": > because a CHK can only be 32KB, Freenet splits larger files up into many > blocks, and then divides those blocks into "segments" of no more than 256 > blocks (128 data blocks, and 128 "check blocks" that are generated for > redundancy). Until 1251, on any splitfile of more than 128 blocks (4MB), all > but the last segment would be exactly 128 data blocks and 128 check blocks. > From this build onwards, all the segments will be roughly the same size. > Plus, we allow up to 131 data blocks (with 125 check blocks) to use fewer > segments (which is a net benefit for reliability), and we add an extra check > block to each segment of less than 128 data blocks. You're only allowing > 128 data blocks on small files, right? With > 520 total data blocks, it should be max 128 data blocks per segment (ie at 5 segments or more). And with less than that, the appropriate max depends on file size. Evan Daniel
[freenet-dev] Freenet 0.7.5 build 1253: UPGRADE NOW!!!
Freenet 0.7.5 build 1253 is now available. This build is immediately mandatory (it will only talk to 1252 or later) because it fixes a critical (potentially remote code execution) bug in FEC, so please upgrade immediately. Major changes: - Even segment splitting. Freenet divides large files up into "splitfiles": because a CHK can only be 32KB, Freenet splits larger files up into many blocks, and then divides those blocks into "segments" of no more than 256 blocks (128 data blocks, and 128 "check blocks" that are generated for redundancy). Until 1251, on any splitfile of more than 128 blocks (4MB), all but the last segment would be exactly 128 data blocks and 128 check blocks. From this build onwards, all the segments will be roughly the same size. Plus, we allow up to 131 data blocks (with 125 check blocks) to use fewer segments (which is a net benefit for reliability), and we add an extra check block to each segment of less than 128 data blocks. - This triggered some rather nasty bugs in the FEC encoding native code. We have a workaround in 1252 but in 1253 native FEC is disabled completely until somebody can get a proper look at the code. These bugs CAN CAUSE SEGMENTATION FAULTS and thus are conceivably exploitable for remote code execution - so you should upgrade NOW! - New content filtering infrastructure. Filtering is now in the client layer rather than part of fproxy, and can be accessed via FCP. Downloads to disk can be filtered. And it's more efficient - future changes based on this code should make it use significantly less I/O, and pave the way for new filters for multimedia files. -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20100614/93958c1c/attachment.pgp>
[freenet-dev] Library version 12
Library version 12 is out. This missed 1252, but will be force-updated in 1253. New-format indexes were being both written and parsed in the local charset, not in UTF-8. If your system locale is anything other than UTF-8, this means the on-disk and on-freenet indexes your Spider has been generating through Library are worthless (unless you convert them). Sorry... -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20100614/e1235b75/attachment.pgp>
[freenet-dev] memory corruption and abort on downloading KSK@frost|message|news|2010.6.13-freenet-38.xml
On 14.06.2010 17:41, Ximin Luo wrote: > does it still happen when you disable libfec? you need to open up > freenet-ext.jar and delete the relevant lines from lib/fec.properties and > lib/native.properties Good catch. Edited lib/native.properties (deleted *linux*), got Unable to find native library for fec8 for platform linux-x86 null Failed to load native FEC: java.lang.UnsatisfiedLinkError: com.onionnetworks.fec.Native8Code.nativeNewFEC(II)I (as expected), and no crash on key download. But this key looks very strange; I've downloaded it, got empty file GET http://127.0.0.1:/KSK at frost%7Cmessage%7Cnews%7C2010.6.13-freenet-38.xml?type=text/plain --> 200 OK [...] Content-Length: 0 Content-Type: text/plain but splitfile consists of 2 data blocks. Something looks really eeky. Tried both data blocks in KeyUtils|Key, got "no data", but (some) checkblocks present. Have no idea where it goes.
Re: [freenet-dev] [freenet-support] Freenet 0.7.5 build 1253: UPGRADE NOW!!!
On Mon, Jun 14, 2010 at 4:56 PM, Matthew Toseland wrote: > - Even segment splitting. Freenet divides large files up into "splitfiles": > because a CHK can only be 32KB, Freenet splits larger files up into many > blocks, and then divides those blocks into "segments" of no more than 256 > blocks (128 data blocks, and 128 "check blocks" that are generated for > redundancy). Until 1251, on any splitfile of more than 128 blocks (4MB), all > but the last segment would be exactly 128 data blocks and 128 check blocks. > From this build onwards, all the segments will be roughly the same size. > Plus, we allow up to 131 data blocks (with 125 check blocks) to use fewer > segments (which is a net benefit for reliability), and we add an extra check > block to each segment of less than 128 data blocks. You're only allowing > 128 data blocks on small files, right? With > 520 total data blocks, it should be max 128 data blocks per segment (ie at 5 segments or more). And with less than that, the appropriate max depends on file size. Evan Daniel ___ Devl mailing list Devl@freenetproject.org http://freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] memory corruption and abort on downloading KSK@frost|message|news|2010.6.13-freenet-38.xml
On Monday 14 June 2010 17:22:11 Yuriy Kaminskiy wrote: > On 14.06.2010 17:41, Ximin Luo wrote: > > does it still happen when you disable libfec? you need to open up > > freenet-ext.jar and delete the relevant lines from lib/fec.properties and > > lib/native.properties > Good catch. Edited lib/native.properties (deleted *linux*), got > Unable to find native library for fec8 for platform linux-x86 > null > Failed to load native FEC: java.lang.UnsatisfiedLinkError: > com.onionnetworks.fec.Native8Code.nativeNewFEC(II)I > (as expected), and no crash on key download. > But this key looks very strange; I've downloaded it, got empty file > GET > http://127.0.0.1:/KSK at > frost%7Cmessage%7Cnews%7C2010.6.13-freenet-38.xml?type=text/plain > --> 200 OK > [...] > Content-Length: 0 > Content-Type: text/plain > but splitfile consists of 2 data blocks. Something looks really eeky. > Tried both data blocks in KeyUtils|Key, got "no data", but (some) checkblocks > present. Have no idea where it goes. Yes it has 2 data blocks 5 check blocks which causes a segfault with the current code. Am in the process of releasing 1251 with a workaround (don't use native FEC unless data blocks >= check blocks). We REALLY need to fix the FEC code... -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20100614/4bd3cc1d/attachment.pgp>
[freenet-dev] memory corruption and abort on downloading KSK@frost|message|news|2010.6.13-freenet-38.xml
I've noticed spam attack on frost reappeared, and out of curiosity looked at some keys. To my horror examining one of keys in KeyUtils caused node restart, with rather scary message in wrapper.log: INFO | jvm 4| 2010/06/12 20:13:32 | Announcement to 94.23.20.44:55705 completed. >INFO | jvm 4| 2010/06/13 17:01:19 | *** glibc detected *** free(): invalid next size (fast): 0x081399b8 *** STATUS | wrapper | 2010/06/13 17:01:25 | JVM received a signal UNKNOWN (6). STATUS | wrapper | 2010/06/13 17:01:25 | JVM process is gone. ERROR | wrapper | 2010/06/13 17:01:25 | JVM exited unexpectedly. STATUS | wrapper | 2010/06/13 17:01:29 | Reloading Wrapper configuration... STATUS | wrapper | 2010/06/13 17:01:31 | Launching a JVM... INFO | jvm 5| 2010/06/13 17:01:32 | WrapperManager: Initializing... Reproducible. Downloading this key: curl 'http://127.0.0.1:/KSK at frost|message|news|2010.6.13-freenet-38.xml?type=text/plain' also causes node restart with same error. Unfortunately, I found nothing useful in node logs (suppose, they are not flushed?) :-\ This is on build1250, i386, linux, sun java 1.6.0_20
[freenet-dev] memory corruption and abort on downloading KSK@frost|message|news|2010.6.13-freenet-38.xml
does it still happen when you disable libfec? you need to open up freenet-ext.jar and delete the relevant lines from lib/fec.properties and lib/native.properties X On 14/06/10 12:35, Yuriy Kaminskiy wrote: > I've noticed spam attack on frost reappeared, and out of curiosity looked at > some keys. To my horror examining one of keys in KeyUtils caused node restart, > with rather scary message in wrapper.log: > INFO | jvm 4| 2010/06/12 20:13:32 | Announcement to 94.23.20.44:55705 > completed. >> INFO | jvm 4| 2010/06/13 17:01:19 | *** glibc detected *** free(): > invalid next size (fast): 0x081399b8 *** > STATUS | wrapper | 2010/06/13 17:01:25 | JVM received a signal UNKNOWN (6). > STATUS | wrapper | 2010/06/13 17:01:25 | JVM process is gone. > ERROR | wrapper | 2010/06/13 17:01:25 | JVM exited unexpectedly. > STATUS | wrapper | 2010/06/13 17:01:29 | Reloading Wrapper configuration... > STATUS | wrapper | 2010/06/13 17:01:31 | Launching a JVM... > INFO | jvm 5| 2010/06/13 17:01:32 | WrapperManager: Initializing... > Reproducible. Downloading this key: > curl > 'http://127.0.0.1:/KSK at > frost|message|news|2010.6.13-freenet-38.xml?type=text/plain' > also causes node restart with same error. > Unfortunately, I found nothing useful in node logs (suppose, they are not > flushed?) :-\ > This is on build1250, i386, linux, sun java 1.6.0_20 > > ___ > Devl mailing list > Devl at freenetproject.org > http://freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Freenet 0.7.5 build 1253: UPGRADE NOW!!!
Freenet 0.7.5 build 1253 is now available. This build is immediately mandatory (it will only talk to 1252 or later) because it fixes a critical (potentially remote code execution) bug in FEC, so please upgrade immediately. Major changes: - Even segment splitting. Freenet divides large files up into "splitfiles": because a CHK can only be 32KB, Freenet splits larger files up into many blocks, and then divides those blocks into "segments" of no more than 256 blocks (128 data blocks, and 128 "check blocks" that are generated for redundancy). Until 1251, on any splitfile of more than 128 blocks (4MB), all but the last segment would be exactly 128 data blocks and 128 check blocks. From this build onwards, all the segments will be roughly the same size. Plus, we allow up to 131 data blocks (with 125 check blocks) to use fewer segments (which is a net benefit for reliability), and we add an extra check block to each segment of less than 128 data blocks. - This triggered some rather nasty bugs in the FEC encoding native code. We have a workaround in 1252 but in 1253 native FEC is disabled completely until somebody can get a proper look at the code. These bugs CAN CAUSE SEGMENTATION FAULTS and thus are conceivably exploitable for remote code execution - so you should upgrade NOW! - New content filtering infrastructure. Filtering is now in the client layer rather than part of fproxy, and can be accessed via FCP. Downloads to disk can be filtered. And it's more efficient - future changes based on this code should make it use significantly less I/O, and pave the way for new filters for multimedia files. signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org http://freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Library version 12
Library version 12 is out. This missed 1252, but will be force-updated in 1253. New-format indexes were being both written and parsed in the local charset, not in UTF-8. If your system locale is anything other than UTF-8, this means the on-disk and on-freenet indexes your Spider has been generating through Library are worthless (unless you convert them). Sorry... signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org http://freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] memory corruption and abort on downloading KSK@frost|message|news|2010.6.13-freenet-38.xml
On Monday 14 June 2010 17:22:11 Yuriy Kaminskiy wrote: > On 14.06.2010 17:41, Ximin Luo wrote: > > does it still happen when you disable libfec? you need to open up > > freenet-ext.jar and delete the relevant lines from lib/fec.properties and > > lib/native.properties > Good catch. Edited lib/native.properties (deleted *linux*), got > Unable to find native library for fec8 for platform linux-x86 > null > Failed to load native FEC: java.lang.UnsatisfiedLinkError: > com.onionnetworks.fec.Native8Code.nativeNewFEC(II)I > (as expected), and no crash on key download. > But this key looks very strange; I've downloaded it, got empty file > GET > http://127.0.0.1:/k...@frost%7cmessage%7cnews%7c2010.6.13-freenet-38.xml?type=text/plain > --> 200 OK > [...] > Content-Length: 0 > Content-Type: text/plain > but splitfile consists of 2 data blocks. Something looks really eeky. > Tried both data blocks in KeyUtils|Key, got "no data", but (some) checkblocks > present. Have no idea where it goes. Yes it has 2 data blocks 5 check blocks which causes a segfault with the current code. Am in the process of releasing 1251 with a workaround (don't use native FEC unless data blocks >= check blocks). We REALLY need to fix the FEC code... signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org http://freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] memory corruption and abort on downloading KSK@frost|message|news|2010.6.13-freenet-38.xml
On 14.06.2010 17:41, Ximin Luo wrote: > does it still happen when you disable libfec? you need to open up > freenet-ext.jar and delete the relevant lines from lib/fec.properties and > lib/native.properties Good catch. Edited lib/native.properties (deleted *linux*), got Unable to find native library for fec8 for platform linux-x86 null Failed to load native FEC: java.lang.UnsatisfiedLinkError: com.onionnetworks.fec.Native8Code.nativeNewFEC(II)I (as expected), and no crash on key download. But this key looks very strange; I've downloaded it, got empty file GET http://127.0.0.1:/k...@frost%7cmessage%7cnews%7c2010.6.13-freenet-38.xml?type=text/plain --> 200 OK [...] Content-Length: 0 Content-Type: text/plain but splitfile consists of 2 data blocks. Something looks really eeky. Tried both data blocks in KeyUtils|Key, got "no data", but (some) checkblocks present. Have no idea where it goes. ___ Devl mailing list Devl@freenetproject.org http://freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] memory corruption and abort on downloading KSK@frost|message|news|2010.6.13-freenet-38.xml
does it still happen when you disable libfec? you need to open up freenet-ext.jar and delete the relevant lines from lib/fec.properties and lib/native.properties X On 14/06/10 12:35, Yuriy Kaminskiy wrote: > I've noticed spam attack on frost reappeared, and out of curiosity looked at > some keys. To my horror examining one of keys in KeyUtils caused node restart, > with rather scary message in wrapper.log: > INFO | jvm 4| 2010/06/12 20:13:32 | Announcement to 94.23.20.44:55705 > completed. >> INFO | jvm 4| 2010/06/13 17:01:19 | *** glibc detected *** free(): > invalid next size (fast): 0x081399b8 *** > STATUS | wrapper | 2010/06/13 17:01:25 | JVM received a signal UNKNOWN (6). > STATUS | wrapper | 2010/06/13 17:01:25 | JVM process is gone. > ERROR | wrapper | 2010/06/13 17:01:25 | JVM exited unexpectedly. > STATUS | wrapper | 2010/06/13 17:01:29 | Reloading Wrapper configuration... > STATUS | wrapper | 2010/06/13 17:01:31 | Launching a JVM... > INFO | jvm 5| 2010/06/13 17:01:32 | WrapperManager: Initializing... > Reproducible. Downloading this key: > curl > 'http://127.0.0.1:/k...@frost|message|news|2010.6.13-freenet-38.xml?type=text/plain' > also causes node restart with same error. > Unfortunately, I found nothing useful in node logs (suppose, they are not > flushed?) :-\ > This is on build1250, i386, linux, sun java 1.6.0_20 > > ___ > Devl mailing list > Devl@freenetproject.org > http://freenetproject.org/cgi-bin/mailman/listinfo/devl ___ Devl mailing list Devl@freenetproject.org http://freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] memory corruption and abort on downloading KSK@frost|message|news|2010.6.13-freenet-38.xml
I've noticed spam attack on frost reappeared, and out of curiosity looked at some keys. To my horror examining one of keys in KeyUtils caused node restart, with rather scary message in wrapper.log: INFO | jvm 4| 2010/06/12 20:13:32 | Announcement to 94.23.20.44:55705 completed. >INFO | jvm 4| 2010/06/13 17:01:19 | *** glibc detected *** free(): invalid next size (fast): 0x081399b8 *** STATUS | wrapper | 2010/06/13 17:01:25 | JVM received a signal UNKNOWN (6). STATUS | wrapper | 2010/06/13 17:01:25 | JVM process is gone. ERROR | wrapper | 2010/06/13 17:01:25 | JVM exited unexpectedly. STATUS | wrapper | 2010/06/13 17:01:29 | Reloading Wrapper configuration... STATUS | wrapper | 2010/06/13 17:01:31 | Launching a JVM... INFO | jvm 5| 2010/06/13 17:01:32 | WrapperManager: Initializing... Reproducible. Downloading this key: curl 'http://127.0.0.1:/k...@frost|message|news|2010.6.13-freenet-38.xml?type=text/plain' also causes node restart with same error. Unfortunately, I found nothing useful in node logs (suppose, they are not flushed?) :-\ This is on build1250, i386, linux, sun java 1.6.0_20 ___ Devl mailing list Devl@freenetproject.org http://freenetproject.org/cgi-bin/mailman/listinfo/devl
