Even if it overflows to floating point, it still gets 52-bit
preceison. In zlib, adler32 checksum is calcualted for each
block, which I guess is 32K or 64K. Therefore the issue of
overflow might well be a non-issue in the context of zlib.
On J32
9!:11[16
255**:65536
1095216660480
2^51x
I'm having a hard time getting a streaming version (one that works on parts of
the data and then recombines.
can only do 4096 bytes for 32 bit J, before it needs do call |.. but it is
fast, and 64bit J can do 2^32 larger byte sizes (over 16TB)
timespacex ' (65521 | [: ({: , [: <: +/) +/\) 1
sorry that version doesn't work on a stream. will post one later.
- Original Message -
From: bill lam
To: 'Pascal Jasmin' via Programming
Cc:
Sent: Sunday, August 24, 2014 11:15:57 AM
Subject: Re: [Jprogramming] Adler-32
Did you mean _4]\ ?
Anyway, from the wikipedia page,
B = (1
Did you mean _4]\ ?
Anyway, from the wikipedia page,
B = (1 + D[1]) + (1 + D[1] + D[2]) + ... + (1 + D[1] + D[2] + ... + D[n]) (mod
65521)
= n×D[1] + (n−1)×D[2] + (n−2)×D[3] + ... + D[n] + n (mod 65521)
if a parallel version is asked for, probably the formula in the
2nd line should be used
Pascal, It seems to me that this is a unique problem with 4 nouns. If you know
J reasonably well, you would build this on the fly, and then build your verb
accordingly.
(<5), (<(<(<"0)4 1),<3),<2
┌─┬─┬─┐
│5│┌─┬─┐│2│
│ ││┌─┬─┐│3││ │
│ │││4│1││ ││ │
│ ││└─┴─┘│ ││ │
│ │└─┴
with 10m bytes, I think 2^19 should work with 32bit J too without needing x:,
and there is negligeable speed difference between _524288 ]/ 1 and _524288000
]/ 1 ,~ Randdata on 64 bit
Randdata =: ?1000$256
65521&|@:([: ({: , [: <: +/) +/\)"1 ] _524288 ]/ 1 ,~ Randdata
0.13647 2.68445e8
> You could try making sure that your alias starts with a capital letter. I
think that is the convention for user (as opposed to system) folders.
Yes, this is correct. You should see the lowercase folder included with
SystemFolders_j_.
There is a good reason for the distinction between SystemFold
On Sun, Aug 24, 2014 at 6:43 AM, mike_liz@tiscali.co.uk
wrote:
> I continue to be puzzled by the poor performance of something
> like add32/\ compared to +/\ .
The answer, here, can be explained in terms of an issue which I remember
being taught in gradeschool - "the associative law". (Of co
an approach that should be fast and also allows resuming where you left off,
and avoiding many |
([: ({: , [: <: +/) +/\) 1, a. i. 'Wikipedia'
920 4582
works even when you split up stream into say 32k bytes, and do the mod
transform between each stream, and the single number reduction at
I realized that my loop translation solution was equally faulty in its
final step. It should be
({: (23 b.) 16&(33 b.)@{.) s2,s1
On Aug 24, 2014 6:40 PM, "bill lam" wrote:
> J32 should give a 32-bit integer so that the last train
> should be
> [: ({: (23 b.) 16&(33 b.)@{.)
> instead of
> 65536
What overflow problem I thought when I came up with
add32 =: 65521&|@:
+ NB. add modulo 65521
a32mdslow =: ((1 add32{:)(+ 16 (33 b.)])
(#add32(add32/)))@:(add32/\)@(a.&i.)
and then I tested it on modest
size input and found it was ORDERS
of magnitude slower than Bill's
loopy version!
So
J32 should give a 32-bit integer so that the last train
should be
[: ({: (23 b.) 16&(33 b.)@{.)
instead of
65536#.
This shoud be good enough. Thanks.
Вс, 24 авг 2014, Jan-Pieter Jacobs написал(а):
> You could incorporate the modulo operation in the summation directly:
>
> As per Raul's solut
You could incorporate the modulo operation in the summation directly:
As per Raul's solution, with the modulo addition:
a32_2=: 65536#. _1 0+ [:((65521|+)/ , {.) [: (65521|+)/\. 1,~ a.i. |.
NB. Test on 10M of random data
Randdata =: a.{~ ?1000$256
timespacex 'r1=:a32_2 Randdata'
7.26671 2.684
Since s2 is quadratic, it overflows quickly in J32. However
ieee floating points have 52 bits mantissa, so it can still
give correct result for medium size data albeit overflowed.
9!:11[16
(3!:0;]) a32 1e6{.a.
Вс, 24 авг 2014, Raul Miller написал(а):
> Hmm... actually there might be a floating
Hmm... actually there might be a floating point overflow problem for
gigabyte length arguments. I do not have the patience, however, to
find what (for example) adler32 1e9#{:a. is.
Thanks,
--
Raul
On Sun, Aug 24, 2014 at 4:37 AM, Raul Miller wrote:
> I believe this is equivalent, and a bit f
I believe this is equivalent, and a bit faster:
a32=: 65536#. 65521| _1 0+ [:(+/ , {.) 65521| [:+/\. 1,~ a.i. |.
a32 'Wikipedia'
300286872
a32 'The quick brown fox jumps over the lazy dog'
1541148634
Thanks,
--
Raul
On Sun, Aug 24, 2014 at 4:05 AM, bill lam wrote:
> Does anyone have a
Does anyone have a J-ish solution to adler32 checksum
and avoid overflow to floating points ?
Adler-32 is composed of two sums accumulated per byte: s1 is
the sum of all bytes, s2 is the sum of all s1 values. Both sums
are done modulo 65521. s1 is initialized to 1, s2 to zero. The
Adler-32 check
I added:
TmpC c:/tmp/
to my folders.cfg and then
dir '~TmpC'
worked fine.
You could try making sure that your alias starts with a capital letter. I
think that is the convention for user (as opposed to system) folders. I
tried the following with no sucess:
tmpC c:/tmp/
On Sat, Aug 23, 2014 at
18 matches
Mail list logo
