Thanks. I realize that. After all, if ?1e6$12345
results in z=:_1e6$(<.1e6%12345)$i.12345,
z would satisfy the chi square test, KS test, etc.,
but we would (should) be highly suspicious of
the "RNG"!
So I am depending on the fact that ?m generates
uniform random numbers as well as having
many other desirable properties. In this case it does,
because ?m uses the Mersenne Twister.
> For y an extended precision number, ?y is computed
> by t=.?y1 where y1 is a little bigger than y,
> repeating until t is less than y. (The same y1
> is used for a given y.)
In detail:
B=: 10000x
rand=: 3 : 0
m=. <.10^.B
t=. (-m) ]\ ((m|-#t)$' '),t=.":y
n=. <:#t
c=. ".{.t NB. leading digit
v=. (c++./'0'~:,}.t),n#B
whilst. y<:z do. z=. B#.?v end.
)
B is the base for extended integers. The loop
repeatedly does z=.B#.?v until y>z . The
number B#.B|v is the y1 that I described.
The bug in the interpreter was that it was
basically doing y<z instead of the correct y<:z .
Examples:
11^20x
672749994932560009201
rand 11^20x
554374486617845425729
rand 11^20x
98359541759946207683
t=: rand"0 ]1e5$12345x
>./t
12344
<./t
0
(c%12344) , (+/c>:t)%#t [ c=: ?12345
0.225697 0.22524
----- Original Message -----
From: Henry Rich <[email protected]>
Date: Saturday, May 21, 2011 10:15
Subject: Re: [Jprogramming] roll
To: Programming forum <[email protected]>
> Even if you know that the distribution of numbers from your RNG
> is
> uniform, that is not sufficient to ensure that there are not
> pernicious
> patterns in the numbers.
>
> For example: multiplicative congruence generators produce
> sequences that
> pass all sorts of tests, but it turns out that if you use sets
> of n
> sequential numbers produced by such an RNG to sample an n-
> dimensional
> space, the n-sets fall on hyperplanes of the n-dimensional
> space,
> leaving large areas unsampled.
>
> I don't know enough to offer any advice about picking an RNG for
> general
> use. My one experience with this sort of thing was when I
> got some
> surprising results using a multiplicative congruence RNG, and
> replaced
> it with Knuth's suggestion, which at the time was to use two
> such RNGs,
> shuffling the results from one using the results of the other,
> and my
> results became reasonable.
>
> Henry Rich
>
> On 5/21/2011 11:30 AM, Roger Hui wrote:
> > Thanks for your reply. To paraphrase Ewart Shaw,
> > my lack of depth in statistics is only ameliorated by
> > the narrowness of my understanding. Regarding the
> > last result:
> >
> >> -.14&chisqcdf f i.5
> >> 0.262214 0.120479 0.124215 0.292923 0.68203
> >
> > What are the numbers saying? Do they indicate that the
> > underlying distribution is nearly uniform or far from uniform?
> > Is it a good or bad thing (or irrelevant) that the 5 numbers
> > are quite different from each other?
> >
> > In "school" I learned that the Kolmogorov-Smirnov test
> > http://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test
> > is good for this kind of thing. What do you think of
> > this test?
> >
> >
> >
> > ----- Original Message -----
> > From: John Randall<[email protected]>
> > Date: Saturday, May 21, 2011 7:52
> > Subject: Re: [Jprogramming] roll
> > To: Programming forum<[email protected]>
> >
> >> The process described should yield uniform random numbers
> >> between 0
> >> and y-1.
> >>
> >> Following up on the tests in the original posting, we can put the
> >> numbers into equal sized bins and then calculate a chi-square
> >> statistic. If this is not significant, we can be somewhat
> >> assuredthat the numbers are random.
> >>
> >> f=:3 : 0"0
> >> y=. 12345
> >> t=. ?2e6$20000
> >> z=. 1e6 {. (12345>t)#t
> >> NB. 12345=15*823
> >> bin=.<.@%&823
> >> observed=.(bin z) #/. z
> >> expected=.(#z)%15
> >> chisquare=.+/(*:observed-expected)%expected
> >> )
> >>
> >> require '~addons/stats/base/distribution.ijs'
> >>
> >> -.14&chisqcdf f i.5
> >> 0.262214 0.120479 0.124215 0.292923 0.68203
> >>
> >> Best wishes,
> >>
> >> John
> >>
> >>
> >>
> >> Roger Hui wrote:
> >>> I have found a bug in ?y where y is an extended
> >>> precision integer. The result should never be y,
> >>> but:
> >>>
> >>> t=: ? 1e6 $ 12345x
> >>> +/t=12345
> >>> 89
> >>>
> >>> The last result should of course be 0.
> >>>
> >>> I know how this can be fixed, but I would like to
> >>> verify that the algorithm for ?y is correct with the
> >>> mathematicians in this forum.
> >>>
> >>> For y an extended precision number, ?y is computed
> >>> by t=.?y1 where y1 is a little bigger than y,
> >>> repeating until t is less than y. (The same y1
> >>> is used for a given y.) Assuming that ?m produces
> >>> uniform random numbers between 0 and m-1, does
> >>> this process give uniform random numbers between
> >>> 0 and y-1?
> >>>
> >>> For example, suppose we want to compute z=:?1e6$y=:12345.
> >>>
> >>> y=: 12345
> >>> t=: ?2e6$20000
> >>> +/12345>t
> >>> 1233158
> >>> z=: 1e6 {. (12345>t)#t
> >>>
> >>> NB. various tests
> >>> (+/z)%#z
> >>> 6170.3
> >>> 12344%2
> >>> 6172
> >>>
> >>> (c%y) , (+/c>:z)%#z [ c=: ? y
> >>> 0.993277 0.993333
> >>> (c%y) , (+/c>:z)%#z [ c=: ? y
> >>> 0.43872 0.438943
> >>> (c%y) , (+/c>:z)%#z [ c=: ? y
> >>> 0.593925 0.594519
> >>> (c%y) , (+/c>:z)%#z [ c=: ? y
> >>> 0.315674 0.315579
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
