Re: [Rd] Extreme bunching of random values from runif with Mersenne-Twister seed
Tirthankar,
"random number generators" do not produce random numbers. Any given
generator produces a fixed sequence of numbers that appear to meet
various tests of randomness. By picking a seed you enter that sequence
in a particular place and subsequent numbers in the sequence appear to
be unrelated. There are no guarantees that if YOU pick a SET of seeds
they won't produce a set of values that are of a similar magnitude.
You can likely solve your problem by following Radford Neal's advice of
not using the the first number from each seed. However, you don't need
to use anything more than the second number. So, you can modify your
function as follows:
function(x) {
set.seed(x, kind = "default")
y = runif(2, 17, 26)
return(y[2])
}
Hope this is helpful,
Dan
--
Daniel Nordlund
Port Townsend, WA USA
On 11/3/2017 11:30 AM, Tirthankar Chakravarty wrote:
Bill,
Appreciate the point that both you and Serguei are making, but the sequence
in question is not a selected or filtered set. These are values as observed
in a sequence from a mechanism described below. The probabilities required
to generate this exact sequence in the wild seem staggering to me.
T
On Fri, Nov 3, 2017 at 11:27 PM, William Dunlap <wdun...@tibco.com> wrote:
Another other generator is subject to the same problem with the same
probabilitiy.
Filter(function(s){set.seed(s, kind="Knuth-TAOCP-2002");runif(1,17,26)>25.99},
1:1)
[1] 280 415 826 1372 2224 2544 3270 3594 3809 4116 4236 5018 5692 7043
7212 7364 7747 9256 9491 9568 9886
Bill Dunlap
TIBCO Software
wdunlap tibco.com
On Fri, Nov 3, 2017 at 10:31 AM, Tirthankar Chakravarty <
tirthankar.li...@gmail.com> wrote:
Bill,
I have clarified this on SO, and I will copy that clarification in here:
"Sure, we tested them on other 8-digit numbers as well & we could not
replicate. However, these are honest-to-goodness numbers generated by a
non-adversarial system that has no conception of these numbers being used
for anything other than a unique key for an entity -- these are not a
specially constructed edge case. Would be good to know what seeds will and
will not work, and why."
These numbers are generated by an application that serves a form, and
associates form IDs in a sequence. The application calls our API depending
on the form values entered by users, which in turn calls our R code that
executes some code that needs an RNG. Since the API has to be stateless, to
be able to replicate the results for possible debugging, we need to draw
random numbers in a way that we can replicate the results of the API
response -- we use the form ID as seeds.
I repeat, there is no design or anything adversarial about the way that
these numbers were generated -- the system generating these numbers and
the users entering inputs have no conception of our use of an RNG -- this
is meant to just be a random sequence of form IDs. This issue was
discovered completely by chance when the output of the API was observed to
be highly non-random. It is possible that it is a 1/10^8 chance, but that
is hard to believe, given that the API hit depends on user input. Note also
that the issue goes away when we use a different RNG as mentioned below.
T
On Fri, Nov 3, 2017 at 9:58 PM, William Dunlap <wdun...@tibco.com> wrote:
The random numbers in a stream initialized with one seed should have
about the desired distribution. You don't win by changing the seed all the
time. Your seeds caused the first numbers of a bunch of streams to be
about the same, but the second and subsequent entries in each stream do
look uniformly distributed.
You didn't say what your 'upstream process' was, but it is easy to come
up with seeds that give about the same first value:
Filter(function(s){set.seed(s);runif(1,17,26)>25.99}, 1:1)
[1] 514 532 1951 2631 3974 4068 4229 6092 6432 7264 9090
Bill Dunlap
TIBCO Software
wdunlap tibco.com
On Fri, Nov 3, 2017 at 12:49 AM, Tirthankar Chakravarty <
tirthankar.li...@gmail.com> wrote:
This is cross-posted from SO (https://stackoverflow.com/q/4
7079702/1414455),
but I now feel that this needs someone from R-Devel to help understand
why
this is happening.
We are facing a weird situation in our code when using R's [`runif`][1]
and
setting seed with `set.seed` with the `kind = NULL` option (which
resolves,
unless I am mistaken, to `kind = "default"`; the default being
`"Mersenne-Twister"`).
We set the seed using (8 digit) unique IDs generated by an upstream
system,
before calling `runif`:
seeds = c(
"86548915", "86551615", "86566163", "86577411", "86584144",
"86584272", "86620568", "86724613", "86756002", "86768593",
"86772411",
"86781516", "86794389", "86805854"
Re: [Rd] Missing Windows binary for R-2.15RC?
-Original Message- From: r-devel-boun...@r-project.org [mailto:r-devel-boun...@r-project.org] On Behalf Of Dan Tenenbaum Sent: Friday, March 23, 2012 12:21 PM To: r-devel@r-project.org Subject: [Rd] Missing Windows binary for R-2.15RC? Hi, The page http://cran.r-project.org/bin/windows/base/rtest.html has a link to: http://cran.r-project.org/bin/windows/base/R-2.15.0rc-win.exe However, clicking on that link gives a 404 Object not found' error. FYI. Dan I experienced the same error you did using the link you provided. However, if you use the CRAN mirror hosted by YOUR organization, you can get the file. :-) Hope this is helpful, Dan Daniel Nordlund Bothell, WA USA __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Missing Windows binary for R-2.15RC?
-Original Message- From: Dan Tenenbaum [mailto:dtene...@fhcrc.org] Sent: Friday, March 23, 2012 5:48 PM To: Daniel Nordlund Cc: r-devel@r-project.org Subject: Re: [Rd] Missing Windows binary for R-2.15RC? On Fri, Mar 23, 2012 at 4:52 PM, Daniel Nordlund djnordl...@frontier.com wrote: -Original Message- From: r-devel-boun...@r-project.org [mailto:r-devel-bounces@r- project.org] On Behalf Of Dan Tenenbaum Sent: Friday, March 23, 2012 12:21 PM To: r-devel@r-project.org Subject: [Rd] Missing Windows binary for R-2.15RC? Hi, The page http://cran.r-project.org/bin/windows/base/rtest.html has a link to: http://cran.r-project.org/bin/windows/base/R-2.15.0rc-win.exe However, clicking on that link gives a 404 Object not found' error. FYI. Dan I experienced the same error you did using the link you provided. However, if you use the CRAN mirror hosted by YOUR organization, you can get the file. :-) I don't think so: http://cran.fhcrc.org/bin/windows/base/R-2.15.0rc-win.exe gives me a 404 as well. Dan I didn't look closely enough at what you were asking for (RC versus beta). R-2.15RC may not have been up-loaded yet. However, I just downloaded it from the original link that was posted, so it appears to be available now. Dan Daniel Nordlund Bothell, WA USA __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] norm_rand() in R-extension vs rnorm in R ---which is better?
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kyeongmi Cheon Sent: Saturday, March 29, 2008 9:09 PM To: r-devel@r-project.org Subject: [Rd] norm_rand() in R-extension vs rnorm in R ---which is better? I need to generate good quality of random numbers from univariate normal distribution for further transformation. I tried rnorm in R but it was not good sometimes. Someone said C++ standard library or Fortran's built-in functions for that are good. I found that there is double norm_rand() in R-extension. Does this function, norm_rand() use the same algorithm as rnorm in R or something similar in C/C++? Or does anyone know which one between norm_rand() in R-extension and rnorm in R is better? If anyone could help, I would appreciate it. Kyeongmi What do you mean when you write that rnorm was not good sometimes? Can you provide an example which demonstrates that rnorm performs poorly? Enquiring minds would like to know. Dan Daniel Nordlund Bothell, WA USA __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] IDE for R C++ package writing ?
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Hin-Tak Leung Sent: Friday, February 23, 2007 5:40 AM To: mel Cc: r-devel@r-project.org Subject: Re: [Rd] IDE for R C++ package writing ? I don't know if ess runs under xemacs, but historically, I have used ess under Xemacs on Windows. I think John Fox still has some documents and files available on the web for setting up Xemacs with ess. You can try searching the R-help archives or probably even just google Fox and Xemacs. Hope this is helpful, Dan Nordlund Bothell, WA USA xemacs (a fork of the emacs code) had windows support earlier than gnu emacs did, and obviously, it is still being worked on as the last version is December 2006. http://www.xemacs.org/Download/win32/ HTH __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Problem with tuned Rblas from CRAN with R-2.4.0
Thanks to Uwe Ligges and Andy Jaworski for their responses. I thought the Rblas I was using was from CRAN master, and I when I encountered the problem originally, I downloaded a new copy from CRAN master. However, I must have misplaced it on download and used an old copy. Long story short, the copy I had was apparently corrupted. I downloaded a new copy from CRAN, and all is functioning. Thanks again for the quick responses. Dan Daniel Nordlund Bothell, WA USA -Original Message- From: Uwe Ligges [mailto:[EMAIL PROTECTED] Sent: Tuesday, December 12, 2006 11:58 PM To: Daniel Nordlund Cc: r-devel@stat.math.ethz.ch Subject: Re: [Rd] Problem with tuned Rblas from CRAN with R-2.4.0 I just looked into the one for P4 and it does export zdrot_ and works for me. Is your one from CRAN master? Uwe Ligges Daniel Nordlund wrote: I encountered the following problem in R-2.4.0 for Windows binary downloaded from CRAN (data from R-help post by Ethan Johnsons). I was also using the contributed binary Rblas.dll for Intel P4 chip. The problem doesn't occur with the default Rblas.dll. x=c(3.05176E-05, 0.000457764, 0.003204346, 0.0138855, 0.04165649, 0.09164429, 0.1527405, 0.1963806, 0.1963806, 0.1527405, 0.09164429, 0.04165649, 0.0138855, 0.003204346, 0.000457764, 3.05176E-05) y=c(0.306, 0.0004566, 0.0031985, 0.0139083, 0.0415539, 0.0917678, 0.1528134, 0.1962831, 0.1962994, 0.1527996, 0.0917336, 0.0415497, 0.0139308, 0.0031917, 0.0004529, 0.301) fit.lm-lm(y~x) summary(fit.lm) I got the following popup window error message The procedure entry point zdrot_ could not be found in the dynamic link library Rblas.dll After closing the popup, the following was printed in the console: Error in chol2inv(Qr$qr[p1, p1, drop = FALSE]) : lapack routines cannot be loaded In addition: Warning message: unable to load shared library 'C:\R\R-2.4.1beta/modules//lapack.dll': LoadLibrary failure: The specified procedure could not be found. I subsequently downloaded R-2.4.1beta from CRAN and had the same problem with the contributed binary Rblas.dll platform i386-pc-mingw32 arch i386 os mingw32 system i386, mingw32 status beta major 2 minor 4.1 year 2006 month 12 day07 svn rev40137 language R version.string R version 2.4.1 beta (2006-12-07 r40137) OS: WinXP Pro service pack 2 I haven't had any problems with the contributed binary up thru R-2.3.1. I checked the changes file and saw that for R-2.4.0 the Rblas.dll would have to be recompiled, but also that the contributed binaries on CRAN had been updated. However, I couldn’t find any Rblas binaries with a date after 2005. Any suggestions on where to find an Rblas binary for WinXP and P4 chip, or do I need to try to compile one on my own? Thanks for any assistance, Dan Daniel Nordlund Bothell, WA USA __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Problem with tuned Rblas from CRAN with R-2.4.0
I encountered the following problem in R-2.4.0 for Windows binary downloaded from CRAN (data from R-help post by Ethan Johnsons). I was also using the contributed binary Rblas.dll for Intel P4 chip. The problem doesn't occur with the default Rblas.dll. x=c(3.05176E-05, 0.000457764, 0.003204346, 0.0138855, 0.04165649, 0.09164429, 0.1527405, 0.1963806, 0.1963806, 0.1527405, 0.09164429, 0.04165649, 0.0138855, 0.003204346, 0.000457764, 3.05176E-05) y=c(0.306, 0.0004566, 0.0031985, 0.0139083, 0.0415539, 0.0917678, 0.1528134, 0.1962831, 0.1962994, 0.1527996, 0.0917336, 0.0415497, 0.0139308, 0.0031917, 0.0004529, 0.301) fit.lm-lm(y~x) summary(fit.lm) I got the following popup window error message The procedure entry point zdrot_ could not be found in the dynamic link library Rblas.dll After closing the popup, the following was printed in the console: Error in chol2inv(Qr$qr[p1, p1, drop = FALSE]) : lapack routines cannot be loaded In addition: Warning message: unable to load shared library 'C:\R\R-2.4.1beta/modules//lapack.dll': LoadLibrary failure: The specified procedure could not be found. I subsequently downloaded R-2.4.1beta from CRAN and had the same problem with the contributed binary Rblas.dll platform i386-pc-mingw32 arch i386 os mingw32 system i386, mingw32 status beta major 2 minor 4.1 year 2006 month 12 day07 svn rev40137 language R version.string R version 2.4.1 beta (2006-12-07 r40137) OS: WinXP Pro service pack 2 I haven't had any problems with the contributed binary up thru R-2.3.1. I checked the changes file and saw that for R-2.4.0 the Rblas.dll would have to be recompiled, but also that the contributed binaries on CRAN had been updated. However, I couldn’t find any Rblas binaries with a date after 2005. Any suggestions on where to find an Rblas binary for WinXP and P4 chip, or do I need to try to compile one on my own? Thanks for any assistance, Dan Daniel Nordlund Bothell, WA USA __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
