[julia-users] Re: Large Data Sets in Julia

2015-11-05 Thread André Lage
hi Viral, Do you have any news on this? André Lage. On Wednesday, July 3, 2013 at 5:12:06 AM UTC-3, Viral Shah wrote: > > Hi all, > > I am cross-posting my reply to julia-stats and julia-users as there was a > separate post on large logistic regressions on julia-users too. > > Just as these que

Re: [julia-users] Re: Large Data Sets in Julia

2015-11-06 Thread Tim Holy
Not sure if it's as high-level as you're hoping for, but julia has great support for arrays that are much bigger than memory. See Mmap.mmap and SharedArray(filename, T, dims). --Tim On Thursday, November 05, 2015 06:33:52 PM André Lage wrote: > hi Viral, > > Do you have any news on this? > >

Re: [julia-users] Re: Large Data Sets in Julia

2015-11-08 Thread André Lage
Thanks! André Lage. On Fri, Nov 6, 2015 at 7:20 AM, Tim Holy wrote: > Not sure if it's as high-level as you're hoping for, but julia has great > support for arrays that are much bigger than memory. See Mmap.mmap and > SharedArray(filename, T, dims). > > --Tim > > On Thursday, November 05, 2015

Re: [julia-users] Re: Large Data Sets in Julia

2015-11-09 Thread John Brock
It looks like SharedArray(filename, T, dims) isn't documented, but SharedArray(T, dims; init=false, pids=Int[]) is. What's the difference? On Friday, November 6, 2015 at 2:21:01 AM UTC-8, Tim Holy wrote: > > Not sure if it's as high-level as you're hoping for, but julia has great > support for

Re: [julia-users] Re: Large Data Sets in Julia

2015-11-11 Thread Tomas Lycken
Everything after the semicolon is keyword arguments, and will dispatch to the same method as if they are left out. Thus, the documentation for SharedArray(T, dims; init=false, pids=[]) is valid for SharedArray(T, dims) too, and the values of init and pids will be the ones given in the signatu

Re: [julia-users] Re: Large Data Sets in Julia

2015-11-11 Thread John Brock
Thanks, Tomas, but I wasn't referring to the keyword arguments. One signature starts with a filename argument, and the other doesn't. What's the difference? Is the filename specifying the location at which to create a memory mapped file? On Wednesday, November 11, 2015 at 3:33:56 AM UTC-8, Toma

Re: [julia-users] Re: Large Data Sets in Julia

2015-11-11 Thread Tim Holy
No, it's for taking an "array" that's already in a file and sharing it among workers via mmap. Docs would be great. I might contribute them, but anyone else is welcome to do so. --Tim On Wednesday, November 11, 2015 04:31:58 PM John Brock wrote: > Thanks, Tomas, but I wasn't referring to the k

Re: [julia-users] Re: Large Data Sets in Julia

2016-01-02 Thread Tim Holy
On Monday, November 09, 2015 12:43:17 PM John Brock wrote: > It looks like SharedArray(filename, T, dims) isn't documented, > but SharedArray(T, dims; init=false, pids=Int[]) is. What's the difference? See #14532. --Tim > > On Friday, November 6, 2015 at 2:21:01 AM UTC-8, Tim Holy wrote: > > No