Re: [Rd] Suggestion: Create On-Disk Dataframes; SparkR

2017-09-04 Thread frederik
What's wrong with SparkR? I never heard of either Spark or SparkR. For on-disk dataframes there is a package called 'ff'. I looked into using it, it works well but there are some drawbacks with the implementation. I think that it should be possible to mmap an object from disk and use it as a

Re: [Rd] Suggestion: Create On-Disk Dataframes

2017-09-04 Thread Dirk Eddelbuettel
On 4 September 2017 at 11:35, Suzen, Mehmet wrote: | It is not needed. There is a large community of developer using SparkR. | https://spark.apache.org/docs/latest/sparkr.html | It does exactly what you want. I hope you are not going to mail a sparkr commercial to this list every day. As the

Re: [Rd] Suggestion: Create On-Disk Dataframes

2017-09-04 Thread Suzen, Mehmet
It is not needed. There is a large community of developer using SparkR. https://spark.apache.org/docs/latest/sparkr.html It does exactly what you want. On 3 September 2017 at 20:38, Juan Telleria wrote: > Dear R Developers, > > I would like to suggest the creation of a new

[Rd] Suggestion: Create On-Disk Dataframes

2017-09-04 Thread Juan Telleria
Dear R Developers, I would like to suggest the creation of a new S4 object class for On-Disk data.frames which do not fit in RAM memory, which could be called disk.data.frame() It could be based in rsqlite for example (By translating R syntax to SQL syntax for example), and the syntax and way of