From: Nisha Muktewar <ni...@cloudera.com>
Sent: Monday, March 26, 2018 2:27:52 PM
To: Josh Goldsborough
Cc: user
Subject: Re: [Spark R]: Linear Mixed-Effects Models in Spark R
Look at LinkedIn's Photon ML package: https://github.com/linkedin/photon-ml
One of the c
Look at LinkedIn's Photon ML package: https://github.com/linkedin/photon-ml
One of the caveats is/was that the input data has to be in Avro in a
specific format.
On Mon, Mar 26, 2018 at 1:46 PM, Josh Goldsborough <
joshgoldsboroughs...@gmail.com> wrote:
> The company I work for is trying to do
SparkR does not mean all libraries of R are executed by magic in a distributed
fashion that scales with the data. In fact that is similar to many other
analytical software. They have the possibility to run things in parallel but
the libraries themselves are not using them. Reason is that it is
The company I work for is trying to do some mixed-effects regression
modeling in our new big data platform including SparkR.
We can run via SparkR's support of native R & use lme4. But it runs single
threaded. So we're looking for tricks/techniques to process large data
sets.
This was asked a
