yes. Create working notes how exactly to do that. (Or, what i am a bit pushing you towards, Spark, since MR is not really iteration friendly platform and it looks like iterations are needed in fastICA.).
On Tue, Jan 7, 2014 at 12:38 PM, Oleksandr Olgashko < [email protected]> wrote: > So the problem is to adapt ICA for MR, am i right? > > > > 2014/1/7 Dmitriy Lyubimov <[email protected]> > > > i already looked at fast ICA. while it claims to be parallel, this work > > doesn't exactly map it into map reduce (or spark) paradigm and from what > i > > can recollect still implies outer iterations for fitting principal > > component vectors one by one. Which means it probably already is > > MR-unfriendly by construction; Spark may show far better promise here but > > still a working notes document is required to show how exactly. that's > what > > i mean. > > > > > > On Tue, Jan 7, 2014 at 1:35 AM, Oleksandr Olgashko < > > [email protected] > > > wrote: > > > > > Could you please take a look on this article? > > > http://cran.r-project.org/web/packages/fastICA/fastICA.pdf > > > I have learned that re-inventing the wheel is wrong for most problems, > > and > > > usually exists a better solution. However, it often needs some > > "grinding", > > > so I may research those ways, in case of approval. > > > > > > About Scala: unfortunately, I have never worked with this language > > before, > > > but wanted to. I'd like to fill that gap in my skills, but I don't know > > > exactly where to start. > > > > > > > > > 2014/1/7 Dmitriy Lyubimov <[email protected]> > > > > > > > ICA is a very useful technique for dimensionality reduction. I > believe > > > > Mahout would benefit from it; however challenges are fairly > significant > > > in > > > > terms of proven parallelization technique and acceptable efficacy, > > which > > > > makes it hard to just "implement" (I am not familiar at this point > with > > > any > > > > concrete work on parallel ICA). So like i said before i am not very > > > > hopeful. However, if one never tries, then nothing will get ever > done. > > > who > > > > knows. > > > > > > > > > > > > On Mon, Jan 6, 2014 at 2:18 PM, Isabel Drost-Fromm < > [email protected] > > > > >wrote: > > > > > > > > > On Mon, Jan 06, 2014 at 10:40:45PM +0200, Oleksandr Olgashko wrote: > > > > > > Returning back to question about theme to work, asked 2 months > ago. > > > > > > What algorithm should I implement? > > > > > > > > > > To be quite frank with you: None. Personally I'd rather see > > > improvements > > > > > (in terms of documentation, integration, stableisation, performance > > > > > optimisation) of the existing Mahout source. > > > > > > > > > > Feel free to take a closer look at the thread concerning "getting > > > > > involved" that we had around Christmas last year for inspiration. > > > > > > > > > > > > > > > Isabel > > > > > > > > > > > > > > > > > > > >
