ECOS Spark Integration

2017-12-17 Thread Debasish Das
Hi, ECOS is a solver for second order conic programs and we showed the Spark integration at 2014 Spark Summit https://spark-summit.org/2014/quadratic-programing-solver-for-non-negative-matrix-factorization/. Right now the examples show how to reformulate matrix factorization as a SOCP and solve

Re: flatMap() returning large class

2017-12-17 Thread Don Drake
Hey Richard, Good to hear from you as well. I thought I would ask if there was something Scala specific I was missing in handling these large classes. I can tweak my job to do a map() and then only one large object will be created at a time and returned, which should allow me to lower my

Re: How to...UNION ALL of two SELECTs over different data sources in parallel?

2017-12-17 Thread Jacek Laskowski
Thanks Silvio! In the meantime, with help of Adam and code review of WholeStageCodegenExec and CollapseCodegenStages, I found out that anything that's codegend is as fast as the tasks in a stage. In this case, union of two codegend subtrees is indeed parallel. Pozdrawiam, Jacek Laskowski

spark + SalesForce SSL HandShake Issue

2017-12-17 Thread kali.tumm...@gmail.com
Hi All, I was trying out spark + SalesforceLibabry on cloudera 5.9 I am having SSL handhshake issue please check out my question on stack over flow no one answered. The library works ok on windows it fails when I try to run on cloudera edge node.

Lucene Index with Spark Cassandra

2017-12-17 Thread Junaid Nasir
Hi everyone, I am trying to run lucene with spark but sparkSQL returns zero results, where when same query is run using cqlsh it returns correct rows. same issue as https://github.com/Stratio/cassandra-lucene-index/issues/79 I can see in spark logs that lucene is working but as mentioned in the