Kristina, Thanks for the discussion. I followed up on your problem and learned that Scala doesn't support multiple implicit conversions in a single expression <http://stackoverflow.com/questions/8068346/can-scala-apply-multiple-implicit-conversions-in-one-expression> for complexity reasons. I'm afraid the solution for now is to do (v1: BV[Double]) + (v1: BV[Double])
On Tue, Aug 25, 2015 at 11:06 AM, Kristina Rogale Plazonic <kpl...@gmail.com > wrote: > YES PLEASE! > > :))))))) > > On Tue, Aug 25, 2015 at 1:57 PM, Burak Yavuz <brk...@gmail.com> wrote: > >> Hmm. I have a lot of code on the local linear algebra operations using >> Spark's Matrix and Vector representations >> done for https://issues.apache.org/jira/browse/SPARK-6442. >> >> I can make a Spark package with that code if people are interested. >> >> Best, >> Burak >> >> On Tue, Aug 25, 2015 at 10:54 AM, Kristina Rogale Plazonic < >> kpl...@gmail.com> wrote: >> >>> However I do think it's easier than it seems to write the implicits; >>>> it doesn't involve new classes or anything. Yes it's pretty much just >>>> what you wrote. There is a class "Vector" in Spark. This declaration >>>> can be in an object; you don't implement your own class. (Also you can >>>> use "toBreeze" to get Breeze vectors.) >>> >>> >>> The implicit conversion with the implicit def happens for the first >>> vector in the sum, but not the second vector (see below). >>> >>> At this point I give up, because I spent way too much time. I am so >>> disappointed. So many times I heard "Spark makes simple things easy and >>> complicated things possible". Well, here is the simplest thing you can >>> imagine in linear algebra, but heck, it is not easy or intuitive. It was >>> easier to run a DeepLearning algo (from another library) than add two >>> vectors. >>> >>> If anybody has a workaround other than implementing your own >>> add/substract/scalarMultiply, PLEASE let me know. >>> >>> Here is the code and error from (freshly started) spark-shell: >>> >>> scala> import breeze.linalg.{DenseVector => BDV, SparseVector => BSV, >>> Vector => BV} >>> import breeze.linalg.{DenseVector=>BDV, SparseVector=>BSV, Vector=>BV} >>> >>> scala> import org.apache.spark.mllib.linalg.Vectors >>> import org.apache.spark.mllib.linalg.Vectors >>> >>> scala> val v1 = Vectors.dense(1.0, 2.0, 3.0) >>> v1: org.apache.spark.mllib.linalg.Vector = [1.0,2.0,3.0] >>> >>> scala> import org.apache.spark.mllib.linalg.{Vector =>SparkVector} >>> import org.apache.spark.mllib.linalg.{Vector=>SparkVector} >>> >>> scala> object MyUtils { >>> | implicit def toBreeze(v:SparkVector) = BV(v.toArray) >>> | } >>> warning: there were 1 feature warning(s); re-run with -feature for >>> details >>> defined module MyUtils >>> >>> scala> import MyUtils._ >>> import MyUtils._ >>> >>> scala> v1:BV[Double] >>> res2: breeze.linalg.Vector[Double] = DenseVector(1.0, 2.0, 3.0) >>> >>> scala> v1 + v1 >>> <console>:30: error: could not find implicit value for parameter op: >>> breeze.linalg.operators.OpAdd.Impl2[breeze.linalg.Vector[Double],org.apache.spark.mllib.linalg.Vector,That] >>> v1 + v1 >>> ^ >>> >>> >>> >> >> >
