Re: mllib.linalg.Vectors vs Breeze?
Oops yes it is. Been working inside the Spark packages too long. Ignore that comment. On Oct 19, 2014 1:42 AM, "Matei Zaharia" wrote: > toBreeze is private within Spark, it should not be accessible to users. If > you want to make a Breeze vector from an MLlib one, it's pretty > straightforward, and you can make your own utility function for it. > > Matei > > > > On Oct 17, 2014, at 5:09 PM, Sean Owen wrote: > > > > Yes, I think that's the logic, but then what do toBreezeVector return > > if it is not based on Breeze? and this is called a lot by client code > > since you often have to do something nontrivial to the vector. I > > suppose you can still have that thing return a Breeze vector and use > > it for no other purpose. I think this abstraction leaks though. > > > > On Fri, Oct 17, 2014 at 7:48 PM, Nicholas Chammas > > wrote: > >> I don't know the answer for sure, but just from an API perspective I'd > guess > >> that the Spark authors don't want to tie their API to Breeze. If at a > future > >> point they swap out a different implementation for Breeze, they don't > have > >> to change their public interface. MLlib's interface remains consistent > while > >> the internals are free to evolve. > >> > >> Nick > >> > >> > >> 2014년 10월 17일 금요일, ll님이 작성한 메시지: > >> > >>> hello... i'm looking at the source code for mllib.linalg.Vectors and it > >>> looks > >>> like it's a wrapper around Breeze with very small changes (mostly > changing > >>> the names). > >>> > >>> i don't have any problem with using spark wrapper around Breeze or > Breeze > >>> directly. i'm just curious to understand why this wrapper was created > vs. > >>> pointing everyone to Breeze directly? > >>> > >>> > >>> > https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala > >>> > >>> > >>> > >>> -- > >>> View this message in context: > >>> > http://apache-spark-user-list.1001560.n3.nabble.com/mllib-linalg-Vectors-vs-Breeze-tp16722.html > >>> Sent from the Apache Spark User List mailing list archive at > Nabble.com. > >>> > >>> - > >>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > >>> For additional commands, e-mail: user-h...@spark.apache.org > >>> > >> > > > > - > > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > > For additional commands, e-mail: user-h...@spark.apache.org > > > >
Re: mllib.linalg.Vectors vs Breeze?
toBreeze is private within Spark, it should not be accessible to users. If you want to make a Breeze vector from an MLlib one, it's pretty straightforward, and you can make your own utility function for it. Matei > On Oct 17, 2014, at 5:09 PM, Sean Owen wrote: > > Yes, I think that's the logic, but then what do toBreezeVector return > if it is not based on Breeze? and this is called a lot by client code > since you often have to do something nontrivial to the vector. I > suppose you can still have that thing return a Breeze vector and use > it for no other purpose. I think this abstraction leaks though. > > On Fri, Oct 17, 2014 at 7:48 PM, Nicholas Chammas > wrote: >> I don't know the answer for sure, but just from an API perspective I'd guess >> that the Spark authors don't want to tie their API to Breeze. If at a future >> point they swap out a different implementation for Breeze, they don't have >> to change their public interface. MLlib's interface remains consistent while >> the internals are free to evolve. >> >> Nick >> >> >> 2014년 10월 17일 금요일, ll님이 작성한 메시지: >> >>> hello... i'm looking at the source code for mllib.linalg.Vectors and it >>> looks >>> like it's a wrapper around Breeze with very small changes (mostly changing >>> the names). >>> >>> i don't have any problem with using spark wrapper around Breeze or Breeze >>> directly. i'm just curious to understand why this wrapper was created vs. >>> pointing everyone to Breeze directly? >>> >>> >>> https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-spark-user-list.1001560.n3.nabble.com/mllib-linalg-Vectors-vs-Breeze-tp16722.html >>> Sent from the Apache Spark User List mailing list archive at Nabble.com. >>> >>> - >>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>> For additional commands, e-mail: user-h...@spark.apache.org >>> >> > > - > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: mllib.linalg.Vectors vs Breeze?
Yes, I think that's the logic, but then what do toBreezeVector return if it is not based on Breeze? and this is called a lot by client code since you often have to do something nontrivial to the vector. I suppose you can still have that thing return a Breeze vector and use it for no other purpose. I think this abstraction leaks though. On Fri, Oct 17, 2014 at 7:48 PM, Nicholas Chammas wrote: > I don't know the answer for sure, but just from an API perspective I'd guess > that the Spark authors don't want to tie their API to Breeze. If at a future > point they swap out a different implementation for Breeze, they don't have > to change their public interface. MLlib's interface remains consistent while > the internals are free to evolve. > > Nick > > > 2014년 10월 17일 금요일, ll님이 작성한 메시지: > >> hello... i'm looking at the source code for mllib.linalg.Vectors and it >> looks >> like it's a wrapper around Breeze with very small changes (mostly changing >> the names). >> >> i don't have any problem with using spark wrapper around Breeze or Breeze >> directly. i'm just curious to understand why this wrapper was created vs. >> pointing everyone to Breeze directly? >> >> >> https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/mllib-linalg-Vectors-vs-Breeze-tp16722.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> - >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> > - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: mllib.linalg.Vectors vs Breeze?
I don't know the answer for sure, but just from an API perspective I'd guess that the Spark authors don't want to tie their API to Breeze. If at a future point they swap out a different implementation for Breeze, they don't have to change their public interface. MLlib's interface remains consistent while the internals are free to evolve. Nick 2014년 10월 17일 금요일, ll님이 작성한 메시지: > hello... i'm looking at the source code for mllib.linalg.Vectors and it > looks > like it's a wrapper around Breeze with very small changes (mostly changing > the names). > > i don't have any problem with using spark wrapper around Breeze or Breeze > directly. i'm just curious to understand why this wrapper was created vs. > pointing everyone to Breeze directly? > > > https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/mllib-linalg-Vectors-vs-Breeze-tp16722.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > - > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
