Additionally, at the higher level, MLlib allocates separate Breeze Vectors/Matrices on a Per-executor basis. The only place I can think of where data structures might be over-written concurrently is in a .aggregate() call, and these calls happen sequentially.
RJ - Do you have a JIRA reference for that bug? Thanks! On Wed, Sep 3, 2014 at 11:50 AM, David Hall <d...@cs.berkeley.edu> wrote: > In general, in Breeze we allocate separate work arrays for each call to > lapack, so it should be fine. In general concurrent modification isn't > thread safe of course, but things that "ought" to be thread safe really > should be. > > > On Wed, Sep 3, 2014 at 10:41 AM, RJ Nowling <rnowl...@gmail.com> wrote: > > > No, it's not in all cases. Since Breeze uses lapack under the hood, > > changes to memory between different threads is bad. > > > > There's actually a potential bug in the KMeans code where it uses += > > instead of +. > > > > > > On Wed, Sep 3, 2014 at 1:26 PM, Ulanov, Alexander < > alexander.ula...@hp.com > > > > > wrote: > > > > > Hi, > > > > > > Is breeze library called thread safe from Spark mllib code in case when > > > native libs for blas and lapack are used? Might it be an issue when > > running > > > Spark locally? > > > > > > Best regards, Alexander > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > > > For additional commands, e-mail: dev-h...@spark.apache.org > > > > > > > > > > > > -- > > em rnowl...@gmail.com > > c 954.496.2314 > > >
