Re: How Spark utilize low-level architecture features?

2016-01-20 Thread Boric Tan
Anyone could shed some light on this? Thanks, Boric On Tue, Jan 19, 2016 at 4:12 PM, Boric Tan wrote: > Hi there, > > I am new to Spark, and would like to get some help to understand if Spark > can utilize the underlying architectures for better performance. If so,

RE: Using CUDA within Spark / boosting linear algebra

2016-01-20 Thread Ulanov, Alexander
Hi Everyone, I’ve updated the benchmark and done experiments with new hardware with 2x Nvidia Tesla K80 (physically 4x Tesla K40) and 2x modern Haswell CPU Intel E5-2650 v3 @ 2.30GHz. This time I computed average and median of 10 runs for each of experiment and approximated FLOPS. Results

Re: spark task scheduling delay

2016-01-20 Thread Stephen Boesch
Which Resource Manager are you using? 2016-01-20 21:38 GMT-08:00 Renu Yadav : > Any suggestions? > > On Wed, Jan 20, 2016 at 6:50 PM, Renu Yadav wrote: > >> Hi , >> >> I am facing spark task scheduling delay issue in spark 1.4. >> >> suppose I have 1600

Re: Removing the Mesos fine-grained mode

2016-01-20 Thread Iulian Dragoș
That'd be great, thanks Adam! On Tue, Jan 19, 2016 at 5:41 PM, Adam McElwee wrote: > Sorry, I never got a chance to circle back with the master logs for this. > I definitely can't share the job code, since it's used to build a pretty > core dataset for my company, but let me

Re: BUILD FAILURE at Spark Project Test Tags for 2.11.7?

2016-01-20 Thread Marcelo Vanzin
On Wed, Jan 20, 2016 at 11:46 AM, Jacek Laskowski wrote: > /Users/jacek/dev/oss/spark/tags/target/scala-2.11/classes... > [error] Cannot run program "javac": error=2, No such file or directory That doesn't exactly look like a Spark problem. -- Marcelo

BUILD FAILURE at Spark Project Test Tags for 2.11.7?

2016-01-20 Thread Jacek Laskowski
Hi, Build is broken again for me :( I build for Scala 2.11.7 and use maven. Is this a known issue? Anyone looking into it? ➜ spark git:(master) ✗ ./build/mvn -Pyarn -Phadoop-2.6 -Dhadoop.version=2.7.1 -Dscala-2.11 -Phive -Phive-thriftserver -DskipTests clean install ... [INFO] ---

Re: BUILD FAILURE at Spark Project Test Tags for 2.11.7?

2016-01-20 Thread Jacek Laskowski
On Wed, Jan 20, 2016 at 8:48 PM, Marcelo Vanzin wrote: > On Wed, Jan 20, 2016 at 11:46 AM, Jacek Laskowski wrote: >> /Users/jacek/dev/oss/spark/tags/target/scala-2.11/classes... >> [error] Cannot run program "javac": error=2, No such file or directory > >

Re: BUILD FAILURE at Spark Project Test Tags for 2.11.7?

2016-01-20 Thread Jacek Laskowski
On Wed, Jan 20, 2016 at 8:48 PM, Marcelo Vanzin wrote: > On Wed, Jan 20, 2016 at 11:46 AM, Jacek Laskowski wrote: >> /Users/jacek/dev/oss/spark/tags/target/scala-2.11/classes... >> [error] Cannot run program "javac": error=2, No such file or directory > >

Re: BUILD FAILURE at Spark Project Test Tags for 2.11.7?

2016-01-20 Thread Sean Owen
That's not a Spark problem. Your compiler was not available. On Wed, Jan 20, 2016 at 10:44 PM, Jacek Laskowski wrote: > On Wed, Jan 20, 2016 at 8:48 PM, Marcelo Vanzin wrote: >> On Wed, Jan 20, 2016 at 11:46 AM, Jacek Laskowski wrote: >>>

Optimized toIndexedRowMatrix

2016-01-20 Thread Driesprong, Fokko
Hi guys, I've been working on an optimized implementation of the toIndexedRowMatrix of the BlockMatrix. I already created a ticket