Scala 2.11 default build

FYI - I just merged Josh's pull request to switch to Scala 2.11 as the default build. https://github.com/apache/spark/pull/10608

Spark not able to fetch events from Amazon Kinesis

Hi All, I have a quick question if anyone has experienced this here. I have been trying to get Spark read events from Kinesis recently but am having problem in receiving the events. While Spark is able to connect to Kinesis and is able to get metadata from Kinesis, Its not able to get events from

Re: Scala 2.11 default build

Nice, good work! I've been using a Docker container to compile against 2.11: https://github.com/fokko/docker-spark Cheers, Fokko 2016-01-30 9:22 GMT+01:00 Reynold Xin : > FYI - I just merged Josh's pull request to switch to Scala 2.11 as the > default build. > > https://github.com/apache/spark

Re: Scala 2.11 default build

Does this mean the following Jenkins builds can be disabled ? https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Compile/job/SPARK-master-COMPILE-MAVEN-SCALA-2.11/ https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Compile/job/SPARK-master-COMPILE-sbt-SCALA-2.11/ Cheers On Sat, Jan 3

Re: Spark not able to fetch events from Amazon Kinesis

w.r.t. protobuf-java version mismatch, I wonder if you can rebuild Spark with the following change (using maven): http://pastebin.com/fVQAYWHM Cheers On Sat, Jan 30, 2016 at 12:49 AM, Yash Sharma wrote: > Hi All, > I have a quick question if anyone has experienced this here. > > I have been tr

Re: Spark not able to fetch events from Amazon Kinesis

Thanks Ted, Rebuilding would not be possible for the setup unfortunately so just wanted to check if the version mismatch is the primary issue here. Wanted to know if anyone has hit across similar issue and how they have solved this. Thanks On Sat, Jan 30, 2016 at 10:23 PM, Ted Yu wrote: > w.r.t

Re: Spark not able to fetch events from Amazon Kinesis

Hi Yash, I've run into multiple problems due to version incompatibilities, either due to protobuf or jackson. That may be your culprit. The problem is that all failures by the Kinesis Client Lib is silent, therefore don't show up on the logs. It's very hard to debug those buggers. Best, Burak On

Proposal

Hi All, As part of my final project at university I would try to build an alternative version of k-means algorithm, it's called k-modes introduced here: Improving the Accuracy and Efficiency of the k-means Clustering Algorithm paper (Link: http://www.iaeng.org/publication/WCE2009/WCE2009_pp308

Re: Spark not able to fetch events from Amazon Kinesis

Thanks Burak, By any chance were you able to work around these errors or get the setup working ? Is there anything else that you might have tried ? Regards On Sun, Jan 31, 2016 at 4:41 AM, Burak Yavuz wrote: > Hi Yash, > > I've run into multiple problems due to version incompatibilities, either

Re: Proposal

Hi there Sounds like a fun project :) I'd recommend getting familiar with the existing k-means implementation as well as bisecting k-means in Spark, and then implementing yours based off that. You should focus on using the new ML pipelines API, and release it as a package on spark-packages.org