RE: Spark and Play
Hi Patrick,
Although we are able to use Spark 1.1.0 with Play 2.2.x, as you mentioned, Akka
incompatibility prevents us from using Spark with the current stable releases
of Play (2.3.6) and Akka (2.3.7). Are there any plans to address this issue in
Spark 1.2?
Thanks,
Mohammed
From: John Meehan [mailto:jnmee...@gmail.com]
Sent: Tuesday, November 11, 2014 11:35 PM
To: Mohammed Guller
Cc: Patrick Wendell; Akshat Aranya; user@spark.apache.org
Subject: Re: Spark and Play
You can also build a Play 2.2.x + Spark 1.1.0 fat jar with sbt-assembly for,
e.g. yarn-client support or using with spark-shell for debugging:
play.Project.playScalaSettings
libraryDependencies ~= { _ map {
case m if m.organization == com.typesafe.play =
m.exclude(commons-logging, commons-logging)
case m = m
}}
assemblySettings
test in assembly := {}
mergeStrategy in assembly = (mergeStrategy in assembly) { (old) =
{
case m if m.toLowerCase.endsWith(manifest.mf) = MergeStrategy.discard
case m if m.startsWith(META-INF) = MergeStrategy.discard
case PathList(javax, servlet, xs @ _*) = MergeStrategy.first
case PathList(org, apache, xs @ _*) = MergeStrategy.first
case PathList(org, jboss, xs @ _*) = MergeStrategy.first
case PathList(org, slf4j, xs @ _*) = MergeStrategy.discard
case about.html = MergeStrategy.rename
case reference.conf = MergeStrategy.concat
case _ = MergeStrategy.first
}
}
On Tue, Nov 11, 2014 at 3:04 PM, Mohammed Guller
moham...@glassbeam.commailto:moham...@glassbeam.com wrote:
Actually, it is possible to integrate Spark 1.1.0 with Play 2.2.x
Here is a sample build.sbt file:
name := xyz
version := 0.1
scalaVersion := 2.10.4
libraryDependencies ++= Seq(
jdbc,
anorm,
cache,
org.apache.spark %% spark-core % 1.1.0,
com.typesafe.akka %% akka-actor % 2.2.3,
com.typesafe.akka %% akka-slf4j % 2.2.3,
org.apache.spark %% spark-sql % 1.1.0
)
play.Project.playScalaSettings
Mohammed
-Original Message-
From: Patrick Wendell [mailto:pwend...@gmail.commailto:pwend...@gmail.com]
Sent: Tuesday, November 11, 2014 2:06 PM
To: Akshat Aranya
Cc: user@spark.apache.orgmailto:user@spark.apache.org
Subject: Re: Spark and Play
Hi There,
Because Akka versions are not binary compatible with one another, it might not
be possible to integrate Play with Spark 1.1.0.
- Patrick
On Tue, Nov 11, 2014 at 8:21 AM, Akshat Aranya
aara...@gmail.commailto:aara...@gmail.com wrote:
Hi,
Sorry if this has been asked before; I didn't find a satisfactory
answer when searching. How can I integrate a Play application with
Spark? I'm getting into issues of akka-actor versions. Play 2.2.x
uses akka-actor 2.0, whereas Play 2.3.x uses akka-actor 2.3.4, neither
of which work fine with Spark 1.1.0. Is there something I should do
with libraryDependencies in my build.sbt to make it work?
Thanks,
Akshat
-
To unsubscribe, e-mail:
user-unsubscr...@spark.apache.orgmailto:user-unsubscr...@spark.apache.org For
additional commands, e-mail:
user-h...@spark.apache.orgmailto:user-h...@spark.apache.org
-
To unsubscribe, e-mail:
user-unsubscr...@spark.apache.orgmailto:user-unsubscr...@spark.apache.org
For additional commands, e-mail:
user-h...@spark.apache.orgmailto:user-h...@spark.apache.org
Re: Spark and Play
Hi Akshat,
If your application is to serve results directly from a SparkContext, you
may want to take a look at http://prediction.io. It integrates Spark with
spray.io (another REST/web toolkit by Typesafe). Some heavy lifting is done
here:
https://github.com/PredictionIO/PredictionIO/blob/develop/core/src/main/scala/workflow/CreateServer.scala
Regards,
Donald
ᐧ
On Tue, Nov 11, 2014 at 11:35 PM, John Meehan jnmee...@gmail.com wrote:
You can also build a Play 2.2.x + Spark 1.1.0 fat jar with sbt-assembly
for, e.g. yarn-client support or using with spark-shell for debugging:
play.Project.playScalaSettings
libraryDependencies ~= { _ map {
case m if m.organization == com.typesafe.play =
m.exclude(commons-logging, commons-logging)
case m = m
}}
assemblySettings
test in assembly := {}
mergeStrategy in assembly = (mergeStrategy in assembly) { (old) =
{
case m if m.toLowerCase.endsWith(manifest.mf) =
MergeStrategy.discard
case m if m.startsWith(META-INF) = MergeStrategy.discard
case PathList(javax, servlet, xs @ _*) = MergeStrategy.first
case PathList(org, apache, xs @ _*) = MergeStrategy.first
case PathList(org, jboss, xs @ _*) = MergeStrategy.first
case PathList(org, slf4j, xs @ _*) = MergeStrategy.discard
case about.html = MergeStrategy.rename
case reference.conf = MergeStrategy.concat
case _ = MergeStrategy.first
}
}
On Tue, Nov 11, 2014 at 3:04 PM, Mohammed Guller moham...@glassbeam.com
wrote:
Actually, it is possible to integrate Spark 1.1.0 with Play 2.2.x
Here is a sample build.sbt file:
name := xyz
version := 0.1
scalaVersion := 2.10.4
libraryDependencies ++= Seq(
jdbc,
anorm,
cache,
org.apache.spark %% spark-core % 1.1.0,
com.typesafe.akka %% akka-actor % 2.2.3,
com.typesafe.akka %% akka-slf4j % 2.2.3,
org.apache.spark %% spark-sql % 1.1.0
)
play.Project.playScalaSettings
Mohammed
-Original Message-
From: Patrick Wendell [mailto:pwend...@gmail.com]
Sent: Tuesday, November 11, 2014 2:06 PM
To: Akshat Aranya
Cc: user@spark.apache.org
Subject: Re: Spark and Play
Hi There,
Because Akka versions are not binary compatible with one another, it
might not be possible to integrate Play with Spark 1.1.0.
- Patrick
On Tue, Nov 11, 2014 at 8:21 AM, Akshat Aranya aara...@gmail.com wrote:
Hi,
Sorry if this has been asked before; I didn't find a satisfactory
answer when searching. How can I integrate a Play application with
Spark? I'm getting into issues of akka-actor versions. Play 2.2.x
uses akka-actor 2.0, whereas Play 2.3.x uses akka-actor 2.3.4, neither
of which work fine with Spark 1.1.0. Is there something I should do
with libraryDependencies in my build.sbt to make it work?
Thanks,
Akshat
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional
commands, e-mail: user-h...@spark.apache.org
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
--
Donald Szeto
PredictionIO
Re: Spark and Play
Hi There, Because Akka versions are not binary compatible with one another, it might not be possible to integrate Play with Spark 1.1.0. - Patrick On Tue, Nov 11, 2014 at 8:21 AM, Akshat Aranya aara...@gmail.com wrote: Hi, Sorry if this has been asked before; I didn't find a satisfactory answer when searching. How can I integrate a Play application with Spark? I'm getting into issues of akka-actor versions. Play 2.2.x uses akka-actor 2.0, whereas Play 2.3.x uses akka-actor 2.3.4, neither of which work fine with Spark 1.1.0. Is there something I should do with libraryDependencies in my build.sbt to make it work? Thanks, Akshat - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
RE: Spark and Play
Actually, it is possible to integrate Spark 1.1.0 with Play 2.2.x Here is a sample build.sbt file: name := xyz version := 0.1 scalaVersion := 2.10.4 libraryDependencies ++= Seq( jdbc, anorm, cache, org.apache.spark %% spark-core % 1.1.0, com.typesafe.akka %% akka-actor % 2.2.3, com.typesafe.akka %% akka-slf4j % 2.2.3, org.apache.spark %% spark-sql % 1.1.0 ) play.Project.playScalaSettings Mohammed -Original Message- From: Patrick Wendell [mailto:pwend...@gmail.com] Sent: Tuesday, November 11, 2014 2:06 PM To: Akshat Aranya Cc: user@spark.apache.org Subject: Re: Spark and Play Hi There, Because Akka versions are not binary compatible with one another, it might not be possible to integrate Play with Spark 1.1.0. - Patrick On Tue, Nov 11, 2014 at 8:21 AM, Akshat Aranya aara...@gmail.com wrote: Hi, Sorry if this has been asked before; I didn't find a satisfactory answer when searching. How can I integrate a Play application with Spark? I'm getting into issues of akka-actor versions. Play 2.2.x uses akka-actor 2.0, whereas Play 2.3.x uses akka-actor 2.3.4, neither of which work fine with Spark 1.1.0. Is there something I should do with libraryDependencies in my build.sbt to make it work? Thanks, Akshat - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: Spark and Play
You can also build a Play 2.2.x + Spark 1.1.0 fat jar with sbt-assembly
for, e.g. yarn-client support or using with spark-shell for debugging:
play.Project.playScalaSettings
libraryDependencies ~= { _ map {
case m if m.organization == com.typesafe.play =
m.exclude(commons-logging, commons-logging)
case m = m
}}
assemblySettings
test in assembly := {}
mergeStrategy in assembly = (mergeStrategy in assembly) { (old) =
{
case m if m.toLowerCase.endsWith(manifest.mf) = MergeStrategy.discard
case m if m.startsWith(META-INF) = MergeStrategy.discard
case PathList(javax, servlet, xs @ _*) = MergeStrategy.first
case PathList(org, apache, xs @ _*) = MergeStrategy.first
case PathList(org, jboss, xs @ _*) = MergeStrategy.first
case PathList(org, slf4j, xs @ _*) = MergeStrategy.discard
case about.html = MergeStrategy.rename
case reference.conf = MergeStrategy.concat
case _ = MergeStrategy.first
}
}
On Tue, Nov 11, 2014 at 3:04 PM, Mohammed Guller moham...@glassbeam.com
wrote:
Actually, it is possible to integrate Spark 1.1.0 with Play 2.2.x
Here is a sample build.sbt file:
name := xyz
version := 0.1
scalaVersion := 2.10.4
libraryDependencies ++= Seq(
jdbc,
anorm,
cache,
org.apache.spark %% spark-core % 1.1.0,
com.typesafe.akka %% akka-actor % 2.2.3,
com.typesafe.akka %% akka-slf4j % 2.2.3,
org.apache.spark %% spark-sql % 1.1.0
)
play.Project.playScalaSettings
Mohammed
-Original Message-
From: Patrick Wendell [mailto:pwend...@gmail.com]
Sent: Tuesday, November 11, 2014 2:06 PM
To: Akshat Aranya
Cc: user@spark.apache.org
Subject: Re: Spark and Play
Hi There,
Because Akka versions are not binary compatible with one another, it might
not be possible to integrate Play with Spark 1.1.0.
- Patrick
On Tue, Nov 11, 2014 at 8:21 AM, Akshat Aranya aara...@gmail.com wrote:
Hi,
Sorry if this has been asked before; I didn't find a satisfactory
answer when searching. How can I integrate a Play application with
Spark? I'm getting into issues of akka-actor versions. Play 2.2.x
uses akka-actor 2.0, whereas Play 2.3.x uses akka-actor 2.3.4, neither
of which work fine with Spark 1.1.0. Is there something I should do
with libraryDependencies in my build.sbt to make it work?
Thanks,
Akshat
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional
commands, e-mail: user-h...@spark.apache.org
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
