[
https://issues.apache.org/jira/browse/SPARK-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jay vyas updated SPARK-5487:
----------------------------
Description:
Building spark's custom shaed akka version is tricky. The code is in
https://github.com/pwendell/akka/ (branch = 2.2.3-shaded-proto) , however, when
attempting to build, I receive some strange errors.
I've attempted to fork off of a Dockerfile for {{SBT 0.12.4}}, which I'll
attach in a snippet just as an example of what we might want to facilitate
building the spark specific akka until SPARK-5293 is completed.
{noformat}
[info] Compiling 6 Scala sources and 1 Java source to
/tmp/akka/akka-multi-node-testkit/target/classes...
[warn] Class com.google.protobuf.MessageLite not found - continuing with a stub.
[error] error while loading ProtobufDecoder, class file
'/root/.ivy2/cache/io.netty/netty/bundles/netty-3.6.6.Final.jar(org/jboss/netty/handler/codec/protobuf/ProtobufDecoder.class)'
is broken
[error] (class java.lang.NullPointerException/null)
[error]
/tmp/akka/akka-multi-node-testkit/src/main/scala/akka/remote/testconductor/RemoteConnection.scala:24:
org.jboss.netty.handler.codec.protobuf.ProtobufDecoder does not have a
constructor
[error] val proto = List(new ProtobufEncoder, new
ProtobufDecoder(TestConductorProtocol.Wrapper.getDefaultInstance))
[error] ^
[error]
/tmp/akka/akka-multi-node-testkit/src/main/scala/akka/remote/testkit/MultiNodeSpec.scala:267:
value await is not a member of
scala.concurrent.Future[Iterable[akka.remote.testconductor.RoleName]]
[error] Note: implicit method awaitHelper is not applicable here because it
comes after the application point and it lacks an explicit result type
[error] testConductor.getNodes.await.filterNot(_ == myself).isEmpty
[error] ^
[error]
/tmp/akka/akka-multi-node-testkit/src/main/scala/akka/remote/testkit/MultiNodeSpec.scala:354:
value await is not a member of scala.concurrent.Future[akka.actor.Address]
[error] Note: implicit method awaitHelper is not applicable here because it
comes after the application point and it lacks an explicit result type
[error] def node(role: RoleName): ActorPath =
RootActorPath(testConductor.getAddressFor(role).await)
[error]
^
[warn] one warning found
[error] four errors found
[info] Updating {file:/tmp/akka/}akka-docs...
[info] Done updating.
[info] Updating {file:/tmp/akka/}akka-contrib...
[info] Done updating.
[info] Updating {file:/tmp/akka/}akka-sample-osgi-dining-hakkers-core...
[info] Done updating.
[info] Compiling 17 Scala sources to /tmp/akka/akka-cluster/target/classes...
[error]
/tmp/akka/akka-cluster/src/main/scala/akka/cluster/protobuf/ClusterMessageSerializer.scala:59:
type mismatch;
[error] found : akka.cluster.protobuf.msg.GossipEnvelope
[error] required: com.google.protobuf_spark.MessageLite
[error] case m: GossipEnvelope ? compress(gossipEnvelopeToProto(m))
[error] ^
[error]
/tmp/akka/akka-cluster/src/main/scala/akka/cluster/protobuf/ClusterMessageSerializer.scala:61:
type mismatch;
[error] found : akka.cluster.protobuf.msg.MetricsGossipEnvelope
[error] required: com.google.protobuf_spark.MessageLite
[error] case m: MetricsGossipEnvelope ?
compress(metricsGossipEnvelopeToProto(m))
[error]
^
[error]
/tmp/akka/akka-cluster/src/main/scala/akka/cluster/protobuf/ClusterMessageSerializer.scala:63:
type mismatch;
[error] found : akka.cluster.protobuf.msg.Welcome
[error] required: com.google.protobuf_spark.MessageLite
[error] case InternalClusterAction.Welcome(from, gossip) ?
compress(msg.Welcome(uniqueAddressToProto(from), gossipToProto(gossip)))
[error]
^
[error]
/tmp/akka/akka-cluster/src/main/scala/akka/cluster/protobuf/ClusterMessageSerializer.scala:257:
type mismatch;
[error] found : com.google.protobuf_spark.ByteString
[error] required: com.google.protobuf.ByteString
[error] msg.NodeMetrics.Number(msg.NodeMetrics.NumberType.Serialized,
None, None, Some(ByteString.copyFrom(bos.toByteArray)))
[error]
^
[error] four errors found
[info] Updating {file:/tmp/akka/}akka-sample-cluster...
[info] Done updating.
[info] Updating {file:/tmp/akka/}akka-sample-osgi-dining-hakkers-integration...
[info] Done updating.
[error] (akka-multi-node-testkit/compile:compile) Compilation failed
[error] (akka-cluster/compile:compile) Compilation failed
[error] Total time: 21 s, completed Jan 29, 2015 5:51:14 PM
root@c938961f45f1:/tmp/akka#
{noformat}
was:
Building spark's custom shaed akka version is tricky. The code is in
https://github.com/pwendell/akka/ (branch = 2.2.3-shaded-proto) , however, when
attempting to build, I receive some strange errors.
I've attempted to fork off of a Dockerfile for {{SBT 0.12.4}}, which I'll
attach in a snippet just as an example of what we might want to facilitate
building the spark specific akka until SPARK-5293 is completed.
{noformat}
[info] Compiling 6 Scala sources and 1 Java source to
/tmp/akka/akka-multi-node-testkit/target/classes...
[warn] Class com.google.protobuf.MessageLite not found - continuing with a stub.
[error] error while loading ProtobufDecoder, class file
'/root/.ivy2/cache/io.netty/netty/bundles/netty-3.6.6.Final.jar(org/jboss/netty/handler/codec/protobuf/ProtobufDecoder.class)'
is broken
[error] (class java.lang.NullPointerException/null)
[error]
/tmp/akka/akka-multi-node-testkit/src/main/scala/akka/remote/testconductor/RemoteConnection.scala:24:
org.jboss.netty.handler.codec.protobuf.ProtobufDecoder does not have a
constructor
[error] val proto = List(new ProtobufEncoder, new
ProtobufDecoder(TestConductorProtocol.Wrapper.getDefaultInstance))
[error] ^
[error]
/tmp/akka/akka-multi-node-testkit/src/main/scala/akka/remote/testkit/MultiNodeSpec.scala:267:
value await is not a member of
scala.concurrent.Future[Iterable[akka.remote.testconductor.RoleName]]
[error] Note: implicit method awaitHelper is not applicable here because it
comes after the application point and it lacks an explicit result type
[error] testConductor.getNodes.await.filterNot(_ == myself).isEmpty
[error] ^
[error]
/tmp/akka/akka-multi-node-testkit/src/main/scala/akka/remote/testkit/MultiNodeSpec.scala:354:
value await is not a member of scala.concurrent.Future[akka.actor.Address]
[error] Note: implicit method awaitHelper is not applicable here because it
comes after the application point and it lacks an explicit result type
[error] def node(role: RoleName): ActorPath =
RootActorPath(testConductor.getAddressFor(role).await)
[error]
^
[warn] one warning found
[error] four errors found
[info] Updating {file:/tmp/akka/}akka-docs...
[info] Done updating.
[info] Updating {file:/tmp/akka/}akka-contrib...
[info] Done updating.
[info] Updating {file:/tmp/akka/}akka-sample-osgi-dining-hakkers-core...
[info] Done updating.
[info] Compiling 17 Scala sources to /tmp/akka/akka-cluster/target/classes...
[error]
/tmp/akka/akka-cluster/src/main/scala/akka/cluster/protobuf/ClusterMessageSerializer.scala:59:
type mismatch;
[error] found : akka.cluster.protobuf.msg.GossipEnvelope
[error] required: com.google.protobuf_spark.MessageLite
[error] case m: GossipEnvelope ? compress(gossipEnvelopeToProto(m))
[error] ^
[error]
/tmp/akka/akka-cluster/src/main/scala/akka/cluster/protobuf/ClusterMessageSerializer.scala:61:
type mismatch;
[error] found : akka.cluster.protobuf.msg.MetricsGossipEnvelope
[error] required: com.google.protobuf_spark.MessageLite
[error] case m: MetricsGossipEnvelope ?
compress(metricsGossipEnvelopeToProto(m))
[error]
^
[error]
/tmp/akka/akka-cluster/src/main/scala/akka/cluster/protobuf/ClusterMessageSerializer.scala:63:
type mismatch;
[error] found : akka.cluster.protobuf.msg.Welcome
[error] required: com.google.protobuf_spark.MessageLite
[error] case InternalClusterAction.Welcome(from, gossip) ?
compress(msg.Welcome(uniqueAddressToProto(from), gossipToProto(gossip)))
[error]
^
[error]
/tmp/akka/akka-cluster/src/main/scala/akka/cluster/protobuf/ClusterMessageSerializer.scala:257:
type mismatch;
[error] found : com.google.protobuf_spark.ByteString
[error] required: com.google.protobuf.ByteString
[error] msg.NodeMetrics.Number(msg.NodeMetrics.NumberType.Serialized,
None, None, Some(ByteString.copyFrom(bos.toByteArray)))
[error]
^
[error] four errors found
[info] Updating {file:/tmp/akka/}akka-sample-cluster...
[info] Done updating.
[info] Updating {file:/tmp/akka/}akka-sample-osgi-dining-hakkers-integration...
[info] Done updating.
[error] (akka-multi-node-testkit/compile:compile) Compilation failed
[error] (akka-cluster/compile:compile) Compilation failed
[error] Total time: 21 s, completed Jan 29, 2015 5:51:14 PM
root@c938961f45f1:/tmp/akka#
> Dockerfile to build spark's custom akka.
> ----------------------------------------
>
> Key: SPARK-5487
> URL: https://issues.apache.org/jira/browse/SPARK-5487
> Project: Spark
> Issue Type: Bug
> Components: Build
> Affects Versions: 1.2.0
> Reporter: jay vyas
>
> Building spark's custom shaed akka version is tricky. The code is in
> https://github.com/pwendell/akka/ (branch = 2.2.3-shaded-proto) , however,
> when attempting to build, I receive some strange errors.
> I've attempted to fork off of a Dockerfile for {{SBT 0.12.4}}, which I'll
> attach in a snippet just as an example of what we might want to facilitate
> building the spark specific akka until SPARK-5293 is completed.
> {noformat}
> [info] Compiling 6 Scala sources and 1 Java source to
> /tmp/akka/akka-multi-node-testkit/target/classes...
> [warn] Class com.google.protobuf.MessageLite not found - continuing with a
> stub.
> [error] error while loading ProtobufDecoder, class file
> '/root/.ivy2/cache/io.netty/netty/bundles/netty-3.6.6.Final.jar(org/jboss/netty/handler/codec/protobuf/ProtobufDecoder.class)'
> is broken
> [error] (class java.lang.NullPointerException/null)
> [error]
> /tmp/akka/akka-multi-node-testkit/src/main/scala/akka/remote/testconductor/RemoteConnection.scala:24:
> org.jboss.netty.handler.codec.protobuf.ProtobufDecoder does not have a
> constructor
> [error] val proto = List(new ProtobufEncoder, new
> ProtobufDecoder(TestConductorProtocol.Wrapper.getDefaultInstance))
> [error] ^
> [error]
> /tmp/akka/akka-multi-node-testkit/src/main/scala/akka/remote/testkit/MultiNodeSpec.scala:267:
> value await is not a member of
> scala.concurrent.Future[Iterable[akka.remote.testconductor.RoleName]]
> [error] Note: implicit method awaitHelper is not applicable here because it
> comes after the application point and it lacks an explicit result type
> [error] testConductor.getNodes.await.filterNot(_ == myself).isEmpty
> [error] ^
> [error]
> /tmp/akka/akka-multi-node-testkit/src/main/scala/akka/remote/testkit/MultiNodeSpec.scala:354:
> value await is not a member of scala.concurrent.Future[akka.actor.Address]
> [error] Note: implicit method awaitHelper is not applicable here because it
> comes after the application point and it lacks an explicit result type
> [error] def node(role: RoleName): ActorPath =
> RootActorPath(testConductor.getAddressFor(role).await)
> [error]
> ^
> [warn] one warning found
> [error] four errors found
> [info] Updating {file:/tmp/akka/}akka-docs...
> [info] Done updating.
> [info] Updating {file:/tmp/akka/}akka-contrib...
> [info] Done updating.
> [info] Updating {file:/tmp/akka/}akka-sample-osgi-dining-hakkers-core...
> [info] Done updating.
> [info] Compiling 17 Scala sources to /tmp/akka/akka-cluster/target/classes...
> [error]
> /tmp/akka/akka-cluster/src/main/scala/akka/cluster/protobuf/ClusterMessageSerializer.scala:59:
> type mismatch;
> [error] found : akka.cluster.protobuf.msg.GossipEnvelope
> [error] required: com.google.protobuf_spark.MessageLite
> [error] case m: GossipEnvelope ? compress(gossipEnvelopeToProto(m))
> [error] ^
> [error]
> /tmp/akka/akka-cluster/src/main/scala/akka/cluster/protobuf/ClusterMessageSerializer.scala:61:
> type mismatch;
> [error] found : akka.cluster.protobuf.msg.MetricsGossipEnvelope
> [error] required: com.google.protobuf_spark.MessageLite
> [error] case m: MetricsGossipEnvelope ?
> compress(metricsGossipEnvelopeToProto(m))
> [error]
> ^
> [error]
> /tmp/akka/akka-cluster/src/main/scala/akka/cluster/protobuf/ClusterMessageSerializer.scala:63:
> type mismatch;
> [error] found : akka.cluster.protobuf.msg.Welcome
> [error] required: com.google.protobuf_spark.MessageLite
> [error] case InternalClusterAction.Welcome(from, gossip) ?
> compress(msg.Welcome(uniqueAddressToProto(from), gossipToProto(gossip)))
> [error]
> ^
> [error]
> /tmp/akka/akka-cluster/src/main/scala/akka/cluster/protobuf/ClusterMessageSerializer.scala:257:
> type mismatch;
> [error] found : com.google.protobuf_spark.ByteString
> [error] required: com.google.protobuf.ByteString
> [error]
> msg.NodeMetrics.Number(msg.NodeMetrics.NumberType.Serialized, None, None,
> Some(ByteString.copyFrom(bos.toByteArray)))
> [error]
> ^
> [error] four errors found
> [info] Updating {file:/tmp/akka/}akka-sample-cluster...
> [info] Done updating.
> [info] Updating
> {file:/tmp/akka/}akka-sample-osgi-dining-hakkers-integration...
> [info] Done updating.
> [error] (akka-multi-node-testkit/compile:compile) Compilation failed
> [error] (akka-cluster/compile:compile) Compilation failed
> [error] Total time: 21 s, completed Jan 29, 2015 5:51:14 PM
> root@c938961f45f1:/tmp/akka#
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
