[jira] [Commented] (SPARK-2805) update akka to version 2.3
[ https://issues.apache.org/jira/browse/SPARK-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14108800#comment-14108800 ] Anand Avati commented on SPARK-2805: [~pwendell] ping > update akka to version 2.3 > -- > > Key: SPARK-2805 > URL: https://issues.apache.org/jira/browse/SPARK-2805 > Project: Spark > Issue Type: Sub-task > Components: Build, Spark Core >Reporter: Anand Avati > > akka-2.3 is the lowest version available in Scala 2.11 > akka-2.3 depends on protobuf 2.5. Hadoop-1 requires protobuf 2.4.1. In order > to reconcile the conflicting dependencies, need to release > akka-2.3.x-shaded-protobuf artifact which has protobuf 2.5 within. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-2812) convert maven to archetype based build
[ https://issues.apache.org/jira/browse/SPARK-2812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14091561#comment-14091561 ] Anand Avati commented on SPARK-2812: According to http://maven.apache.org/archetype/maven-archetype-plugin/examples/create-multi-module-project.html archetype creation should work for multi module project with no extra steps. However while following the steps, 'mvn archetype:create-from-project' seems to succeed (not sure if it has done the right thing, but the command succeeds) but the next step of 'mvn install' fails with the following error: [ERROR] Archetype IT 'basic' failed: org.apache.maven.archetype.exception.ArchetypeGenerationFailure: Error merging velocity templates: Unable to find resource 'archetype-resources/spark-core_2.10/pom.xml' [ERROR] -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-archetype-plugin:2.2:integration-test (default-integration-test) on project spark-parent-archetype: Archetype IT 'basic' failed: org.apache.maven.archetype.exception.ArchetypeGenerationFailure: Error merging velocity templates: Unable to find resource 'archetype-resources/spark-core_2.10/pom.xml' >From the error it appears that it expects modules to be in a sub-directory >with the name of the artifact? On the other hand solving this problem with little shell scripting to re-write pom.xml would be quite simple. Would it be OK if the first step is a script based cross building, and figure out a maven based cross build over time? > convert maven to archetype based build > -- > > Key: SPARK-2812 > URL: https://issues.apache.org/jira/browse/SPARK-2812 > Project: Spark > Issue Type: Sub-task > Components: Build, Spark Core >Reporter: Anand Avati > > In order to support Scala 2.10 and 2.11 parallel builds. > Build profile in pom.xml is insufficient as it is not possible to have > expressions/variables in artifact name of sub-modules. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-2805) update akka to version 2.3
[ https://issues.apache.org/jira/browse/SPARK-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14091195#comment-14091195 ] Anand Avati commented on SPARK-2805: [~pwend...@gmail.com] ping > update akka to version 2.3 > -- > > Key: SPARK-2805 > URL: https://issues.apache.org/jira/browse/SPARK-2805 > Project: Spark > Issue Type: Sub-task > Components: Build, Spark Core >Reporter: Anand Avati > > akka-2.3 is the lowest version available in Scala 2.11 > akka-2.3 depends on protobuf 2.5. Hadoop-1 requires protobuf 2.4.1. In order > to reconcile the conflicting dependencies, need to release > akka-2.3.x-shaded-protobuf artifact which has protobuf 2.5 within. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-2924) Remove use of default arguments where disallowed by 2.11
[ https://issues.apache.org/jira/browse/SPARK-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14091193#comment-14091193 ] Anand Avati commented on SPARK-2924: PR: https://github.com/apache/spark/pull/1704 > Remove use of default arguments where disallowed by 2.11 > > > Key: SPARK-2924 > URL: https://issues.apache.org/jira/browse/SPARK-2924 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: Patrick Wendell >Assignee: Anand Avati >Priority: Blocker > -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-2806) update json4s-jackson to version 3.2.10
[ https://issues.apache.org/jira/browse/SPARK-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Avati resolved SPARK-2806. Resolution: Fixed PR merged > update json4s-jackson to version 3.2.10 > --- > > Key: SPARK-2806 > URL: https://issues.apache.org/jira/browse/SPARK-2806 > Project: Spark > Issue Type: Sub-task > Components: Build, Spark Core >Reporter: Anand Avati >Assignee: Anand Avati > > json4s-jackson 3.2.6 is not available in Scala 2.11 -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-2809) update chill to version 0.4
[ https://issues.apache.org/jira/browse/SPARK-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087365#comment-14087365 ] Anand Avati commented on SPARK-2809: User group post requesting Scala 2.11 support - https://groups.google.com/forum/#!topic/chill-user/7rldCq6FzQo > update chill to version 0.4 > --- > > Key: SPARK-2809 > URL: https://issues.apache.org/jira/browse/SPARK-2809 > Project: Spark > Issue Type: Sub-task > Components: Build, Spark Core >Reporter: Anand Avati > > First twitter chill_2.11 0.4 has to be released -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-2805) update akka to version 2.3
[ https://issues.apache.org/jira/browse/SPARK-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087355#comment-14087355 ] Anand Avati commented on SPARK-2805: Shaded packages can be built from scripts in https://github.com/avati/spark-shaded. Requesting someone with credentials to org.spark-project in maven to build/publish the artifacts from that script. > update akka to version 2.3 > -- > > Key: SPARK-2805 > URL: https://issues.apache.org/jira/browse/SPARK-2805 > Project: Spark > Issue Type: Sub-task > Components: Build, Spark Core >Reporter: Anand Avati > > akka-2.3 is the lowest version available in Scala 2.11 > akka-2.3 depends on protobuf 2.5. Hadoop-1 requires protobuf 2.4.1. In order > to reconcile the conflicting dependencies, need to release > akka-2.3.x-shaded-protobuf artifact which has protobuf 2.5 within. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-2807) update akka-zeromq to version 2.3.4
[ https://issues.apache.org/jira/browse/SPARK-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Avati resolved SPARK-2807. Resolution: Duplicate Shaded packages built for SPARK-2805 solves this issue > update akka-zeromq to version 2.3.4 > --- > > Key: SPARK-2807 > URL: https://issues.apache.org/jira/browse/SPARK-2807 > Project: Spark > Issue Type: Sub-task > Components: Build, Spark Core >Reporter: Anand Avati > > First akka-zeromq_2.11 2.3.4 has to be released -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-2812) convert maven to archetype based build
[ https://issues.apache.org/jira/browse/SPARK-2812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14084963#comment-14084963 ] Anand Avati commented on SPARK-2812: I took Mark's suggestion to use maven-archetype-plugin in order to support multi Scala versions. I am new to it as well. > convert maven to archetype based build > -- > > Key: SPARK-2812 > URL: https://issues.apache.org/jira/browse/SPARK-2812 > Project: Spark > Issue Type: Sub-task > Components: Build, Spark Core >Reporter: Anand Avati > > In order to support Scala 2.10 and 2.11 parallel builds. > Build profile in pom.xml is insufficient as it is not possible to have > expressions/variables in artifact name of sub-modules. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-1997) Update breeze to version 0.8.1
[
https://issues.apache.org/jira/browse/SPARK-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14083843#comment-14083843
]
Anand Avati commented on SPARK-1997:
The reason for upgrade is not for 1.1. It is for Scala 2.11 porting. Only
0.8.1 breeze is available in Scala 2.11. So considering all else being
equal, this is a reason to move to 0.8.1 (at some point eventually).
This update is for master only, not to be back ported to branch-1.1
Thanks
> Update breeze to version 0.8.1
> --
>
> Key: SPARK-1997
> URL: https://issues.apache.org/jira/browse/SPARK-1997
> Project: Spark
> Issue Type: Sub-task
> Components: MLlib
>Reporter: Guoqiang Li
>Assignee: Guoqiang Li
> Fix For: 1.1.0
>
>
> {{breeze 0.7}} does not support {{scala 2.11}} .
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-1997) Update breeze to version 0.8.1
[
https://issues.apache.org/jira/browse/SPARK-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14083823#comment-14083823
]
Anand Avati commented on SPARK-1997:
[~mengxr] I have re-posted the same #940 patch here -
https://github.com/apache/spark/pull/1749
Please let us know if there are any further concerns.
> Update breeze to version 0.8.1
> --
>
> Key: SPARK-1997
> URL: https://issues.apache.org/jira/browse/SPARK-1997
> Project: Spark
> Issue Type: Sub-task
> Components: MLlib
>Reporter: Guoqiang Li
>Assignee: Guoqiang Li
> Fix For: 1.1.0
>
>
> {{breeze 0.7}} does not support {{scala 2.11}} .
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-2805) update akka to version 2.3
[ https://issues.apache.org/jira/browse/SPARK-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14083813#comment-14083813 ] Anand Avati commented on SPARK-2805: Posted https://github.com/apache/spark/pull/1685 > update akka to version 2.3 > -- > > Key: SPARK-2805 > URL: https://issues.apache.org/jira/browse/SPARK-2805 > Project: Spark > Issue Type: Sub-task > Components: Build, Spark Core >Reporter: Anand Avati > > akka-2.3 is the lowest version available in Scala 2.11 > akka-2.3 depends on protobuf 2.5. Hadoop-1 requires protobuf 2.4.1. In order > to reconcile the conflicting dependencies, need to release > akka-2.3.x-shaded-protobuf artifact which has protobuf 2.5 within. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-2806) update json4s-jackson to version 3.2.10
[ https://issues.apache.org/jira/browse/SPARK-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14083814#comment-14083814 ] Anand Avati commented on SPARK-2806: Posted https://github.com/apache/spark/pull/1702 > update json4s-jackson to version 3.2.10 > --- > > Key: SPARK-2806 > URL: https://issues.apache.org/jira/browse/SPARK-2806 > Project: Spark > Issue Type: Sub-task > Components: Build, Spark Core >Reporter: Anand Avati > > json4s-jackson 3.2.6 is not available in Scala 2.11 -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-2810) update scala-maven-plugin to version 3.2.0
[ https://issues.apache.org/jira/browse/SPARK-2810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14083817#comment-14083817 ] Anand Avati commented on SPARK-2810: Posted https://github.com/apache/spark/pull/1711 > update scala-maven-plugin to version 3.2.0 > -- > > Key: SPARK-2810 > URL: https://issues.apache.org/jira/browse/SPARK-2810 > Project: Spark > Issue Type: Sub-task > Components: Build, Spark Core >Reporter: Anand Avati > > Needed for Scala 2.11 'compiler-interface' -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-1812) Support cross-building with Scala 2.11
[ https://issues.apache.org/jira/browse/SPARK-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14083812#comment-14083812 ] Anand Avati commented on SPARK-1812: [~pwendell] I have created sub-tasks for the dependencies and build. All source code changes (warning and error fixes due to 2.11 language changes) will continue against this Jira. > Support cross-building with Scala 2.11 > -- > > Key: SPARK-1812 > URL: https://issues.apache.org/jira/browse/SPARK-1812 > Project: Spark > Issue Type: New Feature > Components: Build, Spark Core >Reporter: Matei Zaharia >Assignee: Prashant Sharma > > Since Scala 2.10/2.11 are source compatible, we should be able to cross build > for both versions. From what I understand there are basically three things we > need to figure out: > 1. Have a two versions of our dependency graph, one that uses 2.11 > dependencies and the other that uses 2.10 dependencies. > 2. Figure out how to publish different poms for 2.10 and 2.11. > I think (1) can be accomplished by having a scala 2.11 profile. (2) isn't > really well supported by Maven since published pom's aren't generated > dynamically. But we can probably script around it to make it work. I've done > some initial sanity checks with a simple build here: > https://github.com/pwendell/scala-maven-crossbuild -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-1997) Update breeze to version 0.8.1
[
https://issues.apache.org/jira/browse/SPARK-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14083811#comment-14083811
]
Anand Avati commented on SPARK-1997:
Ah, you are right, scalalogging was removed, not updated.
In any case can we update to breeze 0.8.1 with the old #940 patch (the goal of
this jira)? The logging changes you suggest to breeze seems orthogonal to 0.7
vs 0.8.1?
> Update breeze to version 0.8.1
> --
>
> Key: SPARK-1997
> URL: https://issues.apache.org/jira/browse/SPARK-1997
> Project: Spark
> Issue Type: Sub-task
> Components: MLlib
>Reporter: Guoqiang Li
>Assignee: Guoqiang Li
> Fix For: 1.1.0
>
>
> {{breeze 0.7}} does not support {{scala 2.11}} .
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-2812) convert maven to archetype based build
Anand Avati created SPARK-2812: -- Summary: convert maven to archetype based build Key: SPARK-2812 URL: https://issues.apache.org/jira/browse/SPARK-2812 Project: Spark Issue Type: Sub-task Reporter: Anand Avati In order to support Scala 2.10 and 2.11 parallel builds. Build profile in pom.xml is insufficient as it is not possible to have expressions/variables in artifact name of sub-modules. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-2809) update chill to version 0.4
[ https://issues.apache.org/jira/browse/SPARK-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14083806#comment-14083806 ] Anand Avati commented on SPARK-2809: [~srowen], spark is currently using twitter chill 0.3.6. Scala 2.11 based chill is likely going to in version 0.4 (the latest 2.10 based release) or later, and therefore will need update of spark's pom.xml in any case. > update chill to version 0.4 > --- > > Key: SPARK-2809 > URL: https://issues.apache.org/jira/browse/SPARK-2809 > Project: Spark > Issue Type: Sub-task > Components: Build, Spark Core >Reporter: Anand Avati > > First twitter chill_2.11 0.4 has to be released -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-2807) update akka-zeromq to version 2.3.4
[ https://issues.apache.org/jira/browse/SPARK-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Avati updated SPARK-2807: --- Summary: update akka-zeromq to version 2.3.4 (was: update akka-zeromq_2.11 to version 2.3.4) > update akka-zeromq to version 2.3.4 > --- > > Key: SPARK-2807 > URL: https://issues.apache.org/jira/browse/SPARK-2807 > Project: Spark > Issue Type: Sub-task > Components: Build, Spark Core >Reporter: Anand Avati > > First akka-zeromq_2.11 2.3.4 has to be released -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-2811) update algebird to 0.7
Anand Avati created SPARK-2811: -- Summary: update algebird to 0.7 Key: SPARK-2811 URL: https://issues.apache.org/jira/browse/SPARK-2811 Project: Spark Issue Type: Sub-task Reporter: Anand Avati First algebird_2.11 0.7.0 has to be released -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-2810) update scala-maven-plugin to version 3.2.0
Anand Avati created SPARK-2810: -- Summary: update scala-maven-plugin to version 3.2.0 Key: SPARK-2810 URL: https://issues.apache.org/jira/browse/SPARK-2810 Project: Spark Issue Type: Sub-task Reporter: Anand Avati Needed for Scala 2.11 'compiler-interface' -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-2809) update chill to version 0.4
Anand Avati created SPARK-2809: -- Summary: update chill to version 0.4 Key: SPARK-2809 URL: https://issues.apache.org/jira/browse/SPARK-2809 Project: Spark Issue Type: Sub-task Reporter: Anand Avati First twitter chill_2.11 0.4 has to be released -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-2808) update kafka to version 0.8.1
Anand Avati created SPARK-2808: -- Summary: update kafka to version 0.8.1 Key: SPARK-2808 URL: https://issues.apache.org/jira/browse/SPARK-2808 Project: Spark Issue Type: Sub-task Reporter: Anand Avati First kafka_2.11 0.8.1 has to be released -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-2807) update kafka to version 0.8.1
[ https://issues.apache.org/jira/browse/SPARK-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Avati updated SPARK-2807: --- Summary: update kafka to version 0.8.1 (was: update kafka to version 2.3) > update kafka to version 0.8.1 > - > > Key: SPARK-2807 > URL: https://issues.apache.org/jira/browse/SPARK-2807 > Project: Spark > Issue Type: Sub-task > Components: Build, Spark Core >Reporter: Anand Avati > > First akka-zeromq_2.11 2.3.4 has to be released -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-2807) update kafka to version 2.3
Anand Avati created SPARK-2807: -- Summary: update kafka to version 2.3 Key: SPARK-2807 URL: https://issues.apache.org/jira/browse/SPARK-2807 Project: Spark Issue Type: Sub-task Reporter: Anand Avati First akka-zeromq_2.11 2.3.4 has to be released -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-2807) update akka-zeromq_2.11 to version 2.3.4
[ https://issues.apache.org/jira/browse/SPARK-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Avati updated SPARK-2807: --- Summary: update akka-zeromq_2.11 to version 2.3.4 (was: update kafka to version 0.8.1) > update akka-zeromq_2.11 to version 2.3.4 > > > Key: SPARK-2807 > URL: https://issues.apache.org/jira/browse/SPARK-2807 > Project: Spark > Issue Type: Sub-task > Components: Build, Spark Core >Reporter: Anand Avati > > First akka-zeromq_2.11 2.3.4 has to be released -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-2806) update json4s-jackson to version 3.2.10
Anand Avati created SPARK-2806: -- Summary: update json4s-jackson to version 3.2.10 Key: SPARK-2806 URL: https://issues.apache.org/jira/browse/SPARK-2806 Project: Spark Issue Type: Sub-task Reporter: Anand Avati json4s-jackson 3.2.6 is not available in Scala 2.11 -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-1997) Update breeze to version 0.8.1
[
https://issues.apache.org/jira/browse/SPARK-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14083784#comment-14083784
]
Anand Avati commented on SPARK-1997:
Please re-open this issue as the patch was reverted. The same patch is good for
re-apply now.
Note that scalalogging is already upgraded to 2.1.2 in master after branch-1.1,
and therefore #940 needs to be re-applied to master soon.
> Update breeze to version 0.8.1
> --
>
> Key: SPARK-1997
> URL: https://issues.apache.org/jira/browse/SPARK-1997
> Project: Spark
> Issue Type: Sub-task
> Components: MLlib
>Reporter: Guoqiang Li
>Assignee: Guoqiang Li
> Fix For: 1.1.0
>
>
> {{breeze 0.7}} does not support {{scala 2.11}} .
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-2805) update akka to version 2.3
Anand Avati created SPARK-2805: -- Summary: update akka to version 2.3 Key: SPARK-2805 URL: https://issues.apache.org/jira/browse/SPARK-2805 Project: Spark Issue Type: Sub-task Reporter: Anand Avati akka-2.3 is the lowest version available in Scala 2.11 akka-2.3 depends on protobuf 2.5. Hadoop-1 requires protobuf 2.4.1. In order to reconcile the conflicting dependencies, need to release akka-2.3.x-shaded-protobuf artifact which has protobuf 2.5 within. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-1812) Support cross-building with Scala 2.11
[ https://issues.apache.org/jira/browse/SPARK-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14083749#comment-14083749 ] Anand Avati commented on SPARK-1812: [~pwendell], sure I will create sub-tasks. Was not aware of this workflow requirement, sorry. > Support cross-building with Scala 2.11 > -- > > Key: SPARK-1812 > URL: https://issues.apache.org/jira/browse/SPARK-1812 > Project: Spark > Issue Type: New Feature > Components: Build, Spark Core >Reporter: Matei Zaharia >Assignee: Prashant Sharma > > Since Scala 2.10/2.11 are source compatible, we should be able to cross build > for both versions. From what I understand there are basically three things we > need to figure out: > 1. Have a two versions of our dependency graph, one that uses 2.11 > dependencies and the other that uses 2.10 dependencies. > 2. Figure out how to publish different poms for 2.10 and 2.11. > I think (1) can be accomplished by having a scala 2.11 profile. (2) isn't > really well supported by Maven since published pom's aren't generated > dynamically. But we can probably script around it to make it work. I've done > some initial sanity checks with a simple build here: > https://github.com/pwendell/scala-maven-crossbuild -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-1842) update scala-logging-slf4j to version 2.1.2
[ https://issues.apache.org/jira/browse/SPARK-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081863#comment-14081863 ] Anand Avati commented on SPARK-1842: Posted https://github.com/apache/spark/pull/1701 > update scala-logging-slf4j to version 2.1.2 > --- > > Key: SPARK-1842 > URL: https://issues.apache.org/jira/browse/SPARK-1842 > Project: Spark > Issue Type: Sub-task > Components: Spark Core >Reporter: Guoqiang Li >Assignee: Guoqiang Li > > scala-logging-slf4j 1.0.1 not support Scala 2.11 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SPARK-2707) Upgrade to Akka 2.3
[ https://issues.apache.org/jira/browse/SPARK-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14079487#comment-14079487 ] Anand Avati commented on SPARK-2707: [~helena_e] - it turned out to be more than just a timeout issue. As described in SPARK-1812 and https://groups.google.com/forum/#!topic/akka-user/cI4CEKEJvfs, this is because of protobuf version mismatch. The combination of https://github.com/avati/spark/commit/f8b5e96fca20c13308cb2a9a6c18049bcdd0a7ba and https://github.com/avati/spark/commit/722aee26399b9bf4b725d17f5cfcfad99464af35 is making akka-2.3 work for me. > Upgrade to Akka 2.3 > --- > > Key: SPARK-2707 > URL: https://issues.apache.org/jira/browse/SPARK-2707 > Project: Spark > Issue Type: New Feature > Components: Spark Core >Affects Versions: 1.0.0 >Reporter: Yardena > > Upgrade Akka from 2.2 to 2.3. We want to be able to use new Akka and Spray > features directly in the same project. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SPARK-1812) Support cross-building with Scala 2.11
[ https://issues.apache.org/jira/browse/SPARK-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078912#comment-14078912 ] Anand Avati commented on SPARK-1812: Figured out. Having two different $protobuf.version in pom.xml (2.4.1 and 2.5.0) and being used for different components seems to be the root cause for akka-2.3 related exceptions. Using 2.5.0 everywhere consistently is making akka-2.3 start working. > Support cross-building with Scala 2.11 > -- > > Key: SPARK-1812 > URL: https://issues.apache.org/jira/browse/SPARK-1812 > Project: Spark > Issue Type: New Feature > Components: Build, Spark Core >Reporter: Matei Zaharia >Assignee: Prashant Sharma > > Since Scala 2.10/2.11 are source compatible, we should be able to cross build > for both versions. From what I understand there are basically three things we > need to figure out: > 1. Have a two versions of our dependency graph, one that uses 2.11 > dependencies and the other that uses 2.10 dependencies. > 2. Figure out how to publish different poms for 2.10 and 2.11. > I think (1) can be accomplished by having a scala 2.11 profile. (2) isn't > really well supported by Maven since published pom's aren't generated > dynamically. But we can probably script around it to make it work. I've done > some initial sanity checks with a simple build here: > https://github.com/pwendell/scala-maven-crossbuild -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SPARK-1812) Support cross-building with Scala 2.11
[ https://issues.apache.org/jira/browse/SPARK-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078887#comment-14078887 ] Anand Avati commented on SPARK-1812: So the akka-2.3.x incompatibility turns out to be because of: java.lang.VerifyError: class akka.remote.WireFormats$AkkaControlMessage overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet; at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:800) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at akka.remote.transport.AkkaPduProtobufCodec$.constructControlMessagePdu(AkkaPduCodec.scala:231) at akka.remote.transport.AkkaPduProtobufCodec$.(AkkaPduCodec.scala:153) at akka.remote.transport.AkkaPduProtobufCodec$.(AkkaPduCodec.scala) at akka.remote.EndpointManager$$anonfun$9.apply(Remoting.scala:714) at akka.remote.EndpointManager$$anonfun$9.apply(Remoting.scala:684) at scala.collection.TraversableLike$WithFilter$$anonfun$map$2.apply(TraversableLike.scala:728) at scala.collection.Iterator$class.foreach(Iterator.scala:743) at scala.collection.AbstractIterator.foreach(Iterator.scala:1174) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at scala.collection.TraversableLike$WithFilter.map(TraversableLike.scala:727) at akka.remote.EndpointManager.akka$remote$EndpointManager$$listens(Remoting.scala:684) at akka.remote.EndpointManager$$anonfun$receive$2.applyOrElse(Remoting.scala:492) at akka.actor.Actor$class.aroundReceive(Actor.scala:465) at akka.remote.EndpointManager.aroundReceive(Remoting.scala:395) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) at akka.actor.ActorCell.invoke(ActorCell.scala:487) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238) at akka.dispatch.Mailbox.run(Mailbox.scala:220) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) I'm not yet familiar what the previous akka-2.2.3-shaded-protobuf was doing under the covers, but I'm guessing that using a vanilla (i.e non -shaded-protobuf) akka-2.3 is making someone not happy. Odd thing, vanilla akka-2.2.3 works fine, just akka-2.3.x seems to have a problem with non -shaded-protobuf. Any pointers will be helpful while I dig further. > Support cross-building with Scala 2.11 > -- > > Key: SPARK-1812 > URL: https://issues.apache.org/jira/browse/SPARK-1812 > Project: Spark > Issue Type: New Feature > Components: Build, Spark Core >Reporter: Matei Zaharia >Assignee: Prashant Sharma > > Since Scala 2.10/2.11 are source compatible, we should be able to cross build > for both versions. From what I understand there are basically three things we > need to figure out: > 1. Have a two versions of our dependency graph, one that uses 2.11 > dependencies and the other that uses 2.10 dependencies. > 2. Figure out how to publish different poms for 2.10 and 2.11. > I think (1) can be accomplished by having a scala 2.11 profile. (2) isn't > really well supported by Maven since published pom's aren't generated > dynamically. But we can probably script around it to make it work. I've done > some initial sanity checks with a simple build here: > https://github.com/pwendell/scala-maven-crossbuild -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SPARK-2707) Upgrade to Akka 2.3
[ https://issues.apache.org/jira/browse/SPARK-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14075796#comment-14075796 ] Anand Avati commented on SPARK-2707: The changes to just get compiled with 2.3.x can be found here - https://github.com/avati/spark/commit/000441bfec9315d1132cd9b785791a6fcbf9d4d4. However that does not work, and "new SparkContext" keeps throwing: java.util.concurrent.TimeoutException: Futures timed out after [1 milliseconds] at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219) at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223) at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107) at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53) at scala.concurrent.Await$.result(package.scala:107) at akka.remote.Remoting.start(Remoting.scala:180) at akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:184) at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:618) at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:615) at akka.actor.ActorSystemImpl._start(ActorSystem.scala:615) I am still investigating what other changes are needed in spark for akka 2.3.x to "work" > Upgrade to Akka 2.3 > --- > > Key: SPARK-2707 > URL: https://issues.apache.org/jira/browse/SPARK-2707 > Project: Spark > Issue Type: New Feature > Components: Spark Core >Affects Versions: 1.0.0 >Reporter: Yardena > > Upgrade Akka from 2.2 to 2.3. We want to be able to use new Akka and Spray > features directly in the same project. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SPARK-1812) Support cross-building with Scala 2.11
[ https://issues.apache.org/jira/browse/SPARK-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14075161#comment-14075161 ] Anand Avati commented on SPARK-1812: FYI - I am working on Scala 2.11 support, ongoing work can be seen at https://github.com/avati/spark/commits/scala-2.11 > Support cross-building with Scala 2.11 > -- > > Key: SPARK-1812 > URL: https://issues.apache.org/jira/browse/SPARK-1812 > Project: Spark > Issue Type: New Feature > Components: Build, Spark Core >Reporter: Matei Zaharia >Assignee: Prashant Sharma > > Since Scala 2.10/2.11 are source compatible, we should be able to cross build > for both versions. From what I understand there are basically three things we > need to figure out: > 1. Have a two versions of our dependency graph, one that uses 2.11 > dependencies and the other that uses 2.10 dependencies. > 2. Figure out how to publish different poms for 2.10 and 2.11. > I think (1) can be accomplished by having a scala 2.11 profile. (2) isn't > really well supported by Maven since published pom's aren't generated > dynamically. But we can probably script around it to make it work. I've done > some initial sanity checks with a simple build here: > https://github.com/pwendell/scala-maven-crossbuild -- This message was sent by Atlassian JIRA (v6.2#6252)
