[jira] [Commented] (SPARK-8095) Spark package dependencies not resolved when package is in local-ivy-cache
[ https://issues.apache.org/jira/browse/SPARK-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594930#comment-14594930 ] Apache Spark commented on SPARK-8095: - User 'brkyvz' has created a pull request for this issue: https://github.com/apache/spark/pull/6923 > Spark package dependencies not resolved when package is in local-ivy-cache > -- > > Key: SPARK-8095 > URL: https://issues.apache.org/jira/browse/SPARK-8095 > Project: Spark > Issue Type: Bug > Components: Spark Submit >Affects Versions: 1.4.0 >Reporter: Eron Wright >Assignee: Burak Yavuz > Fix For: 1.4.1, 1.5.0 > > > Given a dependency expressed with '--packages', the transitive dependencies > are supposed to be automatically included. This is true for most repository > types including local-m2-cache, Spark Packages, and central. For > ivy-local-cache, it is not. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8095) Spark package dependencies not resolved when package is in local-ivy-cache
[ https://issues.apache.org/jira/browse/SPARK-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14583916#comment-14583916 ] Apache Spark commented on SPARK-8095: - User 'brkyvz' has created a pull request for this issue: https://github.com/apache/spark/pull/6788 > Spark package dependencies not resolved when package is in local-ivy-cache > -- > > Key: SPARK-8095 > URL: https://issues.apache.org/jira/browse/SPARK-8095 > Project: Spark > Issue Type: Bug > Components: Spark Submit >Affects Versions: 1.4.0 >Reporter: Eron Wright >Assignee: Burak Yavuz > > Given a dependency expressed with '--packages', the transitive dependencies > are supposed to be automatically included. This is true for most repository > types including local-m2-cache, Spark Packages, and central. For > ivy-local-cache, it is not. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8095) Spark package dependencies not resolved when package is in local-ivy-cache
[ https://issues.apache.org/jira/browse/SPARK-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573784#comment-14573784 ] Apache Spark commented on SPARK-8095: - User 'andrewor14' has created a pull request for this issue: https://github.com/apache/spark/pull/6658 > Spark package dependencies not resolved when package is in local-ivy-cache > -- > > Key: SPARK-8095 > URL: https://issues.apache.org/jira/browse/SPARK-8095 > Project: Spark > Issue Type: Bug > Components: Spark Submit >Affects Versions: 1.4.0 >Reporter: Eron Wright > > Given a dependency expressed with '--packages', the transitive dependencies > are supposed to be automatically included. This is true for most repository > types including local-m2-cache, Spark Packages, and central. For > ivy-local-cache, it is not. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8095) Spark package dependencies not resolved when package is in local-ivy-cache
[
https://issues.apache.org/jira/browse/SPARK-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14572179#comment-14572179
]
Eron Wright commented on SPARK-8095:
-
Tested with 1.4.0RC4.
When package is in local-ivy-cache (incorrect behavior):
{code}
:: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0
confs: [default]
found EronWright#mypackage;0.0.7 in local-ivy-cache
:: resolution report :: resolve 1028ms :: artifacts dl 11ms
:: modules in use:
EronWright#mypackage;0.0.7 from local-ivy-cache in [default]
-
| |modules|| artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
-
| default | 1 | 1 | 0 | 0 || 1 | 1 |
-
{code}
When package is in local-m2-cache (correct behavior):
{code}
:: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0
confs: [default]
found EronWright#mypackage;0.0.7 in local-m2-cache
found org.deeplearning4j#dl4j-spark;0.0.3.3.3.alpha1 in list
...
:: resolution report :: resolve 2509ms :: artifacts dl 101ms
:: modules in use:
EronWright#mypackage;0.0.7 from local-m2-cache in [default]
org.deeplearning4j#dl4j-spark;0.0.3.3.3.alpha1 from list in [default]
...
-
| |modules|| artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
-
| default | 37 | 1 | 1 | 2 || 34 | 1 |
-
{code}
> Spark package dependencies not resolved when package is in local-ivy-cache
> --
>
> Key: SPARK-8095
> URL: https://issues.apache.org/jira/browse/SPARK-8095
> Project: Spark
> Issue Type: Bug
> Components: Spark Submit
>Affects Versions: 1.4.0
>Reporter: Eron Wright
>
> Given a dependency expressed with '--packages', the transitive dependencies
> are supposed to be automatically included. This is true for most repository
> types including local-m2-cache, Spark Packages, and central. For
> ivy-local-cache, it is not.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8095) Spark package dependencies not resolved when package is in local-ivy-cache
[ https://issues.apache.org/jira/browse/SPARK-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14572128#comment-14572128 ] Burak Yavuz commented on SPARK-8095: In the local ivy cache, it should use the `ivy.xml` instead of the pom. That's why, setUsePoms is set to false. At least that was my understanding... Doesn't it work? > Spark package dependencies not resolved when package is in local-ivy-cache > -- > > Key: SPARK-8095 > URL: https://issues.apache.org/jira/browse/SPARK-8095 > Project: Spark > Issue Type: Bug > Components: Spark Submit >Affects Versions: 1.4.0 >Reporter: Eron Wright > > Given a dependency expressed with '--packages', the transitive dependencies > are supposed to be automatically included. This is true for most repository > types including local-m2-cache, Spark Packages, and central. For > ivy-local-cache, it is not. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8095) Spark package dependencies not resolved when package is in local-ivy-cache
[ https://issues.apache.org/jira/browse/SPARK-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14572120#comment-14572120 ] Eron Wright commented on SPARK-8095: - Note that IBiblioResolver::setUsepoms is not set to true on the local-ivy-cache resolver, yet is set on all other resolvers. Speculating about the root cause. > Spark package dependencies not resolved when package is in local-ivy-cache > -- > > Key: SPARK-8095 > URL: https://issues.apache.org/jira/browse/SPARK-8095 > Project: Spark > Issue Type: Bug > Components: Spark Submit >Affects Versions: 1.4.0 >Reporter: Eron Wright > > Given a dependency expressed with '--packages', the transitive dependencies > are supposed to be automatically included. This is true for most repository > types including local-m2-cache, Spark Packages, and central. For > ivy-local-cache, it is not. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
