[ https://issues.apache.org/jira/browse/SPARK-24825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547128#comment-16547128 ]
Stavros Kontopoulos edited comment on SPARK-24825 at 7/17/18 9:31 PM: ---------------------------------------------------------------------- As I mentioned elsewhere one thing that worked for me is modify ./dev/make-distribution.sh to do a clean install instead of clean package. But not sure for the long term for this, although you could purge deps. Test suite has the following spark deps: mvn -f pom.xml dependency:resolve | grep spark [INFO] org.apache.spark:spark-kvstore_2.11:jar:2.4.0-SNAPSHOT:compile [INFO] org.apache.spark:spark-network-shuffle_2.11:jar:2.4.0-SNAPSHOT:compile [INFO] org.apache.spark:spark-unsafe_2.11:jar:2.4.0-SNAPSHOT:compile [INFO] org.apache.spark:spark-core_2.11:jar:2.4.0-SNAPSHOT:compile [INFO] org.apache.spark:spark-launcher_2.11:jar:2.4.0-SNAPSHOT:compile [INFO] org.apache.spark:spark-tags_2.11:jar:2.4.0-SNAPSHOT:compile [INFO] org.apache.spark:spark-tags_2.11:test-jar:tests:2.4.0-SNAPSHOT:compile [INFO] org.spark-project.spark:unused:jar:1.0.0:compile [INFO] org.apache.spark:spark-network-common_2.11:jar:2.4.0-SNAPSHOT:compile was (Author: skonto): As I mentioned elsewhere one thing that worked for me is modify ./dev/make-distribution.sh to do an clean install instead of clean package. But not sure for the long term. Test suite has the following spark deps: mvn -f pom.xml dependency:resolve | grep spark [INFO] org.apache.spark:spark-kvstore_2.11:jar:2.4.0-SNAPSHOT:compile [INFO] org.apache.spark:spark-network-shuffle_2.11:jar:2.4.0-SNAPSHOT:compile [INFO] org.apache.spark:spark-unsafe_2.11:jar:2.4.0-SNAPSHOT:compile [INFO] org.apache.spark:spark-core_2.11:jar:2.4.0-SNAPSHOT:compile [INFO] org.apache.spark:spark-launcher_2.11:jar:2.4.0-SNAPSHOT:compile [INFO] org.apache.spark:spark-tags_2.11:jar:2.4.0-SNAPSHOT:compile [INFO] org.apache.spark:spark-tags_2.11:test-jar:tests:2.4.0-SNAPSHOT:compile [INFO] org.spark-project.spark:unused:jar:1.0.0:compile [INFO] org.apache.spark:spark-network-common_2.11:jar:2.4.0-SNAPSHOT:compile > [K8S][TEST] Kubernetes integration tests don't trace the maven project > dependency structure > ------------------------------------------------------------------------------------------- > > Key: SPARK-24825 > URL: https://issues.apache.org/jira/browse/SPARK-24825 > Project: Spark > Issue Type: Bug > Components: Kubernetes, Tests > Affects Versions: 2.4.0 > Reporter: Matt Cheah > Priority: Critical > > The Kubernetes integration tests will currently fail if maven installation is > not performed first, because the integration test build believes it should be > pulling the Spark parent artifact from maven central. However, this is > incorrect because the integration test should be building the Spark parent > pom as a dependency in the multi-module build, and the integration test > should just use the dynamically built artifact. Or to put it another way, the > integration test builds should never be pulling Spark dependencies from maven > central. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org