[GitHub] spark pull request: [SPARK-5466] Add explicit guava dependencies w...
Github user tsingfu commented on the pull request: https://github.com/apache/spark/pull/4272#issuecomment-92713227 @WangTaoTheTonic Hi, today I got the error when I did a new clean build in updated master branch as follows: ``` cd spark //in master branch, update my local master from remote git pull mvn clean mvn generate-sources mvn package -Dhadoop.version=2.5.0-cdh5.3.1 -DskipTests -Pyarn -Phive-0.13.1 -Phive-thriftserver -Pspark-ganglia-lgpl ``` then the error occured. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5466] Add explicit guava dependencies w...
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/4272#issuecomment-92691600 @tsingfu Could you add more description about how you found the build error? As I got same building error message when I tried to add some code in sql/hive-thriftserver module in which I invoked some method in Utils.class(in core module). I am not sure in which case it will occured and in which not. @vanzin I am not expert on compile thing so please give some explaination about this if you got more infos about it. I will appreciate a lot. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5466] Add explicit guava dependencies w...
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/4272#issuecomment-92720333 @tsingfu Okay I tested it too and the error meesage is: [INFO] compiler plugin: BasicArtifact(org.scalamacros,paradise_2.10.4,2.0.1,null) ^[[0m[^[[0minfo^[[0m] ^[[0mCompiling 8 Scala sources to /home/wangtao111/github/spark/sql/hive-thriftserver/target/scala-2.10/classes...^[[0m ^[[0m[^[[31merror^[[0m] ^[[0mbad symbolic reference. A signature in Utils.class refers to term util^[[0m ^[[0m[^[[31merror^[[0m] ^[[0min package com.google.common which is not available.^[[0m ^[[0m[^[[31merror^[[0m] ^[[0mIt may be completely missing from the current classpath, or the version on^[[0m ^[[0m[^[[31merror^[[0m] ^[[0mthe classpath might be incompatible with the version used when compiling Utils.class.^[[0m ^[[0m[^[[31merror^[[0m] ^[[0m^[[0m ^[[0m[^[[31merror^[[0m] ^[[0m while compiling: /home/wangtao111/github/spark/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLEnv.scala^[[0m ^[[0m[^[[31merror^[[0m] ^[[0mduring phase: erasure^[[0m ^[[0m[^[[31merror^[[0m] ^[[0m library version: version 2.10.4^[[0m ^[[0m[^[[31merror^[[0m] ^[[0mcompiler version: version 2.10.4^[[0m The build command is: ` mvn -Pbigtop-dist -Phive-thriftserver -Phive -Pyarn -Phadoop-2.4 -Dhadoop.version=2.6.0 -DskipTests clean package` Seems like `SparkSQLEnv.scala` used `Utils.class` and there is `import com.google.common.xxx` in `Utils.scala`. So if we shaded guava in core module there will come some problem? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5466] Add explicit guava dependencies w...
Github user tsingfu commented on the pull request: https://github.com/apache/spark/pull/4272#issuecomment-92579447 @vanzin , today I found *sql/hive-thriftserver* also needs explicit guava dependencies. But in `sql/hive-thriftserver/pom.xml` the scope of `com.google.guava.guava` is `runtime`, and this causes build error as same as in [SPARK-5466](https://issues.apache.org/jira/browse/SPARK-5466). my test: I try the scope with `compile`, build works fine. vi sql/hive-thriftserver/pom.xml ``` dependency groupIdcom.google.guava/groupId artifactIdguava/artifactId !-- scoperuntime/scope -- /dependency ``` cc @pwendell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5466] Add explicit guava dependencies w...
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/4272 [SPARK-5466] Add explicit guava dependencies where needed. One side-effect of shading guava is that it disappears as a transitive dependency. For Hadoop 2.x, this was masked by the fact that Hadoop itself depends on guava. But certain versions of Hadoop 1.x also shade guava, leaving either no guava or some random version pulled by another dependency on the classpath. So be explicit about the dependency in modules that use guava directly, which is the right thing to do anyway. You can merge this pull request into a Git repository by running: $ git pull https://github.com/vanzin/spark SPARK-5466 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/4272.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4272 commit d3b2c84eea8ff75d6cdec46c48b8dc8c9aa1bf11 Author: Marcelo Vanzin van...@cloudera.com Date: 2015-01-29T18:10:19Z [SPARK-5466] Add explicit guava dependencies where needed. One side-effect of shading guava is that it disappears as a transitive dependency. For Hadoop 2.x, this was masked by the fact that Hadoop itself depends on guava. But certain versions of Hadoop 1.x also shade guava, leaving either no guava or some random version pulled by another dependency on the classpath. So be explicit about the dependency in modules that use guava directly, which is the right thing to do anyway. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5466] Add explicit guava dependencies w...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4272#issuecomment-72078086 [Test build #26326 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26326/consoleFull) for PR 4272 at commit [`e3f30e5`](https://github.com/apache/spark/commit/e3f30e5735e9a8ceeb8f60903360f701caa10a63). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5466] Add explicit guava dependencies w...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4272#issuecomment-72076351 [Test build #26325 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26325/consoleFull) for PR 4272 at commit [`d3b2c84`](https://github.com/apache/spark/commit/d3b2c84eea8ff75d6cdec46c48b8dc8c9aa1bf11). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5466] Add explicit guava dependencies w...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4272#issuecomment-72081625 Thanks for figuring this out @vanzin! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5466] Add explicit guava dependencies w...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4272#issuecomment-72089456 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26326/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5466] Add explicit guava dependencies w...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4272#issuecomment-72089439 [Test build #26326 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26326/consoleFull) for PR 4272 at commit [`e3f30e5`](https://github.com/apache/spark/commit/e3f30e5735e9a8ceeb8f60903360f701caa10a63). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5466] Add explicit guava dependencies w...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4272#issuecomment-72093311 [Test build #26325 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26325/consoleFull) for PR 4272 at commit [`d3b2c84`](https://github.com/apache/spark/commit/d3b2c84eea8ff75d6cdec46c48b8dc8c9aa1bf11). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5466] Add explicit guava dependencies w...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4272 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
