[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-152891750 @calvinjia can you add "[test-hadoop1.0]" to the title of this PR and then retest it? That will run the tests with hadoop 1. See more info here: https://cwiki.apache.org/confluence/display/SPARK/Useful+Developer+Tools --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user calvinjia commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-152847168 @yhuai Thanks for the the response! The last PR would pass with those parameters, but and would only fail if the entire dev/runtests cycle was ran (specifically if MIMA ran before). I've tested locally that the issues do not occur when running dev/runtests with this change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-152841958 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-152841959 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44756/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-152841909 **[Test build #44756 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44756/consoleFull)** for PR 9395 at commit [`6802ecd`](https://github.com/apache/spark/commit/6802ecd5c957a64bfcb11107ab8390a64a0d4ad6). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_:\n * ` logInfo(s\"Hive class not found $e\")`\n * `logDebug(\"Hive class not found\", e)`\n --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-152833844 btw, the profiles that we used in hadoop 1 tests are `-Phadoop-1 -Dhadoop.version=1.2.1 -Pkinesis-asl -Phive-thriftserver -Phive`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-152833780 cc @pwendell @srowen This is the new pr that upgrades Tachyon. I reverted the original one, which caused hadoop 1 test failures. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-152829433 **[Test build #44756 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44756/consoleFull)** for PR 9395 at commit [`6802ecd`](https://github.com/apache/spark/commit/6802ecd5c957a64bfcb11107ab8390a64a0d4ad6). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-152828790 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-152814340 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user calvinjia commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-152814256 @yhuai Could you take a look? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
GitHub user calvinjia opened a pull request: https://github.com/apache/spark/pull/9395 [SPARK-11236][CORE] Update Tachyon dependency 0.7.1 -> 0.8.1 This is a reopening of #9204 which failed hadoop1 sbt tests. With the original PR, a classpath issue would occur due to the MIMA plugin pulling in hadoop-2.2 dependencies regardless of the hadoop version when building the `oldDeps` project. These affect the hadoop1 sbt build because they are placed in `lib_managed` and Tachyon 0.8.0's default hadoop version is 2.2. You can merge this pull request into a Git repository by running: $ git pull https://github.com/calvinjia/spark spark-11236 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/9395.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #9395 commit 890d48cb5c1d66a1f887b5ad067c0f8d87c76083 Author: Calvin Jia Date: 2015-10-21T18:43:49Z Update Tachyon dependency from 0.7.1 -> 0.8.0. commit 876a5bf4694735ae38e2d00621a7d6085a388e98 Author: Calvin Jia Date: 2015-10-21T19:03:34Z Update make distribution to use Tachyon 0.8.0. commit f7edb372a1963cea4cf116ecb378869a45e33b0e Author: Calvin Jia Date: 2015-10-21T19:31:05Z Style fix. commit a187415013fe61991b71bc780797d0b8bc0a53b6 Author: Calvin Jia Date: 2015-10-30T23:37:19Z Merge branch 'master' into spark-11236 commit 3dad39f93792f04e13e9e067501599fb8f4f7238 Author: Calvin Jia Date: 2015-11-01T07:39:20Z Merge branch 'master' into spark-11236 commit 6802ecd5c957a64bfcb11107ab8390a64a0d4ad6 Author: Calvin Jia Date: 2015-11-01T09:59:02Z Update Tachyon version. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9204#issuecomment-152677014 Reverted by https://github.com/apache/spark/commit/e8ec2a7b01cc86329a6fbafc3d371bdfd79fc1d6. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9204#issuecomment-152671163 This one breaks hadoop 1 tests (see https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/Spark-Master-SBT/3888/AMPLAB_JENKINS_BUILD_PROFILE=hadoop1.0,label=spark-test/#showFailuresLink). I am going to revert it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9204#issuecomment-152415593 @haoyuan hey HY - can you not merge build related patches without asking for feedback from one of the build maintainers (me or @srowen)? This patch makes changes to Spark's dependency graph that need to be audited carefully because they affect all users. There is discussion of the maintainer/review process here: https://cwiki.apache.org/confluence/display/SPARK/Committers#Committers-ReviewProcessandMaintainers I did a post hoc review and it appears this does not change the contents of the assembly jar. So I think it is okay. Separately, it would be good to spin tachyon support out into a package so these changes do not need to go through the upstream review process. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9204 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user calvinjia commented on the pull request: https://github.com/apache/spark/pull/9204#issuecomment-151318905 @pwendell @srowen could either of you take a look? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user haoyuan commented on the pull request: https://github.com/apache/spark/pull/9204#issuecomment-150467834 LGTM @JoshRosen @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user calvinjia commented on the pull request: https://github.com/apache/spark/pull/9204#issuecomment-150277920 @JoshRosen could you take a look? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9204#issuecomment-150033093 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9204#issuecomment-150033096 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44084/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9204#issuecomment-150032938 **[Test build #44084 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44084/consoleFull)** for PR 9204 at commit [`f7edb37`](https://github.com/apache/spark/commit/f7edb372a1963cea4cf116ecb378869a45e33b0e). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_:\n * `case class ClassEncoder[T](`\n * `case class CreateRow(children: Seq[Expression]) extends Expression `\n --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9204#issuecomment-150005057 **[Test build #44084 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44084/consoleFull)** for PR 9204 at commit [`f7edb37`](https://github.com/apache/spark/commit/f7edb372a1963cea4cf116ecb378869a45e33b0e). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9204#issuecomment-150003057 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9204#issuecomment-150003087 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11236][CORE] Update Tachyon dependency ...
GitHub user calvinjia opened a pull request: https://github.com/apache/spark/pull/9204 [SPARK-11236][CORE] Update Tachyon dependency from 0.7.1 -> 0.8.0. Upgrades the tachyon-client version to the latest release. No new dependencies are added and no spark facing APIs are changed. The removal of the `tachyon-underfs-s3` exclusion will enable users to use S3 out of the box and there are no longer any additional external dependencies added by the module. You can merge this pull request into a Git repository by running: $ git pull https://github.com/calvinjia/spark spark-11236 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/9204.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #9204 commit 890d48cb5c1d66a1f887b5ad067c0f8d87c76083 Author: Calvin Jia Date: 2015-10-21T18:43:49Z Update Tachyon dependency from 0.7.1 -> 0.8.0. commit 876a5bf4694735ae38e2d00621a7d6085a388e98 Author: Calvin Jia Date: 2015-10-21T19:03:34Z Update make distribution to use Tachyon 0.8.0. commit f7edb372a1963cea4cf116ecb378869a45e33b0e Author: Calvin Jia Date: 2015-10-21T19:31:05Z Style fix. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
