[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155709277 **[Test build #2039 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2039/consoleFull)** for PR 9618 at commit [`dd1fe92`](https://github.com/apache/spark/commit/dd1fe927fe50f397bd6bf90e6015e8e57f06b26a). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155863448 These two tests need fix: ``` org.apache.spark.sql.execution.metric.SQLMetricsSuite.Project metrics 42 ms 1 org.apache.spark.sql.hive.execution.HiveExplainSuite.explain extended command ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155868367 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155868401 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155869238 **[Test build #45657 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45657/consoleFull)** for PR 9618 at commit [`b296915`](https://github.com/apache/spark/commit/b29691579e5e31bd55731ebfc4b8aa94a8259d2f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155903879 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155903697 **[Test build #45657 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45657/consoleFull)** for PR 9618 at commit [`b296915`](https://github.com/apache/spark/commit/b29691579e5e31bd55731ebfc4b8aa94a8259d2f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155903880 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45657/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9618 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155677601 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9618#discussion_r44504157 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala --- @@ -56,16 +56,6 @@ abstract class SparkPlan extends QueryPlan[SparkPlan] with Logging with Serializ // sqlContext will be null when we are being deserialized on the slaves. In this instance // the value of codegenEnabled/unsafeEnabled will be set by the desserializer after the --- End diff -- done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9618#discussion_r44504179 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -368,8 +367,7 @@ private[sql] abstract class SparkStrategies extends QueryPlanner[SparkPlan] { case logical.Project(projectList, child) => // If unsafe mode is enabled and we support these data types in Unsafe, use the // Tungsten project. Otherwise, use the normal project. -if (sqlContext.conf.unsafeEnabled && - UnsafeProjection.canSupport(projectList) && UnsafeProjection.canSupport(child.schema)) { +if (UnsafeProjection.canSupport(projectList) && UnsafeProjection.canSupport(child.schema)) { --- End diff -- Yea let's do that in a separate PR. If it does support all types, we should also just remove the normal Project. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155678486 **[Test build #45597 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45597/consoleFull)** for PR 9618 at commit [`95a9081`](https://github.com/apache/spark/commit/95a9081d7eafa204e27e019af278f22d0d7114bb). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_:\n * ` case class Metadata(`\n * ` require(className == expectedClassName, s\"Error loading metadata: Expected class name\" +`\n --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155678501 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155678512 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155678521 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155678513 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45597/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155679850 **[Test build #45604 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45604/consoleFull)** for PR 9618 at commit [`dd1fe92`](https://github.com/apache/spark/commit/dd1fe927fe50f397bd6bf90e6015e8e57f06b26a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155680594 **[Test build #2039 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2039/consoleFull)** for PR 9618 at commit [`dd1fe92`](https://github.com/apache/spark/commit/dd1fe927fe50f397bd6bf90e6015e8e57f06b26a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155686323 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155686326 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45604/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155686247 **[Test build #45604 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45604/consoleFull)** for PR 9618 at commit [`dd1fe92`](https://github.com/apache/spark/commit/dd1fe927fe50f397bd6bf90e6015e8e57f06b26a). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_:\n * `public class JavaLBFGSExample `\n * `class LDA @Since(\"1.6.0\") (`\n * ` case class Metadata(`\n * ` require(className == expectedClassName, s\"Error loading metadata: Expected class name\" +`\n --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155672054 cc @yhuai @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/9618 [SPARK-11644][SQL] Remove the option to turn off unsafe and codegen. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rxin/spark SPARK-11644 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/9618.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #9618 commit 888217c9aac665f3bc4c21367fb66bf063559afd Author: Reynold XinDate: 2015-11-11T05:31:43Z [SPARK-11644][SQL] Remove the option to turn off unsafe and codegen. commit 95a9081d7eafa204e27e019af278f22d0d7114bb Author: Reynold Xin Date: 2015-11-11T05:34:05Z Deprecation warning. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155672242 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155672226 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9618#issuecomment-155672601 **[Test build #45597 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45597/consoleFull)** for PR 9618 at commit [`95a9081`](https://github.com/apache/spark/commit/95a9081d7eafa204e27e019af278f22d0d7114bb). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9618#discussion_r44503641 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala --- @@ -56,16 +56,6 @@ abstract class SparkPlan extends QueryPlan[SparkPlan] with Logging with Serializ // sqlContext will be null when we are being deserialized on the slaves. In this instance // the value of codegenEnabled/unsafeEnabled will be set by the desserializer after the --- End diff -- remove this line --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11644][SQL] Remove the option to turn o...
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9618#discussion_r44503762 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -368,8 +367,7 @@ private[sql] abstract class SparkStrategies extends QueryPlanner[SparkPlan] { case logical.Project(projectList, child) => // If unsafe mode is enabled and we support these data types in Unsafe, use the // Tungsten project. Otherwise, use the normal project. -if (sqlContext.conf.unsafeEnabled && - UnsafeProjection.canSupport(projectList) && UnsafeProjection.canSupport(child.schema)) { +if (UnsafeProjection.canSupport(projectList) && UnsafeProjection.canSupport(child.schema)) { --- End diff -- UnsafeProjection should support all the dataType now, we could also remove this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org