[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14797 Just created a JIRA: https://issues.apache.org/jira/browse/SPARK-17409 FYI, Spark 2.0.1 has been fixed by this PR, but Spark 2.1.0 has the issue due to the related codes have been changed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14797 Thanks! Is there a jira? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14797 The test failure in Spark 2.0 has been fixed in https://github.com/apache/spark/pull/14951. However, after this fix, we still optimize the query in CTAS twice, as reported in this PR. Thus, I plan to fix it soon. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14797 @gatorsmile want to put the regression tests at here? Or, you have already have a pr? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14797 This does not catch all the cases. In CTAS, we still optimize the query. By following the way in this PR, I can try to fix that case. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user davies commented on the issue: https://github.com/apache/spark/pull/14797 Merged this into master and 2.0 branch, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user srinathshankar commented on the issue: https://github.com/apache/spark/pull/14797 Looks fine. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user sameeragarwal commented on the issue: https://github.com/apache/spark/pull/14797 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14797 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64791/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14797 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14797 **[Test build #64791 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64791/consoleFull)** for PR 14797 at commit [`02b2896`](https://github.com/apache/spark/commit/02b289673654c2572dc8087e973a714f7c637d36). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14797 **[Test build #64791 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64791/consoleFull)** for PR 14797 at commit [`02b2896`](https://github.com/apache/spark/commit/02b289673654c2572dc8087e973a714f7c637d36). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14797 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64682/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14797 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14797 **[Test build #64682 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64682/consoleFull)** for PR 14797 at commit [`7771c6a`](https://github.com/apache/spark/commit/7771c6af2ca6a32141e2cf8851212e9f302485e1). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14797 **[Test build #64682 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64682/consoleFull)** for PR 14797 at commit [`7771c6a`](https://github.com/apache/spark/commit/7771c6af2ca6a32141e2cf8851212e9f302485e1). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14797 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14797 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64377/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14797 **[Test build #64377 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64377/consoleFull)** for PR 14797 at commit [`25c451a`](https://github.com/apache/spark/commit/25c451a67aeb912c0d215548e49c1a661d308b14). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * ` logWarning(s\"All labels belong to a single class and fitIntercept=false. It's a \" +` * ` * Set thresholds in multiclass (or binary) classification to adjust the probability of` * `class MultinomialLogisticRegression @Since(\"2.1.0\") (` * ` logWarning(s\"All labels belong to a single class and fitIntercept=false. It's\" +` * ` /** Margin (rawPrediction) for each class label. */` * ` /** Score (probability) for each class label. */` * ` class MultinomialLogisticRegressionModelWriter(instance: MultinomialLogisticRegressionModel)` * ` class ALSWrapperWriter(instance: ALSWrapper) extends MLWriter ` * ` class ALSWrapperReader extends MLReader[ALSWrapper] ` * ` class GaussianMixtureWrapperWriter(instance: GaussianMixtureWrapper) extends MLWriter ` * ` class GaussianMixtureWrapperReader extends MLReader[GaussianMixtureWrapper] ` * ` class IsotonicRegressionWrapperWriter(instance: IsotonicRegressionWrapper) extends MLWriter ` * ` class IsotonicRegressionWrapperReader extends MLReader[IsotonicRegressionWrapper] ` * ` class LDAWrapperWriter(instance: LDAWrapper) extends MLWriter ` * ` class LDAWrapperReader extends MLReader[LDAWrapper] ` * ` class MultilayerPerceptronClassifierWrapperWriter(instance: MultilayerPerceptronClassifierWrapper)` * `class JavaClassificationModel(JavaPredictionModel):` * `class LogisticRegressionModel(JavaModel, JavaClassificationModel, JavaMLWritable, JavaMLReadable):` * `class DecisionTreeClassificationModel(DecisionTreeModel, JavaClassificationModel, JavaMLWritable,` * `class RandomForestClassificationModel(TreeEnsembleModel, JavaClassificationModel, JavaMLWritable,` * `class GBTClassificationModel(TreeEnsembleModel, JavaPredictionModel, JavaMLWritable,` * `class NaiveBayesModel(JavaModel, JavaClassificationModel, JavaMLWritable, JavaMLReadable):` * `class MultilayerPerceptronClassificationModel(JavaModel, JavaPredictionModel, JavaMLWritable,` * `class LinearRegressionModel(JavaModel, JavaPredictionModel, JavaMLWritable, JavaMLReadable):` * `class DecisionTreeModel(JavaModel, JavaPredictionModel):` * `class RandomForestRegressionModel(TreeEnsembleModel, JavaPredictionModel, JavaMLWritable,` * `class GBTRegressionModel(TreeEnsembleModel, JavaPredictionModel, JavaMLWritable, JavaMLReadable):` * `class GeneralizedLinearRegressionModel(JavaModel, JavaPredictionModel, JavaMLWritable,` * `class JavaPredictionModel():` * `class SubstituteUnresolvedOrdinals(conf: CatalystConf) extends Rule[LogicalPlan] ` * `case class UnresolvedInlineTable(` * `case class UnresolvedTableValuedFunction(functionName: String, functionArgs: Seq[Expression])` * `case class UnresolvedOrdinal(ordinal: Int)` * `case class Literal (value: Any, dataType: DataType) extends LeafExpression with CodegenFallback ` * `abstract class PlanExpression[T <: QueryPlan[_]] extends Expression ` * `abstract class SubqueryExpression extends PlanExpression[LogicalPlan] ` * `case class ListQuery(plan: LogicalPlan, exprId: ExprId = NamedExpression.newExprId)` * `case class Exists(plan: LogicalPlan, exprId: ExprId = NamedExpression.newExprId)` * `case class SubqueryAlias(` * `class QuantileSummaries(` * ` case class Stats(value: Double, g: Int, delta: Int)` * `class JDBCOptions(` * `abstract class ExecSubqueryExpression extends PlanExpression[SubqueryExec] ` * `class OrcFileFormat extends FileFormat with DataSourceRegister with Serializable ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14797 **[Test build #64377 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64377/consoleFull)** for PR 14797 at commit [`25c451a`](https://github.com/apache/spark/commit/25c451a67aeb912c0d215548e49c1a661d308b14). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14797: [SPARK-17230] [SQL] Should not pass optimized query into...
Github user davies commented on the issue: https://github.com/apache/spark/pull/14797 cc @yhuai @JoshRosen @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
