[GitHub] [spark] SparkQA commented on issue #26331: [SPARK-24203][core] Make executor's bindAddress configurable

[GitBox]

SparkQA commented on issue #26331: [SPARK-24203][core] Make executor's 
bindAddress configurable
URL: https://github.com/apache/spark/pull/26331#issuecomment-548688472
 
 
   **[Test build #113070 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113070/testReport)**
 for PR 26331 at commit 
[`62974f8`](https://github.com/apache/spark/commit/62974f8a7170ae682ff7a5fcdf64134c971dc148).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #26331: [SPARK-24203][core] Make executor's bindAddress configurable

[GitBox]

SparkQA removed a comment on issue #26331: [SPARK-24203][core] Make executor's 
bindAddress configurable
URL: https://github.com/apache/spark/pull/26331#issuecomment-548663478
 
 
   **[Test build #113070 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113070/testReport)**
 for PR 26331 at commit 
[`62974f8`](https://github.com/apache/spark/commit/62974f8a7170ae682ff7a5fcdf64134c971dc148).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26331: [SPARK-24203][core] Make executor's bindAddress configurable

[GitBox]

AmplabJenkins commented on issue #26331: [SPARK-24203][core] Make executor's 
bindAddress configurable
URL: https://github.com/apache/spark/pull/26331#issuecomment-548688878
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113070/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26331: [SPARK-24203][core] Make executor's bindAddress configurable

[GitBox]

AmplabJenkins commented on issue #26331: [SPARK-24203][core] Make executor's 
bindAddress configurable
URL: https://github.com/apache/spark/pull/26331#issuecomment-548688875
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #26353: [SPARK-29590][WEBUI]JDBC/ODBC tab in the spark UI support hide tables, to make it consistent with other tabs.

[GitBox]

SparkQA commented on issue #26353: [SPARK-29590][WEBUI]JDBC/ODBC tab in the 
spark UI support hide tables, to make it consistent with other tabs.
URL: https://github.com/apache/spark/pull/26353#issuecomment-548688871
 
 
   **[Test build #113077 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113077/testReport)**
 for PR 26353 at commit 
[`e86b15a`](https://github.com/apache/spark/commit/e86b15a51b8af06844fcfd5fa75c1115145bd88e).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26353: [SPARK-29590][WEBUI]JDBC/ODBC tab in the spark UI support hide tables, to make it consistent with other tabs.

[GitBox]

AmplabJenkins removed a comment on issue #26353: [SPARK-29590][WEBUI]JDBC/ODBC 
tab in the spark UI support hide tables, to make it consistent with other tabs.
URL: https://github.com/apache/spark/pull/26353#issuecomment-548687376
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17996/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26353: [SPARK-29590][WEBUI]JDBC/ODBC tab in the spark UI support hide tables, to make it consistent with other tabs.

[GitBox]

AmplabJenkins removed a comment on issue #26353: [SPARK-29590][WEBUI]JDBC/ODBC 
tab in the spark UI support hide tables, to make it consistent with other tabs.
URL: https://github.com/apache/spark/pull/26353#issuecomment-548687372
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26331: [SPARK-24203][core] Make executor's bindAddress configurable

[GitBox]

AmplabJenkins removed a comment on issue #26331: [SPARK-24203][core] Make 
executor's bindAddress configurable
URL: https://github.com/apache/spark/pull/26331#issuecomment-548688878
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113070/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26331: [SPARK-24203][core] Make executor's bindAddress configurable

[GitBox]

AmplabJenkins removed a comment on issue #26331: [SPARK-24203][core] Make 
executor's bindAddress configurable
URL: https://github.com/apache/spark/pull/26331#issuecomment-548688875
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] zhengruifeng commented on a change in pull request #26124: [SPARK-29224][ML]Implement Factorization Machines as a ml-pipeline component

[GitBox]

zhengruifeng commented on a change in pull request #26124: 
[SPARK-29224][ML]Implement Factorization Machines as a ml-pipeline component 
URL: https://github.com/apache/spark/pull/26124#discussion_r341470080
 
 

 ##
 File path: 
mllib/src/main/scala/org/apache/spark/ml/classification/FMClassifier.scala
 ##
 @@ -0,0 +1,360 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.classification
+
+import org.apache.hadoop.fs.Path
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.internal.Logging
+import org.apache.spark.ml.linalg._
+import org.apache.spark.ml.param._
+import org.apache.spark.ml.regression._
+import org.apache.spark.ml.util._
+import org.apache.spark.ml.util.Instrumentation.instrumented
+import org.apache.spark.mllib.linalg.{Vector => OldVector}
+import org.apache.spark.mllib.linalg.VectorImplicits._
+import org.apache.spark.mllib.optimization.GradientDescent
+import org.apache.spark.mllib.regression.{LabeledPoint => OldLabeledPoint}
+import org.apache.spark.rdd.RDD
+import org.apache.spark.sql.{Dataset, Row}
+import org.apache.spark.sql.functions.col
+import org.apache.spark.storage.StorageLevel
+
+/**
+ * Params for FMClassifier.
+ */
+private[classification] trait FMClassifierParams extends 
ProbabilisticClassifierParams
+  with FactorizationMachinesParams {
+}
+
+/**
+ * Factorization Machines learning algorithm for classification.
+ * It supports normal gradient descent and AdamW solver.
+ *
+ * The implementation is based upon:
+ * https://www.csie.ntu.edu.tw/~b97053/paper/Rendle2010FM.pdf";>
+ * S. Rendle. "Factorization machines" 2010.
+ *
+ * FM is able to estimate interactions even in problems with huge sparsity
+ * (like advertising and recommendation system).
+ * FM formula is:
+ * {{{
+ *   y = w_0 + \sum\limits^n_{i-1} w_i x_i +
+ * \sum\limits^n_{i=1} \sum\limits^n_{j=i+1} \langle v_i, v_j \rangle x_i 
x_j
+ * }}}
+ * First two terms denote global bias and linear term (as same as linear 
regression),
+ * and last term denotes pairwise interactions term. {{{v_i}}} describes the 
i-th variable
+ * with k factors.
+ *
+ * FM classification model uses logistic loss which can be solved by gradient 
descent method, and
+ * regularization terms like L2 are usually added to the loss function to 
prevent overfitting.
+ *
+ * @note Multiclass labels are not currently supported.
+ */
+@Since("3.0.0")
+class FMClassifier @Since("3.0.0") (
+  @Since("3.0.0") override val uid: String)
+  extends ProbabilisticClassifier[Vector, FMClassifier, FMClassifierModel]
+  with FMClassifierParams with DefaultParamsWritable with Logging {
+
+  import 
org.apache.spark.ml.regression.BaseFactorizationMachinesGradient.{LogisticLoss, 
parseLoss}
+  import org.apache.spark.ml.regression.FMRegressor.initCoefficients
+
+  @Since("3.0.0")
+  def this() = this(Identifiable.randomUID("fmc"))
+
+  /**
+   * Set the dimensionality of the factors.
+   * Default is 8.
+   *
+   * @group setParam
+   */
+  @Since("3.0.0")
+  def setNumFactors(value: Int): this.type = set(numFactors, value)
+  setDefault(numFactors -> 8)
+
+  /**
+   * Set whether to fit global bias term.
+   * Default is true.
+   *
+   * @group setParam
+   */
+  @Since("3.0.0")
+  def setFitBias(value: Boolean): this.type = set(fitBias, value)
+  setDefault(fitBias -> true)
+
+  /**
+   * Set whether to fit linear term.
+   * Default is true.
+   *
+   * @group setParam
+   */
+  @Since("3.0.0")
+  def setFitLinear(value: Boolean): this.type = set(fitLinear, value)
+  setDefault(fitLinear -> true)
+
+  /**
+   * Set the L2 regularization parameter.
+   * Default is 0.0.
+   *
+   * @group setParam
+   */
+  @Since("3.0.0")
+  def setRegParam(value: Double): this.type = set(regParam, value)
+  setDefault(regParam -> 0.0)
+
+  /**
+   * Set the mini-batch fraction parameter.
+   * Default is 1.0.
+   *
+   * @group setParam
+   */
+  @Since("3.0.0")
+  def setMiniBatchFraction(value: Double): this.type = {
+require(value > 0 && value <= 1.0,
+  s"Fraction for mini-batch SGD must be in range (0, 1] but got $value")
+set(miniBatchFraction, value)
+  }
+  setDefault(miniBatchFraction -> 1.0)
+
+  /**
+   * Set the standard d

[GitHub] [spark] zhengruifeng commented on a change in pull request #26124: [SPARK-29224][ML]Implement Factorization Machines as a ml-pipeline component

[GitBox]

zhengruifeng commented on a change in pull request #26124: 
[SPARK-29224][ML]Implement Factorization Machines as a ml-pipeline component 
URL: https://github.com/apache/spark/pull/26124#discussion_r341470080
 
 

 ##
 File path: 
mllib/src/main/scala/org/apache/spark/ml/classification/FMClassifier.scala
 ##
 @@ -0,0 +1,360 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.classification
+
+import org.apache.hadoop.fs.Path
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.internal.Logging
+import org.apache.spark.ml.linalg._
+import org.apache.spark.ml.param._
+import org.apache.spark.ml.regression._
+import org.apache.spark.ml.util._
+import org.apache.spark.ml.util.Instrumentation.instrumented
+import org.apache.spark.mllib.linalg.{Vector => OldVector}
+import org.apache.spark.mllib.linalg.VectorImplicits._
+import org.apache.spark.mllib.optimization.GradientDescent
+import org.apache.spark.mllib.regression.{LabeledPoint => OldLabeledPoint}
+import org.apache.spark.rdd.RDD
+import org.apache.spark.sql.{Dataset, Row}
+import org.apache.spark.sql.functions.col
+import org.apache.spark.storage.StorageLevel
+
+/**
+ * Params for FMClassifier.
+ */
+private[classification] trait FMClassifierParams extends 
ProbabilisticClassifierParams
+  with FactorizationMachinesParams {
+}
+
+/**
+ * Factorization Machines learning algorithm for classification.
+ * It supports normal gradient descent and AdamW solver.
+ *
+ * The implementation is based upon:
+ * https://www.csie.ntu.edu.tw/~b97053/paper/Rendle2010FM.pdf";>
+ * S. Rendle. "Factorization machines" 2010.
+ *
+ * FM is able to estimate interactions even in problems with huge sparsity
+ * (like advertising and recommendation system).
+ * FM formula is:
+ * {{{
+ *   y = w_0 + \sum\limits^n_{i-1} w_i x_i +
+ * \sum\limits^n_{i=1} \sum\limits^n_{j=i+1} \langle v_i, v_j \rangle x_i 
x_j
+ * }}}
+ * First two terms denote global bias and linear term (as same as linear 
regression),
+ * and last term denotes pairwise interactions term. {{{v_i}}} describes the 
i-th variable
+ * with k factors.
+ *
+ * FM classification model uses logistic loss which can be solved by gradient 
descent method, and
+ * regularization terms like L2 are usually added to the loss function to 
prevent overfitting.
+ *
+ * @note Multiclass labels are not currently supported.
+ */
+@Since("3.0.0")
+class FMClassifier @Since("3.0.0") (
+  @Since("3.0.0") override val uid: String)
+  extends ProbabilisticClassifier[Vector, FMClassifier, FMClassifierModel]
+  with FMClassifierParams with DefaultParamsWritable with Logging {
+
+  import 
org.apache.spark.ml.regression.BaseFactorizationMachinesGradient.{LogisticLoss, 
parseLoss}
+  import org.apache.spark.ml.regression.FMRegressor.initCoefficients
+
+  @Since("3.0.0")
+  def this() = this(Identifiable.randomUID("fmc"))
+
+  /**
+   * Set the dimensionality of the factors.
+   * Default is 8.
+   *
+   * @group setParam
+   */
+  @Since("3.0.0")
+  def setNumFactors(value: Int): this.type = set(numFactors, value)
+  setDefault(numFactors -> 8)
+
+  /**
+   * Set whether to fit global bias term.
+   * Default is true.
+   *
+   * @group setParam
+   */
+  @Since("3.0.0")
+  def setFitBias(value: Boolean): this.type = set(fitBias, value)
+  setDefault(fitBias -> true)
+
+  /**
+   * Set whether to fit linear term.
+   * Default is true.
+   *
+   * @group setParam
+   */
+  @Since("3.0.0")
+  def setFitLinear(value: Boolean): this.type = set(fitLinear, value)
+  setDefault(fitLinear -> true)
+
+  /**
+   * Set the L2 regularization parameter.
+   * Default is 0.0.
+   *
+   * @group setParam
+   */
+  @Since("3.0.0")
+  def setRegParam(value: Double): this.type = set(regParam, value)
+  setDefault(regParam -> 0.0)
+
+  /**
+   * Set the mini-batch fraction parameter.
+   * Default is 1.0.
+   *
+   * @group setParam
+   */
+  @Since("3.0.0")
+  def setMiniBatchFraction(value: Double): this.type = {
+require(value > 0 && value <= 1.0,
+  s"Fraction for mini-batch SGD must be in range (0, 1] but got $value")
+set(miniBatchFraction, value)
+  }
+  setDefault(miniBatchFraction -> 1.0)
+
+  /**
+   * Set the standard d

[GitHub] [spark] SparkQA commented on issue #25287: [SPARK-28552][SQL]Identification of different dialects insensitive to case by JDBC URL prefix

[GitBox]

SparkQA commented on issue #25287: [SPARK-28552][SQL]Identification of 
different dialects insensitive to case by JDBC URL prefix
URL: https://github.com/apache/spark/pull/25287#issuecomment-548690511
 
 
   **[Test build #113078 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113078/testReport)**
 for PR 25287 at commit 
[`3e58b81`](https://github.com/apache/spark/commit/3e58b81569ba3a6de7f39642effc45a67e73bfb8).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #25287: [SPARK-28552][SQL]Identification of different dialects insensitive to case by JDBC URL prefix

[GitBox]

AmplabJenkins removed a comment on issue #25287: 
[SPARK-28552][SQL]Identification of different dialects insensitive to case by 
JDBC URL prefix
URL: https://github.com/apache/spark/pull/25287#issuecomment-548690859
 
 
   Merged build finished. Test FAILed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #25287: [SPARK-28552][SQL]Identification of different dialects insensitive to case by JDBC URL prefix

[GitBox]

SparkQA removed a comment on issue #25287: [SPARK-28552][SQL]Identification of 
different dialects insensitive to case by JDBC URL prefix
URL: https://github.com/apache/spark/pull/25287#issuecomment-548690511
 
 
   **[Test build #113078 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113078/testReport)**
 for PR 25287 at commit 
[`3e58b81`](https://github.com/apache/spark/commit/3e58b81569ba3a6de7f39642effc45a67e73bfb8).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #25287: [SPARK-28552][SQL]Identification of different dialects insensitive to case by JDBC URL prefix

[GitBox]

AmplabJenkins commented on issue #25287: [SPARK-28552][SQL]Identification of 
different dialects insensitive to case by JDBC URL prefix
URL: https://github.com/apache/spark/pull/25287#issuecomment-548690859
 
 
   Merged build finished. Test FAILed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #25287: [SPARK-28552][SQL]Identification of different dialects insensitive to case by JDBC URL prefix

[GitBox]

AmplabJenkins commented on issue #25287: [SPARK-28552][SQL]Identification of 
different dialects insensitive to case by JDBC URL prefix
URL: https://github.com/apache/spark/pull/25287#issuecomment-548690865
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113078/
   Test FAILed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #25287: [SPARK-28552][SQL]Identification of different dialects insensitive to case by JDBC URL prefix

[GitBox]

SparkQA commented on issue #25287: [SPARK-28552][SQL]Identification of 
different dialects insensitive to case by JDBC URL prefix
URL: https://github.com/apache/spark/pull/25287#issuecomment-548690854
 
 
   **[Test build #113078 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113078/testReport)**
 for PR 25287 at commit 
[`3e58b81`](https://github.com/apache/spark/commit/3e58b81569ba3a6de7f39642effc45a67e73bfb8).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #25287: [SPARK-28552][SQL]Identification of different dialects insensitive to case by JDBC URL prefix

[GitBox]

AmplabJenkins removed a comment on issue #25287: 
[SPARK-28552][SQL]Identification of different dialects insensitive to case by 
JDBC URL prefix
URL: https://github.com/apache/spark/pull/25287#issuecomment-548690865
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113078/
   Test FAILed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] alfozan commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM with aggregation.

[GitBox]

alfozan commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM with 
aggregation.
URL: https://github.com/apache/spark/pull/25028#issuecomment-548691723
 
 
   Here's what I mean:
   
   test query:
   
   ```
   SELECT TRANSFORM(key, abs(key))
   USING 'cat'
   FROM (SELECT DISTINCT key FROM src);
   ```
   
   The physical plan without this PR (on master):
   ```
   ScriptTransformation [key#7, abs(key#7)], cat, [key#9, value#10], 
HiveScriptIOSchema(List(),List(),Some(org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe),Some(org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe),List((field.delim,
  )),List((field.delim,   
)),Some(org.apache.hadoop.hive.ql.exec.TextRecordReader),Some(org.apache.hadoop.hive.ql.exec.TextRecordWriter),true)
   +- *(2) HashAggregate(keys=[1#16], functions=[], output=[key#7])
  +- Exchange hashpartitioning(1#16, 1), true, [id=#27]
 +- *(1) HashAggregate(keys=[1 AS 1#16], functions=[], output=[1#16])
+- *(1) Scan OneRowRelation[]
   ```
   
   
   
   The physical plan with this PR:
   ```
   ScriptTransformation [key#0, abs(key)#9], cat, [key#2, value#3], 
HiveScriptIOSchema(List(),List(),Some(org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe),Some(org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe),List((field.delim,
   )),List((field.delim,   
)),Some(org.apache.hadoop.hive.ql.exec.TextRecordReader),Some(org.apache.hadoop.hive.ql.exec.TextRecordWriter),true)
   +- *(2) HashAggregate(keys=[1#10], functions=[], output=[key#0, abs(key)#9])
  +- Exchange hashpartitioning(1#10, 1), true, [id=#21]
 +- *(1) HashAggregate(keys=[1 AS 1#10], functions=[], output=[1#10])
+- *(1) Scan OneRowRelation[]
   ```
   
   
   Difference:
   `+- *(2) HashAggregate(keys=[1#16], functions=[], output=[key#7])
   `vs
   `+- *(2) HashAggregate(keys=[1#10], functions=[], output=[key#0, abs(key)#9])
   `
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] alfozan commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM with aggregation.

[GitBox]

alfozan commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM with 
aggregation.
URL: https://github.com/apache/spark/pull/25028#issuecomment-548692408
 
 
   Another issue:
   
   query:
   `MAP k / 10 USING 'cat' AS (aa) from (select 10 as k);
   `
   
   Error in query: cannot resolve '`(k / 10)`' given input columns: [(CAST(k AS 
DOUBLE) / CAST(10 AS DOUBLE))];;


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] alfozan removed a comment on issue #25028: [SPARK-28227][SQL] Support TRANSFORM with aggregation.

[GitBox]

alfozan removed a comment on issue #25028: [SPARK-28227][SQL] Support TRANSFORM 
with aggregation.
URL: https://github.com/apache/spark/pull/25028#issuecomment-548692408
 
 
   Another issue:
   
   query:
   `MAP k / 10 USING 'cat' AS (aa) from (select 10 as k);
   `
   
   Error in query: cannot resolve '`(k / 10)`' given input columns: [(CAST(k AS 
DOUBLE) / CAST(10 AS DOUBLE))];;


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM with aggregation.

[GitBox]

AngersZh commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM 
with aggregation.
URL: https://github.com/apache/spark/pull/25028#issuecomment-548693738
 
 
   > Another issue:
   > 
   > query:
   > `MAP k / 10 USING 'cat' AS (aa) from (select 10 as k); `
   > 
   > Error in query: cannot resolve '`(k / 10)`' given input columns: [(CAST(k 
AS DOUBLE) / CAST(10 AS DOUBLE))];;
   
   This can be solved by 
https://github.com/apache/spark/pull/25028#issuecomment-548675869


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #25287: [SPARK-28552][SQL]Identification of different dialects insensitive to case by JDBC URL prefix

[GitBox]

SparkQA commented on issue #25287: [SPARK-28552][SQL]Identification of 
different dialects insensitive to case by JDBC URL prefix
URL: https://github.com/apache/spark/pull/25287#issuecomment-548693798
 
 
   **[Test build #113079 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113079/testReport)**
 for PR 25287 at commit 
[`ac70da4`](https://github.com/apache/spark/commit/ac70da4f494624368aa1570100e2a00ec0e1c308).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM with aggregation.

[GitBox]

AngersZh commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM 
with aggregation.
URL: https://github.com/apache/spark/pull/25028#issuecomment-548694667
 
 
   > output=[key#7]
   
   This is right since we current tread transform's child logicalplan as an 
entire one.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] alfozan commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM with aggregation.

[GitBox]

alfozan commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM with 
aggregation.
URL: https://github.com/apache/spark/pull/25028#issuecomment-548694618
 
 
   > > Another issue:
   > > query:
   > > `MAP k / 10 USING 'cat' AS (aa) from (select 10 as k); `
   > > Error in query: cannot resolve '`(k / 10)`' given input columns: 
[(CAST(k AS DOUBLE) / CAST(10 AS DOUBLE))];;
   > 
   > This can be solved by [#25028 
(comment)](https://github.com/apache/spark/pull/25028#issuecomment-548675869)
   
   Yes we already discussed a solution in 
https://github.com/apache/spark/pull/25028#discussion_r339890074
   which I can confirm it works.
   
   Currently the open issue is 
https://github.com/apache/spark/pull/25028#issuecomment-548691723


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AngersZhuuuu edited a comment on issue #25028: [SPARK-28227][SQL] Support TRANSFORM with aggregation.

[GitBox]

AngersZh edited a comment on issue #25028: [SPARK-28227][SQL] Support 
TRANSFORM with aggregation.
URL: https://github.com/apache/spark/pull/25028#issuecomment-548694667
 
 
   > Here's what I mean:
   > 
   > test query:
   > 
   > ```
   > SELECT TRANSFORM(key, abs(key))
   > USING 'cat'
   > FROM (SELECT DISTINCT key FROM src);
   > ```
   > 
   > The physical plan without this PR (on master):
   > 
   > ```
   > ScriptTransformation [key#7, abs(key#7)], cat, [key#9, value#10], 
HiveScriptIOSchema(List(),List(),Some(org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe),Some(org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe),List((field.delim,
)),List((field.delim,   
)),Some(org.apache.hadoop.hive.ql.exec.TextRecordReader),Some(org.apache.hadoop.hive.ql.exec.TextRecordWriter),true)
   > +- *(2) HashAggregate(keys=[1#16], functions=[], output=[key#7])
   >+- Exchange hashpartitioning(1#16, 1), true, [id=#27]
   >   +- *(1) HashAggregate(keys=[1 AS 1#16], functions=[], output=[1#16])
   >  +- *(1) Scan OneRowRelation[]
   > ```
   > 
   > The physical plan with this PR:
   > 
   > ```
   > ScriptTransformation [key#0, abs(key)#9], cat, [key#2, value#3], 
HiveScriptIOSchema(List(),List(),Some(org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe),Some(org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe),List((field.delim,
 )),List((field.delim,   
)),Some(org.apache.hadoop.hive.ql.exec.TextRecordReader),Some(org.apache.hadoop.hive.ql.exec.TextRecordWriter),true)
   > +- *(2) HashAggregate(keys=[1#10], functions=[], output=[key#0, 
abs(key)#9])
   >+- Exchange hashpartitioning(1#10, 1), true, [id=#21]
   >   +- *(1) HashAggregate(keys=[1 AS 1#10], functions=[], output=[1#10])
   >  +- *(1) Scan OneRowRelation[]
   > ```
   > 
   > Difference:
   > `+- *(2) HashAggregate(keys=[1#16], functions=[], output=[key#7]) `vs
   > `+- *(2) HashAggregate(keys=[1#10], functions=[], output=[key#0, 
abs(key)#9]) `
   
   This right since we now tread transform's child as an entire logical plan. 
It 's final output is two column, it's right.  And transform use it 's out put 
as transform 's input. Reasonable


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #26352: [SPARK-29696][SQL][TESTS] Port groupingsets.sql

[GitBox]

SparkQA commented on issue #26352: [SPARK-29696][SQL][TESTS] Port 
groupingsets.sql
URL: https://github.com/apache/spark/pull/26352#issuecomment-548696207
 
 
   **[Test build #113067 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113067/testReport)**
 for PR 26352 at commit 
[`6e733fe`](https://github.com/apache/spark/commit/6e733fe510a6f8b1b2aa53d9884865ca1418c560).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #26352: [SPARK-29696][SQL][TESTS] Port groupingsets.sql

[GitBox]

SparkQA removed a comment on issue #26352: [SPARK-29696][SQL][TESTS] Port 
groupingsets.sql
URL: https://github.com/apache/spark/pull/26352#issuecomment-548657216
 
 
   **[Test build #113067 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113067/testReport)**
 for PR 26352 at commit 
[`6e733fe`](https://github.com/apache/spark/commit/6e733fe510a6f8b1b2aa53d9884865ca1418c560).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26352: [SPARK-29696][SQL][TESTS] Port groupingsets.sql

[GitBox]

AmplabJenkins commented on issue #26352: [SPARK-29696][SQL][TESTS] Port 
groupingsets.sql
URL: https://github.com/apache/spark/pull/26352#issuecomment-548696642
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113067/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26352: [SPARK-29696][SQL][TESTS] Port groupingsets.sql

[GitBox]

AmplabJenkins commented on issue #26352: [SPARK-29696][SQL][TESTS] Port 
groupingsets.sql
URL: https://github.com/apache/spark/pull/26352#issuecomment-548696640
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26352: [SPARK-29696][SQL][TESTS] Port groupingsets.sql

[GitBox]

AmplabJenkins removed a comment on issue #26352: [SPARK-29696][SQL][TESTS] Port 
groupingsets.sql
URL: https://github.com/apache/spark/pull/26352#issuecomment-548696640
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26352: [SPARK-29696][SQL][TESTS] Port groupingsets.sql

[GitBox]

AmplabJenkins removed a comment on issue #26352: [SPARK-29696][SQL][TESTS] Port 
groupingsets.sql
URL: https://github.com/apache/spark/pull/26352#issuecomment-548696642
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113067/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #26314: [SPARK-29371][SQL] Support interval field values with fractional parts

[GitBox]

SparkQA commented on issue #26314: [SPARK-29371][SQL] Support interval field 
values with fractional parts
URL: https://github.com/apache/spark/pull/26314#issuecomment-548698845
 
 
   **[Test build #113068 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113068/testReport)**
 for PR 26314 at commit 
[`b071e09`](https://github.com/apache/spark/commit/b071e09a2914dcfb566112cb77d8e187ab0d5a7b).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #26314: [SPARK-29371][SQL] Support interval field values with fractional parts

[GitBox]

SparkQA removed a comment on issue #26314: [SPARK-29371][SQL] Support interval 
field values with fractional parts
URL: https://github.com/apache/spark/pull/26314#issuecomment-548658539
 
 
   **[Test build #113068 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113068/testReport)**
 for PR 26314 at commit 
[`b071e09`](https://github.com/apache/spark/commit/b071e09a2914dcfb566112cb77d8e187ab0d5a7b).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26314: [SPARK-29371][SQL] Support interval field values with fractional parts

[GitBox]

AmplabJenkins commented on issue #26314: [SPARK-29371][SQL] Support interval 
field values with fractional parts
URL: https://github.com/apache/spark/pull/26314#issuecomment-548699258
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113068/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26314: [SPARK-29371][SQL] Support interval field values with fractional parts

[GitBox]

AmplabJenkins commented on issue #26314: [SPARK-29371][SQL] Support interval 
field values with fractional parts
URL: https://github.com/apache/spark/pull/26314#issuecomment-548699252
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26314: [SPARK-29371][SQL] Support interval field values with fractional parts

[GitBox]

AmplabJenkins removed a comment on issue #26314: [SPARK-29371][SQL] Support 
interval field values with fractional parts
URL: https://github.com/apache/spark/pull/26314#issuecomment-548699252
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26314: [SPARK-29371][SQL] Support interval field values with fractional parts

[GitBox]

AmplabJenkins removed a comment on issue #26314: [SPARK-29371][SQL] Support 
interval field values with fractional parts
URL: https://github.com/apache/spark/pull/26314#issuecomment-548699258
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113068/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] viirya opened a new pull request #26354: [SPARK-29353][SQL] Fallback AlterTableAlterColumnStatement to v1 AlterTableChangeColumnCommand

[GitBox]

viirya opened a new pull request #26354: [SPARK-29353][SQL] Fallback 
AlterTableAlterColumnStatement to v1 AlterTableChangeColumnCommand
URL: https://github.com/apache/spark/pull/26354
 
 
   
   
   ### What changes were proposed in this pull request?
   
   
   If the resolved table is v1 table, AlterTableAlterColumnStatement fallbacks 
to v1 AlterTableChangeColumnCommand.
   
   ### Why are the changes needed?
   
   
   To make the catalog/table lookup logic consistent.
   
   ### Does this PR introduce any user-facing change?
   
   
   Yes, a ALTER TABLE ALTER COLUMN command previously fails on v1 tables. After 
this, it falls back to v1 AlterTableChangeColumnCommand.
   
   ### How was this patch tested?
   
   
   Unit test.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #26354: [SPARK-29353][SQL] Fallback AlterTableAlterColumnStatement to v1 AlterTableChangeColumnCommand

[GitBox]

SparkQA commented on issue #26354: [SPARK-29353][SQL] Fallback 
AlterTableAlterColumnStatement to v1 AlterTableChangeColumnCommand
URL: https://github.com/apache/spark/pull/26354#issuecomment-548701092
 
 
   **[Test build #113080 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113080/testReport)**
 for PR 26354 at commit 
[`45ed688`](https://github.com/apache/spark/commit/45ed6887fb20e21f6e52d78c7b32ca67a6e97628).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on issue #24898: [SPARK-22340][PYTHON] Add a mode to pin Python thread into JVM's

[GitBox]

HyukjinKwon commented on issue #24898: [SPARK-22340][PYTHON] Add a mode to pin 
Python thread into JVM's
URL: https://github.com/apache/spark/pull/24898#issuecomment-548701197
 
 
   reteat this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] viirya commented on a change in pull request #26338: [SPARK-29680][SQL] ALTER TABLE CHANGE COLUMN should look up catalog/table like v2 commands

[GitBox]

viirya commented on a change in pull request #26338: [SPARK-29680][SQL] ALTER 
TABLE CHANGE COLUMN should look up catalog/table like v2 commands
URL: https://github.com/apache/spark/pull/26338#discussion_r341480740
 
 

 ##
 File path: 
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
 ##
 @@ -151,7 +151,7 @@ statement
 | ALTER TABLE multipartIdentifier
 (ALTER | CHANGE) COLUMN? qualifiedName
 (TYPE dataType)? (COMMENT comment=STRING)? colPosition?
#alterTableColumn
-| ALTER TABLE tableIdentifier partitionSpec?
+| ALTER TABLE multipartIdentifier partitionSpec?
 
 Review comment:
   Seems we do not really support it, though we allow such syntax:
   
   
https://github.com/apache/spark/blob/7aca0dd658b8bda05574b3df3254aaf66eb2a174/sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala#L343-L351
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26354: [SPARK-29353][SQL] Fallback AlterTableAlterColumnStatement to v1 AlterTableChangeColumnCommand

[GitBox]

AmplabJenkins commented on issue #26354: [SPARK-29353][SQL] Fallback 
AlterTableAlterColumnStatement to v1 AlterTableChangeColumnCommand
URL: https://github.com/apache/spark/pull/26354#issuecomment-548701525
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26354: [SPARK-29353][SQL] Fallback AlterTableAlterColumnStatement to v1 AlterTableChangeColumnCommand

[GitBox]

AmplabJenkins removed a comment on issue #26354: [SPARK-29353][SQL] Fallback 
AlterTableAlterColumnStatement to v1 AlterTableChangeColumnCommand
URL: https://github.com/apache/spark/pull/26354#issuecomment-548701525
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] viirya commented on a change in pull request #26338: [SPARK-29680][SQL] ALTER TABLE CHANGE COLUMN should look up catalog/table like v2 commands

[GitBox]

viirya commented on a change in pull request #26338: [SPARK-29680][SQL] ALTER 
TABLE CHANGE COLUMN should look up catalog/table like v2 commands
URL: https://github.com/apache/spark/pull/26338#discussion_r341480870
 
 

 ##
 File path: 
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
 ##
 @@ -151,7 +151,7 @@ statement
 | ALTER TABLE multipartIdentifier
 (ALTER | CHANGE) COLUMN? qualifiedName
 (TYPE dataType)? (COMMENT comment=STRING)? colPosition?
#alterTableColumn
-| ALTER TABLE tableIdentifier partitionSpec?
+| ALTER TABLE multipartIdentifier partitionSpec?
 
 Review comment:
   Not sure what if the original plan was to support that in the future.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26354: [SPARK-29353][SQL] Fallback AlterTableAlterColumnStatement to v1 AlterTableChangeColumnCommand

[GitBox]

AmplabJenkins commented on issue #26354: [SPARK-29353][SQL] Fallback 
AlterTableAlterColumnStatement to v1 AlterTableChangeColumnCommand
URL: https://github.com/apache/spark/pull/26354#issuecomment-548701526
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17997/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] viirya commented on a change in pull request #26338: [SPARK-29680][SQL] ALTER TABLE CHANGE COLUMN should look up catalog/table like v2 commands

[GitBox]

viirya commented on a change in pull request #26338: [SPARK-29680][SQL] ALTER 
TABLE CHANGE COLUMN should look up catalog/table like v2 commands
URL: https://github.com/apache/spark/pull/26338#discussion_r341480870
 
 

 ##
 File path: 
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
 ##
 @@ -151,7 +151,7 @@ statement
 | ALTER TABLE multipartIdentifier
 (ALTER | CHANGE) COLUMN? qualifiedName
 (TYPE dataType)? (COMMENT comment=STRING)? colPosition?
#alterTableColumn
-| ALTER TABLE tableIdentifier partitionSpec?
+| ALTER TABLE multipartIdentifier partitionSpec?
 
 Review comment:
   Not sure what if the original plan is to support that in the future.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26354: [SPARK-29353][SQL] Fallback AlterTableAlterColumnStatement to v1 AlterTableChangeColumnCommand

[GitBox]

AmplabJenkins removed a comment on issue #26354: [SPARK-29353][SQL] Fallback 
AlterTableAlterColumnStatement to v1 AlterTableChangeColumnCommand
URL: https://github.com/apache/spark/pull/26354#issuecomment-548701526
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17997/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query

[GitBox]

SparkQA commented on issue #22952: [SPARK-20568][SS] Provide option to clean up 
completed files in streaming query
URL: https://github.com/apache/spark/pull/22952#issuecomment-548703401
 
 
   **[Test build #113069 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113069/testReport)**
 for PR 22952 at commit 
[`01f5750`](https://github.com/apache/spark/commit/01f57506c9032d7a31bc383432ed09bf1801d402).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query

[GitBox]

SparkQA removed a comment on issue #22952: [SPARK-20568][SS] Provide option to 
clean up completed files in streaming query
URL: https://github.com/apache/spark/pull/22952#issuecomment-548658566
 
 
   **[Test build #113069 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113069/testReport)**
 for PR 22952 at commit 
[`01f5750`](https://github.com/apache/spark/commit/01f57506c9032d7a31bc383432ed09bf1801d402).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on issue #26340: [WIP][SPARK-29108][SQL] Add new module sql/thriftserver with all code and UT

[GitBox]

AngersZh commented on issue #26340: [WIP][SPARK-29108][SQL] Add new module 
sql/thriftserver  with all code and UT
URL: https://github.com/apache/spark/pull/26340#issuecomment-548703769
 
 
   @juliuszsompolski  Current file tree
   
   ```
   src/
   ├── main
   │   ├── java
   │   │   └── org
   │   │   └── apache
   │   │   └── spark
   │   │   └── sql
   │   │   └── thriftserver
   │   │   ├── AbstractService.java
   │   │   ├── CompositeService.java
   │   │   ├── CookieSigner.java
   │   │   ├── Service.java
   │   │   ├── ServiceException.java
   │   │   ├── ServiceOperations.java
   │   │   ├── ServiceStateChangeListener.java
   │   │   ├── ServiceUtils.java
   │   │   ├── auth
   │   │   │   ├── AnonymousAuthenticationProviderImpl.java
   │   │   │   ├── AuthenticationProviderFactory.java
   │   │   │   ├── CustomAuthenticationProviderImpl.java
   │   │   │   ├── HiveAuthFactory.java
   │   │   │   ├── HttpAuthUtils.java
   │   │   │   ├── HttpAuthenticationException.java
   │   │   │   ├── KerberosSaslHelper.java
   │   │   │   ├── LdapAuthenticationProviderImpl.java
   │   │   │   ├── PamAuthenticationProviderImpl.java
   │   │   │   ├── PasswdAuthenticationProvider.java
   │   │   │   ├── PlainSaslHelper.java
   │   │   │   ├── PlainSaslServer.java
   │   │   │   ├── SaslQOP.java
   │   │   │   ├── TSetIpAddressProcessor.java
   │   │   │   ├── TSubjectAssumingTransport.java
   │   │   │   └── ThriftDelegationTokenManager.java
   │   │   ├── cli
   │   │   │   ├── CLIServiceClient.java
   │   │   │   ├── CLIServiceUtils.java
   │   │   │   ├── FetchOrientation.java
   │   │   │   ├── FetchType.java
   │   │   │   ├── GetInfoType.java
   │   │   │   ├── GetInfoValue.java
   │   │   │   ├── Handle.java
   │   │   │   ├── HandleIdentifier.java
   │   │   │   ├── ICLIService.java
   │   │   │   ├── OperationHandle.java
   │   │   │   ├── OperationState.java
   │   │   │   ├── OperationStatus.java
   │   │   │   ├── OperationType.java
   │   │   │   ├── PatternOrIdentifier.java
   │   │   │   ├── SessionHandle.java
   │   │   │   ├── SparkThriftServerSQLException.java
   │   │   │   ├── Type.java
   │   │   │   ├── operation
   │   │   │   │   └── LogDivertAppender.java
   │   │   │   ├── session
   │   │   │   │   ├── ThriftServerSessionBase.java
   │   │   │   │   └── ThriftServerSessionProxy.java
   │   │   │   ├── thrift
   │   │   │   │   ├── ThriftBinaryCLIService.java
   │   │   │   │   ├── ThriftCLIService.java
   │   │   │   │   ├── ThriftCLIServiceClient.java
   │   │   │   │   ├── ThriftHttpCLIService.java
   │   │   │   │   └── ThriftHttpServlet.java
   │   │   │   └── utils
   │   │   │   └── VariableSubstitution.java
   │   │   └── server
   │   │   ├── ThreadFactoryWithGarbageCleanup.java
   │   │   └── ThreadWithGarbageCleanup.java
   │   └── scala
   │   └── org
   │   └── apache
   │   └── spark
   │   └── sql
   │   └── thriftserver
   │   ├── ReflectionUtils.scala
   │   ├── SparkSQLEnv.scala
   │   ├── cli
   │   │   ├── CLIService.scala
   │   │   ├── ColumnBasedSet.scala
   │   │   ├── ColumnBuffer.scala
   │   │   ├── ColumnDescriptor.scala
   │   │   ├── RowBasedSet.scala
   │   │   ├── RowSet.scala
   │   │   ├── RowSetFactory.scala
   │   │   ├── SchemaMapper.scala
   │   │   ├── TypeDescriptor.scala
   │   │   ├── TypeQualifiers.scala
   │   │   ├── operation
   │    

[GitHub] [spark] AmplabJenkins commented on issue #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query

[GitBox]

AmplabJenkins commented on issue #22952: [SPARK-20568][SS] Provide option to 
clean up completed files in streaming query
URL: https://github.com/apache/spark/pull/22952#issuecomment-548703841
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113069/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query

[GitBox]

AmplabJenkins commented on issue #22952: [SPARK-20568][SS] Provide option to 
clean up completed files in streaming query
URL: https://github.com/apache/spark/pull/22952#issuecomment-548703835
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query

[GitBox]

AmplabJenkins removed a comment on issue #22952: [SPARK-20568][SS] Provide 
option to clean up completed files in streaming query
URL: https://github.com/apache/spark/pull/22952#issuecomment-548703841
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113069/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #22952: [SPARK-20568][SS] Provide option to clean up completed files in streaming query

[GitBox]

AmplabJenkins removed a comment on issue #22952: [SPARK-20568][SS] Provide 
option to clean up completed files in streaming query
URL: https://github.com/apache/spark/pull/22952#issuecomment-548703835
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on issue #26340: [WIP][SPARK-29108][SQL] Add new module sql/thriftserver with all code and UT

[GitBox]

AngersZh commented on issue #26340: [WIP][SPARK-29108][SQL] Add new module 
sql/thriftserver  with all code and UT
URL: https://github.com/apache/spark/pull/26340#issuecomment-548705691
 
 
   ```
   ColumnBasedSet.scala
   ColumnBuffer.scala
ColumnDescriptor.scala
RowBasedSet.scala
   RowSet.scala
   RowSetFactory.scala
   SchemaMapper.scala
   TypeDescriptor.scala
   TypeQualifiers.scala
   ```
   
   Changed a lot with `StructType` `Row` `DataType`


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] MaxGekk opened a new pull request #26355: [SPARK-29651][SQL][2.4] Fix parsing of interval seconds fraction

[GitBox]

MaxGekk opened a new pull request #26355: [SPARK-29651][SQL][2.4] Fix parsing 
of interval seconds fraction
URL: https://github.com/apache/spark/pull/26355
 
 
   ### What changes were proposed in this pull request?
   In the PR, I propose to extract parsing of the seconds interval units to the 
private method `parseNanos` in `CalendarInterval` and modify the code to 
correctly parse the fractional part of the seconds unit of intervals in the 
cases:
   - When the fractional part has less than 9 digits
   - The seconds unit is negative
   
   This is a back port of the commit 
https://github.com/apache/spark/commit/3206a9987001d78cf2f48509a93d73af86f51cfe.
   
   ### Why are the changes needed?
   The changes are needed to fix the issues:
   ```sql
   spark-sql> select interval 10.123456 seconds;
   interval 10 seconds 123 microseconds
   ```
   The correct result must be `interval 10 seconds 123 milliseconds 456 
microseconds`
   ```sql
   spark-sql> select interval -10.123456789 seconds;
   interval -9 seconds -876 milliseconds -544 microseconds
   ```
   but the whole interval should be negated, and the result must be `interval 
-10 seconds -123 milliseconds -456 microseconds`, taking into account the 
truncation to microseconds.
   
   ### Does this PR introduce any user-facing change?
   Yes. After changes:
   ```sql
   spark-sql> select interval 10.123456 seconds;
   interval 10 seconds 123 milliseconds 456 microseconds
   spark-sql> select interval -10.123456789 seconds;
   interval -10 seconds -123 milliseconds -456 microseconds
   ```
   
   ### How was this patch tested?
   By existing test suite, `literals.sql` and new tests in 
`ExpressionParserSuite`.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #26355: [SPARK-29651][SQL][2.4] Fix parsing of interval seconds fraction

[GitBox]

SparkQA commented on issue #26355: [SPARK-29651][SQL][2.4] Fix parsing of 
interval seconds fraction
URL: https://github.com/apache/spark/pull/26355#issuecomment-548706718
 
 
   **[Test build #113081 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113081/testReport)**
 for PR 26355 at commit 
[`56faeb0`](https://github.com/apache/spark/commit/56faeb0a637264fb6245196293afcfe96543be43).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #26134: [SPARK-29486][SQL] CalendarInterval should have 3 fields: months, days and microseconds

[GitBox]

SparkQA commented on issue #26134: [SPARK-29486][SQL] CalendarInterval should 
have 3 fields: months, days and microseconds
URL: https://github.com/apache/spark/pull/26134#issuecomment-548706524
 
 
   **[Test build #113073 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113073/testReport)**
 for PR 26134 at commit 
[`2f90189`](https://github.com/apache/spark/commit/2f901894eb24d7deec31947708b9afdaae2a4866).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #26134: [SPARK-29486][SQL] CalendarInterval should have 3 fields: months, days and microseconds

[GitBox]

SparkQA removed a comment on issue #26134: [SPARK-29486][SQL] CalendarInterval 
should have 3 fields: months, days and microseconds
URL: https://github.com/apache/spark/pull/26134#issuecomment-548674854
 
 
   **[Test build #113073 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113073/testReport)**
 for PR 26134 at commit 
[`2f90189`](https://github.com/apache/spark/commit/2f901894eb24d7deec31947708b9afdaae2a4866).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26134: [SPARK-29486][SQL] CalendarInterval should have 3 fields: months, days and microseconds

[GitBox]

AmplabJenkins commented on issue #26134: [SPARK-29486][SQL] CalendarInterval 
should have 3 fields: months, days and microseconds
URL: https://github.com/apache/spark/pull/26134#issuecomment-548707028
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113073/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26134: [SPARK-29486][SQL] CalendarInterval should have 3 fields: months, days and microseconds

[GitBox]

AmplabJenkins commented on issue #26134: [SPARK-29486][SQL] CalendarInterval 
should have 3 fields: months, days and microseconds
URL: https://github.com/apache/spark/pull/26134#issuecomment-548707020
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26355: [SPARK-29651][SQL][2.4] Fix parsing of interval seconds fraction

[GitBox]

AmplabJenkins commented on issue #26355: [SPARK-29651][SQL][2.4] Fix parsing of 
interval seconds fraction
URL: https://github.com/apache/spark/pull/26355#issuecomment-548707090
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17998/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26134: [SPARK-29486][SQL] CalendarInterval should have 3 fields: months, days and microseconds

[GitBox]

AmplabJenkins removed a comment on issue #26134: [SPARK-29486][SQL] 
CalendarInterval should have 3 fields: months, days and microseconds
URL: https://github.com/apache/spark/pull/26134#issuecomment-548707028
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113073/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26355: [SPARK-29651][SQL][2.4] Fix parsing of interval seconds fraction

[GitBox]

AmplabJenkins commented on issue #26355: [SPARK-29651][SQL][2.4] Fix parsing of 
interval seconds fraction
URL: https://github.com/apache/spark/pull/26355#issuecomment-548707087
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26355: [SPARK-29651][SQL][2.4] Fix parsing of interval seconds fraction

[GitBox]

AmplabJenkins removed a comment on issue #26355: [SPARK-29651][SQL][2.4] Fix 
parsing of interval seconds fraction
URL: https://github.com/apache/spark/pull/26355#issuecomment-548707090
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17998/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on issue #26340: [WIP][SPARK-29108][SQL] Add new module sql/thriftserver with all code and UT

[GitBox]

AngersZh commented on issue #26340: [WIP][SPARK-29108][SQL] Add new module 
sql/thriftserver  with all code and UT
URL: https://github.com/apache/spark/pull/26340#issuecomment-548707215
 
 
   @juliuszsompolski 
   Move SparkThriftServer2 to  SparkThriftServer, and Integrated startup code 
in main method.
   Remove unexecuted  code in SparkThriftServer such as 
`startupShutdownMessage`, replace unexecuted `startHiveServer2()` to 
`startSparkThriftServer()`


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26355: [SPARK-29651][SQL][2.4] Fix parsing of interval seconds fraction

[GitBox]

AmplabJenkins removed a comment on issue #26355: [SPARK-29651][SQL][2.4] Fix 
parsing of interval seconds fraction
URL: https://github.com/apache/spark/pull/26355#issuecomment-548707087
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26134: [SPARK-29486][SQL] CalendarInterval should have 3 fields: months, days and microseconds

[GitBox]

AmplabJenkins removed a comment on issue #26134: [SPARK-29486][SQL] 
CalendarInterval should have 3 fields: months, days and microseconds
URL: https://github.com/apache/spark/pull/26134#issuecomment-548707020
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] LinhongLiu commented on issue #26134: [SPARK-29486][SQL] CalendarInterval should have 3 fields: months, days and microseconds

[GitBox]

LinhongLiu commented on issue #26134: [SPARK-29486][SQL] CalendarInterval 
should have 3 fields: months, days and microseconds
URL: https://github.com/apache/spark/pull/26134#issuecomment-548707716
 
 
   @cloud-fan tests are all pass, could you please check this PR again?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM with aggregation.

[GitBox]

AngersZh commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM 
with aggregation.
URL: https://github.com/apache/spark/pull/25028#issuecomment-548709822
 
 
   @alfozan 
   You can see my branch's newest change. and check the update


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM with aggregation.

[GitBox]

AngersZh commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM 
with aggregation.
URL: https://github.com/apache/spark/pull/25028#issuecomment-548710975
 
 
   > Difference:
   > `+- *(2) HashAggregate(keys=[1#16], functions=[], output=[key#7]) `vs
   > `+- *(2) HashAggregate(keys=[1#10], functions=[], output=[key#0, 
abs(key)#9])`
   
   Seems after my pr. the output is right.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AngersZhuuuu edited a comment on issue #25028: [SPARK-28227][SQL] Support TRANSFORM with aggregation.

[GitBox]

AngersZh edited a comment on issue #25028: [SPARK-28227][SQL] Support 
TRANSFORM with aggregation.
URL: https://github.com/apache/spark/pull/25028#issuecomment-548710975
 
 
   > Difference:
   > `+- *(2) HashAggregate(keys=[1#16], functions=[], output=[key#7]) `vs
   > `+- *(2) HashAggregate(keys=[1#10], functions=[], output=[key#0, 
abs(key)#9])`
   
   Seems after my pr. the output is right. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] fuwhu opened a new pull request #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxDataSourceScanMetadataStringLength'

[GitBox]

fuwhu opened a new pull request #26356: [SPARK-29707][SQL] Add configuration 
'spark.sql.maxDataSourceScanMetadataStringLength'
URL: https://github.com/apache/spark/pull/26356
 
 
   ### What changes were proposed in this pull request?
   Add configuration 'spark.sql.maxDataSourceScanMetadataStringLength' to set 
the max string length of one metadata in DataSourceScanExec.
   
   
   
   ### Why are the changes needed?
   To make the max length of metadata string of `DataSourceScanExec` 
configurable, please see 
[SPARK-29707](https://issues.apache.org/jira/browse/SPARK-29707) for details.
   
   
   ### Does this PR introduce any user-facing change?
   Yes, user can configure the parameter to get more information about data 
source scan metadata.
   
   
   ### How was this patch tested?
   Manual test.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxDataSourceScanMetadataStringLength'

[GitBox]

AmplabJenkins commented on issue #26356: [SPARK-29707][SQL] Add configuration 
'spark.sql.maxDataSourceScanMetadataStringLength'
URL: https://github.com/apache/spark/pull/26356#issuecomment-548714602
 
 
   Can one of the admins verify this patch?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxDataSourceScanMetadataStringLength'

[GitBox]

AmplabJenkins commented on issue #26356: [SPARK-29707][SQL] Add configuration 
'spark.sql.maxDataSourceScanMetadataStringLength'
URL: https://github.com/apache/spark/pull/26356#issuecomment-548714945
 
 
   Can one of the admins verify this patch?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] alfozan commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM with aggregation.

[GitBox]

alfozan commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM with 
aggregation.
URL: https://github.com/apache/spark/pull/25028#issuecomment-548715130
 
 
   > @alfozan
   > You can see my branch's newest change. and check the update
   
   Could you share a link to the new branch/PR?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxDataSourceScanMetadataStringLength'

[GitBox]

AmplabJenkins removed a comment on issue #26356: [SPARK-29707][SQL] Add 
configuration 'spark.sql.maxDataSourceScanMetadataStringLength'
URL: https://github.com/apache/spark/pull/26356#issuecomment-548714602
 
 
   Can one of the admins verify this patch?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #26352: [SPARK-29696][SQL][TESTS] Port groupingsets.sql

[GitBox]

SparkQA commented on issue #26352: [SPARK-29696][SQL][TESTS] Port 
groupingsets.sql
URL: https://github.com/apache/spark/pull/26352#issuecomment-548715516
 
 
   **[Test build #113071 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113071/testReport)**
 for PR 26352 at commit 
[`94b7e79`](https://github.com/apache/spark/commit/94b7e79a97ab303fb227dd2f256990efd3c93849).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #26352: [SPARK-29696][SQL][TESTS] Port groupingsets.sql

[GitBox]

SparkQA removed a comment on issue #26352: [SPARK-29696][SQL][TESTS] Port 
groupingsets.sql
URL: https://github.com/apache/spark/pull/26352#issuecomment-548670945
 
 
   **[Test build #113071 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113071/testReport)**
 for PR 26352 at commit 
[`94b7e79`](https://github.com/apache/spark/commit/94b7e79a97ab303fb227dd2f256990efd3c93849).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26352: [SPARK-29696][SQL][TESTS] Port groupingsets.sql

[GitBox]

AmplabJenkins commented on issue #26352: [SPARK-29696][SQL][TESTS] Port 
groupingsets.sql
URL: https://github.com/apache/spark/pull/26352#issuecomment-548715955
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113071/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26352: [SPARK-29696][SQL][TESTS] Port groupingsets.sql

[GitBox]

AmplabJenkins commented on issue #26352: [SPARK-29696][SQL][TESTS] Port 
groupingsets.sql
URL: https://github.com/apache/spark/pull/26352#issuecomment-548715950
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] wangyum commented on a change in pull request #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxDataSourceScanMetadataStringLength'

[GitBox]

wangyum commented on a change in pull request #26356: [SPARK-29707][SQL] Add 
configuration 'spark.sql.maxDataSourceScanMetadataStringLength'
URL: https://github.com/apache/spark/pull/26356#discussion_r341495574
 
 

 ##
 File path: 
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
 ##
 @@ -58,7 +58,8 @@ trait DataSourceScanExec extends LeafExecNode {
   override def simpleString(maxFields: Int): String = {
 val metadataEntries = metadata.toSeq.sorted.map {
   case (key, value) =>
-key + ": " + StringUtils.abbreviate(redact(value), 100)
+key + ": " + StringUtils.abbreviate(redact(value),
+  SQLConf.get.maxDataSourceScanMetadataStringLength)
 
 Review comment:
   `SQLConf.get.maxDataSourceScanMetadataStringLength)` -> 
`conf.maxDataSourceScanMetadataStringLength)`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26352: [SPARK-29696][SQL][TESTS] Port groupingsets.sql

[GitBox]

AmplabJenkins removed a comment on issue #26352: [SPARK-29696][SQL][TESTS] Port 
groupingsets.sql
URL: https://github.com/apache/spark/pull/26352#issuecomment-548715955
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113071/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26352: [SPARK-29696][SQL][TESTS] Port groupingsets.sql

[GitBox]

AmplabJenkins removed a comment on issue #26352: [SPARK-29696][SQL][TESTS] Port 
groupingsets.sql
URL: https://github.com/apache/spark/pull/26352#issuecomment-548715950
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM with aggregation.

[GitBox]

AngersZh commented on issue #25028: [SPARK-28227][SQL] Support TRANSFORM 
with aggregation.
URL: https://github.com/apache/spark/pull/25028#issuecomment-548716323
 
 
   > > @alfozan
   > > You can see my branch's newest change. and check the update
   > 
   > Could you share a link to the new branch/PR?
   
   
https://github.com/AngersZh/spark/commit/f18a6f99043ec26ac19a8337f41194b6f972090c


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] fuwhu commented on a change in pull request #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxDataSourceScanMetadataStringLength'

[GitBox]

fuwhu commented on a change in pull request #26356: [SPARK-29707][SQL] Add 
configuration 'spark.sql.maxDataSourceScanMetadataStringLength'
URL: https://github.com/apache/spark/pull/26356#discussion_r341498093
 
 

 ##
 File path: 
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
 ##
 @@ -58,7 +58,8 @@ trait DataSourceScanExec extends LeafExecNode {
   override def simpleString(maxFields: Int): String = {
 val metadataEntries = metadata.toSeq.sorted.map {
   case (key, value) =>
-key + ": " + StringUtils.abbreviate(redact(value), 100)
+key + ": " + StringUtils.abbreviate(redact(value),
+  SQLConf.get.maxDataSourceScanMetadataStringLength)
 
 Review comment:
   thanks, udpated


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] wangyum commented on issue #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxDataSourceScanMetadataStringLength'

[GitBox]

wangyum commented on issue #26356: [SPARK-29707][SQL] Add configuration 
'spark.sql.maxDataSourceScanMetadataStringLength'
URL: https://github.com/apache/spark/pull/26356#issuecomment-548718473
 
 
   ok to test


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxDataSourceScanMetadataStringLength'

[GitBox]

AmplabJenkins removed a comment on issue #26356: [SPARK-29707][SQL] Add 
configuration 'spark.sql.maxDataSourceScanMetadataStringLength'
URL: https://github.com/apache/spark/pull/26356#issuecomment-548714945
 
 
   Can one of the admins verify this patch?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxDataSourceScanMetadataStringLength'

[GitBox]

SparkQA commented on issue #26356: [SPARK-29707][SQL] Add configuration 
'spark.sql.maxDataSourceScanMetadataStringLength'
URL: https://github.com/apache/spark/pull/26356#issuecomment-548718809
 
 
   **[Test build #113082 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113082/testReport)**
 for PR 26356 at commit 
[`f207bff`](https://github.com/apache/spark/commit/f207bffbda506d5919ec60357db6a62742c450f7).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxDataSourceScanMetadataStringLength'

[GitBox]

AmplabJenkins commented on issue #26356: [SPARK-29707][SQL] Add configuration 
'spark.sql.maxDataSourceScanMetadataStringLength'
URL: https://github.com/apache/spark/pull/26356#issuecomment-548719186
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxDataSourceScanMetadataStringLength'

[GitBox]

AmplabJenkins commented on issue #26356: [SPARK-29707][SQL] Add configuration 
'spark.sql.maxDataSourceScanMetadataStringLength'
URL: https://github.com/apache/spark/pull/26356#issuecomment-548719188
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17999/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxDataSourceScanMetadataStringLength'

[GitBox]

AmplabJenkins removed a comment on issue #26356: [SPARK-29707][SQL] Add 
configuration 'spark.sql.maxDataSourceScanMetadataStringLength'
URL: https://github.com/apache/spark/pull/26356#issuecomment-548719186
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26356: [SPARK-29707][SQL] Add configuration 'spark.sql.maxDataSourceScanMetadataStringLength'

[GitBox]

AmplabJenkins removed a comment on issue #26356: [SPARK-29707][SQL] Add 
configuration 'spark.sql.maxDataSourceScanMetadataStringLength'
URL: https://github.com/apache/spark/pull/26356#issuecomment-548719188
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17999/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #26338: [SPARK-29680][SQL] ALTER TABLE CHANGE COLUMN should look up catalog/table like v2 commands

[GitBox]

cloud-fan commented on a change in pull request #26338: [SPARK-29680][SQL] 
ALTER TABLE CHANGE COLUMN should look up catalog/table like v2 commands
URL: https://github.com/apache/spark/pull/26338#discussion_r341499950
 
 

 ##
 File path: 
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
 ##
 @@ -151,7 +151,7 @@ statement
 | ALTER TABLE multipartIdentifier
 (ALTER | CHANGE) COLUMN? qualifiedName
 (TYPE dataType)? (COMMENT comment=STRING)? colPosition?
#alterTableColumn
-| ALTER TABLE tableIdentifier partitionSpec?
+| ALTER TABLE multipartIdentifier partitionSpec?
 
 Review comment:
   Let's simply remove this syntax. We have ALTER TABLE CHANGE COLUMN and ALTER 
TABLE RENAME COLUMN, which is good enough to me.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] planga82 commented on issue #26082: [SPARK-29431][WebUI] Improve Web UI / Sql tab visualization with cached dataframes.

[GitBox]

planga82 commented on issue #26082: [SPARK-29431][WebUI] Improve Web UI / Sql 
tab visualization with cached dataframes.
URL: https://github.com/apache/spark/pull/26082#issuecomment-548720773
 
 
   About your questions, yes, it add the relation to the graph always when 
appears InMemoryTableScan node. Looking for this kind of node it seems only 
appear when the chache method is called so there is no other uses of 
InMemorytableScan.
   There is no special indication that this is a cached dataframe, only the 
node type (InMemoryTableScan). We could try to put a diferent color to the node 
to clarify it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #26353: [SPARK-29590][WEBUI]JDBC/ODBC tab in the spark UI support hide tables, to make it consistent with other tabs.

[GitBox]

SparkQA commented on issue #26353: [SPARK-29590][WEBUI]JDBC/ODBC tab in the 
spark UI support hide tables, to make it consistent with other tabs.
URL: https://github.com/apache/spark/pull/26353#issuecomment-548723684
 
 
   **[Test build #113077 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113077/testReport)**
 for PR 26353 at commit 
[`e86b15a`](https://github.com/apache/spark/commit/e86b15a51b8af06844fcfd5fa75c1115145bd88e).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
 * `trait HasRelativeError extends Params `
 * `class _ImputerParams(HasInputCol, HasInputCols, HasOutputCol, 
HasOutputCols, HasRelativeError):`
 * `class _OneHotEncoderParams(HasInputCol, HasInputCols, HasOutputCol, 
HasOutputCols,`
 * `class _RobustScalerParams(HasInputCol, HasOutputCol, HasRelativeError):`
 * `class HasRelativeError(Params):`
 * `  class ResolveCoalesceHints(conf: SQLConf) extends Rule[LogicalPlan] `
 * `case class AlterTableRecoverPartitionsStatement(`
 * `case class AlterTableRenamePartitionStatement(`
 * `case class DropNamespaceStatement(`
 * `case class LoadDataStatement(`
 * `case class ShowColumnsStatement(`
 * `case class DropNamespace(`
 * `case class DataSourceV2ScanRelation(`
 * `case class OptimizeLocalShuffleReader(conf: SQLConf) extends 
Rule[SparkPlan] `
 * `case class LocalShuffleReaderExec(child: QueryStageExec) extends 
UnaryExecNode `
 * `case class DropNamespaceExec(`


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #26353: [SPARK-29590][WEBUI]JDBC/ODBC tab in the spark UI support hide tables, to make it consistent with other tabs.

[GitBox]

SparkQA removed a comment on issue #26353: [SPARK-29590][WEBUI]JDBC/ODBC tab in 
the spark UI support hide tables, to make it consistent with other tabs.
URL: https://github.com/apache/spark/pull/26353#issuecomment-548688871
 
 
   **[Test build #113077 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/113077/testReport)**
 for PR 26353 at commit 
[`e86b15a`](https://github.com/apache/spark/commit/e86b15a51b8af06844fcfd5fa75c1115145bd88e).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #26353: [SPARK-29590][WEBUI]JDBC/ODBC tab in the spark UI support hide tables, to make it consistent with other tabs.

[GitBox]

AmplabJenkins removed a comment on issue #26353: [SPARK-29590][WEBUI]JDBC/ODBC 
tab in the spark UI support hide tables, to make it consistent with other tabs.
URL: https://github.com/apache/spark/pull/26353#issuecomment-548724288
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113077/
   Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #26353: [SPARK-29590][WEBUI]JDBC/ODBC tab in the spark UI support hide tables, to make it consistent with other tabs.

[GitBox]

AmplabJenkins commented on issue #26353: [SPARK-29590][WEBUI]JDBC/ODBC tab in 
the spark UI support hide tables, to make it consistent with other tabs.
URL: https://github.com/apache/spark/pull/26353#issuecomment-548724281
 
 
   Merged build finished. Test PASSed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org