[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23178 thanks for the review, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23178 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99502/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23178 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23178 **[Test build #99502 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99502/testReport)** for PR 23178 at commit [`ad6605e`](https://github.com/apache/spark/commit/ad6605e5f049cc21ed5754b754b695bcac6d4502). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23178 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23178 **[Test build #99502 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99502/testReport)** for PR 23178 at commit [`ad6605e`](https://github.com/apache/spark/commit/ad6605e5f049cc21ed5754b754b695bcac6d4502). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23178 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5569/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23178 +1 as well --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23178 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99457/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23178 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23178 **[Test build #99457 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99457/testReport)** for PR 23178 at commit [`69bc466`](https://github.com/apache/spark/commit/69bc466fc2047fc9912b0ebce3930c2cb57cacb1). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `sealed trait UserDefinedFunction ` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23178 **[Test build #4448 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4448/testReport)** for PR 23178 at commit [`69bc466`](https://github.com/apache/spark/commit/69bc466fc2047fc9912b0ebce3930c2cb57cacb1). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23178 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23178 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99454/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23178 **[Test build #99454 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99454/testReport)** for PR 23178 at commit [`700334f`](https://github.com/apache/spark/commit/700334f3b14cfe88d6141c8a99ec339ec7a16afc). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `// [SPARK-26216][SQL] Do not use case class as public API (UserDefinedFunction)` * `trait UserDefinedFunction ` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23178 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23178 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5525/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23178 **[Test build #99457 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99457/testReport)** for PR 23178 at commit [`69bc466`](https://github.com/apache/spark/commit/69bc466fc2047fc9912b0ebce3930c2cb57cacb1). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/23178
Good idea to have it sealed!
> On Nov 29, 2018, at 7:04 AM, Sean Owen wrote:
>
> @srowen commented on this pull request.
>
> In
sql/core/src/main/scala/org/apache/spark/sql/expressions/UserDefinedFunction.scala:
>
> > if (inputTypes.isDefined) {
>assert(inputTypes.get.length == nullableTypes.get.length)
> }
>
> +val inputsNullSafe = if (nullableTypes.isEmpty) {
> You can use getOrElse here and even inline this into the call below, but
I don't really care.
>
> In
sql/core/src/main/scala/org/apache/spark/sql/expressions/UserDefinedFunction.scala:
>
> > @@ -38,114 +38,108 @@ import org.apache.spark.sql.types.DataType
> * @since 1.3.0
> */
> @Stable
> -case class UserDefinedFunction protected[sql] (
> -f: AnyRef,
> -dataType: DataType,
> -inputTypes: Option[Seq[DataType]]) {
> -
> - private var _nameOption: Option[String] = None
> - private var _nullable: Boolean = true
> - private var _deterministic: Boolean = true
> -
> - // This is a `var` instead of in the constructor for backward
compatibility of this case class.
> - // TODO: revisit this case class in Spark 3.0, and narrow down the
public surface.
> - private[sql] var nullableTypes: Option[Seq[Boolean]] = None
> +trait UserDefinedFunction {
> Should we make this sealed? I'm not sure. Would any user ever extend this
meaningfully? I kind of worry someone will start doing so; maybe they already
subclass it in some cases though. Elsewhere it might help the compiler
understand in match statements that there is only ever one type of UDF class to
match on.
>
> â
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or mute the thread.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23178 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5523/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23178 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23178 **[Test build #99454 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99454/testReport)** for PR 23178 at commit [`700334f`](https://github.com/apache/spark/commit/700334f3b14cfe88d6141c8a99ec339ec7a16afc). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23178: [SPARK-26216][SQL] Do not use case class as public API (...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23178 cc @rxin @srowen @gatorsmile @HyukjinKwon @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
