[GitHub] [spark] SparkQA removed a comment on issue #27842: [SPARK-31078][SQL] Respect aliases in output ordering
SparkQA removed a comment on issue #27842: [SPARK-31078][SQL] Respect aliases in output ordering URL: https://github.com/apache/spark/pull/27842#issuecomment-596163083 **[Test build #119528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119528/testReport)** for PR 27842 at commit [`add187a`](https://github.com/apache/spark/commit/add187ae6443a8db71d24928c625794195604e0c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27842: [SPARK-31078][SQL] Respect aliases in output ordering
SparkQA commented on issue #27842: [SPARK-31078][SQL] Respect aliases in output ordering URL: https://github.com/apache/spark/pull/27842#issuecomment-596177419 **[Test build #119528 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119528/testReport)** for PR 27842 at commit [`add187a`](https://github.com/apache/spark/commit/add187ae6443a8db71d24928c625794195604e0c). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on issue #27821: [SPARK-25908][SQL][FOLLOW-UP] Add Back Multiple Removed APIs
maropu commented on issue #27821: [SPARK-25908][SQL][FOLLOW-UP] Add Back Multiple Removed APIs URL: https://github.com/apache/spark/pull/27821#issuecomment-596176222 btw, could we update the contribution guideline to use a new tag in a title for these kinds of user-facing changes, e.g., `[SPARK-XXX][SQL][API] XXX`? I personally think the tag could make us easily track on-going PRs and commit logs for high-impact changes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #27828: [SPARK-31068][SQL] Avoid IllegalArgumentException in broadcast exchange
maropu commented on a change in pull request #27828: [SPARK-31068][SQL] Avoid
IllegalArgumentException in broadcast exchange
URL: https://github.com/apache/spark/pull/27828#discussion_r389342924
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala
##
@@ -87,9 +87,12 @@ case class BroadcastExchangeExec(
val beforeCollect = System.nanoTime()
// Use executeCollect/executeCollectIterator to avoid conversion
to Scala types
val (numRows, input) = child.executeCollectIterator()
-if (numRows >= 51200) {
+// Since the maximum number of keys that BytesToBytesMap supports
is 1 << 29,
+// and only 70% of the slots can be used before growing in
HashedRelation,
+// here the limitation should not be over 341 million.
+if (numRows >= (1 << 29) / 1.5) {
throw new SparkException(
-s"Cannot broadcast the table with 512 million or more rows:
$numRows rows")
+s"Cannot broadcast the table with 341 million or more rows:
$numRows rows")
Review comment:
Yea, +1 on the Sean's suggestion. btw, could we make this error message
clearer? I personally think the current message looks unobvious for users.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #27842: [SPARK-31078][SQL] Respect aliases in output ordering
maropu commented on a change in pull request #27842: [SPARK-31078][SQL] Respect
aliases in output ordering
URL: https://github.com/apache/spark/pull/27842#discussion_r389342226
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/sources/BucketedReadSuite.scala
##
@@ -604,6 +604,18 @@ abstract class BucketedReadSuite extends QueryTest with
SQLTestUtils {
}
}
+ test("sort should not be introduced when aliases are used") {
+withSQLConf(SQLConf.AUTO_BROADCASTJOIN_THRESHOLD.key -> "0") {
+ withTable("t") {
+df1.repartition(1).write.format("parquet").bucketBy(8,
"i").sortBy("i").saveAsTable("t")
Review comment:
Ah, I see. How about the aggregate case?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
AmplabJenkins removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596173255 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
AmplabJenkins removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596173257 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119530/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
AmplabJenkins commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596173255 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
SparkQA removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596169573 **[Test build #119530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119530/testReport)** for PR 27818 at commit [`46a9f21`](https://github.com/apache/spark/commit/46a9f2132ccaaca4de2bfd1152ffa3f1bdebff86). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
AmplabJenkins commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596173257 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119530/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
SparkQA commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596173191 **[Test build #119530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119530/testReport)** for PR 27818 at commit [`46a9f21`](https://github.com/apache/spark/commit/46a9f2132ccaaca4de2bfd1152ffa3f1bdebff86). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on issue #27829: [SPARK-30841][SQL][DOC][FOLLOW-UP] Add version information to the configuration of SQL
beliefer commented on issue #27829: [SPARK-30841][SQL][DOC][FOLLOW-UP] Add version information to the configuration of SQL URL: https://github.com/apache/spark/pull/27829#issuecomment-596172071 @HyukjinKwon @MaxGekk Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
AmplabJenkins removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596171574 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119529/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
AmplabJenkins removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596171568 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
AmplabJenkins commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596171568 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
SparkQA removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596167752 **[Test build #119529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119529/testReport)** for PR 27818 at commit [`c6f1a3e`](https://github.com/apache/spark/commit/c6f1a3e0ec9206a9fec9757fcb7c33497126b53a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
AmplabJenkins commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596171574 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119529/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
SparkQA commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596171493 **[Test build #119529 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119529/testReport)** for PR 27818 at commit [`c6f1a3e`](https://github.com/apache/spark/commit/c6f1a3e0ec9206a9fec9757fcb7c33497126b53a). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class MultilabelClassificationEvaluator @Since(\"3.0.0\") (@Since(\"3.0.0\") override val uid: String)` * `class RankingEvaluator @Since(\"3.0.0\") (@Since(\"3.0.0\") override val uid: String)` * `class RobustScaler @Since(\"3.0.0\") (@Since(\"3.0.0\") override val uid: String)` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
AmplabJenkins removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596170970 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
AmplabJenkins removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596170971 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24263/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
AmplabJenkins commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596170971 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24263/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
AmplabJenkins commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596170970 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
SparkQA commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596170896 **[Test build #119532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119532/testReport)** for PR 27848 at commit [`c6a36c9`](https://github.com/apache/spark/commit/c6a36c93a36b98ae2488a0207eea6425f7aec86e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
AmplabJenkins commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596170542 Build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
AmplabJenkins removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596170542 Build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
AmplabJenkins removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596170544 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24262/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
AmplabJenkins commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596170544 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24262/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
SparkQA commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596170471 **[Test build #119531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119531/testReport)** for PR 27848 at commit [`993e6d2`](https://github.com/apache/spark/commit/993e6d246c18d0be08a72b66f7b88146bb2fdf3d). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
AmplabJenkins removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596169642 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
AmplabJenkins removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596169643 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24261/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
AmplabJenkins commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596169643 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24261/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
AmplabJenkins commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596169642 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
SparkQA commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596169573 **[Test build #119530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119530/testReport)** for PR 27818 at commit [`46a9f21`](https://github.com/apache/spark/commit/46a9f2132ccaaca4de2bfd1152ffa3f1bdebff86). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
AmplabJenkins removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-596169226 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119526/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
AmplabJenkins removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-596169225 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
AmplabJenkins commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-596169225 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
AmplabJenkins commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-596169226 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119526/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
SparkQA removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-596157539 **[Test build #119526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119526/testReport)** for PR 27278 at commit [`0063736`](https://github.com/apache/spark/commit/00637368512645ce85551c98a36789de17ed10bb). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
SparkQA commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-596169177 **[Test build #119526 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119526/testReport)** for PR 27278 at commit [`0063736`](https://github.com/apache/spark/commit/00637368512645ce85551c98a36789de17ed10bb). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a change in pull request #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
zhengruifeng commented on a change in pull request #27818: [SPARK-30929][ML]
ML, GraphX 3.0 QA: API: New Scala APIs, docs
URL: https://github.com/apache/spark/pull/27818#discussion_r389336697
##
File path:
mllib/src/main/scala/org/apache/spark/ml/regression/AFTSurvivalRegression.scala
##
@@ -195,7 +195,8 @@ class AFTSurvivalRegression @Since("1.6.0")
(@Since("1.6.0") override val uid: S
}
@Since("3.0.0")
Review comment:
If we can safely mark it protected, then do not need this since annotation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a change in pull request #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
zhengruifeng commented on a change in pull request #27818: [SPARK-30929][ML]
ML, GraphX 3.0 QA: API: New Scala APIs, docs
URL: https://github.com/apache/spark/pull/27818#discussion_r389336710
##
File path:
mllib/src/main/scala/org/apache/spark/ml/evaluation/MultilabelClassificationEvaluator.scala
##
@@ -34,12 +34,13 @@ import org.apache.spark.sql.types._
*/
@Since("3.0.0")
@Experimental
-class MultilabelClassificationEvaluator (override val uid: String)
+class MultilabelClassificationEvaluator @Since("3.0.0") (@Since("3.0.0")
override val uid: String)
Review comment:
I also do not feel strongly about it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
AmplabJenkins removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596168066 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
AmplabJenkins commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596168066 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
AmplabJenkins removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596168069 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119527/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
AmplabJenkins commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596168069 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119527/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
SparkQA removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596160099 **[Test build #119527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119527/testReport)** for PR 27848 at commit [`4f8be84`](https://github.com/apache/spark/commit/4f8be8491a3cafc5e46e8db814297661629684ba). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
SparkQA commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596167957 **[Test build #119527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119527/testReport)** for PR 27848 at commit [`4f8be84`](https://github.com/apache/spark/commit/4f8be8491a3cafc5e46e8db814297661629684ba). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
AmplabJenkins removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596167850 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
AmplabJenkins commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596167850 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
AmplabJenkins commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596167852 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24260/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
AmplabJenkins removed a comment on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596167852 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24260/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs
SparkQA commented on issue #27818: [SPARK-30929][ML] ML, GraphX 3.0 QA: API: New Scala APIs, docs URL: https://github.com/apache/spark/pull/27818#issuecomment-596167752 **[Test build #119529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119529/testReport)** for PR 27818 at commit [`c6f1a3e`](https://github.com/apache/spark/commit/c6f1a3e0ec9206a9fec9757fcb7c33497126b53a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on issue #27785: [SPARK-30934][ML][DOCS] Update ml-guide and ml-migration-guide for 3.0 release
huaxingao commented on issue #27785: [SPARK-30934][ML][DOCS] Update ml-guide and ml-migration-guide for 3.0 release URL: https://github.com/apache/spark/pull/27785#issuecomment-596167299 Thank you so much! @srowen @viirya @zhengruifeng This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on issue #27842: [SPARK-31078][SQL] Respect aliases in output ordering
imback82 commented on issue #27842: [SPARK-31078][SQL] Respect aliases in output ordering URL: https://github.com/apache/spark/pull/27842#issuecomment-596164531 cc: @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
beliefer commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596163657 > Let's merge this one with another PR later just to fix them in batch a bit more. OK This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27842: [SPARK-31078][SQL] Respect aliases in output ordering
AmplabJenkins commented on issue #27842: [SPARK-31078][SQL] Respect aliases in output ordering URL: https://github.com/apache/spark/pull/27842#issuecomment-596163156 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27842: [SPARK-31078][SQL] Respect aliases in output ordering
AmplabJenkins commented on issue #27842: [SPARK-31078][SQL] Respect aliases in output ordering URL: https://github.com/apache/spark/pull/27842#issuecomment-596163158 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24259/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27842: [SPARK-31078][SQL] Respect aliases in output ordering
AmplabJenkins removed a comment on issue #27842: [SPARK-31078][SQL] Respect aliases in output ordering URL: https://github.com/apache/spark/pull/27842#issuecomment-596163156 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27842: [SPARK-31078][SQL] Respect aliases in output ordering
AmplabJenkins removed a comment on issue #27842: [SPARK-31078][SQL] Respect aliases in output ordering URL: https://github.com/apache/spark/pull/27842#issuecomment-596163158 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24259/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27842: [SPARK-31078][SQL] Respect aliases in output ordering
SparkQA commented on issue #27842: [SPARK-31078][SQL] Respect aliases in output ordering URL: https://github.com/apache/spark/pull/27842#issuecomment-596163083 **[Test build #119528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119528/testReport)** for PR 27842 at commit [`add187a`](https://github.com/apache/spark/commit/add187ae6443a8db71d24928c625794195604e0c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27842: [SPARK-31078][SQL] Respect aliases in output ordering
imback82 commented on a change in pull request #27842: [SPARK-31078][SQL]
Respect aliases in output ordering
URL: https://github.com/apache/spark/pull/27842#discussion_r389331709
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/sources/BucketedReadSuite.scala
##
@@ -604,6 +604,18 @@ abstract class BucketedReadSuite extends QueryTest with
SQLTestUtils {
}
}
+ test("sort should not be introduced when aliases are used") {
+withSQLConf(SQLConf.AUTO_BROADCASTJOIN_THRESHOLD.key -> "0") {
+ withTable("t") {
+df1.repartition(1).write.format("parquet").bucketBy(8,
"i").sortBy("i").saveAsTable("t")
Review comment:
Do you mean something like the following?
```
val t1 = spark.range(10).selectExpr("id as k1").orderBy("k1")
.selectExpr("k1 as k11").orderBy("k11")
val t2 = spark.range(10).selectExpr("id as k2").orderBy("k2")
t1.join(t2, t1("k11") === t2("k2")).explain(true)
```
Extra `Sort` is optimized away, so it doesn't affect the physical plan (no
extra sort).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #27829: [SPARK-30841][SQL][DOC][FOLLOW-UP] Add version information to the configuration of SQL
HyukjinKwon closed pull request #27829: [SPARK-30841][SQL][DOC][FOLLOW-UP] Add version information to the configuration of SQL URL: https://github.com/apache/spark/pull/27829 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #27847: [SPARK-31002][CORE][DOC] Add version information to the configuration of Core
HyukjinKwon closed pull request #27847: [SPARK-31002][CORE][DOC] Add version information to the configuration of Core URL: https://github.com/apache/spark/pull/27847 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27842: [SPARK-31078][SQL] Respect aliases in output ordering
imback82 commented on a change in pull request #27842: [SPARK-31078][SQL]
Respect aliases in output ordering
URL: https://github.com/apache/spark/pull/27842#discussion_r389331709
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/sources/BucketedReadSuite.scala
##
@@ -604,6 +604,18 @@ abstract class BucketedReadSuite extends QueryTest with
SQLTestUtils {
}
}
+ test("sort should not be introduced when aliases are used") {
+withSQLConf(SQLConf.AUTO_BROADCASTJOIN_THRESHOLD.key -> "0") {
+ withTable("t") {
+df1.repartition(1).write.format("parquet").bucketBy(8,
"i").sortBy("i").saveAsTable("t")
Review comment:
Do you mean something like the following?
```
val t1 = spark.range(10).selectExpr("id as k1").orderBy("k1")
.selectExpr("k1 as k11").orderBy("k11")
val t2 = spark.range(10).selectExpr("id as k2").orderBy("k2")
t1.join(t2, t1("k11") === t2("k2")).explain(true)
```
Extra `Sort` is optimized away, so it doesn't affect the physical plan (no
extra sort) without this change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
HyukjinKwon commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596162499 Let's merge this one with another PR later just to fix them in batch a bit more. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27842: [SPARK-31078][SQL] Respect aliases in output ordering
imback82 commented on a change in pull request #27842: [SPARK-31078][SQL]
Respect aliases in output ordering
URL: https://github.com/apache/spark/pull/27842#discussion_r389331713
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/AliasAwareOutputs.scala
##
@@ -16,16 +16,18 @@
*/
package org.apache.spark.sql.execution
-import org.apache.spark.sql.catalyst.expressions.{Alias, Attribute,
AttributeReference, Expression, NamedExpression}
+import org.apache.spark.sql.catalyst.expressions.{Alias, Attribute,
AttributeReference, Expression, NamedExpression, SortOrder}
import org.apache.spark.sql.catalyst.plans.physical.{HashPartitioning,
Partitioning}
/**
- * A trait that handles aliases in the `outputExpressions` to produce
`outputPartitioning`
- * that satisfies output distribution requirements.
+ * A trait that handles aliases in the `outputExpressions` and
`orderingExpressions` to produce
+ * `outputPartitioning` and `outputOrdering` that satisfy distribution and
ordering requirements.
*/
-trait AliasAwareOutputPartitioning extends UnaryExecNode {
+trait AliasAwareOutputs extends UnaryExecNode {
Review comment:
Yes, good idea.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots
HyukjinKwon commented on a change in pull request #27780: [SPARK-31026] [SQL]
[test-hive1.2] Parquet predicate pushdown on columns with dots
URL: https://github.com/apache/spark/pull/27780#discussion_r389331473
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFiltersBase.scala
##
@@ -45,4 +47,11 @@ trait OrcFiltersBase {
case _: AtomicType => true
case _ => false
}
+
+ /**
+ * The key of the dataTypeMap will be quoted if it contains `dots`.
+ */
+ protected[sql] def quotedDataTypeMap(schema: StructType): Map[String,
DataType] = {
Review comment:
nit: `protected[sql]` -> `protected[orc]`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27780: [SPARK-31026] [SQL] [test-hive1.2] Parquet predicate pushdown on columns with dots
HyukjinKwon commented on a change in pull request #27780: [SPARK-31026] [SQL]
[test-hive1.2] Parquet predicate pushdown on columns with dots
URL: https://github.com/apache/spark/pull/27780#discussion_r389331398
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/sources/filters.scala
##
@@ -32,6 +32,7 @@ import org.apache.spark.annotation.{Evolving, Stable}
sealed abstract class Filter {
/**
* List of columns that are referenced by this filter.
+ * Note that, if a column contains `dots` in name, it will be quoted to
avoid confusion.
Review comment:
This one is actually a pretty breaking change. Not all implementations of
the data sources will have the syntax to handle backquotes - there are so many
non-DBMS implementations outside like elasticsearch, mongodb, etc. which I see
relevant tickets in Spark JIRAs time to time.
In particular, this is a stable API. Can we update the migration guide at
the very least?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
AmplabJenkins commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-596161530 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119525/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
AmplabJenkins commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-596161529 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
AmplabJenkins removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-596161529 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
AmplabJenkins removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-596161530 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119525/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
SparkQA commented on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-596161472 **[Test build #119525 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119525/testReport)** for PR 27570 at commit [`0541f04`](https://github.com/apache/spark/commit/0541f044e669837f813d02b982ef536385ace1d5). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR
SparkQA removed a comment on issue #27570: [SPARK-30820][SPARKR][ML] Add FMClassifier to SparkR URL: https://github.com/apache/spark/pull/27570#issuecomment-596157128 **[Test build #119525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119525/testReport)** for PR 27570 at commit [`0541f04`](https://github.com/apache/spark/commit/0541f044e669837f813d02b982ef536385ace1d5). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR
AmplabJenkins removed a comment on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR URL: https://github.com/apache/spark/pull/27571#issuecomment-596161363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119524/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR
AmplabJenkins removed a comment on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR URL: https://github.com/apache/spark/pull/27571#issuecomment-596161361 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR
AmplabJenkins commented on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR URL: https://github.com/apache/spark/pull/27571#issuecomment-596161361 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR
AmplabJenkins commented on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR URL: https://github.com/apache/spark/pull/27571#issuecomment-596161363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119524/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR
SparkQA removed a comment on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR URL: https://github.com/apache/spark/pull/27571#issuecomment-596157127 **[Test build #119524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119524/testReport)** for PR 27571 at commit [`0adb2cf`](https://github.com/apache/spark/commit/0adb2cfc36623537e91b213297987854838cd642). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR
SparkQA commented on issue #27571: [SPARK-30819][SPARKR][ML] Add FMRegressor wrapper to SparkR URL: https://github.com/apache/spark/pull/27571#issuecomment-596161286 **[Test build #119524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119524/testReport)** for PR 27571 at commit [`0adb2cf`](https://github.com/apache/spark/commit/0adb2cfc36623537e91b213297987854838cd642). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on issue #27834: revert [SPARK-24640][SQL] Return `NULL` from `size(NULL)` by default
HeartSaVioR commented on issue #27834: revert [SPARK-24640][SQL] Return `NULL` from `size(NULL)` by default URL: https://github.com/apache/spark/pull/27834#issuecomment-596161322 > As for the idea to have the size() behavior differ for aggregation vs. non-aggregation processing No I didn't mean it as an idea. I used it to describe the issue where NULL as result wouldn't have. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper
AmplabJenkins removed a comment on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper URL: https://github.com/apache/spark/pull/27593#issuecomment-596161275 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119523/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper
AmplabJenkins commented on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper URL: https://github.com/apache/spark/pull/27593#issuecomment-596161275 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119523/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper
AmplabJenkins removed a comment on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper URL: https://github.com/apache/spark/pull/27593#issuecomment-596161271 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper
AmplabJenkins commented on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper URL: https://github.com/apache/spark/pull/27593#issuecomment-596161271 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper
SparkQA commented on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper URL: https://github.com/apache/spark/pull/27593#issuecomment-596161215 **[Test build #119523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119523/testReport)** for PR 27593 at commit [`33580b0`](https://github.com/apache/spark/commit/33580b0a5fdb9ca0bd7ac2fe65ad0711f46a9249). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper
SparkQA removed a comment on issue #27593: [SPARK-30818][SPARKR][ML] Add SparkR LinearRegression wrapper URL: https://github.com/apache/spark/pull/27593#issuecomment-596157129 **[Test build #119523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119523/testReport)** for PR 27593 at commit [`33580b0`](https://github.com/apache/spark/commit/33580b0a5fdb9ca0bd7ac2fe65ad0711f46a9249). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26815: [SPARK-30189][SQL] Interval from year-month/date-time string should handle whitespaces
AmplabJenkins removed a comment on issue #26815: [SPARK-30189][SQL] Interval from year-month/date-time string should handle whitespaces URL: https://github.com/apache/spark/pull/26815#issuecomment-596160601 Build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26815: [SPARK-30189][SQL] Interval from year-month/date-time string should handle whitespaces
AmplabJenkins commented on issue #26815: [SPARK-30189][SQL] Interval from year-month/date-time string should handle whitespaces URL: https://github.com/apache/spark/pull/26815#issuecomment-596160601 Build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26815: [SPARK-30189][SQL] Interval from year-month/date-time string should handle whitespaces
AmplabJenkins commented on issue #26815: [SPARK-30189][SQL] Interval from year-month/date-time string should handle whitespaces URL: https://github.com/apache/spark/pull/26815#issuecomment-596160602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24258/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26815: [SPARK-30189][SQL] Interval from year-month/date-time string should handle whitespaces
AmplabJenkins removed a comment on issue #26815: [SPARK-30189][SQL] Interval from year-month/date-time string should handle whitespaces URL: https://github.com/apache/spark/pull/26815#issuecomment-596160602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24258/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
AmplabJenkins removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596160167 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24257/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
AmplabJenkins removed a comment on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596160164 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
AmplabJenkins commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596160164 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
AmplabJenkins commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596160167 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24257/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] WeichenXu123 commented on issue #27844: [WIP][SQL] Add back ImageSchema.readImages in Spark 3.0
WeichenXu123 commented on issue #27844: [WIP][SQL] Add back ImageSchema.readImages in Spark 3.0 URL: https://github.com/apache/spark/pull/27844#issuecomment-596160133 Discussed with @gatorsmile , agreed to remove this in 3.0. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] WeichenXu123 closed pull request #27844: [WIP][SQL] Add back ImageSchema.readImages in Spark 3.0
WeichenXu123 closed pull request #27844: [WIP][SQL] Add back ImageSchema.readImages in Spark 3.0 URL: https://github.com/apache/spark/pull/27844 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] WeichenXu123 edited a comment on issue #27844: [WIP][SQL] Add back ImageSchema.readImages in Spark 3.0
WeichenXu123 edited a comment on issue #27844: [WIP][SQL] Add back ImageSchema.readImages in Spark 3.0 URL: https://github.com/apache/spark/pull/27844#issuecomment-596160133 Discussed with @gatorsmile offline, agreed to remove this in 3.0. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
SparkQA commented on issue #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status URL: https://github.com/apache/spark/pull/27848#issuecomment-596160099 **[Test build #119527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119527/testReport)** for PR 27848 at commit [`4f8be84`](https://github.com/apache/spark/commit/4f8be8491a3cafc5e46e8db814297661629684ba). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #27848: [SPARK-30911][CORE][DOC] Add version information to the configuration of Status
beliefer commented on a change in pull request #27848: [SPARK-30911][CORE][DOC]
Add version information to the configuration of Status
URL: https://github.com/apache/spark/pull/27848#discussion_r389329517
##
File path: core/src/main/scala/org/apache/spark/internal/config/Status.scala
##
@@ -22,43 +22,52 @@ import java.util.concurrent.TimeUnit
private[spark] object Status {
val ASYNC_TRACKING_ENABLED =
ConfigBuilder("spark.appStateStore.asyncTracking.enable")
+.version("2.3.0")
.booleanConf
.createWithDefault(true)
val LIVE_ENTITY_UPDATE_PERIOD = ConfigBuilder("spark.ui.liveUpdate.period")
+.version("2.3.0")
.timeConf(TimeUnit.NANOSECONDS)
.createWithDefaultString("100ms")
val LIVE_ENTITY_UPDATE_MIN_FLUSH_PERIOD =
ConfigBuilder("spark.ui.liveUpdate.minFlushPeriod")
.doc("Minimum time elapsed before stale UI data is flushed. This avoids UI
staleness when " +
"incoming task events are not fired frequently.")
+.version("2.4.2")
.timeConf(TimeUnit.NANOSECONDS)
.createWithDefaultString("1s")
val MAX_RETAINED_JOBS = ConfigBuilder("spark.ui.retainedJobs")
+.version("1.2.0")
.intConf
.createWithDefault(1000)
val MAX_RETAINED_STAGES = ConfigBuilder("spark.ui.retainedStages")
+.version("0.9.0")
.intConf
.createWithDefault(1000)
val MAX_RETAINED_TASKS_PER_STAGE = ConfigBuilder("spark.ui.retainedTasks")
+.version("2.0.1")
.intConf
.createWithDefault(10)
val MAX_RETAINED_DEAD_EXECUTORS =
ConfigBuilder("spark.ui.retainedDeadExecutors")
+.version("2.0.0")
.intConf
.createWithDefault(100)
val MAX_RETAINED_ROOT_NODES =
ConfigBuilder("spark.ui.dagGraph.retainedRootRDDs")
+.version("2.1.0")
Review comment:
SPARK-17171, commit ID:
cc87280fcd065b01667ca7a59a1a32c7ab757355#diff-3f492c527ea26679d4307041b28455b8
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
