[GitHub] [spark] AmplabJenkins commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
AmplabJenkins commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-576130571 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
AmplabJenkins commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-576130576 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117048/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser
MaxGekk commented on a change in pull request #27287:
[SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments
accordingly in UnivocityParser
URL: https://github.com/apache/spark/pull/27287#discussion_r368389121
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/UnivocityParser.scala
##
@@ -203,7 +202,11 @@ class UnivocityParser(
}
}
- private val doParse = if (options.columnPruning && requiredSchema.isEmpty) {
+ /**
+ * Parses a single CSV string and turns it into either one resulting row or
no row (if the
+ * the record is malformed).
+ */
+ val parse: String => Option[InternalRow] = if (options.columnPruning &&
requiredSchema.isEmpty) {
Review comment:
I see, let's keep it as is. I would be nice to add a comment about the
reason.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #27263: [SPARK-30433][SQL][FOLLOW-UP] Optimize collect conflict plans
maropu commented on a change in pull request #27263: [SPARK-30433][SQL][FOLLOW-UP] Optimize collect conflict plans URL: https://github.com/apache/spark/pull/27263#discussion_r368389154 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1106,14 +1108,19 @@ class Analyzer( attr } } - (oldVersion, oldVersion.copy(output = newOutput)) + Seq((oldVersion, oldVersion.copy(output = newOutput))) case oldVersion @ Window(windowExpressions, _, _, child) if AttributeSet(windowExpressions.map(_.toAttribute)).intersect(conflictingAttributes) - .nonEmpty => - (oldVersion, oldVersion.copy(windowExpressions = newAliases(windowExpressions))) +.nonEmpty => + Seq((oldVersion, oldVersion.copy(windowExpressions = newAliases(windowExpressions + +case _ => plan.children.flatMap(collectConflictPlans) } + val conflictPlans = collectConflictPlans(right) Review comment: Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
SparkQA commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-576130101 **[Test build #117048 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117048/testReport)** for PR 27019 at commit [`c064d58`](https://github.com/apache/spark/commit/c064d58ee2bc1935b13f6bf06c81e050661e3731). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
SparkQA removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-576078259 **[Test build #117048 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117048/testReport)** for PR 27019 at commit [`c064d58`](https://github.com/apache/spark/commit/c064d58ee2bc1935b13f6bf06c81e050661e3731). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference
AmplabJenkins removed a comment on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27282#issuecomment-576129945 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117082/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference
SparkQA commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27282#issuecomment-576129846 **[Test build #117082 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117082/testReport)** for PR 27282 at commit [`0781899`](https://github.com/apache/spark/commit/0781899eb72b7d7c8d9267bd53aed769c64eaf3a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser
HyukjinKwon commented on a change in pull request #27287:
[SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments
accordingly in UnivocityParser
URL: https://github.com/apache/spark/pull/27287#discussion_r368388820
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/UnivocityParser.scala
##
@@ -203,7 +202,11 @@ class UnivocityParser(
}
}
- private val doParse = if (options.columnPruning && requiredSchema.isEmpty) {
+ /**
+ * Parses a single CSV string and turns it into either one resulting row or
no row (if the
+ * the record is malformed).
+ */
+ val parse: String => Option[InternalRow] = if (options.columnPruning &&
requiredSchema.isEmpty) {
Review comment:
In the current way, it saves one comparison for each record vs the
suggestion needs to compare for every record. It's maybe no big deal but let me
just don't change any behaviours for now in this PR.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference
AmplabJenkins removed a comment on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27282#issuecomment-576129936 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference
SparkQA removed a comment on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27282#issuecomment-576126965 **[Test build #117082 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117082/testReport)** for PR 27282 at commit [`0781899`](https://github.com/apache/spark/commit/0781899eb72b7d7c8d9267bd53aed769c64eaf3a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference
AmplabJenkins commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27282#issuecomment-576129936 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference
AmplabJenkins commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27282#issuecomment-576129945 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117082/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
AmplabJenkins removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576129632 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117074/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
SparkQA removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576120999 **[Test build #117074 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117074/testReport)** for PR 27259 at commit [`4aeb9ec`](https://github.com/apache/spark/commit/4aeb9ecf0388ebd94d64152a530af56c509fdd87). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
SparkQA commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576129514 **[Test build #117074 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117074/testReport)** for PR 27259 at commit [`4aeb9ec`](https://github.com/apache/spark/commit/4aeb9ecf0388ebd94d64152a530af56c509fdd87). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
cloud-fan commented on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework URL: https://github.com/apache/spark/pull/27243#issuecomment-576129640 LGTM except a few minor comments This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
AmplabJenkins removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576129626 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
AmplabJenkins commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576129626 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
AmplabJenkins commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576129632 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117074/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser
MaxGekk commented on a change in pull request #27287:
[SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments
accordingly in UnivocityParser
URL: https://github.com/apache/spark/pull/27287#discussion_r368388359
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/UnivocityParser.scala
##
@@ -203,7 +202,11 @@ class UnivocityParser(
}
}
- private val doParse = if (options.columnPruning && requiredSchema.isEmpty) {
+ /**
+ * Parses a single CSV string and turns it into either one resulting row or
no row (if the
+ * the record is malformed).
+ */
+ val parse: String => Option[InternalRow] = if (options.columnPruning &&
requiredSchema.isEmpty) {
Review comment:
It can be just normal method:
```scala
def parse(input: String): Option[InternalRow] = {
if (options.columnPruning && requiredSchema.isEmpty) {
// If `columnPruning` enabled and partition attributes scanned only,
// `schema` gets empty.
Some(InternalRow.empty)
} else {
// parse if the columnPruning is disabled or requiredSchema is nonEmpty
convert(tokenizer.parseLine(input))
}
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference
AmplabJenkins removed a comment on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27284#issuecomment-576129187 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21848/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference
AmplabJenkins commented on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27284#issuecomment-576129187 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21848/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference
AmplabJenkins commented on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27284#issuecomment-576129179 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27153: [SPARK-20384][SQL] Support value class in schema of Dataset (without breaking existing current projection)
HyukjinKwon commented on issue #27153: [SPARK-20384][SQL] Support value class in schema of Dataset (without breaking existing current projection) URL: https://github.com/apache/spark/pull/27153#issuecomment-576129092 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference
AmplabJenkins removed a comment on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27284#issuecomment-576129179 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
cloud-fan commented on a change in pull request #27243: [SPARK-30535][SQL]
Migrate ALTER TABLE commands to the new framework
URL: https://github.com/apache/spark/pull/27243#discussion_r368388156
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/v2ResolutionPlans.scala
##
@@ -63,14 +73,14 @@ case class ResolvedNamespace(catalog: SupportsNamespaces,
namespace: Seq[String]
*/
case class ResolvedTable(catalog: TableCatalog, identifier: Identifier, table:
Table)
extends LeafNode {
- override def output: Seq[Attribute] = Nil
+ override lazy val output: Seq[Attribute] = table.schema().toAttributes
Review comment:
do we still need this change after
https://github.com/apache/spark/pull/27243#discussion_r368376668 ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
cloud-fan commented on a change in pull request #27243: [SPARK-30535][SQL]
Migrate ALTER TABLE commands to the new framework
URL: https://github.com/apache/spark/pull/27243#discussion_r368387681
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
##
@@ -420,63 +425,71 @@ trait CheckAnalysis extends PredicateHelper {
}
case alter: AlterTable if alter.childrenResolved =>
Review comment:
BTW we can't hit v1 table here right? We should have converted alter table
command to v1 if the table is v1, before we reach here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
cloud-fan commented on a change in pull request #27243: [SPARK-30535][SQL]
Migrate ALTER TABLE commands to the new framework
URL: https://github.com/apache/spark/pull/27243#discussion_r368387490
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
##
@@ -420,63 +425,71 @@ trait CheckAnalysis extends PredicateHelper {
}
case alter: AlterTable if alter.childrenResolved =>
Review comment:
nit: `case alter: AlterTable if alter.childrenResolved &&
alter.table.isInstanceOf[ResolvedTable]`
Then we don't need to change the indentation below and save a lot of code
diff.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27165: [SPARK-28264][PYTHON][SQL] Support type hints in pandas UDF and rename/move inconsistent pandas UDF types
HyukjinKwon commented on issue #27165: [SPARK-28264][PYTHON][SQL] Support type hints in pandas UDF and rename/move inconsistent pandas UDF types URL: https://github.com/apache/spark/pull/27165#issuecomment-576128366 I will probably merge this one in some days given the positive feedback from some people used to Python in the community. But I would love to get some sign-off from committers too honestly :-). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
AmplabJenkins removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576127826 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117072/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on issue #27237: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
beliefer commented on issue #27237: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27237#issuecomment-576127985 cc @cloud-fan @maropu @dongjoon-hyun This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
cloud-fan commented on a change in pull request #27243: [SPARK-30535][SQL]
Migrate ALTER TABLE commands to the new framework
URL: https://github.com/apache/spark/pull/27243#discussion_r368386977
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
##
@@ -86,6 +87,20 @@ trait CheckAnalysis extends PredicateHelper {
}
def checkAnalysis(plan: LogicalPlan): Unit = {
+// Analysis that needs to be performed top down can be added here.
+plan.foreach {
+ case p if p.analyzed => // Skip already analyzed sub-plans
+
+ case alter: AlterTable =>
+alter.table match {
+ case u @ UnresolvedTableWithViewExists(view) if !view.isTempView =>
Review comment:
what's the error message for temp view?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
AmplabJenkins removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576127820 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference
AmplabJenkins removed a comment on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27282#issuecomment-576127394 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21847/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
AmplabJenkins commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576127826 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117072/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
AmplabJenkins commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576127820 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
SparkQA removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576119208 **[Test build #117072 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117072/testReport)** for PR 27259 at commit [`4aeb9ec`](https://github.com/apache/spark/commit/4aeb9ecf0388ebd94d64152a530af56c509fdd87). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
SparkQA commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576127782 **[Test build #117072 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117072/testReport)** for PR 27259 at commit [`4aeb9ec`](https://github.com/apache/spark/commit/4aeb9ecf0388ebd94d64152a530af56c509fdd87). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser
AmplabJenkins removed a comment on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser URL: https://github.com/apache/spark/pull/27287#issuecomment-576127334 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21846/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser
AmplabJenkins removed a comment on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser URL: https://github.com/apache/spark/pull/27287#issuecomment-576127329 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference
AmplabJenkins removed a comment on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27282#issuecomment-576127389 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference
AmplabJenkins commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27282#issuecomment-576127394 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21847/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference
AmplabJenkins commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27282#issuecomment-576127389 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser
AmplabJenkins commented on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser URL: https://github.com/apache/spark/pull/27287#issuecomment-576127334 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21846/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser
AmplabJenkins commented on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser URL: https://github.com/apache/spark/pull/27287#issuecomment-576127329 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
AmplabJenkins removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-576126766 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117044/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
AmplabJenkins removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576126597 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117071/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
SparkQA removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576117528 **[Test build #117071 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117071/testReport)** for PR 27259 at commit [`4aeb9ec`](https://github.com/apache/spark/commit/4aeb9ecf0388ebd94d64152a530af56c509fdd87). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
SparkQA removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-576076002 **[Test build #117044 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117044/testReport)** for PR 27278 at commit [`5d08dfd`](https://github.com/apache/spark/commit/5d08dfd7c614473a54e6fdd4e31fa7ecd94f2b3d). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27263: [SPARK-30433][SQL][FOLLOW-UP] Optimize collect conflict plans
SparkQA commented on issue #27263: [SPARK-30433][SQL][FOLLOW-UP] Optimize collect conflict plans URL: https://github.com/apache/spark/pull/27263#issuecomment-576126998 **[Test build #117083 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117083/testReport)** for PR 27263 at commit [`fbbf785`](https://github.com/apache/spark/commit/fbbf78542b2ceeb4c98e5416d60641fbcac6d0dd). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser
SparkQA commented on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser URL: https://github.com/apache/spark/pull/27287#issuecomment-576126990 **[Test build #117081 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117081/testReport)** for PR 27287 at commit [`e2cf12a`](https://github.com/apache/spark/commit/e2cf12aee9c37f284c4aa2d26bf8946d9bf91242). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
AmplabJenkins removed a comment on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-576126760 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference
SparkQA commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27282#issuecomment-576126965 **[Test build #117082 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117082/testReport)** for PR 27282 at commit [`0781899`](https://github.com/apache/spark/commit/0781899eb72b7d7c8d9267bd53aed769c64eaf3a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
AmplabJenkins removed a comment on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576126592 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
AmplabJenkins commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-576126760 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
AmplabJenkins commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-576126766 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117044/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
AmplabJenkins commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576126592 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
AmplabJenkins commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576126597 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117071/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.
SparkQA commented on issue #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions. URL: https://github.com/apache/spark/pull/27278#issuecomment-576126334 **[Test build #117044 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117044/testReport)** for PR 27278 at commit [`5d08dfd`](https://github.com/apache/spark/commit/5d08dfd7c614473a54e6fdd4e31fa7ecd94f2b3d). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure
SparkQA commented on issue #27259: [DO-NOT-MERGE][WIP] Investigating SPARK-30541 flaky test failure URL: https://github.com/apache/spark/pull/27259#issuecomment-576126471 **[Test build #117071 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117071/testReport)** for PR 27259 at commit [`4aeb9ec`](https://github.com/apache/spark/commit/4aeb9ecf0388ebd94d64152a530af56c509fdd87). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser
MaxGekk commented on a change in pull request #27287:
[SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments
accordingly in UnivocityParser
URL: https://github.com/apache/spark/pull/27287#discussion_r368385464
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/UnivocityParser.scala
##
@@ -232,18 +229,13 @@ class UnivocityParser(
new RuntimeException("Malformed CSV record"))
}
-var checkedTokens = tokens
var badRecordException: Option[Throwable] = None
Review comment:
@HyukjinKwon You address comments before I post them ;-)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser
HyukjinKwon commented on a change in pull request #27287:
[SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments
accordingly in UnivocityParser
URL: https://github.com/apache/spark/pull/27287#discussion_r368385353
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/UnivocityParser.scala
##
@@ -232,18 +229,13 @@ class UnivocityParser(
new RuntimeException("Malformed CSV record"))
}
-var checkedTokens = tokens
var badRecordException: Option[Throwable] = None
Review comment:
Yup, I did :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference
SparkQA removed a comment on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27284#issuecomment-576122961 **[Test build #117075 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117075/testReport)** for PR 27284 at commit [`c54dadd`](https://github.com/apache/spark/commit/c54dadda6e61671adf4ef0b531ef7a169da4c664). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference
AmplabJenkins removed a comment on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27284#issuecomment-576125916 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117075/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference
AmplabJenkins removed a comment on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27284#issuecomment-576125911 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser
MaxGekk commented on a change in pull request #27287:
[SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments
accordingly in UnivocityParser
URL: https://github.com/apache/spark/pull/27287#discussion_r368385157
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/UnivocityParser.scala
##
@@ -232,18 +229,13 @@ class UnivocityParser(
new RuntimeException("Malformed CSV record"))
}
-var checkedTokens = tokens
var badRecordException: Option[Throwable] = None
Review comment:
How about:
```scala
var badRecordException: Option[Throwable] = if (tokens.length !=
parsedSchema.length) {
// If the number of tokens doesn't match the schema, we should treat
it as a malformed record.
Some(new RuntimeException("Malformed CSV record"))
} else None
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference
SparkQA commented on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27284#issuecomment-576125723 **[Test build #117075 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117075/testReport)** for PR 27284 at commit [`c54dadd`](https://github.com/apache/spark/commit/c54dadda6e61671adf4ef0b531ef7a169da4c664). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27285: [SPARK-30578][SQL][TEST] Explicitly set conf to use DSv2 for orc in v2.3/OrcFilterSuite
dongjoon-hyun commented on a change in pull request #27285:
[SPARK-30578][SQL][TEST] Explicitly set conf to use DSv2 for orc in
v2.3/OrcFilterSuite
URL: https://github.com/apache/spark/pull/27285#discussion_r368381893
##
File path:
sql/core/v2.3/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcFilterSuite.scala
##
@@ -44,6 +46,11 @@ import org.apache.spark.sql.types._
*/
class OrcFilterSuite extends OrcTest with SharedSparkSession {
+ override protected def sparkConf: SparkConf =
Review comment:
Please update the PR title and description accordingly.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference
AmplabJenkins commented on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27284#issuecomment-576125911 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference
AmplabJenkins commented on issue #27284: [SPARK-30575][DOC] Document HAVING Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27284#issuecomment-576125916 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117075/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference
AmplabJenkins removed a comment on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27282#issuecomment-576125331 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
AmplabJenkins removed a comment on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework URL: https://github.com/apache/spark/pull/27243#issuecomment-576125390 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21845/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27263: [SPARK-30433][SQL][FOLLOW-UP] Optimize collect conflict plans
AmplabJenkins removed a comment on issue #27263: [SPARK-30433][SQL][FOLLOW-UP] Optimize collect conflict plans URL: https://github.com/apache/spark/pull/27263#issuecomment-576125352 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21844/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27280: [SPARK-30571][SQL] fix splitting shuffle fetch requests
AmplabJenkins removed a comment on issue #27280: [SPARK-30571][SQL] fix splitting shuffle fetch requests URL: https://github.com/apache/spark/pull/27280#issuecomment-576125332 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser
AmplabJenkins removed a comment on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser URL: https://github.com/apache/spark/pull/27287#issuecomment-576125282 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
AmplabJenkins removed a comment on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework URL: https://github.com/apache/spark/pull/27243#issuecomment-576125386 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser
AmplabJenkins removed a comment on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser URL: https://github.com/apache/spark/pull/27287#issuecomment-576125286 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21841/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27280: [SPARK-30571][SQL] fix splitting shuffle fetch requests
AmplabJenkins removed a comment on issue #27280: [SPARK-30571][SQL] fix splitting shuffle fetch requests URL: https://github.com/apache/spark/pull/27280#issuecomment-576125343 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21843/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference
AmplabJenkins removed a comment on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27282#issuecomment-576125339 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21842/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27263: [SPARK-30433][SQL][FOLLOW-UP] Optimize collect conflict plans
AmplabJenkins removed a comment on issue #27263: [SPARK-30433][SQL][FOLLOW-UP] Optimize collect conflict plans URL: https://github.com/apache/spark/pull/27263#issuecomment-576125340 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27263: [SPARK-30433][SQL][FOLLOW-UP] Optimize collect conflict plans
AmplabJenkins commented on issue #27263: [SPARK-30433][SQL][FOLLOW-UP] Optimize collect conflict plans URL: https://github.com/apache/spark/pull/27263#issuecomment-576125352 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21844/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27280: [SPARK-30571][SQL] fix splitting shuffle fetch requests
AmplabJenkins commented on issue #27280: [SPARK-30571][SQL] fix splitting shuffle fetch requests URL: https://github.com/apache/spark/pull/27280#issuecomment-576125343 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21843/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
AmplabJenkins commented on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework URL: https://github.com/apache/spark/pull/27243#issuecomment-576125390 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21845/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser
AmplabJenkins commented on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser URL: https://github.com/apache/spark/pull/27287#issuecomment-576125286 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21841/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser
AmplabJenkins commented on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser URL: https://github.com/apache/spark/pull/27287#issuecomment-576125282 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27263: [SPARK-30433][SQL][FOLLOW-UP] Optimize collect conflict plans
AmplabJenkins commented on issue #27263: [SPARK-30433][SQL][FOLLOW-UP] Optimize collect conflict plans URL: https://github.com/apache/spark/pull/27263#issuecomment-576125340 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference
AmplabJenkins commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27282#issuecomment-576125339 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21842/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
AmplabJenkins commented on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework URL: https://github.com/apache/spark/pull/27243#issuecomment-576125386 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference
AmplabJenkins commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27282#issuecomment-576125331 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27280: [SPARK-30571][SQL] fix splitting shuffle fetch requests
AmplabJenkins commented on issue #27280: [SPARK-30571][SQL] fix splitting shuffle fetch requests URL: https://github.com/apache/spark/pull/27280#issuecomment-576125332 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on a change in pull request #27263: [SPARK-30433][SQL][FOLLOW-UP] Optimize collect conflict plans
Ngone51 commented on a change in pull request #27263: [SPARK-30433][SQL][FOLLOW-UP] Optimize collect conflict plans URL: https://github.com/apache/spark/pull/27263#discussion_r368384580 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1106,14 +1108,19 @@ class Analyzer( attr } } - (oldVersion, oldVersion.copy(output = newOutput)) + Seq((oldVersion, oldVersion.copy(output = newOutput))) case oldVersion @ Window(windowExpressions, _, _, child) if AttributeSet(windowExpressions.map(_.toAttribute)).intersect(conflictingAttributes) - .nonEmpty => - (oldVersion, oldVersion.copy(windowExpressions = newAliases(windowExpressions))) +.nonEmpty => + Seq((oldVersion, oldVersion.copy(windowExpressions = newAliases(windowExpressions + +case _ => plan.children.flatMap(collectConflictPlans) } + val conflictPlans = collectConflictPlans(right) Review comment: Added(on the top of `collectConflictPlans`). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference
SparkQA commented on issue #27282: [SPARK-30573][DOC] Document WHERE Clause of SELECT statement in SQL Reference URL: https://github.com/apache/spark/pull/27282#issuecomment-576125013 **[Test build #117078 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117078/testReport)** for PR 27282 at commit [`f8b7580`](https://github.com/apache/spark/commit/f8b758017771f998fdb4f16d5cbd1de502b4243a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27239: [SPARK-30530][SQL] Fix filter pushdown for bad CSV records
HyukjinKwon commented on a change in pull request #27239: [SPARK-30530][SQL]
Fix filter pushdown for bad CSV records
URL: https://github.com/apache/spark/pull/27239#discussion_r368384640
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/UnivocityParser.scala
##
@@ -230,64 +230,55 @@ class UnivocityParser(
() => getCurrentInput,
() => None,
new RuntimeException("Malformed CSV record"))
-} else if (tokens.length != parsedSchema.length) {
+}
+
+var checkedTokens = tokens
+var badRecordException: Option[Throwable] = None
+
+if (tokens.length != parsedSchema.length) {
// If the number of tokens doesn't match the schema, we should treat it
as a malformed record.
// However, we still have chance to parse some of the tokens, by adding
extra null tokens in
// the tail if the number is smaller, or by dropping extra tokens if the
number is larger.
- val checkedTokens = if (parsedSchema.length > tokens.length) {
+ checkedTokens = if (parsedSchema.length > tokens.length) {
Review comment:
Oops, I already did at https://github.com/apache/spark/pull/27287. Let me
address this comment there.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
SparkQA commented on issue #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework URL: https://github.com/apache/spark/pull/27243#issuecomment-576125029 **[Test build #117080 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117080/testReport)** for PR 27243 at commit [`25ae584`](https://github.com/apache/spark/commit/25ae584ecac5f3a0033c263393f85e4941c4312f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27280: [SPARK-30571][SQL] fix splitting shuffle fetch requests
SparkQA commented on issue #27280: [SPARK-30571][SQL] fix splitting shuffle fetch requests URL: https://github.com/apache/spark/pull/27280#issuecomment-576125026 **[Test build #117079 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117079/testReport)** for PR 27280 at commit [`22c2bd3`](https://github.com/apache/spark/commit/22c2bd39a77c998ca500ff5f394878f108c6c8ba). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser
SparkQA commented on issue #27287: [SPARK-30530][SQL][FOLLOW-UP] Remove unnecessary codes and fix comments accordingly in UnivocityParser URL: https://github.com/apache/spark/pull/27287#issuecomment-576124960 **[Test build #117077 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117077/testReport)** for PR 27287 at commit [`bde0b64`](https://github.com/apache/spark/commit/bde0b64d8fcbe17b5a03aab01e58480b7939e58f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27272: [SPARK-30558][SQL] Avoid rebuilding `AvroOptions` per each partition
HyukjinKwon commented on issue #27272: [SPARK-30558][SQL] Avoid rebuilding `AvroOptions` per each partition URL: https://github.com/apache/spark/pull/27272#issuecomment-576124776 Merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #27272: [SPARK-30558][SQL] Avoid rebuilding `AvroOptions` per each partition
HyukjinKwon closed pull request #27272: [SPARK-30558][SQL] Avoid rebuilding `AvroOptions` per each partition URL: https://github.com/apache/spark/pull/27272 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27243: [SPARK-30535][SQL] Migrate ALTER TABLE commands to the new framework
imback82 commented on a change in pull request #27243: [SPARK-30535][SQL]
Migrate ALTER TABLE commands to the new framework
URL: https://github.com/apache/spark/pull/27243#discussion_r368369253
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##
@@ -514,6 +436,33 @@ class ResolveSessionCatalog(
parseSessionCatalogFunctionIdentifier("CREATE FUNCTION", catalog,
ident)
CreateFunctionCommand(database, function, className, resources, isTemp,
ignoreIfExists,
replace)
+
+case alter: AlterTable => checkAlterTable(alter)
Review comment:
I added this check using `foreach` inside `checkAnalysis`. Please let me
know if this approach is not desired.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
