peter-toth opened a new pull request #34693:
URL: https://github.com/apache/spark/pull/34693
### What changes were proposed in this pull request?
CTE queries are not supported with MSSQL server via JDBC as MSSQL server
doesn't support statements with nested CTEs. When Spark builds the fi
SparkQA commented on pull request #34693:
URL: https://github.com/apache/spark/pull/34693#issuecomment-976453455
**[Test build #145548 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145548/testReport)**
for PR 34693 at commit
[`e2c9577`](https://github.com
SparkQA commented on pull request #34611:
URL: https://github.com/apache/spark/pull/34611#issuecomment-976460698
**[Test build #145540 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145540/testReport)**
for PR 34611 at commit
[`af97fb3`](https://github.co
SparkQA removed a comment on pull request #34611:
URL: https://github.com/apache/spark/pull/34611#issuecomment-976235724
**[Test build #145540 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145540/testReport)**
for PR 34611 at commit
[`af97fb3`](https://gi
SparkQA commented on pull request #34693:
URL: https://github.com/apache/spark/pull/34693#issuecomment-976489807
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50020/
--
This is an automated message from the Apache
AmplabJenkins commented on pull request #34611:
URL: https://github.com/apache/spark/pull/34611#issuecomment-976490264
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145540/
--
This
AmplabJenkins removed a comment on pull request #34611:
URL: https://github.com/apache/spark/pull/34611#issuecomment-976490264
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145540/
-
SparkQA commented on pull request #34677:
URL: https://github.com/apache/spark/pull/34677#issuecomment-976491507
**[Test build #145547 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145547/testReport)**
for PR 34677 at commit
[`0b67651`](https://github.co
SparkQA removed a comment on pull request #34677:
URL: https://github.com/apache/spark/pull/34677#issuecomment-976356000
**[Test build #145547 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145547/testReport)**
for PR 34677 at commit
[`0b67651`](https://gi
SparkQA commented on pull request #34668:
URL: https://github.com/apache/spark/pull/34668#issuecomment-976492795
**[Test build #145549 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145549/testReport)**
for PR 34668 at commit
[`24d3046`](https://github.com
AmplabJenkins commented on pull request #34677:
URL: https://github.com/apache/spark/pull/34677#issuecomment-976494224
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145547/
--
This
AmplabJenkins removed a comment on pull request #34677:
URL: https://github.com/apache/spark/pull/34677#issuecomment-976494224
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145547/
-
thejdeep commented on pull request #34607:
URL: https://github.com/apache/spark/pull/34607#issuecomment-976532846
> > I changed maybeUpdate to update when we encounter a speculative task.
The problem with the previous approach was that - if we have a speculative
task, then future task end
thejdeep commented on a change in pull request #34607:
URL: https://github.com/apache/spark/pull/34607#discussion_r755122578
##
File path: core/src/main/scala/org/apache/spark/status/storeTypes.scala
##
@@ -399,6 +399,20 @@ private[spark] class ExecutorStageSummaryWrapper(
}
SparkQA commented on pull request #34691:
URL: https://github.com/apache/spark/pull/34691#issuecomment-976549263
**[Test build #145544 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145544/testReport)**
for PR 34691 at commit
[`0c393a5`](https://github.co
SparkQA removed a comment on pull request #34691:
URL: https://github.com/apache/spark/pull/34691#issuecomment-976296694
**[Test build #145544 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145544/testReport)**
for PR 34691 at commit
[`0c393a5`](https://gi
SparkQA commented on pull request #34070:
URL: https://github.com/apache/spark/pull/34070#issuecomment-976551994
**[Test build #145545 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145545/testReport)**
for PR 34070 at commit
[`50484e2`](https://github.co
SparkQA removed a comment on pull request #34070:
URL: https://github.com/apache/spark/pull/34070#issuecomment-976297392
**[Test build #145545 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145545/testReport)**
for PR 34070 at commit
[`50484e2`](https://gi
SparkQA commented on pull request #34668:
URL: https://github.com/apache/spark/pull/34668#issuecomment-976553555
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50021/
--
This is an automated message from the Apache
cloud-fan commented on a change in pull request #34668:
URL: https://github.com/apache/spark/pull/34668#discussion_r755141437
##
File path:
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
##
@@ -73,6 +78,19 @@ grammar SqlBase;
return false;
cloud-fan commented on a change in pull request #34668:
URL: https://github.com/apache/spark/pull/34668#discussion_r755141872
##
File path:
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
##
@@ -73,6 +78,19 @@ grammar SqlBase;
return false;
cloud-fan commented on a change in pull request #34668:
URL: https://github.com/apache/spark/pull/34668#discussion_r755142758
##
File path:
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
##
@@ -73,6 +78,19 @@ grammar SqlBase;
return false;
SparkQA commented on pull request #34693:
URL: https://github.com/apache/spark/pull/34693#issuecomment-976568407
Kubernetes integration test status failure
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50020/
--
This is an automated message from the A
cloud-fan commented on a change in pull request #34668:
URL: https://github.com/apache/spark/pull/34668#discussion_r755144886
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParseDriver.scala
##
@@ -78,9 +79,30 @@ abstract class AbstractSqlParser
AmplabJenkins commented on pull request #34693:
URL: https://github.com/apache/spark/pull/34693#issuecomment-976570563
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50020/
--
T
AmplabJenkins commented on pull request #34691:
URL: https://github.com/apache/spark/pull/34691#issuecomment-976570562
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145544/
--
This
AmplabJenkins commented on pull request #34070:
URL: https://github.com/apache/spark/pull/34070#issuecomment-976570564
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145545/
--
This
AmplabJenkins removed a comment on pull request #34691:
URL: https://github.com/apache/spark/pull/34691#issuecomment-976570562
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145544/
-
AmplabJenkins removed a comment on pull request #34070:
URL: https://github.com/apache/spark/pull/34070#issuecomment-976570564
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145545/
-
AmplabJenkins removed a comment on pull request #34693:
URL: https://github.com/apache/spark/pull/34693#issuecomment-976570563
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50020/
cloud-fan commented on a change in pull request #34668:
URL: https://github.com/apache/spark/pull/34668#discussion_r755150850
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParseDriver.scala
##
@@ -78,9 +79,30 @@ abstract class AbstractSqlParser
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755161598
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
##
@@ -194,10 +195,22 @@ case class FileSourceScanExec(
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755163320
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
##
@@ -194,10 +195,22 @@ case class FileSourceScanExec(
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755163320
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
##
@@ -194,10 +195,22 @@ case class FileSourceScanExec(
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755163320
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
##
@@ -194,10 +195,22 @@ case class FileSourceScanExec(
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755164613
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
##
@@ -212,7 +225,16 @@ case class FileSourceScanExec(
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755165986
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
##
@@ -355,7 +377,14 @@ case class FileSourceScanExec(
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755167684
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormat.scala
##
@@ -171,6 +171,28 @@ trait FileFormat {
def sup
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755169306
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala
##
@@ -57,11 +66,15 @@ case class PartitionedFile(
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755169884
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala
##
@@ -57,11 +66,15 @@ case class PartitionedFile(
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755170678
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala
##
@@ -103,6 +116,135 @@ class FileScanRDD(
sarutak commented on pull request #34607:
URL: https://github.com/apache/spark/pull/34607#issuecomment-976610243
> since it would require changing a class val to a var.
What's the problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, pleas
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755175121
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala
##
@@ -103,6 +116,135 @@ class FileScanRDD(
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755176293
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala
##
@@ -103,6 +116,135 @@ class FileScanRDD(
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755178439
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala
##
@@ -103,6 +116,135 @@ class FileScanRDD(
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755180169
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala
##
@@ -103,6 +116,135 @@ class FileScanRDD(
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755181619
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala
##
@@ -103,6 +116,135 @@ class FileScanRDD(
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755183339
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala
##
@@ -103,6 +116,135 @@ class FileScanRDD(
SparkQA commented on pull request #34668:
URL: https://github.com/apache/spark/pull/34668#issuecomment-976633320
Kubernetes integration test status failure
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50021/
--
This is an automated message from the A
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755185991
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala
##
@@ -212,7 +212,9 @@ object FileSourceStrat
cloud-fan commented on a change in pull request #34575:
URL: https://github.com/apache/spark/pull/34575#discussion_r755188315
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileMetadataColumnsSuite.scala
##
@@ -0,0 +1,481 @@
+/*
+ * Licensed t
AmplabJenkins commented on pull request #34668:
URL: https://github.com/apache/spark/pull/34668#issuecomment-97983
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50021/
--
T
AmplabJenkins removed a comment on pull request #34668:
URL: https://github.com/apache/spark/pull/34668#issuecomment-97983
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50021/
cloud-fan commented on a change in pull request #34504:
URL: https://github.com/apache/spark/pull/34504#discussion_r755210674
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##
@@ -1548,6 +1548,31 @@ object PushPredicateThrough
cloud-fan commented on a change in pull request #34504:
URL: https://github.com/apache/spark/pull/34504#discussion_r755213627
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##
@@ -1548,6 +1548,31 @@ object PushPredicateThrough
cloud-fan commented on a change in pull request #34504:
URL: https://github.com/apache/spark/pull/34504#discussion_r755214307
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##
@@ -1548,6 +1548,31 @@ object PushPredicateThrough
cloud-fan commented on pull request #34454:
URL: https://github.com/apache/spark/pull/34454#issuecomment-976686019
Sorry I missed the ping. Merging to master, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use th
cloud-fan closed pull request #34454:
URL: https://github.com/apache/spark/pull/34454
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsu
cloud-fan commented on pull request #34642:
URL: https://github.com/apache/spark/pull/34642#issuecomment-976690695
I'm trying to understand the motivation. Is it because in-memory table can
output rows efficiently? Parquet scan can also output rows but we try our best
to output columnar ba
nchammas commented on pull request #34655:
URL: https://github.com/apache/spark/pull/34655#issuecomment-976699676
@HyukjinKwon - Is there anyone else you think should review this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitH
srowen commented on a change in pull request #34679:
URL: https://github.com/apache/spark/pull/34679#discussion_r755238638
##
File path: pom.xml
##
@@ -3353,11 +3353,6 @@
-
Review comment:
We could possibly leave the profile in and have it still do n
cloud-fan commented on a change in pull request #34634:
URL: https://github.com/apache/spark/pull/34634#discussion_r755239272
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/ShufflePartitionsUtil.scala
##
@@ -316,21 +316,25 @@ object ShufflePartit
srowen commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r755239983
##
File path: hadoop-cloud/pom.xml
##
@@ -201,7 +201,7 @@
enables store-specific committers.
-->
- hadoop-3.2
+ hadoop-3.3
Revi
srowen commented on pull request #34692:
URL: https://github.com/apache/spark/pull/34692#issuecomment-976711598
Because that JIRA is so old, we wouldn't really treat this as part of that
JIRA. I'll just make it a minor docs PR.
--
This is an automated message from the Apache Git Service.
cloud-fan commented on pull request #34684:
URL: https://github.com/apache/spark/pull/34684#issuecomment-976714972
Why is there no problem with AQE off?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
srowen commented on a change in pull request #34692:
URL: https://github.com/apache/spark/pull/34692#discussion_r755242633
##
File path: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala
##
@@ -33,10 +33,9 @@ import org.apache.spark.util.collection.OpenHashSet
/*
gengliangwang commented on a change in pull request #34596:
URL: https://github.com/apache/spark/pull/34596#discussion_r755249223
##
File path:
sql/core/src/test/resources/sql-tests/results/timestampNTZ/timestamp.sql.out
##
@@ -373,17 +374,19 @@ structhttps://user-images.githu
peter-toth commented on pull request #34693:
URL: https://github.com/apache/spark/pull/34693#issuecomment-97672
This change also seem to work with MSSQL's temp table syntax:
```
val withClause = "(SELECT * INTO #TempTable FROM (SELECT * FROM tbl) t)"
val query = "SELECT * FROM #
peter-toth edited a comment on pull request #34693:
URL: https://github.com/apache/spark/pull/34693#issuecomment-97672
This change also seem to work with MSSQL's temp table syntax:
```
val withClause = "(SELECT * INTO #TempTable FROM (SELECT * FROM tbl WHERE x
> 10) t)"
val que
cloud-fan commented on a change in pull request #34596:
URL: https://github.com/apache/spark/pull/34596#discussion_r755252311
##
File path:
sql/core/src/test/resources/sql-tests/results/timestampNTZ/timestamp.sql.out
##
@@ -373,17 +374,19 @@ struct
tgravescs commented on pull request #34622:
URL: https://github.com/apache/spark/pull/34622#issuecomment-976749417
yes, it would be nice to have the actual stagIds in the ui, I'll need to
look closer at the logic though, which likely won't be til next week.
--
This is an automated messag
SparkQA commented on pull request #34671:
URL: https://github.com/apache/spark/pull/34671#issuecomment-976763063
**[Test build #145550 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145550/testReport)**
for PR 34671 at commit
[`61fe9fb`](https://github.com
SparkQA commented on pull request #34671:
URL: https://github.com/apache/spark/pull/34671#issuecomment-976793578
**[Test build #145550 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145550/testReport)**
for PR 34671 at commit
[`61fe9fb`](https://github.co
SparkQA removed a comment on pull request #34671:
URL: https://github.com/apache/spark/pull/34671#issuecomment-976763063
**[Test build #145550 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145550/testReport)**
for PR 34671 at commit
[`61fe9fb`](https://gi
SparkQA commented on pull request #34671:
URL: https://github.com/apache/spark/pull/34671#issuecomment-976826549
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50022/
--
This is an automated message from the Apache
SparkQA commented on pull request #34693:
URL: https://github.com/apache/spark/pull/34693#issuecomment-976827223
**[Test build #145548 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145548/testReport)**
for PR 34693 at commit
[`e2c9577`](https://github.co
SparkQA removed a comment on pull request #34693:
URL: https://github.com/apache/spark/pull/34693#issuecomment-976453455
**[Test build #145548 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145548/testReport)**
for PR 34693 at commit
[`e2c9577`](https://gi
peter-toth commented on pull request #34693:
URL: https://github.com/apache/spark/pull/34693#issuecomment-976828400
Hmm, failures in `ExpressionsSchemaSuite` look unrelated...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub a
AmplabJenkins commented on pull request #34693:
URL: https://github.com/apache/spark/pull/34693#issuecomment-976848358
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145548/
--
This
AmplabJenkins commented on pull request #34671:
URL: https://github.com/apache/spark/pull/34671#issuecomment-976848362
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145550/
--
This
ChenMichael commented on pull request #34684:
URL: https://github.com/apache/spark/pull/34684#issuecomment-976849576
In order for this problem to manifest, we have to do join planning between
the time a InMemoryRelation is converted to an rdd and the time where the job
executing this rdd c
AmplabJenkins removed a comment on pull request #34671:
URL: https://github.com/apache/spark/pull/34671#issuecomment-976848362
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145550/
-
AmplabJenkins removed a comment on pull request #34693:
URL: https://github.com/apache/spark/pull/34693#issuecomment-976848358
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145548/
-
ChenMichael edited a comment on pull request #34684:
URL: https://github.com/apache/spark/pull/34684#issuecomment-976849576
In order for this problem to manifest, we have to do join planning between
the time a InMemoryRelation is converted to an rdd and the time where the job
executing thi
ChenMichael edited a comment on pull request #34684:
URL: https://github.com/apache/spark/pull/34684#issuecomment-976849576
In order for this problem to manifest, we have to do join planning in
between the time an InMemoryRelation is converted to a rdd and the time where
the job executing
tdg5 commented on a change in pull request #29024:
URL: https://github.com/apache/spark/pull/29024#discussion_r755337180
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/connection/ConnectionProvider.scala
##
@@ -18,60 +18,45 @@
package or
ChenMichael edited a comment on pull request #34684:
URL: https://github.com/apache/spark/pull/34684#issuecomment-976849576
In order for this problem to manifest, we have to do join planning in
between the time an InMemoryRelation is converted to a rdd and the time where
the job executing
ChenMichael edited a comment on pull request #34684:
URL: https://github.com/apache/spark/pull/34684#issuecomment-976849576
In order for this problem to manifest, we have to do join planning in
between the time an InMemoryRelation is converted to a RDD and the time where
the job executing
ChenMichael commented on pull request #34684:
URL: https://github.com/apache/spark/pull/34684#issuecomment-976858837
Oh I realized if a plan containing InMemoryRelation is converted to a RDD,
but the RDD is never submitted to DAG scheduler, then the stats would also be
inaccurate. I think
SparkQA commented on pull request #34671:
URL: https://github.com/apache/spark/pull/34671#issuecomment-976884853
Kubernetes integration test status failure
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50022/
--
This is an automated message from the A
SparkQA commented on pull request #34668:
URL: https://github.com/apache/spark/pull/34668#issuecomment-976912054
**[Test build #145549 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145549/testReport)**
for PR 34668 at commit
[`24d3046`](https://github.co
SparkQA removed a comment on pull request #34668:
URL: https://github.com/apache/spark/pull/34668#issuecomment-976492795
**[Test build #145549 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145549/testReport)**
for PR 34668 at commit
[`24d3046`](https://gi
AmplabJenkins commented on pull request #34668:
URL: https://github.com/apache/spark/pull/34668#issuecomment-976940853
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145549/
--
This
AmplabJenkins commented on pull request #34671:
URL: https://github.com/apache/spark/pull/34671#issuecomment-976940851
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50022/
--
T
AmplabJenkins removed a comment on pull request #34668:
URL: https://github.com/apache/spark/pull/34668#issuecomment-976940853
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145549/
-
AmplabJenkins removed a comment on pull request #34671:
URL: https://github.com/apache/spark/pull/34671#issuecomment-976940851
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50022/
sumeetgajjar commented on pull request #34693:
URL: https://github.com/apache/spark/pull/34693#issuecomment-976972982
> This change also seem to work with MSSQL's temp table syntax:
>
> ```
> val withClause = "(SELECT * INTO #TempTable FROM (SELECT * FROM tbl WHERE
x > 10) t)"
sumeetgajjar commented on a change in pull request #34693:
URL: https://github.com/apache/spark/pull/34693#discussion_r755398500
##
File path:
external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MsSqlServerIntegrationSuite.scala
##
@@ -356,4 +357,42 @@ c
viirya commented on pull request #34642:
URL: https://github.com/apache/spark/pull/34642#issuecomment-977016485
> I'm trying to understand the motivation. Is it because in-memory table can
output rows efficiently? Parquet scan can also output rows but we try our best
to output columnar bat
viirya edited a comment on pull request #34642:
URL: https://github.com/apache/spark/pull/34642#issuecomment-977016485
> I'm trying to understand the motivation. Is it because in-memory table can
output rows efficiently? Parquet scan can also output rows but we try our best
to output colum
101 - 200 of 390 matches
Mail list logo