Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22118
@speful looks mistakenly open. mind closing this please?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21320
@mallman, can you close this and put some efforts there in
https://github.com/apache/spark/pull/21889? I see no point of leaving this PR
open.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22117
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/22107#discussion_r210488890
--- Diff: R/pkg/R/DataFrame.R ---
@@ -2876,6 +2905,37 @@ setMethod("except",
dataFrame(excepted)
})
+#'
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/22107#discussion_r210488842
--- Diff: R/pkg/R/DataFrame.R ---
@@ -2848,6 +2848,35 @@ setMethod("intersect",
dataFrame(intersected)
})
+#'
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22117
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/20725#discussion_r210489614
--- Diff: python/pyspark/sql/tests.py ---
@@ -4331,13 +4354,22 @@ def test_createDataFrame_fallback_enabled(self):
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22119
[WIP][SPARK-25129][SQL] Revert mapping com.databricks.spark.avro to
org.apache.spark.sql.avro
## What changes were proposed in this pull request?
In
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22119#discussion_r210491110
--- Diff:
external/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala ---
@@ -503,7 +495,7 @@ class AvroSuite extends QueryTest with
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #4278 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4278/testReport)**
for PR 21889 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21320
**[Test build #4277 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4277/testReport)**
for PR 21320 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22119
For details, see the discussion in the JIRA
https://issues.apache.org/jira/browse/SPARK-24924
---
-
To unsubscribe, e-mail:
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/22110#discussion_r210493260
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/TypeUtils.scala
---
@@ -73,4 +73,14 @@ object TypeUtils {
}
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21537
For 2. and 3., it is harder to say my opinion in the comment. Let me say
short comments at first.
For 2., if I remember correctly, @viirya once wrote the API document in a
JIRA entry. it
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r210492513
--- Diff:
core/src/main/scala/org/apache/spark/internal/config/package.scala ---
@@ -69,6 +69,11 @@ package object config {
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22045
**[Test build #94843 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94843/testReport)**
for PR 22045 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22009
**[Test build #94837 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94837/testReport)**
for PR 22009 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22115
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/22112
> IMO we should traverse the dependency graph and rely on how ShuffledRDD
is configured
A trivial point here - Since `ShuffleDependency` is also a DeveloperAPI,
it's possible for users
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22117
**[Test build #94840 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94840/testReport)**
for PR 22117 at commit
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/22107#discussion_r210490074
--- Diff: R/pkg/R/DataFrame.R ---
@@ -2876,6 +2905,37 @@ setMethod("except",
dataFrame(excepted)
})
+#'
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21835#discussion_r210489980
--- Diff: R/pkg/R/functions.R ---
@@ -3320,7 +3321,7 @@ setMethod("explode",
#' @aliases sequence sequence,Column-method
#' @note sequence
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21537
I wouldn't revert this unless there are specific concerns about this. Do
you see any bug by a mixture
of representation `s""` and `code""`?
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22119
**[Test build #94841 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94841/testReport)**
for PR 22119 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22119
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94841/
Test FAILed.
---
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21868#discussion_r210494497
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
---
@@ -425,12 +426,44 @@ case class FileSourceScanExec(
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22117
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94840/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22117
**[Test build #94840 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94840/testReport)**
for PR 22117 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22031
**[Test build #94839 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94839/testReport)**
for PR 22031 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21889
**[Test build #4278 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4278/testReport)**
for PR 21889 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21320
**[Test build #4277 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4277/testReport)**
for PR 21320 at commit
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22119#discussion_r210495678
--- Diff:
external/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala ---
@@ -503,7 +495,7 @@ class AvroSuite extends QueryTest with
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22031
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22031
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22031
**[Test build #94839 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94839/testReport)**
for PR 22031 at commit
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/22095
@mccheah btw, please add a comment (say "merged to master") after you merge
a PR - just a convention in this project. FYI. thx.
---
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/22112
You are perfectly correct @jiangxb1987, that was a silly mistake on my part
- and not trivial at all !
It should be shuffle dependency we should rely on when traversing the
dependency tree, not
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22119#discussion_r210489530
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -637,6 +635,12 @@ object DataSource extends
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22119
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22119
**[Test build #94841 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94841/testReport)**
for PR 22119 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22119
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22119
@tgravescs @dongjoon-hyun @HyukjinKwon @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21537
Thank for involving me in an important thread. I was busy this morning in
Japan.
I think there are three topics in the thread.
1. Merge or revert this PR
2. Design document
3. IR
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22119
If we all agree this databricks mapping is not reasonable, I think it's ok
to have this inconsistency and remove the mapping for CSV in 3.0.
It's weird to make the same mistake just to
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21221
**[Test build #94842 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94842/testReport)**
for PR 21221 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22119
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22009
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94837/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22009
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22115
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/22112
I am not sure what the definition of `isIdempotent` here is.
For example, from MapPartitionsRDD :
```
override private[spark] def isIdempotent = {
if (inputOrderSensitive) {
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/22117
Jenkins, retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/22107#discussion_r210488641
--- Diff: R/pkg/R/DataFrame.R ---
@@ -2876,6 +2905,37 @@ setMethod("except",
dataFrame(excepted)
})
+#'
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/22107#discussion_r210488754
--- Diff: R/pkg/R/DataFrame.R ---
@@ -2848,6 +2848,35 @@ setMethod("intersect",
dataFrame(intersected)
})
+#'
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/22107#discussion_r210490145
--- Diff: R/pkg/R/DataFrame.R ---
@@ -2848,6 +2848,35 @@ setMethod("intersect",
dataFrame(intersected)
})
+#'
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22119
Sorry if I missed some comments somewhere but just for clarification,
should we do it for CSV in 3.0.0? Inconsistency should also be taken into
account. Actually configuration sounds making
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/22107#discussion_r210490166
--- Diff: R/pkg/R/DataFrame.R ---
@@ -2876,6 +2905,37 @@ setMethod("except",
dataFrame(excepted)
})
+#'
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22119#discussion_r210491239
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -637,6 +635,12 @@ object DataSource extends Logging
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22098
@shaneknapp, seems this was first introduced in
https://issues.apache.org/jira/browse/SPARK-3076 /
https://github.com/apache/spark/pull/1974 for a good reason fwiw.
One thing I am not
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22111
Let us hold these codegen PRs until we see the design doc for building IR
for the codegen?
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21537
If there's a bug, then let's fix in another JIRA. If that's impossible to
fix or sounds super risky and there's something I missed, let's revert.
---
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22119
CSV is loaded by default, while AVRO is not. So having a backward
compatibility mapping in CSV only still makes sense.
Let's remove the mapping for CSV in 3.0.
---
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r210492311
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -216,8 +217,7 @@ private[spark] class Executor(
def stop():
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21221
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22119
This particular inconsistency could confuse users because CSV's one has
existed for a long time. I think configuration makes this safer since both
sides make sense I believe.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22031
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22031
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94839/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22117
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21537
Sounds cool, let's move both to JIRAs or mailing lists.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22107
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94844/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20206
Build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22107
**[Test build #94844 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94844/testReport)**
for PR 22107 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20206
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22107
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22013#discussion_r210524076
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala
---
@@ -497,6 +497,59 @@ case class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22031
**[Test build #94846 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94846/testReport)**
for PR 22031 at commit
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22110#discussion_r210525593
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/TypeUtils.scala
---
@@ -73,4 +73,14 @@ object TypeUtils {
}
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22107
**[Test build #94844 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94844/testReport)**
for PR 22107 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22107
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22085
Thank you so much @mengxr and @jiangxb1987.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user vackosar commented on the issue:
https://github.com/apache/spark/pull/21919
@cloud-fan happy to merge?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/22031
Jenkins, retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22013
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/22031
@techaddict Could you fix the conflicts please? Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/22045
@codeatri Could you fix the conflicts please? Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22107
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21819#discussion_r210500530
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/JsonUtils.scala
---
@@ -95,4 +95,20 @@ private object JsonUtils {
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22031
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22031
**[Test build #94845 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94845/testReport)**
for PR 22031 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22031
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/22013
I'd merge this now.
@mn-mikke @mgaido91 If you have any other comments, let's have a follow-up
pr.
---
-
To unsubscribe,
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/22013
Thanks! merging to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22119
I am not sure how useful the configuration for AVRO is.
For the hive table example @dongjoon-hyun mentioned in
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/21537
Thanks everyone for this discussion. I think we all agree that what we need
as a first thing is a design doc and we can move the discussion there.
@kiszk thank you for your comments. I
Github user mgaido91 closed the pull request at:
https://github.com/apache/spark/pull/22111
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22111
@gatorsmile yes, sure, thanks. I created this before the related discussion
on the other PR. We can keep this as a suggestion while preparing the design
doc I think. I am closing this for the
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22031
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22031
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22013#discussion_r210527183
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala
---
@@ -497,6 +497,59 @@ case class
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/22031
LGTM pending Jenkins.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user dedunumax closed the pull request at:
https://github.com/apache/spark/pull/21120
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
1 - 100 of 440 matches
Mail list logo
