maropu commented on pull request #29771:
URL: https://github.com/apache/spark/pull/29771#issuecomment-693998223
Looks ok otherwise. Nice fix.
This is an automated message from the Apache Git Service.
To respond to the message
maropu commented on a change in pull request #29771:
URL: https://github.com/apache/spark/pull/29771#discussion_r490011151
##
File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
##
@@ -2555,6 +2555,18 @@ class DataFrameSuite extends QueryTest
val d
cloud-fan closed pull request #29316:
URL: https://github.com/apache/spark/pull/29316
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t
AngersZh commented on a change in pull request #29739:
URL: https://github.com/apache/spark/pull/29739#discussion_r490009980
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala
##
@@ -693,4 +694,29 @@ case class HiveTableRelation
cloud-fan commented on pull request #29316:
URL: https://github.com/apache/spark/pull/29316#issuecomment-693988458
thanks, merging to master!
This is an automated message from the Apache Git Service.
To respond to the message
jzc928 commented on pull request #29761:
URL: https://github.com/apache/spark/pull/29761#issuecomment-693988152
@dongjoon-hyun comments fixed.
This is an automated message from the Apache Git Service.
To respond to the messag
AngersZh commented on a change in pull request #29739:
URL: https://github.com/apache/spark/pull/29739#discussion_r490008431
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala
##
@@ -693,4 +694,29 @@ case class HiveTableRelation
cloud-fan commented on a change in pull request #29739:
URL: https://github.com/apache/spark/pull/29739#discussion_r490007609
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala
##
@@ -693,4 +694,29 @@ case class HiveTableRelation(
cloud-fan commented on a change in pull request #29767:
URL: https://github.com/apache/spark/pull/29767#discussion_r490006671
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/streaming/test/DataStreamReaderWriterSuite.scala
##
@@ -815,3 +818,131 @@ class DataStreamRe
maropu commented on a change in pull request #29774:
URL: https://github.com/apache/spark/pull/29774#discussion_r490006677
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala
##
@@ -231,7 +232,9 @@ case class AdaptiveSparkP
HeartSaVioR commented on a change in pull request #29767:
URL: https://github.com/apache/spark/pull/29767#discussion_r490006162
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala
##
@@ -300,54 +301,54 @@ final class DataStreamWriter[T]
cloud-fan commented on a change in pull request #29767:
URL: https://github.com/apache/spark/pull/29767#discussion_r490005962
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/streaming/test/DataStreamReaderWriterSuite.scala
##
@@ -815,3 +818,131 @@ class DataStreamRe
cloud-fan commented on a change in pull request #29767:
URL: https://github.com/apache/spark/pull/29767#discussion_r490004721
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/streaming/test/DataStreamReaderWriterSuite.scala
##
@@ -815,3 +818,131 @@ class DataStreamRe
AmplabJenkins removed a comment on pull request #29767:
URL: https://github.com/apache/spark/pull/29767#issuecomment-693975766
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AngersZh commented on a change in pull request #29739:
URL: https://github.com/apache/spark/pull/29739#discussion_r490003795
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala
##
@@ -693,4 +694,29 @@ case class HiveTableRelation
AmplabJenkins removed a comment on pull request #29774:
URL: https://github.com/apache/spark/pull/29774#issuecomment-693975529
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins commented on pull request #29767:
URL: https://github.com/apache/spark/pull/29767#issuecomment-693975766
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHu
AmplabJenkins commented on pull request #29774:
URL: https://github.com/apache/spark/pull/29774#issuecomment-693975529
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHu
cloud-fan commented on a change in pull request #29767:
URL: https://github.com/apache/spark/pull/29767#discussion_r490003646
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala
##
@@ -300,54 +301,54 @@ final class DataStreamWriter[T] pr
maropu commented on a change in pull request #29774:
URL: https://github.com/apache/spark/pull/29774#discussion_r490003469
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala
##
@@ -231,7 +232,9 @@ case class AdaptiveSparkP
SparkQA commented on pull request #29767:
URL: https://github.com/apache/spark/pull/29767#issuecomment-693974559
**[Test build #128801 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128801/testReport)**
for PR 29767 at commit
[`6444a1e`](https://github.com
SparkQA commented on pull request #29774:
URL: https://github.com/apache/spark/pull/29774#issuecomment-693974560
**[Test build #128800 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128800/testReport)**
for PR 29774 at commit
[`e29626f`](https://github.com
cloud-fan commented on a change in pull request #29739:
URL: https://github.com/apache/spark/pull/29739#discussion_r490001806
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala
##
@@ -693,4 +694,29 @@ case class HiveTableRelation(
AmplabJenkins removed a comment on pull request #29775:
URL: https://github.com/apache/spark/pull/29775#issuecomment-693969011
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins commented on pull request #29775:
URL: https://github.com/apache/spark/pull/29775#issuecomment-693969011
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHu
SparkQA commented on pull request #29775:
URL: https://github.com/apache/spark/pull/29775#issuecomment-693967730
**[Test build #128799 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128799/testReport)**
for PR 29775 at commit
[`3a0f4b0`](https://github.com
cloud-fan commented on a change in pull request #29749:
URL: https://github.com/apache/spark/pull/29749#discussion_r48104
##
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala
##
@@ -47,7 +47,8 @@ private[hive] case class HiveSimpleUDF(
with Hive
sunchao commented on a change in pull request #29775:
URL: https://github.com/apache/spark/pull/29775#discussion_r489996551
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala
##
@@ -103,9 +103,9 @@ object Unwrap
AmplabJenkins removed a comment on pull request #29781:
URL: https://github.com/apache/spark/pull/29781#issuecomment-693952399
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins removed a comment on pull request #29703:
URL: https://github.com/apache/spark/pull/29703#issuecomment-693950638
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins commented on pull request #29781:
URL: https://github.com/apache/spark/pull/29781#issuecomment-693952399
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHu
SparkQA removed a comment on pull request #29781:
URL: https://github.com/apache/spark/pull/29781#issuecomment-693916737
**[Test build #128796 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128796/testReport)**
for PR 29781 at commit
[`b5feb02`](https://gi
SparkQA commented on pull request #29781:
URL: https://github.com/apache/spark/pull/29781#issuecomment-693951695
**[Test build #128796 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128796/testReport)**
for PR 29781 at commit
[`b5feb02`](https://github.co
AmplabJenkins commented on pull request #29703:
URL: https://github.com/apache/spark/pull/29703#issuecomment-693950638
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHu
SparkQA removed a comment on pull request #29703:
URL: https://github.com/apache/spark/pull/29703#issuecomment-693778396
**[Test build #128789 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128789/testReport)**
for PR 29703 at commit
[`033a33e`](https://gi
SparkQA commented on pull request #29703:
URL: https://github.com/apache/spark/pull/29703#issuecomment-693948094
**[Test build #128789 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128789/testReport)**
for PR 29703 at commit
[`033a33e`](https://github.co
cloud-fan commented on a change in pull request #29775:
URL: https://github.com/apache/spark/pull/29775#discussion_r489992483
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala
##
@@ -103,9 +103,9 @@ object Unwr
HeartSaVioR commented on a change in pull request #29767:
URL: https://github.com/apache/spark/pull/29767#discussion_r489991853
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala
##
@@ -300,97 +301,108 @@ final class DataStreamWriter[T]
AmplabJenkins removed a comment on pull request #29773:
URL: https://github.com/apache/spark/pull/29773#issuecomment-693938941
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins commented on pull request #29773:
URL: https://github.com/apache/spark/pull/29773#issuecomment-693938941
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHu
SparkQA commented on pull request #29773:
URL: https://github.com/apache/spark/pull/29773#issuecomment-693936825
**[Test build #128798 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128798/testReport)**
for PR 29773 at commit
[`a7861b9`](https://github.com
HyukjinKwon commented on a change in pull request #29781:
URL: https://github.com/apache/spark/pull/29781#discussion_r489989325
##
File path: python/docs/source/development/setting.rst
##
@@ -0,0 +1,58 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+or m
cloud-fan commented on a change in pull request #29771:
URL: https://github.com/apache/spark/pull/29771#discussion_r489988813
##
File path:
sql/catalyst/src/main/scala-2.12/org/apache/spark/sql/catalyst/expressions/AttributeMap.scala
##
@@ -26,6 +26,8 @@ object AttributeMap {
sunchao commented on a change in pull request #29775:
URL: https://github.com/apache/spark/pull/29775#discussion_r489988859
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala
##
@@ -103,9 +103,9 @@ object Unwrap
cloud-fan commented on a change in pull request #29767:
URL: https://github.com/apache/spark/pull/29767#discussion_r489988109
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala
##
@@ -300,97 +301,108 @@ final class DataStreamWriter[T] p
HyukjinKwon commented on a change in pull request #29781:
URL: https://github.com/apache/spark/pull/29781#discussion_r489987359
##
File path: python/docs/source/development/setting.rst
##
@@ -0,0 +1,58 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+or m
HyukjinKwon commented on pull request #29781:
URL: https://github.com/apache/spark/pull/29781#issuecomment-693923179
Nice, thanks @itholic. Can you also add a link to here at
https://github.com/apache/spark/blob/master/python/docs/source/development/debugging.rst#remote-debugging-pycharm-pr
HyukjinKwon commented on a change in pull request #29781:
URL: https://github.com/apache/spark/pull/29781#discussion_r489985646
##
File path: python/docs/source/development/setting.rst
##
@@ -0,0 +1,58 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+or m
HyukjinKwon commented on a change in pull request #29781:
URL: https://github.com/apache/spark/pull/29781#discussion_r489985515
##
File path: python/docs/source/development/setting.rst
##
@@ -0,0 +1,58 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+or m
cloud-fan commented on pull request #29775:
URL: https://github.com/apache/spark/pull/29775#issuecomment-693920672
> pass timezone info to the generated cast on the literal value
I'd also like to understand this more. `Cast.canonicalize` will drop the
timezone if it's not needed, so
HyukjinKwon commented on a change in pull request #29781:
URL: https://github.com/apache/spark/pull/29781#discussion_r489985388
##
File path: python/docs/source/development/setting.rst
##
@@ -0,0 +1,58 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+or m
dongjoon-hyun edited a comment on pull request #29761:
URL: https://github.com/apache/spark/pull/29761#issuecomment-693912369
@jzc928 . I left a few comments. Please update the PR accordingly. Although
this is different from Parquet, but this is the same with JSON data source. So,
I think
AmplabJenkins removed a comment on pull request #29761:
URL: https://github.com/apache/spark/pull/29761#issuecomment-693918836
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins commented on pull request #29781:
URL: https://github.com/apache/spark/pull/29781#issuecomment-693918935
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHu
AmplabJenkins removed a comment on pull request #29781:
URL: https://github.com/apache/spark/pull/29781#issuecomment-693918935
This is an automated message from the Apache Git Service.
To respond to the message, please log on
HyukjinKwon commented on a change in pull request #29781:
URL: https://github.com/apache/spark/pull/29781#discussion_r489985125
##
File path: python/docs/source/development/setting.rst
##
@@ -0,0 +1,58 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+or m
AmplabJenkins commented on pull request #29761:
URL: https://github.com/apache/spark/pull/29761#issuecomment-693918836
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHu
HyukjinKwon commented on a change in pull request #29781:
URL: https://github.com/apache/spark/pull/29781#discussion_r489984826
##
File path: python/docs/source/development/index.rst
##
@@ -25,3 +25,4 @@ Development
contributing
testing
debugging
+setting
Re
SparkQA commented on pull request #29761:
URL: https://github.com/apache/spark/pull/29761#issuecomment-693916933
**[Test build #128797 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128797/testReport)**
for PR 29761 at commit
[`c3c7f4c`](https://github.com
Ngone51 commented on a change in pull request #29773:
URL: https://github.com/apache/spark/pull/29773#discussion_r489984741
##
File path: core/src/main/scala/org/apache/spark/internal/config/Tests.scala
##
@@ -26,11 +26,11 @@ private[spark] object Tests {
.longConf
.c
SparkQA commented on pull request #29781:
URL: https://github.com/apache/spark/pull/29781#issuecomment-693916737
**[Test build #128796 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128796/testReport)**
for PR 29781 at commit
[`b5feb02`](https://github.com
cloud-fan commented on a change in pull request #29773:
URL: https://github.com/apache/spark/pull/29773#discussion_r489983915
##
File path: core/src/main/scala/org/apache/spark/internal/config/Tests.scala
##
@@ -26,11 +26,11 @@ private[spark] object Tests {
.longConf
dongjoon-hyun commented on pull request #29761:
URL: https://github.com/apache/spark/pull/29761#issuecomment-693912369
@jzc928 . I left a few comments. Please the PR accordingly. Although this is
different from Parquet, but this is the same with JSON data source. So, I think
we can accept
Ngone51 commented on a change in pull request #29773:
URL: https://github.com/apache/spark/pull/29773#discussion_r489983365
##
File path: core/src/main/scala/org/apache/spark/internal/config/Tests.scala
##
@@ -26,11 +26,11 @@ private[spark] object Tests {
.longConf
.c
itholic opened a new pull request #29781:
URL: https://github.com/apache/spark/pull/29781
### What changes were proposed in this pull request?
This PR proposes to document the way of setting PySpark with PyCharm.
**
for PR 29747 at commit
[`d4ce868`](https://gi
SparkQA commented on pull request #29747:
URL: https://github.com/apache/spark/pull/29747#issuecomment-693877716
**[Test build #128786 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128786/testReport)**
for PR 29747 at commit
[`d4ce868`](https://github.co
dongjoon-hyun commented on a change in pull request #29761:
URL: https://github.com/apache/spark/pull/29761#discussion_r489975451
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala
##
@@ -233,6 +233,19 @@ class FileBasedDataSourceSuite ex
viirya commented on pull request #29772:
URL: https://github.com/apache/spark/pull/29772#issuecomment-693863796
> Currently, Spark determines whether UnsafeExternalSorter.SpillableIterator
has spilled already by checking whether upstream is an instance of
UnsafeInMemorySorter.SortedIterato
AmplabJenkins removed a comment on pull request #29779:
URL: https://github.com/apache/spark/pull/29779#issuecomment-693855631
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins commented on pull request #29779:
URL: https://github.com/apache/spark/pull/29779#issuecomment-693855631
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHu
SparkQA removed a comment on pull request #29779:
URL: https://github.com/apache/spark/pull/29779#issuecomment-693803882
**[Test build #128794 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128794/testReport)**
for PR 29779 at commit
[`bd35323`](https://gi
SparkQA commented on pull request #29779:
URL: https://github.com/apache/spark/pull/29779#issuecomment-693854205
**[Test build #128794 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128794/testReport)**
for PR 29779 at commit
[`bd35323`](https://github.co
dongjoon-hyun commented on a change in pull request #29761:
URL: https://github.com/apache/spark/pull/29761#discussion_r489969657
##
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
##
@@ -2206,39 +2206,63 @@ abstract class SQLQuerySui
AmplabJenkins removed a comment on pull request #29780:
URL: https://github.com/apache/spark/pull/29780#issuecomment-693833247
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins commented on pull request #29780:
URL: https://github.com/apache/spark/pull/29780#issuecomment-693833247
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHu
SparkQA commented on pull request #29780:
URL: https://github.com/apache/spark/pull/29780#issuecomment-693832064
**[Test build #128795 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128795/testReport)**
for PR 29780 at commit
[`1bf4f32`](https://github.com
gwax commented on a change in pull request #29720:
URL: https://github.com/apache/spark/pull/29720#discussion_r489967059
##
File path: python/pyspark/sql/types.py
##
@@ -305,7 +305,7 @@ def jsonValue(self):
@classmethod
def fromJson(cls, json):
Review comment:
maropu opened a new pull request #29780:
URL: https://github.com/apache/spark/pull/29780
### What changes were proposed in this pull request?
This PR intends to fix a minor bug when normalizing floats for struct types;
```
scala> import org.apache.spark.sql.execution.agg
ulysses-you commented on pull request #29749:
URL: https://github.com/apache/spark/pull/29749#issuecomment-693825397
also cc @cloud-fan @dongjoon-hyun the similar issue with
[#13930](https://github.com/apache/spark/pull/13930)
--
cloud-fan commented on a change in pull request #29774:
URL: https://github.com/apache/spark/pull/29774#discussion_r489964455
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala
##
@@ -630,8 +639,18 @@ object AdaptiveSparkP
AmplabJenkins removed a comment on pull request #29779:
URL: https://github.com/apache/spark/pull/29779#issuecomment-693817842
This is an automated message from the Apache Git Service.
To respond to the message, please log on
SparkQA removed a comment on pull request #29779:
URL: https://github.com/apache/spark/pull/29779#issuecomment-693792818
**[Test build #128792 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128792/testReport)**
for PR 29779 at commit
[`8d6634b`](https://gi
AmplabJenkins commented on pull request #29779:
URL: https://github.com/apache/spark/pull/29779#issuecomment-693817842
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHu
SparkQA commented on pull request #29779:
URL: https://github.com/apache/spark/pull/29779#issuecomment-693816448
**[Test build #128792 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128792/testReport)**
for PR 29779 at commit
[`8d6634b`](https://github.co
cloud-fan commented on a change in pull request #29774:
URL: https://github.com/apache/spark/pull/29774#discussion_r489964148
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala
##
@@ -534,7 +542,8 @@ case class AdaptiveSpa
1 - 100 of 696 matches
Mail list logo
