[GitHub] spark pull request #21674: [SPARK-24696][SQL] ColumnPruning rule fails to re...

2018-06-29 Thread maryannxue
GitHub user maryannxue opened a pull request: https://github.com/apache/spark/pull/21674 [SPARK-24696][SQL] ColumnPruning rule fails to remove extra Project ## What changes were proposed in this pull request? The ColumnPruning rule tries adding an extra Project if an input

[GitHub] spark issue #21672: [SPARK-24694][K8S] Pass all app args to integration test...

2018-06-29 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21672 @ssuchter @liyinan926 psl review. This is trivial, unless I am missing something. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21662: [SPARK-24662][SQL][SS] Support limit in structure...

2018-06-29 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21662#discussion_r199284336 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingLimitExec.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to

[GitHub] spark pull request #21662: [SPARK-24662][SQL][SS] Support limit in structure...

2018-06-29 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21662#discussion_r199284559 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala --- @@ -805,6 +806,75 @@ class StreamSuite extends StreamTest {

[GitHub] spark pull request #21662: [SPARK-24662][SQL][SS] Support limit in structure...

2018-06-29 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21662#discussion_r199283820 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/MemorySinkSuite.scala --- @@ -70,35 +68,9 @@ class MemorySinkSuite extends

[GitHub] spark issue #21673: SPARK-24697: Fix the reported start offsets in streaming...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21673 **[Test build #92484 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92484/testReport)** for PR 21673 at commit

[GitHub] spark issue #21673: SPARK-24697: Fix the reported start offsets in streaming...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21673 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21673: SPARK-24697: Fix the reported start offsets in streaming...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21673 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21673: SPARK-24697: Fix the reported start offsets in streaming...

2018-06-29 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/spark/pull/21673 @tdas @jose-torres @HeartSaVioR --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21673: SPARK-24697: Fix the reported start offsets in st...

2018-06-29 Thread arunmahadevan
GitHub user arunmahadevan opened a pull request: https://github.com/apache/spark/pull/21673 SPARK-24697: Fix the reported start offsets in streaming query progress ## What changes were proposed in this pull request? Streaming query reports progress during each trigger (e.g.

[GitHub] spark pull request #21662: [SPARK-24662][SQL][SS] Support limit in structure...

2018-06-29 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/21662#discussion_r199279042 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingLimitExec.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to the

[GitHub] spark pull request #21662: [SPARK-24662][SQL][SS] Support limit in structure...

2018-06-29 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/21662#discussion_r199280992 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala --- @@ -615,7 +615,7 @@ class StreamSuite extends StreamTest {

[GitHub] spark pull request #21662: [SPARK-24662][SQL][SS] Support limit in structure...

2018-06-29 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/21662#discussion_r199278369 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -72,6 +72,8 @@ abstract class SparkStrategies extends

[GitHub] spark pull request #21662: [SPARK-24662][SQL][SS] Support limit in structure...

2018-06-29 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/21662#discussion_r199278862 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingLimitExec.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to the

[GitHub] spark pull request #21662: [SPARK-24662][SQL][SS] Support limit in structure...

2018-06-29 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/21662#discussion_r199279216 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingLimitExec.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to the

[GitHub] spark pull request #21662: [SPARK-24662][SQL][SS] Support limit in structure...

2018-06-29 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/21662#discussion_r199278041 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala --- @@ -315,8 +315,10 @@ object

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-06-29 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r199275753 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -3236,13 +3237,50 @@ class Dataset[T] private[sql]( } /**

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-06-29 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/21657 Actually the issue happened because I removed the mapping: https://github.com/apache/spark/commit/64fad0b519cf35b8c0a0dec18dd3df9488a5ed25#diff-d19881aceddcaa5c60620fdcda99b4c4L79 I would

[GitHub] spark issue #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPruning en...

2018-06-29 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/21631 > please take a following action. Will help and check if it's needed. I have opened the issue for uniVocity parser: https://github.com/uniVocity/univocity-parsers/issues/250 ---

[GitHub] spark issue #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Python UDF...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21650 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92482/ Test PASSed. ---

[GitHub] spark issue #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Python UDF...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21650 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Python UDF...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21650 **[Test build #92482 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92482/testReport)** for PR 21650 at commit

[GitHub] spark pull request #21671: [SPARK-24682] [SQL] from_json / to_json now handl...

2018-06-29 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21671#discussion_r199240323 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala --- @@ -317,16 +292,52 @@ class JacksonParser( row

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-06-29 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r199241520 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowConverters.scala --- @@ -183,34 +182,111 @@ private[sql] object

[GitHub] spark issue #20884: [SPARK-23773][SQL] JacksonGenerator does not include key...

2018-06-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20884 Thank you so much for sharing it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21575: [SPARK-24566][CORE] spark.storage.blockManagerSla...

2018-06-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21575 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21575: [SPARK-24566][CORE] spark.storage.blockManagerSlaveTimeo...

2018-06-29 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/21575 LGTM. Thanks! Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21462: [SPARK-24428][K8S] Fix unused code

2018-06-29 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21462#discussion_r199228479 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala --- @@ -46,8 +46,6 @@

[GitHub] spark pull request #21649: [SPARK-23648][R][SQL]Adds more types for hint in ...

2018-06-29 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/21649#discussion_r199225907 --- Diff: R/pkg/R/DataFrame.R --- @@ -3905,6 +3905,18 @@ setMethod("rollup", groupedData(sgd) }) +isTypeAllowed

[GitHub] spark pull request #21666: [SPARK-24535][SPARKR] fix tests on java check err...

2018-06-29 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/21666#discussion_r199224949 --- Diff: R/pkg/R/client.R --- @@ -61,6 +61,11 @@ generateSparkSubmitArgs <- function(args, sparkHome, jars, sparkSubmitOpts, pack }

[GitHub] spark pull request #20884: [SPARK-23773][SQL] JacksonGenerator does not incl...

2018-06-29 Thread makagonov
Github user makagonov closed the pull request at: https://github.com/apache/spark/pull/20884 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21672: [SPARK-24694][K8S] Pass all app args to integration test...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21672 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92483/ Test PASSed. ---

[GitHub] spark issue #21672: [SPARK-24694][K8S] Pass all app args to integration test...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21672 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21672: [SPARK-24694][K8S] Pass all app args to integration test...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21672 **[Test build #92483 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92483/testReport)** for PR 21672 at commit

[GitHub] spark issue #21672: [SPARK-24694][K8S] Pass all app args to integration test...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21672 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21672: [SPARK-24694][K8S] Pass all app args to integration test...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21672 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/590/

[GitHub] spark issue #21672: [SPARK-24694][K8S] Pass all app args to integration test...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21672 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/590/ ---

[GitHub] spark issue #21575: [SPARK-24566][CORE] spark.storage.blockManagerSlaveTimeo...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21575 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92479/ Test PASSed. ---

[GitHub] spark issue #21575: [SPARK-24566][CORE] spark.storage.blockManagerSlaveTimeo...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21575 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21575: [SPARK-24566][CORE] spark.storage.blockManagerSlaveTimeo...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21575 **[Test build #92479 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92479/testReport)** for PR 21575 at commit

[GitHub] spark issue #21672: [SPARK-24694][K8S] Pass all app args to integration test...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21672 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/590/ ---

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92478/ Test PASSed. ---

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #92478 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92478/testReport)** for PR 21596 at commit

[GitHub] spark issue #21672: [SPARK-24694][K8S] Pass all app args to integration test...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21672 **[Test build #92483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92483/testReport)** for PR 21672 at commit

[GitHub] spark pull request #21672: [SPARK-24694][K8S] Pass all app args to integrati...

2018-06-29 Thread skonto
GitHub user skonto opened a pull request: https://github.com/apache/spark/pull/21672 [SPARK-24694][K8S] Pass all app args to integration tests ## What changes were proposed in this pull request? - Allows to pass more than one app args to tests. ## How was this patch tested?

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21439 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92480/ Test PASSed. ---

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21439 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21439 **[Test build #92480 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92480/testReport)** for PR 21439 at commit

[GitHub] spark issue #20884: [SPARK-23773][SQL] JacksonGenerator does not include key...

2018-06-29 Thread sameeragarwal
Github user sameeragarwal commented on the issue: https://github.com/apache/spark/pull/20884 @HyukjinKwon I think it should be okay to close this at least for now. Just to add a little context behind this change, Facebook relies on the `toJSON` method for cross engine (hive/presto

[GitHub] spark pull request #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Pyt...

2018-06-29 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21650#discussion_r199204000 --- Diff: python/pyspark/sql/tests.py --- @@ -5060,6 +5049,138 @@ def test_type_annotation(self): df =

[GitHub] spark pull request #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Pyt...

2018-06-29 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21650#discussion_r199203674 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala --- @@ -94,36 +95,59 @@ object

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-06-29 Thread ifilonenko
Github user ifilonenko commented on the issue: https://github.com/apache/spark/pull/21583 The problem is due to the version of Docker @foxish Can we update the Docker version on the nodes? @shaneknapp --- - To

[GitHub] spark issue #20629: [SPARK-23451][ML] Deprecate KMeans.computeCost

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20629 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92477/ Test PASSed. ---

[GitHub] spark issue #20629: [SPARK-23451][ML] Deprecate KMeans.computeCost

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20629 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20629: [SPARK-23451][ML] Deprecate KMeans.computeCost

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20629 **[Test build #92477 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92477/testReport)** for PR 20629 at commit

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-06-29 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r199193918 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowConverters.scala --- @@ -183,34 +182,111 @@ private[sql] object

[GitHub] spark issue #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Python UDF...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21650 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/589/

[GitHub] spark issue #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Python UDF...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21650 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21671: [SPARK-24682] [SQL] from_json / to_json now handle java....

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21671 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21671: [SPARK-24682] [SQL] from_json / to_json now handle java....

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21671 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Python UDF...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21650 **[Test build #92482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92482/testReport)** for PR 21650 at commit

[GitHub] spark issue #21671: [SPARK-24682] [SQL] from_json / to_json now handle java....

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21671 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21671: [SPARK-24682] [SQL] from_json / to_json now handl...

2018-06-29 Thread patrickmcgloin
GitHub user patrickmcgloin opened a pull request: https://github.com/apache/spark/pull/21671 [SPARK-24682] [SQL] from_json / to_json now handle java.sql.Date/Timestamp as Map key ## What changes were proposed in this pull request? When generating JSON the key is not just

[GitHub] spark issue #21670: [SPARK-24682] [SQL] from_json / to_json now handle java....

2018-06-29 Thread patrickmcgloin
Github user patrickmcgloin commented on the issue: https://github.com/apache/spark/pull/21670 This is mistake. I have not put the correct code changes with the pull request. Will close. --- - To unsubscribe,

[GitHub] spark pull request #21670: [SPARK-24682] [SQL] from_json / to_json now handl...

2018-06-29 Thread patrickmcgloin
Github user patrickmcgloin closed the pull request at: https://github.com/apache/spark/pull/21670 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21670: [SPARK-24682] [SQL] from_json / to_json now handl...

2018-06-29 Thread patrickmcgloin
GitHub user patrickmcgloin opened a pull request: https://github.com/apache/spark/pull/21670 [SPARK-24682] [SQL] from_json / to_json now handle java.sql.Date/Timestamp as Map key ## What changes were proposed in this pull request? When generating JSON the key is not just

[GitHub] spark issue #20629: [SPARK-23451][ML] Deprecate KMeans.computeCost

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20629 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20629: [SPARK-23451][ML] Deprecate KMeans.computeCost

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20629 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92476/ Test FAILed. ---

[GitHub] spark issue #20629: [SPARK-23451][ML] Deprecate KMeans.computeCost

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20629 **[Test build #92476 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92476/testReport)** for PR 20629 at commit

[GitHub] spark issue #21623: [SPARK-24638][SQL] StringStartsWith support push down

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21623 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92474/ Test PASSed. ---

[GitHub] spark issue #21623: [SPARK-24638][SQL] StringStartsWith support push down

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21623 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Pyt...

2018-06-29 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21650#discussion_r199167791 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/python/BatchEvalPythonExecSuite.scala --- @@ -97,6 +103,64 @@ class

[GitHub] spark issue #21623: [SPARK-24638][SQL] StringStartsWith support push down

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21623 **[Test build #92474 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92474/testReport)** for PR 21623 at commit

[GitHub] spark pull request #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Pyt...

2018-06-29 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21650#discussion_r199167359 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala --- @@ -166,8 +190,9 @@ object ExtractPythonUDFs

[GitHub] spark pull request #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Pyt...

2018-06-29 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21650#discussion_r199167215 --- Diff: python/pyspark/sql/tests.py --- @@ -5060,6 +5049,138 @@ def test_type_annotation(self): df =

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21583 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/588/ ---

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21583 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/588/

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21583 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21583 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92473/ Test PASSed. ---

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21583 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21583 **[Test build #92473 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92473/testReport)** for PR 21583 at commit

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21583 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21583 **[Test build #92481 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92481/testReport)** for PR 21583 at commit

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21583 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92481/ Test PASSed. ---

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21583 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/588/ ---

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21583 **[Test build #92481 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92481/testReport)** for PR 21583 at commit

[GitHub] spark issue #21109: [SPARK-24020][SQL] Sort-merge join inner range optimizat...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21109 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21109: [SPARK-24020][SQL] Sort-merge join inner range optimizat...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21109 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92472/ Test PASSed. ---

[GitHub] spark issue #21109: [SPARK-24020][SQL] Sort-merge join inner range optimizat...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21109 **[Test build #92472 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92472/testReport)** for PR 21109 at commit

[GitHub] spark pull request #21624: [SPARK-24639][DOC] Add three config in the doc

2018-06-29 Thread xueyumusic
Github user xueyumusic commented on a diff in the pull request: https://github.com/apache/spark/pull/21624#discussion_r199145469 --- Diff: docs/configuration.md --- @@ -456,6 +456,13 @@ Apart from these, the following properties are also available, and may be useful from

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21439 **[Test build #92480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92480/testReport)** for PR 21439 at commit

[GitHub] spark pull request #21575: [SPARK-24566][CORE] spark.storage.blockManagerSla...

2018-06-29 Thread xueyumusic
Github user xueyumusic commented on a diff in the pull request: https://github.com/apache/spark/pull/21575#discussion_r199143954 --- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala --- @@ -74,17 +75,17 @@ private[spark] class HeartbeatReceiver(sc: SparkContext,

[GitHub] spark issue #21321: [SPARK-24268][SQL] Use datatype.simpleString in error me...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21321 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21321: [SPARK-24268][SQL] Use datatype.simpleString in error me...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21321 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92470/ Test PASSed. ---

[GitHub] spark issue #21321: [SPARK-24268][SQL] Use datatype.simpleString in error me...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21321 **[Test build #92470 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92470/testReport)** for PR 21321 at commit

[GitHub] spark issue #21575: [SPARK-24566][CORE] spark.storage.blockManagerSlaveTimeo...

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21575 **[Test build #92479 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92479/testReport)** for PR 21575 at commit

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #92478 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92478/testReport)** for PR 21596 at commit

[GitHub] spark pull request #21596: [SPARK-24601] Bump Jackson version

2018-06-29 Thread Fokko
Github user Fokko commented on a diff in the pull request: https://github.com/apache/spark/pull/21596#discussion_r199141020 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonBenchmarks.scala --- @@ -25,8 +25,14 @@ import

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21669 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

<    1   2   3   4   >