[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-20 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r158116796 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochCoordinator.scala --- @@ -0,0 +1,195 @@ +/* + *

[GitHub] spark issue #19940: [SPARK-22750][SQL] Reuse mutable states when possible

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19940 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85202/ Test FAILed. ---

[GitHub] spark issue #19940: [SPARK-22750][SQL] Reuse mutable states when possible

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19940 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19940: [SPARK-22750][SQL] Reuse mutable states when possible

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19940 **[Test build #85202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85202/testReport)** for PR 19940 at commit

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2017-12-20 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19993 thanks @hhbyyh, I updated the PR according to your suggestion and previous comments. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19156: [SPARK-19634][SQL][ML][FOLLOW-UP] Improve interfa...

2017-12-20 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/19156#discussion_r158138431 --- Diff: mllib/src/test/scala/org/apache/spark/ml/stat/SummarizerSuite.scala --- @@ -35,237 +34,252 @@ class SummarizerSuite extends SparkFunSuite with

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19993 **[Test build #85209 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85209/testReport)** for PR 19993 at commit

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2017-12-20 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/20023 In am generally in favor of following the SQL standard. How about we do this. Let's make the standard behavior the default, and add a flag to revert to the old behavior. This allows us to ease

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20032 > (And what happens if set to 0/-ve ?) We have a check preventing that in the option itself. The value should be strictly greater than 0 ms. ---

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20032 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85208/ Test PASSed. ---

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20032 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20032 Within the allocator's control loop, it's all asynchronous requests being made for executor pods from the k8s API, so, each loop doesn't take very long. If a user were to set a very low value for

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20032 **[Test build #85208 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85208/testReport)** for PR 20032 at commit

[GitHub] spark pull request #20018: SPARK-22833 [Improvement] in SparkHive Scala Exam...

2017-12-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20018#discussion_r158134032 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/hive/SparkHiveExample.scala --- @@ -102,8 +101,63 @@ object SparkHiveExample { //

[GitHub] spark pull request #20018: SPARK-22833 [Improvement] in SparkHive Scala Exam...

2017-12-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20018#discussion_r158133877 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/hive/SparkHiveExample.scala --- @@ -102,8 +101,63 @@ object SparkHiveExample { //

[GitHub] spark pull request #20018: SPARK-22833 [Improvement] in SparkHive Scala Exam...

2017-12-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20018#discussion_r158133606 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/hive/SparkHiveExample.scala --- @@ -102,8 +101,63 @@ object SparkHiveExample { //

[GitHub] spark pull request #20016: SPARK-22830 Scala Coding style has been improved ...

2017-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20016 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20016: SPARK-22830 Scala Coding style has been improved in Spar...

2017-12-20 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20016 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes.alloca...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20032 **[Test build #85208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85208/testReport)** for PR 20032 at commit

[GitHub] spark issue #20002: [SPARK-22465][Core][WIP] Add a safety-check to RDD defau...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20002 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85200/ Test FAILed. ---

[GitHub] spark issue #20002: [SPARK-22465][Core][WIP] Add a safety-check to RDD defau...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20002 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20002: [SPARK-22465][Core][WIP] Add a safety-check to RDD defau...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20002 **[Test build #85200 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85200/testReport)** for PR 20002 at commit

[GitHub] spark pull request #19992: [SPARK-22805][CORE] Use StorageLevel aliases in e...

2017-12-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19992#discussion_r158131498 --- Diff: core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala --- @@ -2022,12 +1947,7 @@ private[spark] object JsonProtocolSuite extends

[GitHub] spark pull request #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes...

2017-12-20 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/20032#discussion_r158131312 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -217,7

[GitHub] spark issue #20029: [SPARK-22793][SQL]Memory leak in Spark Thrift Server

2017-12-20 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20029 This indeed is the primary change as it's open vs master. https://github.com/apache/spark/pull/19989 had some concerns about whether this affects correctness though? ---

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19876 **[Test build #85207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85207/testReport)** for PR 19876 at commit

[GitHub] spark pull request #20032: [SPARK-22845] [Scheduler] Modify spark.kubernetes...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/20032#discussion_r158129065 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -217,7

[GitHub] spark issue #20039: [SPARK-22850][core] Ensure queued events are delivered t...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20039 **[Test build #85206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85206/testReport)** for PR 20039 at commit

[GitHub] spark pull request #20039: [SPARK-22850][core] Ensure queued events are deli...

2017-12-20 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/20039 [SPARK-22850][core] Ensure queued events are delivered to all event queues. The code in LiveListenerBus was queueing events before start in the queues themselves; so in situations like the

[GitHub] spark pull request #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML expo...

2017-12-20 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r158126871 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -126,15 +180,69 @@ abstract class MLWriter extends BaseReadWrite with

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r158124202 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockHandler.java --- @@ -203,22 +203,23 @@ private

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r158121392 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolver.java --- @@ -165,13 +165,23 @@ public

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r158120856 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockHandler.java --- @@ -203,22 +203,23 @@ private

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r158123441 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -812,10 +812,10 @@ private[spark] object MapOutputTracker extends Logging {

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r158124746 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ShuffleIndexInformation.java --- @@ -59,9 +59,9 @@ public int getSize() {

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r158124569 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolver.java --- @@ -165,13 +165,23 @@ public

[GitHub] spark issue #20037: [SPARK-22849] ivy.retrieve pattern should also consider ...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20037 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85197/ Test FAILed. ---

[GitHub] spark issue #20037: [SPARK-22849] ivy.retrieve pattern should also consider ...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20037 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20037: [SPARK-22849] ivy.retrieve pattern should also consider ...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20037 **[Test build #85197 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85197/testReport)** for PR 20037 at commit

[GitHub] spark pull request #20002: [SPARK-22465][Core][WIP] Add a safety-check to RD...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/20002#discussion_r158108582 --- Diff: core/src/main/scala/org/apache/spark/Partitioner.scala --- @@ -57,7 +60,8 @@ object Partitioner { def defaultPartitioner(rdd: RDD[_],

[GitHub] spark pull request #20002: [SPARK-22465][Core][WIP] Add a safety-check to RD...

2017-12-20 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/20002#discussion_r158119432 --- Diff: core/src/main/scala/org/apache/spark/Partitioner.scala --- @@ -67,6 +71,16 @@ object Partitioner { } } } + +

[GitHub] spark issue #20035: [SPARK-22848][SQL] Eliminate mutable state from Stack

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20035 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85198/ Test FAILed. ---

[GitHub] spark issue #20035: [SPARK-22848][SQL] Eliminate mutable state from Stack

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20035 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20035: [SPARK-22848][SQL] Eliminate mutable state from Stack

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20035 **[Test build #85198 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85198/testReport)** for PR 20035 at commit

[GitHub] spark issue #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19884 **[Test build #85205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85205/testReport)** for PR 19884 at commit

[GitHub] spark pull request #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML expo...

2017-12-20 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r158114844 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -126,15 +180,69 @@ abstract class MLWriter extends BaseReadWrite with

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19876 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19876 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85204/ Test FAILed. ---

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19876 **[Test build #85204 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85204/testReport)** for PR 19876 at commit

[GitHub] spark issue #19993: [SPARK-22799][ML] Bucketizer should throw exception if s...

2017-12-20 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/19993 To make it available for other classes, we need to support checking for both `fit` and `transform`, that means we also need a sample input Dataset, so we may have to add the explicit test in each of

[GitHub] spark issue #20016: SPARK-22830 Scala Coding style has been improved in Spar...

2017-12-20 Thread chetkhatri
Github user chetkhatri commented on the issue: https://github.com/apache/spark/pull/20016 @srowen I think, we can merge this now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19876 **[Test build #85204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85204/testReport)** for PR 19876 at commit

[GitHub] spark pull request #20018: SPARK-22833 [Improvement] in SparkHive Scala Exam...

2017-12-20 Thread chetkhatri
Github user chetkhatri commented on a diff in the pull request: https://github.com/apache/spark/pull/20018#discussion_r158113948 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/hive/SparkHiveExample.scala --- @@ -104,6 +103,60 @@ object SparkHiveExample {

[GitHub] spark issue #20037: [SPARK-22849] ivy.retrieve pattern should also consider ...

2017-12-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20037 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML expo...

2017-12-20 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r158113236 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala --- @@ -994,6 +998,38 @@ class LinearRegressionSuite

[GitHub] spark pull request #20037: [SPARK-22849] ivy.retrieve pattern should also co...

2017-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20037 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19884 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r158112470 --- Diff: python/pyspark/sql/tests.py --- @@ -3356,6 +3356,7 @@ def test_schema_conversion_roundtrip(self): self.assertEquals(self.schema,

[GitHub] spark issue #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19884 **[Test build #85203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85203/testReport)** for PR 19884 at commit

[GitHub] spark issue #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19884 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85203/ Test FAILed. ---

[GitHub] spark pull request #20037: [SPARK-22849] ivy.retrieve pattern should also co...

2017-12-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20037#discussion_r158112362 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -1271,7 +1271,7 @@ private[spark] object SparkSubmitUtils {

[GitHub] spark issue #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19884 **[Test build #85203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85203/testReport)** for PR 19884 at commit

[GitHub] spark issue #19940: [SPARK-22750][SQL] Reuse mutable states when possible

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19940 **[Test build #85202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85202/testReport)** for PR 19940 at commit

[GitHub] spark issue #20037: [SPARK-22849] ivy.retrieve pattern should also consider ...

2017-12-20 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20037 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19876: [WIP][ML][SPARK-11171][SPARK-11239] Add PMML export to S...

2017-12-20 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/19876 @sethah So I'm hesitant to push an API without an implementation to make sure its actually usable for our goal. But I'm fine splitting it out into a separate PR. ---

[GitHub] spark issue #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19884 Ok I did some local testing with these changes and pyarrow 0.8.0 with different combinations of Python and Pandas: **python 3.6.3, pandas 0.19.2** ERROR:

[GitHub] spark issue #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19884 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85195/ Test FAILed. ---

[GitHub] spark issue #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19884 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19884 **[Test build #85195 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85195/testReport)** for PR 19884 at commit

[GitHub] spark issue #20016: SPARK-22830 Scala Coding style has been improved in Spar...

2017-12-20 Thread chetkhatri
Github user chetkhatri commented on the issue: https://github.com/apache/spark/pull/20016 @srowen Thank you for re-run, now it passes all. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20033: [SPARK-22847] [CORE] Remove redundant code in AppStatusL...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20033 **[Test build #4017 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4017/testReport)** for PR 20033 at commit

[GitHub] spark issue #19035: [SPARK-21822][SQL]When insert Hive Table is finished, it...

2017-12-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19035 @figo77 Could you close it? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19035: [SPARK-21822][SQL]When insert Hive Table is finished, it...

2017-12-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19035 I think the problem is gone, right? Could you see the latest file version:

[GitHub] spark issue #20016: SPARK-22830 Scala Coding style has been improved in Spar...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20016 **[Test build #4019 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4019/testReport)** for PR 20016 at commit

[GitHub] spark issue #20036: [SPARK-18016][SQL][FOLLOW-UP] Code Generation: Constant ...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20036 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20036: [SPARK-18016][SQL][FOLLOW-UP] Code Generation: Constant ...

2017-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20036 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85196/ Test FAILed. ---

[GitHub] spark pull request #19725: [DO NOT REVIEW][SPARK-22042] [SQL] Insert shuffle...

2017-12-20 Thread tejasapatil
Github user tejasapatil closed the pull request at: https://github.com/apache/spark/pull/19725 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20036: [SPARK-18016][SQL][FOLLOW-UP] Code Generation: Constant ...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20036 **[Test build #85196 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85196/testReport)** for PR 20036 at commit

[GitHub] spark pull request #20017: [SPARK-22832][ML] BisectingKMeans unpersist unuse...

2017-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20017 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20017: [SPARK-22832][ML] BisectingKMeans unpersist unused datas...

2017-12-20 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20017 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19978: [SPARK-22784][CORE][WIP] Configure reading buffer size i...

2017-12-20 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19978 @MikhailErofeev was the conclusion here that you just want to set spark.taskMetrics.trackUpdatedBlockStatuses in your case? ---

[GitHub] spark pull request #20012: [SPARK-22824] Restore old offset for binary compa...

2017-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20012 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19813 I found a few problems that this PR can ideally solve. If this is not available soon, I will use workaround in upcoming PRs. ---

[GitHub] spark issue #20016: SPARK-22830 Scala Coding style has been improved in Spar...

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20016 **[Test build #4019 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4019/testReport)** for PR 20016 at commit

[GitHub] spark issue #19118: [SPARK-21882][CORE] OutputMetrics doesn't count written ...

2017-12-20 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/19118 The change looks good to me, pending test case; thanks for looking into this @awarrior --- - To unsubscribe, e-mail:

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19813 I agree that string replacement is too dangerous (e.g. `a + 1 = a + 10` with `a + 1`). How about a contract with adding assertions? ---

[GitHub] spark issue #20012: [SPARK-22824] Restore old offset for binary compatibilit...

2017-12-20 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20012 LGTM. Thanks! Merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20018: SPARK-22833 [Improvement] in SparkHive Scala Examples

2017-12-20 Thread chetkhatri
Github user chetkhatri commented on the issue: https://github.com/apache/spark/pull/20018 Adding other contributor of the same file for review. cc\ @cloud-fan @aokolnychyi @liancheng @HyukjinKwon ---

[GitHub] spark issue #20038: [SPARK-22836][ui] Show driver logs in UI when available.

2017-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20038 **[Test build #85201 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85201/testReport)** for PR 20038 at commit

[GitHub] spark pull request #20016: SPARK-22830 Scala Coding style has been improved ...

2017-12-20 Thread chetkhatri
Github user chetkhatri commented on a diff in the pull request: https://github.com/apache/spark/pull/20016#discussion_r157941778 --- Diff: examples/src/main/scala/org/apache/spark/examples/HdfsTest.scala --- @@ -39,7 +39,7 @@ object HdfsTest { val start =

[GitHub] spark pull request #20016: SPARK-22830 Scala Coding style has been improved ...

2017-12-20 Thread chetkhatri
Github user chetkhatri commented on a diff in the pull request: https://github.com/apache/spark/pull/20016#discussion_r157941813 --- Diff: examples/src/main/scala/org/apache/spark/examples/SparkALS.scala --- @@ -100,7 +100,7 @@ object SparkALS { ITERATIONS =

[GitHub] spark pull request #20016: SPARK-22830 Scala Coding style has been improved ...

2017-12-20 Thread chetkhatri
Github user chetkhatri commented on a diff in the pull request: https://github.com/apache/spark/pull/20016#discussion_r157816044 --- Diff: examples/src/main/scala/org/apache/spark/examples/LocalALS.scala --- @@ -95,7 +95,7 @@ object LocalALS { def showWarning() {

[GitHub] spark pull request #20038: [SPARK-22836][ui] Show driver logs in UI when ava...

2017-12-20 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/20038 [SPARK-22836][ui] Show driver logs in UI when available. Port code from the old executors listener to the new one, so that the driver logs present in the application start event are kept. You

[GitHub] spark pull request #20016: SPARK-22830 Scala Coding style has been improved ...

2017-12-20 Thread chetkhatri
Github user chetkhatri commented on a diff in the pull request: https://github.com/apache/spark/pull/20016#discussion_r157794109 --- Diff: examples/src/main/scala/org/apache/spark/examples/DFSReadWriteTest.scala --- @@ -127,11 +125,11 @@ object DFSReadWriteTest {

[GitHub] spark pull request #20016: SPARK-22830 Scala Coding style has been improved ...

2017-12-20 Thread chetkhatri
Github user chetkhatri commented on a diff in the pull request: https://github.com/apache/spark/pull/20016#discussion_r157941808 --- Diff: examples/src/main/scala/org/apache/spark/examples/SparkALS.scala --- @@ -80,7 +80,7 @@ object SparkALS { def showWarning() {

[GitHub] spark pull request #20016: SPARK-22830 Scala Coding style has been improved ...

2017-12-20 Thread chetkhatri
Github user chetkhatri commented on a diff in the pull request: https://github.com/apache/spark/pull/20016#discussion_r157816026 --- Diff: examples/src/main/scala/org/apache/spark/examples/DFSReadWriteTest.scala --- @@ -49,12 +49,10 @@ object DFSReadWriteTest { }

[GitHub] spark pull request #20016: SPARK-22830 Scala Coding style has been improved ...

2017-12-20 Thread chetkhatri
Github user chetkhatri commented on a diff in the pull request: https://github.com/apache/spark/pull/20016#discussion_r157794196 --- Diff: examples/src/main/scala/org/apache/spark/examples/LocalFileLR.scala --- @@ -58,10 +58,10 @@ object LocalFileLR { // Initialize

[GitHub] spark issue #20016: SPARK-22830 Scala Coding style has been improved in Spar...

2017-12-20 Thread chetkhatri
Github user chetkhatri commented on the issue: https://github.com/apache/spark/pull/20016 @srowen Thanks for response, correct - i went through error of jenkins and found error Online[https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4018/console] which fixed

[GitHub] spark pull request #20018: SPARK-22833 [Improvement] in SparkHive Scala Exam...

2017-12-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20018#discussion_r158100765 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/hive/SparkHiveExample.scala --- @@ -104,6 +103,60 @@ object SparkHiveExample { //

[GitHub] spark issue #20018: SPARK-22833 [Improvement] in SparkHive Scala Examples

2017-12-20 Thread chetkhatri
Github user chetkhatri commented on the issue: https://github.com/apache/spark/pull/20018 @srowen Can you please review and if everything seems correct then run test build --- - To unsubscribe, e-mail:

<    1   2   3   4   5   6   7   >