[GitHub] spark pull request #10989: [SPARK-12798] [SQL] generated BroadcastHashJoin

2018-10-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10989#discussion_r222144135 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoin.scala --- @@ -117,6 +120,87 @@ case class BroadcastHashJoin(

[GitHub] spark issue #22047: [SPARK-19851] Add support for EVERY and ANY (SOME) aggre...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22047 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96872/ Test PASSed. ---

[GitHub] spark issue #22047: [SPARK-19851] Add support for EVERY and ANY (SOME) aggre...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22047 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22047: [SPARK-19851] Add support for EVERY and ANY (SOME) aggre...

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22047 **[Test build #96872 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96872/testReport)** for PR 22047 at commit

[GitHub] spark pull request #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilt...

2018-10-02 Thread kmanamcheri
Github user kmanamcheri commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r222140323 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -746,34 +746,20 @@ private[client] class Shim_v0_13 extends

[GitHub] spark issue #22347: [SPARK-25353][SQL] executeTake in SparkPlan is modified ...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22347 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96870/ Test PASSed. ---

[GitHub] spark issue #22347: [SPARK-25353][SQL] executeTake in SparkPlan is modified ...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22347 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22347: [SPARK-25353][SQL] executeTake in SparkPlan is modified ...

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22347 **[Test build #96870 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96870/testReport)** for PR 22347 at commit

[GitHub] spark issue #22491: [SPARK-25483][TEST] Refactor UnsafeArrayDataBenchmark to...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22491 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22491: [SPARK-25483][TEST] Refactor UnsafeArrayDataBenchmark to...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22491 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22500: [SPARK-25488][TEST] Refactor MiscBenchmark to use...

2018-10-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22500#discussion_r222138844 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/MiscBenchmark.scala --- @@ -21,247 +21,125 @@ import

[GitHub] spark issue #22527: [SPARK-17952][SQL] Nested Java beans support in createDa...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22527 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22527: [SPARK-17952][SQL] Nested Java beans support in createDa...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96869/ Test PASSed. ---

[GitHub] spark issue #22527: [SPARK-17952][SQL] Nested Java beans support in createDa...

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22527 **[Test build #96869 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96869/testReport)** for PR 22527 at commit

[GitHub] spark issue #22491: [SPARK-25483][TEST] Refactor UnsafeArrayDataBenchmark to...

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22491 **[Test build #96878 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96878/testReport)** for PR 22491 at commit

[GitHub] spark issue #22488: [SPARK-25479][TEST] Refactor DatasetBenchmark to use mai...

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22488 **[Test build #96879 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96879/testReport)** for PR 22488 at commit

[GitHub] spark issue #22488: [SPARK-25479][TEST] Refactor DatasetBenchmark to use mai...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22488 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22488: [SPARK-25479][TEST] Refactor DatasetBenchmark to use mai...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22488 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22500: [SPARK-25488][TEST] Refactor MiscBenchmark to use main m...

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22500 **[Test build #96877 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96877/testReport)** for PR 22500 at commit

[GitHub] spark issue #22500: [SPARK-25488][TEST] Refactor MiscBenchmark to use main m...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22500 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22500: [SPARK-25488][TEST] Refactor MiscBenchmark to use main m...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22500 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22605: [SPARK-25589][SQL][TEST] Add BloomFilterBenchmark

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22605 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22605: [SPARK-25589][SQL][TEST] Add BloomFilterBenchmark

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22605 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96866/ Test PASSed. ---

[GitHub] spark pull request #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilt...

2018-10-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r222135430 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -746,34 +746,20 @@ private[client] class Shim_v0_13 extends

[GitHub] spark issue #22605: [SPARK-25589][SQL][TEST] Add BloomFilterBenchmark

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22605 **[Test build #96866 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96866/testReport)** for PR 22605 at commit

[GitHub] spark pull request #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilt...

2018-10-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r222134301 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -746,34 +746,20 @@ private[client] class Shim_v0_13 extends

[GitHub] spark pull request #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilt...

2018-10-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r222134090 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -746,34 +746,20 @@ private[client] class Shim_v0_13 extends

[GitHub] spark issue #22616: [SPARK-25586][Core] Remove logdebug statement for outer ...

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22616 **[Test build #96876 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96876/testReport)** for PR 22616 at commit

[GitHub] spark issue #22503: [SPARK-25493][SQL] Use auto-detection for CRLF in CSV da...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22503 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96865/ Test PASSed. ---

[GitHub] spark issue #22503: [SPARK-25493][SQL] Use auto-detection for CRLF in CSV da...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22503 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22616: [SPARK-25586][Core] Remove logdebug statement for...

2018-10-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22616#discussion_r222126406 --- Diff: core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala --- @@ -285,8 +285,6 @@ private[spark] object ClosureCleaner extends Logging {

[GitHub] spark issue #22503: [SPARK-25493][SQL] Use auto-detection for CRLF in CSV da...

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22503 **[Test build #96865 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96865/testReport)** for PR 22503 at commit

[GitHub] spark pull request #22616: [SPARK-25586][Core] Remove logdebug statement for...

2018-10-02 Thread ankuriitg
Github user ankuriitg commented on a diff in the pull request: https://github.com/apache/spark/pull/22616#discussion_r222125433 --- Diff: core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala --- @@ -285,8 +285,6 @@ private[spark] object ClosureCleaner extends Logging {

[GitHub] spark issue #22492: [SPARK-25321][ML] Revert SPARK-14681 to avoid API breaki...

2018-10-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22492 Thank you! Although `master` becomes [3.0.0](https://github.com/apache/spark/pull/22606) already and seems to start to target `Mar 2019`, I agree with you on reverting for rechecking, @mengxr

[GitHub] spark pull request #22604: [SPARK-25586][MLlib][Core] Replace toString metho...

2018-10-02 Thread ankuriitg
Github user ankuriitg closed the pull request at: https://github.com/apache/spark/pull/22604 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilt...

2018-10-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r222123856 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -746,34 +746,20 @@ private[client] class Shim_v0_13 extends

[GitHub] spark pull request #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilt...

2018-10-02 Thread kmanamcheri
Github user kmanamcheri commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r222123426 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -746,34 +746,20 @@ private[client] class Shim_v0_13 extends

[GitHub] spark issue #22605: [SPARK-25589][SQL][TEST] Add BloomFilterBenchmark

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22605 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22605: [SPARK-25589][SQL][TEST] Add BloomFilterBenchmark

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22605 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96861/ Test PASSed. ---

[GitHub] spark issue #22605: [SPARK-25589][SQL][TEST] Add BloomFilterBenchmark

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22605 **[Test build #96861 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96861/testReport)** for PR 22605 at commit

[GitHub] spark pull request #22616: [SPARK-25586][Core] Remove logdebug statement for...

2018-10-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22616#discussion_r222122081 --- Diff: core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala --- @@ -318,19 +316,20 @@ private[spark] object ClosureCleaner extends Logging {

[GitHub] spark pull request #22616: [SPARK-25586][Core] Remove logdebug statement for...

2018-10-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22616#discussion_r222122237 --- Diff: core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala --- @@ -285,8 +285,6 @@ private[spark] object ClosureCleaner extends Logging {

[GitHub] spark pull request #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilt...

2018-10-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r222122774 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -746,34 +746,20 @@ private[client] class Shim_v0_13 extends

[GitHub] spark pull request #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilt...

2018-10-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22614#discussion_r222122400 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -746,34 +746,20 @@ private[client] class Shim_v0_13 extends

[GitHub] spark issue #22616: [SPARK-25586][Core] Remove logdebug statement for outer ...

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22616 **[Test build #96875 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96875/testReport)** for PR 22616 at commit

[GitHub] spark issue #22616: [SPARK-25586][Core] Remove logdebug statement for outer ...

2018-10-02 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22616 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilter shou...

2018-10-02 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22614 Looks ok to me based on discussion in the bug. Will leave here to see if others have any comments. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22492: [SPARK-25321][ML] Revert SPARK-14681 to avoid API breaki...

2018-10-02 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/22492 @cloud-fan said above the next version is very likely to be 2.5.0 instead of 3.0. Well the next version number is not fully discussed yet. For that reason, I think we should revert the changes in

[GitHub] spark issue #22492: [SPARK-25321][ML] Revert SPARK-14681 to avoid API breaki...

2018-10-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22492 May I ask a question, @mengxr ? The master becomes 3.0 today by @gatorsmile 's PR and I saw the following your comment here. > We can keep it in master if the next release is 3.0.

[GitHub] spark issue #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilter shou...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22614 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilter shou...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22614 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96868/ Test PASSed. ---

[GitHub] spark issue #22614: [SPARK-25561][SQL] HiveClient.getPartitionsByFilter shou...

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22614 **[Test build #96868 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96868/testReport)** for PR 22614 at commit

[GitHub] spark issue #22618: [SPARK-25321][ML] Revert SPARK-14681 to avoid API breaki...

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22618 **[Test build #96874 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96874/testReport)** for PR 22618 at commit

[GitHub] spark issue #22618: [SPARK-25321][ML] Revert SPARK-14681 to avoid API breaki...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22618 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22618: [SPARK-25321][ML] Revert SPARK-14681 to avoid API breaki...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22618 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22492: [SPARK-25321][ML] Revert SPARK-14681 to avoid API breaki...

2018-10-02 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/22492 @WeichenXu123 @cloud-fan I made https://github.com/apache/spark/pull/22618 to revert the change in master. --- - To unsubscribe,

[GitHub] spark pull request #22618: [SPARK-25321][ML] Revert SPARK-14681 to avoid API...

2018-10-02 Thread mengxr
GitHub user mengxr opened a pull request: https://github.com/apache/spark/pull/22618 [SPARK-25321][ML] Revert SPARK-14681 to avoid API breaking change ## What changes were proposed in this pull request? This is the same as #22492 but for master branch. Revert SPARK-14681 to

[GitHub] spark issue #22617: [SPARK-25484][TEST] Refactor ExternalAppendOnlyUnsafeRow...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22617 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22617: [SPARK-25484][TEST] Refactor ExternalAppendOnlyUnsafeRow...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22617 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22617: [SPARK-25484][TEST] Refactor ExternalAppendOnlyUnsafeRow...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22617 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22617: [SPARK-25484][TEST] Refactor ExternalAppendOnlyUn...

2018-10-02 Thread peter-toth
GitHub user peter-toth opened a pull request: https://github.com/apache/spark/pull/22617 [SPARK-25484][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark ## What changes were proposed in this pull request? 1. Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark 2.

[GitHub] spark issue #22557: [SPARK-25535][core] Work around bad error handling in co...

2018-10-02 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/22557 lgtm will leave for a day before mergning --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-10-02 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 @holdenk @jkbradley ping... could you please take a look at this PR? Thank you! --- - To unsubscribe, e-mail:

[GitHub] spark issue #22604: [SPARK-25586][MLlib][Core] Replace toString method with ...

2018-10-02 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22604 OK, this PR should be closed then. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #22615: [SPARK-25016][BUILD][CORE] Remove support for Had...

2018-10-02 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22615#discussion_r222097177 --- Diff: docs/building-spark.md --- @@ -49,25 +49,20 @@ To create a Spark distribution like those distributed by the to be runnable, use

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22237 **[Test build #96873 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96873/testReport)** for PR 22237 at commit

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-10-02 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/22237 @HyukjinKwon Please, take a look at the PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22609: [SPARK-25594] [Core] Avoid maintaining task information ...

2018-10-02 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/22609 I will close the PR and document in our tests to use spark.ui.retainedTasks = 1 (or some very low value) : which will, in effect, be equivalent to this PR - and loose information anyway; but will

[GitHub] spark pull request #22609: [SPARK-25594] [Core] Avoid maintaining task infor...

2018-10-02 Thread mridulm
Github user mridulm closed the pull request at: https://github.com/apache/spark/pull/22609 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22429: [SPARK-25440][SQL] Dumping query execution info to a fil...

2018-10-02 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/22429 Is there anything which blocks the PR for now? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22379: [SPARK-25393][SQL] Adding new function from_csv()

2018-10-02 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/22379 Could you take a look at the PR one more time, please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22609: [SPARK-25594] [Core] Avoid maintaining task information ...

2018-10-02 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/22609 > I tried that a lot while writing this code, but the disk store is just too slow That is unfortunate ... I was actually hoping that would be a good compromise if expanded api

[GitHub] spark issue #22347: [SPARK-25353][SQL] executeTake in SparkPlan is modified ...

2018-10-02 Thread Dooyoung-Hwang
Github user Dooyoung-Hwang commented on the issue: https://github.com/apache/spark/pull/22347 Thank you for review. Yes, ThriftServer will use intermediate "collection view" in this PR. And [Original PR of ThriftServer](https://github.com/apache/spark/pull/22219) will be

[GitHub] spark issue #22047: [SPARK-19851] Add support for EVERY and ANY (SOME) aggre...

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22047 **[Test build #96872 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96872/testReport)** for PR 22047 at commit

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22598 **[Test build #96871 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96871/testReport)** for PR 22598 at commit

[GitHub] spark issue #22047: [SPARK-19851] Add support for EVERY and ANY (SOME) aggre...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22047 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22047: [SPARK-19851] Add support for EVERY and ANY (SOME) aggre...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22047 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-10-02 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r222082996 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/KafkaDelegationTokenProvider.scala --- @@ -0,0 +1,66 @@ +/* + * Licensed to

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-10-02 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r222083143 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceProvider.scala --- @@ -560,25 +553,56 @@ private[kafka010]

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-10-02 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r222083063 --- Diff: core/src/test/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManagerSuite.scala --- @@ -46,6 +47,7 @@ class

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-10-02 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r222082853 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSecurityHelper.scala --- @@ -0,0 +1,86 @@ +/* + *

[GitHub] spark issue #22347: [SPARK-25353][SQL] executeTake in SparkPlan is modified ...

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22347 **[Test build #96870 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96870/testReport)** for PR 22347 at commit

[GitHub] spark issue #22527: [SPARK-17952][SQL] Nested Java beans support in createDa...

2018-10-02 Thread michalsenkyr
Github user michalsenkyr commented on the issue: https://github.com/apache/spark/pull/22527 I restructured the code in this commit to allow easier addition of array/list support in the future. --- - To unsubscribe,

[GitHub] spark issue #22609: [SPARK-25594] [Core] Avoid maintaining task information ...

2018-10-02 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22609 Task events still might need to be processed - even for live applications with the UI disabled. See the API that has been mentioned before (`SparkStatusTracker`) for why. Processing task events is

[GitHub] spark issue #22347: [SPARK-25353][SQL] executeTake in SparkPlan is modified ...

2018-10-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22347 Thank you for your first contribution, @Dooyoung-Hwang ! So, this is a spin-off PR from STS? --- - To unsubscribe,

[GitHub] spark pull request #22527: [SPARK-17952][SQL] Nested Java beans support in c...

2018-10-02 Thread michalsenkyr
Github user michalsenkyr commented on a diff in the pull request: https://github.com/apache/spark/pull/22527#discussion_r222079624 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -1100,13 +1101,23 @@ object SQLContext { attrs:

[GitHub] spark issue #22347: [SPARK-25353][SQL] executeTake in SparkPlan is modified ...

2018-10-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22347 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22527: [SPARK-17952][SQL] Nested Java beans support in createDa...

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22527 **[Test build #96869 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96869/testReport)** for PR 22527 at commit

[GitHub] spark issue #22609: [SPARK-25594] [Core] Avoid maintaining task information ...

2018-10-02 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/22609 @vanzin any reason why liveStore is hardcoded to be in-memory ? Any implications of making it disk backed ? That might be another option to unblock - requires a config change to existing

[GitHub] spark issue #22524: [SPARK-25497][SQL] Limit operation within whole stage co...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22524 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96860/ Test PASSed. ---

[GitHub] spark issue #22524: [SPARK-25497][SQL] Limit operation within whole stage co...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22524 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22609: [SPARK-25594] [Core] Avoid maintaining task information ...

2018-10-02 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/22609 @vanzin , @gengliangwang To clarify earlier behavior - if UI is disabled, any task related metrics were reported as zero: granularity of update was at the stage level. UI is typically disabled

[GitHub] spark issue #22524: [SPARK-25497][SQL] Limit operation within whole stage co...

2018-10-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22524 **[Test build #96860 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96860/testReport)** for PR 22524 at commit

[GitHub] spark pull request #22602: [SPARK-25538][SQL] Zero-out all bytes when writin...

2018-10-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22602#discussion_r222071791 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeWriterSuite.scala --- @@ -0,0 +1,48 @@ +/* + *

[GitHub] spark issue #22616: [SPARK-25586][Core] Remove logdebug statement for outer ...

2018-10-02 Thread ankuriitg
Github user ankuriitg commented on the issue: https://github.com/apache/spark/pull/22616 @srowen @yanboliang --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22616: [SPARK-25586][Core] Remove logdebug statement for outer ...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22616 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22616: [SPARK-25586][Core] Remove logdebug statement for outer ...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22616 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22616: [SPARK-25586][Core] Remove logdebug statement for outer ...

2018-10-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22616 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22604: [SPARK-25586][MLlib][Core] Replace toString method with ...

2018-10-02 Thread ankuriitg
Github user ankuriitg commented on the issue: https://github.com/apache/spark/pull/22604 I have opened another PR for this, with the recommended changes: https://github.com/apache/spark/pull/22616 I had to change a few other logDebug statements as well. Please let me know if

[GitHub] spark pull request #22616: [SPARK-25586][Core] Remove logdebug statement for...

2018-10-02 Thread ankuriitg
GitHub user ankuriitg opened a pull request: https://github.com/apache/spark/pull/22616 [SPARK-25586][Core] Remove logdebug statement for outer objects from ClosureCleaner ## What changes were proposed in this pull request? Cause: Recently test_glr_summary failed

<    1   2   3   4   5   >