[GitHub] spark issue #16696: [SPARK-19350] [SQL] Cardinality estimation of Limit and ...

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16696 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16344: [SPARK-18929][ML] Add Tweedie distribution in GLM

2017-01-25 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/16344 This looks good to me. cc @jkbradley @srowen @sethah for another pass if you have time. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on Gi

[GitHub] spark issue #16703: [SPARK-12970][DOCS] Fix the example in SturctType APIs f...

2017-01-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16703 (I just added some links for `StructType` and `StructField` around this examples just for consistency and while I was here) --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #16700: [SPARK-19359][SQL]clear useless path after rename a part...

2017-01-25 Thread windpiger
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/16700 cc @gatorsmile @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] spark issue #16703: [SPARK-12970][DOCS] Fix the example in SturctType APIs f...

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16703 **[Test build #71991 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71991/testReport)** for PR 16703 at commit [`b9a841d`](https://github.com/apache/spark/commit/b9

[GitHub] spark pull request #16308: [SPARK-18936][SQL] Infrastructure for session loc...

2017-01-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16308#discussion_r97753392 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -490,7 +569,11 @@ abstract class UnixTim

[GitHub] spark pull request #16308: [SPARK-18936][SQL] Infrastructure for session loc...

2017-01-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16308#discussion_r97748329 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -490,7 +569,11 @@ abstract class UnixTim

[GitHub] spark pull request #16308: [SPARK-18936][SQL] Infrastructure for session loc...

2017-01-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16308#discussion_r97801907 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastSuite.scala --- @@ -107,108 +109,119 @@ class CastSuite extends Spark

[GitHub] spark pull request #16308: [SPARK-18936][SQL] Infrastructure for session loc...

2017-01-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16308#discussion_r97808039 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala --- @@ -177,180 +177,186 @@ class DateTimeUtilsSuite ex

[GitHub] spark issue #16703: [SPARK-12970][DOCS] Fix the example in SturctType APIs f...

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16703 **[Test build #71992 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71992/testReport)** for PR 16703 at commit [`f6a1059`](https://github.com/apache/spark/commit/f6

[GitHub] spark issue #16572: [SPARK-18863][SQL] Output non-aggregate expressions with...

2017-01-25 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16572 @nsyca it would be nice to make subquery expression part of tree traversal. It does seem risky to me, a lot functions using traversal maintain some state and do not account for the existence of s

[GitHub] spark issue #16697: [SPARK-19358][CORE] LiveListenerBus shall log the event ...

2017-01-25 Thread CodingCat
Github user CodingCat commented on the issue: https://github.com/apache/spark/pull/16697 jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark issue #16697: [SPARK-19358][CORE] LiveListenerBus shall log the event ...

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16697 **[Test build #71993 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71993/testReport)** for PR 16697 at commit [`b4bc5af`](https://github.com/apache/spark/commit/b4

[GitHub] spark pull request #16572: [SPARK-18863][SQL] Output non-aggregate expressio...

2017-01-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16572 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #16572: [SPARK-18863][SQL] Output non-aggregate expressions with...

2017-01-25 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16572 LGTM - merging to master/2.1/2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark pull request #16308: [SPARK-18936][SQL] Infrastructure for session loc...

2017-01-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16308#discussion_r97749254 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1044,7 +1044,8 @@ object DecimalAggregates extends

[GitHub] spark pull request #16308: [SPARK-18936][SQL] Infrastructure for session loc...

2017-01-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16308#discussion_r97749273 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1056,7 +1057,8 @@ object DecimalAggregates extends

[GitHub] spark pull request #16308: [SPARK-18936][SQL] Infrastructure for session loc...

2017-01-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16308#discussion_r97807866 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -250,6 +252,8 @@ class Dataset[T] private[sql]( val hasMoreData = take

[GitHub] spark issue #16572: [SPARK-18863][SQL] Output non-aggregate expressions with...

2017-01-25 Thread nsyca
Github user nsyca commented on the issue: https://github.com/apache/spark/pull/16572 @hvanhovell, I agree it does look risky with this approach. There are a lot of dependencies here. I am pitching in the idea to get your initial thought. Let me do some background and I will share once

[GitHub] spark issue #16660: [SPARK-19311][SQL] fix UDT hierarchy issue

2017-01-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16660 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[GitHub] spark issue #16701: [SPARK-18909][SQL] The error messages in `ExpressionEnco...

2017-01-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16701 Thank you for review, @srowen . Yes. These two are the scope of the issue description of SPARK-18909. @cloud-fan , did I understand correctly? --- If your project is set up for it

[GitHub] spark issue #15821: [SPARK-13534][WIP][PySpark] Using Apache Arrow to increa...

2017-01-25 Thread wesm
Github user wesm commented on the issue: https://github.com/apache/spark/pull/15821 Very nice to see the improved wall clock times. I have been busy engineering the pipeline between the byte stream from Spark and the resulting DataFrame -- the only major thing still left on the table

[GitHub] spark issue #16660: [SPARK-19311][SQL] fix UDT hierarchy issue

2017-01-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16660 Thanks! Merging to master/2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark pull request #16660: [SPARK-19311][SQL] fix UDT hierarchy issue

2017-01-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16660 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #12855: [SPARK-10216][SQL] Avoid creating empty files during ove...

2017-01-25 Thread milanvdmria
Github user milanvdmria commented on the issue: https://github.com/apache/spark/pull/12855 The issue that @jurriaan reported is still there in Spark 2.1.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark issue #16699: [SPARK-18710] Add offset in GLM

2017-01-25 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/16699 can you please: 1.) add [MLLIB] to the title 2.) fix the MiMa tests The MiMa tests error seems to be related to the interface change, probably you need to add an exclude to the file

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-01-25 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/16620 Fail to pass unit test. I will keep working on this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark issue #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16699 **[Test build #71994 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71994/testReport)** for PR 16699 at commit [`d2afcb0`](https://github.com/apache/spark/commit/d2

[GitHub] spark issue #16603: [SPARK-19244][Core] Sort MemoryConsumers according to th...

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16603 **[Test build #71989 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71989/testReport)** for PR 16603 at commit [`083cfb3`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16699 **[Test build #71994 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71994/testReport)** for PR 16699 at commit [`d2afcb0`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16699 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71994/ Test FAILed. ---

[GitHub] spark issue #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16699 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16603: [SPARK-19244][Core] Sort MemoryConsumers according to th...

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16603 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16603: [SPARK-19244][Core] Sort MemoryConsumers according to th...

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16603 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71989/ Test PASSed. ---

[GitHub] spark issue #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16699 **[Test build #71995 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71995/testReport)** for PR 16699 at commit [`9eca1a6`](https://github.com/apache/spark/commit/9e

[GitHub] spark issue #16213: [SPARK-18020][Streaming][Kinesis] Checkpoint SHARD_END t...

2017-01-25 Thread Gauravshah
Github user Gauravshah commented on the issue: https://github.com/apache/spark/pull/16213 facing same issue randomly on prod, can I help in some way to push it ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request #16688: [TESTS][SQL] Setup testdata at the beginning for ...

2017-01-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16688#discussion_r97835936 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala --- @@ -39,6 +39,8 @@ private case class BigData(s: String) class Cache

[GitHub] spark issue #16688: [TESTS][SQL] Setup testdata at the beginning for tests t...

2017-01-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16688 @dilipbiswal Since this PR is to fix the minor test issues, could you do me a favor to fix three comments? Replace `_global_temp` by `global_temp` - https://github.com/apache/sp

[GitHub] spark issue #16696: [SPARK-19350] [SQL] Cardinality estimation of Limit and ...

2017-01-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16696 Sure, will review it tonight. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark issue #16703: [SPARK-12970][DOCS] Fix the example in SturctType APIs f...

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16703 **[Test build #71990 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71990/testReport)** for PR 16703 at commit [`2e9bec8`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #16703: [SPARK-12970][DOCS] Fix the example in SturctType APIs f...

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16703 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71990/ Test PASSed. ---

[GitHub] spark issue #16703: [SPARK-12970][DOCS] Fix the example in SturctType APIs f...

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16703 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16476: [SPARK-19084][SQL] Implement expression field

2017-01-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16476 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #16213: [SPARK-18020][Streaming][Kinesis] Checkpoint SHAR...

2017-01-25 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/16213#discussion_r97841000 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisCheckpointer.scala --- @@ -64,7 +64,18 @@ private[kinesis] class Kine

[GitHub] spark issue #16476: [SPARK-19084][SQL] Implement expression field

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16476 **[Test build #71996 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71996/testReport)** for PR 16476 at commit [`465f7a6`](https://github.com/apache/spark/commit/46

[GitHub] spark issue #16476: [SPARK-19084][SQL] Implement expression field

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16476 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16476: [SPARK-19084][SQL] Implement expression field

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16476 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71996/ Test FAILed. ---

[GitHub] spark issue #16476: [SPARK-19084][SQL] Implement expression field

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16476 **[Test build #71996 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71996/testReport)** for PR 16476 at commit [`465f7a6`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #16213: [SPARK-18020][Streaming][Kinesis] Checkpoint SHARD_END t...

2017-01-25 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/16213 @maropu One last comment. Then I'll merge this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark issue #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16686 **[Test build #71997 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71997/testReport)** for PR 16686 at commit [`3b0d48b`](https://github.com/apache/spark/commit/3b

[GitHub] spark issue #16630: [SPARK-19270][ML] Add summary table to GLM summary

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16630 **[Test build #71998 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71998/testReport)** for PR 16630 at commit [`78bb77f`](https://github.com/apache/spark/commit/78

[GitHub] spark pull request #16582: [SPARK-19220][UI] Make redirection to HTTPS apply...

2017-01-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16582#discussion_r97846126 --- Diff: core/src/test/scala/org/apache/spark/ui/UISuite.scala --- @@ -227,8 +228,55 @@ class UISuite extends SparkFunSuite { assert(newHeader === nu

[GitHub] spark pull request #16603: [SPARK-19244][Core] Sort MemoryConsumers accordin...

2017-01-25 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16603#discussion_r97846569 --- Diff: core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java --- @@ -144,23 +152,49 @@ public long acquireExecutionMemory(long required, Memo

[GitHub] spark pull request #16603: [SPARK-19244][Core] Sort MemoryConsumers accordin...

2017-01-25 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16603#discussion_r97846848 --- Diff: core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java --- @@ -144,23 +152,49 @@ public long acquireExecutionMemory(long required, Memo

[GitHub] spark issue #16703: [SPARK-12970][DOCS] Fix the example in SturctType APIs f...

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16703 **[Test build #71991 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71991/testReport)** for PR 16703 at commit [`b9a841d`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #16703: [SPARK-12970][DOCS] Fix the example in SturctType APIs f...

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16703 **[Test build #71992 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71992/testReport)** for PR 16703 at commit [`f6a1059`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #16703: [SPARK-12970][DOCS] Fix the example in SturctType APIs f...

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16703 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71991/ Test PASSed. ---

[GitHub] spark issue #16703: [SPARK-12970][DOCS] Fix the example in SturctType APIs f...

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16703 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16703: [SPARK-12970][DOCS] Fix the example in SturctType APIs f...

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16703 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16703: [SPARK-12970][DOCS] Fix the example in SturctType APIs f...

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16703 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71992/ Test PASSed. ---

[GitHub] spark issue #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16699 **[Test build #71995 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71995/testReport)** for PR 16699 at commit [`9eca1a6`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16699 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16699 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71995/ Test PASSed. ---

[GitHub] spark issue #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16686 **[Test build #71997 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71997/testReport)** for PR 16686 at commit [`3b0d48b`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16686 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16686 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71997/ Test PASSed. ---

[GitHub] spark issue #16697: [SPARK-19358][CORE] LiveListenerBus shall log the event ...

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16697 **[Test build #71993 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71993/testReport)** for PR 16697 at commit [`b4bc5af`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #16697: [SPARK-19358][CORE] LiveListenerBus shall log the event ...

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16697 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16697: [SPARK-19358][CORE] LiveListenerBus shall log the event ...

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16697 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71993/ Test PASSed. ---

[GitHub] spark pull request #16704: [SPARK-18750][yarn] Follow up: move test to corre...

2017-01-25 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/16704 [SPARK-18750][yarn] Follow up: move test to correct directory in 2.1 branch. You can merge this pull request into a Git repository by running: $ git pull https://github.com/vanzin/spark SPARK-

[GitHub] spark issue #16704: [SPARK-18750][yarn] Follow up: move test to correct dire...

2017-01-25 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16704 fyi @tgravescs I plan to backport the original change and this one to 2.0 after this is pushed. (Merges fine btw.) --- If your project is set up for it, you can reply to this email and hav

[GitHub] spark issue #16704: [SPARK-18750][yarn] Follow up: move test to correct dire...

2017-01-25 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/16704 thanks for fixing, forgot we moved those. +1, go ahead and merge once jenkins passes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

[GitHub] spark issue #16704: [SPARK-18750][yarn] Follow up: move test to correct dire...

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16704 **[Test build #71999 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71999/testReport)** for PR 16704 at commit [`de87fef`](https://github.com/apache/spark/commit/de

[GitHub] spark issue #15821: [SPARK-13534][WIP][PySpark] Using Apache Arrow to increa...

2017-01-25 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/15821 > Parallelizing the record batch conversion and streaming it to Python would be another significant perf win. Right, I should have also mentioned that this PR takes a simplistic approa

[GitHub] spark issue #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-25 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/16699 @zhengruifeng @imatiach-msft Thanks much for pointing out the issue due to the hasOffset trait. This is what caused the test to fail. I have moved it to the GLRBase class. Things are worki

[GitHub] spark issue #16630: [SPARK-19270][ML] Add summary table to GLM summary

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16630 **[Test build #71998 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71998/testReport)** for PR 16630 at commit [`78bb77f`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #16630: [SPARK-19270][ML] Add summary table to GLM summary

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16630 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16630: [SPARK-19270][ML] Add summary table to GLM summary

2017-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16630 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71998/ Test PASSed. ---

[GitHub] spark pull request #16688: [TESTS][SQL] Setup testdata at the beginning for ...

2017-01-25 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/16688#discussion_r97864526 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala --- @@ -39,6 +39,8 @@ private case class BigData(s: String) class Cach

[GitHub] spark issue #16688: [TESTS][SQL] Setup testdata at the beginning for tests t...

2017-01-25 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/16688 @gatorsmile Sure Sean. Will do --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16682: [SPARK-19307][pyspark] Make sure user conf is propagated...

2017-01-25 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16682 Merging to master / 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16682: [SPARK-19307][pyspark] Make sure user conf is pro...

2017-01-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16682 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #16695: [SPARK-19277][yarn] Localize topology scripts ins...

2017-01-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16695#discussion_r97867966 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config.scala --- @@ -349,4 +351,47 @@ package object config { CACHED_FI

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-25 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r97870619 --- Diff: mllib/src/main/scala/org/apache/spark/ml/optim/IterativelyReweightedLeastSquares.scala --- @@ -20,6 +20,7 @@ package org.apache.spark.ml.opt

[GitHub] spark issue #16582: [SPARK-19220][UI] Make redirection to HTTPS apply to all...

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16582 **[Test build #72000 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72000/testReport)** for PR 16582 at commit [`5b65c69`](https://github.com/apache/spark/commit/5b

[GitHub] spark issue #16688: [TESTS][SQL] Setup testdata at the beginning for tests t...

2017-01-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16688 **[Test build #72001 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72001/testReport)** for PR 16688 at commit [`8f9914d`](https://github.com/apache/spark/commit/8f

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-25 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r97872960 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -358,13 +384,13 @@ object GeneralizedLinearRegr

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-25 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r97876098 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -736,15 +762,27 @@ class GeneralizedLinearRegre

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-25 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r97877367 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -94,6 +94,17 @@ private[regression] trait Gene

[GitHub] spark issue #16699: [SPARK-18710][ML] Add offset in GLM

2017-01-25 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/16699 @actuaryzhang added a couple comments, please take a look, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark issue #16557: [SPARK-18693][ML][MLLIB] ML Evaluators should use weight...

2017-01-25 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/16557 ping @sethah @Lewuathe @thunterdb @WeichenXu123 @jkbradley would you be able to take a look at the changes to add a weight column to binary/multiclass/regression evaluators/metrics classes? It

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-25 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/16650#discussion_r97881374 --- Diff: core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala --- @@ -173,6 +174,24 @@ private[scheduler] class BlacklistTracker (

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-25 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/16650#discussion_r97881356 --- Diff: core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala --- @@ -173,6 +174,24 @@ private[scheduler] class BlacklistTracker (

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-25 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/16650#discussion_r97877539 --- Diff: core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala --- @@ -173,6 +174,24 @@ private[scheduler] class BlacklistTracker (

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-25 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/16650#discussion_r97878659 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -528,6 +548,16 @@ class StandaloneDynamicAllocationSuite

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-25 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/16650#discussion_r97877918 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -148,6 +153,12 @@ class CoarseGrainedSchedulerBac

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-25 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/16650#discussion_r97874677 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationClient.scala --- @@ -59,6 +59,19 @@ private[spark] trait ExecutorAllocationClient { de

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-25 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/16650#discussion_r97881401 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -148,6 +153,12 @@ class CoarseGrainedSchedulerBac

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-25 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/16650#discussion_r97875716 --- Diff: core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala --- @@ -187,6 +206,22 @@ private[scheduler] class BlacklistTracker (

[GitHub] spark pull request #16650: [SPARK-16554][CORE] Automatically Kill Executors ...

2017-01-25 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/16650#discussion_r97875223 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -139,6 +139,11 @@ package object config { .timeConf(TimeUnit.MI

<    1   2   3   4   >