[GitHub] spark issue #15264: [SPARK-17477][SQL] SparkSQL cannot handle schema evoluti...

2017-02-02 Thread sameeragarwal
Github user sameeragarwal commented on the issue: https://github.com/apache/spark/pull/15264 @HyukjinKwon I agree. Would you have cycles to re-open https://github.com/apache/spark/pull/14215 by any chance? This is something that'd be great to have that in 2.2. --- If your project is

[GitHub] spark pull request #14412: [SPARK-15355] [CORE] Proactive block replication

2017-02-02 Thread shubhamchopra
Github user shubhamchopra commented on a diff in the pull request: https://github.com/apache/spark/pull/14412#discussion_r99190414 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala --- @@ -188,24 +189,45 @@ class BlockManagerMasterEndpoint(

[GitHub] spark pull request #14412: [SPARK-15355] [CORE] Proactive block replication

2017-02-02 Thread shubhamchopra
Github user shubhamchopra commented on a diff in the pull request: https://github.com/apache/spark/pull/14412#discussion_r99189219 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -1152,20 +1185,25 @@ private[spark] class BlockManager(

[GitHub] spark pull request #14412: [SPARK-15355] [CORE] Proactive block replication

2017-02-02 Thread shubhamchopra
Github user shubhamchopra commented on a diff in the pull request: https://github.com/apache/spark/pull/14412#discussion_r99185105 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -1152,20 +1185,25 @@ private[spark] class BlockManager(

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16733 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72286/ Test PASSed. ---

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16733 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16733 **[Test build #72286 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72286/testReport)** for PR 16733 at commit [`1b9e7a9`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #16779: [SPARK-19437] Rectify spark executor id in HeartbeatRece...

2017-02-02 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16779 @jinxing64 Good catch. Since you are touching this file, could you also replace other `askWithRetry` in this file with `askSync`? --- If your project is set up for it, you can reply to this email a

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16733 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16733 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72285/ Test PASSed. ---

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16733 **[Test build #72285 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72285/testReport)** for PR 16733 at commit [`352163e`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #16781: [SPARK-12297][SQL][POC] Hive compatibility for Parquet T...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16781 **[Test build #72288 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72288/testReport)** for PR 16781 at commit [`5b49ae0`](https://github.com/apache/spark/commit/5b

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16733 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72284/ Test PASSed. ---

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16733 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16733 **[Test build #72284 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72284/testReport)** for PR 16733 at commit [`76b005e`](https://github.com/apache/spark/commit/7

[GitHub] spark pull request #16762: [SPARK-19419] [SPARK-19420] Fix the cross join de...

2017-02-02 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16762#discussion_r99178232 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastNestedLoopJoinExec.scala --- @@ -339,6 +340,18 @@ case class BroadcastNe

[GitHub] spark issue #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-02 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16766 I'd second that. I'd be interested to know if this implementation changes behavior for `coalesce` --- If your project is set up for it, you can reply to this email and have your reply appear o

[GitHub] spark issue #16767: [SPARK-19386][SPARKR][DOC] Bisecting k-means in SparkR d...

2017-02-02 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16767 looking good to me. I'll wait for @wangmiao1981 to review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark issue #16729: [SPARK-19391][SparkR][ML] Tweedie GLM API for SparkR

2017-02-02 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16729 hmm.. I'm not sure we could do that because of Apache licensing policy (unless the owner decides to submit that here). let me think about this a bit. --- If your project is set up for

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-02-02 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r99174674 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/SelectedField.scala --- @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #14412: [SPARK-15355] [CORE] Proactive block replication

2017-02-02 Thread shubhamchopra
Github user shubhamchopra commented on a diff in the pull request: https://github.com/apache/spark/pull/14412#discussion_r99174290 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -1131,14 +1131,47 @@ private[spark] class BlockManager( }

[GitHub] spark pull request #14412: [SPARK-15355] [CORE] Proactive block replication

2017-02-02 Thread shubhamchopra
Github user shubhamchopra commented on a diff in the pull request: https://github.com/apache/spark/pull/14412#discussion_r99174354 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -1131,14 +1131,47 @@ private[spark] class BlockManager( }

[GitHub] spark issue #16779: [SPARK-19437] Rectify spark executor id in HeartbeatRece...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16779 **[Test build #3553 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3553/testReport)** for PR 16779 at commit [`06efcac`](https://github.com/apache/spark/commit/

[GitHub] spark issue #16729: [SPARK-19391][SparkR][ML] Tweedie GLM API for SparkR

2017-02-02 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/16729 @felixcheung The one from statmod will be masked and must be called using `statmod:tweedie`. We can copy the whole `tweedie` function from statmod into `SparkR` and this will avoid the is

[GitHub] spark issue #16781: [SPARK-12297][SQL][POC] Hive compatibility for Parquet T...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16781 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16781: [SPARK-12297][SQL][POC] Hive compatibility for Parquet T...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16781 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72287/ Test FAILed. ---

[GitHub] spark issue #16781: [SPARK-12297][SQL][POC] Hive compatibility for Parquet T...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16781 **[Test build #72287 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72287/testReport)** for PR 16781 at commit [`223ce2c`](https://github.com/apache/spark/commit/22

[GitHub] spark pull request #16781: [SPARK-12297][SQL][POC] Hive compatibility for Pa...

2017-02-02 Thread squito
GitHub user squito opened a pull request: https://github.com/apache/spark/pull/16781 [SPARK-12297][SQL][POC] Hive compatibility for Parquet Timestamps ## What changes were proposed in this pull request? Hive has very strange behavior when writing timestamps to parquet data.

[GitHub] spark issue #12135: [SPARK-14352][SQL] approxQuantile should support multi c...

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/12135 I am fine to create a separate one, but, normally, in Spark SQL, we do not create a separate JIRA for improving the related test case, if the original ones are missing. --- If your project is

[GitHub] spark issue #16738: [SPARK-19398] remove one misleading log in TaskSetManage...

2017-02-02 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/16738 @srowen Thanks a lot. I'll refine : ) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark issue #16780: [SPARK-19438] Both reading and updating executorDataMap ...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16780 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #16780: [SPARK-19438] Both reading and updating executorD...

2017-02-02 Thread jinxing64
GitHub user jinxing64 opened a pull request: https://github.com/apache/spark/pull/16780 [SPARK-19438] Both reading and updating executorDataMap should be guarded by CoarseGrainedSchedulerBackend.this.synchronized when handle RegisterExecutor. ## What changes were proposed in this pu

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16733 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16733 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72283/ Test PASSed. ---

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16733 **[Test build #72283 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72283/testReport)** for PR 16733 at commit [`3e603ee`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16733 **[Test build #72286 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72286/testReport)** for PR 16733 at commit [`1b9e7a9`](https://github.com/apache/spark/commit/1b

[GitHub] spark issue #16775: [SPARK-19433][ML] Periodic checkout datasets for long ml...

2017-02-02 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16775 `StringIndexer` and `OneHotEncoder` are just used as example here. The concept is to have a pipeline with enough long stages. --- If your project is set up for it, you can reply to this email and ha

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16733 **[Test build #72285 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72285/testReport)** for PR 16733 at commit [`352163e`](https://github.com/apache/spark/commit/35

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16733 **[Test build #72284 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72284/testReport)** for PR 16733 at commit [`76b005e`](https://github.com/apache/spark/commit/76

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16733 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72281/ Test PASSed. ---

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16733 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16733 **[Test build #72281 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72281/testReport)** for PR 16733 at commit [`712b80c`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #16778: [BUILD] Close stale PRs

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16778 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16775: [SPARK-19433][ML] Periodic checkout datasets for long ml...

2017-02-02 Thread DavidArenburg
Github user DavidArenburg commented on the issue: https://github.com/apache/spark/pull/16775 Wouldn't it better to Vectorize `StringIndexer` and `OneHotEncoder`? Like for instance `.na.fill` or `.na.replace` operate over the whole data set at once instead of running it in a loop? I fe

[GitHub] spark issue #16778: [BUILD] Close stale PRs

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16778 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72282/ Test FAILed. ---

[GitHub] spark issue #16778: [BUILD] Close stale PRs

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16778 **[Test build #72282 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72282/testReport)** for PR 16778 at commit [`4a54bb4`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #14204: [SPARK-16520] [WEBUI] Link executors to correspon...

2017-02-02 Thread nblintao
Github user nblintao closed the pull request at: https://github.com/apache/spark/pull/14204 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] spark issue #16779: [SPARK-19437] Rectify spark executor id in HeartbeatRece...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16779 **[Test build #3553 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3553/testReport)** for PR 16779 at commit [`06efcac`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #16779: [SPARK-19437] Rectify spark executor id in HeartbeatRece...

2017-02-02 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16779 CC @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if th

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-02 Thread mariusvniekerk
Github user mariusvniekerk commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r99132600 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -823,6 +825,17 @@ case class Rep

[GitHub] spark issue #16779: [SPARK-19437] Rectify spark executor id in HeartbeatRece...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16779 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #16779: [SPARK-19437] Rectify spark executor id in Heartb...

2017-02-02 Thread jinxing64
GitHub user jinxing64 opened a pull request: https://github.com/apache/spark/pull/16779 [SPARK-19437] Rectify spark executor id in HeartbeatReceiverSuite. ## What changes were proposed in this pull request? The current code in `HeartbeatReceiverSuite`, executorId is set as b

[GitHub] spark issue #16778: [BUILD] Close stale PRs

2017-02-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16778 @srowen, I have few candidates we might have to consider. My list is almost overlapped with this. So, I have only the ones I inclusively picked up. I think you already took a look for most of

[GitHub] spark issue #16771: [SPARK-19429][PYTHON][SQL] Support slice arguments in Co...

2017-02-02 Thread zero323
Github user zero323 commented on the issue: https://github.com/apache/spark/pull/16771 cc @holdenk --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if t

[GitHub] spark issue #16652: [SPARK-19234][MLLib] AFTSurvivalRegression should fail f...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16652 **[Test build #3552 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3552/testReport)** for PR 16652 at commit [`c855976`](https://github.com/apache/spark/commit/

[GitHub] spark issue #16777: [SPARK-19435][SQL] Type coercion between ArrayTypes

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16777 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72280/ Test PASSed. ---

[GitHub] spark issue #16777: [SPARK-19435][SQL] Type coercion between ArrayTypes

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16777 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15264: [SPARK-17477][SQL] SparkSQL cannot handle schema evoluti...

2017-02-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15264 BTW, I just wonder if this PR is closable if we want to do this with more types :). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as wel

[GitHub] spark issue #16777: [SPARK-19435][SQL] Type coercion between ArrayTypes

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16777 **[Test build #72280 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72280/testReport)** for PR 16777 at commit [`bd0d9f7`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16733 **[Test build #72283 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72283/testReport)** for PR 16733 at commit [`3e603ee`](https://github.com/apache/spark/commit/3e

[GitHub] spark issue #16778: [BUILD] Close stale PRs

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16778 **[Test build #72282 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72282/testReport)** for PR 16778 at commit [`4a54bb4`](https://github.com/apache/spark/commit/4a

[GitHub] spark pull request #16778: [BUILD] Close stale PRs

2017-02-02 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/16778 [BUILD] Close stale PRs Closes #15736 Closes #16309 Closes #16485 Closes #16502 Closes #16196 Closes #16498 Closes #12380 Closes #16764 You can merge this pull request in

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16733 **[Test build #72281 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72281/testReport)** for PR 16733 at commit [`712b80c`](https://github.com/apache/spark/commit/71

[GitHub] spark pull request #16744: [SPARK-19405][STREAMING] Support for cross-accoun...

2017-02-02 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16744#discussion_r99110527 --- Diff: pom.xml --- @@ -146,6 +146,8 @@ hadoop2 0.7.1 1.6.2 + +1.10.61 --- End diff -- This is probab

[GitHub] spark issue #16652: [SPARK-19234][MLLib] AFTSurvivalRegression should fail f...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16652 **[Test build #3552 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3552/testReport)** for PR 16652 at commit [`c855976`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #16767: [SPARK-19386][SPARKR][DOC] Bisecting k-means in SparkR d...

2017-02-02 Thread krishnakalyan3
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/16767 ping @wangmiao1981 @felixcheung --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark issue #16738: [SPARK-19398] remove one misleading log in TaskSetManage...

2017-02-02 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16738 Rather than remove the comment, maybe it's better to clarify it as you say, to refer to being "completed" rather than "successful". I don't know enough to evaluate whether you're right about that cha

[GitHub] spark issue #16776: [SPARK-19436][SQL] Add missing tests for approxQuantile

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16776 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16776: [SPARK-19436][SQL] Add missing tests for approxQuantile

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16776 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72279/ Test PASSed. ---

[GitHub] spark issue #16776: [SPARK-19436][SQL] Add missing tests for approxQuantile

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16776 **[Test build #72279 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72279/testReport)** for PR 16776 at commit [`917fd6e`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #16776: [SPARK-19436][SQL] Add missing tests for approxQuantile

2017-02-02 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/16776 cc @gatorsmile @HyukjinKwon @holdenk @MLnick --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16776: [SPARK-19436][SQL] Add missing tests for approxQuantile

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16776 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16776: [SPARK-19436][SQL] Add missing tests for approxQuantile

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16776 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72278/ Test PASSed. ---

[GitHub] spark issue #16776: [SPARK-19436][SQL] Add missing tests for approxQuantile

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16776 **[Test build #72278 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72278/testReport)** for PR 16776 at commit [`db9ffc2`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/16733 oh, sorry. I'll add the test soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark issue #16733: [SPARK-19392][SQL] Fix the bug that throws an exception ...

2017-02-02 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16733 @maropu are you able to add a simple test, in the style of other similar tests? this looks like a fine fix. --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark issue #12135: [SPARK-14352][SQL] approxQuantile should support multi c...

2017-02-02 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/12135 @MLnick I created SPARK-19436[https://issues.apache.org/jira/browse/SPARK-19436] for it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #12135: [SPARK-14352][SQL] approxQuantile should support multi c...

2017-02-02 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/12135 @gatorsmile it's a good point about the tests. However this JIRA & PR was for exposing the multi-column functionality of `approxQuantiles`. The missing test cases date back to original impl really. I

[GitHub] spark issue #16775: [SPARK-19433][ML] Periodic checkout datasets for long ml...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16775 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16775: [SPARK-19433][ML] Periodic checkout datasets for long ml...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16775 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72277/ Test PASSed. ---

[GitHub] spark issue #16775: [SPARK-19433][ML] Periodic checkout datasets for long ml...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16775 **[Test build #72277 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72277/testReport)** for PR 16775 at commit [`32c90dd`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #16777: [SPARK-19435][SQL] Type coercion between ArrayTypes

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16777 **[Test build #72280 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72280/testReport)** for PR 16777 at commit [`bd0d9f7`](https://github.com/apache/spark/commit/bd

[GitHub] spark issue #16777: [SPARK-19435][SQL] Type coercion between ArrayTypes

2017-02-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16777 cc @hvanhovell, could you maybe take a look please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #16777: [SPARK-19435][SQL] Type coercion between ArrayTyp...

2017-02-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16777#discussion_r99093765 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercionSuite.scala --- @@ -53,7 +53,8 @@ class TypeCoercionSuite exten

[GitHub] spark pull request #16777: [SPARK-19435][SQL] Type coercion between ArrayTyp...

2017-02-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16777#discussion_r99093557 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -101,24 +101,13 @@ object TypeCoercion {

[GitHub] spark pull request #16777: [SPARK-19435][SQL] Type coercion between ArrayTyp...

2017-02-02 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/16777 [SPARK-19435][SQL] Type coercion between ArrayTypes ## What changes were proposed in this pull request? This PR proposes to support type coercion between `ArrayType`s where the element

[GitHub] spark issue #16776: [SPARK-14352][FOLLOWUP][SQL] update tests for approxQuan...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16776 **[Test build #72279 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72279/testReport)** for PR 16776 at commit [`917fd6e`](https://github.com/apache/spark/commit/91

[GitHub] spark issue #16776: [SPARK-14352][FOLLOWUP][SQL] update tests for approxQuan...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16776 **[Test build #72278 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72278/testReport)** for PR 16776 at commit [`db9ffc2`](https://github.com/apache/spark/commit/db

[GitHub] spark pull request #16776: [SPARK-14352][FOLLOWUP][SQL] add tests for approx...

2017-02-02 Thread zhengruifeng
GitHub user zhengruifeng opened a pull request: https://github.com/apache/spark/pull/16776 [SPARK-14352][FOLLOWUP][SQL] add tests for approxQuantile & ## What changes were proposed in this pull request? 1, check the behavior with illegal `quantiles` and `relativeError` 2, ad

[GitHub] spark pull request #15797: [SPARK-17990][SPARK-18302][SQL] correct several p...

2017-02-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15797#discussion_r99075673 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -810,13 +825,43 @@ private[spark] class HiveExternalCatalog(co

[GitHub] spark pull request #12135: [SPARK-14352][SQL] approxQuantile should support ...

2017-02-02 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/12135#discussion_r99072952 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala --- @@ -75,13 +76,43 @@ final class DataFrameStatFunctions private[sql](d

[GitHub] spark issue #14725: [SPARK-17161] [PYSPARK][ML] Add PySpark-ML JavaWrapper c...

2017-02-02 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14725 For future reference: Merged into master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark pull request #12135: [SPARK-14352][SQL] approxQuantile should support ...

2017-02-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/12135#discussion_r99071851 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala --- @@ -75,13 +76,43 @@ final class DataFrameStatFunctions private[sq

[GitHub] spark issue #16465: [SPARK-19064][PySpark]Fix pip installing of sub componen...

2017-02-02 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16465 Merged into master & branch-2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16775: [SPARK-19433][ML] Periodic checkout datasets for long ml...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16775 **[Test build #72277 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72277/testReport)** for PR 16775 at commit [`32c90dd`](https://github.com/apache/spark/commit/32

[GitHub] spark issue #16765: [SPARK-19425][SQL] Make df.except work for UDT

2017-02-02 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16765 @gatorsmile Thanks for the review. I will update this soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark issue #16775: [SPARK-19433][ML] Periodic checkout datasets for long ml...

2017-02-02 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16775 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, o

[GitHub] spark issue #12135: [SPARK-14352][SQL] approxQuantile should support multi c...

2017-02-02 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/12135 Thanks for the reminder @gatorsmile (it wasn't in the list of things to do when merging so I'll follow up and update the http://spark.apache.org/committers.html docs to add that as a follow up step

[GitHub] spark pull request #12135: [SPARK-14352][SQL] approxQuantile should support ...

2017-02-02 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/12135#discussion_r99070440 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala --- @@ -75,13 +76,43 @@ final class DataFrameStatFunctions private[sql](d

[GitHub] spark issue #16765: [SPARK-19425][SQL] Make df.except work for UDT

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16765 The fix looks good to me. Just the comment about the test case. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

<    1   2   3   4   >