[GitHub] spark issue #9608: [SPARK-11638] [Mesos + Docker Bridge networking]: Run Spa...

2017-02-16 Thread radekg
Github user radekg commented on the issue: https://github.com/apache/spark/pull/9608 @cherryii feel free to reach out if you need additional help. I'm sure your issues can be sorted out. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #16962: [SPARK-18120 ][SPARK-19557][SQL] Call QueryExecutionList...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16962 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73009/ Test PASSed. ---

[GitHub] spark issue #16962: [SPARK-18120 ][SPARK-19557][SQL] Call QueryExecutionList...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16962 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16962: [SPARK-18120 ][SPARK-19557][SQL] Call QueryExecutionList...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16962 **[Test build #73009 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73009/testReport)** for PR 16962 at commit

[GitHub] spark issue #16965: [Spark-18450][ML] Scala API Change for LSH AND-amplifica...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16965 **[Test build #73016 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73016/testReport)** for PR 16965 at commit

[GitHub] spark pull request #16965: [Spark-18450][ML] Scala API Change for LSH AND-am...

2017-02-16 Thread Yunni
GitHub user Yunni opened a pull request: https://github.com/apache/spark/pull/16965 [Spark-18450][ML] Scala API Change for LSH AND-amplification ## What changes were proposed in this pull request? Implemented a new Param numHashFunctions as the dimension of AND-amplification

[GitHub] spark issue #16895: [SPARK-15615][SQL] Add an API to load DataFrame from Dat...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16895 **[Test build #73015 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73015/testReport)** for PR 16895 at commit

[GitHub] spark pull request #13899: [SPARK-16196][SQL] Codegen in-memory scan with Co...

2017-02-16 Thread andrewor14
Github user andrewor14 closed the pull request at: https://github.com/apache/spark/pull/13899 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #13899: [SPARK-16196][SQL] Codegen in-memory scan with ColumnarB...

2017-02-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13899 Closing for now; too many conflicts. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16964: [SPARK-19534][TESTS] Convert Java tests to use lambdas, ...

2017-02-16 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16964 +1, LGTM, too. Very tidy! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16961: [SPARK-19534][EXAMPLES] Convert Java tests to use lambda...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16961 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16961: [SPARK-19534][EXAMPLES] Convert Java tests to use lambda...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16961 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73013/ Test PASSed. ---

[GitHub] spark issue #16961: [SPARK-19534][EXAMPLES] Convert Java tests to use lambda...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16961 **[Test build #73013 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73013/testReport)** for PR 16961 at commit

[GitHub] spark issue #7379: [SPARK-8682][SQL][WIP] Range Join

2017-02-16 Thread zzeekk
Github user zzeekk commented on the issue: https://github.com/apache/spark/pull/7379 Same here. A Workaround is to build blocks and add them as equi-join condition. But then you need to make an additional join on the following block und coalesce the results. --- If your project is

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-02-16 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16944 I've updated the PR based on feedback received. Changes from previous commit: - Fixed a couple indent issues - Clarify some HiveSchemaInferenceSuite comments and general cleanup - Add

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-16 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r101625724 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -296,6 +296,21 @@ object SQLConf { .longConf

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16944 **[Test build #73014 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73014/testReport)** for PR 16944 at commit

[GitHub] spark issue #16961: [SPARK-19534][EXAMPLES] Convert Java tests to use lambda...

2017-02-16 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16961 +1 LGTM! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16961: [SPARK-19534][EXAMPLES] Convert Java tests to use lambda...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16961 **[Test build #73013 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73013/testReport)** for PR 16961 at commit

[GitHub] spark issue #16961: [SPARK-19534][EXAMPLES] Convert Java tests to use lambda...

2017-02-16 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16961 Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16962: [SPARK-18120 ][SPARK-19557][SQL] Call QueryExecutionList...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16962 **[Test build #73012 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73012/testReport)** for PR 16962 at commit

[GitHub] spark pull request #16962: [SPARK-18120 ][SPARK-19557][SQL] Call QueryExecut...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16962#discussion_r101622507 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -573,6 +575,24 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark issue #16623: [SPARK-19066][SPARKR][Backport-2.1]:LDA doesn't set opti...

2017-02-16 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16623 Surely. This is a small enough change (one line) that we could revert easily. But just so I understand, this isn't just a behavior change per se - this is the API not doing that is

[GitHub] spark issue #16949: [SPARK-16122][CORE] Add rest api for job environment

2017-02-16 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/16949 @uncleGen As followup I'd recommend opening a JIRA to research and address the potential security flaws in the env tab. I know we've already merged some fixes to hide passwords in the env tab,

[GitHub] spark pull request #16963: [SPARK-19632] Non hive external catalogs

2017-02-16 Thread robert3005
Github user robert3005 closed the pull request at: https://github.com/apache/spark/pull/16963 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-16 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r101618763 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -158,17 +160,7 @@ trait CheckAnalysis extends

[GitHub] spark pull request #16246: [SPARK-18814][SQL] CheckAnalysis rejects TPCDS qu...

2017-02-16 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/16246#discussion_r101618460 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -136,24 +140,35 @@ trait CheckAnalysis extends

[GitHub] spark issue #16964: [SPARK-19534][TESTS] Convert Java tests to use lambdas, ...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16964 **[Test build #73011 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73011/testReport)** for PR 16964 at commit

[GitHub] spark pull request #16964: [SPARK-19534][TESTS] Convert Java tests to use la...

2017-02-16 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/16964 [SPARK-19534][TESTS] Convert Java tests to use lambdas, Java 8 features ## What changes were proposed in this pull request? Convert tests to use Java 8 lambdas, and modest related fixes to

[GitHub] spark pull request #16690: [SPARK-19347] ReceiverSupervisorImpl can add bloc...

2017-02-16 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16690#discussion_r101616883 --- Diff: core/src/main/scala/org/apache/spark/rpc/RpcEndpointRef.scala --- @@ -75,10 +105,11 @@ private[spark] abstract class RpcEndpointRef(conf:

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-16 Thread ron8hu
Github user ron8hu commented on the issue: https://github.com/apache/spark/pull/16395 Hi @cloud-fan I revised the code using latest Range class. Thanks for reviewing the code. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #9608: [SPARK-11638] [Mesos + Docker Bridge networking]: Run Spa...

2017-02-16 Thread cherryii
Github user cherryii commented on the issue: https://github.com/apache/spark/pull/9608 @radekg, Thank you, I was starting to worry that is the case. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #16963: [SPARK-19632] Non hive external catalogs

2017-02-16 Thread robert3005
GitHub user robert3005 opened a pull request: https://github.com/apache/spark/pull/16963 [SPARK-19632] Non hive external catalogs ## What changes were proposed in this pull request? Open up ExternalCatalog and SessionState in order to allow integrating other catalogs

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16395 **[Test build #73010 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73010/testReport)** for PR 16395 at commit

[GitHub] spark issue #16826: [WIP][SPARK-19540][SQL] Add ability to clone SparkSessio...

2017-02-16 Thread kunalkhamar
Github user kunalkhamar commented on the issue: https://github.com/apache/spark/pull/16826 @rxin The change from `SessionState(SparkSession,parentSessionState)` to `SessionState(SparkSession,SQLConf,SessionCatalog,...)` is mostly done. Still have 4 failing tests (OOMs) and will

[GitHub] spark issue #16947: [SPARK-19617][SS]Fix the race condition when starting an...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16947 **[Test build #3575 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3575/testReport)** for PR 16947 at commit

[GitHub] spark pull request #16962: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-16 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16962#discussion_r101610336 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -211,13 +211,15 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark issue #16962: [SPARK-18120 ][SQL] Call QueryExecutionListener callback...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16962 **[Test build #73009 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73009/testReport)** for PR 16962 at commit

[GitHub] spark issue #16962: [SPARK-18120 ][SQL] Call QueryExecutionListener callback...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16962 cc @salilsurendran @vanzin @gatorsmile @sameeragarwal --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13440 **[Test build #73008 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73008/testReport)** for PR 13440 at commit

[GitHub] spark pull request #16962: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-16 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/16962 [SPARK-18120 ][SQL] Call QueryExecutionListener callback methods for DataFrameWriter methods ## What changes were proposed in this pull request? We only notify `QueryExecutionListener`

[GitHub] spark issue #16945: [SPARK-19616][SparkR]:weightCol and aggregationDepth sho...

2017-02-16 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/16945 I will add tests. Now I am looking for dataset other than iris to be used in the document. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2017-02-16 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/13440 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #9608: [SPARK-11638] [Mesos + Docker Bridge networking]: Run Spa...

2017-02-16 Thread radekg
Github user radekg commented on the issue: https://github.com/apache/spark/pull/9608 It's not going to work with bridge network. The options will work fine with host network. Bridge is a different story. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2017-02-16 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/13440 @erikerlandson I am just helping clearing the stale PRs. :) I have no idea whether they have intention to accept it. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #16960: [SPARK-19447] Make Range operator generate "recordsRead"...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16960 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16960: [SPARK-19447] Make Range operator generate "recordsRead"...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16960 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73005/ Test PASSed. ---

[GitHub] spark issue #16960: [SPARK-19447] Make Range operator generate "recordsRead"...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16960 **[Test build #73005 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73005/testReport)** for PR 16960 at commit

[GitHub] spark issue #16715: [Spark-18080][ML][PYTHON] Python API & Examples for Loca...

2017-02-16 Thread Yunni
Github user Yunni commented on the issue: https://github.com/apache/spark/pull/16715 Sure. Will do. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-16 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r101606197 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -186,8 +212,7 @@ private[hive] class

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-16 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r101605728 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -296,6 +296,17 @@ object SQLConf { .longConf

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-16 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r101605711 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -163,6 +163,10 @@ case class BucketSpec( * @param

[GitHub] spark issue #15125: [SPARK-5484][GraphX] Periodically do checkpoint in Prege...

2017-02-16 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/15125 > I think @mallman is saying he would merge changes to @dding3 branch Yes, or I could do them in a follow up PR. Or @dding3 could do them without my PR. I'm not hung up on getting credit.

[GitHub] spark pull request #16844: [SPARK-19500] [SQL] Fix off-by-one bug in BytesTo...

2017-02-16 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/16844#discussion_r101592592 --- Diff: core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java --- @@ -698,7 +698,7 @@ public boolean append(Object kbase, long koff, int

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13440 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2017-02-16 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/13440 i stopped the build as i need to restart jenkins... i'll retrigger this when we're back up and running. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13440 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73006/ Test FAILed. ---

[GitHub] spark issue #16960: [SPARK-19447] Make Range operator generate "recordsRead"...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16960 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16960: [SPARK-19447] Make Range operator generate "recordsRead"...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16960 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73004/ Test PASSed. ---

[GitHub] spark issue #16960: [SPARK-19447] Make Range operator generate "recordsRead"...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16960 **[Test build #73004 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73004/testReport)** for PR 16960 at commit

[GitHub] spark issue #16947: [SPARK-19617][SS]Fix the race condition when starting an...

2017-02-16 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16947 > It also removes the runUninterruptibly hack from ``HDFSMetadata` I will submit a backport PR for 2.1 to not include this change because this is needed for 2.1 due to HADOOP-10622 (Master

[GitHub] spark pull request #16499: [SPARK-17204][CORE] Fix replicated off heap stora...

2017-02-16 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/16499#discussion_r101602604 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -813,7 +813,14 @@ private[spark] class BlockManager(

[GitHub] spark pull request #16947: [SPARK-19617][SS]Fix the race condition when star...

2017-02-16 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16947#discussion_r101602345 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -250,53 +249,56 @@ class StreamExecution(

[GitHub] spark pull request #16499: [SPARK-17204][CORE] Fix replicated off heap stora...

2017-02-16 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/16499#discussion_r101602014 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -813,7 +813,14 @@ private[spark] class BlockManager(

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-16 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r101594561 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -40,19 +42,179 @@ abstract class PlanExpression[T <:

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-16 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r101596895 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -83,29 +95,150 @@ object RewritePredicateSubquery

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-16 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r101590145 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -40,19 +42,179 @@ abstract class PlanExpression[T <:

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-16 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r101590658 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -40,19 +42,179 @@ abstract class PlanExpression[T <:

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-16 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r101593305 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -40,19 +42,179 @@ abstract class PlanExpression[T <:

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-16 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r101598818 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -365,17 +385,66 @@ object TypeCoercion {

[GitHub] spark issue #16623: [SPARK-19066][SPARKR][Backport-2.1]:LDA doesn't set opti...

2017-02-16 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/16623 @felixcheung Just saw this was backported to 2.1.1. Since this is a fairly significant behavioral change, I recommend we revert this backport. I could imagine workloads working with EM but

[GitHub] spark pull request #16961: [SPARK-19534][EXAMPLES] Convert Java tests to use...

2017-02-16 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16961#discussion_r101597873 --- Diff: examples/src/main/java/org/apache/spark/examples/sql/JavaSQLDataSourceExample.java --- @@ -139,11 +139,9 @@ private static void

[GitHub] spark pull request #16961: [SPARK-19534][EXAMPLES] Convert Java tests to use...

2017-02-16 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16961#discussion_r101597431 --- Diff: examples/src/main/java/org/apache/spark/examples/ml/JavaModelSelectionViaTrainValidationSplitExample.java --- @@ -32,9 +32,6 @@ /** *

[GitHub] spark pull request #16961: [SPARK-19534][EXAMPLES] Convert Java tests to use...

2017-02-16 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16961#discussion_r101597310 --- Diff: examples/src/main/java/org/apache/spark/examples/JavaLogQuery.java --- @@ -40,7 +38,7 @@ */ public final class JavaLogQuery {

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-16 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r101597685 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -83,29 +95,150 @@ object RewritePredicateSubquery

[GitHub] spark pull request #16961: [SPARK-19534][EXAMPLES] Convert Java tests to use...

2017-02-16 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/16961 [SPARK-19534][EXAMPLES] Convert Java tests to use lambdas, Java 8 features ## What changes were proposed in this pull request? Convert Java tests to use lambdas, Java 8 features.

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-16 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r101597000 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala --- @@ -622,7 +622,12 @@ class SubquerySuite extends QueryTest with

[GitHub] spark pull request #16957: [SPARK-19550][HOTFIX][BUILD] Use JAVA_HOME/bin/ja...

2017-02-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16957 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16957: [SPARK-19550][HOTFIX][BUILD] Use JAVA_HOME/bin/java if J...

2017-02-16 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16957 Merged to master, as it's at worst a minor change to mima which makes it a bit more consistent with scripts, and, likely resolves the build problem. --- If your project is set up for it, you can

[GitHub] spark issue #9608: [SPARK-11638] [Mesos + Docker Bridge networking]: Run Spa...

2017-02-16 Thread cherryii
Github user cherryii commented on the issue: https://github.com/apache/spark/pull/9608 I'm already using those three properties and the error I posted above is what happens in bridge networking mode. Note: I'm using spark-submit ... --conf spark.driver.port=$PORT0 etc... [app

[GitHub] spark pull request #16857: [SPARK-19517][SS] KafkaSource fails to initialize...

2017-02-16 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16857#discussion_r101594339 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala --- @@ -141,6 +143,104 @@ class KafkaSourceSuite

[GitHub] spark pull request #16857: [SPARK-19517][SS] KafkaSource fails to initialize...

2017-02-16 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16857#discussion_r101594563 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala --- @@ -97,16 +100,29 @@ private[kafka010] class

[GitHub] spark pull request #16857: [SPARK-19517][SS] KafkaSource fails to initialize...

2017-02-16 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16857#discussion_r101593021 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala --- @@ -88,6 +89,8 @@ private[kafka010] class

[GitHub] spark pull request #16857: [SPARK-19517][SS] KafkaSource fails to initialize...

2017-02-16 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16857#discussion_r101594004 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala --- @@ -97,16 +100,29 @@ private[kafka010] class

[GitHub] spark pull request #16499: [SPARK-17204][CORE] Fix replicated off heap stora...

2017-02-16 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/16499#discussion_r101592331 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -1018,7 +1025,9 @@ private[spark] class BlockManager( try {

[GitHub] spark issue #16947: [SPARK-19617][SS][WIP]Don't interrupt 'mkdirs' to workar...

2017-02-16 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16947 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #16895: [SPARK-15615][SQL] Add an API to load DataFrame f...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16895#discussion_r101591685 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1364,10 +1364,11 @@ class JsonSuite extends

[GitHub] spark pull request #16672: [SPARK-19329][SQL]Reading from or writing to a da...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16672#discussion_r101590857 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -1816,4 +1816,123 @@ class DDLSuite extends QueryTest

[GitHub] spark issue #16946: [SPARK-19554][UI,YARN] Allow SHS URL to be used for trac...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16946 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16946: [SPARK-19554][UI,YARN] Allow SHS URL to be used for trac...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16946 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73007/ Test PASSed. ---

[GitHub] spark issue #16946: [SPARK-19554][UI,YARN] Allow SHS URL to be used for trac...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16946 **[Test build #73007 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73007/testReport)** for PR 16946 at commit

[GitHub] spark pull request #16956: [SPARK-19598][SQL]Remove the alias parameter in U...

2017-02-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16956#discussion_r101587263 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -645,17 +645,21 @@ class AstBuilder extends

[GitHub] spark issue #16956: [SPARK-19598][SQL]Remove the alias parameter in Unresolv...

2017-02-16 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16956 I like this change. To run unit test cases in your local environment, below are some common commands when I develop Spark SQL - To do the local style check dev/lint-scala

[GitHub] spark issue #16946: [SPARK-19554][UI,YARN] Allow SHS URL to be used for trac...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16946 **[Test build #73007 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73007/testReport)** for PR 16946 at commit

[GitHub] spark issue #16946: [SPARK-19554][UI,YARN] Allow SHS URL to be used for trac...

2017-02-16 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16946 @tgravescs pinging you since this is all YARN-side (doesn't really touch the UI). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #9608: [SPARK-11638] [Mesos + Docker Bridge networking]: Run Spa...

2017-02-16 Thread radekg
Github user radekg commented on the issue: https://github.com/apache/spark/pull/9608 No matter if you was to run this in bridge or host mode, the best idea would be to request a number of ports. In case of `HOST` network only the `port` needs to be requested. If you can use Marathon,

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13440 **[Test build #73006 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73006/testReport)** for PR 13440 at commit

[GitHub] spark pull request #16776: [SPARK-19436][SQL] Add missing tests for approxQu...

2017-02-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16776 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16776: [SPARK-19436][SQL] Add missing tests for approxQuantile

2017-02-16 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16776 Thanks! Merging to master. Please continue to finish the work https://issues.apache.org/jira/browse/SPARK-19573 --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #16776: [SPARK-19436][SQL] Add missing tests for approxQuantile

2017-02-16 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16776 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

<    1   2   3   4   5   6   >