[jira] [Assigned] (SPARK-13186) Migrate away from SynchronizedMap which is derpecated

[ https://issues.apache.org/jira/browse/SPARK-13186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13186: Assignee: Apache Spark > Migrate away from SynchronizedMap which is derpecated >

[jira] [Assigned] (SPARK-13186) Migrate away from SynchronizedMap which is derpecated

[ https://issues.apache.org/jira/browse/SPARK-13186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13186: Assignee: (was: Apache Spark) > Migrate away from SynchronizedMap which is derpecated

[jira] [Commented] (SPARK-13186) Migrate away from SynchronizedMap which is derpecated

[ https://issues.apache.org/jira/browse/SPARK-13186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136052#comment-15136052 ] Apache Spark commented on SPARK-13186: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Commented] (SPARK-13171) Update promise & future to Promise and Future as the old ones are deprecated

[ https://issues.apache.org/jira/browse/SPARK-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136006#comment-15136006 ] Ted Yu commented on SPARK-13171:

[jira] [Commented] (SPARK-10935) Avito Context Ad Clicks

[ https://issues.apache.org/jira/browse/SPARK-10935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136053#comment-15136053 ] Ruslan Dautkhanov commented on SPARK-10935: --- I noticed outer joins. Spark before 1.5 used

[jira] [Comment Edited] (SPARK-10935) Avito Context Ad Clicks

[ https://issues.apache.org/jira/browse/SPARK-10935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136053#comment-15136053 ] Ruslan Dautkhanov edited comment on SPARK-10935 at 2/6/16 11:16 PM: I

[jira] [Commented] (SPARK-13171) Update promise & future to Promise and Future as the old ones are deprecated

[ https://issues.apache.org/jira/browse/SPARK-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135979#comment-15135979 ] Jakob Odersky commented on SPARK-13171: --- [~tedyu] On GitHub, [~smilegator] wrote: > BTW, I also

[jira] [Commented] (SPARK-13171) Update promise & future to Promise and Future as the old ones are deprecated

[ https://issues.apache.org/jira/browse/SPARK-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136004#comment-15136004 ] Ted Yu commented on SPARK-13171: No, I have not. bq. Could it be that Hive does some funky reflection

[jira] [Commented] (SPARK-13171) Update promise & future to Promise and Future as the old ones are deprecated

[ https://issues.apache.org/jira/browse/SPARK-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135976#comment-15135976 ] Jakob Odersky commented on SPARK-13171: --- I can't see any reason why the above change should impact

[jira] [Commented] (SPARK-13171) Update promise & future to Promise and Future as the old ones are deprecated

[ https://issues.apache.org/jira/browse/SPARK-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135972#comment-15135972 ] Ted Yu commented on SPARK-13171: The above Jenkins jobs were running Scala 2.11 > Update promise &

[jira] [Resolved] (SPARK-5865) Add doc warnings for methods that return local data structures

[ https://issues.apache.org/jira/browse/SPARK-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5865. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10874

[jira] [Commented] (SPARK-12861) Changes to support KMeans with large feature space

[ https://issues.apache.org/jira/browse/SPARK-12861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135927#comment-15135927 ] Sean Owen commented on SPARK-12861: --- I don't think that's the eventual topic of SPARK-4039 (see the

[jira] [Resolved] (SPARK-4878) driverPropsFetcher causes spurious Akka disassociate errors

[ https://issues.apache.org/jira/browse/SPARK-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4878. -- Resolution: Not A Problem I'm guessing 'not a problem' in that there's no Akka involved anymore >

[jira] [Commented] (SPARK-13198) sc.stop() does not clean up on driver, causes Java heap OOM.

[ https://issues.apache.org/jira/browse/SPARK-13198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135889#comment-15135889 ] Sean Owen commented on SPARK-13198: --- You've described a different problem, which you opened a duplicate

[jira] [Resolved] (SPARK-13218) Executor failed after SparkContext start and start again

[ https://issues.apache.org/jira/browse/SPARK-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13218. --- Resolution: Not A Problem I don't think it's ever been possible to change masters in one app. >

[jira] [Closed] (SPARK-13218) Executor failed after SparkContext start and start again

[ https://issues.apache.org/jira/browse/SPARK-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-13218. - > Executor failed after SparkContext start and start again >

[jira] [Updated] (SPARK-13218) Executor failed after SparkContext stop and start again

[ https://issues.apache.org/jira/browse/SPARK-13218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leo wu updated SPARK-13218: --- Summary: Executor failed after SparkContext stop and start again (was: Executor failed after

[jira] [Updated] (SPARK-5865) Add doc warnings for methods that return local data structures

[ https://issues.apache.org/jira/browse/SPARK-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5865: - Assignee: Nicholas Chammas > Add doc warnings for methods that return local data structures >

[jira] [Resolved] (SPARK-12892) Support plugging in Spark scheduler

[ https://issues.apache.org/jira/browse/SPARK-12892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12892. --- Resolution: Duplicate > Support plugging in Spark scheduler >

[jira] [Assigned] (SPARK-9307) Logging: Make it either stable or private[spark]

[ https://issues.apache.org/jira/browse/SPARK-9307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9307: --- Assignee: (was: Apache Spark) > Logging: Make it either stable or private[spark] >

[jira] [Assigned] (SPARK-9307) Logging: Make it either stable or private[spark]

[ https://issues.apache.org/jira/browse/SPARK-9307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9307: --- Assignee: Apache Spark > Logging: Make it either stable or private[spark] >

[jira] [Commented] (SPARK-9307) Logging: Make it either stable or private[spark]

[ https://issues.apache.org/jira/browse/SPARK-9307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135922#comment-15135922 ] Apache Spark commented on SPARK-9307: - User 'srowen' has created a pull request for this issue:

[jira] [Commented] (SPARK-13062) Overwriting same file with new schema destroys original file.

[ https://issues.apache.org/jira/browse/SPARK-13062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135932#comment-15135932 ] Sean Owen commented on SPARK-13062: --- I'm sure there could be a better error, sure. But I also wouldn't

[jira] [Resolved] (SPARK-13199) Upgrade apache httpclient version to the latest 4.5 for security

[ https://issues.apache.org/jira/browse/SPARK-13199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13199. --- Resolution: Duplicate Ted this is more of what I mean -- this was a duplicate. Please search first

[jira] [Updated] (SPARK-13216) Spark streaming application not honoring --num-executors in restarting of an application from a checkpoint

[ https://issues.apache.org/jira/browse/SPARK-13216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13216: -- Priority: Minor (was: Major) I think checkpointing is for restarting a failed application in as

[jira] [Commented] (SPARK-13226) MLLib PowerIteration Clustering depends on deprecated KMeans setRuns API

[ https://issues.apache.org/jira/browse/SPARK-13226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136113#comment-15136113 ] holdenk commented on SPARK-13226: - Also update the links since the PDF linked to in the model has a new

[jira] [Updated] (SPARK-13226) MLLib PowerIteration Clustering depends on deprecated KMeans setRuns API

[ https://issues.apache.org/jira/browse/SPARK-13226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-13226: Description: The current MLLib PowerIteration clustering implementation sets the number of parallel runs

[jira] [Issue Comment Deleted] (SPARK-13201) Make a private non-deprecated version of setRuns

[ https://issues.apache.org/jira/browse/SPARK-13201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-13201: Comment: was deleted (was: Looking at the only use inside of the MLLib API, I'm pretty convinced its

[jira] [Issue Comment Deleted] (SPARK-13226) MLLib PowerIteration Clustering depends on deprecated KMeans setRuns API

[ https://issues.apache.org/jira/browse/SPARK-13226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-13226: Comment: was deleted (was: Further poking shows that they did both depending on the kmeans algorithm

[jira] [Created] (SPARK-13227) Risky apply() in OpenHashMap

Nan Zhu created SPARK-13227: --- Summary: Risky apply() in OpenHashMap Key: SPARK-13227 URL: https://issues.apache.org/jira/browse/SPARK-13227 Project: Spark Issue Type: Bug Components:

[jira] [Commented] (SPARK-13013) Replace example code in mllib-clustering.md using include_example

[ https://issues.apache.org/jira/browse/SPARK-13013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136167#comment-15136167 ] Xin Ren commented on SPARK-13013: - I'm working on this now, thanks :) > Replace example code in

[jira] [Commented] (SPARK-13226) MLLib PowerIteration Clustering depends on deprecated KMeans setRuns API

[ https://issues.apache.org/jira/browse/SPARK-13226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136118#comment-15136118 ] holdenk commented on SPARK-13226: - Further poking shows that they did both depending on the kmeans

[jira] [Commented] (SPARK-11472) SparkContext creation error after sc.stop() when Spark is compiled for Hive

[ https://issues.apache.org/jira/browse/SPARK-11472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136127#comment-15136127 ] Atkins commented on SPARK-11472: Reproduced in kerberized hadoop cluster with Spark 1.6.0 compiled with

[jira] [Commented] (SPARK-13019) Replace example code in mllib-statistics.md using include_example

[ https://issues.apache.org/jira/browse/SPARK-13019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136166#comment-15136166 ] Apache Spark commented on SPARK-13019: -- User 'keypointt' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13019) Replace example code in mllib-statistics.md using include_example

[ https://issues.apache.org/jira/browse/SPARK-13019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13019: Assignee: (was: Apache Spark) > Replace example code in mllib-statistics.md using

[jira] [Assigned] (SPARK-13019) Replace example code in mllib-statistics.md using include_example

[ https://issues.apache.org/jira/browse/SPARK-13019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13019: Assignee: Apache Spark > Replace example code in mllib-statistics.md using

[jira] [Assigned] (SPARK-13154) Add pydoc lint for docs

[ https://issues.apache.org/jira/browse/SPARK-13154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13154: Assignee: Apache Spark > Add pydoc lint for docs > --- > >

[jira] [Assigned] (SPARK-13154) Add pydoc lint for docs

[ https://issues.apache.org/jira/browse/SPARK-13154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13154: Assignee: (was: Apache Spark) > Add pydoc lint for docs > --- > >

[jira] [Created] (SPARK-13226) MLLib PowerIteration Clustering depends on deprecated KMeans setRuns API

holdenk created SPARK-13226: --- Summary: MLLib PowerIteration Clustering depends on deprecated KMeans setRuns API Key: SPARK-13226 URL: https://issues.apache.org/jira/browse/SPARK-13226 Project: Spark

[jira] [Updated] (SPARK-13201) Make a private non-deprecated version of setRuns

[ https://issues.apache.org/jira/browse/SPARK-13201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-13201: Description: Make a private non-deprecated version of setRuns API so that we can call it from the

[jira] [Comment Edited] (SPARK-13201) Make a private non-deprecated version of setRuns

[ https://issues.apache.org/jira/browse/SPARK-13201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136112#comment-15136112 ] holdenk edited comment on SPARK-13201 at 2/7/16 2:56 AM: - Looking at the only use

[jira] [Commented] (SPARK-13201) Make a private non-deprecated version of setRuns

[ https://issues.apache.org/jira/browse/SPARK-13201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136112#comment-15136112 ] holdenk commented on SPARK-13201: - Looking at the only use inside of the MLLib API, I'm pretty convinced

[jira] [Updated] (SPARK-13201) Make a private non-deprecated version of setRuns

[ https://issues.apache.org/jira/browse/SPARK-13201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-13201: Component/s: (was: ML) > Make a private non-deprecated version of setRuns >

[jira] [Resolved] (SPARK-13103) HashTF dosn't count TF correctly

[ https://issues.apache.org/jira/browse/SPARK-13103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13103. --- Resolution: Cannot Reproduce I can't reproduce this: {code} >>> from pyspark.mllib.feature import

[jira] [Updated] (SPARK-13209) transitive closure on a dataframe

[ https://issues.apache.org/jira/browse/SPARK-13209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Placek updated SPARK-13209: --- Description: When I run the following loop the join gets slower and slower regardless of caching. If I

[jira] [Comment Edited] (SPARK-11472) SparkContext creation error after sc.stop() when Spark is compiled for Hive

[ https://issues.apache.org/jira/browse/SPARK-11472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136127#comment-15136127 ] Atkins edited comment on SPARK-11472 at 2/7/16 4:15 AM: Reproduced in kerberized

[jira] [Commented] (SPARK-12154) Upgrade to Jersey 2

[ https://issues.apache.org/jira/browse/SPARK-12154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136159#comment-15136159 ] Milad Khajavi commented on SPARK-12154: --- Hmm, I can work on this issue, if some expert man give me

[jira] [Assigned] (SPARK-13227) Risky apply() in OpenHashMap

[ https://issues.apache.org/jira/browse/SPARK-13227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13227: Assignee: Apache Spark > Risky apply() in OpenHashMap > > >

[jira] [Commented] (SPARK-13227) Risky apply() in OpenHashMap

[ https://issues.apache.org/jira/browse/SPARK-13227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136160#comment-15136160 ] Apache Spark commented on SPARK-13227: -- User 'CodingCat' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13227) Risky apply() in OpenHashMap

[ https://issues.apache.org/jira/browse/SPARK-13227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13227: Assignee: (was: Apache Spark) > Risky apply() in OpenHashMap >

[jira] [Commented] (SPARK-13154) Add pydoc lint for docs

[ https://issues.apache.org/jira/browse/SPARK-13154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136170#comment-15136170 ] Apache Spark commented on SPARK-13154: -- User 'holdenk' has created a pull request for this issue:

[jira] [Created] (SPARK-13225) [SQL] Support Intersect All/Distinct

Xiao Li created SPARK-13225: --- Summary: [SQL] Support Intersect All/Distinct Key: SPARK-13225 URL: https://issues.apache.org/jira/browse/SPARK-13225 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-13225) [SQL] Support Intersect All/Distinct

[ https://issues.apache.org/jira/browse/SPARK-13225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136093#comment-15136093 ] Apache Spark commented on SPARK-13225: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13225) [SQL] Support Intersect All/Distinct

[ https://issues.apache.org/jira/browse/SPARK-13225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13225: Assignee: Apache Spark > [SQL] Support Intersect All/Distinct >

[jira] [Assigned] (SPARK-13225) [SQL] Support Intersect All/Distinct

[ https://issues.apache.org/jira/browse/SPARK-13225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13225: Assignee: (was: Apache Spark) > [SQL] Support Intersect All/Distinct >

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136132#comment-15136132 ] Xusen Yin commented on SPARK-13178: --- Cheers for the good news! :) > RRDD faces with concurrency issue

[jira] [Updated] (SPARK-5865) Add doc warnings for methods that return local data structures

[ https://issues.apache.org/jira/browse/SPARK-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5865: - Assignee: Tommy Yu (was: Nicholas Chammas) > Add doc warnings for methods that return local data

[jira] [Commented] (SPARK-12469) Consistent Accumulators for Spark

[ https://issues.apache.org/jira/browse/SPARK-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136080#comment-15136080 ] Apache Spark commented on SPARK-12469: -- User 'holdenk' has created a pull request for this issue:

[jira] [Updated] (SPARK-13200) Investigate math.round on integer number in MFDataGenerator.scala:109

[ https://issues.apache.org/jira/browse/SPARK-13200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-13200: Issue Type: Sub-task (was: Improvement) Parent: SPARK-13175 > Investigate math.round on integer

[jira] [Commented] (SPARK-11416) Upgrade kryo package to version 3.0

[ https://issues.apache.org/jira/browse/SPARK-11416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135690#comment-15135690 ] Hitoshi Ozawa commented on SPARK-11416: --- I've used Kryo3Upgrade branch in

[jira] [Assigned] (SPARK-13223) Add stratified sampling to ML feature engineering

[ https://issues.apache.org/jira/browse/SPARK-13223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13223: Assignee: Apache Spark > Add stratified sampling to ML feature engineering >

[jira] [Commented] (SPARK-13223) Add stratified sampling to ML feature engineering

[ https://issues.apache.org/jira/browse/SPARK-13223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135689#comment-15135689 ] Apache Spark commented on SPARK-13223: -- User 'hhbyyh' has created a pull request for this issue:

[jira] [Commented] (SPARK-13171) Update promise & future to Promise and Future as the old ones are deprecated

[ https://issues.apache.org/jira/browse/SPARK-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135784#comment-15135784 ] Ted Yu commented on SPARK-13171: Turns out not that trivial. For build against hadoop 2.4 , some test

[jira] [Comment Edited] (SPARK-11416) Upgrade kryo package to version 3.0

[ https://issues.apache.org/jira/browse/SPARK-11416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135690#comment-15135690 ] Hitoshi Ozawa edited comment on SPARK-11416 at 2/6/16 8:53 AM: --- I've used

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135671#comment-15135671 ] Sun Rui commented on SPARK-13178: - The root cause is that RRDD.compute() uses some instance variables. If