Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/9338#issuecomment-159428812
Hi @jkbradley - the version numbers look OK - but I've been unable to get
the Scala API docs building in my environment (not just this branch - I get the
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8690#issuecomment-154882353
@yu-iskw Thanks for making the changes - LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8690#issuecomment-150954098
@yu-iskw Almost there - you're just missing the version numbers on
`GBTClassifier` class + methods (they should all be set to `1.4.0`), everything
else looks
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8627#issuecomment-150949269
Test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8627#issuecomment-150025192
@yu-iskw Thanks for testing!
@mengxr I've rebased now - the changes were very minor. Should be good to
merge once the tests complete.
---
If your project i
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8690#issuecomment-150023630
@yu-iskw Thanks for removing those methods - but did you loose some of the
version numbers when rebasing? I.e. ```DecisionTreeClassifier.getImpurity()```
has a
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8684#issuecomment-150018838
I think we should keep the comments - but just tweak them so that they're
valid [reStructuredText](http://sphinx-doc.org/rest.html).
In this case all
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8627#issuecomment-149997391
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user noel-smith closed the pull request at:
https://github.com/apache/spark/pull/8855
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
GitHub user noel-smith opened a pull request:
https://github.com/apache/spark/pull/8855
[Doc][PySpark][MLLib] Added newlines to docstrings to fix parameter
formatting (1.5 backport)
Backport of #8851 for 1.5 branch.
You can merge this pull request into a Git repository by running
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8851#issuecomment-142115598
Sure - should be straightforward, I'll take a look.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
GitHub user noel-smith opened a pull request:
https://github.com/apache/spark/pull/8851
[Doc][PySpark][MLLib] Added newlines to docstrings to fix parameter
formatting
Added newlines before `:param ...:` and `:return:` markup. Without these,
parameter lists aren't form
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8571#issuecomment-141709019
Tweaked JavaScript to make it more robust to Sphinx changes. Set minimal
version of Sphinx to `1.2`.
---
If your project is set up for it, you can reply to this
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8690#issuecomment-141213397
The comments on `MultilayerPerceptronClassifier` +
`MultilayerPerceptronClassifierModel` are good. Just need to remove the four
functions noted above (these were
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8690#discussion_r39791016
--- Diff: python/pyspark/ml/classification.py ---
@@ -116,6 +120,37 @@ def setParams(self, featuresCol="features",
labelCol="label"
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8627#issuecomment-140899724
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8695#discussion_r39686456
--- Diff: python/pyspark/ml/util.py ---
@@ -36,6 +39,8 @@ def wrapper(*args, **kwargs):
class Identifiable(object):
"""
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8694#issuecomment-140896835
Just need to replace `.. addedversion::` with `.. versionadded::` and
add a version number to `ParamGridBuilder.build()`.
---
If your project is set up for
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8693#discussion_r39684621
--- Diff: python/pyspark/ml/regression.py ---
@@ -147,6 +159,8 @@ class TreeRegressorParams(object):
class RandomForestParams(object
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8693#discussion_r39684568
--- Diff: python/pyspark/ml/regression.py ---
@@ -154,6 +168,8 @@ class RandomForestParams(object):
class GBTParams(object
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8693#discussion_r39684460
--- Diff: python/pyspark/ml/regression.py ---
@@ -140,6 +150,8 @@ def intercept(self):
class TreeRegressorParams(object
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8693#issuecomment-140892197
Replace `.. addedversion::` with `.. versionadded::`.
Merge from master to remove `setElasticNetParam` and `getElasticNetParam`
functions.
---
If your
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8692#issuecomment-140889476
Need to replace `.. addedversion::` with `.. versionadded::` in the class
docstrings.
---
If your project is set up for it, you can reply to this email and have
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8691#issuecomment-140870734
No problems, LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8626#issuecomment-140868760
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8690#issuecomment-140856366
Some functions have been removed since this PR was created + there are some
new classes (`MultilayerPerceptronClassifier`) without version numbers.
@mengxr
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8690#issuecomment-140853730
Should replace `.. addedversion::` with `.. versionadded::` in all the
class docstrings.
---
If your project is set up for it, you can reply to this email and have
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8689#issuecomment-140836995
That's works too, thanks for resolving - LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8685#issuecomment-140835549
Thanks for the changes - LGTM!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8684#issuecomment-140833937
That's great - LGTM!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8689#issuecomment-140536184
These changes are already included in #8685. Can probably drop this PR and
merge SPARK-10279 with SPARK-10278.
---
If your project is set up for it, you can reply
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8685#issuecomment-140533249
Just minor fixes needed - change `addedversion` to `versionadded` + one
version number alteration.
---
If your project is set up for it, you can reply to this email
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8685#discussion_r39562007
--- Diff: python/pyspark/mllib/util.py ---
@@ -235,6 +248,8 @@ def save(self, sc, path):
class Loader(object):
"""
Mi
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8685#discussion_r39562037
--- Diff: python/pyspark/mllib/util.py ---
@@ -280,15 +297,21 @@ def _load_java(cls, sc, path):
return java_obj.load(sc._jsc.sc(), path
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8685#discussion_r39562027
--- Diff: python/pyspark/mllib/util.py ---
@@ -256,6 +271,8 @@ class JavaLoader(Loader):
"""
Mixin for classes which
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8685#discussion_r39561998
--- Diff: python/pyspark/mllib/util.py ---
@@ -222,9 +231,13 @@ class JavaSaveable(Saveable):
"""
Mixin for models th
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8685#discussion_r39561981
--- Diff: python/pyspark/mllib/util.py ---
@@ -197,6 +204,8 @@ def loadVectors(sc, path):
class Saveable(object):
"""
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8685#discussion_r39561958
--- Diff: python/pyspark/mllib/util.py ---
@@ -32,6 +32,8 @@ class MLUtils(object):
"""
Helper methods to loa
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8685#discussion_r39561631
--- Diff: python/pyspark/mllib/tree.py ---
@@ -431,6 +461,8 @@ class GradientBoostedTrees(object):
Learning algorithm for a gradient boosted
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8685#discussion_r39561561
--- Diff: python/pyspark/mllib/tree.py ---
@@ -418,6 +446,8 @@ class GradientBoostedTreesModel(TreeEnsembleModel,
JavaLoader):
.. note
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8685#discussion_r39561531
--- Diff: python/pyspark/mllib/tree.py ---
@@ -252,6 +276,8 @@ class RandomForest(object):
Learning algorithm for a random forest model for
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8685#discussion_r39561422
--- Diff: python/pyspark/mllib/tree.py ---
@@ -30,6 +30,11 @@
class TreeEnsembleModel(JavaModelWrapper, JavaSaveable
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8685#discussion_r39561444
--- Diff: python/pyspark/mllib/tree.py ---
@@ -72,7 +80,10 @@ class DecisionTreeModel(JavaModelWrapper, JavaSaveable,
JavaLoader):
.. note
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8685#discussion_r39561473
--- Diff: python/pyspark/mllib/tree.py ---
@@ -115,6 +133,8 @@ class DecisionTree(object):
Learning algorithm for a decision tree model for
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8685#discussion_r39561501
--- Diff: python/pyspark/mllib/tree.py ---
@@ -239,6 +261,8 @@ class RandomForestModel(TreeEnsembleModel, JavaLoader):
.. note:: Experimental
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8685#discussion_r39560663
--- Diff: python/pyspark/mllib/tree.py ---
@@ -90,16 +101,23 @@ def predict(self, x):
else:
return self.call("pr
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8684#issuecomment-140524867
Looks good - just need to replace `addedversion` with `versionadded` in the
class docstrings.
---
If your project is set up for it, you can reply to this email and
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8684#discussion_r39558433
--- Diff: python/pyspark/mllib/regression.py ---
@@ -640,6 +736,8 @@ class
StreamingLinearRegressionWithSGD(StreamingLinearAlgorithm):
:param
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8684#discussion_r39558391
--- Diff: python/pyspark/mllib/regression.py ---
@@ -571,8 +641,29 @@ def load(cls, sc, path):
class IsotonicRegression(object
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8684#discussion_r39558375
--- Diff: python/pyspark/mllib/regression.py ---
@@ -523,6 +586,8 @@ class IsotonicRegressionModel(Saveable, Loader):
... rmtree(path
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8684#discussion_r39558409
--- Diff: python/pyspark/mllib/regression.py ---
@@ -590,10 +681,13 @@ class StreamingLinearAlgorithm(object):
Base class that has to be inherited
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8684#discussion_r39558359
--- Diff: python/pyspark/mllib/regression.py ---
@@ -445,8 +497,19 @@ def load(cls, sc, path):
class RidgeRegressionWithSGD(object
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8684#discussion_r39558311
--- Diff: python/pyspark/mllib/regression.py ---
@@ -326,8 +361,19 @@ def load(cls, sc, path):
class LassoWithSGD(object
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8684#discussion_r39558335
--- Diff: python/pyspark/mllib/regression.py ---
@@ -428,14 +474,20 @@ class RidgeRegressionModel(LinearRegressionModelBase):
True
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8684#discussion_r39558268
--- Diff: python/pyspark/mllib/regression.py ---
@@ -198,8 +215,20 @@ def _regression_train_wrapper(train_func, modelClass,
data, initial_weights
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8684#discussion_r39558296
--- Diff: python/pyspark/mllib/regression.py ---
@@ -309,14 +338,20 @@ class LassoModel(LinearRegressionModelBase):
True
>>
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8684#discussion_r39558247
--- Diff: python/pyspark/mllib/regression.py ---
@@ -162,14 +173,20 @@ class
LinearRegressionModel(LinearRegressionModelBase):
True
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8684#discussion_r39552511
--- Diff: python/pyspark/mllib/regression.py ---
@@ -93,8 +101,11 @@ class LinearRegressionModelBase(LinearModel):
True
>>
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8684#discussion_r39552470
--- Diff: python/pyspark/mllib/regression.py ---
@@ -65,6 +67,8 @@ class LinearModel(object):
:param weights: Weights computed for every
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8684#discussion_r39552087
--- Diff: python/pyspark/mllib/regression.py ---
@@ -42,6 +42,8 @@ class LabeledPoint(object):
column matrix)
Note: '
GitHub user noel-smith opened a pull request:
https://github.com/apache/spark/pull/8773
[PySpark][MLlib][Docs] Replaced addversion with versionadded in mllib.random
Missed this when reviewing `pyspark.mllib.random` for SPARK-10275.
You can merge this pull request into a Git
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8677#discussion_r39547963
--- Diff: python/pyspark/mllib/recommendation.py ---
@@ -36,6 +36,8 @@ class Rating(namedtuple("Rating", ["user", "product"
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8677#discussion_r39548087
--- Diff: python/pyspark/mllib/recommendation.py ---
@@ -157,17 +167,25 @@ def recommendProducts(self, user, num):
return list(self.call
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8677#discussion_r39547983
--- Diff: python/pyspark/mllib/recommendation.py ---
@@ -111,13 +113,17 @@ class MatrixFactorizationModel(JavaModelWrapper,
JavaSaveable, JavaLoader
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8677#issuecomment-140230088
LGTM apart from the one minor issue above.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8627#discussion_r39455034
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -129,20 +135,32 @@ def computeCost(self, rdd):
[_convert_to_vector(c
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8627#discussion_r39453363
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -129,20 +135,32 @@ def computeCost(self, rdd):
[_convert_to_vector(c
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8677#discussion_r39452996
--- Diff: python/pyspark/mllib/recommendation.py ---
@@ -157,17 +167,25 @@ def recommendProducts(self, user, num):
return list(self.call
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8666#issuecomment-140215395
All LGTM.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8665#issuecomment-140199152
Sounds good - thanks for confirming - I'll reinstate the thre-part version
numbers in my PRs.
---
If your project is set up for it, you can reply to this emai
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8665#issuecomment-140185860
@mengxr @davies Just to confirm, before @yu-iskw and I make the changes -
we want to stick with the two-part version numbers (`@since(1.4)`) used in
`pyspark.sql
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8665#discussion_r39436349
--- Diff: python/pyspark/mllib/fpm.py ---
@@ -58,6 +61,7 @@ class FPGrowth(object):
"""
@classmethod
+
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8665#discussion_r39436251
--- Diff: python/pyspark/mllib/fpm.py ---
@@ -41,8 +41,11 @@ class FPGrowthModel(JavaModelWrapper):
>>> model = FPGrowth.train(rd
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8665#discussion_r39435855
--- Diff: python/pyspark/mllib/fpm.py ---
@@ -41,8 +41,11 @@ class FPGrowthModel(JavaModelWrapper):
>>> model = FPGrowth.train(rd
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8633#issuecomment-140179397
@mengxr @yu-iskw - Sounds like a plan - I'll take a look.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitH
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8571#issuecomment-139411836
Nice - thanks for finding that!
It probably indicates I need to make the JS logic a bit more robust though.
I'll take a look at the Sphinx history and
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8571#issuecomment-139404962
@davies Could you let me know your browser/platform + whether the example
[link](https://dl.dropboxusercontent.com/u/20821334/pyspark-api-nav-enhance
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8571#issuecomment-139387988
Hmm that's odd - that's exactly the way I'm building too. Did your
generated html pages include the links to the `pyspark.js` and `pyspark.css`
files
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8633#discussion_r39096533
--- Diff: python/pyspark/mllib/feature.py ---
@@ -84,11 +84,14 @@ class Normalizer(VectorTransformer):
>>> nor2 = Normalizer(f
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8623#issuecomment-138812209
I also added a related PR https://github.com/apache/spark/pull/8571 about
highlighting experimental features the API docs. Would be useful to get a
second opinion on
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8657#discussion_r38992400
--- Diff: python/pyspark/__init__.py ---
@@ -48,6 +48,22 @@
from pyspark.status import *
from pyspark.profiler import Profiler, BasicProfiler
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8657#discussion_r38990876
--- Diff: python/pyspark/__init__.py ---
@@ -48,6 +48,22 @@
from pyspark.status import *
from pyspark.profiler import Profiler, BasicProfiler
Github user noel-smith commented on a diff in the pull request:
https://github.com/apache/spark/pull/8633#discussion_r38990506
--- Diff: python/pyspark/__init__.py ---
@@ -51,6 +51,26 @@
# for back compatibility
from pyspark.sql import SQLContext, HiveContext, SchemaRDD
GitHub user noel-smith opened a pull request:
https://github.com/apache/spark/pull/8633
[SPARK-10273] Add @since annotation to pyspark.mllib.feature
Duplicated the @since decorator from pyspark.sql into pyspark (also tweaked
to handle functions without docstrings).
Added
GitHub user noel-smith opened a pull request:
https://github.com/apache/spark/pull/8628
[SPARK-10272] Added @since tags to pyspark.mllib.evaluation
Duplicated the @since decorator from pyspark.sql into pyspark (also tweaked
to handle functions without docstrings).
Added
GitHub user noel-smith opened a pull request:
https://github.com/apache/spark/pull/8627
[SPARK-10271] Added @since tags to pyspark.mllib.clustering
Duplicated the @since decorator from pyspark.sql into pyspark (also tweaked
to handle functions without docstrings).
Added
GitHub user noel-smith reopened a pull request:
https://github.com/apache/spark/pull/8626
Add @since annotation to pyspark.mllib.classification
Duplicated the @since decorator from pyspark.sql into pyspark (also tweaked
to handle functions without docstrings).
Added @since
Github user noel-smith closed the pull request at:
https://github.com/apache/spark/pull/8626
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
GitHub user noel-smith opened a pull request:
https://github.com/apache/spark/pull/8626
Add @since annotation to pyspark.mllib.classification
Duplicated the @since decorator from pyspark.sql into pyspark (also tweaked
to handle functions without docstrings).
Added @since
GitHub user noel-smith opened a pull request:
https://github.com/apache/spark/pull/8623
[SPARK-10094] Pyspark ML Feature transformers marked as experimental
Modified class-level docstrings to mark all feature transformers in
pyspark.ml as experimental.
You can merge this pull
GitHub user noel-smith opened a pull request:
https://github.com/apache/spark/pull/8571
[SPARK-10415][PySpark] Enhance Navigation Sidebar in PySpark API
These are CSS/JavaScript changes to add classes/functions + a few other
tweaks to make navigation in the PySpark API a bit
Github user noel-smith commented on the pull request:
https://github.com/apache/spark/pull/8399#issuecomment-135655500
That would be great - I've just messaged him. If there are any other
changes you need to get this into 1.5 I'll get them in ASAP today.
---
If your proj
GitHub user noel-smith opened a pull request:
https://github.com/apache/spark/pull/8399
[SPARK-10188] [Pyspark] Pyspark CrossValidator with RMSE selects incorrect
model
* Added isLargerBetter() method to Pyspark Evaluator to match the Scala
version.
* JavaEvaluator delegates
93 matches
Mail list logo
