Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21119#discussion_r184839152 --- Diff: python/pyspark/ml/clustering.py --- @@ -1156,6 +1156,204 @@ def getKeepLastCheckpoint(self): return self.getOrDefault(self.keepLastCheckpoint) +@inherit_doc +class PowerIterationClustering(HasMaxIter, HasPredictionCol, JavaTransformer, JavaParams, + JavaMLReadable, JavaMLWritable): + """ + .. note:: Experimental + Power Iteration Clustering (PIC), a scalable graph clustering algorithm developed by + <a href=http://www.icml2010.org/papers/387.pdf>Lin and Cohen</a>. From the abstract: + PIC finds a very low-dimensional embedding of a dataset using truncated power + iteration on a normalized pair-wise similarity matrix of the data. + + PIC takes an affinity matrix between items (or vertices) as input. An affinity matrix + is a symmetric matrix whose entries are non-negative similarities between items. + PIC takes this matrix (or graph) as an adjacency matrix. Specifically, each input row + includes: + + - :py:class:`idCol`: vertex ID + - :py:class:`neighborsCol`: neighbors of vertex in :py:class:`idCol` + - :py:class:`similaritiesCol`: non-negative weights (similarities) of edges between the + vertex in :py:class:`idCol` and each neighbor in :py:class:`neighborsCol` + + PIC returns a cluster assignment for each input vertex. It appends a new column + :py:class:`predictionCol` containing the cluster assignment in :py:class:`[0,k)` for + each row (vertex). + + Notes: + + - [[PowerIterationClustering]] is a transformer with an expensive [[transform]] operation. + Transform runs the iterative PIC algorithm to cluster the whole input dataset. + - Input validation: This validates that similarities are non-negative but does NOT validate + that the input matrix is symmetric. + + @see <a href=http://en.wikipedia.org/wiki/Spectral_clustering> --- End diff -- Use `.. seealso::`?
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org