Hi Jake, i organized things to be a little better/neater/nicer/cleaner
can you take from https://github.com/smarttypes/graphreduce $ python graphreduce_tests.py will reproduce the error On Fri, Dec 9, 2011 at 5:54 PM, Jacob VanderPlas <[email protected]> wrote: > Thanks Timmy, > If you could also include in the gist a minimal python script which > reproduces the error from the included data, that would be very helpful. > Jake > > Timmy Wilson wrote: >> Thanks Jake >> >> here are the files -- https://gist.github.com/1453617 >> >> here's the code creating the input data: >> >> https://github.com/smarttypes/SmartTypes/blob/master/smarttypes/model/twitter_user.py >> >> - def get_adjacency_matrix(self, distance=10): >> >> >> >> On Fri, Dec 9, 2011 at 5:17 PM, Jacob VanderPlas >> <[email protected]> wrote: >> >>> If you have a github account, you could create a gist: >>> https://gist.github.com/ >>> This results in a sort of temporary git repository you can push files to >>> as with any git repository. >>> If you push your data file and python code you use to process and plot >>> it, I can take a look and see if I can figure out what the problem is. >>> Alternatively, you could just push the plot there, but it would be >>> harder for me to experiment with. Thanks! >>> Jake >>> >>> Timmy Wilson wrote: >>> >>>> This was rejected because attachments were > 40K >>>> >>>> What's the preferred way to pass random/tmp files? >>>> >>>> >>>> ---------- Forwarded message ---------- >>>> From: Timmy Wilson <[email protected]> >>>> Date: Fri, Dec 9, 2011 at 8:57 AM >>>> Subject: Re: [Scikit-learn-general] RuntimeError: Factor is exactly >>>> singular >>>> To: [email protected] >>>> >>>> >>>> the eigen_solver='dense' option eliminates the error, but the results >>>> look different >>>> >>>> i attached the offending input (an adjacency matrix from twitter), and >>>> the results of hessian w/ the dense option and standard w/ 30 >>>> neighbors >>>> >>>> >>>> On Thu, Dec 8, 2011 at 11:19 PM, Jacob VanderPlas >>>> <[email protected]> wrote: >>>> >>>> >>>>> Timmy, >>>>> Interesting... this means that none of the earlier ideas were on the >>>>> right track, or else we'd see the same error for standard and modified. >>>>> It turns out that Hessian and LTSA do very similar things. I still >>>>> haven't wrapped my mind around it intuitively, but what you're seeing >>>>> probably means that the hessian estimator is identical in two >>>>> neighborhoods. I'm confused by the fact that adding noise and changing >>>>> the number of neighbors is not affecting things... >>>>> >>>>> Can you try using eigen_solver='dense' (that should get around the >>>>> problem, though it will be slow) and taking a look at the results to see >>>>> if they're anywhere close to what standard and modified LLE give you? >>>>> Jake >>>>> >>>>> Timmy Wilson wrote: >>>>> >>>>> >>>>>> 'standard' + 'modified' both work fine >>>>>> >>>>>> 'hessian' + 'ltsa' both have issues >>>>>> >>>>>> ltsa is printing: >>>>>> >>>>>> RuntimeWarning: Diagonal number 2 is exactly zero. Singular matrix. >>>>>> >>>>>> and then setting everything to -nan >>>>>> >>>>>> i tried adding random noise and increasing n_neighbors -- but no dice >>>>>> >>>>>> Fabian's suggestion is still under review -- >>>>>> https://github.com/scikit-learn/scikit-learn/pull/457 >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Dec 8, 2011 at 11:39 AM, Fabian Pedregosa >>>>>> <[email protected]> wrote: >>>>>> >>>>>> >>>>>> >>>>>>> I reduced the problem to a difference of results with eigsh when >>>>>>> working in shift-invert mode and when not. Notice that the following >>>>>>> works with sigma=None but not with sigma=0. , thus seems to me that we >>>>>>> should fall back to non invert-mode for singular matrices (see >>>>>>> asociated pull request). >>>>>>> >>>>>>> ``` >>>>>>> In [8]: M = sparse.csr_matrix([[1, 1, 1], [1, 1, 1], [1, 1, 1.]]) >>>>>>> >>>>>>> In [9]: linalg.eigsh(M, 1) >>>>>>> Out[9]: >>>>>>> (array([ 3.]), >>>>>>> array([[-0.57735027], >>>>>>> [-0.57735027], >>>>>>> [-0.57735027]])) >>>>>>> >>>>>>> In [10]: linalg.eigsh(M, 1, sigma=0.) >>>>>>> --------------------------------------------------------------------------- >>>>>>> RuntimeError Traceback (most recent call >>>>>>> last) >>>>>>> /home/fabian/dev/sandbox/<ipython-input-10-f646f5be93ec> in <module>() >>>>>>> ----> 1 linalg.eigsh(M, 1, sigma=0.) >>>>>>> >>>>>>> /home/fabian/envs/p26/lib/python2.6/site-packages/scipy/sparse/linalg/eigen/arpack/arpack.pyc >>>>>>> in eigsh(A, k, M, sigma, which, v0, ncv, maxiter, tol, >>>>>>> return_eigenvectors, Minv, OPinv, mode) >>>>>>> 1484 if OPinv is None: >>>>>>> 1485 Minv_matvec = get_OPinv_matvec(A, M, sigma, >>>>>>> -> 1486 symmetric=True, >>>>>>> tol=tol) >>>>>>> 1487 else: >>>>>>> 1488 OPinv = _aslinearoperator_with_dtype(OPinv) >>>>>>> >>>>>>> /home/fabian/envs/p26/lib/python2.6/site-packages/scipy/sparse/linalg/eigen/arpack/arpack.pyc >>>>>>> in get_OPinv_matvec(A, M, sigma, symmetric, tol) >>>>>>> 1004 def get_OPinv_matvec(A, M, sigma, symmetric=False, tol=0): >>>>>>> 1005 if sigma == 0: >>>>>>> -> 1006 return get_inv_matvec(A, symmetric=symmetric, tol=tol) >>>>>>> 1007 >>>>>>> 1008 if M is None: >>>>>>> >>>>>>> /home/fabian/envs/p26/lib/python2.6/site-packages/scipy/sparse/linalg/eigen/arpack/arpack.pyc >>>>>>> in get_inv_matvec(M, symmetric, tol) >>>>>>> 997 if isspmatrix_csr(M) and symmetric: >>>>>>> 998 M = M.T >>>>>>> --> 999 return SpLuInv(M).matvec >>>>>>> 1000 else: >>>>>>> 1001 return IterInv(M, tol=tol).matvec >>>>>>> >>>>>>> /home/fabian/envs/p26/lib/python2.6/site-packages/scipy/sparse/linalg/eigen/arpack/arpack.pyc >>>>>>> in __init__(self, M) >>>>>>> 890 """ >>>>>>> 891 def __init__(self, M): >>>>>>> --> 892 self.M_lu = splu(M) >>>>>>> 893 LinearOperator.__init__(self, M.shape, self._matvec, >>>>>>> dtype=M.dtype) >>>>>>> 894 self.isreal = not np.issubdtype(self.dtype, >>>>>>> np.complexfloating) >>>>>>> >>>>>>> /home/fabian/envs/p26/lib/python2.6/site-packages/scipy/sparse/linalg/dsolve/linsolve.pyc >>>>>>> in splu(A, permc_spec, diag_pivot_thresh, drop_tol, relax, panel_size, >>>>>>> options) >>>>>>> 171 _options.update(options) >>>>>>> 172 return _superlu.gstrf(N, A.nnz, A.data, A.indices, A.indptr, >>>>>>> --> 173 ilu=False, options=_options) >>>>>>> 174 >>>>>>> 175 def spilu(A, drop_tol=None, fill_factor=None, drop_rule=None, >>>>>>> permc_spec=None, >>>>>>> >>>>>>> RuntimeError: Factor is exactly singular >>>>>>> >>>>>>> In [11]: >>>>>>> ``` >>>>>>> >>>>>>> ------------------------------------------------------------------------------ >>>>>>> Cloud Services Checklist: Pricing and Packaging Optimization >>>>>>> This white paper is intended to serve as a reference, checklist and >>>>>>> point of >>>>>>> discussion for anyone considering optimizing the pricing and packaging >>>>>>> model >>>>>>> of a cloud services business. Read Now! >>>>>>> http://www.accelacomm.com/jaw/sfnl/114/51491232/ >>>>>>> _______________________________________________ >>>>>>> Scikit-learn-general mailing list >>>>>>> [email protected] >>>>>>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>>>>>> >>>>>>> >>>>>>> >>>>>> ------------------------------------------------------------------------------ >>>>>> Cloud Services Checklist: Pricing and Packaging Optimization >>>>>> This white paper is intended to serve as a reference, checklist and >>>>>> point of >>>>>> discussion for anyone considering optimizing the pricing and packaging >>>>>> model >>>>>> of a cloud services business. Read Now! >>>>>> http://www.accelacomm.com/jaw/sfnl/114/51491232/ >>>>>> _______________________________________________ >>>>>> Scikit-learn-general mailing list >>>>>> [email protected] >>>>>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>>>>> >>>>>> >>>>>> >>>>> ------------------------------------------------------------------------------ >>>>> Cloud Services Checklist: Pricing and Packaging Optimization >>>>> This white paper is intended to serve as a reference, checklist and point >>>>> of >>>>> discussion for anyone considering optimizing the pricing and packaging >>>>> model >>>>> of a cloud services business. Read Now! >>>>> http://www.accelacomm.com/jaw/sfnl/114/51491232/ >>>>> _______________________________________________ >>>>> Scikit-learn-general mailing list >>>>> [email protected] >>>>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>>>> >>>>> >>>> ------------------------------------------------------------------------------ >>>> Cloud Services Checklist: Pricing and Packaging Optimization >>>> This white paper is intended to serve as a reference, checklist and point >>>> of >>>> discussion for anyone considering optimizing the pricing and packaging >>>> model >>>> of a cloud services business. Read Now! >>>> http://www.accelacomm.com/jaw/sfnl/114/51491232/ >>>> _______________________________________________ >>>> Scikit-learn-general mailing list >>>> [email protected] >>>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>>> >>>> >>> ------------------------------------------------------------------------------ >>> Cloud Services Checklist: Pricing and Packaging Optimization >>> This white paper is intended to serve as a reference, checklist and point of >>> discussion for anyone considering optimizing the pricing and packaging model >>> of a cloud services business. Read Now! >>> http://www.accelacomm.com/jaw/sfnl/114/51491232/ >>> _______________________________________________ >>> Scikit-learn-general mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>> >> >> ------------------------------------------------------------------------------ >> Cloud Services Checklist: Pricing and Packaging Optimization >> This white paper is intended to serve as a reference, checklist and point of >> discussion for anyone considering optimizing the pricing and packaging model >> of a cloud services business. Read Now! >> http://www.accelacomm.com/jaw/sfnl/114/51491232/ >> _______________________________________________ >> Scikit-learn-general mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >> > > ------------------------------------------------------------------------------ > Cloud Services Checklist: Pricing and Packaging Optimization > This white paper is intended to serve as a reference, checklist and point of > discussion for anyone considering optimizing the pricing and packaging model > of a cloud services business. Read Now! > http://www.accelacomm.com/jaw/sfnl/114/51491232/ > _______________________________________________ > Scikit-learn-general mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general ------------------------------------------------------------------------------ Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
