[Numpy-discussion] MaskedArray __setitem__ Performance

[Alexander Michael]

In part of some code I'm rewriting from carrying around a data and
mask array to using MaskedArray, I read data into an array from an
input stream. By its nature this a "one at a time" process, so it is
basically a loop over assigning single elements (in no predetermined
order) of already allocated arrays. Unfortunately, using MaskedArray
in this way is significantly slower. The sample code below
demonstrates that for this particular procedure, filling the
MaskedArray is 32x slower than working with the two arrays I had been
carying around. It appears that I can regain the fill performance by
working on _data and _mask directly. I can guarantee that the
MaskedArrays I'm working with have been created with a dense mask as
I've done below (there are always some masked elements, so there is no
gain in shrinking to nomask). Is this safe? If not, can I make it safe
for this particular performance critical section? I'm assuming that
doing array operations won't incur this sort of penalty when I get
further into my translation. Some overhead is acceptable for the
convenience of not dragging around the mask and thinking about it all
of the time, but hopefully less than 2x slower.

Thanks!
Alex

import numpy

def get_ndarrays():
return (numpy.zeros((5000,500), dtype=float),
numpy.ones((5000,500), dtype=bool))

import timeit

t_base = timeit.Timer(
'a[0,0] = 1.0; m[0,0] = False', 'from __main__ import
get_ndarrays; a,m = get_ndarrays()'
).timeit(1000)/1000
print t_base

6.97574691756e-007

import numpy.ma

def get_maskedarray():
return numpy.ma.MaskedArray(
numpy.zeros((5000,500), dtype=float),
numpy.ones((5000,500), dtype=bool)
)

t_ma = timeit.Timer(
'a[0,0] = 1.0', 'from __main__ import get_maskedarray; a =
get_maskedarray()'
).timeit(1000)/1000
print t_ma, t_ma/t_base

2.26880790715e-005 32.5242290749

t_ma_com = timeit.Timer(
'd[0,0] = 1.0; m[0,0] = False', 'from __main__ import
get_maskedarray, get_setter; a = get_maskedarray(); d,m =
a._data,a._mask'
).timeit(1000)/1000
print t_ma_com, t_ma_com/t_base

7.34450886914e-007 1.05286343612
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] test failures when numpy built without atlas libraries.

[Charles R Harris]

When built without ATLAS, the following tests fail :

==
ERROR: check_testUfuncRegression (numpy.core.tests.test_ma.test_ufuncs)
--
Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/numpy/core/tests/test_ma.py", line
691, in check_testUfuncRegression
self.failUnless(eq(ur.filled(0), mr.filled(0), f))
  File "/usr/lib/python2.5/site-packages/numpy/ma/core.py", line 1552, in
filled
result = self._data.copy()
  File "/usr/lib/python2.5/site-packages/numpy/ma/core.py", line 1474, in
_get_data
return self.view(self._baseclass)
TypeError: Cannot change data-type for object array.

==
FAIL: Ticket #588
--
Traceback (most recent call last):
  File
"/usr/lib/python2.5/site-packages/numpy/core/tests/test_regression.py", line
734, in check_dot_negative_stride
assert_equal(np.dot(x,z),np.dot(x,y2))
  File "/usr/lib/python2.5/site-packages/numpy/testing/utils.py", line 143,
in assert_equal
return assert_array_equal(actual, desired, err_msg)
  File "/usr/lib/python2.5/site-packages/numpy/testing/utils.py", line 225,
in assert_array_equal
verbose=verbose, header='Arrays are not equal')
  File "/usr/lib/python2.5/site-packages/numpy/testing/utils.py", line 217,
in assert_array_compare
assert cond, msg
AssertionError:
Arrays are not equal

(mismatch 100.0%)
 x: array([[ 55924.]])
 y: array([[ 64.]])


When built with ATLAS the second test passes, but the
check_testUfuncRegression error persists. This is at revision 4807.

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] import issue with new Python

[Alan G Isaac]

On Fri, 15 Feb 2008, Dinesh B Vadhia apparently wrote:
> I upgraded to Python 2.5.2c1 today, and got the following error for:
>> import numpy 
>> import scipy 
> Traceback (most recent call last):
>   File "C:\ ... .py", line 19, in 
> import scipy
> ImportError: No module named scipy 
> I'm using Numpy 1.0.4 and Scipy 0.6. 
> Any ideas? 

If you are an Windows, check Lib\site-packages
and see if there are there: looks like not.

If you did a side-by-side install, you'll have
to reinstall SciPy and NumPy on your new Python.

Cheers,
Alan Isaac



___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] import issue with new Python

[Dinesh B Vadhia]

I upgraded to Python 2.5.2c1 today, and got the following error for:

> import numpy
> import scipy

Traceback (most recent call last):
  File "C:\ ... .py", line 19, in 
import scipy
ImportError: No module named scipy

I'm using Numpy 1.0.4 and Scipy 0.6.

Any ideas?

Dinesh

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] MaskedArray __setitem__ for Record Values

[Pierre GM]

On Friday 15 February 2008 14:41:27 Alexander Michael wrote:
> Even better- thanks!

You're welcome.

> >  Mmh. Where should I commit the fix ? Directly to the trunk ?

Done.
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] MaskedArray __setitem__ for Record Values

[Alexander Michael]

On Fri, Feb 15, 2008 at 12:59 PM, Pierre GM <[EMAIL PROTECTED]> wrote:
>  Good call.
>  The easiest is still to replace the line 1361 with:
>  dval = narray(value, copy=False, dtype=self.dtype)
>
>  The problem with the initial method was that the tuple got transformed into a
>  (2,) array whose type could not be changed afterwards. With the new line, we
>  directly transform value to a ndarray of the proper type.

Even better- thanks!

>  Mmh. Where should I commit the fix ? Directly to the trunk ?

I hope so!

Regards,
Alex
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] List Array References?

[Alexander Michael]

On Fri, Feb 15, 2008 at 11:51 AM, Stefan van der Walt <[EMAIL PROTECTED]> wrote:
>  Numpy does complain if you attempt to resize an array with views on
>  it:
>
>   In [8]: x = np.array([1,2,3])
>
>   In [14]: y = x[::-1]
>
>   In [18]: x.resize((4,))
>   ---
>   ValueErrorTraceback (most recent call last)
>
>   /tmp/ in ()
>
>   ValueError: cannot resize an array that has been referenced or is
>   referencing another array in this way.  Use the resize function
>
>  You can catch that exception and work from there.  I hope that is what
>  you had in mind?

Actually, I'm essentially trying to figure out how situations like
that are arising. I'm not
using resize, because it reshapes the array when adding to any
dimension other than
the first. At "the end of the day" I want to enlarge an array without
leaving any
"references" dangling.

Thanks for the suggestion!
Alex
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] MaskedArray __setitem__ for Record Values

[Pierre GM]

On Friday 15 February 2008 10:51:04 Alexander Michael wrote:
> >>> d = numpy.empty((5,4), dtype=[('value', float),('source', int)])
> >>> a = ma.MaskedArray(d, mask=True, fill_value=(0.0,0))
> >>> a[0,0] = (10.0, 1)
>
> numpy\ma\core.py in __setitem__(self, indx, value)
>1359 return
>1360 #
> -> 1361 dval = getdata(value).astype(self.dtype)
>1362 valmask = getmask(value)
>1363 if self._mask is nomask:
>
> TypeError: expected a readable buffer object

Good call.
The easiest is still to replace the line 1361 with:
dval = narray(value, copy=False, dtype=self.dtype)

The problem with the initial method was that the tuple got transformed into a 
(2,) array whose type could not be changed afterwards. With the new line, we 
directly transform value to a ndarray of the proper type.

Mmh. Where should I commit the fix ? Directly to the trunk ?
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] resizing arrays

[Charles R Harris]

On Fri, Feb 15, 2008 at 6:49 AM, Neal Becker <[EMAIL PROTECTED]> wrote:

> I have a situation where I'm going to save some data sets to plot, but I
> don't know a-priori how many sets there will be.  I'm using this code:
>
> try:
>  shape = list(phase_plots.shape)
>  shape[0] += 1
>  phase_plots.resize (shape, refcheck=0)
> except NameError:
>  phase_plots = empty ((1, 2*iterations+1, l))
>
> This works, I'm just wondering if this is a reasonable approach or if
> maybe
> something else would be better (or more efficient).
>
> I'd just append the data to a list. Arrays aren't meant to be dynamic and
what you end up with is a bunch of reallocations and copies.

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] List Array References?

[Stefan van der Walt]

On Fri, Feb 15, 2008 at 08:28:08AM -0500, Alexander Michael wrote:
> On Fri, Feb 15, 2008 at 7:12 AM, Stefan van der Walt <[EMAIL PROTECTED]> 
> wrote:
> >  As far as I know, the reference count to the array is increased when
> >  you create a view, but the views themselves are not tracked anywhere.
> >  You can therefore say whether there are references around, but you
> >  cannot identify the Python objects.
> 
> I would like to occasionally dynamically grow an array (i.e. add
> length to existing dimensions) as I do not know th ultimately required
> length beforehand, but I have a good guess so I won't be need to
> reallocate that often if at all. The only way I know how to do this is
> numpy is to create a new larger array with the new dimensions and copy
> the existing data from the smaller array into it. Perhaps there is a
> better way that doesn't invalidate views on the array? I want to make
> sure that there are no outstanding references to the old array (i.e.
> views on it, etc.) so that I can raise a helpful exception while
> developing.

Numpy does complain if you attempt to resize an array with views on
it:

  In [8]: x = np.array([1,2,3])

  In [14]: y = x[::-1]

  In [18]: x.resize((4,))
  ---
  ValueErrorTraceback (most recent call last)

  /tmp/ in ()

  ValueError: cannot resize an array that has been referenced or is
  referencing another array in this way.  Use the resize function

You can catch that exception and work from there.  I hope that is what
you had in mind?

Regards
Stéfan
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Is anyone knowledgeable about dll deployment on windows ?

[David Cournapeau]

On Sat, Feb 16, 2008 at 12:43 AM, Christian Heimes <[EMAIL PROTECTED]> wrote:
>
>  The MSDN website has some articles about SxS DLLs though. I had to read
>  about ten articles to get the big picture. The information is scattered
>  all over the place. :/

10 pages, xml files and not even compatible across OS versions: it it
what is meant by MS when they talk about rich API ? :) I won't read
those pages to understand something which is available on unix for 20
years and understandable in 5 minutes. I wasted one half day, I won't
waste more than that. Well, this is it, I give up, I won't support
linking dll with MS compilers. I don't have easy access to windows,
and I don't even use this crap.

Thanks for your help, though

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] MaskedArray __setitem__ for Record Values

[Alexander Michael]

I am attempting to work with field dtypes (i.e. record arrays) as
MaskedArrays. I don't want to use MaskedRecords because I want the
mask to apply to the whole record since the first field is primary and
the rest are "tag along" values associated with the primary data
value. The numpy.ndarray supports the ability to set a record with a
(heterogenous) sequence, but MaskedArray does not:

>>> import numpy
>>> from numpy import ma

>>> d = numpy.empty((5,4), dtype=[('value', float),('source', int)])
>>> a = ma.MaskedArray(d, mask=True, fill_value=(0.0,0))
>>> a[0,0] = (10.0, 1)

numpy\ma\core.py in __setitem__(self, indx, value)
   1359 return
   1360 #
-> 1361 dval = getdata(value).astype(self.dtype)
   1362 valmask = getmask(value)
   1363 if self._mask is nomask:

TypeError: expected a readable buffer object

I suggest replacing " dval = getdata(value).astype(self.dtype)" with
something along the lines of:

dval = None
if self.dtype.fields is None:
dval = getdata(value).astype(self.dtype)
else:
dval = value

but I do not understand all of the activity in the __setitem__ method,
so this may be too naive
a fix (but it *appears* to work).

Thanks,
Alex
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Is anyone knowledgeable about dll deployment on windows ?

[Christian Heimes]

David Cournapeau wrote:
> Do you have a link to the related python ML discussion by any chance ?

No, I'm sorry. It was a private chat between between Guido, Martin and
me during the release phase of Python 3.0a2.

The MSDN website has some articles about SxS DLLs though. I had to read
about ten articles to get the big picture. The information is scattered
all over the place. :/

Christian

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] String sort

[Francesc Altet]

A Friday 15 February 2008, Charles R Harris escrigué:
> On Fri, Feb 15, 2008 at 5:09 AM, Francesc Altet <[EMAIL PROTECTED]> 
wrote:
> > Hi Chuck,
> >
> > I've given more testing to the new quicksort routines for strings
> > in the forthcoming NumPy.  I've run the indexing test units in
> > PyTables Pro (they stress the sorting routines a lot) against the
> > current version of NumPy in the repository, for the complete set of
> > quicksort, mergesort and heapsort of the new implementation, and
> > I'm happy to say to everything went very smoothly, i.e. more than
> > 1000 tests with different input arrays has passed flawlessly.  Good
> > job!
>
> Hi Francesc,
>
> Thanks for the thorough testing. It makes me feel much more
> comfortable this close to the release of 1.0.5.

You are welcome.  I forgot to mention that I tested out both direct 
(.sort()) and indirect (.argsort()) methods.

Cheers,

-- 
>0,0<   Francesc Altet     http://www.carabos.com/
V   V   Cárabos Coop. V.   Enjoy Data
 "-"
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] MLPY - Machine Learning Py - Python/NumPy based package for machine learning

[Davide Albanese]

Ok, sorry! The cost function is embedded into the C smo() function. I 
think that you cannot access it directly.

Matthieu Brucher ha scritto:
> Well this is an input parameter, I'd like to access the cost function 
> directly so that I can use it to follow its gradient to the limit 
> between the two classes.
>
> Matthieu
>
> 2008/2/15, Davide Albanese <[EMAIL PROTECTED] >:
>
> Yes: https://mlpy.fbk.eu/wiki/MlpyExamplesWithDoc
>
> * svm()
> Initialize the svm class.
>
> Inputs:
> ...
> cost - for cost-sensitive classification [-1.0, 1.0]
>
>
>
> Matthieu Brucher ha scritto:
>
> > OK, I'll try it then :)
> >
> > Is there an access to the underlying cost function ? (this is mainly
> > what I need)
> >
> > Matthieu
> >
>
> > 2008/2/15, Davide Albanese <[EMAIL PROTECTED]
>   >>:
>
> >
> > I don't know very well libsvm too, the core of svm-mlpy is
> written
> > in C
> > and was developed by Stefano Merler ([EMAIL PROTECTED]
> 
>
> > >).
>
> > I have simply wrapped it into svm() Python class.
> >
> > Regards,
> >
> > /* da */
> >
> >
> >
> > --
> > French PhD student
> > Website : http://matthieu-brucher.developpez.com/
> > Blogs : http://matt.eifelle.com and
> http://blog.developpez.com/?blog=92
> > LinkedIn : http://www.linkedin.com/in/matthieubrucher
>
>
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org 
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
>
> -- 
> French PhD student
> Website : http://matthieu-brucher.developpez.com/
> Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
> LinkedIn : http://www.linkedin.com/in/matthieubrucher 

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] MLPY - Machine Learning Py - Python/NumPy based package for machine learning

[Matthieu Brucher]

Well this is an input parameter, I'd like to access the cost function
directly so that I can use it to follow its gradient to the limit between
the two classes.

Matthieu

2008/2/15, Davide Albanese <[EMAIL PROTECTED]>:
>
> Yes: https://mlpy.fbk.eu/wiki/MlpyExamplesWithDoc
>
> * svm()
> Initialize the svm class.
>
> Inputs:
> ...
> cost - for cost-sensitive classification [-1.0, 1.0]
>
>
>
> Matthieu Brucher ha scritto:
>
> > OK, I'll try it then :)
> >
> > Is there an access to the underlying cost function ? (this is mainly
> > what I need)
> >
> > Matthieu
> >
>
> > 2008/2/15, Davide Albanese <[EMAIL PROTECTED] >:
>
> >
> > I don't know very well libsvm too, the core of svm-mlpy is written
> > in C
> > and was developed by Stefano Merler ([EMAIL PROTECTED]
>
> > ).
>
> > I have simply wrapped it into svm() Python class.
> >
> > Regards,
> >
> > /* da */
> >
> >
> >
> > --
> > French PhD student
> > Website : http://matthieu-brucher.developpez.com/
> > Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
> > LinkedIn : http://www.linkedin.com/in/matthieubrucher
>
>
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>



-- 
French PhD student
Website : http://matthieu-brucher.developpez.com/
Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn : http://www.linkedin.com/in/matthieubrucher
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] resizing arrays

[Neal Becker]

I have a situation where I'm going to save some data sets to plot, but I
don't know a-priori how many sets there will be.  I'm using this code:

try:
  shape = list(phase_plots.shape)
  shape[0] += 1
  phase_plots.resize (shape, refcheck=0)
except NameError:
  phase_plots = empty ((1, 2*iterations+1, l))

This works, I'm just wondering if this is a reasonable approach or if maybe
something else would be better (or more efficient).


___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] List Array References?

[Alexander Michael]

On Fri, Feb 15, 2008 at 7:12 AM, Stefan van der Walt <[EMAIL PROTECTED]> wrote:
>  As far as I know, the reference count to the array is increased when
>  you create a view, but the views themselves are not tracked anywhere.
>  You can therefore say whether there are references around, but you
>  cannot identify the Python objects.

I would like to occasionally dynamically grow an array (i.e. add
length to existing dimensions) as I do not know th ultimately required
length beforehand, but I have a good guess so I won't be need to
reallocate that often if at all. The only way I know how to do this is
numpy is to create a new larger array with the new dimensions and copy
the existing data from the smaller array into it. Perhaps there is a
better way that doesn't invalidate views on the array? I want to make
sure that there are no outstanding references to the old array (i.e.
views on it, etc.) so that I can raise a helpful exception while
developing.

Robin and Davide-
Thanks for the pointer to the numpy.who method, while that only
searches in a specified dictionary, it was the method that I thought I
had encountered before.
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Is anyone knowledgeable about dll deployment on windows ?

[David Cournapeau]

Christian Heimes wrote:
>
> That's not enough for some DLLs. .NET assemblies as well as Side-by-Side
> dlls (SxS) must be registered properly. You can install a SxS dll in
> PATH but simple copying the DLL isn't enough. It also depends on the OS.

Ah, that reminds me of something I tried, that is using the global cache 
something, with gacutil, which is supposed to work with 'bare' dll. Of 
course, it did not work as said on MSDN, and I got some obscure error 
message (I am starting to wonder if anything coming from MS works at all).

>
> Have fun! The SxS issue has bitten us in the a.. when we delivered the
> second beta of Python 3.0.
>

Do you have a link to the related python ML discussion by any chance ?

cheers,

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Is anyone knowledgeable about dll deployment on windows ?

[Christian Heimes]

Matthieu Brucher wrote:
> When Visual Studio 2008 will be used, there might be a way of using the
> manifest files (that were created for a similar purpose).
> For the moment, All I know is that you must put the dll in the
> Windows/system32 folder or somewhere in the PATH.

That's not enough for some DLLs. .NET assemblies as well as Side-by-Side
dlls (SxS) must be registered properly. You can install a SxS dll in
PATH but simple copying the DLL isn't enough. It also depends on the OS.

Have fun! The SxS issue has bitten us in the a.. when we delivered the
second beta of Python 3.0.

Christian

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] String sort

[Charles R Harris]

On Fri, Feb 15, 2008 at 5:09 AM, Francesc Altet <[EMAIL PROTECTED]> wrote:

> Hi Chuck,
>
> I've given more testing to the new quicksort routines for strings in the
> forthcoming NumPy.  I've run the indexing test units in PyTables Pro
> (they stress the sorting routines a lot) against the current version of
> NumPy in the repository, for the complete set of quicksort, mergesort
> and heapsort of the new implementation, and I'm happy to say to
> everything went very smoothly, i.e. more than 1000 tests with different
> input arrays has passed flawlessly.  Good job!
>

Hi Francesc,

Thanks for the thorough testing. It makes me feel much more comfortable this
close to the release of 1.0.5.

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Is anyone knowledgeable about dll deployment on windows ?

[David Cournapeau]

Matthieu Brucher wrote:
> When Visual Studio 2008 will be used, there might be a way of using 
> the manifest files (that were created for a similar purpose).
> For the moment, All I know is that you must put the dll in the 
> Windows/system32 folder or somewhere in the PATH.

Do you know where to find some informations about that ? I tried to find 
something readable, but I could only find some msdn pages with almost no 
concrete information on how to use those manifest files.

cheers,

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] List Array References?

[Davide Albanese]

Whith "standard" Python:
 >>> who()

Robin ha scritto:
> On Thu, Feb 14, 2008 at 8:43 PM, Alexander Michael <[EMAIL PROTECTED]> wrote:
>   
>> Is there a way to list all of the arrays that are referencing a given
>>  array? Similarly, is there a way to get a list of all arrays that are
>>  currently in memory?
>> 
>
> For the second question, if you are working interactively in Ipython, you can 
> do
> %who ndarray
> or
> %whos ndarray
> to get a list of arrays.
>
> It might be possible to get this functionality within a script/program
> through the ipython api, but I'm afraid I don't know much about that.
>
> Cheers,
>
> Robin
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>   

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Is anyone knowledgeable about dll deployment on windows ?

[David Cournapeau]

Jon Wright wrote:
>
> Hi,
>
> To be honest I'd be nervous about spreading your depedencies all over 
> the disk, the potential interactions go as n^2. 

Well, I am more than nervous, this is totally insane, and I tried 
different things (registry: per application path, etc...). But it just 
looks like windows is even more f***-up than I thought. I am tired of 
wasting my time for this broken platform: I am really near giving up 
supporting MS compilers at all.

> According to msdn there 
> is a function which might work:
>
> "SetDllDirectory: Adds a directory to the search path used to locate 
> DLLs for the application."
> http://msdn2.microsoft.com/en-us/library/ms686203.aspx

Well, this does not sound right to me either; anyway, this is a C 
function which does not seem to be available from python.

> Otherwise you might have to copy the 
> mkl dll into the "directory from which the application loaded". This 
> would seem like a better choice, but I do wonder where that is for python.

This directory depends on how you start python, I think, so this is not 
better. Having looked at how installers do it, there does not seem to be 
much choice: either use PATH (actually, that's what the MKL is doing), 
or put everything in hardcoded directories (system32 and co). The other 
solution I see is to add the paths to the registry for the python 
application (it seems that windows has a registry subtree App Paths to 
handle per application dll path, but touching the registry makes me 
quite nervous too).
>
> Hope this helps. I'm certainly not an expert, but keen to try to support 
> your work on help the masses access optimised libraries.

Thanks,

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] MLPY - Machine Learning Py - Python/NumPy based package for machine learning

[Davide Albanese]

Yes: https://mlpy.fbk.eu/wiki/MlpyExamplesWithDoc

* svm()
Initialize the svm class.

Inputs:
...
cost - for cost-sensitive classification [-1.0, 1.0]



Matthieu Brucher ha scritto:
> OK, I'll try it then :)
>
> Is there an access to the underlying cost function ? (this is mainly 
> what I need)
>
> Matthieu
>
> 2008/2/15, Davide Albanese <[EMAIL PROTECTED] >:
>
> I don't know very well libsvm too, the core of svm-mlpy is written
> in C
> and was developed by Stefano Merler ([EMAIL PROTECTED]
> ).
> I have simply wrapped it into svm() Python class.
>
> Regards,
>
> /* da */
>
>
>
> -- 
> French PhD student
> Website : http://matthieu-brucher.developpez.com/
> Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
> LinkedIn : http://www.linkedin.com/in/matthieubrucher 

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Is anyone knowledgeable about dll deployment on windows ?

[Matthieu Brucher]

When Visual Studio 2008 will be used, there might be a way of using the
manifest files (that were created for a similar purpose).
For the moment, All I know is that you must put the dll in the
Windows/system32 folder or somewhere in the PATH.

Matthieu

2008/2/15, David Cournapeau <[EMAIL PROTECTED]>:
>
> Hi,
>
> My head hurt trying to understand dll management with windows. I
> wanted to find a sane way to use dll for numscons, but I can't see how
> to do this, so I was wondering if anyone on this ML had any deep
> knowledge on how to install dll, and reuse them with python extensions ?
> The problem is the following:
> - you have some dll (say MKL) in a path (say C:\program files\MKL).
> - how do you tell another dll (more exactly a python extension, a
> .pyd) to look for mkl dll into the MKL path ?
>
> It seems that there is no rpath-like capability on windows, and the only
> way to load dll is to put them into some windows directory, or to put
> the dll path into PATH. Is it really the only way ?
>
> cheers,
>
> David
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>



-- 
French PhD student
Website : http://matthieu-brucher.developpez.com/
Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn : http://www.linkedin.com/in/matthieubrucher
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Is anyone knowledgeable about dll deployment on windows ?

[Jon Wright]

David Cournapeau wrote:
> Hi,
> 
> My head hurt trying to understand dll management with windows. I 
> wanted to find a sane way to use dll for numscons, but I can't see how 
> to do this, so I was wondering if anyone on this ML had any deep 
> knowledge on how to install dll, and reuse them with python extensions ? 
> The problem is the following:
> - you have some dll (say MKL) in a path (say C:\program files\MKL).
> - how do you tell another dll (more exactly a python extension, a 
> .pyd) to look for mkl dll into the MKL path ?

Hi,

To be honest I'd be nervous about spreading your depedencies all over 
the disk, the potential interactions go as n^2. According to msdn there 
is a function which might work:

"SetDllDirectory: Adds a directory to the search path used to locate 
DLLs for the application."
http://msdn2.microsoft.com/en-us/library/ms686203.aspx

Perhaps this is only for vista/xp? Otherwise you might have to copy the 
mkl dll into the "directory from which the application loaded". This 
would seem like a better choice, but I do wonder where that is for python.

Hope this helps. I'm certainly not an expert, but keen to try to support 
your work on help the masses access optimised libraries.

Best,

Jon

> 
> It seems that there is no rpath-like capability on windows, and the only 
> way to load dll is to put them into some windows directory, or to put 
> the dll path into PATH. Is it really the only way ?
> 
> cheers,
> 
> David
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion


___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] String sort

[Francesc Altet]

A Thursday 14 February 2008, Charles R Harris escrigué:
> On Thu, Feb 14, 2008 at 1:03 PM, Francesc Altet <[EMAIL PROTECTED]> 
wrote:
> > Maybe I'd also be interested in trying insertion sort out.  During
> > the optimization process of an OPSI index, there is a need to sort
> > out a slice of data that is already made of smaller chunks that are
> > already sorted, so chances are that insertion sort could be
> > significantly faster than the merge sort (or even the quick sort)
> > in this scenario.
> >
> > But this is becoming an OT.  However, I'd be glad to further dicuss
> > this privately, if you like to.
>
> Well, I don't have much more to say. If you do decide that insertion
> sort will be useful you won't have to twist my arm much to get it,
> but I think it is most useful when the data never has to move far. In
> the case of quicksort and mergesort it is called to deal with small
> unsorted chunks, but the chunks themselves are already in the right
> place. Some kind of multi merge sort might be more appropriate to the
> OPSI index.

OK, thanks, I'll have you in mind ;)  And yes, multi-way merge seems 
very interesting for OPSI indeed.  Eventually, it might even be a good 
candidate for taking advantage of the several cores in modern CPU's;  
so it would be really interesting to check out this path.

Thanks for very insightful hints!

-- 
>0,0<   Francesc Altet     http://www.carabos.com/
V   V   Cárabos Coop. V.   Enjoy Data
 "-"
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] List Array References?

[Robin]

On Thu, Feb 14, 2008 at 8:43 PM, Alexander Michael <[EMAIL PROTECTED]> wrote:
> Is there a way to list all of the arrays that are referencing a given
>  array? Similarly, is there a way to get a list of all arrays that are
>  currently in memory?

For the second question, if you are working interactively in Ipython, you can do
%who ndarray
or
%whos ndarray
to get a list of arrays.

It might be possible to get this functionality within a script/program
through the ipython api, but I'm afraid I don't know much about that.

Cheers,

Robin
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Is anyone knowledgeable about dll deployment on windows ?

[David Cournapeau]

Hi,

My head hurt trying to understand dll management with windows. I 
wanted to find a sane way to use dll for numscons, but I can't see how 
to do this, so I was wondering if anyone on this ML had any deep 
knowledge on how to install dll, and reuse them with python extensions ? 
The problem is the following:
- you have some dll (say MKL) in a path (say C:\program files\MKL).
- how do you tell another dll (more exactly a python extension, a 
.pyd) to look for mkl dll into the MKL path ?

It seems that there is no rpath-like capability on windows, and the only 
way to load dll is to put them into some windows directory, or to put 
the dll path into PATH. Is it really the only way ?

cheers,

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] List Array References?

[Stefan van der Walt]

Hi Alexander

On Thu, Feb 14, 2008 at 03:43:46PM -0500, Alexander Michael wrote:
> Is there a way to list all of the arrays that are referencing a given
> array? Similarly, is there a way to get a list of all arrays that are
> currently in memory?

As far as I know, the reference count to the array is increased when
you create a view, but the views themselves are not tracked anywhere.
You can therefore say whether there are references around, but you
cannot identify the Python objects.

I'm curious why you need this information -- maybe there is an
alternative solution to your problem?

Regards
Stéfan
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] String sort

[Francesc Altet]

Hi Chuck,

I've given more testing to the new quicksort routines for strings in the 
forthcoming NumPy.  I've run the indexing test units in PyTables Pro 
(they stress the sorting routines a lot) against the current version of 
NumPy in the repository, for the complete set of quicksort, mergesort 
and heapsort of the new implementation, and I'm happy to say to 
everything went very smoothly, i.e. more than 1000 tests with different 
input arrays has passed flawlessly.  Good job!

Cheers,

-- 
>0,0<   Francesc Altet     http://www.carabos.com/
V   V   Cárabos Coop. V.   Enjoy Data
 "-"
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] MLPY - Machine Learning Py - Python/NumPy based package for machine learning

[Matthieu Brucher]

OK, I'll try it then :)

Is there an access to the underlying cost function ? (this is mainly what I
need)

Matthieu

2008/2/15, Davide Albanese <[EMAIL PROTECTED]>:
>
> I don't know very well libsvm too, the core of svm-mlpy is written in C
> and was developed by Stefano Merler ([EMAIL PROTECTED]).
> I have simply wrapped it into svm() Python class.
>
> Regards,
>
> /* da */



-- 
French PhD student
Website : http://matthieu-brucher.developpez.com/
Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn : http://www.linkedin.com/in/matthieubrucher
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] MLPY - Machine Learning Py - Python/NumPy based package for machine learning

[Davide Albanese]

I don't know very well libsvm too, the core of svm-mlpy is written in C 
and was developed by Stefano Merler ([EMAIL PROTECTED]).
I have simply wrapped it into svm() Python class.

Regards,
/* da */

Matthieu Brucher ha scritto:
> Thanks for the reference :)
>
> I should have asked in other terms : how does it compare to libsvm, 
> which is one of the most known packages for SVMs ?
>
> Matthieu
>
> 2008/2/15, Davide Albanese <[EMAIL PROTECTED] >:
>
> Dear Matthieu,
> I don't know very well scikit.
> The Svm is implemented by Sequential Minimal Optimization (SMO).
> As for Terminated Ramps (TR) you can read this paper:
> /S. Merler and G. Jurman/* Terminated Ramp - Support Vector Machine: a
> nonparametric data dependent kernel* Neural Network, 19(10),
> 1597-1611,
> 2006.
>
> /* da */
>
>
>
>
> -- 
> French PhD student
> Website : http://matthieu-brucher.developpez.com/
> Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
> LinkedIn : http://www.linkedin.com/in/matthieubrucher 

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] MLPY - Machine Learning Py - Python/NumPy based package for machine learning

[Matthieu Brucher]

Thanks for the reference :)

I should have asked in other terms : how does it compare to libsvm, which is
one of the most known packages for SVMs ?

Matthieu

2008/2/15, Davide Albanese <[EMAIL PROTECTED]>:
>
> Dear Matthieu,
> I don't know very well scikit.
> The Svm is implemented by Sequential Minimal Optimization (SMO).
> As for Terminated Ramps (TR) you can read this paper:
> /S. Merler and G. Jurman/* Terminated Ramp - Support Vector Machine: a
> nonparametric data dependent kernel* Neural Network, 19(10), 1597-1611,
> 2006.
>
> /* da */




-- 
French PhD student
Website : http://matthieu-brucher.developpez.com/
Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn : http://www.linkedin.com/in/matthieubrucher
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] MLPY - Machine Learning Py - Python/NumPy based package for machine learning

[Davide Albanese]

Dear Matthieu,
I don't know very well scikit.
The Svm is implemented by Sequential Minimal Optimization (SMO).
As for Terminated Ramps (TR) you can read this paper:
/S. Merler and G. Jurman/* Terminated Ramp - Support Vector Machine: a 
nonparametric data dependent kernel* Neural Network, 19(10), 1597-1611, 
2006.

/* da */


Matthieu Brucher ha scritto:
> Hi,
>
> How does it compare to the elarn scikit, especially for the SVM part ? 
> How was it implemented ?
>
> Matthieu
>
> 2008/2/14, Davide Albanese <[EMAIL PROTECTED] >:
>
> *Machine Learning Py* (MLPY) is a *Python/NumPy* based package for
> machine learning.
> The package now includes:
>
> * *Support Vector Machines* (linear, gaussian, polinomial,
>   terminated ramps) for 2-class problems
> * *Fisher Discriminant Analysis* for 2-class problems
> * *Iterative Relief* for feature weighting for 2-class problems
> * *Feature Ranking* methods based on Recursive Feature Elimination
>   (rfe, onerfe, erfe, bisrfe, sqrtrfe) and Recursive Forward
>   Selection (rfs)
> * *Input Data* functions
> * *Confidence Interval* functions
>
> Requires Python  >= 2.4 and NumPy
>  >= 1.0.3.*
> MLPY* is a project of MPBA Group  (mpa.fbk.eu) at
> Fondazione Bruno Kessler (www.fbk.eu). *
> MLPY* is free software. It is licensed under the GNU General Public
> License (GPL) version 3 .
>
> HomePage: mlpy.fbk.eu
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org 
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
>
> -- 
> French PhD student
> Website : http://matthieu-brucher.developpez.com/
> Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
> LinkedIn : http://www.linkedin.com/in/matthieubrucher 

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion