Re: [Numpy-discussion] How can I constrain linear_least_squares to integer solutions?

[Charles R Harris]

On Nov 27, 2007 9:51 PM, Mark Schmucker <[EMAIL PROTECTED]>
wrote:

>  Hi,
>
>
>
> I have successfully used LinearAlgebra.linear_least_squares to estimate
> solutions to continuous functions. The coefficients returned in that case
> are of course floating-point values.
>
>
>
> Now I have a problem where some of the terms are continuous but some must
> be constrained to integer multiples. As a simple example,
>
>
>
> y = c1 * x^2 + c0 * x
>
>
>
> where c1 is floating-point in the usual way, but c0 can only be
> integer-valued. (The actual problem has many more terms.)
>
>
>
> Can anyone tell me how to constrain some of the coefficients to integer
> values?
>
> This is not a trivial problem, as you can see by googling mixed integer
least squares (MILS). Much will depend on the nature of the parameters, the
number of variables you are using in the fit, and how exact the solution
needs to be. One approach would be to start by rounding the coefficients
that must be integer and improve the solution using annealing or genetic
algorithms to jig the integer coefficients while fitting the remainder in
the usual least square way, but that wouldn't have the elegance of some of
the specific methods used for this sort of problem. However, I don't know of
a package in scipy that implements those more sophisticated algorithms,
perhaps someone else on this list who knows more about these things than I
can point you in the right direction.

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] How can I constrain linear_least_squares to integer solutions?

[Mark Schmucker]

Hi,

 

I have successfully used LinearAlgebra.linear_least_squares to estimate
solutions to continuous functions. The coefficients returned in that
case are of course floating-point values.

 

Now I have a problem where some of the terms are continuous but some
must be constrained to integer multiples. As a simple example,

 

y = c1 * x^2 + c0 * x

 

where c1 is floating-point in the usual way, but c0 can only be
integer-valued. (The actual problem has many more terms.)

 

Can anyone tell me how to constrain some of the coefficients to integer
values?

 

Thank you for any insight.

 

Mark

 

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] how do I speed up this?

[Timothy Hochberg]

On Nov 27, 2007 12:41 PM, Giorgio F. Gilestro <[EMAIL PROTECTED]> wrote:

> Thanks Tim,
> shape of the array is variable but follows the rule (x, y*1440) with
> both x an y being values greater than 1 (usually around 10 or 20).


OK. That's helpful.


> So as result second dimension is always much bigger. (and the bug you
> foresaw is actually taken care of)


Are you sure? This still looks problematic.


>
> I figured out that somehow removing the unnecessary *1 multiplication
> after the boolean verification will increase speed by almost 100% (wow!).
> I guess if I can get rid of the iteration too and use some internal
> function I'd probably get even faster.
> Could .flat() help me somehow?


I doubt it. I'll look at this a little later if I can find some time and see
what I can come up with.



>
>
> Timothy Hochberg wrote:
> >
> >
> > On Nov 27, 2007 11:38 AM, Giorgio F. Gilestro <[EMAIL PROTECTED]
> > > wrote:
> >
> > Hello everyone,
> >
> > ma and new_ma are bi-dimensional array with shape (a1, a2) on
> > which I am
> > performing the following iteration:
> >
> > for fd in range(a1):
> > new_ma[fd] = [( ma[fd][i-5:i+5].sum() == 0 )*1 for i in range
> > (a2)]
> >
> >
> > This looks like it's probably buggy; when, for example,, a2 == 0, I
> > don't think ma[fd][-5:5] is going to return what you want.
> >
> >
> > Is there any faster more elegant way to do that with numpy?
> >
> >
> > There are faster ways, I'm not sure that they are more elegant. It
> > looks like what you want is essentially a moving average, possible
> > with periodic boundary conditions. There are a few different tricks
> > that can make this fast; which is appropriate depends on what the
> > shape of ma is expected to be. If 'a1' if large, then you can
> > profitably remove the outer loop; something vaguely like:
> >
> > for i in range(a2):
> > new_ma[:, i] = ( ma[fd][i-5:i+5].sum() == 0)
> >
> > (This is still buggy as above, since I'm not sure what kind of
> > boundary conditions you need).
> >
> > If that's not applicable, another approach is to loop over the offset,
> > in this case -5 to 5, and remove the loop over a2. That method is more
> > complicated and I'm going to avoid describing it in detail unless
> > needed. One might even be able to do both, but that's probably overkill.
> >
> > HTH
> >
> > --
> > .  __
> > .   |-\
> > .
> > .  [EMAIL PROTECTED] 
> > 
> >
> > ___
> > Numpy-discussion mailing list
> > Numpy-discussion@scipy.org
> > http://projects.scipy.org/mailman/listinfo/numpy-discussion
> >
>
>
> --
> [EMAIL PROTECTED]
> http://www.cafelamarck.it
>
>
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>



-- 
.  __
.   |-\
.
.  [EMAIL PROTECTED]
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How to remove loops over inner()

[Timothy Hochberg]

On Nov 26, 2007 2:30 PM, Hans-Andreas Engel <[EMAIL PROTECTED]>
wrote:

> Dear all:
>
> After using numpy for several weeks, I am very happy about it and
> deeply impressed about the performance improvements it brings in my
> python code.  Now I have stumbled upon a problem, where I cannot use
> numpy to eliminate all my loops in python.
>
> Currently the return value of inner(a, b) is defined as
>   inner(a, b)[I, J] = sum_k  a[I, k] * b[J, k],
> for some super indices I and J.  Somewhat more general is the
> tensordot() function that allows to specify over which axes K is
> summed over.
>
> However, if I understand numpy correctly, the following more general
> version is currently missing:
>  inner(a, b, keep_axis=0)[H, I, J] = sum_k a[H, I, k] * b[H, J, k].
> Here H would be an additional super index (specified via the keep_axis
> keyword), on which no outer product is taken, i.e., the same index is
> used for a[] and b[].
>
> This more general definition would allow elimination of an extra level
> of loops.  For example, I wish to calculate the following
>   a = rand(200, 5, 2)
>   b = rand(200, 4, 2)
>   r = empty(a.shape[:-1] + b.shape[1:-1])
>   for h in range(a.shape[0]):
>   r[h] = inner(a[h], b[h])
> How could I eliminate the loop?  It would be great if there would be
> the mentioned generalized version of the inner() [or tensordot()]
> function, since it would eliminate this loop and make my code much
> faster.
>
> What are your opinions?  Would such a feature be desirable (or is it
> already implemented)?


Essentially, you want to operate on a stack of two dimensional arrays,
correct? I'd be mildly supportive of something like this for tensordot; I'd
prefer more descriptive name for keep_axis, but I don't know what it would
be off the top of my head. In any event it should be XXX_axes and optionally
take a sequence of axes so that more than one can be ignored. You could
trivially build more specific functions on top of tensordot, so I don't see
that inner needs to be changed as it's basically a convenience function
anyway.

-- 
.  __
.   |-\
.
.  [EMAIL PROTECTED]
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] how do I speed up this?

[Giorgio F. Gilestro]

Thanks Tim,
shape of the array is variable but follows the rule (x, y*1440) with 
both x an y being values greater than 1 (usually around 10 or 20).
So as result second dimension is always much bigger. (and the bug you 
foresaw is actually taken care of)
I figured out that somehow removing the unnecessary *1 multiplication 
after the boolean verification will increase speed by almost 100% (wow!).
I guess if I can get rid of the iteration too and use some internal 
function I'd probably get even faster.
Could .flat() help me somehow?

Timothy Hochberg wrote:
>
>
> On Nov 27, 2007 11:38 AM, Giorgio F. Gilestro <[EMAIL PROTECTED] 
> > wrote:
>
> Hello everyone,
>
> ma and new_ma are bi-dimensional array with shape (a1, a2) on
> which I am
> performing the following iteration:
>
> for fd in range(a1):
> new_ma[fd] = [( ma[fd][i-5:i+5].sum() == 0 )*1 for i in range
> (a2)] 
>
>
> This looks like it's probably buggy; when, for example,, a2 == 0, I 
> don't think ma[fd][-5:5] is going to return what you want.
>
>
> Is there any faster more elegant way to do that with numpy?
>
>
> There are faster ways, I'm not sure that they are more elegant. It 
> looks like what you want is essentially a moving average, possible 
> with periodic boundary conditions. There are a few different tricks 
> that can make this fast; which is appropriate depends on what the 
> shape of ma is expected to be. If 'a1' if large, then you can 
> profitably remove the outer loop; something vaguely like:
>
> for i in range(a2):
> new_ma[:, i] = ( ma[fd][i-5:i+5].sum() == 0)
>
> (This is still buggy as above, since I'm not sure what kind of 
> boundary conditions you need).
>
> If that's not applicable, another approach is to loop over the offset, 
> in this case -5 to 5, and remove the loop over a2. That method is more 
> complicated and I'm going to avoid describing it in detail unless 
> needed. One might even be able to do both, but that's probably overkill.
>  
> HTH
>
> -- 
> .  __
> .   |-\
> .
> .  [EMAIL PROTECTED] 
> 
>
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>   


-- 
[EMAIL PROTECTED]
http://www.cafelamarck.it


___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] how do I speed up this?

[Timothy Hochberg]

On Nov 27, 2007 11:38 AM, Giorgio F. Gilestro <[EMAIL PROTECTED]> wrote:

> Hello everyone,
>
> ma and new_ma are bi-dimensional array with shape (a1, a2) on which I am
> performing the following iteration:
>
> for fd in range(a1):
> new_ma[fd] = [( ma[fd][i-5:i+5].sum() == 0 )*1 for i in range (a2)]


This looks like it's probably buggy; when, for example,, a2 == 0, I don't
think ma[fd][-5:5] is going to return what you want.

>
> Is there any faster more elegant way to do that with numpy?


There are faster ways, I'm not sure that they are more elegant. It looks
like what you want is essentially a moving average, possible with periodic
boundary conditions. There are a few different tricks that can make this
fast; which is appropriate depends on what the shape of ma is expected to
be. If 'a1' if large, then you can profitably remove the outer loop;
something vaguely like:

for i in range(a2):
new_ma[:, i] = ( ma[fd][i-5:i+5].sum() == 0)

(This is still buggy as above, since I'm not sure what kind of boundary
conditions you need).

If that's not applicable, another approach is to loop over the offset, in
this case -5 to 5, and remove the loop over a2. That method is more
complicated and I'm going to avoid describing it in detail unless needed.
One might even be able to do both, but that's probably overkill.

HTH

-- 
.  __
.   |-\
.
.  [EMAIL PROTECTED]
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] how do I speed up this?

[Giorgio F. Gilestro]

Hello everyone,

ma and new_ma are bi-dimensional array with shape (a1, a2) on which I am 
performing the following iteration:

for fd in range(a1):
 new_ma[fd] = [( ma[fd][i-5:i+5].sum() == 0 )*1 for i in range (a2)]

Is there any faster more elegant way to do that with numpy?

Thanks a lot!
Giorgio

-- 
[EMAIL PROTECTED]
http://www.cafelamarck.it


___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Appending a numpy array to existing text file

[LB]

If you just want to add your matrix to an existing ascii file, you can
open this file in append mode and give the file handle to
numpy.savetxt :

f_handle = file('my_file.dat', 'a')
savetxt(f_handle, my_matrix)
f_handle.close()

HTH

--
LB
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Appending a numpy array to existing text file

[Andy Cheesman]

Hi people,

Just a quick question, how do I append a numpy array to an existing 
ascii output file? I've looked at the numpy.savetxt function and it 
performs all the formating functions I desire for my output file but it 
doesn't let me specify how I save my array type
Is there a clever work around for this or does anyone have any suggestions?

Many Thanks for your help

Andy
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] ANN: PyTables & PyTables Pro 2.0.2 are out!

[Ivan Vilata i Balaguer]

Hi everyone,

We at Carabos are happy to announce the simultaneous release of the new
2.0.2 versions of both PyTables and PyTables Pro.  They are mainly
bugfix releases, and users of previous versions are encouraged to
upgrade.

And now the official announce:


 Announcing PyTables and PyTables Pro 2.0.2


PyTables is a library for managing hierarchical datasets and designed to
efficiently cope with extremely large amounts of data with support for
full 64-bit file addressing.  PyTables runs on top of the HDF5 library
and NumPy package for achieving maximum throughput and convenient use.
PyTables Pro adds OPSI, a powerful indexing engine for executing
very fast queries in large tables.

In this version, some bugs have been fixed, being the most important a
problem when moving or renaming a group.  Some small improvements have
been added as well.  Besides, a *critical* bug has been fixed in the Pro
version (the problem arose when doing repeated queries using the same
index). Because of this, an upgrade is strongly recommended.

In case you want to know more in detail what has changed in this
version, have a look at ``RELEASE_NOTES.txt``.  Find the HTML version
for this document at:
http://www.pytables.org/moin/ReleaseNotes/Release_2.0.2

You can download a source package of the version 2.0.2 with
generated PDF and HTML docs and binaries for Windows from
http://www.pytables.org/download/stable/

For an on-line version of the manual, visit:
http://www.pytables.org/docs/manual-2.0.2


Migration Notes for PyTables 1.x users
==

If you are a user of PyTables 1.x, probably it is worth for you to look
at ``MIGRATING_TO_2.x.txt`` file where you will find directions on how
to migrate your existing PyTables 1.x apps to the 2.x versions.  You can
find an HTML version of this document at
http://www.pytables.org/moin/ReleaseNotes/Migrating_To_2.x


Resources
=

Go to the PyTables web site for more details:

http://www.pytables.org

About the HDF5 library:

http://hdfgroup.org/HDF5/

About NumPy:

http://numpy.scipy.org/

To know more about the company behind the development of PyTables, see:

http://www.carabos.com/

Acknowledgments
===

Thanks to many users who provided feature improvements, patches, bug
reports, support and suggestions.  See the ``THANKS`` file in the
distribution package for a (incomplete) list of contributors.  Many
thanks also to SourceForge who have helped to make and distribute this
package!  And last, but not least thanks a lot to the HDF5 and NumPy
(and numarray!) makers. Without them, PyTables simply would not exist.

Share your experience
=

Let us know of any bugs, suggestions, gripes, kudos, etc. you may
have.



  **Enjoy data!**

  -- The PyTables Team

::

Ivan Vilata i Balaguer   >qo<   http://www.carabos.com/
   Cárabos Coop. V.  V  V   Enjoy Data
  ""


signature.asc
Description: Digital signature
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion