Rahul Garg wrote:
> a) Can you guys tell me briefly about the kind of problems you are
> tackling with numpy and scipy?
mainly timeseries of Remote Sensing data ('satellite images')
processing. No really fancy math, but huge (sometimes multiple
gigabytes) multidimensional (date, bands, y, x: order of magnitude:
[hundreds, some, tenthousands or more, idem]) datasets. Some tasks are
long-running, not because of the complex math involved, but because of
the amount of data.
> b) Have you ever felt that numpy/scipy was slow and had to switch to
> C/C++/Fortran?
Yes, but usually the overhead of programming the whole thing in C is not
worth the speedup of processing, especially with modern fast computers
and easy ways to parallelize stuff (ipython and/or parallelpython).
> c) Do you use any form of parallel processing? Multicores? SMPs?
> Clusters? If yes how did u utilize them?
See above. Just started to use parallelpython here, currently not yet
for production work, but in testing/prototyping phase. Using both smp,
multicore and multiple machines ('cluster'). PP doesn't make any
difference between them.
How? Just cut the data in suitable pieces and throw them as a pp job to
the cluster :-) It's really that simple nowadays. And most of our
processing is very parallel in nature.
Cheers,
Vincent Schut.
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
