On Tuesday, 10 November 2015 at 10:13:46 UTC, Russel Winder wrote:
On Mon, 2015-11-09 at 19:31 +0000, Márcio Martins via
Digitalmars-d wrote:
I have been running some MCMC simulations in Python and it's
hard
to cope with how unbelievably slow it is.
Takes me almost a minute to run a few hundred thousand samples
on
my laptop whereas I can run the same simulation with a million
samples in under 100ms, on my phone with JavaScript on a
browser.
Are you using NumPy with Python, or pure Python? In either case
you will be better served by profiling you code to find out
whoch is actually the performance bottleneck and then doing one
of:
1. Use Numba to JIT the Python and achieve native code speed.
2. Replace the Python code with D code using either CFFI or
PyD. 3. Replace the Python code with Chapel code using either
CFFI or PyChapel.
You could also use C++ or Rust via CFFI, but I have little
experience using Rust with Python and am trying to avoid C++.
Then, you spend a minute waiting for the simulation to finish,
to find out you had an error in your report code that would
have been easily caught with static typing. So annoying...
Is anyone doing similar stuff with D? Unfortunately, I
couldn't find any plotting libraries nor MATLAB-like
numerical/stats libs in dub.
No need if you leave the coordination code in Python and then
use Matplotlib, etc. Just put the computational expensive code
into Chapel or D, leave the rest of your code in Python.
This seems like another area where D could easily pick up
momentum with RDMD and perhaps an integration with Jupyter
which is becoming very very popular.
Jupyter (née IPython) is only popular in one workflow, creting
papers for people to read and play with the executable code
fragments. This is a big area but only one of many.
Numba worked great! I had never heard of it before, but seems
like an elegant enough and practical solution. Still slightly
slower than JavaScript but fast enough for prototyping.
No wonder Python is so popular despite being so slow - everything
just works and feels elegant with no frustration. From not even
having Python installed to having everything setup with Miniconda
and having my code running took under 10 minutes and the only
thing that wasn't smooth was really the speed, which was really
frustrating a the time, but in a way was to be expected, and now
was also elegantly fixed with Numba. No hacks, no platform
issues, just silky smoothness :)
Thanks!