On Tue, 14 Mar 2023 at 16:27, Alexander Nestorov <alexander...@gmail.com> wrote: > > I'm working on an NLP and I got bitten by an unreasonably slow behaviour in > Python while operating with small amounts of numbers. > > I have the following code: > > ```python > import random, time > from functools import reduce > > def trainPerceptron(perceptron, data): > learningRate = 0.002 > weights = perceptron['weights'] > error = 0 > for chunk in data: > input = chunk['input'] > output = chunk['output'] > > # 12x slower than equivalent JS > sum_ = 0 > for key in input: > v = weights[key] > sum_ += v
In Python something like your task here would usually use something along the lines of NumPy. Your two innermost loops involve adding up a subset of numbers from a list chosen using a list of indices. This is something that numpy can do much more efficiently with its fancy indexing e.g.: In [3]: a = np.array([1, 2, 3, 4, 5, 6, 7]) In [4]: b = np.array([0, 3, 5]) In [5]: a[b] Out[5]: array([1, 4, 6]) In [6]: a[b].sum() Out[6]: 11 This a[b].sum() operation in your code would be weights[input].sum() and would be much faster than the loop shown (the speed difference will be larger if you increase the size of the input array). -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
