On 7/30/16, 5:26 PM, "bilbosax" <[email protected]> wrote:
>So my application is a HUGE number cruncher. But since it is running on >one >thread, the deployed version still takes about an hour to process my >data. >If at all possible, I would like to get this down to about 15 minutes so I >could run it several times a day. My computer has two XEON Quad Cores and >almost 24GB of RAM, so I am not worried about overworking my machine. >Currently, my main application is running at about 150MB. > >So I am interested in speeding up my application using Workers. I have >never >tried using Workers to increase speed, so I wanted to ask people with >experience with Workers if you think it is worth the effort. By splitting >the processing load to four threads, do you think I could whittle the >processing time down to 15 minutes, or is there a point at which adding >more >threads just increases overhead, and therefore you don't gain much? Or is >it >hugely advantageous? It depends... My first question is how your app can run for more than a minute without a script timeout. Do you already have some pseudo-threading going on? Workers should be faster in most cases, but understanding how you keep your code running might give you a better idea of any trade-offs. As Gary says in his post, your number crunching must involve things that can be computed in parallel. If the workers are going to wait on a other workers the waiting will affect your performance results. -Alex
