Thanks for the idea. It may not apply in my particular case. I forgot to mention that my memory is 32GB on the machine I am using. So swapping with the hard drive does not seem likely.
Given that the second execution of the function is much faster (1 second vs 5 min), my best guess is that the long initial run might be due to compiling. On Tuesday, January 26, 2016 at 12:12:41 PM UTC-5, nae...@ic.ufal.br wrote: > > Hello Christopher, > > I saw you code and you noticed you are reading all the data(3GB) locally, > maybe it's causing your machine to use swap memory, which leads to the > performance issue. > You can check that by > 1: open htop in one terminal (apt-get install -y htop) > 2: run your application > 3: check the memory usage on htop, if it use 100% of memory, then the > machine will start to swap > > Please, correct me if I'm wrong. >
