It is likely an issue with the GC. An easy way to detect where your program spend the most time is to send a SIGINT (CTRL+C on the terminal) and analyse the traceback.
I repeated the operation a few times and it was always inside some method (grow, CellSet) in gc.nim I quickly checked with another GC --gc:markAndSweep and the problem doesn't occur. It is roughly as fast as with a seq
