Dear all,
for quite a while now, I have experienced this issue with some curiosity;
yesterday I had it again, when a program that took well over one hour
before only needed about ten minutes, after a system reboot (recent Ubuntu)
and with no browser started -- finally deciding to post this.
I
On Sat, Feb 2, 2013 at 3:19 PM, Nick Rudnick nick.rudn...@gmail.com wrote:
Roughly, I would say the differences in runtime can reach a factor as much
as 1:10 at many times -- and so I am curious whether this subject has
already been observed or even better discussed elsewhere. I have spoken to
Hi Gwern,
thanks for the interesting info. I quite often have processing of CSV file
data of about 100M-1G done.
Thanks a lot, Nick
2013/2/2 Gwern Branwen gwe...@gmail.com
On Sat, Feb 2, 2013 at 3:19 PM, Nick Rudnick nick.rudn...@gmail.com
wrote:
Roughly, I would say the differences in
Nick Rudnick wrote:
thanks for the interesting info. I quite often have processing of CSV file
data of about 100M-1G done.
What library are you using to process the CSV? I have had problems
with excessive laziness causing processing of a 75Meg CSV file
consuming 500+ megabytes and after I
If you are doing row-by-row transformations, I would recommend giving a try to
my csv-conduit or csv-enumerator packages on Hackage. They were designed with
constant space operation in mind, which may help you here.
If you're keeping an accumulator around, however, you may still run into
On Sat, Feb 2, 2013 at 5:14 PM, Ozgun Ataman ozata...@gmail.com wrote:
If you are doing row-by-row transformations, I would recommend giving a try
to my csv-conduit or csv-enumerator packages on Hackage. They were designed
with constant space operation in mind, which may help you here.
If
Ozgun Ataman wrote:
If you are doing row-by-row transformations, I would recommend
giving a try to my csv-conduit
I was using csv-conduit.
If you're keeping an accumulator around, however, you may still
run into issues with too much laziness.
This was the problem which I solved with
