Sounds like Ed is right and you should be doing the append as add-a-new-column instead of overwrite-existing-column.
On Tue, May 31, 2011 at 8:36 PM, Marcus Bointon <mar...@synchromedia.co.uk> wrote: > On 31 May 2011, at 23:03, Dan Kuebrich wrote: > >> I think perhaps OP meant O(N * M), where N is number of rows and M is total >> bytes. > > That's probably more accurate. > > This is what it was doing: Say I repeatedly append 100 bytes to the same 1000 > records. First time around that's 100,000 bytes to transfer. Second time > around it has to read the first 100k, then write it back, plus the next 100k, > making 300k. Next time it's read 200k, write 200k+100k, making 500k. After > 100 iterations you're transferring 20Mb for a 100k update. MySQL really does > do this; It's very painful. In my real case a 50 byte update to 500k records > went from taking a few seconds to several hours, with no prospect of ever > improving. > > Marcus -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
