SSTableWriter and Bulk Loading life cycle enhancement

2012-05-03 Thread Benoit Perroud
Hi All, I'm bulk loading (a lot of) data from Hadoop into Cassandra 1.0.x. The provided CFOutputFormat is not the best case here, I wanted to use the bulk loading feature. I know 1.1 comes with a BulkOutputFormat but I wanted to propose a simple enhancement to SSTableSimpleUnsortedWriter that coul

Re: SSTableWriter and Bulk Loading life cycle enhancement

2012-05-07 Thread aaron morton
Can you copy the sstables as a task after the load operation ? You should know where the files are. The are multiple files may be created by the writer during the loading process. So running code that performs a long running action will impact on the time taken to pump data through the SSTable