An alternative approach is to form another new cluster, leave the original cluster alive (many times it's a must since it needs to be 24x7 online). Double write to the two clusters and later migrate the data to it. Either by taking a snapshot and pass those files to the new cluster or with sstableloader. With this procedure, you'll need to have the same token range ownership.
Another solution is to migrate using Spark which will full-table-scan. We have generic code that does it and we can open source it. This way the new cluster can be of any size and speed is also good with large amount of data (100s of TB). This process is also restartable as it takes days to transfer such amount of data. Good luck On Tue, Dec 4, 2018 at 9:04 PM dinesh.jo...@yahoo.com.INVALID <dinesh.jo...@yahoo.com.invalid> wrote: > Thanks, nice summary of the overall process. > > Dinesh > > > On Tuesday, December 4, 2018, 9:38:47 PM EST, Jonathan Koppenhofer < > j...@koppedomain.com> wrote: > > > Unfortunately, we found this to be a little tricky. We did migrations from > DSE 4.8 and 5.0 to OSS 3.0.x, so you may run into additional issues. I will > also say your best option may be to install a fresh cluster and stream the > data. This wasn't feasible for us at the size and scale in the time frames > and infrastructure restrictions we had. I will have to review my notes for > more detail, but off the top of my head, for an in place migration... > > Pre-upgrade > * Be sure you are not using any Enterprise features like Search or Graph. > Not only are there not equivalent features in open source, but theses > features require proprietary classes to be in the classpath, or Cassandra > will not even start up. > * By default, I think DSE uses their own custom authenticators, > authorizors, and such. Make sure what you are doing has an open source > equivalent. > * The DSE system keyapaces use custom replication strategies. Convert > these to NTS before upgrade. > * Otherwise, follow the same processes you would do before an upgrade > (repair, snapshot, etc) > > Upgrade > * The easy part is just replacing the binaries as you would in normal > upgrade. Drain and stop the existing node first. You can also do this same > process in a rolling fashion to maintain availability. In our case, we were > doing an in-place upgrade and reusing the same IPs > * DSE unfortunately creates a custom column in a system table that > requires you to remove one (or more) system tables (peers?) to be able to > start the node. You delete these system tables by removing the sstbles on > disk while the node is down. This is a bit of a headache if using vnodes. > As we are using vnodes, it required us to manually specify num tokens, and > the specific tokens the node was responsible for in Cassandra.yaml. You > have to do this before you start the node. If not using vnodes, this is > simpler, but we used vnodes. Again, I'll double check my notes. Once the > node is up, you can revert to your normal vnodes/num tokens settings. > > Post upgrade: > * Drop DSE system tables. > > I'll revert with more detail if needed. > > On Tue, Dec 4, 2018, 5:46 PM Nandakishore Tokala < > nandakishore.tok...@gmail.com wrote: > > HI All, > > we are migrating from DSE to open source Cassandra. if anyone has recently > migrated, Can you please share their experience, steps you followed and > challenges you guys faced. > > we want to migrate to the same computable version in open source, can you > give us version number(even with the minor version) for DSE 5.1.2 > > 5.1 DSE production-certified 3.10 + enhancements 3.4 + enhancements big m > > -- > Thanks & Regards, > Nanda Kishore > >
