At high level, in the production line, machine will provide the data in the form of CSV in every 1 sec to 1 minutes to 1 day ( depending on machine type used in the line operations). I need to parse those files and load it to DB and build and API layer expose it to downstream systems.
Number of files to be processed 13,889,660,134 per day Each file could range from 20 KB to 600MB which will translate into few hundred rows to millions of rows. High availability with high write. Read is less compare to write. While extracting the rows, few validation to be performed. Build an API layer on top of the data to be persisted in the DB. Now, tell me what would be the best choice... From: Russell Bateman [mailto:r...@windofkeltia.com] Sent: Thursday, May 31, 2018 7:36 PM To: user@cassandra.apache.org Subject: Re: Mongo DB vs Cassandra Sudhakar, MongoDB will accommodate loading CSV without regard to schema while still creating identifiable "columns" in the database, but you'll have to predict or back-impose some schema later if you're going to create indices for fast searching of the data. You can perform searching of data without indexing in MongoDB, but it's slower. Cassandra will require you to understand the schema, i.e.: what the columns are up front unless you're just going to store the data without schema and, therefore, without ability to search effectively. As suggested already, you should share more detail if you want good advice. Both DBs are excellent. Both do different things in different ways. Hope this helps, Russ On 05/31/2018 05:49 AM, Sudhakar Ganesan wrote: Team, I need to make a decision on Mongo DB vs Cassandra for loading the csv file data and store csv file as well. If any of you did such study in last couple of months, please share your analysis or observations. Regards, Sudhakar Legal Disclaimer : The information contained in this message may be privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and delete or destroy any copy of this message!
