Re: Which Database when lot of insert / update queries to execute
Hello, I am designing a solution which will need me to import from CSV, i am using my JAVA code to parse. CSV file has 500K rows, and i need to do it thrice an hour, for 10 hours a day. try to use `LOAD DATA INFILE' to import from CSV file. http://dev.mysql.com/doc/refman/5.5/en/load-data.html Also , if i need to do 'group by', on a column on a large table what should i keep in mind, is it advisable, create index on columns used by 'group by' and columns used by aggregate functions. regards, 2012/6/15 abhishek jain abhishek.netj...@gmail.com: hi, I am biased on mysql, and hence i am asking this on mysql forum first. I am designing a solution which will need me to import from CSV, i am using my JAVA code to parse. CSV file has 500K rows, and i need to do it thrice an hour, for 10 hours a day. The Queries will mainly be update but select and insert also at times, The database size will be estimated to be about 5GB. I need to know is this a classic case for a NOSQL database or mysql is a good option. Also , if i need to do 'group by', on a column on a large table what should i keep in mind, is it advisable, Please advice, -- Thanks and kind Regards, Abhishek jain -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql
RE: Which Database when lot of insert / update queries to execute
Let's see SHOW CREATE TABLE ... SELECT ... It sounds doable with MySQL; might be too big for NOSQL. -Original Message- From: abhishek jain [mailto:abhishek.netj...@gmail.com] Sent: Friday, June 15, 2012 1:57 AM To: mysql@lists.mysql.com Subject: Which Database when lot of insert / update queries to execute hi, I am biased on mysql, and hence i am asking this on mysql forum first. I am designing a solution which will need me to import from CSV, i am using my JAVA code to parse. CSV file has 500K rows, and i need to do it thrice an hour, for 10 hours a day. The Queries will mainly be update but select and insert also at times, The database size will be estimated to be about 5GB. I need to know is this a classic case for a NOSQL database or mysql is a good option. Also , if i need to do 'group by', on a column on a large table what should i keep in mind, is it advisable, Please advice, -- Thanks and kind Regards, Abhishek jain -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql
Re: Which Database when lot of insert / update queries to execute
2012/06/15 18:14 +0900, Tsubasa Tanaka try to use `LOAD DATA INFILE' to import from CSV file. http://dev.mysql.com/doc/refman/5.5/en/load-data.html Try is the operative word: MySQL s character format is _like_ CSV, but not the same. The treatment of NULL is doubtless the biggest stumbling-block. -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql
Re: Which database?
You're on the MySQL list, so expect biased answers. Both certainly have their merits. PostgreSQL is currently more advanced, at least if you go by their claims. Hoever, there seems to be much less support for PostgreSQL than MySQL, be it from books or other users. While I expect to use PostgreSQL at some point in time, if only for curiousity's sake, I'm glad I started with MySQL. Edward Dudlik Becoming Digital www.becomingdigital.com - Original Message - From: Kaarel [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, 10 June, 2003 07:38 Subject: Which database? Hi I am fairly new to databases, but my boss has assigned me to choose a database system for our company. I know of Oracle databases but they are way too expensive so it's not an option for us. I did some research and came up with two free database systems that seemed to be more popular - MySQL and PostgreSQL. The problem is that I can't make a decision between the two because I have practically no experience. Some say MySQL is easier to use, some say PostgreSQL is more robust etc. Most articles seem to celarly favor one or the other. It is very hard to find an objective comparison. And I have to explain my boss why I chose either one. So perhaps some of you can give me some reasons why to choose MySQL over Postgre. Thanks, Kaarel [EMAIL PROTECTED] -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED] -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Which database?
# [EMAIL PROTECTED] / 2003-06-10 14:38:49 +0300: I am fairly new to databases, but my boss has assigned me to choose a database system for our company. I know of Oracle databases but they are way too expensive so it's not an option for us. I did some research and came up with two free database systems that seemed to be more popular - MySQL and PostgreSQL. The problem is that I can't make a decision between the two because I have practically no experience. Some say MySQL is easier to use, some say PostgreSQL is more robust etc. Most articles seem to celarly favor one or the other. It is very hard to find an objective comparison. And I have to explain my boss why I chose either one. So perhaps some of you can give me some reasons why to choose MySQL over Postgre. I won't recommend one or the other, but here's why I've been using MySQL: I started programming with ~4 years ago, when all I knew was MS Windows. MySQL ran on NT with no fuss, while you needed cygwin and whatnot to run PostgreSQL. I remember I couldn't manage to install it successfully. MySQL was a snap. At that time, large portions of the PostgreSQL manual said just this section needs to be written, are you volunteering?. For a database newbie, this is deadly. PostgreSQL seemed to require more administration than MySQL (driving a space shuttle requires more knowledge than driving a bicycle... ok, so this comparison is flawed, but you get the point). Now, I've been peeking at PostgreSQL again. Looks like the manual has grown substantially, I seem to recall reading about a native win32 port (not that I would care these days), and perhaps the best of all, PostgreSQL is BSD licensed which means you can safely use it for whatever purpose, in any programming language or environment whatsoever. The same can't be said for MySQL as it is GPL'd. Actually, the license might turn out to be your biggest problem, especially if arter six months of development it turns that you either have to pay for MySQL or rewrite your application using another database. Note that I don't know the MySQL AB pricing scheme, and I'm sure it'd be a fraction of what you'd have to pay for Informix or Oracle at worst. :) -- If you cc me or remove the list(s) completely I'll most likely ignore your message.see http://www.eyrie.org./~eagle/faqs/questions.html -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Which database?
Actually, the license might turn out to be your biggest problem, especially if arter six months of development it turns that you either have to pay for MySQL or rewrite your application using another database. Note that I don't know the MySQL AB pricing scheme, and I'm sure it'd be a fraction of what you'd have to pay for Informix or Oracle at worst. :) Check the prices - about 2 orders of magnitude less than Oracle. Frankly, if you are doing real commercial work, MySQL's license is so trivial as to be unnoticable. To answere the original question, I explain the difference between MySQL and PostgreSQL by analogy: MySQL is an offroad vehicle - simple, powerful, indestructible. PostgreSQL is a limousine - very highly featured, but not as fast and not as rugged. Which you need depends upon your application. One thing I would say in favour of MySQL if you are doing commercial work is that the support is excellent - both community support via this list and the paid-for support from MySQL AB. I don't think PostgreSQL has a support company at the moment; I couldn't comment on its community support. But if I were starting a new project at the moment, that alone would swing me to MySQL. Alec -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
RE: Which database?
Hi I would echo that support is probably the most important aspect of your choice. Unless you have a particularly technically demanding application that needs features only found in Postgres you would be best with MySql You can get an answer to most support questions via this list, or MySql themselves if you have a support contract, within hours or even minutes, from experts. For a newbie, support can make or break a project. I am willing to be corrected, but Postres smaller userbase and lack of commercial support company means you are unlikely to get the same level of support. Peter --- Excellence in internet and open source software --- Sunmaia Birmingham UK www.sunmaia.net tel. 0121-242-1473 International +44-121-242-1473 --- -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: 10 June 2003 14:04 To: [EMAIL PROTECTED] Cc: Kaarel; [EMAIL PROTECTED] Subject: Re: Which database? Actually, the license might turn out to be your biggest problem, especially if arter six months of development it turns that you either have to pay for MySQL or rewrite your application using another database. Note that I don't know the MySQL AB pricing scheme, and I'm sure it'd be a fraction of what you'd have to pay for Informix or Oracle at worst. :) Check the prices - about 2 orders of magnitude less than Oracle. Frankly, if you are doing real commercial work, MySQL's license is so trivial as to be unnoticable. To answere the original question, I explain the difference between MySQL and PostgreSQL by analogy: MySQL is an offroad vehicle - simple, powerful, indestructible. PostgreSQL is a limousine - very highly featured, but not as fast and not as rugged. Which you need depends upon your application. One thing I would say in favour of MySQL if you are doing commercial work is that the support is excellent - both community support via this list and the paid-for support from MySQL AB. I don't think PostgreSQL has a support company at the moment; I couldn't comment on its community support. But if I were starting a new project at the moment, that alone would swing me to MySQL. Alec -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED] -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Which database?
# [EMAIL PROTECTED] / 2003-06-10 14:11:39 +0100: (...) lack of commercial support company means you are unlikely to get the same level of support. unlikely? perhaps. but... how many OS programs do you use, how many of them have commercial support, and how do those that have compare to those that don't? is the help and expertise provided e. g. by [EMAIL PROTECTED] any worse than that of this list? I would disagree. -- If you cc me or remove the list(s) completely I'll most likely ignore your message.see http://www.eyrie.org./~eagle/faqs/questions.html -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
RE: Which database?
smaller userbase and no commercial support *combined* mean fewer sources of support. With MySql, support from this *superb* MySQL list :) and MySql AB is likely to be better than Postgres support. Open source support (free or paid for) in general seems to be better than closed source equivalents Peter -Original Message- From: Roman Neuhauser [mailto:[EMAIL PROTECTED] Sent: 10 June 2003 14:43 To: [EMAIL PROTECTED] Subject: Re: Which database? # [EMAIL PROTECTED] / 2003-06-10 14:11:39 +0100: (...) lack of commercial support company means you are unlikely to get the same level of support. unlikely? perhaps. but... how many OS programs do you use, how many of them have commercial support, and how do those that have compare to those that don't? is the help and expertise provided e. g. by [EMAIL PROTECTED] any worse than that of this list? I would disagree. -- If you cc me or remove the list(s) completely I'll most likely ignore your message.see http://www.eyrie.org./~eagle/faqs/questions.html -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED] -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Which database?
Hey Kaarel- I've been sticking with MySQL mostly for it's support. Large community, lots of documentation, and they have a future plan (which they tend to actually complete ahead of schedule). While the featureset is 'supposedly' not as advanced as pgsql, mysql does in fact work. I hit the thing with an excess of 2k queries/sec, and run DBs in excess of 5gb. Never have a problem with it (4.12). While Postgres has some advantages, mysql support and organization wins.. P Kaarel [EMAIL PROTECTED] 06/10/2003 07:38 AM To: [EMAIL PROTECTED] cc: Subject:Which database? Hi I am fairly new to databases, but my boss has assigned me to choose a database system for our company. I know of Oracle databases but they are way too expensive so it's not an option for us. I did some research and came up with two free database systems that seemed to be more popular - MySQL and PostgreSQL. The problem is that I can't make a decision between the two because I have practically no experience. Some say MySQL is easier to use, some say PostgreSQL is more robust etc. Most articles seem to celarly favor one or the other. It is very hard to find an objective comparison. And I have to explain my boss why I chose either one. So perhaps some of you can give me some reasons why to choose MySQL over Postgre. Thanks, Kaarel [EMAIL PROTECTED] -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Which database?
On Tue, Jun 10, 2003 at 10:54:52AM -0400, Peter J. Milanese wrote: Hey Kaarel- I've been sticking with MySQL mostly for it's support. Large community, lots of documentation, and they have a future plan (which they tend to actually complete ahead of schedule). Wow. You must be the first person in the history of this mailing list to accuse them of getting development done ahead of time! :-) -- Jeremy D. Zawodny | Perl, Web, MySQL, Linux Magazine, Yahoo! [EMAIL PROTECTED] | http://jeremy.zawodny.com/ MySQL 4.0.13: up 7 days, processed 223,418,033 queries (351/sec. avg) -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Which database?
MySQL has the backing of many large corporations (for which MySQL was initially written). MySQL support/userbase will not go away that easily. They have a plan for enhancements. They accomplish these enhancements ahead of schedule. This is how I translate 'lack of commercial support company means you are unlikely to get the same level of support.' P Roman Neuhauser [EMAIL PROTECTED] 06/10/2003 09:42 AM To: [EMAIL PROTECTED] cc: Subject:Re: Which database? # [EMAIL PROTECTED] / 2003-06-10 14:11:39 +0100: (...) lack of commercial support company means you are unlikely to get the same level of support. unlikely? perhaps. but... how many OS programs do you use, how many of them have commercial support, and how do those that have compare to those that don't? is the help and expertise provided e. g. by [EMAIL PROTECTED] any worse than that of this list? I would disagree. -- If you cc me or remove the list(s) completely I'll most likely ignore your message.see http://www.eyrie.org./~eagle/faqs/questions.html -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Which database?
subselects were slated for release with 5.0 Not 4.1 P Jeremy Zawodny [EMAIL PROTECTED] 06/10/2003 11:21 AM Please respond to mysql To: Peter J. Milanese [EMAIL PROTECTED] cc: Kaarel [EMAIL PROTECTED], [EMAIL PROTECTED] Subject:Re: Which database? On Tue, Jun 10, 2003 at 10:54:52AM -0400, Peter J. Milanese wrote: Hey Kaarel- I've been sticking with MySQL mostly for it's support. Large community, lots of documentation, and they have a future plan (which they tend to actually complete ahead of schedule). Wow. You must be the first person in the history of this mailing list to accuse them of getting development done ahead of time! :-) -- Jeremy D. Zawodny | Perl, Web, MySQL, Linux Magazine, Yahoo! [EMAIL PROTECTED] | http://jeremy.zawodny.com/ MySQL 4.0.13: up 7 days, processed 223,418,033 queries (351/sec. avg) -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Which database?
there's a pretty good comparison on the MySQL website and they were quite fair about it. MySQL has been adding features slowly because, above all, they don't want to sacrifice performance for the feature. Each database has its strengths. The bottom line is that you need to figure out just what your requirements are and then see which database suites your needs. If you need to work with extremely large databases (multi GB) I would go with MySQL. It scales to large files extremely well. Curtis Maurand On Tue, 10 Jun 2003, Becoming Digital wrote: You're on the MySQL list, so expect biased answers. Both certainly have their merits. PostgreSQL is currently more advanced, at least if you go by their claims. Hoever, there seems to be much less support for PostgreSQL than MySQL, be it from books or other users. While I expect to use PostgreSQL at some point in time, if only for curiousity's sake, I'm glad I started with MySQL. Edward Dudlik Becoming Digital www.becomingdigital.com - Original Message - From: Kaarel [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, 10 June, 2003 07:38 Subject: Which database? Hi I am fairly new to databases, but my boss has assigned me to choose a database system for our company. I know of Oracle databases but they are way too expensive so it's not an option for us. I did some research and came up with two free database systems that seemed to be more popular - MySQL and PostgreSQL. The problem is that I can't make a decision between the two because I have practically no experience. Some say MySQL is easier to use, some say PostgreSQL is more robust etc. Most articles seem to celarly favor one or the other. It is very hard to find an objective comparison. And I have to explain my boss why I chose either one. So perhaps some of you can give me some reasons why to choose MySQL over Postgre. Thanks, Kaarel [EMAIL PROTECTED] -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED] -- -- Curtis Maurand mailto:[EMAIL PROTECTED] http://www.maurand.com -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Which database?
At 11:39 -0400 6/10/03, Peter J. Milanese wrote: subselects were slated for release with 5.0 Not 4.1 P The earliest that subselects were planned for that I know of was 3.23. Then 3.24 (during 3.23 development). 3.24 got renamed to 4.0 (I guess), and subselects didn't appear there, either. :-) Now we have them in 4.1. I'd say that subselects are not the classic example of a feature that was completed early. :-) Jeremy Zawodny [EMAIL PROTECTED] 06/10/2003 11:21 AM Please respond to mysql To: Peter J. Milanese [EMAIL PROTECTED] cc: Kaarel [EMAIL PROTECTED], [EMAIL PROTECTED] Subject:Re: Which database? On Tue, Jun 10, 2003 at 10:54:52AM -0400, Peter J. Milanese wrote: Hey Kaarel- I've been sticking with MySQL mostly for it's support. Large community, lots of documentation, and they have a future plan (which they tend to actually complete ahead of schedule). Wow. You must be the first person in the history of this mailing list to accuse them of getting development done ahead of time! :-) -- Jeremy D. Zawodny | Perl, Web, MySQL, Linux Magazine, Yahoo! [EMAIL PROTECTED] | http://jeremy.zawodny.com/ MySQL 4.0.13: up 7 days, processed 223,418,033 queries (351/sec. avg) -- Paul DuBois, Senior Technical Writer Madison, Wisconsin, USA MySQL AB, www.mysql.com Are you MySQL certified? http://www.mysql.com/certification/ -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
