Re: [Tutor] Which non SQL Database ?
On Sat, 4 Dec 2010, Jorge Biquez wrote: What would do you suggest to take a look? If possible available under the 3 plattforms. I would second the use of SQLite. It's built into Python now, on all platforms. But you specified non SQL, so one other thing I'd suggest is to just create the data structure you need in Python and use pickle to save it. I recently had an exercise of recovering files from a damaged hard drive. The problem is, it recovered a lot of legitimately deleted files along with the recovered live files. All the files had generic names, with only filetypes to guide me for content, like 028561846.avi instead of descriptive names. I wrote a program to read every single one of these files and determine its MD5 checksum; I stored the results in a dictionary. The key to the dictionary was the checksum; and the value was a list of files that had that checksum; the list was usually, but not always, only one element. Then I pickled that dictionary. In another program, I ran os.walk against my archive CDROMs/DVDRROMs, or some other directories on my hard drive, finding the MD5 of each file; and if it corresponded to a rescued file, it deleted the rescued file. Ideally, I would have also updated the dictionary to drop the files I'd cleaned up, and at the end of processing, re-pickle the edited dictionary; but that wasn't an option as I usually had 2 or 3 instances of the program running simultaneously, each processing a different directory of CD/DVD. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which non SQL Database ?
Am 04.12.2010 23:42, schrieb Jorge Biquez: Hello all. Newbie question. Sorry. As part of my process to learn python I am working on two personal applications. Both will do it fine with a simple structure of data stored in files. I now there are lot of databases around I can use but I would like to know yoor advice on what other options you would consider for the job (it is training so no pressure on performance). One application will run as a desktop one,under Windows, Linux, Macintosh, being able to update data, not much, not complex, not many records. The second application, running behind web pages, will do the same, I mean, process simple data, updating showing data. not much info, not complex. As an excersice it is more than enough I guess and will let me learn what I need for now. Talking with a friend about what he will do (he use C only) he suggest to take a look on dBase format file since it is a stable format, fast and the index structure will be fine or maybe go with BD (Berkley) database file format (I hope I understood this one correctly) . Plain files it is not an option since I would like to have option to do rapid searches. What would do you suggest to take a look? If possible available under the 3 plattforms. As a non SQL Database I can recommend MongoDB. It's a document store, which uses binary json as format. You can create secondary indexes and also perform more sophisticated queris. The main advantage from my point of view is the ease of use. Installation was a breeze and it has an easy to use python driver. Note that you need a 64-bit OS to use it with bigger data volumes (I remember 3 GB, but it's all on their website). Thanks in advance for your comments. Jorge Biquez ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which non SQL Database ?
Jorge Biquez jbiq...@icsmx.com wrote Talking with a friend about what he will do (he use C only) he suggest to take a look on dBase format file since it is a stable True enough and Dabo is worth a look since it provides not only a dbase format db engine but a good GUI builder tool too. However (Berkley) database file format (I hope I understood this one correctly) . Plain files it is not an option since I would like to have option to do rapid searches. Why not use SQL? SQLlite comes with Python, is small, easy to use and if necessary can be used in-memory and as such fast. Once you have decided to go down the database route then not using SQL nowadays is a very odd decision. Its rather liike going out looking for a car that has solid types instead of pneumatics. Why would you want to? HTH, -- Alan Gauld Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which non SQL Database ?
Am 05.12.2010 10:41, schrieb Alan Gauld: Jorge Biquez jbiq...@icsmx.com wrote Talking with a friend about what he will do (he use C only) he suggest to take a look on dBase format file since it is a stable True enough and Dabo is worth a look since it provides not only a dbase format db engine but a good GUI builder tool too. However (Berkley) database file format (I hope I understood this one correctly) . Plain files it is not an option since I would like to have option to do rapid searches. Why not use SQL? SQLlite comes with Python, is small, easy to use and if necessary can be used in-memory and as such fast. Once you have decided to go down the database route then not using SQL nowadays is a very odd decision. Its rather liike going out looking for a car that has solid types instead of pneumatics. Why would you want to? The NoSQL databases seem to be en vogue. Some big internet companies have started to use them, like Google with its Big Table. That's how I stumbled across them. There're a lot of discussions going on on the internet which is better for which use case. There're some cases where one kind of the database type is the clear winner, but most use cases can be modelled with both. My conclusion and what draw me finally to MongoDB over PostgreSQL and SQLAlqemy was: 1) Really the ease of use (from setting up the database and later when it comes to administration and scaling, which is particularly easy). MongoDB is very useful for prototyping. 2) I don't need complex transactions and the ability to perform complex, not in advanced known queries. I'd say SQL-DBs are more sophisticated but more complex to use. For the sake of learning, go with SQL. It doesn't hurt having a driver license. If you need to go to the bakery around the corner the driver license doesn't prevent you to jump on your bicycle. Cheers, Jan HTH, ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which non SQL Database ?
Knacktus knack...@googlemail.com wrote Why not use SQL? SQLlite comes with Python, is small, easy to use and if necessary can be used in-memory and as such fast. The NoSQL databases seem to be en vogue. Some big internet companies have started to use them, like Google with its Big Table. There are cases where it makes sense but they are nearly all cases where extreme performance is required for a small fixed set of queries. In these cases the data structures can be optimised to match the queries and low level file access can provide optimum access speed. But unless you absolutely know you need that a lightweight SQL database like SQLlite or firebird will be easier to configure, easier to query and be much more scaleable and flexible. (Plus you get for free the ability to run ad-hoc MIS style reports of the back of the SQL engine.) one kind of the database type is the clear winner, but most use cases can be modelled with both. Yes, its like OOP v procedural, both can be used for any problem but usually OOP is is the more powerful, flexibly and easier option. But for specific, small or high performance apps procedural can be better. My conclusion and what draw me finally to MongoDB over PostgreSQL and SQLAlqemy was: I've no idea what MongoDB is like but PostGrres is one of the more complex (and powerful) SQL options. Certainly not my first chooice for a small, single user app. 1) Really the ease of use (from setting up the database and later when it comes to administration and scaling, which is particularly easy). Most SQL databases come with GUI tools for all of that. MongoDB is very useful for prototyping. 2) I don't need complex transactions and the ability to perform complex, not in advanced known queries. Fixed in advance, simple, single dimensional queries are the area where non SQL can be a valid choice. Any kind of multi dimensional query will quickly become difficult to code and maintain, and thats where SQL wins. In my experience the majority of so-called simple database applications very quickly become complicated! Of course if all you need is data persistence then flat files are a perfectly valid option. But I'm assuming here that database implies off-disk data search and access. -- Alan Gauld Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Which non SQL Database ?
Hello all. Newbie question. Sorry. As part of my process to learn python I am working on two personal applications. Both will do it fine with a simple structure of data stored in files. I now there are lot of databases around I can use but I would like to know yoor advice on what other options you would consider for the job (it is training so no pressure on performance). One application will run as a desktop one,under Windows, Linux, Macintosh, being able to update data, not much, not complex, not many records. The second application, running behind web pages, will do the same, I mean, process simple data, updating showing data. not much info, not complex. As an excersice it is more than enough I guess and will let me learn what I need for now. Talking with a friend about what he will do (he use C only) he suggest to take a look on dBase format file since it is a stable format, fast and the index structure will be fine or maybe go with BD (Berkley) database file format (I hope I understood this one correctly) . Plain files it is not an option since I would like to have option to do rapid searches. What would do you suggest to take a look? If possible available under the 3 plattforms. Thanks in advance for your comments. Jorge Biquez ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which non SQL Database ?
On Sat, Dec 4, 2010 at 5:42 PM, Jorge Biquez jbiq...@icsmx.com wrote: Newbie question. Sorry. If it isn't you're on the wrong list :) training so no pressure on performance). One application will run as a desktop one,under Windows, Linux, Macintosh, being able to update data, not much, not complex, not many records. The important details here are: simple data, low-volume. I'm assuming this is single-user (as in, each instance of your application has it's own DB) The second application, running behind web pages, will do the same, Is this multiple users, each accessing the same DB? That really changes what you are looking for. If you are dealing with single-user, or only a few users, I'd say look into SQLite - It uses SQL syntax but doesn't run as a server and stores the database as a single file. It's great to use in small projects because the syntax is the same as larger projects, and you can replace with a full-blown multi-user SQL DB if you ever need to without having to rework everything. It's also very simple to use. I believe SQLite (sqlite3) is part of the core library in recent Python versions, or available as a package for older pythons. Berkeley DB is pretty much interchangeable with SQLite in terms of functionality. I much prefer SQLite. If your web application intends to have multiple users interacting with the same data, neither is probably a good fit. -- Brett Ritter / SwiftOne swift...@swiftone.org ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
