On Wed, Jul 27, 2011 at 2:48 PM, Michael Bayer <mike...@zzzcomputing.com> wrote: > > On Jul 27, 2011, at 3:21 PM, Kent Tenney wrote: > >> Howdy, >> >> I'm aggregating data from several Sqlite files into a Postgres db. >> The sqlite files are storage for several apps I use: Shotwell, >> Firefox, Zotero, Banshee ... I just watch and pull from them. >> >> I've been using "import sqlite3" so far, dumping sql from sqlite, >> using it to create the Postgres tables. I then add columns to meet >> my own needs. I now can diff 2 sqlite files, so I know what rows >> need updating and adding in the Postgres tables. >> >> I feel I should be using Sqlalchemy, > > ok well what problems do you have currently that you'd like to solve ?
With some proof-of-concept working, looking ahead to increasing complexity, my sense is that SA can hide some complexity behind a layer which moves data into Postgres. I think this is what you suggest in your final paragraph. > > >> but have been intimidated by >> the wealth of choices SA offers. > > It offers "choices" in that you can A. use core only or B. the ORM, as well > as "choices" in that it works with whatever kind of schema you'd like, I'll be studying doc for a better idea of the A/B distinction, as well as what kinds of schemas there are. I have the O'Reilly book, I understand SA has changed considerably since then, am I better off sticking with current doc? which is the same "choice" you have anyway, and in this case it seems you have already made. Your app sounds like kind of a nuts and bolts table-to-table thing, i.e. is SQL centric, so using constructs like table.select() and table.insert() could perhaps reduce the verbosity of generating those statements by hand, the Table construct itself can turn the equation of "what columns am i dealing with here?" into a data driven one (the Table is a datastructure, which stores a list of Column objects - a data driven description of a schema). > >> >> - comfortable in Python, SQL not so much > > dont sell yourself short, you're moving rows and adding columns and that's a > fair degree of knowledge right there. > >> - pull into the Postgres db from other sources >> - file system content >> - email >> - other db's: Mysql, rdf, ... >> - feed Sphinxsearch from the Postgres db > > for all of these you'd probably want to figure out some intermediary format > that everything goes into, then goes to the database. Depending on how much > this format is hardwired to the SQL schema or not, as well as if you're > generally dealing with "one big table" to store a format versus many tables > storing a more normalized structure, would determine how well the ORM may or > may not be useful. The ORM is good when you have multiple tables in some > hierarchical structure that is to be related to an object hierarchy. For a > straight up "I'd like the columns in this Excel spreadsheet to become columns > in a new database table", it might be overkill. I appreciate the advice, I'll be back. Thanks, Kent > > > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to sqlalchemy@googlegroups.com. > To unsubscribe from this group, send email to > sqlalchemy+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
