> First recommendation: Less layers. Instead of SQLAlchemy, just import > sqlite3 and use it directly. You should be able to switch out "import > sqlite as db" for "import psycopg2 as db" or any other Python DB API > module, and still have most/all of the benefit of the extra layer, > without any extra imports.
Yeah, I am a fan of apsw but this is what I have now done. While I am fine with writing my own abstractions, its certainly less overhead using an orm. > Secondly, see if you can keep the Python process running, instead of > separately invoking it for every operation. This will mean some > structural changes, but if you're specifying Python as the plugin > interpreter (as opposed to, say, "execute this program with these > args", which is far more general), you can probably set up a wrapper > script that calls on the user code repeatedly. Perhaps you could have > a multiprocessing.Queue feeding tasks to the subprocess, via a little > parent Python process. If you can't make those kinds of changes to the > application, the next best would be a stubby driver process that feeds > jobs to your real subprocess; the real work might be done by a > Unix/TCP socket server, and the stub connects, feeds it some > information, gets back a result, and disconnects. The stub will have > an extremely minimal set of imports (the bear necessities of life, if > you like), and the real process can import as much as it likes, > because it won't be doing it for every operation. I am stuck with the current architecture, but the idea you propose has been thrown around, truth is I am not certain if we are enduring the effort of such a large rewrite that Python is the tool to use (this is a Windows application) but regardless your idea is one that is proposed. Thanks, jlc -- https://mail.python.org/mailman/listinfo/python-list
