Re: [python-sqlite] Re: pysqlite 2.8.0 released
Yes, I forgot to "setup.py sdist upload". It's fixed now. Sorry for the
trouble.
I'm of course looking forward to hear if SQLAlchemy still works ok with
this release.
On Wed, Aug 19, 2015 at 10:10 PM, wrote:
> Hi Gerhard -
>
> is the download missing? On Pypi I see 2.8.0 is registered but no
> download file:
>
> https://pypi.python.org/pypi/pysqlite/2.8.0
>
> pip fails:
>
> $ ./bin/pip install pysqlite==2.8.0 --upgrade --force
> Collecting pysqlite==2.8.0
> Could not find a version that satisfies the requirement pysqlite==2.8.0
> (from versions: 2.5.6, 2.6.0, 2.6.3, 2.7.0)
> Some externally hosted files were ignored as access to them may be
> unreliable (use --allow-external to allow).
> No distributions matching the version for pysqlite==2.8.0
>
>
>
> On Tuesday, August 18, 2015 at 8:17:46 PM UTC-4, Gerhard Häring wrote:
>>
>> NEW FEATURES
>>
>> - No new features, but tons of bugfixes. These mean that things now work
>> that
>> didn't before:
>> - Transactional DDL now works
>> - You can use SAVEPOINTs now
>>
>>
>> BUILD PROCESS
>>
>> - Python 2.7.x is now required. If trying to use it with Python 3, print a
>> useful error message. Integrated all fixes from the sqlite3 module in
>> Python
>> 2.7.10.
>>
>>
>> MAJOR IMPROVEMENTS
>>
>> - Completety got rid of statement parsing. We now use SQLite functions to
>> determine if a statement modifies the database or not. If a statement
>> modifies the database, then we implicitly start a transaction. For
>> backwards
>> compatibility reasons, we do NOT implicitly start a transaction if we
>> encounter a DDL statement.
>>
>> You can, however, now have transactional DDL if you want to:
>>
>> cur = con.cursor()
>> cur.execute("begin")
>> cur.execute("create table foo(bar)")
>> con.rollback()
>>
>> This also means that people can now finally use SAVEPOINTS.
>>
>> - Use sqlite3_get_autocommit() to determine if we are within a transaction
>> instead of trying to be smart.
>>
>> - Switch to v2 statement API. This simplified the code and will increase
>> stability.
>>
>> MINOR IMPROVEMENTS
>>
>> - You can use unicode strings as index for Row objects.
>>
>>
>> BUGFIXES
>>
>> - Fixed a regression: statements should not be reset after a commit.
>>
>>
>> GENERAL CLEANUP AND DEPRECATIONS
>>
>> - Since december 2005, row_factory is a feature of the Connection class
>> instead
>> of the Cursor class. It was kept in the Cursor class for backwards
>> compatibility. Now it was time to finally remove it from the Cursor
>> class.
>> - DEPRECATE converters and adapters.
>> - DEPRECATE text_factory.
>> - Remove compatibility workarounds for old Python versions.
>> - Remove workarounds for old SQLite versions.
>> - Remove apsw related code.
>>
>> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "python-sqlite" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to python-sqlite+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
https://mail.python.org/mailman/listinfo/python-list
pysqlite 2.8.0 released
NEW FEATURES
- No new features, but tons of bugfixes. These mean that things now work
that
didn't before:
- Transactional DDL now works
- You can use SAVEPOINTs now
BUILD PROCESS
- Python 2.7.x is now required. If trying to use it with Python 3, print a
useful error message. Integrated all fixes from the sqlite3 module in
Python
2.7.10.
MAJOR IMPROVEMENTS
- Completety got rid of statement parsing. We now use SQLite functions to
determine if a statement modifies the database or not. If a statement
modifies the database, then we implicitly start a transaction. For
backwards
compatibility reasons, we do NOT implicitly start a transaction if we
encounter a DDL statement.
You can, however, now have transactional DDL if you want to:
cur = con.cursor()
cur.execute("begin")
cur.execute("create table foo(bar)")
con.rollback()
This also means that people can now finally use SAVEPOINTS.
- Use sqlite3_get_autocommit() to determine if we are within a transaction
instead of trying to be smart.
- Switch to v2 statement API. This simplified the code and will increase
stability.
MINOR IMPROVEMENTS
- You can use unicode strings as index for Row objects.
BUGFIXES
- Fixed a regression: statements should not be reset after a commit.
GENERAL CLEANUP AND DEPRECATIONS
- Since december 2005, row_factory is a feature of the Connection class
instead
of the Cursor class. It was kept in the Cursor class for backwards
compatibility. Now it was time to finally remove it from the Cursor class.
- DEPRECATE converters and adapters.
- DEPRECATE text_factory.
- Remove compatibility workarounds for old Python versions.
- Remove workarounds for old SQLite versions.
- Remove apsw related code.
--
https://mail.python.org/mailman/listinfo/python-list
Re: are there pros or contras, keeping a connection to a (sqlite) database ?
On Thu, Sep 9, 2010 at 12:29 AM, CM wrote: > [...] > I'm not even sure what a "connection" really is; I assumed it was > nothing more than a rule that says to write to the database with the > file named in the parentheses. [...] The following list is not exclusive, but these are the first things that I can think of. - SQLite connections have state like uncommitted transactions - SQLite connections have page caches and metadata caches (tables, structures and indices) Opening and closing SQLite connections is very cheap, but keeping connections open is usually a better approach. You have to have a "global" setting like the path to the database file anyway; so you can wrap a "global" connection object in a factory function just as well. In database applications that use the raw DB-APi i usually start with something like: def get_con(): # return database connection ... HTH -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: rich comparison fscks up reference count?
On Apr 12, 4:27 pm, Gerhard Häring wrote: > Maybe somebody can enlighten me here. I can't figure out why doing a > rich comparison on my object decreases the total reference count by 1. [...] Doh! It turned out the strange effect was due to my particular build process. My Python 2.6/3.1 are built using our own SCons-based build and the corresponding Makefile that distutils then uses didn't have the same DEFINEs. So, the Python was built with Py_REF_DEBUG, but Py_REF_DEBUG was not defined when I built my extension module. Which fscked up the *total* reference counting (sys.gettotalrefcount()). -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: rich comparison fscks up reference count?
Can be run like this: ghaer...@ws124~/src/gh/test$ python3 setup.py build_ext --inplace running build_ext building 'foo' extension gcc -fno-strict-aliasing -g -fwrapv -O0 -Wall -Wstrict-prototypes - arch i386 -m32 -I/opt/jetstream/include/python3.1 -c foo.c -o build/ temp.macosx-10.4-i386-3.1-pydebug/foo.o foo.c: In function ‘PyInit_foo’: foo.c:109: warning: return makes pointer from integer without a cast gcc -bundle -undefined dynamic_lookup -arch i386 -m32 build/ temp.macosx-10.4-i386-3.1-pydebug/foo.o -o /Users/ghaering/src/gh/test/ foo.so [50367 refs] ghaer...@ws124~/src/gh/test$ python3 setup.py test running test total refcount 65304 total refcount 65305 total refcount 65304 total refcount 65303 total refcount 65302 total refcount 65301 total refcount 65300 total refcount 65299 total refcount 65298 total refcount 65297 [45322 refs] ghaer...@ws124~/src/gh/test$ -- http://mail.python.org/mailman/listinfo/python-list
rich comparison fscks up reference count?
Maybe somebody can enlighten me here. I can't figure out why doing a rich comparison on my object decreases the total reference count by 1. Linked is the minimal test case with a C exension that compiles under both Python 2.6 and 3.1. No external dependencies, except a DEBUG build of Python to see the effect (Py_REF_DEBUG defined). http://cdn.ghaering.de/foo.c http://cdn.ghaering.de/setup.py Builtin types like list, dict, etc. don't seem to have the same effect. I've also run the test a few thousand times; then the total refcount is like -5, but Python does not crash. Also if I print all objects and their respective refcounts using the gc module, none of them seems to have a negative refcount, but the total refcount is still way negative. How could that be? -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
[ANN] pysqlite 2.6.0 released
pysqlite 2.6.0 released === Release focus: Synchronize with sqlite3 module in Python trunk. pysqlite is a DB-API 2.0-compliant database interface for SQLite. SQLite is a in-process library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine. Go to http://pysqlite.googlecode.com/ for downloads, online documentation and for reporting bugs. Changes === - pysqlite can now be built against Python versions without thread support - PyDict_SetItem is now checked so you cannot shoot yourself in the foot, even if you try to - More checks for closed connections were added to avoid possible segfaults Compatibility = Warning: pysqlite is not supported for Python < 2.5 any longer. This release still builds against Python 2.3 and 2.4, but I won't bother with these in the future. -- http://mail.python.org/mailman/listinfo/python-list
Building Python with Scons
I'm setting up a third-party library project (similar to the one in Google Chromium) where I use SCons as build tool. Now I need to integrate Python, too. Has anybody written a Scons script for Python 2.x or 3.x, yet? -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Why Doesn't XP Pro Show Size, Time and Date Mod of a Created File?
W. eWatson wrote:
> I created a folder, and wrote a file to it. When I look at what files
> are in it, they are correct. However, The Size, Type, and Date Mod are
> not shown. Why am I missing those columns? I'm writing files with a
> suffix of dat, which seem only to match up with video CD movie.
That's probably a Windows Explorer thing. The column may be hidden or
moved away to the far right.
As far as Python is concerned, you can fetch this kind of information
with the stat() function of the os module.
import os, time
stat_info = os.stat("x")
print "size:", stat_info.st_size
print "modification time:", time.strftime("%Y-%m-%d %H:%M",
time.localtime(stat_info.st_mtime))
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: editor with autocompletion
Siva B wrote: > Hi friends, > > I am writing a new language. > So I want an editor with auto complete. > I there any such tool in Python ?(not only in python any other) > I want it for my new lang IDLE, the Integrated Development Environment included with your Python installation nowadays has autocompletion (I just check with the one included in Python 2.6). -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: dbapi2 select where IN (...)
yota.n...@gmail.com wrote:
> hello,
>
> I couldn't find how the dbapi2 planned to handle the sql IN statement.
>
> ex :
> SELECT * FROM table WHERE num IN (2,3,8,9);
>
> I'd be glad to take advantage of the ? mechanism, but what about
> tuples !
>
> execute("""SELECT * FROM table WHERE num IN ?;""" ,
> ((2,3,8,9),))...fail... [...]
You cannot use parameter binding when the number of parameters is
unknown in advance. So you'll have to create this part of the SQL query
differently.
For example:
ids = [2, 3, 8, 9]
in_clause = " (" + ",".join([str(id) for id in ids]) + ")"
query = "select * from table where num in" + in_clause
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: IDE+hg
Rhodri James wrote: > On Mon, 23 Nov 2009 19:20:27 -, NiklasRTZ wrote: > >> Dear experts, >> Since no py IDE I found has easy hg access. IDEs PIDA and Eric claim >> Mercurial support not found i.e. buttons to clone, commit and push to >> repositories to define dev env dvcs, editor and deployment all in 1. > > I don't really understand this urge to cram everything into a single > program, since that inevitably leads to compromises that will compromise > just how much of Mercurial's useful and interesting functionality you > can get at. Still, if you really must, Emacs (and presumably vim) seems > to be capable of working with most source control systems. I prefer the commandline tools, too. FWIW, Eclipse supports Mercurial through http://www.vectrace.com/mercurialeclipse/ -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Python & OpenOffice Spreadsheets
Is there a *simple* way to read OpenOffice spreadsheets?
Bonus: write them, too?
I mean something like:
doc.cells[0][0] = "foo"
doc.save("xyz.ods")
>From a quick look, pyodf offers little more than just using a XML parser
directly.
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: python and web pages
Daniel Dalton wrote: > Hi, > > Here is my situation: > I'm using the command line, as in, I'm not starting gnome or kde (I'm on > linux.) > I have a string of text attached to a variable,. So I need to use one of > the browsers on linux, that run under the command line, eg. lynx, > elinks, links, links2 and do the following. No, you don't need any text mode browser. It's much easier to do with modules from the Python standard library. > 1. Open a certain web page and find the first text box on the page, and > put this text into the form. I assume you know HTML and HTTP. You can use urllib or urllib2 to submit the form. > 2. Press the submit button, and wait for the result page to load. > 3. Click on the 15th link down the page. You will then get the returned HTML from a function and you will have to parse it for the link you're interested in. There are many approaches to get there. Manual parsing, regular expressions or one of the XML parsers in the standard library (etree is the easiest). > So, how do I do this, can anyone point me to some docs or modules that > may help out here? While all of this may seem overwhelming at first, I guess the solution can be written in 20 - 30 lines of code. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Pyfora, a place for python
Lorenzo Gatti wrote: > On Nov 1, 8:06 am, Saketh wrote: >> Hi everyone, >> >> I am proud to announce the release of Pyfora (http://pyfora.org), an >> online community of Python enthusiasts to supplement comp.lang.python >> and #python. While the site is small right now, please feel free to >> register and post any questions or tips you may have. > > I'll feel free to not even bookmark it. I'm sorry, but it is just a > bad idea. [...] I agree. > Your forum cannot (and should not) compete either with Python's > official newsgroup, IRC channel and mailing list or with popular, well- > made and well-frequented general programming sites like > stackoverflow.com. [...] The good thing is, unless something the announced new forum gets critical mass, it will just slowly (or not-so-slowly die). But even though I'm an old-timer who still prefers newsgroups/mailing lists, I think that there should be something better, browser based. In particular supporting moderation/voting and tagging/filtering. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Python: Text file insert to MySQL
On Wed, Oct 7, 2009 at 1:32 PM, Schedule wrote:
> That was great ! Now I am able to insert the values from the file.
>
> Somehow I am not able to update a specific field with all the vaues in the
> file. For eg:
> [...]
> c.execute("UPDATE a SET last = %s", row)
The database does what you say, not what you mean ;-)
SQL commands like SELECT, UPDATE, DELETE work on sets. If you don't
qualify the set any further, you always operate on the whole set.
To restrict the set, apply a WHERE clause. I. e.
Perhaps you should try a SQL tutorial before jumping into SQL +
Python. Here's one:
http://sqlzoo.net/
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python: Text file insert to MySQL
Schedule wrote:
> Hello,
>
> I am currenty using MySQL 5.1 community server and trying to import the
> data of the comma delimited text file into the table using python 2.6
> scripts. I have installed Mysqldb 1.2.2.
>
> follwoing is my script:
> [...]
>7.
> c.execute("INSERT INTO a (first, last) VALUES (%s, %s), row")
> [...]
> When I execute the statement I get the following error:
>
> [...]
> _mysql_exceptions.ProgrammingError: (1064, "You have an error in your
> SQL syntax; check the manual tha
> t corresponds to your MySQL server version for the right syntax to use
> near '%s, %s), row' at line 1")
You misplaced the closing quote.
wrong: c.execute("INSERT INTO a (first, last) VALUES (%s, %s), row")
correct: c.execute("INSERT INTO a (first, last) VALUES (%s, %s)", row)
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: Compile kinterbasdb with mingw32 and python 2.6 - DLL load failed
Laszlo Nagy wrote: > >>> The building and installation went find. But I cannot "import >>> kinterbasdb" >>> because I get a "DLL load failed" error. I figured out that has >>> something to >>> do with msvcr90 and "_ftime". Can you please give me some advice how to >>> solve this problem? >>> >> >> Download Microsoft Visual C++.2008 Express Edition >> > > Its license does not allow us to create services for internet if we > charge fees. At least this is what I was told. Does this mean that using > kinterbasdb (compiled with MSVC express) I'm forbidden to write > commercial internet services? :-s I see three options: 1) Ask a laywer. 2) Download the MSVC express compiled version from somewhere else. ; adhere to the kinterbasdb license; let MS license issues not be your problem - or at least now having "plausible deniability". 3) F*ck license issues. You're not redistributing anything, who will know what software compiled with which compilers you're using? Well, or just use mingw. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: file comparison
learner learner wrote: > Hi all, > > I want to compare two text files line by line and eliminate the > matching/repeated line and store the unmatched/leftout lines into a > third file or overwrite into one of them. gl & hf! -- http://mail.python.org/mailman/listinfo/python-list
Re: non-owning references?
Utpal Sarkar wrote: > Hi, > [...] You're looking for the weakref module. What you're describing there sounds like a nice exercise, but I cannot imagine why you'd really need to clean it up, if it really is a singleton. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating xml
Greg Lindstrom wrote: > It's been a while since I've played with XML using Python but I've been > asked to create XML using data from our postgres database. Currently we > are generating XML directly from the database using a set of stored > procedures but it is too slow (yes, I have numbers). I have been asked > to create a routine to generate the XML to see if it will be faster that > the sprocs (which I think it will be, based on other routines we have > running). But I digress... I write to a cStringIO.StringIO instance to create XML. Like this (pseudo-code): class Foo: def toxml(self, stream): print >> stream, "" If you're concerned about producing valid XML, you can write tests ;-) > Using my favorite search engine I see that there is a lot of material on > Python and XML. At the risk of starting something (which is not my > intention), what are my options if I want to create xml? Ceratinly > writing my own routine is an option, but I bet there are better ones :-) > > How about if I need/want to parse or process an XML file? Use lxml. It's blazingly fast and its XPath support makes parsing XML a snap. If you don't want to depend on anything not in the standard library, use the etree module. Code will be slightly longer because you don't have XPath support and performance will be only "ok". Not "super-fast", like with lxml. HTH -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: [RELEASED] Python 3.1 final
Scott David Daniels wrote: > Nobody wrote: >> On Sat, 27 Jun 2009 16:12:10 -0500, Benjamin Peterson wrote: >> >> >> That's a significant improvement >> All in all, Python 3.x still has a long way to go before it will be >> suitable for real-world use. > > Fortunately, I have assiduously avoided the real word, and am happy to > embrace the world from our 'bot overlords. > > Congratulations on another release from the hydra-like world of > multi-head development. +1 QOTW -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Column types with DB API
Thomas Robitaille wrote: > Hi, > > I'm trying to use DB API compliant database modules > (psycopg2,MySQLdb,SQLite) to access SQL databases, and I am trying to > determine the type of each column in a table. The DB API defines > cursor.description which contains information about the column names and > types (once .execute() has been used to select part or all of the table), > but the types are expressed as integers. I realize that these numbers can be > checked against the type objects such as MySQLdb.NUMBER to determine whether > a given column is a number, a date, text, etc. However, is there any way to > determine what the sub-type of a column is, for example if a column is a > NUMBER, is there a way to determine if it is a 2, 4, or 8-byte integer or > real number? Not with the DB-API. You'll have to implement all this for each database separately. For some databases (like SQLite) it's even impossible. I suggest you don't bother with cursor.description at all. Depending on it doesn't harmonize with the way Python is typically used: duck typing. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: sqlite3, OperationalError: no such column, shouldn't that ne a ProgrammingError?
John Machin wrote: > Hi Gerhard, > [...] > In http://www.sqlite.org/c3ref/prepare.html it says """When an error > occurs, sqlite3_step() will return one of the detailed error codes or > extended error codes. The legacy behavior was that sqlite3_step() > would only return a generic SQLITE_ERROR result code and you would > have to make a second call to sqlite3_reset() in order to find the > underlying cause of the problem. With the "v2" prepare interfaces, the > underlying reason for the error is returned immediately.""" > > Are you using sqlite3_prepare() or sqlite3_prepare_v2()? > sqlite3_errcode() or sqlite3_extended_errcode()? Because of compatibility with older SQLite versions, I'm using the old API only. I'll change this one day and then I'll be able to throw a lot of compatibility code out of the window, too. But I certainly won't go #ifdef hell and implement both. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: sqlite3, OperationalError: no such column, shouldn't that ne a ProgrammingError?
Gabriel Rossetti wrote: > Hello everyone, > > I get an OperationalError with sqlite3 if I put the wrong column name, > but shouldn't that be a ProgrammingError instead? I read PEP 249 and it > says : > > "OperationalError >Exception raised for errors that are related to the >database's operation and not necessarily under the control >of the programmer, e.g. an unexpected disconnect occurs, >the data source name is not found, a transaction could not >be processed, a memory allocation error occurred during >processing, etc. It must be a subclass of DatabaseError. > ProgrammingError >Exception raised for programming errors, e.g. table not >found or already exists, syntax error in the SQL >statement, wrong number of parameters specified, etc. It >must be a subclass of DatabaseError. > " > > and to me it sounds more like a programming error than an operational > error. I agree. But I can't help you there. See _pysqlite_seterror() at http://oss.itsystementwicklung.de/trac/pysqlite/browser/src/util.c What I do is map SQLite error codes to DB-API exceptions. And SQLite reports your error as generic SQLITE_ERROR, which is mapped to OperationalError. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Observer implementations
Tobias Weber wrote: > Hi, > how to use the Observer pattern in Python? Implement it in your classes? > I found PubSub and PyDispatcher, both of which are abandoned. [...] I haven't searched for these, but googling for "python observer pattern" yields http://code.activestate.com/recipes/131499/ and this seems like a good inspiritation for owns own implementation. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: command prompt: the ntvdm cpu has encountered an illegal instruction
Daniel wrote: > If I try to invoke python via the command prompt I get an error > "command prompt: the ntvdm cpu has encountered an illegal > instruction..." > > I don't get this problem if I first cd to the python directory. I am > running python 3.0 on windows. Running Python from the Cygwin shell? Try from outside Cygwin, then. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Learning C++ for Python Development
joshua.pea...@gmail.com wrote: > I am a recovering C# web developer who has recently picked up Django > and I'm loving it. > > I would eventually like to get a job as a Django/Python developer. It > seems that many Python jobs require that you also be a C++ developer. I've seen the C++/Python combination in job descriptions, too. ISTM that these are about systems that are written in C++ and then scripted in Python. For speeding up Python applications. > While I want to remain primarily a web developer, I don't want be > stuck doing CRUD applications, so I would like to learn C++ for Python > development. In the web development field, the probability for doing custom C/C++ development is very low in my experience. > I have taken two basic programming courses in straight up > C++, no STL, Boost or anything like that, but I do have a very basic > knowledge of the language. Learning progamming languages never hurts, but I'd recommend you learn C instead, perhaps by using the Python C API (writing small extension modules). There are not so many Python extensions written in C++, most are written in C. C++ usually doesn't buy you enough to warrant using it instead of C here. So my advice: go with C. Or learn something more relevant to web programming. Like improve your JavaScript skills ;-) > [...] -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: update python version
km wrote: > Hi all, > > Is there a way to update python 2.6.1 to 2.6.2 using easy_install ? No, easy_install installs Python packages. It doesn't upgrade Python itself. If this is Windows, just install the newer Python version. No need to uninstall the 2.6.1 first. If this is some Unix variant, hopefully they will provide updated Python packages soon. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: configuring python with disable-thread in Ubuntu
Deepak Chandran wrote: > Hello, > > I am embedding python inside a C++ program. For some reason (I think > libxml2), I am getting Segmentation fault at PyThread_release_lock. > > The solution I found online was to configure python with --disable-thread. That doesn't sound like a solution, but like a kludge. > I used "apt-get install python-dev" to install python. > How do I re-configure it using the --disable-thread option? You don't want your system Python to be built with "--disable-thread". Other utilities might depend on threads being available in Python. If you really want to go this route, download Python from python.org and build it from source with the required options to "./configure". -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Web framework for embedded system
Thomas Heller wrote: > I'm looking for a lightweight web-framework for an embedded system. > The system is running a realtime linux-variant on a 200 MHz ARM > processor, Python reports a performance of around 500 pystones. > > The web application will not be too fancy, no databases involved > for example, but it will need to control some simple peripherals > (some parallel in/out, some dacs) attached to the system. > > It should provide a user interface showing the current state and > allowing to manipulate it via a browser, and I will probably need > to support some rpc as well. > > Does this sound sensible at all? Any suggestions? I'd try first what I know best. In my case Django. Despite the restricted hardware, it might just work fine because you'll only use a tiny fraction of it. If this doesn't work because of CPU or memory requirements, I'd next try to build my own using components like - CherryPy for the web server, it should also be able to handle your (XML)RPC requirements - some templating language like Mako (my favourite) It doesn't sound like you need much more. You could even build upon wsgiref, string.Template and the XMLRPC stuff in the standard library, but that's for people who like tinkering ;-) -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: value error
Francesco Pietra wrote:
> hi:
> with script
>
> data = open('134-176_rectified_edited.pdb', 'r')
> outp = open('134-176_renumbered.pdb', 'w')
>
> for L in data:
>if L[3] == 'M':
> L = L[:24] + "%4d" % (int(L[24-28])+133) + L[28:]
>outp.write(L)
>
>
> i wanted to modify lines of the type:
> ATOM 1 HH31 ACE 1 1.573 1.961 0.769 1.00 0.00 H
>
> to add 133 to column 25, getting 134 there, and so on for next lines 2
> -> 135, 3 -> 136, etc.
>
>
> i must have heavily messed things because the file was not even read:
>
> $ python renumber.py 134-176_rectified.pdb
> Traceback (most recent call last):
> File "renumber.py", line 6, in
> L = L[:24] + "%4d" % (int(L[24-28])+133) + L[28:]
> ValueError: invalid literal for int() with base 10: ''
Instead of L[24-28] you want L[24:28]. Otherwise it's L[-4] ;-)
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: Memory problems (garbage collection)
Here's a link for you: http://wiki.python.org/moin/PythonSpeed/PerformanceTips which also talks about string concatenation and othere do's and don'ts. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory problems (garbage collection)
Carbon Man wrote:
> Very new to Python, running 2.5 on windows.
> I am processing an XML file (7.2MB). Using the standard library I am
> recursively processing each node and parsing it. The branches don't go
> particularly deep. What is happening is that the program is running really
> really slowly, so slow that even running it over night, it still doesn't
> finish.
> Stepping through it I have noticed that memory usage has shot up from 190MB
> to 624MB and continues to climb.
That sounds indeed like a problem in the code. But even if the XML file
is only 7.2 MB the XML structures and what you create out of them have
some overhead.
> If I set a break point and then stop the
> program the memory is not released. It is not until I shutdown PythonWin
> that the memory gets released.
Then you're apparently looking at VSIZE or whatever it's called on
Windows. It's the maximum memory the process ever allocated. And this
usually *never* decreases, no matter what the application (Python or
otherwise).
> [GC experiments]
Unless you have circular references, in my experience automatic garbage
collection in Python works fine. I never had to mess with it myself in
10 years of Python usage.
> If I have the program at a break and do gc.collect() it doesn't fix it, so
> whatever referencing is causing problems is still active.
> My program is parsing the XML and generating a Python program for
> SQLalchemy, but the program never gets a chance to run the memory problem is
> prior to that. It probably has something to do with the way I am string
> building.
Yes, you're apparently concatenating strings. A lot. Don't do that. At
least not this way:
s = ""
s += "something"
s += "else"
instead do this:
from cStringIO import StringIO
s = StringIO()
s.write("something")
s.write("else")
...
s.seek(0)
print s.read()
or
lst = []
lst.append("something")
lst.append("else")
print "".join(lst)
> My apologies for the long post but without being able to see the code I
> doubt anyone can give me a solid answer so here it goes (sorry for the lack
> of comments): [...]
Code snipped.
Two tips: Use one of the above methods for concatenating strings. This
is a common problem in Python (and other languages, Java and C# also
have StringBuilder classes because of this).
If you want to speed up your XML processing, use the ElementTree module
in the standard library. It's a lot easier to use and also faster than
what you're using currently. A bonus is it can be swapped out for the
even faster lxml module (externally available, not in the standard
library) by changing a single import for another noticable performance
improvement.
HTH
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: and [True,True] --> [True, True]?????
Martin v. Löwis wrote: >> Are they widespread? I haven't noticed, yet. >> >> I prefer to write it explicitly: >> >> if len(lst) > 0: > > I prefer to test explicitly for the truth value of the > list. I don't want to test whether the length of the list > is greater than 0 (in fact, I don't care about the length > property of the list at all) - I want to know whether the > list is empty (or not empty). The Python syntax for this > test is > > if lst: > # not empty > > or > > if not list: > #empty > [...] You're right - as most of the time ;-) This makes a lot of sense to me. The reason I preferred len(), btw., was only that len() make it clear that the argument is a sequence. Maybe I was just too annoyed by lots of Python code I read that looked like this: def foo(x, y, z): if x: ... else: ... with poorly named variables where I didn't know what the heck the variables are (bool, list, instance, ...). I hate it when I have to look for the actual method calls to figure out what's going in. Better variable naming and small comments would often help. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: when can i expect libraries and third party tools to be updated for python 3 ?
Deep_Feelings wrote: > every one is telling "dont go with python 3 , 3rd party tools and > libraries have no compitability with python 3" > > so from previous experience : when can i expect libraries and third > party tools to be updated for python 3 ? (especially libraries ) The problem is: there is no previous experience. At least not in Python-land. Despite its major number, Python 2.0 was not disruptive. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: and [True,True] --> [True, True]?????
Peter Otten wrote: > bdb112 wrote: > >> Your explanation of Boolean ops on lists was clear. >> It leads to some intriguing results: >> >> bool([False]) >> --> True >> >> I wonder if python 3 changes any of this? > > No. Tests like > > if items: >... > > to verify that items is a non-empty list are a widespread idiom in Python. > They rely on the behaviour you observe. Are they widespread? I haven't noticed, yet. I prefer to write it explicitly: if len(lst) > 0: ... if item is None: ... etc. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
WHIFF - was: Re: Using Python after a few years of Ruby
Aaron Watters wrote: > On Apr 15, 3:49 am, Tim Hoffman wrote: > >> There are plenty of python web frameworks, some have quite different >> approaches, >> what suits you will depend very much on your own bias, interest. > > I've had a lot of luck with WHIFF > ( http://whiff.sourceforge.net ) > Of course I wrote it and just released it publicly > recently. > > I'm a bit disappointed because I know a lot of people have looked > at the docs and demos and there have been a number of downloads, > but I haven't gotten one comment yet :( zilch nada. (There is > one comment attached to one of the docs, but that was me testing > the comments feature :) ). > > Back in the old days you could at least count on someone yelling > at you about how your design was such a bad idea [Just read the docs for ca. 5 minutes.] That's perhaps because a lot of people had the same thought after short reading/trying out as me: WTF?! This is weird stuff! Why the hell would I use this instead of a Python web framework like Django/Pylons/etc. You should perhaps contrast WHIFF with the other offerings for creating web applications. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: [ANN] Falcon - powering innovation
Kless wrote: > If anybody is interesed in new technologies, you'll love this new > language called Falcon [1], which has been open sourced ago little > time. > > Falcon is a scripting engine ready to empower mission-critical > multithreaded applications. "Mission-critical" and "empower" sound like a good start for bullshit bingo :-P > It provides six integrated programming > paradigms: procedural, object oriented, prototype oriented, > functional, tabular and message oriented. You use what you prefer. I've never heard of tabular programming before. > To know more about its design, read the interview to the Falcon author > that has published ComputerWorld Australia [2]. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: C API String Parsing/Returning
k3xji wrote:
> Hi all,
>
> This might be a newbie question. I am trying to implement a simple
> string decoder/encoder algorithm. Just suppose I am substrcating some
> values from the string passed as a parameter to the function and I
> want the function to return encoded/decoded version of the string.
>
> Here is the call:
> ss= esauth.penc('s')
> st = esauth.pdec(ss)
>
> static PyObject *
> pdec(PyObject *self, PyObject *args)
> {
> unsigned char *s= NULL;
>
> unsigned int v,len,i = 0;
>
> if (!PyArg_ParseTuple(args, "s", &s))
> return NULL;
> if (!s)
> return NULL;
These two lines are superfluous. s now points to the contents of the
Python string (which must not contain any 0 characters, else a TypeError
is raised instead). Python strings are immutable, so you should *not
modify this C string*.
> len = strlen(s);
>
> for(i=0;i if (s[i] > 10)
> s[i] = s[i] - 10;
> }
>
> return Py_BuildValue("s",s);
> }
>
>
> This is returning the original string. I mean the parameter is changed
> but the Py_BuildValue is returning the original string passed in as
> param. [...]
Yes, that's because you're returning a Python string from the string
passed in ;-)
You should do something else instead:
char* buf = strdup(s);
if (!buf) {
PyErr_SetString(PyExc_MemoryError, "Out of memory: strdup failed");
return NULL;
}
/* TODO: your string manipulation */
return PyString_FromString(buf); /* return Py_BuildValue("s", buf); */
If you want to cope with Python strings that may contain 0 bytes, parse
them with "s#" instead. This should normally be better because you avoid
the strlen() this way.
HTH
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: group several methods under a attribute
jelle wrote: > Hi, > > I'm working on a pretty large class Can you describe what its purpose is? > and I'd like to group several methods under a attribute. That doesn't work in Python without bending the Python. > Its not convenient to chop up the class in several smaller classes, But that's almost certainly the right thing to do. > nor would mixins really solve the issue. > So, what is a pythonic way of grouping several methods under a > attribute? Whatever it is, you should find a better way instead of cramming everything into a single class. That smells of the God Object antipattern (http://en.wikipedia.org/wiki/God_object). -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: HTTP Authentication
Lakshman wrote:
> Whats is the python urllib2 equivallent of
>
> curl -u username:password status="abcd" http://example.com/update.json
>
> I did this:
>
> handle = urllib2.Request(url)
> authheader = "Basic %s" % base64.encodestring('%s:%s' % (username,
> password))
> handle.add_header("Authorization", authheader)
>
> Is there a better / simpler way?
Better? Yes.
Simpler? No.
Actually, the proper way using the urllib2 API is more code.
When I need it some time ago, I googled and used this recipe:
http://www.voidspace.org.uk/python/articles/urllib2.shtml#id6
# create a password manager
password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
# Add the username and password.
# If we knew the realm, we could use it instead of ``None``.
top_level_url = "http://example.com/foo/";
password_mgr.add_password(None, top_level_url, username, password)
handler = urllib2.HTTPBasicAuthHandler(password_mgr)
# create "opener" (OpenerDirector instance)
opener = urllib2.build_opener(handler)
# use the opener to fetch a URL
opener.open(a_url)
# Install the opener.
# Now all calls to urllib2.urlopen use our opener.
urllib2.install_opener(opener)
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: Cannot find text in *.py files with Windows Explorer?
John Machin wrote: > On Apr 4, 3:21 pm, John Doe wrote: >> Anybody have a solution for Windows (XP) Explorer search not finding >> ordinary text in *.py files? >> > > Get a grep on yourself! > > http://gnuwin32.sourceforge.net/packages/grep.htm There's something even better: "ack -- better than grep, a power search tool for programmers" http://betterthangrep.com/ -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Compatible JS Lib for Django
Daniel Fetchinson wrote: >> Does anyone have experience with using JS Libraries with Django? >> Do some work better than others and are easier to code with? > > You might want to ask this on the django list. Or on a JavaScript list ;-) It doesn't matter much in what context you use the JavaScript library (Django, Ruby on Rails, PHP, ...). FWIW I'm using JQuery with great success. It's really amazing what you can do with it with very little code. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: PyPy Progress (and Psyco)
andrew cooke wrote: This looks very promising - http://www.voidspace.org.uk/python/weblog/arch_d7_2009_03_14.shtml#e1063 I am really looking forwards to PyPy having a final release. I hope it happens. Me too. I doubt it, though. From an outside view, the project seems to lack focus. To me, it looks like a research platform, and producing a successor to CPython seems to be just one out of a dozen projects. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
[ANN] pysqlite 2.5.2
pysqlite 2.5.2 released
===
Release focus: minor bugfixes, minor new features.
pysqlite is a DB-API 2.0-compliant database interface for SQLite.
SQLite is a in-process library that implements a self-contained,
serverless, zero-configuration, transactional SQL database
engine.
Go to http://pysqlite.org/ for downloads, online documentation and
for reporting bugs.
Changes
===
- Like on Connection.rollback(), Connection.commit() now resets
all statements associated to the connection, so that the
commit() should always succeed (unless other connections create
trouble).
- pysqlite used to deliver bogus results on cursors that still
have unfetched data when a rollback() was done on the
connection. A commit() or rollback() now puts the cursor into a
"reset" state. If you try to fetch data after commit() or
rollback() you will now get an InterfaceError exception instead
of bogus data.
- For better DB-API compliance, operations on closed cursors now
raise exceptions.
- Add amalgamation directory to include path when building
statically against amalgamation files.
Otherwise, building against the amalgamation only worked if you
already had the SQLite3 header files in the search path. Like
on my development system ;-)
- Made sure HAVE_LOAD_EXTENSION is not defined twice. Also made
sure that it's OFF if the OMIT macro is defined.
- Fixed check if non-UTF8 strings are acceptable input. The check
was wrong for OptimizedUnicode. Also added the missing tests
for this feature.
- Wrap routine sqlite3_load_extension as method load_extension of
the Connection object.
Compatibility
=
The fact that commit() and rollback() now reset all associated cursors
changes
the behaviour of pysqlite. Some code that previously worked will now raise
InterfaceError exceptions. OTOH the code did not really *work*, because it
produced wrong results.
In previous pysqlite versions:
>>> from pysqlite2 import dbapi2 as sqlite3
>>> con = sqlite3.connect(":memory:")
>>> con.executescript("create table t(c); insert into t(c) values (1);
insert into t(c) values (2); insert into t(c) values (3);")
>>> cur = con.cursor()
>>> cur.execute("insert into t values (4)")
>>> cur.execute("select * from t")
>>> con.rollback ()
>>> print cur.fetchall ()
[(1,), (1,), (2,), (3,)]
^ ^
!! Notice the duplicate rows with values (1,) !! This code produced
wrong results.
With this release, the last cur.fetchall() will raise an exception:
>>> print cur.fetchall ()
Traceback (most recent call last):
File "", line 1, in
pysqlite2.dbapi2.InterfaceError: Cursor needed to be reset because of
commit/rollback and can no longer be fetched from.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Display a list using PyQt
member Basu wrote:
> I'm writing an application with PyQt as the GUI toolkit. I have a
> function that returns a simple list and I would like to update a List
> View widget with it. However I can't get it to work. I'm not sure if I
> need to subclass one of the Model classes, because it is just a list.
> Can someone point me to a simple tutorial or give me instructions on how
> to go about doing this?
The attached example should help. Also see the examples included with PyQt.
-- Gerhard
from PyQt4 import QtCore, QtGui
import sys
class MyDialog(QtGui.QDialog):
def __init__(self, parent):
super(QtGui.QDialog, self).__init__(parent)
self.layout = QtGui.QVBoxLayout(self)
self.resize(400, 300)
self.listWidget = QtGui.QListWidget(self)
self.layout.addWidget(self.listWidget)
buttonBox = QtGui.QDialogButtonBox(self)
buttonBox.setStandardButtons(QtGui.QDialogButtonBox.Ok)
self.layout.addWidget(buttonBox)
QtCore.QObject.connect(buttonBox, QtCore.SIGNAL("accepted()"), self.accept)
def set_list(self, lst):
for item in lst:
listItem = QtGui.QListWidgetItem(str(item), self.listWidget)
def accept(self):
selected_items = ", ".join([str(item.text()) for item in self.listWidget.selectedItems()])
print "selected:", selected_items
self.close()
if __name__ == "__main__":
app = QtGui.QApplication(sys.argv)
dlg = MyDialog(None)
lst = [3,4,5]
dlg.set_list(lst)
dlg.exec_()
--
http://mail.python.org/mailman/listinfo/python-list
Re: Which core am I running on?
psaff...@googlemail.com wrote:
> On 9 Feb, 12:24, Gerhard Häring wrote:
>> http://objectmix.com/python/631346-parallel-python.html
>>
>
> Hmm. In fact, this doesn't seem to work for pp. When I run the code
> below, it says everything is running on the one core.
>
> import pp
> import random
> import time
> from string import lowercase
>
> ncpus = 3
>
> def timedCharDump(waittime, char):
> time.sleep(waittime)
> mycore = open("/proc/%i/stat" % os.getpid()).read().split()[39]
> print "I'm doing stuff!", mycore, char
> return char
>
> job_server = pp.Server(ncpus, ppservers=())
>
> jobdetails = [ (random.random(), letter) for letter in lowercase ]
>
> jobs = [ job_server.submit(timedCharDump,(jinput1, jinput2), (),
> ("os", "time",)) for jinput1, jinput2 in jobdetails ]
>
> for job in jobs:
> print job()
Quick look again found this:
http://brokestream.com/procstat.html
I guess I counted wrong and it's actually column 38, not 39.
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: Which core am I running on?
psaff...@googlemail.com wrote: > Is there some way I can get at this information at run-time? I'd like > to use it to tag diagnostic output dumped during runs using Parallel > Python. There should be a way, but not with the Python standard library. It's also platform-specific. What are you using? Linux, MacOS X, FreeBSD, Solaris, or maybe Windows? -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Which core am I running on?
psaff...@googlemail.com wrote: > Is there some way I can get at this information at run-time? I'd like > to use it to tag diagnostic output dumped during runs using Parallel > Python. Looks like I have answered a similar question once, btw. ;-) http://objectmix.com/python/631346-parallel-python.html -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Local python web server problem
Farsheed Ashouri wrote: > Hi everyone. I have started to develop a web base software for > renderfarm managing. > I run the cherrypy hello world! example and when I visit > 127.0.0.1:8080 > on firefox, a nice "Hello World" appears. I am on ubuntu and my ip in > local network is 192.168.100.18 > but when I visit 192.168.100.18, there is another page: "It Works!". > What is "It Works!"? and when I type 192.168.100.18:8080 in another > local pc, the page didn't load? [...] Services like HTTP servers "bind" not only to a port, but to a combination of IP address and port. The CherryPy tutorial configuration http://www.cherrypy.org/browser/trunk/cherrypy/tutorial/tutorial.conf binds to 127.0.0.1 and 8080. This means the application can only be accessed from the local machine. To bind to all network interfaces, (i. e. Ethernet etc., and not just the loopback device), you can change server.socket_host to "0.0.0.0". -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: updating nntplib
Travis wrote: > Hello all, > > There are some notable deficiencies in nntlib. Here are two: [...] Be sure to add a bug report/patch to the Python bug tracker. http://bugs.python.org/ Anything else will most likely be overlooked or forgotten. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there anyway Vpython and pyODE can be made to work with newer versions of Python 2.6.1 etc. without a lot of changes to source code?
Casey Hawthorne wrote: Is there anyway Vpython and pyODE can be made to work with newer versions of Python 2.6.1 etc. without a lot of changes to source code? I suppose I'm thinking of an extra layer of indirection, which might slow things down to much. Aren't this just Python libraries that include extension modules (i. e. modules written in C to interface external C libraries). If so, there is a good chance that just compiling them against Python 2.6 works out of the box. Otherwise the adjustments necessary will be just minor ones. Python 3.0 compatibility is a different issue, though (more work). -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: PyQt4 on Windows ?
Linuxguy123 wrote:
What does it take to get a PyQt4 application running on a Windows
machine ?
To run, installing Python + PyQt4 ;-)
To create a binary wrapper, I use py2exe (I also tried cx_Freeze, both
work the same). There's a gotcha with PyQt4 - snippet follows:
setup(
options = {"py2exe": {"compressed": 1,
"optimize": 2,
"includes": ["sip"],
"excludes": []}},
service = [],
com_server = [],
console = [],
windows = [modeler],
)
You need to include the "sip" module manually, it's not automatically
discovered.
To create an installer, I use NSIS.
I'm sorry if this is a redundant question, but I've searched this and I
am not finding a comprehensive answer.
If anyone is running a PyQt4 application on a Windows (XP or Vista)
machine, I'd love to know how it works for you
It works fine in a test setup. I didn't pursue this any further at the
moment, because for the project in question we decided to deploy on
MacOS X first and delay the Windows version.
and how extensive the application is and how much of the Qt library it uses.
The application is currently 1200 LOC, 200 of which are autogenerated
from Qt designer files. The final application will probably be about
three times as large.
It's a modeler application and I most of the code is thus centered
around QGraphicsScene and two custom QGraphicsItems - one for nodes, one
for edges. It's been a real pleasure that I get most functionality for
the modeling app for free with (Py)Qt.
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: Encrypted Logging in python
koranth...@gmail.com wrote: >I was wondering if there is a mechanism to encrypt logging > automatically in python. Python's standard library doesn't include any "strong" symmetric ciphers. But if you include for example a cryptographic module for AES, for example, it should be easy (I guess 10 lines of code, yes, the issue always is *which* 10 lines) to write a custom logger that encrypts using a hardcoded key. As others have said, this is not really secure, so you could just as well use something stupid like rot13 or base64 instead. >The issue is as follows: > (a) An application (after py2exe) will go as executable and there > is no need for the user to know that it is written in python. If an > exception occurs and it is logged, then the user can understand it is > written in python. In 99.326 % of all cases, the answer is: so what? > (b) A security threat. If an exception occurs, the code is seen by > the user - and possibly be misused. Simply make the user not see the exception, but use a fallback exception handler that does whatever you want to. Write to a log file. Or write to an encrypted log file if you still think that helps. >Base64 encoding somewhat helps - which is supported by logging > module - but even that is not very secure. If there can be an option - > wherein we send in the password and the logging is encrypted - it > might be better. [...] As I said before, that should be trivial to program if you look up the documentation about the logging module. Just subclass FileHandler. And make sure your class is then used. That's probably the hardest part ;-) That all being said, I have one final advise: Your time is probably much better spent on *real* issues. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Apache Handler
Scooter wrote: > Does anyone have any good examples, or links thereto for using python > as an Apache handler? And I should qualify all of this by saying I'm a > python newbie, and while having experience with Apache, I've never > done anything outside whats "in the box" . > > What I'm looking for is how one might use Python not from the CGI/ > presentation side but more on the backend...i.e. for each page Apache > serves up examine the request and update some headers, or add a cookie > to the response. I vaguely remembered that mod_python can do such things. Looking again it seems to be the case: http://www.modpython.org/live/current/doc-html/pyapi-filter.html > Or possibly use Python for writing a custom Apache > logger. [...] Maybe http://www.modpython.org/live/current/doc-html/dir-handlers-plh.html The documentation about this is a joke, though. Remember that you can write custom Apache loggers quite easily with "piped logs": http://httpd.apache.org/docs/2.2/logs.html This looks roughly like this: CustomLog "|/path/to/my_script.py" common ## begin my_script.py ## import sys for line in sys.stdin: # do stuff ## end ## -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: pg_result_status() alternative?
Qian Xu wrote: Steve Holden wrote: Without knowing the full details of that particular module I would hazard a guess that any database errors will raise exceptions in Python. No exceptions means your database operation worked fine. result status is not an exception. It means the information of frontend/backend protocal, after a SQL-statement is execute. The following is a list of backend IPC commands (v3) Z - Zero / Ready for Query [...] For instance: SELECT * FROM my_table; The backend protocal should return T (Row Description) and the frontend protocal should return Q (Query) DROP TABLE my_table; The backend protocal should return C (Complete) and the frontend protocal should return Q (Query) What are you testing, really? "Normal" Python code should use a PostgreSQL DB-API module or a wrapper on top of it, like SQLAlchemy. Generally, you shouldn't have to drop down to protocol-level functions, unless you're developing a PostgreSQL adapter yourself. Care to explain why you're messing around with this low-level functions? -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better algorithm?
Kottiyath wrote: I have the following list of tuples: L = [(1, 2), (3, 4, 5), (6, 7)] I want to loop through the list and extract the values. The only algorithm I could think of is: [...] If this is part of a real program, instead of an exercise, you should fix the code that creates this list of tuples so that they have a uniform length of 3. And if the third element is missing, it should be None. This saves lots of trouble later on. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Easy-to-use Python GUI
Dotan Cohen wrote: I have been following this thread with interest. Is there a way to build Qt apps with relative easy? I use KDE and would prefer the Qt toolkit for my GUI apps. Thanks. A few years ago, I've had bad experiences with wxPython (random things not actually working on Linux, only on Windows; getting segfaults when using not exactly the right values for API calls). So, when I had to decide for a toolkit for a new application I'm developing on my job which required: - ability to run on Windows - ability to run on MacOS - time to develop is short I recommended to go with PyQt. I remembered it was warmly recommended by Alex Martelli and others a few years ago. So far, it's been nothing but joy. We bought the book "Rapid GUI Programming with Python and Qt" (http://www.qtrac.eu/pyqtbook.html) which is *really* well written. It's probably the best tech book I ever had. The author formerly did Qt documentation for Trolltech, so he has deep understanding of what he's writing about. There may be not so many third-party add-ons for PyQt like for wxPython, but in my opinion, the quality of Qt, PyQt and documentation like the book make up for it. And, it's really extensive. So far I've found everything in Qt/PyQt I wanted/needed: - MDI workspaces - Dock windows - I needed something like wxPythons wxOGL for a process modeler and after looking around for two days, I found out that it's already all there in Qt 4.4: QGraphicsScene/QGraphicsView - etc. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: SQL, lite lite lite
Bruno Desthuilliers wrote: Aaron Brady a écrit : Hi all, (snip) > I don't think relational data can be read and written very easily in Python. Did you try SQLAlchemy or Django's ORM ? [...] Using an ORM when you don't grasp the relational model and/or the SQL query language is futile. That's probably the case for many other abstraction layers, too. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: turtle ?
sai wrote: > python newbie here :-) > > I am trying to get turtle to run but got stuck here: > > $ python > Python 2.5.2 (r252:60911, Aug 5 2008, 16:17:28) > [GCC 4.2.2 20071128 (prerelease) (4.2.2-3.1mdv2008.0)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. import turtle > Traceback (most recent call last): > File "", line 1, in > ImportError: No module named turtle > > I have used google to search the web and the newsgroup but got > nothing. The Linux distro is Mandriva, if that makes a difference. Mandriva have probably split Python into several packages. If so, chances are one of them contains the tkinter GUI library. You should look in your package management tool for a package called python-tkinter, python-tk or similar and install it. HTH Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: RELEASED Python 3.0 final
Iain King wrote: > [...] Props. I just looked through the What's New and the change log, but I > couldn't find the answer to something: has any change been made to > how tabs and spaces are used as indentation? Can they still be > (inadvisably) mixed in one file? Or, more extremely, has one or the > other been abolished? As you have probably guessed: nothing changed here. Also see: http://www.python.org/dev/peps/pep-0666/ -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic design patterns
Slaunger wrote: > Hi comp.lang.python > > I am this novice Python programmer, who is not educated as a computer > scientist (I am a physicist), and who (regrettably) has never read the > GOF on design patterns. [...] > I guess I could boost my productivity by learning these well-proven > and well-established design patterns by heart. At least for me I only getter better by actually training. For programming this means: write code. Revisit it later, improve on it. Of course one often wants to apply all cool new tricks you've learnt. That's normal, but remember: """ Brian Kernighan Law of Debugging Difficulty Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. -- Brian Kernighan """ > I was therefore wondering if you could recommend a book or a resource > concerning design patterns with special focus on the possibilities in > Python? [...] I have this in my bookmarks: http://www.suttoncourtenay.org.uk/duncan/accu/pythonpatterns.html -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: building an extension module with autotools?
Michael George wrote: Hello, (Please CC me in replies, as I am off-list) Ok, but please reply publicly. I'm building an application (a game) in python, with a single C module containing some performance-critical code. I'm trying to figure out the best way to set it up to build. Use distutils and keep your sanity. Distutils seems to be designed only for building and distributing code that lives in site-packages. I'd prefer not to install it in the global site-packages, but rather group all of the files together. Then just use the build step of distutils and do the rest from your build script (Python-based, make based, whatever you prefer). I've tried using automake, In my opinion, this is serious overkill. automake is good for making stuff work on a herd of different Unixen with various combinations of libc functions available etc. But for developing a Python extension, it doesn't help much at all. All you need to know about Python is available via macros if you import Python.h. however I'm worried about libtool not getting the options right while building my module. It seems to me like the ideal frankenstein monster would be for automake to invoke distutils to do the actual building. However I'm relatively new to both autotools and distutils, so getting this rigged up has been a bit of a challenge. Can anybody point me to a good example of something like this or give me any pointers? My recommendation is to start simple instead of wasting too much time upfront for probably nothing. Here's what I'd start with: #!/bin/sh python setup.py build cp build/lib.*/*.so . python test.py HTH -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: python an sqlite objects
azrael wrote:
It logical that it would be more efficient and logical to use a object
oriented database, but in this case I ask because of the portable
nature of sqlite.
so, if I get it right, this should be possible [...]
Did you try it? Did it work? If so,it was pure luck. Attached is a
script that shows how to do it right.
-- Gerhard
# This is an example for storing pickleable Python objects in a SQLite
# database
import cPickle as pickle
try:
from pysqlite2 import dbapi2 as sqlite3
except ImportError:
import sqlite3
class Point(object):
def __init__(self, x, y):
self.x, self.y = x, y
def __repr__(self):
return "" % (self.x, self.y)
def test():
con = sqlite3.connect(":memory:")
cur = con.cursor()
# Make sure you store your pickled
# cur.execute("create table pickled(id integer primary key, data blob)")
cur.execute("create table pickled(id integer primary key, data blob)")
# Here we force pickle to use the efficient binary protocol
# (protocol=2). This means you absolutely must use an SQLite BLOB field
# and make sure you use sqlite3.Binary() to bind a BLOB parameter.
p1 = Point(3, 4)
cur.execute("insert into pickled(data) values (?)", (sqlite3.Binary(pickle.dumps(p1, protocol=2)),))
# If we use old pickle protocol (protocol=0, which is also the default),
# we get away with sending ASCII bytestrings to SQLite.
p2 = Point(-5, 3.12)
cur.execute("insert into pickled(data) values (?)", (pickle.dumps(p2, protocol=0),))
# Fetch the BLOBs back from SQLite
cur.execute("select data from pickled")
for row in cur:
serialized_point = row[0]
# Deserialize the BLOB to a Python object - # pickle.loads() needs a
# bytestring.
point = pickle.loads(str(serialized_point))
print "got point back from database", point
if __name__ == "__main__":
test()
--
http://mail.python.org/mailman/listinfo/python-list
Re: python an sqlite objects
[EMAIL PROTECTED] wrote:
> azrael> is it possible to save a python object into a sqlite database as
> azrael> an atribute of type BLOB
>
> Sure. Just pickle the object and save the resulting string.
Be sure to save it as BLOB, not TEXT.
Suppose you have serialized your object as Python bytestring.
serialized = ...
... .execute("insert into mytable(mycolumn) values (?)",
(sqlite3.Binary(serialized),))
This way you will get a BLOB in the form of a Python buffer object when
you later select it from the database, which you can then deserialize to
a Python object.
If you don't go the BLOB way, you may get an exception, because SQLite
assumes all text is UTF-8 encoded, which it isn't necessarily when you
put arbitrary serialized strings into the database.
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: pysqlite install error on hp ux (ld: Can't find library for -lpython2.5)
Geon. wrote: hi everyone! when i install pysqlite i meet bellow error. ( use easy_install and source code building same problem ) ld: Can't find library for -lpython2.5 what mean this message? and what i do? my system is hp-ux 11i v3. and python2.5 is installed. ld command also avaliable. I don't know much about HPUX, but here are some ideas. Is Python's shared library built at all? Or is Python, for whatever reason, built statically? The command to check on Linux is "ldd /path/to/python_binary". I don't know if it's ldd on HP UX too or if it's something different. If Python is built as a shared library, is the shared library somewhere the system's dynamic linker will find it? If not, you either have to set some environment variable (LD_LIBRARY_PATH on Linux) or configure it (/etc/ld.so.conf on Linux). -- Gerhard PS: On some exotic systems where Java is already installed I found it easier to get Jython (the Python implementation for the Java Virtual Machine) to run than compiling Python. -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with sqlite3 cursor and imbricated for loop
Charles V. wrote:
Hi,
Both may be standard compliant, but if you're depending on
implementation details, you may still get different behaviour.
I'm pretty sure that MySQLdb always fetches the entire resultset from
the server. The sqlite3 module uses what would have been called
"server-side cursors" in real databases, i. e. it only fetches rows on
demand. To fetch everything in one go with the sqlite3 module, you have
to call fetchall() explicitly.
You are right: the default Cursor in MySQLdb fetches the complete set on the
client (It explains why I have a "correct" answer with MySQLdb). As having
multiple cursor isn't an option for me and using non-standard execute on the
connection neither, I tried to modify the Cursor class to store results on
the client side.
class NewCursor(sqlite3.Cursor):
def __iter__(self):
return iter(self.fetchall())
conn = sqlite3.connect(':memory:')
> [...[]
Do you think it is a good solution (any drawbacks ?) ?
It's at least incomplete. fetchone() and fetchmany() won't work any longer.
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: Problem with sqlite3 cursor and imbricated for loop
Steve Holden wrote:
[...]
I feel with you. The fact that cursors, and not connection objects have
the executeXXX methods is totally braindead.
So you'd rather have to use separate connections? That would make
isloated transaction processing a little tricky ...
No, I just find code like:
con = ...connect(...)
cursor1 = con.cursor()
cursor1.execute("select ...")
for row in cursor1:
cursor2 = con.cursor()
cursor2.execute("...)
quite verbose compared to:
con = ...connect()
for row in con.execute("select ...")
con.execute("...")
Both of which work with pysqlite (*).
Granted, the second form works way better with a reference-counting
garbage-collector like CPython has ;-)
That's why pysqlite (sqlite3) has alternative nonstandard executeXXX
methods in the connection object that return cursors.
It's also why SQLite's not a real RDBMS. Fortunately this doesn't stop
it being very useful.
What does pysqlite extending the DB-API have to do with *SQLite* not
being a "real" RDBMS?
-- Gerhard
(*) And writing a wrapper for other modules that does the same is trivial.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Problem with sqlite3 cursor and imbricated for loop
Charles V. wrote: Hi, Thank for replying. Either use a second cursor OR ensure you fetch all the data from the first .execute() first: Are these really the only solutions ? Yes. I was expecting the same behavior than MySQLdb module, which is, as sqlite3, DB-API 2.0 compatible. Both may be standard compliant, but if you're depending on implementation details, you may still get different behaviour. I'm pretty sure that MySQLdb always fetches the entire resultset from the server. The sqlite3 module uses what would have been called "server-side cursors" in real databases, i. e. it only fetches rows on demand. To fetch everything in one go with the sqlite3 module, you have to call fetchall() explicitly. It means a program written for MySQLdb won't be compatible with sqlite3 (even if I am using standard SQL). In fact I don't really understand why the iterator isn't in some way "encapsulated". [...] I feel with you. The fact that cursors, and not connection objects have the executeXXX methods is totally braindead. That's why pysqlite (sqlite3) has alternative nonstandard executeXXX methods in the connection object that return cursors. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: PySqlite - division of real numbers without decimal fractions
Astley Le Jasper wrote:
I've been getting errors recently when using pysqlite. I've declared
the table columns as real numbers to 2 decimal places (I'm dealing
with money),
MySQL doesn't have any MONEY type. All it has is INTEGER, REAL, TEXT,
BLOB and NULL types.
but when doing division on two numbers that happen to
have no decimal fractions, the results through pysqlite are coming
through as integers. The funny thing is that when looking at the
database using SQLite Manager or SQLite Pro the results there are
displayed correctly. As a temporary fix I've had to multiply one of
the numbers with 1.0 which has fixed it but I wonder if there is
something else I'm doing wrong.
Perhaps using SQLite's column affinity would help? Let the type be named
"real" instead of anything fancy:
>>> from pysqlite2 import dbapi2 as sqlite3
>>> con = sqlite3.connect(":memory:")
>>> con.execute("create table foo(x number(10,2))")
>>> con.executemany("insert into foo(x) values (?)", [(3,), (3.0,)])
>>> print con.execute("select x from foo").fetchall()
[(3,), (3,)] # < !!!
>>> con.execute("create table bar(x real)")
>>> con.executemany("insert into bar(x) values (?)", [(3,), (3.0,)])
>>> print con.execute("select x from bar").fetchall()
[(3.0,), (3.0,)] # < !!!
Do you see the difference?
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is psyco available for python 2.6?
[EMAIL PROTECTED] wrote: sert: I used the windows installer for the latest version of psyco, which is labeled as compatible with 2.5, but it gives the following error: ImportError: DLL load failed: The specified module could not be found. (check that the compiled extension 'C:\Python26\lib\site- packages\psyco\_psyco.pyd' is for the correct Python version; this is Python 2.6) I think you have tried to install something compiled for Python 2.5 on Python 2.6, therefore it doesn't work. At the moment Psyco isn't available for Python 2.6, you will probably have to wait some months (or use Python 2.5+Psyco in the meantime). psyco seems to just work on Linux with Python 2.6. So it is probably "only" a matter of compiling it on Windows for Python 2.6. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: list versions of all installed modules
John [H2O] wrote: Is there a quick way to list the version of each installed module? $ sudo easy_install yolk $ yolk -l -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.5 + sqlite full text search possible?
Guillermo wrote:
Hi!
Is it possible to use the full-text module of SQLite with the sqlite3
module? I've done a bit of investigation and it seems the stand-alone
distribution of SQLite is compiled without it,
Yes, though compiling using the amalgamation and defining
SQLITE_ENABLE_FTS3 helps.
and so does the version bundled with Python.
True.
I'm too lazy to build a SQLite3 DLL with FTS enabled, I'm pretty sure
those can be found on the net.
But I've just patched pysqlite with one line:
+ext.define_macros.append(("SQLITE_ENABLE_FTS3", "1")) #
build with fulltext search enabled
which helped its super-crazy script mkwin32.py to build Windows binaries
*on Linux* that fetch the SQLite amalgamation and build Windows
binaries for Python 2.3, 2.4 and 2.5 (no 2.6, yet).
Just go here
http://oss.itsystementwicklung.de/download/pysqlite/2.5/2.5.0/win32_fts/
download and install the binary for Python 2.5 and off you go:
from pysqlite2 import dbapi2 as sqlite3
con = sqlite3.connect(":memory:")
# example from SQLite wiki
con.execute("create virtual table recipe using fts3(name, ingredients)")
con.executescript("""
insert into recipe (name, ingredients) values ('broccoli stew', 'broccoli
peppers cheese tomatoes');
insert into recipe (name, ingredients) values ('pumpkin stew', 'pumpkin
onions garlic celery');
insert into recipe (name, ingredients) values ('broccoli pie', 'broccoli
cheese onions flour');
insert into recipe (name, ingredients) values ('pumpkin pie', 'pumpkin
sugar flour butter');
""")
for row in con.execute("select rowid, name, ingredients from recipe where name match
'pie'"):
print row
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.5 + sqlite full text search possible?
Guillermo wrote: Hi! Is it possible to load the full-text search module for the SQLite version bundled with Python 2.5? [...] I'm on Windows XP. Yes, it's possible. But not easily. You have to replace the sqlite3.dll that comes with Python 2.5 with one that includes fulltext search. If you can't or don't want to build it yourself, you could perhaps kindly ask on the SQLite mailing list. The DLL has to include fulltext search already, because only the very latest release of pysqlite wraps the method to enable loading of SQLite extensions. But still then, you'd need to have a custom built fts2.dll to load it into SQLite. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: [newbie] Right way to access item in array?
Diez B. Roggisch wrote:
Gilles Ganault wrote:
Hello
I'd like to know what the right way is to access an item in a row as
returned by a database:
=
import apsw
connection=apsw.Connection("test.sqlite")
cursor=connection.cursor()
rows=cursor.execute("SELECT isbn,price FROM books WHERE price IS
NULL")
for row in rows:
#Is this right?
for isbn in row:
No. This will iterate though all columns, not just the isbn.
You can use tuple-unpacking in such cases:
for row in rows:
isbn, price = row
...
You can do it even in one step with APSW (and pysqlite, and others):
for isbn, price in cur.execute("select isbn, price ..."):
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: conversion to and from unicode strings
abhishek wrote:
hello group,
i want to represent and store a string u'\x00\x07\xa7' as
'\x00\x07\xa7'. any ideas on how to achieve this.
You want to store it in the form of the repr() of the string? It is
possible to use repr() to get a bytestring to store and to use eval() to
create a unicode string of that bytestring again. But that's just bad.
It's much better to use a encoding that can represent all Unicode
characters like UTF-8.
>>> s = u'\x00\x07\xa7'
>>> bytestr = s.encode("utf-8")
>>> bytestr
'\x00\x07\xc2\xa7'
>>> out_str = unicode(bytestr, "utf-8")
>>> out_str == s
True
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: look-behind fixed width issue (package re)
MRAB wrote: On Oct 24, 6:29 am, Peng Yu <[EMAIL PROTECTED]> wrote: Hi, It seem that the current python requires fixed-width pattern for look- behind. I'm wondering if there is any newly development which make variable-width pattern available for look-behind. The re module is currently being worked on, but unfortunately it won't appear until Python 2.7. Variable-width look-behind is one of the improvements. Most probably a backport to Python 2.6 or even 2.5 under a different module name like re_ng wouldn't be too difficult to do for anybody that needs the new functionality and knows a bit about building extension modules. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: [APSW] SELECT COUNT(*) not succesfull?
Gilles Ganault wrote: On Thu, 23 Oct 2008 00:24:01 -0200, "Gabriel Genellina" <[EMAIL PROTECTED]> wrote: In case you didn't notice, B.D. already provided the answer you're after - reread his 3rd paragraph from the end. Yes, but it doesn't work with this wrapper (APSW version 3.5.9-r1): The recommended way is to pass the arguments to cursor.execute, ie: I'm getting an error when doing it this way: === isbn = "123" sql = "SELECT COUNT(*) FROM books WHERE isbn='%s'" #Incorrect number of bindings supplied. The current statement uses 0 and there are 1 supplied. Current offset is 0 cursor.execute(sql, (isbn,)) === I don't know enough about Python and this wrapper to tell why it triggers an error. you want: row = cursor.fetchone() count = row[0] if not count: This wrapper doesn't seem to support fetchone(): = #AttributeError: 'apsw.Cursor' object has no attribute 'fetchone' row = cursor.fetchone() [...] Directly calling next() should probably do the trick with APSW. Its cursors support the iterator interface and iterators are implemented by providing __iter__() and next() methods. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: [APSW] SELECT COUNT(*) not succesfull?
Dennis Lee Bieber wrote: On Thu, 23 Oct 2008 09:26:54 +0200, Gilles Ganault <[EMAIL PROTECTED]> declaimed the following in comp.lang.python: Yes, but it doesn't work with this wrapper (APSW version 3.5.9-r1): APSW is not, so far as I recall, a "DB-API 2" adapter -- it is a touch more low-level (closer to the raw C-interface). pysqlite2 IS a DB-API 2 adapter. For APSW, one will need to read the specific documentation on all the calls to determine behavior (even if the same person is now maintaining both APSW and pysqlite2 ) Maintainership of pysqlite or APSW hasn't changed. pysqlite is still maintained by me and APSW still by Roger Binns. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: like using python
[EMAIL PROTECTED] wrote: as the subject me2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Race condition when generating .pyc files
[EMAIL PROTECTED] wrote: I have a large body of Python code which runs on many different (Unix) machines concurrently. Part of the code lives in one place, but most of it lives in directories which I find at runtime. I only have one copy of each Python source file and I think I'm hitting a race condition where two hosts attempt to import the same module at the same time. My import fails on one of the machines and the following exception is thrown: EOFError: EOF read where object expected My hypothesis is that there's contention between the two (or more) hosts when the module's .pyc file is generated. Possible solutions I see: 1) Running a cron job before my code executes which compiles all the python source first. 2) Making separate copies of all the .py files for each host running the code - I'd rather not do this, it seems like a big pain. 3) Inhibiting the generation of .pyc files altogether if that's even possible [...] If you don't want pyc files to be created, you could set Unix permissions such that Python cannot write to the directory. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Pure Python interface to MySQL?
Gerhard Häring wrote: James Mills wrote: On Tue, Oct 7, 2008 at 9:15 AM, Roy Smith <[EMAIL PROTECTED]> wrote: Does there exist a pure Python version of a MySQL module? I've got a data logging application that needs to run on a whole bunch of OSs, ranging from Windows to a dozen different unix flavors on all sorts of hardware. Portability is much more important than performance for this application. We're only inserting a few hundred records a day from each system, but the ability to quickly deploy to anywhere I've already got Python running is key. My solution (tm): You could implement a proxy server/client sub-system [...] Or instead of reinventing the wheel, you could use SQLRelay (http://sqlrelay.sourceforge.net/), which has a pure-Python DB-API module. Unfortunately, the Python interface for SQLRelay seems to include parts written in C. What a pity. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Pure Python interface to MySQL?
James Mills wrote: On Tue, Oct 7, 2008 at 9:15 AM, Roy Smith <[EMAIL PROTECTED]> wrote: Does there exist a pure Python version of a MySQL module? I've got a data logging application that needs to run on a whole bunch of OSs, ranging from Windows to a dozen different unix flavors on all sorts of hardware. Portability is much more important than performance for this application. We're only inserting a few hundred records a day from each system, but the ability to quickly deploy to anywhere I've already got Python running is key. My solution (tm): You could implement a proxy server/client sub-system [...] Or instead of reinventing the wheel, you could use SQLRelay (http://sqlrelay.sourceforge.net/), which has a pure-Python DB-API module. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Cannot install pysqlite on Cygwin
Tilman Kispersky wrote: I am trying to install sqlite for use with python on cygwin. I have installed the sqlite packages from cygwin (that is libsqlite3-devel and libsqlite3_0). When attempting to easy_install pysqlite I get: [...] build/temp.cygwin-1.5.25-i686-2.5/src/connection.o: In function `pysqlite_enable_load_extension': /cygdrive/c/Users/Tilman/AppData/Local/Temp/easy_install-876nHz/ pysqlite-2.5.0/src/connection.c:922: undefined reference to `_sqlite3_enable_load_extension' [...] It might be that Cygwin's SQLite doesn't support loading extensions. Either: Rebuild SQLite from source on Cygwin and be sure to include --enable-load-extension when calling ./configure. Or hack the pysqlite sources and remove these lines in src/connection.c: #if SQLITE_VERSION_NUMBER >= 3003008 #define HAVE_LOAD_EXTENSION #endif I'm very interested how you can fix the problem. -- Gerhar -- http://mail.python.org/mailman/listinfo/python-list
Re: finding domain name
Bobby Roberts wrote:
hi group. I'm new to python and need some help and hope you can
answer this question. I have a situation in my code where i need to
create a file on the server and write to it. That's not a problem if
i hard code the path. However, the domain name needs to be dynamic so
it is picked up automatically. The path to our websites is
home/sites/x/
where x represents the domain name.
How can I find the domain name of the current url being viewed.
Depends on the technology/web framework. If you use WSGI, you should use
something like:
host_name = environ.get("HTTP_HOST", None) or environ["SERVER_NAME"]
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: Comparing float and decimal
D'Arcy J.M. Cain wrote:
I'm not sure I follow this logic. Can someone explain why float and
integer can be compared with each other and decimal can be compared to
integer but decimal can't be compared to float?
from decimal import Decimal
i = 10
f = 10.0
d = Decimal("10.00")
i == f
True
i == d
True
f == d
False
I can give you the technical answer after reading the sources of the
decimal module: you can only compare to Decimal what can be converted to
Decimal. And that is int, long and another Decimal.
Everything else will return False when comparing.
This seems to break the rule that if A is equal to B and B is equal to
C then A is equal to C.
Yes, but only if comparison from type(A) to type(C) is supported at all.
Instead of raising ValueError or NotImplementedError, the decimal
module returns False here.
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: adding in-place operator to Python
Arash Arfaee wrote: Hi All, Is there anyway to add new in-place operator to Python? You can't create new syntax, like %= Or is there any way to redefine internal in-place operators? What you can do is give your objects the ability to use these operators. See http://docs.python.org/ref/numeric-types.html for __iadd_ (+=) and friends. You could implement something like a string buffer this way: class Buffer: def __init__(self): self.buf = [] def __iadd__(self, item): self.buf.append(item) return self def __str__(self): return "".join(self.buf) if __name__ == "__main__": buf = Buffer() buf += "str1" buf += "str2" print str(buf) -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: how dump a program which is running in memory
ruqiang826 wrote:
hi
I have written a service running backgroud to do something in linux.
unfortunately，I deleted the source code by mistake, and I can still
see the process running background using "ps aux" :
username 13820 0.0 0.0 60368 2964 ?SAug20 0:33
python ./UpdateJobStatus.py
I wonder if there is some way to dump the programme
"UpdateJobStatus.py" and get the source code back?
Often, there is a way by accessing /proc/{pid}/fd/
But I believe you're out of luck with this method because apparently the
Python interpreter closes the source file after parsing it.
You can still try to find an undeletion utility for your filesystem.
Avoid writing to disk in the meantime to not overwrite the deleted file
accidentally, of course. There's such a utility for ext2, but I don't
know if that works ok with ext3. For other filesystems, I have no idea.
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: [ANN] pysqlite 2.5.0 released
Matthias Huening wrote:
Gerhard Häring (08.09.2008 10:12):
Error is:
con.execute("select load_extension('./fts3.so')")
pysqlite2._sqlite.OperationalError: Das angegebene Modul wurde nicht
gefunden.
Where should I look for the module?
The sources are in ext/fts3 in the SQLite source tree. I haven't found
any Makefile, so I it myself using this gcc command:
$ cd .../ext/fts3
$ gcc -shared -o ~/src/gh/pysqlite/build/lib.linux-i686-2.5/fts3.so
*.c -lsqlite3
Thanks!
Will fts3 be integrated in the Python 2.6 release?
No (only relevant on win32, where we also ship the SQLite DLL). Neither
will be the ability to load extensions modules. It's just too late to
add features now.
But AFAIK it's possible to compile a custom SQLite with appropriate
flags to ./configure that will include the fulltext search extension.
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: [ANN] pysqlite 2.5.0 released
Matthias Huening wrote:
Hi,
- - Connection.enable_load_extension(enabled) to allow/disallow extension
loading. Allows you to use fulltext search extension, for example ;-)
The following code (from the docs) produces an error:
from pysqlite2 import dbapi2 as sqlite3
con = sqlite3.connect(":memory:")
# Load the fulltext search extension
con.enable_load_extension(True)
con.execute("select load_extension('./fts3.so')")
con.enable_load_extension(False)
Error is:
con.execute("select load_extension('./fts3.so')")
pysqlite2._sqlite.OperationalError: Das angegebene Modul wurde nicht
gefunden.
Where should I look for the module?
The sources are in ext/fts3 in the SQLite source tree. I haven't found
any Makefile, so I it myself using this gcc command:
$ cd .../ext/fts3
$ gcc -shared -o ~/src/gh/pysqlite/build/lib.linux-i686-2.5/fts3.so *.c
-lsqlite3
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
Re: embed python in ms-word?
oyster wrote: > In my ms-word documnet, there are some calculation whihc I have to > change due to different argumnet. is there any way to embed python > code in word, so that I can write the following as a macro or > something else, then the result (i.e. 2) is shown in the word > documnet? > > def f(n): > if n<2: > return 1 > else: > return f(n-1)+f(n-2) > main() > return 'fib(3)=%0i' % f(3) > > if that is impossible, does there exist such word-addons can do that? thanx With Python and the win32 extensions you can write COM servers. These you can use from Visual Basic for Applications (VBA). But it's really only worth the effort if you're doing something less trivial than above ;-) -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: sqlite3 import performance
Ben Lee wrote: > hi folks -- > > a quick python and sqlite3 performance question. i find that > inserting a million rows of in-memory data into an in-memory database > via a single executemany() is about 30% slower than using the sqlite3 > CLI and the .import command (reading the same data from a disk file, > even.) i find this surprising, executemany() i assume is using a > prepared statement and this is exactly what the .import command does > (based on my quick perusal of the source.) > > is this discrepancy to be expected? where is the overhead coming > from? [...] Ok, I'll bite. Well, first, the a 30 % slowdown with a Python DB-API wrapper compared to the native commandline tool of the database is to be considered still quite reasonable, in my opinion. About a year ago I compared the performance of pysqlite vs. the other SQLite wrapper, APSW. At the time, APSW was a bit faster, not excessively, but measurable. In meaningless benchmarks like yours ;-) So I changed pysqlite here and there to get the same performance as APSW. Only minor tweaks, nothing spectacular. And a few hardcore tricks as well, like special-casing *not-subclassed* classes. The result was pysqlite 2.3.5: http://oss.itsystementwicklung.de/trac/pysqlite/wiki/2.3.5_Changelog These optmizations are not yet in Python 2.5.x, but they can be found in the sqlite3 module of the Python 2.6/3.0 betas. Well, here are my results of your benchmark :-) -- Gerhard # with Python 2.5.2's sqlite3 module [EMAIL PROTECTED]:~/tmp$ python t.py generating data... done! testing 100 inserts... [42.795290946960449, 44.337385892868042, 46.35642409324646] # with pysqlite 2.5.0, which I released earlier today [EMAIL PROTECTED]:~/tmp$ python t.py generating data... done! testing 100 inserts... [33.027599096298218, 32.73675012588501, 32.823790073394775] # SQLite commandline [EMAIL PROTECTED]:~/tmp$ time sqlite3 -init sqlcmds ':memory:' '.quit' real0m32.514s [EMAIL PROTECTED]:~/tmp$ time sqlite3 -init sqlcmds ':memory:' '.quit' real0m32.576s [EMAIL PROTECTED]:~/tmp$ time sqlite3 -init sqlcmds ':memory:' '.quit' real0m32.604s -- http://mail.python.org/mailman/listinfo/python-list
[ANN] pysqlite 2.5.0 released
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 pysqlite 2.5.0 released === I'm pleased to announce the availability of pysqlite 2.5.0. This is a release with major new features. Go to http://pysqlite.org/ for downloads, online documentation and reporting bugs. What is pysqlite? pysqlite is a DB-API 2.0-compliant database interface for SQLite. SQLite is a in-process library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine. pysqlite makes this powerful embedded SQL engine available to Python programmers. It stays compatible with the Python database API specification 2.0 as much as possible, but also exposes most of SQLite's native API, so that it is for example possible to create user-defined SQL functions and aggregates in Python. If you need a relational database for your applications, or even small tools or helper scripts, pysqlite is often a good fit. It's easy to use, easy to deploy, and does not depend on any other Python libraries or platform libraries, except SQLite. SQLite itself is ported to most platforms you'd ever care about. It's often a good alternative to MySQL, the Microsoft JET engine or the MSDE, without having any of their license and deployment issues. pysqlite can be downloaded from http://pysqlite.org/ - Sources and Windows binaries for Python 2.5, 2.4 and Python 2.3 are available. === CHANGES === - - Windows binaries are now cross-built using mingw on Linux - - import various fixes from Python 2.6 version - - Connection has new method iterdump() that allows you to create a script file that can be used to clone a database - - the docs are now built using Sphinx and were imported from Python 2.6's sqlite3 module - - Connection.enable_load_extension(enabled) to allow/disallow extension loading. Allows you to use fulltext search extension, for example ;-) - - Give the remaining C functions used in multiple .c source files the pysqlite_ prefix. - - Release GIL during sqlite3_prepare() calls for better concurrency. - - Automatically download the SQLite amalgamation when building statically. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIwZV3dIO4ozGCH14RAp1YAJwPIdgtCZY7E8YcDUjO/dzoAThblgCggfhs OATfXAb6JYXqb8eTadl9k74= =KU3f -END PGP SIGNATURE- -- http://mail.python.org/mailman/listinfo/python-list
Re: cPickle
gopal mishra wrote: I have 3 objects and want to save in one pickle file. I used cPickle to dump 3 objects in a pkl file i.e cPickle.dump(object1, fileobject, -1) cPickle.dump(object2, fileobject, -1) cPickle.dump(object3, fileobject, -1) I have changed the 3^rd object and want to save the updated 3^rd object in the pickle file. I have to dump all the 3 objects again. Is there any way to dump only the 3^rd object in to the pkl file. No, there isn't. You could, of course, save each object in its own pickle file. Or use an object database like ZODB instead. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: use of Queue
Alexandru Mosoi wrote: how is Queue intended to be used? I found the following code in python manual, but I don't understand how to stop consumers after all items have been produced. I tried different approaches but all of them seemed incorrect (race, deadlock or duplicating queue functionality) def worker(): while True: item = q.get() if item is None: break do_work(item) q.task_done() q = Queue() for i in range(num_worker_threads): t = Thread(target=worker) t.setDaemon(True) t.start() for item in source(): q.put(item) # stop all consumers for i in range(num_worker_threads): q.put(None) q.join() # block until all tasks are done This is how I do it. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: Non-evil multithreaded WSGI server?
Gabriel Genellina wrote: En Tue, 26 Aug 2008 03:20:53 -0300, Gerhard Häring <[EMAIL PROTECTED]> escribi�: In a recent experiment I've done this: from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler from wsgiref.simple_server import make_server, demo_app from SocketServer import ThreadingMixIn # Let's make a WSGI server that can use multiple threads. class ThreadedHTTPServer(ThreadingMixIn, HTTPServer): """Handle requests in a separate thread.""" # Evil! ;-) from wsgiref.simple_server import WSGIServer as MyWSGIServer MyWSGIServer.__bases__ = (ThreadedHTTPServer,) Now I wonder if there's a less evil way that does not involve copy & paste of the WSGIServer code (only couple of lines, but I hate duplication)?! I'm not sure I understand completely the question - does the code below work for you? class MyWSGIServer(ThreadingMixIn, wsgiref.simple_server.WSGIServer): pass Yes, it does! I was totally on the wrong track. Of course the ThreadingMixin can also be "mixed in" later in the chain. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Non-evil multithreaded WSGI server?
In a recent experiment I've done this: from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler from wsgiref.simple_server import make_server, demo_app from SocketServer import ThreadingMixIn # Let's make a WSGI server that can use multiple threads. class ThreadedHTTPServer(ThreadingMixIn, HTTPServer): """Handle requests in a separate thread.""" # Evil! ;-) from wsgiref.simple_server import WSGIServer as MyWSGIServer MyWSGIServer.__bases__ = (ThreadedHTTPServer,) Now I wonder if there's a less evil way that does not involve copy & paste of the WSGIServer code (only couple of lines, but I hate duplication)?! -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: rspec for python
Rustom Mody wrote: Is there anything equivalent to rspec for python? I had to Google to see what it is: "A Behaviour Driven Development" framework for Ruby. In a blog article from Ian Bicking says that it is impossible to have a Python port of this because Python doesn't allow you to inject methods into the object class and suggests Python's doctest module as an alternative. Have you actually used this "rspec" thing in Ruby? I always wonder with such things. Same with all the other hyped technologies of yesteryear. Anybody out there who really uses model-driven development? -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie question
sharon k wrote:
hi all,
i am new to python.
>
i fetch a webpage with urllib, extract a few numbers in a format as follow;
10,884
24,068
my question is how to remove the comma between the number, since i have
to add them up later.
Strings have a replace method. Calling replace(",", "") on the string
will do the trick here.
-- Gerhard
--
http://mail.python.org/mailman/listinfo/python-list
