Re: Setting the encoding in pysqlite2

2005-08-26 Thread Gerhard Haering
On Thu, Aug 25, 2005 at 01:15:55AM -0700, Michele Simionato wrote:
> An easy question, but I don't find the answer in the docs :-(
> I have a sqlite3 database containing accented characters (latin-1).
> How do I set the right encoding? For instance if I do this: [...]

You cannot set the encoding directly, because TEXT data in SQLite3
databases is expected to be in UTF-8 encoding. If you store "weird"
TEXT, you can work around it by using a custom converter in pysqlite2,
like in the following example:

#-*- encoding: latin-1 -*-
from pysqlite2 import dbapi2 as sqlite

# Register an additional converter for plain bytestrings
sqlite.register_converter("bytestring", str)

con = sqlite.connect(":memory:", detect_types=sqlite.PARSE_COLNAMES)
cur = con.cursor()
cur.execute("create table test(t)")

testdata = "H?ring" # bytestring in ISO-8859-1 encoding

cur.execute("insert into test(t) values (?)", (testdata,))

# Try to retrieve the test data, will fail
try:
cur.execute("select t from test")
except UnicodeDecodeError:
print "Could not decode latin1 as utf-8 (as expected)"

# Via the PARSE_COLNAMES trick, explicitly choose the bytestring converter
# instead of the default unicode one:
cur.execute('select t as "t [bytestring]" from test')
result = cur.fetchone()[0]
assert testdata == result
print "Correctly retrieved test data"

HTH,

-- Gerhard
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: dictionary as property

2005-07-19 Thread Gerhard Haering
On Tue, Jul 19, 2005 at 07:56:20PM +0300, Thanos Tsouanas wrote:
> Hello.
> 
> (How) can I have a class property d, such that d['foo'] = 'bar' will run
> a certain function of the class with 'foo' and 'bar' as it's arguments?

You could implement a custom container type that will do what you want.
See http://docs.python.org/ref/sequence-types.html for __setitem__.

Quick hack:

>>> class Foo:
... def __init__(self):
... self.d = self
... def __setitem__(self, key, value):
... getattr(self, key)(value)
... def bar(self, param):
... print "bar got called with", param
...
>>> foo = Foo()
>>> foo.d["bar"] = 42
bar got called with 42
>>>

-- Gerhard
-- 
Gerhard Häring - [EMAIL PROTECTED] - Python, web & database development


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: goto

2005-07-18 Thread Gerhard Haering
On Mon, Jul 18, 2005 at 08:40:16AM -0700, Kay Schluehr wrote:
> Hayri ERDENER schrieb:
> > hi,
> > what is the equivalent of C languages' goto  statement in python?
> > best regards
> 
> No, but some of goto's use cases can be covered by unconditional jumps
> provided by exceptions. [...]

I like the "named loops" concept of other HLL like Ada 95 or Java better
than either goto or exceptions. It allows you to use "break" and
"continue" for other than the innermost loops, too:

break;  => break out of inner loop
break loop_name;=> break out of named loop "loop_name"

OTOH it's not used *that* often, so I won't argue for including it in
Python ;)

-- Gerhard
-- 
Gerhard Häring - [EMAIL PROTECTED] - Python, web & database development


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What is your favorite Python web framework?

2005-07-18 Thread Gerhard Haering
On Mon, Jul 18, 2005 at 09:06:21AM -0400, Peter Hansen wrote:
> JZ wrote:
> > I think Django is more mature than Subway or CherryPy and can quickly
> > become the black horse in area of pythonic frameworks.
> 
> I'm not familiar with this expression.  What do you mean by "black horse"?

Maybe "the Ferrari of pythonic frameworks" (black horse on yellow
background being the symbol of Ferrari).

That's what I thought of first when I tried to parse the sentence ;-)

-- Gerhard
-- 
Gerhard Häring - [EMAIL PROTECTED] - Python, web & database development


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: psycopg simplest problem

2005-07-08 Thread Gerhard Haering
On Fri, Jul 08, 2005 at 04:23:50PM +0200, Glauco wrote:
> [...]
> My problem is to do a middle layer over pycopg for eliminate type 
> casting problem in postgres in all direction.
> 
> i've resolved this doing a C extension in python and manipulating only 
> string and int in my application.
> 
> this is my example:
> import sqlvar
> 
> sql = """insert into mytable (myint, mytext, maydate)
>   values
>   (%d,%s,%s);""" % (sqlvar.number(myvalue1), 
> sqlvar.text(myvalue2), sqlvar.date(myvalue3) )
> 
> all problem concerning quoting, " ' -> ''", null, None, 0, empty string 
> is solved by the sqlvar lib. [...]

Instead of quoting Python values yourself appropriately for each type,
just let the DB-API module do its work. Use parametrized queries and
then supply the arguments to the query:

cur = con.cursor()
intVal = 42
textVal = "Jason's house"
dateVal = datetime.date(2005, 7, 8)
cur.execute("""
insert into mytable(myint, mytext, mydate)
values (%s, %s, %s)
""", (intval, textVal, dateVal))

The execute(many) method has an optional second parameter, which is a
tuple of values for your parametrized query.

There are different styles to parametrize queries among the various
DB-API modules. psycopg uses the pyformat one, where you just use %s as
placeholders.

It will happily quote all Python values of types int, str, date, float,
etc.  for you. It's also possible to teach it how to quote other custom
data types, but you'll normally not need this.

HTH,

-- Gerhard
-- 
Gerhard Häring - [EMAIL PROTECTED] - Python, web & database development


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: VC++ linking problem

2005-07-07 Thread Gerhard Haering
On Thu, Jul 07, 2005 at 04:52:11AM -0700, J wrote:
> Hi everyone,
> 
> I started embedding python into a 3D graphics App and I came
> across this linking problem.
> 
> 
> SO.lib(ScnGlobal.obj) : error LNK2001: unresolved external symbol
> __imp__Py_InitModule4TraceRefs
> SO.lib(ScnNode.obj) : error LNK2001: unresolved external symbol
> __imp___Py_RefTotal
> SO.lib(ScnLight.obj) : error LNK2001: unresolved external symbol
> __imp___Py_RefTotal
> 
> I am linking against python24.lib using VC++ 6.0. Before I got to this
> point it couldn't find python24_d.lib. After searching around
> for a while I came across a solution that included changing
> python24_d.lib to python24.lib in one of the header files. I hope that
> didn't have a negative effect. I would really appreciate
> some help

It *does* have a negative effect. Python debug and non-debug libraries
are not binary compatible, even when otherwise compiled with the same
settings. I think that's the reason for the _d suffixes for Python debug
binaries on Windows.

The solution for you is to compile yourself a debug version of Python
for testing your whole app in debug mode.

AFAIR in python-dev Python 2.4 can still be compiled with MSVC6, even
though the official binaries are built with MSVC 7.1.

-- Gerhard
-- 
Gerhard Häring - [EMAIL PROTECTED] - Python, web & database development


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

[ANN] pysqlite 2.0.3 released

2005-05-28 Thread Gerhard Haering
This is a minor bugfix release.

Wiki, bugtracker, downloads at http://pysqlite.org/

Changes since 2.0.2
===

The changes for prefetching in 2.0.2 were incomplete. A check that made
sense before had to be removed, otherwise fetchone() / fetchmany() /
fetchall() could raise ProgrammingErrors instead of returning None or an
empty list if no more rows are available.

Users of pysqlite 2.0.2 should definitely upgrade!
-- 
Gerhard Häring - [EMAIL PROTECTED] - Python, web & database development


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: write to the same file from multiple processes at the same time?

2005-05-27 Thread Gerhard Haering
On Fri, May 27, 2005 at 09:27:38AM -0400, Roy Smith wrote:
> Peter Hansen <[EMAIL PROTECTED]> wrote:
> > The OP was probably on the right track when he suggested that things 
> > like SQLite (conveniently wrapped with PySQLite) had already solved this 
> > problem.
> 
> Perhaps, but a relational database seems like a pretty heavy-weight 
> solution for a log file.

On the other hand, it works ;-)

-- Gerhard
-- 
Gerhard Häring - [EMAIL PROTECTED] - Python, web & database development


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: SQL Query via python

2005-05-23 Thread Gerhard Haering
On Mon, May 23, 2005 at 04:12:31PM +, Austyn Bontrager wrote:
> How about:
> 
> cursor.execute("""
>   SELECT name, month, day ,category, city FROM bday
>   WHERE %(col_name)s = %%s
>   """ % dict(col_name=arg1),
>   (arg2)
>   )
> 
> The "%(col_name)s" will be replaced by normal Python string 
> substitution, while the "%%s" will be quoted by the db module.
> 
> Watch out for SQL injection in arg1, though! Maybe check beforehand that 
> it is a string containing only word characters...

Maybe what you really need is the ability to search for two fields, or
both fields?

Here's an approach that I usually use:

- Write a search function accepting the parameters you search for.
- If you fill one of the parameters with None (in SQL: NULL), you don't
  care for its value
- This way, you can narrow your search as much as you like

Here's a quick test script, using pysqlite2 to demonstrate the approach.
It's also not meant as production code, but at least it only works with
SQL parameter binding.

- SQLite uses :name for named parameters instead of %(name)s.
- locals() is a neat hack to access the local variables as a dictionary

#v+
from pysqlite2 import dbapi2 as sqlite

con = sqlite.connect(":memory:")
cur = con.cursor()
cur.execute("create table test(a, b)")
cur.execute("insert into test(a, b) values (1, 2)")
cur.execute("insert into test(a, b) values (1, 3)")
cur.execute("insert into test(a, b) values (2, 3)")

def search(a, b):
global cur
cur.execute("""
select a, b from test
where (:a is null or a=:a)
  and (:b is null or b=:b)
  """, locals())
return cur.fetchall()

print search(2, None)
print "-" * 50
print search(None, 3)
print "-" * 50
print search(2, 3)
#v-

-- Gerhard
-- 
Gerhard Häring - [EMAIL PROTECTED] - Python, web & database development


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

[ANN] pysqlite 2.0.2 released

2005-05-20 Thread Gerhard Haering
This is a minor bugfix release.

Wiki, bugtracker, downloads at http://pysqlite.org/

If you missed 2.0.1, it fixed a bug that could happen if user-defined
functions/aggregates were getting out of scope. It's a fatal bug that
will crash your application if you encounter it.

- Code changes to allow compilation on MacOS X Tiger

- Fixed a few compilation warnings that appeared under GCC 4.0.

- Refactored code so cursors are closed ASAP. This means that statements are
  now finalized as soon as the last row was fetched. In order to achieve this,
  pysqlite is always prefetching exactly one row.

- Removed the type objects STRING, NUMBER, DATETIME, ... from the module,
  because pysqlite cannot deliver the type code anyway, so these type objects
  are useless, if not even confusing.
-- 
Gerhard Häring - [EMAIL PROTECTED] - Python, web & database development


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [pysqlite] How do I use pysqlite in a multi-threading env.?

2005-05-20 Thread Gerhard Haering
On Fri, May 20, 2005 at 08:42:54AM +0200, F. GEIGER wrote:
> In my wxPython-app a part of it gathers data, when a button is pressed, and
> stores it into a db.
> 
> The GUI part should display the stuff being stored in the db.
> 
> When both parts work on the same connection, I get "SQL statements in
> progress errors". Seems ok to me, you can't do that.

You get that error message with which pysqlite call?

> So, next step: Both parts get a separate connection. Now I get "Database
> locked" errors. Hmm, yes, it's GUI stuff after all, being all in the same
> thread.

One connection did not commit() or rollback() within the default
timeout (5 seconds). SQLite is not a good solution if you need
long transactions, because other SQLite connections are blocked, then.

> So, next step: I put the data gathering part into a real thread, that's
> started, wehn the "Scan" button is pressed. This way the "Database locked"-
> errors should got away. But now I get "SQLite objects created in a thread
> can only be used in that same thread.The object was created in thread id
> 3576 and this is thread id 1040". Hmm, true, I want to display db content,
> that was stored by an other thread. But that's what multi-threading
> capabilities are for!

Yes, but pysqlite's threadsafety level is 1, that means you cannot share
a connection object among multiple threads. In order to ensure that and
to avoid that you get random and unexplainable crashes, I implemented
this warning.

> I must be missing something here. Anyway, I'm a bit lost now, really. So,
> how are you doing such stuff with pysqlite?

Use multiple connections and commit/rollback early.

HTH,

-- Gerhard
-- 
Gerhard Häring - [EMAIL PROTECTED] - Python, web & database development


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [pysqlite] pysqlite2.dbapi2.OperationalError: cannot commit transaction - SQL statements in progress

2005-05-19 Thread Gerhard Haering
On Wed, May 18, 2005 at 09:41:39PM +0200, F. GEIGER wrote:
> I've troubles to let my app take off using pysqlite.
> 
> What I wonder most for now is that "pysqlite2.dbapi2.OperationalError:
> cannot commit transaction - SQL statements in progress" when I do this:
> 

Urgh! I would have preferred simplified demo code. But after a little
thinking, I guessed right.

> [...]

pysqlite 2 currently has problems doing .commit() .rollback() on the
connection object after a cur.execute("select ...") in a few cases.

I know why and I will fix it in 2.0.2 (originally only MacOS X fixes
planned) or if it takes longer to test, in 2.0.3.

Current workarounds:

- don't commit/rollback after SELECT. It doesn't make much sense anyway.
  .commit()/.rollback() after your DML statements instead
  (INSERT/UPDATE/DELETE).

- Use fetchall()[0] for fetching single rows instead of fetchone() for
  queries like your SELECT COUNT.

-- Gerhard
-- 
Gerhard Häring - [EMAIL PROTECTED] - Python, web & database development


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

[ANN] pysqlite 2.0.0 final released!

2005-05-16 Thread Gerhard Haering
Hello everyone,

After pondering about a redesign of pysqlite for years, and now after half a
year of development, I am happy to finally announce the first stable release of
pysqlite2.

pysqlite a DB-API 2.0-compliant database interface for SQLite.

SQLite is a relational database management system contained in a relatively
small C library. It is a public domain project created by D. Richard Hipp.
Unlike the usual client-server paradigm, the SQLite engine is not a standalone
process with which the program communicates, but is linked in and thus becomes
an integral part of the program. The library implements most of SQL-92
standard, including transactions, triggers and most of complex queries.

pysqlite makes this powerful embedded SQL engine available to Python
programmers. It stays compatible with the Python database API specification 2.0
as much as possible, but also exposes most of SQLite's native API, so that it
is for example possible to create user-defined SQL functions and aggregates in
Python.

If you need a relational database for your applications, or even small tools or
helper scripts, pysqlite is often a good fit. It's easy to use, easy to deploy,
and does not depend on any other Python libraries or platform libraries, except
SQLite. SQLite itself is ported to most platforms you'd ever care about.

It's often a good alternative to MySQL, the Microsoft JET engine or the MSDE,
without having any of their license and deployment issues.

pysqlite homepage: http://pysqlite.org/

On the homepage, there's also a bug tracking system and a wiki.


Sources:

http://initd.org/pub/software/pysqlite/releases/2.0/2.0.0/pysqlite-2.0.0.tar.gz

Windows binaries for Python 2.3:

http://initd.org/pub/software/pysqlite/releases/2.0/2.0.0/pysqlite-2.0.0.win32-py2.3.exe
 

Windows binaries for Python 2.4:

http://initd.org/pub/software/pysqlite/releases/2.0/2.0.0/pysqlite-2.0.0.win32-py2.4.exe


Advantages of pysqlite 2.0 over pysqlite 1.x


- Straightforward:
  No surprises: pysqlite 2.0 does not convert any types behind your back. With
  default settings, it only supports the database types SQLite supports out of
  the box: INTEGER, REAL, TEXT, BLOB and NULL.

- Documentation:
  pysqlite 2.0 now has usable documentation. The usage manual covers the full
  API.

- Advanced type system:
  It is, however, possible to turn on type detection like in the old pysqlite.
  Types can be detected by their declared type in the "CREATE TABLE" statement.
  Or, for values that don't originate directly from tables or views, it's
  possible to detect the type from the column name via a neat trick. For
  details, look into the pysqlite usage manual. No more "-- types" hack like in
  the old pysqlite.

  Type conversion from Python to SQLite works via PEP-246-like object
  adaptation.

- Fine-grained transaction control:
  pysqlite 2.0 allows to control which type of transactions are opened via the
  transaction_isolation property - None for auto-commit mode or one of SQLite's
  transaction types "DEFERRED", "IMMEDIATE", "EXCLUSIVE".

- executemany() uses precompiled statements for optimal speed.

- Result sets are not prefetched any more, rows are only fetched on demand. So,
  pysqlite 2.0 behaves a lot nicer with respect to memory usage.

- pysqlite 2.0 supports both the "qmark" and "named" paramstyle. So you can
  supply query parameters as sequences or as mappings.

- Performance: 
  pysqlite 2.0 is almost entirely written in hand-coded C. Under most
  circumstances, it is noticeably faster than the old pysqlite. On the pysqlite
  wiki, there's a page with benchmarks:
http://initd.org/tracker/pysqlite/wiki/PysqliteBenchmarks

  The benchmark shows that executemany() is 5 times as fast as in pysqlite 1.1.

Open issues:


pysqlite 2.0 does currently not compile under MacOS X Tiger (10.2 seems to work
for me on the SF compile farm), because of unresolved symbols. Unfortunately, I
don't have access to that platform. I will need a patch with a real fix from a
MacOS X users to fix the problem.


And, for those who have followed the alpha/beta testing:

Changes since pysqlite 2.0beta1
===

- Removed dead code.
- Improved error handling.
- Fixed a leak that occurred when erroneous SQL was sent to execute(many)().
- Recognize REPLACE as DML statement and start transactions appropriately.
- Issue a Warning when users send more than one SQL statement to 
execute(many)().
- Changed a few mappings SQLite error => DB-API exception.
- Added lots of new unit tests so all testable code paths are tested/executed.
  This was done when doing coverage testing using gcov.
- Implemented a nonstandard convenience method cursor.executescript(sql) to
  execute several SQL statements in a bunch, for example for creating a
  database initially.
- The converters dictionary was moved from the Connection object into the
  DB-API module. Register converters with

Re: sync dir's

2005-05-10 Thread Gerhard Haering
On Tue, May 10, 2005 at 05:21:30PM +1000, Timothy Smith wrote:
> i need something which has a python library, so far i haven't seen 
> anything like that for rsync. i'll check out the others but

http://osx.freshmeat.net/projects/pysync/

"""
Pysync has both a demonstration implementation of the rsync and
related algorithms in pure Python, and a high speed librsync Python
extension. The pure Python is not fast and is not optimized, however
it does work and provides a simple implementation of the algorithm
for reference and experimentation. It includes a combination of
ideas taken from librsync, xdelta, and rsync. The librsync Python
extension is less flexible and harder to understand, but is very
fast.
"""
-- 
Gerhard Häring - [EMAIL PROTECTED] - Python, web & database development

pysqlite - Powerful and fast embedded database engine "SQLite" for Python.


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

[ANN] pysqlite 2.0.beta1

2005-05-07 Thread Gerhard Haering
==
pysqlite 2.0.beta1
==

I'm glad to announce pysqlite 2.0.beta1. The API is 98 % stable now. And
pysqlite itself should be a lot more stable too, now.

The most notable changes are a lot of fixed refcount bugs, and the added
documentation.

Download the release here:

Sources: 
http://initd.org/pub/software/pysqlite/releases/2.0/2.0.beta1/pysqlite-2.0.beta1.tar.gz
win32 binaries for Python 2.3: 
http://initd.org/pub/software/pysqlite/releases/2.0/2.0.beta1/pysqlite-2.0.beta1.win32-py2.3.exe
win32 binaries for Python 2.4: 
http://initd.org/pub/software/pysqlite/releases/2.0/2.0.beta1/pysqlite-2.0.beta1.win32-py2.4.exe

pysqlite homepage, bug tracker, wiki: http://pysqlite.org/


Changes since 2.0.alpha4:
=

- Added pysqlite 2.0 documentation: usage guide and source installation guide.
  Adapted from kinterbasdb documentation with the permission of David Rushby.

- Fixed several refcount problems. Per test suite run, lose 0 references
  instead of 550 per test suite run like in alpha4.

- If a database file cannot be opened, raise an OperationalError specifically
  instead of a DatabaseError.

- Call the row factory with (cursor, row_tuple) instead of (row_tuple).

- Fixed a crash in .connect() when you tried to set a keyword argument. It's
  quite annoying that Python doesn't offer a method to extract a single keyword
  argument at C-level easily. Argh! So I bit the bullet and duplicated the
  parameter extraction code.

- The type name of PrepareProtocol was corrected. Only interesting for
  introspection.

- Added more tests to the test suite.

- Implemented cursor.arraysize.

- cursor.rowcount is now -1 instead of None in case of "not determined", like
  the DB-API requires.

- Implemented autocommit mode which replaces the ''no_implicit_begin''
  parameter to the module-level connect(). This way, we're more compatible with
  other DB-API modules. autocommit parameter in module-level connect and also
  an autocommit property of connections. -- The "begin" method of connections
  is gone.


- Completely reworked the advanced type detection:
o connection.register_converter is gone
o instead, the dictionary connection.converters is exposed directly.
o The parameter more_types to the module-level connect is gone.
o Instead, use any combination of PARSE_DECLTYPES and PARSE_COLNAMES for
  the new paramter detect_types.

  PARSE_DECLTYPES will parse out the first word of a declared type and look
  up a converter in connection.converters:

create table foo(col mytype not null)

The decltype would be "mytype not null", but PARSE_DECLTYPES will cut
out "mytype" and look for a converter in converters["mytype"]. If it
finds any, it will use it to convert the value. Otherwise, the standard
SQLite manifest typing will be used.

PARSE_COLNAMES will parse out the column names and look up a converter in
connection.converters:

  cur.execute("select 1 as "colname [mytype]")

  the column names will be parsed for [...], in this case mytype will be
  found as the type for the colum, and the converters dictionary will be
  consulted for an appropriate converter function. If none is found, the
  standard SQLite manifest typing will be used.

   Also, the column names in cursor.description will only consist of the first
   word. So it would be "colname" in our example, not "colname [mytype]".

- cursor.coltypes is gone.

- The connection attribute of cursors is now made available at Python level.
  That's an optional DB-API extension.

- The exception classes are now attributes of the connection objects. That's an
  optional DB-API extension.

- Optimized the _sqlite_step_with_busyhandler() function by making time.time()
  and time.sleep() available at module import instead of importing the "time"
  module each time and getting out the "time" and "sleep" functions each time.
  Big performance improvement.

- Some work on the benchmarks.


- Made the destructor of the Cursor class more stable. It used to crash when an
  error occured in the Cursor *constructor*.

- Implemented a check that the parameter for Cursor() is actually an instance
  of the Connection class (or a subclass thereof).


- Allow long integers as parameters. Re-enable test cases for this.
-- 
Gerhard Häring - [EMAIL PROTECTED] - Python, web & database development

pysqlite - Powerful and fast embedded database engine "SQLite" for Python.


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [ANN] pysqlite 2.0.alpha3

2005-04-21 Thread Gerhard Haering
On Fri, Apr 22, 2005 at 01:34:32AM +0200, Gerhard Haering wrote:
> ===
> pysqlite 2.0.alpha3
> ===
> 
> I'm glad to announce pysqlite 2.0.alpha3, which will be the last alpha
> release.  Documentation and more testing will be what I'm concentrating
> on in the beta test phase, so here's the explanation of the new features
> within the change log.
> 
> Download it from Sourceforge:
> http://sourceforge.net/project/showfiles.php?group_id=54058
> [...]

Forgot the pysqlite homepage: http://pysqlite.org/ with Wiki, BugTracker
and Subversion repository.

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

[ANN] pysqlite 2.0.alpha3

2005-04-21 Thread Gerhard Haering
===
pysqlite 2.0.alpha3
===

I'm glad to announce pysqlite 2.0.alpha3, which will be the last alpha
release.  Documentation and more testing will be what I'm concentrating
on in the beta test phase, so here's the explanation of the new features
within the change log.

Download it from Sourceforge:
http://sourceforge.net/project/showfiles.php?group_id=54058

Changes since 2.0.alpha2


- Fixed various memory leaks and refcount problems.

- Connection, Cursor and row factories.

o Connection factory: Subclass pysqlite2.dbapi2.Connection and
provide the parameter to pysqlite2.dbapi2.connect to use that class
instead.

o Cursor factory: Subclass pysqlite2.dbapi2.Cursor and provide the
parameter to the cursor() method of the Connection class so that
cursors of your class are created.

o Row factory: set the row_factory attribute in your cursor to a
callable that accepts a tuple and returns the "real" row object.

Combine the three for maximimal power and convenience like in the
following example, where we transparently use the db_row module from
http://opensource.theopalgroup.com/ to be able to access columns by
name instead of by index. A fast alternative to PgResultSet in
pysqlite 1.x.

from pysqlite2 import dbapi2 as sqlite
import db_row

class DbRowConnection(sqlite.Connection):
def __init__(self, *args, **kwargs):
sqlite.Connection.__init__(self, *args, **kwargs)

def cursor(self):
return DbRowCursor(self)

class DbRowCursor(sqlite.Cursor):
def execute(self, *args, **kwargs):
sqlite.Cursor.execute(self, *args, **kwargs)
if self.description:
self.row_factory = db_row.IMetaRow(self.description) 

con = sqlite.connect(":memory:") #, factory=DbRowConnection)
cur = con.cursor(factory=DbRowCursor)
cur.execute("create table test(i integer)")
cur.execute("select 4+5 as foo")
for row in cur:
print ">", row
print row["foo"]
print cur.fetchone()["foo"]

- The first parameter to .execute() and .executemany() can now also be a
  Unicode string:

  cur.execute(u"insert into person(name) values ('Häring')")

- The conversion in SQLite types mode for TEXT is now unicode, not UTF-8
  encoded bytestrings.

  i. e. if the column type is TEXT, then you get back unicode objects now.

- Implemented user functions and aggregates.

Where a number of parameters is expected, -1 means variable number
of arguments.

The functions and aggregates must return values of the type NoneType, int,
float, str, unicode or buffer (BLOB).

Example code:

def f(*args):
return sum(args)

class Sum:
def __init__(self):
self.sum = 0

def step(self, x):
self.sum += int(x)

def finalize(self):
return self.sum


con = sqlite.connect(":memory:")

# Syntax: function name, number of parameters, callable
con.create_function("myfunc", -1, f)

# Syntax: aggregate name, number of parameters, aggregate class
con.create_aggregate("myaggr", 1, Sum)

- Added MANIFEST.in file, so that bdist_rpm and sdist will work.

- Release GIL during SQLite calls.

- After a cursor.executemany, cursor.rowcount now delivers the sum of all
  changes, not only the changes in the last call to the prepared statement.


- Implemented checks that the SQLite objects are used from the same
  thread they were created in. Otherwise raise a ProgrammingError.

  If you're 100% sure that you want to use the same connection object in
  multiple threads, because you use proper locking, you can turn the
  check of by using the parameter ''check_same_thread=0''.

- Allow for parameters to be dictionaries, too. Then, the name-based
  binding in SQLite 3 is used:

cur.execute("select name from test where name=:name", {"name": "foo"})


- Improved error handling.

- Allow for explicit transaction start:

>>> con = sqlite.connect(":memory:", no_implicit_begin=True)
...
>>> con.begin()
...
>>> con.rollback()
...
>>> con.commit()

- Implemented cursor.lastrowid

- Avoid code duplication: handle execute() and executemany() internally
  in the same function.



signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: DBAPI Paramstyle

2005-03-28 Thread Gerhard Haering
On Mon, Mar 28, 2005 at 01:43:28PM -0800, Andy Dustman wrote:
> Tim Roberts wrote:
> > [prepared statements]
> 
> mx.ODBC does, since it is an ODBC implementation. I would be very
> surprised if the Oracle adapter did not. MySQLdb does not yet, but
> probably will by the end of summer (with MySQL-4.1 or newer).

pysqlite 2.0 (alpha) does this (compiled statements) too, but currently
only for .executemany().

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: fastest postgresql module

2005-03-17 Thread Gerhard Haering
On Fri, Mar 18, 2005 at 09:31:45AM +1000, Timothy Smith wrote:
> Leif B. Kristensen wrote:
> 
> >Timothy Smith skrev:
> >>has anyone got some hard numbers on which pg access module is the
> >>fastest, i currently use pypgsql, but i can't help but feel it's a
> >>little slow.
> >>3 seconds to connect, send one query, get th return data.
> >>i'm on a adsl 1.5mbit/256kbit link, the server is on a 10/10mbit, and
> >>i have a ping of 245ms average.
> >>maybe pypgsql does too much backand forth, i don't know.
> >>   
> >>
> >
> >You might want to try psycopg, it's claimed to be optimized for speed.
> > 
> >
> my only issue with psycopg, is last time i looked they had no win32 port?

It has one now.

But (of course depending on your query), your issue is not dependant on
the particular PostgreSQL, I think.

You'll probably also have it with executing a plain vanilla script via
psql.

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

[ANN] pysqlite 2.0.alpha2

2005-03-13 Thread Gerhard Haering
pysqlite 2.0.alpha2
===

The last release was back in Nov. 2004, but pysqlite 2 development is going on
faster now. So it's time for another alpha release to collect feedback.

Please report any bugs you find on the pysqlite mailing list, or even better on
the trac tracking system at http://initd.org/tracker/pysqlite

Downloads at Sourceforge:
http://sourceforge.net/project/showfiles.php?group_id=54058&package_id=134545

The project itself at http://pysqlite.org/ 



Changes since pysqlite 2.0.alpha1
=

Namespaces change:
- the DB-API module is now called pysqlite2.dbapi2 instead of sqlite, you
  typically import it using "from pysqlite2 import dbapi2 as sqlite"

DB-API compliance:
- implemented all module-level constants and functions that are required
  for DB-API compliance

Type system:
<< Type conversion SQLite => Python >>

** SQLite types mode (default **
- nothing happens behind your back. if you SELECT, you get back the types
  SQLite reports. So, you will only get strings, ints, floats, and BLOBs
  (BLOBs as Python buffer objects)

** pysqlite types mode (have to activate explicitly) **
- the returned type depends on the decleared column for the SELECTed data.
  To use it successfully, you must register converters for the column types
  (see below). You can also set the declared column types explicitly using
  the coltypes property of cursors (see below)
- new method register_converter(coltypes, converter) for connection objects:
* con.register_converter("int": int)
* con.register_converter("unicode": unicode)
* con.register_converter("boolean": bool)
* con.register_converter("foo": lambda x: "<%s>" % x)
* class Bar: ...
  con.register_converter("bar": Bar)
- new property coltypes of cursor objects:
cur.coltypes = {"col1": int}
cur.execute("select foo as col1 from bar")

<< Type conversion Python => SQLite >>
- Like in PEP 246 (Object Adaptation)
- the default protocol does nothing, except handle Unicode strings
- the protocol's __adapt__ returns the SQLite value directly at the moment
  (this will perhaps change in the next release)
- example protocol:
class P(dbapi2.SQLitePrepareProtocol):
def __adapt__(self, obj):
if isinstance(obj, Point):
return "(%s/%s)" % (obj.x, obj.y)
else:
return dbapi2.SQLitePrepareProtocol.__adapt__(self, obj)
con = sqlite.connect(":memory:", more_types=True, prepareProtocol=P())

Connect call:
- Syntax is now:
con = sqlite.connect(database, timeout, protocol, more_types)

* timeout: timeout parameter in seconds, until error raised if
  concurrent access. in alpha1, it just failed immediately
* protocol: see Type system
* more_types: set to true if you want to use the pysqlite types mode
  instead of the default SQLite one

Misc.:
- pysqlite.dbapi2 now has constants version and version_info


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: User Identification

2005-02-08 Thread Gerhard Haering
On Tue, Feb 08, 2005 at 12:29:48PM -, Bob Parnes wrote:
> I have a python program on a server of an all-linux network. It needs to 
> record the user name running it. Is there a way for the program to extract
> the name from the system without a separate log-in dialog?

os.environ["USER"] for a start?

os.getuid() will give you the user id, but I don't know if Python has
methods to look up more information from that from /etc/passwd or
whereever from.

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 2.4 binaries for accessing PostgreSQL from Windows?

2005-02-04 Thread Gerhard Haering
On Fri, Feb 04, 2005 at 02:24:50AM -0800, Frank Millman wrote:
> Hi all
> 
> The subject line says it all.
> 
> I have been using pypgsql to access PostgreSQL from Linux and from
> Windows, and it works fine.
> 
> I am upgrading to Python 2.4. I can recompile pypgsql for Linux, but I
> do not have a Windows compiler. SourceForge has a binary for Python
> 2.3, which is the one I have been using. It does not have one for 2.4.
> 
> I tried the psycopg site, but it does not seem to have binaries at all.
> 
> Does anyone know if either of these will be available in binary form
> for Python 2.4 on Windows?

http://ghaering.de/pypgsql/

These are SSL-enabled dynamically linked and need the PostgreSQL client
libraries installed. I recommend the ones from PgAdmin3, which have SSL
support. They were only tested with these.

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Debian says "Warning! you are running an untested version of Python." on 2.3

2005-01-13 Thread Gerhard Haering
On Thu, Jan 13, 2005 at 12:56:38PM -, Alex Stapleton wrote:
> Whenever I run python I get
> 
> "Warning! you are running an untested version of Python."
> 
> prepended to the start of any output on stdout. [...]

ROFL. Are you using testing, sid or experimental?  I expect overzealous
patching from Debian developers, but this is the worst I've heard of.

To look what's going on and how to fix it, you could try to download the
sources with "apt-get source ...", which will give you the standard
Python tarball + the Debian patches. Then you can look into the Debian
patches wtf they're doing there.

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python.org, Website of Satan

2005-01-12 Thread Gerhard Haering
On Wed, Jan 12, 2005 at 10:15:34AM -0500, Jane wrote:
> [...] Some people have too much time on their hands...

OMG, PyPy is full of evil, too!!!1

print sum([ord(x) for x in "PyPy"])

or, if you haven't upgraded to 2.4, yet:

import operator
print reduce(operator.add, [ord(x) for x in "PyPy"])

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Embedding a restricted python interpreter

2005-01-06 Thread Gerhard Haering
On Thu, Jan 06, 2005 at 07:32:25AM -0800, Paul Rubin wrote:
> Jp Calderone <[EMAIL PROTECTED]> writes:
> >   A Python sandbox would be useful, but the hosting provider's excuse
> > for not allowing you to use mod_python is completely bogus.  All the 
> > necessary security tools for that situation are provided by the 
> > platform in the form of process and user separation.
> 
> But mod_python is an apache module and runs in the same apache process
> with other users' scripts.

Which is why it's a good idea for each customer to have it's own system user
and their virtual hosts running under this uid. Which was the idea for the
perchild MPM for Apache 2 - which is abandoned now :-( muxmpm is a replacement
project in beta.

This really sucks when you use Apache2. I myself did make the switch some time
ago, then noticed that this (for me) important feature was missing. It now
works, somehow, but to make it work properly I'd need to either:

- go back to Apache 1.3.x, missing some nice improvements
- use different webservers per user, put them together with mod_proxy (yuck!)

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: DB-API 2.0 in pysqlite and pgdb

2005-01-04 Thread Gerhard Haering
On Sat, Jan 01, 2005 at 06:33:24PM +0300, Roman Suzi wrote:
> 
> Happy New Year to all Pythoneers!
> 
> I am playing with pysqlite and pgdb and their DB-API conformancy.
> It was quite interesting to know:
> 
>  - sqlite doesn't have mandatory helper-functions Date, Tim, etc.
>(due to an error int it's __init__, but this is quite obvious to correct
>or just to use mx.Date, mx.Time)

Yes, that's an oversight.

> more serious mishaps with pgdb (postgresql-python):
> it doesn't know how to quote date-time data for the objects
> it has constructors itself. [...]

You may have better luck with pyPgSQL or psycopg.

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Probleme mit der Installation der openSource Bittorrent.... python vs JAVA

2004-12-30 Thread Gerhard Haering
On Fri, Dec 31, 2004 at 12:02:24AM +0800, Craig Ringer wrote:
> On Thu, 2004-12-30 at 23:31, JZ wrote:
> > Dnia 30 Dec 2004 07:24:45 -0800, xunling napisa??(a):
> > 
> > > ich hätte da mal eine Frage zum Azureus bttrn client.
> > 
> > This is not German newsgroup! Write English or none.
> 
> While that may be true, there are nicer ways to express the
> sentiment. I don't know about you, but I frequently feel rather
> poorly about demanding that others speak my language when I have no
> (or in this case very, very minimal) ability to speak theirs. [...]

I can understand your emotions here. OTOH it's typical for Germans
(and even more so French) to assume that everybody speaks their
language. And if they don't, to make an effort to figure out what the
heck they want. I'm sure it's not only not knowing the medium and
environment [*], but also plain arrogance.

-- Gerhard

[*] The very minimum of netiquette would require to read a few
messages in the newsgroup/mailing list and check what it is about and
in what language it is.


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

[OT] Azureus Java BitTorrent client - was: Re: Probleme mit der Installation der openSource Bittorrent.... python vs JAVA

2004-12-30 Thread Gerhard Haering
On Thu, Dec 30, 2004 at 07:24:45AM -0800, xunling wrote:
> Hallo,
> 
> ich hätte da mal eine Frage zum Azureus bttrn client. [...]

[I'd have a question about the Azureus BitTorrent client.]

While the original BitTorrent is implemented in Python, this is not
the right mailing list/newsgruop for Java BitTorrent clients.

The Azureus homepage has a guide on how to get it running on various
operating systems. Perhaps these instructions will help you:
http://azureus.sourceforge.net/howto_win.php

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Problem with msvcrt60 vs. msvcr71 vs. strdup/free

2004-12-21 Thread Gerhard Haering
Hello,

I used to build Python extension modules with mingw. Now, Python has
switched to the MSVCR71 runtime with version 2.4, and I thought mingw
has support for this. But I get problems with symbols being referenced
from the wrong DLLs.

You can see the problem by compiling this:

##
#include 

int main()
{
char* s;
int i;


for (i = 0; i < 10; i++) {
s = strdup("foo");
free(s);
}

return 0;
}
##

with gcc x.c -lmsvcr71

Then if you run a.exe it crashes.

If you use depends.exe on it, you see that it resolves strdup() via
msvcrt, but the rest with msvcr71.dll. That's why strdup() is using
the one malloc, but free() a different free() from the other DLL,
which is undoubtedly the reason for the crash.

Is there any way I can force mingw to not link in msvcr for things
like strdup?

I think I found a very strange and ugly workaround: if I use _strdup()
instead of strdup(), it is resolved using the msvcr71 DLL.

I hope it is (soon) possible to use only the msvcr71 runtime from the
mingw compiler.

-- Gerhard
-- 
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with msvcrt60 vs. msvcr71 vs. strdup/free

2004-12-21 Thread Gerhard Haering
Of not so much interest to most Pythoneers, but ...

I cross-posted this to python-list to make people aware that thare are
*real* problems with mingw + Python 2.4, not only theoretical ones.

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Best GUI for small-scale accounting app?

2004-12-20 Thread Gerhard Haering
On Mon, Dec 20, 2004 at 05:04:33PM +0100, Thomas Heller wrote:
> Bulba! <[EMAIL PROTECTED]> writes:
> 
> > I'll soon start development of a specialized small app and need 
> > to choose GUI for it. 
> 
> Quoting a somewhat popular german blogger, on the state of cross
> platform Python GUI toolkits (http://blog.schockwellenreiter.de/7282):
> 
> Ansonsten habe ich nach einiger Überlegung entschieden, daß ich in
> Zukunft Programme, die graphische Ausgaben und eine GUI benötige, in
> Java schreiben werde. Ich habe zu lange auf Besserung der
> (plattformübergreifenden) GUI-Situation in Python gewartet. Tkinter ist
> nur noch krank, wxPython scheint zwar nicht schlecht zu sein, doch es
> mangelt an jeder Dokumentation (und ich habe weder Zeit noch Lust, mich
> durchzuwursteln) und PyQt ist zwar etwas besser dokumentiert, aber nicht
> so weit verbreitet und es besitzt eine etwas seltsame Lizenz. Swing
> dagegen ist hervorragend dokumentiert, Eclipse hat sich zu einer
> brauchbaren IDE gemausert und Java selber hat -- trotz der geschweiften
> Klammern -- doch soviel von Oberon geerbt, daß ich nicht gleich Pickel
> bekomme, wenn ich darin programmieren muß. ;o)

Quick and dirty translation for those who don't read German:

Besides, after some consideration I decided for programs that need graphical
output and GUI I will use Java. I have waited too long for improvements of the
(cross-platform) GUI situation in Python. Tkinter is just sick, wxPython
doesn't seem bad, but it lacks any documentation (and I have neither time nor
do I fancy muddling through that) and PyQt is a little better documented, but
not so widespread and has a somewhat strange license. Swing on the other hand
is superbly documented, Eclipse has become a sable IDE and Java itself -
despite the curly braces - has inherited so much from Oberon that I don't get
spots when I need to program in it. ;o)

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: BASIC vs Python

2004-12-17 Thread Gerhard Haering
On Fri, Dec 17, 2004 at 11:49:22AM +0100, Peter Otten wrote:
> Peter Hickman wrote:
> > [..] Basic has progressed much since you last looked at it, time
> > to update your facts. Basic has recursion, it compiles to native
> > code, it has objects, can be event driven and everything else you
> > would expect of a language.
> > 
> > Computing changes too fast to allow you to think you know it all. Keep up
> > to date granddad.
> > 
> > However what basic doesn't have is a portable language definition.
> 
> May you could give us an idea of the current state of basic affairs then by
> translating the following example snippet: [...]

IIRC BASIC does have a portable language definition:
ANSI BASIC, which is the old crap with GOTO and GOSUB that nobody in
their right mind would want to use nowadays ...

I only know about Visual Basic 5.0/6.0 and a little about Visual Basic
.NET and thanks to the .NET standard library it's possible to write
the Python code you gave us in relatively clean VB.NET with the
collections classes.

In VB6, it would an exercise of working around the limitations of the
data structures.

-- Gerhard
-- 
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: why not arrays?

2004-12-16 Thread Gerhard Haering
On Thu, Dec 16, 2004 at 05:51:18AM -0800, Rahul wrote:
> Hi.
> I just wanted to know why arrays have not been included as a builtin
> datatype like lists or dictionaries? The numpy extension shows that it
> can be implemented. then why not include arrays in core python?

Arrays are included in a module in the standard module called 'array'.

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: problem with datetime

2004-12-10 Thread Gerhard Haering
On Fri, Dec 10, 2004 at 04:19:56AM -0800, [EMAIL PROTECTED] wrote:
> Relatively new to python.  I can get the following to work from the
> command line:
> 
> Python 2.3.4 (#2, Aug 18 2004, 21:49:15)
> [GCC 3.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import datetime
> >>> d = datetime.datetime.today()
> >>> d
> datetime.datetime(2004, 12, 10, 6, 13, 28, 154472)
> >>>
> 
> But when I try to run the following small program I get the following
> results:
> 
> import datetime
> d = datetime.datetime.today()
> print d
> 
> Traceback (most recent call last):
> File "datetime.py", line 1, in ?
> import datetime
> File "/home/bob/pyshow/datetime.py", line 3, in ?
 ^^^

> d = datetime.datetime.today()
> AttributeError: 'module' object has no attribute 'today'
> 
> Do I need to change a path?
> Running python 2.3.4 on linux redhat 7.3.

Change your script's name from datetime.py to something else. Otherwise the
script will import itself, and not the datetime module from the standard
library.

-- Gerhard
-- 
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: I need to create the table and I want to edit its content from www level.

2004-12-08 Thread Gerhard Haering
On Wed, Dec 08, 2004 at 09:03:54AM -0800, Rootshell wrote:
> Hello.
> 
> I have one more problem with 'tabla.py' file: 

We don't know about the files on your harddisk ;-)

> Can't do the compilation 'cause something wrong is happening with
> module 'posix'.

Whoever wrote tabla.py probably didn't read the documentation for the
posix module, which says:

*Do not import this module directly*. Instead, import the module
os, which provides a portable version of this interface.

> The message is as follows: "no module name posix".
> I guess that it is necessary to import it.
> Unfortunately 'posix' seems to be the unix module and my platform is WinXP.
> I'll appreciate any help.

Try replacing posix with os in the source code.

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PySQLLite Speed

2004-12-03 Thread Gerhard Haering
On Fri, Dec 03, 2004 at 06:06:11AM -0500, Kent Johnson wrote:
> If your data is (or can be) created by an iterator, you can use this recipe 
> to group the data into batches of whatever size you choose and write the 
> individual batches to the db.
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/303279

If your data is (or can be) created by an iterator, then you might find it
interesting that *pysqlite2*'s .executemany() not only works on lists, but also
on iterators.

Example:

import pysqlite2.dbapi2 as sqlite
...
# A generator function (which returns an iterator)
def gen():
for i in xrange(5):
yield (5, 'foo')

cu.executemany("insert into foo(x, y) values (?, ?)", gen())

So, in pysqlite2, .executemany() and iterators provide best
performance.  .executemany() reuses the compiled SQL statement (so the
engine only needs to parse it once), and the iterator, if used
smartly, reduces the amount of memory used because you don't need to
construct large lists any more.

I hope I don't create too much confusion here ;-)

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PySQLLite Speed

2004-12-03 Thread Gerhard Haering
On Thu, Dec 02, 2004 at 08:39:31PM -0800, Kevin wrote:
> Hello All,
> 
> I wanted to thank Roger Binn for his email.  He had
> the answer to my issue with writing speed.  It's
> actual made an incredible change in the preformace.  I
> didn't have to go all the way to implementing the
> synchronous mode(for my app).  Previously, I was
> insert one record at a time.  The key was to write
> them all at one time.  I moved up to a 13 meg file and
> wrote it to the db in secs.  Now the issue is the 120
> meg of RAM consumed by PyParse to read in a 13 meg
> file.  If anyone has thoughts on that, it would be
> great.  Otherwise, I will repost under a more specific
> email.
> 
> Thanks,
>   Kevin
> 
> 
> 
> db.execute("begin")
> 
> while i < TriNum
>db.execute("""insert into TABLE(V1_x)
> values(%f),""" (data[i]))
> i = i + 1

If you're using pysqlite 2.0alpha, then .executemany() will boost performance
*a lot*. For pysqlite 1.x, unfortunately, it won't make any difference. But
generally, .executemany() is a good idea.

Also note that the preferred way of using transactions is to let the DB-API
adapter BEGIN the connection for you, then invoke .commit() on the connection
object.

Sending BEGIN/ROLLBACK/COMMIT via .execute() is bad.

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PySQLLite Speed

2004-12-03 Thread Gerhard Haering
On Thu, Dec 02, 2004 at 04:22:25PM -0800, Kevin wrote:
> Hello All,
> 
> I wondering if anyone has encountered the same issue
> with using PySQL.  This is my first time using this DB
> so this preformance may be typical.  I'm reading an
> ASCII file through PyParse that contains about 1.3Meg
> of Float data.  8000 Triangles (3 Vertexes and 1
> Normal).  This file reads fine the PC has a
> utilization of 50% for Python.  Then when it starts to
> write the Database, the PC Util drops to 1-2% and it
> takes forever.  I'm not PC related preformance
> barriers that I'm aware of.
> 
> Regards,
>Kevin
> 
> OS : Windows
> PySQLLite Version: 1.1.5

You're not doing silly things like using pysqlite's autocommit mode or
.commit()-ing too often, are you?

Like already mentioned, PRAGMA SYNCHRONOUS should boost performance.

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: installer

2004-12-02 Thread Gerhard Haering
On Sat, Dec 04, 2004 at 12:15:04AM +0530, km wrote:
> Hi all,
> 
> does python have a default module installer inbuilt as for perl in
> windows ? 

No, there is no CPAN-like system for Python. You will have to search,
download and install your third-party Python libraries yourself.

OTOH, there are some good places to look for these, like
http://www.python.org/pypi?:action=browse and the libraries are
usually packaged with distutils into win32 installers that you can
then easily install/uninstall.

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: exec size

2004-12-02 Thread Gerhard Haering
On Fri, Dec 03, 2004 at 10:53:30PM +0530, km wrote:
> Hi Gerhard,
> ya have tried stripping python2.4 it came to 952K.  now does the size
> diference before and after strip affect the runtime of a python program ? 

No, it only saves some space on disk. The interpreter doesn't need any less
memory at runtime just because the executable size is less now. The OS wouldn't
load the symbols into memory, unless they're used for debugging, anyway.

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: exec size

2004-12-02 Thread Gerhard Haering
On Fri, Dec 03, 2004 at 10:14:48PM +0530, km wrote:
> Hi all,
> just curious to know why /usr/bin/python2.3  is 890K and
> /usr/bin/python2.4 is 3.5M  in linux ? 

Did you try already removing the debug symbols with strip(1)?

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: 2.4 or 2.3.4 for 2.3 software?

2004-12-01 Thread Gerhard Haering
On Wed, Dec 01, 2004 at 10:23:55AM +0100, Jens Bloch Helmers wrote:
> Can we expect the current release of 2.4 to be just as reliable as
> 2.3.4 for 2.3 compliant software?

Only time will tell.

I myself had never had any problems with 2.x.0 versions of Python. Only early
2.0.x had a few problems in the C API for me.

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Building web graphics with Python

2004-11-29 Thread Gerhard Haering
On Mon, Nov 29, 2004 at 10:35:00AM +0100, Peter Maas wrote:
> Steven Feil schrieb:
> >I am wondering if there is a light weight Python library for producing
> >web graphics on-the-fly. There is a C-language library called gd that
> >can be used in a CGI program to produce gif images.  The Library can
> >be used to produce graphics images that are dynamically generated.  I
> >was wondering if there is something similar for Python.
> 
> You can create a Python wrapper for gd with SWIG or Pyrex. PIL (Python
> Imging Library) and Piddle are native Python solutions.

No need to reimplement a gd wrapper:

http://dmoz.org/Computers/Programming/Languages/Python/Modules/Graphics/
http://newcenturycomputers.net/projects/gdmodule.html

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Psycopg2 mirror availability

2004-11-29 Thread Gerhard Haering
On Sun, Nov 28, 2004 at 10:06:20PM +, Andrew James wrote:
> Gerhard,
> What's your public key? Your messages are all showing up as having an
> invalid signature!

The key should be on the keyservers now.

-- Gerhard


signature.asc
Description: Digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list