Important news!
1) The new DAL (dal.py) passes all the tests I have.
2) The new DAL has integrated GAE support (datastore) without need for
contrib/gql.py
3) The new DAL is better because more customizable, smaller (10%), and
more readable
4) The new DAL is one single file and it does not depend on web2py.
You can use it with other frameworks out of the box without tweaking.
TODO:
- test test test and more tests
- benckmark old dal vs new dal
- replace old dal with new dal in stable web2py
I may keep working on this make more corrections/improvements.
Massimo
On Dec 6, 1:12 pm, ron_m <ron.mco...@gmail.com> wrote:
> Ok that worked for starting the welcome app. As Jonathan suggests the
> self.uri in ConnectionPool.pools[self.uri] needs to be instance.uri as
> well.
>
> I will look to see why the application I am working on won't connect
> to postgresql - timeout after 5 tries and get back if I find the
> cause.
>
> Thanks
> Ron
>
> On Dec 6, 9:25 am, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > please try replace
>
> > @staticmethod
> > def close_all_instances(action):
> > """ to close cleanly databases in a multithreaded environment
> > """
> > if not hasattr(thread,'instances'):
> > return
> > while thread.instances:
> > instance = thread.instances.pop()
> > action(instance)
> > # ## if you want pools, recycle this
> > connection
> > really = True
> > if instance.pool_size:
> > sql_locker.acquire()
> > pool = ConnectionPool.pools[self.uri]
> > if len(pool) < instance.pool_size:
> > pool.append(self.connection)
> > really = False
> > sql_locker.release()
> > if really:
> > self.connection.close()
> > return
>
> > with
>
> > @staticmethod
> > def close_all_instances(action):
> > """ to close cleanly databases in a multithreaded environment
> > """
> > if not hasattr(thread,'instances'):
> > return
> > while thread.instances:
> > instance = thread.instances.pop()
> > action(instance)
> > # ## if you want pools, recycle this
> > connection
> > really = True
> > if instance.pool_size:
> > sql_locker.acquire()
> > pool = ConnectionPool.pools[self.uri]
> > if len(pool) < instance.pool_size:
> > pool.append(instance.connection)
> > really = False
> > sql_locker.release()
> > if really:
> > instance.connection.close()
> > return
>
> > I think this is global replace error.
>
> > On Dec 6, 11:13 am, ron_m <ron.mco...@gmail.com> wrote:
>
> > > I did an update of trunk using hg and copied dal.py over sql.py in
> > > gluon and start the web2py.py server and still get
>
> > > Traceback (most recent call last):
> > > File "/home/camcentral/Dev/web2py_hg/gluon/main.py", line 446, in
> > > wsgibase
> > > BaseAdapter.close_all_instances(BaseAdapter.commit)
> > > File "/home/camcentral/Dev/web2py_hg/gluon/sql.py", line 276, in
> > > close_all_instances
> > > self.connection.close()
> > > NameError: global name 'self' is not defined
>
> > > main is calling BaseAdapter.close_all_instances() as a class method so
> > > the self reference at line 276 blows up because there is no instance
> > > of a class. I see from the latest post we can test dal.py standalone,
> > > are we not to be testing as part of web2py yet because of integration
> > > issues?
>
> > > Ron
>
> > > On Dec 5, 7:11 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > > > started work integrating with gluon/contrib/gql.py do
>
> > > > please keep testing dal.py
>
> > > > now it no longer requires web2py and you can do
>
> > > > % python>>> from dal import DAL, Field
> > > > >>> db=DAL('sqlite://file.sqlite')
> > > > >>> db.define_table('person',Field('name'))
>
> > > > etc etc.
>
> > > > you ONLY need dal.py
>
> > > > On Dec 5, 1:56 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > > > > in trunk!
>
> > > > > On Dec 5, 1:38 pm, "mr.freeze" <nat...@freezable.com> wrote:
>
> > > > > > Auth.define_tables doesn't have a fake_migrate attribute so I added
> > > > > > one (want a patch?). Once I did, my .table files were re-created
> > > > > > when
> > > > > > migrate=True and fake_migrate=True. It still works once I set
> > > > > > fake_migrate to False. I will test it against the new DAL now.
>
> > > > > > On Dec 5, 1:24 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > > > > > > can you try migrate=True, fake_migrate=True?
>
> > > > > > > On Dec 5, 1:05 pm, "mr.freeze" <nat...@freezable.com> wrote:
>
> > > > > > > > I tried and the file modification times of the .table files do
> > > > > > > > not
> > > > > > > > change. I deleted them (safely backed up) and they are not
> > > > > > > > re-created.
>
> > > > > > > > On Dec 5, 12:56 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > > > > > > > > It is the same. try without delete them.
>
> > > > > > > > > On Dec 5, 12:53 pm, "mr.freeze" <nat...@freezable.com> wrote:
>
> > > > > > > > > > I reverted to the old DAL and did the steps you outlined.
> > > > > > > > > > Same result.
> > > > > > > > > > Should I delete the .table files first?
>
> > > > > > > > > > On Dec 5, 12:41 pm, mdipierro <mdipie...@cs.depaul.edu>
> > > > > > > > > > wrote:
>
> > > > > > > > > > > for the troublesome table (auth_user) set migrate=False,
> > > > > > > > > > > fake_migrate=True reload, then migrate=True and remove
> > > > > > > > > > > fake_migrate.
> > > > > > > > > > > It should fix your broken .table.
>
> > > > > > > > > > > Should work with old and with new dal.
>
> > > > > > > > > > > On Dec 5, 11:47 am, "mr.freeze" <nat...@freezable.com>
> > > > > > > > > > > wrote:
>
> > > > > > > > > > > > I seem to remember this failing before though.
> > > > > > > > > > > > Something about
> > > > > > > > > > > > my .TABLE files being out of sync. What is the
> > > > > > > > > > > > procedure to re-sync
> > > > > > > > > > > > them?
>
> > > > > > > > > > > > On Dec 5, 10:45 am, "mr.freeze" <nat...@freezable.com>
> > > > > > > > > > > > wrote:
>
> > > > > > > > > > > > > I have a global variable migrate_db which is passed
> > > > > > > > > > > > > to all
> > > > > > > > > > > > > define_table functions. I switched it to false and
> > > > > > > > > > > > > get this:
> > > > > > > > > > > > > Traceback (most recent call last):
> > > > > > > > > > > > > File "C:\web2py\gluon\restricted.py", line 188, in
> > > > > > > > > > > > > restricted
> > > > > > > > > > > > > exec ccode in environment
> > > > > > > > > > > > > File "C:/web2py/applications/main/models/db.py",
> > > > > > > > > > > > > line 34, in
> > > > > > > > > > > > <module>
> > > > > > > > > > > > > readable=False, default=""),migrate=migrate_db)
> > > > > > > > > > > > > File "C:\web2py\gluon\sql.py", line 1406, in
> > > > > > > > > > > > > define_table
> > > > > > > > > > > > > obj = str(obj)
> > > > > > > > > > > > > File "C:\web2py\gluon\sql.py", line 1858, in _create
> > > > > > > > > > > > > 'boolean': 'CHAR(1)',
> > > > > > > > > > > > > File "C:\web2py\gluon\sql.py", line 1024, in
> > > > > > > > > > > > > <lambda>
> > > > > > > > > > > > > 'string': 'CHAR(%(length)s)',
> > > > > > > > > > > > > File
> > > > > > > > > > > > > "C:\Python26\lib\site-packages\MySQLdb\cursors.py",
> > > > > > > > > > > > > line 166,
> > > > > > > > > > > > > in execute
> > > > > > > > > > > > > self.errorhandler(self, exc, value)
> > > > > > > > > > > > > File
> > > > > > > > > > > > > "C:\Python26\lib\site-packages\MySQLdb\connections.py",
> > > > > > > > > > > > > line
> > > > > > > > > > > > > 35, in defaulterrorhandler
> > > > > > > > > > > > > raise errorclass, errorvalue
> > > > > > > > > > > > > OperationalError: (1050, "Table 'auth_user' already
> > > > > > > > > > > > > exists")
>
> > > > > > > > > > > > > On Dec 5, 10:37 am, "mr.freeze"
> > > > > > > > > > > > > <nat...@freezable.com> wrote:
>
> > > > > > > > > > > > > > Let me backup the web2pyslice.com database and I
> > > > > > > > > > > > > > will try.
>
> > > > > > > > > > > > > > On Dec 5, 10:36 am, mdipierro
> > > > > > > > > > > > > > <mdipie...@cs.depaul.edu> wrote:
>
> > > > > > > > > > > > > > > can you try migrate=True (add a dummy field and
> > > > > > > > > > > > > > > the remove it) with
> > > > > > > > > > > > > > > mysql?
>
> > > > > > > > > > > > > > > On Dec 5, 10:20 am, "mr.freeze"
> > > > > > > > > > > > > > > <nat...@freezable.com> wrote:
>
> > > > > > > > > > > > > > > > Working with mysql also. I tested with
> > > > > > > > > > > > > > > > migrate=False. Nice work!
>
> > > > > > > > > > > > > > > > On Dec 5, 10:12 am, "mr.freeze"
> > > > > > > > > > > > > > > > <nat...@freezable.com> wrote:
>
> > > > > > > > > > > > > > > > > Okay, web2pyslices.com source runs against it
> > > > > > > > > > > > > > > > > with a sqlite database.
> > > > > > > > > > > > > > > > > I will test with mysql.
>
> > > > > > > > > > > > > > > > > On Dec 4, 11:02 pm, mdipierro
> > > > > > > > > > > > > > > > > <mdipie...@cs.depaul.edu> wrote:
>
> > > > > > > > > > > > > > > > > > One more test please. I do not have mysql
> > > > > > > > > > > > > > > > > > installed here so I did not
> > > > > > > > > > > > > > > > > > test pooling. This helps a lot. thanks.
>
> > > > > > > > > > > > > > > > > > On Dec 4, 10:33 pm, "mr.freeze"
> > > > > > > > > > > > > > > > > > <nat...@freezable.com> wrote:
>
> > > > > > > > > > > > > > > > > > > Looking closer, there is a syntax error
> > > > > > > > > > > > > > > > > > > on line 189: COnnectionPool
> > > > > > > > > > > > > > > > > > > After I fixed that, I get:
> > > > > > > > > > > > > > > > > > > Traceback (most recent call last):
> > > > > > > > > > > > > > > > > > > File "C:\web2py\gluon\main.py", line
> > > > > > > > > > > > > > > > > > > 488, in wsgibase
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > BaseAdapter.close_all_instances(BaseAdapter.rollback)
> > > > > > > > > > > > > > > > > > > File "C:\web2py\gluon\sql.py", line
> > > > > > > > > > > > > > > > > > > 163, in close_all_instances
> > > > > > > > > > > > > > > > > > > pool =
> > > > > > > > > > > > > > > > > > > ConnectionPool._pools[instance._uri]
> > > > > > > > > > > > > > > > > > > AttributeError: 'SQLiteAdapter' object
> > > > > > > > > > > > > > > > > > > has no attribute '_uri'
>
> > > > > > > > > > > > > > > > > > > On Dec 4, 10:26 pm, "mr.freeze"
> > > > > > > > > > > > > > > > > > > <nat...@freezable.com> wrote:
>
> > > > > > > > > > > > > > > > > > > > I get this now:
> > > > > > > > > > > > > > > > > > > > Traceback (most recent call last):
> > > > > > > > > > > > > > > > > > > > File "C:\web2py\gluon\restricted.py",
> > > > > > > > > > > > > > > > > > > > line 188, in restricted
> > > > > > > > > > > > > > > > > > > > exec ccode in environment
> > > > > > > > > > > > > > > > > > > > File
> > > > > > > > > > > > > > > > > > > > "C:/web2py/applications/main/models/db.py",
> > > > > > > > > > > > > > > > > > > > line 22, in
> > > > > > > > > > > > > > > > > > > > <module>
> > > > > > > > > > > > > > > > > > > > db =
> > > > > > > > > > > > > > > > > > > > SQLDB(dal_connection,pool_size=10)
> > > > > > > > > > > > > > > > > > > > File "C:\web2py\gluon\sql.py", line
> > > > > > > > > > > > > > > > > > > > 2405, in __init__
> > > > > > > > > > > > > > > > > > > > raise RuntimeError, "Failure to
> > > > > > > > > > > > > > > > > > > > connect, tried 5 times"
>
> ...
>
> read more »
