[web2py] Re: Lazy table bug in define table?
Sure, you could use an integer instead of a "reference" and there would be no circular reference (since the DAL doesn't "know" the integer refers to another table). But you would also lose the power of having a foreign key defined on the table. It may be the best compromise in this case. Massimo and I exchanged some messages on the topic and we decided to just let people know "don't use circular references and lazy_tables" for now. -- Joe On Thursday, December 6, 2012 4:56:25 PM UTC-8, Bill Thayer wrote: > > Would using an integer field or a list:reference help out? --
[web2py] Re: Lazy table bug in define table?
Would using an integer field or a list:reference help out? --
[web2py] Re: Lazy table bug in define table?
Oh yes, I am missing entire freight trains of stuff...
Here is what I understand now:
def __getattr__(self, key):
if ogetattr(self,'_lazy_tables') and \
key in ogetattr(self,'_LAZY_TABLES'):
tablename, fields, args = self._LAZY_TABLES.pop(key)
return self.lazy_define_table(tablename,*fields,**args)
return ogetattr(self, key)
- The "hasattr" bug is not a bug at all. It is supposed to call
_getattr_ and check for an exception, which it does. Case closed.
- I'm seeing the error as a result of the _getattr_ function calling
lazy_define_table(). It pops the table from the list of _LAZY_TABLES
and "processes" it. But in the course of processing...
- The table definition calls init() on Table class and builds a table.
In the course of building the table, the code calls sqlhtml_validators on
all fields, including those which reference other tables. The validator
code calls for any table that was referenced.
- The code above is re-entered, this time for the new table. This new
table has a reference for the previous table (yes, circular references)
Now we are in trouble.
- The code falls through the _getattr_ once more, this time looking for
the original table name. It has been removed from _LAZY_TABLES because
it's been here before. But it is now stuck in the middle of
lazy_define_table() and has not gotten to the step where the table is
added as an attribute to db. This time thru we throw an exception.
I'm not sure how to fix this, other than to avoid circular references. But
the circular references may be vey long and this problem could still
surface. I hope someone a lot smarter can figure out how to break this.
Joe
On Tuesday, November 27, 2012 4:42:29 AM UTC-8, Joe Barnhart wrote:
>
> I'm running into a weird "lazy tables" bug. I have about 32 tables
> defined and I'm running into a error when I try to open a controller on a
> table that references another. The referenced table doesn't show up in the
> DAL and web2py throws an error.
>
> I can see the table during "define_table" and I see it added to the
> _LAZY_TABLES list. But later, when the table is referenced, it does NOT
> show up in the list when I am performing _getattr_ on the name of the table
> in the DAL instance. It's like it magically disappears between the
> "define_table" and the time the controller is executed.
>
> It seems like the problem is here, in the dal.py file:
>
>
> def define_table(
> self,
> tablename,
> *fields,
> **args
> ):
> if not isinstance(tablename,str):
> raise SyntaxError, "missing table name"
> elif hasattr(self,tablename) or tablename in self.tables:
> if not args.get('redefine',False):
> raise SyntaxError, 'table already defined: %s' % tablename
> ...etc
>
>
> The call to "hasattr" is processed as a call to the routine "__getattr__"
> shown below:
>
>
> def __getattr__(self, key):
> if ogetattr(self,'_lazy_tables') and \
> key in ogetattr(self,'_LAZY_TABLES'):
> tablename, fields, args = self._LAZY_TABLES.pop(key)
> return self.lazy_define_table(tablename,*fields,**args)
> return ogetattr(self, key)
>
>
> But __getattr__ throws an exception if the key is not present, yet the
> define_table code which calls it does not look for or process the
> exception. Something is amiss in this code but I'm not in tune enough to
> figure out the original intent of "hasattr" in this context.
>
> Am I missing something? (probably)
>
> -- Joe B.
>
>
>
--
[web2py] Re: Lazy table bug in define table?
One artifact of my fix -- the "database administration" page only shows the auth tables now! Now THAT'S a bunch of lazy tables! --
[web2py] Re: Lazy table bug in define table?
In answer to my own question -- YES, I was missing a few things. 1. The implementation of "hasattr" itself calls the _getattr_ and looks for the exception. It is working as expected. There is no "hasattr" problem. 2. The table that is failing is called "club". When it gets its turn to be defined, it is happily added to the _LAZY_TABLES list. I put a watch on that list to see when it goes away. 3. The lazy version of "club" is removed later when another table definition looks for "reference club". In that lookup, the _getattr_ routine falls through to the point where lazy_define_table is called. 4. Looking at the code, this call is happening in a "chain" of lazy_define_table calls -- all are happening because a lazy table is referenced by another table. The reference when it is looked up instantiates the table. 5. But! The _getattr_ call to lazy_define_table does *not add the newly-instantiated table to the list of actual tables in the DAL*. So at this point, the lazy table has been removed from the _LAZY_TABLES list and instantiated, but the resulting structure was thrown away instead of being saved in the DAL. def __getattr__(self, key): if ogetattr(self,'_lazy_tables') and \ key in ogetattr(self,'_LAZY_TABLES'): tablename, fields, args = self._LAZY_TABLES.pop(key) return self.lazy_define_table(tablename,*fields,**args) return ogetattr(self, key) As a possible remedy, I took the following two lines from "define_table" and put them at the end of "lazy_define_table": if not tablename in self.tables: self.tables.append(tablename) This ensures that the table structure is not thrown away when the table name is probed as an attribute of the DAL, causing the table to be instantiated. It fixes my code and makes the lazy tables work properly in my context. My fix might have a negative effect on the time savings, however. It may be better to re-work the logic so that a reference to a table does not need to cause it to be built. --
Re: [web2py] Re: Lazy table bug in define table?
Indeed, it does not happen if you run web2py on the shell. For example,
from sources with:
./web2py.py -S test_app -B
and then input the two simple commands above. But it happens with the web
one, even on PythonAnywhere
2012/11/27 Niphlod
> weird, I can't reproduce on the shell (not the web one, I never use that)
> with trunk or stable.
>
>
> On Tuesday, November 27, 2012 9:57:52 PM UTC+1, Nico Zanferrari wrote:
>>
>>
>>
>> Il giorno martedì 27 novembre 2012 13:42:29 UTC+1, Joe Barnhart ha
>> scritto:
>>>
>>> I'm running into a weird "lazy tables" bug.
>>>
>>
>>
>> I think the bug is real. A simple way to reproduce it is with a new
>> simple application; if you enter its the interactive web shell:
>>
>> db=DAL('sqlite://storage.db')
>> print db._uri
>>
>>
>> you get a similar error ("AttributeError: 'DAL' object has no attribute
>> '_lazy_tables').
>> I've just opened the ticket
>> 1192about this. It
>> does not happen on version 2.0.9.
>>
>> Nico
>>
>> --
>
>
>
>
--
[web2py] Re: Lazy table bug in define table?
you really should post what you're trying to do with the table.
lazy_tables=True "activates" the lazy definition of the table, meaning that
it gets really defined only if it's accessed as db.tablename, db[tablename]
or db.__getattr__(tablename).
The only thing you can't do "in the same way" with lazy_tables=True is
tablename = db.define_table('whatever', Field('whatever'))
With lazy_tables=False, tablename is a reference to db.whatever
(db['whatever'], db.__getattr__('whatever')).
with lazy_tables=True, the variable tablename is None, because define_table
with lazy_tables doesn't return anything (as it's supposed to).
On Tuesday, November 27, 2012 1:42:29 PM UTC+1, Joe Barnhart wrote:
>
> I'm running into a weird "lazy tables" bug. I have about 32 tables
> defined and I'm running into a error when I try to open a controller on a
> table that references another. The referenced table doesn't show up in the
> DAL and web2py throws an error.
>
> I can see the table during "define_table" and I see it added to the
> _LAZY_TABLES list. But later, when the table is referenced, it does NOT
> show up in the list when I am performing _getattr_ on the name of the table
> in the DAL instance. It's like it magically disappears between the
> "define_table" and the time the controller is executed.
>
> It seems like the problem is here, in the dal.py file:
>
>
> def define_table(
> self,
> tablename,
> *fields,
> **args
> ):
> if not isinstance(tablename,str):
> raise SyntaxError, "missing table name"
> elif hasattr(self,tablename) or tablename in self.tables:
> if not args.get('redefine',False):
> raise SyntaxError, 'table already defined: %s' % tablename
> ...etc
>
>
> The call to "hasattr" is processed as a call to the routine "__getattr__"
> shown below:
>
>
> def __getattr__(self, key):
> if ogetattr(self,'_lazy_tables') and \
> key in ogetattr(self,'_LAZY_TABLES'):
> tablename, fields, args = self._LAZY_TABLES.pop(key)
> return self.lazy_define_table(tablename,*fields,**args)
> return ogetattr(self, key)
>
>
> But __getattr__ throws an exception if the key is not present, yet the
> define_table code which calls it does not look for or process the
> exception. Something is amiss in this code but I'm not in tune enough to
> figure out the original intent of "hasattr" in this context.
>
> Am I missing something? (probably)
>
> -- Joe B.
>
>
>
--
[web2py] Re: Lazy table bug in define table?
weird, I can't reproduce on the shell (not the web one, I never use that)
with trunk or stable.
On Tuesday, November 27, 2012 9:57:52 PM UTC+1, Nico Zanferrari wrote:
>
>
>
> Il giorno martedì 27 novembre 2012 13:42:29 UTC+1, Joe Barnhart ha scritto:
>>
>> I'm running into a weird "lazy tables" bug.
>>
>
>
> I think the bug is real. A simple way to reproduce it is with a new simple
> application; if you enter its the interactive web shell:
>
> db=DAL('sqlite://storage.db')
> print db._uri
>
>
> you get a similar error ("AttributeError: 'DAL' object has no attribute
> '_lazy_tables').
> I've just opened the ticket
> 1192about this. It
> does not happen on version 2.0.9.
>
> Nico
>
>
--
[web2py] Re: Lazy table bug in define table?
Il giorno martedì 27 novembre 2012 13:42:29 UTC+1, Joe Barnhart ha scritto:
>
> I'm running into a weird "lazy tables" bug.
>
I think the bug is real. A simple way to reproduce it is with a new simple
application; if you enter its the interactive web shell:
db=DAL('sqlite://storage.db')
print db._uri
you get a similar error ("AttributeError: 'DAL' object has no attribute
'_lazy_tables').
I've just opened the ticket
1192about this. It does
not happen on version 2.0.9.
Nico
--
