[web2py] SQLTABLE and db.table.field.count()

[Kurt Grutzmacher]

It seems that change 
14bcad629eb2b73266b11b4c9dbf63499c73d38b
 modified 
the way SQLTABLE handled column names by using this regex for table.field:

REGEX_TABLE_DOT_FIELD = re.compile('^(\w+)\.([^.]+)$')


This has caused a backwards compatibility problem when doing the following:

In [1]: count = db.auth_user.id.count()

In [2]: a = db().select(db.auth_user.first_name, count, 
groupby=db.auth_user.first_name)

In [3]: SQLTABLE(a)
---
AttributeErrorTraceback (most recent call last)
/Users/oldgrutz/Documents/SPA/Kvasir/grutz-kvasir-virtenv/web2py/applications/kvgrutz/models/scheduler.py
 
in ()
> 1 SQLTABLE(a)

/Users/oldgrutz/Documents/SPA/Kvasir/grutz-kvasir-virtenv/web2py/gluon/sqlhtml.py
 
in __init__(self, sqlrows, linkto, upload, orderby, headers, truncate, 
columns, th_link, extracolumns, selectid, renderstyle, cid, colgroup, 
**attributes)
   2912 return
   2913 if not columns:
-> 2914 columns = 
['.'.join(sqlrows.db._adapter.REGEX_TABLE_DOT_FIELD.match(c).groups()) for 
c in sqlrows.colnames]
   2915 if headers == 'fieldname:capitalize':
   2916 headers = {}

AttributeError: 'NoneType' object has no attribute 'groups'

In [4]:


 

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Kvasir: Penetration test-driven data management in web2py

[Kurt Grutzmacher]

Thanks everyone!

A video overview of the features is planned. If anyone is going to ToorCon San 
Diego in October I will be giving a brief presentation on Oct 20.

António Ramos wrote:
> Only a video presentation is missing.
>
> Nice work 
>
>
> 2013/9/24 step mailto:step.l...@gmail.com>>
>
> nice and comprehensive intro in the first linked post!
>
> --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to the Google
> Groups "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to web2py+unsubscr...@googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
> -- 
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to the Google
> Groups "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[web2py] Kvasir: Penetration test-driven data management in web2py

[Kurt Grutzmacher]

Hey everyone,

A tool I created to help manage Penetration Test-related data was pushed
this morning to Github. It's written in web2py which was (and still is)
the best tool for the job when we started this back in 2010. I'm very
proud to finally be releasing Kvasir publicly under the 3-clause BSD
license.

Since I'm not a professional programmer (I'm a professional hacker!) I
know there are plenty of areas for code improvement. Hackers generally
use twig and bubble gum to piece together tools that work for them, and
Kvasir is certainly guilty of that. Any feedback is certainly welcomed
and fully appreciated!

http://blogs.cisco.com/security/introducing-kvasir/
https://github.com/KvasirSecurity/Kvasir

-- 
- grutz;

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: $.web2py.ajax_page and response.flash / response.js

[Kurt Grutzmacher]

Yes, that's what's happening: X-Requested-Header is not being sent when
I manually use $.web2py.ajax_page(...) in my HTML.


Niphlod wrote:
> I'm missing a point . the only "reliable" way to detect a request
> cming from ajax is indeed the presence of the x-requested-with header
> . are you saying that you can replicate consistently a behaviour
> where jquery doesn't add the x-requested-with header ?
>
> Il giorno giovedě 19 settembre 2013 08:05:46 UTC+2, Kurt Grutzmacher
> ha scritto:
>
> Changeset a51d0877979c0846b420bee31102d162030f79ca in
> gluon/main.py now
> checks for response.ajax being true before adding response headers
> for
> web2py-component-flash and web2py-component-command. Unfortunately
> calls
> manually utilizing web2py_ajax_page( ... ) are not being seen as
> "ajax"
> because jQuery does not send the X-Requested-With header.
>
> For instance in the I have this this:
>
> var data = "article=" + checks;
> url = "{{=URL('hosts', 'delete.json', user_signature=True)}}"
> $.web2py.ajax_page( 'POST', url, data );
>
> If I look at the request headers being send, X-Requested-With is not
> set. Ajax calls that are set up by web2py functions such as LOAD(..,
> ajax=True) seem to be ok.
>
> This is acting like jQuery believes I'm sending a cross-domain ajax,
> which would should not send the header.
>
> A work-around can be done by setting both the
> response.flash/response.js
> entries as well as the corresponding header entries but that seems
> counterintuitive since main.py/wsgibase <http://main.py/wsgibase>
> should be setting these accordingly.
>
> -- 
> - grutz;
>
> -- 
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to the Google
> Groups "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

-- 
- grutz;

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[web2py] $.web2py.ajax_page and response.flash / response.js

[Kurt Grutzmacher]

Changeset a51d0877979c0846b420bee31102d162030f79ca in gluon/main.py now
checks for response.ajax being true before adding response headers for
web2py-component-flash and web2py-component-command. Unfortunately calls
manually utilizing web2py_ajax_page( ... ) are not being seen as "ajax"
because jQuery does not send the X-Requested-With header.

For instance in the I have this this:

var data = "article=" + checks;
url = "{{=URL('hosts', 'delete.json', user_signature=True)}}"
$.web2py.ajax_page( 'POST', url, data );

If I look at the request headers being send, X-Requested-With is not
set. Ajax calls that are set up by web2py functions such as LOAD(..,
ajax=True) seem to be ok.

This is acting like jQuery believes I'm sending a cross-domain ajax,
which would should not send the header.

A work-around can be done by setting both the response.flash/response.js
entries as well as the corresponding header entries but that seems
counterintuitive since main.py/wsgibase should be setting these accordingly.

-- 
- grutz;

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: DB insert confusion

[Kurt Grutzmacher]

Yeah, yeah, yeah.. My on-the-fly testing code wasn't correct. ;)

Checking with isinstance() is good but fails if params is just a string
due to the data.get('params', '') a few lines up.

Changing that to data.get('params', []) will resolve this or just drop
elif isinstance(params, list): and change it to an else: catch

||
ifisinstance(params,dict):
s =methods[method](**params)
else:
s =methods[method](*params)

This way the called method will take defaults or error out if variables
are not defined. Either way seems to work fine on my end.


Niphlod wrote:
> ok, let's try to not make any further confusion. Let's agree on what
> web2py needs to provide and how to call it.
> First things first: gluon.contrib.simplejsonrpc supports only 1.1, so
> *only* positional arguments sent as lists.
> jsonrpclib instead supports 2.0 spec, so *both* positional arguments
> sent as lists and keyword arguments sent as dicts.
>
> Second thing: it's hard to follow some example if the example is never
> going to work :P
>
> ||
> data =db(db.t_test.ALL).select()
> returndict(data=data)
>
> will go on exception no matter what.
> ||
> data =db().select(db.t_test.ALL)
> returndict(data=data)
>
> works as intended.
>
> Third: decorated functions usually need to have arguments.
> can we set on a
> ||
> @service.jsonrpc
> deftestlist(arg1,arg2):
> data =db().select(db.t_test.ALL)
> returndict(data=data,arg=arg1,arg2=arg2)
> ? Right!
>
> Now, I think with a simple patch we can escape the "pass parameters as
> a mapping or as a list" problem in gluon/tools.py
> ||
> ifisinstance(params,dict):
> s =methods[method](**params)
> elifisinstance(params,list):
> s =methods[method](*params)
>
> That should restore functionality for who calls functions only with
> "positional" style, while retaining the "keywords" style feature. This
> means that using contrib.simplejsonrpc you'll be able to call
>
> ||
> s.testlist(1,2)
>
> but not
> ||
>
> s.testlist(arg1=1,arg2=2)
>
>
> NB: This is just because simplejsonrpc adheres only to the 1.1 spec
>
> With jsonrpclib, however, both will work ok (yeah!)
>
> If this is fine, we can further fine-tune responses (e.g. 'version':
> '2.0' instead of 'version': '1.1') or having 2 separate @service
> decorators as Jonathan suggested.
>
> Do we have a deal ?
>
> -- 
>  
>  
>  

-- 
- grutz;

-- 

Re: [web2py] Re: DB insert confusion

[Kurt Grutzmacher]

Any movement on this? I can't seem to get jsonrpclib to force parameterized 
queries. Replicating this is fairly simple:

1. Create a new app, in default.py add a function. Replace t_test with auth or 
whatever:

@service.jsonrpc
def list():
data = db(db.t_test.ALL).select()
return dict(data=data)

2. Drop to a python shell:

>>> import jsonrpclib
>>> j = 
>>> jsonrpclib.Server(uri='http://localhost:8000/appname/default/call/jsonrpc')
>>> j.list()
Traceback (most recent call last):
  File "", line 1, in 
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/jsonrpclib/jsonrpc.py",
 line 276, in __call__
return self.__send(self.__name, kwargs)
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/jsonrpclib/jsonrpc.py",
 line 225, in _request
check_for_errors(response)
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/jsonrpclib/jsonrpc.py",
 line 529, in check_for_errors
raise ProtocolError((code, message))
ProtocolError: (100, u'TypeError: list() argument after ** must be a mapping, 
not str')

Same problem with simplejsonrpc in gluon/contrib...


Niphlod wrote:
> uhm. The "problem" is that even though every jsonrpc interface I
> worked with was parameter-based, for jsonrpc2 position-based and
> parameter-based are both valid.
>
> On Tuesday, December 11, 2012 10:42:06 PM UTC+1, Jonathan Lundell wrote:
>
> On 11 Dec 2012, at 1:09 PM, Niphlod  > wrote:
>> I took some time to watch at the jsonrpc specs. Right now I had
>> experience with jsonrpc only in the "named parameters" format. I
>> thought it was the standard, my bad.
>> However, I found out that 2.0 introduced "named parameters" that
>> were not supported - explicitely - in 1.0
>>
>> --> {"jsonrpc": "2.0", "method": "subtract", "params": [42, 23], "id": 1}
>> <-- {"jsonrpc": "2.0", "result": 19, "id": 1}
>>
>> --> {"jsonrpc": "2.0", "method": "subtract", "params": {"subtrahend": 
>> 23, "minuend": 42}, "id": 3}
>> <-- {"jsonrpc": "2.0", "result": 19, "id": 3}
>>
>> are both valid.
>>
>> Maybe revert this and make a jsonrpc2 decorator to support named
>> parameters explicitely would be a better solution?
>
> How about both jsonrpc1 and jsonrpc2, and then jsonrpc = jsonrpc2?
> (I think it'd be better to make v2 the default.)
>
>>
>> On Tuesday, December 11, 2012 9:57:50 PM UTC+1, Kurt Grutzmacher
>> wrote:
>>
>> I don't think this is a good JSON-RPC example as the change
>> broke our app that uses simplejsonrpc or jsonrpclib to make
>> API calls.
>>
>> Based on the jsonrpclib python module
>> @ https://code.google.com/p/jsonrpclib/
>> <https://code.google.com/p/jsonrpclib/> requests look like:
>>
>> >>> import jsonrpclib
>> >>> server = jsonrpclib.Server('http://localhost:8080 
>> <http://localhost:8080/>')
>> >>> server.add(5,6)
>> 11
>> >>> print jsonrpclib.history.request
>> {"jsonrpc": "2.0", "params": [5, 6], "id": "gb3c9g37", "method": 
>> "add"}
>> >>> print jsonrpclib.history.response
>> {'jsonrpc': '2.0', 'result': 11, 'id': 'gb3c9g37'}
>>
>>
>> And the JSON-RPC spec states params should be "An Array of
>> objects to pass as arguments to the method."
>> -- http://json-rpc.org/wiki/specification
>> <http://json-rpc.org/wiki/specification>
>>
>> However the actual spec doesn't specify array, dict or
>> whatever as it tries to be universal: "A Structured value
>> that holds the parameter values to be used during the
>> invocation of the method. This member MAY be omitted."
>>  http://www.jsonrpc.org/specification#request_object
>> <http://www.jsonrpc.org/specification#request_object>
>>
>> For simplejsonrpc and jsonrpclib to work we have to undo this
>> change in gluon/tools.py
>>
>>
>&

[web2py] Re: Python keywords in db Field names

[Kurt Grutzmacher]

True, I was just hoping there was a way DAL could use an "alias" for the 
field name so we could do db.table.password but the Field() would actually 
reference the "pass" field in the table. Seems like an odd corner case that 
I'll have to work around with using executesql instead of full DAL 
abstraction for now.


On Saturday, December 15, 2012 7:30:36 AM UTC-8, Massimo Di Pierro wrote:
>
> pass is a python keyword. You would would not be able to do db.table.pass:
>
> >>> class A(object): pass
> ... 
> >>> a=A()
> >>> a.pass
>   File "", line 1
> a.pass
>  ^
> SyntaxError: invalid syntax
>
>
>
> On Saturday, 15 December 2012 08:51:28 UTC-6, Kurt Grutzmacher wrote:
>>
>> Ok.
>>
>> I want to utilize DAL to read data from a database, build forms, grid, 
>> tables, etc. One of the tables looks like this:
>>
>> db.define_table('credentials',
>>   Field('id', type='id'),
>>   Field('user', type='string'),
>>   Field('pass', type='string'),
>> )
>>
>> The 'pass' field does not pass the REGEX_PYTHON_KEYWORDS match on line 
>> 8548 of dal.py.
>>
>>
>> On Saturday, December 15, 2012 6:09:21 AM UTC-8, Niphlod wrote:
>>>
>>> tell us more about what you're trying to do, and the model, also if 
>>> throws errors, that are you willing to use
>>>
>>> On Saturday, December 15, 2012 3:23:23 AM UTC+1, Kurt Grutzmacher wrote:
>>>>
>>>> I need to access a static third-party data source that uses some Python 
>>>> keywords in the Field name. This obviously is not good and dal.py syntax 
>>>> errors when loading the app.
>>>>
>>>> Any way I can work around this?
>>>>
>>>>
>>>>
>>>>

-- 

[web2py] Re: Python keywords in db Field names

[Kurt Grutzmacher]

Ok.

I want to utilize DAL to read data from a database, build forms, grid, 
tables, etc. One of the tables looks like this:

db.define_table('credentials',
  Field('id', type='id'),
  Field('user', type='string'),
  Field('pass', type='string'),
)

The 'pass' field does not pass the REGEX_PYTHON_KEYWORDS match on line 8548 
of dal.py.


On Saturday, December 15, 2012 6:09:21 AM UTC-8, Niphlod wrote:
>
> tell us more about what you're trying to do, and the model, also if throws 
> errors, that are you willing to use
>
> On Saturday, December 15, 2012 3:23:23 AM UTC+1, Kurt Grutzmacher wrote:
>>
>> I need to access a static third-party data source that uses some Python 
>> keywords in the Field name. This obviously is not good and dal.py syntax 
>> errors when loading the app.
>>
>> Any way I can work around this?
>>
>>
>>
>>

-- 

[web2py] Python keywords in db Field names

[Kurt Grutzmacher]

I need to access a static third-party data source that uses some Python 
keywords in the Field name. This obviously is not good and dal.py syntax 
errors when loading the app.

Any way I can work around this?



-- 

Re: [web2py] Re: DB insert confusion

[Kurt Grutzmacher]

I don't think this is a good JSON-RPC example as the change broke our app 
that uses simplejsonrpc or jsonrpclib to make API calls.

Based on the jsonrpclib python module 
@ https://code.google.com/p/jsonrpclib/ requests look like:

>>> import jsonrpclib
>>> server = jsonrpclib.Server('http://localhost:8080')
>>> server.add(5,6)
11
>>> print jsonrpclib.history.request
{"jsonrpc": "2.0", "params": [5, 6], "id": "gb3c9g37", "method": "add"}
>>> print jsonrpclib.history.response
{'jsonrpc': '2.0', 'result': 11, 'id': 'gb3c9g37'}


And the JSON-RPC spec states params should be "An Array of objects to pass 
as arguments to the method." -- http://json-rpc.org/wiki/specification

However the actual spec doesn't specify array, dict or whatever as it tries 
to be universal: "A Structured value that holds the parameter values to be 
used during the invocation of the method. This member MAY be omitted."  
http://www.jsonrpc.org/specification#request_object

For simplejsonrpc and jsonrpclib to work we have to undo this change in 
gluon/tools.py


On Wednesday, November 14, 2012 1:25:22 PM UTC-8, Mike Anson wrote:
>
> Thanks very much for your help Niphlod.
>
> On Wednesday, 14 November 2012 16:10:35 UTC-5, Niphlod wrote:
>>
>>
>> Yes I understand your point. The reason it is currently like this is 
>>> because if I use your suggestion (which I obviously had originally)
>>>
>>> {"id": 1, "method": "savemessage", "params": { "*message*": 
>>> "variableholdingmessage", "*uid*" : "variableholdingmail"}}
>>>
>>> I get message and uid as the values in the DB. So I switched them.
>>>
>>> {"id": 1, "method": "savemessage", "params": { "variableholdingmessage": 
>>> "mymessage", "variableholdinguid" : "myemail@localhost"}}
>>>
>>> The result means that variableholdingmessage is saved as the message and 
>>> not the expected "mymessage". Exactly the same for uid.
>>>
>>> Unfortunately jsonrpc call method has a bug. in web2py 2.2.1, in 
>> gluon/tools.py, line 4231 should be 
>>
>> s = methods[method](**params)
>>
>> instead of
>>
>> s = methods[method](*params)
>>
>>
>> sending a patch to Massimo right now!
>>  
>>
>>>
>>> re: PS -- haha yes I know. I did try it with single quotes and it 
>>> crapped out?? So just kept the doubles. It's not that bad!
>>>
>>> re: PS2 -- have you any recommendations to solve this special character 
>>> potential problem?
>>>
>>>  
>> Normally with jsonrpc you use something that is not curl, e.g. a 
>> programming language that supports json (python?!)
>> Escaping on bash without awk, sed, etc is always problematic but if 
>> you're willing to have as only limitation the " character that is less 
>> frequent to use within a message, why don't you use one of the methods not 
>> requiring a json body to be posted ? e.g. @service.xml, @service.csv or 
>> @service.json
>>
>> curl -v --get --data-urlencode \"uid=$uid\" --data-urlencode 
>> \"message=$message\" $url
>>
>> here curl takes care of urlencoding the message and the uid parameters.
>>
>>
>>  
>>
>

-- 

[web2py] Re: On storing datetime in the database, and textual representation of fields

[Kurt Grutzmacher]

Hi Massimo,

I updated to the the latest trunk and my imports no longer have the problem 
in Issue #403. I swear nick and I didn't collude to report similar issues! 
:)

On another system running web2py 1.98.2 (2011-08-31 20:40:46) and postgresql 
8.4 I do not receive the same error. With the same web2py code and 
postgreslql 9.0.1 I do. Your assumption about 9.0.1 ignoring 
standard_conforming_strings 
is most likely correct (and answers why some of my colleagues haven't seen 
the same error!)

grutz

[web2py] VirtualFields are really slow with large datasets

[Kurt Grutzmacher]

I have three tables that aren't particularly large (1500, 2500, and
3500 records in each). With VirtualFields enabled a select() query
take a very long time to process.

class VirtualFields(object):
def name(self):
return self.table1.table2.name
def email(self):
return self.table1.table3.email
def phone(self):
return self.table1.table3.phone
db.table1.virtualfields.append(VirtualFields())

Query: db(db.table1.id <= 1500).select()

Virtual fields enabled: 69185322 function calls (69042017 primitive
calls) in 99.450 seconds
No virtual fields: 357819 function calls (357815 primitive calls) in
0.376 seconds

cProfile output with virtual fields enabled:

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall
filename:lineno(function)
10.0010.001   99.450   99.450 :1()
   2583/10.0220.000   99.449   99.449 dal.py:1004(select)
 25830.0070.0001.1760.000 dal.py:1008(response)
7749/25830.0250.0000.0490.000 dal.py:1045(tables)
 25830.0080.0001.1350.000 dal.py:1086(log_execute)
 25830.0040.0001.1390.000 dal.py:1090(execute)
 25830.0100.0000.0170.000 dal.py:1093(represent)
 25830.0010.0000.0010.000 dal.py:
1160(represent_exceptions)
 25830.0010.0000.0010.000 dal.py:1169(rowslice)
   2583/10.7010.000   99.436   99.436 dal.py:1173(parse)
  8397955   12.3720.000   13.6360.000 dal.py:3225(__getitem__)
  50891225.7000.0005.7000.000 dal.py:3234(__setitem__)
  7244.1730.000   14.8860.000 dal.py:3237(__getattr__)
 77460.0050.0000.0130.000 dal.py:3240(__setattr__)
   1458220.1820.0000.2520.000 dal.py:3510(__getitem__)
 25830.0060.0000.0070.000 dal.py:3513(__setitem__)
489960.0330.0000.1110.000 dal.py:3516(__getattr__)
 25830.0040.0000.0100.000 dal.py:3519(__setattr__)
 25830.0040.0000.0080.000 dal.py:3528(__call__)
  5432.2800.0005.6620.000 dal.py:3631(__allocate)
  5435.5430.000   13.6790.000 dal.py:3637(__getattr__)
258200.0290.0000.0400.000 dal.py:3643(__setattr__)
144605/193780.2880.0003.4220.000 dal.py:
3827(__getitem__)
206580.0160.0000.0770.000 dal.py:3895(__getattr__)
503610.0360.0000.1630.000 dal.py:3903(__iter__)
142010.0440.0000.0930.000 dal.py:4113(__eq__)
 25830.0080.0000.0170.000 dal.py:4116(__ne__)
10.0000.0000.0000.000 dal.py:4122(__le__)
503610.0370.0000.0370.000 dal.py:4488(__str__)
167850.0180.0000.0180.000 dal.py:4509(__init__)
142020.0120.0000.0120.000 dal.py:4565(__init__)
   2583/10.0090.000   99.449   99.449 dal.py:4598(select)
 25830.0030.0000.0030.000 dal.py:4663(__init__)
 1291   19.5830.015   99.0710.077 dal.py:
4677(setvirtualfields)
 25820.0070.0000.0140.000 dal.py:4718(__getitem__)
 25820.0040.0000.0180.000 dal.py:4742(first)
 25820.0040.0000.0320.000 dal.py:761(EQ)
10.0000.0000.0000.000 dal.py:774(LE)
55527/503610.0540.0000.1230.000 dal.py:807(expand)
 25830.1160.0000.5350.000 dal.py:911(_select)
 25830.0150.0000.0150.000 dal.py:
998(select_limitby)
  1816.1400.000   16.1330.000 db_02_autospang.py:
181(f_nexid)
  1815.0440.000   11.3450.000 db_02_autospang.py:
185(f_proto)
  1815.3420.000   13.8680.000 db_02_autospang.py:
189(f_number)
296270.0180.0000.0730.000 utf_8.py:15(decode)
296270.0550.0000.0550.000 {_codecs.utf_8_decode}
619680.0800.0000.0800.000 {built-in method match}
  181   18.9710.000   18.9710.000 {dir}
  7241.3840.0001.3840.000 {getattr}
  66693063.8940.0007.6510.000 {hasattr}
   5473590.1300.0000.1300.000 {isinstance}
 51650.0010.0000.0010.000 {len}
 25830.0000.0000.0000.000 {method 'add' of 'set'
objects}
555240.0100.0000.0100.000 {method 'append' of
'list' objects}
296270.0350.0000.1090.000 {method 'decode' of
'str' objects}
10.0000.0000.0000.000 {method 'disable' of
'_lsprof.Profiler' objects}
296270.0420.0000.0420.000 {method 'encode' of
'unicode' objects}
 25831.1160.0001.1160.000 {method 'execute' of
'psycopg2._psycopg.cursor' objects}
 25830.0290.0000.0290.000 {method 'fetchall' of
'psycopg2._psycopg.cursor' objects}
 134186623.7400.0003.7400.000 

[web2py] Re: Create databases folder

[Kurt Grutzmacher]

If migrate is True on your Fields then web2py will want to have the
databases directory existing first as it tries to open databases/
sql.log first. If this directory doesn't exist then an exception is
thrown.

web2py$ ls -FC applications/newapp/
ABOUT   __init__.py controllers/
languages/  modules/views/
LICENSE cache/  cron/   models/ 
static/ wizard.metadata
web2py$ python web2py.py -S newapp -M
web2py Enterprise Web Framework
Created by Massimo Di Pierro, Copyright 2007-2011
Version 1.91.6 (2011-02-01 17:13:19)
Database drivers available: SQLite3, pymysql, PostgreSQL
Traceback (most recent call last):
  File "/Users/grutz/src/web2py/gluon/restricted.py", line 188, in
restricted
exec ccode in environment
  File "applications/newapp/models/db.py", line 80, in 
 
auth.define_tables(migrate=settings.migrate)
# creates all needed tables
  File "/Users/grutz/src/web2py/gluon/tools.py", line 1198, in
define_tables
format = '%(role)s (%(id)s)')
  File "/Users/grutz/src/web2py/gluon/dal.py", line 3490, in
define_table
polymodel=polymodel)
  File "/Users/grutz/src/web2py/gluon/dal.py", line 522, in
create_table
logfile = self.file_open(table._loggername, 'a')
  File "/Users/grutz/src/web2py/gluon/dal.py", line 359, in file_open
fileobj = open(filename,mode)
IOError: [Errno 2] No such file or directory: 'applications/newapp/
databases/sql.log'

web2py$ mkdir applications/newapp/databases
web2py$ python web2py.py -S newapp -M
web2py Enterprise Web Framework
Created by Massimo Di Pierro, Copyright 2007-2011
Version 1.91.6 (2011-02-01 17:13:19)
Database drivers available: SQLite3, pymysql, PostgreSQL
Python 2.6.4 (r264:75821M, Oct 27 2009, 19:48:32)
Type "copyright", "credits" or "license" for more information.

IPython 0.10.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help  -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints
more.

In [1]:

[web2py] Exception during file autodelete

[Kurt Grutzmacher]

Using latest web2py trunk.

Upload Field configured for autodelete=True and executing "del
db.t_file_uploads[form.vars.id]" after processing the file:

File "/Users/grutz/src/web2py/gluon/dal.py", line 3696, in __delitem__
elif not str(key).isdigit() or not self._db(self.id ==
key).delete():
  File "/Users/grutz/src/web2py/gluon/dal.py", line 4381, in delete
self.delete_uploaded_files()
  File "/Users/grutz/src/web2py/gluon/dal.py", line 4412, in
delete_uploaded_files
uploadfolder = os.path.join(self.db._folder, '..', 'uploads')
  File "/Users/grutz/src/web2py/gluon/dal.py", line 3329, in
__getattr__
return dict.__getitem__(self,key)
KeyError: '_folder'

FIX: Line 4412 in dal.py should be:

uploadfolder = os.path.join(self.db._adapter.folder, '..', 'uploads')

[web2py] CRUD json/xml/etc not using format

[Kurt Grutzmacher]

I'm wondering if this is expected behavior or not with CRUD. Given two
tables:

db.define_table('hash_types',
Field('htype', 'string', requires=IS_NOT_EMPTY()),
format='%(htype)s',
)

db.define_table('hashes',
Field('enchash', required=True, unique=True, label="Encrypted
hash"),
Field('htype', 'reference hash_types', label="Hash type", ),
)

And a CRUD read controller:

def read():
return dict(hashes=crud.read(db.hashes, request.args(0)))

If I perform a regular request (http://server:8000/app/default/read/1)
then the db.hashes.htype returns the string.

If I request read.json version (http://server:8000/app/default/
read.json/1) then htype return the id number, not the string/format:

{"hashes": {"enchash": "string", "htype": 1, "id": 1}}

Is that to be expected?