[web2py] web2py best practices/patterns
Hi, working with web2py is fun and I've learned a lot about web development and Python. So 'yes', web2py is a great teaching tool and the docs are very good too. Small projects are a breeze to realize and MVC separates the concerns nicely. In larger projects however, I still manage to end up in a bit of a mess. What I lack is a set of w2p design principles that go beyond MVC. The type of questions I had are: - What should one never place into a Model, View, Controller, Module and why? - What should one always place into a Model, View, Controller, Module and why? - What known exceptions are there to these rules? - Should a view ideally be assembled entirely from components? - At what granularity should components, controllers and plugins operate? - Should one try to create a (Rest) API first and then consume it oneself? ... Here's how I would answer some of them today: - Create a distinct controller for every business function (blog, shopping cart, ...) - Keep controller actions as small as possible. - Move all utility functions into custom modules that don't depend on w2p (i.e. they don't import gluon). - Allow strictly no logic in views except for loops and branches. - If DAL is too low-level or an alternative storage backend is used create a ORM-type wrapper as a custom module. - Assemble pages from components. ... This is how I would go about my next w2p project today but I am sure that I would have to learn some new lessons the hard way. How do you structure your apps to avoid spaghetti logic and to keep them maintainable? I would also be interested to hear the kind of questions you have? Maybe we can collect our lessons learned and compile a nice document/catalogue. If there's already such a doc, please let me know. Best regards, HC --
[web2py] Re: CouchDB & NoSQLAdapter.represent for datetime
Hi Massimo,
I tested and get a error now. The problem is that value is None.
Function argument list(self=,
obj=None, fieldtype='reference auth_user')
Code listing line 3531 pp
-
value = NoSQLAdapter.represent(self, obj, fieldtype) if
fieldtype=='id': return repr(str(int(value))) elif
fieldtype in ('date','time','datetime','boolean'): return
serializers.json(value) return repr(not
isinstance(value,unicode) and value or value.encode('utf8')) #ERROR
value has not attribute encode
The following tables and records are created in a fresh couchdb after
trying to register a new user.
NAME Number of records
auth_cas 0
auth_event 0
auth_group 1
auth_membership 0
auth_permission 0
auth_user1
I tested this serverside as my local env isn't setup for couchdb. Let
me set it up and then I can also debug but perhaps you have a
suspicion already. Thanks for your help.
On a different note, I realize that the adapter is still experimental
but before we can use it in production we have to address the
temporary views, which are to expensive to run in prod. Not unlike GAE
and index.yaml, CouchDb views have to be defined upfront. This is not
the thread to discuss it at length but I wonder whether one should
allow temporary views at all. Perhaps DAL should error out if it does
not find a permanent view to to answer a query (one could tell the
developer what view needs to be added...).
Best regards,
HC
On Oct 25, 3:51 am, Massimo Di Pierro
wrote:
> I made some changes in trunk that should have fixed the date/datetime
> issue but I am not sure. Can you try it?
>
> On Oct 24, 7:25 pm, "H.C. v. Stockhausen" wrote:
>
>
>
>
>
>
>
> > Hi Massimo,
>
> > thanks for your reply. I don't think CouchDB has a datetime type but
> > the CouchDB-0.8 driver itself maps to and from iso string
> > representations '2007-04-01T15:30:00Z'. See Chapter 4 "Mapping CouchDB
> > documents to Python objects: couchdb.mapping"
> > athttp://packages.python.org/CouchDB/mapping.html
> > Would it make sense for DAL to use couchdb.mapping directly?
>
> > I'd like to help but I don't understand dal well enough yet and in
> > particular the importance of the base class NoSQLAdapter. Does the
> > couchDB adapter really need to derive from it?
>
> > Best regards,
> > HC
>
> > On Oct 23, 5:48 pm, Massimo Di Pierro
> > wrote:
>
> > > This is very helpful. I was expecting this problem. How are datetimes
> > > supposed to be stores in CouchDB?
>
> > > On Oct 22, 3:13 am, "H.C. v. Stockhausen" wrote:
>
> > > > Hi
>
> > > > I am just testing the CouchDB Adapter and it fails when registering a
> > > > new user. I tired both w2py's default auth and janrain. The problem
> > > > appears to be the time_stamp field.
>
> > > > - w2py Version 1.99.2 (2011-09-26 06:55:33) stable
> > > > - driver CouchDB-0.8-py2.6.egg
>
> > > > Traceback
> > > >
> > > > File "/home/hcvst/web2py/gluon/dal.py", line 3510, in insert
> > > > ctable.save(values)
> > > > File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
> > > > couchdb/client.py", line 407, in save
> > > > _, _, data = func(body=doc, **options)
> > > > File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
> > > > couchdb/http.py", line 405, in put_json
> > > > status, headers, data = self.put(*a, **k)
> > > > File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
> > > > couchdb/http.py", line 384, in put
> > > > return self._request('PUT', path, body=body, headers=headers,
> > > > **params)
> > > > File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
> > > > couchdb/http.py", line 419, in _request
> > > > credentials=self.credentials)
> > > > File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
> > > > couchdb/http.py", line 239, in request
> > > > resp = _try_request_with_retries(iter(self.retry_delays))
> > > > File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
> > > > couchdb/http.py", line 196, in _try_request_with_retries
> > > > return _try_request()
> > > > File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.
[web2py] Re: CouchDB & NoSQLAdapter.represent for datetime
Hi Massimo,
thanks for your reply. I don't think CouchDB has a datetime type but
the CouchDB-0.8 driver itself maps to and from iso string
representations '2007-04-01T15:30:00Z'. See Chapter 4 "Mapping CouchDB
documents to Python objects: couchdb.mapping" at
http://packages.python.org/CouchDB/mapping.html
Would it make sense for DAL to use couchdb.mapping directly?
I'd like to help but I don't understand dal well enough yet and in
particular the importance of the base class NoSQLAdapter. Does the
couchDB adapter really need to derive from it?
Best regards,
HC
On Oct 23, 5:48 pm, Massimo Di Pierro
wrote:
> This is very helpful. I was expecting this problem. How are datetimes
> supposed to be stores in CouchDB?
>
> On Oct 22, 3:13 am, "H.C. v. Stockhausen" wrote:
>
>
>
>
>
>
>
> > Hi
>
> > I am just testing the CouchDB Adapter and it fails when registering a
> > new user. I tired both w2py's default auth and janrain. The problem
> > appears to be the time_stamp field.
>
> > - w2py Version 1.99.2 (2011-09-26 06:55:33) stable
> > - driver CouchDB-0.8-py2.6.egg
>
> > Traceback
> >
> > File "/home/hcvst/web2py/gluon/dal.py", line 3510, in insert
> > ctable.save(values)
> > File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
> > couchdb/client.py", line 407, in save
> > _, _, data = func(body=doc, **options)
> > File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
> > couchdb/http.py", line 405, in put_json
> > status, headers, data = self.put(*a, **k)
> > File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
> > couchdb/http.py", line 384, in put
> > return self._request('PUT', path, body=body, headers=headers,
> > **params)
> > File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
> > couchdb/http.py", line 419, in _request
> > credentials=self.credentials)
> > File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
> > couchdb/http.py", line 239, in request
> > resp = _try_request_with_retries(iter(self.retry_delays))
> > File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
> > couchdb/http.py", line 196, in _try_request_with_retries
> > return _try_request()
> > File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
> > couchdb/http.py", line 222, in _try_request
> > chunk = body.read(CHUNK_SIZE)
> > AttributeError: 'dict' object has no attribute 'read'
>
> > DAL CODE (dal.py at line 3510)
> > =
> > def insert(self,table,fields):
> > id = uuid2int(web2py_uuid())
> > ctable = self.connection[table._tablename]
> > values = dict((k.name,NoSQLAdapter.represent(self,v,k.type))
> > for k,v in fields)
> > values['_id'] = str(id)
> > ctable.save(values) # line 3510
>
> > DRIVER CODE (..egg/couchdb/http.py in _try_request at line 222)
> >
> > if body is not None:
> > if isinstance(body, str):
> > conn.sock.sendall(body)
> > else: # assume a file-like object and send in
> > chunks
> > while 1:
> > chunk = body.read(CHUNK_SIZE) # line 222
>
> > BODY VARIABLE
> > ==
> > {'_id': '156782505411822007491552899341462059095', 'client_ip':
> > u'127.0.0.1', 'description': u'Group
> > 128304130898558275345572010972780625739 created', 'origin': u'auth',
> > 'time_stamp': datetime.datetime(2011, 10, 22, 3, 48, 48, 413381),
> > 'user_id': None}
>
> > The problem is probably that the BODY time_stamp attribute of type
> > datetime cannot be JSON serialized but I don't know dal.py well enough
> > to say what NoSQLAdapter.represent should return.
>
> > Best regards,
> > HC
[web2py] CouchDB & NoSQLAdapter.represent for datetime
Hi
I am just testing the CouchDB Adapter and it fails when registering a
new user. I tired both w2py's default auth and janrain. The problem
appears to be the time_stamp field.
- w2py Version 1.99.2 (2011-09-26 06:55:33) stable
- driver CouchDB-0.8-py2.6.egg
Traceback
File "/home/hcvst/web2py/gluon/dal.py", line 3510, in insert
ctable.save(values)
File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
couchdb/client.py", line 407, in save
_, _, data = func(body=doc, **options)
File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
couchdb/http.py", line 405, in put_json
status, headers, data = self.put(*a, **k)
File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
couchdb/http.py", line 384, in put
return self._request('PUT', path, body=body, headers=headers,
**params)
File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
couchdb/http.py", line 419, in _request
credentials=self.credentials)
File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
couchdb/http.py", line 239, in request
resp = _try_request_with_retries(iter(self.retry_delays))
File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
couchdb/http.py", line 196, in _try_request_with_retries
return _try_request()
File "/usr/local/lib/python2.6/dist-packages/CouchDB-0.8-py2.6.egg/
couchdb/http.py", line 222, in _try_request
chunk = body.read(CHUNK_SIZE)
AttributeError: 'dict' object has no attribute 'read'
DAL CODE (dal.py at line 3510)
=
def insert(self,table,fields):
id = uuid2int(web2py_uuid())
ctable = self.connection[table._tablename]
values = dict((k.name,NoSQLAdapter.represent(self,v,k.type))
for k,v in fields)
values['_id'] = str(id)
ctable.save(values) # line 3510
DRIVER CODE (..egg/couchdb/http.py in _try_request at line 222)
if body is not None:
if isinstance(body, str):
conn.sock.sendall(body)
else: # assume a file-like object and send in
chunks
while 1:
chunk = body.read(CHUNK_SIZE) # line 222
BODY VARIABLE
==
{'_id': '156782505411822007491552899341462059095', 'client_ip':
u'127.0.0.1', 'description': u'Group
128304130898558275345572010972780625739 created', 'origin': u'auth',
'time_stamp': datetime.datetime(2011, 10, 22, 3, 48, 48, 413381),
'user_id': None}
The problem is probably that the BODY time_stamp attribute of type
datetime cannot be JSON serialized but I don't know dal.py well enough
to say what NoSQLAdapter.represent should return.
Best regards,
HC
