[web2py] testing routes.py patterns
I've been advocating the use of doctests in routes.py to debug problems. Here's
another doctest you can use (I'll send a patch to Massimo that adds an example).
compile_re('.*http://otherdomain.com.* (?Pany.*)',
'/app/ctr\gany')[0].pattern
'^.*http://otherdomain.com.* (?Pany.*)$'
compile_re('.*http://otherdomain.com.* (?Pany.*)',
'/app/ctr\gany')[1]
'/app/ctrgany'
compile_re('/$c/$f', '/init/$c/$f')[0].pattern
'^.*?:https?://[^:/]+:[a-z]+ /(?Pc[w_]+)/(?Pf[w_]+)$'
compile_re('/$c/$f', '/init/$c/$f')[1]
'/init/gc/gf'
For the trunk version, the import line needs to change to this:
from gluon.rewrite import select, load, filter_url, filter_out, filter_err,
compile_re
Notice that the backslash escaping gets a little confusing. I can't figure out
how to use raw strings properly here; so it goes.
Re: [web2py] testing routes.py patterns
On Aug 8, 2010, at 8:31 AM, Jonathan Lundell wrote:
I've been advocating the use of doctests in routes.py to debug problems.
Here's another doctest you can use (I'll send a patch to Massimo that adds an
example).
compile_re('.*http://otherdomain.com.* (?Pany.*)',
'/app/ctr\gany')[0].pattern
'^.*http://otherdomain.com.* (?Pany.*)$'
compile_re('.*http://otherdomain.com.* (?Pany.*)', '/app/ctr\gany')[1]
'/app/ctrgany'
compile_re('/$c/$f', '/init/$c/$f')[0].pattern
'^.*?:https?://[^:/]+:[a-z]+ /(?Pc[w_]+)/(?Pf[w_]+)$'
compile_re('/$c/$f', '/init/$c/$f')[1]
'/init/gc/gf'
For the trunk version, the import line needs to change to this:
from gluon.rewrite import select, load, filter_url, filter_out,
filter_err, compile_re
Notice that the backslash escaping gets a little confusing. I can't figure
out how to use raw strings properly here; so it goes.
One more note. All the other doctests in routes.example.py use the patterns
defined earlier in the file. The compile_re tests are entirely self-contained,
so you can easily add them without any other editing. Just copy
routes.example.py to, say, routes.test.py, and add some compile_re tests, and
you're in business.
Re: [web2py] Why is routes.py not recommended for production?
On Aug 8, 2010, at 2:35 PM, mwolfe02 wrote: According to the web2py book, routes.py should not be used in production environment (http://web2py.com/book/default/chapter/04? search=lighttpd). Instead, Apache/lighttpd web server rewrite is suggested. I assumed this was due to some overhead that using routes.py would incur. However, massimo's response in this post from January (http:// groups.google.com/group/web2py/browse_thread/thread/39e72dc4a68f33a1) seems to suggest that's not the case. His response to the question, Why is routes.py not preferred? No reason. No overhead. If that's the case, then that's great news. I would much rather rewrite my urls inside web2py. If there is overhead involved, how does it compare to whatever overhead may be involved with Django's urls.py? Is there a fundamental difference between how the two frameworks implement url rewriting (other than the fact that it is required in Django and optional in web2py)? The reason (I think) is that Apache can serve certain static files and the like directly, without invoking web2py at all. It's not the overhead of using routes.py; it's the overhead of having web2py serve stuff that it doesn't need to. There might be more to it than that.
Re: [web2py] routes_in 153, me 0
On Aug 8, 2010, at 5:10 PM, Andrew Thompson wrote:
I can not wrap my head around routes_in. I'm attempting to host 3 sites via
lighttpd and fcgihandler.py (is this the best way?)
I ripped this mostly from a mdpierro post, and tweaked it for my domain, but
I'm just not making progress:
('(.*):https?://(.*)mysite\.com:(.*) /favicon.ico',
'/mysite/static/favicon.ico'),
http://mysite.com/mysite/static/favicon.ico -- this link works.
http://myste.com/favicon.ico -- Invalid request
You meant 'mysite', right?
I've been playing with the builtin doctest, and this passes:
filter_url('http://mysite.com/favicon.ico')
'http://mysite.com/mysite/static/favicon.ico'
Any ideas what I'm doing wrong?
You're restarting web2py when you change routes.py?
I think it'd be useful to be able to turn on logging of URL rewriting. It'd be
pretty verbose, I suppose, but useful for this kind of problem.
I have two more lines commented out that I also want to put in:
('(.*):https?://(.*)mysite\.com:(.*) /$anything', '/mysite/$anything'),
('(.*):https?://(.*)mysite\.com:(.*) /images/$anything',
'/mysite/static/images/$anything'),
I'm not sure, is $anything is compatible with all those .*'es?
I think so. No need to parenthesize them, since there's no capture going on. I
might use .*? instead. But .* should work in this case.
I do need the / to /mysite's to keep the sites separated from each other,
don't I?
I don't follow the question.
Any help appreciated.
--
Andrew Thompson
http://aktzero.com/
Re: [web2py] routes_in 153, me 0
On Aug 8, 2010, at 5:33 PM, Thadeus Burgess wrote:
The routes.example.py contains some unit tests. I have found this to
be the most effective way of testing my routes to make sure they do
what I want.
Also, I sent a note (yesterday?) to this list on how to use compile_re() in
doctest to see what the regex turned into (web2py munges it a little). That can
be helpful.
filter_url in the doctest has to convert the incoming URL to a set of env
variables, in imitation of the web server. So the action of the server isn't in
the loop, which could make a difference in some circumstances, I suppose.
That's why I'd like to see logging.
rewrite.py already does some logging. I'll take a look at doing something
simpleminded. Maybe a logging-enable variable in routes.py
I'm hacking in there anyway at the moment.
--
Thadeus
On Sun, Aug 8, 2010 at 7:30 PM, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 8, 2010, at 5:10 PM, Andrew Thompson wrote:
I can not wrap my head around routes_in. I'm attempting to host 3 sites via
lighttpd and fcgihandler.py (is this the best way?)
I ripped this mostly from a mdpierro post, and tweaked it for my domain,
but I'm just not making progress:
('(.*):https?://(.*)mysite\.com:(.*) /favicon.ico',
'/mysite/static/favicon.ico'),
http://mysite.com/mysite/static/favicon.ico -- this link works.
http://myste.com/favicon.ico -- Invalid request
You meant 'mysite', right?
I've been playing with the builtin doctest, and this passes:
filter_url('http://mysite.com/favicon.ico')
'http://mysite.com/mysite/static/favicon.ico'
Any ideas what I'm doing wrong?
You're restarting web2py when you change routes.py?
I think it'd be useful to be able to turn on logging of URL rewriting. It'd
be pretty verbose, I suppose, but useful for this kind of problem.
I have two more lines commented out that I also want to put in:
('(.*):https?://(.*)mysite\.com:(.*) /$anything', '/mysite/$anything'),
('(.*):https?://(.*)mysite\.com:(.*) /images/$anything',
'/mysite/static/images/$anything'),
I'm not sure, is $anything is compatible with all those .*'es?
I think so. No need to parenthesize them, since there's no capture going on.
I might use .*? instead. But .* should work in this case.
I do need the / to /mysite's to keep the sites separated from each other,
don't I?
I don't follow the question.
Any help appreciated.
--
Andrew Thompson
http://aktzero.com/
Re: [web2py] routes_in 153, me 0
On Aug 8, 2010, at 6:00 PM, Andrew Thompson wrote:
On 8/8/2010 8:30 PM, Jonathan Lundell wrote:
On Aug 8, 2010, at 5:10 PM, Andrew Thompson wrote:
I can not wrap my head around routes_in. I'm attempting to host 3 sites via
lighttpd and fcgihandler.py (is this the best way?)
I ripped this mostly from a mdpierro post, and tweaked it for my domain,
but I'm just not making progress:
('(.*):https?://(.*)mysite\.com:(.*) /favicon.ico',
'/mysite/static/favicon.ico'),
http://mysite.com/mysite/static/favicon.ico-- this link works.
http://myste.com/favicon.ico-- Invalid request
You meant 'mysite', right?
Yeah, examplification typo.
I've been playing with the builtin doctest, and this passes:
filter_url('http://mysite.com/favicon.ico')
'http://mysite.com/mysite/static/favicon.ico'
Any ideas what I'm doing wrong?
You're restarting web2py when you change routes.py?
Yes, I am.
I think it'd be useful to be able to turn on logging of URL rewriting. It'd
be pretty verbose, I suppose, but useful for this kind of problem.
I do need the / to /mysite's to keep the sites separated from each other,
don't I?
I don't follow the question.
Ignore it, I realized the initial domain to site mapping is happening in
lighttpd.
Is it possible that lighttpd is altering the URL enough that it doesn't match
what you've got in routes.py?
Re: [web2py] routes_in 153, me 0
On Aug 8, 2010, at 6:15 PM, Andrew Thompson wrote: On 8/8/2010 9:10 PM, Jonathan Lundell wrote: Is it possible that lighttpd is altering the URL enough that it doesn't match what you've got in routes.py? Perhaps. I've got to figure out how to test for that. I'll try to send Massimo a logging patch tomorrow, but if you look at rewrite.py, it should be semi-clear how to do it yourself. filter_in is where you want to log, and I'd log the arguments and return value of the filter_url call. In your case, the return value 'path' is what you want to see.
Re: [web2py] Managing routes.py with mercurial checkout
On Aug 9, 2010, at 8:20 AM, mwolfe02 wrote: I'm just looking for a best practice here. I am running web2py using a clone of the repository https://web2py.googlecode.com/hg/. This has worked really well for me. However, I just added a routes.py file. Clearly, I want to version control this file, but that requires committing the change to my local web2py repository. That's not a problem, except that every time I update to the latest web2py version I'll have two heads and have to merge. The merge should always be done without conflicts, but it would be an extra step I'd have to do each time. Also, if I wanted to send patches in at some point in the future, would those extra changesets in my local repository cause problems? I can describe what I do. It would have to be elaborated a bit to handle multiple applications. I keep a separate repository for my application, with a soft link from applications/ to tie it into web2py. I keep my routes.py in applications/myapp/private/, and move or link it to the web2py root as part of installation. App-specific routes.py will change this a bit, but it won't completely address the issue, since we still need a base routes.py at the web2py root.
Re: [web2py] Managing routes.py with mercurial checkout
On Aug 9, 2010, at 9:06 AM, Michael Wolfe wrote: OK, but that still leaves a link to routes.py in the web2py repository. Do you use the Pull -- Update -- Merge -- Commit approach then when you update your web2py repository? Actually, I misdescribed my configuration slightly. I run my app linked to a normally downloaded-and-installed web2py stable, not the hg pull. I use the hg-pulled copy for working on web2py, and not my application. If I had to keep the two projects (web2py and my app) in one tree, I think I might use .hgignore in the web2py root to exclude routes.py and my app folder. That's not a complete solution, because .hgignore is itself in web2py's repository. On Mon, Aug 9, 2010 at 11:40 AM, Jonathan Lundell jlund...@pobox.com wrote: On Aug 9, 2010, at 8:20 AM, mwolfe02 wrote: I'm just looking for a best practice here. I am running web2py using a clone of the repository https://web2py.googlecode.com/hg/. This has worked really well for me. However, I just added a routes.py file. Clearly, I want to version control this file, but that requires committing the change to my local web2py repository. That's not a problem, except that every time I update to the latest web2py version I'll have two heads and have to merge. The merge should always be done without conflicts, but it would be an extra step I'd have to do each time. Also, if I wanted to send patches in at some point in the future, would those extra changesets in my local repository cause problems? I can describe what I do. It would have to be elaborated a bit to handle multiple applications. I keep a separate repository for my application, with a soft link from applications/ to tie it into web2py. I keep my routes.py in applications/myapp/private/, and move or link it to the web2py root as part of installation. App-specific routes.py will change this a bit, but it won't completely address the issue, since we still need a base routes.py at the web2py root.
Re: [web2py] testing app-specific routing
On Aug 9, 2010, at 10:25 PM, Thadeus Burgess wrote: Where is the documentation on how to use it? I sent an email on the subject, and there are comments in routes.example.py. Tomorrow I'll try to add some info to the book in time for Massimo's deadline. Hang on just a minute and I'll try to find the messsage ... a minute passes ... here we are: 1. If you don't explicitly invoke any of the new features, routing should behave identically to before. If you see any different, please let us know asap. 2. You can now have a routes.py in the top level folder of an application, and it will be used *instead* of the base routes.py. However, it's not enough to simply have the file there; you must inform the routing logic about it. 3. The way you inform the routing logic is with a new element in the base routes.py: routes_app. routes_app is processed identically to routes_in, but the output must be an app name (or nothing). routes_app is processed at the beginning of a request. If it produces an app name, and that app has an app-specific routes.py (that is, applications/appname/routes.py), then that routes.py is used instead of the base routes.py. 4. In an unrelated change, there are three other new elements in routes.py: default_application = init default_controller = default default_function = index Note that default_application doesn't interact with app-specfic routing, since it's used after rewrite has taken place. default_controller and default_function should normally be used only in an app-specific routes.py, because, in the base routes.py, they will apply to all apps *without* an app-specific routes.py. That would probably lead to confusion when running admin or examples; at the very least their defaults would break. 5. As usual, I suggest that when you edit routes.example.py to generate a new routes.py, you also edit the doctest at the end, and use it to verify that you're getting what you expect. To run the doctest, just do python routes.py. There's also a patch pending that logs all rewrite operations. MUCH too verbose for production work, but could be useful in diagnosing problems with routes.py. (I think there's a problem with logging in general, though, so some more changes may be necessary.) It's largely untested, but I'll try to respond quickly to bug reports.
[web2py] web2py logging
web2py makes semi-extensive use of the Python logging module, but (near as I can tell) it never changes the output away from the default stdout. Is this true, and if so isn't it a problem?
Re: [web2py] Cyclic tables
On Aug 10, 2010, at 10:55 AM, Oatman wrote:
Hi all,
How do I create two tables that both refer to each other? Here's a
simple example:
db.define_table(
'item',
Field('current_alias', 'db.item_alias')
)
db.define_table(
'item_alias',
Field('name'),
Field('item', 'db.item')
)
This throws
SyntaxError: Field: unknown field type: db.item_alias for
current_alias
because they both need the other to be created first!
How can I create a simple relationship like this? Thanks in advance!
If the answer to this question is 'reference'
http://web2py.com/book/default/chapter/06?search=define_table#Self-Reference-and-Aliases,
perhaps we could add a line or three in the manual section to explicitly
mention forward references, and not only self references.
Re: [web2py] using URL() inside a component view
On Aug 10, 2010, at 5:48 PM, Miguel Lopes wrote:
I'm pretty sure this as been asked, but I can't find it anywhere.
I would like to use the URL function to construct a link inside a component
view (i.e. views/components/view_x.load). This would be a link to a
completely new page, to be loaded in the browser and not in the component
div. However, the default behavior is that all links inside a component are
trapped and loaded into the same component, thus if I have in the component
view:
a href={{URL(c='default', f='view_detail',
args=rec.contact.id)}}{{rec.contact.name}}/a
Are you sure this is the code? It doesn't seem right; I'd expect a syntax error
with no application or request specified.
At any rate, you can suppress the extension with extension= or
extension=False.
This gets translated into:
a href=/app/default/view_detail.load/6Clockwork Orange/a
And I would like to get:
a href=/init/default/view_detail/6Clockwork Orange/a
How can I achieve this?
Re: [web2py] using URL() inside a component view
On Aug 10, 2010, at 7:11 PM, Bruno Rocha wrote: extension=False is new to me, Thats not covered by the Book I found and reported that before : http://groups.google.com/group/web2py/browse_thread/thread/99c2f5b8db562fba/8b82a682dff3fc58?lnk=gstq=URL+changed#8b82a682dff3fc58 It's a side effect of the way URL fills in its values. A slight oversimplification: if you don't specify an extension, you'll get the current extension (.load in your case, more often .html). Specifying a value other than None (which leads to the default behavior) that evaluates to False causes the extension to be omitted.
Re: [web2py] using URL() inside a component view
On Aug 10, 2010, at 7:37 PM, Bruno Rocha wrote: Thank you agains 4 the explanation, now, this should be documented anywhere, as this is not on the book ( http://web2py.com/book/default/search?search=extension ) also not in AlterEgo ( http://www.web2py.com/AlterEgo/default/search?search=extension ) I've just added it to the end of the URL section of Chapter 4. Massimo, would you please check that I've got it right? 2010/8/10 Jonathan Lundell jlund...@pobox.com On Aug 10, 2010, at 7:11 PM, Bruno Rocha wrote: extension=False is new to me, Thats not covered by the Book I found and reported that before : http://groups.google.com/group/web2py/browse_thread/thread/99c2f5b8db562fba/8b82a682dff3fc58?lnk=gstq=URL+changed#8b82a682dff3fc58 It's a side effect of the way URL fills in its values. A slight oversimplification: if you don't specify an extension, you'll get the current extension (.load in your case, more often .html). Specifying a value other than None (which leads to the default behavior) that evaluates to False causes the extension to be omitted. -- http://rochacbruno.com.br
Re: [web2py] using URL() inside a component view
On Aug 10, 2010, at 7:59 PM, Bruno Rocha wrote: I thought your explanation is very well written, we need always maintain the practice of looking for something relevant to be documented when a new thread emerges, and document as quickly as possible, so soon web2py will be the framework with the best online documentation (if not already) Agreed. I'm not satisfied with unexplained solutions on this list; I want to know *why* something works. Or doesn't. And the book is a good place to document that stuff.
Re: [web2py] Re: Cyclic tables
On Aug 11, 2010, at 2:48 AM, Oatman wrote:
Jonathan,
Thanks very much for you pointing out the reference keyword, that
solves the problem for self-referencing tables, but do you know of a
solution to my two-tables-referencing-each-other problem? I admit my
simplistic alias example could be solved very elegantly by a self-
referencing table, but I'd like to know how to do it with two tables,
if I wanted.
I'm pretty sure that the subject has come up on this list, but I don't recall
when. I'm hoping that someone (Massimo when he comes up for air, perhaps) will
chime in, and then we can add it to the manual section.
I'm wondering, though, whether you can't use 'reference' in your first table to
refer to the second one.
Any thoughts? Thanks again
On Aug 10, 7:16 pm, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 10, 2010, at 10:55 AM, Oatman wrote:
Hi all,
How do I create two tables that both refer to each other? Here's a
simple example:
db.define_table(
'item',
Field('current_alias', 'db.item_alias')
)
db.define_table(
'item_alias',
Field('name'),
Field('item', 'db.item')
)
This throws
SyntaxError: Field: unknown field type: db.item_alias for
current_alias
because they both need the other to be created first!
How can I create a simple relationship like this? Thanks in advance!
If the answer to this question is 'reference'
http://web2py.com/book/default/chapter/06?search=define_table#Self-Re...,
perhaps we could add a line or three in the manual section to explicitly
mention forward references, and not only self references.
Re: [web2py] Chapter3: Form self-submission
On Aug 11, 2010, at 6:05 AM, Hector Oron wrote:
Hello,
Following the example on Chapter 3 [1] about Form self-submission, I
get Internal Error after visiting first and introducing the name,
getting the following traceback:
What version of web2py are you running?
If you're not running something fairly current, try adding r=request to your
URL() calls (it won't do any harm in later versions, but shouldn't be
necessary).
Traceback (most recent call last):
File /home/zumbi/TCL/SRC_test/web2py/gluon/restricted.py, line
178, in restricted
exec ccode in environment
File
/home/zumbi/TCL/SRC_test/web2py/applications/mytcl/controllers/default.py,
line 22, in module
File /home/zumbi/TCL/SRC_test/web2py/gluon/globals.py, line 96, in lambda
self._caller = lambda f: f()
File
/home/zumbi/TCL/SRC_test/web2py/applications/mytcl/controllers/default.py,
line 14, in first
redirect(URL('second'))
File /home/zumbi/TCL/SRC_test/web2py/gluon/html.py, line 182, in URL
raise SyntaxError, 'not enough information to build the url'
SyntaxError: not enough information to build the url
# -*- coding: utf-8 -*-
#
## This is a samples controller
## - index is the default action of any application
## - user is required for authentication and authorization
## - download is for downloading files uploaded in the db (does streaming)
## - call exposes all registered services (none by default)
#
def first():
if request.vars.visitor_name:
session.visitor_name = request.vars.visitor_name
redirect(URL('second'))
return dict()
def second():
if not request.function=='first' and not session.visitor_name:
redirect(URL('first'))
return dict()
response._vars=response._caller(first)
^^^-- This line does not exist on my controller code.
controllers/default.py:
def first():
if request.vars.visitor_name:
session.visitor_name = request.vars.visitor_name
redirect(URL('second'))
return dict()
def second():
if not request.function=='first' and not session.visitor_name:
redirect(URL('first'))
return dict()
default/first:
{{extend 'layout.html'}}
What is your name?
form
input name=visitor_name /
input type=submit /
/form
default/second:
{{extend 'layout.html'}}
h1Hello {{=session.visitor_name or anonymous}}/h1
[1] http://web2py.com/book/default/chapter/03
Cheers,
--
Héctor Orón
Our Sun unleashes tremendous flares expelling hot gas into the Solar
System, which one day will disconnect us.
-- Day DVB-T stop working nicely
Video flare: http://antwrp.gsfc.nasa.gov/apod/ap100510.html
Re: [web2py] Re: web2py logging
On Aug 11, 2010, at 6:00 AM, mdipierro wrote: we could specify line option to change it but why not use regular stdout redirection? I suppose we could, but at least when we start via web2py.py, it's good to see the startup messages directed to stdout before web2py daemonizes. The nice thing about having a standard logging configuration file is that it gives the user a lot more control over things. For example, if I understand the logging module correctly, it could be used to send logging to syslog. A question, though: web2py can run in various deployment modes. I'm wondering if there's a central place to configure logging, common to all modes? rocket does installs logging.NullHandler, for example. I'm unclear on the implications of that for web2py applications. On Aug 10, 12:28 pm, Jonathan Lundell jlund...@pobox.com wrote: web2py makes semi-extensive use of the Python logging module, but (near as I can tell) it never changes the output away from the default stdout. Is this true, and if so isn't it a problem?
Re: [web2py] Re: web2py logging (and Rocket: ahoy, Tim!)
On Aug 11, 2010, at 8:21 AM, mdipierro wrote: We could have a standard web2py logging file such as errors.log I think that would be a good default, but I'd rather have a standard web2py logging configuration file, web2py/logging.conf or the like. It could point to errors.log by default, but the user could edit the configuration file to change that. The logging module also supports rotating log files, which would be a good default, I think, to prevent runaway log file growth. I was just looking at the RHEL init startup scripts (this applies to Fedora as well); when web2py is started that way (which is what I do in production), stdout and stderr are automatically redirected to /dev/null (in the daemon function), so redirection in the startup script will be overridden. I'd like Tim's input on this, since Rocket has some logging initialization logic as well. Tim: the subject is consistent (and configurable) runtime logging for the web2py core as well as applications. On Aug 11, 10:14 am, Jonathan Lundell jlund...@pobox.com wrote: On Aug 11, 2010, at 6:00 AM, mdipierro wrote: we could specify line option to change it but why not use regular stdout redirection? I suppose we could, but at least when we start via web2py.py, it's good to see the startup messages directed to stdout before web2py daemonizes. The nice thing about having a standard logging configuration file is that it gives the user a lot more control over things. For example, if I understand the logging module correctly, it could be used to send logging to syslog. A question, though: web2py can run in various deployment modes. I'm wondering if there's a central place to configure logging, common to all modes? rocket does installs logging.NullHandler, for example. I'm unclear on the implications of that for web2py applications. On Aug 10, 12:28 pm, Jonathan Lundell jlund...@pobox.com wrote: web2py makes semi-extensive use of the Python logging module, but (near as I can tell) it never changes the output away from the default stdout. Is this true, and if so isn't it a problem?
Re: [web2py] Re: Cyclic tables
On Aug 11, 2010, at 8:29 AM, mdipierro wrote:
It has come up. reference table does not solve this problem. The
problem can be solved with the Table keyword but I think tables should
not refer to each other. Use a link table of set one of the references
to an int and use a validator to make it a reference.
If someone will test an example of this, I'll add it to the book. It's too
late, I assume, to make the next print edition, but at least we'll have it on
line, and it'll make the next one.
On Aug 11, 9:56 am, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 11, 2010, at 2:48 AM, Oatman wrote:
Jonathan,
Thanks very much for you pointing out the reference keyword, that
solves the problem for self-referencing tables, but do you know of a
solution to my two-tables-referencing-each-other problem? I admit my
simplistic alias example could be solved very elegantly by a self-
referencing table, but I'd like to know how to do it with two tables,
if I wanted.
I'm pretty sure that the subject has come up on this list, but I don't
recall when. I'm hoping that someone (Massimo when he comes up for air,
perhaps) will chime in, and then we can add it to the manual section.
I'm wondering, though, whether you can't use 'reference' in your first table
to refer to the second one.
Any thoughts? Thanks again
On Aug 10, 7:16 pm, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 10, 2010, at 10:55 AM, Oatman wrote:
Hi all,
How do I create two tables that both refer to each other? Here's a
simple example:
db.define_table(
'item',
Field('current_alias', 'db.item_alias')
)
db.define_table(
'item_alias',
Field('name'),
Field('item', 'db.item')
)
This throws
SyntaxError: Field: unknown field type: db.item_alias for
current_alias
because they both need the other to be created first!
How can I create a simple relationship like this? Thanks in advance!
If the answer to this question is 'reference'
http://web2py.com/book/default/chapter/06?search=define_table#Self-Re...,
perhaps we could add a line or three in the manual section to explicitly
mention forward references, and not only self references.
Re: [web2py] Re: Is there a way how to specifiy, which controller act as default?
On Aug 6, 2010, at 9:40 AM, Phyo Arkar wrote: i voted this too. I just dont like the controller named default and i want it different on each apps. On 8/6/10, mr.freeze nat...@freezable.com wrote: Massimo, why not make the default application,controller and function configurable in an external file like options.py or parameters_.py? This is now in the trunk. See routes.example.py for documentation (there's also a note in the online book).
Re: [web2py] routes_in 153, me 0
On Aug 8, 2010, at 6:29 PM, Jonathan Lundell wrote: On Aug 8, 2010, at 6:15 PM, Andrew Thompson wrote: On 8/8/2010 9:10 PM, Jonathan Lundell wrote: Is it possible that lighttpd is altering the URL enough that it doesn't match what you've got in routes.py? Perhaps. I've got to figure out how to test for that. I'll try to send Massimo a logging patch tomorrow, but if you look at rewrite.py, it should be semi-clear how to do it yourself. filter_in is where you want to log, and I'd log the arguments and return value of the filter_url call. In your case, the return value 'path' is what you want to see. The logging patch is now in the trunk. Enable it via routes.py (see routes.example.py to see how). I'm a little hazy on how well logging works generally in web2py, but I *think* that if you run web2py.py from the command line you ought to see it. Set logging to error if you need to be sure it's showing up. Also, you can enable logging in conjunction with running the routes.py doctest, if you have that configured, and see soem of what's going on without running the main web2py. But it's not as complete a test.
Re: [web2py] Re: doc2py - php.net like python module documentation
A minor suggestion: when an arg default is [] or {}, it would enhance
readability to stick a space in there. Or set it in a monospace font.
(I can't say I'm a big fan of Andale Mono, though.)
Re: [web2py] Re: web2py logging
On Aug 11, 2010, at 8:21 AM, mdipierro wrote: We could have a standard web2py logging file such as errors.log I think I have a straightforward way of doing it. I'm not sure I can get to it this week, but it should be straightforward. In the meantime, I haven't tested it, but I think console logging will work OK when web2py.py is run from the command line. On Aug 11, 10:14 am, Jonathan Lundell jlund...@pobox.com wrote: On Aug 11, 2010, at 6:00 AM, mdipierro wrote: we could specify line option to change it but why not use regular stdout redirection? I suppose we could, but at least when we start via web2py.py, it's good to see the startup messages directed to stdout before web2py daemonizes. The nice thing about having a standard logging configuration file is that it gives the user a lot more control over things. For example, if I understand the logging module correctly, it could be used to send logging to syslog. A question, though: web2py can run in various deployment modes. I'm wondering if there's a central place to configure logging, common to all modes? rocket does installs logging.NullHandler, for example. I'm unclear on the implications of that for web2py applications. On Aug 10, 12:28 pm, Jonathan Lundell jlund...@pobox.com wrote: web2py makes semi-extensive use of the Python logging module, but (near as I can tell) it never changes the output away from the default stdout. Is this true, and if so isn't it a problem?
Re: [web2py] Re: proper way to transform a query GAE #1
On Aug 11, 2010, at 5:14 PM, Rob wrote: Where is this syntax documented? db(query1)(query2) Is this the same as 'db(query1 query2)' except web2py does the join and not the database? It's identical. db(query) returns a Set instance. And calling a Set instance that already has a query combines the queries with : def __call__(self, query): if self._query: return Set(self._db, self._query query) else: return Set(self._db, query) (this is from dal.py; sql.py isn't quite so readable) As for documentation, I think that all these DAL behaviors ought to be better documented. Maybe the next rev of the book
Re: [web2py] Re: Is there a way how to specifiy, which controller act as default?
On Aug 11, 2010, at 6:43 PM, mr.freeze wrote: Very nice. I think this is more congruous with web2py's philosophy of ease of use. It seems clean enough, I think. One thing to keep in mind (I think I've mentioned this somewhere) is that the default_application setting is effective only after rewriting. And with app-specific routing, of course, the other defaults can and should be in the app-specific routes.py, if there is one. On Aug 11, 11:12 am, Jonathan Lundell jlund...@pobox.com wrote: On Aug 6, 2010, at 9:40 AM, Phyo Arkar wrote: i voted this too. I just dont like thecontrollernameddefaultand i want it different on each apps. On 8/6/10, mr.freeze nat...@freezable.com wrote: Massimo, why not make thedefaultapplication,controllerand function configurable in an external file like options.py or parameters_.py? This is now in the trunk. See routes.example.py for documentation (there'salso a note in the online book).
Re: [web2py] web2py FORM, concurrency and failed requests
On Aug 12, 2010, at 8:44 AM, David Marko wrote: Failed requests:19 (Connect: 0, Receive: 0, Length: 19, Exceptions: 0) http://www.celebrazio.net/tech/unix/apache_bench.html In the above example, if the server returns dynamic content, the file size may be different from one request to another. One difference, for example, is ads on the page - they vary per request and have different byte size. When this happens, Apache bench notes it as Failed request for Length (byte size) being different. We can ignore that.
Re: [web2py] web2py FORM, concurrency and failed requests
On Aug 12, 2010, at 8:44 AM, David Marko wrote: Failed requests:19 (Connect: 0, Receive: 0, Length: 19, Exceptions: 0) May not actually be a problem. Reason? It might be a site that serves dynamic context (such as different cookie IDs mentioned) where the file size changes between each query. http://www.celebrazio.net/tech/unix/apache_bench.html Easy way to verify: Code: $ wget url Repeat that twice, then: Code: $ diff retrieved filename retrieved filename.1 See if there are differences. If yes, then you can ignore the length-related failures. If no, it's some other cause and need to investigate further. http://www.linode.com/forums/archive/o_t/t_1633/apache_benchmark_high_
Re: [web2py] web2py FORM, concurrency and failed requests
On Aug 12, 2010, at 8:44 AM, David Marko wrote: Failed requests:19 (Connect: 0, Receive: 0, Length: 19, Exceptions: 0) Even more reassurance: http://stackoverflow.com/questions/1512304/failed-requests-by-length-in-my-apachebench-load-test-result http://alwaysthecritic.typepad.com/atc/2009/04/apache-bench-notes.html The length variation should be checked, of course, but it may well be harmless. If the length variation is under our control (cookie format, maybe?), perhaps we could make an effort to make it the same.
Re: [web2py] Re: web2py FORM, concurrency and failed requests
On Aug 12, 2010, at 10:00 AM, mdipierro wrote: You are the man. For the page I am considering the fail requests are not a real failure but declare a content-length of 19383 (wrong) instead of 19384 (correct). Let's continue investigate... I'm seeing two variations (using curl -i). One is harmless: divdatetime.datetime(2010, 8, 12, 18, 0, 36, 68094)/div vs divdatetime.datetime(2010, 8, 12, 18, 3, 15, 325125)/div However, the other is odd: tmp $ grep -i Transfer-Encoding ? 2:Transfer-Encoding: chunked 4:Transfer-Encoding: chunked 5:Transfer-Encoding: chunked 6:Transfer-Encoding: chunked tmp $ grep -i Transfer-Length ? tmp $ grep -i Content-Length ? 1:Content-Length: 24163 3:Content-Length: 24164 (filenames 1-6 are my curl -i captures) Apparently at random, I see chunked returns. My request is: curl -i http://web2py.com/examples/form_examples/form
Re: [web2py] Re: web2py FORM, concurrency and failed requests
On Aug 12, 2010, at 10:00 AM, mdipierro wrote: For the page I am considering the fail requests are not a real failure but declare a content-length of 19383 (wrong) instead of 19384 (correct). Let's continue investigate... Once the chunked encoding question is answered, it seems to me that we ought to create some pages (in examples?) expressly for the purpose of benchmarking. Make sure they're the same length, so we don't get the bogus errors, and create some variations: with and without session.forget, with and without a database access, etc.
Re: [web2py] Re: Can you share some web2py performance info?
On Aug 12, 2010, at 10:29 AM, mdipierro wrote: The fail requests are due to the fact that dynamical pages may have different length at each time and ab reports them as failure. It is not an issue. http://groups.google.com/group/web2py/msg/e6509b1d543696b3 But why the erratic transfer-encoding: chunked? On Aug 12, 11:13 am, Thadeus Burgess thade...@thadeusb.com wrote: Thank god I'm not the only one =) -- Thadeus On Thu, Aug 12, 2010 at 10:35 AM, mdipierro mdipie...@cs.depaul.edu wrote: There numbers are more typical. Consider that those example page also have a db connection and load/store sessions. I do not understand the 30% fail requests on form page. Can you reproduce the problem using Rocket instead of Apache? what is the exact ab statement. I want to try reproduce the problem. Massimo On Aug 12, 10:29 am, David Marko dma...@tiscali.cz wrote: I just tried to bechmark standard web2py examples on the same machine. This example /examples/simple_examples/hello3.html gives me 70 req/sec and it also returns almost 7kB of page data. and /examples/form_examples/form gives me 25 req/sec but with 30% of fail requests and this returns of 25kB of data, so its very nice. (I have reporetd this previously .. its strange that pages woth forms causes failed requests) David On 12 srp, 17:13, mdipierro mdipie...@cs.depaul.edu wrote: It seems that it takes 600ms to serve a page when you are serving 10 at the time. Considering that Python because of the GIL probably not taking advantage of multiple cores you may have this is an effective time per request of 600/10=60ms. I still think it is too high considering you have a fast CPU. Can you check memory and cpu usage wit top? Do you have many model files? What is in there? What is the apache configuration (processes or threads)? Massimo On Aug 12, 10:05 am, David Marko dma...@tiscali.cz wrote: This is what I can see in log. The first two lines are when served just one request = simple page reload. The rest is with apache benchmark with concurrency set to 10. 192.168.2.62, 2010-08-12 19:00:35, GET, /init/default/index, HTTP/1.0, 200, 0.082553 192.168.2.62, 2010-08-12 19:01:02, GET, /init/default/index, HTTP/1.0, 200, 0.086724 192.168.2.62, 2010-08-12 19:01:31, GET, /init/default/index, HTTP/1.0, 200, 0.504790 192.168.2.62, 2010-08-12 19:01:31, GET, /init/default/index, HTTP/1.0, 200, 0.506875 192.168.2.62, 2010-08-12 19:01:31, GET, /init/default/index, HTTP/1.0, 200, 0.516474 192.168.2.62, 2010-08-12 19:01:31, GET, /init/default/index, HTTP/1.0, 200, 0.599019 192.168.2.62, 2010-08-12 19:01:31, GET, /init/default/index, HTTP/1.0, 200, 0.597636 192.168.2.62, 2010-08-12 19:01:31, GET, /init/default/index, HTTP/1.0, 200, 0.622482 192.168.2.62, 2010-08-12 19:01:31, GET, /init/default/index, HTTP/1.0, 200, 0.629780 192.168.2.62, 2010-08-12 19:01:31, GET, /init/default/index, HTTP/1.0, 200, 0.660393 192.168.2.62, 2010-08-12 19:01:31, GET, /init/default/index, HTTP/1.0, 200, 0.677426 192.168.2.62, 2010-08-12 19:01:31, GET, /init/default/index, HTTP/1.0, 200, 0.712054 On 12 srp, 16:56, mdipierro mdipie...@cs.depaul.edu wrote: Here are the logs for my web server running web2py.com. The log only shows dynamic pages most of which (like the book) use db. http://web2py.com/examples/static/logs.txt Most pages take around ~20ms but this is a 600MHz VPS with 384MB Ram. On Aug 12, 9:48 am, mdipierro mdipie...@cs.depaul.edu wrote: Something is wrong. In wsgiserver.py set LOGGING = True and look at web2py/httpserver.log what times do you get? do you have very large model files? Massimo On Aug 12, 9:43 am, David Marko dma...@tiscali.cz wrote: I just moved my first real-life app into production (its a private app so I cant share screens etc.). Its running latest stable web2py on Debian 5 + Python 2.6.5 + modwsgi 3.3 . When testing application home page, where are no sql commands, session is disabled (by session.forget()) and migration is disabled, I can get, using apache benchmark, only 12 req/sec. When I compile app, it increases up to 16 req/sec, but both numbers seems to me very low. Can you share some your experience what one can get from web2py? Using browser, the application feels very responsive, but I'm just scared (a bit) how this will change when all users start using the app. My server HW is one processor 3GHz, 1.5 GB RAM. David
Re: [web2py] Re: web2py FORM, concurrency and failed requests
On Aug 12, 2010, at 10:53 AM, mdipierro wrote: The chunked issue needs to be investigated. Are you using rocket or apache? I also noticed rocket responds always with http/1.1 which (allows the server to decide on the option of chunked encoding and it may decide based on the length of content). The problem is that ab - I think - asks for 1.0 since according to the docs does not support 1.x fully. curl -i http://web2py.com/examples/form_examples/form You'd know better than I. According to the headers, Apache and http 1.1: HTTP/1.1 200 OK Date: Thu, 12 Aug 2010 17:00:36 GMT Server: Apache/2.2.8 (Ubuntu) mod_wsgi/3.2-BRANCH Python/2.5.2 mod_ssl/2.2.8 OpenSSL/0.9.8g Expires: Thu, 12 Aug 2010 17:00:36 GMT^M Pragma: no-cache Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Set-Cookie: session_id_examples=70-0-249-65-52b1ec97-d08b-435e-a1b7-ddc1f87e0ddd; Path=/ Content-Length: 24163 Content-Type: text/html; charset=utf-8 HTTP/1.1 200 OK Date: Thu, 12 Aug 2010 17:00:39 GMT Server: Apache/2.2.8 (Ubuntu) mod_wsgi/3.2-BRANCH Python/2.5.2 mod_ssl/2.2.8 OpenSSL/0.9.8g Expires: Thu, 12 Aug 2010 17:00:39 GMT Pragma: no-cache Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Set-Cookie: session_id_examples=70-0-249-65-3139077f-0ef8-4e39-8ddf-6bc40690414f; Path=/ Transfer-Encoding: chunked Content-Type: text/html; charset=utf-8 Massimo On Aug 12, 12:08 pm, Jonathan Lundell jlund...@pobox.com wrote: On Aug 12, 2010, at 10:00 AM, mdipierro wrote: You are the man. For the page I am considering the fail requests are not a real failure but declare a content-length of 19383 (wrong) instead of 19384 (correct). Let's continue investigate... I'm seeing two variations (using curl -i). One is harmless: divdatetime.datetime(2010, 8, 12, 18, 0, 36, 68094)/div vs divdatetime.datetime(2010, 8, 12, 18, 3, 15, 325125)/div However, the other is odd: tmp $ grep -i Transfer-Encoding ? 2:Transfer-Encoding: chunked 4:Transfer-Encoding: chunked 5:Transfer-Encoding: chunked 6:Transfer-Encoding: chunked tmp $ grep -i Transfer-Length ? tmp $ grep -i Content-Length ? 1:Content-Length: 24163 3:Content-Length: 24164 (filenames 1-6 are my curl -i captures) Apparently at random, I see chunked returns. My request is: curl -ihttp://web2py.com/examples/form_examples/form
Re: [web2py] Re: web2py FORM, concurrency and failed requests
On Aug 12, 2010, at 5:13 PM, mdipierro wrote: I looked at the Rocket web server: 1) it always responds with HTTP/1.1 even if client is HTTP/1.0. I am not sure this is allowed. I think not. A response must follow a protocol that is same or lower then requested protocol 2) it seems Rocket responds with chunked encoding in two cases: a) the request was chunked; b) if the wsgi app returns a list with more than one element (web2py can do this but normally does not do it). There is one caveat abdou 2) There is this code def write(self, data, sections=None): Write the data to the output socket. if not self.headers_sent: self.send_headers(data, sections) If it happens that self.send_headers(data, None) is called, it will force chunked-encoding. I do not see how that can happen but I cannot exclude since I do not see the logic behind calling send_headers in two places. What's odd is that we see the different cases on identical calls to the same URL. I found the chunked logic a little confusing, probably because I don't really understand what it's trying to do, or how it gets called. At least it seems easy to reproduce. At least in my tests (through web2py.com), I'm seeing Apache. Is that mod_proxy? On Aug 12, 12:48 pm, Thadeus Burgess thade...@thadeusb.com wrote: In my testings the errors were actually 500 internal server errors, not content length issues. These pages were dynamic in that they returned information from the database, but it was always the exact same 10 records. The content length was not one of the reported failed requests types. I can even replicate this with the welcome app, just add a friends table, and a sqltable on the default/index page, insert some records, then run ab on the index page, it always return sthe same thing, but will generate roughly 15% error ratios not related to content length (since it stays the same) -- Thadeus On Thu, Aug 12, 2010 at 12:27 PM, mdipierro mdipie...@cs.depaul.edu wrote: I think I figure it. It is not a bug in web2py. The Content-Length is not computed by web2py but it is computed by the web server and it is computed correctly. The length is actually different at each request. This is because: 1) in forms it contains the CSRF token with is a uuid and is different at each request 2) forms that contain a date may have different rounding for seconds 3) pages with display [request], [response], etc also contain datetime info which have different length at every request The second link Jonathan posted says: Quite often you may see in the statistics Failed requests: 5 or similar, followed by a list of the types of failure: (Connect: 0, Receive: 0, Length: 5, Exceptions: 0). If the only type of failure that actually occurred is 'Length' then don't be alarmed. This simply means that each request (for the same URL) returned a different length response, which ab regards as suspicious. However it's perfectly normal for dynamic webpages, especially if they include the time or other very dynamic data on the page. This our case. Case closed? Massimo On Aug 12, 12:00 pm, mdipierro mdipie...@cs.depaul.edu wrote: You are the man. For the page I am considering the fail requests are not a real failure but declare a content-length of 19383 (wrong) instead of 19384 (correct). Let's continue investigate... Massimo On Aug 12, 11:53 am, Jonathan Lundell jlund...@pobox.com wrote: On Aug 12, 2010, at 8:44 AM, David Marko wrote: Failed requests:19 (Connect: 0, Receive: 0, Length: 19, Exceptions: 0) Even more reassurance: http://stackoverflow.com/questions/1512304/failed-requests-by-length-... http://alwaysthecritic.typepad.com/atc/2009/04/apache-bench-notes.html The length variation should be checked, of course, but it may well be harmless. If the length variation is under our control (cookie format, maybe?), perhaps we could make an effort to make it the same.
Re: [web2py] Re: web2py FORM, concurrency and failed requests
On Aug 12, 2010, at 11:59 PM, mdipierro wrote: I originally misunderstood the issue. I thought the problem was with Rocket. web2py.com runs on Apache+mod_wsgi. In this case Apache decides on chunking, not web2py. It is strange but I do not know what is the Apache policy in this respect. There's a bit of explanation here, along with a fragment of relevant Apache source (in the comments): http://bytes.com/topic/php/answers/10395-chunked-encoding-php-apache2 Massimo On Aug 12, 9:59 pm, Jonathan Lundell jlund...@pobox.com wrote: On Aug 12, 2010, at 5:13 PM, mdipierro wrote: I looked at the Rocket web server: 1) it always responds with HTTP/1.1 even if client is HTTP/1.0. I am not sure this is allowed. I think not. A response must follow a protocol that is same or lower then requested protocol 2) it seems Rocket responds with chunked encoding in two cases: a) the request was chunked; b) if the wsgi app returns a list with more than one element (web2py can do this but normally does not do it). There is one caveat abdou 2) There is this code def write(self, data, sections=None): Write the data to the output socket. if not self.headers_sent: self.send_headers(data, sections) If it happens that self.send_headers(data, None) is called, it will force chunked-encoding. I do not see how that can happen but I cannot exclude since I do not see the logic behind calling send_headers in two places. What's odd is that we see the different cases on identical calls to the same URL. I found the chunked logic a little confusing, probably because I don't really understand what it's trying to do, or how it gets called. At least it seems easy to reproduce. At least in my tests (through web2py.com), I'm seeing Apache. Is that mod_proxy? On Aug 12, 12:48 pm, Thadeus Burgess thade...@thadeusb.com wrote: In my testings the errors were actually 500 internal server errors, not content length issues. These pages were dynamic in that they returned information from the database, but it was always the exact same 10 records. The content length was not one of the reported failed requests types. I can even replicate this with the welcome app, just add a friends table, and a sqltable on the default/index page, insert some records, then run ab on the index page, it always return sthe same thing, but will generate roughly 15% error ratios not related to content length (since it stays the same) -- Thadeus On Thu, Aug 12, 2010 at 12:27 PM, mdipierro mdipie...@cs.depaul.edu wrote: I think I figure it. It is not a bug in web2py. The Content-Length is not computed by web2py but it is computed by the web server and it is computed correctly. The length is actually different at each request. This is because: 1) in forms it contains the CSRF token with is a uuid and is different at each request 2) forms that contain a date may have different rounding for seconds 3) pages with display [request], [response], etc also contain datetime info which have different length at every request The second link Jonathan posted says: Quite often you may see in the statistics Failed requests: 5 or similar, followed by a list of the types of failure: (Connect: 0, Receive: 0, Length: 5, Exceptions: 0). If the only type of failure that actually occurred is 'Length' then don't be alarmed. This simply means that each request (for the same URL) returned a different length response, which ab regards as suspicious. However it's perfectly normal for dynamic webpages, especially if they include the time or other very dynamic data on the page. This our case. Case closed? Massimo On Aug 12, 12:00 pm, mdipierro mdipie...@cs.depaul.edu wrote: You are the man. For the page I am considering the fail requests are not a real failure but declare a content-length of 19383 (wrong) instead of 19384 (correct). Let's continue investigate... Massimo On Aug 12, 11:53 am, Jonathan Lundell jlund...@pobox.com wrote: On Aug 12, 2010, at 8:44 AM, David Marko wrote: Failed requests:19 (Connect: 0, Receive: 0, Length: 19, Exceptions: 0) Even more reassurance: http://stackoverflow.com/questions/1512304/failed-requests-by-length-... http://alwaysthecritic.typepad.com/atc/2009/04/apache-bench-notes.html The length variation should be checked, of course, but it may well be harmless. If the length variation is under our control (cookie format, maybe?), perhaps we could make an effort to make it the same.
Re: [web2py] Re: Problem with IS_STRONG() and CRYPT() methods
On Aug 13, 2010, at 2:19 AM, mdipierro wrote: @elfuogo1, let us know if the problems are solved. Following up on Mr Freeze's comment, IS_STRONG is a bit problematical, or at least it doesn't fit so well into the pattern of the other validators. There's no good way to translate its messages, for example, without reducing them to a single message (which isn't at all what it does by default). I don't have a solution, but it might be worthwhile documenting its quirks. On Aug 12, 10:36 pm, mr.freeze nat...@freezable.com wrote: IS_STRONG is failing for a different reason but displaying your error message. The defaults are: min=8, max=20, upper=1, lower=1, number=1, special=1 If you remove your error message, you will get a descriptive message for each failure. You can set each parameter to 0 to disallow and to None to not check. db.auth_user.password.requires = [IS_STRONG(min=8,max=None,upper=None, lower=None,special=None,number=None,error_message='Too short'), CRYPT(auth.settings.hmac_key)] Perhaps the defaults should be less aggressive. Not sure on the CRYPT, it hashes the password for me. On Aug 12, 9:53 pm, elfuego1 elfue...@gmail.com wrote: Hi, I have a problem with two things in registration form. 1. Definition for password field in database looks as follows: db.auth_user.password.requires = [IS_STRONG(min=8 ,error_message='Your password is too short!'), CRYPT(auth.settings.hmac_key)] But the form is not accepting passwords. Each time I want to send a form it shows me error message: 'Your password is too short', even if the password is much longer than required 8 signs. After removing 'min=8' parameter I'm able to save my form in database. 2. Although I have provided an encryption setting: auth.settings.hmac_key='sha512:something password is not encrypted in the database. Can you help me and tell me what I'm doing wrong? Am I missing some parameters?
[web2py] widget.py: -f folder
Remind me, please, what the function of the -f folder command line option is. All the other startup method appear to use the location of the startup file (eg wsgihandler.py).
Re: [web2py] Re: is it possible to pass string C++ as URL argument?
On Aug 14, 2010, at 6:10 AM, mdipierro wrote:
On Aug 14, 5:55 am, Jurgis Pralgauskis jurgis.pralgaus...@gmail.com
wrote:
On Aug 5, 3:41 pm, mdipierro mdipie...@cs.depaul.edu wrote:
the problem is not the escaping (which is correct, + must be escaped)
the problem is thta web2py urls do not allow non-alphanumeric chars.
but for arguments there seems to be no need for such restriction, or
is there?
It is our policy that arguments should be alphanumeric. This is
because they can be used to id records or access the file system and
we do not worry you to have to worry about validating then. This does
not mean you cannot have c++ in a URL. It just means you must use
routes to map an arg into a var.:
routes_in =[('/app/c/f/$anything','/app/c/f/data=$anything')]
Down the road, I hope to introduce a URL validation rewrite subsystem that
will allow the user a little more flexibility in this respect; the current
restriction doesn't make sense in all cases (or could be enforced more locally).
Re: [web2py] Re: is it possible to pass string C++ as URL argument?
On Aug 14, 2010, at 10:53 AM, mdipierro wrote:
perhaps app level regex validation config?
We need to first do standards-based URL parsing and decoding, and then apply
validation as a subsequent step.
One option would be to accept anything that's legal in a URL; the app would do
its own validation. This would be useful for a query string that needed
different validation for different args/vars.
We could automate that, too, or at least provide helpers. Perhaps a list of
var-key patterns and corresponding validation patterns for the arguments.
On Aug 14, 11:04 am, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 14, 2010, at 6:10 AM, mdipierro wrote:
On Aug 14, 5:55 am, Jurgis Pralgauskis jurgis.pralgaus...@gmail.com
wrote:
On Aug 5, 3:41 pm, mdipierro mdipie...@cs.depaul.edu wrote:
the problem is not the escaping (which is correct, + must be escaped)
the problem is thta web2py urls do not allow non-alphanumeric chars.
but for arguments there seems to be no need for such restriction, or
is there?
It is our policy that arguments should be alphanumeric. This is
because they can be used to id records or access the file system and
we do not worry you to have to worry about validating then. This does
not mean you cannot have c++ in a URL. It just means you must use
routes to map an arg into a var.:
routes_in =[('/app/c/f/$anything','/app/c/f/data=$anything')]
Down the road, I hope to introduce a URL validation rewrite subsystem that
will allow the user a little more flexibility in this respect; the current
restriction doesn't make sense in all cases (or could be enforced more
locally).
Re: [web2py] Re: Translated sets
On Aug 15, 2010, at 6:48 AM, mdipierro wrote: IS_IN_SET(set,[T(x) for x in set]) How do the language files get updated for something dynamic like this? On Aug 15, 7:54 am, Fabio Alessandro Locati floc...@grimp.eu wrote: Hi, I've created a set in 'modules/sets.py'. I use this set for a db field (I use db.table.field.require = IS_IN_SET(set)). Is possible to internazionalize it? If yes, how? Ps: another question about internazionalization: it seems that pressing 'update all languages' web2py adds to each lang file the definitions the user added since last update. It seems, to me, that it does not remove the unused ones. Is this a feature or a bug? Is there a way to force the deletion too? Thank you very much :) Fale -- Inviato dal mio dispositivo mobile
Re: [web2py] Re: ?code=400ticket=Nonerequested_uri=None
On Aug 16, 2010, at 11:56 AM, mdipierro wrote: This is important and should be fixed. we will look into it. Perhaps Jonathan who has worked on routes extensively can look into this. I'm a little in the dark on routes_onerror; I recall looking at it a while back when I noticed that its doctest was failing, but I didn't figure it out. I looked at some of the logic now, though, and maybe there's a problem buried inside. 1. main.wsgibase populates request.env by lowercasing environ items (and replacing dots with underscores). 2. main.parse_url uses path_info in preference to request_uri. If path_info is missing, it parses request_uri into path_info and query_string. 3. If we're filtering routes_in, AND the rewritten path has a ? in it, we rewrite REQUEST_URI. But otherwise, we never do. I wonder if we shouldn't rewrite request_uri in filter_in regardless of whether there's a query string, and perhaps in parse_url, write something into request_uri if it's empty. The non-standardization of these variables is a PITA, from what I can tell. Another workaround might be for Rocket to go ahead and initialize request_uri, I suppose, but who knows what all the other web servers do? We could also try to rationalize the environment very early in an incoming request, perhaps just before filter_in. I hesitate to mess around in this logic, since it's so central to everybody's operation, and I have no idea what accidental features people might be depending on. Mike, on the basis of the above, and just as an experiment, you might try the same test but with a query string as part of your URL. If you're rewriting incoming URLs, that should cause request_uri to be initialized, and confirm what I've written above, or at least part of it. In that case, I think it'd be safe enough to write request_uri in the non-query-string case of filter_in. Massimo On Aug 16, 1:54 pm, mwolfe02 michael.joseph.wo...@gmail.com wrote: Just tested this in my production environment (apache) and it works as expected. I'm having the problem in my dev environment which is running the built-in rocket server on windows. It looks like the Rocket server is not providing request_uri as apache with mod_wsgi does. If you just output the contents of request.env in the two different environments you will see the differences. According to the WSGI PEP (http://www.python.org/dev/peps/pep-0333/#environ-variables), request_uri is not a required environment variable. I don't think the 'requested_uri' functionality for routes_onerror is mentioned in the documentation anywhere, but if and when it is a caveat about its limitations with respect to different web servers should be mentioned. Unfortunately, I spent several hours trying to figure out what I was doing wrong before realizing it was not something I had any control over. I hope I can save someone else some trouble in the future. -Mike On Aug 16, 12:52 pm, mwolfe02 michael.joseph.wo...@gmail.com wrote: I'm trying to use routes_onerror, but requested_uri keeps coming through as None (subject of this e-mail is the query string from the redirect). I searched through the source code and could not find where this is supposed to be set. I stepped through using WinPDB and there is simply no entry in request.env for request_uri. There is an entry for path_info which appears to be the re-written uri (ie, post routes.py processing). I'm running from trunk. Please let me know what other info is needed to troubleshoot. Thanks, Mike
Re: [web2py] Re: ?code=400ticket=Nonerequested_uri=None
On Aug 16, 2010, at 12:41 PM, Michael Wolfe wrote: I'm not sure anything can be done about it from within web2py (though Jonathan could certainly answer that better than I). It appears to be a limitation of the rocket web server. It may need to be taken up with the development team working on Rocket. Of course, if it's a problem with Rocket it could be a problem with other web servers. It seems like the safest thing would be to use one of the environment variables required by the PEP. PATH_INFO almost works, but it is returning the URL _after_ it has been rewritten by routes.py. Maybe we can intercept and set the variable before the url is rewritten? Maybe we would only do that if the web server did not define a request_uri environment variable? Under certain circumstances, we write WEB2PY_ORIGINAL_URI into the environment. But not always. And in error handling, it's not obvious (to me) that we necessarily want the original (vs rewritten) URL. Maybe the query string (as in the current subject line) ought to include them both.
Re: [web2py] Logging in web2py
On Aug 16, 2010, at 1:31 PM, Yarin wrote: I've posted an updated slice with an in-depth treatment of logging in web2py. http://web2pyslices.com/main/slices/take_slice/91 I've got a patch pending that adds a (standard format) logging configuration file, and converts all of web2py's logging calls to use it. It has a logger named web2py.app, so you can getLogger(web2py.app.myapp) and log calls will propagate to that logger (or you can create a new web2py.app.myapp logger definition in the config file). Initialization happens when web2py loads, so you don't have to test for configuration. Based on our earlier discussion http://groups.google.com/group/web2py/browse_thread/thread/d3b534113a904cd7/4d3f9f10d402210d?lnk=gstq=logging#4d3f9f10d402210d The slice addresses the issues of application-level logging, once-only configuration, and simpler syntax. It offers an approach that fully leverages Python's native logging capabilities, allows for full flexibility in configuration, and doesn't interfere with existing logging implementations. Some notes: -I decided against creating a separate contrib module for the code as it would impede flexibility and obscure the simplicity of solution. If included in the framework, I think it should be as model code. -I abandoned the cache approach to creating singleton loggers, and just inspect handlers instead. (Logger caching was causing problems on GAE) Let me know what you think. Thanks Massimo, Iceberg...
Re: [web2py] Re: Logging in web2py
On Aug 16, 2010, at 5:25 PM, Yarin wrote: @Jonathan- I'd like to see what you've got- keep me updated The patch is here, if anyone would like to review it. http://web.me.com/jlundell/filechute/logging.zip Most of the changes are to use a specific logger instead of making a generic logging call. The only interesting code is in main.py, along with logging.example.conf. Nothing much to it: just loading a configuration file, and using specific loggers. On Aug 16, 5:03 pm, Jonathan Lundell jlund...@pobox.com wrote: On Aug 16, 2010, at 1:31 PM, Yarin wrote: I've posted an updated slice with an in-depth treatment of logging in web2py. http://web2pyslices.com/main/slices/take_slice/91 I've got a patch pending that adds a (standard format) logging configuration file, and converts all of web2py's logging calls to use it. It has a logger named web2py.app, so you can getLogger(web2py.app.myapp) and log calls will propagate to that logger (or you can create a new web2py.app.myapp logger definition in the config file). Initialization happens when web2py loads, so you don't have to test for configuration. Based on our earlier discussion http://groups.google.com/group/web2py/browse_thread/thread/d3b534113a... The slice addresses the issues of application-level logging, once-only configuration, and simpler syntax. It offers an approach that fully leverages Python's native logging capabilities, allows for full flexibility in configuration, and doesn't interfere with existing logging implementations. Some notes: -I decided against creating a separate contrib module for the code as it would impede flexibility and obscure the simplicity of solution. If included in the framework, I think it should be as model code. -I abandoned the cache approach to creating singleton loggers, and just inspect handlers instead. (Logger caching was causing problems on GAE) Let me know what you think. Thanks Massimo, Iceberg...
Re: [web2py] Re: PAPERBACK web2py book
On Aug 17, 2010, at 4:49 AM, mdipierro wrote: As I explained in a previous thread there seem to be some legal problems. Lulu is the publisher and they offer amazon or ibookstore. They do not seem to allow both and I cannot redistribute the book without them using the same isbn. yet ibookstore requires an isbn. Let me keep looking if you have any advice let me know. While I prefer iBooks for reading, if I were publishing and had to make that choice, I'd go with Amazon, given that Kindle readers are fairly widespread. Presumably that will change, but for now Massimo On Aug 17, 6:22 am, David Mitchell monch1...@gmail.com wrote: Hi Massimo, Any chance of an EPUB format? I've promised myself I won't buy any more technical books in dead-tree format, and PDFs are just too hard to read on my ebook reader. Regards David Mitchell On Aug 13, 5:36 pm, mdipierro mdipie...@cs.depaul.edu wrote: The 3rd edition of the book is finally available in paperback. http://www.lulu.com/product/paperback/web2py-(3rd-edition)/12199578 The 3rd edition has 40% more text and it is 30% cheaper than the 2nd edition. Massimo
Re: [web2py] Re: Logging in web2py
On Aug 17, 2010, at 9:16 PM, Yarin wrote: @Jonathan- Like your use of config files for default configurations, but beyond that I'm not clear on what your patch is meant to solve- What's your reason for introducing named loggers and sub-loggers at the site level? Couldn't the root logger just handle all this? The main purpose of the configuration file is to allow a site to configure the logging destination. The current logging all goes to stdout (which in the case of Fedora or RHEL, at least, is /dev/null for system-started daemons). The configuration file lets us log to files, rotating files, or syslog, local or remote. The reason for named loggers is twofold. One is trivial: to identify the source of log messages. The other is not so trivial, but mainly for development: to let us have different log levels for different loggers. If I'm debugging URL rewrite, I can lower the level of the web2py.rewrite logger to (say) debug, without enabling all the other debug/info loggers in the system. Note that having named loggers is a pretty trivial addition, once we have a config file; it's not central, but it's a distinct convenience. Note also that we already have named loggers (eg Rocket and its kin), but no way to configure them. My intention was to use named loggers to allow for logging control at the application level, but for site level logging I see no reason to stray from root. @Iceberg- Yeah, I played around with abstracting get_configured_logger() into a contrib module, but it got messy because a) The request object we use to get application name and log file path isn't available outside the mvc classes, and would have to be passed in. b) More importantly, it killed the flexibility of being able to do custom configuration, use custom or multiple handlers, etc. The GAEHandler and others I include in the link are useful, but they're not web2py specific so Massimo might not want to clutter the framework with them. On Aug 17, 2:32 pm, Iceberg iceb...@21cn.com wrote: Nice work, Yarin! You use a more decent way, logging.getLogger(request.application), to achive app-wide logger. It is better than my cache.ram() trick. Oh, I was so blind, man. :-) And I wish GAEHandler and get_configured_logger() could go to web2py trunk, and the scaffold's models/model.py contains just one extra line: logger = get_configured_logger(folder = 'static', maxBytes=1024, backupCount=5) so it will be even easier to use. Best regards, Iceberg, 2010-Aug-18, 02:20(AM), Wed --- Original Message --- From:Yarin ykess...@gmail.com To: web2py-users web2py@googlegroups.com Cc: iceb...@21cn.com Date:Mon, 16 Aug 2010 13:31:16 -0700 (PDT) Subject: Logging in web2py --- I've posted an updated slice with an in-depth treatment of logging in web2py. http://web2pyslices.com/main/slices/take_slice/91 Based on our earlier discussion http://groups.google.com/group/web2py/browse_thread/thread/d3b534113a... The slice addresses the issues of application-level logging, once-only configuration, and simpler syntax. It offers an approach that fully leverages Python's native logging capabilities, allows for full flexibility in configuration, and doesn't interfere with existing logging implementations. Some notes: -I decided against creating a separate contrib module for the code as it would impede flexibility and obscure the simplicity of solution. If included in the framework, I think it should be as model code. -I abandoned the cache approach to creating singleton loggers, and just inspect handlers instead. (Logger caching was causing problems on GAE) Let me know what you think. Thanks Massimo, Iceberg...
Re: [web2py] Re: Logging in web2py
On Aug 18, 2010, at 6:36 AM, Yarin wrote: The reason for named loggers is twofold. One is trivial: to identify the source of log messages. Logging already tracks module, function, lineno... we can even insert a stack trace The other ... to let us have different log levels for different loggers. But it looks like you're introducing several named loggers only to log several messages at startup. For example, your web2py.sql logger's only function is to log a single driver exception. As far as I can tell, only the web2py.rewrite logger does any continuous work. And I assume the Rocket logger does a lot of work. But beyond that I can't see the justification in forcing the framework to create and configure a number of different loggers that will only be used once, if at all? Mostly for purposes of example. The web2py.foo loggers needn't appear in the config file; they can all use the web2py logger (I keep writing 'blogger'). The Rocket module is (I think) independent of web2py; if not, I'd have changed its logger (or asked Tim to do so) to something like 'web2py.rocket.*' (maybe we could pass a logger base name in to it). Identifying the source of the log messages through the logger name is useful for more than human consumption. log levels is one use; another is choice of handler. I'd have no objection to consolidating the startup-only loggers into 'web2py.startup', though I think that a noisy logger (even if noisy only at startup) ought to have its own name.
Re: [web2py] Re: Bug in URL(..,..,vars={})
On Aug 20, 2010, at 8:29 AM, mdipierro wrote:
I changed and changed back. I am waiting for comments form other
developers. I am not sure what the best course of action is.
What's an example of the framework breakage using the helper form of URL?
Massimo
On Aug 20, 8:23 am, Phyo Arkar phyo.arkarl...@gmail.com wrote:
So the way it worked before was a bug and now (rev 853) it is a feature?
On 8/20/10, mdipierro mdipie...@cs.depaul.edu wrote:
On a second thought we should leave it alone. Else we break the
scaffolding app, i.e. 90% os apps out there.. Anyway,... I'd still
like to hear your opinions.
On Aug 19, 8:38 pm, mdipierro mdipie...@cs.depaul.edu wrote:
There is a problem (in 1.83.2 and earlier version)
URL()
returns a string, not a helper. If you want to include the string
without escaping twice you have to do
{{=XML(URL(...))}}
In trunk, I modified the definition of URL so that it returns the
XML(..) helper wrapping the string. This will prevent
{{=URL(...)}}
from double escaping but will prevent you from doing
'http://127.0.0.:8000'+URL(..)
you would have to do
'http://127.0.0.:8000%s'%URL(..)
we cannot have it both ways.
URL is either a helper or a string... The more I think about it the
more I am convinced URL should be a helper not a string otherwise all
the examples in which we use {{=URL(...)}} are buggy. Therefore this
change should be considered a bug fix and not a breaking of backward
compatibility.
What do other people think? Should we change this as in trunk or leave
it alone?
Massimo
On Aug 19, 1:39 pm, Phyo Arkar phyo.arkarl...@gmail.com wrote:
Trunk version Rev 853
in view:
$(#list).jqGrid({
url:{{=URL(r=request,f='listMIME.json',vars={'extracted_path':extracted_path,'source_path':source_path})}},
HTML Result:
$(#list).jqGrid({
url:/sExtract/extraction/listMIME.json?source_path=home*amp*
;extracted_path=target,
it insert amp;instead of
that screwed up all my sites :D ..
have not tested with released version ..
Re: [web2py] Re: Bug in URL(..,..,vars={})
On Aug 19, 2010, at 6:38 PM, mdipierro wrote:
In trunk, I modified the definition of URL so that it returns the
XML(..) helper wrapping the string. This will prevent
{{=URL(...)}}
from double escaping but will prevent you from doing
'http://127.0.0.:8000'+URL(..)
Couldn't this be fixed by implementing URL.__radd__() ?
def __add__(self,other):
return '%s%s' % (self,other)
def __radd__(self,other):
return '%s%s' % (other,self)
you would have to do
'http://127.0.0.:8000%s' % URL(..)
we cannot have it both ways.
Re: [web2py] Re: Bug in URL(..,..,vars={})
On Aug 20, 2010, at 9:22 AM, Jonathan Lundell wrote:
On Aug 19, 2010, at 6:38 PM, mdipierro wrote:
In trunk, I modified the definition of URL so that it returns the
XML(..) helper wrapping the string. This will prevent
{{=URL(...)}}
from double escaping but will prevent you from doing
'http://127.0.0.:8000'+URL(..)
Couldn't this be fixed by implementing URL.__radd__() ?
I meant XML.__radd__()
def __add__(self,other):
return '%s%s' % (self,other)
def __radd__(self,other):
return '%s%s' % (other,self)
you would have to do
'http://127.0.0.:8000%s' % URL(..)
we cannot have it both ways.
Re: [web2py] Re: Bug in URL(..,..,vars={})
On Aug 20, 2010, at 10:17 AM, mdipierro wrote:
I did as you suggest. I also had to add lower(), upper() and __len__
methods to the XML class.
I think this is a good solution. Thanks Jonathan.
oops:
+def upper(self):
+return str(self).lower()
Massimo
On Aug 20, 11:31 am, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 20, 2010, at 9:22 AM, Jonathan Lundell wrote:
On Aug 19, 2010, at 6:38 PM, mdipierro wrote:
In trunk, I modified the definition of URL so that it returns the
XML(..) helper wrapping the string. This will prevent
{{=URL(...)}}
from double escaping but will prevent you from doing
'http://127.0.0.:8000'+URL(..)
Couldn't this be fixed by implementing URL.__radd__() ?
I meant XML.__radd__()
def __add__(self,other):
return '%s%s' % (self,other)
def __radd__(self,other):
return '%s%s' % (other,self)
you would have to do
'http://127.0.0.:8000%s'% URL(..)
we cannot have it both ways.
Re: [web2py] Re: Should we have a feature freeze and stability maintenance period in future?
On Aug 20, 2010, at 2:40 PM, mdipierro wrote: I see a lot of value in - bug-squishing-contest , - Stress test, Test everything , try to crash web2py etc. - fix bugs, fix performance issues , improve performance - code cleanup , documentation. we can set deadlines for that. This means we would stress test and improve features existing at a certain date and we would only add new features tagged as experimental that do not interfere with parts that are being stress tested. Makes sense? I think so, as long as the terms are clear. In particular, it should be clear how the experimental features would move to stable (perhaps just a time limit on the stress test, or some more specific condition). Perhaps in going through an exercise like this, we could also think about something like it could be incorporated into the normal development cycle, on an ongoing basis rather than as a one-shot project.
Re: [web2py] Re: TypeError: 'XML' object is unsubscriptable
On Aug 21, 2010, at 10:02 PM, mdipierro wrote: Did you try 819? I think XML needs this: def __getitem__(self, key): return str(self).__getitem__(key) Massimo On Aug 21, 11:01 pm, mr.freeze nat...@freezable.com wrote: I get the same thing on a newly created app when running from trunk: Traceback (most recent call last): File C:\web2py\gluon\restricted.py, line 188, in restricted exec ccode in environment File C:\web2py\applications\crashtest/views\default/index.html, line 27, in module TypeError: 'XML' object is unsubscriptable On Aug 21, 9:58 pm, mdipierro mdipie...@cs.depaul.edu wrote: Which version. If not web2py stable, please check latest trunk. On Aug 21, 8:33 pm, Jose jjac...@gmail.com wrote: Hi, (I can not find the thread where it was this error) From admin create a new application: apptest go to:http://127.0.0.1:8000/apptest produces the following error: TypeError: 'XML' object is unsubscriptable if type(code) == types.CodeType: ccode = code else: ccode = compile2(code,layer) exec ccode in environment except HTTP: raise except Exception: # XXX Show exception in Wing IDE if running in debugger
Re: [web2py] Re: TypeError: 'XML' object is unsubscriptable
On Aug 21, 2010, at 10:41 PM, mdipierro wrote: I had it, together with __getslice__, I thought I pushed in 819 but perhaps I did not. Pushing again. That's better. On Aug 22, 12:17 am, Jonathan Lundell jlund...@pobox.com wrote: On Aug 21, 2010, at 10:02 PM, mdipierro wrote: Did you try 819? I think XML needs this: def __getitem__(self, key): return str(self).__getitem__(key) Massimo On Aug 21, 11:01 pm, mr.freeze nat...@freezable.com wrote: I get the same thing on a newly created app when running from trunk: Traceback (most recent call last): File C:\web2py\gluon\restricted.py, line 188, in restricted exec ccode in environment File C:\web2py\applications\crashtest/views\default/index.html, line 27, in module TypeError: 'XML' object is unsubscriptable On Aug 21, 9:58 pm, mdipierro mdipie...@cs.depaul.edu wrote: Which version. If not web2py stable, please check latest trunk. On Aug 21, 8:33 pm, Jose jjac...@gmail.com wrote: Hi, (I can not find the thread where it was this error) From admin create a new application: apptest go to:http://127.0.0.1:8000/apptest produces the following error: TypeError: 'XML' object is unsubscriptable if type(code) == types.CodeType: ccode = code else: ccode = compile2(code,layer) exec ccode in environment except HTTP: raise except Exception: # XXX Show exception in Wing IDE if running in debugger
Re: [web2py] Re: Should we have a feature freeze and stability maintenance period in future?
On Aug 24, 2010, at 11:29 AM, mart wrote: Well, this is all interesting :) After reading Michele's email, I just had to spend hours looking at Mercurial ( the like) as deeply as the day would let me (I'm on PTO, so I can do this). I thought I had a good idea about distributed version control system but, as it turns out, a few surprises were there waiting for me. I have always been a believer in development against the known reference version (now, that's me being old fashioned). So, yes... Mercurial must be a painful thing. Based on what I have read, mercurial seems a little too chaotic for my taste. I do believe there is one way to manage a source repository (did I really just say that?) wrt to input/output. Well, honestly, I am way too big of a exactly what is going to any of the products I build junky, to even accept some of the basic premises of Mercurial. Like any workingDirectory is a potential branch? The O'Reilly Hg book Mercurial: The Definitive Guide is a really good reference here, not just for day-to-day help, but for real insight into the advantages and strategies of distributed version control. Highly recommended.
Re: [web2py] dashes in controller names?
On Aug 24, 2010, at 6:47 PM, Cory Coager wrote: How do you use dashes in controller names? I get errors, Invalid request. You can't. The URL parser allows only \w+ (alphanumeric or underscore).
Re: [web2py] Re: dashes in controller names?
On Aug 24, 2010, at 8:03 PM, Kevin wrote:
As far as I can find, you have to set up web2py routes to do this --
it's pretty easy to do redirection using routes (using routes_in), but
I'm not aware of any way off-hand to do the equivalent of rewriting/
translation the URL for inbound requests (routes_out does to
translation).
I think it would be difficult, unless you explicitly translated known
controller names. Which you could do...
We could translate incoming hyphens to underscores, I suppose, without too much
effort, but we wouldn't know whether to reverse the translation on the outbound
side.
Come the revolution (new rewrite logic), all things will be possible. But no
promises on a schedule.
We *could* simply allow hyphens in controller names, without translation. We
can't do that with functions, since they need to function as Python
identifiers.
This is something I (and doubtless) many others need as well, and it
really should be a (default enabled) option for web2py to
automatically treat URL-embedded hyphens as though they were
underscores for matching app, controller and function names, and an
optionally enablable setting to make all generated routes use hyphens
instead of underscores for apps, controllers, and functions.
The open-and-shut arguments for using hyphens boil down to (in order
of severity):
* Underscores are obscured when URLs are underlined
* Hyphens are easier to type than underscores – no need for the shift
key
* Hyphens are easier to read
* Underscores are a typographical construction invented primarily for
underlining characters (key backspace underscore on a
typewriter), and today is used primarily in programming and other
technical contexts. In short, there is little justification for
exposing back-end considerations to the world.
Many references can be found on this subject:
*
http://pylonsbook.com/en/1.1/urls-routing-and-dispatch.html#choosing-good-urls
* http://semicolons.org/post/256699383/friendly-urls
*
http://www.wordsellinc.com/blog/content-optimization/use-hyphens-not-underscores-in-urls/
This is one of the sore points I have with anything I put on the web
-- to provide what I feel is a correct URL scheme to the user, I'll
bypass or override half of the backend code if necessary, which is
suboptimal.
Web2py is very close to making it easy to provide URLs that'll
withstand the scrutiny of bloggers -- besides the hyphen bit, all I
really see as being needed is:
* The aforementioned settings for hyphenization.
* in routes_in and routes_out add the ability to supply a function/
lambda in the second index of any inner two-tuple, which would receive
a match object and return a string. For example, the following could
be a stop-gap way to perform this kind of underscore-to-hyphen
conversion:
routes_out = ( (r'/([^/]*)/([^/]*)/([^/]*)(/?Pany.*)?',
lambda match: '/'.join(part.replace('_', '-') for part in
match.groups()) + match.group('any')), )
On Aug 24, 7:47 pm, Cory Coager ccoa...@gmail.com wrote:
How do you use dashes in controller names? I get errors, Invalid
request.
Re: [web2py] Re: dashes in controller names?
On Aug 25, 2010, at 1:42 AM, Kevin wrote:
It would be safe for the routing code to treat hyphens as underscores
when matching for app, controller, or function, though -- since web2py
(and python) semantics require that the code itself uses underscores.
Of course underscores in the rest of the URL would have to pass
through untouched.
Are Python restrictions relevant for applications and controllers? Or just
filename restrictions? I'm not sure.
Reverse translation is not an issue, since I could just do something
like:
{{= DASH(URL(...)) }}
Where DASH replaces all underscores with hyphens (which is acceptable
for me, since I don't use any underscores in URLs), though it would be
handy to make the routes.py configurable to do that automatically in
the URL call itself if I wanted.
It could be an option to URL, I suppose. I'm thinking that you wouldn't want to
translate underscores in the query string.
On Aug 24, 9:46 pm, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 24, 2010, at 8:03 PM, Kevin wrote:
As far as I can find, you have to set up web2py routes to do this --
it's pretty easy to do redirection using routes (using routes_in), but
I'm not aware of any way off-hand to do the equivalent of rewriting/
translation the URL for inbound requests (routes_out does to
translation).
I think it would be difficult, unless you explicitly translated known
controller names. Which you could do...
We could translate incoming hyphens to underscores, I suppose, without too
much effort, but we wouldn't know whether to reverse the translation on the
outbound side.
Come the revolution (new rewrite logic), all things will be possible. But no
promises on a schedule.
We *could* simply allow hyphens in controller names, without translation. We
can't do that with functions, since they need to function as Python
identifiers.
This is something I (and doubtless) many others need as well, and it
really should be a (default enabled) option for web2py to
automatically treat URL-embedded hyphens as though they were
underscores for matching app, controller and function names, and an
optionally enablable setting to make all generated routes use hyphens
instead of underscores for apps, controllers, and functions.
The open-and-shut arguments for using hyphens boil down to (in order
of severity):
* Underscores are obscured when URLs are underlined
* Hyphens are easier to type than underscores – no need for the shift
key
* Hyphens are easier to read
* Underscores are a typographical construction invented primarily for
underlining characters (key backspace underscore on a
typewriter), and today is used primarily in programming and other
technical contexts. In short, there is little justification for
exposing back-end considerations to the world.
Many references can be found on this subject:
*http://pylonsbook.com/en/1.1/urls-routing-and-dispatch.html#choosing-...
*http://semicolons.org/post/256699383/friendly-urls
*http://www.wordsellinc.com/blog/content-optimization/use-hyphens-not-...
This is one of the sore points I have with anything I put on the web
-- to provide what I feel is a correct URL scheme to the user, I'll
bypass or override half of the backend code if necessary, which is
suboptimal.
Web2py is very close to making it easy to provide URLs that'll
withstand the scrutiny of bloggers -- besides the hyphen bit, all I
really see as being needed is:
* The aforementioned settings for hyphenization.
* in routes_in and routes_out add the ability to supply a function/
lambda in the second index of any inner two-tuple, which would receive
a match object and return a string. For example, the following could
be a stop-gap way to perform this kind of underscore-to-hyphen
conversion:
routes_out = ( (r'/([^/]*)/([^/]*)/([^/]*)(/?Pany.*)?',
lambda match: '/'.join(part.replace('_', '-') for part in
match.groups()) + match.group('any')), )
On Aug 24, 7:47 pm, Cory Coager ccoa...@gmail.com wrote:
How do you use dashes in controller names? I get errors, Invalid
request.
Re: [web2py] Re: IS_IMAGE() broke?
On Aug 25, 2010, at 8:11 AM, Joe Wakefield wrote:
The assertion errors are blank, so I added some extra output:
-IS_IMAGE STARTED-
filename: 1.png
extension: png
self.extensions: jpeg
ERROR IN IS_IMAGE:
-IS_IMAGE STARTED-
filename: llamas.jpg
extension: jpg
self.extensions: jpeg
__jpeg called
width, height: 400, 300
new: users.photo.81f708977b7f7851.6c6c616d61732e6a7067.jpgold:
-IS_IMAGE STARTED-
filename: 1.png
extension: png
self.extensions: jpeg
ERROR IN IS_IMAGE:
new: users.photo.8ef32f83074cd5bf.312e706e67.pngold: users.photo.
81f708977b7f7851.6c6c616d61732e6a7067.jpg
-IS_IMAGE STARTED-
filename: test.zip
extension: zip
self.extensions: jpeg
ERROR IN IS_IMAGE:
new: users.photo.a4b74cd11efaf711.746573742e7a6970.zipold:
users.photo.8ef32f83074cd5bf.312e706e67.png
This would suggest that the IS_IMAGE validator is working fine, and
I'm guessing the problem would be in sqlform.accepts()?
My desk-check of IS_IMAGE suggested that it was OK, too. It could use a couple
of documenting doctests, I think.
You're deliberately restricting it to jpeg above, right? Hence the rejection of
png?
On Aug 25, 10:17 am, mdipierro mdipie...@cs.depaul.edu wrote:
can you help me debug?
in gluon/validators.py
classIS_IMAGE:
def __call__(self,value):
except:
return (value, self.error_message)
replace last two lines with
except Exception, e:
return (value, str(r))
what does the error say?
On Aug 23, 4:06 pm, mdipierro mdipie...@cs.depaul.edu wrote:
Something is wrong here
IS_IMAGE(extensions=('jpeg'))
should be
IS_IMAGE(extensions=('jpeg',))
A tuple of one element must contain a comma. Not sure if that may be
the cause of your problem.
On Aug 23, 3:16 pm, Joe Wakefield coffeeburr...@gmail.com wrote:
I was also experiencing this issue, but found that rebuilding the
database only worked until the first valid upload, after which it is
broken again.
In my case, I am using:
Field('photo', 'upload', uploadfield='photo_data',
requires=IS_IMAGE(extensions=('jpeg'))),
Field('photo_data', 'blob', requires=IS_IMAGE(extensions=('jpeg'))),
And was using SQLFORM to display the update form.
When I first added this, it was accepting absolutely everything (pdf,
odt, zip) as an upload. I deleted my database entirely, and had it
rebuilt. I first noticed that it started rejecting non-image uploads.
However, once I had uploaded an image, all subsequent uploads of non-
image types were allowed again. A second database build showed the
exact same behaviour; proper rejection until the first valid upload.
I then made a backup of my web2py folder, and extracted today's
nightly build over my folder. Rebuilding my database one last time, it
shows the exact same behaviour.
On Jul 11, 2:56 pm, Rob r...@rmdashr.com wrote:
I just recently added:
db.Item.image.requires =IS_IMAGE()
The records that existed prior to adding this line does not obey
theIS_IMAGE() (ie: they still allow me to upload a PDF). All new records
created DO work - they force me to select an image or else they show
an error.
steps to reproduce (untested)
1) create DB
db.define_table('Item',
Field('description'),
Field('need', 'boolean'),
Field('image', 'upload'))
2) add rows to the table
3) add rules:
db.Item.category.requires = IS_IN_DB(db, db.Category.id,'%(name)s')
db.Item.description.requires = IS_NOT_EMPTY()
db.Item.image.requires =IS_IMAGE()
4) go back to the rows you added to the Item table and add non-image
files - notice it works
Does that help?
On Jul 11, 11:42 am, mdipierro mdipie...@cs.depaul.edu wrote:
Please tell us more. When do you get an error? What is the error?
On 11 Lug, 11:57, Rob r...@rmdashr.com wrote:
This issue only happens for records that were created before I added
the .requires fields. Error handling on new records works as
expected... so... is it still a bug?
On Jul 11, 9:15 am, Rob r...@rmdashr.com wrote:
db.define_table('Category',
Field('name'))
db.define_table('Item',
Field('category', db.Category),
Field('description'),
Field('need', 'boolean'),
Field('image', 'upload'))
db.Item.category.requires = IS_IN_DB(db, db.Category.id)
db.Item.description.requires = IS_NOT_EMPTY()
db.Item.image.requires =IS_IMAGE()
def details():
item = request.args(0)
form = crud.update(db.Item, item, next=URL(r=request, args=item))
return dict(form=form)
It allows me to upload PDFs and flashes 'record updated'
Re: [web2py] Re: IS_IMAGE() broke?
On Aug 25, 2010, at 8:33 AM, Joe Wakefield wrote:
I'll do you one better. The problem is in sqlhtml.py at lines
909-917. It changes ret=False to ret=True if the field already has a
value.
Good catch. This is a case where a comment documenting the intent of this hunk
of code would come in handy for those of us reading it. The error is cancelled
under certain conditions, but what's the intent?
===
ret, errors: False, Storage {'photo': 'invalid image'}
new ret:False
===
ret, errors: True, Storage {}
new ret:True
new: users.photo.8a5327ab32003765.6c6c616d61732e6a7067.jpgold:
===
ret, errors: False, Storage {'photo': 'invalid image'}
new ret:True
new: users.photo.b6369c7a23ae4813.312e706e67.pngold: users.photo.
8a5327ab32003765.6c6c616d61732e6a7067.jpg
On Aug 25, 10:17 am, mdipierro mdipie...@cs.depaul.edu wrote:
can you help me debug?
in gluon/validators.py
classIS_IMAGE:
def __call__(self,value):
except:
return (value, self.error_message)
replace last two lines with
except Exception, e:
return (value, str(r))
what does the error say?
On Aug 23, 4:06 pm, mdipierro mdipie...@cs.depaul.edu wrote:
Something is wrong here
IS_IMAGE(extensions=('jpeg'))
should be
IS_IMAGE(extensions=('jpeg',))
A tuple of one element must contain a comma. Not sure if that may be
the cause of your problem.
On Aug 23, 3:16 pm, Joe Wakefield coffeeburr...@gmail.com wrote:
I was also experiencing this issue, but found that rebuilding the
database only worked until the first valid upload, after which it is
broken again.
In my case, I am using:
Field('photo', 'upload', uploadfield='photo_data',
requires=IS_IMAGE(extensions=('jpeg'))),
Field('photo_data', 'blob', requires=IS_IMAGE(extensions=('jpeg'))),
And was using SQLFORM to display the update form.
When I first added this, it was accepting absolutely everything (pdf,
odt, zip) as an upload. I deleted my database entirely, and had it
rebuilt. I first noticed that it started rejecting non-image uploads.
However, once I had uploaded an image, all subsequent uploads of non-
image types were allowed again. A second database build showed the
exact same behaviour; proper rejection until the first valid upload.
I then made a backup of my web2py folder, and extracted today's
nightly build over my folder. Rebuilding my database one last time, it
shows the exact same behaviour.
On Jul 11, 2:56 pm, Rob r...@rmdashr.com wrote:
I just recently added:
db.Item.image.requires =IS_IMAGE()
The records that existed prior to adding this line does not obey
theIS_IMAGE() (ie: they still allow me to upload a PDF). All new records
created DO work - they force me to select an image or else they show
an error.
steps to reproduce (untested)
1) create DB
db.define_table('Item',
Field('description'),
Field('need', 'boolean'),
Field('image', 'upload'))
2) add rows to the table
3) add rules:
db.Item.category.requires = IS_IN_DB(db, db.Category.id,'%(name)s')
db.Item.description.requires = IS_NOT_EMPTY()
db.Item.image.requires =IS_IMAGE()
4) go back to the rows you added to the Item table and add non-image
files - notice it works
Does that help?
On Jul 11, 11:42 am, mdipierro mdipie...@cs.depaul.edu wrote:
Please tell us more. When do you get an error? What is the error?
On 11 Lug, 11:57, Rob r...@rmdashr.com wrote:
This issue only happens for records that were created before I added
the .requires fields. Error handling on new records works as
expected... so... is it still a bug?
On Jul 11, 9:15 am, Rob r...@rmdashr.com wrote:
db.define_table('Category',
Field('name'))
db.define_table('Item',
Field('category', db.Category),
Field('description'),
Field('need', 'boolean'),
Field('image', 'upload'))
db.Item.category.requires = IS_IN_DB(db, db.Category.id)
db.Item.description.requires = IS_NOT_EMPTY()
db.Item.image.requires =IS_IMAGE()
def details():
item = request.args(0)
form = crud.update(db.Item, item, next=URL(r=request, args=item))
return dict(form=form)
It allows me to upload PDFs and flashes 'record updated'
Re: [web2py] Re: Web2py and threads
On Aug 25, 2010, at 9:00 AM, Phyo Arkar wrote: Did I Read that reading files inside controller will block web2py , Does it? Thats a bad news.. i am doing a file crawler and while crawling , web2py is blocked even tho the process talke only 25% of 1 out of 4 CPUs .. This stuff gets a little coverage in the book's deployment chapter, but it could use a systematic discussion. What are the implication for web2py apps of http server policies, database locks (sqlite especially), session locking, the GIL, etc? With a section on best practices. On 8/25/10, pierreth pierre.thibau...@gmail.com wrote: I would appreciate a good reference to understand the concepts you are talking about. It is something new to me and I don't understand. On 25 août, 11:22, John Heenan johnmhee...@gmail.com wrote: No, nothing that abstract. Using WSGI forces a new thread for each request. This is is a simple and inefficient brute force approach that really only suits the simplest Python applications and where only a small number of concurrent connection might be expected. Any application that provides web services is going to OS block on file reading (and writing) and on database access. Using threads is a classic and easy way out that carries a lot of baggage. Windows has had a way out of this for years with its asynch (or event) notification set up through an OVERLAPPED structure. Lightttpd makes use of efficient event notification schemes like kqueue and epoll. Apache only uses such schemes for listening and Keep- Alives. No matter how careful one is with threads and processes there always appears to be unexpected gotchas. Python has a notorious example, the now fixed 'Beazly Effect' that affected the GIL. Also I don't think there is a single experienced Python user that trusts the GIL. John Heenan
Re: [web2py] Re: Web2py and threads
On Aug 25, 2010, at 1:41 PM, mdipierro wrote: call session._unlock() if you do not need session locking If you do that (without calling session.forget), what will happen in _try_store_on_disk when cPickle.dump(dict(self), response.session_file) is called with a None file argument? Or is cPickle.dump cool with that? Or am I misreading the logic? On Aug 25, 11:38 am, Phyo Arkar phyo.arkarl...@gmail.com wrote: Yes may be session was locked , thats why session.current=processing_path not working But then again , while processing files i try opening separate page , to other controller , it was waited till the first (file Crawler) page finished parsing. ok i will make a separate thread about this. On 8/25/10, mdipierro mdipie...@cs.depaul.edu wrote: On Aug 25, 11:00 am, Phyo Arkar phyo.arkarl...@gmail.com wrote: Did I Read that reading files inside controller will block web2py , Does it? No web2py does not block. web2py only locks sessions that means one user cannot request two concurrent pages because there would be a race condition in saving sessions. Two user can request different pages which open the same file unless the file is explicitly locked by your code. Thats a bad news.. i am doing a file crawler and while crawling , web2py is blocked even tho the process talke only 25% of 1 out of 4 CPUs .. Tell us more or I cannot help. On 8/25/10, pierreth pierre.thibau...@gmail.com wrote: I would appreciate a good reference to understand the concepts you are talking about. It is something new to me and I don't understand. On 25 août, 11:22, John Heenan johnmhee...@gmail.com wrote: No, nothing that abstract. Using WSGI forces a new thread for each request. This is is a simple and inefficient brute force approach that really only suits the simplest Python applications and where only a small number of concurrent connection might be expected. Any application that provides web services is going to OS block on file reading (and writing) and on database access. Using threads is a classic and easy way out that carries a lot of baggage. Windows has had a way out of this for years with its asynch (or event) notification set up through an OVERLAPPED structure. Lightttpd makes use of efficient event notification schemes like kqueue and epoll. Apache only uses such schemes for listening and Keep- Alives. No matter how careful one is with threads and processes there always appears to be unexpected gotchas. Python has a notorious example, the now fixed 'Beazly Effect' that affected the GIL. Also I don't think there is a single experienced Python user that trusts the GIL. John Heenan
Re: [web2py] Re: Web2py and threads
On Aug 25, 2010, at 6:37 PM, mdipierro wrote: The problem is only if have two http request from the same client in the same session Thanks for that; I was wondering under which conditions unlocking might be permissible (and I'm still not entirely clear, but never mind for now). My concern is this. Here's unlock: def _unlock(self, response): if response and response.session_file: try: portalocker.unlock(response.session_file) response.session_file.close() del response.session_file - except: ### this should never happen but happens in Windows pass Now we save the session file: def _try_store_on_disk(self, request, response): if response._dbtable_and_field \ or not response.session_id \ or self._forget: self._unlock(response) return if response.session_new: # Tests if the session folder exists, if not, create it session_folder = os.path.dirname(response.session_filename) response.session_file = open(response.session_filename, 'wb') portalocker.lock(response.session_file, portalocker.LOCK_EX) cPickle.dump(dict(self), response.session_file) self._unlock(response) But response.session_file is None at this point. A arrives loads session and unlocks B arrives loads session and unlocks A change session and saves it B changes session and saves it Nothing breaks but B never sees changes made by A and they are overwritten by B. With locks A arrives loads session B arrives and waits A change session and saves it B loads session (with changes made by A) B changes session and saves it On Aug 25, 3:52 pm, Jonathan Lundell jlund...@pobox.com wrote: On Aug 25, 2010, at 1:41 PM, mdipierro wrote: call session._unlock() if you do not need session locking If you do that (without calling session.forget), what will happen in _try_store_on_disk when cPickle.dump(dict(self), response.session_file) is called with a None file argument? Or is cPickle.dump cool with that? Or am I misreading the logic? On Aug 25, 11:38 am, Phyo Arkar phyo.arkarl...@gmail.com wrote: Yes may be session was locked , thats why session.current=processing_path not working But then again , while processing files i try opening separate page , to other controller , it was waited till the first (file Crawler) page finished parsing. ok i will make a separate thread about this. On 8/25/10, mdipierro mdipie...@cs.depaul.edu wrote: On Aug 25, 11:00 am, Phyo Arkar phyo.arkarl...@gmail.com wrote: Did I Read that reading files inside controller will block web2py , Does it? No web2py does not block. web2py only locks sessions that means one user cannot request two concurrent pages because there would be a race condition in saving sessions. Two user can request different pages which open the same file unless the file is explicitly locked by your code. Thats a bad news.. i am doing a file crawler and while crawling , web2py is blocked even tho the process talke only 25% of 1 out of 4 CPUs .. Tell us more or I cannot help. On 8/25/10, pierreth pierre.thibau...@gmail.com wrote: I would appreciate a good reference to understand the concepts you are talking about. It is something new to me and I don't understand. On 25 août, 11:22, John Heenan johnmhee...@gmail.com wrote: No, nothing that abstract. Using WSGI forces a new thread for each request. This is is a simple and inefficient brute force approach that really only suits the simplest Python applications and where only a small number of concurrent connection might be expected. Any application that provides web services is going to OS block on file reading (and writing) and on database access. Using threads is a classic and easy way out that carries a lot of baggage. Windows has had a way out of this for years with its asynch (or event) notification set up through an OVERLAPPED structure. Lightttpd makes use of efficient event notification schemes like kqueue and epoll. Apache only uses such schemes for listening and Keep- Alives. No matter how careful one is with threads and processes there always appears to be unexpected gotchas. Python has a notorious example, the now fixed 'Beazly Effect' that affected the GIL. Also I don't think there is a single experienced Python user that trusts the GIL. John Heenan
Re: [web2py] Re: Web2py and threads
On Aug 25, 2010, at 7:56 PM, mdipierro wrote: This is a bug. I fixed it in trunk. Thanks Jonathan. It's fixed in the sense that it won't raise an exception. But now how is calling _unlock different from calling forget? On Aug 25, 9:30 pm, Jonathan Lundell jlund...@pobox.com wrote: On Aug 25, 2010, at 6:37 PM, mdipierro wrote: The problem is only if have two http request from the same client in the same session Thanks for that; I was wondering under which conditions unlocking might be permissible (and I'm still not entirely clear, but never mind for now). My concern is this. Here's unlock: def _unlock(self, response): if response and response.session_file: try: portalocker.unlock(response.session_file) response.session_file.close() del response.session_file - except: ### this should never happen but happens in Windows pass Now we save the session file: def _try_store_on_disk(self, request, response): if response._dbtable_and_field \ or not response.session_id \ or self._forget: self._unlock(response) return if response.session_new: # Tests if the session folder exists, if not, create it session_folder = os.path.dirname(response.session_filename) response.session_file = open(response.session_filename, 'wb') portalocker.lock(response.session_file, portalocker.LOCK_EX) cPickle.dump(dict(self), response.session_file) self._unlock(response) But response.session_file is None at this point. A arrives loads session and unlocks B arrives loads session and unlocks A change session and saves it B changes session and saves it Nothing breaks but B never sees changes made by A and they are overwritten by B. With locks A arrives loads session B arrives and waits A change session and saves it B loads session (with changes made by A) B changes session and saves it On Aug 25, 3:52 pm, Jonathan Lundell jlund...@pobox.com wrote: On Aug 25, 2010, at 1:41 PM, mdipierro wrote: call session._unlock() if you do not need session locking If you do that (without calling session.forget), what will happen in _try_store_on_disk when cPickle.dump(dict(self), response.session_file) is called with a None file argument? Or is cPickle.dump cool with that? Or am I misreading the logic? On Aug 25, 11:38 am, Phyo Arkar phyo.arkarl...@gmail.com wrote: Yes may be session was locked , thats why session.current=processing_path not working But then again , while processing files i try opening separate page , to other controller , it was waited till the first (file Crawler) page finished parsing. ok i will make a separate thread about this. On 8/25/10, mdipierro mdipie...@cs.depaul.edu wrote: On Aug 25, 11:00 am, Phyo Arkar phyo.arkarl...@gmail.com wrote: Did I Read that reading files inside controller will block web2py , Does it? No web2py does not block. web2py only locks sessions that means one user cannot request two concurrent pages because there would be a race condition in saving sessions. Two user can request different pages which open the same file unless the file is explicitly locked by your code. Thats a bad news.. i am doing a file crawler and while crawling , web2py is blocked even tho the process talke only 25% of 1 out of 4 CPUs .. Tell us more or I cannot help. On 8/25/10, pierreth pierre.thibau...@gmail.com wrote: I would appreciate a good reference to understand the concepts you are talking about. It is something new to me and I don't understand. On 25 août, 11:22, John Heenan johnmhee...@gmail.com wrote: No, nothing that abstract. Using WSGI forces a new thread for each request. This is is a simple and inefficient brute force approach that really only suits the simplest Python applications and where only a small number of concurrent connection might be expected. Any application that provides web services is going to OS block on file reading (and writing) and on database access. Using threads is a classic and easy way out that carries a lot of baggage. Windows has had a way out of this for years with its asynch (or event) notification set up through an OVERLAPPED structure. Lightttpd makes use of efficient event notification schemes like kqueue and epoll. Apache only uses such schemes for listening and Keep- Alives. No matter how careful one is with threads and processes there always appears to be unexpected gotchas. Python has a notorious example, the now fixed 'Beazly Effect' that affected the GIL. Also I don't think there is a single experienced Python user that trusts the GIL. John Heenan
Re: [web2py] Re: IS_IMAGE() broke?
On Aug 26, 2010, at 6:56 AM, Joe Wakefield wrote:
Massimo,
The new code works great! Thanks for the quick fix.
And yes, the comment is helpful.
Regards,
Joe
On Aug 25, 10:51 pm, mdipierro mdipie...@cs.depaul.edu wrote:
Please check if this is fixed in trunk and if the comment I added
makes any sense.
Massimo
On Aug 25, 10:59 am, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 25, 2010, at 8:33 AM, Joe Wakefield wrote:
I'll do you one better. The problem is in sqlhtml.py at lines
909-917. It changes ret=False to ret=True if the field already has a
value.
Good catch. This is a case where a comment documenting the intent of this
hunk of code would come in handy for those of us reading it. The error is
cancelled under certain conditions, but what's the intent?
===
ret, errors: False, Storage {'photo': 'invalid image'}
new ret:False
===
ret, errors: True, Storage {}
new ret:True
new: users.photo.8a5327ab32003765.6c6c616d61732e6a7067.jpgold:
===
ret, errors: False, Storage {'photo': 'invalid image'}
new ret:True
new: users.photo.b6369c7a23ae4813.312e706e67.pngold: users.photo.
8a5327ab32003765.6c6c616d61732e6a7067.jpg
On Aug 25, 10:17 am, mdipierro mdipie...@cs.depaul.edu wrote:
can you help me debug?
in gluon/validators.py
classIS_IMAGE:
def __call__(self,value):
except:
return (value, self.error_message)
replace last two lines with
except Exception, e:
return (value, str(r))
what does the error say?
On Aug 23, 4:06 pm, mdipierro mdipie...@cs.depaul.edu wrote:
Something is wrong here
IS_IMAGE(extensions=('jpeg'))
should be
IS_IMAGE(extensions=('jpeg',))
A tuple of one element must contain a comma. Not sure if that may be
the cause of your problem.
On Aug 23, 3:16 pm, Joe Wakefield coffeeburr...@gmail.com wrote:
I was also experiencing this issue, but found that rebuilding the
database only worked until the first valid upload, after which it is
broken again.
In my case, I am using:
Field('photo', 'upload', uploadfield='photo_data',
requires=IS_IMAGE(extensions=('jpeg'))),
Field('photo_data', 'blob', requires=IS_IMAGE(extensions=('jpeg'))),
And was using SQLFORM to display the update form.
When I first added this, it was accepting absolutely everything (pdf,
odt, zip) as an upload. I deleted my database entirely, and had it
rebuilt. I first noticed that it started rejecting non-image uploads.
However, once I had uploaded an image, all subsequent uploads of non-
image types were allowed again. A second database build showed the
exact same behaviour; proper rejection until the first valid upload.
I then made a backup of my web2py folder, and extracted today's
nightly build over my folder. Rebuilding my database one last time, it
shows the exact same behaviour.
On Jul 11, 2:56 pm, Rob r...@rmdashr.com wrote:
I just recently added:
db.Item.image.requires =IS_IMAGE()
The records that existed prior to adding this line does not obey
theIS_IMAGE() (ie: they still allow me to upload a PDF). All new
records
created DO work - they force me to select an image or else they show
an error.
steps to reproduce (untested)
1) create DB
db.define_table('Item',
Field('description'),
Field('need', 'boolean'),
Field('image', 'upload'))
2) add rows to the table
3) add rules:
db.Item.category.requires = IS_IN_DB(db, db.Category.id,'%(name)s')
db.Item.description.requires = IS_NOT_EMPTY()
db.Item.image.requires =IS_IMAGE()
4) go back to the rows you added to the Item table and add non-image
files - notice it works
Does that help?
On Jul 11, 11:42 am, mdipierro mdipie...@cs.depaul.edu wrote:
Please tell us more. When do you get an error? What is the error?
On 11 Lug, 11:57, Rob r...@rmdashr.com wrote:
This issue only happens for records that were created before I added
the .requires fields. Error handling on new records works as
expected... so... is it still a bug?
On Jul 11, 9:15 am, Rob r...@rmdashr.com wrote:
db.define_table('Category',
Field('name'))
db.define_table('Item',
Field('category', db.Category),
Field('description'),
Field('need', 'boolean'),
Field('image', 'upload'))
db.Item.category.requires = IS_IN_DB(db, db.Category.id)
db.Item.description.requires = IS_NOT_EMPTY()
db.Item.image.requires =IS_IMAGE()
def details():
item = request.args(0)
form = crud.update(db.Item, item, next=URL(r=request,
args=item))
return dict(form=form)
It allows me to upload PDFs and flashes 'record updated'
Re: [web2py] Re: Web2py and threads
On Aug 25, 2010, at 8:12 PM, Jonathan Lundell wrote: On Aug 25, 2010, at 7:56 PM, mdipierro wrote: This is a bug. I fixed it in trunk. Thanks Jonathan. It's fixed in the sense that it won't raise an exception. But now how is calling _unlock different from calling forget? Nag. On Aug 25, 9:30 pm, Jonathan Lundell jlund...@pobox.com wrote: On Aug 25, 2010, at 6:37 PM, mdipierro wrote: The problem is only if have two http request from the same client in the same session Thanks for that; I was wondering under which conditions unlocking might be permissible (and I'm still not entirely clear, but never mind for now). My concern is this. Here's unlock: def _unlock(self, response): if response and response.session_file: try: portalocker.unlock(response.session_file) response.session_file.close() del response.session_file - except: ### this should never happen but happens in Windows pass Now we save the session file: def _try_store_on_disk(self, request, response): if response._dbtable_and_field \ or not response.session_id \ or self._forget: self._unlock(response) return if response.session_new: # Tests if the session folder exists, if not, create it session_folder = os.path.dirname(response.session_filename) response.session_file = open(response.session_filename, 'wb') portalocker.lock(response.session_file, portalocker.LOCK_EX) cPickle.dump(dict(self), response.session_file) self._unlock(response) But response.session_file is None at this point. A arrives loads session and unlocks B arrives loads session and unlocks A change session and saves it B changes session and saves it Nothing breaks but B never sees changes made by A and they are overwritten by B. With locks A arrives loads session B arrives and waits A change session and saves it B loads session (with changes made by A) B changes session and saves it On Aug 25, 3:52 pm, Jonathan Lundell jlund...@pobox.com wrote: On Aug 25, 2010, at 1:41 PM, mdipierro wrote: call session._unlock() if you do not need session locking If you do that (without calling session.forget), what will happen in _try_store_on_disk when cPickle.dump(dict(self), response.session_file) is called with a None file argument? Or is cPickle.dump cool with that? Or am I misreading the logic? On Aug 25, 11:38 am, Phyo Arkar phyo.arkarl...@gmail.com wrote: Yes may be session was locked , thats why session.current=processing_path not working But then again , while processing files i try opening separate page , to other controller , it was waited till the first (file Crawler) page finished parsing. ok i will make a separate thread about this. On 8/25/10, mdipierro mdipie...@cs.depaul.edu wrote: On Aug 25, 11:00 am, Phyo Arkar phyo.arkarl...@gmail.com wrote: Did I Read that reading files inside controller will block web2py , Does it? No web2py does not block. web2py only locks sessions that means one user cannot request two concurrent pages because there would be a race condition in saving sessions. Two user can request different pages which open the same file unless the file is explicitly locked by your code. Thats a bad news.. i am doing a file crawler and while crawling , web2py is blocked even tho the process talke only 25% of 1 out of 4 CPUs .. Tell us more or I cannot help. On 8/25/10, pierreth pierre.thibau...@gmail.com wrote: I would appreciate a good reference to understand the concepts you are talking about. It is something new to me and I don't understand. On 25 août, 11:22, John Heenan johnmhee...@gmail.com wrote: No, nothing that abstract. Using WSGI forces a new thread for each request. This is is a simple and inefficient brute force approach that really only suits the simplest Python applications and where only a small number of concurrent connection might be expected. Any application that provides web services is going to OS block on file reading (and writing) and on database access. Using threads is a classic and easy way out that carries a lot of baggage. Windows has had a way out of this for years with its asynch (or event) notification set up through an OVERLAPPED structure. Lightttpd makes use of efficient event notification schemes like kqueue and epoll. Apache only uses such schemes for listening and Keep- Alives. No matter how careful one is with threads and processes there always appears to be unexpected gotchas. Python has a notorious example, the now fixed 'Beazly Effect' that affected the GIL. Also I don't think there is a single experienced Python user that trusts the GIL. John Heenan
Re: [web2py] Re: best grid system
On Aug 28, 2010, at 10:07 AM, mdipierro wrote: datatable -- i can do all i want, nice documentation/example good for searching but only for small datasets Supports server-side filtering, though: http://www.datatables.net/usage/server-side jqgrid -- i cant find a good documentation, i can do only the basic configuration wery well documented and good implementation: http://www.trirand.com/jqgridwiki/doku.php?id=wiki:jqgriddocs powerfuls confifurations, seach and ajax features. Scales well with large datasets (because ajax search). intergrated with plugin_wiki (look at source code). You can use it without much JS programming. flexigrid -- wonderfull from screenshot, but i can't find nothing on it Requires lots of JS on your side to use it. I gave up.
Re: [web2py] Re: please check 1.84.0 (beta)
On Aug 29, 2010, at 12:02 AM, mart wrote: 2. changed .bash_profile to point to python 2.7 (even if nor recommended, 1.83 worked just fine on 2.7 for me). This time got the 'No handlers could be found for logger web2py' error and returned the terminal to the cursor (web2py stopped immediately - which I believe should be expected because of the unsupported python version (although 1.83 worked fine) Try making a copy of logging.example.conf and naming it logging.conf, and see what happens. I haven't done any testing with 2.7. Quite possibly if there isn't a logging.conf we ought to install an explicit default logger.
Re: [web2py] Re: please check 1.84.0 (beta)
On Aug 29, 2010, at 9:46 AM, mart wrote:
So, I just downloaded the src from
http://web2py.com/examples/static/nightly/web2py_src.zip,
just in case there are changes since last night. I am on mac os 10.6.4
and this is what I get (copied from terminal):
macmart:web2py_184 mart$ python web2py.py
Warning: web2py requires Python 2.4, 2.5 (recommended), or 2.6 but you
are running:
2.7 (r27:82508, Jul 3 2010, 21:12:11)
[GCC 4.0.1 (Apple Inc. build 5493)]web2py Enterprise Web Framework
Created by Massimo Di Pierro, Copyright 2007-2010
Version 1.84.0 (2010-08-29 06:19:04)
Database drivers available: SQLite3
Starting hardcron...
No handlers could be found for logger web2py
choose a password:*
please visit:
http://127.0.0.1:8000
use kill -SIGTERM 1746 to shutdown the web2py server
macmart:web2py_184 mart$
So, we still notice the same error No handlers could be found for
logger web2py and the I am brought back to the terminal cursor.
So, now on to Jonathan's suggestion.
SO, now I/ looking for logging.example.conf but can't find it, does it
usually come with the src? Or what should the contents be? I did see
some references to that file in a few place though...
It should be in the root directory, as part of the distribution (it's a new
file); if it's not, it must have something to do with how the distribution is
created. Massimo?
In the meantime, try the following (plain text file):
[loggers]
keys=root,rocket,markdown,web2py,rewrite,app,welcome
# the default configuration is console-based (stdout) for backward compatibility
#
# note that file-based handlers are thread-safe but not mp-safe;
# for mp-safe logging, configure the appropriate syslog handler
[handlers]
keys=consoleHandler
#keys=consoleHandler,rotatingFileHandler
#keys=osxSysLogHandler
[formatters]
keys=simpleFormatter
[logger_root]
level=WARNING
handlers=consoleHandler
[logger_web2py]
level=WARNING
handlers=consoleHandler
qualname=web2py
propagate=0
[logger_rewrite]
level=WARNING
qualname=web2py.rewrite
handlers=consoleHandler
propagate=0
# generic app handler
[logger_app]
level=WARNING
qualname=web2py.app
handlers=consoleHandler
propagate=0
# welcome app handler
[logger_welcome]
level=WARNING
qualname=web2py.app.welcome
handlers=consoleHandler
propagate=0
# loggers for legacy getLogger calls: Rocket and markdown
[logger_rocket]
level=WARNING
handlers=consoleHandler
qualname=Rocket
propagate=0
[logger_markdown]
level=WARNING
handlers=consoleHandler
qualname=markdown
propagate=0
[handler_consoleHandler]
class=StreamHandler
level=WARNING
formatter=simpleFormatter
args=(sys.stdout,)
# Rotating file handler
# mkdir logs in the web2py base directory if not already present
# args: (filename[, mode[, maxBytes[, backupCount[, encoding[, delay])
#
[handler_rotatingFileHandler]
class=handlers.RotatingFileHandler
level=INFO
formatter=simpleFormatter
args=(logs/web2py.log, a, 100, 5)
[handler_osxSysLogHandler]
class=handlers.SysLogHandler
level=WARNING
formatter=simpleFormatter
args=(/var/run/syslog, handlers.SysLogHandler.LOG_DAEMON)
[handler_linuxSysLogHandler]
class=handlers.SysLogHandler
level=WARNING
formatter=simpleFormatter
args=(/dev/log, handlers.SysLogHandler.LOG_DAEMON)
[handler_remoteSysLogHandler]
class=handlers.SysLogHandler
level=WARNING
formatter=simpleFormatter
args=(('sysloghost.domain.com', handlers.SYSLOG_UDP_PORT),
handlers.SysLogHandler.LOG_DAEMON)
[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=
Re: [web2py] Re: please check 1.84.0 (beta)
On Aug 29, 2010, at 9:46 AM, mart wrote:
So, we still notice the same error No handlers could be found for
logger web2py and the I am brought back to the terminal cursor.
There may be two problems here.
One is that the no handlers message may be normal, but shouldn't result in
termination. So there may be a separate 2.7 issue that's causing you to
terminate that has nothing to do with the message.
Try running this from the python command line:
import logging
l = logging.getLogger('foobar')
l.warning('bar')
No handlers could be found for logger foobar
This is 2.6.4 on OS X. We get the message, but no termination. How about 2.7?
Re: [web2py] Re: please check 1.84.0 (beta)
On Aug 29, 2010, at 10:26 AM, mart wrote:
Working great now! Thanks for that :)
A favor, please, Mart. In main.py, you'll find this logging init code:
logpath = os.path.join(web2py_path, logging.conf)
if os.path.exists(logpath):
logging.config.fileConfig(os.path.join(web2py_path, logging.conf))
logger = logging.getLogger(web2py)
In the configuration that terminates for you (2.7 and no logging.conf, I
believe), try adding an else clause to the above, thus:
logpath = os.path.join(web2py_path, logging.conf)
if os.path.exists(logpath):
logging.config.fileConfig(os.path.join(web2py_path, logging.conf))
else:
logging.basicConfig()
logger = logging.getLogger(web2py)
Hopefully that will solve the termination problem, in which case we'll want
that patch for 2.7 (and it should do no harm before that). I have to think it's
a 2.7 bug, but who knows?
Also, Mart, can you try that python command-line test under 2.7?
On Aug 29, 1:06 pm, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 29, 2010, at 9:46 AM, mart wrote:
So, I just downloaded the src
fromhttp://web2py.com/examples/static/nightly/web2py_src.zip,
just in case there are changes since last night. I am on mac os 10.6.4
and this is what I get (copied from terminal):
macmart:web2py_184 mart$ python web2py.py
Warning: web2py requires Python 2.4, 2.5 (recommended), or 2.6 but you
are running:
2.7 (r27:82508, Jul 3 2010, 21:12:11)
[GCC 4.0.1 (Apple Inc. build 5493)]web2py Enterprise Web Framework
Created by Massimo Di Pierro, Copyright 2007-2010
Version 1.84.0 (2010-08-29 06:19:04)
Database drivers available: SQLite3
Starting hardcron...
No handlers could be found for logger web2py
choose a password:*
please visit:
http://127.0.0.1:8000
use kill -SIGTERM 1746 to shutdown the web2py server
macmart:web2py_184 mart$
So, we still notice the same error No handlers could be found for
logger web2py and the I am brought back to the terminal cursor.
So, now on to Jonathan's suggestion.
SO, now I/ looking for logging.example.conf but can't find it, does it
usually come with the src? Or what should the contents be? I did see
some references to that file in a few place though...
It should be in the root directory, as part of the distribution (it's a new
file); if it's not, it must have something to do with how the distribution
is created. Massimo?
In the meantime, try the following (plain text file):
[loggers]
keys=root,rocket,markdown,web2py,rewrite,app,welcome
# the default configuration is console-based (stdout) for backward
compatibility
#
# note that file-based handlers are thread-safe but not mp-safe;
# for mp-safe logging, configure the appropriate syslog handler
[handlers]
keys=consoleHandler
#keys=consoleHandler,rotatingFileHandler
#keys=osxSysLogHandler
[formatters]
keys=simpleFormatter
[logger_root]
level=WARNING
handlers=consoleHandler
[logger_web2py]
level=WARNING
handlers=consoleHandler
qualname=web2py
propagate=0
[logger_rewrite]
level=WARNING
qualname=web2py.rewrite
handlers=consoleHandler
propagate=0
# generic app handler
[logger_app]
level=WARNING
qualname=web2py.app
handlers=consoleHandler
propagate=0
# welcome app handler
[logger_welcome]
level=WARNING
qualname=web2py.app.welcome
handlers=consoleHandler
propagate=0
# loggers for legacy getLogger calls: Rocket and markdown
[logger_rocket]
level=WARNING
handlers=consoleHandler
qualname=Rocket
propagate=0
[logger_markdown]
level=WARNING
handlers=consoleHandler
qualname=markdown
propagate=0
[handler_consoleHandler]
class=StreamHandler
level=WARNING
formatter=simpleFormatter
args=(sys.stdout,)
# Rotating file handler
# mkdir logs in the web2py base directory if not already present
# args: (filename[, mode[, maxBytes[, backupCount[, encoding[, delay])
#
[handler_rotatingFileHandler]
class=handlers.RotatingFileHandler
level=INFO
formatter=simpleFormatter
args=(logs/web2py.log, a, 100, 5)
[handler_osxSysLogHandler]
class=handlers.SysLogHandler
level=WARNING
formatter=simpleFormatter
args=(/var/run/syslog, handlers.SysLogHandler.LOG_DAEMON)
[handler_linuxSysLogHandler]
class=handlers.SysLogHandler
level=WARNING
formatter=simpleFormatter
args=(/dev/log, handlers.SysLogHandler.LOG_DAEMON)
[handler_remoteSysLogHandler]
class=handlers.SysLogHandler
level=WARNING
formatter=simpleFormatter
args=(('sysloghost.domain.com', handlers.SYSLOG_UDP_PORT),
handlers.SysLogHandler.LOG_DAEMON)
[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=
Re: [web2py] Re: please check 1.84.0 (beta)
On Aug 29, 2010, at 11:12 AM, mart wrote:
hey Jonathan,
renamed logging.conf to logging_x.conf, modified main.py, and re-ran
web2py. Worked great for me! :)
Thanks, I appreciate it.
Thanks,
Mart :)
On Aug 29, 1:42 pm, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 29, 2010, at 10:26 AM, mart wrote:
Working great now! Thanks for that :)
A favor, please, Mart. In main.py, you'll find this logging init code:
logpath = os.path.join(web2py_path, logging.conf)
if os.path.exists(logpath):
logging.config.fileConfig(os.path.join(web2py_path, logging.conf))
logger = logging.getLogger(web2py)
In the configuration that terminates for you (2.7 and no logging.conf, I
believe), try adding an else clause to the above, thus:
logpath = os.path.join(web2py_path, logging.conf)
if os.path.exists(logpath):
logging.config.fileConfig(os.path.join(web2py_path, logging.conf))
else:
logging.basicConfig()
logger = logging.getLogger(web2py)
Hopefully that will solve the termination problem, in which case we'll want
that patch for 2.7 (and it should do no harm before that). I have to think
it's a 2.7 bug, but who knows?
Also, Mart, can you try that python command-line test under 2.7?
On Aug 29, 1:06 pm, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 29, 2010, at 9:46 AM, mart wrote:
So, I just downloaded the src
fromhttp://web2py.com/examples/static/nightly/web2py_src.zip,
just in case there are changes since last night. I am on mac os 10.6.4
and this is what I get (copied from terminal):
macmart:web2py_184 mart$ python web2py.py
Warning: web2py requires Python 2.4, 2.5 (recommended), or 2.6 but you
are running:
2.7 (r27:82508, Jul 3 2010, 21:12:11)
[GCC 4.0.1 (Apple Inc. build 5493)]web2py Enterprise Web Framework
Created by Massimo Di Pierro, Copyright 2007-2010
Version 1.84.0 (2010-08-29 06:19:04)
Database drivers available: SQLite3
Starting hardcron...
No handlers could be found for logger web2py
choose a password:*
please visit:
http://127.0.0.1:8000
use kill -SIGTERM 1746 to shutdown the web2py server
macmart:web2py_184 mart$
So, we still notice the same error No handlers could be found for
logger web2py and the I am brought back to the terminal cursor.
So, now on to Jonathan's suggestion.
SO, now I/ looking for logging.example.conf but can't find it, does it
usually come with the src? Or what should the contents be? I did see
some references to that file in a few place though...
It should be in the root directory, as part of the distribution (it's a
new file); if it's not, it must have something to do with how the
distribution is created. Massimo?
In the meantime, try the following (plain text file):
[loggers]
keys=root,rocket,markdown,web2py,rewrite,app,welcome
# the default configuration is console-based (stdout) for backward
compatibility
#
# note that file-based handlers are thread-safe but not mp-safe;
# for mp-safe logging, configure the appropriate syslog handler
[handlers]
keys=consoleHandler
#keys=consoleHandler,rotatingFileHandler
#keys=osxSysLogHandler
[formatters]
keys=simpleFormatter
[logger_root]
level=WARNING
handlers=consoleHandler
[logger_web2py]
level=WARNING
handlers=consoleHandler
qualname=web2py
propagate=0
[logger_rewrite]
level=WARNING
qualname=web2py.rewrite
handlers=consoleHandler
propagate=0
# generic app handler
[logger_app]
level=WARNING
qualname=web2py.app
handlers=consoleHandler
propagate=0
# welcome app handler
[logger_welcome]
level=WARNING
qualname=web2py.app.welcome
handlers=consoleHandler
propagate=0
# loggers for legacy getLogger calls: Rocket and markdown
[logger_rocket]
level=WARNING
handlers=consoleHandler
qualname=Rocket
propagate=0
[logger_markdown]
level=WARNING
handlers=consoleHandler
qualname=markdown
propagate=0
[handler_consoleHandler]
class=StreamHandler
level=WARNING
formatter=simpleFormatter
args=(sys.stdout,)
# Rotating file handler
# mkdir logs in the web2py base directory if not already present
# args: (filename[, mode[, maxBytes[, backupCount[, encoding[,
delay])
#
[handler_rotatingFileHandler]
class=handlers.RotatingFileHandler
level=INFO
formatter=simpleFormatter
args=(logs/web2py.log, a, 100, 5)
[handler_osxSysLogHandler]
class=handlers.SysLogHandler
level=WARNING
formatter=simpleFormatter
args=(/var/run/syslog, handlers.SysLogHandler.LOG_DAEMON)
[handler_linuxSysLogHandler]
class=handlers.SysLogHandler
level=WARNING
formatter=simpleFormatter
args=(/dev/log, handlers.SysLogHandler.LOG_DAEMON)
[handler_remoteSysLogHandler]
class=handlers.SysLogHandler
level=WARNING
formatter=simpleFormatter
args=(('sysloghost.domain.com', handlers.SYSLOG_UDP_PORT),
handlers.SysLogHandler.LOG_DAEMON)
[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=
Re: [web2py] Re: please check 1.84.0 (beta)
On Aug 29, 2010, at 3:40 PM, mart wrote: Just downloaded src from same location. All is working great :) however... i am still weeing the warning 'No handlers could be found for logger web2py' Do have a logging.conf, and/or do you see the change in main.py? thanks, Mart :) On Aug 29, 6:34 pm, mart msenecal...@gmail.com wrote: Any time :) On Aug 29, 4:40 pm, Jonathan Lundell jlund...@pobox.com wrote: On Aug 29, 2010, at 11:12 AM, mart wrote: hey Jonathan, renamed logging.conf to logging_x.conf, modified main.py, and re-ran web2py. Worked great for me! :) Thanks, I appreciate it. Thanks, Mart :) On Aug 29, 1:42 pm, Jonathan Lundell jlund...@pobox.com wrote: On Aug 29, 2010, at 10:26 AM, mart wrote: Working great now! Thanks for that :) A favor, please, Mart. In main.py, you'll find this logging init code: logpath = os.path.join(web2py_path, logging.conf) if os.path.exists(logpath): logging.config.fileConfig(os.path.join(web2py_path, logging.conf)) logger = logging.getLogger(web2py) In the configuration that terminates for you (2.7 and no logging.conf, I believe), try adding an else clause to the above, thus: logpath = os.path.join(web2py_path, logging.conf) if os.path.exists(logpath): logging.config.fileConfig(os.path.join(web2py_path, logging.conf)) else: logging.basicConfig() logger = logging.getLogger(web2py) Hopefully that will solve the termination problem, in which case we'll want that patch for 2.7 (and it should do no harm before that). I have to think it's a 2.7 bug, but who knows? Also, Mart, can you try that python command-line test under 2.7? On Aug 29, 1:06 pm, Jonathan Lundell jlund...@pobox.com wrote: On Aug 29, 2010, at 9:46 AM, mart wrote: So, I just downloaded the src fromhttp://web2py.com/examples/static/nightly/web2py_src.zip, just in case there are changes since last night. I am on mac os 10.6.4 and this is what I get (copied from terminal): macmart:web2py_184 mart$ python web2py.py Warning: web2py requires Python 2.4, 2.5 (recommended), or 2.6 but you are running: 2.7 (r27:82508, Jul 3 2010, 21:12:11) [GCC 4.0.1 (Apple Inc. build 5493)]web2py Enterprise Web Framework Created by Massimo Di Pierro, Copyright 2007-2010 Version 1.84.0 (2010-08-29 06:19:04) Database drivers available: SQLite3 Starting hardcron... No handlers could be found for logger web2py choose a password:* please visit: http://127.0.0.1:8000 use kill -SIGTERM 1746 to shutdown the web2py server macmart:web2py_184 mart$ So, we still notice the same error No handlers could be found for logger web2py and the I am brought back to the terminal cursor. So, now on to Jonathan's suggestion. SO, now I/ looking for logging.example.conf but can't find it, does it usually come with the src? Or what should the contents be? I did see some references to that file in a few place though... It should be in the root directory, as part of the distribution (it's a new file); if it's not, it must have something to do with how the distribution is created. Massimo? In the meantime, try the following (plain text file): [loggers] keys=root,rocket,markdown,web2py,rewrite,app,welcome # the default configuration is console-based (stdout) for backward compatibility # # note that file-based handlers are thread-safe but not mp-safe; # for mp-safe logging, configure the appropriate syslog handler [handlers] keys=consoleHandler #keys=consoleHandler,rotatingFileHandler #keys=osxSysLogHandler [formatters] keys=simpleFormatter [logger_root] level=WARNING handlers=consoleHandler [logger_web2py] level=WARNING handlers=consoleHandler qualname=web2py propagate=0 [logger_rewrite] level=WARNING qualname=web2py.rewrite handlers=consoleHandler propagate=0 # generic app handler [logger_app] level=WARNING qualname=web2py.app handlers=consoleHandler propagate=0 # welcome app handler [logger_welcome] level=WARNING qualname=web2py.app.welcome handlers=consoleHandler propagate=0 # loggers for legacy getLogger calls: Rocket and markdown [logger_rocket] level=WARNING handlers=consoleHandler qualname=Rocket propagate=0 [logger_markdown] level=WARNING handlers=consoleHandler qualname=markdown propagate=0 [handler_consoleHandler] class=StreamHandler level=WARNING formatter=simpleFormatter args=(sys.stdout,) # Rotating file handler # mkdir logs in the web2py base directory if not already present # args: (filename[, mode[, maxBytes[, backupCount[, encoding[, delay]) # [handler_rotatingFileHandler] class=handlers.RotatingFileHandler level=INFO formatter=simpleFormatter args=(logs/web2py.log, a, 100, 5) [handler_osxSysLogHandler] class=handlers.SysLogHandler level=WARNING formatter=simpleFormatter args=(/var/run/syslog, handlers.SysLogHandler.LOG_DAEMON) [handler_linuxSysLogHandler] class=handlers.SysLogHandler level
Re: [web2py] Re: please check 1.84.0 (beta)
On Aug 29, 2010, at 6:24 PM, mart wrote:
hey again...
So, I should have looked a little deeper earlier when I saw it working
but still getting the error/warning...
so, not too sure what's happening right now (will deeper later this
evening).
but for now. I synced the src from same location specified at the
beginning of the thread. extracted in a different location as
yesterday's instance. the web2py_path/gluon/main.py does not contain
your new else statement, but the older version:
logpath = os.path.join(web2py_path, logging.conf)
if os.path.exists(logpath):
logging.config.fileConfig(os.path.join(web2py_path,
logging.conf))
logger = logging.getLogger(web2py)
and no logging.conf exists.
Just for the record: the release will have logging.example.conf, which you'll
need to copy to logging.conf to have it take effect.
But the change to main.py should give a (possibly) reasonable default in the
absence of a logging.conf, as long as it's possible to write to stdout.
I noticed that os.environ is being used to set cwd, so I checked the
output, which gives me this:
I'm suspicious of the cwd logic, too. But that's (maybe) a question for another
day.
macmart:web2py_2_184 mart$ python
Python 2.7 (r27:82508, Jul 3 2010, 21:12:11)
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type help, copyright, credits or license for more information.
import os
for item in os.environ:
... print({0} = {1}.format(item,os.environ[item]))
...
LANG = en_CA.UTF-8
TERM = xterm-color
Apple_PubSub_Socket_Render = /tmp/launch-nyryAG/Render
SHLVL = 1
OLDPWD = /Users/mart/Downloads
SSH_AUTH_SOCK = /tmp/launch-xtTLKc/Listeners
TERM_PROGRAM_VERSION = 273
__CF_USER_TEXT_ENCODING = 0x1F6:0:0
PWD = /Users/mart/Downloads/web2py_2_184
SHELL = /bin/bash
LOGNAME = mart
USER = mart
HOME = /Users/mart
PATH = /Library/Frameworks/Python.framework/Versions/2.7/bin:/Library/
Frameworks/Python.framework/Versions/2.6/bin:/usr/bin:/bin:/usr/sbin:/
sbin:/usr/local/bin:/usr/local/git/bin:/usr/X11/bin
COMMAND_MODE = unix2003
_ = /Library/Frameworks/Python.framework/Versions/2.7/bin/python
DISPLAY = /tmp/launch-fj9Ipo/org.x:0
TMPDIR = /var/folders/gL/gLsicOEFGNC3VqZE1yVgmU+++TM/-Tmp-/
TERM_PROGRAM = Apple_Terminal
So, no mention of previous web2py install path (unless I;m not looking
correctly), but it still works. To be sure, I renamed the previous
web2py's folder name (just in case, something had picked it up)... but
still working (and getting the warning)...
So, not too sure why this isn't failing as it did, this morning...
Thanks,
Mart :)
On Aug 29, 6:59 pm, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 29, 2010, at 3:40 PM, mart wrote:
Just downloaded src from same location. All is working great :)
however... i am still weeing the warning 'No handlers could be found
for logger web2py'
Do have a logging.conf, and/or do you see the change in main.py?
thanks,
Mart :)
On Aug 29, 6:34 pm, mart msenecal...@gmail.com wrote:
Any time :)
On Aug 29, 4:40 pm, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 29, 2010, at 11:12 AM, mart wrote:
hey Jonathan,
renamed logging.conf to logging_x.conf, modified main.py, and re-ran
web2py. Worked great for me! :)
Thanks, I appreciate it.
Thanks,
Mart :)
On Aug 29, 1:42 pm, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 29, 2010, at 10:26 AM, mart wrote:
Working great now! Thanks for that :)
A favor, please, Mart. In main.py, you'll find this logging init code:
logpath = os.path.join(web2py_path, logging.conf)
if os.path.exists(logpath):
logging.config.fileConfig(os.path.join(web2py_path, logging.conf))
logger = logging.getLogger(web2py)
In the configuration that terminates for you (2.7 and no logging.conf,
I believe), try adding an else clause to the above, thus:
logpath = os.path.join(web2py_path, logging.conf)
if os.path.exists(logpath):
logging.config.fileConfig(os.path.join(web2py_path, logging.conf))
else:
logging.basicConfig()
logger = logging.getLogger(web2py)
Hopefully that will solve the termination problem, in which case we'll
want that patch for 2.7 (and it should do no harm before that). I have
to think it's a 2.7 bug, but who knows?
Also, Mart, can you try that python command-line test under 2.7?
On Aug 29, 1:06 pm, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 29, 2010, at 9:46 AM, mart wrote:
So, I just downloaded the src
fromhttp://web2py.com/examples/static/nightly/web2py_src.zip,
just in case there are changes since last night. I am on mac os
10.6.4
and this is what I get (copied from terminal):
macmart:web2py_184 mart$ python web2py.py
Warning: web2py requires Python 2.4, 2.5 (recommended), or 2.6 but
you
are running:
2.7 (r27:82508, Jul 3 2010, 21:12:11)
[GCC 4.0.1 (Apple Inc. build 5493)]web2py Enterprise Web Framework
Created by Massimo Di Pierro, Copyright 2007-2010
Re: [web2py] Re: per app routes
On Aug 31, 2010, at 9:31 AM, mwolfe02 wrote: I'm still debugging now, but there seems to be an issue using the special $anything token inside app-specific routes_in. It appears that the incoming URL is being applied to each app-specific routes_in one at a time before it is applied against routes_app. I'll try to include more details later, but I wanted to bring it to your attention now. Thanks. I'll wait to hear from you. It should look at the global routes_app first (because it doesn't know which app-specific routes to use), and if that resolves to an app that has its own routes file, then use that for routes_in. -Mike On Aug 7, 12:36 pm, Jonathan Lundell jlund...@pobox.com wrote: On Aug 7, 2010, at 9:32 AM, David Marko wrote: Hello, have you tested performance impact on application. Do you assume some noticeable slowdown when usingroutes? I have not measured it, but I'd expect the effect to be trivial, perhaps unmeasurable in that it'd be in the noise. In particular, the routing files are read and the regexes compiled only once, when web2py starts up, so the per-request overhead is quite low. david On 7 srp, 18:26, Jonathan Lundell jlund...@pobox.com wrote: On Aug 7, 2010, at 9:03 AM, mdipierro wrote: Thanks to Jonathan Lundell we have an experimental version in trunk of applevelroutes. To understand how it works readroutes.example.py and comments in the file gluon/rewrite.py If you test it please report your findings here. *Very* experimental, mostly not tested. I'll describe some of the changes here. 1. If you don't explicitly invoke any of the new features, routing should behave identically to before. If you see any different, please let us know asap. 2. You can now have aroutes.py in the top level folder of an application, and it will be used *instead* of the baseroutes.py. However, it's not enough to simply have the file there; you must inform the routing logic about it. 3. The way you inform the routing logic is with a new element in the baseroutes.py: routes_app. routes_app is processed identically to routes_in, but the output must be anappname (or nothing). routes_app is processed at the beginning of a request. If it produces anappname, and thatapphas anapp-specificroutes.py (that is, applications/appname/routes.py), then thatroutes.py is used instead of the baseroutes.py. 4. In an unrelated change, there are three other new elements inroutes.py: default_application = init default_controller = default default_function = index Note that default_application doesn't interact withapp-specfic routing, since it's used after rewrite has taken place. default_controller and default_function should normally be used only in anapp-specificroutes.py, because, in the baseroutes.py, they will apply to all apps *without* anapp-specificroutes.py. That would probably lead to confusion when running admin or examples; at the very least their defaults would break. 5. As usual, I suggest that when you editroutes.example.py to generate a newroutes.py, you also edit the doctest at the end, and use it to verify that you're getting what you expect. To run the doctest, just do pythonroutes.py. Note also that I have a more far-reaching change in mind, but don't have it worked out yet. The new version will move away from regexes (though the old logic will remain in place for compatibility). It's supposed to be more flexible and much easier to use, and also handle URL encoding decoding better. But this change should help in the meantime.
Re: [web2py] Re: per app routes
On Aug 31, 2010, at 10:01 AM, mwolfe02 wrote: More clues... When web2py loads, rewrite.params.routes_in gets set as follows: - items from base routes_in are appended first - then items from app-specific routes_in are appended in alphabetical order by application This seems to be causing my problems. More info to follow... I think I may see the problem. If I'm right, when we initialize a new set of params from the default set, we need to do a deeper copy than we're doing now. Can I send you a replacement rewrite.py to try out? It'd be from the trunk, which ought to be equivalent, for our purposes, to the current nightly and close enough to the last stable release. -Mike On Aug 31, 12:31 pm, mwolfe02 michael.joseph.wo...@gmail.com wrote: I'm still debugging now, but there seems to be an issue using the special $anything token insideapp-specificroutes_in. It appears that the incoming URL is being applied to eachapp-specificroutes_in one at a time before it is applied against routes_app. I'll try to include more details later, but I wanted to bring it to your attention now. -Mike On Aug 7, 12:36 pm, Jonathan Lundell jlund...@pobox.com wrote: On Aug 7, 2010, at 9:32 AM, David Marko wrote: Hello, have you tested performance impact on application. Do you assume some noticeable slowdown when usingroutes? I have not measured it, but I'd expect the effect to be trivial, perhaps unmeasurable in that it'd be in the noise. In particular, the routing files are read and the regexes compiled only once, when web2py starts up, so the per-request overhead is quite low. david On 7 srp, 18:26, Jonathan Lundell jlund...@pobox.com wrote: On Aug 7, 2010, at 9:03 AM, mdipierro wrote: Thanks to Jonathan Lundell we have an experimental version in trunk of applevelroutes. To understand how it works readroutes.example.py and comments in the file gluon/rewrite.py If you test it please report your findings here. *Very* experimental, mostly not tested. I'll describe some of the changes here. 1. If you don't explicitly invoke any of the new features, routing should behave identically to before. If you see any different, please let us know asap. 2. You can now have aroutes.py in the top level folder of an application, and it will be used *instead* of the baseroutes.py. However, it's not enough to simply have the file there; you must inform the routing logic about it. 3. The way you inform the routing logic is with a new element in the baseroutes.py: routes_app. routes_app is processed identically to routes_in, but the output must be anappname (or nothing). routes_app is processed at the beginning of a request. If it produces anappname, and thatapphas anapp-specificroutes.py (that is, applications/appname/routes.py), then thatroutes.py is used instead of the baseroutes.py. 4. In an unrelated change, there are three other new elements inroutes.py: default_application = init default_controller = default default_function = index Note that default_application doesn't interact withapp-specfic routing, since it's used after rewrite has taken place. default_controller and default_function should normally be used only in anapp-specificroutes.py, because, in the baseroutes.py, they will apply to all apps *without* anapp-specificroutes.py. That would probably lead to confusion when running admin or examples; at the very least their defaults would break. 5. As usual, I suggest that when you editroutes.example.py to generate a newroutes.py, you also edit the doctest at the end, and use it to verify that you're getting what you expect. To run the doctest, just do pythonroutes.py. Note also that I have a more far-reaching change in mind, but don't have it worked out yet. The new version will move away from regexes (though the old logic will remain in place for compatibility). It's supposed to be more flexible and much easier to use, and also handle URL encoding decoding better. But this change should help in the meantime.
Re: [web2py] Re: per app routes
On Aug 31, 2010, at 12:53 PM, mwolfe02 wrote: default_function does not seem to be recognized properly in app- specific routes.py. I'm thinking default_controller may have a similar problem, but I'm not really redefining it. My base routes.py has default_application set to 'my_app' (and nothing set for default_controller or default_function). In the routes.py file for my 'my_app' I have the following set: default_controller = 'default' # ordinarily set in app-specific routes.py default_function = 'search' # ordinarily set in app-specific routes.py When I visit http://domain.com/ I receive the 'invalid function' page instead of rewriting to http://domain.com/my_app/default/search/. I'm debugging now and will post back when I learn more. You'll need to define default_application in the app-specific routes file. I know that doesn't make a lot of sense; I'll look at changing that so that it shows up as the current app by default. Let me know if that works.
Re: [web2py] Re: per app routes
On Aug 31, 2010, at 12:53 PM, mwolfe02 wrote: default_function does not seem to be recognized properly in app- specific routes.py. I'm thinking default_controller may have a similar problem, but I'm not really redefining it. My base routes.py has default_application set to 'my_app' (and nothing set for default_controller or default_function). In the routes.py file for my 'my_app' I have the following set: default_controller = 'default' # ordinarily set in app-specific routes.py default_function = 'search' # ordinarily set in app-specific routes.py When I visit http://domain.com/ I receive the 'invalid function' page instead of rewriting to http://domain.com/my_app/default/search/. I'm debugging now and will post back when I learn more. OK, making the current app the default turned out to be pretty straightforward, and even if that's not the problem you're having, I think it makes sense to do. Here's the new rewrite.py: http://web.me.com/jlundell/filechute/rewrite.zip
Re: [web2py] Re: per app routes
On Aug 31, 2010, at 2:20 PM, Michael Wolfe wrote:
That didn't seem to quite do it. Visiting http://domain.com/ rewrites
to http://domain.com/my_app/default/index/ instead of
http://domain.com/my_app/default/search/. The URL is being
substantively rewritten in the parse_url function (lines 802-807) of
gluon/main.py:
request.application = \
regex_space.sub('_', match.group('a') or
rewrite.params.default_application)
request.controller = \
regex_space.sub('_', match.group('c') or
rewrite.params.default_controller)
request.function = \
regex_space.sub('_', match.group('f') or
rewrite.params.default_function)
The problem being that rewrite.params.default_function is not using
the default_function specified in my app-specific routes.py.
The parse_url function is being called from line 326 of gluon/main.py:
# ##
# invoke the legacy URL parser and serve static file
# ##
static_file = parse_url(request, environ)
To be clear, /my_app/default/search/ is not a static file; parse_url
appears to do double-duty identifying static files and performing
simple URL re-writes.
On a side note, I'll be heading home for the day soon and won't be
working on this project again until Thursday. So if you don't get a
response from me for awhilethat's why.
OK. I'll take a closer look. It's helpful to know that it's getting 'index' in
this case.
One final thing: what's your routes_app? Is http://domain.com/ resulting in
my_app? Maybe you could send me, privately if you like, your global and my_app
routes.py.
-Mike
On Tue, Aug 31, 2010 at 4:09 PM, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 31, 2010, at 12:53 PM, mwolfe02 wrote:
default_function does not seem to be recognized properly in app-
specific routes.py. I'm thinking default_controller may have a
similar problem, but I'm not really redefining it.
My base routes.py has default_application set to 'my_app' (and nothing
set for default_controller or default_function). In the routes.py
file for my 'my_app' I have the following set:
default_controller = 'default' # ordinarily set in app-specific
routes.py
default_function = 'search' # ordinarily set in app-specific
routes.py
When I visit http://domain.com/ I receive the 'invalid function' page
instead of rewriting to http://domain.com/my_app/default/search/. I'm
debugging now and will post back when I learn more.
OK, making the current app the default turned out to be pretty
straightforward, and even if that's not the problem you're having, I think
it makes sense to do. Here's the new rewrite.py:
http://web.me.com/jlundell/filechute/rewrite.zip
[web2py] rewrite.py patch
One more change to rewrite.py. http://web.me.com/jlundell/filechute/rewrite.zip I took out the custom logging logic, which was confusing, because now that we have generalized logging it's redundant. However, there's a remaining problem to be resolved. Maybe.
[web2py] bug in app-specific routing
Massimo, it dawned on me (literally--I was lying in bed with the sun coming up) that we can't use the global rewrite.params to store the app-specific routing parameters because it's not thread-safe. Right? But I can't think of where we *could* keep that info, given the places that need to use it. So: help!
Re: [web2py] Re: bug in app-specific routing
On Sep 1, 2010, at 7:01 AM, mdipierro wrote: No we cannot. rewrite.params[appname]? That structure exists, but where would appname come from (in general)? I think that's the same problem. We could maybe stick it in request or response, but params is used a couple of places where that's not available. Maybe the environment that carries request and the other globals? On Sep 1, 8:46 am, Jonathan Lundell jlund...@pobox.com wrote: Massimo, it dawned on me (literally--I was lying in bed with the sun coming up) that we can't use the global rewrite.params to store the app-specific routing parameters because it's not thread-safe. Right? But I can't think of where we *could* keep that info, given the places that need to use it. So: help!
Re: [web2py] Re: bug in app-specific routing
On Sep 1, 2010, at 7:32 AM, mdipierro wrote:
request.application is already there. Where is it used and the
appmname is not available?
If request were always available, we could store the appropriate params link
there as request.params.
Here's an example (not, I think, the only one):
streamer.stream_file_or_304_or_206() looks like this:
def stream_file_or_304_or_206(
static_file,
chunk_size = DEFAULT_CHUNK_SIZE,
request = None,
headers = {},
error_message = rewrite.params.error_message,
):
We could default error_message to None, and fill in the default
programmatically, but it looks to me like we can't count on having request here.
Similarly, in rewrite.filter_out, the first thing we look for is
params.routes_out, but can we count on any global context at all?
On Sep 1, 9:17 am, Jonathan Lundell jlund...@pobox.com wrote:
On Sep 1, 2010, at 7:01 AM, mdipierro wrote:
No we cannot.
rewrite.params[appname]?
That structure exists, but where would appname come from (in general)? I
think that's the same problem.
We could maybe stick it in request or response, but params is used a couple
of places where that's not available. Maybe the environment that carries
request and the other globals?
On Sep 1, 8:46 am, Jonathan Lundell jlund...@pobox.com wrote:
Massimo, it dawned on me (literally--I was lying in bed with the sun
coming up) that we can't use the global rewrite.params to store the
app-specific routing parameters because it's not thread-safe. Right?
But I can't think of where we *could* keep that info, given the places
that need to use it.
So: help!
Re: [web2py] Re: rewrite.py patch
On Sep 1, 2010, at 5:14 PM, mart wrote: just a quick question: is this related to the 1.84 build issue on Mac os with python 2.7? Not exactly, I don't think. That was fixed, afaik, by the default logger patch. This is a bug in app-specific routing that we don't have a solution to yet. No problem if you're not using it, or if you have only one application running. thanks, Mart :) On Aug 31, 9:08 pm, Jonathan Lundell jlund...@pobox.com wrote: One more change to rewrite.py. http://web.me.com/jlundell/filechute/rewrite.zip I took out the custom logging logic, which was confusing, because now that we have generalized logging it's redundant. However, there's a remaining problem to be resolved. Maybe.
Re: [web2py] Re: per app routes
On Sep 2, 2010, at 6:40 AM, mwolfe02 wrote:
Actually, I had commented out routes_app altogether. When I restored
it to this, things seemed to work again:
routes_app = ((r'/(?Pappwelcome|admin|examples|app)\b.*',
r'\gapp'),
(r'(.*)', r'my_app'),
(r'/?(.*)', r'my_app'))
I was thinking that the logic would work as follows:
1. if base routes_in exists check URL against base routes_in (if match
found then rewrite else continue)
2. if routes_app exists check URL against routes_app (if match found
then load app-specific routes_in or app-specific default controller/
function else continue)
3. if default_app specified then load app-specific routes_in or app-
specific default controller/function for the default_app
4. if no default_app specified and URL does not match base routes_in
or routes_app return error
First of all, default_* doesn't really have anything to do with rewriting. I
added them to routes.py as a convenience, to be able to override the default
init/default/index logic that happens after all the routes_in is complete. I
would recommend using routes_* or default_*, but not both.
routes_app is the first thing we look at. It completely determines which
routes.py (base or app-specific) we'll use for the entire request and response.
If there's no routes_app, then we'll always use the base routes.py.
So the rule is actually pretty simple:
1. If routes_app produces an application name, and that application has its own
routes.py, then use that app-specific routes.py. In all other cases, use the
base routes.py. (This decision is final for the entire request.)
2. Using the routes.py determined in (1), apply routes_in to the URL. The best
practice, in my view, is for routes_in to always product a complete URL
(a/c/f/...).
3. If the URL does not have all three routing elements /a/c/f, complete it with
default_* from the selected routes.py (defaulting in the code to
/init/default/index if not overridden).
4. All subsequent rewriting (routes_out, error rewriting, etc) uses the
routes.py selected in (1).
Note that an app-specfic routes.py is all or nothing. If (1) selects an
app-specific routes.py and that routes.py does not contain (say) a routes_out,
we do *not* fall back on the base routes_out. Similarly for default_*.
Once URL rewriting has been redirected to a specific app (as in step 2
or 3 above) do the following:
1. if app-specific routes_in exists check URL against app-specific
routes_in (if match found then rewrite else continue)
2. if URL maps to an existing controller/function, then call that
controller/function else continue
3. if default_controller specified, prepend default_controller to URL
and try step 2 else continue
4. if default_controller and default_function specified, prepend
default_controller/default_function to URL and try step 2 else
continue
5. if default_function specified, assume first part of URL is
controller, insert default_function after assumed controller and
before any potential function arguments
6. if nothing matches, return error
Obviously my assumptions were not entirely correct. I'm wondering if
you could pass along a brief overview of how the routes_app,
default_app, default_controller, and default_function parameters all
actually do interact in terms of URL rewriting.
Thanks again,
-Mike
On Aug 31, 5:34 pm, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 31, 2010, at 2:20 PM, Michael Wolfe wrote:
That didn't seem to quite do it. Visitinghttp://domain.com/rewrites
tohttp://domain.com/my_app/default/index/instead of
http://domain.com/my_app/default/search/. The URL is being
substantively rewritten in the parse_url function (lines 802-807) of
gluon/main.py:
request.application = \
regex_space.sub('_', match.group('a') or
rewrite.params.default_application)
request.controller = \
regex_space.sub('_', match.group('c') or
rewrite.params.default_controller)
request.function = \
regex_space.sub('_', match.group('f') or
rewrite.params.default_function)
The problem being that rewrite.params.default_function is not using
the default_function specified in my app-specific routes.py.
The parse_url function is being called from line 326 of gluon/main.py:
# ##
# invoke the legacy URL parser and serve static file
# ##
static_file = parse_url(request, environ)
To be clear, /my_app/default/search/ is not a static file; parse_url
appears to do double-duty identifying static files and performing
simple URL re-writes.
On a side note, I'll be heading home for the day soon and won't be
working on this project again until Thursday. So if you don't get a
response from me for awhilethat's why.
OK. I'll take a closer look. It's helpful to know that it's
Re: [web2py] Re: per app routes
On Sep 2, 2010, at 7:40 AM, Michael Wolfe wrote:
Great info. Thanks for taking the time to explain.
You're welcome.
A caveat: I don't believe that the current logic is thread-safe. Consequently,
if you have simultaneous requests on different threads that use different
routes.py files, one of them could end up using the wrong logic for some or all
of its translations.
I intend to fix it, but I haven't figured out how yet. So if you're using
multiple apps in production, I recommend sticking to the base routes.py for now.
-Mike
On Thu, Sep 2, 2010 at 10:16 AM, Jonathan Lundell jlund...@pobox.com wrote:
On Sep 2, 2010, at 6:40 AM, mwolfe02 wrote:
Actually, I had commented out routes_app altogether. When I restored
it to this, things seemed to work again:
routes_app = ((r'/(?Pappwelcome|admin|examples|app)\b.*',
r'\gapp'),
(r'(.*)', r'my_app'),
(r'/?(.*)', r'my_app'))
I was thinking that the logic would work as follows:
1. if base routes_in exists check URL against base routes_in (if match
found then rewrite else continue)
2. if routes_app exists check URL against routes_app (if match found
then load app-specific routes_in or app-specific default controller/
function else continue)
3. if default_app specified then load app-specific routes_in or app-
specific default controller/function for the default_app
4. if no default_app specified and URL does not match base routes_in
or routes_app return error
First of all, default_* doesn't really have anything to do with rewriting. I
added them to routes.py as a convenience, to be able to override the default
init/default/index logic that happens after all the routes_in is complete. I
would recommend using routes_* or default_*, but not both.
routes_app is the first thing we look at. It completely determines which
routes.py (base or app-specific) we'll use for the entire request and
response. If there's no routes_app, then we'll always use the base routes.py.
So the rule is actually pretty simple:
1. If routes_app produces an application name, and that application has its
own routes.py, then use that app-specific routes.py. In all other cases, use
the base routes.py. (This decision is final for the entire request.)
2. Using the routes.py determined in (1), apply routes_in to the URL. The
best practice, in my view, is for routes_in to always product a complete URL
(a/c/f/...).
3. If the URL does not have all three routing elements /a/c/f, complete it
with default_* from the selected routes.py (defaulting in the code to
/init/default/index if not overridden).
4. All subsequent rewriting (routes_out, error rewriting, etc) uses the
routes.py selected in (1).
Note that an app-specfic routes.py is all or nothing. If (1) selects an
app-specific routes.py and that routes.py does not contain (say) a
routes_out, we do *not* fall back on the base routes_out. Similarly for
default_*.
Once URL rewriting has been redirected to a specific app (as in step 2
or 3 above) do the following:
1. if app-specific routes_in exists check URL against app-specific
routes_in (if match found then rewrite else continue)
2. if URL maps to an existing controller/function, then call that
controller/function else continue
3. if default_controller specified, prepend default_controller to URL
and try step 2 else continue
4. if default_controller and default_function specified, prepend
default_controller/default_function to URL and try step 2 else
continue
5. if default_function specified, assume first part of URL is
controller, insert default_function after assumed controller and
before any potential function arguments
6. if nothing matches, return error
Obviously my assumptions were not entirely correct. I'm wondering if
you could pass along a brief overview of how the routes_app,
default_app, default_controller, and default_function parameters all
actually do interact in terms of URL rewriting.
Thanks again,
-Mike
On Aug 31, 5:34 pm, Jonathan Lundell jlund...@pobox.com wrote:
On Aug 31, 2010, at 2:20 PM, Michael Wolfe wrote:
That didn't seem to quite do it. Visitinghttp://domain.com/rewrites
tohttp://domain.com/my_app/default/index/instead of
http://domain.com/my_app/default/search/. The URL is being
substantively rewritten in the parse_url function (lines 802-807) of
gluon/main.py:
request.application = \
regex_space.sub('_', match.group('a') or
rewrite.params.default_application)
request.controller = \
regex_space.sub('_', match.group('c') or
rewrite.params.default_controller)
request.function = \
regex_space.sub('_', match.group('f') or
rewrite.params.default_function)
The problem being that rewrite.params.default_function is not using
the default_function specified in my app-specific routes.py.
The parse_url function is being called from line 326 of gluon/main.py
Re: [web2py] Using SOLR as the web2py backend
On Sep 2, 2010, at 2:05 PM, harryf wrote: This is kind of a re-post of a question I asked on stackoverflow - http://stackoverflow.com/questions/3630641/whats-the-most-productive-frontend-framework-to-use-with-solr-as-the-backend/ which Massimo invited me to ask here. To repeat, I want to build a web app using SOLR as the backend ( no RDBMS or other backend ). Most of the data will be stored in SOLR via offline jobs but there is some need for CRUD from the web app. The schema will probably move fairly slowly ( in fact it already exists ) so creating / changing models manually is acceptable from a maintenance point of view. Have only got so far as a web2py Hello World so don't have deep insight but would appreciate any hints on how this might be accomplished in web2py. Also if anyone has any general experience of using web2py with a RESTful backend as the primary data source, would be great to hear about it. My main web2py application is a manager for a collection of servers running my company's network services. The server API is xml-rpc, but I imagine that a RESTful API would work just as well. I have a local database for users, and another one that basically provides access info for the managed servers. The servers themselves I treat as a kind of distributed database. Not that I try to do SQL or anything, but in that they store their own data, and I don't try to keep local copies of it (except for backup restore, which is another matter). It works quite well. I encapsulate each managed server in its own object, and use threads so that I can talk to a bunch of them in parallel when I need to. I'm pleased with the way it turned out.
Re: [web2py] RESTful URLs
On Sep 3, 2010, at 9:28 PM, bally boy wrote: Hi is there a way to generate RESTful URLS. Let me explain:- 1. I have an app called 'app' 2. a controller called 'cont' 3. a function called 'func' Instead of being called by the url http://.../app/cont/func Can it be called like http://../func directly.. I am sure that it can be done ... just that I would have to spend some time on the docs.. can someone please just link me to the correct section http://web2py.com/book/default/chapter/04?search=routes.py URL Rewrite
Re: [web2py] What is polymodel?
On Sep 6, 2010, at 9:10 AM, Jose wrote: What is polymodel? http://code.google.com/appengine/articles/polymodel.html Do you use? How is it used?
Re: [web2py] Re: Routes.py on GAE
On Sep 9, 2010, at 6:27 AM, mdipierro wrote:
I do not understand where unable to import Rocket comes from.
That's a strange error. Also, the syntax error on routes.py is on line 3, which
is empty; there's no code until line 4.
My advice is to resolve the first error (Rocket) before worrying about the
routes error.
On Sep 9, 8:03 am, Richard richar...@gmail.com wrote:
I am using the following simple routes.py and find it works fine on
GAE with latest trunk:
routes_in = (
('/', '/init/default/index'),
('/topics', '/init/topics/index'),
)
routes_out = [(second, first) for (first, second) in routes_in]
On Sep 9, 9:49 pm, mdipierro mdipie...@cs.depaul.edu wrote:
i cannot reproduce this problem. Can you send me your routes?
On Sep 7, 1:28 am, Miguel Goncalves goncalvesmig...@gmail.com wrote:
Hi
I guess this bug has not been fixed yet?
I am getting the following error:
unable to import Rocket
Your routes.py has a syntax error Please fix it before you restart web2py
Traceback (most recent call last):
File
/base/data/home/apps/reviewround/1.344628390884008259/gluon/rewrite.py,
line 106, in load
exec routesfp.read() in symbols
File string, line 3
^
SyntaxError: invalid syntax
type 'exceptions.SyntaxError': invalid syntax (string, line 3)
Traceback (most recent call last):
File
/base/data/home/apps/reviewround/1.344628390884008259/gaehandler.py,
line 52, in module
import gluon.main
File
/base/data/home/apps/reviewround/1.344628390884008259/gluon/main.py,
line 66, in module
rewrite.load()
File
/base/data/home/apps/reviewround/1.344628390884008259/gluon/rewrite.py,
line 114, in load
raise e
type 'exceptions.SyntaxError': invalid syntax (string, line 3)
The routes.py looks like :
#!/usr/bin/python
# -*- coding: utf-8 -*-
default_application = 'reviewround' # ordinarily set in base routes.py
default_controller = 'default' # ordinarily set in app-specific
routes.py
default_function = 'index'
routes_in = ( ('/', '/reviewround/default/index'),)
routes_out = ( ('/reviewround/default/index', '/'),)
thanks
Miguel
Re: [web2py] Re: Routes.py on GAE
On Sep 9, 2010, at 7:49 AM, mdipierro wrote:
gaehanlder.py does not import Rocket.
main imports Rocket
Is it possible gaehandler.py or
app.yaml were modified?
On Sep 9, 9:40 am, Jonathan Lundell jlund...@pobox.com wrote:
On Sep 9, 2010, at 6:27 AM, mdipierro wrote:
I do not understand where unable to import Rocket comes from.
That's a strange error. Also, the syntax error on routes.py is on line 3,
which is empty; there's no code until line 4.
My advice is to resolve the first error (Rocket) before worrying about the
routes error.
On Sep 9, 8:03 am, Richard richar...@gmail.com wrote:
I am using the following simple routes.py and find it works fine on
GAE with latest trunk:
routes_in = (
('/', '/init/default/index'),
('/topics', '/init/topics/index'),
)
routes_out = [(second, first) for (first, second) in routes_in]
On Sep 9, 9:49 pm, mdipierro mdipie...@cs.depaul.edu wrote:
i cannot reproduce this problem. Can you send me your routes?
On Sep 7, 1:28 am, Miguel Goncalves goncalvesmig...@gmail.com wrote:
Hi
I guess this bug has not been fixed yet?
I am getting the following error:
unable to import Rocket
Your routes.py has a syntax error Please fix it before you restart web2py
Traceback (most recent call last):
File
/base/data/home/apps/reviewround/1.344628390884008259/gluon/rewrite.py,
line 106, in load
exec routesfp.read() in symbols
File string, line 3
^
SyntaxError: invalid syntax
type 'exceptions.SyntaxError': invalid syntax (string, line 3)
Traceback (most recent call last):
File
/base/data/home/apps/reviewround/1.344628390884008259/gaehandler.py,
line 52, in module
import gluon.main
File
/base/data/home/apps/reviewround/1.344628390884008259/gluon/main.py,
line 66, in module
rewrite.load()
File
/base/data/home/apps/reviewround/1.344628390884008259/gluon/rewrite.py,
line 114, in load
raise e
type 'exceptions.SyntaxError': invalid syntax (string, line 3)
The routes.py looks like :
#!/usr/bin/python
# -*- coding: utf-8 -*-
default_application = 'reviewround' # ordinarily set in base
routes.py
default_controller = 'default' # ordinarily set in app-specific
routes.py
default_function = 'index'
routes_in = ( ('/', '/reviewround/default/index'),)
routes_out = ( ('/reviewround/default/index', '/'),)
thanks
Miguel
Re: [web2py] Re: Routes.py on GAE
On Sep 9, 2010, at 9:17 AM, mdipierro wrote:
try:
import rocket
except:
logging.warn('unable to import Rocket')
True. This is supposed to fail on GAE. I will change it. No need to
try the import and issue a warning on GAE.
...which leaves the mystery of the routes syntax error.
I'd try it with the first three lines (two comments and blank line) removed,
and see what happens.
I'd also carefully check routes.py for invisible garbage characters, perhaps
from some word processor. Word's idea of a non-breaking space, for example.
If possible, zip the routes.py that you're uploading to GAE, and send me a copy.
On Sep 9, 9:57 am, Jonathan Lundell jlund...@pobox.com wrote:
On Sep 9, 2010, at 7:49 AM, mdipierro wrote:
gaehanlder.py does not import Rocket.
main imports Rocket
Is it possible gaehandler.py or
app.yaml were modified?
On Sep 9, 9:40 am, Jonathan Lundell jlund...@pobox.com wrote:
On Sep 9, 2010, at 6:27 AM, mdipierro wrote:
I do not understand where unable to import Rocket comes from.
That's a strange error. Also, the syntax error on routes.py is on line 3,
which is empty; there's no code until line 4.
My advice is to resolve the first error (Rocket) before worrying about the
routes error.
On Sep 9, 8:03 am, Richard richar...@gmail.com wrote:
I am using the following simple routes.py and find it works fine on
GAE with latest trunk:
routes_in = (
('/', '/init/default/index'),
('/topics', '/init/topics/index'),
)
routes_out = [(second, first) for (first, second) in routes_in]
On Sep 9, 9:49 pm, mdipierro mdipie...@cs.depaul.edu wrote:
i cannot reproduce this problem. Can you send me your routes?
On Sep 7, 1:28 am, Miguel Goncalves goncalvesmig...@gmail.com wrote:
Hi
I guess this bug has not been fixed yet?
I am getting the following error:
unable to import Rocket
Your routes.py has a syntax error Please fix it before you restart
web2py
Traceback (most recent call last):
File
/base/data/home/apps/reviewround/1.344628390884008259/gluon/rewrite.py,
line 106, in load
exec routesfp.read() in symbols
File string, line 3
^
SyntaxError: invalid syntax
type 'exceptions.SyntaxError': invalid syntax (string, line 3)
Traceback (most recent call last):
File
/base/data/home/apps/reviewround/1.344628390884008259/gaehandler.py,
line 52, in module
import gluon.main
File
/base/data/home/apps/reviewround/1.344628390884008259/gluon/main.py,
line 66, in module
rewrite.load()
File
/base/data/home/apps/reviewround/1.344628390884008259/gluon/rewrite.py,
line 114, in load
raise e
type 'exceptions.SyntaxError': invalid syntax (string, line 3)
The routes.py looks like :
#!/usr/bin/python
# -*- coding: utf-8 -*-
default_application = 'reviewround' # ordinarily set in base
routes.py
default_controller = 'default' # ordinarily set in
app-specific
routes.py
default_function = 'index'
routes_in = ( ('/', '/reviewround/default/index'),)
routes_out = ( ('/reviewround/default/index', '/'),)
thanks
Miguel
Re: [web2py] Re: Routes.py on GAE
On Sep 9, 2010, at 9:07 PM, Miguel Goncalves wrote:
in my case I was using the following routes.py
What I'm looking for is the file itself (preferably zipped). I'm wondering if
there might be something in it that isn't surviving a paste into email, because
I don' t seen anything in line 3 that would cause a syntax error. (Notice that
this is a Python syntax error, not a complaint from the rewrite code.)
#!/usr/bin/python
# -*- coding: utf-8 -*-
default_application = 'reviewround' # ordinarily set in base routes.py
default_controller = 'default' # ordinarily set in app-specific
routes.py
default_function = 'index'
routes_in = ( ('/', '/reviewround/default/index'),)
routes_out = ( ('/reviewround/default/index', '/'),)
-Miguel
On Thu, Sep 9, 2010 at 9:28 AM, Jonathan Lundell jlund...@pobox.com wrote:
On Sep 9, 2010, at 9:17 AM, mdipierro wrote:
try:
import rocket
except:
logging.warn('unable to import Rocket')
True. This is supposed to fail on GAE. I will change it. No need to
try the import and issue a warning on GAE.
...which leaves the mystery of the routes syntax error.
I'd try it with the first three lines (two comments and blank line) removed,
and see what happens.
I'd also carefully check routes.py for invisible garbage characters, perhaps
from some word processor. Word's idea of a non-breaking space, for example.
If possible, zip the routes.py that you're uploading to GAE, and send me a
copy.
On Sep 9, 9:57 am, Jonathan Lundell jlund...@pobox.com wrote:
On Sep 9, 2010, at 7:49 AM, mdipierro wrote:
gaehanlder.py does not import Rocket.
main imports Rocket
Is it possible gaehandler.py or
app.yaml were modified?
On Sep 9, 9:40 am, Jonathan Lundell jlund...@pobox.com wrote:
On Sep 9, 2010, at 6:27 AM, mdipierro wrote:
I do not understand where unable to import Rocket comes from.
That's a strange error. Also, the syntax error on routes.py is on line
3, which is empty; there's no code until line 4.
My advice is to resolve the first error (Rocket) before worrying about
the routes error.
On Sep 9, 8:03 am, Richard richar...@gmail.com wrote:
I am using the following simple routes.py and find it works fine on
GAE with latest trunk:
routes_in = (
('/', '/init/default/index'),
('/topics', '/init/topics/index'),
)
routes_out = [(second, first) for (first, second) in routes_in]
On Sep 9, 9:49 pm, mdipierro mdipie...@cs.depaul.edu wrote:
i cannot reproduce this problem. Can you send me your routes?
On Sep 7, 1:28 am, Miguel Goncalves goncalvesmig...@gmail.com wrote:
Hi
I guess this bug has not been fixed yet?
I am getting the following error:
unable to import Rocket
Your routes.py has a syntax error Please fix it before you restart
web2py
Traceback (most recent call last):
File
/base/data/home/apps/reviewround/1.344628390884008259/gluon/rewrite.py,
line 106, in load
exec routesfp.read() in symbols
File string, line 3
^
SyntaxError: invalid syntax
type 'exceptions.SyntaxError': invalid syntax (string, line 3)
Traceback (most recent call last):
File
/base/data/home/apps/reviewround/1.344628390884008259/gaehandler.py,
line 52, in module
import gluon.main
File
/base/data/home/apps/reviewround/1.344628390884008259/gluon/main.py,
line 66, in module
rewrite.load()
File
/base/data/home/apps/reviewround/1.344628390884008259/gluon/rewrite.py,
line 114, in load
raise e
type 'exceptions.SyntaxError': invalid syntax (string, line 3)
The routes.py looks like :
#!/usr/bin/python
# -*- coding: utf-8 -*-
default_application = 'reviewround' # ordinarily set in base
routes.py
default_controller = 'default' # ordinarily set in
app-specific
routes.py
default_function = 'index'
routes_in = ( ('/', '/reviewround/default/index'),)
routes_out = ( ('/reviewround/default/index', '/'),)
thanks
Miguel
[web2py] app-specific routing
Massimo has applied (after a false start on my part) the final portion of the app-specific routing patch. There are no API changes (unless you're directly looking at rewrite.params, in which case ask about the new interface), but the new code is thread-safe and should be good to go. Have a look at the web2py book or at the comments in routes.example.py for general information on how to use the feature. (This is not the next-gen URL-rewrite code, which is still a good intention merely. It's just the app-specific version of the existing URL-rewrite logic.)
Re: [web2py] Re: app-specific routing
On Sep 10, 2010, at 1:20 PM, _po wrote: Hi, I am having trouble with this patch. (rev d38a7099445b) The recently modified /gluon/streamer.py seems to prevent web2py from starting : Thanks; I'll have a look. $ hg up d38a7099445b 4 files updated, 0 files merged, 0 files removed, 0 files unresolved $ python web2py.py Traceback (most recent call last): File web2py.py, line 17, in module import gluon.widget File /media/DONNEES/Mes documents/Documents/Prog/web2py/gluon/ widget.py, line 28, in module import main File /media/DONNEES/Mes documents/Documents/Prog/web2py/gluon/ main.py, line 48, in module from globals import Request, Response, Session File /media/DONNEES/Mes documents/Documents/Prog/web2py/gluon/ globals.py, line 19, in module from streamer import streamer, stream_file_or_304_or_206, DEFAULT_CHUNK_SIZE File /media/DONNEES/Mes documents/Documents/Prog/web2py/gluon/ streamer.py, line 46, in module error_message = main.thread.routes.error_message % 'invalid request', AttributeError: 'thread._local' object has no attribute 'routes' On 10 sep, 19:10, Jonathan Lundell jlund...@pobox.com wrote: Massimo has applied (after a false start on my part) the final portion of the app-specific routing patch. There are no API changes (unless you're directly looking at rewrite.params, in which case ask about the new interface), but the new code is thread-safe and should be good to go. Have a look at the web2py book or at the comments in routes.example.py for general information on how to use the feature. (This is not the next-gen URL-rewrite code, which is still a good intention merely. It's just the app-specific version of the existing URL-rewrite logic.)
Re: [web2py] Re: app-specific routing
On Sep 10, 2010, at 1:20 PM, _po wrote: Hi, I am having trouble with this patch. (rev d38a7099445b) The recently modified /gluon/streamer.py seems to prevent web2py from starting : I found this problem (and another one), and will send Massimo a patch shortly. $ hg up d38a7099445b 4 files updated, 0 files merged, 0 files removed, 0 files unresolved $ python web2py.py Traceback (most recent call last): File web2py.py, line 17, in module import gluon.widget File /media/DONNEES/Mes documents/Documents/Prog/web2py/gluon/ widget.py, line 28, in module import main File /media/DONNEES/Mes documents/Documents/Prog/web2py/gluon/ main.py, line 48, in module from globals import Request, Response, Session File /media/DONNEES/Mes documents/Documents/Prog/web2py/gluon/ globals.py, line 19, in module from streamer import streamer, stream_file_or_304_or_206, DEFAULT_CHUNK_SIZE File /media/DONNEES/Mes documents/Documents/Prog/web2py/gluon/ streamer.py, line 46, in module error_message = main.thread.routes.error_message % 'invalid request', AttributeError: 'thread._local' object has no attribute 'routes' On 10 sep, 19:10, Jonathan Lundell jlund...@pobox.com wrote: Massimo has applied (after a false start on my part) the final portion of the app-specific routing patch. There are no API changes (unless you're directly looking at rewrite.params, in which case ask about the new interface), but the new code is thread-safe and should be good to go. Have a look at the web2py book or at the comments in routes.example.py for general information on how to use the feature. (This is not the next-gen URL-rewrite code, which is still a good intention merely. It's just the app-specific version of the existing URL-rewrite logic.)
Re: [web2py] Re: Scaffolding app with Twitter OAuth1.0a auth
Folks playing with Twitter OAuth might care to read this: http://arstechnica.com/security/guides/2010/09/twitter-a-case-study-on-how-to-do-oauth-wrong.ars/
Re: [web2py] TAG() throws a ticket when string too short
On Sep 12, 2010, at 8:57 AM, weheh wrote: I'm flattening some text, assuming it's html, and get this error message when the text input is less than 4 characters: input='abc' stream=cStringIO.StringIO(TAG(input).flatten()) What are you expecting this to generate? At first glance, it seems like an abuse of TAG(). File C:/web2py/applications/myapp/controllers/myfunc.py, line 311, in process_text_in stream=cStringIO.StringIO(TAG(input).flatten()) File C:\web2py\gluon\html.py, line 821, in __call__ return web2pyHTMLParser(decoder.decoder(html)).tree File C:\web2py\gluon\decoder.py, line 72, in decoder encoding = autoDetectXMLEncoding(buffer) File C:\web2py\gluon\decoder.py, line 38, in autoDetectXMLEncoding bytes = (byte1, byte2, byte3, byte4) = tuple(map(ord, buffer[0:4])) ValueError: need more than 3 values to unpack 4 or more characters works OK.
Re: [web2py] Re: Routes.py on GAE
On Sep 12, 2010, at 11:46 AM, Miguel Goncalves wrote:
I attached the zipped routed file to this email.
Thank you. It looks OK.
Miguel, if I send you a small patch for rewrite.py, to add a little debugging,
could you run it for me?
What I have in mind (and feel free to do this for yourself if you like) is
printing a hex dump of the relevant string, in the exception handler.
You could also try deleting the first three lines of routes.py (two comments
and a blank line) to see if that makes any difference.
The patch would be something like this. Here's the stock code:
try:
routesfp = open(path, 'r')
exec routesfp.read() in symbols
routesfp.close()
logger.info('URL rewrite is on. configuration in %s' % path)
except SyntaxError, e:
routesfp.close()
logger.error('Your %s has a syntax error ' % path + \
'Please fix it before you restart web2py\n' + \
traceback.format_exc())
raise e
The debug code:
try:
routesfp = open(path, 'r')
routestring = routesfp.read()
exec routestring in symbols
routesfp.close()
logger.info('URL rewrite is on. configuration in %s' % path)
except SyntaxError, e:
routesfp.close()
hex =
for c in routestring:
hex += 0123456789abcdef[ord(c)//16] +
0123456789abcdef[ord(c)%16] +
logger.error('Your %s has a syntax error ' % path + \
'Please fix it before you restart web2py\n' + \
'File: %s\n' % hex + \
traceback.format_exc())
raise e
thanks
Miguel
On Thu, Sep 9, 2010 at 9:24 PM, Jonathan Lundell jlund...@pobox.com wrote:
On Sep 9, 2010, at 9:07 PM, Miguel Goncalves wrote:
in my case I was using the following routes.py
What I'm looking for is the file itself (preferably zipped). I'm wondering if
there might be something in it that isn't surviving a paste into email,
because I don' t seen anything in line 3 that would cause a syntax error.
(Notice that this is a Python syntax error, not a complaint from the rewrite
code.)
#!/usr/bin/python
# -*- coding: utf-8 -*-
default_application = 'reviewround' # ordinarily set in base routes.py
default_controller = 'default' # ordinarily set in app-specific
routes.py
default_function = 'index'
routes_in = ( ('/', '/reviewround/default/index'),)
routes_out = ( ('/reviewround/default/index', '/'),)
-Miguel
On Thu, Sep 9, 2010 at 9:28 AM, Jonathan Lundell jlund...@pobox.com wrote:
On Sep 9, 2010, at 9:17 AM, mdipierro wrote:
try:
import rocket
except:
logging.warn('unable to import Rocket')
True. This is supposed to fail on GAE. I will change it. No need to
try the import and issue a warning on GAE.
...which leaves the mystery of the routes syntax error.
I'd try it with the first three lines (two comments and blank line) removed,
and see what happens.
I'd also carefully check routes.py for invisible garbage characters, perhaps
from some word processor. Word's idea of a non-breaking space, for example.
If possible, zip the routes.py that you're uploading to GAE, and send me a
copy.
On Sep 9, 9:57 am, Jonathan Lundell jlund...@pobox.com wrote:
On Sep 9, 2010, at 7:49 AM, mdipierro wrote:
gaehanlder.py does not import Rocket.
main imports Rocket
Is it possible gaehandler.py or
app.yaml were modified?
On Sep 9, 9:40 am, Jonathan Lundell jlund...@pobox.com wrote:
On Sep 9, 2010, at 6:27 AM, mdipierro wrote:
I do not understand where unable to import Rocket comes from.
That's a strange error. Also, the syntax error on routes.py is on line
3, which is empty; there's no code until line 4.
My advice is to resolve the first error (Rocket) before worrying about
the routes error.
On Sep 9, 8:03 am, Richard richar...@gmail.com wrote:
I am using the following simple routes.py and find it works fine on
GAE with latest trunk:
routes_in = (
('/', '/init/default/index'),
('/topics', '/init/topics/index'),
)
routes_out = [(second, first) for (first, second) in routes_in]
On Sep 9, 9:49 pm, mdipierro mdipie...@cs.depaul.edu wrote:
i cannot reproduce this problem. Can you send me your routes?
On Sep 7, 1:28 am, Miguel Goncalves goncalvesmig...@gmail.com
wrote:
Hi
I guess this bug has not been fixed yet?
I am getting the following error:
unable to import Rocket
Your routes.py has a syntax error Please fix it before you restart
web2py
Traceback (most recent call last):
File
/base/data/home/apps/reviewround/1.344628390884008259/gluon/rewrite.py,
line 106, in load
exec routesfp.read() in symbols
File string, line 3
^
SyntaxError: invalid syntax
type 'exceptions.SyntaxError': invalid syntax (string, line 3)
Traceback (most
