Re: [web2py] Re: Moving away from define_table
On 10 May 2012 22:10, Anthony abasta...@gmail.com wrote: You can turn off migrations globally for the entire db connection via DAL(..., migrate_enabled=False). Anthony p.s., Even people with a programming background find the DAL useful. :-) I agree. It is useful, but also limiting: e.g. To work with SQLFORM.grid/smartgrid/plugin_wiki's jqgrid you depend on DAL. I very often find that for complex queries I spend a lot of time experimenting to get the DAL version producing the same results as the raw SQL and there are cases where you just cannot do in DAL what can be done in SQL. In such case the use of tools like grid/smartgid is not possible in this stage. Regards Johann -- Because experiencing your loyal love is better than life itself, my lips will praise you. (Psalm 63:3)
Re: [web2py] Load data from sqlite file?
On 11 May 2012 01:38, Dave davidramsayreinh...@gmail.com wrote: Any examples or suggestions on how to load data from a user-uploaded .sqlite database? I will have the database structure of the uploaded files ahead of time (they are being generated by a mobile app), so I need a way to transfer the data from an uploaded .sqlite file to my web2py tables. is it possible to just use the uploaded file as a new dal, then transfer the data to the main dal, all within a function? If the structure of the database file is compatible with your DAL setup, try and use it as a second database. If it does not work, open it with a tool like SQLITEBROWSER, export the tables to csv-files and import it on the other side. Regards Johann -- Because experiencing your loyal love is better than life itself, my lips will praise you. (Psalm 63:3)
Re: [web2py] Need a little help in code review (a function eating up all memory)
The problem is not that if I call the controller function if consumes
memory. My problem is that there is something that still reference
something after the execution is finished so the consumed memory never get
released / reused...
2012. május 10., csütörtök 23:27:32 UTC+2 időpontban Martin.Mulone a
következőt írta:
The problem is fetchall(), is getting all the records to memory.
If you want to iterate a large dataset or not so heavy, but have a table
with many fields, perhaps you can do it by blocks.
For example using raw sql with dal: example: this fetch by 1000 records
queryraw = select * from mytable %(limit_query)s
def records(query):
rows = db.executesql(query)
if rows:
return True
else:
return False
rec = 0
stage = 0
block = 1000while True:
offset = stage * block
query = queryraw % {'limitquery': 'LIMIT %s, %s' % (offset, block)}
if not records(query):
break
rows = db.executesql(query)
for row in rows:
rec += 1
# do something
stage += 1
This is not fast but keep memory low.
2012/5/10 szimszon szims...@gmail.com
Could it be related?
https://groups.google.com/forum/#!topic/web2py/hmsupVHdDHo/discussion
(Memory
leak in standalone DAL (issue #731), can you please help test?)
2012. május 10., csütörtök 22:32:53 UTC+2 időpontban szimszon a
következőt írta:
Okay. It's clear.
I'm only puzzled about why the memory didn't get freed or reused after
execution is finished. And if I execute the controller function in 1-2min
interval mem is still not reused.
So I understand it can eat up the memory but why is all memory locked
forever and didn't get reused - in my understanding (it's not much) in my
case GC do not free mem at all in python level. So some reference is still
intact after function is finished.
2012. május 10., csütörtök 21:53:06 UTC+2 időpontban Richard a
következőt írta:
Ok, you don't need it to works all the time.
Did you get it to update your records?
If not, and if as you said it is a one trip, you can just treat the
whole records batch by batch...
Look here : http://web2py.com/books/**default/chapter/29/14#**
Populating-database-with-**dummy-datahttp://web2py.com/books/default/chapter/29/14#Populating-database-with-dummy-data
for i in range(10):
populate(db.mytable,100)
db.commit()
In this example only 100 records are populated at a time between
db.commit()... So maybe you just have to wrap for loop that will
db.commit() a couples of time during your processing.
Richard
On Thu, May 10, 2012 at 3:32 PM, szimszon szims...@gmail.com wrote:
I had to store files and a lot of properties for it. It was in csv.
But after I processed it we figured out that not all value was correct in
csv but it was a bit redundant. So I can correct it by go through all the
records row by row. So that was a one time trip.
I just realized after the process I had no memory left. So now I'm
investigating what happened...
2012. május 10., csütörtök 21:00:05 UTC+2 időpontban Richard a
következőt írta:
Yes but in this case it is not for the entire reecords...
Why would you return a full list of all the records?
I don't understand what is the purpose of listar that you return in
the view under a html table, why do you need to return all the 10+
entries?
Richard
On Thu, May 10, 2012 at 2:56 PM, szimszon szims...@gmail.com wrote:
In book it is a recommended way to iterate over sql results:
http://web2py.com/books/**defaul**t/chapter/29/6http://web2py.com/books/default/chapter/29/6
You can do all the steps in one statement:
1.
2.
3.
for row in db(db.person.name=='Alex').select():
print row.name
Alex
2012. május 10., csütörtök 20:42:22 UTC+2 időpontban Bruce Wade a
következőt írta:
Sorry, you really need to read more about how python works. If you
learn how for loops work and memory you will understand the problem.
One solution do the query before the for loop then loop through the
objects. This may help at bit. Research xrange vs range
On Thu, May 10, 2012 at 11:30 AM, szimszon szims...@gmail.comwrote:
Sorry I don't understand. What do you mean achieve with join?
There is an empty for loop with db.executesql() without join. And
it is eating up the memory. :(
2012. május 10., csütörtök 19:12:30 UTC+2 időpontban Richard a
következőt írta:
You can't manage what you want to achieve with join?
Richard
On Thu, May 10, 2012 at 10:48 AM, szimszon szims...@gmail.comwrote:
Sorry for my dumbness but if something is wrong with my code
please point me the right line. I'm not so good in English if it
comes to
object instance count and so. Yeah I know I should go and do some
milkmaid job :) but I'm curious.
I'm just define some variable:
lista = list()
last_row = None
next_page_number = 0
Go in a for loop that just assign the db
Re: [web2py] Need a little help in code review (a function eating up all memory)
OK. Finally I have it :)
It was only the amount of memory it was consumed each time. I tried it with
a smaller but a significant number of records and finally I figured out
that there is a limit and after that memory consumption is ok. I had only
reach that count of browser request :-D
Sorry for your time and thanks for help. =--=
2012. május 11., péntek 9:30:28 UTC+2 időpontban szimszon a következőt írta:
The problem is not that if I call the controller function if consumes
memory. My problem is that there is something that still reference
something after the execution is finished so the consumed memory never get
released / reused...
2012. május 10., csütörtök 23:27:32 UTC+2 időpontban Martin.Mulone a
következőt írta:
The problem is fetchall(), is getting all the records to memory.
If you want to iterate a large dataset or not so heavy, but have a table
with many fields, perhaps you can do it by blocks.
For example using raw sql with dal: example: this fetch by 1000 records
queryraw = select * from mytable %(limit_query)s
def records(query):
rows = db.executesql(query)
if rows:
return True
else:
return False
rec = 0
stage = 0
block = 1000while True:
offset = stage * block
query = queryraw % {'limitquery': 'LIMIT %s, %s' % (offset, block)}
if not records(query):
break
rows = db.executesql(query)
for row in rows:
rec += 1
# do something
stage += 1
This is not fast but keep memory low.
2012/5/10 szimszon szims...@gmail.com
Could it be related?
https://groups.google.com/forum/#!topic/web2py/hmsupVHdDHo/discussion
(Memory
leak in standalone DAL (issue #731), can you please help test?)
2012. május 10., csütörtök 22:32:53 UTC+2 időpontban szimszon a
következőt írta:
Okay. It's clear.
I'm only puzzled about why the memory didn't get freed or reused after
execution is finished. And if I execute the controller function in 1-2min
interval mem is still not reused.
So I understand it can eat up the memory but why is all memory locked
forever and didn't get reused - in my understanding (it's not much) in my
case GC do not free mem at all in python level. So some reference is still
intact after function is finished.
2012. május 10., csütörtök 21:53:06 UTC+2 időpontban Richard a
következőt írta:
Ok, you don't need it to works all the time.
Did you get it to update your records?
If not, and if as you said it is a one trip, you can just treat the
whole records batch by batch...
Look here : http://web2py.com/books/**default/chapter/29/14#**
Populating-database-with-**dummy-datahttp://web2py.com/books/default/chapter/29/14#Populating-database-with-dummy-data
for i in range(10):
populate(db.mytable,100)
db.commit()
In this example only 100 records are populated at a time between
db.commit()... So maybe you just have to wrap for loop that will
db.commit() a couples of time during your processing.
Richard
On Thu, May 10, 2012 at 3:32 PM, szimszon szims...@gmail.com wrote:
I had to store files and a lot of properties for it. It was in csv.
But after I processed it we figured out that not all value was correct
in
csv but it was a bit redundant. So I can correct it by go through all
the
records row by row. So that was a one time trip.
I just realized after the process I had no memory left. So now I'm
investigating what happened...
2012. május 10., csütörtök 21:00:05 UTC+2 időpontban Richard a
következőt írta:
Yes but in this case it is not for the entire reecords...
Why would you return a full list of all the records?
I don't understand what is the purpose of listar that you return in
the view under a html table, why do you need to return all the 10+
entries?
Richard
On Thu, May 10, 2012 at 2:56 PM, szimszon szims...@gmail.comwrote:
In book it is a recommended way to iterate over sql results:
http://web2py.com/books/**defaul**t/chapter/29/6http://web2py.com/books/default/chapter/29/6
You can do all the steps in one statement:
1.
2.
3.
for row in db(db.person.name=='Alex').select():
print row.name
Alex
2012. május 10., csütörtök 20:42:22 UTC+2 időpontban Bruce Wade a
következőt írta:
Sorry, you really need to read more about how python works. If you
learn how for loops work and memory you will understand the problem.
One solution do the query before the for loop then loop through
the objects. This may help at bit. Research xrange vs range
On Thu, May 10, 2012 at 11:30 AM, szimszon szims...@gmail.comwrote:
Sorry I don't understand. What do you mean achieve with join?
There is an empty for loop with db.executesql() without join. And
it is eating up the memory. :(
2012. május 10., csütörtök 19:12:30 UTC+2 időpontban Richard a
következőt írta:
You can't manage what you want to achieve with join?
Richard
On Thu, May 10,
Re: [web2py] Need a little help in code review (a function eating up all memory)
I had only to reach that count of browser request :-D
[web2py] Changing the controller on the fly
I need to be able to dispatch to a different controller based on a database
lookup. So a user will go to a url (say '/dispatch'), and we'll look up in
the database some information on that user, choose a new controller and
function, and call that controller and function with its view.
I've almost got this working below, but the models are not being loaded
into the new controller. Is there a way to fix that?
In default.py:
def dispatch():
controller,function = ... load these from the database ...
response.view = '%s/%s.html' % (controller,
function)
if not os.path.exists(request.folder + '/views/' + response.view):
response.view = 'generic.html'
from gluon.shell import exec_environment
controller = exec_environment('%s/controllers/%s.py'
% (request.folder,
controller),
request=request,
response=response,
session=session)
return controller[request.task_function]()
Unfortunately, the controller being called has access to request, response,
and session, but none of the global variables defined in my models. Is
there a way to get exec_environment() to run a function in another
controller WITHOUT losing all the model definitions?
Or is there a better way to do this?
[web2py] Re: Changing the controller on the fly
You can do:
def dispatch():
controller,function = ... load these from the database ...
redirect(URL(c=controller, f=function, vars=request.vars,
args=request.args))
On Friday, 11 May 2012 10:17:19 UTC+1, Michael Toomim wrote:
I need to be able to dispatch to a different controller based on a
database lookup. So a user will go to a url (say '/dispatch'), and we'll
look up in the database some information on that user, choose a new
controller and function, and call that controller and function with its
view.
I've almost got this working below, but the models are not being loaded
into the new controller. Is there a way to fix that?
In default.py:
def dispatch():
controller,function = ... load these from the database ...
response.view = '%s/%s.html' % (controller,
function)
if not os.path.exists(request.folder + '/views/' + response.view):
response.view = 'generic.html'
from gluon.shell import exec_environment
controller = exec_environment('%s/controllers/%s.py'
% (request.folder,
controller),
request=request,
response=response,
session=session)
return controller[request.task_function]()
Unfortunately, the controller being called has access to request,
response, and session, but none of the global variables defined in my
models. Is there a way to get exec_environment() to run a function in
another controller WITHOUT losing all the model definitions?
Or is there a better way to do this?
[web2py] Re: Load data from sqlite file?
http://web2py.com/books/default/chapter/29/6#Legacy-databases-and-keyed-tables On Friday, 11 May 2012 00:38:41 UTC+1, Dave wrote: Any examples or suggestions on how to load data from a user-uploaded .sqlite database? I will have the database structure of the uploaded files ahead of time (they are being generated by a mobile app), so I need a way to transfer the data from an uploaded .sqlite file to my web2py tables. is it possible to just use the uploaded file as a new dal, then transfer the data to the main dal, all within a function? Thanks in advance, Dave
[web2py] Re: MongoDB Adapter error in select
Thanks Massimo. Please let me know if you need more help in debugging it
On Friday, May 11, 2012 12:40:35 AM UTC+1, Massimo Di Pierro wrote:
Now I understand better where all these problems come from. I shall fix it
tonight.
On Thursday, 10 May 2012 18:02:24 UTC-5, Francisco Costa wrote:
i didn't understand your question..
This is the complete code
import sys
import time
from gluon.dal import DAL, Field
mongo = DAL('mongodb://localhost:27017/sapo')
mongo.define_table('user',
Field('name', 'text'),
Field('age', 'integer'),
Field('city', 'string')
)
def insert_users():
mongo.user.insert(name='John', age=66, city='Toronto')
mongo.user.insert(name='Mark', age=43, city='Boston')
mongo.user.insert(name='Tom', age=43, city='Detroit')
mongo.user.insert(name='Jim', age=18, city='Detroit')
mongo.user.insert(name='Jack', age=18)
mongo.user.insert(name='Eric', city='Boston')
return 'users in database'
def find_users():
users = mongo(mongo.user.age==18).select()
return dict(users=users)
[web2py] No error_message displayed when using IS_LIST_OF(IS_ALPHANUMERIC()) validator
Hi,
I have no error message when I wrongly fill a list with spaces / special
characters.
However the validator works, as I need to respect it to update the record.
db.define_table('parameter',
Field('allowed_users', type='list:string'),
)
db.parameter.allowed_users.requires = IS_LIST_OF(IS_ALPHANUMERIC())
# I first inserted a row with the default DB administration tool. But no
error message also there.
form = crud.update(db.parameter, 1, deletable=False)
I did try quite everything already :
- Redefining error_message at the IS_ALPHANUMERIC level or at the IS_LISTOF.
- defining the validator at the table level
- Removing the IS_LIST_OF..
But no success, it 's validating, but no error message displayed. This then
becomes quite unusable.
[web2py] Re: SQLFORM.smartgrid : error when setting fields to be displayed as a fields dictionary.
Sorry Massimo, I didn't found any Bug Reporting Tool on the web2py web site or on the presentation message of this list. Where can I report the bug ? Thanks, Le jeudi 10 mai 2012 02:08:31 UTC+2, François Delpierre a écrit : Hi, When I try to select the fields to be displayed as follow : form = SQLFORM.smartgrid( db.t_xlsfile, fields=dict( t_xlsfile=[ db.t_xlsfile.f_xlsfile, ], ), I immediately get this strange error : Traceback (most recent call last): File /home/www-data/web2py/gluon/restricted.py, line 205, inrestricted exec ccode in environment File /home/www-data/web2py/applications/init/controllers/default.py,line 425, in module File /home/www-data/web2py/gluon/globals.py, line 173, in lambda self._caller = lambda f: f() File /home/www-data/web2py/gluon/tools.py, line 2575, in f return action(*a, **b) File /home/www-data/web2py/applications/init/controllers/default.py,line 257, in posting_history t_xlsfile='Table of Excel files', File /home/www-data/web2py/gluon/sqlhtml.py, line 1989, in smartgrid user_signature=user_signature,**kwargs) File /home/www-data/web2py/gluon/sqlhtml.py, line 1517, in grid if field._tablename in tablenames] AttributeError: 'str' object has no attribute '_tablename' Any idea ??
[web2py] [Fixed] SQLFORM.smartgrid : error when setting fields to be displayed as a fields dictionary.
Le jeudi 10 mai 2012 02:08:31 UTC+2, François Delpierre a écrit : Hi, When I try to select the fields to be displayed as follow : form = SQLFORM.smartgrid( db.t_xlsfile, fields=dict( t_xlsfile=[ db.t_xlsfile.f_xlsfile, ], ), I immediately get this strange error : Traceback (most recent call last): File /home/www-data/web2py/gluon/restricted.py, line 205, inrestricted exec ccode in environment File /home/www-data/web2py/applications/init/controllers/default.py,line 425, in module File /home/www-data/web2py/gluon/globals.py, line 173, in lambda self._caller = lambda f: f() File /home/www-data/web2py/gluon/tools.py, line 2575, in f return action(*a, **b) File /home/www-data/web2py/applications/init/controllers/default.py,line 257, in posting_history t_xlsfile='Table of Excel files', File /home/www-data/web2py/gluon/sqlhtml.py, line 1989, in smartgrid user_signature=user_signature,**kwargs) File /home/www-data/web2py/gluon/sqlhtml.py, line 1517, in grid if field._tablename in tablenames] AttributeError: 'str' object has no attribute '_tablename' Any idea ??
[web2py] Re: problem related to GAE server
I don't i can say this problem but in GAE it accept only .png file and other expect .jpg i don't get the perfect solution. On Thursday, 10 May 2012 11:33:17 UTC+5:30, Prakhar Srivastava wrote: what is the max image size we can upload through the img tag ? Because my application is working fine the local server and rock server when i upload my application on GAE server then it's not showing properly www.pixelofn.appspot.com this is my application actually it stack because he one photo should appear in the middle then navigation come from right hand side. HOW CAN I SOLVE THIS PROBLEM ???
[web2py] Re: GAE
I can't say this problem solve but in GAE it accept only .png file and other expect .jpg i don't get the perfect solution. On Thursday, 10 May 2012 12:06:08 UTC+5:30, Prakhar Srivastava wrote: when i upload my application this waring come up 2012-05-09 23:02:24.992 /pixelofn/static/images/album/thumbs/PixelOfNature_fm_03.jpg 404 27ms 0kb 223.183.189.220 - - [09/May/2012:23:02:24 -0700] GET /pixelofn/static/images/album/thumbs/PixelOfNature_fm_03.jpg HTTP/1.1 404 0 http://www.pixelofn.appspot.com/; - www.pixelofn.appspot.com ms=27 cpu_ms=0 api_cpu_ms=0 cpm_usd=0.74 W 2012-05-09 23:02:24.992 Static file referenced by handler not found: applications/pixelofn/static/images/album/thumbs/PixelOfNature_fm_03.jpg
[web2py] Re: Changing the controller on the fly
Or to avoid a redirect, you can change the function and controller in a
model file:
db = DAL(...)
if request.function == 'dispatch':
request.controller, request.function = [fetch from db]
response.view = '%s/%s.%s' % (request.controller, request.function,request
.extension)
response.generic_patterns = ['html'] # to enable the generic.html view
if needed
Anthony
On Friday, May 11, 2012 6:07:56 AM UTC-4, simon wrote:
You can do:
def dispatch():
controller,function = ... load these from the database ...
redirect(URL(c=controller, f=function, vars=request.vars,
args=request.args))
On Friday, 11 May 2012 10:17:19 UTC+1, Michael Toomim wrote:
I need to be able to dispatch to a different controller based on a
database lookup. So a user will go to a url (say '/dispatch'), and we'll
look up in the database some information on that user, choose a new
controller and function, and call that controller and function with its
view.
I've almost got this working below, but the models are not being loaded
into the new controller. Is there a way to fix that?
In default.py:
def dispatch():
controller,function = ... load these from the database ...
response.view = '%s/%s.html' % (controller,
function)
if not os.path.exists(request.folder + '/views/' + response.view):
response.view = 'generic.html'
from gluon.shell import exec_environment
controller = exec_environment('%s/controllers/%s.py'
% (request.folder,
controller),
request=request,
response=response,
session=session)
return controller[request.task_function]()
Unfortunately, the controller being called has access to request,
response, and session, but none of the global variables defined in my
models. Is there a way to get exec_environment() to run a function in
another controller WITHOUT losing all the model definitions?
Or is there a better way to do this?
[web2py] Re: [Fixed] SQLFORM.smartgrid : error when setting fields to be displayed as a fields dictionary.
here http://code.google.com/p/web2py/issues/list Il giorno venerdì 11 maggio 2012 13:32:21 UTC+2, François Delpierre ha scritto: Le jeudi 10 mai 2012 02:08:31 UTC+2, François Delpierre a écrit : Hi, When I try to select the fields to be displayed as follow : form = SQLFORM.smartgrid( db.t_xlsfile, fields=dict( t_xlsfile=[ db.t_xlsfile.f_xlsfile, ], ), I immediately get this strange error : Traceback (most recent call last): File /home/www-data/web2py/gluon/restricted.py, line 205, inrestricted exec ccode in environment File /home/www-data/web2py/applications/init/controllers/default.py,line 425, in module File /home/www-data/web2py/gluon/globals.py, line 173, in lambda self._caller = lambda f: f() File /home/www-data/web2py/gluon/tools.py, line 2575, in f return action(*a, **b) File /home/www-data/web2py/applications/init/controllers/default.py,line 257, in posting_history t_xlsfile='Table of Excel files', File /home/www-data/web2py/gluon/sqlhtml.py, line 1989, in smartgrid user_signature=user_signature,**kwargs) File /home/www-data/web2py/gluon/sqlhtml.py, line 1517, in grid if field._tablename in tablenames] AttributeError: 'str' object has no attribute '_tablename' Any idea ??
[web2py] To uuid or not?
I developed my original app to use uuid-crossreferences in stead of the normal 'id'-field. The reason was that I wanted it to be consistent when I move the database to another computer. Now I am rewriting the app to make it more efficient and to make the code cleaner - applying some new features that came into web2py recently - e.g. archiving. I am considering it to drop the dependence on uuid-references and move back to id-based references and to keep the present references in the database valid I am thinking (as an experiment) to do it in the following way: * add a computed field to each table that use uuid-based references. This computation will then lookup the id in the other table based on the uuid-reference. * after confirming that the references work well, I plan to drop the uuid-fields and uuid-based reference fields from each table using them. and change the computed field to normal reference-field. * remove archiving methods based on uuid which I have developed and use the new archiving in the trunk. Am I overlooking something? I would like the opinion of some experts in this list on what I have in mind please. Is there still a place for uuid-based references? When? Regards Johann -- Because experiencing your loyal love is better than life itself, my lips will praise you. (Psalm 63:3)
[web2py] colorific
This may be useful to automate css creation form logo images: http://99designs.com/tech-blog/blog/2012/05/11/color-analysis/
[web2py] Re: No error_message displayed when using IS_LIST_OF(IS_ALPHANUMERIC()) validator
Yes, this is a known issue when applying IS_LIST_OF to list: type fields.
Previously, you could not even use the IS_LIST_OF validator with list:
fields -- we added support for that case but did not work out the
displaying of errors on the form (which is a bit tricky given how the list:
widget works). So, if you want to use that validator, you are responsible
for explicitly checking for errors post validation and displaying the
errors yourself (after a failed validation, the errors will be in
form.errors, so you can pull them from there and decide how you want to
show them on the form).
Anthony
On Friday, May 11, 2012 7:27:08 AM UTC-4, François Delpierre wrote:
Hi,
I have no error message when I wrongly fill a list with spaces / special
characters.
However the validator works, as I need to respect it to update the record.
db.define_table('parameter',
Field('allowed_users', type='list:string'),
)
db.parameter.allowed_users.requires = IS_LIST_OF(IS_ALPHANUMERIC())
# I first inserted a row with the default DB administration tool. But no
error message also there.
form = crud.update(db.parameter, 1, deletable=False)
I did try quite everything already :
- Redefining error_message at the IS_ALPHANUMERIC level or at the
IS_LISTOF.
- defining the validator at the table level
- Removing the IS_LIST_OF..
But no success, it 's validating, but no error message displayed. This
then becomes quite unusable.
Re: [web2py] Re: Nginx-uwsgi problem.
Bruce, Are you building dict with query for represent or other use? Recently I solve speed problem I had by caching dict building query. I never thought that building a dictionary could be that expensive in term of cpu load. Richard On Thu, May 10, 2012 at 5:13 PM, Bruce Wade bruce.w...@gmail.com wrote: Yes there are a LOT of wait state on the web2py nodes and high CPU I will try your suggestions. Thanks, Bruce On Thu, May 10, 2012 at 2:10 PM, Michele Comitini michele.comit...@gmail.com wrote: The high load on web2py nodes seems to point to code in web2py. If it were a problem with postgres you would have a high load on postgresql and a lot of wait state and little CPU time resulting in little uptime on web2py nodes but long page rendering times. I suggest to try to convert some logic to use raw resultsets using executesql instead of DAL Row objects. But before doing that try the query on postgres directly: you can use the _select() method to obtain the query generated by the DAL. If postgresql answers slowly try adding indexes on columns as requested by EXPLAIN. If postgresql answers fast try the guilty query with the DAL in a python shell (i.e. python web2py.py -M -S appname). If it slow than you have found the cause. Else keep using top to find if other processes are infesting the CPU maybe it is a simple problem of ping pong or swappiness. Simple tuning of uWSGI could suffice. As a rule of thumb you should not have the number of web2py processes be more than twice the number of cores. mic 2012/5/10 Bruce Wade bruce.w...@gmail.com: Web2py is on 3 different servers/nodes, postgresql is on it's own node with 8GB ram. CPU is being used by uwsgi so web2py. The slowness I think is from DB queries as when you load a page without the DB involved much it loads quickly The serving ads part is not a problem it is the other pages on the website. At least not the adviewer the banner ads are new. The adviewer has served over 29 million ads. I will try disabling the banner ads for now and set them so they are querying from a completely different server, maybe using mongodb and node.js On Thu, May 10, 2012 at 11:28 AM, pbreit pbreitenb...@gmail.com wrote: Is your traffic from serving ads or users coming to your web site? Have you exhausted caching opportunities? -- -- Regards, Bruce Wade http://ca.linkedin.com/in/brucelwade http://www.wadecybertech.com http://www.fittraineronline.com - Fitness Personal Trainers Online http://www.warplydesigned.com -- -- Regards, Bruce Wade http://ca.linkedin.com/in/brucelwade http://www.wadecybertech.com http://www.fittraineronline.com - Fitness Personal Trainers Online http://www.warplydesigned.com
Re: [web2py] Re: Nginx-uwsgi problem.
Maybe in some places of the code but not everywhere. The problem is when there is a large load all 3 servers get very slow on every page. I think it is the DB layer as we have 90 tables in one database 45 in another. I am also using connection pooling which I think is causing problems. Because the DAL is loaded with every request, wouldn't that means the pool is open for each request? Or should there only ever be 10 connections open even if I have 1000 concurrent connections? On Fri, May 11, 2012 at 6:51 AM, Richard Vézina ml.richard.vez...@gmail.com wrote: Bruce, Are you building dict with query for represent or other use? Recently I solve speed problem I had by caching dict building query. I never thought that building a dictionary could be that expensive in term of cpu load. Richard On Thu, May 10, 2012 at 5:13 PM, Bruce Wade bruce.w...@gmail.com wrote: Yes there are a LOT of wait state on the web2py nodes and high CPU I will try your suggestions. Thanks, Bruce On Thu, May 10, 2012 at 2:10 PM, Michele Comitini michele.comit...@gmail.com wrote: The high load on web2py nodes seems to point to code in web2py. If it were a problem with postgres you would have a high load on postgresql and a lot of wait state and little CPU time resulting in little uptime on web2py nodes but long page rendering times. I suggest to try to convert some logic to use raw resultsets using executesql instead of DAL Row objects. But before doing that try the query on postgres directly: you can use the _select() method to obtain the query generated by the DAL. If postgresql answers slowly try adding indexes on columns as requested by EXPLAIN. If postgresql answers fast try the guilty query with the DAL in a python shell (i.e. python web2py.py -M -S appname). If it slow than you have found the cause. Else keep using top to find if other processes are infesting the CPU maybe it is a simple problem of ping pong or swappiness. Simple tuning of uWSGI could suffice. As a rule of thumb you should not have the number of web2py processes be more than twice the number of cores. mic 2012/5/10 Bruce Wade bruce.w...@gmail.com: Web2py is on 3 different servers/nodes, postgresql is on it's own node with 8GB ram. CPU is being used by uwsgi so web2py. The slowness I think is from DB queries as when you load a page without the DB involved much it loads quickly The serving ads part is not a problem it is the other pages on the website. At least not the adviewer the banner ads are new. The adviewer has served over 29 million ads. I will try disabling the banner ads for now and set them so they are querying from a completely different server, maybe using mongodb and node.js On Thu, May 10, 2012 at 11:28 AM, pbreit pbreitenb...@gmail.com wrote: Is your traffic from serving ads or users coming to your web site? Have you exhausted caching opportunities? -- -- Regards, Bruce Wade http://ca.linkedin.com/in/brucelwade http://www.wadecybertech.com http://www.fittraineronline.com - Fitness Personal Trainers Online http://www.warplydesigned.com -- -- Regards, Bruce Wade http://ca.linkedin.com/in/brucelwade http://www.wadecybertech.com http://www.fittraineronline.com - Fitness Personal Trainers Online http://www.warplydesigned.com -- -- Regards, Bruce Wade http://ca.linkedin.com/in/brucelwade http://www.wadecybertech.com http://www.fittraineronline.com - Fitness Personal Trainers Online http://www.warplydesigned.com
[web2py] web2py - form to pass input to matplotlib
Hi All,
i'm tring to learn web2py, thanks for the wonderful book and for the really
nice tutorial!
i'm a bit 'stuck' on how to perform a simple task (i guess it is simple)
because of my ignorance.
.. i'm bring to update dynamic a graph-chart (generated with matplotlib) using
a form to pass the input the draw-function.
what i have is a controller :
# template_example.py that 'espone' a function 'variables' , it will display
an image generated by matplotlib using the 'stem_plot' function (in the same
.py file)
# import a lib that generate the data from the url gived as input
from ecoopclimate import Climate
#import some matplotlib methodsm numpy ...
from pylab import stem, setp, grid, savefig, show, gca, subplot, subplots_adjust
import matplotlib as mpl
import matplotlib.pyplot as plt
import datetime as dt
import numpy as np
# plot function
def stem_plot(data, name):
data[np.isnan(data)]=0
fig = plt.figure()
ax = fig.add_subplot(211)
mindate = int(data[0][0])
maxdate = int(data[0][-1])
date2_1 = dt.datetime(mindate, 1, 1)
date2_2 = dt.datetime(maxdate, 1, 1)
delta2 = dt.timedelta(days=365)
dates2 = mpl.dates.drange(date2_1, date2_2, delta2)
dateFmt = mpl.dates.DateFormatter('%Y')
ax.xaxis.set_major_formatter(dateFmt)
fig.autofmt_xdate(bottom=0.1)
x_p = dates2[np.where(data[1]=0)[0]]
y_p = data[1][np.where(data[1]=0)[0]]
x_n = dates2[np.where(data[1]0)[0]]
y_n = data[1][np.where(data[1]0)[0]]
markerline, stemlines, baseline = stem(x_p, y_p, 'r-')
setp(markerline, 'markerfacecolor', 'b')
setp(baseline, 'color', 'r', 'linewidth', 2)
setp(stemlines, 'linewidth', 1)
markerline, stemlines, baseline = stem(x_n, y_n, 'b-')
setp(markerline, 'markerfacecolor', 'b')
setp(baseline, 'color', 'r', 'linewidth', 2)
setp(stemlines, 'linewidth', 1)
grid()
setp(gca(), 'xlabel', 'Year', 'ylabel', name)
fig = '/Users/epifanio/web2py/applications/welcome/static/'+name+'.png'
dateFmt = mpl.dates.DateFormatter('%Y')
savefig(fig)
return fig
#show()
# function connected to the view : variables.html
def variables():
nao_url =
'https://climatedataguide.ucar.edu/sites/default/files/cas_data_files/asphilli/nao_station_djfm.txt'
clm = Climate()
NAO = clm.nao(nao_url, 'nao.txt')
image = stem_plot(NAO['nao'], 'nao')
return dict(d=URL('static', 'nao.png'))
the view template_example/variables.html is :
{{extend 'layout.html'}}
h1Image generated with matplotlib/h1
img src={{=d}} alt=Red dot /
As you can see from the code in stem_plot it takes as input : 'data' and 'name'
i'm hardcoding the var inside the function 'variables'
i need to modify that code adding a 'form' in the view that will take 2 string
as input
nao_url =
'https://climatedataguide.ucar.edu/sites/default/files/cas_data_files/asphilli/nao_station_djfm.txt'
image_name = 'imagename'
and pass this input to the main function that generate the graph.
Have you any hints on how to do that ?
i'd like to have the form in the same page where the image is displayed
so that i can update the image every time i'll resubmit the form.
i'm a beginner, an example that does something like that will be really helpful
:)
Many many Thanks for your help!
Massimo.
[web2py] denormalizing a list:reference -- need a bit of help
db.define_table('entity', Field('name'), format='%(name)s')
auth_user_table = db.define_table(
auth.settings.table_user_name,
...
Field('entity', 'list:reference db.entity',
requires=IS_IN_DB(db, 'entity.id','%(name)s', multiple=True),
...
)
Later, I want to get a list of entities by name from the list: reference
entry for the current user.
I would think I could do this:
user = db(db.auth_user.id == auth.user_id).select().first()
entities = db.auth_user.entity.represent(user.entity)
but I get a ticket:
File N:/web2py/applications/myapp/controllers/mycontroller.py, line 15,
in myfunc
return dict(etext=db.auth_user.entity.represent(user.entity))
TypeError: 'NoneType' object is not callable
I've tried a few different variations on the theme, but none working so
far. Any help would be appreciated.
There are obvious slower ways to do this, but inelegant. I want the
fastest, tightest solution.
[web2py] Re: denormalizing a list:reference -- need a bit of help
If you need to fetch all entities knowing the user in advance there is a
ultra-simple way to fetch entities in a single query
Actually I don't think anyone will come up with a solution involving more
than 1 additional query to the one required for selecting the user.
user = db.auth_user(auth.user_id)
entities = db(db.entity.id.belongs(user.entity)).select(db.entity.name)
Mind that the auth_user line is copied into session, so if that is always
available you can skip the first query and do
entities = db(db.entity.id.belongs(auth.user.entity)).select(db.entity.name)
Il giorno venerdì 11 maggio 2012 17:28:38 UTC+2, weheh ha scritto:
db.define_table('entity', Field('name'), format='%(name)s')
auth_user_table = db.define_table(
auth.settings.table_user_name,
...
Field('entity', 'list:reference db.entity',
requires=IS_IN_DB(db, 'entity.id','%(name)s', multiple=True),
...
)
Later, I want to get a list of entities by name from the list: reference
entry for the current user.
I would think I could do this:
user = db(db.auth_user.id == auth.user_id).select().first()
entities = db.auth_user.entity.represent(user.entity)
but I get a ticket:
File N:/web2py/applications/myapp/controllers/mycontroller.py, line 15
, in myfunc
return dict(etext=db.auth_user.entity.represent(user.entity))
TypeError: 'NoneType' object is not callable
I've tried a few different variations on the theme, but none working so
far. Any help would be appreciated.
There are obvious slower ways to do this, but inelegant. I want the
fastest, tightest solution.
[web2py] Re: MongoDB Adapter error in select
the latest DAL produces this error
users = mongo(mongo.user.age==18).select()
File /opt/web2py/gluon/dal.py, line 8004, in select
return adapter.select(self.query,fields,attributes)
File /opt/web2py/gluon/dal.py, line 4528, in select
colnames = [fix(column) for column in mongo_list_dicts[0]]
File
/usr/local/lib/python2.7/dist-packages/pymongo-2.1.1-py2.7-linux-x86_64.egg/pymongo/cursor.py,
line 402, in __getitem__
raise IndexError(no such item for Cursor instance)
IndexError: no such item for Cursor instance
On Friday, May 11, 2012 12:40:35 AM UTC+1, Massimo Di Pierro wrote:
Now I understand better where all these problems come from. I shall fix it
tonight.
On Thursday, 10 May 2012 18:02:24 UTC-5, Francisco Costa wrote:
i didn't understand your question..
This is the complete code
import sys
import time
from gluon.dal import DAL, Field
mongo = DAL('mongodb://localhost:27017/sapo')
mongo.define_table('user',
Field('name', 'text'),
Field('age', 'integer'),
Field('city', 'string')
)
def insert_users():
mongo.user.insert(name='John', age=66, city='Toronto')
mongo.user.insert(name='Mark', age=43, city='Boston')
mongo.user.insert(name='Tom', age=43, city='Detroit')
mongo.user.insert(name='Jim', age=18, city='Detroit')
mongo.user.insert(name='Jack', age=18)
mongo.user.insert(name='Eric', city='Boston')
return 'users in database'
def find_users():
users = mongo(mongo.user.age==18).select()
return dict(users=users)
On Thursday, May 10, 2012 11:58:36 PM UTC+1, Massimo Di Pierro wrote:
What's the model?
On Thursday, 10 May 2012 16:27:51 UTC-5, Francisco Costa wrote:
I found another problem!!
if you do this:
mongo.user.insert(name='Jim', age=18, city='Detroit')
mongo.user.insert(name='Jack', age=18)
and then select
users = mongo(mongo.user.age==18).select()
you don't have the field city in one of the rows, so you get this
error:
users = mongo(mongo.user.age==18).select()
File /opt/web2py/gluon/dal.py, line 8123, in select
return adapter.select(self.query,fields,attributes)
File /opt/web2py/gluon/dal.py, line 4700, in select
return processor(rows,fields,colnames,False)
File /opt/web2py/gluon/dal.py, line 1681, in parse
self.parse_value(value, fields[j].type,blob_decode)
File /opt/web2py/gluon/dal.py, line 1575, in parse_value
return self.parsemap[key](value,field_type)
File /opt/web2py/gluon/dal.py, line 1641, in parse_id
return int(value)
ValueError: invalid literal for int() with base 10: 'Jack'
On Thursday, May 10, 2012 10:16:32 PM UTC+1, Francisco Costa wrote:
Just add a '\' at the end of line 4664 and your are done!
Great job Massimo!
On Thursday, May 10, 2012 9:12:48 PM UTC+1, Massimo Di Pierro wrote:
This helps. I have another attempt to fix this in trunk. Hard to test
it since I do not have mongodb installed.
On Thursday, 10 May 2012 08:39:22 UTC-5, Francisco Costa wrote:
Still not working. I think Fields order is wrong.
At the end of the select() function I write this:
a = []
for f in fields:
a.append(f.type)
return dict(rows=rows, fields=a, colnames=colnames)
and I got this:
colnames:
city
age
_id
name
fields:
id
text
integer
string
rows:
Toronto
66L
24652490551171733682233802752L
John
Re: [web2py] Crontab generator
Pretty cool! Richard On Fri, May 11, 2012 at 12:05 PM, Bruno Rocha rochacbr...@gmail.com wrote: Useful application: http://corntab.com/pages/crontab-gui -- Bruno Rocha [http://rochacbruno.com.br]
[web2py] Re: Looking for a way to combine auto-complete with adding a non-existing item.
Are you trying to auto-complete a reference field, and you want to be able
to add new records to the referenced table? The built-in autocomplete
widget doesn't handle that out of the box, but upon submission, you could
check the request variables, and if the submitted value doesn't belong to
an existing record in the referenced table, you could do an insert:
Model:
db.define_table('category', Field(name))
db.define_table('article',
Field('title'),
Field('body', 'text'),
Field('category', db.category,
widget=SQLFORM.widgets.autocomplete(request, db.category.name,id_field
=db.category.id)))
Controller:
def myfunction():
if request.post_vars._autocomplete_name_aux and not request.post_vars.
category:
request.post_vars.category = db.category.insert(name=request.
post_vars._autocomplete_name_aux)
return dict(form=SQLFORM(db.article).process())
When you use the autocomplete widget on a reference field, it submits the
display value in a special field named
_autocomplete_[display_field_name]_aux, and the actual field value is the
record ID of the referenced record associated with the display value.
However, if you enter and submit a display value that does not have an
associated record in the referenced table, the reference field value will
simply be empty (though the _autocomplete_..._aux field will contain the
submitted display value).
For example, when submitting a new article, suppose you enter programming
in the category input field, but programming does not yet exist in the
db.category table. In that case, request.post_vars.category will be empty,
and request.post_vars._autocomplete_name_aux will contain the word
programming. In that case, the above code inserts programming into the
db.category table and stores the new record ID in
request.post_vars.category. When the form is then processed, the new
db.article record will be created, with the new db.category record ID
stored in the db.article.category field.
Perhaps the autocomplete widget should handle this automatically (with an
optional setting).
Anthony
On Thursday, May 10, 2012 12:44:51 PM UTC-4, gfdgdgfdg wrote:
Hello everyone,
I'm looking for a way to get an auto-complete field combined with adding
non-existing item to the DB. I've seen a website which
uses this functionality (demo: http://www.tellmehow.nl/video.html )
I have already tried SELECT_OR_ADD_OPTION but i don't like drop-downs...
Thanks in advance!
Remco
[web2py] Self-submitting Form and using dict update method
I am brand new to web2py and relatively new to python and I'm stuck, mainly
on a python issue.
My goal is to have a self-submitting form that allows a user to re-submit
the same simple form any number of times, capture the inputs each time and
display a list of the values entered, with the intent of acting on those
values when the user is done adding them. The filters will essentially be
where clause statements to execute a long running query, but I'm stuck on
the form aspects.
As you can see below, I use session.no_filters(number of filters) to change
the name of the form elements each time. I then try to use the dict update
method to add the request.vars dict to the dict that is supposed to hold
all the values (session.filter_list). The problem is that in the piece
I've called out here, I always hit the AttributeError. I know that x={1:2}
y={2:3} x.update(y) makes x={1: 2, 2: 3}, so from my perspective, this
should work, but I don't know what I'm talking about so...please help.
This is the piece that doesn't work, called out from the below full code
listing for this piece.
if form.accepts(request,session):
session.no_filters = session.no_filters+1
try:
session.filter_list.update(request.vars)
except NameError:
session.filter_list = request.vars
except AttributeError:
session.filter_list = 'This is an attribute error'
Controller code for add_filters
def add_filters():
try:
session.no_filters
except NameError:
session.no_filters = 0
response.flash=add filters
#rows= db().select(db.tablecols.ALL)
form = FORM(
TR(LABEL('Filter Source'), \
SELECT(
_name='filter_src' + str(session.no_filters))),
TR(LABEL('Operator'), \
SELECT(
OPTION('='),
OPTION(''),
OPTION(''),
OPTION('in'),
_name='operator' + str(session.no_filters))),
TR(LABEL('Value'),INPUT(_name='value')),
INPUT(_type='submit',_value='Save Filter'),
INPUT(_type='button',_value='Add Filter',_onclick='addfilter2'),
INPUT(_type='button',_value='Create Segments',_onclick='createsegment'))
# if form correct, perform the insert
if form.accepts(request,session):
session.no_filters = session.no_filters+1
try:
session.filter_list.update(request.vars)
except NameError:
session.filter_list = request.vars
except AttributeError:
session.filter_list = 'This is an attribute error'
elif form.errors:
response.flash=form is invalid
else:
response.flash=please fill in the form
return dict(form=form,output=form.vars)
The view code is really just:
{{=BEAUTIFY(session.filter_list)}}
Thanks in advance!
[web2py] class 'sqlite3.ProgrammingError' Cannot operate on a closed database.
Here is my complete model code for a sqlite logging handler. This works the
first time the program is run, and after that I get an error:
class 'sqlite3.ProgrammingError' Cannot operate on a closed database.
*model mylogging.py:*
import logging
import logging.handlers
class AppHandler(logging.Handler): # Inherit from logging.Handler
def __init__(self):
logging.Handler.__init__(self)
self._db = db
def emit(self, record):
args = {}
args['loggername'] = record.name
args['srclineno'] = record.lineno
args['func'] = record.funcName
args['level'] = record.levelname
args['msg'] = record.msg
try:
args['type'] = record.args[0]
except:
args['type'] = None
self._db.log.insert(**args)
log_db = DAL('sqlite://log.sqlite')
log_db.define_table('log',
Field('loggername', 'string'), #unique=True
Field('srclineno', 'string'),
Field('func', 'string'),
Field('level', 'string'),
Field('msg', 'string'),
Field('type', 'string'),
)
import logging
logger = logging.getLogger(web2py.app.myapp)
logger.setLevel(logging.DEBUG)
logger.addHandler(AppHandler(log_db))
logger.debug('test log')
I don't understand how I'm causing this, as all as im doing is creating a
second db instance and inserting using web2py DAL methods?
[web2py] Re: jqgrid assistance
got it working by using the wizard to create a new version of my app.
On Thursday, May 10, 2012 4:16:00 PM UTC-4, Larry Wapnitsky wrote:
the whole plugin_wiki thing is really starting to get my goat. Now I'm
getting errors trying to delete my test wiki page:
class 'gluon.contrib.pymysql.err.IntegrityError' (1452, u'Cannot add or
update a child row: a foreign key constraint fails
(`rbl`.`plugin_wiki_page_archive`, CONSTRAINT
`plugin_wiki_page_archive_ibfk_1` FOREIGN KEY (`current_record`) REFERENCES
`plugin_wiki_page` (`id`) ON DELETE CASCADE)')
On Thursday, May 10, 2012 3:44:49 PM UTC-4, Massimo Di Pierro wrote:
If you use chrome check the developer tools. There should be some ajax
request to get data. Does it fail? Why?
To check you can also delete your tickets in errors/* and see if any new
one shows up.
On Thursday, 10 May 2012 08:31:15 UTC-5, Larry Wapnitsky wrote:
I'm trying out jqgrid (yes, I finally got plugin_wiki working ;)), but
can't seem to get it to display data. Here is my code:
*{{=plugin_wiki.widget('jqgrid', db.ips)}}*
I see a table with MOST of my fields showing (not all, strangely), but
no data. I know for a fact that the database is populated.
Ideas?
Thanks
[web2py] Re: jqgrid assistance
nix that. the updated version in plugin_wiki is still giving me grief.
the standard version you can install from the wizard (jqgrid) works except
for individual column sizing.
On Friday, May 11, 2012 3:42:04 PM UTC-4, Larry Wapnitsky wrote:
got it working by using the wizard to create a new version of my app.
On Thursday, May 10, 2012 4:16:00 PM UTC-4, Larry Wapnitsky wrote:
the whole plugin_wiki thing is really starting to get my goat. Now I'm
getting errors trying to delete my test wiki page:
class 'gluon.contrib.pymysql.err.IntegrityError' (1452, u'Cannot add or
update a child row: a foreign key constraint fails
(`rbl`.`plugin_wiki_page_archive`, CONSTRAINT
`plugin_wiki_page_archive_ibfk_1` FOREIGN KEY (`current_record`) REFERENCES
`plugin_wiki_page` (`id`) ON DELETE CASCADE)')
On Thursday, May 10, 2012 3:44:49 PM UTC-4, Massimo Di Pierro wrote:
If you use chrome check the developer tools. There should be some ajax
request to get data. Does it fail? Why?
To check you can also delete your tickets in errors/* and see if any new
one shows up.
On Thursday, 10 May 2012 08:31:15 UTC-5, Larry Wapnitsky wrote:
I'm trying out jqgrid (yes, I finally got plugin_wiki working ;)), but
can't seem to get it to display data. Here is my code:
*{{=plugin_wiki.widget('jqgrid', db.ips)}}*
I see a table with MOST of my fields showing (not all, strangely), but
no data. I know for a fact that the database is populated.
Ideas?
Thanks
[web2py] Re: MongoDB Adapter error in select
One more try please.
On Friday, 11 May 2012 06:04:52 UTC-5, Francisco Costa wrote:
Thanks Massimo. Please let me know if you need more help in debugging it
On Friday, May 11, 2012 12:40:35 AM UTC+1, Massimo Di Pierro wrote:
Now I understand better where all these problems come from. I shall fix
it tonight.
On Thursday, 10 May 2012 18:02:24 UTC-5, Francisco Costa wrote:
i didn't understand your question..
This is the complete code
import sys
import time
from gluon.dal import DAL, Field
mongo = DAL('mongodb://localhost:27017/sapo')
mongo.define_table('user',
Field('name', 'text'),
Field('age', 'integer'),
Field('city', 'string')
)
def insert_users():
mongo.user.insert(name='John', age=66, city='Toronto')
mongo.user.insert(name='Mark', age=43, city='Boston')
mongo.user.insert(name='Tom', age=43, city='Detroit')
mongo.user.insert(name='Jim', age=18, city='Detroit')
mongo.user.insert(name='Jack', age=18)
mongo.user.insert(name='Eric', city='Boston')
return 'users in database'
def find_users():
users = mongo(mongo.user.age==18).select()
return dict(users=users)
Re: [web2py] Crontab generator
This is just JS. We can integrate it in admin. We use crontab syntax. On Friday, 11 May 2012 11:28:56 UTC-5, Richard wrote: Pretty cool! Richard On Fri, May 11, 2012 at 12:05 PM, Bruno Rocha rochacbr...@gmail.comwrote: Useful application: http://corntab.com/pages/crontab-gui -- Bruno Rocha [http://rochacbruno.com.br]
[web2py] Re: class 'sqlite3.ProgrammingError' Cannot operate on a closed database.
Is this one or two files?
If the model is passing db to the logger then the db is closed when the
first request responds.
The logger must make its own connection to the db and commit
On Friday, 11 May 2012 14:28:22 UTC-5, Yarin wrote:
Here is my complete model code for a sqlite logging handler. This works
the first time the program is run, and after that I get an error:
class 'sqlite3.ProgrammingError' Cannot operate on a closed database.
*model mylogging.py:*
import logging
import logging.handlers
class AppHandler(logging.Handler): # Inherit from logging.Handler
def __init__(self):
logging.Handler.__init__(self)
self._db = db
def emit(self, record):
args = {}
args['loggername'] = record.name
args['srclineno'] = record.lineno
args['func'] = record.funcName
args['level'] = record.levelname
args['msg'] = record.msg
try:
args['type'] = record.args[0]
except:
args['type'] = None
self._db.log.insert(**args)
log_db = DAL('sqlite://log.sqlite')
log_db.define_table('log',
Field('loggername', 'string'), #unique=True
Field('srclineno', 'string'),
Field('func', 'string'),
Field('level', 'string'),
Field('msg', 'string'),
Field('type', 'string'),
)
import logging
logger = logging.getLogger(web2py.app.myapp)
logger.setLevel(logging.DEBUG)
logger.addHandler(AppHandler(log_db))
logger.debug('test log')
I don't understand how I'm causing this, as all as im doing is creating a
second db instance and inserting using web2py DAL methods?
[web2py] Re: jqgrid assistance
Screenshot of the issue
herehttp://www.zimagez.com/zimage/screenshot-05112012-041826pm.php
:
http://www.zimagez.com/full/85e6a64dc62ed33f329a618ec91320d41abfadb28052b4b86924c0fb25bf0a9d45207eef74a4ebe8b70e615d5809904e0acef0c8ca77eb20.php
On Friday, May 11, 2012 4:08:01 PM UTC-4, Larry Wapnitsky wrote:
nix that. the updated version in plugin_wiki is still giving me grief.
the standard version you can install from the wizard (jqgrid) works except
for individual column sizing.
On Friday, May 11, 2012 3:42:04 PM UTC-4, Larry Wapnitsky wrote:
got it working by using the wizard to create a new version of my app.
On Thursday, May 10, 2012 4:16:00 PM UTC-4, Larry Wapnitsky wrote:
the whole plugin_wiki thing is really starting to get my goat. Now
I'm getting errors trying to delete my test wiki page:
class 'gluon.contrib.pymysql.err.IntegrityError' (1452, u'Cannot add
or update a child row: a foreign key constraint fails
(`rbl`.`plugin_wiki_page_archive`, CONSTRAINT
`plugin_wiki_page_archive_ibfk_1` FOREIGN KEY (`current_record`) REFERENCES
`plugin_wiki_page` (`id`) ON DELETE CASCADE)')
On Thursday, May 10, 2012 3:44:49 PM UTC-4, Massimo Di Pierro wrote:
If you use chrome check the developer tools. There should be some ajax
request to get data. Does it fail? Why?
To check you can also delete your tickets in errors/* and see if any
new one shows up.
On Thursday, 10 May 2012 08:31:15 UTC-5, Larry Wapnitsky wrote:
I'm trying out jqgrid (yes, I finally got plugin_wiki working ;)), but
can't seem to get it to display data. Here is my code:
*{{=plugin_wiki.widget('jqgrid', db.ips)}}*
I see a table with MOST of my fields showing (not all, strangely), but
no data. I know for a fact that the database is populated.
Ideas?
Thanks
[web2py] cookbook component upload recipe
I'm trying to use the recipe on p. 142 of the web2py cookbook (Uploading
files using a LOADed component) and I'm wondering if there's an error in
the replacement web2py_trap_form function. Line 4 of the replacement
provides an 'if' condition with no statements following (no { } at all). Is
this correct?
function web2py_trap_form(action,target) {
jQuery('#'+target+' form').each(function(i){
var form=jQuery(this);
if(!form.hasClass('no_trap'))
if(form.find('.upload').length0) {
form.ajaxForm({
url: action,
success: function(data, statusText, xhr) {
jQuery('#'+target).html(xhr.responseText);
web2py_trap_form(action,target);
web2py_ajax_init();
}
});
} else {
form.submit(function(e){
jQuery('.flash').hide().html('');
web2py_ajax_page('post',action,form.serialize(),target);
e.preventDefault();
});
}
});
}
Thanks,
Ian
[web2py] Re: class 'sqlite3.ProgrammingError' Cannot operate on a closed database.
Massimo- see revised - this is one model file and now I'm defining the db
in the constructor, but I get the Cannot operate on a closed database
error immediately.
import logging
import logging.handlers
class AppHandler(logging.Handler): # Inherit from logging.Handler
def __init__(self):
logging.Handler.__init__(self)
self.log_db = DAL('sqlite://log.sqlite')
self.log_db.define_table('log',
Field('loggername', 'string'),
Field('srclineno', 'string'),
Field('func', 'string'),
Field('level', 'string'),
Field('msg', 'string'),
Field('type', 'string')
)
def emit(self, record):
args = {}
args['loggername'] = record.name
args['srclineno'] = record.lineno
args['func'] = record.funcName
args['level'] = record.levelname
args['msg'] = record.msg
try:
args['type'] = record.args[0]
except:
args['type'] = None
self.log_db.log.insert(**args)
logger = logging.getLogger(web2py.app.myapp)
logger.setLevel(logging.DEBUG)
logger.addHandler(AppHandler())
logger.debug('test log')
Are you saying I need to make a connection and commit manually? How come
this isn't required in the normal db.py file's db definition- there we just
define it and go?
On Friday, May 11, 2012 4:19:22 PM UTC-4, Massimo Di Pierro wrote:
Is this one or two files?
If the model is passing db to the logger then the db is closed when the
first request responds.
The logger must make its own connection to the db and commit
On Friday, 11 May 2012 14:28:22 UTC-5, Yarin wrote:
Here is my complete model code for a sqlite logging handler. This works
the first time the program is run, and after that I get an error:
class 'sqlite3.ProgrammingError' Cannot operate on a closed database.
*model mylogging.py:*
import logging
import logging.handlers
class AppHandler(logging.Handler): # Inherit from logging.Handler
def __init__(self):
logging.Handler.__init__(self)
self._db = db
def emit(self, record):
args = {}
args['loggername'] = record.name
args['srclineno'] = record.lineno
args['func'] = record.funcName
args['level'] = record.levelname
args['msg'] = record.msg
try:
args['type'] = record.args[0]
except:
args['type'] = None
self._db.log.insert(**args)
log_db = DAL('sqlite://log.sqlite')
log_db.define_table('log',
Field('loggername', 'string'), #unique=True
Field('srclineno', 'string'),
Field('func', 'string'),
Field('level', 'string'),
Field('msg', 'string'),
Field('type', 'string'),
)
import logging
logger = logging.getLogger(web2py.app.myapp)
logger.setLevel(logging.DEBUG)
logger.addHandler(AppHandler(log_db))
logger.debug('test log')
I don't understand how I'm causing this, as all as im doing is creating a
second db instance and inserting using web2py DAL methods?
Re: [web2py] Re: Nginx-uwsgi problem.
2012/5/11 Bruce Wade bruce.w...@gmail.com: Maybe in some places of the code but not everywhere. The problem is when there is a large load all 3 servers get very slow on every page. I think it is the DB layer as we have 90 tables in one database 45 in another. I am also using connection pooling which I think is causing problems. Because the DAL is loaded with every request, wouldn't that means the pool is open for each request? Or should there only ever be 10 connections open even if I have 1000 concurrent connections? Connections = (pool_size) * (number of web2py processes) So if you have 10 threads and pool_size = 4 1 * 4 = 4 connections If you have 10 processes (each with 6 threads): 10 * 4 = 40 connections As you can see the number of processes is not a term of the computation. You must count the number of concurrent processes, the number of threads does not count, same for the number of requests in the nginx queue. If the db seems to be locked you can do on the db server host: ps ax | grep TRANSACTION you should get many postgres processes IDLE IN TRANSACTION. It is a symptom of web2py taking long to commit the transaction. If you do not use the db in some complex view you can try to put a db.rollback() at the beginning of the controller. Are you using any web2py scripts (cron or the like)? check that you do not keep the transaction open if the process is long. Use alway db.commit! mic
Re: [web2py] Re: Nginx-uwsgi problem.
Connections = (pool_size) * (number of web2py processes) So if you have 10 threads and pool_size = 4 1 * 4 = 4 connections If you have 10 processes (each with 6 threads): 10 * 4 = 40 connections As you can see the number of processes is not a term of the computation. You must count the number of concurrent processes, the number of threads does not count, same for the number of requests in the nginx queue. AHEM... its too late here... ...the number of processes is not a term ... should read ...the number of threads is not a term ... If the db seems to be locked you can do on the db server host: ps ax | grep TRANSACTION you should get many postgres processes IDLE IN TRANSACTION. It is a symptom of web2py taking long to commit the transaction. If you do not use the db in some complex view you can try to put a db.rollback() at the beginning of the controller. Are you using any web2py scripts (cron or the like)? check that you do not keep the transaction open if the process is long. Use alway db.commit! mic
[web2py] Re: jqgrid assistance
This is supposed to be `` name: jqgrid table: ips col_width: 80 width: 700 height: 300 ``:widget On Thursday, 10 May 2012 15:11:32 UTC-5, Larry Wapnitsky wrote: Even this fails on a wiki page: `` name: jqgrid table: db.ips col_width: 80 width: 700 height: 300 ``:widget Traceback (most recent call last): File /home/lwapnitsky/web2py/applications/rbl_web2py/models/plugin_wiki.py, line 635, in render_widget html = getattr(PluginWikiWidgets,name)(**args) File /home/lwapnitsky/web2py/applications/rbl_web2py/models/plugin_wiki.py, line 245, in jqgrid fields = [x.strip() for x in db[table].fields if db[table][x.strip()].readable] File /home/lwapnitsky/web2py/gluon/dal.py, line 6773, in __getitem__ return dict.__getitem__(self, str(key)) KeyError: 'db.ips'
[web2py] Re: jqgrid assistance
What do you get if you call this yourself? Anyway, I do not see the signature. Are you logged in? jqgrid in plugin_wiki requires login if I remember it right. On Thursday, 10 May 2012 15:02:27 UTC-5, Larry Wapnitsky wrote: Also just found this via the dev tools:: Request URL: http://127.0.0.1:8000/rbl_web2py/plugin_wiki/jqgrid?columns=ipaddress%2Cattacknotes%2Cb_or_w%2CupdCountfieldname=fieldvalue=Nonetablename=ips_search=falsend=1336679760463rows=10page=1sidx=sord=asc
Re: [web2py] Re: jqgrid assistance
I'll try that, but even adjusting, this (in the controller) does not work:
wl2 = plugin_wiki.widget('jqgrid', db.ips, fieldname=b_or_w,
fieldvalue=w, fields=[id, ipaddress, attacknotes])
nor does:
wl2 = plugin_wiki.widget('jqgrid', ips, fieldname=b_or_w, fieldvalue=w,
fields=[id, ipaddress, attacknotes])
On Fri, May 11, 2012 at 6:01 PM, Massimo Di Pierro
massimo.dipie...@gmail.com wrote:
What do you get if you call this yourself?
Anyway, I do not see the signature. Are you logged in?
jqgrid in plugin_wiki requires login if I remember it right.
On Thursday, 10 May 2012 15:02:27 UTC-5, Larry Wapnitsky wrote:
Also just found this via the dev tools::
Request URL:
http://127.0.0.1:8000/rbl_**web2py/plugin_wiki/jqgrid?**
columns=ipaddress%**2Cattacknotes%2Cb_or_w%**2CupdCountfieldname=**
fieldvalue=Nonetablename=ips**_search=falsend=**
1336679760463rows=10page=1**sidx=sord=aschttp://127.0.0.1:8000/rbl_web2py/plugin_wiki/jqgrid?columns=ipaddress%2Cattacknotes%2Cb_or_w%2CupdCountfieldname=fieldvalue=Nonetablename=ips_search=falsend=1336679760463rows=10page=1sidx=sord=asc
[web2py] Re: class 'sqlite3.ProgrammingError' Cannot operate on a closed database.
After
self.log_db.log.insert(**args)
you need
self.log_db.commit()
every http request, the models, are executed, DAL(...) connects you to the
db (or recycles a connection from the pool), then when the action is done,
it automatically commits or rollsback, then web2py closes the connection.
If you import a module, Python caches the module. A db connection defined
in the module, lives as long as the module lives. A db connection defined
in a model lives only as long as the http request lives. If you pass one to
the other you run into trouble.
One more complication is that if you use sqlite, if one process/thread
opens the file and tries write into it, the file gets locked.
On Friday, 11 May 2012 15:40:39 UTC-5, Yarin wrote:
Massimo- see revised - this is one model file and now I'm defining the db
in the constructor, but I get the Cannot operate on a closed database
error immediately.
import logging
import logging.handlers
class AppHandler(logging.Handler): # Inherit from logging.Handler
def __init__(self):
logging.Handler.__init__(self)
self.log_db = DAL('sqlite://log.sqlite')
self.log_db.define_table('log',
Field('loggername', 'string'),
Field('srclineno', 'string'),
Field('func', 'string'),
Field('level', 'string'),
Field('msg', 'string'),
Field('type', 'string')
)
def emit(self, record):
args = {}
args['loggername'] = record.name
args['srclineno'] = record.lineno
args['func'] = record.funcName
args['level'] = record.levelname
args['msg'] = record.msg
try:
args['type'] = record.args[0]
except:
args['type'] = None
self.log_db.log.insert(**args)
logger = logging.getLogger(web2py.app.myapp)
logger.setLevel(logging.DEBUG)
logger.addHandler(AppHandler())
logger.debug('test log')
Are you saying I need to make a connection and commit manually? How come
this isn't required in the normal db.py file's db definition- there we just
define it and go?
On Friday, May 11, 2012 4:19:22 PM UTC-4, Massimo Di Pierro wrote:
Is this one or two files?
If the model is passing db to the logger then the db is closed when the
first request responds.
The logger must make its own connection to the db and commit
On Friday, 11 May 2012 14:28:22 UTC-5, Yarin wrote:
Here is my complete model code for a sqlite logging handler. This works
the first time the program is run, and after that I get an error:
class 'sqlite3.ProgrammingError' Cannot operate on a closed database.
*model mylogging.py:*
import logging
import logging.handlers
class AppHandler(logging.Handler): # Inherit from logging.Handler
def __init__(self):
logging.Handler.__init__(self)
self._db = db
def emit(self, record):
args = {}
args['loggername'] = record.name
args['srclineno'] = record.lineno
args['func'] = record.funcName
args['level'] = record.levelname
args['msg'] = record.msg
try:
args['type'] = record.args[0]
except:
args['type'] = None
self._db.log.insert(**args)
log_db = DAL('sqlite://log.sqlite')
log_db.define_table('log',
Field('loggername', 'string'), #unique=True
Field('srclineno', 'string'),
Field('func', 'string'),
Field('level', 'string'),
Field('msg', 'string'),
Field('type', 'string'),
)
import logging
logger = logging.getLogger(web2py.app.myapp)
logger.setLevel(logging.DEBUG)
logger.addHandler(AppHandler(log_db))
logger.debug('test log')
I don't understand how I'm causing this, as all as im doing is creating
a second db instance and inserting using web2py DAL methods?
[web2py] Table/grid with checkboxes for selecting records?
How can I create a table/grid with a checkbox in front of each row so that multiple records can be selected for update/delete? Something like your standard webmail inbox - check off a series of messages and then you can do something with them. I've got a version working with SQLFORM.factory and a field that uses widget=SQLFORM.widgets.checkboxes.widget but it isn't anywhere near as polished of a look as I would like (my checkbox labels end up being rediculously long and really should be separate columns). Is there a nifty web2py way of doing this or do I just need to create the form/checkboxes myself in the view and leave behind the web2py form validation? Thanks, Brian
[web2py] storing more than 1 millions uploaded files
Hi All, I need to sore a lot of file... up to few millions. Just wondering if that can be done with web2py out of the box without the need of adding extra code for storing them in a sub folders tree structure I've read that web2py can organize the files uploaded in folders. I've created a simple file upload app and it looks like it creates only one level of subfolder. Something like uploads/tablename/fieldname/xx/ where xx seams been the first two characters of the file uuid. As the filesystem starts to get slow with more than 1k files per folder, that means that after 1k x 1k = 1M files the system will get slow... Am I correct ? is it possible to configure web2py to use more than one level (per table/field) ? thanks ? -- Sebastian E. Ovide
Re: [web2py] Table/grid with checkboxes for selecting records?
A working example on plugin BadMin https://github.com/rochacbruno/badmin On Fri, May 11, 2012 at 7:13 PM, Brian M bmere...@gmail.com wrote: How can I create a table/grid with a checkbox in front of each row so that multiple records can be selected for update/delete? Something like your standard webmail inbox - check off a series of messages and then you can do something with them. I've got a version working with SQLFORM.factory and a field that uses widget=SQLFORM.widgets.checkboxes.widget but it isn't anywhere near as polished of a look as I would like (my checkbox labels end up being rediculously long and really should be separate columns). Is there a nifty web2py way of doing this or do I just need to create the form/checkboxes myself in the view and leave behind the web2py form validation? Thanks, Brian -- Bruno Rocha [http://rochacbruno.com.br]
[web2py] Re: Load data from sqlite file?
So the user is going to be uploading a .sqlite file which you want to connect to with the DAL and import? You should be able to create a new DAL connection within a controller function or probably better yet a module instead of in a model file (which would have been run too early in the request process). So save the uploaded .sqlite file then find out the filename that web2py gave it and then pass that filename (and path) to a new dal connection, proceed to define the tables as usual and import away. On Thursday, May 10, 2012 6:38:41 PM UTC-5, Dave wrote: Any examples or suggestions on how to load data from a user-uploaded .sqlite database? I will have the database structure of the uploaded files ahead of time (they are being generated by a mobile app), so I need a way to transfer the data from an uploaded .sqlite file to my web2py tables. is it possible to just use the uploaded file as a new dal, then transfer the data to the main dal, all within a function? Thanks in advance, Dave
[web2py] Re: Self-submitting Form and using dict update method
if form.accepts(request,session): session.no_filters = session.no_filters+1 try: session.filter_list.update(request.vars) except NameError: session.filter_list = request.vars except AttributeError: session.filter_list = 'This is an attribute error' The session object is a gluon.storage.Storage object, as discussed here: http://web2py.com/books/default/chapter/29/4#session. In that section, it mentions: Because session is a Storage object, trying to access an attribute/key that has not been set does not raise an exception; it returns None instead. In your code, you are expecting session.filter_list.update(request.vars) to raise a NameError before you have assigned any value to filter_list. Instead, session.filter_list simply returns None, and then you are attempting to call the .update() method on None, which is what is raising the AttributeError. If you don't catch the AttributeError, you should see that it says 'NoneType' object has no attribute 'update'. Instead, you can do: if session.filter_list: session.filter_list.update(request.vars) else: session.filter_list = request.vars Anthony
[web2py] Remembering selections for form submission
I'm creating a form with drop down boxes like this:
SELECT(*options, _name='q%d' % (q_num), requires=IS_IN_SET(['1', '2', '3',
'4', '5'], error_message=Oops, looks like you missed this one.)
When options looks like this:
options = ['0', '1', '2', '3', '4', '5']
everything is fine. In particular, if there is a form validation error, the
form remembers which options was selected for each item.
However, if options looks like this:
options = [OPTION('', _value='0'),
OPTION('Never have the thought', _value='1'),
OPTION('Rarely have the thought ', _value='2'),
OPTION('Sometimes have the thought', _value='3'),
OPTION('Often have the thought', _value='4'),
OPTION('Always have the thought', _value='5')]
The form initially displays correctly, but when there is a validation
error, it forgets which option was previously selected. In particular, it
always says selected=0.
Is this a bug, or am I doing something wrong? Is there a workaround?
Neil
[web2py] Re: Remembering selections for form submission
You need
name = 'q%d' % (q_num)
SELECT(,value=request.vars.name or '0',...)
Only SQLFORM remembers the state. primitive tags like SELECT, INPUT,
TEXTAREA that do not know state. you must pass state.
On Friday, 11 May 2012 15:52:21 UTC-5, Neil wrote:
I'm creating a form with drop down boxes like this:
SELECT(*options, _name='q%d' % (q_num), requires=IS_IN_SET(['1', '2', '3',
'4', '5'], error_message=Oops, looks like you missed this one.)
When options looks like this:
options = ['0', '1', '2', '3', '4', '5']
everything is fine. In particular, if there is a form validation error,
the form remembers which options was selected for each item.
However, if options looks like this:
options = [OPTION('', _value='0'),
OPTION('Never have the thought', _value='1'),
OPTION('Rarely have the thought ', _value='2'),
OPTION('Sometimes have the thought', _value='3'),
OPTION('Often have the thought', _value='4'),
OPTION('Always have the thought', _value='5')]
The form initially displays correctly, but when there is a validation
error, it forgets which option was previously selected. In particular, it
always says selected=0.
Is this a bug, or am I doing something wrong? Is there a workaround?
Neil
Re: [web2py] Re: jqgrid assistance
The second argument is the name of the table, not the table.
wl2 = plugin_wiki.widget('jqgrid', 'ips', fieldname=b_or_w,
fieldvalue=w, fields=[id, ipaddress, attacknotes])
On Friday, 11 May 2012 17:07:02 UTC-5, Larry Wapnitsky wrote:
I'll try that, but even adjusting, this (in the controller) does not work:
wl2 = plugin_wiki.widget('jqgrid', db.ips, fieldname=b_or_w,
fieldvalue=w, fields=[id, ipaddress, attacknotes])
nor does:
wl2 = plugin_wiki.widget('jqgrid', ips, fieldname=b_or_w,
fieldvalue=w, fields=[id, ipaddress, attacknotes])
On Fri, May 11, 2012 at 6:01 PM, Massimo Di Pierro
massimo.dipie...@gmail.com wrote:
What do you get if you call this yourself?
Anyway, I do not see the signature. Are you logged in?
jqgrid in plugin_wiki requires login if I remember it right.
On Thursday, 10 May 2012 15:02:27 UTC-5, Larry Wapnitsky wrote:
Also just found this via the dev tools::
Request URL:
http://127.0.0.1:8000/rbl_**web2py/plugin_wiki/jqgrid?**
columns=ipaddress%**2Cattacknotes%2Cb_or_w%**2CupdCountfieldname=**
fieldvalue=Nonetablename=ips**_search=falsend=**
1336679760463rows=10page=1**sidx=sord=aschttp://127.0.0.1:8000/rbl_web2py/plugin_wiki/jqgrid?columns=ipaddress%2Cattacknotes%2Cb_or_w%2CupdCountfieldname=fieldvalue=Nonetablename=ips_search=falsend=1336679760463rows=10page=1sidx=sord=asc
[web2py] Re: storing more than 1 millions uploaded files
yes.
Field('name','upload',uploadseparate=True)
Mind that you have to use this from the beginning when there is nothing in
uploads, else you will not be able to download previously uploaded files.
On Friday, 11 May 2012 17:19:07 UTC-5, sebastian wrote:
Hi All,
I need to sore a lot of file... up to few millions.
Just wondering if that can be done with web2py out of the box without the
need of adding extra code for storing them in a sub folders tree
structure I've read that web2py can organize the files uploaded in
folders. I've created a simple file upload app and it looks like it creates
only one level of subfolder. Something like uploads/tablename/fieldname/xx/
where xx seams been the first two characters of the file uuid. As the
filesystem starts to get slow with more than 1k files per folder, that
means that after 1k x 1k = 1M files the system will get slow...
Am I correct ? is it possible to configure web2py to use more than one
level (per table/field) ?
thanks ?
--
Sebastian E. Ovide
[web2py] Re: storing more than 1 millions uploaded files
On Friday, May 11, 2012 7:25:21 PM UTC-4, Massimo Di Pierro wrote:
yes.
Field('name','upload',uploadseparate=True)
He's aware of that but seems to think that one level of sub-folders won't
be enough (he's expecting millions of files, so still more than 1000 files
per sub-folder, even with uploadseparate=True).
I don't think web2py includes any out-of-the-box solution for generating
deeper levels of sub-folders for uploaded files. Maybe subclass Field for
that upload field and roll your own .store() and .retrieve() methods.
Anthony
Re: [web2py] Re: jqgrid assistance
better, but still no data showing up.
On Fri, May 11, 2012 at 7:23 PM, Massimo Di Pierro
massimo.dipie...@gmail.com wrote:
The second argument is the name of the table, not the table.
wl2 = plugin_wiki.widget('jqgrid', 'ips', fieldname=b_or_w,
fieldvalue=w, fields=[id, ipaddress, attacknotes])
On Friday, 11 May 2012 17:07:02 UTC-5, Larry Wapnitsky wrote:
I'll try that, but even adjusting, this (in the controller) does not work:
wl2 = plugin_wiki.widget('jqgrid', db.ips, fieldname=b_or_w,
fieldvalue=w, fields=[id, ipaddress, attacknotes])
nor does:
wl2 = plugin_wiki.widget('jqgrid', ips, fieldname=b_or_w,
fieldvalue=w, fields=[id, ipaddress, attacknotes])
On Fri, May 11, 2012 at 6:01 PM, Massimo Di Pierro
massimo.dipie...@gmail.com wrote:
What do you get if you call this yourself?
Anyway, I do not see the signature. Are you logged in?
jqgrid in plugin_wiki requires login if I remember it right.
On Thursday, 10 May 2012 15:02:27 UTC-5, Larry Wapnitsky wrote:
Also just found this via the dev tools::
Request URL:
http://127.0.0.1:8000/rbl_**web2**py/plugin_wiki/jqgrid?**columns=**
ipaddress%**2Cattacknotes%2Cb_**or_w%**2CupdCountfieldname=**fie**
ldvalue=Nonetablename=ips**_**search=falsend=**1336679760463**
rows=10page=1**sidx=sord=aschttp://127.0.0.1:8000/rbl_web2py/plugin_wiki/jqgrid?columns=ipaddress%2Cattacknotes%2Cb_or_w%2CupdCountfieldname=fieldvalue=Nonetablename=ips_search=falsend=1336679760463rows=10page=1sidx=sord=asc
Re: [web2py] Table/grid with checkboxes for selecting records?
Thanks Bruno! So it looks like building the form myself in HTML is probably the way to go then. Would be nice to be able to have web2py do the form validation, but I'll survive. I'm thinking that I might be able to mix and match - use form.custom to start the form and show some widgets for some other fields and then build the check boxes manually as long as I use the same field naming conventions. On Friday, May 11, 2012 5:20:58 PM UTC-5, rochacbruno wrote: A working example on plugin BadMin https://github.com/rochacbruno/badmin On Fri, May 11, 2012 at 7:13 PM, Brian M wrote: How can I create a table/grid with a checkbox in front of each row so that multiple records can be selected for update/delete? Something like your standard webmail inbox - check off a series of messages and then you can do something with them. I've got a version working with SQLFORM.factory and a field that uses widget=SQLFORM.widgets.checkboxes.widget but it isn't anywhere near as polished of a look as I would like (my checkbox labels end up being rediculously long and really should be separate columns). Is there a nifty web2py way of doing this or do I just need to create the form/checkboxes myself in the view and leave behind the web2py form validation? Thanks, Brian -- Bruno Rocha [http://rochacbruno.com.br]
[web2py] Re: storing more than 1 millions uploaded files
Oops. Sorry. I read it in a hurry.
I would not use the file system for something like this. Anyway, upload
separate creates up to 1296 subfolder per table.field. This number can be
increased.
On Friday, 11 May 2012 18:33:22 UTC-5, Anthony wrote:
On Friday, May 11, 2012 7:25:21 PM UTC-4, Massimo Di Pierro wrote:
yes.
Field('name','upload',uploadseparate=True)
He's aware of that but seems to think that one level of sub-folders won't
be enough (he's expecting millions of files, so still more than 1000 files
per sub-folder, even with uploadseparate=True).
I don't think web2py includes any out-of-the-box solution for generating
deeper levels of sub-folders for uploaded files. Maybe subclass Field for
that upload field and roll your own .store() and .retrieve() methods.
Anthony
Re: [web2py] Table/grid with checkboxes for selecting records?
WhooHoo! Keeping the same SQLFORM in the controller and then just using form.custom (see http://web2py.com/books/default/chapter/29/7?search=form.custom) for my non-checkbox elements and then simply building the desired grid with check boxes myself in the HTML works beautifully. Just have to make sure that the input fields (check box elements) you create yourself use the same name as SQLFORM expects and form processing works as usual. (You loose the ability to preserve which items were selected if there is a form error, but for me that's minor and you could probably work around that if needed). On Friday, May 11, 2012 7:53:51 PM UTC-5, Brian M wrote: Thanks Bruno! So it looks like building the form myself in HTML is probably the way to go then. Would be nice to be able to have web2py do the form validation, but I'll survive. I'm thinking that I might be able to mix and match - use form.custom to start the form and show some widgets for some other fields and then build the check boxes manually as long as I use the same field naming conventions. On Friday, May 11, 2012 5:20:58 PM UTC-5, rochacbruno wrote: A working example on plugin BadMin https://github.com/rochacbruno/badmin On Fri, May 11, 2012 at 7:13 PM, Brian M wrote: How can I create a table/grid with a checkbox in front of each row so that multiple records can be selected for update/delete? Something like your standard webmail inbox - check off a series of messages and then you can do something with them. I've got a version working with SQLFORM.factory and a field that uses widget=SQLFORM.widgets.checkboxes.widget but it isn't anywhere near as polished of a look as I would like (my checkbox labels end up being rediculously long and really should be separate columns). Is there a nifty web2py way of doing this or do I just need to create the form/checkboxes myself in the view and leave behind the web2py form validation? Thanks, Brian -- Bruno Rocha [http://rochacbruno.com.br]
[web2py] Tagging View
I have a question I have not been able to answer for myself.
In the web2py cookbook, there is this suggested model for handling tags:
db.define_table('data', Field('value'))
db.define_table('tag', Field('record_id', db.data), Field('name'))
I like that. But I have struggled with the correct way to handle the form
for the data entity. I want a create/edit form that allows the entry of
any fields in the data entities table as well as tags for the entity
(like, say, Wordpress). I am using SQLFORM to handle the form processing of
the data table form elements. But how would I handle the tags input? What
is the recommended way of creating/updating the tags?
Thanks
Rod
[web2py] Re: Tagging View
That is done here:
https://github.com/mdipierro/w2cms
Look into the action tags()
On Friday, 11 May 2012 21:06:43 UTC-5, Rod Watkins wrote:
I have a question I have not been able to answer for myself.
In the web2py cookbook, there is this suggested model for handling tags:
db.define_table('data', Field('value'))
db.define_table('tag', Field('record_id', db.data), Field('name'))
I like that. But I have struggled with the correct way to handle the form
for the data entity. I want a create/edit form that allows the entry of
any fields in the data entities table as well as tags for the entity
(like, say, Wordpress). I am using SQLFORM to handle the form processing of
the data table form elements. But how would I handle the tags input? What
is the recommended way of creating/updating the tags?
Thanks
Rod
[web2py] Integrating error pages in web2py on app engine
Hi All, I have two static custom errors pages for all 4XX and 5XX error codes. I want to integrate them with my web2py application which needs to be deployed on app engine. I tried playing around with routes.py as well as routers.py but still couldn't fix the same. Can anyone please tell me the correct way to achieve the above ? Thanks and Regards, Sushant Taneja
[web2py] Re: Integrating error pages in web2py on app engine
There's no such thing as routers.py -- if using the parameter-based rewrite system, you still use routes.py. Anyway, did you try adding routes_onerror to routes.py: http://web2py.com/books/default/chapter/29/4#Routes-on-error? Note, you have to restart the app for routes changes to take effect. Anthony On Friday, May 11, 2012 11:06:18 PM UTC-4, Sushant Taneja wrote: Hi All, I have two static custom errors pages for all 4XX and 5XX error codes. I want to integrate them with my web2py application which needs to be deployed on app engine. I tried playing around with routes.py as well as routers.py but still couldn't fix the same. Can anyone please tell me the correct way to achieve the above ? Thanks and Regards, Sushant Taneja
[web2py] Re: Integrating error pages in web2py on app engine
Yes I did configure the routes_onerror in routes.py But whenever I entered the URL of the page which does not exist, it would still return me *invalid request.* On Saturday, May 12, 2012 8:40:35 AM UTC+5:30, Anthony wrote: There's no such thing as routers.py -- if using the parameter-based rewrite system, you still use routes.py. Anyway, did you try adding routes_onerror to routes.py: http://web2py.com/books/default/chapter/29/4#Routes-on-error? Note, you have to restart the app for routes changes to take effect. Anthony On Friday, May 11, 2012 11:06:18 PM UTC-4, Sushant Taneja wrote: Hi All, I have two static custom errors pages for all 4XX and 5XX error codes. I want to integrate them with my web2py application which needs to be deployed on app engine. I tried playing around with routes.py as well as routers.py but still couldn't fix the same. Can anyone please tell me the correct way to achieve the above ? Thanks and Regards, Sushant Taneja
[web2py] Re: Changing the controller on the fly
This is working great! It's exactly what I needed, and makes my code much
simpler.
Thank you very much! I love it!
On Friday, May 11, 2012 5:03:51 AM UTC-7, Anthony wrote:
Or to avoid a redirect, you can change the function and controller in a
model file:
db = DAL(...)
if request.function == 'dispatch':
request.controller, request.function = [fetch from db]
response.view = '%s/%s.%s' % (request.controller, request.function,request
.extension)
response.generic_patterns = ['html'] # to enable the generic.html
view if needed
Anthony
On Friday, May 11, 2012 6:07:56 AM UTC-4, simon wrote:
You can do:
def dispatch():
controller,function = ... load these from the database ...
redirect(URL(c=controller, f=function, vars=request.vars,
args=request.args))
On Friday, 11 May 2012 10:17:19 UTC+1, Michael Toomim wrote:
I need to be able to dispatch to a different controller based on a
database lookup. So a user will go to a url (say '/dispatch'), and we'll
look up in the database some information on that user, choose a new
controller and function, and call that controller and function with its
view.
I've almost got this working below, but the models are not being loaded
into the new controller. Is there a way to fix that?
In default.py:
def dispatch():
controller,function = ... load these from the database ...
response.view = '%s/%s.html' % (controller,
function)
if not os.path.exists(request.folder + '/views/' + response.view):
response.view = 'generic.html'
from gluon.shell import exec_environment
controller = exec_environment('%s/controllers/%s.py'
% (request.folder,
controller),
request=request,
response=response,
session=session)
return controller[request.task_function]()
Unfortunately, the controller being called has access to request,
response, and session, but none of the global variables defined in my
models. Is there a way to get exec_environment() to run a function in
another controller WITHOUT losing all the model definitions?
Or is there a better way to do this?
[web2py] Is there a way to use an Alias for a COUNT() field
Having the following
def by_country():
count = db.procesados.idpublicacion.count()
rows = db().select(db.procesados.pais, count,
groupby=db.procesados.pais)
return rows.json()
It generates the JSON:
[{pais: , COUNT(procesados.idpublicacion): 236}, {pais: AE,
COUNT(procesados.idpublicacion): 3}]
Is there a way to assign an Alias to the COUNT field so I won't have that long
name COUNT(procesados.idpublicacion) in the JSON output
Thanks
[web2py] Re: Is there a way to use an Alias for a COUNT() field
No, but you can do
for row in rows: row['count'] = row[count]
On Friday, 11 May 2012 23:44:57 UTC-5, Franklin Freitas wrote:
Having the following
def by_country():
count = db.procesados.idpublicacion.count()
rows = db().select(db.procesados.pais, count,
groupby=db.procesados.pais)
return rows.json()
It generates the JSON:
[{pais: , COUNT(procesados.idpublicacion): 236}, {pais: AE,
COUNT(procesados.idpublicacion): 3}]
Is there a way to assign an Alias to the COUNT field so I won't have that
long name COUNT(procesados.idpublicacion) in the JSON output
Thanks
[web2py] Re: Tagging View
Wow, very cool. One question. In the cms app, a page (with an id) already exists so the ajax post to the addtag action can add the tag to the page. In my case the create version of the form will not have an entity with an id yet since it does not exist in the db. Am I wrong to think than that this solution won't work on the create entity page? I would need a pre-existing entity, yes? Thanks Rod
[web2py] Re: Is there a way to use an Alias for a COUNT() field
If I use what you recommend then what would be the best way to generate the
JSON output
Thanks Massimo
On Saturday, May 12, 2012 12:29:09 AM UTC-4:30, Massimo Di Pierro wrote:
No, but you can do
for row in rows: row['count'] = row[count]
On Friday, 11 May 2012 23:44:57 UTC-5, Franklin Freitas wrote:
Having the following
def by_country():
count = db.procesados.idpublicacion.count()
rows = db().select(db.procesados.pais, count,
groupby=db.procesados.pais)
return rows.json()
It generates the JSON:
[{pais: , COUNT(procesados.idpublicacion): 236}, {pais: AE,
COUNT(procesados.idpublicacion): 3}]
Is there a way to assign an Alias to the COUNT field so I won't have that
long name COUNT(procesados.idpublicacion) in the JSON output
Thanks
