[web2py] Re: SQLForm.grid() + LEFT JOIN + common_filters
Added a ticket with the patch that worked for me. Thanks! On Wednesday, March 27, 2013 1:05:22 PM UTC-3, Felipe Meirelles wrote: > > Actualy it worked but not for reason i expected. Ill run some more tests > here and let you know. > > On Wednesday, March 27, 2013 1:03:26 PM UTC-3, Massimo Di Pierro wrote: >> >> Please open a ticket about this and include the patch. I will review it >> and apply it asap. >> >> Massimo >> >> On Wednesday, 27 March 2013 10:57:54 UTC-5, Felipe Meirelles wrote: >>> >>> What is happening is the complete oposite. Its aplying to the leftjoin >>> table, but not to the main table. >>> >>> The problem is on dal.py line 1553: >>> tables_to_merge.keys() contains the main table, so the main table is not >>> put on excluded var in line 1567 and therefore is not used on line 1574 to >>> set the common filters. >>> >>> I patched it adding >>> >>> excluded + tablenames >>> >>> on line 1572 >>> >>> Can some one else test it and see if it will not impact on other places? >>> On Wednesday, March 27, 2013 12:40:10 PM UTC-3, Massimo Di Pierro wrote: >>>> >>>> yes. I believe common filter are ignored for left joins. >>>> >>>> https://code.google.com/p/web2py/issues/detail?id=992&q=common >>>> >>>> On Wednesday, 27 March 2013 09:13:20 UTC-5, Felipe Meirelles wrote: >>>>> >>>>> I'm facing a problem where if I have something like this: >>>>> >>>>> grid = SQLFORM.grid(db.table1, >>>>> left=[db.table2.on(db.table2.id==db.table1.reference)], >>>>> ) >>>>> >>>>> my common filters are not processed for table1, only for table2! >>>>> including is_active filter for version recording. >>>>> >>>>> Is there a bug? >>>>> >>>> -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: SQLForm.grid() + LEFT JOIN + common_filters
Actualy it worked but not for reason i expected. Ill run some more tests here and let you know. On Wednesday, March 27, 2013 1:03:26 PM UTC-3, Massimo Di Pierro wrote: > > Please open a ticket about this and include the patch. I will review it > and apply it asap. > > Massimo > > On Wednesday, 27 March 2013 10:57:54 UTC-5, Felipe Meirelles wrote: >> >> What is happening is the complete oposite. Its aplying to the leftjoin >> table, but not to the main table. >> >> The problem is on dal.py line 1553: >> tables_to_merge.keys() contains the main table, so the main table is not >> put on excluded var in line 1567 and therefore is not used on line 1574 to >> set the common filters. >> >> I patched it adding >> >> excluded + tablenames >> >> on line 1572 >> >> Can some one else test it and see if it will not impact on other places? >> On Wednesday, March 27, 2013 12:40:10 PM UTC-3, Massimo Di Pierro wrote: >>> >>> yes. I believe common filter are ignored for left joins. >>> >>> https://code.google.com/p/web2py/issues/detail?id=992&q=common >>> >>> On Wednesday, 27 March 2013 09:13:20 UTC-5, Felipe Meirelles wrote: >>>> >>>> I'm facing a problem where if I have something like this: >>>> >>>> grid = SQLFORM.grid(db.table1, >>>> left=[db.table2.on(db.table2.id==db.table1.reference)], >>>> ) >>>> >>>> my common filters are not processed for table1, only for table2! >>>> including is_active filter for version recording. >>>> >>>> Is there a bug? >>>> >>> -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: SQLForm.grid() + LEFT JOIN + common_filters
What is happening is the complete oposite. Its aplying to the leftjoin table, but not to the main table. The problem is on dal.py line 1553: tables_to_merge.keys() contains the main table, so the main table is not put on excluded var in line 1567 and therefore is not used on line 1574 to set the common filters. I patched it adding excluded + tablenames on line 1572 Can some one else test it and see if it will not impact on other places? On Wednesday, March 27, 2013 12:40:10 PM UTC-3, Massimo Di Pierro wrote: > > yes. I believe common filter are ignored for left joins. > > https://code.google.com/p/web2py/issues/detail?id=992&q=common > > On Wednesday, 27 March 2013 09:13:20 UTC-5, Felipe Meirelles wrote: >> >> I'm facing a problem where if I have something like this: >> >> grid = SQLFORM.grid(db.table1, >> left=[db.table2.on(db.table2.id==db.table1.reference)], >> ) >> >> my common filters are not processed for table1, only for table2! >> including is_active filter for version recording. >> >> Is there a bug? >> > -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] SQLForm.grid() + LEFT JOIN + common_filters
I'm facing a problem where if I have something like this: grid = SQLFORM.grid(db.table1, left=[db.table2.on(db.table2.id==db.table1.reference)], ) my common filters are not processed for table1, only for table2! including is_active filter for version recording. Is there a bug? -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: VirtualFields on select options
I was using a outdated version of web2py, in the new version the line is 511 On Wednesday, February 20, 2013 12:42:05 PM UTC-3, Felipe Meirelles wrote: > > Hi, > > Just found a bug in validators.py, line 468. > > If I use a virtualfield to format the select options, it will drop an > exception, trying to order the select by de virtual field. > > Can you patch it in trunk? > > from gluon.dal import FieldVirtual > fields = filter(lambda f: type(f) != FieldVirtual, fields) > > Thanks! > -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] VirtualFields on select options
Hi, Just found a bug in validators.py, line 468. If I use a virtualfield to format the select options, it will drop an exception, trying to order the select by de virtual field. Can you patch it in trunk? from gluon.dal import FieldVirtual fields = filter(lambda f: type(f) != FieldVirtual, fields) Thanks! -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Record versioning on delete CASCADE
Good point. But in my point of view you wont use record versioning at all your tables, just in the ones that have sensitive data. And yet, you dont use ondelete cascade in all tables too... I still think record versioning should have this standard behaviour. On Sunday, February 3, 2013 1:14:21 AM UTC-2, Massimo Di Pierro wrote: > > I should explain my answer. In order to have record versioning, record > must be copied. To have a cascade delete we would have to loop and copy > each individual record. Those records themselves could have references. > More looping. etc. This are even more complex since there may be > non-trivial circular references. > > On Saturday, 2 February 2013 21:10:56 UTC-6, Massimo Di Pierro wrote: >> >> It is possible but very very inefficient since it would not be handled at >> the database level. >> >> On Saturday, 2 February 2013 15:44:20 UTC-6, Felipe Meirelles wrote: >>> >>> I think they should, to be compatible with the default behaviour. Is >>> this possible to do? >>> >>> On Saturday, February 2, 2013 6:41:25 PM UTC-2, Massimo Di Pierro wrote: >>>> >>>> Good point. No. they are not set to is_active=False. Should they? >>>> >>>> On Saturday, 2 February 2013 06:20:47 UTC-6, Felipe Meirelles wrote: >>>>> >>>>> No, I mean, when a record is "delete" aka is_active is seted to False, >>>>> the other records there make reference to it are seted false too? >>>>> >>>>> On Friday, February 1, 2013 6:19:52 PM UTC-2, Massimo Di Pierro wrote: >>>>>> >>>>>> When using versioning. Latest versions of records are never deleted, >>>>>> only filtered out. I think cascade should work as expected. >>>>>> >>>>>> On Friday, 1 February 2013 11:54:25 UTC-6, Felipe Meirelles wrote: >>>>>>> >>>>>>> The cascade feature should work for record versioning enabled tables? >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>> -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Record versioning on delete CASCADE
I think they should, to be compatible with the default behaviour. Is this possible to do? On Saturday, February 2, 2013 6:41:25 PM UTC-2, Massimo Di Pierro wrote: > > Good point. No. they are not set to is_active=False. Should they? > > On Saturday, 2 February 2013 06:20:47 UTC-6, Felipe Meirelles wrote: >> >> No, I mean, when a record is "delete" aka is_active is seted to False, >> the other records there make reference to it are seted false too? >> >> On Friday, February 1, 2013 6:19:52 PM UTC-2, Massimo Di Pierro wrote: >>> >>> When using versioning. Latest versions of records are never deleted, >>> only filtered out. I think cascade should work as expected. >>> >>> On Friday, 1 February 2013 11:54:25 UTC-6, Felipe Meirelles wrote: >>>> >>>> The cascade feature should work for record versioning enabled tables? >>>> >>>> Thanks. >>>> >>> -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Record versioning on delete CASCADE
No, I mean, when a record is "delete" aka is_active is seted to False, the other records there make reference to it are seted false too? On Friday, February 1, 2013 6:19:52 PM UTC-2, Massimo Di Pierro wrote: > > When using versioning. Latest versions of records are never deleted, only > filtered out. I think cascade should work as expected. > > On Friday, 1 February 2013 11:54:25 UTC-6, Felipe Meirelles wrote: >> >> The cascade feature should work for record versioning enabled tables? >> >> Thanks. >> > -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Record versioning on delete CASCADE
The cascade feature should work for record versioning enabled tables? Thanks. -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: Grid with multiple left joins
Thanks Massimo, all working (: On Wed, Jan 30, 2013 at 2:38 PM, Mark wrote: > Yes, it works. Thanks! > > > On Wednesday, January 30, 2013 10:28:23 AM UTC-5, Massimo Di Pierro wrote: >> >> This is in trunk. Can you please check it. >> >> On Wednesday, 23 January 2013 10:02:03 UTC-6, Mark wrote: >>> >>> I also hope this can be done. In sqlhtml.py, simply change following >>> code: >>> >>> if left is not None: >>> tablenames += db._adapter.tables(left) >>> >>> into: >>> >>> if left is not None: >>> if isinstance(left, (list)): >>> for l in left: >>> tablenames += db._adapter.tables(l) >>> else: >>> tablenames += db._adapter.tables(left) >>> >>> >>> >>> On Friday, January 18, 2013 11:17:19 AM UTC-5, Massimo Di Pierro wrote: >>>> >>>> Looks good. >>>> Can you please send me a patch applied to the trunk version? >>>> >>>> On Friday, 18 January 2013 05:47:20 UTC-6, Felipe Meirelles wrote: >>>>> >>>>> Did anyone saw it? >>>>> >>>>> On Monday, January 14, 2013 5:41:36 PM UTC-2, Felipe Meirelles wrote: >>>>>> >>>>>> Please desconsider the sorter_icons parameter, its a custom >>>>>> implementation of mine. >>>>>> I can post this patch too if you think its relevant. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> On Monday, January 14, 2013 5:38:53 PM UTC-2, Felipe Meirelles wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> When you try to make more than one left join on a SQLFORM.grid as: >>>>>>> >>>>>>> grid = SQLFORMCustom.grid(db.device, >>>>>>> fields=[db.device.serial, db.device.device_type, >>>>>>> db.device.vehicle, db.vehicle.plate, db.chip.imei, ], >>>>>>> field_id=db.device.id, >>>>>>> left=[db.chip.on(db.chip.id==**db.device.chip), >>>>>>> db.vehicle.on(db.vehicle.id==**db.device.vehicle)], >>>>>>> paginate=max_results, >>>>>>> showbuttontext=False, >>>>>>> formstyle='divs', >>>>>>> paginate_icons=paginate_icons, >>>>>>> sorter_icons=sorter_icons >>>>>>> ) >>>>>>> >>>>>>> it can't display the join fields as it dosen't find the fields on >>>>>>> the query in sqlhtml.py line 1083. >>>>>>> >>>>>>> Can you apply this patch on line 1083? >>>>>>> >>>>>>> if left is not None: >>>>>>> if isinstance(left, (list)): >>>>>>> for l in left: >>>>>>> tablenames += db._adapter.tables(l) >>>>>>> else: >>>>>>> tablenames += db._adapter.tables(left) >>>>>>> >>>>>> -- > > --- > You received this message because you are subscribed to the Google Groups > "web2py-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to web2py+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- Att, Felipe Meirelles. -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Grid with multiple left joins
Did anyone saw it? On Monday, January 14, 2013 5:41:36 PM UTC-2, Felipe Meirelles wrote: > > Please desconsider the sorter_icons parameter, its a custom implementation > of mine. > I can post this patch too if you think its relevant. > > Thanks! > > On Monday, January 14, 2013 5:38:53 PM UTC-2, Felipe Meirelles wrote: >> >> Hi, >> >> When you try to make more than one left join on a SQLFORM.grid as: >> >> grid = SQLFORMCustom.grid(db.device, >> fields=[db.device.serial, db.device.device_type, >> db.device.vehicle, db.vehicle.plate, db.chip.imei, ], >> field_id=db.device.id, >> left=[db.chip.on(db.chip.id==db.device.chip), db.vehicle.on( >> db.vehicle.id==db.device.vehicle)], >> paginate=max_results, >> showbuttontext=False, >> formstyle='divs', >> paginate_icons=paginate_icons, >> sorter_icons=sorter_icons >> ) >> >> it can't display the join fields as it dosen't find the fields on the >> query in sqlhtml.py line 1083. >> >> Can you apply this patch on line 1083? >> >> if left is not None: >> if isinstance(left, (list)): >> for l in left: >> tablenames += db._adapter.tables(l) >> else: >> tablenames += db._adapter.tables(left) >> > --
[web2py] Re: Grid with multiple left joins
Please desconsider the sorter_icons parameter, its a custom implementation of mine. I can post this patch too if you think its relevant. Thanks! On Monday, January 14, 2013 5:38:53 PM UTC-2, Felipe Meirelles wrote: > > Hi, > > When you try to make more than one left join on a SQLFORM.grid as: > > grid = SQLFORMCustom.grid(db.device, > fields=[db.device.serial, db.device.device_type, > db.device.vehicle, db.vehicle.plate, db.chip.imei, ], > field_id=db.device.id, > left=[db.chip.on(db.chip.id==db.device.chip), db.vehicle.on( > db.vehicle.id==db.device.vehicle)], > paginate=max_results, > showbuttontext=False, > formstyle='divs', > paginate_icons=paginate_icons, > sorter_icons=sorter_icons > ) > > it can't display the join fields as it dosen't find the fields on the > query in sqlhtml.py line 1083. > > Can you apply this patch on line 1083? > > if left is not None: > if isinstance(left, (list)): > for l in left: > tablenames += db._adapter.tables(l) > else: > tablenames += db._adapter.tables(left) > --
[web2py] Grid with multiple left joins
Hi, When you try to make more than one left join on a SQLFORM.grid as: grid = SQLFORMCustom.grid(db.device, fields=[db.device.serial, db.device.device_type, db.device.vehicle, db.vehicle.plate, db.chip.imei, ], field_id=db.device.id, left=[db.chip.on(db.chip.id==db.device.chip), db.vehicle.on(db.vehicle.id==db.device.vehicle)], paginate=max_results, showbuttontext=False, formstyle='divs', paginate_icons=paginate_icons, sorter_icons=sorter_icons ) it can't display the join fields as it dosen't find the fields on the query in sqlhtml.py line 1083. Can you apply this patch on line 1083? if left is not None: if isinstance(left, (list)): for l in left: tablenames += db._adapter.tables(l) else: tablenames += db._adapter.tables(left) --
Re: [web2py] Re: Add a WHERE clause to every request
The tested solution: for table in db.tables: if('auth_view' in db[table]): filter_query = db[table].auth_view.belongs(db.view_membership.auth_user == auth.user.id) db[table]._common_filter = lambda query, f=filter_query: f On Friday, January 11, 2013 5:39:05 PM UTC-2, Felipe Meirelles wrote: > > It worked. Is there a way to debug SQL on appengine. My DEBUG env_var is > set to True but it dosen't output any SQL. > > On Friday, January 11, 2013 4:52:54 PM UTC-2, Felipe Meirelles wrote: >> >> I'll test. Thanks (: >> >> On Friday, January 11, 2013 3:21:40 PM UTC-2, rochacbruno wrote: >>> >>> little fix. >>> >>> On Fri, Jan 11, 2013 at 3:21 PM, Bruno Rocha wrote: >>> >>>> for table in db.tables: >>>> db[table]._common_filter = lambda query: db[table].is_active == >>>> True >>>> >>> >>> >>> --
Re: [web2py] Re: Add a WHERE clause to every request
It worked. Is there a way to debug SQL on appengine. My DEBUG env_var is set to True but it dosen't output any SQL. On Friday, January 11, 2013 4:52:54 PM UTC-2, Felipe Meirelles wrote: > > I'll test. Thanks (: > > On Friday, January 11, 2013 3:21:40 PM UTC-2, rochacbruno wrote: >> >> little fix. >> >> On Fri, Jan 11, 2013 at 3:21 PM, Bruno Rocha wrote: >> >>> for table in db.tables: >>> db[table]._common_filter = lambda query: db[table].is_active == True >>> >> >> >> --
Re: [web2py] Re: Add a WHERE clause to every request
I'll test. Thanks (: On Friday, January 11, 2013 3:21:40 PM UTC-2, rochacbruno wrote: > > little fix. > > On Fri, Jan 11, 2013 at 3:21 PM, Bruno Rocha > > wrote: > >> for table in db.tables: >> db[table]._common_filter = lambda query: db[table].is_active == True >> > > > --
[web2py] Add a WHERE clause to every request
Is there possible add a WHERE clause to every query that runs in the application (like record versioning is_active)? --
[web2py] Re: GAE + Cloud SQL: Tickets
Now its working for all requests, but the solution is a little odd: First try to get the table, then try to define it for the first time and if it fails, try to just define the table, with no migrate. Is this OK? def _get_table(self, db, tablename, app): tablename = tablename + '_' + app logging.info(tablename) table = None try: table = db[tablename] except: db.rollback() # not necessary but one day # any app may store tickets on DB logging.info("Trying to create") try: db.define_table( tablename, db.Field('ticket_id', length=100), db.Field('ticket_data', 'text'), db.Field('created_datetime', 'datetime'), ) table = db[tablename] except: try: db.define_table( tablename, db.Field('ticket_id', length=100), db.Field('ticket_data', 'text'), db.Field('created_datetime', 'datetime'), migrate=False ) table = db[tablename] except: pass return table On Tuesday, January 8, 2013 4:34:39 PM UTC-2, Felipe Meirelles wrote: > > Never mind, this solution just worked the first time, when I need to > define the table, the second time it fails too... > > On Tuesday, January 8, 2013 4:21:01 PM UTC-2, Felipe Meirelles wrote: >> >> Well, the real problem is I'm using GAE with Google Cloud SQL, so I have >> the migrations problem but here is the fix: >> >> On restricted.py line 72: >> >> def _get_table(self, db, tablename, app): >> tablename = tablename + '_' + app >> try: >> table = db[tablename] >> except: >> db.rollback() # not necessary but one day >> # any app may store tickets on DB >> table = db.define_table( >> tablename, >> db.Field('ticket_id', length=100), >> db.Field('ticket_data', 'text'), >> db.Field('created_datetime', 'datetime'), >> ) >> logging.info(table) >> return table >> >> The problem is web2py was trying to create the table in every request, >> even when it already exists (don't know why, but with lazy tables >> db.get(tablename, None) was returning none always). So I swaped to >> db[tablename] and added a exception handling for it. Seem to work just fine >> now. >> >> Can you take a look and see if this dosen't conflicts with another DBs to >> move it to trunk? Thanks! >> >> Thanks. >> >> On Friday, January 4, 2013 10:58:07 PM UTC-2, Massimo Di Pierro wrote: >>> >>> Admin is readonly on GAE and by default not deployed. >>> >>> On Friday, 4 January 2013 17:10:56 UTC-6, Alan Etkin wrote: >>>> >>>> El jueves, 3 de enero de 2013 11:31:45 UTC-3, Felipe Meirelles escribió: >>>>> >>>>> Well, the problem is when the ticket is saved, self.db points to >>>> uri="google:sql://**:novello-solutionworkshop:novello-solutionworkshop/novello_test"> >>>>> >>>>> but when load() is called, it points to . >>>> >>>> >>>> AFAIK admin is db less. It uses the db of the app it manages. Besides, >>>> wasn't admin disabled for GAE? >>>> >>>> --
[web2py] Re: GAE + Cloud SQL: Tickets
Never mind, this solution just worked the first time, when I need to define the table, the second time it fails too... On Tuesday, January 8, 2013 4:21:01 PM UTC-2, Felipe Meirelles wrote: > > Well, the real problem is I'm using GAE with Google Cloud SQL, so I have > the migrations problem but here is the fix: > > On restricted.py line 72: > > def _get_table(self, db, tablename, app): > tablename = tablename + '_' + app > try: > table = db[tablename] > except: > db.rollback() # not necessary but one day > # any app may store tickets on DB > table = db.define_table( > tablename, > db.Field('ticket_id', length=100), > db.Field('ticket_data', 'text'), > db.Field('created_datetime', 'datetime'), > ) > logging.info(table) > return table > > The problem is web2py was trying to create the table in every request, > even when it already exists (don't know why, but with lazy tables > db.get(tablename, None) was returning none always). So I swaped to > db[tablename] and added a exception handling for it. Seem to work just fine > now. > > Can you take a look and see if this dosen't conflicts with another DBs to > move it to trunk? Thanks! > > Thanks. > > On Friday, January 4, 2013 10:58:07 PM UTC-2, Massimo Di Pierro wrote: >> >> Admin is readonly on GAE and by default not deployed. >> >> On Friday, 4 January 2013 17:10:56 UTC-6, Alan Etkin wrote: >>> >>> El jueves, 3 de enero de 2013 11:31:45 UTC-3, Felipe Meirelles escribió: >>>> >>>> Well, the problem is when the ticket is saved, self.db points to >>> uri="google:sql://**:novello-solutionworkshop:novello-solutionworkshop/novello_test"> >>>> >>>> but when load() is called, it points to . >>> >>> >>> AFAIK admin is db less. It uses the db of the app it manages. Besides, >>> wasn't admin disabled for GAE? >>> >>> --
[web2py] Re: GAE + Cloud SQL: Tickets
Well, the real problem is I'm using GAE with Google Cloud SQL, so I have the migrations problem but here is the fix: On restricted.py line 72: def _get_table(self, db, tablename, app): tablename = tablename + '_' + app try: table = db[tablename] except: db.rollback() # not necessary but one day # any app may store tickets on DB table = db.define_table( tablename, db.Field('ticket_id', length=100), db.Field('ticket_data', 'text'), db.Field('created_datetime', 'datetime'), ) logging.info(table) return table The problem is web2py was trying to create the table in every request, even when it already exists (don't know why, but with lazy tables db.get(tablename, None) was returning none always). So I swaped to db[tablename] and added a exception handling for it. Seem to work just fine now. Can you take a look and see if this dosen't conflicts with another DBs to move it to trunk? Thanks! Thanks. On Friday, January 4, 2013 10:58:07 PM UTC-2, Massimo Di Pierro wrote: > > Admin is readonly on GAE and by default not deployed. > > On Friday, 4 January 2013 17:10:56 UTC-6, Alan Etkin wrote: >> >> El jueves, 3 de enero de 2013 11:31:45 UTC-3, Felipe Meirelles escribió: >>> >>> Well, the problem is when the ticket is saved, self.db points to >> uri="google:sql://**:novello-solutionworkshop:novello-solutionworkshop/novello_test"> >>> >>> but when load() is called, it points to . >> >> >> AFAIK admin is db less. It uses the db of the app it manages. Besides, >> wasn't admin disabled for GAE? >> >> --
[web2py] Re: GAE + Cloud SQL: Tickets
Well, the problem is when the ticket is saved, self.db points to but when load() is called, it points to . Where the admin application DAL is defined? On Thursday, January 3, 2013 11:47:38 AM UTC-2, Felipe Meirelles wrote: > > Actualy, it is displaying the SQL as: > > select [, 0x126F9170>, , object at 0x126F9210>] where u'127.0.0.1.2013-01-03.13-22-37.4b65b4db-1e4f-48f1-baca-5f0201fdb53a': 'unicode'>)]> > > > On Thursday, January 3, 2013 11:36:15 AM UTC-2, Felipe Meirelles wrote: >> >> Just added a self.db.commit() after the insert command, and it worked as >> spected, but the load() function cant retrieve the ticket data... >> >> I've tried to inspect the select stmt with _select(), but it dosent >> returns the SQL. Any ideas? >> >> On Thursday, January 3, 2013 11:07:04 AM UTC-2, Felipe Meirelles wrote: >>> >>> Massimo, just figured it out: >>> >>> Using App Engine + Lazy Tables results on _get_table() in restricted.py >>> line 77 returning None as the tablename all the times. >>> >>> So I tried to set lazy_tables = False, then the table is found, but when >>> >>> table.insert(ticket_id=ticket_id, >>> ticket_data=cPickle.dumps(ticket_data), >>> created_datetime=request.now) >>> >>> is called, it returns the right ID, but is not inserted on the >>> database... >>> >>> Can you guys take a look? >>> >>> Thanks >>> >>> >>> On Tuesday, December 18, 2012 2:52:56 PM UTC-2, Massimo Di Pierro wrote: >>>> >>>> If there is no ticket here is a problem talking to database or other >>>> internal web2py error, not an application error. Please check for >>>> traceback >>>> in the app engine logs. >>>> >>>> On Tuesday, 18 December 2012 06:08:56 UTC-6, Felipe Meirelles wrote: >>>>> >>>>> Hi, I'm using GAE + Cloud SQL but every time I get an error, there is >>>>> no ticket. Can I set web2py to save tickets to database as in Google >>>>> NoSQL >>>>> (big table)? >>>> >>>> --
[web2py] Re: GAE + Cloud SQL: Tickets
Actualy, it is displaying the SQL as: select [, , , ] where )]> On Thursday, January 3, 2013 11:36:15 AM UTC-2, Felipe Meirelles wrote: > > Just added a self.db.commit() after the insert command, and it worked as > spected, but the load() function cant retrieve the ticket data... > > I've tried to inspect the select stmt with _select(), but it dosent > returns the SQL. Any ideas? > > On Thursday, January 3, 2013 11:07:04 AM UTC-2, Felipe Meirelles wrote: >> >> Massimo, just figured it out: >> >> Using App Engine + Lazy Tables results on _get_table() in restricted.py >> line 77 returning None as the tablename all the times. >> >> So I tried to set lazy_tables = False, then the table is found, but when >> >> table.insert(ticket_id=ticket_id, >> ticket_data=cPickle.dumps(ticket_data), >> created_datetime=request.now) >> >> is called, it returns the right ID, but is not inserted on the database... >> >> Can you guys take a look? >> >> Thanks >> >> >> On Tuesday, December 18, 2012 2:52:56 PM UTC-2, Massimo Di Pierro wrote: >>> >>> If there is no ticket here is a problem talking to database or other >>> internal web2py error, not an application error. Please check for traceback >>> in the app engine logs. >>> >>> On Tuesday, 18 December 2012 06:08:56 UTC-6, Felipe Meirelles wrote: >>>> >>>> Hi, I'm using GAE + Cloud SQL but every time I get an error, there is >>>> no ticket. Can I set web2py to save tickets to database as in Google NoSQL >>>> (big table)? >>> >>> --
[web2py] Re: GAE + Cloud SQL: Tickets
Just added a self.db.commit() after the insert command, and it worked as spected, but the load() function cant retrieve the ticket data... I've tried to inspect the select stmt with _select(), but it dosent returns the SQL. Any ideas? On Thursday, January 3, 2013 11:07:04 AM UTC-2, Felipe Meirelles wrote: > > Massimo, just figured it out: > > Using App Engine + Lazy Tables results on _get_table() in restricted.py > line 77 returning None as the tablename all the times. > > So I tried to set lazy_tables = False, then the table is found, but when > > table.insert(ticket_id=ticket_id, > ticket_data=cPickle.dumps(ticket_data), > created_datetime=request.now) > > is called, it returns the right ID, but is not inserted on the database... > > Can you guys take a look? > > Thanks > > > On Tuesday, December 18, 2012 2:52:56 PM UTC-2, Massimo Di Pierro wrote: >> >> If there is no ticket here is a problem talking to database or other >> internal web2py error, not an application error. Please check for traceback >> in the app engine logs. >> >> On Tuesday, 18 December 2012 06:08:56 UTC-6, Felipe Meirelles wrote: >>> >>> Hi, I'm using GAE + Cloud SQL but every time I get an error, there is no >>> ticket. Can I set web2py to save tickets to database as in Google NoSQL >>> (big table)? >> >> --
[web2py] Re: GAE + Cloud SQL: Tickets
Massimo, just figured it out: Using App Engine + Lazy Tables results on _get_table() in restricted.py line 77 returning None as the tablename all the times. So I tried to set lazy_tables = False, then the table is found, but when table.insert(ticket_id=ticket_id, ticket_data=cPickle.dumps(ticket_data), created_datetime=request.now) is called, it returns the right ID, but is not inserted on the database... Can you guys take a look? Thanks On Tuesday, December 18, 2012 2:52:56 PM UTC-2, Massimo Di Pierro wrote: > > If there is no ticket here is a problem talking to database or other > internal web2py error, not an application error. Please check for traceback > in the app engine logs. > > On Tuesday, 18 December 2012 06:08:56 UTC-6, Felipe Meirelles wrote: >> >> Hi, I'm using GAE + Cloud SQL but every time I get an error, there is no >> ticket. Can I set web2py to save tickets to database as in Google NoSQL >> (big table)? > > --
[web2py] Re: Encoding works on production but fails in development
Hey hey, just solved it, if you can patch: dal.py line 4104: def connector(driver_args=driver_args): return rdbms.connect(charset='utf8', **driver_args) the credits goes to this post as in cloud sql documentation is never mentioned this charset parameter to the development db: https://groups.google.com/forum/?fromgroups=#!search/cloud$20sql$20utf8/google-cloud-sql-discuss/cmmNTT1lJ18/ZBzmsYdcdNwJ Thanks all. On Thursday, December 27, 2012 4:47:08 PM UTC-2, Felipe Meirelles wrote: > > It also fails when rendering the grid, but again, only in development > server, wich makes me think its something related to google sdk. > Is there some ideia? > > On Thursday, December 27, 2012 12:51:23 PM UTC-2, Felipe Meirelles wrote: >> >> I'm using GAE + Google Cloud SQL and when I insert some data to local >> server with special chars like in "Água" it saves the data, but when i try >> to retrieve it i get an encoding error from python. >> In production it is automaticaly encoded to \u00c1gua and saves to DB. >> Is it some mysql feature that Google Cloud SQL has as default or some >> diference in the implementation of the adapters? >> >> Thanks >> > --
[web2py] Re: Encoding works on production but fails in development
It also fails when rendering the grid, but again, only in development server, wich makes me think its something related to google sdk. Is there some ideia? On Thursday, December 27, 2012 12:51:23 PM UTC-2, Felipe Meirelles wrote: > > I'm using GAE + Google Cloud SQL and when I insert some data to local > server with special chars like in "Água" it saves the data, but when i try > to retrieve it i get an encoding error from python. > In production it is automaticaly encoded to \u00c1gua and saves to DB. Is > it some mysql feature that Google Cloud SQL has as default or some > diference in the implementation of the adapters? > > Thanks > --
[web2py] Encoding works on production but fails in development
I'm using GAE + Google Cloud SQL and when I insert some data to local server with special chars like in "Água" it saves the data, but when i try to retrieve it i get an encoding error from python. In production it is automaticaly encoded to \u00c1gua and saves to DB. Is it some mysql feature that Google Cloud SQL has as default or some diference in the implementation of the adapters? Thanks --
[web2py] GAE + Cloud SQL: Tickets
Hi, I'm using GAE + Cloud SQL but every time I get an error, there is no ticket. Can I set web2py to save tickets to database as in Google NoSQL (big table)? --
Re: [web2py] Re: Login form password widget chaned
Well, and is there possible to set his placeholder this way? On Fri, Nov 30, 2012 at 2:31 PM, Niphlod wrote: > yep, now it carries the script to enable the entropy color codes. > > Il giorno venerdì 30 novembre 2012 17:21:15 UTC+1, Felipe Meirelles ha > scritto: >> >> def index(): >> form = auth() >> logging.info(form.custom) >> >> , >> 'widget': , >> 'id': '', 'email': }>, 'end': >> , 'linkto': , 'begin': >> , 'submit': > at 0x04A50830>, 'label': > 'email': 'E-mail'}>, 'deletable': '', 'inpval': > 'id': '', 'email': ''}>, 'dspval': > object at 0x04A50C70>, 'id': , >> 'email': }>}> >> >> That password widget used to be a gluon.html.INPUT same as email but now >> it is a CAT >> >> >> On Fri, Nov 30, 2012 at 1:12 PM, Massimo Di Pierro > > wrote: >> >>> What change are you referring to? Can you provide more details? >>> >>> >>> On Friday, November 30, 2012 9:01:18 AM UTC-6, Felipe Meirelles wrote: >>>> >>>> Did you changed the password widget on the login form from INPUT to >>>> CAT? Is there a reason to this change? >>>> >>>> I had a placeholder on it and since DIVs dont have this property, my >>>> placeholders don't work any more. >>>> >>> -- >>> >>> >>> >>> >> >> >> >> -- >> Att, >> Felipe Meirelles. >> >> -- > > > > -- Att, Felipe Meirelles. --
Re: [web2py] Re: Login form password widget chaned
def index(): form = auth() logging.info(form.custom) , 'widget': , 'id': '', 'email': }>, 'end': , 'linkto': , 'begin': , 'submit': , 'label': , 'deletable': '', 'inpval': , 'dspval': , 'id': , 'email': }>}> That password widget used to be a gluon.html.INPUT same as email but now it is a CAT On Fri, Nov 30, 2012 at 1:12 PM, Massimo Di Pierro < massimo.dipie...@gmail.com> wrote: > What change are you referring to? Can you provide more details? > > > On Friday, November 30, 2012 9:01:18 AM UTC-6, Felipe Meirelles wrote: >> >> Did you changed the password widget on the login form from INPUT to CAT? >> Is there a reason to this change? >> >> I had a placeholder on it and since DIVs dont have this property, my >> placeholders don't work any more. >> > -- > > > > -- Att, Felipe Meirelles. --
[web2py] Login form password widget chaned
Did you changed the password widget on the login form from INPUT to CAT? Is there a reason to this change? I had a placeholder on it and since DIVs dont have this property, my placeholders don't work any more. --
Re: [web2py] CASE Tool for python (preferably web2py)
Well, its a good tool but I'm looking for a more complete tool. StarUML is a complete tool and very customizable but i didnt found a pattern to generate python code. If there was one I could modify it to generate web2py DAL definitions. On Tuesday, November 27, 2012 6:00:27 PM UTC-2, Richard wrote: > > Don't know if it still working but SQL Designer was include in web2py in > the pass... Not sure if it was possible to generate model from it... > > Here a article but it in portuguese I think : > > http://www.tuxtilt.com/web2py-modelagem-de-dados/ > > Richard > > On Tue, Nov 27, 2012 at 2:02 PM, Felipe Meirelles < > fel...@felipemeirelles.com.br > wrote: > >> So, is there any tool capable of generating python code from diagrams? >> I was taking a look at Xtext, is there some MDA tool in python language? >> Or any tool designed to generate web2py models from diagrams? >> Or even generate diagrams from web2py models? >> >> Thanks >> >> -- >> >> >> >> > > --
[web2py] CASE Tool for python (preferably web2py)
So, is there any tool capable of generating python code from diagrams? I was taking a look at Xtext, is there some MDA tool in python language? Or any tool designed to generate web2py models from diagrams? Or even generate diagrams from web2py models? Thanks --
[web2py] NDB Support
Hey guys, Is there some effort going on (or at least planed) to support NDB on appengine? Thanks! --
Re: [web2py] Re: Lazy Tables and Cached Results on GAE
Oh, thats nice, all just works the way I use to work with GAE cache. Great work Massimo, hope I can start to contribute soon. Thank you. On Mon, Sep 10, 2012 at 1:08 PM, Massimo Di Pierro < massimo.dipie...@gmail.com> wrote: > You cannot invalidate a single row if you are caching the all set of rows. > > Mind that if you do rows.update_records(...) and rows is cached, it should > update the cached row as well. > > > On Monday, 10 September 2012 08:16:07 UTC-5, Felipe Meirelles wrote: > >> But it will invalidate all the cache data. I want to invalidate just one >> row cache, is this possible? >> >> On Mon, Sep 10, 2012 at 10:14 AM, Massimo Di Pierro < >> massimo@gmail.com> wrote: >> >>> If you use cache.ram or cache.disk you can do >>> >>> cache.ram.clear() >>> cache.disk.clear() >>> >>> this will clear all ram. You can also pass a regex to clear to clear >>> only some keys. >>> >>> On Monday, 10 September 2012 07:49:27 UTC-5, Felipe Meirelles wrote: >>>> >>>> Only one more question, can I enforce the cache to invalidate (as when >>>> I know the record has changed) or update the select cache manulay? >>>> >>>> On Monday, September 10, 2012 9:44:18 AM UTC-3, Massimo Di Pierro wrote: >>>>> >>>>> 2.0.8 is known to work fine on GAE. >>>>> >>>>> On Monday, 10 September 2012 07:05:11 UTC-5, Felipe Meirelles wrote: >>>>>> >>>>>> Anyone tested the new 2.0.2 changes on GAE? Should this work fine? >>>>>> >>>>>> Can someone tell me how the internals work for this two features and >>>>>> how to use it? >>>>>> >>>>>> Thanks. >>>>>> >>>>> -- >>> >>> >>> >>> >> >> >> >> -- >> Att, >> Felipe Meirelles. >> >> -- > > > > -- Att, Felipe Meirelles. --
Re: [web2py] Re: Lazy Tables and Cached Results on GAE
But it will invalidate all the cache data. I want to invalidate just one row cache, is this possible? On Mon, Sep 10, 2012 at 10:14 AM, Massimo Di Pierro < massimo.dipie...@gmail.com> wrote: > If you use cache.ram or cache.disk you can do > > cache.ram.clear() > cache.disk.clear() > > this will clear all ram. You can also pass a regex to clear to clear only > some keys. > > On Monday, 10 September 2012 07:49:27 UTC-5, Felipe Meirelles wrote: >> >> Only one more question, can I enforce the cache to invalidate (as when I >> know the record has changed) or update the select cache manulay? >> >> On Monday, September 10, 2012 9:44:18 AM UTC-3, Massimo Di Pierro wrote: >>> >>> 2.0.8 is known to work fine on GAE. >>> >>> On Monday, 10 September 2012 07:05:11 UTC-5, Felipe Meirelles wrote: >>>> >>>> Anyone tested the new 2.0.2 changes on GAE? Should this work fine? >>>> >>>> Can someone tell me how the internals work for this two features and >>>> how to use it? >>>> >>>> Thanks. >>>> >>> -- > > > > -- Att, Felipe Meirelles. --
[web2py] Re: Lazy Tables and Cached Results on GAE
Only one more question, can I enforce the cache to invalidate (as when I know the record has changed) or update the select cache manulay? On Monday, September 10, 2012 9:44:18 AM UTC-3, Massimo Di Pierro wrote: > > 2.0.8 is known to work fine on GAE. > > On Monday, 10 September 2012 07:05:11 UTC-5, Felipe Meirelles wrote: >> >> Anyone tested the new 2.0.2 changes on GAE? Should this work fine? >> >> Can someone tell me how the internals work for this two features and how >> to use it? >> >> Thanks. >> > --
[web2py] Lazy Tables and Cached Results on GAE
Anyone tested the new 2.0.2 changes on GAE? Should this work fine? Can someone tell me how the internals work for this two features and how to use it? Thanks. --
Re: [web2py] SQLFORM.smartgrid field labels not I18N
Nah, I'll do it, the community is so supportive and its make me on the mood to contribute (: I'll just will need some help to send you a patch, never did it on github On Thu, Aug 16, 2012 at 10:53 AM, Massimo Di Pierro < massimo.dipie...@gmail.com> wrote: > That would be great. Or simply list the parts not internationalized > > -- > > > > -- Att, Felipe Meirelles. --
[web2py] SQLFORM.smartgrid field labels not I18N
Yarrr! So, I'm using the smartgrid beta and noticed that some parts of it are not I18N. Should I fix it and send a patch or you guys are working on it already? Thanks --
Re: [web2py] Re: Auth and Model less
I'm also using memcache to store the session as follows: from gluon.contrib.gae_memcache import MemcacheClient from gluon.contrib.memdb import MEMDB cache.memcache = MemcacheClient(request) cache.ram = cache.disk = cache.memcache session.connect(request,response,MEMDB(cache.memcache)) On Tue, Aug 14, 2012 at 1:52 PM, Felipe Meirelles wrote: > Well, but even initializing db and auth for evey request I still get the > not validating bug... > > ill attach my code to you to take a look. > > > On Tue, Aug 14, 2012 at 1:44 PM, Felipe Meirelles wrote: > >> Sorry, the DB was still static. Just created a init_db() and worked as >> spected. >> >> >> On Tue, Aug 14, 2012 at 1:18 PM, Anthony wrote: >> >>> Can you show the exact code you're using in the module and controller? >>> >>> >>> On Tuesday, August 14, 2012 12:07:15 PM UTC-4, Felipe Meirelles wrote: >>> >>>> Now I'm getting this error when I call the initialization method: >>>> >>>> SyntaxError: invalid table name: auth_user >>>> >>>> On Tue, Aug 14, 2012 at 10:38 AM, Anthony wrote: >>>> >>>>> On Tuesday, August 14, 2012 9:14:21 AM UTC-4, Felipe Meirelles wrote: >>>>>> >>>>>> But the DB object can be static through requests as well or I should >>>>>> instantiate it on every request too? >>>>>> >>>>> >>>>> You should instantiate on every request (that needs the db). >>>>> >>>>> Anthony >>>>> >>>>> -- >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Att, >>>> Felipe Meirelles. >>>> >>>> -- >>> >>> >>> >>> >> >> >> >> -- >> Att, >> Felipe Meirelles. >> >> > > > -- > Att, > Felipe Meirelles. > > -- Att, Felipe Meirelles. --
Re: [web2py] Re: Auth and Model less
Well, but even initializing db and auth for evey request I still get the not validating bug... ill attach my code to you to take a look. On Tue, Aug 14, 2012 at 1:44 PM, Felipe Meirelles wrote: > Sorry, the DB was still static. Just created a init_db() and worked as > spected. > > > On Tue, Aug 14, 2012 at 1:18 PM, Anthony wrote: > >> Can you show the exact code you're using in the module and controller? >> >> >> On Tuesday, August 14, 2012 12:07:15 PM UTC-4, Felipe Meirelles wrote: >> >>> Now I'm getting this error when I call the initialization method: >>> >>> SyntaxError: invalid table name: auth_user >>> >>> On Tue, Aug 14, 2012 at 10:38 AM, Anthony wrote: >>> >>>> On Tuesday, August 14, 2012 9:14:21 AM UTC-4, Felipe Meirelles wrote: >>>>> >>>>> But the DB object can be static through requests as well or I should >>>>> instantiate it on every request too? >>>>> >>>> >>>> You should instantiate on every request (that needs the db). >>>> >>>> Anthony >>>> >>>> -- >>>> >>>> >>>> >>>> >>> >>> >>> >>> -- >>> Att, >>> Felipe Meirelles. >>> >>> -- >> >> >> >> > > > > -- > Att, > Felipe Meirelles. > > -- Att, Felipe Meirelles. -- {{=form}} {{if request.args(0)=='login':}} {{if 0:}} {{if not 'register' in auth.settings.actions_disabled:}} {{=T("register")}} {{pass}} {{if not 'request_reset_password' in auth.settings.actions_disabled:}} {{=T("lost password")}} {{pass}} {{pass}} {{pass}} auth.py Description: Binary data db.py Description: Binary data default.py Description: Binary data
Re: [web2py] Re: Auth and Model less
Sorry, the DB was still static. Just created a init_db() and worked as spected. On Tue, Aug 14, 2012 at 1:18 PM, Anthony wrote: > Can you show the exact code you're using in the module and controller? > > > On Tuesday, August 14, 2012 12:07:15 PM UTC-4, Felipe Meirelles wrote: > >> Now I'm getting this error when I call the initialization method: >> >> SyntaxError: invalid table name: auth_user >> >> On Tue, Aug 14, 2012 at 10:38 AM, Anthony wrote: >> >>> On Tuesday, August 14, 2012 9:14:21 AM UTC-4, Felipe Meirelles wrote: >>>> >>>> But the DB object can be static through requests as well or I should >>>> instantiate it on every request too? >>>> >>> >>> You should instantiate on every request (that needs the db). >>> >>> Anthony >>> >>> -- >>> >>> >>> >>> >> >> >> >> -- >> Att, >> Felipe Meirelles. >> >> -- > > > > -- Att, Felipe Meirelles. --
Re: [web2py] Re: Auth and Model less
Now I'm getting this error when I call the initialization method: SyntaxError: invalid table name: auth_user On Tue, Aug 14, 2012 at 10:38 AM, Anthony wrote: > On Tuesday, August 14, 2012 9:14:21 AM UTC-4, Felipe Meirelles wrote: >> >> But the DB object can be static through requests as well or I should >> instantiate it on every request too? >> > > You should instantiate on every request (that needs the db). > > Anthony > > -- > > > > -- Att, Felipe Meirelles. --
Re: [web2py] Re: Auth and Model less
But the DB object can be static through requests as well or I should instantiate it on every request too? On Tue, Aug 14, 2012 at 10:02 AM, Anthony wrote: > modules/web2py/auth.py: >> >> auth = Auth(db, hmac_key=Auth.get_or_create_**key()) >> > > Where does that db object come from? Note, global objects are defined only > once when the module is first imported, so any such objects are associated > with the first request and are not re-created with each request. Instead, > you should put the Auth code inside a function or class and pass in the db > object from the controller: > > module: > > def initialize_auth(db): > auth = Auth(db, ...) > auth.define_tables > return auth > > controller: > > import initialize_auth > auth = initialize_auth(db) > > Anthony > > -- > > > > -- Att, Felipe Meirelles. --
[web2py] Auth and Model less
Hello again, I'm having some trouble with my model less aproch with auth module. I've put him on a separeted file on modules folder, and, when needed, I import it as follows: modules/web2py/auth.py: auth = Auth(db, hmac_key=Auth.get_or_create_key()) controllers/tracking.py: from applications.main.modules.web2py.auth import auth But some times (its not every time) when I try to loggin, the form just don't validate and don't pass through the login/password for checking (I'm also using the default login form from docs). Maybe this have some relation with imports being done only once on appengine? And, if is it, what can I do about (since not every page on my app needs auth) Thanks --
Re: [web2py] Re: Appengine CPU cycles
Just a last update about the tests, my overall application cost (in dolars from appengine) droped 25% with the change, so, at least in appengine lazy models are a must. On Wednesday, August 8, 2012 12:24:37 PM UTC-3, Jonathan Lundell wrote: > > On 8 Aug 2012, at 7:59 AM, Massimo Di Pierro > > > wrote: > > It is likely Django uses simplejson binary while web2py (for portability) > only ships with simplejson in pure python. This may be another performance > loss if you use json a lot. We need to fix this at the web2py level. > > > This is fixed in at least some places in web2py, where we try to import > json from the standard library before falling back on python solutions. But > not everywhere, I think. > --
Re: [web2py] Re: GAE datastore: how to unindex a field?
Just reviving this topic. Its not setable via index.yaml. On Wednesday, May 16, 2012 1:07:10 PM UTC-3, howesc wrote: > > actually the question at hand here i think is whether or not web2py can > support indexed=False in the property constructor when defining GAE > models: > > https://developers.google.com/appengine/docs/python/datastore/propertyclass > > class Property(verbose_name=None, name=None, default=None, > required=False, validator=None, choices=None, indexed=True) > > i was hoping there was a way to specify that in index.yaml so that we > didn't have to do extra work in the web2py GAE dal. i have not found > that there is or is not a way to specify that in index.yaml. > > cfh > > On 5/16/12 8:39 , Wikus van de Merwe wrote: > > This is really a GAE question, not related to web2py. By default the > > indexes are set automatically when you run your app in dev environment > > depending on what queries you use. But you can do it manually to by > editing > > the index.yaml (above the #AUTOGENERATED line). On the next deploy the > > indexes will be updated. > > > > To disable the autogenerated indexes and test your index definitions, > you > > can run your app locally with: > > dev_appserver.py --require_indexes > > > > Read this article for more details: > > https://developers.google.com/appengine/articles/indexselection > > > > > --
Re: [web2py] Re: Appengine CPU cycles
Sorry about the URL confusion. The processing on web2py version is done on /queues/position, while in the django one is done in /api/position as I didnt implemented queues on the django version. Json dumps and loads are only used on web2py version, but not using the shipped simplejson (it realy had a performace issue) its done using native json lib for python 2.7. I just posted the django performance for a matter of rough comparsion, as the code and the aproch for both are sightly diferent, so the performance should be too. The main comparsion is about web2py with or without the models. On Wed, Aug 8, 2012 at 11:59 AM, Massimo Di Pierro < massimo.dipie...@gmail.com> wrote: > I do not understand. When you compare Django vs web2p[y performance, you > seem to be comparing the MCycles for > /api/1.0/position<https://appengine.google.com/logs?version_id=production.359553040742132058&app_id=s%7Egps-holdsat-hrd&filter_type=labels&filter=path%3A%22%2Fapi%2F1.0%2Fposition%22&severity_level_override=1&view=Search> > (in > case of Django) with > /main/queues/position<https://appengine.google.com/logs?version_id=testing.360875204440667710&app_id=s%7Eapache-felipemeirelles&filter_type=labels&filter=path%3A%22%2Fmain%2Fqueues%2Fposition%22&severity_level_override=1&view=Search> > (in > the case of web2py). Shouldn't you be comparing with > /main/api/position<https://appengine.google.com/logs?version_id=testing.360875204440667710&app_id=s%7Eapache-felipemeirelles&filter_type=labels&filter=path%3A%22%2Fmain%2Fapi%2Fposition%22&severity_level_override=1&view=Search>? > Or perhaps the names are reversed? > > Do they perform the same DB IO? Do they both store sessions in DB (web2py > does)? > > It is likely Django uses simplejson binary while web2py (for portability) > only ships with simplejson in pure python. This may be another performance > loss if you use json a lot. We need to fix this at the web2py level. > > Massimo > > On Wednesday, 8 August 2012 08:06:05 UTC-5, Felipe Meirelles wrote: >> >> Well, after 24h running the new model less code I think I can post some >> data: >> >> First of all I'll explain my application archtecture: >> >> I have a cloud server gattering information from around 150 gps on some >> asserts. Each assert sends information every 30s, and some sends additional >> reading information, wich give me a total arround 150k request/day. This >> cloud server is integrated with appengine trought a REST API, speaching >> Json to each other. >> >> Before the model less refactor I had the following status: >> *Current Load [image: >> help]<https://developers.google.com/appengine/kb/general#currentload> >> * URI Req/Seccurrent Requestslast 6 hrs Runtime MCycleslast hr Avg >> Latency >> /main/queues/position<https://appengine.google.com/logs?version_id=testing.360875204440667710&app_id=s%7Eapache-felipemeirelles&filter_type=labels&filter=path%3A%22%2Fmain%2Fqueues%2Fposition%22&severity_level_override=1&view=Search> >> 1.2 48.78K 351 1200ms >> /main/api/position<https://appengine.google.com/logs?version_id=testing.360875204440667710&app_id=s%7Eapache-felipemeirelles&filter_type=labels&filter=path%3A%22%2Fmain%2Fapi%2Fposition%22&severity_level_override=1&view=Search> >> 0.9 44.37K 267 600 ms >> As you can see, I have controllers, the /api/ receives a call from the >> cloud server and enqueues a process request on appengine (this is a very >> simple controller, no data processing at all), and the /queues/ wich >> actualy process the data, make some inserts/updates and handle the memcache. >> >> After the refactor: >> >> *Current Load [image: >> help]<https://developers.google.com/appengine/kb/general#currentload> >> * URI Req/Seccurrent Requestslast 6 hrs Runtime MCycleslast hr Avg >> Latency >> /main/queues/position<https://appengine.google.com/logs?version_id=testing.360875204440667710&app_id=s%7Eapache-felipemeirelles&filter_type=labels&filter=path%3A%22%2Fmain%2Fqueues%2Fposition%22&severity_level_override=1&view=Search> >> 1.1 49.34K 180 404 ms >> /main/api/position<https://appengine.google.com/logs?version_id=testing.360875204440667710&app_id=s%7Eapache-felipemeirelles&filter_type=labels&filter=path%3A%22%2Fmain%2Fapi%2Fposition%22&severity_level_override=1&view=Search> >> 0.9 44.80K 78 139 ms >> Im only posting the position URL becouse its the most complex and most >> hitted. >> >> In comparsion, the Django project (same logic, I have it still runing >> while the web2py
Re: [web2py] Re: Appengine CPU cycles
Well, after 24h running the new model less code I think I can post some data: First of all I'll explain my application archtecture: I have a cloud server gattering information from around 150 gps on some asserts. Each assert sends information every 30s, and some sends additional reading information, wich give me a total arround 150k request/day. This cloud server is integrated with appengine trought a REST API, speaching Json to each other. Before the model less refactor I had the following status: *Current Load [image: help]<https://developers.google.com/appengine/kb/general#currentload> *URI Req/SeccurrentRequestslast 6 hrsRuntime MCycleslast hrAvg Latency /main/queues/position<https://appengine.google.com/logs?version_id=testing.360875204440667710&app_id=s%7Eapache-felipemeirelles&filter_type=labels&filter=path%3A%22%2Fmain%2Fqueues%2Fposition%22&severity_level_override=1&view=Search> 1.248.78K3511200ms /main/api/position<https://appengine.google.com/logs?version_id=testing.360875204440667710&app_id=s%7Eapache-felipemeirelles&filter_type=labels&filter=path%3A%22%2Fmain%2Fapi%2Fposition%22&severity_level_override=1&view=Search> 0.944.37K267600 ms As you can see, I have controllers, the /api/ receives a call from the cloud server and enqueues a process request on appengine (this is a very simple controller, no data processing at all), and the /queues/ wich actualy process the data, make some inserts/updates and handle the memcache. After the refactor: *Current Load [image: help]<https://developers.google.com/appengine/kb/general#currentload> *URI Req/SeccurrentRequestslast 6 hrsRuntime MCycleslast hrAvg Latency /main/queues/position<https://appengine.google.com/logs?version_id=testing.360875204440667710&app_id=s%7Eapache-felipemeirelles&filter_type=labels&filter=path%3A%22%2Fmain%2Fqueues%2Fposition%22&severity_level_override=1&view=Search> 1.149.34K180404 ms /main/api/position<https://appengine.google.com/logs?version_id=testing.360875204440667710&app_id=s%7Eapache-felipemeirelles&filter_type=labels&filter=path%3A%22%2Fmain%2Fapi%2Fposition%22&severity_level_override=1&view=Search> 0.944.80K78139 ms Im only posting the position URL becouse its the most complex and most hitted. In comparsion, the Django project (same logic, I have it still runing while the web2py one is not ready): *Current Load [image: help]<https://developers.google.com/appengine/kb/general#currentload> *URI Req/SeccurrentRequestslast 6 hrsRuntime MCycleslast hrAvg Latencylast hr /api/1.0/position<https://appengine.google.com/logs?version_id=production.359553040742132058&app_id=s%7Egps-holdsat-hrd&filter_type=labels&filter=path%3A%22%2Fapi%2F1.0%2Fposition%22&severity_level_override=1&view=Search> 0.850.13K92406 ms On the django version I didnt had the queue yet, so this one URL proccess the request at all. Final toughts are, after a model less aproch, I reached half of the performance from django (still, some parts of the code are sightly diferent, and a lot more Json conversion and memcache is used in favour of reducing datastore access, wich can explain the loss on processing performance). Also, after the model less aproch, compared to web2py it self, I got a huge performance gain, and expect a solid cost reduction (I'll post the cost reduction latter, when appengine consolidates the billing). Thanks you all for sharing your toughts. On Tuesday, August 7, 2012 6:56:15 PM UTC-3, Massimo Di Pierro wrote: > > Please share you benchmarks when you think you are done. > > On Tuesday, 7 August 2012 14:29:51 UTC-5, Felipe Meirelles wrote: >> >> Well, I'm still in the middle of tests but all points to a huge drop on >> cpu usage using a model less aproch. I've made, in a few days ill post >> numbers, but the cpu_ms is arround half of it was. >> >> I've set all db.define_table() in separete models, and just include them >> where i use them. Also, all the configs from 0.py were split in files, and >> are just called when I need them. >> >> Thanks for the help! >> >> On Tue, Aug 7, 2012 at 4:25 PM, Anthony > >wrote: >> >>> The app I'm developing is highly dependent on auth. >>>> >>>> Does it mean that any request that uses auth won't be faster than 200ms >>>> on GAE? >>>> >>> >>> Note, if you're just checking for login, I don't think you need to call >>> auth.define_tables() (that should only be needed for registration, login, >>> and checking permissions/membership). Also, make sure to use >>> auth.define_tables(migrate=False) to avoid migrations on the auth tables in >>> production. >>> >>> Anthony >>> >>> -- >>> >>> >>> >>> >> >> >> >> -- >> Att, >> Felipe Meirelles. >> >> --
Re: [web2py] Re: Appengine CPU cycles
Well, I'm still in the middle of tests but all points to a huge drop on cpu usage using a model less aproch. I've made, in a few days ill post numbers, but the cpu_ms is arround half of it was. I've set all db.define_table() in separete models, and just include them where i use them. Also, all the configs from 0.py were split in files, and are just called when I need them. Thanks for the help! On Tue, Aug 7, 2012 at 4:25 PM, Anthony wrote: > The app I'm developing is highly dependent on auth. >> >> Does it mean that any request that uses auth won't be faster than 200ms >> on GAE? >> > > Note, if you're just checking for login, I don't think you need to call > auth.define_tables() (that should only be needed for registration, login, > and checking permissions/membership). Also, make sure to use > auth.define_tables(migrate=False) to avoid migrations on the auth tables in > production. > > Anthony > > -- > > > > -- Att, Felipe Meirelles. --
Re: [web2py] Re: Appengine CPU cycles
Actualy Django uses the datastore too. I'm using the memcache for all things its possible (it already reduce the old django version database access on 90% in this new web2py version). I think the overhead is realy from loading all models, grids and menus I've defined for my app. I'll try a Model less aproch, but tying to right a less complex code, using most of the web2py api as possible. As soon as I finish, I'll post here (: Thank you all. On Monday, August 6, 2012 7:50:59 PM UTC-3, Massimo Di Pierro wrote: > > You should check the {{=response.toolbar()}} because there may be more DB > io than you think in the models. > > You should also try remove the setting of model attributes (requires=, > models=, ...) and move them in the controllers that need them. > > Also mind that web2py on GAE has sessions turned on by default and they > are stored and retrieved from datastore. > > I do not know how Django stores sessions on GAE by default. > > massimo > > > > > > > On Monday, 6 August 2012 14:19:35 UTC-5, Felipe Meirelles wrote: >> >> Without any model I have a huge drop on the cpu use, from around 300ms to >> around 60ms. Still higher than with Django, but its acceptable by the >> concept of the framework. >> Ill make some debug on my models and update the topic. >> >> On Monday, August 6, 2012 4:05:56 PM UTC-3, rochacbruno wrote: >>> >>> What do you have in models? >>> >>> All model files runs alphabetically for each request, so we need to know >>> what are you doing on model files. >>> >>> Can you try to test with an empty brand new app, remove all files from >>> models folder and try your simple controller >>> >>> import logging >>> >>> def test(): >>> logging.info("Just making a performace test") >>> >>> Makes difference? >>> >>> >>> >>> On Mon, Aug 6, 2012 at 3:59 PM, Felipe Meirelles < >>> fel...@felipemeirelles.com.br> wrote: >>> >>>> Even the simplest controller give me a high cpu usage: >>>> >>>> import logging >>>> >>>> def test(): >>>> logging.info("Just making a performace test") >>>> >>>> >>>> ms=559 cpu_ms=612 api_cpu_ms=245 >>>> >>>> >>>> On Monday, August 6, 2012 3:37:38 PM UTC-3, Felipe Meirelles wrote: >>>>> >>>>> Just compiled the app and deployed again, seems to make no diference. >>>>> App engine uses the bytecode generated on web2py? Did I need to change >>>>> some >>>>> config at app.yaml? >>>>> >>>>> On Monday, August 6, 2012 3:21:29 PM UTC-3, Felipe Meirelles wrote: >>>>>> >>>>>> Pre compiled by the admin interface? Not yet, is there some impove >>>>>> with the precompiled code? >>>>>> >>>>>> On Mon, Aug 6, 2012 at 3:16 PM, Derek wrote: >>>>>> >>>>>>> Have you compiled your app? >>>>>>> >>>>>>> On Monday, August 6, 2012 10:46:33 AM UTC-7, Felipe Meirelles wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I'm using web2py in a project that stands on top of a REST api. The >>>>>>>> project is write intensive (around 110k requests a day, with all the >>>>>>>> requests making at least 2 updates to the datastore). >>>>>>>> All my requests seems to uso at least 200 cpu_ms (even dummy >>>>>>>> requests that only places a item on the queue to be processeced >>>>>>>> later). Is >>>>>>>> this behaviour right? I used to get around 20 cpu_ms while using >>>>>>>> django >>>>>>>> with the same application (I'm currently re-writing it on web2py). >>>>>>>> Is there some way to optimize web2py for a appengine deploy (I >>>>>>>> already followed all the documentation) >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>> -- >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Att, >>>>>> Felipe Meirelles. >>>>>> >>>>>> -- >>>> >>>> >>>> >>>> >>> >>> --
Re: [web2py] Re: Appengine CPU cycles
With the basic models (i have 10+ tables on my app) the cpu time jumps from 60ms to 200ms. I have some grids and menus too, if I add that, it goes to my 300ms for a basic controller. Ill try the model less aproch and post the results again. Thanks. On Monday, August 6, 2012 4:27:41 PM UTC-3, rochacbruno wrote: > > On Mon, Aug 6, 2012 at 4:19 PM, Felipe Meirelles < > fel...@felipemeirelles.com.br> wrote: > >> Without any model I have a huge drop on the cpu use, from around 300ms to >> around 60ms. Still higher than with Django, but its acceptable by the >> concept of the framework. > > > Because of that I am using a model-less approach[1], Please share your > tests here, we really need to find the better way to avoid the models and > put some model-less pattern in a documentation. > > [1] > http://www.web2pyslices.com/slice/show/1479/model-less-apps-using-data-models-and-modules-in-web2py > > https://github.com/rochacbruno/web2py_model_less_app > > > --
Re: [web2py] Re: Appengine CPU cycles
Without any model I have a huge drop on the cpu use, from around 300ms to around 60ms. Still higher than with Django, but its acceptable by the concept of the framework. Ill make some debug on my models and update the topic. On Monday, August 6, 2012 4:05:56 PM UTC-3, rochacbruno wrote: > > What do you have in models? > > All model files runs alphabetically for each request, so we need to know > what are you doing on model files. > > Can you try to test with an empty brand new app, remove all files from > models folder and try your simple controller > > import logging > > def test(): > logging.info("Just making a performace test") > > Makes difference? > > > > On Mon, Aug 6, 2012 at 3:59 PM, Felipe Meirelles < > fel...@felipemeirelles.com.br> wrote: > >> Even the simplest controller give me a high cpu usage: >> >> import logging >> >> def test(): >> logging.info("Just making a performace test") >> >> >> ms=559 cpu_ms=612 api_cpu_ms=245 >> >> >> On Monday, August 6, 2012 3:37:38 PM UTC-3, Felipe Meirelles wrote: >>> >>> Just compiled the app and deployed again, seems to make no diference. >>> App engine uses the bytecode generated on web2py? Did I need to change some >>> config at app.yaml? >>> >>> On Monday, August 6, 2012 3:21:29 PM UTC-3, Felipe Meirelles wrote: >>>> >>>> Pre compiled by the admin interface? Not yet, is there some impove with >>>> the precompiled code? >>>> >>>> On Mon, Aug 6, 2012 at 3:16 PM, Derek wrote: >>>> >>>>> Have you compiled your app? >>>>> >>>>> On Monday, August 6, 2012 10:46:33 AM UTC-7, Felipe Meirelles wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> I'm using web2py in a project that stands on top of a REST api. The >>>>>> project is write intensive (around 110k requests a day, with all the >>>>>> requests making at least 2 updates to the datastore). >>>>>> All my requests seems to uso at least 200 cpu_ms (even dummy requests >>>>>> that only places a item on the queue to be processeced later). Is this >>>>>> behaviour right? I used to get around 20 cpu_ms while using django with >>>>>> the >>>>>> same application (I'm currently re-writing it on web2py). >>>>>> Is there some way to optimize web2py for a appengine deploy (I >>>>>> already followed all the documentation) >>>>>> >>>>>> Thanks! >>>>>> >>>>> -- >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Att, >>>> Felipe Meirelles. >>>> >>>> -- >> >> >> >> > > --
Re: [web2py] Re: Appengine CPU cycles
Even the simplest controller give me a high cpu usage: import logging def test(): logging.info("Just making a performace test") ms=559 cpu_ms=612 api_cpu_ms=245 On Monday, August 6, 2012 3:37:38 PM UTC-3, Felipe Meirelles wrote: > > Just compiled the app and deployed again, seems to make no diference. App > engine uses the bytecode generated on web2py? Did I need to change some > config at app.yaml? > > On Monday, August 6, 2012 3:21:29 PM UTC-3, Felipe Meirelles wrote: >> >> Pre compiled by the admin interface? Not yet, is there some impove with >> the precompiled code? >> >> On Mon, Aug 6, 2012 at 3:16 PM, Derek wrote: >> >>> Have you compiled your app? >>> >>> On Monday, August 6, 2012 10:46:33 AM UTC-7, Felipe Meirelles wrote: >>>> >>>> Hi, >>>> >>>> I'm using web2py in a project that stands on top of a REST api. The >>>> project is write intensive (around 110k requests a day, with all the >>>> requests making at least 2 updates to the datastore). >>>> All my requests seems to uso at least 200 cpu_ms (even dummy requests >>>> that only places a item on the queue to be processeced later). Is this >>>> behaviour right? I used to get around 20 cpu_ms while using django with >>>> the >>>> same application (I'm currently re-writing it on web2py). >>>> Is there some way to optimize web2py for a appengine deploy (I already >>>> followed all the documentation) >>>> >>>> Thanks! >>>> >>> -- >>> >>> >>> >>> >> >> >> >> -- >> Att, >> Felipe Meirelles. >> >> --
Re: [web2py] Re: Appengine CPU cycles
Just compiled the app and deployed again, seems to make no diference. App engine uses the bytecode generated on web2py? Did I need to change some config at app.yaml? On Monday, August 6, 2012 3:21:29 PM UTC-3, Felipe Meirelles wrote: > > Pre compiled by the admin interface? Not yet, is there some impove with > the precompiled code? > > On Mon, Aug 6, 2012 at 3:16 PM, Derek wrote: > >> Have you compiled your app? >> >> On Monday, August 6, 2012 10:46:33 AM UTC-7, Felipe Meirelles wrote: >>> >>> Hi, >>> >>> I'm using web2py in a project that stands on top of a REST api. The >>> project is write intensive (around 110k requests a day, with all the >>> requests making at least 2 updates to the datastore). >>> All my requests seems to uso at least 200 cpu_ms (even dummy requests >>> that only places a item on the queue to be processeced later). Is this >>> behaviour right? I used to get around 20 cpu_ms while using django with the >>> same application (I'm currently re-writing it on web2py). >>> Is there some way to optimize web2py for a appengine deploy (I already >>> followed all the documentation) >>> >>> Thanks! >>> >> -- >> >> >> >> > > > > -- > Att, > Felipe Meirelles. > > --
Re: [web2py] Re: Appengine CPU cycles
Pre compiled by the admin interface? Not yet, is there some impove with the precompiled code? On Mon, Aug 6, 2012 at 3:16 PM, Derek wrote: > Have you compiled your app? > > On Monday, August 6, 2012 10:46:33 AM UTC-7, Felipe Meirelles wrote: >> >> Hi, >> >> I'm using web2py in a project that stands on top of a REST api. The >> project is write intensive (around 110k requests a day, with all the >> requests making at least 2 updates to the datastore). >> All my requests seems to uso at least 200 cpu_ms (even dummy requests >> that only places a item on the queue to be processeced later). Is this >> behaviour right? I used to get around 20 cpu_ms while using django with the >> same application (I'm currently re-writing it on web2py). >> Is there some way to optimize web2py for a appengine deploy (I already >> followed all the documentation) >> >> Thanks! >> > -- > > > > -- Att, Felipe Meirelles. --
[web2py] Appengine CPU cycles
Hi, I'm using web2py in a project that stands on top of a REST api. The project is write intensive (around 110k requests a day, with all the requests making at least 2 updates to the datastore). All my requests seems to uso at least 200 cpu_ms (even dummy requests that only places a item on the queue to be processeced later). Is this behaviour right? I used to get around 20 cpu_ms while using django with the same application (I'm currently re-writing it on web2py). Is there some way to optimize web2py for a appengine deploy (I already followed all the documentation) Thanks! --
[web2py] GAE datastore: how to unindex a field?
Hi, I'm porting a commercial project from django + djangoappengine ( http://www.allbuttonspressed.com/) to web2py and one thing I could'nt find at all is how to manage per field indexes on App Engine. My project is database intensive so every entity is written hundreds of thousend times a day and, as you may know, by default all datastore fields are indexed on Bigtable and all indexed fields consume one "Datastore write" tick, and that means I've to pay for it. Is there some way to unindex fields on web2py? Thanks!