[web2py] Filter with IS_IN_DB without changing record presentation format
Hi, I have a field in table 'expenses' defined as: db.define_table('expenses', Field('gang', db.gangs, required=True, notnull=True), ... Field('payer', db.gang_users, label=T('Payer')), ... Table gang_users has a format like: db.define_table('gang_users', Field('gang', db.gangs, required=True, notnull=True), Field('auth_user', db.auth_user, required=True, notnull=True), Field('nickname', 'string', required=False, notnull=False), format = lambda r: %s (%s) % (r.auth_user.email, r.nickname) if r.nickname else r.auth_user.email ) When the payer field is shown in a SQLFORM, it will display a dropdown with the list of users from the table gang_users, formatted according to the auth.user.email (nickname) rule defined in the format parameter of gang_users table. However, I want to filter out this users, by doing in the controller: db.expenses.payer.requires = IS_IN_DB(db(db.gang_users.gang == gang.id), db.gang_users.nickname) When the validator is set, only users from the given gang are displayed in the dropdown, but I can display the nickname or id or any column from the gang_users table. I would like to keep the auth_users.email (gang_users.nickname) format as without the validator. Is it possible? Thanks very much. -- (:=:) Alvaro J. Iradier Muro - airad...@gmail.com -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[web2py] Re: Web2Py Pages module (basic CMS)
Back in 2011 I proposed using quoted identifiers for databases supporting it: https://groups.google.com/d/msg/web2py/aSPtD_mGXdM/4Eflv-edEUEJ I think the check_reserved option is a valid fix, but databases like PostgreSql that allow using any word as identifier by quoting it should use this instead of checking for reserved words. It is a pain to hit this kind of issue when migrating from one database to another. Could this be rethinked again? Thanks. El martes, 6 de mayo de 2014 19:24:15 UTC+2, Rene Dohmen escribió: Hi Villas, page is a restricted word in postgres and some other DB's I think. We started development in web2py 1.99.1 with sqlite. it's a valid sqlite tableName. The problem for us is that we can't rename it very easing without breaking a lot of apps. The problem that we had with building a web2py plugin is that it's not so nice to overwrite db.py; and that has strict sql keyword checks in there by default. I will create a w2p file from it tomorrow. For now you can copy the git tree over an app; and it should work (when you have the needed deps) Online references: http://www.formatics.nl (it's 95% page module) Kind Regards Rene On Friday, February 28, 2014 1:34:50 AM UTC+1, villas wrote: Hi Rene Is there a plugin now? BTW the whole idea of using check_reserved is so you can avoid using reserved names with new projects. Finding a way to avoid that so you can use 'page' negates the purpose. Thanks for your effort to release this work and it will be very nice to see it in action! Best wishes, D On Thursday, 27 February 2014 20:07:21 UTC, Rene Dohmen wrote: This was solved a while ago :) On Friday, December 20, 2013 9:34:21 AM UTC+1, Ruud Schroen wrote: We are still working on a plugin. But we are facing a problem regarding reserved keywords which prevents us from creating a plugin. db = DAL('sqlite://storage.sqlite',pool_size=1,check_reserved=['all']) *check_reserved=['all'] *needs to be removed in order for it to work right now, the problem is you can't overrule it somewhere. If anyone knows a solution that will be greatly appreciated, and we can finally make a plugin for it. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [web2py] Detect web2py main process exit on cron process
Proposition: I have made a small proposed patch to newcron.py, in order to keep a list of cron subprocesses and send a terminate signal when web2py exists. See attached newcron.py.diff. A lock should be added when accessing the _cron_subprocs list, to avoid thread concurrency. Also, I noticed that newcron.stopcron() is called when closing the widget window if the web server was started. But if the web server was not started, it will not be called, which is strange, as cron is started before the web server. The second attached patch (widget.py.diff) fixes this to call newcron.stopcron() on exit. I guess same fix should be applied for winservice.py, and similar for the --nogui case. What do you think about this proposed changes? Thanks. On Tue, Mar 5, 2013 at 10:23 PM, Álvaro José Iradier airad...@gmail.comwrote: Hi, on my web2py application on windows, I have a cron task that runs for a long time. I noticed a new process is spawned for every cron job.. If the main web2py process exists (for example if I try to terminate the process, or close the web2py window), it will not exit until the cron process is finished (is it waiting for the child processes to finish?). Am I right, or do I get something wrong? So my question is, Is there a way to detect from the cron process that the main web2py process is exiting, in order to finish inmediately, but clean? I thought of using the getppid() from the cron process for getting the parent process id, but it is not available on windows. Could the main web2py process send a terminate or interrupt signal or similar to the still running cron processes? This way the process would be killed, or could trap the signal and exit cleanly. Thanks very much. -- --- You received this message because you are subscribed to a topic in the Google Groups web2py-users group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/NzETk-m6Liw/unsubscribe?hl=en. To unsubscribe from this group and all its topics, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- (:=:) Alvaro J. Iradier Muro - airad...@gmail.com -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. newcron.py.diff Description: Binary data widget.py.diff Description: Binary data
Re: [web2py] Re: Import problem in Web2py 2.3.2 vs 1.99.4
Works, but the identation at the else: on line 84 in custom_import.py is wrong. FIxed identation, and it apparently works perfect. Thanks very much! On Wed, Feb 6, 2013 at 6:35 PM, Massimo Di Pierro massimo.dipie...@gmail.com wrote: I agree. I am have attempted a fix. Can you try it? On Wednesday, 6 February 2013 01:23:06 UTC-6, Álvaro José Iradier wrote: I agree they should get priority over system wide modules, but not over imports for files on the same directory. So in Geraldo reports, it does not make sense that in the situation: .../site-packages/geraldo/**generators/__init__.py - from pdf import PDFGenerator .../site-packages/geraldo/**geraldo/generators/pdf.py - PDFGenerator class is in here .../myapp/modules/pdf.py - Nothing to do with Geraldo the pdf.py from modules/ is higher priority than the pdf.py from the geraldo.generators package, when being imported from __init__.py in the same folder. This behavior could unexpectedly break any module or package just because the application uses a modules/whatever.py file, whatever being the same name as any other package. Furthermore, in this case the error when importing geraldo.generators was something like Can not import module geraldo, which is quite misleading, and it took me some work to figure out what was wrong. Maybe the rule should apply when importing from the web2py application, or using local_import, but never when importing from other module files where a simple filename collision inside any package can break the module. Thanks. On Tue, Feb 5, 2013 at 11:24 PM, Massimo Di Pierro massimo@gmail.com wrote: The rule is that if something is imported web2py should first look in app/modules/ because app-level modules should get priority else you cannot override system wide modules. So 2.3.2 is doing the right thing. This did not work well in previous versions. On Tuesday, 5 February 2013 14:40:41 UTC-6, Álvaro José Iradier wrote: Recently I updated web2py from 1.99.4 to 2.3.2. Suddenly, a PDF report stopped working. Digging into the problem, I found there is a *pdf.py* file in my application *modules/* folder. Also, I am using Geraldo Reports in * web2py/site-packages*. Geraldo reports has the following file: *geraldo/generators/pdf.py* and the __init__.py in geraldo/generators/__init__.py does: from pdf import PDFGenerator but when doing *import geraldo.generators*, that line fails, so importing geraldo.generators fails. It fails in 2.3.2, but works in 1.99.4. The fix has been to rename modules/pdf.py in my application to modules/pdfreport.py, because it looks like web2py is trying to import it in the geraldo/generators/__init.py sentence. ¿Is this the expected behavior, or should it look in the __init__,py folder first, as it did in 1.99.4? Thanks. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@**googlegroups.com. For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out . -- (:**=:) Alvaro J. Iradier Muro - aira...@gmail.com -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- (:=:) Alvaro J. Iradier Muro - airad...@gmail.com -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: Import problem in Web2py 2.3.2 vs 1.99.4
I agree they should get priority over system wide modules, but not over imports for files on the same directory. So in Geraldo reports, it does not make sense that in the situation: .../site-packages/geraldo/generators/__init__.py - from pdf import PDFGenerator .../site-packages/geraldo/geraldo/generators/pdf.py - PDFGenerator class is in here .../myapp/modules/pdf.py - Nothing to do with Geraldo the pdf.py from modules/ is higher priority than the pdf.py from the geraldo.generators package, when being imported from __init__.py in the same folder. This behavior could unexpectedly break any module or package just because the application uses a modules/whatever.py file, whatever being the same name as any other package. Furthermore, in this case the error when importing geraldo.generators was something like Can not import module geraldo, which is quite misleading, and it took me some work to figure out what was wrong. Maybe the rule should apply when importing from the web2py application, or using local_import, but never when importing from other module files where a simple filename collision inside any package can break the module. Thanks. On Tue, Feb 5, 2013 at 11:24 PM, Massimo Di Pierro massimo.dipie...@gmail.com wrote: The rule is that if something is imported web2py should first look in app/modules/ because app-level modules should get priority else you cannot override system wide modules. So 2.3.2 is doing the right thing. This did not work well in previous versions. On Tuesday, 5 February 2013 14:40:41 UTC-6, Álvaro José Iradier wrote: Recently I updated web2py from 1.99.4 to 2.3.2. Suddenly, a PDF report stopped working. Digging into the problem, I found there is a *pdf.py* file in my application *modules/* folder. Also, I am using Geraldo Reports in * web2py/site-packages*. Geraldo reports has the following file: *geraldo/generators/pdf.py* and the __init__.py in geraldo/generators/__init__.py does: from pdf import PDFGenerator but when doing *import geraldo.generators*, that line fails, so importing geraldo.generators fails. It fails in 2.3.2, but works in 1.99.4. The fix has been to rename modules/pdf.py in my application to modules/pdfreport.py, because it looks like web2py is trying to import it in the geraldo/generators/__init.py sentence. ¿Is this the expected behavior, or should it look in the __init__,py folder first, as it did in 1.99.4? Thanks. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- (:=:) Alvaro J. Iradier Muro - airad...@gmail.com -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Web2py upload behind apache mod proxy
Hi, I have a testing web2py environment running on a system on port 8000, and then apache2 using mod_proxy to redirect requests to a virtual host on port 80 to the web2py rocket server. This is the apache config for the vhost: --- VirtualHost *:80 ... Proxy * Order deny,allow Allow from all /Proxy ProxyPass / http://127.0.0.1:8000/ ProxyPassReverse / http://127.0.0.1:8000/ /VirtualHost --- When I upload a file to a form, I get this error from the Framework: Traceback (most recent call last): File /var/www/web2py_klnetcenter/gluon/main.py, line 447, in wsgibase parse_get_post_vars(request, environ) File /var/www/web2py_klnetcenter/gluon/main.py, line 275, in parse_get_post_vars request.body = copystream_progress(request) ### stores request body File /var/www/web2py_klnetcenter/gluon/main.py, line 143, in copystream_progress copystream(source, dest, size, chunk_size) File /var/www/web2py_klnetcenter/gluon/fileutils.py, line 376, in copystream data = src.read(chunk_size) File /usr/lib/python2.6/socket.py, line 353, in read data = self._sock.recv(left) timeout: timed out It seems to be happening even before my controler is run, just when receiving the post variables. ¿Any idea what might be wrong? Thanks very much.
[web2py] Re: Web2py binary distribution and psycopg2 driver
Thanks for your comments. Do you remember how you did it? I can make it work correctly when using the source version of web2py, and I install psycopg under the python installation (c:\python27). However, when trying the binary distribution, I have tried extracting the files from the psycopg version for python 2.5, and putting them: a) Inside web2py\site-packages: They work if I start a shell and try import psycopg2, psycopg2.connect(...), etc. However they are not detected as an available driver, and trying to connect the DAL to Postgres fails with Unable to import driver b) I tried putting the files inside the library.zip file created by py2exe, but it just didn't work even when importing from the application shell... I have managed to get it working using portable python + web2py source distribution, but I think it would be easier to bundle it with web2py binary. Greets. On Jan 11, 11:55 am, Alan Etkin spame...@gmail.com wrote: I think i installed one of the listed binary files listed here without problems:http://stickpeople.com/projects/python/win-psycopg/ Regards On 10 ene, 17:14, Álvaro J. Iradier airad...@gmail.com wrote: Hi, I was trying to distribute an application with web2py binary for windows, but I required the PostgreSQL driver. So I copied the psycopg2 files to site-packages in the web2py binary folder. If I start a shell from an application, I can import psycopg2 and connect to the database. However, it looks like the DAL does not recognize the psycopg2 driver. I am getting the error: Traceback (most recent call last): File gluon/restricted.py, line 204, in restricted File C:/Users/airadier/Downloads/web2py_win/web2py/applications/ init/models/10_db.py, line 13, in module File gluon/dal.py, line 4749, in __init__ RuntimeError: Failure to connect, tried 5 times: Traceback (most recent call last): File gluon/dal.py, line 4736, in __init__ File gluon/dal.py, line 1867, in __init__ RuntimeError: Unable to import driver My guess is when DAL is being initialized, the site-packages folder is not yet in the sys.path, so the following fails: try: import psycopg2 from psycopg2.extensions import adapt as psycopg2_adapt drivers.append('PostgreSQL') except ImportError: logger.debug('no psycopg2 driver') Is there a reason for this? Is it a bug or a feature? Is there other way to acomplish these? And is it allowed to distribute a binary web2py including the PostgreSQL driver? Thanks in advance.
[web2py] Web2py binary distribution and psycopg2 driver
Hi, I was trying to distribute an application with web2py binary for windows, but I required the PostgreSQL driver. So I copied the psycopg2 files to site-packages in the web2py binary folder. If I start a shell from an application, I can import psycopg2 and connect to the database. However, it looks like the DAL does not recognize the psycopg2 driver. I am getting the error: Traceback (most recent call last): File gluon/restricted.py, line 204, in restricted File C:/Users/airadier/Downloads/web2py_win/web2py/applications/ init/models/10_db.py, line 13, in module File gluon/dal.py, line 4749, in __init__ RuntimeError: Failure to connect, tried 5 times: Traceback (most recent call last): File gluon/dal.py, line 4736, in __init__ File gluon/dal.py, line 1867, in __init__ RuntimeError: Unable to import driver My guess is when DAL is being initialized, the site-packages folder is not yet in the sys.path, so the following fails: try: import psycopg2 from psycopg2.extensions import adapt as psycopg2_adapt drivers.append('PostgreSQL') except ImportError: logger.debug('no psycopg2 driver') Is there a reason for this? Is it a bug or a feature? Is there other way to acomplish these? And is it allowed to distribute a binary web2py including the PostgreSQL driver? Thanks in advance.
[web2py] Re: Web2py binary distribution and psycopg2 driver
Does setup_exe.py work at all? When I run it, A PyFPDF Template Designer pop ups. The line causing it is: from gluon.import_all import base_modules, contributed_modules Trying setup_exe_2.6.py, there is a syntax error in line 113: if os.path.exists('dist/site-packages') shutil.rmtree('dist/site-packages') the : is missing at the end of the line. Fixing it, I get same problem as with setup_exe.py. Are these scripts mantained? Did anyone get them running in latest web2py? Thanks very much. On Jan 11, 2:20 am, Brian M bmere...@gmail.com wrote: You should be able to build your own web2py executable using setup_exe.py. I've been able to include pyodbc that way for a project that needed MS SQL access - see no reason it shouldn't also work for the PostrgreSQL driver too. ~Brian
[web2py] Re: migrating to postgres error
There is also an old thread I started some time ago about quoting identifiers: http://www.mail-archive.com/web2py@googlegroups.com/msg68661.html Greets. On 31 dic 2011, 15:15, Anthony abasta...@gmail.com wrote: On Saturday, December 31, 2011 7:33:30 AM UTC-5, Plumo wrote: ah thanks - that was it. I heard a year back the new DAL would take care of these name conflicts. You have to set check_reserved:http://web2py.com/books/default/chapter/29/6#Reserved-keywords
[web2py] Re: URL inside .load component
Thanks very much, that made it. I couldn't find documentation about the extension=False option for the URL function, it would be nice to have it in the book. Greets. On 23 ago, 18:47, Anthony abasta...@gmail.com wrote: On Tuesday, August 23, 2011 12:14:43 PM UTC-4, Álvaro J. Iradier wrote: Hi, I've created a component mycomponent.load to be used with the LOAD() method. But inside the mycomponent.load I want to have a link to another function in the same controller. When I do: {{=URL(r=request, f='otherfunction')}}, the .load extension is automatically added, so I end up being linked to the nonexisting URL http://.../mycontroller/otherfunction.load;. Is this the intended behavior? I can guess there is something about the extension from this fragment of the book: If you LOAD a component having the .load extension and the corresponding controller function redirects to another action (for example a login form), the .load extension propagates and the new url (the one to redirect too) is also loaded with a .load extension. but is there a way to remove the .load extension when using URL inside a component? Your component isn't redirecting, so this is not due to the propogation of the .load extension upon redirect. However, the URL function will automatically use the extension of the current request if the extension isn't specified (unless the extension of the current request is html). Since the request that loads the component has a .load extension, the URL function will automatically add a .load extension as well. To avoid this, just specify an explicit extension (e.g., extension='html'). Note, if you want the link to point to an html page, but you don't want the resulting URL to actually include the .html extension, then you can explicitly set extension=False. In that case, web2py will default to requesting the .html view (as it always does when there is no url extension), but it won't show .html in the URL. See near the end of this section in the book:http://web2py.com/book/default/chapter/04#URL. Anthony
[web2py] URL inside .load component
Hi, I've created a component mycomponent.load to be used with the LOAD() method. But inside the mycomponent.load I want to have a link to another function in the same controller. When I do: {{=URL(r=request, f='otherfunction')}}, the .load extension is automatically added, so I end up being linked to the nonexisting URL http://.../mycontroller/otherfunction.load;. Is this the intended behavior? I can guess there is something about the extension from this fragment of the book: If you LOAD a component having the .load extension and the corresponding controller function redirects to another action (for example a login form), the .load extension propagates and the new url (the one to redirect too) is also loaded with a .load extension. but is there a way to remove the .load extension when using URL inside a component? Thanks.
[web2py] Re: Postgres quoted identifiers
Hi Massimo, any progress on this? I need to migrato from MySql to Postgres and this issue is stopping me (I'd like to avoid making changes on the database right now). If you think it might be good to include this, I can start working on it and provide patches. I don't like the idea of keeping my own modified version of web2py forever, that is why I am asking if this might be included in a future version. Just let me know. Greets. On 29 jul, 12:03, Massimo Di Pierro massimo.dipie...@gmail.com wrote: This is a good idea. Let me give this some thought. On Jul 28, 10:35 am, Álvaro J. Iradier airad...@gmail.com wrote: Couldn't it be implemented by calling a engine-dependent method like sql_quote(name), which in other engines would by default return the unquoted string, but return the quoted name in mysql andpostgres? Greets. On 27 jul, 22:16, Massimo Di Pierro massimo.dipie...@gmail.com wrote: Yes. The dal was rewritten to allow this. The reason it was never implemented fully is that I cannot find documentation about this for all supported db engines. Implementing this only for mysql and pgsql is a pain. On Jul 27, 2:34 pm, Álvaro J. Iradier airad...@gmail.com wrote: Hi, migrating my application toPostgres, I've been hit by the user reserved keyword problem previously commented onhttp://groups.google.com/group/web2py/browse_thread/thread/f23c03ff81 According to PostgreSQL documentation,http://www.postgresql.org/docs/8.2/static/sql-syntax-lexical.html, section 4.1.1: - There is a second kind of identifier: the delimited identifier or quoted identifier. It is formed by enclosing an arbitrary sequence of characters in double-quotes (). A delimited identifier is always an identifier, never a key word. So select could be used to refer to a column or table named select, whereas an unquoted select would be taken as a key word and would therefore provoke a parse error when used where a table or column name is expected. The example can be written with quoted identifiers like this: UPDATE my_table SET a = 5; - Shouldn't web2py use this syntax by default in order to avoid keyword collisions? Probably there is a similar syntax for other DB engines as well (for example, quote using [name] in SQL Server, etc.). Is there something against this? Thanks.
[web2py] Re: Postgres quoted identifiers
Couldn't it be implemented by calling a engine-dependent method like sql_quote(name), which in other engines would by default return the unquoted string, but return the quoted name in mysql and postgres? Greets. On 27 jul, 22:16, Massimo Di Pierro massimo.dipie...@gmail.com wrote: Yes. The dal was rewritten to allow this. The reason it was never implemented fully is that I cannot find documentation about this for all supported db engines. Implementing this only for mysql and pgsql is a pain. On Jul 27, 2:34 pm, Álvaro J. Iradier airad...@gmail.com wrote: Hi, migrating my application to Postgres, I've been hit by the user reserved keyword problem previously commented onhttp://groups.google.com/group/web2py/browse_thread/thread/f23c03ff81 According to PostgreSQL documentation,http://www.postgresql.org/docs/8.2/static/sql-syntax-lexical.html, section 4.1.1: - There is a second kind of identifier: the delimited identifier or quoted identifier. It is formed by enclosing an arbitrary sequence of characters in double-quotes (). A delimited identifier is always an identifier, never a key word. So select could be used to refer to a column or table named select, whereas an unquoted select would be taken as a key word and would therefore provoke a parse error when used where a table or column name is expected. The example can be written with quoted identifiers like this: UPDATE my_table SET a = 5; - Shouldn't web2py use this syntax by default in order to avoid keyword collisions? Probably there is a similar syntax for other DB engines as well (for example, quote using [name] in SQL Server, etc.). Is there something against this? Thanks.
[web2py] Postgres quoted identifiers
Hi, migrating my application to Postgres, I've been hit by the user reserved keyword problem previously commented on http://groups.google.com/group/web2py/browse_thread/thread/f23c03ff8143b55/15602d3158ae5e6c. According to PostgreSQL documentation, http://www.postgresql.org/docs/8.2/static/sql-syntax-lexical.html, section 4.1.1: - There is a second kind of identifier: the delimited identifier or quoted identifier. It is formed by enclosing an arbitrary sequence of characters in double-quotes (). A delimited identifier is always an identifier, never a key word. So select could be used to refer to a column or table named select, whereas an unquoted select would be taken as a key word and would therefore provoke a parse error when used where a table or column name is expected. The example can be written with quoted identifiers like this: UPDATE my_table SET a = 5; - Shouldn't web2py use this syntax by default in order to avoid keyword collisions? Probably there is a similar syntax for other DB engines as well (for example, quote using [name] in SQL Server, etc.). Is there something against this? Thanks.
[web2py] Re: Inner join on subselect
At first I thought this was exactly what I was looking for... but after trying, sorry, but this query doesn't work for me... it returns 0 records, while the previous one is returning the right number. It looks like the GROUP BY selects one of the dates (the first record found) for the grouping, then the HAVING clause filters all grouping records having a selected date max(date), leaving 0 records. Trying: SELECT param, date, max(date) , value FROM param_values GROUP BY param HAVING date=max(date) (that is, add the 'date' column in the select) you can see what date is selected, and why the HAVING clause is filtering everything out. So that's why I need to use the subselect... Right now I got it working perfectly using raw db.executesql(...). In other thread I found a different but similar problem, that needed something not implemented that would be like Subtables or similar... Greets. On 18 jul, 00:46, Vasile Ermicioi elff...@gmail.com wrote: you can do that with only one query SELECT param, max(date) , value FROM param_values GROUP BY param HAVING date=max(date) _max_date = db.param_values.date.max() rows = db(db.param_values).select(db.param_values.param,_max_date, db.param_values.value, groupby=db.param_values.param, having=db.param_values.date=_max_date)
[web2py] Inner join on subselect
Hi, I have a table with a record of values read from a device. Table has a parameter ID, the record date, and the parameter value: db.define_table('param_values', Field('param', db.dev_parameter, required=True, notnull=True), Field('date', 'datetime', required=True, notnull=True), Field('value', 'integer', required=True, notnull=True), ) I'm trying to get the latest recorded value (according to the 'date' field) for each parameter, in a single query. In MySQL, I can run the following query: SELECT p.param, p.date, p.value FROM param_values p INNER JOIN (SELECT param, max(date) maxdate from param_values GROUP BY device, param) m ON m.param = p.param AND m.maxdate = p.date but I can't find a way to build a similar query in Web2py DAL, as the inner joins in DAL are really built using a WHERE. The problem is I want to join with two values in the subquery. I must use a subquery in order to use the grouping to get the max date for each different param... Any idea how could I do something similar using the DAL? Or will I need to use raw SQL? Thanks very much!
[web2py] Re: Inner join on subselect
Sorry, the previous query was wrong, the GROUP BY device was left from a previous more complex example. So, I want to get an equivalent to: SELECT p.param, p.date, p.value FROM param_values p INNER JOIN (SELECT param, max(date) maxdate from param_values GROUP BY param) m ON m.param = p.param AND m.maxdate = p.date It looks like this query is also equivalent: SELECT p.param, p.date, p.value FROM param_values p, (SELECT param, max(date) maxdate from param_values GROUP BY param) m WHERE p.param=m.param and p.date = m.maxdate but I still can't find how to do this using the DAL. Greets. On 17 jul, 22:45, Álvaro J. Iradier airad...@gmail.com wrote: Hi, I have a table with a record of values read from a device. Table has a parameter ID, the record date, and the parameter value: db.define_table('param_values', Field('param', db.dev_parameter, required=True, notnull=True), Field('date', 'datetime', required=True, notnull=True), Field('value', 'integer', required=True, notnull=True), ) I'm trying to get the latest recorded value (according to the 'date' field) for each parameter, in a single query. In MySQL, I can run the following query: SELECT p.param, p.date, p.value FROM param_values p INNER JOIN (SELECT param, max(date) maxdate from param_values GROUP BY device, param) m ON m.param = p.param AND m.maxdate = p.date but I can't find a way to build a similar query in Web2py DAL, as the inner joins in DAL are really built using a WHERE. The problem is I want to join with two values in the subquery. I must use a subquery in order to use the grouping to get the max date for each different param... Any idea how could I do something similar using the DAL? Or will I need to use raw SQL? Thanks very much!
[web2py] Re: Broken pipe with mysql
I agree there's not way to notify web2py that the connection is lost, but isn't it possible to detect that the connection broke after the error is raised, and try to reconnect? Thanks. On 15 jun, 03:25, Massimo Di Pierro massimo.dipie...@gmail.com wrote: This presents logical problems. It just should not happen. Edit the mysql config file and change the timeout. If the database is restarted, web2py needs restarting because there is no mechanism in the database connection API to notify web2py (or python) that a connection lost. On Jun 14, 4:06 pm, Álvaro J. Iradier airad...@gmail.com wrote: As seen in this thread: http://groups.google.com/group/web2py/browse_thread/thread/c6653dadbe... I'm getting broken pipe messages when mysql is restarted, and under other circunstances. Going to a shell, I tried doing a query, for example: db(db.location).select() gluon.dal.Rows object at 0xa2056ec everything goes ok. Then I restart Mysql, and running the same query I get: db(db.location).select() Traceback (most recent call last): File console, line 1, in module File /var/www/web2py_klnetcenter/gluon/dal.py, line 5382, in select return self.db._adapter.select(self.query,fields,attributes) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1159, in select rows = response(sql) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1149, in response self.execute(sql) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1234, in execute return self.log_execute(*a, **b) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1229, in log_execute ret = self.cursor.execute(*a,**b) File /var/www/web2py_klnetcenter/gluon/contrib/pymysql/cursors.py, line 108, in execute self.errorhandler(self, exc, value) File /var/www/web2py_klnetcenter/gluon/contrib/pymysql/ connections.py, line 184, in defaulterrorhandler raise errorclass, errorvalue OperationalError: (2013, 'Lost connection to MySQL server during query') trying to run it again, I only get: db(db.location).select() Traceback (most recent call last): File console, line 1, in module File /var/www/web2py_klnetcenter/gluon/dal.py, line 5382, in select return self.db._adapter.select(self.query,fields,attributes) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1159, in select rows = response(sql) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1149, in response self.execute(sql) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1234, in execute return self.log_execute(*a, **b) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1229, in log_execute ret = self.cursor.execute(*a,**b) File /var/www/web2py_klnetcenter/gluon/contrib/pymysql/cursors.py, line 108, in execute self.errorhandler(self, exc, value) File /var/www/web2py_klnetcenter/gluon/contrib/pymysql/ connections.py, line 182, in defaulterrorhandler raise Error(errorclass, errorvalue) Error: (class 'socket.error', error(32, 'Broken pipe')) forever!. However, if I do: db._adapter.connection.ping() True db(db.location).select() gluon.dal.Rows object at 0xa20596c so it looks like pinging does a reconnect. If the connection is closed manually, it won't reconnect either: db(db.location).select() gluon.dal.Rows object at 0xa205a0c db._adapter.connection.close() print db(db.location).select() Traceback (most recent call last): File console, line 1, in module File /var/www/web2py_klnetcenter/gluon/dal.py, line 5382, in select return self.db._adapter.select(self.query,fields,attributes) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1159, in select rows = response(sql) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1149, in response self.execute(sql) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1234, in execute return self.log_execute(*a, **b) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1229, in log_execute ret = self.cursor.execute(*a,**b) File /var/www/web2py_klnetcenter/gluon/contrib/pymysql/cursors.py, line 108, in execute self.errorhandler(self, exc, value) File /var/www/web2py_klnetcenter/gluon/contrib/pymysql/ connections.py, line 184, in defaulterrorhandler raise errorclass, errorvalue InterfaceError: (0, '') forever again! Is this the expected behavior? Is web2py or the adapter not detecting it is not connected and not trying to reconnect unless you ping the connection? I think this needs fixing. I have a long running shell process, and every night I find it stopped working and broken pipe errors are all around... I'm trying to workaround it pinging the connection, but I think that's not a good fix... Greets.
[web2py] Re: Broken pipe with mysql
I agree with Ron. If connection is broken between 4 and 5, I would expect the transaction to be rollback (it was not commited), and an error raised. Probably that request will fail, but at least the connection is not dropped and not recoverable. I guess in a server restart or shutdown, the server will probably (or should) wait until active connections are finished, and stop accepting new connections meanwhile. So the problem is likely to happen when there are idle connections in the pool, and the server is restarted. The connnections in the pool are invalid and can't be used anymore, but they won't recover, as web2py won't notice they are disconnected... Greets. On 15 jun, 17:28, ron_m ron.mco...@gmail.com wrote: Unless step 6 is reached and completed if the client closed then I would expect the database server to detect the closed connection and rollback on its own since there is no way to recover the state built on that connection or if the database server is restarted then the database server should perform startup recovery and rollback anything that isn't completed but the client still needs to clean up. What this smells like is a connection pool will hold open connections for reuse, the database server is restarted because of a maintenance script and now the pool has stale connections. When one of these pooled connections is given to a user task the connection should be detected as stale in the pool logic and a reconnect attempted before handing it over to the user. What Alvaro describes is similar but it looks like the DAL won't initiate a new connection after a server restart unless a ping is performed. Is it possible there is state being held in the DAL or the mysqldb driver that says there is a connection but is not cleared to allow an attempt to get a fresh open on the database?
[web2py] Broken pipe with mysql
As seen in this thread: http://groups.google.com/group/web2py/browse_thread/thread/c6653dadbea77f0d/06ed6fe085eae33c I'm getting broken pipe messages when mysql is restarted, and under other circunstances. Going to a shell, I tried doing a query, for example: db(db.location).select() gluon.dal.Rows object at 0xa2056ec everything goes ok. Then I restart Mysql, and running the same query I get: db(db.location).select() Traceback (most recent call last): File console, line 1, in module File /var/www/web2py_klnetcenter/gluon/dal.py, line 5382, in select return self.db._adapter.select(self.query,fields,attributes) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1159, in select rows = response(sql) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1149, in response self.execute(sql) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1234, in execute return self.log_execute(*a, **b) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1229, in log_execute ret = self.cursor.execute(*a,**b) File /var/www/web2py_klnetcenter/gluon/contrib/pymysql/cursors.py, line 108, in execute self.errorhandler(self, exc, value) File /var/www/web2py_klnetcenter/gluon/contrib/pymysql/ connections.py, line 184, in defaulterrorhandler raise errorclass, errorvalue OperationalError: (2013, 'Lost connection to MySQL server during query') trying to run it again, I only get: db(db.location).select() Traceback (most recent call last): File console, line 1, in module File /var/www/web2py_klnetcenter/gluon/dal.py, line 5382, in select return self.db._adapter.select(self.query,fields,attributes) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1159, in select rows = response(sql) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1149, in response self.execute(sql) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1234, in execute return self.log_execute(*a, **b) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1229, in log_execute ret = self.cursor.execute(*a,**b) File /var/www/web2py_klnetcenter/gluon/contrib/pymysql/cursors.py, line 108, in execute self.errorhandler(self, exc, value) File /var/www/web2py_klnetcenter/gluon/contrib/pymysql/ connections.py, line 182, in defaulterrorhandler raise Error(errorclass, errorvalue) Error: (class 'socket.error', error(32, 'Broken pipe')) forever!. However, if I do: db._adapter.connection.ping() True db(db.location).select() gluon.dal.Rows object at 0xa20596c so it looks like pinging does a reconnect. If the connection is closed manually, it won't reconnect either: db(db.location).select() gluon.dal.Rows object at 0xa205a0c db._adapter.connection.close() print db(db.location).select() Traceback (most recent call last): File console, line 1, in module File /var/www/web2py_klnetcenter/gluon/dal.py, line 5382, in select return self.db._adapter.select(self.query,fields,attributes) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1159, in select rows = response(sql) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1149, in response self.execute(sql) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1234, in execute return self.log_execute(*a, **b) File /var/www/web2py_klnetcenter/gluon/dal.py, line 1229, in log_execute ret = self.cursor.execute(*a,**b) File /var/www/web2py_klnetcenter/gluon/contrib/pymysql/cursors.py, line 108, in execute self.errorhandler(self, exc, value) File /var/www/web2py_klnetcenter/gluon/contrib/pymysql/ connections.py, line 184, in defaulterrorhandler raise errorclass, errorvalue InterfaceError: (0, '') forever again! Is this the expected behavior? Is web2py or the adapter not detecting it is not connected and not trying to reconnect unless you ping the connection? I think this needs fixing. I have a long running shell process, and every night I find it stopped working and broken pipe errors are all around... I'm trying to workaround it pinging the connection, but I think that's not a good fix... Greets.
[web2py] Re: Project: pydev extension for web2py in Eclipse
No, I wasn't aware... sorry, I can't follow the current web2py development at that detail. Could you point me to the information or make a small brief? It sounds like it could be quite useful. Greets. On 10 jun, 04:52, Bruno Rocha rochacbr...@gmail.com wrote: Are you aware about the changes included in the latest web2py version? I am talking specially about the new importer, the 'current' object and the models subfolders. Is that being considerated? On Thu, Jun 9, 2011 at 6:11 PM, Álvaro J. Iradier airad...@gmail.comwrote: Yes, I checked that project, but these way would require not adding anything at the headers, and it also parses the models file. It requires no modification in web2py, just in Pydev. Talking to the pydev author in the pydev mailing list, he might consider including support for web2py if we manage to get it working fine, and we are quite there. Greets. On 9 jun, 20:43, Pierre Thibault pierre.thibau...@gmail.com wrote: You can use this project:http://code.google.com/p/neo-insert-imports/to add automatically the necessary imports for the static analyzer. -- A+ - Pierre My blog and profile (http://pierrethibault.posterous.com) http://pierrethibault.posterous.com YouTube page (http://www.youtube.com/user/tubetib) http://www.youtube.com/user/tubetib Twitter (http://twitter.com/pierreth2) http://twitter.com/pierreth2
[web2py] Re: Project: pydev extension for web2py in Eclipse
After some talking to the Pydev author, we've managed to make a patch to a file in Pydev that calls the build_environment() and run_models_in() functions from web2py, and adds the resulting environment into the __builtins__. The result is you automagically get a fully working web2py environment in eclipse, with autocompletion, etc. You can follow the development on this feature request on Pydev issue tracker: https://sourceforge.net/tracker/?func=detailatid=577332aid=3294887group_id=85796 The procedure is quite manual yet, I hope it can be improved so it is customizable per project. Comments and help are welcome! Greets. On 29 abr, 10:51, Álvaro J. Iradier alvaro.irad...@polartech.es wrote: Hi, I am trying to develop an extension (a Jython script[1]) for Pydev[2] to improve web2py integration into Eclipse (I'm not sattisfied with the existing recipes). I've just started, and you can follow what's going on on the following thread in the Pydev-code mailing list: https://sourceforge.net/mailarchive/forum.php?thread_name=BANLkTik-Sj... If someone is interested on helping, please let me know, I don't have much time available to work on it. Greets. [1]http://pydev.org/manual_articles_scripting.html [2]http://pydev.org/ -- Álvaro J. Iradier Muro Departamento de Desarrollo alvaro.irad...@polartech.es Polar Technologies T +34 976 527 952 F +34 976 466 125www.polartech.es Antes de imprimir este mensaje, por favor, compruebe que es verdaderamente necesario. El medioambiente es cosa de todos. AVISO LEGAL Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener información confidencial, siendo para uso exclusivo del destinatario, quedando prohibida su divulgación, copia o distribución a terceros sin la autorización expresa del remitente. Si Vd. ha recibido este mensaje erróneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboración.
[web2py] Re: Project: pydev extension for web2py in Eclipse
Yes, I checked that project, but these way would require not adding anything at the headers, and it also parses the models file. It requires no modification in web2py, just in Pydev. Talking to the pydev author in the pydev mailing list, he might consider including support for web2py if we manage to get it working fine, and we are quite there. Greets. On 9 jun, 20:43, Pierre Thibault pierre.thibau...@gmail.com wrote: You can use this project:http://code.google.com/p/neo-insert-imports/to add automatically the necessary imports for the static analyzer. -- A+ - Pierre My blog and profile (http://pierrethibault.posterous.com)http://pierrethibault.posterous.com YouTube page (http://www.youtube.com/user/tubetib)http://www.youtube.com/user/tubetib Twitter (http://twitter.com/pierreth2) http://twitter.com/pierreth2
[web2py] Project: pydev extension for web2py in Eclipse
Hi, I am trying to develop an extension (a Jython script[1]) for Pydev[2] to improve web2py integration into Eclipse (I'm not sattisfied with the existing recipes). I've just started, and you can follow what's going on on the following thread in the Pydev-code mailing list: https://sourceforge.net/mailarchive/forum.php?thread_name=BANLkTik-SjO6XQ3uUBwfqfJU9LZsMOO_pw%40mail.gmail.comforum_name=pydev-code If someone is interested on helping, please let me know, I don't have much time available to work on it. Greets. [1] http://pydev.org/manual_articles_scripting.html [2] http://pydev.org/ -- Álvaro J. Iradier Muro Departamento de Desarrollo alvaro.irad...@polartech.es Polar Technologies T +34 976 527 952 F +34 976 466 125 www.polartech.es Antes de imprimir este mensaje, por favor, compruebe que es verdaderamente necesario. El medioambiente es cosa de todos. AVISO LEGAL Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener información confidencial, siendo para uso exclusivo del destinatario, quedando prohibida su divulgación, copia o distribución a terceros sin la autorización expresa del remitente. Si Vd. ha recibido este mensaje erróneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboración.
[web2py] Obtain label for field with IS_IN_SET
I found thread where someone asked exactly the same question I'm trying to solve: http://groups.google.com/group/web2py/browse_thread/thread/ba58ec07a23b0393/ but the question wasn't answered, quoting from the last post: --- ... In a controller I defined a form: form=SQLFORM.factory(Field('bedrijf',requires=IS_IN_SET([2,3,4,5,6,7],labels=['Fitnesscentrum','Fysiofitnesscentrum','Personal Trainer','Dietist(e)','Voedingsconsulent(e)','Yogaschool'],zero=None))) ... Massimo, I am not sure you did understand my question correctly. When in the form the user selects 'Fitnesscentre' in the drop box, the function that selects all the Fitnesscentres takes 2 as its argument. I would like the flash in the view to display something like: Fitnesscentres in Amsterdam request.vars.city gives me Amsterdam, what I am looking for is something similar for Fitnesscentres, however, request.vars.bedrijf gives me 2. --- Is it possible? Thanks very much,
[web2py] Re: Obtain label for field with IS_IN_SET
Thanks Massimo, that worked perfectly. I wasn't aware of the [('id1', 'label1'), ('id2', 'label2')] syntax for IS_IN_SET. Thanks for the quick response! On 20 ene, 22:30, Massimo Di Pierro massimo.dipie...@gmail.com wrote: I am still not sure but I will give it a try: options=[(2,'Fitnesscentrum'),(3,'Fysiofitnesscentrum'),(4,'Personal Trainer'),(5,'Dietist(e)'),(6,'Voedingsconsulent(e)'), (7,'Yogaschool')] def index(): form=SQLFORM.factory(Field('bedrijf',requires=IS_IN_SET(options)), Field('city')) if form.accepts(request): response.flash='%s in %s' (dict(options) [int(form.vars.bedrijf)],form.vars.city) return dict(form=form) form=SQLFORM.factory(Field('bedrijf',requires=IS_IN_SET([2,3,4,5,6,7],label s=['Fitnesscentrum','Fysiofitnesscentrum','Personal Trainer','Dietist(e)','Voedingsconsulent(e)','Yogaschool'],zero=None))) On Jan 20, 3:15 pm, Álvaro J. Iradier airad...@gmail.com wrote: I found thread where someone asked exactly the same question I'm trying to solve: http://groups.google.com/group/web2py/browse_thread/thread/ba58ec07a2... but the question wasn't answered, quoting from the last post: --- ... In a controller I defined a form: form=SQLFORM.factory(Field('bedrijf',requires=IS_IN_SET([2,3,4,5,6,7],label s=['Fitnesscentrum','Fysiofitnesscentrum','Personal Trainer','Dietist(e)','Voedingsconsulent(e)','Yogaschool'],zero=None))) ... Massimo, I am not sure you did understand my question correctly. When in the form the user selects 'Fitnesscentre' in the drop box, the function that selects all the Fitnesscentres takes 2 as its argument. I would like the flash in the view to display something like: Fitnesscentres in Amsterdam request.vars.city gives me Amsterdam, what I am looking for is something similar for Fitnesscentres, however, request.vars.bedrijf gives me 2. --- Is it possible? Thanks very much,
[web2py] Re: The stability of web2py releases
Given the pace of quick adoption for new versions of web2py, I think it would be good enough to mark a new release (except if it's just a bug fix release) like latest or edge release, and just keep the previous one as recommended. After some days (for example one week), without new releases, the edge would become recommended. If bugs are found in the edge, like it happened in 1.90 after the DAL rewrite, then just release 1.90.2 as edge, 1.90.3, 1.90.4, ... without promiting them as recommended. After one week of no new releases, the current 1.90.x would become recommended (there would be no edge download until a new release is done). This wouldn't add extra work to the them. Just changing the label of the release in the download page, and sometimes keep 2 download links instead of one. Greets. On 22 dic, 20:31, Branko Vukelić stu...@brankovukelic.com wrote: On Wed, Dec 22, 2010 at 8:29 PM, mdipierro mdipie...@cs.depaul.edu wrote: The idea was to have stable and nightly-build. The problem is that very few people check the nightly build. Well, yeah, it's because it's sounds like a nightly TRUNK dump. :) It's better to make a 'incubation release' or something like that, so it's obvious that it's a release. And when it's hatched, you can label it safe-for-production. I don't know if people would use them, though. They might still go yuck and decide it's just like nightly, with a fancy name. :D -- Branko Vukelic stu...@brankovukelic.comhttp://www.brankovukelic.com/
[web2py] Patch for cron not working on compiled apps
Hi, cron jobs won't work on a bytecode compiled application after unpacked. When web2py starts, the following error(s) is(are) displayed: invalid application name: testapp/cron/crontask the crontab file looks like: #crontab * * * * * root *cron/crontask it looks like gluon/shell.py is not working for bytecode compiled applications, as it's searching for the .py file: cfile = os.path.join('applications', a, 'controllers', c + '.py') Suggested patch attached. Greets. -- Álvaro J. Iradier Muro Departamento de Desarrollo alvaro.irad...@polartech.es Polar Technologies T +34 976 527 952 F +34 976 466 125 www.polartech.es Antes de imprimir este mensaje, por favor, compruebe que es verdaderamente necesario. El medioambiente es cosa de todos. AVISO LEGAL Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener información confidencial, siendo para uso exclusivo del destinatario, quedando prohibida su divulgación, copia o distribución a terceros sin la autorización expresa del remitente. Si Vd. ha recibido este mensaje erróneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboración. shell.py.patch Description: Binary data
[web2py] Re: cron @reboot not working on 1.87.3
Suggested patch: *** newcron.old.py Fri Oct 15 11:27:34 2010 --- newcron.py Fri Oct 15 11:29:08 2010 *** *** 175,181 task[id] = [] vals = s.split(',') for val in vals: ! if '-' in val and '/' not in val: val = '%s/1' % val if '/' in val: task[id] += rangetolist(val, id) --- 175,181 task[id] = [] vals = s.split(',') for val in vals: ! if val != '-1' and '-' in val and '/' not in val: val = '%s/1' % val if '/' in val: task[id] += rangetolist(val, id) Please confirm if you're going to apply it. Greets. On 14 oct, 22:43, Álvaro J. Iradier airad...@gmail.com wrote: Ok I found the problem, in rev 770b1f1298: http://code.google.com/p/web2py/source/diff?spec=svn770b1f129819eb108... in newcron.py: if val.find('/') -1: was changed to: if '-' in val and '/' not in val: val = '%s/1' % val if '/' in val: and it looks like there's something wrong with val = '-1/1', coming from the previous: 153 if line.startswith('@reboot'): 154 line=line.replace('@reboot', '-1 * * * *') However, I can't guess what the fix is... Greets. On 14 oct, 22:34, Álvaro J. Iradier airad...@gmail.com wrote: Thanks very much. Ok, so after trying: Works in 1.83.2. Doesn't work in 1.84.1 or newer. I don't see anything on the changelog for 1.84.1-4 related to cron, but clearly something was changed that stopped it from working. Greets. On 14 oct, 22:00, Bruno Rocha rochacbr...@gmail.com wrote: http://www.web2py.com/examples/static/version/web2py_src.zip http://www.web2py.com/examples/static/1.81.4/web2py_src.zip 2010/10/14 Álvaro J. Iradier airad...@gmail.com After upgrading web2py from 1.81.4 to latest 1.87.3, cron jobs at @reboot stopped working. This is my crontab file: #crontab * * * * * root *cron/pollsensors * * * * * root *cron/checkschedule 0 0 * * * root *cron/cleandata @reboot root *cron/cleandata @reboot root *cron/pollingloop The two last jobs simply won't start. Reverting to 1.81.4 works. Any idea? How can I download previous versions to track where it stopped working? Thanks very much. -- http://rochacbruno.com.br
[web2py] cron @reboot not working on 1.87.3
After upgrading web2py from 1.81.4 to latest 1.87.3, cron jobs at @reboot stopped working. This is my crontab file: #crontab * * * * * root *cron/pollsensors * * * * * root *cron/checkschedule 0 0 * * * root *cron/cleandata @reboot root *cron/cleandata @reboot root *cron/pollingloop The two last jobs simply won't start. Reverting to 1.81.4 works. Any idea? How can I download previous versions to track where it stopped working? Thanks very much.
[web2py] Re: cron @reboot not working on 1.87.3
Thanks very much. Ok, so after trying: Works in 1.83.2. Doesn't work in 1.84.1 or newer. I don't see anything on the changelog for 1.84.1-4 related to cron, but clearly something was changed that stopped it from working. Greets. On 14 oct, 22:00, Bruno Rocha rochacbr...@gmail.com wrote: http://www.web2py.com/examples/static/version/web2py_src.zip http://www.web2py.com/examples/static/1.81.4/web2py_src.zip 2010/10/14 Álvaro J. Iradier airad...@gmail.com After upgrading web2py from 1.81.4 to latest 1.87.3, cron jobs at @reboot stopped working. This is my crontab file: #crontab * * * * * root *cron/pollsensors * * * * * root *cron/checkschedule 0 0 * * * root *cron/cleandata @reboot root *cron/cleandata @reboot root *cron/pollingloop The two last jobs simply won't start. Reverting to 1.81.4 works. Any idea? How can I download previous versions to track where it stopped working? Thanks very much. -- http://rochacbruno.com.br
[web2py] Re: cron @reboot not working on 1.87.3
Ok I found the problem, in rev 770b1f1298: http://code.google.com/p/web2py/source/diff?spec=svn770b1f129819eb108fe7de0c6c5ffebc2ef7317dr=770b1f129819eb108fe7de0c6c5ffebc2ef7317dformat=sidepath=/gluon/newcron.py in newcron.py: if val.find('/') -1: was changed to: if '-' in val and '/' not in val: val = '%s/1' % val if '/' in val: and it looks like there's something wrong with val = '-1/1', coming from the previous: 153 if line.startswith('@reboot'): 154 line=line.replace('@reboot', '-1 * * * *') However, I can't guess what the fix is... Greets. On 14 oct, 22:34, Álvaro J. Iradier airad...@gmail.com wrote: Thanks very much. Ok, so after trying: Works in 1.83.2. Doesn't work in 1.84.1 or newer. I don't see anything on the changelog for 1.84.1-4 related to cron, but clearly something was changed that stopped it from working. Greets. On 14 oct, 22:00, Bruno Rocha rochacbr...@gmail.com wrote: http://www.web2py.com/examples/static/version/web2py_src.zip http://www.web2py.com/examples/static/1.81.4/web2py_src.zip 2010/10/14 Álvaro J. Iradier airad...@gmail.com After upgrading web2py from 1.81.4 to latest 1.87.3, cron jobs at @reboot stopped working. This is my crontab file: #crontab * * * * * root *cron/pollsensors * * * * * root *cron/checkschedule 0 0 * * * root *cron/cleandata @reboot root *cron/cleandata @reboot root *cron/pollingloop The two last jobs simply won't start. Reverting to 1.81.4 works. Any idea? How can I download previous versions to track where it stopped working? Thanks very much. -- http://rochacbruno.com.br
[web2py] Re: Sharing data between background worker and requests
Thanks Massimo, I forgot to mention I'm using the integrated rocket web server (not mod_python, cgi or whatever). So I was just curious if there was a way of sharing data between the hard cron thread and the thread where the requests are run, without needing to serialize informate (picke- unpickle or anything). I also thought of using a socket, instead of named pipes. Basically, I want to avoid database writing because the application is run from a SD card, to avoid wearing. However, if there exists not a way for sharing class instances or plain variables between cron and request threads, I'm going to take a look at using a database from RAM or from a tmpfs. I was thinking of using :memory: in-memory sqlite database, but I would hit the same problem: I can't share the db_in_memory from the cron thread to the request thread. Thanks very much. On 21 sep, 22:28, mdipierro mdipie...@cs.depaul.edu wrote: If not and you have a single web2py installation you can use a named pipe http://stackoverflow.com/questions/1430446/create-a-temporary-fifo-na... has pros and cons. data can be lost or corrupted if the reading/ writing app crashes. I would use the database because of transcaction safety. Massimo On Sep 21, 3:21 pm, Álvaro J. Iradier airad...@gmail.com wrote: Hi, I've setup a background worker thread using cron @reboot. The background worker is polling some serial sensors all the time (there's a loop in class reading the sensor values). On some moment, the values are recorded and an alarm bit is set. Now, what I need is a way to share this information with a web request without writing it to the database. I've tried so far using class variables, and cache.ram, but none of them work. I think the request is being processed in a clean environment, am I right? So, what would be a good way of passingsome information to the request thread from the background worker class? Thanks very much. Álvaro Iradier.
[web2py] Re: Sharing data between background worker and requests
Ok, so after cheking newcron.py I can see the main issue here is that cron tasks are run in a totally independent process, via subprocess module, so it's impossible to use threading shared data. Has rocket a maximum request timeout? If not, I could use cron @reboot to launch a request to a special URL where a infinite duration task is started, so I could use that thread to do the sensor polling and share data with the other threads using cache.ram, or in-memory sqlite databases. I'll give a try to that approach, any suggestions are appreciated. Thanks very much. On 22 sep, 08:04, Álvaro J. Iradier airad...@gmail.com wrote: Thanks Massimo, I forgot to mention I'm using the integrated rocket web server (not mod_python, cgi or whatever). So I was just curious if there was a way of sharing data between the hard cron thread and the thread where the requests are run, without needing to serialize informate (picke- unpickle or anything). I also thought of using a socket, instead of named pipes. Basically, I want to avoid database writing because the application is run from a SD card, to avoid wearing. However, if there exists not a way for sharing class instances or plain variables between cron and request threads, I'm going to take a look at using a database from RAM or from a tmpfs. I was thinking of using :memory: in-memory sqlite database, but I would hit the same problem: I can't share the db_in_memory from the cron thread to the request thread. Thanks very much. On 21 sep, 22:28, mdipierro mdipie...@cs.depaul.edu wrote: If not and you have a single web2py installation you can use a named pipe http://stackoverflow.com/questions/1430446/create-a-temporary-fifo-na... has pros and cons. data can be lost or corrupted if the reading/ writing app crashes. I would use the database because of transcaction safety. Massimo On Sep 21, 3:21 pm, Álvaro J. Iradier airad...@gmail.com wrote: Hi, I've setup a background worker thread using cron @reboot. The background worker is polling some serial sensors all the time (there's a loop in class reading the sensor values). On some moment, the values are recorded and an alarm bit is set. Now, what I need is a way to share this information with a web request without writing it to the database. I've tried so far using class variables, and cache.ram, but none of them work. I think the request is being processed in a clean environment, am I right? So, what would be a good way of passingsome information to the request thread from the background worker class? Thanks very much. Álvaro Iradier.
[web2py] Sharing data between background worker and requests
Hi, I've setup a background worker thread using cron @reboot. The background worker is polling some serial sensors all the time (there's a loop in class reading the sensor values). On some moment, the values are recorded and an alarm bit is set. Now, what I need is a way to share this information with a web request without writing it to the database. I've tried so far using class variables, and cache.ram, but none of them work. I think the request is being processed in a clean environment, am I right? So, what would be a good way of passingsome information to the request thread from the background worker class? Thanks very much. Álvaro Iradier.
[web2py] Problem with crud.create and duplicate _formname
Hi, I think I found a problem (bug?) with crud.create and form.custom.end: I have the following model: db.define_table('contents', Field('name', 'string', required=True, notnull=True, requires=IS_NOT_EMPTY()), Field('ordering', 'integer', required=True, requires=IS_NOT_EMPTY()), Field('duration', 'integer', required=True, requires=IS_NOT_EMPTY()), Field('enabled', 'boolean', required=True), Field('file', 'upload', required=True, requires=IS_NOT_EMPTY(), autodelete=True) ) The following controller: def index(): form = crud.create(db.contents) if form.accepts(request.vars): response.flash='Inserted' return dict(form=form) and the serialization of form.custom.end is: div class=hiddeninput name=_next type=hidden /input name=_formkey type=hidden value=b1b19180-5226-48da-8d26-da29ed904177 /input name=_formname type=hidden value=contents_create //divdiv class=hiddeninput name=_next type=hidden /input name=_formkey type=hidden value=b1b19180-5226-48da-8d26-da29ed904177 /input name=_formname type=hidden value=contents/None //div/form Notice the output is duplicated, specially _formname is repeated with a value of contents/None, which makes the form not working. Am I doing something wrong, or is this a bug? Thanks very much. -- (:=:) Alvaro J. Iradier Muro - airad...@gmail.com
[web2py] Re: Problem with crud.create and duplicate _formname
Thanks Massimo. It's impossible to have better support than web2py, you and this community are impressing, five stars! So, assumming I'm doing other things between the crud.create and the return: def index(): form = crud.create(db.contents,message='Inserted') ... #other code return dict(form=form, other_variables...) Question is: what's the best way for checking if the crud accepted the input, and a new record was created? I just got it working by checking if form.vars.id != None, but is there a better or official way of doing it? Thanks. On 9 sep, 22:31, mdipierro mdipie...@cs.depaul.edu wrote: because you cannot .accepts a crud forms. It is implicit. You can do def index(): form = crud.create(db.contents,message='Inserted') return dict(form=form) On Sep 9, 2:51 pm, Álvaro J. Iradier airad...@gmail.com wrote: Hi, I think I found a problem (bug?) with crud.create and form.custom.end: I have the following model: db.define_table('contents', Field('name', 'string', required=True, notnull=True, requires=IS_NOT_EMPTY()), Field('ordering', 'integer', required=True, requires=IS_NOT_EMPTY()), Field('duration', 'integer', required=True, requires=IS_NOT_EMPTY()), Field('enabled', 'boolean', required=True), Field('file', 'upload', required=True, requires=IS_NOT_EMPTY(), autodelete=True) ) The following controller: def index(): form = crud.create(db.contents) if form.accepts(request.vars): response.flash='Inserted' return dict(form=form) and the serialization of form.custom.end is: div class=hiddeninput name=_next type=hidden /input name=_formkey type=hidden value=b1b19180-5226-48da-8d26-da29ed904177 /input name=_formname type=hidden value=contents_create //divdiv class=hiddeninput name=_next type=hidden /input name=_formkey type=hidden value=b1b19180-5226-48da-8d26-da29ed904177 /input name=_formname type=hidden value=contents/None //div/form Notice the output is duplicated, specially _formname is repeated with a value of contents/None, which makes the form not working. Am I doing something wrong, or is this a bug? Thanks very much. -- (:=:) Alvaro J. Iradier Muro - airad...@gmail.com
[web2py] Re: Table migration problems with Sqlite
I think if the migration can't be done, it would be better an error or exception than a warning. If the changes in the DB can't be migrated, it's better for the developer to know the problem and find an alternate way. I'll try and send you a patch, and then we can decide if it's better to give a warning or an exception. If it's not possible to drop columns in Sqlite, what about creating a new temp table, inserting data from old table into the new one, removing the old table and renaming the temp table into the old table's name? That would allow this kind of migrations in Sqlite. I could try this too, and suggest a patch, if you agree this is s a correct approach. Greets. On 29 ago, 07:12, mdipierro mdipie...@cs.depaul.edu wrote: I agree .table should not we rewritten all the time. If it does so it is a bug, I should double check, and I will take a patch to fix it. I am not sure about dropping columns and SQLite. The problem only appears when a user tries to re-define a column that was dropped. Perhaps a warning instead of an exception? I would take a patch to issue a logging.warn. On Aug 28, 12:32 pm, Álvaro J. Iradier airad...@gmail.com wrote: Thanks Massimo, that answers my first question. But what about the .tables being written on every request, with the old fields? if query: ... if key in sql_fields: sql_fields_old[key] = sql_fields[key] else: del sql_fields_old[key] tfile = open(self._dbt, 'w') portalocker.lock(tfile, portalocker.LOCK_EX) cPickle.dump(sql_fields_old, tfile) portalocker.unlock(tfile) tfile.close() For sqlite, query == None, so sql_fields_old is unchanged, and the original sql_fields_old is cpickled to the .table file. So, the change is detected on every request, and the .table file is rewritten all the time. I think the correct fix should be, if sqlite doesn't support dropping a column, then an exception should be raised, so the developer knows the migration has to be done manually. What do you think? Thanks very much. On 27 ago, 23:31, mdipierro mdipie...@cs.depaul.edu wrote: so, for SQL databases, if a column is removed or dropped, it's not migrated for sqlite. Is there a reason for this? sqlite does not support drop columns. On Aug 27, 2:03 pm, Álvaro J. Iradier alvaro.irad...@polartech.es wrote: Hi everyone, this is a long one today I noticed one of my _settings.table file was written everytime. The table definition was changed a long ago, from the old chema: CREATE TABLE settings( id INTEGER PRIMARY KEY AUTOINCREMENT, key CHAR(512), value CHAR(512) ); to the new: CREATE TABLE settings( id INTEGER PRIMARY KEY AUTOINCREMENT, key CHAR(512) NOT NULL UNIQUE, value CHAR(512) ); the change is, key column was made NOT NULL UNIQUE. So first question I noticed. In the migrate procedure, I noticed the following code: for key in keys: if not key in sql_fields_old: query = ['ALTER TABLE %s ADD %s %s;' % \ (self._tablename, key, sql_fields_aux[key].replace(', ', new_add))] elif self._db._dbname == 'sqlite': query = None ... so, for SQL databases, if a column is removed or dropped, it's not migrated for sqlite. Is there a reason for this? Then I tried commenting the if self._db._dbname == 'sqlite': line, and I got the following SQL error: OperationalError: Cannot add a UNIQUE column which makes sense, it's not trivial to add a UNIQUE column to an existing database... But what makes me worry is, at the end of the _migrate method, I find this: if query: ... if key in sql_fields: sql_fields_old[key] = sql_fields[key] else: del sql_fields_old[key] tfile = open(self._dbt, 'w') portalocker.lock(tfile, portalocker.LOCK_EX) print Here2:, self._dbt, sql_fields, sql_fields_old cPickle.dump(sql_fields_old, tfile) portalocker.unlock(tfile) tfile.close() First, sql_fields_old is updated with the migrated value ONLY if query is not None. For sqlite it's None for changed columns. So, later, cPickle dumps the value of sql_fields_old, so the file _settings.table is written with the same old values, as this field was not migrated. So, for every request, web2py detects a migration is needed, but it does nothing but writing the .table file with the old values. I think the correct behaviour should be throwing an error if migration can't be done. Can you suggest a fix
[web2py] Re: Table migration problems with Sqlite
Thanks Massimo, that answers my first question. But what about the .tables being written on every request, with the old fields? if query: ... if key in sql_fields: sql_fields_old[key] = sql_fields[key] else: del sql_fields_old[key] tfile = open(self._dbt, 'w') portalocker.lock(tfile, portalocker.LOCK_EX) cPickle.dump(sql_fields_old, tfile) portalocker.unlock(tfile) tfile.close() For sqlite, query == None, so sql_fields_old is unchanged, and the original sql_fields_old is cpickled to the .table file. So, the change is detected on every request, and the .table file is rewritten all the time. I think the correct fix should be, if sqlite doesn't support dropping a column, then an exception should be raised, so the developer knows the migration has to be done manually. What do you think? Thanks very much. On 27 ago, 23:31, mdipierro mdipie...@cs.depaul.edu wrote: so, for SQL databases, if a column is removed or dropped, it's not migrated for sqlite. Is there a reason for this? sqlite does not support drop columns. On Aug 27, 2:03 pm, Álvaro J. Iradier alvaro.irad...@polartech.es wrote: Hi everyone, this is a long one today I noticed one of my _settings.table file was written everytime. The table definition was changed a long ago, from the old chema: CREATE TABLE settings( id INTEGER PRIMARY KEY AUTOINCREMENT, key CHAR(512), value CHAR(512) ); to the new: CREATE TABLE settings( id INTEGER PRIMARY KEY AUTOINCREMENT, key CHAR(512) NOT NULL UNIQUE, value CHAR(512) ); the change is, key column was made NOT NULL UNIQUE. So first question I noticed. In the migrate procedure, I noticed the following code: for key in keys: if not key in sql_fields_old: query = ['ALTER TABLE %s ADD %s %s;' % \ (self._tablename, key, sql_fields_aux[key].replace(', ', new_add))] elif self._db._dbname == 'sqlite': query = None ... so, for SQL databases, if a column is removed or dropped, it's not migrated for sqlite. Is there a reason for this? Then I tried commenting the if self._db._dbname == 'sqlite': line, and I got the following SQL error: OperationalError: Cannot add a UNIQUE column which makes sense, it's not trivial to add a UNIQUE column to an existing database... But what makes me worry is, at the end of the _migrate method, I find this: if query: ... if key in sql_fields: sql_fields_old[key] = sql_fields[key] else: del sql_fields_old[key] tfile = open(self._dbt, 'w') portalocker.lock(tfile, portalocker.LOCK_EX) print Here2:, self._dbt, sql_fields, sql_fields_old cPickle.dump(sql_fields_old, tfile) portalocker.unlock(tfile) tfile.close() First, sql_fields_old is updated with the migrated value ONLY if query is not None. For sqlite it's None for changed columns. So, later, cPickle dumps the value of sql_fields_old, so the file _settings.table is written with the same old values, as this field was not migrated. So, for every request, web2py detects a migration is needed, but it does nothing but writing the .table file with the old values. I think the correct behaviour should be throwing an error if migration can't be done. Can you suggest a fix for this? Thanks very much. -- Álvaro J. Iradier Muro Departamento de Desarrollo alvaro.irad...@polartech.es
[web2py] Table migration problems with Sqlite
Hi everyone, this is a long one today I noticed one of my _settings.table file was written everytime. The table definition was changed a long ago, from the old chema: CREATE TABLE settings( id INTEGER PRIMARY KEY AUTOINCREMENT, key CHAR(512), value CHAR(512) ); to the new: CREATE TABLE settings( id INTEGER PRIMARY KEY AUTOINCREMENT, key CHAR(512) NOT NULL UNIQUE, value CHAR(512) ); the change is, key column was made NOT NULL UNIQUE. So first question I noticed. In the migrate procedure, I noticed the following code: for key in keys: if not key in sql_fields_old: query = ['ALTER TABLE %s ADD %s %s;' % \ (self._tablename, key, sql_fields_aux[key].replace(', ', new_add))] elif self._db._dbname == 'sqlite': query = None ... so, for SQL databases, if a column is removed or dropped, it's not migrated for sqlite. Is there a reason for this? Then I tried commenting the if self._db._dbname == 'sqlite': line, and I got the following SQL error: OperationalError: Cannot add a UNIQUE column which makes sense, it's not trivial to add a UNIQUE column to an existing database... But what makes me worry is, at the end of the _migrate method, I find this: if query: ... if key in sql_fields: sql_fields_old[key] = sql_fields[key] else: del sql_fields_old[key] tfile = open(self._dbt, 'w') portalocker.lock(tfile, portalocker.LOCK_EX) print Here2:, self._dbt, sql_fields, sql_fields_old cPickle.dump(sql_fields_old, tfile) portalocker.unlock(tfile) tfile.close() First, sql_fields_old is updated with the migrated value ONLY if query is not None. For sqlite it's None for changed columns. So, later, cPickle dumps the value of sql_fields_old, so the file _settings.table is written with the same old values, as this field was not migrated. So, for every request, web2py detects a migration is needed, but it does nothing but writing the .table file with the old values. I think the correct behaviour should be throwing an error if migration can't be done. Can you suggest a fix for this? Thanks very much. -- Álvaro J. Iradier Muro Departamento de Desarrollo alvaro.irad...@polartech.es
[web2py] Cron task run twice when init is symbolic link
I went nuts until I found why one cron task in my application was run twice every minute, instead of once. The reason was plain simple: I had a symbolic link to make it the 'init' application. Is this the expected behavior? Should web2py detect symbolic links to a duplicate application (that is applications/init - applications/myapplication) and skip running one of the crontabs? Thanks. -- Álvaro J. Iradier Muro Departamento de Desarrollo alvaro.irad...@polartech.es Polar Technologies T +34 976 527 952 F +34 976 466 125 www.polartech.es Antes de imprimir este mensaje, por favor, compruebe que es verdaderamente necesario. El medioambiente es cosa de todos. AVISO LEGAL Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener información confidencial, siendo para uso exclusivo del destinatario, quedando prohibida su divulgación, copia o distribución a terceros sin la autorización expresa del remitente. Si Vd. ha recibido este mensaje erróneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboración.
[web2py] Re: Web2py on port 80 with non-root, setuid, possible?
Well the problem is non-root users can't use ports 1024, and I want to use port 80, so when running: ./web2py.py -p 80 --nogui web2py Enterprise Web Framework Created by Massimo Di Pierro, Copyright 2007-2010 Version 1.79.2 (2010-06-08 22:45:26) Database drivers available: SQLite3 Starting hardcron... choose a password:** please visit: http://127.0.0.1:80 use kill -SIGTERM 2958 to shutdown the web2py server ERROR:Rocket.Errors.ThreadPool:Socket 127.0.0.1:80 in use by other process and it won't share. CRITICAL:Rocket.Errors.ThreadPool:No interfaces to listen on...closing. - the error message is bogus because the binding is just failing because of the non-root user. Some web servers allow you to set a user in config. You start them as root, and after the port binding the server setuids to another user. I can't seem to find this option in web2py / rocket server, but it would be useful. Or is there other way I'm missing to achieve this? Thanks very much. On 9 jul, 02:03, mdipierro mdipie...@cs.depaul.edu wrote: You NEVER run it as root user. You can run web2py as any user. You should run it as the user with the lowest permissions. What problem do you experience? On 8 Lug, 03:38, Álvaro J. Iradier alvaro.irad...@polartech.es wrote: Hi everybody, I need to run web2py integrated web server, and I'd like to run it on port 80 as a non-root user, for security reasons. Is it possible? Can web2py be started as root, and then setuid as a different user? If it's possible, I didn't find how to do it, can you point me on the right direction? Thanks very much. -- Álvaro J. Iradier Muro Departamento de Desarrollo alvaro.irad...@polartech.es Polar Technologies T +34 976 527 952 F +34 976 466 125www.polartech.es
[web2py] Web2py on port 80 with non-root, setuid, possible?
Hi everybody, I need to run web2py integrated web server, and I'd like to run it on port 80 as a non-root user, for security reasons. Is it possible? Can web2py be started as root, and then setuid as a different user? If it's possible, I didn't find how to do it, can you point me on the right direction? Thanks very much. -- Álvaro J. Iradier Muro Departamento de Desarrollo alvaro.irad...@polartech.es Polar Technologies T +34 976 527 952 F +34 976 466 125 www.polartech.es