Re: [web2py] Re: New Relic module causing import error
Yes, we have httplib2 installed in our modules folder, and the
python-oauth2 lib runs fine if we don't import the new relic agent first.
Likewise, the new relic works fine and we get no errors if we never call
import oauth2. It's only when both are imported.
On Mon, Apr 29, 2013 at 3:51 PM, Niphlod niph...@gmail.com wrote:
are you positive that you have httplib2 installed (and available)?
custom_import machinery is tricky and the most complete picture should be
in Massimo's end.
for starters, let's be sure that httplib2 is installed ^_^
On Monday, April 29, 2013 9:45:21 PM UTC+2, Yarin wrote:
We're trying to integrate New Relic with web2py. We installed it per
their instructions for web2py running on
mod_wsgihttps://newrelic.com/docs/python/python-agent-and-web2py,
adding the following lines in wsgihandler.py:
import newrelic.agent
newrelic.agent.initialize('/**opt/web-apps/web2py/newrelic.**ini')
right before
import gluon.main
This seems to work fine, until we import another library our app uses
called python-oauth2 https://github.com/simplegeo/python-oauth2. Then
we get the following error:
import oauth2 as oauth
File /opt/web-apps/web2py/gluon/**custom_import.py, line 95, in
custom_importer
raise e2 # there is an error in the module
KeyError: 'httplib2'
I want to report this problem to New Relic, but am unsure what the root
cause might be. Is anyone familiar enough with the custom_importer to
venture a guess?
--
---
You received this message because you are subscribed to a topic in the
Google Groups web2py-users group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/web2py/5r2YXTa9vAo/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to
web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: New Relic module causing import error
Cliff- No, we are not using virtualenv On Mon, Apr 29, 2013 at 10:23 PM, Cliff Kachinske cjk...@gmail.com wrote: Are you using a virtualenv? Maybe you having a version clash between libs. -- --- You received this message because you are subscribed to a topic in the Google Groups web2py-users group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/5r2YXTa9vAo/unsubscribe?hl=en. To unsubscribe from this group and all its topics, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: New Relic module causing import error
As I mentioned earlier this is no longer an issue since we've installed httplib2 via pip. It was happening when httplib2 was being loaded from our app's directory, but I'm honestly not even sure what module version that was. At this point we're just assuming it was a problem with an older module and leaving it at that. On Mon, Apr 29, 2013 at 11:49 PM, Massimo Di Pierro massimo.dipie...@gmail.com wrote: Is this only with web2py? What if you import newrelic followed by httplib2 in from shell or in wsgi before importing web2py? On Monday, 29 April 2013 21:28:24 UTC-5, Yarin wrote: Cliff- No, we are not using virtualenv On Mon, Apr 29, 2013 at 10:23 PM, Cliff Kachinske cjk...@gmail.comwrote: Are you using a virtualenv? Maybe you having a version clash between libs. -- --- You received this message because you are subscribed to a topic in the Google Groups web2py-users group. To unsubscribe from this topic, visit https://groups.google.com/d/** topic/web2py/5r2YXTa9vAo/**unsubscribe?hl=enhttps://groups.google.com/d/topic/web2py/5r2YXTa9vAo/unsubscribe?hl=en . To unsubscribe from this group and all its topics, send an email to web2py+un...@**googlegroups.com. For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out . -- --- You received this message because you are subscribed to a topic in the Google Groups web2py-users group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/5r2YXTa9vAo/unsubscribe?hl=en. To unsubscribe from this group and all its topics, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: Scheduler: help us test it while learning
Sweet- looking forward to using the API. Schema changes a pain but done for right reasons. Can you give more explanation of the immediate=True param? As for patterns- a basic event calendar would be good demo Thanks for the great work Niphlod On Sat, Feb 23, 2013 at 1:21 PM, Niphlod niph...@gmail.com wrote: resuming historic thread. Latest commits added a few features, and changed schemas a little (my fault, sorry). Now db schema complies with check_reserved=['all'], so should work in any RDBMS out there: - scheduler_run.output -- scheduler_run.run_output - scheduler_run.result -- scheduler_run.run_result - scheduler_run.scheduler_task -- scheduler_run.task_id New features: - API available: .queue_task(), .task_status(), .resume(), .disable(), .terminate(), .kill() - W2P_TASK variable injected into tasks - a new immediate=True parameter to queue_task in order to wake up a nothing to do... looping worker The app on Github has been updated with all the new features (use web2py's trunk, not stable). I'm planning to include also common patterns using the scheduler. As always, feel free (or, should I say, compelled ? :P) to propose your most wanted patterns (e.g. manage an email queue) -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: Why aren't email/username fields unique?
Agree that is should be caught by validation- but why not enforce uniqueness on the DB as well to prevent data corruption in case the db gets hit by another source/ edited manually/ etc. Why have DB-level constraints if we don't use them in cases like this? On Sun, Feb 10, 2013 at 11:31 AM, Anthony abasta...@gmail.com wrote: The code is here: http://code.google.com/p/web2py/source/browse/gluon/tools.py#1452 Uniqueness is checked via the form validators rather than unique=True (which is enforced at the database level). The validators are more useful because they display a user friendly error message on the form rather than causing a database operational error and a 500 response. Anthony On Sunday, February 10, 2013 11:11:39 AM UTC-5, Yarin wrote: Why are email/username fields not created with (unique=True) by default? Since these fields are use to identify users, non-unique values would clearly break auth. (Side note: where is the code for defining the default auth table fields located?- can't find it anywhere) -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: calendar.js incompatible with moment.js
Not errors, just incorrect results- fails to parse dates, and returns isValid() false when testing valid date strings- see my bug report for a reproducable example- On Sat, Dec 22, 2012 at 10:27 PM, Massimo Di Pierro massimo.dipie...@gmail.com wrote: Do you get any error in the chrome console? On Saturday, 22 December 2012 20:41:43 UTC-6, Yarin wrote: We've found that the popular date library moment.js fails on date parsing when used alongside the calendar.js file included by default with web2py projects. I've submitted a bug report: http://code.google.com/p/**web2py/issues/detail?id=1243http://code.google.com/p/web2py/issues/detail?id=1243 We removed calendar.js from our projects, but I'm unclear as to what it's there for? Where is it used, and why is it included by default? Is there a reason web2py doesn't use jquery's datepicker instead? -- --
Re: [web2py] Re: Improving SSL Support
Awesome thanks Massimo- will test tonight On Thu, Oct 4, 2012 at 4:13 PM, Massimo Di Pierro massimo.dipie...@gmail.com wrote: OK. check trunk. Auth(db,secure=True). On Thursday, 4 October 2012 13:01:51 UTC-5, Yarin wrote: Yes exactly On Thursday, October 4, 2012 12:38:34 PM UTC-4, Massimo Di Pierro wrote: So... would replaing this in gluon.main.py is_https = env.wsgi_url_scheme in ['https', 'HTTPS'] or env.https=='on') with is_https = env.wsgi_url_scheme in ['https', 'HTTPS'] or env.https=='on' or request.env.http_x_**forwarded_proto in ['https', '**HTTPS'] address the first issue? Massimo On Thursday, 4 October 2012 07:05:17 UTC-5, Yarin wrote: I'm revising my stance on this. After further digging around, I'm gonna go with Niphlod's position that securing only the login traffic without securing the entire session is for the most part pretty worthless. While this might have value to some sites that have to deal with mixed content, the complexity it introduces isn't worth it. I'm also taking back my recommendation that we need to have a setting to explicitly allow SSL traffic. I think it's fine to just check the headers for forwarded SSL traffic and trust that it is. Yes, headers can be spoofed, but I can't think of how this could be exploited on the user end- So that leaves only *two recommended changes:* - When checking whether HTTPS, check for forwarded SSL headers with if request.env.http_x_**forwarded_proto in ['https', '**HTTPS']: - Add a auth.secure = True convenience setting, which would call requires_https() while the user is logged in, and on all login/registration methods. I'll update the ticket On Friday, September 21, 2012 2:26:36 PM UTC-4, Yarin wrote: Done http://code.google.com/p/**web2py/issues/detail?id=1023http://code.google.com/p/web2py/issues/detail?id=1023 On Friday, September 21, 2012 2:05:41 PM UTC-4, Massimo Di Pierro wrote: Yarin, please open an issue on google code as suggested enhancement so ti does not get lost. Also feel free to move the discussion on web2py developers. On Friday, 21 September 2012 12:22:57 UTC-5, Yarin wrote: Here's a complete example of our own implementation (simplified, untested) using the proposed auth settings: *In our model:* def force_https(trust_proxy = False, secure_session = False): Enforces HTTPS in appropriate environments Args: trust_proxy: Can we trust proxy header 'http_x_forwarded_proto' to determine SSL. (Set this only if ALL your traffic comes via trusted proxy.) secure_session: Secure the session as well. (Do this only when enforcing SSL throughout the session) # If cronjob or scheduler, exit: cronjob = request.global_settings.cronjo**b cmd_options = request.global_settings.cmd_**options if cronjob or (cmd_options and cmd_options.scheduler): return # If local host, exit: if request.env.remote_addr == 127.0.0.1: return # If already HTTPS, exit: if request.env.wsgi_url_scheme in ['https', 'HTTPS']: if secure_session: current.session.secure() return # If HTTPS request forwarded over HTTP via a SSL-terminating proxy, exit: if trust_proxy and request.env.http_x_forwarded_**proto in [ 'https', 'HTTPS']: if secure_session: current.session.secure() return # Redirect to HTTPS: redirect(URL(scheme='https', args=request.args, vars=request. vars)) # If a login function, force SSL: if request.controller == 'default' and request.function == 'user' and auth.settings.force_ssl_login: force_https(trust_proxy = auth.settings.is_proxied, secu**re_session = auth.settings.force_ssl_**session) # If user is logged in and we're enforcing a full SSL session: elif auth.is_logged_in() and auth.settings.force_ssl_**session: force_https(trust_proxy = auth.settings.is_proxied,secure_session = True) def on_login(form): Post login redirection # If we're enforcing SSL on login only, redirect from HTTPS to HTTP immediately after login: if auth.settings.force_ssl_login is True and auth.settings. force_ssl_**session is False: if request.env.wsgi_url_scheme in ['https', 'HTTPS'] orrequest .env.http_x_forwarded_**proto in ['https', 'HTTPS']: # Extract the post-login url value from auth # (hack - look at end of login() function in tools.py. This belongs in Auth itself.): login_next_path = auth.next or auth.settings.login_next # Build an absolute, HTTP url from it: login_next_url = URL(scheme='http',c='default',**f= 'index') + login_next_path[1:] # Redirect to the HTTP URL: redirect(login_next_url) auth.settings.login_onaccept = on_login On Friday, September 21, 2012 12:35:37 PM UTC-4, Yarin wrote: You can't detect this- it must be a setting.
Re: [web2py] Re: Monitoring scheduler with supervisord
You the man On Thu, Sep 20, 2012 at 12:13 PM, Niphlod niph...@gmail.com wrote: no free time at home to test on every platform, but the code part is done. in a few days. On Thursday, September 20, 2012 6:09:59 PM UTC+2, Yarin wrote: Any luck with this Simone? On Friday, September 14, 2012 12:39:19 PM UTC-4, Niphlod wrote: ok, give me a few hours, I need to get home first :P -- --
Re: [web2py] Re: Accessing auth_user the first time a user logs in
Niphlod- there's no auth object or way to access the user id (as far as I could tell) on the registration verification event, because the user is not yet logged in. But you guys are right- checking the trial or auth_event is easy enough to do on login.. On Thu, Aug 30, 2012 at 2:11 PM, Niphlod niph...@gmail.com wrote: If this is going to be required one-time-only as soon as someone verifies the registration, why don't you do this when that someone verifies the registration ? However, if trial_expires is not checkable (as Derek suggested) you can also hook into the on_login and check if auth_event holds a record of the user logged in. On Thursday, August 30, 2012 6:41:27 PM UTC+2, Yarin wrote: We have to perform a series of checks and updates on an auth_user record as soon as their registration has been verified. Previously, we had not required verification and had done everything in register_onaccept: def first_thing(form): Performs a series of checks for newly registered users. # Send welcome email: notify(subject= Welcome,message=Welcome to our app,email=auth. user.email) # Set trial expiration: today = datehelper.get_utc_midnight() # Midnight of current UTC day trial_expires_date = today+relativedelta.relativede**lta(days=+15) db(db.auth_user.id==auth.user_**id).update(trial_expires= ** trial_expires_date) auth.settings.register_**onaccept = first_thing But now that we require verification on registration, we changed the function to be called on verify_email_onaccept: auth.settings.verify_email_**onaccept = first_thing The problem is that now we get an error because Auth is None, as the user still needs to log in. So what would be the best way to update an auth_user record the first time a user actually logs in? -- --
Re: [web2py] Re: Defining stored procedures
+1 On Sun, Aug 26, 2012 at 8:55 PM, Andrew awillima...@gmail.com wrote: Thanks Anthony, Wasn't aware of that one, and it looks quite useful. On Monday, August 27, 2012 6:29:55 AM UTC+12, Anthony wrote: Often wondered about this too. You would also have to call them with executesql. So should the dal API support stored procedure , database macro definitions and execution? Would require work in each database adapter, but could we come up with a single interface ? Do you mean for creating stored procedures, or calling them? To call them, you can use db.executesql(). If you want the returned data to be parsed into a DAL Rows object like a regular select() would be, you can now specify a fields argument to executesql(). Here's the docstring explaining its usage: Added 2012-08-24 fields optional argument. If not None, the results cursor returned by the DB driver will be converted to a DAL Rows object using the db._adapter.parse() method. This requires specifying the fields argument as a list of DAL Field objects that match the fields returned from the DB. The Field objects should be part of one or more Table objects defined on the DAL object. The fields list can include one or more DAL Table objects in addition to or instead of including Field objects, or it can be just a single table (not in a list). In that case, the Field objects will be extracted from the table(s). The field names will be extracted from the Field objects, or optionally, a list of field names can be provided (in tablename.fieldname format) via the colnames argument. Note, the fields and colnames must be in the same order as the fields in the results cursor returned from the DB. Anthony -- --
Re: [web2py] Re: DAL not returning results on temp table query
Massimo- good, that's help- thanks
On Sat, Aug 25, 2012 at 4:39 PM, Massimo Di Pierro
massimo.dipie...@gmail.com wrote:
Exately. On top of this until yesterday executesql had a try fetchall()
except return None. which is now gone. If you get the latest web2py from
trunk it should give you an error.
On Saturday, 25 August 2012 13:52:42 UTC-5, Niphlod wrote:
I think the DBAPI doesn't support several results set in a single
statement.
If you cared to do those one line at a time you'd noticed that:
create temporary table tmp like people does not return results.
insert into tmp select * from people returns the just inserted values
select * from tmp returns your lines.
I think that as long as you have only one statement (and it's the last )
in your executesql() wrapped string you'll be fine. If some of your lines
returns results, then the driver doesn't know what set to actually return.
Anyway splitting those lines in multiple db.executesql()s is working
great.
On Saturday, August 25, 2012 5:58:40 PM UTC+2, Yarin wrote:
I'm trying to execute some SQL from the DAL that relies on a temp table:
CREATE TEMPORARY TABLE tmp LIKE people;
INSERT INTO tmp SELECT * FROM people;
INSERT INTO tmp SELECT * FROM people;
SELECT * FROM tmp;
This code works fine in a SQL panel, but fails when used in the DAL:
def test():
db_test = DAL('mysql://root:root@**localhost/test')
sql = CREATE TEMPORARY TABLE tmp LIKE people;
INSERT INTO tmp SELECT * FROM people;
INSERT INTO tmp SELECT * FROM people;
SELECT * FROM tmp;
results = db_test.executesql(sql)
Results always returns None. No idea why or how to proceed.
--
--
Re: [web2py] Re: Defining stored procedures
Yeah I could create them outside of web2py, was just wondering if there was a good way to perform SQL 'prep' on a db along with the table migrations.. not critical though. On Sat, Aug 25, 2012 at 6:29 PM, Anthony abasta...@gmail.com wrote: Do they need to be created dynamically by your app? If not, you could just create them outside of web2py. Side note -- if you want web2py to generate SQL for any queries, you can form the queries as usual and call the ._select() method instead of .select() -- that will return the SQL as a string rather than actually running the query. Anthony On Saturday, August 25, 2012 5:56:10 PM UTC-4, Yarin wrote: I'm assuming there's probably no support for defining stored procedures using DAL functions. Instead I was going to try to create them in raw sql using executesql whenever migrate was turned on. Just wondering if this is the best technique, or someone has another idea -- --
Re: [web2py] Re: Scheduler: help us test it while learning
I think retry_failed and repeats are two distinct concepts and shouldn't be mixed. For example, a task set to (repeats=0, retry_failed=0, period=3600) should be able to fail at 2:00pm, but will try again at 3:00pm regardless of what happened at 2:00. Likewise, if it was set to (repeats=0, retry_failed=2,period=3600), and failed all three times at 2:00pm, the retry count should be reset on the next go around. I think it's safer to presume that if a task is set up for indefinite repitition, a failure on one repeat should not bring down the whole task- rather the transactional unit that constitutes a failure should be limited to the any given attempt, repeated or not. This was one of the reasons i pressed for renaming repeats_failed to retry_failed- distinct concepts On Sat, Aug 18, 2012 at 2:56 PM, Niphlod niph...@gmail.com wrote: Can you elaborate further on the inconsistent behaviour ? repeats requeues the task n times (defaults to only completed tasks) and retry_failed make them requeued if execution fails. You have parameters to let the task be like a cron one (repeats=0, retry_failed=-1). You have also all the bits to manage your tasks (and I don't catch the inconsistency). Are you seeking for supporting some kind of requeue task only if failed at most 2 times in a 2 minutes timeframe ? On Saturday, August 18, 2012 7:45:46 PM UTC+2, Yarin wrote: I've noticed that repeating tasks that fail during a certain period are no longer repeated and the task is turned to FAILED. I think this is inconsistent behavior. The better approach would be: - Allow a periodic task to fail during a given period - Reset the task to QUEUED, just like when a periodic task completes - Have the scheduler_run table record the failure In other words, retry_failed should apply to the current repeated attempt, not to the totality of the task. Another smaller issue- in the scheduler_task table definition, can we place the last_run_time field betwen the start_time and next_run_time fields. This way they are grouped clearly in the appadmin screens. Thanks-- On Thursday, July 12, 2012 4:36:38 PM UTC-4, Niphlod wrote: Hello everybody, in the last month several changes were commited to the scheduler, in order to improve it. Table schemas were changed, to add some features that were missed by some users. On the verge of releasing web2py v.2.0.0, and seeing that the scheduler potential is often missed by regular web2py users, I created a test app with two main objectives: documenting the new scheduler and test the features. App is available on github (https://github.com/niphlod/** w2p_scheduler_tests https://github.com/niphlod/w2p_scheduler_tests). All you need is download the trunk version of web2py, download the app and play with it. Current features: - one-time-only tasks - recurring tasks - possibility to schedule functions at a given time - possibility to schedule recurring tasks with a stop_time - can operate distributed among machines, given a database reachable for all workers - group_names to divide tasks among different workers - group_names can also influence the percentage of assigned tasks to similar workers - simple integration using modules for embedded tasks (i.e. you can use functions defined in modules directly in your app or have them processed in background) - configurable heartbeat to reduce latency: with sane defaults and not t many tasks queued normally a queued task doesn't exceed 5 seconds execution times - option to start it, process all available tasks and then die automatically - integrated tracebacks - monitorable as state is saved on the db - integrated app environment if started as web2py.py -K - stop processes immediately (set them to KILL) - stop processes gracefully (set them to TERMINATE) - disable processes (set them to DISABLED) - functions that doesn't return results do not generate a scheduler_run entry - added a discard_results parameter that doesn't store results no matter what - added a uuid record to tasks to simplify checkings of unique tasks - task_name is not required anymore - you can skip passing the function to the scheduler istantiation: functions can be dinamically retrieved in the app's environment So, your mission is: - test the scheduler with the app and familiarize with it Secondary mission is: - report any bug you find here or on github (https://github.com/niphlod/ **w2p_scheduler_tests/issueshttps://github.com/niphlod/w2p_scheduler_tests/issues ) - propose new examples to be embedded in the app, or correct the current docs (English is not my mother tongue) Once approved, docs will be probably embedded in the book ( http://web2py.com/book) Feel free to propose features you'd like to see in the scheduler, I have some time to spend implementing it. -- --
Re: [web2py] Re: Scheduler: help us test it while learning
And the reason i think the behavior's inconsistent is because when you complete an attempt on a repeating task it is immediately requeued to fulfill its repeat obligations, and the last go-around is forgotton- so i think failures should be handled the same way. On Sat, Aug 18, 2012 at 4:32 PM, Yarin Kessler ykess...@gmail.com wrote: I think retry_failed and repeats are two distinct concepts and shouldn't be mixed. For example, a task set to (repeats=0, retry_failed=0, period=3600) should be able to fail at 2:00pm, but will try again at 3:00pm regardless of what happened at 2:00. Likewise, if it was set to (repeats=0, retry_failed=2,period=3600), and failed all three times at 2:00pm, the retry count should be reset on the next go around. I think it's safer to presume that if a task is set up for indefinite repitition, a failure on one repeat should not bring down the whole task- rather the transactional unit that constitutes a failure should be limited to the any given attempt, repeated or not. This was one of the reasons i pressed for renaming repeats_failed to retry_failed- distinct concepts On Sat, Aug 18, 2012 at 2:56 PM, Niphlod niph...@gmail.com wrote: Can you elaborate further on the inconsistent behaviour ? repeats requeues the task n times (defaults to only completed tasks) and retry_failed make them requeued if execution fails. You have parameters to let the task be like a cron one (repeats=0, retry_failed=-1). You have also all the bits to manage your tasks (and I don't catch the inconsistency). Are you seeking for supporting some kind of requeue task only if failed at most 2 times in a 2 minutes timeframe ? On Saturday, August 18, 2012 7:45:46 PM UTC+2, Yarin wrote: I've noticed that repeating tasks that fail during a certain period are no longer repeated and the task is turned to FAILED. I think this is inconsistent behavior. The better approach would be: - Allow a periodic task to fail during a given period - Reset the task to QUEUED, just like when a periodic task completes - Have the scheduler_run table record the failure In other words, retry_failed should apply to the current repeated attempt, not to the totality of the task. Another smaller issue- in the scheduler_task table definition, can we place the last_run_time field betwen the start_time and next_run_time fields. This way they are grouped clearly in the appadmin screens. Thanks-- On Thursday, July 12, 2012 4:36:38 PM UTC-4, Niphlod wrote: Hello everybody, in the last month several changes were commited to the scheduler, in order to improve it. Table schemas were changed, to add some features that were missed by some users. On the verge of releasing web2py v.2.0.0, and seeing that the scheduler potential is often missed by regular web2py users, I created a test app with two main objectives: documenting the new scheduler and test the features. App is available on github (https://github.com/niphlod/** w2p_scheduler_tests https://github.com/niphlod/w2p_scheduler_tests). All you need is download the trunk version of web2py, download the app and play with it. Current features: - one-time-only tasks - recurring tasks - possibility to schedule functions at a given time - possibility to schedule recurring tasks with a stop_time - can operate distributed among machines, given a database reachable for all workers - group_names to divide tasks among different workers - group_names can also influence the percentage of assigned tasks to similar workers - simple integration using modules for embedded tasks (i.e. you can use functions defined in modules directly in your app or have them processed in background) - configurable heartbeat to reduce latency: with sane defaults and not t many tasks queued normally a queued task doesn't exceed 5 seconds execution times - option to start it, process all available tasks and then die automatically - integrated tracebacks - monitorable as state is saved on the db - integrated app environment if started as web2py.py -K - stop processes immediately (set them to KILL) - stop processes gracefully (set them to TERMINATE) - disable processes (set them to DISABLED) - functions that doesn't return results do not generate a scheduler_run entry - added a discard_results parameter that doesn't store results no matter what - added a uuid record to tasks to simplify checkings of unique tasks - task_name is not required anymore - you can skip passing the function to the scheduler istantiation: functions can be dinamically retrieved in the app's environment So, your mission is: - test the scheduler with the app and familiarize with it Secondary mission is: - report any bug you find here or on github ( https://github.com/niphlod/**w2p_scheduler_tests/issueshttps://github.com/niphlod/w2p_scheduler_tests/issues ) - propose new examples to be embedded in the app, or correct the current docs (English
Re: [web2py] Re: Scheduler: help us test it while learning
10 4 - thanks On Tue, Aug 14, 2012 at 7:48 PM, niphlod niph...@gmail.com wrote: Nope, that goes wyyy over the scheduler responsibility. Prune all records, prune only completed, prune only failed, requeue timeoutted, prune every day, every hour, etc, etc, etc these are implementation details that belongs to the application. We though that since it is all recorded and timestamped it's a matter of: timelimit = datetime.datetime.utcnow() - datetime.timedelta(days=15) db((db.scheduler_task.status == 'COMPLETED') (db.scheduler_task.last_run_time timelimit)).delete() that is actually not so hard (scheduler_run records should be pruned away automatically because they are referenced) (I like to have a function maintenance fired off every now and then with these things on it.) 2012/8/15 Yarin ykess...@gmail.com Niphlod- has there been any discussion about a param for clearing out old records on the runs and tasks tables? Maybe a retain_results or retain_completed value that specifies a period for which records will be kept? On Thursday, July 12, 2012 4:36:38 PM UTC-4, Niphlod wrote: Hello everybody, in the last month several changes were commited to the scheduler, in order to improve it. Table schemas were changed, to add some features that were missed by some users. On the verge of releasing web2py v.2.0.0, and seeing that the scheduler potential is often missed by regular web2py users, I created a test app with two main objectives: documenting the new scheduler and test the features. App is available on github (https://github.com/niphlod/** w2p_scheduler_tests https://github.com/niphlod/w2p_scheduler_tests). All you need is download the trunk version of web2py, download the app and play with it. Current features: - one-time-only tasks - recurring tasks - possibility to schedule functions at a given time - possibility to schedule recurring tasks with a stop_time - can operate distributed among machines, given a database reachable for all workers - group_names to divide tasks among different workers - group_names can also influence the percentage of assigned tasks to similar workers - simple integration using modules for embedded tasks (i.e. you can use functions defined in modules directly in your app or have them processed in background) - configurable heartbeat to reduce latency: with sane defaults and not t many tasks queued normally a queued task doesn't exceed 5 seconds execution times - option to start it, process all available tasks and then die automatically - integrated tracebacks - monitorable as state is saved on the db - integrated app environment if started as web2py.py -K - stop processes immediately (set them to KILL) - stop processes gracefully (set them to TERMINATE) - disable processes (set them to DISABLED) - functions that doesn't return results do not generate a scheduler_run entry - added a discard_results parameter that doesn't store results no matter what - added a uuid record to tasks to simplify checkings of unique tasks - task_name is not required anymore - you can skip passing the function to the scheduler istantiation: functions can be dinamically retrieved in the app's environment So, your mission is: - test the scheduler with the app and familiarize with it Secondary mission is: - report any bug you find here or on github (https://github.com/niphlod/ **w2p_scheduler_tests/issueshttps://github.com/niphlod/w2p_scheduler_tests/issues ) - propose new examples to be embedded in the app, or correct the current docs (English is not my mother tongue) Once approved, docs will be probably embedded in the book ( http://web2py.com/book) Feel free to propose features you'd like to see in the scheduler, I have some time to spend implementing it. -- -- --
Re: [web2py] Re: Requiring password on registration
Grazie On Sun, Aug 12, 2012 at 12:09 PM, Anthony abasta...@gmail.com wrote: Note, this has now been fixed in trunk: http://code.google.com/p/web2py/source/detail?r=73b2137eb32e61abc47b491786643d46c199 On Saturday, August 11, 2012 11:21:12 PM UTC-4, Anthony wrote: Looks like a bug in the wizard: db.auth_user.password.requires = CRYPT(key=auth.settings.hmac_**key) should be: db.auth_user.password.requires = CRYPT(key=auth.settings.hmac_**key,min_length =4) Actually, maybe the wizard should simply use auth.define_tables() instead of manually creating the auth_user table -- would be easier to keep the wizard consistent with standard apps. Anthony On Saturday, August 11, 2012 8:40:12 PM UTC-4, Yarin wrote: Anthony- I'm on 1.99.7 stable. Just tried it it with a fresh basic app and there the problem doesn't exist. However, if you use the *app wizard *, and just click through accepting all the defaults, it will recreate the issue. Note also that it DOES allow logging in with no password subsequent to the registration. Adding the requires.insert(0,IS_**LENGTH (minsize=5)) has no effect. I've attached the app I just generated with the wizard on all default settings. On Saturday, August 11, 2012 7:10:29 PM UTC-4, Anthony wrote: Anthony's correct- this is in Auth, but is not being respected: settings.password_min_length = 4 I just created a fresh app using trunk, and when I try to register without a password, I get a too short error message on the password field. What version of web2py are you using? Can we see your Auth related code? Or maybe you can pack and attach a simple app that exhibits the behavior. Massimo - For security reason it will not be accepted for logging anyway. - Not sure what you meant but it registered me and logged me in with an empty pass. By default, when you first register, you are automatically logged in upon registration (this can be disabled). I think Massimo meant that subsequently you will not be able to continue to login with an empty password. I also tried inserting: db.auth_user.password.requires**.insert(0,IS_LENGTH(minsize=5)**) What happens whey you try that -- does it generate an error, or simply allow you to register without a password? Anthony -- --
Re: [web2py] Re: Strange behavior db( ... and ... and ...) vs. db( ... or ... or ...)
Sorry- I missed that altogether. On Sat, Aug 11, 2012 at 11:25 PM, Anthony abasta...@gmail.com wrote: http://web2py.com/books/default/chapter/29/6#Logical-operators Specifically: Due to Python restrictions in overloading and and or operators, these cannot be used in forming queries. The binary operators and | must be used instead. Note that these operators (unlike and and or) have higher precedence than comparison operators, so the extra parentheses in the above examples are mandatory. Did you have something else in mind? Anthony On Saturday, August 11, 2012 8:57:38 PM UTC-4, Yarin wrote: Massimo, could we highlight this issue in the documentation somewhere, and/or show the right way to do a compound query. There's no documentation on this stuff, it's very easy to accidentally do because won't throw a syntax error, and we almost just shipped some production code with 'and' instead of On Thursday, June 18, 2009 11:43:37 AM UTC-4, mdipierro wrote: follow up query1 or query2 IS INVALID SYNTAX query1|query2 IS CORRECT or cannot be overwritten in Python so it is still valid but it does not do what you want. query1 or query2 ALWAYS returns just query1. Massimo On Jun 18, 10:43 am, mdipierro mdipie...@cs.depaul.edu wrote: query1 and query2 IS INVALID SYNTAX query1query2 IS CORRECT and cannot be overwritten in Python so it is still valid but it does not do what you want. query1 and query2 ALWAYS returns just query2. Massimo On Jun 18, 8:58 am, weheh richard_gor...@verizon.net wrote: I'm using web2py 1.64.1. I'm seeing a strange behavior in my query. I'm trying to isolate a record using a compound and clause ... very simple stuff: records = db( (db.x.a == i) and (db.x.b==0) and (db.x.c==None) and ...).select(db.x.ALL) But, it returns a whole bunch of records that have nothing to do with my match criteria. I redid the search with an or clause instead, (don't ask me why I did this 'cause it makes no sense to me): records = db( (db.x.a == i) or (db.x.b==0) or (db.x.c==None) or ...).select(db.x.ALL) and believe it or not, this works fine. I think I'm going nuts! Why should this work? -- --
Re: [web2py] Re: Cookie chaos- please see this example
The book doesn't describe the defaults. I think mentioning that cookies will be set to the specific URL, args and all, would be helpful. On Thu, Aug 9, 2012 at 12:30 PM, Anthony abasta...@gmail.com wrote: To me a default where apps share cookies doesn't seem out of with line with the other things apps share- the default logger for example. Cookies are for end-user functionality; the logger is for development/debugging. For the most part, the apps don't share anything. Just as making an app-specific logger can be achieved by passing request.application to a new instance, so we can prefix app-specific cookies with the app name. To me it's the more obvious default. If the apps are truly independent and unrelated, I would think using a cookie would more likely be for some app-specific functionality. On the other hand, in most cases, you probably don't have multiple apps running on the same domain, or if you do, those apps are related and might need to share cookies. But it still might be safer to require the developer to set the path explicitly. Note, I think this is reasonably well documented in the book: http://web2py.com/books/default/chapter/29/4#Cookies. Anthony -- --
Re: [web2py] Re: Cookie chaos- please see this example
Thanks Anthony- much clearer On Thu, Aug 9, 2012 at 1:22 PM, Anthony abasta...@gmail.com wrote: On Thursday, August 9, 2012 12:59:13 PM UTC-4, Yarin wrote: The book doesn't describe the defaults. I think mentioning that cookies will be set to the specific URL, args and all, would be helpful. Note, web2py isn't setting the path at all -- if you don't set a path, there is none. In that case, browsers assume the path of the requested resource -- see http://en.wikipedia.org/wiki/HTTP_cookie#Domain_and_Path. This is not specific to web2py but simply how cookies work. Anyway, I went ahead and added a note to the book pointing this out. Anthony -- --
Re: [web2py] Re: Scheduler: help us test it while learning
Alan- the scheduler relies on a normalized table structure that's impossible to implement in GAE, but the GAE has its own task scheduler if I recall. EC2 should be fine as long as you've got a supported DB somewhere. On Mon, Aug 6, 2012 at 9:05 AM, Alan Etkin spame...@gmail.com wrote: Feel free to propose features you'd like to see in the scheduler, I have some time to spend implementing it. Will (or could) scheduler support multi-platform apps? (EC2, GAE, ...)? -- --
Re: [web2py] Re: Scheduler won't run with request.requires_https()
Massimo- I don't understand your answer- My question was about how to run scheduler WITH request.requires_https() turned on. Are you saying this is not possible? My main app needs https protection. My scheduler makes use of the db/models/methods of the main app, so what is your recommendation? Do I need to separate the scheduler into a separate app and make cross-app calls? On Thu, Jun 21, 2012 at 2:36 PM, Massimo Di Pierro massimo.dipie...@gmail.com wrote: You have this line in request.requires_https() In myapp/models/db.**py, line 10, in module it prevents the scheduler form importing the modules. On Thursday, 21 June 2012 11:15:32 UTC-5, Yarin wrote: I can't get the scheduler to run when I turn on request.requires_https(). $ /opt/web-apps/web2py/web2py.py -K myapp Result: starting scheduler for myapp... Currently running 1 scheduler processes Processes started Process Process-1: Traceback (most recent call last): File /usr/lib64/python2.6/**multiprocessing/process.py, line 232, in _bootstrap self.run() File /usr/lib64/python2.6/**multiprocessing/process.py, line 88, in run self._target(*self._args, **self._kwargs) File /opt/web-apps/web2py/gluon/**shell.py, line 189, in run _env = env(a, c=c, import_models=import_models) File /opt/web-apps/web2py/gluon/**shell.py, line 131, in env run_models_in(environment) File /opt/web-apps/web2py/gluon/**compileapp.py, line 513, in run_models_in restricted(code, environment, layer=model) File /opt/web-apps/web2py/gluon/**restricted.py, line 205, in restricted exec ccode in environment File applications/myapp/models/db.**py, line 10, in module request.requires_https() File /opt/web-apps/web2py/gluon/**globals.py, line 119, in requires_https redirect(URL(scheme='https', args=self.args, vars=self.vars)) File /opt/web-apps/web2py/gluon/**http.py, line 128, in redirect Location=location) HTTP: 303 SEE OTHER LAMP/mod_wsgi environment. Anyone deal with this already? -- --
[web2py] Re: Re[2]: SQLite Logging
@Iceberg- Thanks for all the input- I'll review your latest and update the
code later this weekend. With respect to log file location though, don't
think it makes sense to put it in static since it's a file that will be
written to (i.e. non static). It may need its own location, a var/ or
log/... Massimo, thoughts on this?
On Fri, Jul 23, 2010 at 3:02 PM, Iceberg iceb...@21cn.com wrote:
Well done Yarin! Some comments.
1. Change those time_expire= into time_expire=None please. The
clumsy trick was my early invention before I know the
time_expire=None approach.
2. My another later improvement is to write the log file at {web2py
path}/applications/{your app}/static/app.log
so that developer can easily access the log file by
http://your_host/your_app/static/app.log
Well, you can refer to the attachment for my latest implementation. Some
more trick inside.
3. @Massimo, I suggested to include the applog.py into
welcome/model/applog.py since day1 I invented this little component. Only
this way people will have an official place to always have the latest code,
rather than dig some not-the-latest post from different places. So please do
consider it this time.
4. @Yarin, when last time I said putting the web2py db instance, rather
than a filename, to initialize your SQLiteHandler() class, I mean something
like:
def _init_sqlite_log(level=logging.DEBUG):
handler = SQLiteHandler( db ) # db is defined by: db =
DAL('sqlite://storage.sqlite')
# or
handler = SQLiteHandler( DAL('sqlite://log.sqlite') )
There is some more implementation to do, but you get the idea.
5. With respect to GAE, this was my concept vehicle:
if request.env.web2py_runtime_gae: # if running on Google App Engine
handler=logging.handlers.HTTPHandler(request.env.http_host,URL(r=request,f='log'))
# assuming there is an optional log action
but I never test it. I think a practical solution would be implement some
log handler which wraps GAE's own log facility.
Best regards,
Iceberg, 2010-Jul-24, 02:35(AM), Sat
--- Original Message ---
From:Yarin Kessler ykess...@gmail.com
To: Massimo Di Pierro mdipie...@cs.depaul.edu
Cc: web2py-users web2py@googlegroups.com, Iceberg iceb...@21cn.com,
Hans johann.scheibelho...@easytouch-edv.com, Richard
richar...@gmail.com, MikeEllis michael.f.el...@gmail.com, cjparsons
cjparso...@yahoo.co.uk
Date:Fri, 23 Jul 2010 09:56:17 -0400
Subject: Re: SQLite Logging
---
Glad you like- I'd love to see this as part of the core. Let me know if
there's anything else I can do.
I am not so keen to include the sqlite part because it is too specific
Is this because it requires the extra module? I kept it separate because
it's a Python, not web2py, specific class, but we could easily bake it
into
log.py.
Personally I'm a big fan of SQLite for logging- what would it take to
make
it less specific and ready for inclusion?
we would need a way to deal with GAE/
Hans/Iceberg had a GAE solution in their orig file, but I haven't tested
it:
if request.env.web2py_runtime_gae: # if running on Google App Engine
handler=logging.handlers.HTTPHandler(request.env.http_host,URL(r=request,f='log'))
# assuming there is an optional log action
else:
handler=logging.handlers.RotatingFileHandler(os.path.join(request.folder,filename),maxBytes=maxBytes,backupCount=backupCount)
I'll re-introduce it into the code though.
On Fri, Jul 23, 2010 at 9:38 AM, Massimo Di Pierro
mdipie...@cs.depaul.eduwrote:
Very good work. We should think about a way to include this in web2py
core.
I am not so keen to include the sqlite part because it is too specific
but:
I see the log.py file could go in contrib with minimal changes (it it
were
implemented as a function that takes the request and cache objects)
we would need a way to deal with GAE/
On Jul 23, 2010, at 8:24 AM, Yarin wrote:
Added a slice: Application Logging
http://web2pyslices.com/main/slices/take_slice/91
It covers logging to both a file and SQLite. The code is well
documented- use this instead of the previous code. Would love to get
your comments.
@Iceberg, btw apparently the SQLite cross-thread issues weren't solved
by using the default DB after all. I added a thread-safe work around,
but suggestions are welcome.
