[web2py] Re: Scheduler as upstart job
I got the "error cleaning up" message after updating the web2py. I had set user specific permissions, thats what happened. Well its resolved now. Thanks for all the support given by the web2py group. On Tuesday, 17 September 2013 17:50:58 UTC+5:30, Niphlod wrote: > > catching those kind of errors is going to be a real PITA... every backend > (i.e. database engine) logs the error as it wishes, so we don't have a > "method" for pointing what is going wrong. No logs you reported included > "error cleaning up": if you told me earlier, I would have pointed you in > the right direction. > > PS: every web2py part is kinda "required" to have read and write > permission everywhere nobody really had any problem with it ^_^ > > On Tuesday, September 17, 2013 1:47:34 PM UTC+2, ajith c t wrote: >> >> Hi Niphlod, >> Got my scheduler working correctly. >> I updated my web2py version to 2.5.1 from 2.2.1 as you mentioned >> earlier. After that started two upstart jobs, one for my app and another >> for the scheduler. Still it returned an error "error cleaning up.". >> I couldn't find an appropriate solution for it net so checked the >> code of scheduler.py file in gluon folder. Found that an exception raises >> this error at line number 797. The exception doesn't say anything about the >> error, just prints the message. After calling the Exception class and >> printing the exception, found out that it was a permission problem. I had >> not given the DELETE permission to the user. Once that was corrected, >> everything worked fine as I needed, >> >> It would be helpful if you could also log the exception that displays the >> message from the scheduler.py file. I had to spend lot of time to find this >> error, maybe my mistake as I am a web2py beginner, but it would really help >> if the cause of the exception is also logged, >> >> On Thursday, 12 September 2013 12:36:33 UTC+5:30, ajith c t wrote: >>> >>> Thank you for the response, will try everything you said and post it if >>> every thing goes correct >>> >>> On Thursday, 12 September 2013 00:34:31 UTC+5:30, Niphlod wrote: On Wednesday, September 11, 2013 5:44:00 PM UTC+2, ajith c t wrote: > > sorry I am not executing the code in the load balancer. Let me say > once more, so it will be clear. > > I dont have any web2py code in the load balancer. > > What I meant is I have my code in another server(say X) , to which > the user is redirected by the load balancer(say Y). > > when I execute the webpy application from server X , the index > function , where my scheduler.queue_task call comes,in the default.py > controller will not be called as I cant access it from X. That is > expected. > So what I asked was , for breaking down purpose shouldn't I move that > call > into the scheduler.py file in the models folder as the model file will be > called. > > I really don't get this part. Every controller in web2py gets executed as long as the request comes to the server. If you can reach a page and in your controller for that page you use queue_task(), you should definitely be able to enqueue a task. queue_task() is NOT meant to be used in models, because you'd queue a task for EACH and EVERY request coming in. > > And I will use the "seperation of duties" approach. but can you just > confirm that the commands are correct. > > web part: python web2py.py -a yourpassword -p 8000 -i 0.0.0.0 scheduler python web2py.py -K appname let me stress it out once more: serving a web app in production with the included webserver is not going to provide stellar performances, although it definitely works. >>> -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
catching those kind of errors is going to be a real PITA... every backend (i.e. database engine) logs the error as it wishes, so we don't have a "method" for pointing what is going wrong. No logs you reported included "error cleaning up": if you told me earlier, I would have pointed you in the right direction. PS: every web2py part is kinda "required" to have read and write permission everywhere nobody really had any problem with it ^_^ On Tuesday, September 17, 2013 1:47:34 PM UTC+2, ajith c t wrote: > > Hi Niphlod, > Got my scheduler working correctly. > I updated my web2py version to 2.5.1 from 2.2.1 as you mentioned > earlier. After that started two upstart jobs, one for my app and another > for the scheduler. Still it returned an error "error cleaning up.". > I couldn't find an appropriate solution for it net so checked the > code of scheduler.py file in gluon folder. Found that an exception raises > this error at line number 797. The exception doesn't say anything about the > error, just prints the message. After calling the Exception class and > printing the exception, found out that it was a permission problem. I had > not given the DELETE permission to the user. Once that was corrected, > everything worked fine as I needed, > > It would be helpful if you could also log the exception that displays the > message from the scheduler.py file. I had to spend lot of time to find this > error, maybe my mistake as I am a web2py beginner, but it would really help > if the cause of the exception is also logged, > > On Thursday, 12 September 2013 12:36:33 UTC+5:30, ajith c t wrote: >> >> Thank you for the response, will try everything you said and post it if >> every thing goes correct >> >> On Thursday, 12 September 2013 00:34:31 UTC+5:30, Niphlod wrote: >>> >>> >>> >>> On Wednesday, September 11, 2013 5:44:00 PM UTC+2, ajith c t wrote: sorry I am not executing the code in the load balancer. Let me say once more, so it will be clear. I dont have any web2py code in the load balancer. What I meant is I have my code in another server(say X) , to which the user is redirected by the load balancer(say Y). when I execute the webpy application from server X , the index function , where my scheduler.queue_task call comes,in the default.py controller will not be called as I cant access it from X. That is expected. So what I asked was , for breaking down purpose shouldn't I move that call into the scheduler.py file in the models folder as the model file will be called. >>> I really don't get this part. Every controller in web2py gets executed >>> as long as the request comes to the server. If you can reach a page and in >>> your controller for that page you use queue_task(), you should definitely >>> be able to enqueue a task. queue_task() is NOT meant to be used in models, >>> because you'd queue a task for EACH and EVERY request coming in. >>> >>> And I will use the "seperation of duties" approach. but can you just confirm that the commands are correct. >>> web part: >>> python web2py.py -a yourpassword -p 8000 -i 0.0.0.0 >>> >>> scheduler >>> python web2py.py -K appname >>> >>> let me stress it out once more: serving a web app in production with the >>> included webserver is not going to provide stellar performances, although >>> it definitely works. >>> >> -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
Hi Niphlod, Got my scheduler working correctly. I updated my web2py version to 2.5.1 from 2.2.1 as you mentioned earlier. After that started two upstart jobs, one for my app and another for the scheduler. Still it returned an error "error cleaning up.". I couldn't find an appropriate solution for it net so checked the code of scheduler.py file in gluon folder. Found that an exception raises this error at line number 797. The exception doesn't say anything about the error, just prints the message. After calling the Exception class and printing the exception, found out that it was a permission problem. I had not given the DELETE permission to the user. Once that was corrected, everything worked fine as I needed, It would be helpful if you could also log the exception that displays the message from the scheduler.py file. I had to spend lot of time to find this error, maybe my mistake as I am a web2py beginner, but it would really help if the cause of the exception is also logged, On Thursday, 12 September 2013 12:36:33 UTC+5:30, ajith c t wrote: > > Thank you for the response, will try everything you said and post it if > every thing goes correct > > On Thursday, 12 September 2013 00:34:31 UTC+5:30, Niphlod wrote: >> >> >> >> On Wednesday, September 11, 2013 5:44:00 PM UTC+2, ajith c t wrote: >>> >>> sorry I am not executing the code in the load balancer. Let me say once >>> more, so it will be clear. >>> >>> I dont have any web2py code in the load balancer. >>> >>> What I meant is I have my code in another server(say X) , to which the >>> user is redirected by the load balancer(say Y). >>> >>> when I execute the webpy application from server X , the index function >>> , where my scheduler.queue_task call comes,in the default.py controller >>> will not be called as I cant access it from X. That is expected. So what I >>> asked was , for breaking down purpose shouldn't I move that call into the >>> scheduler.py file in the models folder as the model file will be called. >>> >>> >> I really don't get this part. Every controller in web2py gets executed as >> long as the request comes to the server. If you can reach a page and in >> your controller for that page you use queue_task(), you should definitely >> be able to enqueue a task. queue_task() is NOT meant to be used in models, >> because you'd queue a task for EACH and EVERY request coming in. >> >> >>> >>> And I will use the "seperation of duties" approach. but can you just >>> confirm that the commands are correct. >>> >>> >> web part: >> python web2py.py -a yourpassword -p 8000 -i 0.0.0.0 >> >> scheduler >> python web2py.py -K appname >> >> let me stress it out once more: serving a web app in production with the >> included webserver is not going to provide stellar performances, although >> it definitely works. >> > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
Thank you for the response, will try everything you said and post it if every thing goes correct On Thursday, 12 September 2013 00:34:31 UTC+5:30, Niphlod wrote: > > > > On Wednesday, September 11, 2013 5:44:00 PM UTC+2, ajith c t wrote: >> >> sorry I am not executing the code in the load balancer. Let me say once >> more, so it will be clear. >> >> I dont have any web2py code in the load balancer. >> >> What I meant is I have my code in another server(say X) , to which the >> user is redirected by the load balancer(say Y). >> >> when I execute the webpy application from server X , the index function , >> where my scheduler.queue_task call comes,in the default.py controller will >> not be called as I cant access it from X. That is expected. So what I asked >> was , for breaking down purpose shouldn't I move that call into the >> scheduler.py file in the models folder as the model file will be called. >> >> > I really don't get this part. Every controller in web2py gets executed as > long as the request comes to the server. If you can reach a page and in > your controller for that page you use queue_task(), you should definitely > be able to enqueue a task. queue_task() is NOT meant to be used in models, > because you'd queue a task for EACH and EVERY request coming in. > > >> >> And I will use the "seperation of duties" approach. but can you just >> confirm that the commands are correct. >> >> > web part: > python web2py.py -a yourpassword -p 8000 -i 0.0.0.0 > > scheduler > python web2py.py -K appname > > let me stress it out once more: serving a web app in production with the > included webserver is not going to provide stellar performances, although > it definitely works. > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
On Wednesday, September 11, 2013 5:44:00 PM UTC+2, ajith c t wrote: > > sorry I am not executing the code in the load balancer. Let me say once > more, so it will be clear. > > I dont have any web2py code in the load balancer. > > What I meant is I have my code in another server(say X) , to which the > user is redirected by the load balancer(say Y). > > when I execute the webpy application from server X , the index function , > where my scheduler.queue_task call comes,in the default.py controller will > not be called as I cant access it from X. That is expected. So what I asked > was , for breaking down purpose shouldn't I move that call into the > scheduler.py file in the models folder as the model file will be called. > > I really don't get this part. Every controller in web2py gets executed as long as the request comes to the server. If you can reach a page and in your controller for that page you use queue_task(), you should definitely be able to enqueue a task. queue_task() is NOT meant to be used in models, because you'd queue a task for EACH and EVERY request coming in. > > And I will use the "seperation of duties" approach. but can you just > confirm that the commands are correct. > > web part: python web2py.py -a yourpassword -p 8000 -i 0.0.0.0 scheduler python web2py.py -K appname let me stress it out once more: serving a web app in production with the included webserver is not going to provide stellar performances, although it definitely works. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
ok. the scheduler from the console runs right. Now you have to test if queuing a task from web2py gets the task processed. Then you can do the same thing with the scheduler started by upstart, and verify that it's working. Next is to verify that you can queue tasks from the web2py web server started by upstart. Next to verify that the task queued gets processed by the scheduler in the console Next to verify that it gets processed by the scheduler in upstart Something along the way should reveal where the breakage that you experiment using both the webserver and the scheduler in upstart ^_^ -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
Agreed on the first point. Here the tasks are not getting assigned because I am calling the queue_task from the controller. I believe the controller function 'index' didn't not load as app cannot be opened in the browser. And the second point, yes the web2py version is a bit old, I had a little fear if I update the app midway. Well I will update it tonight . On Wednesday, 11 September 2013 20:16:46 UTC+5:30, Niphlod wrote: > > so, the scheduler is working pretty fine: it just doesn't find any task to > process. > > PS: you are on a really old release (2.2.1). That's almost a year ago! > > On Wednesday, September 11, 2013 4:23:07 PM UTC+2, ajith c t wrote: >> >> ubuntu@ip-10-0-0-13:/srv/trustvouch-fe$ sudo python web2py.py -K app -D 0 >> web2py Web Framework >> Created by Massimo Di Pierro, Copyright 2007-2013 >> Version 2.2.1 (2012-10-21 16:57:04) stable >> Database drivers available: SQLite(sqlite3), MySQL(pymysql), >> PostgreSQL(pg8000), IMAP(imaplib) >> starting single-scheduler for "app"... >> /usr/local/lib/python2.7/dist-packages/riak-2.0.1.post5-py2.7.egg/riak/util.py:148: >> >> UserWarning: ``solr`` is deprecated, use ``fulltext_search``, >> ``fulltext_add`` and ``fulltext_delete`` directly >> value = self.fget(obj) >> >> >> >> >> 2013-09-11 14:21:20,236 - DEBUG - web2py.scheduler - defining tables >> (migrate=True) >> 2013-09-11 14:21:20,242 - DEBUG - web2py.scheduler - looping... >> 2013-09-11 14:21:20,246 - DEBUG - web2py.scheduler - nothing to do >> 2013-09-11 14:21:20,246 - DEBUG - web2py.scheduler - sleeping... >> 2013-09-11 14:21:20,246 - DEBUG - web2py.scheduler - thread building own >> DAL object >> 2013-09-11 14:21:20,249 - DEBUG - web2py.scheduler - defining tables >> (migrate=False) >> 2013-09-11 14:21:20,254 - DEBUG - web2py.scheduler - freeing workers >> that have not sent heartbeat >> 2013-09-11 14:21:23,249 - DEBUG - web2py.scheduler - looping... >> 2013-09-11 14:21:23,254 - DEBUG - web2py.scheduler - nothing to do >> 2013-09-11 14:21:23,254 - DEBUG - web2py.scheduler - sleeping... >> 2013-09-11 14:21:23,263 - DEBUG - web2py.scheduler - recording >> heartbeat >> 2013-09-11 14:21:26,257 - DEBUG - web2py.scheduler - looping... >> 2013-09-11 14:21:26,261 - DEBUG - web2py.scheduler - nothing to do >> 2013-09-11 14:21:26,262 - DEBUG - web2py.scheduler - sleeping... >> 2013-09-11 14:21:26,270 - DEBUG - web2py.scheduler - recording >> heartbeat >> 2013-09-11 14:21:29,265 - DEBUG - web2py.scheduler - looping... >> 2013-09-11 14:21:29,269 - DEBUG - web2py.scheduler - nothing to do >> 2013-09-11 14:21:29,269 - DEBUG - web2py.scheduler - sleeping... >> 2013-09-11 14:21:29,277 - DEBUG - web2py.scheduler - recording >> heartbeat >> >> >> above the out put of the log, >> >> On Wednesday, 11 September 2013 19:48:06 UTC+5:30, Niphlod wrote: >>> >>> let's solve an issue at a time. If your scheduler can't process the >>> tasks that are in the queue when you launch it from the console, something >>> is wrong with it and there's nothing that upstart can "fix". >>> If you launch your scheduler from the console with >>> web2py.py -K yourappname -D 0 >>> what is the result ? >>> >> -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
sorry I am not executing the code in the load balancer. Let me say once more, so it will be clear. I dont have any web2py code in the load balancer. What I meant is I have my code in another server(say X) , to which the user is redirected by the load balancer(say Y). when I execute the webpy application from server X , the index function , where my scheduler.queue_task call comes,in the default.py controller will not be called as I cant access it from X. That is expected. So what I asked was , for breaking down purpose shouldn't I move that call into the scheduler.py file in the models folder as the model file will be called. And I will use the "seperation of duties" approach. but can you just confirm that the commands are correct. On Wednesday, 11 September 2013 21:02:05 UTC+5:30, Niphlod wrote: > > > > >> Just a doubt, >> >> " ok. the scheduler from the console runs right. Now you have to test if >> queuing a task from web2py gets the task processed. " >> >> Since I am having a load balancer, and the web2py folder is in another >> server, when running directly, I believe I will have to queue the task from >> the model file itself. Am I correct (as I wont be getting the controller of >> the app browser) >> >> you can't execute any web2py activity in any server that doesn't have > web2py code. The load balancer is NOT the place to open a console or to > configure an upstart job. It just load-balance all requests and splits them > on one server or the other. That is what load balancers are for. when you > reach the loadbalancer address the loadbalancer redirects the requests to > the backends, so it is on those servers that the activity takes place. > > >> >> " Then you can do the same thing with the scheduler started by upstart, >> and verify that it's working " >> >> Which will be the better one, using the command >> exec python -- /srv/trustvouch-fe/web2py.py --nogui -a -p 8000 -i >> 0.0.0.0 -K app -X >> >> or having two upstart jobs (and I am not sure if these are the commands >> to start, can you confirm it also) >> one to run the app >> exec python -- /srv/trustvouch-fe/web2py.py --nogui -a -p 8000 -i >> 0.0.0.0 >> >> and one to start the scheduler >> exec python -- /srv/trustvouch-fe/web2py.py -K app >> >> >> > definitely the second one, for "separation of duties". > > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
> Just a doubt, > > " ok. the scheduler from the console runs right. Now you have to test if > queuing a task from web2py gets the task processed. " > > Since I am having a load balancer, and the web2py folder is in another > server, when running directly, I believe I will have to queue the task from > the model file itself. Am I correct (as I wont be getting the controller of > the app browser) > > you can't execute any web2py activity in any server that doesn't have web2py code. The load balancer is NOT the place to open a console or to configure an upstart job. It just load-balance all requests and splits them on one server or the other. That is what load balancers are for. when you reach the loadbalancer address the loadbalancer redirects the requests to the backends, so it is on those servers that the activity takes place. > > " Then you can do the same thing with the scheduler started by upstart, > and verify that it's working " > > Which will be the better one, using the command > exec python -- /srv/trustvouch-fe/web2py.py --nogui -a -p 8000 -i > 0.0.0.0 -K app -X > > or having two upstart jobs (and I am not sure if these are the commands to > start, can you confirm it also) > one to run the app > exec python -- /srv/trustvouch-fe/web2py.py --nogui -a -p 8000 -i > 0.0.0.0 > > and one to start the scheduler > exec python -- /srv/trustvouch-fe/web2py.py -K app > > > definitely the second one, for "separation of duties". -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
Thanks for the support. Yeah I will workout the steps you said one by one and breakdown the problem. Just a doubt, " ok. the scheduler from the console runs right. Now you have to test if queuing a task from web2py gets the task processed. " Since I am having a load balancer, and the web2py folder is in another server, when running directly, I believe I will have to queue the task from the model file itself. Am I correct (as I wont be getting the controller of the app browser) " Then you can do the same thing with the scheduler started by upstart, and verify that it's working " Which will be the better one, using the command exec python -- /srv/trustvouch-fe/web2py.py --nogui -a -p 8000 -i 0.0.0.0 -K app -X or having two upstart jobs (and I am not sure if these are the commands to start, can you confirm it also) one to run the app exec python -- /srv/trustvouch-fe/web2py.py --nogui -a -p 8000 -i 0.0.0.0 and one to start the scheduler exec python -- /srv/trustvouch-fe/web2py.py -K app On Wednesday, 11 September 2013 20:35:25 UTC+5:30, Niphlod wrote: > > ok. the scheduler from the console runs right. Now you have to test if > queuing a task from web2py gets the task processed. Then you can do the > same thing with the scheduler started by upstart, and verify that it's > working. > Next is to verify that you can queue tasks from the web2py web server > started by upstart. > Next to verify that the task queued gets processed by the scheduler in the > console > Next to verify that it gets processed by the scheduler in upstart > > Something along the way should reveal where the breakage that you > experiment using both the webserver and the scheduler in upstart ^_^ > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
so, the scheduler is working pretty fine: it just doesn't find any task to process. PS: you are on a really old release (2.2.1). That's almost a year ago! On Wednesday, September 11, 2013 4:23:07 PM UTC+2, ajith c t wrote: > > ubuntu@ip-10-0-0-13:/srv/trustvouch-fe$ sudo python web2py.py -K app -D 0 > web2py Web Framework > Created by Massimo Di Pierro, Copyright 2007-2013 > Version 2.2.1 (2012-10-21 16:57:04) stable > Database drivers available: SQLite(sqlite3), MySQL(pymysql), > PostgreSQL(pg8000), IMAP(imaplib) > starting single-scheduler for "app"... > /usr/local/lib/python2.7/dist-packages/riak-2.0.1.post5-py2.7.egg/riak/util.py:148: > > UserWarning: ``solr`` is deprecated, use ``fulltext_search``, > ``fulltext_add`` and ``fulltext_delete`` directly > value = self.fget(obj) > > > > > 2013-09-11 14:21:20,236 - DEBUG - web2py.scheduler - defining tables > (migrate=True) > 2013-09-11 14:21:20,242 - DEBUG - web2py.scheduler - looping... > 2013-09-11 14:21:20,246 - DEBUG - web2py.scheduler - nothing to do > 2013-09-11 14:21:20,246 - DEBUG - web2py.scheduler - sleeping... > 2013-09-11 14:21:20,246 - DEBUG - web2py.scheduler - thread building own > DAL object > 2013-09-11 14:21:20,249 - DEBUG - web2py.scheduler - defining tables > (migrate=False) > 2013-09-11 14:21:20,254 - DEBUG - web2py.scheduler - freeing workers > that have not sent heartbeat > 2013-09-11 14:21:23,249 - DEBUG - web2py.scheduler - looping... > 2013-09-11 14:21:23,254 - DEBUG - web2py.scheduler - nothing to do > 2013-09-11 14:21:23,254 - DEBUG - web2py.scheduler - sleeping... > 2013-09-11 14:21:23,263 - DEBUG - web2py.scheduler - recording > heartbeat > 2013-09-11 14:21:26,257 - DEBUG - web2py.scheduler - looping... > 2013-09-11 14:21:26,261 - DEBUG - web2py.scheduler - nothing to do > 2013-09-11 14:21:26,262 - DEBUG - web2py.scheduler - sleeping... > 2013-09-11 14:21:26,270 - DEBUG - web2py.scheduler - recording > heartbeat > 2013-09-11 14:21:29,265 - DEBUG - web2py.scheduler - looping... > 2013-09-11 14:21:29,269 - DEBUG - web2py.scheduler - nothing to do > 2013-09-11 14:21:29,269 - DEBUG - web2py.scheduler - sleeping... > 2013-09-11 14:21:29,277 - DEBUG - web2py.scheduler - recording > heartbeat > > > above the out put of the log, > > On Wednesday, 11 September 2013 19:48:06 UTC+5:30, Niphlod wrote: >> >> let's solve an issue at a time. If your scheduler can't process the tasks >> that are in the queue when you launch it from the console, something is >> wrong with it and there's nothing that upstart can "fix". >> If you launch your scheduler from the console with >> web2py.py -K yourappname -D 0 >> what is the result ? >> > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
let's solve an issue at a time. If your scheduler can't process the tasks that are in the queue when you launch it from the console, something is wrong with it and there's nothing that upstart can "fix". If you launch your scheduler from the console with web2py.py -K yourappname -D 0 what is the result ? -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
ubuntu@ip-10-0-0-13:/srv/trustvouch-fe$ sudo python web2py.py -K app -D 0 web2py Web Framework Created by Massimo Di Pierro, Copyright 2007-2013 Version 2.2.1 (2012-10-21 16:57:04) stable Database drivers available: SQLite(sqlite3), MySQL(pymysql), PostgreSQL(pg8000), IMAP(imaplib) starting single-scheduler for "app"... /usr/local/lib/python2.7/dist-packages/riak-2.0.1.post5-py2.7.egg/riak/util.py:148: UserWarning: ``solr`` is deprecated, use ``fulltext_search``, ``fulltext_add`` and ``fulltext_delete`` directly value = self.fget(obj) 2013-09-11 14:21:20,236 - DEBUG - web2py.scheduler - defining tables (migrate=True) 2013-09-11 14:21:20,242 - DEBUG - web2py.scheduler - looping... 2013-09-11 14:21:20,246 - DEBUG - web2py.scheduler - nothing to do 2013-09-11 14:21:20,246 - DEBUG - web2py.scheduler - sleeping... 2013-09-11 14:21:20,246 - DEBUG - web2py.scheduler - thread building own DAL object 2013-09-11 14:21:20,249 - DEBUG - web2py.scheduler - defining tables (migrate=False) 2013-09-11 14:21:20,254 - DEBUG - web2py.scheduler - freeing workers that have not sent heartbeat 2013-09-11 14:21:23,249 - DEBUG - web2py.scheduler - looping... 2013-09-11 14:21:23,254 - DEBUG - web2py.scheduler - nothing to do 2013-09-11 14:21:23,254 - DEBUG - web2py.scheduler - sleeping... 2013-09-11 14:21:23,263 - DEBUG - web2py.scheduler - recording heartbeat 2013-09-11 14:21:26,257 - DEBUG - web2py.scheduler - looping... 2013-09-11 14:21:26,261 - DEBUG - web2py.scheduler - nothing to do 2013-09-11 14:21:26,262 - DEBUG - web2py.scheduler - sleeping... 2013-09-11 14:21:26,270 - DEBUG - web2py.scheduler - recording heartbeat 2013-09-11 14:21:29,265 - DEBUG - web2py.scheduler - looping... 2013-09-11 14:21:29,269 - DEBUG - web2py.scheduler - nothing to do 2013-09-11 14:21:29,269 - DEBUG - web2py.scheduler - sleeping... 2013-09-11 14:21:29,277 - DEBUG - web2py.scheduler - recording heartbeat above the out put of the log, On Wednesday, 11 September 2013 19:48:06 UTC+5:30, Niphlod wrote: > > let's solve an issue at a time. If your scheduler can't process the tasks > that are in the queue when you launch it from the console, something is > wrong with it and there's nothing that upstart can "fix". > If you launch your scheduler from the console with > web2py.py -K yourappname -D 0 > what is the result ? > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
When I start directly from the web2py folder, my load balancer gives me a bad gateway error. When I run the web2py shell , I see the scheduler tables are created. ubuntu@ip-10-0-0-13:/srv/tv-fe$ sudo python web2py.py -K app web2py Web Framework Created by Massimo Di Pierro, Copyright 2007-2013 Version 2.2.1 (2012-10-21 16:57:04) stable Database drivers available: SQLite(sqlite3), MySQL(pymysql), PostgreSQL(pg8000), IMAP(imaplib) starting single-scheduler for "app"... /usr/local/lib/python2.7/dist-packages/riak-2.0.1.post5-py2.7.egg/riak/util.py:148: UserWarning: ``solr`` is deprecated, use ``fulltext_search``, ``fulltext_add`` and ``fulltext_delete`` directly value = self.fget(obj) The scheduler_worker has record But other tables are empty as expected for me. My apps workflow is as follows, I dont understand if that is creating the problem, I have a load balancer at a public ip, which has the nginx that acts as a reverse proxy for the frontend server. The web2py folder is in this server and the models folder contains a scheduler.py file in which the tasks are defined and the scheduler class is called. Now in the controller default.py, index function calls the scheduler.queue_task. Since I started it from the web2py folder and it resulted in a bad gateway, the index function could not be called so the tasks is not scheduled. But when I am using with upstart, there is no bad gateway error, everything goes fine except that the tasks in the scheduler_task is not assigned a worker. On Wednesday, 11 September 2013 19:03:19 UTC+5:30, Niphlod wrote: > > and when you start it without using upstart it works or not ? > > On Wednesday, September 11, 2013 2:22:32 PM UTC+2, ajith c t wrote: >> >> Sorry missed the issue, the issue is the task just doesnt get called up. >> I believe the scheduler_task will have the worker that picked up the task >> in the assigned_worker_name field. In my case it remains empty >> >> On Wednesday, 11 September 2013 17:45:11 UTC+5:30, Niphlod wrote: >>> >>> what you posted doesn't indicate any issues. The task is there and the >>> worker too. >>> What are you experiencing instead ? >>> >>> -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
and when you start it without using upstart it works or not ? On Wednesday, September 11, 2013 2:22:32 PM UTC+2, ajith c t wrote: > > Sorry missed the issue, the issue is the task just doesnt get called up. I > believe the scheduler_task will have the worker that picked up the task in > the assigned_worker_name field. In my case it remains empty > > On Wednesday, 11 September 2013 17:45:11 UTC+5:30, Niphlod wrote: >> >> what you posted doesn't indicate any issues. The task is there and the >> worker too. >> What are you experiencing instead ? >> >> -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
Sorry missed the issue, the issue is the task just doesnt get called up. I believe the scheduler_task will have the worker that picked up the task in the assigned_worker_name field. In my case it remains empty On Wednesday, 11 September 2013 17:45:11 UTC+5:30, Niphlod wrote: > > what you posted doesn't indicate any issues. The task is there and the > worker too. > What are you experiencing instead ? > > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
what you posted doesn't indicate any issues. The task is there and the worker too. What are you experiencing instead ? -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
Hi Niphlod, I changed my .conf file a littile by reading the web2py online book, Created a new web2py-scheduler.conf file start on runlevel [2345] stop on runlevel [016] respawn limit 8 60 script exec sudo -u root python /srv/tv-fe/web2py.py -K app end script respawn and another conf file tv-fe.conf start on runlevel [2345] # When to stop the service stop on runlevel [016] # Automatically restart process if crashed respawn respawn limit 5 60 # Specify resource limits # Make sure you have set the system limits accordingly limit nofile 65536 65536 # Modify scheduling priority, ranges from -20 (most favorable) to 19 (least # favorable) nice -10 # Out Of Memory (OOM) killing score, ranges from -999 (very unlikely to be # killed by OOM killer) to 1000 (very likely) or 'never' oom score -555 # Essentially lets upstart know the process will detach itself to the background expect fork expect daemon # Run before process pre-start script [ -d /var/log/tv/frontend ] || mkdir -p /var/log/tv/frontend end script # Start the process script exec python -- /srv/tv-fe/web2py.py --nogui -a '' -p 8000 -i 0.0.0.0 & end script The log from the upstart web2py-scheduler.log is web2py Web Framework Created by Massimo Di Pierro, Copyright 2007-2013 Version 2.2.1 (2012-10-21 16:57:04) stable Database drivers available: SQLite(sqlite3), MySQL(pymysql), PostgreSQL(pg8000), IMAP(imaplib) starting single-scheduler for "app"... /usr/local/lib/python2.7/dist-packages/riak-2.0.1.post5-py2.7.egg/riak/util.py:148: UserWarning: ``solr`` is deprecated, use ``fulltext_search``, ``fulltext_add`` and ``fulltext_delete`` directly value = self.fget(obj) the tv-fe.log is ubuntu@ip-XX-X-X-XX:/var/log/upstart$ sudo tail -f tv-fe.log value = self.fget(obj) web2py Web Framework Created by Massimo Di Pierro, Copyright 2007-2013 Version 2.2.1 (2012-10-21 16:57:04) stable Database drivers available: SQLite(sqlite3), MySQL(pymysql), PostgreSQL(pg8000), IMAP(imaplib) 2013-09-11 11:38:54,936 - WARNING - web2py - GUI not available because Tk library is not installed no password, no admin interface please visit: http://0.0.0.0:8000 use "kill -SIGTERM 11998" to shutdown the web2py server /usr/local/lib/python2.7/dist-packages/riak-2.0.1.post5-py2.7.egg/riak/util.py:148: UserWarning: ``solr`` is deprecated, use ``fulltext_search``, ``fulltext_add`` and ``fulltext_delete`` directly value = self.fget(obj) Entry in the scheduler worker table >>> workers = db(db.scheduler_worker).select() >>> for each in workers: ... print each ... Entry in the scheduler_task table am I doing it the correct order, if yes then why doesnt the worker pickup the task. I am not sure whether the conf files I have created is right. I run the services as sudo service tv-fe start sudo service web2py-scheduler start On Tuesday, 10 September 2013 23:34:56 UTC+5:30, Niphlod wrote: > > you may be missing some bits. on production nginx acts as a proxy > between uwsgi and the webyou don't need (again, usually) web2py to > start its own webserver the -X switch is only useful if you want a > single commandline to start BOTH the scheduler and the webserver, and it's > meant to be used only in development. > > Usually you'd have uwsgi serving the "web part" and a script to run the > scheduler (hence, just web2py.py -K appname) . > BTW, upstart creates logs by default that are named after the servicename, > so something like /var/log/upstart/yourservicename.log gets created: you > may inspect that to see what is going on your server. > > On Tuesday, September 10, 2013 3:28:13 PM UTC+2, ajith c t wrote: >> >> ok i will try making two separate services. and which log are you >> checking out, I maintain my custom logging mechanism. The only thing that >> comes in it is >> >> DEBUG - web2py.scheduler - defining tables (migrate=True) >> >> the app doesnt assign a worker in the scheduler_task table. The column >> shows a NULL value. >> >> I am using nginx, when I remove the -X parameter it results in a bad >> gateway, >> >> >> >> On Tuesday, 10 September 2013 18:21:36 UTC+5:30, Niphlod wrote: >>> >>> why don't you schedule two separate services (the webserver and the >>> scheduler), so the issues will be easier to track down ? what's in the logs >>> ? >>> >>> On Tuesday, September 10, 2013 1:42:11 PM UTC+2, ajith c t wrote: Hi All, I have a scheduler in my app in the models folder with some specific task. I need to run the app as a service in the server. I cant figure out how to run start the web2py with the arguments. script exec python -- /srv/tv-fe/web2py.py -K app -X --nogui -p 8000 -i 0.0.0.0 -a '' & end script The above script is used to run the app. Upon running it , it doesnt gen
[web2py] Re: Scheduler as upstart job
Few suggestions... 1. Try running the job in the foreground. Upstart docs. recommend this wherever its an option.. If you do run the job in the background, be sure you have an "expect fork" or "expect daemon" stanza so Upstart can follow the process. 2. When running in the foreground, if you have a "console log" stanza all of the console output will go to /var/log/upstart/-jobname-.log... this can be helpful. 3. You don't have any environment variables unless you set them with "env VAR=x" etc this includes HOME and PATH etc. It seems rather counterintuitive to not run your daemon as a daemon... but it solved a lot of the issues I was having with jobs that just wouldn't start. Hope it helps. On Tuesday, September 10, 2013 6:28:13 AM UTC-7, ajith c t wrote: > > ok i will try making two separate services. and which log are you checking > out, I maintain my custom logging mechanism. The only thing that comes in > it is > > DEBUG - web2py.scheduler - defining tables (migrate=True) > > the app doesnt assign a worker in the scheduler_task table. The column > shows a NULL value. > > I am using nginx, when I remove the -X parameter it results in a bad > gateway, > > > > On Tuesday, 10 September 2013 18:21:36 UTC+5:30, Niphlod wrote: >> >> why don't you schedule two separate services (the webserver and the >> scheduler), so the issues will be easier to track down ? what's in the logs >> ? >> >> On Tuesday, September 10, 2013 1:42:11 PM UTC+2, ajith c t wrote: >>> >>> Hi All, >>> >>> I have a scheduler in my app in the models folder with some >>> specific task. I need to run the app as a service in the server. I cant >>> figure out how to run start the web2py with the arguments. >>> >>> >>> script >>> exec python -- /srv/tv-fe/web2py.py -K app -X --nogui -p 8000 -i >>> 0.0.0.0 -a '' & >>> end script >>> >>> The above script is used to run the app. Upon running it , it doesnt >>> generate error, but the worker is not created in the scheduler_worker >>> table. So none of my task gets executed. >>> >>> Where am I getting wrong with the parameters >>> >>> Thanks and regards >>> Ajith >>> >>> -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
you may be missing some bits. on production nginx acts as a proxy between uwsgi and the webyou don't need (again, usually) web2py to start its own webserver the -X switch is only useful if you want a single commandline to start BOTH the scheduler and the webserver, and it's meant to be used only in development. Usually you'd have uwsgi serving the "web part" and a script to run the scheduler (hence, just web2py.py -K appname) . BTW, upstart creates logs by default that are named after the servicename, so something like /var/log/upstart/yourservicename.log gets created: you may inspect that to see what is going on your server. On Tuesday, September 10, 2013 3:28:13 PM UTC+2, ajith c t wrote: > > ok i will try making two separate services. and which log are you checking > out, I maintain my custom logging mechanism. The only thing that comes in > it is > > DEBUG - web2py.scheduler - defining tables (migrate=True) > > the app doesnt assign a worker in the scheduler_task table. The column > shows a NULL value. > > I am using nginx, when I remove the -X parameter it results in a bad > gateway, > > > > On Tuesday, 10 September 2013 18:21:36 UTC+5:30, Niphlod wrote: >> >> why don't you schedule two separate services (the webserver and the >> scheduler), so the issues will be easier to track down ? what's in the logs >> ? >> >> On Tuesday, September 10, 2013 1:42:11 PM UTC+2, ajith c t wrote: >>> >>> Hi All, >>> >>> I have a scheduler in my app in the models folder with some >>> specific task. I need to run the app as a service in the server. I cant >>> figure out how to run start the web2py with the arguments. >>> >>> >>> script >>> exec python -- /srv/tv-fe/web2py.py -K app -X --nogui -p 8000 -i >>> 0.0.0.0 -a '' & >>> end script >>> >>> The above script is used to run the app. Upon running it , it doesnt >>> generate error, but the worker is not created in the scheduler_worker >>> table. So none of my task gets executed. >>> >>> Where am I getting wrong with the parameters >>> >>> Thanks and regards >>> Ajith >>> >>> -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
ok i will try making two separate services. and which log are you checking out, I maintain my custom logging mechanism. The only thing that comes in it is DEBUG - web2py.scheduler - defining tables (migrate=True) the app doesnt assign a worker in the scheduler_task table. The column shows a NULL value. I am using nginx, when I remove the -X parameter it results in a bad gateway, On Tuesday, 10 September 2013 18:21:36 UTC+5:30, Niphlod wrote: > > why don't you schedule two separate services (the webserver and the > scheduler), so the issues will be easier to track down ? what's in the logs > ? > > On Tuesday, September 10, 2013 1:42:11 PM UTC+2, ajith c t wrote: >> >> Hi All, >> >> I have a scheduler in my app in the models folder with some >> specific task. I need to run the app as a service in the server. I cant >> figure out how to run start the web2py with the arguments. >> >> >> script >> exec python -- /srv/tv-fe/web2py.py -K app -X --nogui -p 8000 -i >> 0.0.0.0 -a '' & >> end script >> >> The above script is used to run the app. Upon running it , it doesnt >> generate error, but the worker is not created in the scheduler_worker >> table. So none of my task gets executed. >> >> Where am I getting wrong with the parameters >> >> Thanks and regards >> Ajith >> >> -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler as upstart job
why don't you schedule two separate services (the webserver and the scheduler), so the issues will be easier to track down ? what's in the logs ? On Tuesday, September 10, 2013 1:42:11 PM UTC+2, ajith c t wrote: > > Hi All, > > I have a scheduler in my app in the models folder with some specific > task. I need to run the app as a service in the server. I cant figure out > how to run start the web2py with the arguments. > > > script > exec python -- /srv/tv-fe/web2py.py -K app -X --nogui -p 8000 -i > 0.0.0.0 -a '' & > end script > > The above script is used to run the app. Upon running it , it doesnt > generate error, but the worker is not created in the scheduler_worker > table. So none of my task gets executed. > > Where am I getting wrong with the parameters > > Thanks and regards > Ajith > > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
