Re: [web2py] Re: Pattern to run async proccess
I've been studying the scheduler, but I think it fails in one of the things I need: The web page must be updated as the process is being done i.e. if the scheduler is doiing some background task, I need to update the web page with the progress of the task, but I don't find how to do it with the scheduler 2013/2/1 Massimo Di Pierro massimo.dipie...@gmail.com: yes. If the worker is not busy it starts the task immediately. You can also have more than one worker. On Friday, 1 February 2013 11:10:11 UTC-6, José Luis Redrejo Rodríguez wrote: Thanks for your advice Massimo, but does the scheduler start inmediately when no worker has been used before? 2013/2/1 Massimo Di Pierro massimo@gmail.com: All that you ask can be done using the scheduler except that your app does not start the process, but submits a request to the scheduler. The scheduler runs the app when a worker is available. This is to prevent spikes in resource utilization when multiple processes start. The task can communicate with the app vid database and/or filesystem (which is ok but not 100% satisfactory). Web2py can monitor and kill running scheduler tasks. This works well for most types of tasks but not for tasks that need a lot of IO with your application. I do not have a satisfactory solution in that case. You want the tasks to have some way to communicate asynchronously with the client and this present major issues, some related with security. On Friday, 1 February 2013 10:22:35 UTC-6, José Luis Redrejo Rodríguez wrote: Hi, This is a question that has been asked several times in the list, and I have also had to implement this kind of app in the past. Now I'm also facing to another application where I need to run a resource_and_time_consuming process managed from web2py. The exact problem is: - From a web page, a long process must be started - The web page must be updated as the process is being done - The web page must be able to cancel the process. In the past I have had to deal with the fact of sessions lockings: web2py server doesn't react while the process is being executed. I've solved this by using session.forget(response), but this solution avoids the use of session variables to update the process in the original web page. I've used background processes, queues, etc, These solutions work when time is not an issue, but not when the synchronization between the process and the webpage must be fast and accurate I wonder if someone has a definitive pattern to do this kind of action. Regards José L. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: Pattern to run async proccess
you must save somewhere in your long_time_consuming_function() the progress and let the page show that. If you manage that function outside web2py (using the scheduler or not), you need something readable by your page and writeable by the external process. A record on the db, a file, something. The scheduler has a facility you can use for that http://web2py.com/books/default/chapter/29/04#Scheduler-%28experimental%29 under the section reporting percentages . if you want to get familiar with the scheduler you can look at this learn by trial application https://github.com/niphlod/w2p_scheduler_tests let me know if you need further assistance on the scheduler's side. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: Pattern to run async proccess
Niphlod Thanks very much , your learn by trial application has been gold for me. Catched!! 2013/2/2 Niphlod niph...@gmail.com: you must save somewhere in your long_time_consuming_function() the progress and let the page show that. If you manage that function outside web2py (using the scheduler or not), you need something readable by your page and writeable by the external process. A record on the db, a file, something. The scheduler has a facility you can use for that http://web2py.com/books/default/chapter/29/04#Scheduler-%28experimental%29 under the section reporting percentages . if you want to get familiar with the scheduler you can look at this learn by trial application https://github.com/niphlod/w2p_scheduler_tests let me know if you need further assistance on the scheduler's side. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: Pattern to run async proccess
You can also use Redis Queue http://rochacbruno.com.br/web2py-and-redis-queue/ -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Pattern to run async proccess
All that you ask can be done using the scheduler except that your app does not start the process, but submits a request to the scheduler. The scheduler runs the app when a worker is available. This is to prevent spikes in resource utilization when multiple processes start. The task can communicate with the app vid database and/or filesystem (which is ok but not 100% satisfactory). Web2py can monitor and kill running scheduler tasks. This works well for most types of tasks but not for tasks that need a lot of IO with your application. I do not have a satisfactory solution in that case. You want the tasks to have some way to communicate asynchronously with the client and this present major issues, some related with security. On Friday, 1 February 2013 10:22:35 UTC-6, José Luis Redrejo Rodríguez wrote: Hi, This is a question that has been asked several times in the list, and I have also had to implement this kind of app in the past. Now I'm also facing to another application where I need to run a resource_and_time_consuming process managed from web2py. The exact problem is: - From a web page, a long process must be started - The web page must be updated as the process is being done - The web page must be able to cancel the process. In the past I have had to deal with the fact of sessions lockings: web2py server doesn't react while the process is being executed. I've solved this by using session.forget(response), but this solution avoids the use of session variables to update the process in the original web page. I've used background processes, queues, etc, These solutions work when time is not an issue, but not when the synchronization between the process and the webpage must be fast and accurate I wonder if someone has a definitive pattern to do this kind of action. Regards José L. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: Pattern to run async proccess
Thanks for your advice Massimo, but does the scheduler start inmediately when no worker has been used before? 2013/2/1 Massimo Di Pierro massimo.dipie...@gmail.com: All that you ask can be done using the scheduler except that your app does not start the process, but submits a request to the scheduler. The scheduler runs the app when a worker is available. This is to prevent spikes in resource utilization when multiple processes start. The task can communicate with the app vid database and/or filesystem (which is ok but not 100% satisfactory). Web2py can monitor and kill running scheduler tasks. This works well for most types of tasks but not for tasks that need a lot of IO with your application. I do not have a satisfactory solution in that case. You want the tasks to have some way to communicate asynchronously with the client and this present major issues, some related with security. On Friday, 1 February 2013 10:22:35 UTC-6, José Luis Redrejo Rodríguez wrote: Hi, This is a question that has been asked several times in the list, and I have also had to implement this kind of app in the past. Now I'm also facing to another application where I need to run a resource_and_time_consuming process managed from web2py. The exact problem is: - From a web page, a long process must be started - The web page must be updated as the process is being done - The web page must be able to cancel the process. In the past I have had to deal with the fact of sessions lockings: web2py server doesn't react while the process is being executed. I've solved this by using session.forget(response), but this solution avoids the use of session variables to update the process in the original web page. I've used background processes, queues, etc, These solutions work when time is not an issue, but not when the synchronization between the process and the webpage must be fast and accurate I wonder if someone has a definitive pattern to do this kind of action. Regards José L. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: Pattern to run async proccess
yes. If the worker is not busy it starts the task immediately. You can also have more than one worker. On Friday, 1 February 2013 11:10:11 UTC-6, José Luis Redrejo Rodríguez wrote: Thanks for your advice Massimo, but does the scheduler start inmediately when no worker has been used before? 2013/2/1 Massimo Di Pierro massimo@gmail.com javascript:: All that you ask can be done using the scheduler except that your app does not start the process, but submits a request to the scheduler. The scheduler runs the app when a worker is available. This is to prevent spikes in resource utilization when multiple processes start. The task can communicate with the app vid database and/or filesystem (which is ok but not 100% satisfactory). Web2py can monitor and kill running scheduler tasks. This works well for most types of tasks but not for tasks that need a lot of IO with your application. I do not have a satisfactory solution in that case. You want the tasks to have some way to communicate asynchronously with the client and this present major issues, some related with security. On Friday, 1 February 2013 10:22:35 UTC-6, José Luis Redrejo Rodríguez wrote: Hi, This is a question that has been asked several times in the list, and I have also had to implement this kind of app in the past. Now I'm also facing to another application where I need to run a resource_and_time_consuming process managed from web2py. The exact problem is: - From a web page, a long process must be started - The web page must be updated as the process is being done - The web page must be able to cancel the process. In the past I have had to deal with the fact of sessions lockings: web2py server doesn't react while the process is being executed. I've solved this by using session.forget(response), but this solution avoids the use of session variables to update the process in the original web page. I've used background processes, queues, etc, These solutions work when time is not an issue, but not when the synchronization between the process and the webpage must be fast and accurate I wonder if someone has a definitive pattern to do this kind of action. Regards José L. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com javascript:. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
