[web2py] Re: Scheduler Replacement

[Boris Aramis Aguilar Rodríguez]

Thanks for your answer!, I'm a poor worker. lol.

Really, really thanks, I'm tempted to take the road of doing some 
"interface" code to emulate the same functionality the scheduler currently 
does but using mrq.io library/framework as a backend... that to avoid 
changing the current code within the project, but I'm currently trying to 
see if it is feasible, and how much time that would take. 

Thanks again Niphlod for the hints and info; it would be awesome if the 
scheduler limits where documented somewhere (I would do it but I've never 
done it before); anyway, thanks a lot :)

El viernes, 23 de noviembre de 2018, 2:20:59 (UTC-6), Niphlod escribió:
>
> jokes aside, yeah, you definitely hit scheduler's limits, or, for better 
> saying, limits using a relational database as a queue table.
> web2py's scheduler can still be optimized, and I feel that 30k tasks are 
> manageable, if they are spread throughout the day (20 tasks a minute if 
> math is not failing me)
> Managing 15-20 workers is not a problem with a good database backend, 
> 30ish or more is asking for disasters to happen, and that's why 
> redis_scheduler was born.
> The redis backed one moves to redis the heavy concurrency part, which is 
> the scheduler_worker table and the polling algo for looking for new tasks, 
> but it doesn't move EVERYTHING out of the database.
> 60-80 workers is probably the limit for redis_backed.
> Both of them are prolly less than 2k lines of code and they use DAL and 
> redis, but nothing else from the standard library so they're definitely 
> good, but not on par with "specialized" alternatives.
>
> One "grin" of the implementation is that it forks a new process every 
> task, and that's for a basic design principle others do not enforce which 
> is to terminate a long running process going into timeout (and that's 
> because python can't kill a thread, just a process). But that's a problem 
> for people running 40 tasks per second, not 20 a minute.
>
> Things to look out for when you reach high numbers like you (i.e. you can 
> see if it helps "staying with the scheduler"):
> - trim the scheduler_task regularly (having 1m rows in the scheduler_task 
> table, if you do not need them for reporting, can definitely bring down 
> performances)
> - same deal for scheduler_run (do you need the result, and if you need it, 
> do you need for how much time?!)
> - create indexes on it-s- the scheduler_task table (one on status, 
> assigned_worker_name helps the workers,  on stop_time, next_run_time and 
> status helps the ticker)
>
> took that out of the table, you can quite easily jump on a different ship 
> (my advice would be to look at rq before celery, but YMMV) which are HUGE 
> projects.
>
> The only problem you may have is that if you use the same backend to store 
> results for your tasks (because they're needed)  you *may* hit the same 
> limits, i.e. you may have your backend not sized up for the work your apps 
> need to do.
>
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: common filter error on many to many table

[icodk]

PROBLEM SOLVED
I might be wrong but common filter can be an expression that refers ONLY to 
the filtered   table so it is not a general  DAL query.
(please correct me if I am wrong).
Solution:
Create a function that returns an expression  with only the filtered table 
and assign this function name to the common filter
The function:
def filter_person(query):

avail_groups=db((db.person_group.id==auth_user_person_group.person_group)&( 
db.auth_user_person_group.auth_user==auth.user.id))._select(db.person_group.id)
return db.person.person_group.belongs(avail_groups)


Now assign it to the common filter:


db.person._common_filter = filter_person





On Wednesday, November 28, 2018 at 5:54:31 PM UTC+1, icodk wrote:
>
> Getting the following error when setting a common filter on a table:
>
> *Query Not Supported: current transaction is aborted, commands ignored 
>> until end of transaction block*
>
>
> Looking in the database log  i can see an error on a missing table name in 
> a query:
>
> *ERROR:  missing FROM-clause entry for table "auth_user_person_group" at 
>> character ...*
>
>
> Table definitions:
> db.define_table('person_group',
> Field('name'))
>
> db.define_table('person',
> Field('name'),
> Field('group', 'reference person_group'),
>
>
>  The following table is the standard auth_user table and is not part of 
> the issue. Mentioned here  just for completeness
> db.define_table('auth_user',
> ...
> )
>
> The following table is a many to many that connects personal groups to 
> auth_user:
> db.define_table('auth_user_person_group',
> Field('auth_user', 'reference auth_user'),
> Field('group', 'reference person_group'),
> )
>
> ..And this is the common filter I try to use (that gives the above 
> mentioned error)
> It's propose is to limit the persons from the person table  to those 
> belongs to groups that the current auth.user is connected through the many 
> to m many table above (auth_user_person_group)
> db.person._common_filter = lambda query: (db.person.person_group == 
> db.auth_user_person_group.person_group)& 
> (db.auth_user_person_group.auth_user == auth.user.id)
>
> The query that reaches the database is missing the auth_user_person_group 
> table in the FROM part
>
> Help will be greatly appreciated
>
>
>
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: AWS Lambda Deployment Recipe

[appjarbiz]

Thank you Anthony, I added the wsgibase wrapper after my initial post.  I 
was modifying gluon/main.py but I added the wrapper (lambda_handler.py) to 
avoid that.

I confirmed which environ variables are coming through as unicode and 
submitted an issue to zappa.
https://github.com/Miserlou/Zappa/issues/1713

I'll also modify the recipe to make it more clear that this is likely a 
zappa and not a web2py issue.

On Wednesday, November 28, 2018 at 8:39:51 AM UTC-6, Anthony wrote:
>
> On Tuesday, November 27, 2018 at 6:51:54 PM UTC-5, appj...@gmail.com 
>  wrote:
>>
>> I've created a first pass at an AWS Lambda deployment recipe which uses 
>> an open-source tool called Zappa.
>>
>> I welcome any comments or suggested improvements.  In particular, step 3 
>> requires modifying gluon/main.py which I would like to avoid.
>>
>
> I wouldn't say that is modifying gluon/main.py -- it is simply providing a 
> wrapper around gluon.main.wsgibase. In fact, the web2py gaehandler.py 
> handler does something very similar: 
> https://github.com/web2py/web2py/blob/master/handlers/gaehandler.py#L83.
>
> Note, according to PEP , strings passed to the server should not be 
> unicode: https://www.python.org/dev/peps/pep-/#unicode-issues.
>
> Actually, it looks like Zappa is already handling this here: 
> https://github.com/Miserlou/Zappa/blob/master/zappa/wsgi.py#L110 (see 
> also, https://github.com/Miserlou/Zappa/blob/master/zappa/wsgi.py#L110). 
> Perhaps there is still some unicode getting through from Zappa, so maybe 
> you can open an issue with Zappa.
>
> Anthony
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: common filter error on many to many table

[icodk]

Correction to the common filter 
It is  :
db.person._common_filter = lambda query: (db.person.group == db.
auth_user_person_group.person_group)& (db.auth_user_person_group.auth_user 
== auth.user.id)

This error is only in the issue description here. So I still get the error 
in my code


On Wednesday, November 28, 2018 at 5:54:31 PM UTC+1, icodk wrote:
>
> Getting the following error when setting a common filter on a table:
>
> *Query Not Supported: current transaction is aborted, commands ignored 
>> until end of transaction block*
>
>
> Looking in the database log  i can see an error on a missing table name in 
> a query:
>
> *ERROR:  missing FROM-clause entry for table "auth_user_person_group" at 
>> character ...*
>
>
> Table definitions:
> db.define_table('person_group',
> Field('name'))
>
> db.define_table('person',
> Field('name'),
> Field('group', 'reference person_group'),
>
>
>  The following table is the standard auth_user table and is not part of 
> the issue. Mentioned here  just for completeness
> db.define_table('auth_user',
> ...
> )
>
> The following table is a many to many that connects personal groups to 
> auth_user:
> db.define_table('auth_user_person_group',
> Field('auth_user', 'reference auth_user'),
> Field('group', 'reference person_group'),
> )
>
> ..And this is the common filter I try to use (that gives the above 
> mentioned error)
> It's propose is to limit the persons from the person table  to those 
> belongs to groups that the current auth.user is connected through the many 
> to m many table above (auth_user_person_group)
> db.person._common_filter = lambda query: (db.person.person_group == 
> db.auth_user_person_group.person_group)& 
> (db.auth_user_person_group.auth_user == auth.user.id)
>
> The query that reaches the database is missing the auth_user_person_group 
> table in the FROM part
>
> Help will be greatly appreciated
>
>
>
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Select_or_add

['Matthew J Watts' via web2py-users]

Hi all

Anyone using this feature with the current version of web2py. I tried to 
get the feature working using the code from the link below but it isnt 
working propery i.e. can´t close pop up box, form doesn't refesh when i 
insert a new entry. I still have to learn Jquery and javascript

https://github.com/mdipierro/web2py-recipes-source/blob/master/source/05_adding_ajax_effects/02_Creating_a_select_or_add_widget/default.py

cheers

Matt


-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] common filter error on many to many table

[icodk]

Getting the following error when setting a common filter on a table:

*Query Not Supported: current transaction is aborted, commands ignored 
> until end of transaction block*


Looking in the database log  i can see an error on a missing table name in 
a query:

*ERROR:  missing FROM-clause entry for table "auth_user_person_group" at 
> character ...*


Table definitions:
db.define_table('person_group',
Field('name'))

db.define_table('person',
Field('name'),
Field('group', 'reference person_group'),
   

 The following table is the standard auth_user table and is not part of the 
issue. Mentioned here  just for completeness
db.define_table('auth_user',
...
)

The following table is a many to many that connects personal groups to 
auth_user:
db.define_table('auth_user_person_group',
Field('auth_user', 'reference auth_user'),
Field('group', 'reference person_group'),
)

..And this is the common filter I try to use (that gives the above 
mentioned error)
It's propose is to limit the persons from the person table  to those 
belongs to groups that the current auth.user is connected through the many 
to m many table above (auth_user_person_group)
db.person._common_filter = lambda query: (db.person.person_group == 
db.auth_user_person_group.person_group)& 
(db.auth_user_person_group.auth_user == auth.user.id)

The query that reaches the database is missing the auth_user_person_group 
table in the FROM part

Help will be greatly appreciated




-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] mobile responsive table

[lbjc1978]

I'd like to have this table to fit in any media screen including ipad and  
mobile phone.
How do I do it. This is the code:
{{extend 'layout.html'}}

.btn{
border-radius: .28rem;
}
h3 { 
display: block;
font-size: 1.17em;
margin-top: 0.5em;
margin-bottom: 1em;
margin-left: 0;
margin-right: 0;
font-weight: light;
font-family: "Josefin Slab";
color:black;
}


We'd love to have data about your state. Please enter or confirm your 
details, then scroll down to submit.


#table {
position: absolute;
top: 12em;
left: 16%;
padding: 1em 3em 3em 3em;
max-width:60em;
max-height:29em;
margin-top: -9em; /*set to a negative number 1/2 of your height*/
margin-left: -15em; /*set to a negative number 1/2 of your width*/
background-color: #DFF0D8;
color:black;
font-family: "Josefin Slab";
border-radius:18px;
overflow:scroll;
}
::-webkit-scrollbar {
width: 0px;  /* remove scrollbar space */
background: transparent; 
}
/* optional: show position indicator in red */
::-webkit-scrollbar-thumb {
background: #FF;
}
table {
width: 100%;
}
tr {
text-align: left;
}


{{=form}}



-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: AWS Lambda Deployment Recipe

[Anthony]

On Tuesday, November 27, 2018 at 6:51:54 PM UTC-5, appjar...@gmail.com 
wrote:
>
> I've created a first pass at an AWS Lambda deployment recipe which uses an 
> open-source tool called Zappa.
>
> I welcome any comments or suggested improvements.  In particular, step 3 
> requires modifying gluon/main.py which I would like to avoid.
>

I wouldn't say that is modifying gluon/main.py -- it is simply providing a 
wrapper around gluon.main.wsgibase. In fact, the web2py gaehandler.py 
handler does something very 
similar: 
https://github.com/web2py/web2py/blob/master/handlers/gaehandler.py#L83.

Note, according to PEP , strings passed to the server should not be 
unicode: https://www.python.org/dev/peps/pep-/#unicode-issues.

Actually, it looks like Zappa is already handling this 
here: https://github.com/Miserlou/Zappa/blob/master/zappa/wsgi.py#L110 (see 
also, https://github.com/Miserlou/Zappa/blob/master/zappa/wsgi.py#L110). 
Perhaps there is still some unicode getting through from Zappa, so maybe 
you can open an issue with Zappa.

Anthony

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] RuntimeError: cannot load dispatch table from pyexpat

[Yebach]

I am getting the following error after I reinstalled my machine. I am using 
Python 27 (32-bit)


Traceback (most recent call last):
  File "C:\workspaces\Python\iRazpored\gluon\restricted.py", line 217, in 
restricted
exec ccode in environment
  File 
"C:/workspaces/Python/iRazpored/applications/bilbanstevci/controllers/stevci.py"
 , 
line 7, in 
from openpyxl import Workbook
  File "C:\workspaces\Python\iRazpored\gluon\custom_import.py", line 94, in 
custom_importer
raise e2  # there is an error in the module
RuntimeError: cannot load dispatch table from pyexpat


Any suggestions

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.