Re: [web2py] Re: detect a specific device

[paolo.vall...@gmail.com]

hi, thanks everyone for the answers
@Willoughby I don't think the mac address is sent/received in an http
request, isn't ?
@dave your suggestion is in the right direction but how can you
automatically tells the browser to send this token?
@niphlod authentication is not enough, an user can still use his/her
credentials to log in in a different tablet, ip address would work but it
is too intrusive for an already present intranet.

Following @dave suggestion, I thought about a cookie that never expire, the
first time the tablet start the 'a setup app' stores the cookie on both the
client browser and in the server, however I'm not convinced this solution
would work either; the 'setup app' is not foresee at all. I've to verify if
there're ways to pre-install the cookie in the tablet?



 Paolo


2014/1/14 Niphlod 

> this is an architectural problem: what does the device send to your app to
> be able to identify it ?
> If it's a webapp and it is accessed via the browser, and there are no
> running bits on the device itself speaking to your webapp, then the headers
> and the env are the only thing you can rely onto (fortunately, browsers are
> fully isolated).
>
> Hello, authentication (or fixed ip address)!
>
>
> On Tuesday, January 14, 2014 8:05:17 PM UTC+1, Paolo Valleri wrote:
>
>> Hi,
>> what you posted can be a starting point but it is not enough because it
>> doesn't guarantee that a different android tablet has been purchased.
>> The requirement is, only a well defined set of tablets can access the
>> webapp.
>>
>>  Paolo
>>
>>
>> 2014/1/14 Leonel Câmara 
>>
>>>  I guess you can use something like:
>>>
>>> request.user_agent().is_tablet and request.user_agent().dist.name== 
>>> 'Android'
>>>
>>>
>>>  --
>>> Resources:
>>> - http://web2py.com
>>> - http://web2py.com/book (Documentation)
>>> - http://github.com/web2py/web2py (Source code)
>>> - https://code.google.com/p/web2py/issues/list (Report Issues)
>>> ---
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "web2py-users" group.
>>> To unsubscribe from this topic, visit https://groups.google.com/d/
>>> topic/web2py/3Q7izsidMXM/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to
>>> web2py+un...@googlegroups.com.
>>>
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>  --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/3Q7izsidMXM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: detect a specific device

[paolo.vall...@gmail.com]

Hi,
what you posted can be a starting point but it is not enough because it
doesn't guarantee that a different android tablet has been purchased.
The requirement is, only a well defined set of tablets can access the
webapp.

 Paolo


2014/1/14 Leonel Câmara 

> I guess you can use something like:
>
> request.user_agent().is_tablet and request.user_agent().dist.name ==
> 'Android'
>
>
>  --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/3Q7izsidMXM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Migrating a hude database from Spring to web2py

[paolo.vall...@gmail.com]

Then set the three variables Michele mentioned to false directly, it is not
necessary to create any metadata,
the following code snippet print the station name from a remote database
(vpn address), the powerful thing is that the table station has several
fields but I defined only once :)

db = DAL('postgres://web2py:web2py@10.8.0.26:5432/postgis',
migrate=False,
migrate_enabled=False,
lazy_tables=True,
pool_size=3
)

db.define_table('station',
Field('name')
)

print db(db.station).select()


 Paolo


2014/1/9 Tushar Tuteja 

> No,
> I don't want to alter the 5 tables schema.
> But I want to perform CRUD operations.
> And it would be great if somehow I can restrict web2py from altering table
> schemas all together as a J2EE application is running on the same the DB
> and DB is a remote DB.
> thanks,
> Tushar Tuteja
>
>
> On 9 January 2014 18:45, Paolo Valleri  wrote:
>
>> Mind that DAL should be instantiated with fake_migrate_all;
>> run your app once, then you can set it to false. This operation will
>> create several files (one for each defined table) in the app/databases/
>> directory
>>
>> moreover, do you need to alter the 5 tables you mentioned?
>>
>> Paolo
>>
>>
>> On Thursday, January 9, 2014 12:59:18 PM UTC+1, Michele Comitini wrote:
>>
>>> Instantiate the DAL with fake_migrate=True, migrate=False,migrate_enable=
>>> False
>>> Define the tables with only the fields you are going to need in a model
>>> file.
>>> This is going to create the metadata needed by web2py without touching
>>> your legacy database schema.
>>>
>>>
>>>
>>>
>>> 2014/1/9 Tushar Tuteja 
>>>
  I have a J2EE application and a database which has 200 tables in it.
 I want to develop a new application using web2py, which would use only
 5 tables.
 I want to know how should I go about it. As I am not clear about how I
 would use database without mapping all the 200 tables which is a huge task.
 and I don't want web2py to make any changes on the database tables and
 their schema.
 only crud operations on the records of the 5 tables.

 Thanks,
 Regards,
 Tushar Tuteja

 --
 Resources:
 - http://web2py.com
 - http://web2py.com/book (Documentation)
 - http://github.com/web2py/web2py (Source code)
 - https://code.google.com/p/web2py/issues/list (Report Issues)
 ---
 You received this message because you are subscribed to the Google
 Groups "web2py-users" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to web2py+un...@googlegroups.com.

 For more options, visit https://groups.google.com/groups/opt_out.

>>>
>>>  --
>> Resources:
>> - http://web2py.com
>> - http://web2py.com/book (Documentation)
>> - http://github.com/web2py/web2py (Source code)
>> - https://code.google.com/p/web2py/issues/list (Report Issues)
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "web2py-users" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/web2py/BPabc0lulcw/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> web2py+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>
>
> --
> Tushar Tuteja
> Undergraduate Student(Fourth Year)
> Civil Engineering
> IIT Delhi
> +919953061943
> ttut...@gmail.com 
> ce1090...@civil.iitd.ernet.in
> "The question isn't who is going to let me; its who is going to stop me!"
> - Howard Roark
>
> --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/BPabc0lulcw/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: Integrated IDE does not color javascript code

[paolo.vall...@gmail.com]

Have you upgraded web2py?
I've just double checked with web2py 2.8.2 and it works well

 Paolo


2013/12/23 António Ramos 

> I dont see it working.
> What is the status of javascript coloring?
>
> Thank you
>
>
> 2013/10/18 paolo.vall...@gmail.com 
>
>> It should be fixed in trunk. Please have a try
>>
>> Paolo
>>
>> --
>> Resources:
>> - http://web2py.com
>> - http://web2py.com/book (Documentation)
>> - http://github.com/web2py/web2py (Source code)
>> - https://code.google.com/p/web2py/issues/list (Report Issues)
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "web2py-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to web2py+unsubscr...@googlegroups.com.
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>  --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/CA2vixMUSZk/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: Can't edit default.py after upgrade to 2.8.2

[paolo.vall...@gmail.com]

No but I am not a master of mod_wsgi.
everything works but the editor, right?

 Paolo


2013/12/19 Richard Brown 

> Hi Paolo, this is a web site so presumably has to accessible from the
> outside?
>
> We should probably keep this discussion separate and concentrate on the
> main issue - is there anything from my list of files with wsgi references
> which you need to see the errors?
>
>  --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/fIuUAqdSieQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: Can't edit default.py after upgrade to 2.8.2

[paolo.vall...@gmail.com]

there are plenty how-to for improving the security of your network/rpi. I
don't know your network configuration, given that I can't tell you what to
do.
Surely you have apache accessible from 'outside' if this is not intentional
I would fix it as the first thing to do.

Moreover, I found this, maybe it could help
http://two-bulls.com/2011/02/to-solve-invalid-method-request-x16x03x01/
Finally, in your error.log line 6-34 there are several sudo messages, check
on google if you can find out more related to the rpi.



 Paolo


2013/12/18 Richard Brown 

> Hi Paolo, I think I'm getting a bit out of my depth here... I can't
> specifically find any wsgi log files, I did a  'find'for filenames
> containing 'wsgi' and the results are attached, with the command I used for
> the search at the top of the file. Please tell me what I need to send.
>
> Last night I rebooted my RPi and was able to work for a couple of hours
> before I got a 'Communication Error' when trying to revert a file and that
> was it - no more editing.
>
> Could you recommend any useful tips/links regarding internet security
> please?
>
> Thanks again,  Richard
>
>  --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/fIuUAqdSieQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: Can't edit default.py after upgrade to 2.8.2

[paolo.vall...@gmail.com]

You should locate the log of mod_wsgi, please post those as soon you find
them, from the apache log everything seems to work well (I see 304 for
cached stuff and 200 when you edit file), for example line 720-725.
Be aware that, you got a lot of requests from external ip which are sending
against your apache fake/malformed messages, see lines 497, 498, 516 and so
on

 Paolo


2013/12/17 Richard Brown 

>
> Hi Paolo, I've attached the error.log and access.log files from
> /var/log/apache2 as requested. The logs were capture after a couple of
> failed attempts to edit files at about 20:18.The entries at around 19:30
> were to the target site via my smartphone.
>
> The folder /var/log/wsgi does not exist.
>
> The AliasMatch parameter is exactly as per the script in the link.
>
> I am about to reboot the RPi again to see if I can do some actual
> development.
>
> 
>
> Leonel - I am aware of the concerns regarding db access from multiple
> threads, but my project (Heating Controller) specifically creates and
> modifies events which the Scheduler scans and acts upon so I'm not sure how
> separate databases would work. To avoid blurring the lines I was going to
> start a separate post on this topic once I am able to edit files...
> However, is there a high-level idiot's guide/explanation/diagram anywhere
> for novices like me as to how databases are stored and accessed?
>
> Richard
>
> --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/fIuUAqdSieQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: Can't edit default.py after upgrade to 2.8.2

[paolo.vall...@gmail.com]

the editor requires several files, I'm just wondering if you get all of
them, can you check if there are error in Chrome console?
Moreover, please check apache logs, if something is not working there.
have you enabled map_hyphen ?

 Paolo


2013/12/17 Richard Brown 

> Hi Paolo, as a C programming, PC-using hardware engineer bumbling around
> in Linux-land I came across this sequence of commands:
>
> wget http://web2py.googlecode.com/hg/scripts/setup-web2py-ubuntu.sh
> chmod +x setup-web2py-ubuntu.sh
> sudo ./setup-web2py-ubuntu.sh
>
> So whatever it does that is what I have installed. The webserver is
> Apache. I first ran this install a year or so ago and it was all working
> fine until I had some database locking problems when using the Scheduler so
> I tried to upgrade Web2py to see if it fixed it. However I kept getting
> tickets and couldn't work out what was wrong, so I reran the above
> commands, which presumably re-installed everything, and it all sprang into
> life - but with the file editing problem currently under discussion.
>
> The RPi is running Debian, but not the latest version as I didn't want to
> change anything that was working at the time. Maybe I should update now.
>
> Hope this helps, Richard.
>
>
>
>
> On Monday, 9 December 2013 22:04:31 UTC, Richard Brown wrote:
>
>> I am running Web2py on a Raspberry Pi and accessing the Admin
>> Interface via a browser on a Windows 8.1 PC. Since upgrading to the
>> latest version I can no longer edit default.py. When I try I get a 'blank'
>> edit screen with the 'Files Toggle', + A -' etc buttons at the top and the
>> 'Powered by Web2py' footer - but nothing in between. I can edit my Views,
>> db files and Modules, and I can peek default.py but always this blank
>> screen whenever I try to edit it. It is my longest file at just over 800
>> lines, but it worked fine before I upgraded.
>>
>> I have tried both IE and Chrome with the same results. Any ideas?
>>
>> Richard
>>
>  --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/fIuUAqdSieQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: Can't edit default.py after upgrade to 2.8.2

[paolo.vall...@gmail.com]

Hi Richard,
which webserver are you using? Let us know more info about the
infrastructure you have build on the rpi


 Paolo


2013/12/16 Richard Brown 

> OK, to answer Paolo's question on Friday as to what had changed to make it
> not work at all - nothing, only the passage of time. So as an experiment I
> rebooted my RPi once again and behold it all sprang into life. I have
> rebooted several times since the Web2py upgrade and each time it works to
> some extent for a while, and then stops. I was then away until Sunday
> evening but it was still working when I tried it then, and so I was
> actually able to do some development work. However tonight when I logged in
> - nothing again. Not a single file at all will open for editing.
>
> Could this be some sort of timing (or timeout) problem rather than a hard
> 'bug'? File loading, when it does work, seems much slower than with the
> previous version. I've supplied some debug info in previous posts, but if
> there is anything else I can look at then please let me know.
>
>  --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/fIuUAqdSieQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: Wappalyzer

[paolo.vall...@gmail.com]

It is here https://github.com/ElbertF/Wappalyzer/pull/397

 Paolo


2013/12/16 Paolo Valleri 

> hi,
> we can list all apps that web2py implies, for sure python, if we check the
> presence of web2py.js I would also set jquery, what else (see all apps here
> https://github.com/ElbertF/Wappalyzer/blob/master/share/apps.json)?
> We could also check for the presence of either "X-Powered-By:web2py" in
> the header or for the global js variable $.web2py (only for recent web2py
> app).
> I'll work on a PR later today.
> Paolo
>
> On Monday, December 16, 2013 2:42:19 PM UTC+1, webpypy wrote:
>>
>> Hi,
>>
>> I wish I could help, but I don't know what is it about.
>>
>> Good luck.
>>
>> Ashraf
>>
>  --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/aGHop5pYBxs/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: new feature in trunk: TODO panel in admin editor

[paolo.vall...@gmail.com]

Hi,
At the moment it is placed in the admin editor. Have a look there,in the
footer there is a button called “todo“ ,click there :-P
Il giorno 10/dic/2013 15:02, "Gael Princivalle" 
ha scritto:

> Hi Paolo.
>
> Sorry, where is this TODO panel ?
>
> Regards.
>
> Il giorno domenica 8 dicembre 2013 10:36:00 UTC+1, Paolo Valleri ha
> scritto:
>>
>> Dear all,
>> the online editor has got a new TODO panel in which are listed all TODO
>> you might have in the current application.
>> For the time being it highlights only TODO in python files, in the short
>> future we will support other file types and special keywords, along with
>> the keyword TODO I've seen users using FIXME (please let me know the ones
>> you use most).
>>
>> It is still in a work in progress feature, given that please let me know
>> not only any issue you will find but also all extensions you foresee.
>>
>> Paolo
>>
>  --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/DJ_pQAjMhUU/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: Proposal for big but simple improvement: [add view] in IDE view

[paolo.vall...@gmail.com]

You are right, technically speaking it is true we already have the name of
views and we can avoid to ask them again to the users.
However what you proposed It's not so straightforward  with the current
editor structure, I don't say it is impossible. At first sight we should:

1- migrate the file list sidebar into an indipendent component (currently
it is part of the editor page) (backend part)
2- for each opened file(tab) stores in a javascript variable the current
available views (js)
3- periodically or on demand, parse the python file to find the current
declared endpoints (js)
4- remove from the declared endpoints(discovered in point3) those already
available (point 2). (js)
5- feed the dropdown menu with those discovered in point4   (js)
6- when a new file is created, add the new view to the available
views(point3) and update the file list sidebar(point 1).(backend/js)

I start with 1, I can work on the other points in the coming days/week.


 Paolo


2013/12/9 Mika Sjöman 

> Well, I was thinking that usually you write the controller first. If you
> click an [Add view] button, it could check which functions are missing a
> view and give you two options in a popup, or pop-down? (don't know what it
> is called);
>
> -
> Add view for function:
>   [Link: missing_view_function1]
>   [Link: missing_view_function2]
>
> OR add one with name:
> Name: [  ] [Add View]
> --
>
> Mostly you would just click on the missing view function link and you
> would not misspell the file name, which is a quite common error and it
> would reduce writing. Also you would not add one by mistake if there is one
> already.
>
> // cheers
>
>
>
>
> On Monday, December 9, 2013 5:10:20 PM UTC+1, Paolo Valleri wrote:
>>
>> It is a good idea that could speed up the initial development phase;
>> We could start by implementing some like that: while you edit the
>> controller test.py at the top you will get a button 'add view' that when
>> clicked creates a file (a view actually) in the directory app/views/test;
>> the file name could be get by a dialog. Let me think about that a bit more.
>>
>> For the [Add or +] in the file list, if I remember correctly Roberto (in
>> CC) has started something a while ago.
>>
>> Paolo
>>
>> On Monday, December 9, 2013 4:57:25 PM UTC+1, Mika Sjöman wrote:
>>>
>>> Hi
>>>
>>> I just noticed watching Massimo's video tutorials, how often we need to
>>> open a second tab in Chrome when we want to create a view/controller/model
>>> file. It often leads us to get conflicts since we do not want to close the
>>> previous (having multiple tabs open we are working on).
>>>
>>> If we could add a view with a button "add view" when looking at a
>>> controller and the list of views, we could basically stop opening multiple
>>> chrome tabs and also reduce the merges we need to do.
>>>
>>> *Edit views:* view1, view2, [Add view]
>>>
>>> Clicking it would open the new text file immediately in another tab.
>>>
>>> It should also reload the files toggle view on the left. An [add or +]
>>> button could also be added after each section in the files toggle view to
>>> the left, and it would feel really complete. Even though I guess we would
>>> still open it sometime, it would reduce the times we need to click Edit
>>> link at the top bar.
>>>
>>  --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/JO4PrRnDpnU/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: Integrated IDE does not color javascript code

[paolo.vall...@gmail.com]

It should be fixed in trunk. Please have a try

Paolo

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: Best approach to render fragments

[paolo.vall...@gmail.com]

I don't know rails at all, and actually I don't even like to compare
frameworks, but with flexible and easier I meant something like (rails):
"render layout: false". In a talk I attended I remember that symfony has
something very similar as well.
Given that, I know they are completely different frameworks but personally
I don't know how to simply solve the example proposed in a cleaner way.

 Paolo


2013/8/27 Anthony 

>
> The first approach is difficult to maintain, the example is very simple,
>> the chain can be arbitrary long. The second one doesn't compile, and
>> empty.html is very ugly. I am wondering if could be possible to have extend
>> and include more flexible and easier.
>>
>
> What exactly do you envision? What do you mean by "more flexible and
> easier"?
>
> Note, rather than nesting lots of separate views, you might consider using
> Python logic within a given view. You could also have relatively simple
> request return rendered results directly rather than passing data to a view
> (if you return a string or HTML helper object, you don't need a view).
>
> Anthony
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/NhgqZlbu8L8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: Can page/chapter titles be added to the online book ?

[paolo.vall...@gmail.com]

Anthony we can have both, chapter numbers and slug in the url. If we don't
change the args position former links should work too. isn't it?

 Paolo


2013/6/24 Anthony 

> I have argued for slugs *instead* of chapter numbers in the URL's. There
> are many links to the book on the internet (here, Stack Overflow, etc.),
> and if we ever re-number the chapters (which we have done in the past), all
> the old links break.
>
> Also, when a section title ends with "(experimental)", it would be nice to
> keep that out of the URL, so when we finally remove the "(experimental)",
> it won't break existing links to the section (not to mention the fact that
> the parentheses present problems when pasting the URL into posts on Stack
> Overflow and other places).
>
> Anthony
>
>
> On Monday, June 24, 2013 2:55:03 PM UTC-4, Paolo valleri wrote:
>>
>> I would add the title in the url, not only for a better seo but also for
>> a faster in-browser url matching against visited url while typing,
>> from:
>> http://web2py.com/books/**default/chapter/29/02
>> to:
>> http://web2py.com/books/**default/chapter/29/02/The
>> database abstraction 
>> layer
>>
>> actually even better without the 'default'
>>
>> Paolo
>>
>> On Monday, June 24, 2013 8:31:11 PM UTC+2, Massimo Di Pierro wrote:
>>>
>>> Good idea. Asap. It is easy to do.
>>>
>>> On Monday, 24 June 2013 05:01:11 UTC-5, Holger Schurig wrote:

 As I'm new to web2py, I usually have several tabs open to various
 web2py chapters from the online book.

 Unfortunately, the page title for each tab is "web2py". See the
 attached screenshot.

 Can those book pages get more descriptive titles?

>>>  --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/NPpBGRBY1nI/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: disable left side panel in editor in 2.5>

[paolo.vall...@gmail.com]

I see your point, just a question, which web2py version are you using?
In trunk there is a newer editor with tabs, it allows you to edit few files
in parallel, by removing the left side panel your won't be able to use such
feature.

 Paolo


2013/6/24 lucas 

> hard-code or button?  doesn't matter to me, just a way to do it and how to
> get it done.  cool?  but, whatever the method, it be permanent and across
> all editable files.  lucas
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/kBinmkCYkpo/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: admin access password file

[paolo.vall...@gmail.com]

Massimo I discovered the problem, the server is behind a reserve proxy and
this latter wasn't properly configured, namely requests to the external
port 443 were directed to the local 80.
Sorry for the troubles!

 Paolo


2013/6/17 Massimo Di Pierro 

> I think the port is set by your script: 9001 and it is the same for http
> and https.
>
>
> On Monday, 17 June 2013 09:24:36 UTC-5, Paolo valleri wrote:
>>
>> Hi Massimo, I've just tried but the issue hasn't gone :(
>> what can I do to debug it more, by who is the request.env.server_port
>> setter ?
>>
>> Paolo
>>
>> On Sunday, June 16, 2013 10:32:37 AM UTC+2, Massimo Di Pierro wrote:
>>>
>>> After
>>>
>>>  uwsgi_param UWSGI_SCHEME $scheme;
>>>
>>> you must add
>>>
>>>  uwsgi_param HTTP_X_FORWARDED_PROTO https;
>>>
>>>
>>>
>>> On Wednesday, 12 June 2013 09:09:01 UTC-5, Paolo valleri wrote:

 Hi Massimo, I used https. I expired the same behavior on both apache
 and nginx.
 please find attached the condiguration file of nginx

  Paolo


 2013/6/12 Massimo Di Pierro 

> In the URL, are you using http or https? Assuming you use apache, can
> you show the config file?
>
> On Wednesday, 12 June 2013 01:53:25 UTC-5, Paolo valleri wrote:
>>
>> Hi all, I have to migrate a web2py app to a centos server. It has
>> python 2.6.6, uwsgi 1.9.12.  admin fails saying:
>>
>> "admin disabled because unable to access password file"
>>
>> the file (parameters_443.py) is there. I tried to debug the failure
>> and I discover that, despite the fact that I use https, it tries to 
>> access
>> parameters_80.py.
>> Thus, I created the file parameters_80.py but then it fails saying to
>> use https :(.
>>
>> Paolo
>>
>>
>>
>>  --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/**
> topic/web2py/1PDpL5yRKMs/**unsubscribe?hl=en
> .
> To unsubscribe from this group and all its topics, send an email to
> web2py+un...@googlegroups.com.
> For more options, visit 
> https://groups.google.com/**groups/opt_out
> .
>
>
>

  --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/1PDpL5yRKMs/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: [web2py-dev] Re: Spammers on web2pyslices.com

[paolo.vall...@gmail.com]

I don't see an approach able to tackle the issue at all,we should implement
several techniques together.
Anyway, what shall we do when a bot is detected? Have we got a sort of
blacklist? If so,instead of starting with an empty list,  we could think to
start from a public available blacklist of bot.
Il giorno 15/giu/2013 15:51, "Niphlod"  ha scritto:

> I have an unrelated (on web2py's side) website that uses captchas from
> google and bots are successfully registering to it (of course, they need to
> be approved first but it's a PITA to remove them anyway).
> There are captcha services that decode the images for you (and your bot).
>
> I'm working on a threaded comments plugin on my spare time and for spam
> prevention I just add some hidden fields that needs javascript to be
> filled. Given that bots running javascript code are a little percentage,
> this should mitigate the issue (at least, a similar technique on the
> aforementioned site is keeping spambots away).
> Small problem, though, users with javascript disabled are left alone. If
> that is fine, I can share the draft code (was waiting to complete the
> plugin before posting to github and here, but if needed that's not a big
> deal)
>
> On Saturday, June 15, 2013 3:37:26 PM UTC+2, Alan Etkin wrote:
>>
>> Is it possible these are not being posted by bots?
>>
>>
>> It would take a very smart bot to pass captcha (no?). Maybe it is
>> possible to change type of captcha used (i.e. random visual tests like
>> those of arithmetics with objects, etc.)?. I'm clueless about
>> authentication beyond the built-in web2py features but I can help running
>> tests against the web2pyslices app if needed.
>>
>> If so, we might need another tactic, such as requiring that a new user's
>>> first post be approved by a moderator.
>>
>>
>> +1
>>
>  --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/M2HlsCpqHbM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: admin access password file

[paolo.vall...@gmail.com]

Hi Massimo, I used https. I expired the same behavior on both apache and
nginx.
please find attached the condiguration file of nginx

 Paolo


2013/6/12 Massimo Di Pierro 

> In the URL, are you using http or https? Assuming you use apache, can you
> show the config file?
>
> On Wednesday, 12 June 2013 01:53:25 UTC-5, Paolo valleri wrote:
>>
>> Hi all, I have to migrate a web2py app to a centos server. It has python
>> 2.6.6, uwsgi 1.9.12.  admin fails saying:
>>
>> "admin disabled because unable to access password file"
>>
>> the file (parameters_443.py) is there. I tried to debug the failure and I
>> discover that, despite the fact that I use https, it tries to access
>> parameters_80.py.
>> Thus, I created the file parameters_80.py but then it fails saying to use
>> https :(.
>>
>> Paolo
>>
>>
>>
>>  --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/1PDpL5yRKMs/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.




ds.integreen-life.bz.it.conf
Description: Binary data

Re: [web2py] Re: scheduler sincronization

[paolo.vall...@gmail.com]

Hi Niphlod, sorry for the delay I had time only on the weekend to run few
tests. On my pc, I run it for hours using postgres. The maximum delay found
in starting a new task was 17s.

 Paolo


2013/4/4 paolo.vall...@gmail.com 

> Unfortunately I can't install any dbms on that machine, I will make a test
> on an other 'similar' machine where I have postgres installed.
>
>  Paolo
>
>
> 2013/4/3 Niphlod 
>
>> ok, at least it seems consistent:
>> there's:
>> - no send_heartbeat(), do_assign_tasks=True line
>> - no Inside If, do_assign_tasks=True line
>> in all the log.
>>
>> PS: I still can't reproduce the "erratic" behaviour, but a baseline
>> question rises: do you get this also using another db engine ?
>>
>>  --
>>
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "web2py-users" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/web2py/u_PgzKLuQmw/unsubscribe?hl=en.
>>  To unsubscribe from this group and all its topics, send an email to
>> web2py+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: scheduler sincronization

[paolo.vall...@gmail.com]

Unfortunately I can't install any dbms on that machine, I will make a test
on an other 'similar' machine where I have postgres installed.

 Paolo


2013/4/3 Niphlod 

> ok, at least it seems consistent:
> there's:
> - no send_heartbeat(), do_assign_tasks=True line
> - no Inside If, do_assign_tasks=True line
> in all the log.
>
> PS: I still can't reproduce the "erratic" behaviour, but a baseline
> question rises: do you get this also using another db engine ?
>
>  --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/u_PgzKLuQmw/unsubscribe?hl=en.
>  To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: scheduler sincronization

[paolo.vall...@gmail.com]

Right, I updated the log too, now I have:

self.do_assign_tasks = False
logger.debug('send_heartbeat(), do_assign_tasks=%s', self.do_assign_tasks )
if counter % 5 == 0 or mybackedstatus == PICK:

and

if self.worker_status[0] == ACTIVE:
logger.debug('Inside If, do_assign_tasks=%s', self.do_assign_tasks )
self.do_assign_tasks = True

I can send a patch with the added logs if it helps, and the complete log
file as well.
The log are:

2013-04-03 08:49:47,990 - web2py.scheduler.mapserver#5108 - DEBUG -
recording heartbeat (ACTIVE)
2013-04-03 08:49:47,990 - web2py.scheduler.mapserver#5108 - DEBUG -
send_heartbeat(), do_assign_tasks=False
2013-04-03 08:49:47,990 - web2py.scheduler.mapserver#5108 - DEBUG -
freeing workers that have not sent heartbeat
2013-04-03 08:49:47,992 - web2py.scheduler.mapserver#5108 - INFO - TICKER:
I'm a ticker
2013-04-03 08:49:48,029 - web2py.scheduler.mapserver#5108 - DEBUG -
looping...
2013-04-03 08:49:48,029 - web2py.scheduler.mapserver#5108 - INFO - pop task
True False
2013-04-03 08:49:48,143 - web2py.scheduler.mapserver#5108 - DEBUG -
is_a_ticker := True, worker_status[0] := ACTIVE
2013-04-03 08:49:48,143 - web2py.scheduler.mapserver#5108 - DEBUG - Inside
If, do_assign_tasks=False
2013-04-03 08:49:48,159 - web2py.scheduler.mapserver#5108 - INFO - nothing
to do
2013-04-03 08:49:48,159 - web2py.scheduler.mapserver#5108 - DEBUG -
sleeping...
2013-04-03 08:49:51,147 - web2py.scheduler.mapserver#5108 - DEBUG -
recording heartbeat (ACTIVE)
2013-04-03 08:49:51,148 - web2py.scheduler.mapserver#5108 - DEBUG -
send_heartbeat(), do_assign_tasks=False
2013-04-03 08:49:51,162 - web2py.scheduler.mapserver#5108 - DEBUG -
looping...
2013-04-03 08:49:51,162 - web2py.scheduler.mapserver#5108 - INFO - pop task
True False
2013-04-03 08:49:51,292 - web2py.scheduler.mapserver#5108 - INFO - nothing
to do
2013-04-03 08:49:51,292 - web2py.scheduler.mapserver#5108 - DEBUG -
sleeping...
2013-04-03 08:49:54,288 - web2py.scheduler.mapserver#5108 - DEBUG -
recording heartbeat (ACTIVE)
2013-04-03 08:49:54,289 - web2py.scheduler.mapserver#5108 - DEBUG -
send_heartbeat(), do_assign_tasks=False
2013-04-03 08:49:54,295 - web2py.scheduler.mapserver#5108 - DEBUG -
looping...
2013-04-03 08:49:54,295 - web2py.scheduler.mapserver#5108 - INFO - pop task
True False
2013-04-03 08:49:54,475 - web2py.scheduler.mapserver#5108 - INFO - nothing
to do
2013-04-03 08:49:54,475 - web2py.scheduler.mapserver#5108 - DEBUG -
sleeping...
2013-04-03 08:49:57,430 - web2py.scheduler.mapserver#5108 - DEBUG -
recording heartbeat (ACTIVE)
2013-04-03 08:49:57,430 - web2py.scheduler.mapserver#5108 - DEBUG -
send_heartbeat(), do_assign_tasks=False
2013-04-03 08:49:57,479 - web2py.scheduler.mapserver#5108 - DEBUG -
looping...
2013-04-03 08:49:57,479 - web2py.scheduler.mapserver#5108 - INFO - pop task
True False
2013-04-03 08:49:57,583 - web2py.scheduler.mapserver#5108 - INFO - nothing
to do
2013-04-03 08:49:57,584 - web2py.scheduler.mapserver#5108 - DEBUG -
sleeping...
2013-04-03 08:50:00,579 - web2py.scheduler.mapserver#5108 - DEBUG -
recording heartbeat (ACTIVE)
2013-04-03 08:50:00,579 - web2py.scheduler.mapserver#5108 - DEBUG -
send_heartbeat(), do_assign_tasks=False
2013-04-03 08:50:00,587 - web2py.scheduler.mapserver#5108 - DEBUG -
looping...
2013-04-03 08:50:00,587 - web2py.scheduler.mapserver#5108 - INFO - pop task
True False
2013-04-03 08:50:00,717 - web2py.scheduler.mapserver#5108 - INFO - nothing
to do
2013-04-03 08:50:00,717 - web2py.scheduler.mapserver#5108 - DEBUG -
sleeping...
2013-04-03 08:50:03,711 - web2py.scheduler.mapserver#5108 - DEBUG -
recording heartbeat (ACTIVE)
2013-04-03 08:50:03,712 - web2py.scheduler.mapserver#5108 - DEBUG -
send_heartbeat(), do_assign_tasks=False



Regards,





 Paolo


2013/4/2 Niphlod 

> I can't tell what logs line have been added (I see new lines but I don't
> know what is the line following "if counter " and what is the one
> following "if self.worker_status[0] ")
>
>
> On Tuesday, April 2, 2013 11:05:11 AM UTC+2, Paolo valleri wrote:
>
>> Hi Niphlod, sorry for the late answer, I hadn't access to the involved
>> server.
>> I added the two logs lines you proposed, below the logs:
>> 2013-04-02 09:38:14,561 - web2py.scheduler.mapserver#**4162 - DEBUG -
>> recording heartbeat (ACTIVE)
>> 2013-04-02 09:38:14,562 - web2py.scheduler.mapserver#**4162 - DEBUG -
>> send_heartbeat(), do_assign_tasks=False
>> 2013-04-02 09:38:14,562 - web2py.scheduler.mapserver#**4162 - DEBUG -
>>   freeing workers that have not sent heartbeat
>> 2013-04-02 09:38:14,564 - web2py.scheduler.mapserver#**4162 - INFO -
>> TICKER: I'm a ticker
>> 2013-04-02 09:38:14,644 - web2py.scheduler.mapserver#**4162 - DEBUG -
>> looping...
>> 2013-04-02 09:38:14,652 - web2py.scheduler.mapserver#**4162 - INFO - pop
>> task True False
>> 2013-04-02 09:38:14,723 - web2py.scheduler.mapserver#**4162 - DEBUG -
>> is_a_ticker := True, worker_status[0] := ACTIVE
>> 2013-04-02 

Re: [web2py] Re: scheduler sincronization

[paolo.vall...@gmail.com]

Hi Niphlod, sorry for the late answer, I hadn't access to the involved
server.
I added the two logs lines you proposed, below the logs:
2013-04-02 09:38:14,561 - web2py.scheduler.mapserver#4162 - DEBUG -
recording heartbeat (ACTIVE)
2013-04-02 09:38:14,562 - web2py.scheduler.mapserver#4162 - DEBUG -
send_heartbeat(), do_assign_tasks=False
2013-04-02 09:38:14,562 - web2py.scheduler.mapserver#4162 - DEBUG -
freeing workers that have not sent heartbeat
2013-04-02 09:38:14,564 - web2py.scheduler.mapserver#4162 - INFO - TICKER:
I'm a ticker
2013-04-02 09:38:14,644 - web2py.scheduler.mapserver#4162 - DEBUG -
looping...
2013-04-02 09:38:14,652 - web2py.scheduler.mapserver#4162 - INFO - pop task
True False
2013-04-02 09:38:14,723 - web2py.scheduler.mapserver#4162 - DEBUG -
is_a_ticker := True, worker_status[0] := ACTIVE
2013-04-02 09:38:14,732 - web2py.scheduler.mapserver#4162 - INFO - nothing
to do
2013-04-02 09:38:14,732 - web2py.scheduler.mapserver#4162 - DEBUG -
sleeping...
2013-04-02 09:38:17,727 - web2py.scheduler.mapserver#4162 - DEBUG -
recording heartbeat (ACTIVE)
2013-04-02 09:38:17,728 - web2py.scheduler.mapserver#4162 - DEBUG -
send_heartbeat(), do_assign_tasks=True
2013-04-02 09:38:17,735 - web2py.scheduler.mapserver#4162 - DEBUG -
looping...
2013-04-02 09:38:17,736 - web2py.scheduler.mapserver#4162 - INFO - pop task
True False
2013-04-02 09:38:17,916 - web2py.scheduler.mapserver#4162 - INFO - nothing
to do
2013-04-02 09:38:17,916 - web2py.scheduler.mapserver#4162 - DEBUG -
sleeping...
2013-04-02 09:38:20,871 - web2py.scheduler.mapserver#4162 - DEBUG -
recording heartbeat (ACTIVE)
2013-04-02 09:38:20,872 - web2py.scheduler.mapserver#4162 - DEBUG -
send_heartbeat(), do_assign_tasks=False
2013-04-02 09:38:20,919 - web2py.scheduler.mapserver#4162 - DEBUG -
looping...
2013-04-02 09:38:20,920 - web2py.scheduler.mapserver#4162 - INFO - pop task
True False
2013-04-02 09:38:21,050 - web2py.scheduler.mapserver#4162 - INFO - nothing
to do
2013-04-02 09:38:21,050 - web2py.scheduler.mapserver#4162 - DEBUG -
sleeping...
2013-04-02 09:38:24,045 - web2py.scheduler.mapserver#4162 - DEBUG -
recording heartbeat (ACTIVE)
2013-04-02 09:38:24,046 - web2py.scheduler.mapserver#4162 - DEBUG -
send_heartbeat(), do_assign_tasks=False
2013-04-02 09:38:24,054 - web2py.scheduler.mapserver#4162 - DEBUG -
looping...
2013-04-02 09:38:24,054 - web2py.scheduler.mapserver#4162 - INFO - pop task
True False
2013-04-02 09:38:24,234 - web2py.scheduler.mapserver#4162 - INFO - nothing
to do
2013-04-02 09:38:24,234 - web2py.scheduler.mapserver#4162 - DEBUG -
sleeping...
2013-04-02 09:38:27,193 - web2py.scheduler.mapserver#4162 - DEBUG -
recording heartbeat (ACTIVE)
2013-04-02 09:38:27,193 - web2py.scheduler.mapserver#4162 - DEBUG -
send_heartbeat(), do_assign_tasks=False
2013-04-02 09:38:27,238 - web2py.scheduler.mapserver#4162 - DEBUG -
looping...
2013-04-02 09:38:27,238 - web2py.scheduler.mapserver#4162 - INFO - pop task
True False
2013-04-02 09:38:27,368 - web2py.scheduler.mapserver#4162 - INFO - nothing
to do
2013-04-02 09:38:27,368 - web2py.scheduler.mapserver#4162 - DEBUG -
sleeping...
2013-04-02 09:38:30,360 - web2py.scheduler.mapserver#4162 - DEBUG -
recording heartbeat (ACTIVE)
2013-04-02 09:38:30,361 - web2py.scheduler.mapserver#4162 - DEBUG -
send_heartbeat(), do_assign_tasks=False
2013-04-02 09:38:30,361 - web2py.scheduler.mapserver#4162 - DEBUG -
freeing workers that have not sent heartbeat
2013-04-02 09:38:30,363 - web2py.scheduler.mapserver#4162 - INFO - TICKER:
I'm a ticker



 Paolo


2013/3/27 Niphlod 

> it was just a trial.
> Explaining it further, let's see if someone spots the **supposed**
> implementation error
>
> In order to separate transactions and avoid "task contention" among
> several workers, we need the **master** steps to both assign tasks and
> process tasks.
>
> The **thread** steps inserts a virtual "assign the task" job setting the
> do_assign_task flag to True when necessary.
> When that is found to be true, **worker** assigns tasks and sleeps for 3
> seconds, giving the change to the send_heartbeat() to reset the
> do_assign_task to False, so you don't have consecutive "assign_tasks" over
> and over.
>
> Skipping over the actual steps taken, a **master** does a loop and sleeps.
>
> In that loop, if do_assign_task is True, it assign tasks, returns "None"
> --> goes to sleep for 3 seconds
>
> A **thread** does some cleanup, sleeps for 3 seconds and sets the
> do_assign_task every 5 cycles.
>
> Ok, to be fair it's not guaranteed that a loop in either of those
> completes in 3 seconds, but the **steps** for both (when the **worker**
> doesn't anything) take a few ms (i.e. scheduler doesn't take into account
> how many ms passed between the start of either loop before setting the
> sleep of 3 seconds...)
>
> So, self.sleep() in normal condition gets called:
> - at every loop of the **thread** (at the end of the send_heartbeat())
> - at every loop of the **ma

Re: [web2py] Re: scheduler sincronization

[paolo.vall...@gmail.com]

I tried it, it seems to be more sporadic but the problem is not gone.
log:
2013-03-27 18:07:11,809 - web2py.scheduler.mapserver#30515 - DEBUG -
freeing workers that have not sent heartbeat
2013-03-27 18:07:11,811 - web2py.scheduler.mapserver#30515 - INFO - TICKER:
I'm a ticker
2013-03-27 18:07:11,843 - web2py.scheduler.mapserver#30515 - DEBUG -
looping...
2013-03-27 18:07:11,853 - web2py.scheduler.mapserver#30515 - INFO - pop
task True False
2013-03-27 18:07:11,970 - web2py.scheduler.mapserver#30515 - DEBUG -
is_a_ticker := True, worker_status[0] := ACTIVE
2013-03-27 18:07:11,983 - web2py.scheduler.mapserver#30515 - INFO - nothing
to do
2013-03-27 18:07:11,983 - web2py.scheduler.mapserver#30515 - DEBUG -
sleeping...
2013-03-27 18:07:14,971 - web2py.scheduler.mapserver#30515 - DEBUG -
recording heartbeat (ACTIVE)
2013-03-27 18:07:14,986 - web2py.scheduler.mapserver#30515 - DEBUG -
looping...
2013-03-27 18:07:15,009 - web2py.scheduler.mapserver#30515 - INFO - pop
task True False
2013-03-27 18:07:15,190 - web2py.scheduler.mapserver#30515 - INFO - nothing
to do
2013-03-27 18:07:15,190 - web2py.scheduler.mapserver#30515 - DEBUG -
sleeping...
2013-03-27 18:07:18,148 - web2py.scheduler.mapserver#30515 - DEBUG -
recording heartbeat (ACTIVE)
2013-03-27 18:07:18,192 - web2py.scheduler.mapserver#30515 - DEBUG -
looping...
2013-03-27 18:07:18,193 - web2py.scheduler.mapserver#30515 - INFO - pop
task True False
2013-03-27 18:07:18,323 - web2py.scheduler.mapserver#30515 - INFO - nothing
to do
2013-03-27 18:07:18,324 - web2py.scheduler.mapserver#30515 - DEBUG -
sleeping...
2013-03-27 18:07:21,314 - web2py.scheduler.mapserver#30515 - DEBUG -
recording heartbeat (ACTIVE)
2013-03-27 18:07:21,327 - web2py.scheduler.mapserver#30515 - DEBUG -
looping...
2013-03-27 18:07:21,327 - web2py.scheduler.mapserver#30515 - INFO - pop
task True False
2013-03-27 18:07:21,457 - web2py.scheduler.mapserver#30515 - INFO - nothing
to do
2013-03-27 18:07:21,458 - web2py.scheduler.mapserver#30515 - DEBUG -
sleeping...
2013-03-27 18:07:24,454 - web2py.scheduler.mapserver#30515 - DEBUG -
recording heartbeat (ACTIVE)
2013-03-27 18:07:24,461 - web2py.scheduler.mapserver#30515 - DEBUG -
looping...
2013-03-27 18:07:24,461 - web2py.scheduler.mapserver#30515 - INFO - pop
task True False
2013-03-27 18:07:24,641 - web2py.scheduler.mapserver#30515 - INFO - nothing
to do
2013-03-27 18:07:24,642 - web2py.scheduler.mapserver#30515 - DEBUG -
sleeping...
2013-03-27 18:07:27,604 - web2py.scheduler.mapserver#30515 - DEBUG -
recording heartbeat (ACTIVE)
2013-03-27 18:07:27,605 - web2py.scheduler.mapserver#30515 - DEBUG -
freeing workers that have not sent heartbeat
2013-03-27 18:07:27,607 - web2py.scheduler.mapserver#30515 - INFO - TICKER:
I'm a ticker


 Paolo


2013/3/27 Niphlod 

> the do_assign_task variable gets eventually shifted (read True to False,
> False to True) every loop.
> The issue is  how can the execution be pushed far (in your case even
> 60 seconds) with no errors whatsoever ?
>
> Usually when a TICKER has been chosen (or re-confirmed), do_assign_task is
> set to True and the thread is put to sleep for 3 seconds.
> There's quite no logic between the do_assign_task = True and the
> self.sleep() line, so there's a "sure" timeframe of 3 seconds for the main
> process to pick up the tasks.
> Are you suggesting that sleeping the main loop and the heartbeat thread
> "at the same pace" generates the problem you're seeing ?
> If so, as a trial, can you please replace the self.sleep() call on at the
> end of the loop() function to be, e.g. time.sleep(3.5) ?
>
>
> On Wednesday, March 27, 2013 1:42:22 PM UTC+1, Paolo valleri wrote:
>
>> So, I added: logger.debug('is_a_**ticker := %s, worker_status[0] := %s',
>> self.is_a_ticker, self.worker_status[0]) before the last if of relevant
>> snippet you posted,logs:
>>
>> 2013-03-27 13:25:01,785 - web2py.scheduler.mapserver#**29806 - INFO -
>> TICKER: I'm a ticker
>> 2013-03-27 13:25:01,797 - web2py.scheduler.mapserver#**29806 - DEBUG -
>> looping...
>> 2013-03-27 13:25:01,798 - web2py.scheduler.mapserver#**29806 - INFO -
>> pop task True False
>> 2013-03-27 13:25:01,950 - web2py.scheduler.mapserver#**29806 - DEBUG -
>> is_a_ticker := True, worker_status[0] := ACTIVE
>> 2013-03-27 13:25:01,978 - web2py.scheduler.mapserver#**29806 - INFO -
>> nothing to do
>>
>> I supposed to see "is_a_ticker := True, worker_status[0] := ACTIVE" after
>> "INFO - TICKER: I'm a ticker" and then something like:  'DEBUG - Assigning
>> tasks..' but I didn't see it because I have 'INFO - pop task True False',
>> the last False, means that self.do_assign_tasks was False. Later, the 
>> worker_status[0]
>> is ACTIVE so it would set do_assign_tasks to True but it is too late.
>> Given that, in some cases pop_task comes before the do_assign_tasks =
>> True statement. This could be the problem, Is It correct?
>>
>>
>>
>>
>>  Paolo
>>
>>
>> 2013/3/27 Niphlod 
>>
>>> yeps.
>>> this is expected, but ev

Re: [web2py] Re: scheduler sincronization

[paolo.vall...@gmail.com]

So, I added: logger.debug('is_a_ticker := %s, worker_status[0] := %s',
self.is_a_ticker, self.worker_status[0]) before the last if of relevant
snippet you posted,logs:

2013-03-27 13:25:01,785 - web2py.scheduler.mapserver#29806 - INFO - TICKER:
I'm a ticker
2013-03-27 13:25:01,797 - web2py.scheduler.mapserver#29806 - DEBUG -
looping...
2013-03-27 13:25:01,798 - web2py.scheduler.mapserver#29806 - INFO - pop
task True False
2013-03-27 13:25:01,950 - web2py.scheduler.mapserver#29806 - DEBUG -
is_a_ticker := True, worker_status[0] := ACTIVE
2013-03-27 13:25:01,978 - web2py.scheduler.mapserver#29806 - INFO - nothing
to do

I supposed to see "is_a_ticker := True, worker_status[0] := ACTIVE" after
"INFO - TICKER: I'm a ticker" and then something like:  'DEBUG - Assigning
tasks..' but I didn't see it because I have 'INFO - pop task True False',
the last False, means that self.do_assign_tasks was False. Later, the
worker_status[0]
is ACTIVE so it would set do_assign_tasks to True but it is too late.
Given that, in some cases pop_task comes before the do_assign_tasks = True
statement. This could be the problem, Is It correct?




 Paolo


2013/3/27 Niphlod 

> yeps.
> this is expected, but every ~15 seconds that turns to "True" (meaning more
> or less "I'm the Ticker, I'm free, I was free in the last 15 seconds, let's
> see if there are any submitted (or repeating) tasks to assign").
> I don't have a python interpreter ATM but let's break up the logic, so
> everyone can follow (and explaining most of the times lights up the
> underlying problem, let's hope so)
>
> Who "tells" if that process is a Ticker is the "being_a_ticker()" function.
> being_a_ticker() is wrapped in a try_except clause, so you should see the
> error logging in case something goes wrong --> Error coordinating TICKER
>
> one level up, do_assign_task is set every 5 cycles (or when "PICK" is
> found), with the "prune dead workers" logic in the same statements...that
> too is wrapped and should log --> Error cleaning up
>
> one level up, the whole thread checking is wrapped and should log -->
> Error retrieving status
>
> Now, you get "I'm a TICKER", so the being_a_ticker() is getting executed,
> and you don't have any errors reported.. this is the relevant snippet
> try:
> self.is_a_ticker = self.being_a_ticker()
> except:
> logger.error('Error coordinating TICKER')
> if self.worker_status[0] == ACTIVE:
> self.do_assign_tasks = True
>
> So.. either the worker status is not ACTIVE at that moment, or
> self.do_assign_task is set to True .
>
>
>
>
>
>  --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/u_PgzKLuQmw/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: scheduler sincronization

[paolo.vall...@gmail.com]

odd behavior, is possible that the function is not called at all?
I added a log line in pop_task(), in the else branch of the first test, to
understand why fails the test: The statement is logger.info("pop task %s
%s" % (self.is_a_ticker , self.do_assign_tasks))
The log says:
2013-03-27 11:38:01,276 - web2py.scheduler.mapserver#17922 - DEBUG -
recording heartbeat (ACTIVE)
2013-03-27 11:38:01,277 - web2py.scheduler.mapserver#17922 - DEBUG -
freeing workers that have not sent heartbeat
2013-03-27 11:38:01,278 - web2py.scheduler.mapserver#17922 - INFO - TICKER:
I'm a ticker
2013-03-27 11:38:01,315 - web2py.scheduler.mapserver#17922 - DEBUG -
looping...
2013-03-27 11:38:01,316 - web2py.scheduler.mapserver#17922 - INFO - pop
task True False
2013-03-27 11:38:01,445 - web2py.scheduler.mapserver#17922 - INFO - nothing
to do
2013-03-27 11:38:01,445 - web2py.scheduler.mapserver#17922 - DEBUG -
sleeping...
2013-03-27 11:38:04,432 - web2py.scheduler.mapserver#17922 - DEBUG -
recording heartbeat (ACTIVE)
2013-03-27 11:38:04,449 - web2py.scheduler.mapserver#17922 - DEBUG -
looping...
2013-03-27 11:38:04,461 - web2py.scheduler.mapserver#17922 - INFO - pop
task True False
2013-03-27 11:38:04,590 - web2py.scheduler.mapserver#17922 - INFO - nothing
to do
2013-03-27 11:38:04,590 - web2py.scheduler.mapserver#17922 - DEBUG -
sleeping...
2013-03-27 11:38:07,581 - web2py.scheduler.mapserver#17922 - DEBUG -
recording heartbeat (ACTIVE)
2013-03-27 11:38:07,594 - web2py.scheduler.mapserver#17922 - DEBUG -
looping...
2013-03-27 11:38:07,610 - web2py.scheduler.mapserver#17922 - INFO - pop
task True False
2013-03-27 11:38:07,740 - web2py.scheduler.mapserver#17922 - INFO - nothing
to do
2013-03-27 11:38:07,740 - web2py.scheduler.mapserver#17922 - DEBUG -
sleeping...
2013-03-27 11:38:10,731 - web2py.scheduler.mapserver#17922 - DEBUG -
recording heartbeat (ACTIVE)
2013-03-27 11:38:10,741 - web2py.scheduler.mapserver#17922 - DEBUG -
looping...
2013-03-27 11:38:10,741 - web2py.scheduler.mapserver#17922 - INFO - pop
task True False
2013-03-27 11:38:10,921 - web2py.scheduler.mapserver#17922 - INFO - nothing
to do
2013-03-27 11:38:10,922 - web2py.scheduler.mapserver#17922 - DEBUG -
sleeping...
2013-03-27 11:38:13,888 - web2py.scheduler.mapserver#17922 - DEBUG -
recording heartbeat (ACTIVE)
2013-03-27 11:38:13,925 - web2py.scheduler.mapserver#17922 - DEBUG -
looping...
2013-03-27 11:38:13,925 - web2py.scheduler.mapserver#17922 - INFO - pop
task True False
2013-03-27 11:38:14,055 - web2py.scheduler.mapserver#17922 - INFO - nothing
to do
2013-03-27 11:38:14,055 - web2py.scheduler.mapserver#17922 - DEBUG -
sleeping...
2013-03-27 11:38:17,047 - web2py.scheduler.mapserver#17922 - DEBUG -
recording heartbeat (ACTIVE)
2013-03-27 11:38:17,048 - web2py.scheduler.mapserver#17922 - DEBUG -
freeing workers that have not sent heartbeat

so, pop_task is not called because do_assign_task is False, hope it helps


 Paolo


2013/3/27 Niphlod 

> well, something dark and obscure is happening ..
> either
> logger.error('Error cleaning up')
> logger.error('Error retrieving status')
> logger.error('error popping tasks')
> logger.debug('Assigning tasks...')
> "must" happen
>
>
> On Tuesday, March 26, 2013 11:02:22 PM UTC+1, Paolo valleri wrote:
>
>> I will try to update sqlite as you suggested asap.
>> I tried your scheduler but I cannot see any error. In the meanwhile I
>> have seen that when the following logs are missing
>> "INFO - TICKER: workers are 1"
>> "INFO - TICKER: tasks are 0"
>> even the log: "DEBUG - Assigning tasks..." is missing.
>> could this mean that the function wrapped_assign_tasks is not called at
>> all?
>>
>>
>>
>>
>>  Paolo
>>
>>
>> 2013/3/26 Niphlod 
>>
>> here's the patch. I purposedly blocked the underlying db from another
>>> terminal to see what could be the issue, but I can't reproduce in other way
>>> what is happening on your system. Enough said, as soon as the db is
>>> unlocked, normal operations resume.
>>> There are a few error() calls to the logger, now if something goes wrong
>>> it's reported accordingly.
>>>
>>> PS: if you want to stick to SQLite, it's better to install the most
>>> updated sqlite adapter, on unix-like OSes is as simple as (giving standard
>>> build tools are available)
>>>
>>> pip install http://pysqlite.googlecode.**com/files/pysqlite-2.6.3.tar.**
>>> gz 
>>> --global-option 
>>> build_static
>>>
>>> and then activating WAL (reduces the chances of a locked db. Not lock
>>> free, but certainly helps out).
>>> WAL can be activated once on every db with a simple
>>> PRAGMA journal_mode=WAL
>>> or, if you are on a recent web2py distribution
>>>
>>> def activate_wal(db_instance):
>>> db_instance.execute('PRAGMA journal_mode=WAL;')
>>>
>>> db = DAL('sqlite://storage.sqlite', after_connection=activate_wal)
>>>
>>>
>>>
>>> On Tuesday, March 26, 2013 8:05:44 PM UTC+1, Paolo valleri wrote:
>>>

Re: [web2py] Re: scheduler sincronization

[paolo.vall...@gmail.com]

I will try to update sqlite as you suggested asap.
I tried your scheduler but I cannot see any error. In the meanwhile I have
seen that when the following logs are missing
"INFO - TICKER: workers are 1"
"INFO - TICKER: tasks are 0"
even the log: "DEBUG - Assigning tasks..." is missing.
could this mean that the function wrapped_assign_tasks is not called at all?




 Paolo


2013/3/26 Niphlod 

> here's the patch. I purposedly blocked the underlying db from another
> terminal to see what could be the issue, but I can't reproduce in other way
> what is happening on your system. Enough said, as soon as the db is
> unlocked, normal operations resume.
> There are a few error() calls to the logger, now if something goes wrong
> it's reported accordingly.
>
> PS: if you want to stick to SQLite, it's better to install the most
> updated sqlite adapter, on unix-like OSes is as simple as (giving standard
> build tools are available)
>
> pip install 
> http://pysqlite.googlecode.com/files/pysqlite-2.6.3.tar.gz--global-option 
> build_static
>
> and then activating WAL (reduces the chances of a locked db. Not lock
> free, but certainly helps out).
> WAL can be activated once on every db with a simple
> PRAGMA journal_mode=WAL
> or, if you are on a recent web2py distribution
>
> def activate_wal(db_instance):
> db_instance.execute('PRAGMA journal_mode=WAL;')
>
> db = DAL('sqlite://storage.sqlite', after_connection=activate_wal)
>
>
>
> On Tuesday, March 26, 2013 8:05:44 PM UTC+1, Paolo valleri wrote:
>
>> I can make few tests but only tomorrow, I will be out for the rest of the
>> week.
>> If you send me a patch with the new log statement, I will come back with
>> the result asap.
>>
>>  Paolo
>>
>>
>> 2013/3/26 Niphlod 
>>
>> whoa. seems that something wrong is happening trying to assing new
>>> tasks
>>> normally, every
>>> web2py.scheduler.mapserver#**7791 - INFO - TICKER: I'm a ticker
>>> should be followed closely by the lines
>>> web2py.scheduler.mapserver#**7791 - INFO - TICKER: workers are 1
>>> web2py.scheduler.mapserver#**7791 - INFO - TICKER: tasks are 0
>>> While in your case for several times those lines are not present
>>> The fact is that the assignment is wrapped yet in a try except clause
>>> and every exception should be logged as well, but your log doesn't show
>>> anything of that.
>>>
>>> I can add more debug lines to the scheduler but this didn't ever happen
>>> on all my platforms, so without reproducing it I'm a little bit unsure what
>>> the fix can be.
>>>
>>>
>>> On Tuesday, March 26, 2013 4:26:11 PM UTC+1, Paolo valleri wrote:
>>>
 the flle! sorry...

  Paolo


 2013/3/26 paolo@gmail.com 

> If can be useful, I attached part of the log file in which demo1 is
> executed.
> First execution: 2013-03-26 15:52:31
> second execution: 2013-03-26 15:58:55 (+384s)
>
>  Paolo
>
>
> 2013/3/26 Paolo valleri 
>
> >>> import sqlite3
>> >>> print sqlite3.version
>> 2.6.0
>> >>> print sqlite3.sqlite_version
>> 3.7.9
>> But, if the db lock is not the problem, the test application is very
>> easy, where is it supposed to be the problem?
>>
>>
>> On Tuesday, March 26, 2013 2:32:50 PM UTC+1, Niphlod wrote:
>>>
>>> I find hard to believe that with a single worker, with that function
>>> that basically just prints something and an execution every 300 seconds 
>>> the
>>> problem lies into a lock, unless the SQLite library available on your
>>> system is reallly old.
>>>
>>> On Tuesday, March 26, 2013 2:21:21 PM UTC+1, Paolo valleri wrote:

 When yesterday I saw demo1 in timeout with ps auxf I have seen that
 a new process was created. For this reason I started to debug 
 scheduler and
 I asked how to log etc.
 Moreover, I restarted the scheduler manually so I am not able to
 understand if the other different names are for an internal problem or
 something different.
 Do you think that should be fixed by using a different db engine?

 Paolo

 On Tuesday, March 26, 2013 12:42:14 PM UTC+1, Niphlod wrote:
>
> with the default logging.conf the timestamp is present as in all
> other web2py-related logging 
>
> PS: are you sure that the worker is not killed/restarted by any
> chance (see the worker_name in the scheduler_run table)
>
> On Tuesday, March 26, 2013 11:33:53 AM UTC+1, Paolo valleri wrote:
>>
>> I executed again demo1, I run it several times, I got even in
>> this case elapsed time between two consecutive executions around 360 
>> and
>> even more instead of 300. What can I do to understand what is not 
>> working
>> correctly?
>> Moreover, I would suggest to add the timestamp to the scheduler
>> debug log.
>

Re: [web2py] Re: scheduler sincronization

[paolo.vall...@gmail.com]

I can make few tests but only tomorrow, I will be out for the rest of the
week.
If you send me a patch with the new log statement, I will come back with
the result asap.

 Paolo


2013/3/26 Niphlod 

> whoa. seems that something wrong is happening trying to assing new
> tasks
> normally, every
> web2py.scheduler.mapserver#7791 - INFO - TICKER: I'm a ticker
> should be followed closely by the lines
> web2py.scheduler.mapserver#7791 - INFO - TICKER: workers are 1
> web2py.scheduler.mapserver#7791 - INFO - TICKER: tasks are 0
> While in your case for several times those lines are not present
> The fact is that the assignment is wrapped yet in a try except clause and
> every exception should be logged as well, but your log doesn't show
> anything of that.
>
> I can add more debug lines to the scheduler but this didn't ever happen on
> all my platforms, so without reproducing it I'm a little bit unsure what
> the fix can be.
>
>
> On Tuesday, March 26, 2013 4:26:11 PM UTC+1, Paolo valleri wrote:
>
>> the flle! sorry...
>>
>>  Paolo
>>
>>
>> 2013/3/26 paolo@gmail.com 
>>
>>> If can be useful, I attached part of the log file in which demo1 is
>>> executed.
>>> First execution: 2013-03-26 15:52:31
>>> second execution: 2013-03-26 15:58:55 (+384s)
>>>
>>>  Paolo
>>>
>>>
>>> 2013/3/26 Paolo valleri 
>>>
>>> >>> import sqlite3
 >>> print sqlite3.version
 2.6.0
 >>> print sqlite3.sqlite_version
 3.7.9
 But, if the db lock is not the problem, the test application is very
 easy, where is it supposed to be the problem?


 On Tuesday, March 26, 2013 2:32:50 PM UTC+1, Niphlod wrote:
>
> I find hard to believe that with a single worker, with that function
> that basically just prints something and an execution every 300 seconds 
> the
> problem lies into a lock, unless the SQLite library available on your
> system is reallly old.
>
> On Tuesday, March 26, 2013 2:21:21 PM UTC+1, Paolo valleri wrote:
>>
>> When yesterday I saw demo1 in timeout with ps auxf I have seen that a
>> new process was created. For this reason I started to debug scheduler 
>> and I
>> asked how to log etc.
>> Moreover, I restarted the scheduler manually so I am not able to
>> understand if the other different names are for an internal problem or
>> something different.
>> Do you think that should be fixed by using a different db engine?
>>
>> Paolo
>>
>> On Tuesday, March 26, 2013 12:42:14 PM UTC+1, Niphlod wrote:
>>>
>>> with the default logging.conf the timestamp is present as in all
>>> other web2py-related logging 
>>>
>>> PS: are you sure that the worker is not killed/restarted by any
>>> chance (see the worker_name in the scheduler_run table)
>>>
>>> On Tuesday, March 26, 2013 11:33:53 AM UTC+1, Paolo valleri wrote:

 I executed again demo1, I run it several times, I got even in this
 case elapsed time between two consecutive executions around 360 and 
 even
 more instead of 300. What can I do to understand what is not working
 correctly?
 Moreover, I would suggest to add the timestamp to the scheduler
 debug log.


  Paolo


 2013/3/25 Niphlod 

>
>
> On Monday, March 25, 2013 10:46:12 PM UTC+1, Paolo valleri wrote:
>>
>> I didn't get your point, with one repetitive task, should I start
>> the scheduler with two or more workers? If so, I will try it.
>>
>
> The point is that the thread that manages some logic every
> heartbeat seconds is the one in charge of "waiting" 5 loops to 
> trigger the
> additional logic to pick up new tasks (a repetitive task is just a 
> new task
> to execute). If the process "doing the work" is busy processing the 
> task
> and the underlying thread reaches the "let's assign tasks" loop, the 
> logic
> will be skipped (it's unuseful to assign tasks if a worker is already
> processing them). So it can happen that even if the "assignment" time 
> has
> come, if the worker is processing tasks it will skip the "assignment"
>
> Actually I have just seen the stop time, on average the task
>> completes it cycle in just a few seconds (~1-2). Given that,  is 
>> what you
>> have suggested still valid?
>>
>
> Nope. As I said it guaranteed that even in the case that the
> assignment loop falls into the timeframe of a RUNNING task, at the 
> next
> round it will be picked up
>
>
>> Last but not least, demo1 has gone in timeout after one
>> successful cycle, this is very odd, How I can debug the scheduler
>> application and find its errors?
>> I am 

Re: [web2py] Re: scheduler sincronization

[paolo.vall...@gmail.com]

If can be useful, I attached part of the log file in which demo1 is
executed.
First execution: 2013-03-26 15:52:31
second execution: 2013-03-26 15:58:55 (+384s)

 Paolo


2013/3/26 Paolo valleri 

> >>> import sqlite3
> >>> print sqlite3.version
> 2.6.0
> >>> print sqlite3.sqlite_version
> 3.7.9
> But, if the db lock is not the problem, the test application is very easy,
> where is it supposed to be the problem?
>
>
> On Tuesday, March 26, 2013 2:32:50 PM UTC+1, Niphlod wrote:
>>
>> I find hard to believe that with a single worker, with that function that
>> basically just prints something and an execution every 300 seconds the
>> problem lies into a lock, unless the SQLite library available on your
>> system is reallly old.
>>
>> On Tuesday, March 26, 2013 2:21:21 PM UTC+1, Paolo valleri wrote:
>>>
>>> When yesterday I saw demo1 in timeout with ps auxf I have seen that a
>>> new process was created. For this reason I started to debug scheduler and I
>>> asked how to log etc.
>>> Moreover, I restarted the scheduler manually so I am not able to
>>> understand if the other different names are for an internal problem or
>>> something different.
>>> Do you think that should be fixed by using a different db engine?
>>>
>>> Paolo
>>>
>>> On Tuesday, March 26, 2013 12:42:14 PM UTC+1, Niphlod wrote:

 with the default logging.conf the timestamp is present as in all other
 web2py-related logging 

 PS: are you sure that the worker is not killed/restarted by any chance
 (see the worker_name in the scheduler_run table)

 On Tuesday, March 26, 2013 11:33:53 AM UTC+1, Paolo valleri wrote:
>
> I executed again demo1, I run it several times, I got even in this
> case elapsed time between two consecutive executions around 360 and even
> more instead of 300. What can I do to understand what is not working
> correctly?
> Moreover, I would suggest to add the timestamp to the scheduler debug
> log.
>
>
>  Paolo
>
>
> 2013/3/25 Niphlod 
>
>>
>>
>> On Monday, March 25, 2013 10:46:12 PM UTC+1, Paolo valleri wrote:
>>>
>>> I didn't get your point, with one repetitive task, should I start
>>> the scheduler with two or more workers? If so, I will try it.
>>>
>>
>> The point is that the thread that manages some logic every heartbeat
>> seconds is the one in charge of "waiting" 5 loops to trigger the 
>> additional
>> logic to pick up new tasks (a repetitive task is just a new task to
>> execute). If the process "doing the work" is busy processing the task and
>> the underlying thread reaches the "let's assign tasks" loop, the logic 
>> will
>> be skipped (it's unuseful to assign tasks if a worker is already 
>> processing
>> them). So it can happen that even if the "assignment" time has come, if 
>> the
>> worker is processing tasks it will skip the "assignment"
>>
>> Actually I have just seen the stop time, on average the task
>>> completes it cycle in just a few seconds (~1-2). Given that,  is what 
>>> you
>>> have suggested still valid?
>>>
>>
>> Nope. As I said it guaranteed that even in the case that the
>> assignment loop falls into the timeframe of a RUNNING task, at the next
>> round it will be picked up
>>
>>
>>> Last but not least, demo1 has gone in timeout after one successful
>>> cycle, this is very odd, How I can debug the scheduler application and 
>>> find
>>> its errors?
>>> I am running scheduler as a linux service, as described here:
>>> http://web2py.com/books/**defaul**t/chapter/29/13#Start-**the-**
>>> scheduler-as-a-Linux-**service-%**28upstart%29
>>>
>>>
>>
>> SQLite locking is the most probable cause.
>> The fastest way is to see what's happening is starting the scheduler
>> with debug logging 
>> web2py.py -K appname -D 0
>>
>> --
>>
>> ---
>> You received this message because you are subscribed to a topic in
>> the Google Groups "web2py-users" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/**
>> topic/web2py/u_PgzKLuQmw/**unsubscribe?hl=en
>> .
>> To unsubscribe from this group and all its topics, send an email to
>> web2py+un...@googlegroups.com.
>> For more options, visit 
>> https://groups.google.com/**groups/opt_out
>> .
>>
>>
>>
>
>  --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/u_PgzKLuQmw/unsubscribe?hl=en.
> To unsubscribe from this group and all its topi

Re: [web2py] Re: scheduler sincronization

[paolo.vall...@gmail.com]

I executed again demo1, I run it several times, I got even in this case
elapsed time between two consecutive executions around 360 and even more
instead of 300. What can I do to understand what is not working correctly?
Moreover, I would suggest to add the timestamp to the scheduler debug log.


 Paolo


2013/3/25 Niphlod 

>
>
> On Monday, March 25, 2013 10:46:12 PM UTC+1, Paolo valleri wrote:
>>
>> I didn't get your point, with one repetitive task, should I start the
>> scheduler with two or more workers? If so, I will try it.
>>
>
> The point is that the thread that manages some logic every heartbeat
> seconds is the one in charge of "waiting" 5 loops to trigger the additional
> logic to pick up new tasks (a repetitive task is just a new task to
> execute). If the process "doing the work" is busy processing the task and
> the underlying thread reaches the "let's assign tasks" loop, the logic will
> be skipped (it's unuseful to assign tasks if a worker is already processing
> them). So it can happen that even if the "assignment" time has come, if the
> worker is processing tasks it will skip the "assignment"
>
> Actually I have just seen the stop time, on average the task completes it
>> cycle in just a few seconds (~1-2). Given that,  is what you have suggested
>> still valid?
>>
>
> Nope. As I said it guaranteed that even in the case that the assignment
> loop falls into the timeframe of a RUNNING task, at the next round it will
> be picked up
>
>
>> Last but not least, demo1 has gone in timeout after one successful cycle,
>> this is very odd, How I can debug the scheduler application and find its
>> errors?
>> I am running scheduler as a linux service, as described here:
>> http://web2py.com/books/**default/chapter/29/13#Start-**
>> the-scheduler-as-a-Linux-**service-%28upstart%29
>>
>>
>
> SQLite locking is the most probable cause.
> The fastest way is to see what's happening is starting the scheduler with
> debug logging 
> web2py.py -K appname -D 0
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/u_PgzKLuQmw/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: scheduler sincronization

[paolo.vall...@gmail.com]

I didn't get your point, with one repetitive task, should I start the
scheduler with two or more workers? If so, I will try it.
Actually I have just seen the stop time, on average the task completes it
cycle in just a few seconds (~1-2). Given that,  is what you have suggested
still valid?
Last but not least, demo1 has gone in timeout after one successful cycle,
this is very odd, How I can debug the scheduler application and find its
errors?
I am running scheduler as a linux service, as described here:
http://web2py.com/books/default/chapter/29/13#Start-the-scheduler-as-a-Linux-service-%28upstart%29


 Paolo


2013/3/25 Niphlod 

> I don't think that's the issue cause next_time_run gets calculated from
> the start_time of the previous task, i.e. if I put time.sleep(60) in
> demo1(), I still get the same execution times...
>
> The only bit missing at this point is remembering that if you have a
> single worker it can't ASSIGN a new task while a task is RUNNING, so if,
> e.g., your task is still RUNNING while the scheduler reaches "the 5th
> round" it will skip the assignment and do it on the 10th, i.e. the first
> time it's not processing any task and 15 seconds are passed (still, worst
> case scenario)
>
>
> On Monday, March 25, 2013 10:07:45 PM UTC+1, Paolo valleri wrote:
>
>> My task is available here: https://github.com/ilvalle/**
>> scapp/blob/master/models/**scheduler.py
>> As you can see the difference is related to the external request. Asap I
>> will try your app too.
>>
>>  Paolo
>>
>>
>> 2013/3/25 Niphlod 
>>
>> can't reproduce the issue, even with SQLite.
>>> Using the demo1() function from the w2p_scheduler_test app, if I
>>> schedule it with period=300, repeats = 0, this is what I get back
>>>
>>> r1: 2013-03-25 21:32:56
>>> r2: 2013-03-25 21:38:12  -- 316s
>>> r3: 2013-03-25 21:43:28  -- 316s
>>> r4: 2013-03-25 21:48:31  -- 303s
>>> r5: 2013-03-25 21:53:36  -- 305s
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Monday, March 25, 2013 8:57:09 PM UTC+1, Paolo valleri wrote:
>>>
 Hi, given the fact that I left untouched the heartbeats value and that
 my worker was performing only the test task, I got odd starting values:
 t1: 2013-03-25 15:34:43
 t2: 2013-03-25 15:40:48  ( t2 started after 6m + 5s = 365seconds )
 t3: 2013-03-25 15:46:52  ( t3 started after 6m + 4s = 364seconds )
 t4: 2013-03-25 15:52:08  ( t4 started after 5m +16s = 316seconds )
 t5: 2013-03-25 15:57:22  ( t5 started after 5m +14s = 314seconds )

 Given these starting values, it seems that only the last one is under
 315. I think we should investigate more how scheduler runs repetitive
 tasks.



  Paolo


 2013/3/25 Niphlod 

> The scheduler is not as "precise" as you would because there are some
> design considerations to think of
>
> The "uber-costraint"  is that a worker can execute only one task at a
> time --> if the scheduler is busy with something else, queued tasks can be
> "delayed" down the line
> If the worker is free, new tasks are checked every 5*heartbeats
> seconds, so (unless immediate=True) you can get a timeframe for execution
> that spans in the worst case scenario with the default values by 15 
> seconds
>
> This means that in the worst case scenario you could have a repeating
> task with a 300 seconds period that are actually executed every 315 
> seconds.
>
> To clear out doubts, for repetitive tasks, as documented in more
> detail in the w2p_scheduler_tests app, the start time of the next 
> execution
> is calculated adding period seconds after the start time of the current
> execution.
> Lockings on the db can add more "imprecision", cause by the default
> the scheduler is eager to store/update whatever it needs to, to ensure
> consinstency between the real executions and the data stored on the 
> tables.
>
>
> On Monday, March 25, 2013 4:23:56 PM UTC+1, Paolo valleri wrote:
>>
>> Dear all,
>> I started to use scheduler. I've created a simple starting example
>> task: https://github.com/ilvalle/**scapp/blob/master/models/**
>> scheduler.py
>> and with the great niphlod's plugin cs_monitor_plugin I created an
>> initial repetitive task.
>> However, although I set repeats: 0 and period: 300, tasks are not
>> repeated as expected exactly every 300s. The first five tasks were 
>> executed
>> at:
>> 2013-03-25 15:34:43
>> 2013-03-25 15:40:48
>> 2013-03-25 15:46:52
>> 2013-03-25 15:52:08
>> 2013-03-25 15:57:22
>> The time between two consecutive execution is not so straightforward.
>> what am I missing? Is there a w

Re: [web2py] Re: scheduler sincronization

[paolo.vall...@gmail.com]

My task is available here:
https://github.com/ilvalle/scapp/blob/master/models/scheduler.py
As you can see the difference is related to the external request. Asap I
will try your app too.

 Paolo


2013/3/25 Niphlod 

> can't reproduce the issue, even with SQLite.
> Using the demo1() function from the w2p_scheduler_test app, if I schedule
> it with period=300, repeats = 0, this is what I get back
>
> r1: 2013-03-25 21:32:56
> r2: 2013-03-25 21:38:12  -- 316s
> r3: 2013-03-25 21:43:28  -- 316s
> r4: 2013-03-25 21:48:31  -- 303s
> r5: 2013-03-25 21:53:36  -- 305s
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Monday, March 25, 2013 8:57:09 PM UTC+1, Paolo valleri wrote:
>
>> Hi, given the fact that I left untouched the heartbeats value and that my
>> worker was performing only the test task, I got odd starting values:
>> t1: 2013-03-25 15:34:43
>> t2: 2013-03-25 15:40:48  ( t2 started after 6m + 5s = 365seconds )
>> t3: 2013-03-25 15:46:52  ( t3 started after 6m + 4s = 364seconds )
>> t4: 2013-03-25 15:52:08  ( t4 started after 5m +16s = 316seconds )
>> t5: 2013-03-25 15:57:22  ( t5 started after 5m +14s = 314seconds )
>>
>> Given these starting values, it seems that only the last one is under
>> 315. I think we should investigate more how scheduler runs repetitive
>> tasks.
>>
>>
>>
>>  Paolo
>>
>>
>> 2013/3/25 Niphlod 
>>
>>> The scheduler is not as "precise" as you would because there are some
>>> design considerations to think of
>>>
>>> The "uber-costraint"  is that a worker can execute only one task at a
>>> time --> if the scheduler is busy with something else, queued tasks can be
>>> "delayed" down the line
>>> If the worker is free, new tasks are checked every 5*heartbeats seconds,
>>> so (unless immediate=True) you can get a timeframe for execution that spans
>>> in the worst case scenario with the default values by 15 seconds
>>>
>>> This means that in the worst case scenario you could have a repeating
>>> task with a 300 seconds period that are actually executed every 315 seconds.
>>>
>>> To clear out doubts, for repetitive tasks, as documented in more detail
>>> in the w2p_scheduler_tests app, the start time of the next execution is
>>> calculated adding period seconds after the start time of the current
>>> execution.
>>> Lockings on the db can add more "imprecision", cause by the default the
>>> scheduler is eager to store/update whatever it needs to, to ensure
>>> consinstency between the real executions and the data stored on the tables.
>>>
>>>
>>> On Monday, March 25, 2013 4:23:56 PM UTC+1, Paolo valleri wrote:

 Dear all,
 I started to use scheduler. I've created a simple starting example
 task: https://github.com/ilvalle/**sca**pp/blob/master/models/**
 schedule**r.py
 and with the great niphlod's plugin cs_monitor_plugin I created an
 initial repetitive task.
 However, although I set repeats: 0 and period: 300, tasks are not
 repeated as expected exactly every 300s. The first five tasks were executed
 at:
 2013-03-25 15:34:43
 2013-03-25 15:40:48
 2013-03-25 15:46:52
 2013-03-25 15:52:08
 2013-03-25 15:57:22
 The time between two consecutive execution is not so straightforward.
 what am I missing? Is there a way to impose more accuracy ?

 paolo

>>>  --
>>>
>>> ---
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "web2py-users" group.
>>> To unsubscribe from this topic, visit https://groups.google.com/d/**
>>> topic/web2py/u_PgzKLuQmw/**unsubscribe?hl=en
>>> .
>>>  To unsubscribe from this group and all its topics, send an email to
>>> web2py+un...@**googlegroups.com.
>>>
>>> For more options, visit 
>>> https://groups.google.com/**groups/opt_out
>>> .
>>>
>>>
>>>
>>
>>  --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/u_PgzKLuQmw/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: scheduler sincronization

[paolo.vall...@gmail.com]

Hi, given the fact that I left untouched the heartbeats value and that my
worker was performing only the test task, I got odd starting values:
t1: 2013-03-25 15:34:43
t2: 2013-03-25 15:40:48  ( t2 started after 6m + 5s = 365seconds )
t3: 2013-03-25 15:46:52  ( t3 started after 6m + 4s = 364seconds )
t4: 2013-03-25 15:52:08  ( t4 started after 5m +16s = 316seconds )
t5: 2013-03-25 15:57:22  ( t5 started after 5m +14s = 314seconds )

Given these starting values, it seems that only the last one is under 315.
I think we should investigate more how scheduler runs repetitive tasks.



 Paolo


2013/3/25 Niphlod 

> The scheduler is not as "precise" as you would because there are some
> design considerations to think of
>
> The "uber-costraint"  is that a worker can execute only one task at a time
> --> if the scheduler is busy with something else, queued tasks can be
> "delayed" down the line
> If the worker is free, new tasks are checked every 5*heartbeats seconds,
> so (unless immediate=True) you can get a timeframe for execution that spans
> in the worst case scenario with the default values by 15 seconds
>
> This means that in the worst case scenario you could have a repeating task
> with a 300 seconds period that are actually executed every 315 seconds.
>
> To clear out doubts, for repetitive tasks, as documented in more detail in
> the w2p_scheduler_tests app, the start time of the next execution is
> calculated adding period seconds after the start time of the current
> execution.
> Lockings on the db can add more "imprecision", cause by the default the
> scheduler is eager to store/update whatever it needs to, to ensure
> consinstency between the real executions and the data stored on the tables.
>
>
> On Monday, March 25, 2013 4:23:56 PM UTC+1, Paolo valleri wrote:
>>
>> Dear all,
>> I started to use scheduler. I've created a simple starting example task:
>> https://github.com/ilvalle/**scapp/blob/master/models/**scheduler.py
>> and with the great niphlod's plugin cs_monitor_plugin I created an
>> initial repetitive task.
>> However, although I set repeats: 0 and period: 300, tasks are not
>> repeated as expected exactly every 300s. The first five tasks were executed
>> at:
>> 2013-03-25 15:34:43
>> 2013-03-25 15:40:48
>> 2013-03-25 15:46:52
>> 2013-03-25 15:52:08
>> 2013-03-25 15:57:22
>> The time between two consecutive execution is not so straightforward.
>> what am I missing? Is there a way to impose more accuracy ?
>>
>> paolo
>>
>  --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/u_PgzKLuQmw/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: scheduler sincronization

[paolo.vall...@gmail.com]

Accordingly to the book
http://web2py.com/books/default/chapter/29/04?search=scheduler#Scheduler-(experimental)
It doesn't start when the previous task ends but when the previous task
starts instead.

 Paolo


2013/3/25 szimszon 

> I think the period starts as the last execution ends.
>
> Sometimes I miss the feature to start the task exactly at fixed interval
> rater than fixed interval starting after the last run ends but I can live
> without it. :) It's a great tool anyway :-D
>
> 2013. március 25., hétfő 16:23:56 UTC+1 időpontban Paolo valleri a
> következőt írta:
>
>> Dear all,
>> I started to use scheduler. I've created a simple starting example task:
>> https://github.com/ilvalle/**scapp/blob/master/models/**scheduler.py
>> and with the great niphlod's plugin cs_monitor_plugin I created an
>> initial repetitive task.
>> However, although I set repeats: 0 and period: 300, tasks are not
>> repeated as expected exactly every 300s. The first five tasks were executed
>> at:
>> 2013-03-25 15:34:43
>> 2013-03-25 15:40:48
>> 2013-03-25 15:46:52
>> 2013-03-25 15:52:08
>> 2013-03-25 15:57:22
>> The time between two consecutive execution is not so straightforward.
>> what am I missing? Is there a way to impose more accuracy ?
>>
>> paolo
>>
>  --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/u_PgzKLuQmw/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: Cache performance with computed columns

[paolo.vall...@gmail.com]

Hi Niphlod, thanks for the answer. I am not blaming on web2py I am just
sharing with you all these results.
In both the cases I have the time constrain, in one case it is in the query
(Q1) while in the second it is explicit(Q2). With cache enabled, Q1 is no
longer checking the constrain while Q2 is still checking all the constrain
for all rows. Moreover, computing the .epoch() should take a well defined
amount of time but for queries without cache, with cache it should be
stored the computed result.

To sum up, when cache is enabled, with the same amount of rows (nearly
300):
- Q2 is taking less time than Q1 even if it has to compute explicitly the
time constrain
- Q1 is taking more time than Q2, but it has to retrieve from cache only
one field more than Q1 among a dozen of fields

Given that, retrieving more data from memcached can be slower than
re-computing it for each request.


 Paolo


2013/3/25 paolo.vall...@gmail.com 

>
>
>  Paolo
>
>
> 2013/3/25 Niphlod 
>
>> I didn't get the point. 
>> Q1 is slower (obviously) cause of the epoch() function and the additional
>> filtering
>> Q1 with cache shows more or less the same duration as Q2 with cache.
>> the key point is "even when the cache is enabled the first query is
>> taking longer" ... longer than what ?
>> Q1 with cache is speedier that:
>> - Q1 without cache, and that's ok
>> - Q2 without cache, and that's ok
>> but it's slower than
>> - Q2 with cache
>> however 3 ms (assuming those timings are in seconds) with all the moving
>> parts of a memcache instance seems unfair to blame entirely on web2py
>>
>>
>>
>> On Monday, March 25, 2013 9:48:14 AM UTC+1, Paolo valleri wrote:
>>>
>>> Dear all, I want to share with you some results. I implemented a query
>>> in two different ways. Given the following common code:
>>> start = db.record.with_alias('start_**point')
>>> end = db.record.with_alias('end_**point')
>>> elapsed_time = end.gathered_on.epoch() - start.gathered_on.epoch()
>>>
>>> The first query is (the constrain is in the query):
>>> rows = db( query & (elapsed_time  < 86400) ).select(
>>> start.ALL,
>>> end.ALL,
>>>  start.gathered_on.epoch(),
>>> end.gathered_on.epoch(),
>>> elapsed_time,
>>>  orderby=start.gathered_on.**epoch(),
>>> left=start.on( (start.mac == end.mac) & (start.gathered_on <
>>> end.gathered_on)),
>>>  cache=(cache.memcache, 3600),
>>> cacheable = True
>>> )
>>> The second one is (the constrain is explicitly tested latter):
>>> rows = db( query ).select(
>>> start.ALL,
>>> end.ALL,
>>> start.gathered_on.epoch(),
>>>  end.gathered_on.epoch(),
>>> elapsed_time,
>>> orderby=start.gathered_on.**epoch(),
>>>  left=start.on( (start.mac == end.mac) & (start.gathered_on <
>>> end.gathered_on)),
>>> cache=(cache.memcache, 3600),
>>>  cacheable = True
>>> )
>>> rows2 = [r for r in rows if (r.end_point.gathered_on -
>>> r.start_point.gathered_on < datetime.timedelta(days=1)) ]
>>>
>>> From the timing results I got that the second query is always faster,
>>> with or without cache:
>>> Q_1: 0.273243904114
>>> Q_1 with cache: 0.0182011127472
>>> Q_2: 0.250607967377
>>> Q_2 with cache: 0.0158171653748
>>>
>>> Beside the fact that they are just a few milliseconds of difference and
>>> that all the rows satisfy the constrain, what is not clear to me is why
>>> even when the cache is enabled the first query is taking longer. The
>>> question that came to my mind is about computed columns, are they cached?
>>>
>>> Paolo
>>>
>>  --
>>
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "web2py-users" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/web2py/9RfFmNQc-tE/unsubscribe?hl=en.
>> To unsubscribe from this group and all its topics, send an email to
>> web2py+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: Cache performance with computed columns

[paolo.vall...@gmail.com]

 Paolo


2013/3/25 Niphlod 

> I didn't get the point. 
> Q1 is slower (obviously) cause of the epoch() function and the additional
> filtering
> Q1 with cache shows more or less the same duration as Q2 with cache.
> the key point is "even when the cache is enabled the first query is taking
> longer" ... longer than what ?
> Q1 with cache is speedier that:
> - Q1 without cache, and that's ok
> - Q2 without cache, and that's ok
> but it's slower than
> - Q2 with cache
> however 3 ms (assuming those timings are in seconds) with all the moving
> parts of a memcache instance seems unfair to blame entirely on web2py
>
>
>
> On Monday, March 25, 2013 9:48:14 AM UTC+1, Paolo valleri wrote:
>>
>> Dear all, I want to share with you some results. I implemented a query in
>> two different ways. Given the following common code:
>> start = db.record.with_alias('start_**point')
>> end = db.record.with_alias('end_**point')
>> elapsed_time = end.gathered_on.epoch() - start.gathered_on.epoch()
>>
>> The first query is (the constrain is in the query):
>> rows = db( query & (elapsed_time  < 86400) ).select(
>> start.ALL,
>> end.ALL,
>>  start.gathered_on.epoch(),
>> end.gathered_on.epoch(),
>> elapsed_time,
>>  orderby=start.gathered_on.**epoch(),
>> left=start.on( (start.mac == end.mac) & (start.gathered_on <
>> end.gathered_on)),
>>  cache=(cache.memcache, 3600),
>> cacheable = True
>> )
>> The second one is (the constrain is explicitly tested latter):
>> rows = db( query ).select(
>> start.ALL,
>> end.ALL,
>> start.gathered_on.epoch(),
>>  end.gathered_on.epoch(),
>> elapsed_time,
>> orderby=start.gathered_on.**epoch(),
>>  left=start.on( (start.mac == end.mac) & (start.gathered_on <
>> end.gathered_on)),
>> cache=(cache.memcache, 3600),
>>  cacheable = True
>> )
>> rows2 = [r for r in rows if (r.end_point.gathered_on -
>> r.start_point.gathered_on < datetime.timedelta(days=1)) ]
>>
>> From the timing results I got that the second query is always faster,
>> with or without cache:
>> Q_1: 0.273243904114
>> Q_1 with cache: 0.0182011127472
>> Q_2: 0.250607967377
>> Q_2 with cache: 0.0158171653748
>>
>> Beside the fact that they are just a few milliseconds of difference and
>> that all the rows satisfy the constrain, what is not clear to me is why
>> even when the cache is enabled the first query is taking longer. The
>> question that came to my mind is about computed columns, are they cached?
>>
>> Paolo
>>
>  --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/9RfFmNQc-tE/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: wiki: how to cache media files?

[paolo.vall...@gmail.com]

Niphlod has written exactly what I have in mind.
Given the fact that distinct requests from the same user/session are
serialized as Massimo has outlined from my point of view we should add
session.forget() or session._unlock(request) as early as possible in the
download() function of globals.py
@Massimo, which are the difference between session.forget(request) and
session._unlock(request)? Which one is the better to use in this case?

 Paolo


2013/2/5 Niphlod 

>
>
> On Tuesday, February 5, 2013 5:30:11 PM UTC+1, Massimo Di Pierro wrote:
>>
>> Not sure I understand the question.
>
>
> I think the doubt lies into the fact that given 1 request for the wiki
> page and ,e.g., 4 requests for 4 embedded images the 5 requests would be
> concurrent or serialized. In my apps whenever I don't use session I put at
> the top a session.forget(), and I know for a fact that static files are
> handled concurrently (they are outside session logic in main.py).
>
> In theory to download something (either in the default download() or in
> the catch of the wiki) session is not touched, so in case the requests are
> in fact serialized a session.forget() could/should be added.
>
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: wiki: how to cache media files?

[paolo.vall...@gmail.com]

I am wondering to understand if from the theoretical point of view is true
or not that parallel requests are not handled in parallel. In the meanwhile
I will try to monitor the gain that we can have by session.forget in all
the downloads().

 Paolo


2013/2/5 Niphlod 

>
>
> On Tuesday, February 5, 2013 2:49:24 PM UTC+1, Paolo valleri wrote:
>>
>> Great! I've just tested it and it seems to work very well, I am getting
>> the 304 that I was looking for. Now many resources are no longer sent to
>> the client saving bandwidth, I think many web2py applications will
>> appreciate this speed up.
>>
>> @Niphlod, the header you have mentioned will force to keep the same file
>> even if I update it, this is not the behaviour that I need in my app.
>>
>
> Tune it s you like. From my POV an embedded media never changes. What
> changes is the media itself (i.e. if I want to change the image in a page,
> I upload a new image and change the pointers to it).
> If it's not your usecase, send cache headers at least one day in the
> future to avoid users scanning through your wiki to request media again and
> again.
>
>
>>
>> One more think about the download function.
>> Do we support parallel requests? Namely, I have recently read that the
>> default web2py approach is to serializes parallel ajax requests due to the
>> session lock. If that is true for download as well, I think we should add
>> an session.forget somwhere.
>> What do you think?
>>
>
> Didn't test it, but you can of course. measure it with
> session.forget() in the if zero.isdigit() and without
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: wiki: how to cache media files?

[paolo.vall...@gmail.com]

Great! I've just tested it and it seems to work very well, I am getting the
304 that I was looking for. Now many resources are no longer sent to the
client saving bandwidth, I think many web2py applications will appreciate
this speed up.

@Niphlod, the header you have mentioned will force to keep the same file
even if I update it, this is not the behaviour that I need in my app.

One more think about the download function.
Do we support parallel requests? Namely, I have recently read that the
default web2py approach is to serializes parallel ajax requests due to the
session lock. If that is true for download as well, I think we should add
an session.forget somwhere.
What do you think?


 Paolo


2013/2/4 Massimo Di Pierro 

> In trunk!
>
>
> On Monday, 4 February 2013 15:05:05 UTC-6, Niphlod wrote:
>>
>> Uhm, I was just thinking to return a string instead of the open stream.
>> attached the patch (some refactoring bits added)
>>
>> @Paolo: turns out that wiki has it's own "download()" function to handle
>> files . in order to return cache headers you should do:
>>
>> def yourwikiindex():
>> zero = request.args(0) or 'index'
>> if zero and zero.isdigit():
>> response.headers['Cache-**Control'] = 'max-age=31536'
>> response.headers['Expires'] = 'Thu, 31 Dec 2037 23:59:59 GMT'
>> return auth.wiki()
>>
>>
>>
>> On Monday, February 4, 2013 7:34:58 PM UTC+1, Massimo Di Pierro wrote:
>>>
>>> Honestly I would like to refactor all this logic in web2py. It was
>>> already done for web3py:
>>>
>>> https://github.com/mdipierro/**web3py/blob/master/web3py/**stream.py
>>>
>>> Anyway, if you can help fix it in web2py, we can merge everything later
>>>
>>> Massimo
>>>
>>> On Monday, 4 February 2013 11:46:03 UTC-6, Paolo valleri wrote:

 I thought the same about postponing the opening of the file. Moreover,
 just after globals:344 in the stream(...) function I have added

 if isinstance(stream, file):
 stream_file_or_304_or_206(stre**am.name
 ,
   **chunk_size=chunk_size,
   **request=request,
   **headers=headers,
   **status=self.status)

 and finally I got a 304 :-). As a result we are on the right direction,
 although there is no reason to open the file and take only its name. We
 need definitely something better



  Paolo


 2013/2/4 Niphlod 

>
>
> On Monday, February 4, 2013 3:38:29 PM UTC+1, Paolo valleri wrote:
>>
>> Hi Niphlod, I am still trying to understand why the wiki uploads are
>> not returned as 304.
>> The function in charged to return either 200 or 304 is
>> stream_file_or_304_or_206 in streamer.py. Moreover, this function is only
>> called in globals.py inside stream(...). Please have a look at
>> globals.py:344, In order to be called the parameter 'stream' must be a
>> string but with 'common' downloads the stream is an open file. This is 
>> also
>> documented by the comment in dal.py:9096. is what I've written correct ?
>>
>
> seems so, but in this case I think it should be patched (as soon as
> I'll get home).
>
> For downloads that on the table are merely strings (i.e. on the table
> there's the path to the actual file, usually in the "uploads" folder) the
> correct way should be to "defer" the opening of the file down the line to
> gluon.streamer.
> A lot of logic has been added to store files on pyfilesystem (and
> without much tests on that, I hope I won't brake anything) but I think 
> that
> for default "upload" file the logic needs to be revisited.
>
>  --
>
> ---
> You received this message because you are subscribed to the Google
> Groups "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to web2py+un...@googlegroups.com.
> For more options, visit 
> https://groups.google.com/**groups/opt_out
> .
>
>
>

  --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: wiki: how to cache media files?

[paolo.vall...@gmail.com]

I thought the same about postponing the opening of the file. Moreover, just
after globals:344 in the stream(...) function I have added

if isinstance(stream, file):
stream_file_or_304_or_206(stream.name,
  chunk_size=chunk_size,
  request=request,
  headers=headers,
  status=self.status)

and finally I got a 304 :-). As a result we are on the right direction,
although there is no reason to open the file and take only its name. We
need definitely something better



 Paolo


2013/2/4 Niphlod 

>
>
> On Monday, February 4, 2013 3:38:29 PM UTC+1, Paolo valleri wrote:
>>
>> Hi Niphlod, I am still trying to understand why the wiki uploads are not
>> returned as 304.
>> The function in charged to return either 200 or 304 is
>> stream_file_or_304_or_206 in streamer.py. Moreover, this function is only
>> called in globals.py inside stream(...). Please have a look at
>> globals.py:344, In order to be called the parameter 'stream' must be a
>> string but with 'common' downloads the stream is an open file. This is also
>> documented by the comment in dal.py:9096. is what I've written correct ?
>>
>
> seems so, but in this case I think it should be patched (as soon as I'll
> get home).
>
> For downloads that on the table are merely strings (i.e. on the table
> there's the path to the actual file, usually in the "uploads" folder) the
> correct way should be to "defer" the opening of the file down the line to
> gluon.streamer.
> A lot of logic has been added to store files on pyfilesystem (and without
> much tests on that, I hope I won't brake anything) but I think that for
> default "upload" file the logic needs to be revisited.
>
>  --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: wiki: how to cache media files?

[paolo.vall...@gmail.com]

Hi Niphlod, I am still trying to understand why the wiki uploads are not
returned as 304.
The function in charged to return either 200 or 304 is
stream_file_or_304_or_206 in streamer.py. Moreover, this function is only
called in globals.py inside stream(...). Please have a look at
globals.py:344, In order to be called the parameter 'stream' must be a
string but with 'common' downloads the stream is an open file. This is also
documented by the comment in dal.py:9096. is what I've written correct ?



 Paolo


2013/2/1 Niphlod 

> "modified_on" is basically the same info that is the mtime on the file.
> This tells you when the file has been modified, so you can return a 304,
> but doesn't help you with the cache. In theory all files served through
> download() (minus the one stored on a blob in a table) should work ok
> without needing any further patch (i.e. "the automation is there")
>
> What is needed to set cache headers (and avoid the browser to request
> files) is some sort of "will_be_modified_on", so you can set correctly the
> expiration without "guessing".
>
>
> On Friday, February 1, 2013 5:42:59 PM UTC+1, Paolo valleri wrote:
>
>> Something that handles the headers automatically accordingly to the wiki
>> media modified_on field. I will try somenthing next week (i am out this
>> weekend)
>> Il giorno 01/feb/2013 16:59, "Niphlod"  ha scritto:
>>
>> what "automation" ?
>>>
>>> On Friday, February 1, 2013 4:51:54 PM UTC+1, Paolo valleri wrote:

 I get it,thank for explaing.
 I tryed a test,it seems that main.py overwrites the default headers
 setting them as no-cache,no-store and so on. I have to test it more.
 Moreover wiki media are handled by users,i don't know how they behave
 with them. Something 'automatic' would be really nice to have.
 Paolo
 Il giorno 01/feb/2013 15:47, "Niphlod"  ha scritto:

> let me explain better (PS: untested but "should" work that way)
>
> every file that is returned by the download() function does not carry
> any of the cache headers (to be fair, it includes one that basically says
> "it expires now")
>
> 304 + no content or 200 + the entire content is a step "lower"  
> when a request is made with the If-Modified-Since header will check the
> mtime of the file and response.stream() will handle that (the logic is
> defined in gluon/streamer.py), returning 304 if the file has not been
> modified or a 200 if the file is indeed more recent.
>
> However, cache headers are useful to avoid even the requests that will
> end in a probable 304.
> In a static wiki with images, if you're sure that the published images
> will never change, you can enforce cache headers that will set the item to
> expire in some point in the future  when cache headers are returned 
> the
> browser will avoid a request with the if-modified-since header entirely.
>
> To return cache headers, you can use something like (taken from static
> asset management in gluon/main.py)
>
> response.headers['Cache-**Contro**l'] = 'max-age=31536'
> response.headers['Expires'] = 'Thu, 31 Dec 2037 23:59:59 GMT'
>
> This will ensure that the item would not been requested until 31 Dec
> 2037 (the effect is that a user will download the image one time only and
> will never request that image to the server again).
>
> On Friday, February 1, 2013 1:40:07 PM UTC+1, Paolo valleri wrote:
>>
>> Hi Niphlod, I don't know very well which header I have to set, can
>> you give me an example?
>> The 304 is sent by the server to the user accordingly to the user's
>> request.
>> Given that, If my upload file as a modified_on field I can actually
>> understand if I have to return the whole file (http: 200) or a 304.
>> Is it correct?
>> If yes, I would propose to implement something like that for
>> download():
>> if the field as the modified_on field we can decide between 200/304,
>> is the field doesn't have the modified_on field we return always a 200.
>>
>>  Paolo
>>
>>
>> 2013/2/1 Niphlod 
>>
>>> you have to alter the default download() function to return cache
>>> headers.
>>> The "theoretical" problem is that web2py (and then the wiki) doesn't
>>> know when the image stored in a table would be updated, so it doesn't 
>>> issue
>>> any cache headers by default.
>>>
>>> On Thursday, January 31, 2013 9:49:21 PM UTC+1, Paolo valleri wrote:

 Hi all,
 today I discovered that wiki media, actually they are all images,
 they are always requested to the server and never cached by the 
 client. In
 other words, even if I have already visited the page I do a get to 
 download
 all the images instead of using those in the browser cache as commonly
 happens with for js or css. Is it a problem of m

Re: [web2py] Re: wiki: how to cache media files?

[paolo.vall...@gmail.com]

Something that handles the headers automatically accordingly to the wiki
media modified_on field. I will try somenthing next week (i am out this
weekend)
Il giorno 01/feb/2013 16:59, "Niphlod"  ha scritto:

> what "automation" ?
>
> On Friday, February 1, 2013 4:51:54 PM UTC+1, Paolo valleri wrote:
>>
>> I get it,thank for explaing.
>> I tryed a test,it seems that main.py overwrites the default headers
>> setting them as no-cache,no-store and so on. I have to test it more.
>> Moreover wiki media are handled by users,i don't know how they behave
>> with them. Something 'automatic' would be really nice to have.
>> Paolo
>> Il giorno 01/feb/2013 15:47, "Niphlod"  ha scritto:
>>
>>> let me explain better (PS: untested but "should" work that way)
>>>
>>> every file that is returned by the download() function does not carry
>>> any of the cache headers (to be fair, it includes one that basically says
>>> "it expires now")
>>>
>>> 304 + no content or 200 + the entire content is a step "lower"  
>>> when a request is made with the If-Modified-Since header will check the
>>> mtime of the file and response.stream() will handle that (the logic is
>>> defined in gluon/streamer.py), returning 304 if the file has not been
>>> modified or a 200 if the file is indeed more recent.
>>>
>>> However, cache headers are useful to avoid even the requests that will
>>> end in a probable 304.
>>> In a static wiki with images, if you're sure that the published images
>>> will never change, you can enforce cache headers that will set the item to
>>> expire in some point in the future  when cache headers are returned the
>>> browser will avoid a request with the if-modified-since header entirely.
>>>
>>> To return cache headers, you can use something like (taken from static
>>> asset management in gluon/main.py)
>>>
>>> response.headers['Cache-**Control'] = 'max-age=31536'
>>> response.headers['Expires'] = 'Thu, 31 Dec 2037 23:59:59 GMT'
>>>
>>> This will ensure that the item would not been requested until 31 Dec
>>> 2037 (the effect is that a user will download the image one time only and
>>> will never request that image to the server again).
>>>
>>> On Friday, February 1, 2013 1:40:07 PM UTC+1, Paolo valleri wrote:

 Hi Niphlod, I don't know very well which header I have to set, can you
 give me an example?
 The 304 is sent by the server to the user accordingly to the user's
 request.
 Given that, If my upload file as a modified_on field I can actually
 understand if I have to return the whole file (http: 200) or a 304.
 Is it correct?
 If yes, I would propose to implement something like that for
 download():
 if the field as the modified_on field we can decide between 200/304, is
 the field doesn't have the modified_on field we return always a 200.

  Paolo


 2013/2/1 Niphlod 

> you have to alter the default download() function to return cache
> headers.
> The "theoretical" problem is that web2py (and then the wiki) doesn't
> know when the image stored in a table would be updated, so it doesn't 
> issue
> any cache headers by default.
>
> On Thursday, January 31, 2013 9:49:21 PM UTC+1, Paolo valleri wrote:
>>
>> Hi all,
>> today I discovered that wiki media, actually they are all images,
>> they are always requested to the server and never cached by the client. 
>> In
>> other words, even if I have already visited the page I do a get to 
>> download
>> all the images instead of using those in the browser cache as commonly
>> happens with for js or css. Is it a problem of my configuration or of 
>> wiki
>> it self?
>>
>> paolo
>>
>  --
>
> ---
> You received this message because you are subscribed to the Google
> Groups "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to web2py+un...@**googlegroups.com.
> For more options, visit 
> https://groups.google.com/**grou**ps/opt_out
> .
>
>
>

  --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "web2py-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to web2py+un...@**googlegroups.com.
>>> For more options, visit 
>>> https://groups.google.com/**groups/opt_out
>>> .
>>>
>>>
>>>
>>  --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group 

Re: [web2py] Re: wiki: how to cache media files?

[paolo.vall...@gmail.com]

I get it,thank for explaing.
I tryed a test,it seems that main.py overwrites the default headers setting
them as no-cache,no-store and so on. I have to test it more.
Moreover wiki media are handled by users,i don't know how they behave with
them. Something 'automatic' would be really nice to have.
Paolo
Il giorno 01/feb/2013 15:47, "Niphlod"  ha scritto:

> let me explain better (PS: untested but "should" work that way)
>
> every file that is returned by the download() function does not carry any
> of the cache headers (to be fair, it includes one that basically says "it
> expires now")
>
> 304 + no content or 200 + the entire content is a step "lower"   when
> a request is made with the If-Modified-Since header will check the mtime of
> the file and response.stream() will handle that (the logic is defined in
> gluon/streamer.py), returning 304 if the file has not been modified or a
> 200 if the file is indeed more recent.
>
> However, cache headers are useful to avoid even the requests that will end
> in a probable 304.
> In a static wiki with images, if you're sure that the published images
> will never change, you can enforce cache headers that will set the item to
> expire in some point in the future  when cache headers are returned the
> browser will avoid a request with the if-modified-since header entirely.
>
> To return cache headers, you can use something like (taken from static
> asset management in gluon/main.py)
>
> response.headers['Cache-Control'] = 'max-age=31536'
> response.headers['Expires'] = 'Thu, 31 Dec 2037 23:59:59 GMT'
>
> This will ensure that the item would not been requested until 31 Dec 2037
> (the effect is that a user will download the image one time only and will
> never request that image to the server again).
>
> On Friday, February 1, 2013 1:40:07 PM UTC+1, Paolo valleri wrote:
>>
>> Hi Niphlod, I don't know very well which header I have to set, can you
>> give me an example?
>> The 304 is sent by the server to the user accordingly to the user's
>> request.
>> Given that, If my upload file as a modified_on field I can actually
>> understand if I have to return the whole file (http: 200) or a 304.
>> Is it correct?
>> If yes, I would propose to implement something like that for download():
>> if the field as the modified_on field we can decide between 200/304, is
>> the field doesn't have the modified_on field we return always a 200.
>>
>>  Paolo
>>
>>
>> 2013/2/1 Niphlod 
>>
>>> you have to alter the default download() function to return cache
>>> headers.
>>> The "theoretical" problem is that web2py (and then the wiki) doesn't
>>> know when the image stored in a table would be updated, so it doesn't issue
>>> any cache headers by default.
>>>
>>> On Thursday, January 31, 2013 9:49:21 PM UTC+1, Paolo valleri wrote:

 Hi all,
 today I discovered that wiki media, actually they are all images, they
 are always requested to the server and never cached by the client. In other
 words, even if I have already visited the page I do a get to download all
 the images instead of using those in the browser cache as commonly happens
 with for js or css. Is it a problem of my configuration or of wiki it self?

 paolo

>>>  --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "web2py-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to web2py+un...@**googlegroups.com.
>>> For more options, visit 
>>> https://groups.google.com/**groups/opt_out
>>> .
>>>
>>>
>>>
>>
>>  --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: wiki: how to cache media files?

[paolo.vall...@gmail.com]

Hi Niphlod, I don't know very well which header I have to set, can you give
me an example?
The 304 is sent by the server to the user accordingly to the user's
request.
Given that, If my upload file as a modified_on field I can actually
understand if I have to return the whole file (http: 200) or a 304.
Is it correct?
If yes, I would propose to implement something like that for download():
if the field as the modified_on field we can decide between 200/304, is the
field doesn't have the modified_on field we return always a 200.

 Paolo


2013/2/1 Niphlod 

> you have to alter the default download() function to return cache headers.
> The "theoretical" problem is that web2py (and then the wiki) doesn't know
> when the image stored in a table would be updated, so it doesn't issue any
> cache headers by default.
>
> On Thursday, January 31, 2013 9:49:21 PM UTC+1, Paolo valleri wrote:
>>
>> Hi all,
>> today I discovered that wiki media, actually they are all images, they
>> are always requested to the server and never cached by the client. In other
>> words, even if I have already visited the page I do a get to download all
>> the images instead of using those in the browser cache as commonly happens
>> with for js or css. Is it a problem of my configuration or of wiki it self?
>>
>> paolo
>>
>  --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: auth.wiki how to format all fields

[paolo.vall...@gmail.com]

for me is ok too.

 Paolo


2013/1/30 Massimo Di Pierro 

> I am suggesting the opposite. Not wrapping any field in XML.
>
>
> On Wednesday, January 30, 2013 1:50:40 PM UTC-6, Paolo valleri wrote:
>
>> I agree too in breaking the backward compatibility as soon as is marked
>> experimental, however I don't see any other field that can be wrapped
>> around XML.
>>
>>  Paolo
>>
>>
>> 2013/1/30 Alan Etkin 
>>
>>> > I have applied the patch. I am uneasy with the fact that only one
>>> field is wrapped around XML and the others are not.
>>>
>>> Agree.
>>>
>>> I would make any change necessary to have a more clean api while it is
>>> an experimental feature.
>>>
>>>  --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "web2py-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to web2py+un...@**googlegroups.com.
>>>
>>> For more options, visit 
>>> https://groups.google.com/**groups/opt_out
>>> .
>>>
>>>
>>>
>>
>>  --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: auth.wiki how to format all fields

[paolo.vall...@gmail.com]

I agree too in breaking the backward compatibility as soon as is marked
experimental, however I don't see any other field that can be wrapped
around XML.

 Paolo


2013/1/30 Alan Etkin 

> > I have applied the patch. I am uneasy with the fact that only one field
> is wrapped around XML and the others are not.
>
> Agree.
>
> I would make any change necessary to have a more clean api while it is an
> experimental feature.
>
>  --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [web2py] Re: auth.wiki how to format all fields

[paolo.vall...@gmail.com]

Hi Massimo, I would avoid to write explicit code, please find attached a
potential patch. what about that?



 Paolo


2013/1/30 Massimo Di Pierro 

> db.auth_wiki(slug=slug).title ?
>
>
> On Sunday, 27 January 2013 11:46:47 UTC-6, Paolo valleri wrote:
>
>> The problem is that with the current approach i set the title field but i
>> have not found yer the possibility to render it :-(
>>
>> Looking to the code a very dirty but working workaround is to unset the
>> current.request.extension. get the wiki and set extension as before. In
>> this way i get all fields see gluon/tools.py:5090-5107. I think that we
>> just need to change a little bit the current read.
>>
>> Paolo
>> Il giorno 27/gen/2013 18:30, "Massimo Di Pierro" 
>> ha scritto:
>>
>> I agree that is what it does and that is what is suposed to do. Paolo
>>> tells us it should work differently. I think that would be best done with a
>>> different API.
>>>
>>> Massimo
>>>
>>>
 def wiki():
 return auth.wiki() # no "render" argument

 The output is:

 {'content': }

 I guess this is the normal behavior for all .wiki calls (just the
 content).

>>>  --
>>>
>>>
>>>
>>>
>>   --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.




tools.py.patch
Description: Binary data

Re: [web2py] Re: auth.wiki how to format all fields

[paolo.vall...@gmail.com]

The problem is that with the current approach i set the title field but i
have not found yer the possibility to render it :-(

Looking to the code a very dirty but working workaround is to unset the
current.request.extension. get the wiki and set extension as before. In
this way i get all fields see gluon/tools.py:5090-5107. I think that we
just need to change a little bit the current read.

Paolo
Il giorno 27/gen/2013 18:30, "Massimo Di Pierro" 
ha scritto:

> I agree that is what it does and that is what is suposed to do. Paolo
> tells us it should work differently. I think that would be best done with a
> different API.
>
> Massimo
>
>
>> def wiki():
>> return auth.wiki() # no "render" argument
>>
>> The output is:
>>
>> {'content': }
>>
>> I guess this is the normal behavior for all .wiki calls (just the
>> content).
>>
>  --
>
>
>
>

-- 

Re: [web2py] Re: Auth with migrate=False

[paolo.vall...@gmail.com]

Hi Massimo, just wondering if you have found something better for this.
Paolo


2013/1/17 Massimo Di Pierro 

> Me neither. Let's think of a solution on web2py_developers.
>
>
> On Wednesday, 16 January 2013 01:59:27 UTC-6, Paolo valleri wrote:
>>
>> Hi Massimo, personally I don't like this:
>>
>>> db._LAZY_TABLES['**yourtablename'][2]['on_define'**] = 
>>>
>> it is neither easy to remember nor easy to understand what it dows.
>> Something more friendly would be better from my point of view, If adding an
>> on_define could create misunderstanding since the parameter is different
>> from the 'common' on_define, we could use an other totally fresh name,
>> i.e., auth.set_settings, auth.customize and so on to do the same thing.
>>
>> --
>>  Paolo
>>
>  --
>
>
>
>



-- 
 Paolo

-- 

Re: [web2py] Re: new setup-web2py-nginx-uwsgi-ubuntu.sh

[paolo.vall...@gmail.com]

yep, I've added it under the location / and it worked. You are right it
might be placed in the main configuration, on the other hand today at work
I was blocked because unable to upload files greaten than 1M. From my point
of view we should at least mention it. For what concerns gzip is that on
ubuntu server I can see that is already enabled by default.
paolo


2013/1/23 niphlod 

> uhm. this is outside the scope of the location  I mean... probably
> you'd like to turn on gzip too, but that is a feature to enable on nginx
> "main" conf, not in the "web2py" one 
> Anyway, did you put this setting inside the file
> /etc/nginx/sites-available/web2py and it just works ok ?
>
>
> 2013/1/23 paolo.vall...@gmail.com 
>
>> In the meanwhile I discovered that if you upload a file greater than 1M,
>> I tried with a 1.5M, you will get a 413 error. In order to avoid this error
>> for file not so big, I suggest you to add the directive *client_max_body_size
>> *and to set it at something more 'common' for me few MB is already ok.
>> see here: http://wiki.nginx.org/HttpCoreModule#client_max_body_size
>> paolo
>>
>>
>>
>> 2013/1/23 Niphlod 
>>
>>> ok, I'll test it more @home and then add a commented section to the
>>> script
>>>
>>> PS: that site you posted . turn off response.optimize_js = 'inline'
>>> pleeease :P
>>>
>>>
>>> On Wednesday, January 23, 2013 11:19:34 AM UTC+1, Paolo valleri wrote:
>>>
>>>> Just tested, it works, I've already upgraded the production server too
>>>> :-P
>>>> I would suggest to add it (maybe as a comment) to the script.
>>>> Thanks, Paolo
>>>>
>>>>
>>>> 2013/1/22 Niphlod 
>>>>
>>>> maybe I have a fix .
>>>>> Can you please test it ?
>>>>> assuming an example of routes.py like
>>>>>
>>>>> myapp = dict(languages=['en', 'it', 'jp'], default_language='en')
>>>>>
>>>>>
>>>>> this means that navigating to:
>>>>>  /myapp/ --> the static files will be referenced as
>>>>> web2py_home/applications/**myapp/en/static/whatever.css
>>>>>  /myapp/it/ --> the static files will be referenced as
>>>>> web2py_home/applications/**myapp/it/static/whatever.css
>>>>>
>>>>> however, direct requests to /myapp/static/whatever.css should map to
>>>>> web2py_home/applications/**myapp/en/static/whatever.css
>>>>>
>>>>> additionally, if /myapp/it/static/whatever.css is requested, if not
>>>>> found web2py_home/applications/**myapp/static/whatever.css should be
>>>>> served.
>>>>>
>>>>> Now, I think that adding this section before the standard one will fix
>>>>> the issue
>>>>>
>>>>> location ~* /(\w+)/(en|it|jp)/static/(.*)$ {
>>>>> alias /home/www-data/web2py/applicat**ions/$1/;
>>>>> try_files static/$2/$3 static/$3 =404;
>>>>> }
>>>>>
>>>>> (en|it|jp) here is the regex matching the "languages" list of the
>>>>> routes app's dict  This will set for the request
>>>>> /myapp/it/static/whatever.css   the alias to the web2py_home/applications/
>>>>> **myapp/ and then try to serve static/it/whatever.css, if not found,
>>>>> it will fall back to a "normal" static/whatever.css, and if that one is 
>>>>> not
>>>>> found will return a 404.
>>>>>
>>>>> @Jonathan: please stop me if I did miss something in the routes logic
>>>>> @all : seems a simple fix - maybe too simple... please help test it
>>>>>
>>>>>
>>>>> On Tuesday, January 22, 2013 9:14:26 PM UTC+1, Niphlod wrote:
>>>>>>
>>>>>> got it. can you post the routes.py you're using ?
>>>>>>
>>>>>> The problem lies in the fact that routes.py is really flexible and
>>>>>> adapting that logic using only rewrite or alias statements in nginx is
>>>>>> cumbersome.
>>>>>> Standing on one feet (i.e. without tests) I'd say that the regex
>>>>>> checking static files
>>>>>>
>>>>>> location ~* /(\w+)/static/ {
>>>>>> root 

Re: [web2py] Re: new setup-web2py-nginx-uwsgi-ubuntu.sh

[paolo.vall...@gmail.com]

In the meanwhile I discovered that if you upload a file greater than 1M, I
tried with a 1.5M, you will get a 413 error. In order to avoid this error
for file not so big, I suggest you to add the directive *client_max_body_size
*and to set it at something more 'common' for me few MB is already ok. see
here: http://wiki.nginx.org/HttpCoreModule#client_max_body_size
paolo


2013/1/23 Niphlod 

> ok, I'll test it more @home and then add a commented section to the script
>
> PS: that site you posted . turn off response.optimize_js = 'inline'
> pleeease :P
>
>
> On Wednesday, January 23, 2013 11:19:34 AM UTC+1, Paolo valleri wrote:
>
>> Just tested, it works, I've already upgraded the production server too :-P
>> I would suggest to add it (maybe as a comment) to the script.
>> Thanks, Paolo
>>
>>
>> 2013/1/22 Niphlod 
>>
>> maybe I have a fix .
>>> Can you please test it ?
>>> assuming an example of routes.py like
>>>
>>> myapp = dict(languages=['en', 'it', 'jp'], default_language='en')
>>>
>>>
>>> this means that navigating to:
>>>  /myapp/ --> the static files will be referenced as
>>> web2py_home/applications/**myapp/en/static/whatever.css
>>>  /myapp/it/ --> the static files will be referenced as
>>> web2py_home/applications/**myapp/it/static/whatever.css
>>>
>>> however, direct requests to /myapp/static/whatever.css should map to
>>> web2py_home/applications/**myapp/en/static/whatever.css
>>>
>>> additionally, if /myapp/it/static/whatever.css is requested, if not
>>> found web2py_home/applications/**myapp/static/whatever.css should be
>>> served.
>>>
>>> Now, I think that adding this section before the standard one will fix
>>> the issue
>>>
>>> location ~* /(\w+)/(en|it|jp)/static/(.*)$ {
>>> alias /home/www-data/web2py/applicat**ions/$1/;
>>> try_files static/$2/$3 static/$3 =404;
>>> }
>>>
>>> (en|it|jp) here is the regex matching the "languages" list of the routes
>>> app's dict  This will set for the request /myapp/it/static/whatever.css
>>>   the alias to the web2py_home/applications/**myapp/ and then try to
>>> serve static/it/whatever.css, if not found, it will fall back to a "normal"
>>> static/whatever.css, and if that one is not found will return a 404.
>>>
>>> @Jonathan: please stop me if I did miss something in the routes logic
>>> @all : seems a simple fix - maybe too simple... please help test it
>>>
>>>
>>> On Tuesday, January 22, 2013 9:14:26 PM UTC+1, Niphlod wrote:

 got it. can you post the routes.py you're using ?

 The problem lies in the fact that routes.py is really flexible and
 adapting that logic using only rewrite or alias statements in nginx is
 cumbersome.
 Standing on one feet (i.e. without tests) I'd say that the regex
 checking static files

 location ~* /(\w+)/static/ {
 root /home/www-data/web2py/**applicat**ions/;
 #remove next comment on production


 #expires max;
 }

 is catching it as the app name and not the language "trick" . so, for a 
 request going to /it/app/static/whatever.css is looking into



 /home/www-data/web2py/**applicat**ions/*it/*appname/**static/**whatever.css
  instead of  . (remove the bold part, "*it/*").

 If you remove those lines web2py will take charge for static files
 serving, so the issue will be temporarily fixed.

 I'll try to set up some additional rules to make nginx behave like
 routes.py, but it will take some time.

 PS: please mind that the script it's a template it's not meant to
 fullfill every custom installation patterns out there without further
 tuning.
 language routing is probably the most difficult out there because for a
 request
 /it/appname/static/**whateverfil**e.css
 web2py looks into /web2py/appname/static/it/**what**everfile.css and
 then if not found into /web2py/appname/static/**whateve**rfile.css (a
 nice fallback, but hard to map with a simple nginx statement)

 On Tuesday, January 22, 2013 6:14:55 PM UTC+1, Paolo valleri wrote:
>
> Hi Massimo, for example, if I visit this traffic.integreen-life.bz.it/
> **d**efault/wiki/about
>  (default is the controller and wiki is the function) it works while
> traffic.integreen-life.bz.it/**i**t/default/wiki/aboutthe
>  application works very well but the static links do not.
>
> Paolo
>
>
> 2013/1/22 Massimo Di Pierro 
>
>> can you show an example?
>>
>>
>> On Tuesday, 22 January 2013 07:36:39 UTC-6, Paolo valleri wrote:
>>
>>> Hi all, I've just discovered that the regex used to retrieve the
>>> static files doesn't work if I use languages abbreviation in urls.
>>> How can we fix this?
>>>
>>>
>>>
>>> 20

Re: [web2py] Re: new setup-web2py-nginx-uwsgi-ubuntu.sh

[paolo.vall...@gmail.com]

Just tested, it works, I've already upgraded the production server too :-P
I would suggest to add it (maybe as a comment) to the script.
Thanks, Paolo


2013/1/22 Niphlod 

> maybe I have a fix .
> Can you please test it ?
> assuming an example of routes.py like
>
> myapp = dict(languages=['en', 'it', 'jp'], default_language='en')
>
>
> this means that navigating to:
>  /myapp/ --> the static files will be referenced as
> web2py_home/applications/myapp/en/static/whatever.css
>  /myapp/it/ --> the static files will be referenced as
> web2py_home/applications/myapp/it/static/whatever.css
>
> however, direct requests to /myapp/static/whatever.css should map to
> web2py_home/applications/myapp/en/static/whatever.css
>
> additionally, if /myapp/it/static/whatever.css is requested, if not found
> web2py_home/applications/myapp/static/whatever.css should be served.
>
> Now, I think that adding this section before the standard one will fix the
> issue
>
> location ~* /(\w+)/(en|it|jp)/static/(.*)$ {
> alias /home/www-data/web2py/applications/$1/;
> try_files static/$2/$3 static/$3 =404;
> }
>
> (en|it|jp) here is the regex matching the "languages" list of the routes
> app's dict  This will set for the request /myapp/it/static/whatever.css
>   the alias to the web2py_home/applications/myapp/ and then try to serve
> static/it/whatever.css, if not found, it will fall back to a "normal"
> static/whatever.css, and if that one is not found will return a 404.
>
> @Jonathan: please stop me if I did miss something in the routes logic
> @all : seems a simple fix - maybe too simple... please help test it
>
>
> On Tuesday, January 22, 2013 9:14:26 PM UTC+1, Niphlod wrote:
>>
>> got it. can you post the routes.py you're using ?
>>
>> The problem lies in the fact that routes.py is really flexible and
>> adapting that logic using only rewrite or alias statements in nginx is
>> cumbersome.
>> Standing on one feet (i.e. without tests) I'd say that the regex checking
>> static files
>>
>> location ~* /(\w+)/static/ {
>> root /home/www-data/web2py/**applications/;
>> #remove next comment on production
>> #expires max;
>> }
>>
>> is catching it as the app name and not the language "trick" . so, for a 
>> request going to /it/app/static/whatever.css is looking into
>>
>> /home/www-data/web2py/**applications/*it/*appname/**static/whatever.css 
>> instead of  . (remove the bold part, "*it/*").
>>
>> If you remove those lines web2py will take charge for static files
>> serving, so the issue will be temporarily fixed.
>>
>> I'll try to set up some additional rules to make nginx behave like
>> routes.py, but it will take some time.
>>
>> PS: please mind that the script it's a template it's not meant to
>> fullfill every custom installation patterns out there without further
>> tuning.
>> language routing is probably the most difficult out there because for a
>> request
>> /it/appname/static/**whateverfile.css
>> web2py looks into /web2py/appname/static/it/**whateverfile.css and then
>> if not found into /web2py/appname/static/**whateverfile.css (a nice
>> fallback, but hard to map with a simple nginx statement)
>>
>> On Tuesday, January 22, 2013 6:14:55 PM UTC+1, Paolo valleri wrote:
>>>
>>> Hi Massimo, for example, if I visit this traffic.integreen-life.bz.it/**
>>> default/wiki/about 
>>> (default is the controller and wiki is the function) it works while
>>> traffic.integreen-life.bz.it/**it/default/wiki/aboutthe
>>>  application works very well but the static links do not.
>>>
>>> Paolo
>>>
>>>
>>> 2013/1/22 Massimo Di Pierro 
>>>
 can you show an example?


 On Tuesday, 22 January 2013 07:36:39 UTC-6, Paolo valleri wrote:

> Hi all, I've just discovered that the regex used to retrieve the
> static files doesn't work if I use languages abbreviation in urls.
> How can we fix this?
>
>
>
> 2012/12/28 Richard Vézina 
>
>>  Hello,
>>
>> I publish a new script that allow deployment of Redmine beside web2py.
>>
>> Here : https://groups.google.com/**fo**rum/?fromgroups=#!searchin/**
>> web**2py/redmine/web2py/**ZqL7Si8Khbo**/Es-wK1yXdgQJ
>>
>> Notice : After some read, I choose Unicorn over Phussion Passenger.
>>
>> Richard
>>
>>
>> On Thu, Dec 20, 2012 at 9:57 AM, Massimo Di Pierro <
>> massimo@gmail.com> wrote:
>>
>>> please email me the patch of latest file when ready for inclusion.
>>>
>>>
>>> On Thursday, 20 December 2012 03:18:20 UTC-6, Niphlod wrote:

 perfect, Ccing Massimo on this final one.

 Il giorno giovedì 20 dicembre 2012 00:16:25 UTC+1, Paolo ha scritto:
>>>

Re: [web2py] Re: new setup-web2py-nginx-uwsgi-ubuntu.sh

[paolo.vall...@gmail.com]

Hi Niphlod, once I discovered that making a symbolic link for each language
in the app directory pointing to the app directory worked well.
Namely something like that app_dir/it ---> app_dir . However  it would be
better to find something cleaner but for that I have to look at nginx but
right now I don't access to the virtual machine.
I guess we can add a new directive for serving static files with language
in url. will it works?
Paolo


2013/1/22 Niphlod 

> got it. can you post the routes.py you're using ?
>
> The problem lies in the fact that routes.py is really flexible and
> adapting that logic using only rewrite or alias statements in nginx is
> cumbersome.
> Standing on one feet (i.e. without tests) I'd say that the regex checking
> static files
>
> location ~* /(\w+)/static/ {
> root /home/www-data/web2py/applications/;
> #remove next comment on production
> #expires max;
> }
>
> is catching it as the app name and not the language "trick" . so, for a 
> request going to /it/app/static/whatever.css is looking into
>
> /home/www-data/web2py/applications/*it/*appname/static/whatever.css instead 
> of  . (remove the bold part, "*it/*").
>
> If you remove those lines web2py will take charge for static files
> serving, so the issue will be temporarily fixed.
>
> I'll try to set up some additional rules to make nginx behave like
> routes.py, but it will take some time.
>
> PS: please mind that the script it's a template it's not meant to
> fullfill every custom installation patterns out there without further
> tuning.
> language routing is probably the most difficult out there because for a
> request
> /it/appname/static/whateverfile.css
> web2py looks into /web2py/appname/static/it/whateverfile.css and then if
> not found into /web2py/appname/static/whateverfile.css (a nice fallback,
> but hard to map with a simple nginx statement)
>
>
> On Tuesday, January 22, 2013 6:14:55 PM UTC+1, Paolo valleri wrote:
>
>> Hi Massimo, for example, if I visit this traffic.integreen-life.bz.it/**
>> default/wiki/about 
>> (default is the controller and wiki is the function) it works while
>> traffic.integreen-life.bz.it/**it/default/wiki/aboutthe
>>  application works very well but the static links do not.
>>
>> Paolo
>>
>>
>> 2013/1/22 Massimo Di Pierro 
>>
>> can you show an example?
>>>
>>>
>>> On Tuesday, 22 January 2013 07:36:39 UTC-6, Paolo valleri wrote:
>>>
 Hi all, I've just discovered that the regex used to retrieve the static
 files doesn't work if I use languages abbreviation in urls.
 How can we fix this?



 2012/12/28 Richard Vézina 

>  Hello,
>
> I publish a new script that allow deployment of Redmine beside web2py.
>
> Here : https://groups.google.com/**fo**rum/?fromgroups=#!searchin/**
> web**2py/redmine/web2py/**ZqL7Si8Khbo**/Es-wK1yXdgQJ
>
> Notice : After some read, I choose Unicorn over Phussion Passenger.
>
> Richard
>
>
> On Thu, Dec 20, 2012 at 9:57 AM, Massimo Di Pierro <
> massimo@gmail.com> wrote:
>
>> please email me the patch of latest file when ready for inclusion.
>>
>>
>> On Thursday, 20 December 2012 03:18:20 UTC-6, Niphlod wrote:
>>>
>>> perfect, Ccing Massimo on this final one.
>>>
>>> Il giorno giovedì 20 dicembre 2012 00:16:25 UTC+1, Paolo ha scritto:

 Hi
 I was trying with the script that comes with the stable web2py.
 with the one on dropbox I problem has gone.

 Paolo


 2012/12/19 Niphlod 

>
> @Simone, an other improvement to the script could be to combine
>> into a single server the 80, and 443 to avoid duplicating 
>> configuration, as
>> stated here :
>> http://nginx.org/en/docs/http/configuring_https_servers.*
>> *html**#single_http_https_server
>> I don't had test this.
>>
>> Also, here the code for permanent redirection
>> server {
>>   server_name $hostname;
>>   listen  80;
>>   return 301 http*s*://$hostname$request_uri; # NOTE: I am not
>> sure for $hostname here, because I didn't set hostname for my VM 
>> until now,
>> as in the example (URL below) we can use domainName.com instead if 
>> properly
>> configure in nginx
>> ...
>> }
>>
>>
> People may want separate configs for http and https. The script
> objective is to have a working copy of web2py. If we start to follow 
> 

Re: [web2py] Re: new setup-web2py-nginx-uwsgi-ubuntu.sh

[paolo.vall...@gmail.com]

Hi Massimo, for example, if I visit this
traffic.integreen-life.bz.it/default/wiki/about  (default is the controller
and wiki is the function) it works while
traffic.integreen-life.bz.it/it/default/wiki/about the application works
very well but the static links do not.

Paolo


2013/1/22 Massimo Di Pierro 

> can you show an example?
>
>
> On Tuesday, 22 January 2013 07:36:39 UTC-6, Paolo valleri wrote:
>
>> Hi all, I've just discovered that the regex used to retrieve the static
>> files doesn't work if I use languages abbreviation in urls.
>> How can we fix this?
>>
>>
>>
>> 2012/12/28 Richard Vézina 
>>
>>>  Hello,
>>>
>>> I publish a new script that allow deployment of Redmine beside web2py.
>>>
>>> Here : https://groups.google.com/**forum/?fromgroups=#!searchin/**
>>> web2py/redmine/web2py/**ZqL7Si8Khbo/Es-wK1yXdgQJ
>>>
>>> Notice : After some read, I choose Unicorn over Phussion Passenger.
>>>
>>> Richard
>>>
>>>
>>> On Thu, Dec 20, 2012 at 9:57 AM, Massimo Di Pierro <
>>> massimo@gmail.com> wrote:
>>>
 please email me the patch of latest file when ready for inclusion.


 On Thursday, 20 December 2012 03:18:20 UTC-6, Niphlod wrote:
>
> perfect, Ccing Massimo on this final one.
>
> Il giorno giovedì 20 dicembre 2012 00:16:25 UTC+1, Paolo ha scritto:
>>
>> Hi
>> I was trying with the script that comes with the stable web2py. with
>> the one on dropbox I problem has gone.
>>
>> Paolo
>>
>>
>> 2012/12/19 Niphlod 
>>
>>>
>>> @Simone, an other improvement to the script could be to combine into
 a single server the 80, and 443 to avoid duplicating configuration, as
 stated here :
 http://nginx.org/en/docs/http/**configuring_https_servers.**
 html#single_http_https_server
 I don't had test this.

 Also, here the code for permanent redirection
 server {
   server_name $hostname;
   listen  80;
   return 301 http*s*://$hostname$request_uri; # NOTE: I am not
 sure for $hostname here, because I didn't set hostname for my VM until 
 now,
 as in the example (URL below) we can use domainName.com instead if 
 properly
 configure in nginx
 ...
 }


>>> People may want separate configs for http and https. The script
>>> objective is to have a working copy of web2py. If we start to follow 
>>> such
>>> requests, we'd end up installing postgresql and redis too :P
>>>
>>> @Paolo: try the script found at the dropbox link. If the same error
>>> happens I think we need the log of pip and a hand by Roberto on the
>>> specific error...
>>>
>>> --
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>>  Paolo
>>
>  --




>>>
>>>  --
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>>  Paolo
>>
>  --
>
>
>
>



-- 
 Paolo

-- 

Re: [web2py] Re: new setup-web2py-nginx-uwsgi-ubuntu.sh

[paolo.vall...@gmail.com]

Hi all, I've just discovered that the regex used to retrieve the static
files doesn't work if I use languages abbreviation in urls.
How can we fix this?



2012/12/28 Richard Vézina 

> Hello,
>
> I publish a new script that allow deployment of Redmine beside web2py.
>
> Here :
> https://groups.google.com/forum/?fromgroups=#!searchin/web2py/redmine/web2py/ZqL7Si8Khbo/Es-wK1yXdgQJ
>
> Notice : After some read, I choose Unicorn over Phussion Passenger.
>
> Richard
>
>
> On Thu, Dec 20, 2012 at 9:57 AM, Massimo Di Pierro <
> massimo.dipie...@gmail.com> wrote:
>
>> please email me the patch of latest file when ready for inclusion.
>>
>>
>> On Thursday, 20 December 2012 03:18:20 UTC-6, Niphlod wrote:
>>>
>>> perfect, Ccing Massimo on this final one.
>>>
>>> Il giorno giovedì 20 dicembre 2012 00:16:25 UTC+1, Paolo ha scritto:

 Hi
 I was trying with the script that comes with the stable web2py. with
 the one on dropbox I problem has gone.

 Paolo


 2012/12/19 Niphlod 

>
> @Simone, an other improvement to the script could be to combine into a
>> single server the 80, and 443 to avoid duplicating configuration, as 
>> stated
>> here :
>> http://nginx.org/en/docs/http/configuring_https_servers.**html**
>> #single_http_https_server
>> I don't had test this.
>>
>> Also, here the code for permanent redirection
>> server {
>>   server_name $hostname;
>>   listen  80;
>>   return 301 http*s*://$hostname$request_uri; # NOTE: I am not sure
>> for $hostname here, because I didn't set hostname for my VM until now, as
>> in the example (URL below) we can use domainName.com instead if properly
>> configure in nginx
>> ...
>> }
>>
>>
> People may want separate configs for http and https. The script
> objective is to have a working copy of web2py. If we start to follow such
> requests, we'd end up installing postgresql and redis too :P
>
> @Paolo: try the script found at the dropbox link. If the same error
> happens I think we need the log of pip and a hand by Roberto on the
> specific error...
>
> --
>
>
>
>



 --
  Paolo

>>>  --
>>
>>
>>
>>
>
>  --
>
>
>
>



-- 
 Paolo

-- 

Re: [web2py] Re: problems with wiki create

[paolo.vall...@gmail.com]

Hi Alan, good catch,
I don't know how wiki works but if it is only to add a field to the table
and handle it during insert/update it is worth to do that. isn't it?
When a user is editing a page he/she can choose from a dropdow list the
render engine to use.

Paolo


2013/1/21 Alan Etkin 

> ... In the meanwhile I have opened a issue for having the render option at
>> page ...
>>
>
> Perhaps it's possible to add a .render record field and have the wiki
> collect the option from the record on document reads. When calling
> auth.wiki(render=...) the wiki object should set the .render field default
> attribute to the value passed (unless the page was retrieved from a query,
> in which case the record render has precedence). This would require adding
> a render field to the insert/update page form
>
>  --
>
>
>
>



-- 
 Paolo

-- 

Re: [web2py] Re: auth.wiki menu

[paolo.vall...@gmail.com]

yep, this fix the problem but it is something ad-hoc, a general procedure
would be better.


2013/1/16 Andrew W 

> I just tried something like this:
>
> if request.function != 'wiki':
>response.menu += Wiki(auth).menu(controller="**default", function=
> "wiki")
>
>
>
> On Wednesday, January 16, 2013 10:07:06 PM UTC+13, Paolo valleri wrote:
>
>> Hi Alan, thank for the tip :-), it seems to work correctly.
>> The only issue is that when I call the function 'wiki' I get two menu,
>> one from the ad-hoc import, the other one from the common wiki.
>> What shoud be nice is to explicitly tell auth.wiki when it has to
>> populate the response.menu and when not.
>> Paolo
>>
>>
>> 2013/1/15 Alan Etkin 
>>
>> > Now I want to have response.menu populated with the wiki pages too  in
>>> every page I have, is there a way to populate it calling
>>> > somehow auth.wiki or I have to do that by hand, updating each time
>>> menu.py?
>>>
>>> If you put this on top of the controller (or model), you get the wiki
>>> menu
>>>
>>> from gluon.tools import Wiki
>>> response.menu += Wiki(auth).menu(controller="**default", function="wiki"
>>> )
>>>
>>> The controller and / or function is required since otherwise, the wiki
>>> object cannot resolve redirection to edit/search/etc., and that address
>>> should be an action that calls auth.wiki(). And also you must check the
>>> action name to avoid duplicating the sub-menu
>>>
>>> What about an extra set of keyword arguments to activate the menu?:
>>>
>>> auth.wiki(resolve=False, menu=True, controller=..., function=)
>>>
>>>
>>>  --
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>>  Paolo
>>
>  --
>
>
>
>



-- 
 Paolo

-- 

Re: [web2py] Re: auth.wiki menu

[paolo.vall...@gmail.com]

Hi Alan, thank for the tip :-), it seems to work correctly.
The only issue is that when I call the function 'wiki' I get two menu, one
from the ad-hoc import, the other one from the common wiki.
What shoud be nice is to explicitly tell auth.wiki when it has to populate
the response.menu and when not.
Paolo


2013/1/15 Alan Etkin 

> > Now I want to have response.menu populated with the wiki pages too  in
> every page I have, is there a way to populate it calling
> > somehow auth.wiki or I have to do that by hand, updating each time
> menu.py?
>
> If you put this on top of the controller (or model), you get the wiki menu
>
> from gluon.tools import Wiki
> response.menu += Wiki(auth).menu(controller="default", function="wiki")
>
> The controller and / or function is required since otherwise, the wiki
> object cannot resolve redirection to edit/search/etc., and that address
> should be an action that calls auth.wiki(). And also you must check the
> action name to avoid duplicating the sub-menu
>
> What about an extra set of keyword arguments to activate the menu?:
>
> auth.wiki(resolve=False, menu=True, controller=..., function=)
>
>
>  --
>
>
>
>



-- 
 Paolo

-- 

Re: [web2py] Re: Auth with migrate=False

[paolo.vall...@gmail.com]

Hi Anthony, I agree with you. It would be nice to have such a on_define
function, this will not only optimize the code but also makes the code more
readable. I prefer the function rather than distributing in several
controllers the code that forces the definition of Auth


2013/1/15 Anthony 

> On Tuesday, January 15, 2013 9:27:55 AM UTC-5, Paolo valleri wrote:
>
>> I am using lazy_tables=True at DAL definition, actually I don't know how
>> to enable lazy_table for auth. Does it exist?
>
>
> Setting lazy_tables at the DAL definition affects all tables, including
> the Auth tables.
>
>
>> As I can see, auth.define_tables doesn't take any lazy_table parameter,
>> moreover in my code I have a few lines of auth.settings.* which needs auth.
>>
>
> If you have code that directly refers to an Auth table after it has been
> defined lazily, then the table will be fully defined. In that case, you
> could move that code to the action that actually needs it (e.g., the user()
> action). Perhaps we should add an _on_define attribute to tables so it is
> possible to add an on_define function after the initial lazy table
> definition.
>
> Anthony
>
>>   --
>
>
>
>



-- 
 Paolo

-- 

Re: [web2py] Re: Auth with migrate=False

[paolo.vall...@gmail.com]

One or an other is ok, they lead to the same thing! I am adding them to my
project :)
However, after I first test, i could see any difference, I will investigate
more asap


2013/1/15 Niphlod 

> I'd rather specify also that auth.define_tables(migrate=False) is the way
> to go to turn migrations off of all the auth_* tables
>
> Il giorno martedì 15 gennaio 2013 14:11:48 UTC+1, Paolo valleri ha scritto:
>>
>> Thanks for clarifying it.
>> So that, to speed up web2py, we must use migrate_enabled=False instead of
>> migrate=False at DAL definition. Otherwise for all the auth_* tables,
>> web2py will access the disk to verify if the relative file in
>> databases/hash_tablename.table exists and is updated, right?
>> If what I have said is correct I would suggest to update the book,
>> http://web2py.com/books/**default/chapter/29/13#**Efficiency-tricks
>> changing migrate=False with migrate_enabled=False
>>
>> Paolo
>>
>>
>> 2013/1/15 Anthony 
>>
>> DAL(..., migrate=False) does not completely turn off migrations but
>>> merely sets the default migrate value for each table to False if it is not
>>> set explicitly. The auth.define_tables() method, however, explicitly sets
>>> migrate to True for each Auth table unless otherwise specified, so you
>>> would have to do auth.define_tables(..., migrate=False). Another option is
>>> to completely disable all migrations, regardless of individual table
>>> settings, via:
>>>
>>> DAL(..., migrate_enabled=False)
>>>
>>> Anthony
>>>
>>>
>>> On Tuesday, January 15, 2013 7:35:14 AM UTC-5, Paolo valleri wrote:

 is on the db definition.
 db = DAL('postgres://web2py:web2py@localhost:5432/traffic',
  migrate=False,
 lazy_tables=True,
 )

 Paolo

 On Tuesday, January 15, 2013 11:38:49 AM UTC+1, Niphlod wrote:
>
> migrate=False is on the db or on Auth define_table() call ?
>
>
>
> Il giorno martedì 15 gennaio 2013 11:07:49 UTC+1, Paolo valleri ha
> scritto:
>>
>> Hi all, I tried something odd.
>> I pulled a fresh copy of my code from git and I connected the app to
>> an already 'working' database. 'working' in the meaning that both tables
>> and data are already in the database. In web2py this approach
>> fails, because at the first run it tries to create all the tables, and 
>> the
>> db raises an exception like 'table already exist'.
>> In order to fix that I thought that migrate=False would help, however
>> it fails again. due to Auth that tries to create the tables. Ticket:
>>
>> Traceback (most recent call last):
>>
>>
>>
>>   File "/home/pvalleri/src/web2py/**glu**on/restricted.py", line 212, in 
>> restricted
>>
>>
>>
>> exec ccode in environment
>>
>>
>>
>>   File 
>> "/home/pvalleri/src/web2py/**app**lications/vtraffic/**controllers**/default.py"
>>  
>> ,
>>  line 6, in 
>>
>>
>>
>> start = db.record.with_alias('start_**po**int')
>>
>>
>>
>>   File "/home/pvalleri/src/web2py/**glu**on/dal.py", line 7221, in 
>> __getattr__
>>
>>
>>
>> return self.lazy_define_table(tablename,*fields,**args)
>>
>>
>>
>>   File "/home/pvalleri/src/web2py/**glu**on/dal.py", line 7174, in 
>> lazy_define_table
>>
>>
>>
>> table = table_class(self, tablename, *fields, **args)
>>
>>
>>
>>   File "/home/pvalleri/src/web2py/**glu**on/dal.py", line 7594, in 
>> __init__
>>
>>
>>
>> field.requires = sqlhtml_validators(field)
>>
>>
>>
>>   File "/home/pvalleri/src/web2py/**glu**on/dal.py", line 6338, in 
>> sqlhtml_validators
>>
>>
>>
>> referenced = db[field_type[10:]]
>>
>>
>>
>>   File "/home/pvalleri/src/web2py/**glu**on/dal.py", line 7215, in 
>> __getitem__
>>
>>
>>
>> return self.__getattr__(str(key))
>>
>>
>>
>>   File "/home/pvalleri/src/web2py/**glu**on/dal.py", line 7221, in 
>> __getattr__
>>
>>
>>
>> return self.lazy_define_table(tablename,*fields,**args)
>>
>>
>>
>>   File "/home/pvalleri/src/web2py/**glu**on/dal.py", line 7177, in 
>> lazy_define_table
>>
>>
>>
>> table._create_references() # must follow above line to handle self 
>> references
>>
>>
>>
>>   File "/home/pvalleri/src/web2py/**glu**on/dal.py", line 7663, in 
>> _create_references
>>
>>
>>
>> rtable = db[rtablename]
>>
>>
>>
>>   File "/home/pvalleri/src/web2py/**glu**on/dal.py", line 7215, in 
>> __getitem__
>>
>>
>>
>> return self.__getattr__(str(key))
>>
>>
>>
>>   File "/home/pvalleri/src/web2p

Re: [web2py] Re: Auth with migrate=False

[paolo.vall...@gmail.com]

Thanks for clarifying it.
So that, to speed up web2py, we must use migrate_enabled=False instead of
migrate=False at DAL definition. Otherwise for all the auth_* tables,
web2py will access the disk to verify if the relative file in
databases/hash_tablename.table exists and is updated, right?
If what I have said is correct I would suggest to update the book,
http://web2py.com/books/default/chapter/29/13#Efficiency-tricks
changing migrate=False with migrate_enabled=False

Paolo


2013/1/15 Anthony 

> DAL(..., migrate=False) does not completely turn off migrations but merely
> sets the default migrate value for each table to False if it is not set
> explicitly. The auth.define_tables() method, however, explicitly sets
> migrate to True for each Auth table unless otherwise specified, so you
> would have to do auth.define_tables(..., migrate=False). Another option is
> to completely disable all migrations, regardless of individual table
> settings, via:
>
> DAL(..., migrate_enabled=False)
>
> Anthony
>
>
> On Tuesday, January 15, 2013 7:35:14 AM UTC-5, Paolo valleri wrote:
>>
>> is on the db definition.
>> db = DAL('postgres://web2py:web2py@**localhost:5432/traffic',
>>  migrate=False,
>> lazy_tables=True,
>> )
>>
>> Paolo
>>
>> On Tuesday, January 15, 2013 11:38:49 AM UTC+1, Niphlod wrote:
>>>
>>> migrate=False is on the db or on Auth define_table() call ?
>>>
>>>
>>>
>>> Il giorno martedì 15 gennaio 2013 11:07:49 UTC+1, Paolo valleri ha
>>> scritto:

 Hi all, I tried something odd.
 I pulled a fresh copy of my code from git and I connected the app to an
 already 'working' database. 'working' in the meaning that both tables and
 data are already in the database. In web2py this approach fails, because at
 the first run it tries to create all the tables, and the db raises an
 exception like 'table already exist'.
 In order to fix that I thought that migrate=False would help, however
 it fails again. due to Auth that tries to create the tables. Ticket:

 Traceback (most recent call last):

   File "/home/pvalleri/src/web2py/**gluon/restricted.py", line 212, in 
 restricted

 exec ccode in environment

   File 
 "/home/pvalleri/src/web2py/**applications/vtraffic/**controllers/default.py"
  
 ,
  line 6, in 

 start = db.record.with_alias('start_**point')

   File "/home/pvalleri/src/web2py/**gluon/dal.py", line 7221, in 
 __getattr__

 return self.lazy_define_table(tablena**me,*fields,**args)

   File "/home/pvalleri/src/web2py/**gluon/dal.py", line 7174, in 
 lazy_define_table

 table = table_class(self, tablename, *fields, **args)

   File "/home/pvalleri/src/web2py/**gluon/dal.py", line 7594, in __init__

 field.requires = sqlhtml_validators(field)

   File "/home/pvalleri/src/web2py/**gluon/dal.py", line 6338, in 
 sqlhtml_validators

 referenced = db[field_type[10:]]

   File "/home/pvalleri/src/web2py/**gluon/dal.py", line 7215, in 
 __getitem__

 return self.__getattr__(str(key))

   File "/home/pvalleri/src/web2py/**gluon/dal.py", line 7221, in 
 __getattr__

 return self.lazy_define_table(tablena**me,*fields,**args)

   File "/home/pvalleri/src/web2py/**gluon/dal.py", line 7177, in 
 lazy_define_table

 table._create_references() # must follow above line to handle self 
 references

   File "/home/pvalleri/src/web2py/**gluon/dal.py", line 7663, in 
 _create_references

 rtable = db[rtablename]

   File "/home/pvalleri/src/web2py/**gluon/dal.py", line 7215, in 
 __getitem__

 return self.__getattr__(str(key))

   File "/home/pvalleri/src/web2py/**gluon/dal.py", line 7221, in 
 __getattr__

 return self.lazy_define_table(tablena**me,*fields,**args)

   File "/home/pvalleri/src/web2py/**gluon/dal.py", line 7189, in 
 lazy_define_table

 polymodel=polymodel)

   File "/home/pvalleri/src/web2py/**gluon/dal.py", line 935, in 
 create_table

 self.create_sequence_and_**triggers(query,table)

   File "/home/pvalleri/src/web2py/**gluon/dal.py", line 2483, in 
 create_sequence_and_triggers

 self.execute(query)

   File "/home/pvalleri/src/web2py/**gluon/dal.py", line 1709, in execute

 return self.log_execute(*a, **b)

   File "/home/pvalleri/src/web2py/**gluon/dal.py", line 1703, in 
 log_execute

 ret = self.cursor.execute(*a, **b)

   File "/home/pvalleri/src/web2py/**gluon/contrib/pg8000/dbapi.py"**, line 
 246, in _fn

 return fn(self, *args, **kwargs)

   File "/home/pvalleri/src/web2py/**gluon/contrib/pg8000/dbapi.py"**, line 
 317, in execute
>>>

Re: [web2py] geodal

[paolo.vall...@gmail.com]

Done,
http://code.google.com/p/web2py/issues/detail?id=1259&thanks=1259&ts=1357666735
Thanks

Paolo


2013/1/8 Massimo Di Pierro 

> Please open a ticket so I will look into it asap. I would add the st_x/y
> to all GIS adapters.
>
>
> On Tuesday, January 8, 2013 9:36:57 AM UTC-6, Paolo valleri wrote:
>>
>> Hi Massimo, good to know that the patch is in trunk but I it is only for
>> the postgres adapter, I can try to define the same functions for the other
>> adapter but then someone has to check if they works or not.
>>
>> For what concern the other issue I will open I ticket, the main problem
>> is that web2py makes all the common fields name lower while it store the
>> 'gis' fields as is. I have just tried this:
>> db.define_table('test_geo1_**name',
>> Field('name_P', 'string'),
>> Field('location_P','geometry()**'),
>> )
>> In the db I have the field *name_p* and *location_P* respectively. To
>> fix it, I added in dal.py .lower() at line 832:
>> ftype = ftype % dict(schema=schema,
>>  tablename=tablename,
>>  fieldname=field_name.lower(), srid=srid,
>>  dimension=dimension)
>> And now in the db I have the field *name_p* and *location_p* respectively
>>
>> Paolo
>>
>>
>>
>>
>>
>> On Tuesday, January 8, 2013 3:48:56 PM UTC+1, Massimo Di Pierro wrote:
>>>
>>> In trunk. How about the previous issue? Is that still open? If so, can
>>> you please open a ticket? Thanks.
>>>
>>> On Tuesday, 8 January 2013 05:31:28 UTC-6, Paolo valleri wrote:

 Hi all, I have a simple problem, how I can extract values of a single
 point?
 I found that there are the functions ST_X, and ST_Y.
 Please find attached a patch that add them to dal and to the postgres
 adapter, I haven't put them neither to MSSQLAdapter nor to
 SQLITESpatiaLiteAdapter because I don't have those db to run tests.
 Now you can do something like:
 print db(db.city).select(db.city.lat**lng,
  db.city.latlng.st_x(),
  db.city.latlng.st_y())

 city.latlng,ST_X(city.latlng),**ST_Y(city.latlng)
 POINT(51.507335 -0.127683),51.507335,-0.127683
 POINT(41.901514 12.460774),41.901514,12.460774
 POINT(40.851775 14.268124),40.851775,14.268124
 The geometry field MUST contain only point otherwise it raises an error.

 Hope it helps

 Paolo



 On Sunday, January 6, 2013 5:29:52 PM UTC+1, Paolo valleri wrote:
>
> Hi Massimo, I found an other strange behavior. I tried to capitalize
> the name of a field, as follows:
> db.define_table('test_geo',
> Field('loc_Test','geometry()')**,
> )
> and I get this error:
>
> Traceback (most recent call last):
>
>   File "/home/paolo/Dropbox/git/**web2py/gluon/restricted.py", line 212, 
> in restricted
>
> exec ccode in environment
>
>   File 
> "/home/paolo/Dropbox/git/**web2py/applications/bikend/**models/db.py" 
> , line 585, 
> in 
>
> db.test_geo.insert(loc_Test=ge**oPoint(45.89096,11.0401399))
>
>   File "/home/paolo/Dropbox/git/**web2py/gluon/dal.py", line 7977, in 
> insert
>
> ret =  self._db._adapter.insert(self**,self._listify(fields))
>
>   File "/home/paolo/Dropbox/git/**web2py/gluon/dal.py", line 1175, in 
> insert
>
> raise e
> ProgrammingError: column "loc_test" of relation "test_geo" does not exist
>
> LINE 1: INSERT INTO test_geo(loc_Test) VALUES (ST_GeomFromText('POIN...
>
> and so I tried to insert without the capitalize and I get this error:
>
> Traceback (most recent call last):
>
>   File "/home/paolo/Dropbox/git/**web2py/gluon/restricted.py", line 212, 
> in restricted
>
> exec ccode in environment
>
>   File 
> "/home/paolo/Dropbox/git/**web2py/applications/bikend/**models/db.py" 
> , line 585, 
> in 
>
> db.test_geo.insert(loc_test=ge**oPoint(45.89096,11.0401399))
>
>   File "/home/paolo/Dropbox/git/**web2py/gluon/dal.py", line 7977, in 
> insert
>
> ret =  self._db._adapter.insert(self**,self._listify(fields))
>
>   File "/home/paolo/Dropbox/git/**web2py/gluon/dal.py", line 7915, in 
> _listify
>
> 'Field %s does not belong to the table' % name)
> SyntaxError: Field loc_test does not belong to the table
>
>
> finally, I redefined the table as was before, (without any capital
> latter) and I get this:
>
> Traceback (most recent call last):
>
>   File "/home/paolo/Dropbox/git/**web2py/gluon/restricted.py", line 212, 
> in restricted
>
> exec ccode in environment
>
>   File 
> "/home/paolo/Dropbox/git/**web2py/applications/bikend/**m

Re: [web2py] Re: self join on postgres

[paolo.vall...@gmail.com]

Thanks for this explanations! To add more info, I can say that the elements
on left column are naturally ordered, I have thousands of
records(~5000record/day) for each station but only a few
station_id(~5-10stations totally).
Instead of making a complex query, a cleaner approach would be to run the
query with only the join and then do a loop in python to remove the
unnecessary lines.

Moreover, right now, the worst think is that since I switched to postgres
the initial query (the one with only the leftjoin) is taking minutes to be
executed (almost 4m) against just few seconds on sqlite. I set an index on
the mac field through pgAdmin and the results are really better,
unfortunately this hack is not portable, is there a way to define indexes
from web2py?

Paolo


2013/1/8 Niphlod 

> after a full day at work I may lack the usual fantasy, but what you're
> trying to do can't be even achieved by a "relatively simple" windowed
> function
> What you're trying to do is recursing. way out of DAL reach (if you
> want to do it in a single query)
> Trying to explain better
>
> Real data helps.
> For every station_id record with the same mac address you want to find the
> "min gathered_on" record from the same table (with another station_id) and
> "subtract it" for every next possible match.
>
> One thing is requiring
>
> "2013-01-21 11:23:35";"a";127167;"2013-01-21 11:23:45";"a";127168
> "2013-01-21 11:23:00";"a";127169;"2013-01-21 11:23:45";"a";127168
> That can be accomplished by something like this
>
> select * from (
> selectstart_point.gathered_on,start_point.mac,start_point.id,
> end_point.gathered_on,end_point.mac,end_point.id,
> row_number() over (partition by start_point.id order by end_point.
> gathered_on) as filter_field
> from record as start_point
> inner join
> record as end_point
> on start_point.mac = end_point.mac
> and start_point.gathered_on <= end_point.gathered_on
> where start_point.station_id = 13
> and end_point.station_id = 14
> ) as q
> where q.filter_field = 1
>
> because for the record 127167 the next record with another station_id is
> 127168, but then for the 127169 you don't want the 127168, you want 127170
> because 127168 "has been booked before" by 127169.
>
> Honestly, (beware of the lack of fantasy :P) I'd do a loop in python
> instead of using recursing in the db itself unless you have zillions of
> "windows" (i.e. you have 1000 station_id = 13 and 1000 station_id = 14, and
> 1000 distinct station_id), just because it's more readable than what it
> would be needed in raw sql
>
> --
>
>
>
>



-- 
 Paolo

-- 

Re: [web2py] Re: delete uploads with SQLFORM.factory

[paolo.vall...@gmail.com]

Hi, I had a look at sqlform.py, the error comes from the fact that if a
default value is set for the field upload, web2py tries to read it and
store it as the new value for the upload field. As a result it is not
possible to me to edit forms and remove uploads by setting the default
value, this trick doesn't work.
Moreover, I tried to fix the initial error about the missing file, by
setting db.test_img['picture'].default  equal to real path of the file.
This worked well but than, if I understood correctly how web2py works, it
tried to store again the default value as the new value, starting a kind of
loop! That was wrong, and actually this has never worked, it stopped due to
the 'error filename' to long.

The good thing is that by changing this two line (line 1394 of sqlhtml.py)
f = self.table[fieldname].default or ''
fields[fieldname] = f
to:
fields[fieldname] = self.table[fieldname].default
It seems that this solved the issue, but of course, I need something better.


2012/12/27 Paolo 

> Hi Massimo, thank for the answer, I updated the code but the problem is
> still there, I made several tests, no one figured out what is wrong. So
> that I tried to simplify the code as follows:
> db.define_table('test_img',
>
> Field("picture", "upload",
>
>   uploadfolder=os.path.join(request.folder,'uploads','pictures'),
>   uploadseparate=True),
> Field('created_on', 'datetime', default=request.now, writable=False)
> )
>
> def sqlform_add():
> form = SQLFORM.factory(db.test_img, _class='well',
>
>upload=URL('default', 'download'),
>
>uploadfolder=os.path.join(request.folder,
> 'uploads','pictures'),
>uploadseparate=True, table_name='test')
> if form.process(dbio=False).accepted:
> id = db.test_img.insert(**dict(form.vars))
> return response.render('generic.html', dict(form=form))
>
> def sqlform_edit():
> img = db(db.test_img.id>0).select().first()
> db.test_img['picture'].default = img['picture']
> form = SQLFORM.factory(db.test_img, _class='well',
>
>upload=URL('default', 'download'),
>
>uploadfolder=os.path.join(request.folder,
> 'uploads','pictures'),
>uploadseparate=True, table_name='test')
> if form.process(dbio=False).accepted:
> print 'form accepted'
> return response.render('generic.html', dict(form=form))
>
> what I got:
> - adding an img with sqlform_add and then removing it with appadmin  -->
> worked
> - adding an img with appadmin and then removing it with sqlform_edit -->
> failed
> - using both sqlform_add and sqlform_edit --> failed
> Now I am thinking about the way I set the default value, maybe for a field
> as an upload I have to set something more?
>
> Paolo
>
>
> On Thursday, December 27, 2012 9:28:54 PM UTC+1, Massimo Di Pierro wrote:
>>
>> perhaps not the cause of the problem but
>>
>> request.folder+'uploads/**pictures'
>>
>> should be
>>
>> os.path.join(request.folder,'**uploads','pictures')
>>
>> On Thursday, 27 December 2012 00:30:00 UTC-6, Paolo wrote:
>>>
>>> Hi Massimo,
>>> I've just tried to post and then edit with both SQLFORM.factory having
>>> uploadseparate but nothing has changed. The problem is still there,
>>> actually I am making explicitly inserts and  edits.
>>> The whole SQLFORM.factory to edit a field is this:
>>> for field in ['title','description', 'user_contact',
>>> 'picture' ]:
>>> db.club[field].default = club[field]
>>> for field in ['name']:
>>> db.cities[field].default = city[field]
>>> form = SQLFORM.factory(db.club, db.cities, _class='well',
>>>formstyle='bootstrap', showid= False,
>>>upload=URL('default', 
>>> 'download'),uploadfolder
>>> =request.folder+'**uploads/pictures',
>>>uploadseparate=True, 
>>> autodelete=True,table_name
>>> ='club')
>>>
>>>
>>> Paolo
>>>
>>>
>>>
>>> On Thursday, December 27, 2012 12:47:19 AM UTC+1, Massimo Di Pierro
>>> wrote:

 Did you upload the file first and then add uploadseparate/uploadfolder?


 On Wednesday, 26 December 2012 12:49:01 UTC-6, Paolo wrote:
>
> Hi all,
> it seems to me that SQLFORM.factory doesn't honor the uploadseparate
> option because I'am not able to delete the uploaded file with
> SQLFORM.factory
>
> form = SQLFORM.factory(db.club, db.cities, _class='well',
>formstyle='bootstrap', showid= False,
>upload=URL('default', 'download'), uploadfolder
> =request.folder+'**uploads/pictures',
>uploadseparate=True, autodelete=True,table_name
> ='club'  )
>
>
>
> The error:
> Enter code here...2012-12-26 19:38:07,728 - web2py - ERROR - Traceback
> (mo

Re: [web2py] Re: new setup-web2py-nginx-uwsgi-ubuntu.sh

[paolo.vall...@gmail.com]

Hi
I was trying with the script that comes with the stable web2py. with the
one on dropbox I problem has gone.

Paolo


2012/12/19 Niphlod 

>
> @Simone, an other improvement to the script could be to combine into a
>> single server the 80, and 443 to avoid duplicating configuration, as stated
>> here :
>> http://nginx.org/en/docs/http/**configuring_https_servers.**
>> html#single_http_https_server
>> I don't had test this.
>>
>> Also, here the code for permanent redirection
>> server {
>>   server_name $hostname;
>>   listen  80;
>>   return 301 http*s*://$hostname$request_uri; # NOTE: I am not sure for
>> $hostname here, because I didn't set hostname for my VM until now, as in
>> the example (URL below) we can use domainName.com instead if properly
>> configure in nginx
>> ...
>> }
>>
>>
> People may want separate configs for http and https. The script objective
> is to have a working copy of web2py. If we start to follow such requests,
> we'd end up installing postgresql and redis too :P
>
> @Paolo: try the script found at the dropbox link. If the same error
> happens I think we need the log of pip and a hand by Roberto on the
> specific error...
>
> --
>
>
>
>



-- 
 Paolo

-- 

Re: [web2py] Re: Time aggregations

[paolo.vall...@gmail.com]

Hi Niphlod, thanks for clarifying me this. I will go for the aggregation by
hand, which is more portable rather than database view. However, we could
provide something like that in the web2py library or somewhere, to speedup
the work for future developments.



2012/12/14 Niphlod 

> there are no "native" functions on databases to group by, e.g., a quarter
> hour.
> For those kind of things, either you aggregate by hand in python or code a
> view directly in the database to exploit its functions (depending on the
> engine, could be either an impossible or a semi-complicated task :-P) and
> access it with migrate=False within web2py.
>
>
> On Friday, December 14, 2012 12:53:43 PM UTC+1, Paolo wrote:
>>
>> Dear all,
>> I am looking for some advice in how to perform time aggregations, so far
>> I have been able to aggregate only by hours or minutes but nothing between,
>> for example every 10m, 15m and so on. For example this query count the
>> number of fields aggregated by hours:
>> c = db.record.id.count()
>> s = db.record.created_on.year() | db.record.created_on.month() | db.
>> record.created_on.day() | db.record.created_on.hour()
>> rows = db(db.record.id > 0).select(db.record.created_on**, c, groupby=s)
>> while in this case:
>> c = db.record.id.count()
>> s = db.record.created_on.year() | db.record.created_on.month() | db.
>> record.created_on.day() | db.record.created_on.hour() | db.record.
>> created_on.minutes()
>> rows = db(db.record.id > 0).select(db.record.created_on**, c, groupby=s)
>> How I can make the above queries more flexible? Namely, in aggregating
>> for different time frames, i.e., 10minutes, 400seconds and so?
>> So far, I've solved this issue by making a simple query but aggregation
>> and then, counting and aggregating by hand. I hope there is a better and
>> linear approach for that.
>>
>> Regards
>> Paolo
>>
>>
>>
>  --
>
>
>
>



-- 
 Paolo

-- 

Re: [web2py] Re: new setup-web2py-nginx-uwsgi-ubuntu.sh

[paolo.vall...@gmail.com]

yeap :)


2012/12/12 Niphlod 

> meaning checking if the script is run as root ?
>
> Il giorno mercoledì 12 dicembre 2012 11:06:42 UTC+1, Paolo ha scritto:
>>
>> sorry, one more thing, what about checking if you have the permission to
>> install stuff and so on?
>>
>>
>> 2012/12/12 Niphlod 
>>
>> mee too, didn't notice in the cut-paste madness.
>>>
>>>
>>> On Wednesday, December 12, 2012 9:57:06 AM UTC+1, Paolo wrote:

 Hi and great job!

 One thing that I don't like is at line 10
 apt-get -y dist-upgrade

 this was also in the previous version, actually I see this too
 restrictive, for example I have ubuntu 12.04 and don't want to move to 
 12.10

 Paolo


 On Wednesday, December 12, 2012 12:06:16 AM UTC+1, Niphlod wrote:
>
> I run uwsgi using the emperor mode for some time now based on my
> config I tried to upgrade the existing one. Result is available at
> https://www.dropbox.com/s/**n7ch**teos9sh6p2h/setup-web2py-**nginx**
> -uwsgi-ubuntu.sh
>
> I'm looking forward to improve things both on uwsgi niceties and
> general bash customization (the moment I saw fabricI 
> almost stopped writing bash :-P)
>
> Improvements:
> - no uwsgi from debian packages (overcomplicated configurations).
> uwsgi is installed from pip (so you can upgrade with a simple pip install
> --upgrade uwsgi)
> - optional response.static_version friendly static directories
> configuration (added also cache headers as an option)
> - improved syntax for web2py.xml (no app, no plugin)
> - example of uwsgi cron facility (clean sessions script included)
> - uwsgi emperor mode, managed with Upstart
> - optional upstart pre-script command to fix permissions and compress
> static files
>
> If interested, I can make a fabfile for that.
>
>  I use Redis as cache but for the "I love embedded" guys
> uwsgi has a cache framework. I'm accepting votes to make a uwsgi's cache
> compatible module for web2py, but you will have to test it (no time for
> that, sorry)
> 
>
  --
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>>  Paolo
>>
>  --
>
>
>
>



-- 
 Paolo

-- 

Re: [web2py] Re: new setup-web2py-nginx-uwsgi-ubuntu.sh

[paolo.vall...@gmail.com]

sorry, one more thing, what about checking if you have the permission to
install stuff and so on?


2012/12/12 Niphlod 

> mee too, didn't notice in the cut-paste madness.
>
>
> On Wednesday, December 12, 2012 9:57:06 AM UTC+1, Paolo wrote:
>>
>> Hi and great job!
>>
>> One thing that I don't like is at line 10
>> apt-get -y dist-upgrade
>>
>> this was also in the previous version, actually I see this too
>> restrictive, for example I have ubuntu 12.04 and don't want to move to 12.10
>>
>> Paolo
>>
>>
>> On Wednesday, December 12, 2012 12:06:16 AM UTC+1, Niphlod wrote:
>>>
>>> I run uwsgi using the emperor mode for some time now based on my
>>> config I tried to upgrade the existing one. Result is available at
>>> https://www.dropbox.com/s/**n7chteos9sh6p2h/setup-web2py-**
>>> nginx-uwsgi-ubuntu.sh
>>>
>>> I'm looking forward to improve things both on uwsgi niceties and general
>>> bash customization (the moment I saw fabric  I
>>> almost stopped writing bash :-P)
>>>
>>> Improvements:
>>> - no uwsgi from debian packages (overcomplicated configurations). uwsgi
>>> is installed from pip (so you can upgrade with a simple pip install
>>> --upgrade uwsgi)
>>> - optional response.static_version friendly static directories
>>> configuration (added also cache headers as an option)
>>> - improved syntax for web2py.xml (no app, no plugin)
>>> - example of uwsgi cron facility (clean sessions script included)
>>> - uwsgi emperor mode, managed with Upstart
>>> - optional upstart pre-script command to fix permissions and compress
>>> static files
>>>
>>> If interested, I can make a fabfile for that.
>>>
>>>  I use Redis as cache but for the "I love embedded" guys
>>> uwsgi has a cache framework. I'm accepting votes to make a uwsgi's cache
>>> compatible module for web2py, but you will have to test it (no time for
>>> that, sorry)
>>> 
>>>
>>  --
>
>
>
>



-- 
 Paolo

-- 

Re: [web2py] Re: Lazy_tables with virtual fields

[paolo.vall...@gmail.com]

Hi Massimo, first of all thank you for spending time on it!
The first test ends as the others, namely the first time it works,
then it raises an error.
The the second test works without any problem :-)
To solve the issue, I will change the code to follow what you have
suggested in the second test.

Cheers, Paolo

2012/9/1 Massimo Di Pierro :
> One more test:
>
> def getdb(): return db
> db.reviews.pos = Field.Lazy(lambda row,db=getdb:
> db()((db().reviews_like.review_id == row.reviews.id) &
> (db().reviews_like.helpful == True)).count() )
> db.reviews.neg = Field.Lazy(lambda row,db=getdb:
> db()((db().reviews_like.review_id == row.reviews.id) &
> (db().reviews_like.helpful == False)).count() )
>
> Notice you can also do
>
> db.reviews.pos = Field.Virtual(lambda row: db((db.reviews_like.review_id ==
> row.reviews.id) & (db.reviews_like.helpful == True)).count() )
> db.reviews.neg = Field.Virtual(lambda row: db((db.reviews_like.review_id ==
> row.reviews.id) & (db.reviews_like.helpful == False)).count() )
>
> and use rows.pos instead of rows.pos()
>
> In this case the value is cached, not the callback function.
>
>
> On Saturday, 1 September 2012 07:40:20 UTC-5, Paolo wrote:
>>
>> Hi Massimo,
>> I've just tried both strategies you suggested but I got the same
>> behavior; namely the first time it works, the second rises the error.
>>
>> Paolo
>>
>> 2012/9/1 Massimo Di Pierro :
>> > I do not fully understand this. It mush have something to do with the
>> > scoping of variables in python.
>> >
>> > Can you try replace
>> >
>> > db.reviews.pos = Field.Lazy(lambda row: db((db.reviews_like.review_id ==
>> > row.reviews.id) & (db.reviews_like.helpful == True)).count() )
>> > db.reviews.neg = Field.Lazy(lambda row: db((db.reviews_like.review_id ==
>> > row.reviews.id) & (db.reviews_like.helpful == False)).count() )
>> >
>> > with
>> >
>> > db.reviews.pos = Field.Lazy(lambda row,db=db:
>> > db((db.reviews_like.review_id
>> > == row.reviews.id) & (db.reviews_like.helpful == True)).count() )
>> > db.reviews.neg = Field.Lazy(lambda row,db=db:
>> > db((db.reviews_like.review_id
>> > == row.reviews.id) & (db.reviews_like.helpful == False)).count() )
>> >
>> > or with
>> >
>> > def lazy1(row,db=db): return db((db.reviews_like.review_id ==
>> > row.reviews.id) & (db.reviews_like.helpful == True)).count()
>> > def lazy2(row,db=db): return db((db.reviews_like.review_id ==
>> > row.reviews.id) & (db.reviews_like.helpful == False)).count()
>> > db.reviews.pos = Field.Lazy(lazy1)
>> > db.reviews.neg = Field.Lazy(lazy2)
>> >
>> >
>> > On Saturday, 1 September 2012 05:50:35 UTC-5, João Saraiva wrote:
>> >>
>> >> Hello,
>> >>
>> >> Could it be something related to Dropbox's sync mechanism? Have you
>> >> tried
>> >> running your app in another folder outside of Dropbox's reach? Would
>> >> probably explain why your example works fine on other user's machines.
>> >>
>> >> Best regards,
>> >> JS
>> >>
>> >>
>> >> On Saturday, September 1, 2012 6:34:39 AM UTC+1, Paolo wrote:
>> >>>
>> >>> Hi Massimo,  the error is: Cannot operate on a closed database.
>> >>> ticket:
>> >>> Traceback (most recent call last):
>> >>>   File "/home/paolo/Dropbox/git/pp/web2py/gluon/restricted.py", line
>> >>> 209, in restricted
>> >>> exec ccode in environment
>> >>>   File
>> >>>
>> >>> "/home/paolo/Dropbox/git/pp/web2py/applications/test_vf/views/default/test.html",
>> >>> line 3, in 
>> >>> {{pass}}
>> >>>   File "/home/paolo/Dropbox/git/pp/web2py/gluon/dal.py", line 8825, in
>> >>> __call__
>> >>> return self.method(self.row,*args,**kwargs)
>> >>>   File
>> >>> "/home/paolo/Dropbox/git/pp/web2py/applications/test_vf/models/db.py",
>> >>> line 129, in 
>> >>> db.reviews_s.pos = Field.Lazy(lambda row:
>> >>> db((db.reviews_like_s.review_id == row.reviews_s.id) &
>> >>> (db.reviews_like_s.helpful == True)).count() )
>> >>>   File "/home/paolo/Dropbox/git/pp/web2py/gluon/dal.py", line 8690, in
>> >>> count
>> >>> return db._adapter.count(self.query,distinct)
>> >>>   File "/home/paolo/Dropbox/git/pp/web2py/gluon/dal.py", line 1597, in
>> >>> count
>> >>> self.execute(self._count(query, distinct))
>> >>>   File "/home/paolo/Dropbox/git/pp/web2py/gluon/dal.py", line 1653, in
>> >>> execute
>> >>> return self.log_execute(*a, **b)
>> >>>   File "/home/paolo/Dropbox/git/pp/web2py/gluon/dal.py", line 1647, in
>> >>> log_execute
>> >>> ret = self.cursor.execute(*a, **b)
>> >>> ProgrammingError: Cannot operate on a closed database.
>> >>>
>> >>> I put the code of the simple app in a new app with the latest web2py,
>> >>> the first time I accessed the page all worked well but
>> >>> when I reloaded it, I got the error. Hope this can help you.
>> >>>
>> >>> Best,
>> >>> Paolo
>> >>>
>> >>>
>> >>>
>> >>> 2012/8/31 Massimo Di Pierro :
>> >>> > What's the error? Works for me.
>> >>> >
>> >>> >
>> >>> > On Friday, 31 August 2012 16:00:24 UTC-5, Paolo wrote:
>> >>> >>
>> >>> >> Hi Massimo,
>> >>> >> I made a simple app, accordingly this 

Re: [web2py] Re: Lazy_tables with virtual fields

[paolo.vall...@gmail.com]

Hi Massimo,
I've just tried both strategies you suggested but I got the same
behavior; namely the first time it works, the second rises the error.

Paolo

2012/9/1 Massimo Di Pierro :
> I do not fully understand this. It mush have something to do with the
> scoping of variables in python.
>
> Can you try replace
>
> db.reviews.pos = Field.Lazy(lambda row: db((db.reviews_like.review_id ==
> row.reviews.id) & (db.reviews_like.helpful == True)).count() )
> db.reviews.neg = Field.Lazy(lambda row: db((db.reviews_like.review_id ==
> row.reviews.id) & (db.reviews_like.helpful == False)).count() )
>
> with
>
> db.reviews.pos = Field.Lazy(lambda row,db=db: db((db.reviews_like.review_id
> == row.reviews.id) & (db.reviews_like.helpful == True)).count() )
> db.reviews.neg = Field.Lazy(lambda row,db=db: db((db.reviews_like.review_id
> == row.reviews.id) & (db.reviews_like.helpful == False)).count() )
>
> or with
>
> def lazy1(row,db=db): return db((db.reviews_like.review_id ==
> row.reviews.id) & (db.reviews_like.helpful == True)).count()
> def lazy2(row,db=db): return db((db.reviews_like.review_id ==
> row.reviews.id) & (db.reviews_like.helpful == False)).count()
> db.reviews.pos = Field.Lazy(lazy1)
> db.reviews.neg = Field.Lazy(lazy2)
>
>
> On Saturday, 1 September 2012 05:50:35 UTC-5, João Saraiva wrote:
>>
>> Hello,
>>
>> Could it be something related to Dropbox's sync mechanism? Have you tried
>> running your app in another folder outside of Dropbox's reach? Would
>> probably explain why your example works fine on other user's machines.
>>
>> Best regards,
>> JS
>>
>>
>> On Saturday, September 1, 2012 6:34:39 AM UTC+1, Paolo wrote:
>>>
>>> Hi Massimo,  the error is: Cannot operate on a closed database.
>>> ticket:
>>> Traceback (most recent call last):
>>>   File "/home/paolo/Dropbox/git/pp/web2py/gluon/restricted.py", line
>>> 209, in restricted
>>> exec ccode in environment
>>>   File
>>> "/home/paolo/Dropbox/git/pp/web2py/applications/test_vf/views/default/test.html",
>>> line 3, in 
>>> {{pass}}
>>>   File "/home/paolo/Dropbox/git/pp/web2py/gluon/dal.py", line 8825, in
>>> __call__
>>> return self.method(self.row,*args,**kwargs)
>>>   File
>>> "/home/paolo/Dropbox/git/pp/web2py/applications/test_vf/models/db.py",
>>> line 129, in 
>>> db.reviews_s.pos = Field.Lazy(lambda row:
>>> db((db.reviews_like_s.review_id == row.reviews_s.id) &
>>> (db.reviews_like_s.helpful == True)).count() )
>>>   File "/home/paolo/Dropbox/git/pp/web2py/gluon/dal.py", line 8690, in
>>> count
>>> return db._adapter.count(self.query,distinct)
>>>   File "/home/paolo/Dropbox/git/pp/web2py/gluon/dal.py", line 1597, in
>>> count
>>> self.execute(self._count(query, distinct))
>>>   File "/home/paolo/Dropbox/git/pp/web2py/gluon/dal.py", line 1653, in
>>> execute
>>> return self.log_execute(*a, **b)
>>>   File "/home/paolo/Dropbox/git/pp/web2py/gluon/dal.py", line 1647, in
>>> log_execute
>>> ret = self.cursor.execute(*a, **b)
>>> ProgrammingError: Cannot operate on a closed database.
>>>
>>> I put the code of the simple app in a new app with the latest web2py,
>>> the first time I accessed the page all worked well but
>>> when I reloaded it, I got the error. Hope this can help you.
>>>
>>> Best,
>>> Paolo
>>>
>>>
>>>
>>> 2012/8/31 Massimo Di Pierro :
>>> > What's the error? Works for me.
>>> >
>>> >
>>> > On Friday, 31 August 2012 16:00:24 UTC-5, Paolo wrote:
>>> >>
>>> >> Hi Massimo,
>>> >> I made a simple app, accordingly this example the error seems related
>>> >> to the cache, because without the cache all works fine.
>>> >>
>>> >> The model:
>>> >> db.define_table('reviews_like_s',
>>> >>  Field('review_id', 'reference reviews'),
>>> >>  Field('helpful', 'boolean'))
>>> >>
>>> >> db.define_table('reviews_s',
>>> >> Field('title'))
>>> >> db.reviews_s.pos = Field.Lazy(lambda row:
>>> >> db((db.reviews_like_s.review_id == row.reviews_s.id) &
>>> >> (db.reviews_like_s.helpful == True)).count() )
>>> >> db.reviews_s.neg = Field.Lazy(lambda row:
>>> >> db((db.reviews_like_s.review_id == row.reviews_s.id) &
>>> >> (db.reviews_like_s.helpful == False)).count() )
>>> >>
>>> >>
>>> >> Function:
>>> >> def test():
>>> >> id = db.reviews_s.insert(title='xxx')
>>> >> db.reviews_like_s.insert(review_id=id, helpful=True)
>>> >> rows = db(db.reviews_s.id > 0).select( cache=(cache.ram,3600) )
>>> >> return dict(reviews=rows)
>>> >>
>>> >> The view:
>>> >> {{for review in reviews:}}
>>> >> {{=review.pos()}}
>>> >> {{pass}}
>>> >>
>>> >> On my side the first time works well, the second rises the error.
>>> >>
>>> >> Best,
>>> >> Paolo
>>> >>
>>> >>
>>> >> 2012/8/30 paolo@gmail.com :
>>> >> > Hi Massimo,
>>> >> > no, I do not have multiple models, I have defined the string
>>> >> > connection only once in db.py, that's all.
>>> >> > I have a few try...except but in other places, I do not thing they
>>> >> > are
>>> >> > connected.
>>> >> > I will tr

Re: [web2py] Re: Lazy_tables with virtual fields

[paolo.vall...@gmail.com]

Hi Massimo,  the error is: Cannot operate on a closed database.
ticket:
Traceback (most recent call last):
  File "/home/paolo/Dropbox/git/pp/web2py/gluon/restricted.py", line
209, in restricted
exec ccode in environment
  File 
"/home/paolo/Dropbox/git/pp/web2py/applications/test_vf/views/default/test.html",
line 3, in 
{{pass}}
  File "/home/paolo/Dropbox/git/pp/web2py/gluon/dal.py", line 8825, in __call__
return self.method(self.row,*args,**kwargs)
  File "/home/paolo/Dropbox/git/pp/web2py/applications/test_vf/models/db.py",
line 129, in 
db.reviews_s.pos = Field.Lazy(lambda row:
db((db.reviews_like_s.review_id == row.reviews_s.id) &
(db.reviews_like_s.helpful == True)).count() )
  File "/home/paolo/Dropbox/git/pp/web2py/gluon/dal.py", line 8690, in count
return db._adapter.count(self.query,distinct)
  File "/home/paolo/Dropbox/git/pp/web2py/gluon/dal.py", line 1597, in count
self.execute(self._count(query, distinct))
  File "/home/paolo/Dropbox/git/pp/web2py/gluon/dal.py", line 1653, in execute
return self.log_execute(*a, **b)
  File "/home/paolo/Dropbox/git/pp/web2py/gluon/dal.py", line 1647, in
log_execute
ret = self.cursor.execute(*a, **b)
ProgrammingError: Cannot operate on a closed database.

I put the code of the simple app in a new app with the latest web2py,
the first time I accessed the page all worked well but
when I reloaded it, I got the error. Hope this can help you.

Best,
Paolo



2012/8/31 Massimo Di Pierro :
> What's the error? Works for me.
>
>
> On Friday, 31 August 2012 16:00:24 UTC-5, Paolo wrote:
>>
>> Hi Massimo,
>> I made a simple app, accordingly this example the error seems related
>> to the cache, because without the cache all works fine.
>>
>> The model:
>> db.define_table('reviews_like_s',
>>  Field('review_id', 'reference reviews'),
>>  Field('helpful', 'boolean'))
>>
>> db.define_table('reviews_s',
>> Field('title'))
>> db.reviews_s.pos = Field.Lazy(lambda row:
>> db((db.reviews_like_s.review_id == row.reviews_s.id) &
>> (db.reviews_like_s.helpful == True)).count() )
>> db.reviews_s.neg = Field.Lazy(lambda row:
>> db((db.reviews_like_s.review_id == row.reviews_s.id) &
>> (db.reviews_like_s.helpful == False)).count() )
>>
>>
>> Function:
>> def test():
>> id = db.reviews_s.insert(title='xxx')
>> db.reviews_like_s.insert(review_id=id, helpful=True)
>> rows = db(db.reviews_s.id > 0).select( cache=(cache.ram,3600) )
>> return dict(reviews=rows)
>>
>> The view:
>> {{for review in reviews:}}
>> {{=review.pos()}}
>> {{pass}}
>>
>> On my side the first time works well, the second rises the error.
>>
>> Best,
>> Paolo
>>
>>
>> 2012/8/30 paolo@gmail.com :
>> > Hi Massimo,
>> > no, I do not have multiple models, I have defined the string
>> > connection only once in db.py, that's all.
>> > I have a few try...except but in other places, I do not thing they are
>> > connected.
>> > I will try to make a simple app ables to reproduce the problem as soon
>> > as possible.
>> >
>> > Paolo
>> >
>> > 2012/8/30 Massimo Di Pierro :
>> >> Can you make a simple app to reproduce this?
>> >>
>> >> Do you have try  except anywhere? Do you have multiple models wich
>> >> define the same db = DAL(...) connection?
>> >>
>> >>
>> >> On Thursday, 30 August 2012 15:49:01 UTC-5, Paolo wrote:
>> >>>
>> >>> Hi Bruno, a simple db with sqlite without modules
>> >>>
>> >>> db = DAL('sqlite://storage.sqlite', lazy_tables=True )
>> >>>
>> >>> sorry, but I've just discovered that the same problem is happened even
>> >>> with lazy_tables=False
>> >>> Paolo
>> >>>
>> >>> 2012/8/30 Bruno Rocha :
>> >>> > How are you defining the db connection? are you using models or
>> >>> > modules?
>> >>> > which db?
>> >>> >
>> >>> > --
>> >>> >
>> >>> >
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>>  Paolo
>> >>
>> >> --
>> >>
>> >>
>> >>
>> >
>> >
>> >
>> > --
>> >  Paolo
>>
>>
>>
>> --
>>  Paolo
>
> --
>
>
>



-- 
 Paolo

-- 

Re: [web2py] Re: Lazy_tables with virtual fields

[paolo.vall...@gmail.com]

Hi Massimo,
I made a simple app, accordingly this example the error seems related
to the cache, because without the cache all works fine.

The model:
db.define_table('reviews_like_s',
 Field('review_id', 'reference reviews'),
 Field('helpful', 'boolean'))

db.define_table('reviews_s',
Field('title'))
db.reviews_s.pos = Field.Lazy(lambda row:
db((db.reviews_like_s.review_id == row.reviews_s.id) &
(db.reviews_like_s.helpful == True)).count() )
db.reviews_s.neg = Field.Lazy(lambda row:
db((db.reviews_like_s.review_id == row.reviews_s.id) &
(db.reviews_like_s.helpful == False)).count() )


Function:
def test():
id = db.reviews_s.insert(title='xxx')
db.reviews_like_s.insert(review_id=id, helpful=True)
rows = db(db.reviews_s.id > 0).select( cache=(cache.ram,3600) )
return dict(reviews=rows)

The view:
{{for review in reviews:}}
{{=review.pos()}}
{{pass}}

On my side the first time works well, the second rises the error.

Best,
Paolo


2012/8/30 paolo.vall...@gmail.com :
> Hi Massimo,
> no, I do not have multiple models, I have defined the string
> connection only once in db.py, that's all.
> I have a few try...except but in other places, I do not thing they are
> connected.
> I will try to make a simple app ables to reproduce the problem as soon
> as possible.
>
> Paolo
>
> 2012/8/30 Massimo Di Pierro :
>> Can you make a simple app to reproduce this?
>>
>> Do you have try  except anywhere? Do you have multiple models wich
>> define the same db = DAL(...) connection?
>>
>>
>> On Thursday, 30 August 2012 15:49:01 UTC-5, Paolo wrote:
>>>
>>> Hi Bruno, a simple db with sqlite without modules
>>>
>>> db = DAL('sqlite://storage.sqlite', lazy_tables=True )
>>>
>>> sorry, but I've just discovered that the same problem is happened even
>>> with lazy_tables=False
>>> Paolo
>>>
>>> 2012/8/30 Bruno Rocha :
>>> > How are you defining the db connection? are you using models or modules?
>>> > which db?
>>> >
>>> > --
>>> >
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>>  Paolo
>>
>> --
>>
>>
>>
>
>
>
> --
>  Paolo



-- 
 Paolo

-- 

Re: [web2py] Re: Lazy_tables with virtual fields

[paolo.vall...@gmail.com]

Hi Massimo,
no, I do not have multiple models, I have defined the string
connection only once in db.py, that's all.
I have a few try...except but in other places, I do not thing they are
connected.
I will try to make a simple app ables to reproduce the problem as soon
as possible.

Paolo

2012/8/30 Massimo Di Pierro :
> Can you make a simple app to reproduce this?
>
> Do you have try  except anywhere? Do you have multiple models wich
> define the same db = DAL(...) connection?
>
>
> On Thursday, 30 August 2012 15:49:01 UTC-5, Paolo wrote:
>>
>> Hi Bruno, a simple db with sqlite without modules
>>
>> db = DAL('sqlite://storage.sqlite', lazy_tables=True )
>>
>> sorry, but I've just discovered that the same problem is happened even
>> with lazy_tables=False
>> Paolo
>>
>> 2012/8/30 Bruno Rocha :
>> > How are you defining the db connection? are you using models or modules?
>> > which db?
>> >
>> > --
>> >
>> >
>> >
>>
>>
>>
>> --
>>  Paolo
>
> --
>
>
>



-- 
 Paolo

-- 

Re: [web2py] Re: Lazy_tables with virtual fields

[paolo.vall...@gmail.com]

Hi Bruno, a simple db with sqlite without modules

db = DAL('sqlite://storage.sqlite', lazy_tables=True )

sorry, but I've just discovered that the same problem is happened even
with lazy_tables=False
Paolo

2012/8/30 Bruno Rocha :
> How are you defining the db connection? are you using models or modules?
> which db?
>
> --
>
>
>



-- 
 Paolo

-- 

Re: [web2py] Re: Lazy_tables with virtual fields

[paolo.vall...@gmail.com]

Hi Massimo, actually I do not know, it seems to be a very odd error;
I've just restarted web2py and the problem has seemed gone but
unfortunately, after a while it came back in a totally different
fashion:

Traceback (most recent call last):
  File "/home/paolo/Dropbox/git/web2py/gluon/restricted.py", line 209,
in restricted
exec ccode in environment
  File 
"/home/paolo/Dropbox/git/web2py/applications/bikend/views/route/routesListWrapper.html",
line 981, in 
  File "/home/paolo/Dropbox/git/web2py/applications/bikend/models/routeDB.py",
line 27, in get_route_a
content = get_route_img(id=route.route.id)
  File "/home/paolo/Dropbox/git/web2py/applications/bikend/models/routeDB.py",
line 56, in get_route_img
img = get_img(route.route.photo_id, True)
  File "/home/paolo/Dropbox/git/web2py/applications/bikend/models/1_extra.py",
line 94, in get_img
if picture and picture.picture:
  File "/home/paolo/Dropbox/git/web2py/gluon/dal.py", line 7273, in __getattr__
self.__allocate()
  File "/home/paolo/Dropbox/git/web2py/gluon/dal.py", line 7266, in __allocate
self._record = self._table[int(self)]
  File "/home/paolo/Dropbox/git/web2py/gluon/dal.py", line 7551, in __getitem__
return self._db(self._id == key).select(limitby=(0,1)).first()
  File "/home/paolo/Dropbox/git/web2py/gluon/dal.py", line 8682, in select
return adapter.select(self.query,fields,attributes)
  File "/home/paolo/Dropbox/git/web2py/gluon/dal.py", line 2022, in select
return super(SQLiteAdapter, self).select(query, fields, attributes)
  File "/home/paolo/Dropbox/git/web2py/gluon/dal.py", line 1522, in select
return self._select_aux(sql,fields,attributes)
  File "/home/paolo/Dropbox/git/web2py/gluon/dal.py", line 1495, in _select_aux
self.execute(sql)
  File "/home/paolo/Dropbox/git/web2py/gluon/dal.py", line 1598, in execute
return self.log_execute(*a, **b)
  File "/home/paolo/Dropbox/git/web2py/gluon/dal.py", line 1592, in log_execute
ret = self.cursor.execute(*a, **b)
ProgrammingError: Cannot operate on a closed database.

I do not know, maybe there is something wrong with my application
rather than with web2py
Paolo


2012/8/30 Massimo Di Pierro :
> I tried this (2.0.2) and I cannot reproduce the problem:
>
> db = DAL(lazy_tables=True)
>
> db.define_table('reviews_like',
>  Field('review_id', 'reference reviews'),
>  Field('helpful', 'boolean'))
>
> db.define_table('reviews',
> Field('title'))
>
> db.reviews.pos = Field.Lazy(lambda row: db((db.reviews_like.review_id == ro\
> w.reviews.id) & (db.reviews_like.helpful == True)).count() )
> db.reviews.neg = Field.Lazy(lambda row: db((db.reviews_like.review_id == ro\
> w.reviews.id) & (db.reviews_like.helpful == False)).count() )
> id = db.reviews.insert(title='xxx')
> db.reviews_like.insert(review_id=id, helpful=True)
> print db(db.reviews_like.review_id==db.reviews.id).select()
>
>
>
> On Thursday, 30 August 2012 11:16:27 UTC-5, Paolo wrote:
>>
>> Dear all,
>> I just updated web2py and after turning on the lazy_table I got:
>>
>> Traceback (most recent call last):
>>   File "/home/paolo/Dropbox/git/web2py/gluon/restricted.py", line 209, in
>> restricted
>> exec ccode in environment
>>   File
>> "/home/paolo/Dropbox/git/web2py/applications/bikend/views/load/reviewsFilter.html",
>> line 57, in 
>> var str = '{{=T('Route ')}}NAME 
>> '.replace(/NAME/gi, $(this).text());
>>   File "/home/paolo/Dropbox/git/web2py/gluon/dal.py", line 8810, in
>> __call__
>> return self.method(self.row,*args,**kwargs)
>>   File "/home/paolo/Dropbox/git/web2py/applications/bikend/models/db.py",
>> line 427, in 
>> db.reviews.pos = Field.Lazy(lambda row: db((db.reviews_like.review_id
>> == row.reviews.id) & (db.reviews_like.helpful == True)).count() )
>>   File "/home/paolo/Dropbox/git/web2py/gluon/dal.py", line 8675, in count
>> return db._adapter.count(self.query,distinct)
>>   File "/home/paolo/Dropbox/git/web2py/gluon/dal.py", line 1542, in count
>> self.execute(self._count(query, distinct))
>>   File "/home/paolo/Dropbox/git/web2py/gluon/dal.py", line 1598, in
>> execute
>> return self.log_execute(*a, **b)
>>   File "/home/paolo/Dropbox/git/web2py/gluon/dal.py", line 1592, in
>> log_execute
>> ret = self.cursor.execute(*a, **b)
>> ProgrammingError: Cannot operate on a closed database.
>>
>>
>> The table involved in the error is defined as follows:
>>
>> db.define_table('reviews_like',
>>  Field('review_id', 'reference reviews'),
>>  Field('helpful', 'boolean',
>>widget = BuildRadioButtonWidget),
>>  Field('created_on', 'datetime', default=request.now,
>>writable = False, readable = False),
>>  Field('user_id', db.auth_user, default=auth.user_id,
>>writable = False, readable = False)
>> )
>>
>> db.define_table('reviews',
>> Field('route_id', '

Re: [web2py] Re: Upload error field *_newfilename does not belong to the table

[paolo.vall...@gmail.com]

Hi Massimo,
With the last trunk is now fixed. thank you!
For what concerns the backward compatibility I haven't used this field either.
Regards, paolo

2012/8/7 Massimo Di Pierro :
> I think this is fixed in trunk, although fixing this may be a backward
> compatibility issues.
>
> When web2py uploads a file (in you case in field picture) the new file name
> is stored in both
>
> form.vars.picture
> form.vars.picture_newfilename (for backward compatibility)
>
> I see the latter is problematic and nobody is probably using it anyway. I am
> removing unless there is a major complaint. It could be treated as a bug.
>
> massimo
>
>
> On Tuesday, 7 August 2012 12:34:16 UTC-5, Paolo wrote:
>>
>> Hi all,
>> I've just checked out the last trunk, I got the following error:
>>
>> 2012-08-07 19:26:01,063 - web2py - ERROR - Traceback (most recent call
>> last):
>>   File "/home/paolo/Desktop/git/web2py/gluon/restricted.py", line 205, in
>> restricted
>> exec ccode in environment
>>   File
>> "/home/paolo/Dropbox/git/web2py/applications/bikend/controllers/club.py",
>> line 125, in 
>>   File "/home/paolo/Desktop/git/web2py/gluon/globals.py", line 182, in
>> 
>> self._caller = lambda f: f()
>>   File "/home/paolo/Desktop/git/web2py/gluon/tools.py", line 2722, in f
>> return call_or_redirect(
>>   File
>> "/home/paolo/Dropbox/git/web2py/applications/bikend/controllers/club.py",
>> line 37, in item
>> m = modal()
>>   File "/home/paolo/Desktop/git/web2py/gluon/tools.py", line 2722, in f
>> return call_or_redirect(
>>   File
>> "/home/paolo/Dropbox/git/web2py/applications/bikend/controllers/club.py",
>> line 57, in modal
>> d['form'] = club_form()
>>   File
>> "/home/paolo/Dropbox/git/web2py/applications/bikend/controllers/club.py",
>> line 104, in club_form
>> form.vars.id = db.club.insert(**dict(form.vars))
>>   File "/home/paolo/Desktop/git/web2py/gluon/dal.py", line 7514, in insert
>>   File "/home/paolo/Desktop/git/web2py/gluon/dal.py", line 7475, in
>> _listify
>> if name != 'id':
>> SyntaxError: Field picture_newfilename does not belong to the table
>>
>> The tables involved are defined as follows:
>>
>> pictures_fields = db.Table(db, 'picture',
>> Field('picture_desc', 'text', requires=[IS_TRIM(),
>> IS_LENGTH(140,error_message=T('Warning, description too long'))]),
>> Field("picture", "upload", autodelete=True, requires=[IS_NOT_EMPTY(),
>> IS_IMAGE(),
>> IS_UPLOAD_FILENAME(extension='jpg|jpeg|png'),IS_IMAGE(extensions=('jpeg',
>> 'png'))]),
>> Field("normal", "upload", autodelete=True, requires=[IS_IMAGE()]),
>> Field("thumbnail", "upload", autodelete=True, requires=[IS_IMAGE()]),
>> )
>> pictures_fields.thumbnail.compute = lambda row: THUMBER(row.picture, 200,
>> 200, name='thumb')
>> pictures_fields.normal.compute = lambda row: THUMBER(row.picture, 600,
>> 600, name='normal')
>> pictures_fields.picture_desc.label = T('Picture description')
>>
>>
>> db.define_table('club',
>> Field('title', 'string', requires=[IS_TRIM(),
>> IS_LENGTH(150,error_message=T('Warning, title too long'))]),
>> Field('description', 'text',
>> requires=[IS_NOT_EMPTY(error_message=T('Please, enter a description')),
>>
>> IS_LENGTH(250,error_message=T('Warning, description too long')),
>>IS_LENGTH(minsize=75,
>> error_message=T('Warning, the description is too short, reviews must be at
>> least 75 characters long.')),
>>IS_TRIM()]),
>> Field("type", requires=IS_IN_SET(['cerco', 'offro']),
>> default='offro'),
>> Field("location", 'string'),
>> pictures_fields,
>> auth.signature
>> )
>> db.club.picture.requires = IS_EMPTY_OR(db.club.picture.requires)
>> db.club.picture_desc.readable = False
>> db.club.picture_desc.writable = False
>> db.club._common_filter = lambda query: (db.club.is_active==True)
>>
>> What could be the problem?
>>
>> Regards,
>> paolo
>
> --
>
>
>



-- 
 Paolo

--