Re: [web2py] Re: wiki: how to cache media files?
Great! I've just tested it and it seems to work very well, I am getting the 304 that I was looking for. Now many resources are no longer sent to the client saving bandwidth, I think many web2py applications will appreciate this speed up. @Niphlod, the header you have mentioned will force to keep the same file even if I update it, this is not the behaviour that I need in my app. One more think about the download function. Do we support parallel requests? Namely, I have recently read that the default web2py approach is to serializes parallel ajax requests due to the session lock. If that is true for download as well, I think we should add an session.forget somwhere. What do you think? Paolo 2013/2/4 Massimo Di Pierro massimo.dipie...@gmail.com In trunk! On Monday, 4 February 2013 15:05:05 UTC-6, Niphlod wrote: Uhm, I was just thinking to return a string instead of the open stream. attached the patch (some refactoring bits added) @Paolo: turns out that wiki has it's own download() function to handle files . in order to return cache headers you should do: def yourwikiindex(): zero = request.args(0) or 'index' if zero and zero.isdigit(): response.headers['Cache-**Control'] = 'max-age=31536' response.headers['Expires'] = 'Thu, 31 Dec 2037 23:59:59 GMT' return auth.wiki() On Monday, February 4, 2013 7:34:58 PM UTC+1, Massimo Di Pierro wrote: Honestly I would like to refactor all this logic in web2py. It was already done for web3py: https://github.com/mdipierro/**web3py/blob/master/web3py/**stream.pyhttps://github.com/mdipierro/web3py/blob/master/web3py/stream.py Anyway, if you can help fix it in web2py, we can merge everything later Massimo On Monday, 4 February 2013 11:46:03 UTC-6, Paolo valleri wrote: I thought the same about postponing the opening of the file. Moreover, just after globals:344 in the stream(...) function I have added if isinstance(stream, file): stream_file_or_304_or_206(stre**am.namehttp://stream.name , **chunk_size=chunk_size, **request=request, **headers=headers, **status=self.status) and finally I got a 304 :-). As a result we are on the right direction, although there is no reason to open the file and take only its name. We need definitely something better Paolo 2013/2/4 Niphlod nip...@gmail.com On Monday, February 4, 2013 3:38:29 PM UTC+1, Paolo valleri wrote: Hi Niphlod, I am still trying to understand why the wiki uploads are not returned as 304. The function in charged to return either 200 or 304 is stream_file_or_304_or_206 in streamer.py. Moreover, this function is only called in globals.py inside stream(...). Please have a look at globals.py:344, In order to be called the parameter 'stream' must be a string but with 'common' downloads the stream is an open file. This is also documented by the comment in dal.py:9096. is what I've written correct ? seems so, but in this case I think it should be patched (as soon as I'll get home). For downloads that on the table are merely strings (i.e. on the table there's the path to the actual file, usually in the uploads folder) the correct way should be to defer the opening of the file down the line to gluon.streamer. A lot of logic has been added to store files on pyfilesystem (and without much tests on that, I hope I won't brake anything) but I think that for default upload file the logic needs to be revisited. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com. For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out . -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: wiki: how to cache media files?
On Tuesday, February 5, 2013 2:49:24 PM UTC+1, Paolo valleri wrote: Great! I've just tested it and it seems to work very well, I am getting the 304 that I was looking for. Now many resources are no longer sent to the client saving bandwidth, I think many web2py applications will appreciate this speed up. @Niphlod, the header you have mentioned will force to keep the same file even if I update it, this is not the behaviour that I need in my app. Tune it s you like. From my POV an embedded media never changes. What changes is the media itself (i.e. if I want to change the image in a page, I upload a new image and change the pointers to it). If it's not your usecase, send cache headers at least one day in the future to avoid users scanning through your wiki to request media again and again. One more think about the download function. Do we support parallel requests? Namely, I have recently read that the default web2py approach is to serializes parallel ajax requests due to the session lock. If that is true for download as well, I think we should add an session.forget somwhere. What do you think? Didn't test it, but you can of course. measure it with session.forget() in the if zero.isdigit() and without -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: wiki: how to cache media files?
I am wondering to understand if from the theoretical point of view is true or not that parallel requests are not handled in parallel. In the meanwhile I will try to monitor the gain that we can have by session.forget in all the downloads(). Paolo 2013/2/5 Niphlod niph...@gmail.com On Tuesday, February 5, 2013 2:49:24 PM UTC+1, Paolo valleri wrote: Great! I've just tested it and it seems to work very well, I am getting the 304 that I was looking for. Now many resources are no longer sent to the client saving bandwidth, I think many web2py applications will appreciate this speed up. @Niphlod, the header you have mentioned will force to keep the same file even if I update it, this is not the behaviour that I need in my app. Tune it s you like. From my POV an embedded media never changes. What changes is the media itself (i.e. if I want to change the image in a page, I upload a new image and change the pointers to it). If it's not your usecase, send cache headers at least one day in the future to avoid users scanning through your wiki to request media again and again. One more think about the download function. Do we support parallel requests? Namely, I have recently read that the default web2py approach is to serializes parallel ajax requests due to the session lock. If that is true for download as well, I think we should add an session.forget somwhere. What do you think? Didn't test it, but you can of course. measure it with session.forget() in the if zero.isdigit() and without -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: wiki: how to cache media files?
Not sure I understand the question. There are two issues. One is about sessions. Distinct requests from the same user/session are serialized unless session._unlock(request) is called. Requests belonging to different session are executed in different threads but to a max number of threads specified by the web server. They are only serielized if they use sqlite and perform write operations because sqlite blocks. Mind that multiple threads running on the same host may be executed under the same python process. In this case they will be concurrent but not parallel because of python GIL. This can be avoided by forking multiple processes. On Tuesday, 5 February 2013 09:14:17 UTC-6, Paolo valleri wrote: I am wondering to understand if from the theoretical point of view is true or not that parallel requests are not handled in parallel. In the meanwhile I will try to monitor the gain that we can have by session.forget in all the downloads(). Paolo 2013/2/5 Niphlod nip...@gmail.com javascript: On Tuesday, February 5, 2013 2:49:24 PM UTC+1, Paolo valleri wrote: Great! I've just tested it and it seems to work very well, I am getting the 304 that I was looking for. Now many resources are no longer sent to the client saving bandwidth, I think many web2py applications will appreciate this speed up. @Niphlod, the header you have mentioned will force to keep the same file even if I update it, this is not the behaviour that I need in my app. Tune it s you like. From my POV an embedded media never changes. What changes is the media itself (i.e. if I want to change the image in a page, I upload a new image and change the pointers to it). If it's not your usecase, send cache headers at least one day in the future to avoid users scanning through your wiki to request media again and again. One more think about the download function. Do we support parallel requests? Namely, I have recently read that the default web2py approach is to serializes parallel ajax requests due to the session lock. If that is true for download as well, I think we should add an session.forget somwhere. What do you think? Didn't test it, but you can of course. measure it with session.forget() in the if zero.isdigit() and without -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com javascript:. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: wiki: how to cache media files?
On Tuesday, February 5, 2013 5:30:11 PM UTC+1, Massimo Di Pierro wrote: Not sure I understand the question. I think the doubt lies into the fact that given 1 request for the wiki page and ,e.g., 4 requests for 4 embedded images the 5 requests would be concurrent or serialized. In my apps whenever I don't use session I put at the top a session.forget(), and I know for a fact that static files are handled concurrently (they are outside session logic in main.py). In theory to download something (either in the default download() or in the catch of the wiki) session is not touched, so in case the requests are in fact serialized a session.forget() could/should be added. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: wiki: how to cache media files?
Niphlod has written exactly what I have in mind. Given the fact that distinct requests from the same user/session are serialized as Massimo has outlined from my point of view we should add session.forget() or session._unlock(request) as early as possible in the download() function of globals.py @Massimo, which are the difference between session.forget(request) and session._unlock(request)? Which one is the better to use in this case? Paolo 2013/2/5 Niphlod niph...@gmail.com On Tuesday, February 5, 2013 5:30:11 PM UTC+1, Massimo Di Pierro wrote: Not sure I understand the question. I think the doubt lies into the fact that given 1 request for the wiki page and ,e.g., 4 requests for 4 embedded images the 5 requests would be concurrent or serialized. In my apps whenever I don't use session I put at the top a session.forget(), and I know for a fact that static files are handled concurrently (they are outside session logic in main.py). In theory to download something (either in the default download() or in the catch of the wiki) session is not touched, so in case the requests are in fact serialized a session.forget() could/should be added. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: wiki: how to cache media files?
I agree. In trunk. they both unlock but session.forget(response) does not save the session. session._unlock(response) instead still saves the session when the request is completed. On Tuesday, 5 February 2013 12:33:03 UTC-6, Paolo valleri wrote: Niphlod has written exactly what I have in mind. Given the fact that distinct requests from the same user/session are serialized as Massimo has outlined from my point of view we should add session.forget() or session._unlock(request) as early as possible in the download() function of globals.py @Massimo, which are the difference between session.forget(request) and session._unlock(request)? Which one is the better to use in this case? Paolo 2013/2/5 Niphlod nip...@gmail.com javascript: On Tuesday, February 5, 2013 5:30:11 PM UTC+1, Massimo Di Pierro wrote: Not sure I understand the question. I think the doubt lies into the fact that given 1 request for the wiki page and ,e.g., 4 requests for 4 embedded images the 5 requests would be concurrent or serialized. In my apps whenever I don't use session I put at the top a session.forget(), and I know for a fact that static files are handled concurrently (they are outside session logic in main.py). In theory to download something (either in the default download() or in the catch of the wiki) session is not touched, so in case the requests are in fact serialized a session.forget() could/should be added. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com javascript:. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: wiki: how to cache media files?
Hi Niphlod, I am still trying to understand why the wiki uploads are not returned as 304. The function in charged to return either 200 or 304 is stream_file_or_304_or_206 in streamer.py. Moreover, this function is only called in globals.py inside stream(...). Please have a look at globals.py:344, In order to be called the parameter 'stream' must be a string but with 'common' downloads the stream is an open file. This is also documented by the comment in dal.py:9096. is what I've written correct ? Paolo 2013/2/1 Niphlod niph...@gmail.com modified_on is basically the same info that is the mtime on the file. This tells you when the file has been modified, so you can return a 304, but doesn't help you with the cache. In theory all files served through download() (minus the one stored on a blob in a table) should work ok without needing any further patch (i.e. the automation is there) What is needed to set cache headers (and avoid the browser to request files) is some sort of will_be_modified_on, so you can set correctly the expiration without guessing. On Friday, February 1, 2013 5:42:59 PM UTC+1, Paolo valleri wrote: Something that handles the headers automatically accordingly to the wiki media modified_on field. I will try somenthing next week (i am out this weekend) Il giorno 01/feb/2013 16:59, Niphlod nip...@gmail.com ha scritto: what automation ? On Friday, February 1, 2013 4:51:54 PM UTC+1, Paolo valleri wrote: I get it,thank for explaing. I tryed a test,it seems that main.py overwrites the default headers setting them as no-cache,no-store and so on. I have to test it more. Moreover wiki media are handled by users,i don't know how they behave with them. Something 'automatic' would be really nice to have. Paolo Il giorno 01/feb/2013 15:47, Niphlod nip...@gmail.com ha scritto: let me explain better (PS: untested but should work that way) every file that is returned by the download() function does not carry any of the cache headers (to be fair, it includes one that basically says it expires now) 304 + no content or 200 + the entire content is a step lower when a request is made with the If-Modified-Since header will check the mtime of the file and response.stream() will handle that (the logic is defined in gluon/streamer.py), returning 304 if the file has not been modified or a 200 if the file is indeed more recent. However, cache headers are useful to avoid even the requests that will end in a probable 304. In a static wiki with images, if you're sure that the published images will never change, you can enforce cache headers that will set the item to expire in some point in the future when cache headers are returned the browser will avoid a request with the if-modified-since header entirely. To return cache headers, you can use something like (taken from static asset management in gluon/main.py) response.headers['Cache-**Contro**l'] = 'max-age=31536' response.headers['Expires'] = 'Thu, 31 Dec 2037 23:59:59 GMT' This will ensure that the item would not been requested until 31 Dec 2037 (the effect is that a user will download the image one time only and will never request that image to the server again). On Friday, February 1, 2013 1:40:07 PM UTC+1, Paolo valleri wrote: Hi Niphlod, I don't know very well which header I have to set, can you give me an example? The 304 is sent by the server to the user accordingly to the user's request. Given that, If my upload file as a modified_on field I can actually understand if I have to return the whole file (http: 200) or a 304. Is it correct? If yes, I would propose to implement something like that for download(): if the field as the modified_on field we can decide between 200/304, is the field doesn't have the modified_on field we return always a 200. Paolo 2013/2/1 Niphlod nip...@gmail.com you have to alter the default download() function to return cache headers. The theoretical problem is that web2py (and then the wiki) doesn't know when the image stored in a table would be updated, so it doesn't issue any cache headers by default. On Thursday, January 31, 2013 9:49:21 PM UTC+1, Paolo valleri wrote: Hi all, today I discovered that wiki media, actually they are all images, they are always requested to the server and never cached by the client. In other words, even if I have already visited the page I do a get to download all the images instead of using those in the browser cache as commonly happens with for js or css. Is it a problem of my configuration or of wiki it self? paolo -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@**googlegroups.com. For more options, visit https://groups.google.com/**grou ps/opt_out https://groups.google.com/groups/opt_out. -- --- You
Re: [web2py] Re: wiki: how to cache media files?
On Monday, February 4, 2013 3:38:29 PM UTC+1, Paolo valleri wrote: Hi Niphlod, I am still trying to understand why the wiki uploads are not returned as 304. The function in charged to return either 200 or 304 is stream_file_or_304_or_206 in streamer.py. Moreover, this function is only called in globals.py inside stream(...). Please have a look at globals.py:344, In order to be called the parameter 'stream' must be a string but with 'common' downloads the stream is an open file. This is also documented by the comment in dal.py:9096. is what I've written correct ? seems so, but in this case I think it should be patched (as soon as I'll get home). For downloads that on the table are merely strings (i.e. on the table there's the path to the actual file, usually in the uploads folder) the correct way should be to defer the opening of the file down the line to gluon.streamer. A lot of logic has been added to store files on pyfilesystem (and without much tests on that, I hope I won't brake anything) but I think that for default upload file the logic needs to be revisited. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: wiki: how to cache media files?
I thought the same about postponing the opening of the file. Moreover, just after globals:344 in the stream(...) function I have added if isinstance(stream, file): stream_file_or_304_or_206(stream.name, chunk_size=chunk_size, request=request, headers=headers, status=self.status) and finally I got a 304 :-). As a result we are on the right direction, although there is no reason to open the file and take only its name. We need definitely something better Paolo 2013/2/4 Niphlod niph...@gmail.com On Monday, February 4, 2013 3:38:29 PM UTC+1, Paolo valleri wrote: Hi Niphlod, I am still trying to understand why the wiki uploads are not returned as 304. The function in charged to return either 200 or 304 is stream_file_or_304_or_206 in streamer.py. Moreover, this function is only called in globals.py inside stream(...). Please have a look at globals.py:344, In order to be called the parameter 'stream' must be a string but with 'common' downloads the stream is an open file. This is also documented by the comment in dal.py:9096. is what I've written correct ? seems so, but in this case I think it should be patched (as soon as I'll get home). For downloads that on the table are merely strings (i.e. on the table there's the path to the actual file, usually in the uploads folder) the correct way should be to defer the opening of the file down the line to gluon.streamer. A lot of logic has been added to store files on pyfilesystem (and without much tests on that, I hope I won't brake anything) but I think that for default upload file the logic needs to be revisited. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: wiki: how to cache media files?
Honestly I would like to refactor all this logic in web2py. It was already done for web3py: https://github.com/mdipierro/web3py/blob/master/web3py/stream.py Anyway, if you can help fix it in web2py, we can merge everything later Massimo On Monday, 4 February 2013 11:46:03 UTC-6, Paolo valleri wrote: I thought the same about postponing the opening of the file. Moreover, just after globals:344 in the stream(...) function I have added if isinstance(stream, file): stream_file_or_304_or_206(stream.name, chunk_size=chunk_size, request=request, headers=headers, status=self.status) and finally I got a 304 :-). As a result we are on the right direction, although there is no reason to open the file and take only its name. We need definitely something better Paolo 2013/2/4 Niphlod nip...@gmail.com javascript: On Monday, February 4, 2013 3:38:29 PM UTC+1, Paolo valleri wrote: Hi Niphlod, I am still trying to understand why the wiki uploads are not returned as 304. The function in charged to return either 200 or 304 is stream_file_or_304_or_206 in streamer.py. Moreover, this function is only called in globals.py inside stream(...). Please have a look at globals.py:344, In order to be called the parameter 'stream' must be a string but with 'common' downloads the stream is an open file. This is also documented by the comment in dal.py:9096. is what I've written correct ? seems so, but in this case I think it should be patched (as soon as I'll get home). For downloads that on the table are merely strings (i.e. on the table there's the path to the actual file, usually in the uploads folder) the correct way should be to defer the opening of the file down the line to gluon.streamer. A lot of logic has been added to store files on pyfilesystem (and without much tests on that, I hope I won't brake anything) but I think that for default upload file the logic needs to be revisited. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com javascript:. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: wiki: how to cache media files?
Uhm, I was just thinking to return a string instead of the open stream.
attached the patch (some refactoring bits added)
@Paolo: turns out that wiki has it's own download() function to handle
files . in order to return cache headers you should do:
def yourwikiindex():
zero = request.args(0) or 'index'
if zero and zero.isdigit():
response.headers['Cache-Control'] = 'max-age=31536'
response.headers['Expires'] = 'Thu, 31 Dec 2037 23:59:59 GMT'
return auth.wiki()
On Monday, February 4, 2013 7:34:58 PM UTC+1, Massimo Di Pierro wrote:
Honestly I would like to refactor all this logic in web2py. It was already
done for web3py:
https://github.com/mdipierro/web3py/blob/master/web3py/stream.py
Anyway, if you can help fix it in web2py, we can merge everything later
Massimo
On Monday, 4 February 2013 11:46:03 UTC-6, Paolo valleri wrote:
I thought the same about postponing the opening of the file. Moreover,
just after globals:344 in the stream(...) function I have added
if isinstance(stream, file):
stream_file_or_304_or_206(stream.name,
chunk_size=chunk_size,
request=request,
headers=headers,
status=self.status)
and finally I got a 304 :-). As a result we are on the right direction,
although there is no reason to open the file and take only its name. We
need definitely something better
Paolo
2013/2/4 Niphlod nip...@gmail.com
On Monday, February 4, 2013 3:38:29 PM UTC+1, Paolo valleri wrote:
Hi Niphlod, I am still trying to understand why the wiki uploads are
not returned as 304.
The function in charged to return either 200 or 304 is
stream_file_or_304_or_206 in streamer.py. Moreover, this function is only
called in globals.py inside stream(...). Please have a look at
globals.py:344, In order to be called the parameter 'stream' must be a
string but with 'common' downloads the stream is an open file. This is
also
documented by the comment in dal.py:9096. is what I've written correct ?
seems so, but in this case I think it should be patched (as soon as I'll
get home).
For downloads that on the table are merely strings (i.e. on the table
there's the path to the actual file, usually in the uploads folder) the
correct way should be to defer the opening of the file down the line to
gluon.streamer.
A lot of logic has been added to store files on pyfilesystem (and
without much tests on that, I hope I won't brake anything) but I think that
for default upload file the logic needs to be revisited.
--
---
You received this message because you are subscribed to the Google
Groups web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send
an email to web2py+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
diff -r f137cf18b70a gluon/dal.py
--- a/gluon/dal.py lun feb 04 10:03:29 2013 -0600
+++ b/gluon/dal.py lun feb 04 22:01:31 2013 +0100
@@ -876,7 +876,7 @@
# The reason is that we do not want to trigger
# a migration simply because a default value changes.
not_null = self.NOT_NULL(field.default, field_type)
-ftype = ftype.replace('NOT NULL', not_null)
+ftype = ftype.replace('NOT NULL', not_null)
sql_fields_aux[field_name] = dict(sql=ftype)
# Postgres - PostGIS:
# geometry fields are added after the table has been created, not now
@@ -1234,7 +1234,7 @@
self.expand('%'+second, 'string'))
def CONTAINS(self, first, second, case_sensitive=False):
-if isinstance(second,Expression):
+if isinstance(second,Expression):
field = self.expand(first)
expr = self.expand(second,'string')
if first.type.startswith('list:'):
@@ -3265,7 +3265,7 @@
def execute(self,a):
return self.log_execute(a.decode('utf8'))
-
+
class SybaseAdapter(MSSQLAdapter):
drivers = ('Sybase',)
@@ -5037,7 +5037,7 @@
'list:reference': list,
}
-error_messages = {javascript_needed: This must yet be replaced +
+error_messages = {javascript_needed: This must yet be replaced +
with javascript in order to work.}
def __init__(self,db,uri='mongodb://127.0.0.1:5984/db',
@@ -5116,12 +5116,12 @@
raise ValueError(
invalid objectid
Re: [web2py] Re: wiki: how to cache media files?
In trunk! On Monday, 4 February 2013 15:05:05 UTC-6, Niphlod wrote: Uhm, I was just thinking to return a string instead of the open stream. attached the patch (some refactoring bits added) @Paolo: turns out that wiki has it's own download() function to handle files . in order to return cache headers you should do: def yourwikiindex(): zero = request.args(0) or 'index' if zero and zero.isdigit(): response.headers['Cache-Control'] = 'max-age=31536' response.headers['Expires'] = 'Thu, 31 Dec 2037 23:59:59 GMT' return auth.wiki() On Monday, February 4, 2013 7:34:58 PM UTC+1, Massimo Di Pierro wrote: Honestly I would like to refactor all this logic in web2py. It was already done for web3py: https://github.com/mdipierro/web3py/blob/master/web3py/stream.py Anyway, if you can help fix it in web2py, we can merge everything later Massimo On Monday, 4 February 2013 11:46:03 UTC-6, Paolo valleri wrote: I thought the same about postponing the opening of the file. Moreover, just after globals:344 in the stream(...) function I have added if isinstance(stream, file): stream_file_or_304_or_206(stream.name, chunk_size=chunk_size, request=request, headers=headers, status=self.status) and finally I got a 304 :-). As a result we are on the right direction, although there is no reason to open the file and take only its name. We need definitely something better Paolo 2013/2/4 Niphlod nip...@gmail.com On Monday, February 4, 2013 3:38:29 PM UTC+1, Paolo valleri wrote: Hi Niphlod, I am still trying to understand why the wiki uploads are not returned as 304. The function in charged to return either 200 or 304 is stream_file_or_304_or_206 in streamer.py. Moreover, this function is only called in globals.py inside stream(...). Please have a look at globals.py:344, In order to be called the parameter 'stream' must be a string but with 'common' downloads the stream is an open file. This is also documented by the comment in dal.py:9096. is what I've written correct ? seems so, but in this case I think it should be patched (as soon as I'll get home). For downloads that on the table are merely strings (i.e. on the table there's the path to the actual file, usually in the uploads folder) the correct way should be to defer the opening of the file down the line to gluon.streamer. A lot of logic has been added to store files on pyfilesystem (and without much tests on that, I hope I won't brake anything) but I think that for default upload file the logic needs to be revisited. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: wiki: how to cache media files?
you have to alter the default download() function to return cache headers. The theoretical problem is that web2py (and then the wiki) doesn't know when the image stored in a table would be updated, so it doesn't issue any cache headers by default. On Thursday, January 31, 2013 9:49:21 PM UTC+1, Paolo valleri wrote: Hi all, today I discovered that wiki media, actually they are all images, they are always requested to the server and never cached by the client. In other words, even if I have already visited the page I do a get to download all the images instead of using those in the browser cache as commonly happens with for js or css. Is it a problem of my configuration or of wiki it self? paolo -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: wiki: how to cache media files?
Hi Niphlod, I don't know very well which header I have to set, can you give me an example? The 304 is sent by the server to the user accordingly to the user's request. Given that, If my upload file as a modified_on field I can actually understand if I have to return the whole file (http: 200) or a 304. Is it correct? If yes, I would propose to implement something like that for download(): if the field as the modified_on field we can decide between 200/304, is the field doesn't have the modified_on field we return always a 200. Paolo 2013/2/1 Niphlod niph...@gmail.com you have to alter the default download() function to return cache headers. The theoretical problem is that web2py (and then the wiki) doesn't know when the image stored in a table would be updated, so it doesn't issue any cache headers by default. On Thursday, January 31, 2013 9:49:21 PM UTC+1, Paolo valleri wrote: Hi all, today I discovered that wiki media, actually they are all images, they are always requested to the server and never cached by the client. In other words, even if I have already visited the page I do a get to download all the images instead of using those in the browser cache as commonly happens with for js or css. Is it a problem of my configuration or of wiki it self? paolo -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: wiki: how to cache media files?
let me explain better (PS: untested but should work that way) every file that is returned by the download() function does not carry any of the cache headers (to be fair, it includes one that basically says it expires now) 304 + no content or 200 + the entire content is a step lower when a request is made with the If-Modified-Since header will check the mtime of the file and response.stream() will handle that (the logic is defined in gluon/streamer.py), returning 304 if the file has not been modified or a 200 if the file is indeed more recent. However, cache headers are useful to avoid even the requests that will end in a probable 304. In a static wiki with images, if you're sure that the published images will never change, you can enforce cache headers that will set the item to expire in some point in the future when cache headers are returned the browser will avoid a request with the if-modified-since header entirely. To return cache headers, you can use something like (taken from static asset management in gluon/main.py) response.headers['Cache-Control'] = 'max-age=31536' response.headers['Expires'] = 'Thu, 31 Dec 2037 23:59:59 GMT' This will ensure that the item would not been requested until 31 Dec 2037 (the effect is that a user will download the image one time only and will never request that image to the server again). On Friday, February 1, 2013 1:40:07 PM UTC+1, Paolo valleri wrote: Hi Niphlod, I don't know very well which header I have to set, can you give me an example? The 304 is sent by the server to the user accordingly to the user's request. Given that, If my upload file as a modified_on field I can actually understand if I have to return the whole file (http: 200) or a 304. Is it correct? If yes, I would propose to implement something like that for download(): if the field as the modified_on field we can decide between 200/304, is the field doesn't have the modified_on field we return always a 200. Paolo 2013/2/1 Niphlod nip...@gmail.com javascript: you have to alter the default download() function to return cache headers. The theoretical problem is that web2py (and then the wiki) doesn't know when the image stored in a table would be updated, so it doesn't issue any cache headers by default. On Thursday, January 31, 2013 9:49:21 PM UTC+1, Paolo valleri wrote: Hi all, today I discovered that wiki media, actually they are all images, they are always requested to the server and never cached by the client. In other words, even if I have already visited the page I do a get to download all the images instead of using those in the browser cache as commonly happens with for js or css. Is it a problem of my configuration or of wiki it self? paolo -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com javascript:. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: wiki: how to cache media files?
I get it,thank for explaing. I tryed a test,it seems that main.py overwrites the default headers setting them as no-cache,no-store and so on. I have to test it more. Moreover wiki media are handled by users,i don't know how they behave with them. Something 'automatic' would be really nice to have. Paolo Il giorno 01/feb/2013 15:47, Niphlod niph...@gmail.com ha scritto: let me explain better (PS: untested but should work that way) every file that is returned by the download() function does not carry any of the cache headers (to be fair, it includes one that basically says it expires now) 304 + no content or 200 + the entire content is a step lower when a request is made with the If-Modified-Since header will check the mtime of the file and response.stream() will handle that (the logic is defined in gluon/streamer.py), returning 304 if the file has not been modified or a 200 if the file is indeed more recent. However, cache headers are useful to avoid even the requests that will end in a probable 304. In a static wiki with images, if you're sure that the published images will never change, you can enforce cache headers that will set the item to expire in some point in the future when cache headers are returned the browser will avoid a request with the if-modified-since header entirely. To return cache headers, you can use something like (taken from static asset management in gluon/main.py) response.headers['Cache-Control'] = 'max-age=31536' response.headers['Expires'] = 'Thu, 31 Dec 2037 23:59:59 GMT' This will ensure that the item would not been requested until 31 Dec 2037 (the effect is that a user will download the image one time only and will never request that image to the server again). On Friday, February 1, 2013 1:40:07 PM UTC+1, Paolo valleri wrote: Hi Niphlod, I don't know very well which header I have to set, can you give me an example? The 304 is sent by the server to the user accordingly to the user's request. Given that, If my upload file as a modified_on field I can actually understand if I have to return the whole file (http: 200) or a 304. Is it correct? If yes, I would propose to implement something like that for download(): if the field as the modified_on field we can decide between 200/304, is the field doesn't have the modified_on field we return always a 200. Paolo 2013/2/1 Niphlod nip...@gmail.com you have to alter the default download() function to return cache headers. The theoretical problem is that web2py (and then the wiki) doesn't know when the image stored in a table would be updated, so it doesn't issue any cache headers by default. On Thursday, January 31, 2013 9:49:21 PM UTC+1, Paolo valleri wrote: Hi all, today I discovered that wiki media, actually they are all images, they are always requested to the server and never cached by the client. In other words, even if I have already visited the page I do a get to download all the images instead of using those in the browser cache as commonly happens with for js or css. Is it a problem of my configuration or of wiki it self? paolo -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@**googlegroups.com. For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out . -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: wiki: how to cache media files?
what automation ? On Friday, February 1, 2013 4:51:54 PM UTC+1, Paolo valleri wrote: I get it,thank for explaing. I tryed a test,it seems that main.py overwrites the default headers setting them as no-cache,no-store and so on. I have to test it more. Moreover wiki media are handled by users,i don't know how they behave with them. Something 'automatic' would be really nice to have. Paolo Il giorno 01/feb/2013 15:47, Niphlod nip...@gmail.com javascript: ha scritto: let me explain better (PS: untested but should work that way) every file that is returned by the download() function does not carry any of the cache headers (to be fair, it includes one that basically says it expires now) 304 + no content or 200 + the entire content is a step lower when a request is made with the If-Modified-Since header will check the mtime of the file and response.stream() will handle that (the logic is defined in gluon/streamer.py), returning 304 if the file has not been modified or a 200 if the file is indeed more recent. However, cache headers are useful to avoid even the requests that will end in a probable 304. In a static wiki with images, if you're sure that the published images will never change, you can enforce cache headers that will set the item to expire in some point in the future when cache headers are returned the browser will avoid a request with the if-modified-since header entirely. To return cache headers, you can use something like (taken from static asset management in gluon/main.py) response.headers['Cache-Control'] = 'max-age=31536' response.headers['Expires'] = 'Thu, 31 Dec 2037 23:59:59 GMT' This will ensure that the item would not been requested until 31 Dec 2037 (the effect is that a user will download the image one time only and will never request that image to the server again). On Friday, February 1, 2013 1:40:07 PM UTC+1, Paolo valleri wrote: Hi Niphlod, I don't know very well which header I have to set, can you give me an example? The 304 is sent by the server to the user accordingly to the user's request. Given that, If my upload file as a modified_on field I can actually understand if I have to return the whole file (http: 200) or a 304. Is it correct? If yes, I would propose to implement something like that for download(): if the field as the modified_on field we can decide between 200/304, is the field doesn't have the modified_on field we return always a 200. Paolo 2013/2/1 Niphlod nip...@gmail.com you have to alter the default download() function to return cache headers. The theoretical problem is that web2py (and then the wiki) doesn't know when the image stored in a table would be updated, so it doesn't issue any cache headers by default. On Thursday, January 31, 2013 9:49:21 PM UTC+1, Paolo valleri wrote: Hi all, today I discovered that wiki media, actually they are all images, they are always requested to the server and never cached by the client. In other words, even if I have already visited the page I do a get to download all the images instead of using those in the browser cache as commonly happens with for js or css. Is it a problem of my configuration or of wiki it self? paolo -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@**googlegroups.com. For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out . -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com javascript:. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: wiki: how to cache media files?
Something that handles the headers automatically accordingly to the wiki media modified_on field. I will try somenthing next week (i am out this weekend) Il giorno 01/feb/2013 16:59, Niphlod niph...@gmail.com ha scritto: what automation ? On Friday, February 1, 2013 4:51:54 PM UTC+1, Paolo valleri wrote: I get it,thank for explaing. I tryed a test,it seems that main.py overwrites the default headers setting them as no-cache,no-store and so on. I have to test it more. Moreover wiki media are handled by users,i don't know how they behave with them. Something 'automatic' would be really nice to have. Paolo Il giorno 01/feb/2013 15:47, Niphlod nip...@gmail.com ha scritto: let me explain better (PS: untested but should work that way) every file that is returned by the download() function does not carry any of the cache headers (to be fair, it includes one that basically says it expires now) 304 + no content or 200 + the entire content is a step lower when a request is made with the If-Modified-Since header will check the mtime of the file and response.stream() will handle that (the logic is defined in gluon/streamer.py), returning 304 if the file has not been modified or a 200 if the file is indeed more recent. However, cache headers are useful to avoid even the requests that will end in a probable 304. In a static wiki with images, if you're sure that the published images will never change, you can enforce cache headers that will set the item to expire in some point in the future when cache headers are returned the browser will avoid a request with the if-modified-since header entirely. To return cache headers, you can use something like (taken from static asset management in gluon/main.py) response.headers['Cache-**Control'] = 'max-age=31536' response.headers['Expires'] = 'Thu, 31 Dec 2037 23:59:59 GMT' This will ensure that the item would not been requested until 31 Dec 2037 (the effect is that a user will download the image one time only and will never request that image to the server again). On Friday, February 1, 2013 1:40:07 PM UTC+1, Paolo valleri wrote: Hi Niphlod, I don't know very well which header I have to set, can you give me an example? The 304 is sent by the server to the user accordingly to the user's request. Given that, If my upload file as a modified_on field I can actually understand if I have to return the whole file (http: 200) or a 304. Is it correct? If yes, I would propose to implement something like that for download(): if the field as the modified_on field we can decide between 200/304, is the field doesn't have the modified_on field we return always a 200. Paolo 2013/2/1 Niphlod nip...@gmail.com you have to alter the default download() function to return cache headers. The theoretical problem is that web2py (and then the wiki) doesn't know when the image stored in a table would be updated, so it doesn't issue any cache headers by default. On Thursday, January 31, 2013 9:49:21 PM UTC+1, Paolo valleri wrote: Hi all, today I discovered that wiki media, actually they are all images, they are always requested to the server and never cached by the client. In other words, even if I have already visited the page I do a get to download all the images instead of using those in the browser cache as commonly happens with for js or css. Is it a problem of my configuration or of wiki it self? paolo -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@**googlegroups.com. For more options, visit https://groups.google.com/**grou**ps/opt_outhttps://groups.google.com/groups/opt_out . -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@**googlegroups.com. For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out . -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: wiki: how to cache media files?
modified_on is basically the same info that is the mtime on the file. This tells you when the file has been modified, so you can return a 304, but doesn't help you with the cache. In theory all files served through download() (minus the one stored on a blob in a table) should work ok without needing any further patch (i.e. the automation is there) What is needed to set cache headers (and avoid the browser to request files) is some sort of will_be_modified_on, so you can set correctly the expiration without guessing. On Friday, February 1, 2013 5:42:59 PM UTC+1, Paolo valleri wrote: Something that handles the headers automatically accordingly to the wiki media modified_on field. I will try somenthing next week (i am out this weekend) Il giorno 01/feb/2013 16:59, Niphlod nip...@gmail.com javascript: ha scritto: what automation ? On Friday, February 1, 2013 4:51:54 PM UTC+1, Paolo valleri wrote: I get it,thank for explaing. I tryed a test,it seems that main.py overwrites the default headers setting them as no-cache,no-store and so on. I have to test it more. Moreover wiki media are handled by users,i don't know how they behave with them. Something 'automatic' would be really nice to have. Paolo Il giorno 01/feb/2013 15:47, Niphlod nip...@gmail.com ha scritto: let me explain better (PS: untested but should work that way) every file that is returned by the download() function does not carry any of the cache headers (to be fair, it includes one that basically says it expires now) 304 + no content or 200 + the entire content is a step lower when a request is made with the If-Modified-Since header will check the mtime of the file and response.stream() will handle that (the logic is defined in gluon/streamer.py), returning 304 if the file has not been modified or a 200 if the file is indeed more recent. However, cache headers are useful to avoid even the requests that will end in a probable 304. In a static wiki with images, if you're sure that the published images will never change, you can enforce cache headers that will set the item to expire in some point in the future when cache headers are returned the browser will avoid a request with the if-modified-since header entirely. To return cache headers, you can use something like (taken from static asset management in gluon/main.py) response.headers['Cache-**Control'] = 'max-age=31536' response.headers['Expires'] = 'Thu, 31 Dec 2037 23:59:59 GMT' This will ensure that the item would not been requested until 31 Dec 2037 (the effect is that a user will download the image one time only and will never request that image to the server again). On Friday, February 1, 2013 1:40:07 PM UTC+1, Paolo valleri wrote: Hi Niphlod, I don't know very well which header I have to set, can you give me an example? The 304 is sent by the server to the user accordingly to the user's request. Given that, If my upload file as a modified_on field I can actually understand if I have to return the whole file (http: 200) or a 304. Is it correct? If yes, I would propose to implement something like that for download(): if the field as the modified_on field we can decide between 200/304, is the field doesn't have the modified_on field we return always a 200. Paolo 2013/2/1 Niphlod nip...@gmail.com you have to alter the default download() function to return cache headers. The theoretical problem is that web2py (and then the wiki) doesn't know when the image stored in a table would be updated, so it doesn't issue any cache headers by default. On Thursday, January 31, 2013 9:49:21 PM UTC+1, Paolo valleri wrote: Hi all, today I discovered that wiki media, actually they are all images, they are always requested to the server and never cached by the client. In other words, even if I have already visited the page I do a get to download all the images instead of using those in the browser cache as commonly happens with for js or css. Is it a problem of my configuration or of wiki it self? paolo -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@**googlegroups.com. For more options, visit https://groups.google.com/**grou**ps/opt_outhttps://groups.google.com/groups/opt_out . -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@**googlegroups.com. For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out . -- --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving
