Re: [RFC] HttpResponse: freeze API, read_once
Hi Luke, On Tue, Sep 14, 2010 at 01:22:37AM +0100, Luke Plant wrote: > you wrote: > > > > I'd like to return to the problem of middleware, view decorators, etc. > > affecting > > responses in negative ways. > > Thanks so much for all your hard work and persistence on this issue. Thanks. Persistence is one of my few good traits. ;) > Although I haven't gone through this in detail, I think this is arriving > at a good solution now. I massively prefer the explicit 'freeze' API on > HttpResponse to subclasses that do different things. It's no silver bullet, but it's probably good enough to cover the majority of use cases without making things complicated. I'm curious to hear what people think. > It would be extremely useful to me if you could go through the use cases > linked on the previous thread and see if your solution addressed them > all. I'll get an e-mail out tomorrow with a good overview of each use case I know of and how well the freeze API would deal with them. > > I am more than happy to provide an implementation of the above, but I'd > > like to > > get an idea of what chance it has of being accepted before I start working > > on > > it. I am also happy to maintain it outside of Django svn for a while so > > that > > people can test it out. > > From my perspective that would be great. One thing that would be > brilliant whatever the eventual solution would be a set of use cases > encoded into some tests - I suspect that whatever solution/API is chosen > it would be worth doing the tests first so that you can see if it is > going to work in practice, and so that other people can see how to use > it. That's a great idea. Probably step one then would be to write up some test cases and make sure we all agree about those. Once I hear what people have to say about the proposed API, I'll start on that. Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: get_next_by_FOO allows nulls
On Sun, Sep 12, 2010 at 3:13 PM, Alex Gaynor wrote: > On Sun, Sep 12, 2010 at 1:09 PM, Carl Karsten wrote: >> On Thu, Aug 26, 2010 at 8:12 PM, Russell Keith-Magee >> wrote: >>> On Fri, Aug 27, 2010 at 2:12 AM, Carl Karsten >>> wrote: I would like to discus my patch to add support of nullable fields to get_next_by_FOO http://code.djangoproject.com/ticket/13611 > You say "we" discussed this on IRC, but you don't say who "we" are. I forget who was involved in the IRC discussion - the point was that it was a lengthy debate and a conclusion was reached, which means quite a bit of thought and research was put into it; so yes: I am passionate about this and will be happy to discuss it more. >>> >>> Sure - but it makes a difference if "we" is you and three close >>> friends with no history of involvement in the Django community, or you >>> and three other core developers. >>> >>> I've said my piece on the ticket; at this point, I need to hear the >>> opinion of others. >>> >>> >> >> Alex, >> >> You have at least thought about this, so what's your opinion? >> >> >> -- >> Carl K >> > > Basically my thoughts amount to: it should be possible, I'm not > convinced on which direction we should assume nulls to be, whether > they should be included by default, and whether or not we just choose > a position, or add some flags for it. > > Alex If the code is going to address how nulls are handled, I vote strongly for adding flags. I like this idea too - it basically moves the 12 lines of fairly generic app code into core which will make my app code much easier to read. I also think it will make the docs easier to understand: my current implementation results in: "you figure it out and code it in your app" which would leave me wondering wtf. having the 3 options already coded and documented results in the docs saying "Here are your 3 options... pick 1." So lets talk api. backwards compatibility: It only has to work the same for non nulable fields. If someone is using get_next_by_foo changes foo to allow nulls, I am not sure what to do. currently that will prevent get_next_by_foo from being created, so their code will AttributeError. I don't think we need to maintain that behaviour. But I am not against throwing some exception because I am hesitant to pass nulls back to the calling code, so I would say the default is option 3 below. behaviour: I see 3 choices: 1. nulls come before values. 2. nulls come after values. 3. you can't get there from here. It will be like there are 2 sequences: one for the values, one for the nulls. This is how my current implementation works: if you are on the last value and call _next you get DoesNotExist exception. Same if you are on the last null (nulls ordered by ID.) So if you are on a value you will never end up on a null, if you are on a null you will never end up on a value. If the field does not support nulls, all of the options will behave the same. So we need to do something like: get_next_by_foo( nulls_come=BEFORE ) get_next_by_foo( nulls_come=AFTER ) get_next_by_foo( nulls_come=IN_ANOTHER_UNIVERSE ) I could use some help with the last one, and maybe the exact wording of all 3. Is this in mind with what you were thinking? If so, I'll be happy to get coding. -- Carl K -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: [RFC] HttpResponse: freeze API, read_once
Hi Forest, you wrote: > > I'd like to return to the problem of middleware, view decorators, etc. > affecting > responses in negative ways. Thanks so much for all your hard work and persistence on this issue. Although I haven't gone through this in detail, I think this is arriving at a good solution now. I massively prefer the explicit 'freeze' API on HttpResponse to subclasses that do different things. It would be extremely useful to me if you could go through the use cases linked on the previous thread and see if your solution addressed them all. > I am more than happy to provide an implementation of the above, but I'd like > to > get an idea of what chance it has of being accepted before I start working on > it. I am also happy to maintain it outside of Django svn for a while so that > people can test it out. >From my perspective that would be great. One thing that would be brilliant whatever the eventual solution would be a set of use cases encoded into some tests - I suspect that whatever solution/API is chosen it would be worth doing the tests first so that you can see if it is going to work in practice, and so that other people can see how to use it. Many thanks. Luke -- When you're great, people often mistake candor for bragging. -- Calvin and Hobbes Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: My problem with proposed Custom FilterSpec implementation
Maybe code talks? I added a patch to http://code.djangoproject.com/ticket/5833 which sums up the kind of change I am gunning for. If anyone has some feedback, awesome. -Steve On Sep 7, 3:22 pm, "subs...@gmail.com" wrote: > Hello there, > > I'm watching #5833 and I have a bit of a problem with the > implementation. A good example of their proposal is here: > > http://code.djangoproject.com/ticket/5833#comment:68 > > The API is remarkably similar to Gaynor's django-filters. My problem > with this is that your filtering essentially becomes > Model.objects.filter(x__y=x).filter(y__x=y) which creates a join each > time, whereas you may not really want a join. > > To get around this limitation I propose having the FilterSpec's return > Q objects and then passing them all into filter(), however I've been > stumped as to how to do this in such a way that you allow people to > use AND or OR, depending on their preference. > > I'm afraid if this issue isn't worked out the current proposal (which > is Accepted) will then become gospel and many uses cases (mine!) will > be left out. > > -Steve -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Tickets #12979, #11557
Generally, the best bet is to find someone who does NOT have commit access (and is thus not quite so busy) to review the patches, run the tests, and mark them as RFC. Once that has happened, the patch will get looked at by a committer, often in reasonably short order. Good etiquette dictates that the person marking as RFC shouldn't be someone who opened the ticket or who submitted the patch. Often asking in #django-dev or #django-social will turn someone up. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Tickets #12979, #11557
Hi, Does anyone with commit access have a few minutes to look over the following tickets? * http://code.djangoproject.com/ticket/12979 * http://code.djangoproject.com/ticket/11557 Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: [RFC] HttpResponse: freeze API, read_once
Hi Santiago,
On Mon, Sep 13, 2010 at 03:58:12PM -0300, Santiago Perez wrote:
> The biggest problem with the approach I've proposed is that it is not
> sufficiently granular to handle the case where I don't want the response
> content
> modified (e.g. compressed) but I *do* want to return HTTP 304 Not Modified
> where
> appropriate. That's the only use case I can think of right now that is
> not
> addressed by my proposal. I have a few ideas about this but I'm curious
> to
> see
> what ideas other people have and I think it can be addressed later.
>
>
> Another missing use case is when you have streaming content that you don't
> want
> to load in memory but you still want it to be modified (compressed for
> instance) in ways that can be done during the streaming.
Good point.
But it can be added fairly easily if we also stipulate that with iterator
content the iterator itself is available as attribute "content_iterator".
def compress_content(response):
if response.header_is_frozen('Content-Encoding') \
or response.header_is_frozen('Content-Length') \
or response.content_is_frozen():
return response
if response.read_once:
response.content_iterator =
compress_iterator(response.content_iterator)
# We cannot know the response length.
if 'Content-Length' in response:
del response['Content-Length']
else:
response.content = compress_string(response.content)
response['Content-Length'] = str(len(response.content))
response['Content-Encoding'] = 'gzip'
return response
Thanks,
Forest
--
Forest Bond
http://www.alittletooquiet.net
http://www.pytagsfs.org
signature.asc
Description: Digital signature
Re: [RFC] HttpResponse: freeze API, read_once
> > The biggest problem with the approach I've proposed is that it is not > sufficiently granular to handle the case where I don't want the response > content > modified (e.g. compressed) but I *do* want to return HTTP 304 Not Modified > where > appropriate. That's the only use case I can think of right now that is not > addressed by my proposal. I have a few ideas about this but I'm curious to > see > what ideas other people have and I think it can be addressed later. > Another missing use case is when you have streaming content that you don't want to load in memory but you still want it to be modified (compressed for instance) in ways that can be done during the streaming. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
[RFC] HttpResponse: freeze API, read_once
Hi,
I'd like to return to the problem of middleware, view decorators, etc. affecting
responses in negative ways.
Background
==
Previous discussion on django-developers (this has come up more than once,
though, I think):
*
http://groups.google.com/group/django-developers/browse_thread/thread/f5ba6df0400a9c57
Some relevant tickets (there are probably more):
* http://code.djangoproject.com/ticket/6527
* http://code.djangoproject.com/ticket/6027
* http://code.djangoproject.com/ticket/12214
This originally came up for me with streaming responses (with iterator content),
but since then I've seen a few new situations where I wanted to be able to
prevent middleware from doing certain things to certain responses.
Russell had proposed a capabilities-based approach to dealing with this problem
(see previous discussion). Ultimately, I don't think this is the correct way to
go simply because I don't think the problem is limited to the question of what
you *can* do to a response. It's also a question of what you *should* do to a
response.
In fact, the only real "capability" I'm aware of is the case where the response
content is an iterator. Everything else is a matter of developer preference,
which he would most likely want to express in his view or using a view
decorator. To accommodate this, we need an API that the developer can use to
express his preferences for how a particular response should be handled.
So here are the things that I think we need to be able to communicate about a
response:
1. Don't set, change, or remove header X.
2. Don't modify, replace, or remove the response content.
3. Response content can only be read once, don't try to read it.
Response Freezing
=
To address #1 and #2, I'd like to propose a response "freezing" API. The basic
idea is that I can freeze a particular header expect that that header will not
be modified by middleware or view decorators. I can also freeze the response
content to indicate that the content should not be modified.
The relevant functions would be:
* HttpResponse.freeze_header(header)
* HttpResponse.freeze_all_headers()
* HttpResponse.header_is_frozen(header)
* HttpResponse.freeze_content()
* HttpResponse.content_is_frozen()
Any response-handling code should check to see if the things it wants to modify
are frozen before actually doing anything. If they are, it should not modify
the response at all.
If middleware tries to modify something that has been frozen, an exception
should be raised. This is backwards compatible because the freeze API is new,
and so long as no one calls a freeze method, responses will continue to behave
as they do now. However, if there are concerns about the transition, we can
emit a DeprecationWarning instead for a few releases. In any case, middleware
included with Django should be updated to respect frozen requests.
To be clear with regard to backwards compatibility: no views, decorators, or
middleware included with Django should call the freeze_* functions, at least
until the freeze API has been out long enough for people to fix their
middleware.
Iterator Protection
===
To address #3, we need a way to protect responses with iterator content from
being drained prematurely. For this, I propose a boolean attribute on
HttpResponse instances called "read_once". This will default to False to
preserve backwards compatibility, but if it is set to True, an exception should
be raised when the content is read. Middleware can inspect response.read_once
to see if it is okay to read the response content.
Obviously, there will need to be some special handling by the HTTP handler so
that it can access the content so it can feed it to the client without tripping
protection.
Examples
Some simple functions that demonstrate how middleware/view decorators would need
to handle responses:
def set_etag(response):
if response.read_once or response.header_is_frozen('ETag'):
return response
response['ETag'] = calculate_etag(response.content)
return response
def set_content_length(response):
if response.read_once or response.header_is_frozen('Content-Length'):
return response
response['Content-Length'] = str(len(response.content))
return response
def compress_content(response):
# Note that we don't need to check response.read_once here since we are
# replacing the content with a non-iterator. If the user didn't want us
# to do this, he would have frozen the content.
if response.header_is_frozen('Content-Encoding') \
or response.header_is_frozen('Content-Length') \
or response.content_is_frozen():
return response
response.content = compress_string(response.content)
response['Content-Encoding'] = 'gzip'
response['Content-Length'] = str(len(response.content))
return response
Shortcomings
The biggest problem with the approach I
ANN: intermittent downtime on djangoproject.com this week
Hi folks -- Over the course of this week I'm going to be upgrading and improving the djangoproject.com infrastructure. We'll get some Trac upgrade action (thanks to Jeremy Dunck, Jannis, and a few others), a better /community page (thanks to Justin Lilly), and I'll be setting the stage for a sysadmin *team* so that I'm not a SPOF any more. That's going to mean some downtime this week, possibly a couple of hours here and there, affecting pretty much all parts of djangoproject.com (SVN, Trac, docs, etc.) at one point or another. Most of this downtime will be evenings US central time. See you on the other side, Jacob -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: CI server - IRC bot
On Sun, Sep 12, 2010 at 11:55 PM, Tobias McNulty wrote: > At the DjangoCon sprint this weekend I setup Django on our CI server for > OSS and had it reporting on builds in the #django-sprint IRC channel. Would > that be useful to continue post-sprint, and should I have it report to the > #django-dev channel as well? > I got an unsolicited +1 from Carl and Alex in IRC and lazy consensus is good enough for me...so I'll go ahead and add it. Feel free to object if it gets out of hand. Cheers, Tobias -- Tobias McNulty, Managing Partner Caktus Consulting Group, LLC http://www.caktusgroup.com -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: A prompt 1.2.3 release
Hi Eric, Does this mean that the account we set up on our server is no longer necessary? Stephen On 11 Sep 2010, at 01:09, Eric Holscher wrote: There was a hudson server running IIRC, but http://hudson.djangoproject.com/ is not responding to me. I took the hudson instance down because nobody was using it and it was costing me a decent amount of money to run per month. I still have the images around on Rackspace Cloud if the DSF or anyone actually wants to pay to have good CI. Just as a data point, I took it down about a month ago, and people only just noticed during the sprints. I don't know how to fix that particular problem. Cheers, Eric -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com . To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/django-developers?hl=en . -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
