Re: Exceptions in model._meta._property_names with some 3rd-pty libraries

[Carl Meyer]

Hi Zack,

On 06/02/2017 02:02 PM, Zack Voase wrote:
> Hi all,
> 
> I'm encountering exceptions in a number of popular third-party Django
> libraries when upgrading from 1.8 to 1.11. The code path is typically
> `MyModel.objects.get_or_create(...)`, which causes
> `model._meta._property_names` to be checked (to make sure we're not
> querying/setting a property). The problem is, `_property_names` is
> implemented as follows:
> 
> |
> # in django/db/models/options.py:
> def_property_names(self):
> returnfrozenset({
>attr forattr in
>dir(self.model)ifisinstance(getattr(self.model,attr),property)
> })
> |
> 
> The problem is when a custom field installs a field descriptor on the
> model class (during `contribute_to_class()`), with a `__get__()` method
> like this:
> 
> |
> classSomeFieldDescriptor(object):
> # ...
> def__get__(self,instance,type=None):
> ifinstance isNone:
> raiseAttributeError("Can only be accessed via an instance.")
> # ...
> |

I can see two things here that could be done better, one on Django side
and one on third-party app side.

1) I've never seen a good reason for a descriptor to raise an
AttributeError like that. Typically `return self` is a much more useful
option for the "access on the class" scenario, if the descriptor can't
provide useful behavior in that case, as it allows introspecting the
class and finding the descriptor object.

2) On the Django side, I think we should switch to using
`inspect.getattr_static`, to avoid any possibility of triggering
side-effecty descriptor code of any kind. AttributeError is only the
most visible problem here; there could be much subtler problems (e.g.
performance issues) caused by e.g. accessing a descriptor that does a
database query or something.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/fbd639ad-6a51-7657-f647-753cbc3b6b90%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Exceptions in model._meta._property_names with some 3rd-pty libraries

[Sergey Fursov]

Hi everyone,

I just want to mention here that in django-jsonfield repo proposed solution
to solve this problem on third party side:
https://github.com/dmkoch/django-jsonfield/issues/189. Maybe this is the
better way.

Thanks,
Sergey

2017-06-03 0:02 GMT+03:00 Zack Voase :

> Hi all,
>
> I'm encountering exceptions in a number of popular third-party Django
> libraries when upgrading from 1.8 to 1.11. The code path is typically
> `MyModel.objects.get_or_create(...)`, which causes
> `model._meta._property_names` to be checked (to make sure we're not
> querying/setting a property). The problem is, `_property_names` is
> implemented as follows:
>
> # in django/db/models/options.py:
> def _property_names(self):
> return frozenset({
>attr for attr in
>dir(self.model) if isinstance(getattr(self.model, attr), property)
> })
>
> The problem is when a custom field installs a field descriptor on the
> model class (during `contribute_to_class()`), with a `__get__()` method
> like this:
>
> class SomeFieldDescriptor(object):
> # ...
> def __get__(self, instance, type=None):
> if instance is None:
> raise AttributeError("Can only be accessed via an instance.")
> # ...
>
> Libraries which install such descriptors include [django-fsm](
> https://github.com/kmmbvnr/django-fsm/blob/2d2eaee/django_fsm/__init__.
> py#L225) and [django-jsonfield](https://github.com/dmkoch/django-
> jsonfield/blob/2.0.1/jsonfield/subclassing.py#L35). I think the problem
> is that we can't assume that all results of `dir(model_class)` can be
> accessed via `getattr(model_class, attr)` without exception; indeed, the
> Python docs state (c.f. https://docs.python.org/2/
> library/functions.html#dir):
>
> > Because dir() is supplied primarily as a convenience for use at an
> interactive prompt, it tries to supply an interesting set of names more
> than it tries to supply a rigorously or consistently defined set of names,
> and its detailed behavior may change across releases.
>
> A potential fix would be to adjust the definition of `_property_names`
> like so:
>
> def _property_names(self):
> attrs = []
> for attr in dir(self.model):
> try:
> if isinstance(getattr(self.model, attr), property):
> attrs.append(attr)
> except AttributeError:
> pass
> return frozenset(attrs)
>
> Does this seem like a good solution, or even a problem worth solving?
>
> Thanks!
> -- Zack
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/ec0e2506-4c4b-441f-b005-
> 5623d6521ae3%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAG-zk-BDvJOQfhgNBFBpRCBuPkB7MTtJCYLp6D1jxgbppMdr%2BQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Exceptions in model._meta._property_names with some 3rd-pty libraries

[Curtis Maloney]

What about using inspect.getmembers ?

https://docs.python.org/3/library/inspect.html#inspect.getmembers

In other code I've also used inspect.classify_class_attrs but it seems 
to be undocumented :/


If nothing else, it could be used as a guide on how to do this.

--
Curtis


On 03/06/17 08:52, Adam Johnson wrote:

This is my bad, I did the refactoring :) You're right, the original
version

did in fact use a try..except AttributeError and that should be
preserved for cases like this.

I've made a ticket here https://code.djangoproject.com/ticket/28269 . If
you want to make the PR copying in your fix and adding a test that would
be neat, you've done 90% of the work already! :)


On 2 June 2017 at 22:02, Zack Voase >
wrote:

Hi all,

I'm encountering exceptions in a number of popular third-party
Django libraries when upgrading from 1.8 to 1.11. The code path is
typically `MyModel.objects.get_or_create(...)`, which causes
`model._meta._property_names` to be checked (to make sure we're not
querying/setting a property). The problem is, `_property_names` is
implemented as follows:

|
# in django/db/models/options.py:
def_property_names(self):
returnfrozenset({
   attr forattr in
   dir(self.model)ifisinstance(getattr(self.model,attr),property)
})
|

The problem is when a custom field installs a field descriptor on
the model class (during `contribute_to_class()`), with a `__get__()`
method like this:

|
classSomeFieldDescriptor(object):
# ...
def__get__(self,instance,type=None):
ifinstance isNone:
raiseAttributeError("Can only be accessed via an instance.")
# ...
|

Libraries which install such descriptors include

[django-fsm](https://github.com/kmmbvnr/django-fsm/blob/2d2eaee/django_fsm/__init__.py#L225

)
and

[django-jsonfield](https://github.com/dmkoch/django-jsonfield/blob/2.0.1/jsonfield/subclassing.py#L35

).
I think the problem is that we can't assume that all results of
`dir(model_class)` can be accessed via `getattr(model_class, attr)`
without exception; indeed, the Python docs state (c.f.
https://docs.python.org/2/library/functions.html#dir
):

> Because dir() is supplied primarily as a convenience for use at an
interactive prompt, it tries to supply an interesting set of names
more than it tries to supply a rigorously or consistently defined
set of names, and its detailed behavior may change across releases.

A potential fix would be to adjust the definition of
`_property_names` like so:

|
def_property_names(self):
attrs =[]
forattr indir(self.model):
try:
ifisinstance(getattr(self.model,attr),property):
attrs.append(attr)
exceptAttributeError:
pass
returnfrozenset(attrs)
|

Does this seem like a good solution, or even a problem worth solving?

Thanks!
-- Zack

--
You received this message because you are subscribed to the Google
Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to django-developers+unsubscr...@googlegroups.com
.
To post to this group, send email to
django-developers@googlegroups.com
.
Visit this group at
https://groups.google.com/group/django-developers
.
To view this discussion on the web visit

https://groups.google.com/d/msgid/django-developers/ec0e2506-4c4b-441f-b005-5623d6521ae3%40googlegroups.com

.
For more options, visit https://groups.google.com/d/optout
.




--
Adam

--
You received this message because you are subscribed to the Google
Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to django-developers+unsubscr...@googlegroups.com
.
To post to this group, send email to django-developers@googlegroups.com
.
Visit this group at https://groups.google.com/group/django-developers.
To 

Re: Exceptions in model._meta._property_names with some 3rd-pty libraries

[Adam Johnson]

This is my bad, I did the refactoring :) You're right, the original version

did in fact use a try..except AttributeError and that should be preserved
for cases like this.

I've made a ticket here https://code.djangoproject.com/ticket/28269 . If
you want to make the PR copying in your fix and adding a test that would be
neat, you've done 90% of the work already! :)


On 2 June 2017 at 22:02, Zack Voase  wrote:

> Hi all,
>
> I'm encountering exceptions in a number of popular third-party Django
> libraries when upgrading from 1.8 to 1.11. The code path is typically
> `MyModel.objects.get_or_create(...)`, which causes
> `model._meta._property_names` to be checked (to make sure we're not
> querying/setting a property). The problem is, `_property_names` is
> implemented as follows:
>
> # in django/db/models/options.py:
> def _property_names(self):
> return frozenset({
>attr for attr in
>dir(self.model) if isinstance(getattr(self.model, attr), property)
> })
>
> The problem is when a custom field installs a field descriptor on the
> model class (during `contribute_to_class()`), with a `__get__()` method
> like this:
>
> class SomeFieldDescriptor(object):
> # ...
> def __get__(self, instance, type=None):
> if instance is None:
> raise AttributeError("Can only be accessed via an instance.")
> # ...
>
> Libraries which install such descriptors include [django-fsm](
> https://github.com/kmmbvnr/django-fsm/blob/2d2eaee/django_fsm/__init__.
> py#L225) and [django-jsonfield](https://github.com/dmkoch/django-
> jsonfield/blob/2.0.1/jsonfield/subclassing.py#L35). I think the problem
> is that we can't assume that all results of `dir(model_class)` can be
> accessed via `getattr(model_class, attr)` without exception; indeed, the
> Python docs state (c.f. https://docs.python.org/2/
> library/functions.html#dir):
>
> > Because dir() is supplied primarily as a convenience for use at an
> interactive prompt, it tries to supply an interesting set of names more
> than it tries to supply a rigorously or consistently defined set of names,
> and its detailed behavior may change across releases.
>
> A potential fix would be to adjust the definition of `_property_names`
> like so:
>
> def _property_names(self):
> attrs = []
> for attr in dir(self.model):
> try:
> if isinstance(getattr(self.model, attr), property):
> attrs.append(attr)
> except AttributeError:
> pass
> return frozenset(attrs)
>
> Does this seem like a good solution, or even a problem worth solving?
>
> Thanks!
> -- Zack
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/ec0e2506-4c4b-441f-b005-
> 5623d6521ae3%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM3ZqsL5JStFfuGG%3DM%2BAmOcmkOLGAkmTRDkVfUYXnMqcYA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Exceptions in model._meta._property_names with some 3rd-pty libraries

[Zack Voase]

Hi all,

I'm encountering exceptions in a number of popular third-party Django 
libraries when upgrading from 1.8 to 1.11. The code path is typically 
`MyModel.objects.get_or_create(...)`, which causes 
`model._meta._property_names` to be checked (to make sure we're not 
querying/setting a property). The problem is, `_property_names` is 
implemented as follows:

# in django/db/models/options.py:
def _property_names(self):
return frozenset({
   attr for attr in
   dir(self.model) if isinstance(getattr(self.model, attr), property)
})

The problem is when a custom field installs a field descriptor on the model 
class (during `contribute_to_class()`), with a `__get__()` method like this:

class SomeFieldDescriptor(object):
# ...
def __get__(self, instance, type=None):
if instance is None:
raise AttributeError("Can only be accessed via an instance.")
# ...

Libraries which install such descriptors include 
[django-fsm](https://github.com/kmmbvnr/django-fsm/blob/2d2eaee/django_fsm/__init__.py#L225)
 
and 
[django-jsonfield](https://github.com/dmkoch/django-jsonfield/blob/2.0.1/jsonfield/subclassing.py#L35).
 
I think the problem is that we can't assume that all results of 
`dir(model_class)` can be accessed via `getattr(model_class, attr)` without 
exception; indeed, the Python docs state (c.f. 
https://docs.python.org/2/library/functions.html#dir):

> Because dir() is supplied primarily as a convenience for use at an 
interactive prompt, it tries to supply an interesting set of names more 
than it tries to supply a rigorously or consistently defined set of names, 
and its detailed behavior may change across releases.

A potential fix would be to adjust the definition of `_property_names` like 
so:

def _property_names(self):
attrs = []
for attr in dir(self.model):
try:
if isinstance(getattr(self.model, attr), property):
attrs.append(attr)
except AttributeError:
pass
return frozenset(attrs)

Does this seem like a good solution, or even a problem worth solving?

Thanks!
-- Zack

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ec0e2506-4c4b-441f-b005-5623d6521ae3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: On ASGI...

[Andrew Godwin]

Right. I'll try and get a full async example up in channels-examples soon
to show off how this might work; I did introduce a Worker class into the
asgiref package last week as well, and the two things that you need to
override on that are "the list of channels to listen to" and "handle this
message".

While the consumer interface as it stands is tied into Django (where we
_could_ allow coroutines, with some difficulty if I am to continue
supporting Python 2.7 for Django 1.11), there's also room for asgiref to
have either:

 - Something substantially similar to Django's consumer system, or
 - The ability to just plug in a callable/coroutine that gets called with
messages as they arrive, and which provides you with channel names to
listen on as an attribute

This second one is perhaps my preferred approach, as it's getting very
close to the design of WSGI, and it would not be hard to re-layer Channels
to exist behind this pattern. I think at that point the main difference to
WSGI is the need to persist state somewhere outside the locals() of the
function (as you'll still have separate coroutines for
connect/receive/disconnect).

Andrew

On Fri, Jun 2, 2017 at 8:21 AM, Tom Christie  wrote:

> > I suspect there is potential for a very fast async-only layer that can
> trigger the await that's hanging in a receive_async() directly from a
> send() to a related channel, rather than sticking it onto a memory location
> and waiting
>
> Yup. Something that the gunicorn worker doesn't currently provide is
> `receive()/receive_async()` hooks, and instead current just  rely on
> `send()`ing every incoming message. For eg. HTTP the one thing that doesn't
> give you so easily is request body buffering or streaming, but you *could*
> handle that on the application side if you really wanted.
>
> I think that the sensible next step for me will be to work towards adding
> that interface in, at which point we ought to have an implementation
> capable of running a Django service using ASGIHandler. (And yeah, I see
> that you'd be adding channel queues at that point.) ayncio.Queue is looking
> like a mightily useful bit of tooling right now.
>
> > the design of an ASGI-direct protocol
>
> I think the important part of this is "how does asyncio code look in the
> context of an ASGI consumer interface" rather than specifics about
> server-direct-to-consumer configurations. Not unreasonable that an "An
> asyncio ASGI app, running behind a full channel layer" could be a thing
> too, in the future.
>
> Perhaps a good way of narrowing this down would be to consider an asyncio
> runworker: Is the application responsible for pushing coroutines onto the
> event loop (advantage: more narrowly scoped), or do you extend the ASGI
> consumer callable definition to also allow it to be a coroutine
> (advantages: more natural interface, server can manage task cancellations).
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/ms
> gid/django-developers/16f87149-6896-40a2-a1fd-028446a171a3%
> 40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1uqqXFc7tKfU4b3uL%3DGdPUBNHBAS1aURP15Sa56nmT2czg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Tabs in Django Admin forms?

[Adam Johnson]

I agree, it sounds like feature creep to me. It could probably be provided
by code in a third party package too, the admin is very extensible.

On 2 June 2017 at 16:42, Tim Graham  wrote:

> I think Django can't reasonably add every feature under the sun to the
> admin. If you need a pixel perfect UI, use your own views.
>
> On Tuesday, May 30, 2017 at 4:50:16 AM UTC-4, Óscar M. Lage wrote:
>>
>> Oh, sorry, something like this:
>>
>>
>> 
>>
>> I think we've the ability to put the fields side-by-side but don't know
>> if there is a way to group the fields in tabs.
>>
>> El martes, 30 de mayo de 2017, 10:33:07 (UTC+2), Florian Apolloner
>> escribió:
>>>
>>> > to group form fields in tabs (as django-crispy-form does)?.
>>>
>>> For those who do not know django-crispy-form, how does that look like?
>>>
>>> Cheers,
>>> Florian
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/8185ce8c-b582-4072-9a08-
> fe84f1bd5380%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM2n55J8aK5aXOgU_dgQx%2BO7QToxfYBn34O5OL46Ama%3DNA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Incorrect escaping in Django SQL query

[Tim Graham]

Cross posted to django-users: 
https://groups.google.com/d/msg/django-users/nNgcv2QajCI/hPRwIeVNBAAJ

In the future, please keep the conversation in one place. django-users is 
fine for asking "is it a bug?" questions.

On Friday, June 2, 2017 at 11:26:22 AM UTC-4, Tim Graham wrote:
>
> I couldn't find any testing of TruncDay with Value().
>
> Is there a difference between the query you're trying to construct and 
> MyModel.objects.filter(created__lt=timezone.now().date()) ?
>
> On Thursday, June 1, 2017 at 7:35:25 AM UTC-4, Roshan Raghupathy wrote:
>>
>> Hi,
>>
>> I came across an issue yesterday caused by TruncDay function. You can 
>> find it here: 
>> https://stackoverflow.com/questions/44287443/incorrect-escaping-in-django-sql-query
>>
>> Copy pasting from there:
>>
>> Here's the query I'm trying to run:
>>
>> MyModel.objects.filter(created__lt=functions.TruncDay(Value(timezone.now(), 
>> output_field=DateTimeField(
>>
>> It translates to:
>>
>> SELECT  FROM "mymodel" WHERE "mymodel"."created" < 
>> (DATE_TRUNC('day', %%s AT TIME ZONE %s))
>>
>> before Django performs parameter substitution. Note that the first %s has 
>> been escaped to %%s. This causes the parameter substitution to throw an 
>> exception.
>>
>> Is this intended behaviour or a bug?
>>
>>
>> I dug into the internals a bit to figure this out and the issue comes 
>> from this line 
>> 
>>  where 
>> the query till that point is escaped but later, they are never escaped back 
>> to the original form for correct parameter substitution. I went ahead and 
>> added code to replace `%%s` with `%s` before parameter substitution and it 
>> worked fine. Am I missing something here or should I file a bug(has it been 
>> filed already?)?
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/85487850-4b39-4453-a3d8-a63ada279c02%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Are there use cases for storing null bytes in CharField/TextField?

[Tim Graham]

I found a PostgreSQL bug report requesting removal of the restriction. 
Here's the final reply:

Franklin Schmidt wrote:
> I agree that storing 0x00 in a UTF8 string is weird, but I am
> converting a huge database to postgres, and in a huge database, weird
> things happen.  Using bytea for a text field just because one in a
> million records has a 0x00 doesn't make sense to me.  I did hack
> around it in my conversion code to remove the 0x00 but I expect that
> anyone else who tries converting a big database to postgres will also
> confront this issue.

That's the right solution. If you have 0x00 bytes in your text fields, 
you're much better off cleaning them away anyway, than trying to work 
around them.

-Heikki Linnakangas


https://www.postgresql.org/message-id/200712170734.lBH7YdG9034458%40wwwmaster.postgresql.org

I also found a possible related discussion about supporting \u in JSON 
values [1]. PostgreSQL tried to support it but had to remove that support 
because it caused ambiguity.

https://www.postgresql.org/message-id/e1yhhv8-00032a...@gemulon.postgresql.org

On Wednesday, May 31, 2017 at 8:27:49 PM UTC-4, Jon Dufresne wrote:
>
> On Mon, May 15, 2017 at 10:30 AM, Tim Chase  > wrote:
>
>> On 2017-05-15 08:54, Tim Graham wrote:
>> > Does anyone know of a use case for using null bytes in
>> > CharField/TextField?
>>
>> Is this not what BinaryField is for?  It would seem to me that
>> attempting to store binary NULL bytes in a CharField/TextField should
>> result in an error condition.
>>
>
> The null byte is also a valid Unicode code point [0].
>
> I guess I'm a bit surprised that a valid code point can't be stored in a 
> PostgreSQL text column. This does appear to be documented for the char(int) 
> string function [1], although without justification.
>
> > The NULL (0) character is not allowed because text data types cannot 
> store such bytes.
>
> I'm curious behind PostgreSQL's decision to prohibit this code point. If 
> anyone has additional information to share on their reason, please pass it 
> along.
>
>
> [0] http://www.fileformat.info/info/unicode/char//index.htm
> [1] https://www.postgresql.org/docs/current/static/functions-string.html
>
> Cheers
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/450da309-c2af-43a9-8026-51d23994f8c9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Tests failing on postgres 9.6

[Tim Graham]

Closing the loop for anyone who finds this message and doesn't click 
through to the ticket:

It seems to be a locale problem. Cleaned up everything (removed 
/usr/local/var/postgres), installed postgres again with export 
LC_ALL=en_US.UTF-8; export LANG=en_US.UTF-8 and it worked.

On Tuesday, May 23, 2017 at 11:33:18 PM UTC-4, André Ericson wrote:
>
> Hi,
>
> I'm having troubles with a couple of tests on postgres 9.6. I've created a 
> ticket here: https://code.djangoproject.com/ticket/28234
>
> Because Jenkins is still on 9.5 I would appreciate if someone could try to 
> reproduce it on their setup with 9.6.
>
> Thanks.
>
> André Ericson
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a8ba5921-73ec-41f5-9f62-82463f742c70%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Tabs in Django Admin forms?

[Tim Graham]

I think Django can't reasonably add every feature under the sun to the 
admin. If you need a pixel perfect UI, use your own views.

On Tuesday, May 30, 2017 at 4:50:16 AM UTC-4, Óscar M. Lage wrote:
>
> Oh, sorry, something like this:
>
>
> 
>
> I think we've the ability to put the fields side-by-side but don't know if 
> there is a way to group the fields in tabs.
>
> El martes, 30 de mayo de 2017, 10:33:07 (UTC+2), Florian Apolloner 
> escribió:
>>
>> > to group form fields in tabs (as django-crispy-form does)?.
>>
>> For those who do not know django-crispy-form, how does that look like?
>>
>> Cheers,
>> Florian
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8185ce8c-b582-4072-9a08-fe84f1bd5380%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: migrate id field from integer to biginter (probable bug)

[Tim Graham]

Yes, it looks like a bug.

The incorrect SQL is added here:
https://github.com/django/django/blob/8149bd00d8b8af816a683e5ecc0e204f344616f5/django/db/backends/base/schema.py#L745-L749

On Wednesday, May 31, 2017 at 1:34:07 PM UTC-4, drakkan wrote:
>
> Hi,
>
> I have two models like these:
>
> class Allarme(models.Model):
> 
>
> class Registrazione(models.Model):
>allarme = models.ForeignKey(Allarme, blank=True, null=True, 
> on_delete=models.DO_NOTHING, db_constraint=False)
>
>
> now I want to change id field for these models from int to bigint so I 
> added to both
>
> id = models.BigAutoField(primary_key=True)
>
> and I did the migration,
>
> python manage.py sqlmigrate
>
> show something like this:
>
> BEGIN;
> --
> -- Alter field id on allarme
> --
> ALTER TABLE "multimedia_allarmi" ALTER COLUMN "id" TYPE bigint USING 
> "id"::bigint;
> DROP SEQUENCE IF EXISTS "multimedia_allarmi_id_seq" CASCADE;
> CREATE SEQUENCE "multimedia_allarmi_id_seq";
> ALTER TABLE "multimedia_allarmi" ALTER COLUMN "id" SET DEFAULT 
> nextval('"multimedia_allarmi_id_seq"');
> SELECT setval('"multimedia_allarmi_id_seq"', MAX("id")) FROM 
> "multimedia_allarmi";
> ALTER TABLE "multimedia_registrazioni" ALTER COLUMN "allarme_id" TYPE 
> bigint USING "allarme_id"::bigint;
> *ALTER TABLE "multimedia_registrazioni" ADD CONSTRAINT 
> "multimedia_registrazioni_allarme_id_4919213e_fk" FOREIGN KEY 
> ("allarme_id") REFERENCES "multimedia_allarmi" ("id") DEFERRABLE INITIALLY 
> DEFERRED;*
> --
> -- Alter field id on registrazione
> --
> ALTER TABLE "multimedia_registrazioni" ALTER COLUMN "id" TYPE bigint USING 
> "id"::bigint;
> DROP SEQUENCE IF EXISTS "multimedia_registrazioni_id_seq" CASCADE;
> CREATE SEQUENCE "multimedia_registrazioni_id_seq";
> ALTER TABLE "multimedia_registrazioni" ALTER COLUMN "id" SET DEFAULT 
> nextval('"multimedia_registrazioni_id_seq"');
> SELECT setval('"multimedia_registrazioni_id_seq"', MAX("id")) FROM 
> "multimedia_registrazioni";
> COMMIT;
>
> so as you can see a foreign key is added for a model field with 
> db_constraint=False, 
>
> can you please confirm that this is a bug? I'm using django 1.11.1
>
> thanks!
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5f1f024e-f737-4ee3-ac83-b92e9c956a7b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: should nonexistent template filter arguments resolve to string_if_invalid rather than raise VariableDoesNotExist?

[Vlastimil Zíma]

Django now reports invalid variable names by 'VariableDoesNotExist' 
exception, which is logged. So it is possible to track down invalid 
variable names. The problem I see here is inconsistency in handling invalid 
variable names. If invalid variable is rendered or used in an 'if' tag, it 
is silently turned into a string_if_invalid, if it's as a filter argument, 
it suddenly raises error.

Shai: Values are sometimes turned into an empty string in case of 
UnicodeDecodeError, see 
https://github.com/django/django/blob/master/django/template/base.py#L994-L998

IMO invalid variable should behave the same way regardless of whether it's 
printed or used as an filter argument.

Vlastik

Dne pátek 2. června 2017 8:37:35 UTC+2 Shai Berger napsal(a):
>
> I favor raising an exception in this case, because values in this context 
> are 
> not coerced to strings (unlike the case in {{ }} references). 
>
> IMO, and in order to protect the writers of all 3rd-party filters, the 
> only way 
> to solve this without raising an exception is to resolve the whole filter 
> expression to string_if_invalid, rather than just the missing var. 
>
> Shai 
>
> On Thursday 01 June 2017 03:10:39 Jon Dufresne wrote: 
> > Just my opinion, but I think raising an exception is more helpful to 
> > developers and will result in fewer unnoticed bugs. 
> > 
> > More generally, I know the template engine has a history of silently 
> > converting unknown variables to string_if_invalid but this is more 
> harmful 
> > than helpful in my experience. It continues to be a source of hard to 
> catch 
> > bugs on the projects I work on. So I'd prefer we avoid introducing more 
> of 
> > this pattern if possible. 
> > 
> > On Wed, May 31, 2017 at 8:03 AM, Tim Graham  > wrote: 
> > > Should nonexistent template filter arguments raise an exception? 
> Current 
> > > behavior: 
> > > 
> > > {{ value|filter:nonexistent_template_var}} 
> > > 
> > > raises VariableDoesNotExist for nonexistent_template_var. 
> > > 
> > > I guess the proposal would be to make nonexistent_tempatle_var resolve 
> to 
> > > string_if_invalid. As for me, I think the current behavior is less 
> error 
> > > prone. 
> > > 
> > > Related tickets: 
> > > 
> > > https://code.djangoproject.com/ticket/13167 
> > > This was first closed incorrectly as "fixed" by Jacob (actually a 
> > > different issue was fixed), then amended as wontfix by Karen with the 
> > > note, "I'm fine with wontfixing that, though it does rather seem to go 
> > > against "template errors don't raise exceptions" philosophy." 
> > > 
> > > https://code.djangoproject.com/ticket/28172 
> > > A follow up ticket requesting the same thing as #13167. 
> > > 
> > > -- 
> > > You received this message because you are subscribed to the Google 
> Groups 
> > > "Django developers (Contributions to Django itself)" group. 
> > > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > > email to django-develop...@googlegroups.com . 
> > > To post to this group, send email to django-d...@googlegroups.com 
> . 
> > > Visit this group at https://groups.google.com/group/django-developers. 
>
> > > To view this discussion on the web visit https://groups.google.com/d/ 
> > > msgid/django-developers/02727bd0-5a67-4cf9-9c3f- 
> > > b0a0a7ea0a3a%40googlegroups.com 
> > > <
> https://groups.google.com/d/msgid/django-developers/02727bd0-5a67-4cf9-9 
> > > c3f-b0a0a7ea0a3a%40googlegroups.com?utm_medium=email_source=footer> 
> . 
> > > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/163523b9-00f5-4327-9803-6eca49f668da%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Incorrect escaping in Django SQL query

[Tim Graham]

I couldn't find any testing of TruncDay with Value().

Is there a difference between the query you're trying to construct and 
MyModel.objects.filter(created__lt=timezone.now().date()) ?

On Thursday, June 1, 2017 at 7:35:25 AM UTC-4, Roshan Raghupathy wrote:
>
> Hi,
>
> I came across an issue yesterday caused by TruncDay function. You can find 
> it here: 
> https://stackoverflow.com/questions/44287443/incorrect-escaping-in-django-sql-query
>
> Copy pasting from there:
>
> Here's the query I'm trying to run:
>
> MyModel.objects.filter(created__lt=functions.TruncDay(Value(timezone.now(), 
> output_field=DateTimeField(
>
> It translates to:
>
> SELECT  FROM "mymodel" WHERE "mymodel"."created" < 
> (DATE_TRUNC('day', %%s AT TIME ZONE %s))
>
> before Django performs parameter substitution. Note that the first %s has 
> been escaped to %%s. This causes the parameter substitution to throw an 
> exception.
>
> Is this intended behaviour or a bug?
>
>
> I dug into the internals a bit to figure this out and the issue comes from 
> this 
> line 
> 
>  where 
> the query till that point is escaped but later, they are never escaped back 
> to the original form for correct parameter substitution. I went ahead and 
> added code to replace `%%s` with `%s` before parameter substitution and it 
> worked fine. Am I missing something here or should I file a bug(has it been 
> filed already?)?
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f6776bdc-366a-4b38-ae73-02d87e4401c4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: On ASGI...

[Tom Christie]

> I suspect there is potential for a very fast async-only layer that can 
trigger the await that's hanging in a receive_async() directly from a 
send() to a related channel, rather than sticking it onto a memory location 
and waiting

Yup. Something that the gunicorn worker doesn't currently provide is 
`receive()/receive_async()` hooks, and instead current just  rely on 
`send()`ing every incoming message. For eg. HTTP the one thing that doesn't 
give you so easily is request body buffering or streaming, but you *could* 
handle that on the application side if you really wanted.

I think that the sensible next step for me will be to work towards adding 
that interface in, at which point we ought to have an implementation 
capable of running a Django service using ASGIHandler. (And yeah, I see 
that you'd be adding channel queues at that point.) ayncio.Queue is looking 
like a mightily useful bit of tooling right now.

> the design of an ASGI-direct protocol

I think the important part of this is "how does asyncio code look in the 
context of an ASGI consumer interface" rather than specifics about 
server-direct-to-consumer configurations. Not unreasonable that an "An 
asyncio ASGI app, running behind a full channel layer" could be a thing 
too, in the future.

Perhaps a good way of narrowing this down would be to consider an asyncio 
runworker: Is the application responsible for pushing coroutines onto the 
event loop (advantage: more narrowly scoped), or do you extend the ASGI 
consumer callable definition to also allow it to be a coroutine 
(advantages: more natural interface, server can manage task cancellations).

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/16f87149-6896-40a2-a1fd-028446a171a3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Value of tightening URLValidator/EmailValidator regular expressions?

[Tim Graham]

Aymeric, did anything specific change your mind from your March 2016 mail:

"Indeed, for some reason, the URL and email validators get anywhere from 2 
to 8 changes in every Django version, and there’s no end in sight. (I 
contributed to this. Sorry.) Like James, I’m in favor of making the 
validation much more simple and documenting it. This seems better than 
perpetually modifying it at the risk of introducing regressions."

How should we make a determination about future Email/URLValidator changes? 
Put a halt to them completely? I've closed a few tickets about 
EmailValidator (e.g. [1]) as wontfix under the assumption that the regex 
will be simplified.

[1] https://code.djangoproject.com/ticket/25452

On Thursday, June 1, 2017 at 5:27:10 PM UTC-4, Aymeric Augustin wrote:
>
> I agree with Claude.
>
> -- 
> Aymeric.
>
>
>
> On 1 Jun 2017, at 09:50, Claude Paroz  
> wrote:
>
> As for me, I still think the current validator is valid for 99% of use 
> cases. And 99% of the time, an email address with dot-less domain is a user 
> input error.
>
> So I would prefer fixing #25594 (validator propagation from db field to 
> form field), adding a "looser" validator in validators.py and better 
> documenting usage of alternate validators for EmailFields.
> But I won't block the boat if I'm in the minority!
>
> Claude
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-develop...@googlegroups.com .
> To post to this group, send email to django-d...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/c3138fc8-fb78-4ff3-898d-2ed92433b13e%40googlegroups.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/d9cb4c7c-c3ae-4e33-921e-552c49f33626%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Deprecating AuthenticationMiddleware

[Tim Graham]

Curtis started a PR [0] that replaces the functionality of 
AuthenticationMiddleware.process_request():

request.user = SimpleLazyObject(lambda: get_user(request))

with this line in AuthConfig.ready():

HttpRequest.user = cached_property(get_user, 'user')

Before finishing the patch, I'd like to get a few more eyeballs on it to 
make sure there aren't any downsides to this approach. Thanks!

[0] https://github.com/django/django/pull/8568

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3f443ea4-8963-4609-8325-0c9464030d25%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ feature request ] Ability to add plugin child restricted in interface mode edit

[Nicolas Pascal]

Ability to add  plugin child restricted in interface edit mode, such {%  
render_plugin_child_add  %}.

This is to avoid access to the structure mode for a user who enriches only 
the exsiting content.

1)
In the template parent plugin, we could add {% render_plugin_child_add  
'samplechildAplugin'  'samplechildBplugin' %}, and a list of restricted 
plugin choices appears under the + symbol.

or
 
2)
 {% Render_plugin_child_add %} which would inherit from child plugins 
allowed structure mode, and a list of restricted plugin choices appears 
under the + symbol.

According to my research, I am not sure it is possible at this time.
I think this will be handy when the structure mode is not activated and 
when it is just a matter of adding a child plugin.

Can this be envisaged ?

-- 
Message URL: 
https://groups.google.com/d/msg/django-cms-developers/topic-id/message-id
Unsubscribe: send a message to 
django-cms-developers+unsubscr...@googlegroups.com
--- 
You received this message because you are subscribed to the Google Groups 
"django CMS developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-cms-developers+unsubscr...@googlegroups.com.
To view this discussion on the web, visit 
https://groups.google.com/d/msgid/django-cms-developers/339d5fb8-6153-4c01-9d0b-85e3c8700367%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: should nonexistent template filter arguments resolve to string_if_invalid rather than raise VariableDoesNotExist?

[Shai Berger]

I favor raising an exception in this case, because values in this context are 
not coerced to strings (unlike the case in {{ }} references).

IMO, and in order to protect the writers of all 3rd-party filters, the only way 
to solve this without raising an exception is to resolve the whole filter 
expression to string_if_invalid, rather than just the missing var.

Shai

On Thursday 01 June 2017 03:10:39 Jon Dufresne wrote:
> Just my opinion, but I think raising an exception is more helpful to
> developers and will result in fewer unnoticed bugs.
> 
> More generally, I know the template engine has a history of silently
> converting unknown variables to string_if_invalid but this is more harmful
> than helpful in my experience. It continues to be a source of hard to catch
> bugs on the projects I work on. So I'd prefer we avoid introducing more of
> this pattern if possible.
> 
> On Wed, May 31, 2017 at 8:03 AM, Tim Graham  wrote:
> > Should nonexistent template filter arguments raise an exception? Current
> > behavior:
> > 
> > {{ value|filter:nonexistent_template_var}}
> > 
> > raises VariableDoesNotExist for nonexistent_template_var.
> > 
> > I guess the proposal would be to make nonexistent_tempatle_var resolve to
> > string_if_invalid. As for me, I think the current behavior is less error
> > prone.
> > 
> > Related tickets:
> > 
> > https://code.djangoproject.com/ticket/13167
> > This was first closed incorrectly as "fixed" by Jacob (actually a
> > different issue was fixed), then amended as wontfix by Karen with the
> > note, "I'm fine with wontfixing that, though it does rather seem to go
> > against "template errors don't raise exceptions" philosophy."
> > 
> > https://code.djangoproject.com/ticket/28172
> > A follow up ticket requesting the same thing as #13167.
> > 
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django developers (Contributions to Django itself)" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to django-developers+unsubscr...@googlegroups.com.
> > To post to this group, send email to django-developers@googlegroups.com.
> > Visit this group at https://groups.google.com/group/django-developers.
> > To view this discussion on the web visit https://groups.google.com/d/
> > msgid/django-developers/02727bd0-5a67-4cf9-9c3f-
> > b0a0a7ea0a3a%40googlegroups.com
> >  > c3f-b0a0a7ea0a3a%40googlegroups.com?utm_medium=email_source=footer> .
> > For more options, visit https://groups.google.com/d/optout.